AI content creation with agents: three use cases that survive production

The demo is easy. A model reads a ticket, writes a reply, everyone claps. Six weeks later the same agent is quietly switched off because it invented a material number, opened fifty pull requests nobody reviewed, and nobody could say what it cost.

The patterns below share one property: the agent produces an artefact a human already knows how to check — a diff, a test report, a draft, a migration worklist. That single constraint removes most of the risk and all of the mystique. What follows is the architecture, the tool surface and the failure modes for each of the three sections on this site.

The foundation: an agent is a loop with brakes

Strip the marketing away and a working agent is five things:

1. A goal stated as a verifiable outcome ("all failing tests green and no new lint errors"), not a vibe.
2. A tool surface — narrow, typed, permissioned. Each tool does one thing and validates its own input.
3. State kept outside the model: a task record, a worklog, a JSON file. The context window is a scratchpad, never the system of record.
4. A brake — a budget (tokens, tool calls, wall clock) and a gate (human, CI, policy) that can stop the loop.
5. A trace — every prompt, tool call, result and decision stored, replayable, attributable.

If you cannot point at all five in your design, you have a prompt, not an agent. That distinction matters in an audit.

Rule I use on every engagement: the agent may read anything it has been granted, propose anything, and change nothing without a gate. Read, propose, gated write. In that order.

Use case 1 — Agents inside an SAP S/4HANA migration

A migration is a giant, repetitive, evidence-hungry activity: tens of thousands of custom objects, hundreds of test scripts, reconciliation worksheets nobody wants to own. It is close to an ideal agent playing field, because the work is bounded by rules SAP already codifies.

Where the agent earns its keep

The architecture that works

Keep the model away from the productive system. The pattern is a read-only extract, offline reasoning, gated write-back:

SAP (DEV/QAS, read-only RFC/OData)
   │  extract: ATC findings, object list, usage data
   ▼
Landing zone (Parquet / SQLite)  ←  agent works here, not in SAP
   │
   ▼
Agent: classify → cluster → propose remedy → draft worklist
   │
   ▼
Artefact: CSV/Markdown worklist + Jira issues + Confluence page
   │
   ▼
Human gate: architect reviews, edits, approves
   │
   ▼
Transport of record (only after approval)

The classification step is where you should spend your prompt engineering. Force structured output and a fixed taxonomy, and let the model abstain:

REMEDY = Literal["retire", "standard_replacement", "adapt", "rewrite_rap", "unknown"]

class Finding(BaseModel):
    object_name: str
    object_type: Literal["PROG", "CLAS", "FUGR", "INCLUDE", "CDS"]
    atc_priority: int
    message_ids: list[str]
    remedy: REMEDY
    rationale: str = Field(max_length=400)
    confidence: float = Field(ge=0, le=1)
    needs_evidence: bool          # true -> route to a human, do not guess

PROMPT = """You triage ABAP findings for an S/4HANA conversion.

Rules:
- Base the decision only on the ATC message ids and the code excerpt given.
- If the excerpt is insufficient, set needs_evidence=true and remedy="unknown".
- Never propose a remedy that deletes business logic without citing the message id.
- Prefer 'standard_replacement' when a released API covers the message.
"""

What to measure

• Classification agreement against a hand-labelled sample of 300 objects. Below 85 % on any remedy class, do not scale it out.
• Abstention rate — a healthy agent says "unknown" on 10–25 % of objects and routes them to a human. An agent that never abstains is guessing.
• Cycle time from "findings exported" to "worklist approved", which is the number your programme manager actually cares about.

Use case 2 — AI integrated into Microsoft Teams

Teams is where your users already are, which makes it the highest-leverage surface and the easiest place to build something nobody opens twice. The difference between the two is almost never the model — it's whether the assistant answers inside the conversation where the work happens, with the permissions of the person asking.

Three shapes, in increasing order of value

1. Q&A over your own corpus. A retrieval assistant over SharePoint, Confluence and policy documents, scoped by Entra ID groups. Useful, cheap, and the one everybody builds first. Ceiling: it only answers questions.

2. Action bot. The assistant can do things through typed tools: raise a Jira ticket, check a purchase order status, request an SAP transport, schedule a cutover dry run. This is where adoption jumps, because people stop switching tabs.

3. Ambient participant. The bot sits in a channel, watches the conversation, and offers the artefact before anyone asks — meeting notes with decisions extracted, a proposed ticket from a complaint, a runbook link when an alert fires. Powerful and the one most likely to annoy people. Default it to silence and let it be summoned.

The implementation stack that holds up

• Identity: Microsoft Entra ID with on-behalf-of flow. The bot must never have a super-user token; it acts as the user, so file-level permissions are enforced by Graph, not by your prompt.
• Channel: Teams AI Library (or Bot Framework if you need the older surface) with a manifest scoped to a single team for the pilot.
• Knowledge: retrieval over sanctioned sources only. Chunk on headings, keep the source URL and last-modified date in every chunk, and always return citations.
• Actions: each tool wraps an existing API with its own validation and audit entry. Never let the model compose raw HTTP.
• Data boundary: classify the corpus first. Anything above "internal" gets summarised in your tenant and never leaves it — that is a procurement question, not an engineering one, but it kills projects late if ignored.

# Manifest skeleton — keep the scope minimal for the pilot
{
  "manifestVersion": "1.17",
  "id": "<app-guid>",
  "name": { "short": "Programme Assistant" },
  "bots": [{
    "botId": "<bot-guid>",
    "scopes": ["team", "personal"],
    "supportsFiles": false,
    "commandLists": [{
      "scopes": ["team"],
      "commands": [
        { "title": "status",  "description": "Where is PO 4500012345?" },
        { "title": "ticket",  "description": "Draft a Jira issue from this thread" },
        { "title": "runbook", "description": "Find the cutover runbook for module X" }
      ]
    }]
  }],
  "permissions": ["identity", "messageTeamMembers"]
}

What makes users keep it

Three details, all unglamorous:

• Answer in the thread, not in a tab. If the user has to leave the conversation to get value, they will stop.
• Show your sources, inline. One line under the answer with the document name and date. It is the difference between trust and a novelty.
• Say "I don't know" fast. A confident wrong answer about a purchase order costs more than a hundred "not found in the indexed sources".

Use case 3 — Agents working together: Jira, Confluence, testing, releases

This is the interesting one, and the one most teams get wrong by starting with five agents instead of one. The working shape is a supervisor with specialists, not a democracy of peers: one orchestrator owns the plan and the state, specialists own narrow skills, and the orchestrator decides what happens next.

The flow, ticket to production

1. Triage. Orchestrator pulls a Jira issue via webhook, reads the description and linked Confluence page, and writes an execution plan: which specialists, in what order, what "done" means.
2. Spec. Spec agent turns the story into acceptance criteria and an explicit list of files it expects to touch. This is the contract everything else is measured against.
3. Build. Code agent works on a branch, produces a diff plus tests, and must run the test suite locally before handing over. No diff, no handover.
4. Verify. QA agent runs the pipeline, and on failure reproduces the bug in isolation, clusters the failures and writes a diagnosis. It may not "fix" anything — separation of powers, cheaply enforced.
5. Document. Docs agent writes the Confluence update and release note from the diff and the plan, not from imagination.
6. Release. Release agent drafts the changelog, opens the MR, tags, and deploys to staging. Promotion to production waits for the human gate.

The orchestrator, stripped down

class Squad:
    """Supervisor loop. State lives on disk, not in the context window."""

    def __init__(self, ticket, budget_tools=60, budget_usd=3.00):
        self.state = {"ticket": ticket, "plan": [], "artefacts": [], "spent": 0.0}
        self.max_tools = budget_tools
        self.max_usd = budget_usd

    def run(self):
        self.state["plan"] = planner(ticket=self.state["ticket"])   # tool call
        for step in self.state["plan"]:
            agent = AGENTS[step["agent"]]                             # spec|code|qa|docs|release
            result = agent.invoke(step, context=self.state)            # typed in, typed out
            self.state["artefacts"].append(result)
            self.state["spent"] += result.cost
            if not result.ok or self.state["spent"] > self.max_usd:
                return self.escalate(result)                          # brake, not retry-forever
            persist(self.state)                                       # survives a crash
        return self.state["artefacts"]

    def escalate(self, result):
        jira.comment(self.state["ticket"], summary=result.summary,
                     artefacts=result.links, needs="human decision")
        return {"status": "escalated", "why": result.reason}

Tool contracts: the part everyone skips

Write the tool layer before you write a single prompt. Every tool: typed arguments, validated response, an audit log line, and an idempotency key so a retried call cannot open the same ticket twice.

That last row is the whole design. The agent does not have a production deployment tool, so "the agent deployed to prod" is not a failure mode — it is an impossibility. Capability is a security boundary, not a prompt instruction.

Evaluation, or you are flying blind

Keep a small, growing set of real tickets with known-good outcomes — 30 is enough to start. Every change to a prompt or a model re-runs the set. Score three things:

• Task success — did the artefact pass the same review a human's would? (merged MR, accepted ticket, green suite)
• Regression — did it break something that used to pass? Track flaky vs real.
• Cost and steps — tokens and tool calls per accepted ticket. A 10 % quality gain for 3× the cost is usually a losing trade in production.

Version the eval set in the repo, next to the prompts, and make the pipeline fail when success drops more than a couple of points. This is the difference between an engineering programme and a science project.

The operating model nobody writes about

Four questions decide whether any of the above is still running in a year:

1. Who owns it? A named person, in a real team, with the pager. "The AI CoE" is not an owner.
2. What does it cost per month, and who approves that number? Model spend is an operating expense with a budget line, not a pilot footnote.
3. How do you switch it off? A feature flag per agent, default off, documented in the runbook. If disabling it breaks a process, you built a dependency without a plan.
4. What is it not allowed to do? Written down, enforced in code, tested like any other control.

Ship checklist

---

The next sections go deeper on the hardware side: the reconnaissance drone build, and on the programme side: S/4HANA migration decisions. If you want the agent architecture reviewed against a specific programme, the contact section is the shortest path.