The AI Ops Team of One: How a Solo Platform Engineer Runs 40 Agents in Production
How one platform engineer runs 40 AI agents in production alone—shared primitives, guardrails, evals, and observability that make a team of one work.
One engineer. Forty AI agents. Production SLOs. A documented escalation path instead of a 24/7 pager rotation.
That's the operating reality for the platform engineer we're calling "Mara" (a pseudonym; her employer asked us not to name the internal system). She owns the AI agent platform for a mid-size B2B SaaS company — roughly 1,200 employees, a customer base in the tens of thousands — and she is the only person whose full-time job is running the agents themselves. Forty of them are in production today, handling support triage, code review, data enrichment, and a long tail of internal workflows. (All figures here are self-reported by the operator.)
This isn't a story about heroics or 80-hour weeks. It's a story about an operating model for AI agent operations: what she built, what she deliberately refused to build, and how observability, standardization, and a clear escalation path let one person hold the line. We spoke with her over two sessions about architecture, alerting, evals, and the buyer conversations she keeps having with engineering leaders who want to copy the model. (See also: AI agent observability: what to instrument first.)
Meet the Operator: The Interview Setup
Who They Are and What "Team of One" Actually Means
Mara has been a platform engineer for eleven years, the last three focused on AI infrastructure. She reports into the platform organization, but her scope is narrow and specific: the agent runtime, the guardrails, the observability stack, and the lifecycle of every agent in production.
To be honest about the math, "team of one" excludes the shared infrastructure she rides on — Kubernetes, the cloud account, the identity provider, and the data warehouse are all run by other teams. She also leans on two part-time contributors who review prompt changes and serve as backup approvers. But no one else is on the hook for agent reliability, and there is no rotation behind her. Her coverage model is business-hours ownership with a documented escalation path into the platform on-call for infrastructure failures — more on that below.
The 40-Agent Portfolio at a Glance
The agents fall into four functional buckets:
- Customer operations (~14): support ticket triage, draft responses, refund eligibility checks, escalation routing.
- Engineering productivity (~11): code review comments, test-gap flagging, changelog drafting, dependency-update triage.
- Data and enrichment (~9): entity resolution, CRM hygiene, account scoring, document extraction.
- Internal workflows (~6): procurement summaries, meeting-note routing, policy Q&A.
Most are low-risk and read-only. Six can take consequential actions — issuing refunds, merging code, writing to the CRM — and those carry approval gates. "The portfolio is deliberately lopsided," she said. "The read-only agents are where the platform earns its keep. The risky ones get the scrutiny." (Related: How to tier AI agent risk before you scale.)
Why a Team of One Is a Design Choice, Not a Staffing Gap
The Constraints That Forced the Architecture
Mara didn't set out to run 40 agents alone. The first six arrived organically as teams asked for automation, and she quickly realized that per-agent bespoke builds would not scale past ten. "Every agent was a snowflake — its own prompt store, its own retry logic, its own way of calling tools," she said. "I was the human integration layer."
The constraint that changed everything was headcount, not budget. Leadership offered to hire, but Mara argued for platform investment instead: a shared gateway, a common memory layer, a tool registry, and a single observability pipeline. The pitch was simple — one well-instrumented platform could absorb new agents at marginal cost, whereas a team of five would still be maintaining snowflakes. (Background: Building a shared LLM gateway for multiple agents.)
What They Deliberately Did Not Build
The discipline shows in the deferral list. Mara has no custom chat UI (agents live in Slack, email, and existing internal tools). No per-team agent frameworks. No multi-region deployment — single region with a documented failover runbook. No fine-tuned models; everything runs on hosted APIs with prompt-level customization.
"Every feature I don't build is a feature I don't get paged about. The deferral list is the most important document I own."
She revisits it quarterly. So far, only one deferred item — a self-service prompt editor for product teams — has been pulled forward, and it came with a hard constraint: changes ship through the same eval gate as everything else.
The Architecture: Running 40 Agents Without a 40-Person Team
Shared Primitives: Gateway, Memory, and Tool Registry
Every agent calls the same three primitives. The LLM gateway handles provider routing, retries, rate limits, token accounting, and prompt-version pinning. The memory layer keeps two things distinct: short-term conversation state (scoped to a session) and long-term retrieval (a managed index), so no agent team rolls its own vector store. The tool registry is the important one: each tool is declared once with a schema, an owner, a rate limit, and a risk classification, and agents can only invoke tools they've been explicitly granted.
The result is that a new agent is mostly configuration. "Onboarding a new agent takes about two days, and one of those days is writing evals," Mara said. "The infrastructure part is an afternoon."
Standardizing Agent Templates and Guardrails
Agents are built from templates, not from scratch. A typical template — say, the "read-and-summarize with escalation" pattern used by most support agents — ships with:
- A system prompt scaffold with mandatory sections (role, tools, refusal behavior, escalation criteria).
- A tool allowlist enforced at the registry, not the prompt. This matters for security: prompt-level allowlists are bypassable via prompt injection, while a registry check is a hard control the model cannot talk its way past.
- Rate limits per agent and per downstream tool.
- An approval gate for any write action, routed to the owning team's queue with the proposed action, the diff it would apply, and a one-click approve/reject.
- Idempotency keys on every write call, so retries and duplicate invocations don't double-issue a refund or re-merge a branch. Effectively-once behavior comes from idempotency plus a dedupe window, not from trusting the model to behave.
- A per-agent kill switch that disables the agent at the gateway without touching its config, so a misbehaving agent can be stopped in seconds.
Prompt Versioning, Rollback, and the Eval Gate
Every prompt is a versioned artifact pinned by the gateway, and every change moves through the same pipeline:
- Offline evals against a labeled set per agent (golden cases plus regressions pulled from past incidents).
- Shadow/canary run on a slice of live traffic, with outputs compared against the current production version.
- Promotion only if eval scores hold and the canary shows no regression in error rate or escalation rate.
- Instant rollback by repinning to the prior version — a one-line change, not a redeploy.
"The gate is the thing that lets me sleep," she said. "I don't have to be in the room for every prompt change. I have to be in the room when the gate fails."
Observability and Alerting That a Solo Operator Can Actually Carry
The observability pipeline is the piece that makes the whole model work. Every agent invocation emits a structured trace — trace ID propagated through every tool call, model and prompt version, token counts, latency, tool outcomes, and the final disposition (auto-resolved, escalated, refused). Traces roll up into per-agent dashboards and, critically, into SLOs with error budgets.
Alerting is tiered so it doesn't page a single human for noise:
- Tier 1 (page Mara, business hours; escalate to platform on-call after hours): error-budget burn on the six write-capable agents, gateway failures, or a spike in approval-gate rejections.
- Tier 2 (ticket, next business day): quality drift on read-only agents, cost anomalies, latency regressions.
- Tier 3 (dashboard only): individual failed invocations, retry storms within tolerance.
"The trick is that most agents are allowed to fail quietly and retry," she said. "Only the six that can move money or code are allowed to wake anyone up."
Cost and Capacity Control
Token spend is metered at the gateway and budgeted per agent, with hard caps that fail closed — an agent that exceeds its monthly budget stops rather than silently burning spend. Per-agent and per-tool rate limits double as backpressure against downstream systems, so a runaway loop degrades one agent instead of taking out a shared dependency.
What Generalizes — and What Doesn't
The model is portable, but not every part of it transfers cleanly. What generalizes:
- A shared gateway, memory layer, and tool registry as the substrate for every agent.
- Registry-enforced tool permissions and idempotent writes as non-negotiable controls.
- An eval gate plus versioned prompts with instant rollback.
- Tiered, SLO-based alerting sized to what one owner can actually carry.
What's context-specific to Mara's shop: the single-region posture (acceptable given her failover runbook and read-heavy workload), the hosted-API-only stance (no fine-tuning), and the business-hours ownership model, which depends on a platform on-call team existing behind her. A team without that backstop needs a different coverage plan before it copies this.
The Buyer Conversation
Engineering leaders keep asking Mara the same question: how few people can run this? Her answer is consistent — the headcount question is the wrong first question. "People want to know if one person can run forty agents. The honest answer is that one person can run the platform and the agents if the platform does the boring work. If you skip the platform, you don't need one person — you need ten, and they'll all be firefighting."
The order of operations she recommends: build the gateway and tool registry first, instrument before you scale, put the eval gate in place before the second write-capable agent ships, and write the deferral list before anyone asks for a feature. The team of one is the output of those decisions, not the input.
Editor's note: figures are self-reported by an anonymized operator; the employer and internal system names are withheld at the source's request. Replace bracketed (#) links with final URLs before publish.