Inside the AgentOps Trenches: An Interview with an Enterprise AI Lead on Shipping Production Agents

An Algorithmine interviews feature. Q&A with Maya Chen, Head of Applied AI at Meridian Analytics.

Meta description: An enterprise AI lead gets candid about what it really takes to ship and operate production AI agents in 2026 — failures, evaluation, observability, guardrails, team structure, and the true cost of AgentOps.

For the better part of two years, the loudest conversation in AI has been about what agents could do. Chatbots auto-resolved tickets on stage; slide decks showed fleets of autonomous workers handling customer support, underwriting, and back-office operations. In 2026, that conversation has moved somewhere far less glamorous and far more consequential: production. The teams that actually shipped agents are no longer talking about demos. They're talking about error rates, token budgets, human escalation loops, and the quiet discipline required to keep a fleet of autonomous systems from burning the company's money and its customers' trust.

Maya Chen is exactly that kind of practitioner. As Head of Applied AI at Meridian Analytics — a large US enterprise software firm — and previously a leader in the applied-AI group of a top-twenty financial services company, Maya has spent 2024 through 2026 taking agents from promising prototypes to workloads that real customers depend on. She agreed to talk to us with the kind of bluntness you rarely hear from vendor marketing materials.

The core thesis of this interview: AgentOps — the discipline of operating agents reliably at scale — is where the real margin and the real risk live. The models matter, but they're no longer the hard part. Everything around them is.

How production agents became a real workload

Algorithmine: You've been shipping agents since before it was fashionable. When did "agents" stop being a demo and become a real workload for you?

Maya Chen: Around the middle of 2024, honestly. The inflection wasn't a model release — it was when we stopped asking "can the model do this task?" and started asking "can we run it ten thousand times a day without supervision?" Those are completely different problems.

A demo agent completes one task with a human watching. A production agent completes a task thousands of times, with nobody watching most of the time, and the failures show up in a Slack alert at 3 a.m. The models got good enough in 2024 that the bottleneck stopped being intelligence and became operations. That's exactly when I stopped calling it "building AI" and started calling it "running AI."

A: What does "production agent" actually mean in your world? Everyone defines it differently.

MC: For me, three things have to be true. First, it's attached to real systems — it can call APIs, read and write to databases, send messages, do something with consequence. Second, it runs on a schedule or in response to real events, not because a human clicked "run." Third, and this is the one people miss — it has a cost and a failure mode that we measure continuously. If you're not tracking how often it succeeds and what it costs when it doesn't, it's not in production. It's a script with better vibes.

Right now at Meridian I operate a portfolio of agents across support triage, internal knowledge retrieval, document processing, and a few customer-facing assistants. We're talking hundreds of thousands of agent executions a month across the fleet. Volume is what forces discipline.

A: You mentioned the shift from MLOps to AgentOps. Is that a real distinction or just a buzzword?

MC: It's real, and it's not subtle. MLOps was built around serving a model: you train or fine-tune, you serve an endpoint, you monitor latency and drift. A model is a stateless function — input in, output out.

An agent is not stateless. It loops. It calls tools. It makes decisions between calls. It can get stuck in a loop, spend your entire budget on one task, or produce a confident but completely wrong answer. So the operating concerns change completely. With MLOps you worry about model latency and accuracy. With AgentOps you worry about agent state, tool-call traces, token spend per task, loop detection, and escalation behavior. That's a different discipline, and the tooling reflects it.

AgentOps vs MLOps: side-by-side diagram comparing model serving with agent operation across plan, tool calls, loop detection, and escalation layers

The biggest failures — and what they cost

A: You're known for being candid about failures. What's the most expensive mistake you've shipped?

MC: The one that taught me the most was a customer-facing support agent in financial services. We'd validated it on a test set, it looked great, and we let it handle a real tier of inbound tickets with only light supervision. The model was strong, so most of the time it was fine. But it had a failure mode: when it hit a genuinely ambiguous request, instead of escalating, it filled in the gaps with a plausible-sounding answer. Wrong account. Wrong balance. Confidently.

We caught it before it did lasting damage, but that incident cost us weeks of trust with the business line, a full re-audit, and a very uncomfortable retro. The dollar figure was real, but the real cost was that the stakeholder who'd championed the project nearly pulled the funding. The most expensive failure wasn't the bad answer — it was losing the belief that the system could be trusted.

A: What patterns do you see failing most often in production?

MC: I'd rank them. First, silent confidence — the agent produces a wrong answer and nothing flags it, because nobody wired up verification. Second, agent loops and spirals — the agent gets stuck repeating tool calls, burning tokens and money while you sleep. Third, scope creep — the agent does more than the task asked, touching systems it shouldn't. And fourth, cost blowups — a single long agent run that costs more than the human task it replaced.

Every one of these is an operations problem, not a model problem. The models are impressive. What kills you is that you've handed autonomy to something with no instinct for cost or consequence, and you forgot to add those boundaries yourself.

Cost-of-failure breakdown chart showing the four production agent failure modes: silent confidence, loops and spirals, scope creep, and cost blowups

Evaluation is the real bottleneck

A: You've said the "eval problem" is the hardest part of AgentOps. Walk us through that.

MC: People assume evaluation is a solved problem because we had it in ML. It isn't. With a model you evaluate offline: you hold out a test set, measure accuracy, and you're mostly done. With an agent, offline evaluation is almost worthless for predicting production behavior, because the agent's behavior depends on the path it takes — what tools it calls, what order, how it reacts to tool responses.

So we had to build production-grounded evaluation. We don't just measure outcomes. We trace each agent run and score the reasoning path, the tool-call sequence, and the final result. And crucially, we measure what I think of as the human-edit rate — how often a human has to change what the agent produced before it ships. That's the single most honest signal of quality, because it's the ground truth of whether your output was actually acceptable.

A: What metrics should a team that's starting AgentOps track from day one?

MC: Start small and make them parasitic — piggyback on the humans who already review the work. Track four things:

Completion rate — did the agent finish the task end to end without intervention?
Human-edit rate — of the completed tasks, how many needed a human change?
Escalation rate — how often did the agent correctly hand off to a human because it knew it was out of its depth?
Cost per resolved task — total tokens and infra divided by tasks actually completed successfully.

That last one is the one everyone forgets. A cheap agent that fails often is more expensive than an expensive agent that rarely needs rework. If you optimize token price alone, you'll optimize straight into a quality cliff.

Agent evaluation funnel: agent runs flowing through completion, human-edit, escalation, and cost-per-task metrics with pass and fail thresholds

Observability and how she debugs agents

A: When an agent misbehaves at 3 a.m., how do you actually debug it?

MC: You need a trace. Period. If you can't replay what the agent thought, called, and decided at every step, you're debugging blind. For every run we capture the reasoning, every tool call with inputs and outputs, the token spend at each step, and the final state. When something breaks, we can pull it up like a debugger view and see exactly where the path went wrong.

A: Build or buy for AgentOps tooling?

MC: It's a spectrum, and I've done both. When the workflows are idiosyncratic — and ours were — you end up building the layer that's specific to your systems: the eval harness, the escalation routing, the integration with your approval and audit systems. The generic observability and tracing pieces, you can absolutely buy, and you should, because building telemetry from scratch is a time sink.

The rule I give my team: buy the commodity, build the moat. Don't build a tracing framework — that's commodity. Do build the evaluation logic that encodes your definition of a good outcome, because that's your proprietary understanding of your own business, and it's hard to replicate.

A: Anything you'd tell teams to skip?

MC: Skip the dashboard that shows you green checkmarks. If all your metrics are green, either you're doing something unbelievably well, which is rare, or — far more likely — you're measuring the wrong things and hiding the real problems. The most valuable observability output I get is the red — the escalation spikes, the edit-rate creep, the unusual token spend. That's where the signal lives.

Guardrails, safety, and human escalation

A: Where do guardrails live in your stack, and how strict are they?

MC: Guardrails aren't a single layer, they're a set of fences at different distances. There are hard, deterministic fences at the system level — an agent physically cannot touch certain APIs, write to certain databases, or exceed a token budget. Those are non-negotiable and enforced in code, not by the model.

Then there are softer, model-level fences — instruction-based boundaries about scope and tone. Those are useful but fallible, so they can't be your only defense.

The third fence is human escalation, and it's the one that matters most for production. We define explicit conditions under which an agent must hand off: low confidence, high stakes, sensitive data, regulatory territory, or cost above a threshold. A well-calibrated agent knows when it's out of its depth. The ones that get it right on the escalation call are the ones I trust with autonomy elsewhere.

A: How does compliance and audit fit in?

MC: Heavily. In financial services especially, everything an agent does has to be explainable and logged for audit. That's not a burden on top of the work — it is the work. We log every step specifically so that, two years from now, if a customer or a regulator asks why the agent made a decision, we can answer with a complete, human-readable record. That auditability turned out to be the thing that unlocked approval for autonomy. You can't give an agent freedom you can't explain. And the moment you can explain it, governance stops being a blocker and becomes your license to operate.

Layered guardrail diagram showing three fences: system-level hard rules, model-level soft policies, and human escalation thresholds protecting the agent core

Team structure, costs, and the ROI question

A: How do you staff an AgentOps team? These are new skills.

MC: You don't need an army, and you don't need everyone to be an LLM researcher. I run a small, senior team. The roles that matter:

Prompt/agent engineers who understand how to build reliable agent logic on top of models.
Platform engineers who own the infra, tooling, and integrations — these are the people who make agents actually connect to your systems.
Evaluation specialists who build and maintain the eval harnesses. This is the most underrated role; it's what decides whether you can trust scale.
An applied researcher who understands model behavior well enough to know when a problem is a prompting issue vs a genuine model limitation.

The through-line is reliability engineering, not ML research. I tell people: agent engineers are more like site reliability engineers who happen to be fluent in AI than they are like ML researchers.

A: Let's talk unit economics. What does a production agent actually cost?

MC: Cost per resolved task is the number I live on. It's the sum of model inference across every reasoning and tool-call step, tool execution, human oversight minutes, and rework. Reasoning-heavy agents that plan and verify cost more per task — but they replace higher-value work, so the comparison against the human alternative is what matters.

Two things surprise people. First, reasoning tokens dominate. The agent's internal planning can cost several times the final answer's tokens. Second, failure is the real expense. A task that fails and has to be re-run at human cost can wipe out the savings from ten cheap successes. That's why I'm allergic to optimizing for the cheapest possible tokens — you save pennies and pay dollars in rework.

A: When is an agent NOT worth it?

MC: Be honest about that, because it saves you from expensive hobbies. If the task is fully deterministic and scriptable, a script beats an agent — you don't pay reasoning tokens and there's no hallucination risk. If the task is so rare or so unpredictable that you can't easily evaluate success, you can't build confidence either. And if the cost of a single error is catastrophic, no amount of guardrail enthusiasm is going to make autonomy comfortable. Agents shine in high-volume, semi-structured tasks with well-defined success criteria and affordable failures. Point them there and let them earn their keep.

Advice for teams starting now

A: If a company is shipping agents for the first time in 2026, what's your one piece of advice?

MC: Instrument before you automate. Wire up tracing, evaluation, and escalation before you let the agent touch anything real. It's boring, it's unglamorous, and it's the difference between a smooth rollout and a trust-destroying incident. The teams I see succeed are the ones who treat the first sixty days as an operations project, not an AI project.

A: What would you have done differently?

MC: Started the human-edit-rate measurement earlier. It's embarrassing in hindsight — we had great latency dashboards and almost no honest output-quality signal for the longest time. I'd also have fought harder for senior evaluation talent up front instead of treating it as a junior task. It's not. Evaluation is the thing that decides whether your AgentOps bet pays off, and it deserves your best people.

Key takeaways

AgentOps is an operations discipline, not a model problem. The hard part of production agents is state, tool calls, cost, and escalation — not intelligence.
Measure the honest signals: completion rate, human-edit rate, escalation rate, and cost per resolved task. Ignore vanity dashboards.
Trace everything. If you can't replay an agent's reasoning and tool calls, you can't debug it.
Guardrails are layered fences + human escalation, and auditability is what unlocks autonomy.
Buy the commodity, build the moat — buy telemetry, build the evaluation logic that encodes your definition of a good outcome.
Cost per resolved task beats cost per token. Failures are the real expense.

Want more practitioner-grounded analysis like this? Subscribe to the Algorithmine portal for deep interviews, research, and market briefs on the people actually building and operating production AI systems.

FAQ

Q: What is AgentOps, and how is it different from MLOps? A: AgentOps is the discipline of operating autonomous AI agents reliably in production — handling agent state, tool calls, token spend, loop detection, evaluation, and human escalation. MLOps serves stateless models; AgentOps operates stateful, decision-making systems that take actions, which changes the monitoring and reliability concerns entirely.

Q: How do you measure whether a production agent is actually good? A: Track completion rate, human-edit rate (how often a human must change the output), escalation rate (how often the agent correctly hands off), and cost per resolved task. Human-edit rate is often the most honest quality signal because it's ground-truth acceptability.

Q: What's the most common reason production agents fail? A: Silent confidence — an agent produces a confidently wrong answer and nothing is wired up to flag it. Loop/spiral behavior, scope creep, and cost blowups are close behind. All are operational problems, not model intelligence problems.

Q: How much supervision do production agents need? A: It depends on task stakes and error cost. The winning pattern is targeted autonomy: full autonomy for high-volume, affordable-failure tasks, and strict human escalation for high-stakes or low-confidence ones. Oversight should be calibrated per task, not per platform.

Q: Do production agents save money? A: Yes, when pointed at the right workloads, but the economics are about cost per resolved task, not token price. Reasoning tokens and rework from failures dominate costs. Agents earn their keep in high-volume, semi-structured tasks with clear success criteria and affordable failures.