Inside a B2B Team That Replaced First-Call SDR Qualification With an AI Agent
Inside a B2B SaaS team that moved first-call SDR qualification to an AI agent—what broke, what held, and the cost and acceptance numbers.
This is a structural story, not a tooling story. A mid-market B2B SaaS company—call them Northwind, a 180-person vendor selling to operations leaders at $28K average contract value—removed human SDRs from first-call qualification entirely. An AI agent now owns the stage that used to consume the first 15 minutes of every SDR conversation: responding to inbound, running discovery-style qualification, scoring fit, and booking the meeting.
We spoke with Northwind's VP of Revenue Operations and one of its two remaining SDRs (now working as a "conversation designer" and QA lead) about what the change actually required. The interview was candid about the failures. The agent over-qualified early, hallucinated a competitor comparison, and triggered a rep revolt in week three. It also cut cost per qualified meeting by more than half and improved AE acceptance of booked leads, measured over two full quarters.
The stakes matter for anyone reading this: moving qualification to an AI agent changes the shape of the GTM org, not just the top of the funnel. Here is what that looked like inside one team. (See also: How AI SDR Agents Work: A Practical Guide for Revenue Teams)
The Setup — Why This Team Ripped Out First-Call SDR Qualification
The Team and the Problem
Northwind runs a mid-market motion with a six-person SDR team, eight AEs, and roughly 1,400 inbound and outbound-sourced conversations per quarter. The funnel was not broken in the dramatic sense—it was expensive and inconsistent.
Three problems compounded:
- SDR ramp time. New SDRs took 11–14 weeks to reach quota, and two of the last five hires washed out before month six.
- Inconsistent qualification quality. AEs complained that "qualified" meant something different depending on which SDR booked the meeting. Some leads arrived with a clean BANT note; others arrived with a calendar invite and a one-line description.
- Cost per qualified meeting. Fully loaded, Northwind was spending roughly $410 per qualified meeting once SDR salary, tooling, and management overhead were allocated.
First-call no-shows hovered around 18%, and AE feedback on SDR-booked meetings ran to a recurring phrase in call reviews: "this wasn't a real opportunity."
The Trigger for Change
The catalyst was a hiring freeze, not an AI initiative. In Q1, Northwind's CFO froze two open SDR requisitions and asked RevOps to hold pipeline coverage with the existing team. The VP of RevOps had already been running a small experiment: an AI voice and chat agent handling after-hours inbound only, booking meetings without human involvement.
That pilot booked 44 meetings in six weeks. AEs accepted 31 of them as legitimate first calls—a 70% acceptance rate against a 54% baseline from human SDRs in the same period. Read that as a directional signal, not a benchmark: 44 meetings is a small sample, and after-hours inbound skews toward higher intent than the blended human-SDR pool, so some of the gap is selection, not agent skill. It was enough to justify a real test, not enough to prove a result.
"We didn't set out to remove SDRs. We set out to survive a hiring freeze. The pilot made the harder question unavoidable: if the agent can do first-call qualification at 70% acceptance, what exactly are we paying six people to do in that first call?" — VP of Revenue Operations
How the AI Agent Actually Works
What the Agent Handles Before a Human Ever Joins
The agent owns the entire pre-AE stage. Concretely, it handles:
- Inbound response across web chat, inbound phone, and form-triggered callbacks, with a target response time under 30 seconds. On voice, the team budgets separately: speech-to-text plus LLM turn plus text-to-speech has to land under roughly 1.2 seconds of perceived latency per turn, or callers talk over the agent. They hit that budget with streaming STT, a fast model tier for conversational turns, and a slower model tier only for end-of-call summarization.
- Discovery-style qualification using a custom rubric derived from the team's own BANT variant—budget signal, authority (who the prospect says owns the decision and who signs), timeline, and a firmographic/technographic fit check against ICP criteria. The agent surfaces stated authority signals; it does not verify them.
- Fit scoring against a weighted rubric: company size, industry, current stack, stated pain, and urgency. Scores below threshold route to nurture; scores above route to scheduling.
- Scheduling and routing directly onto AE calendars, with routing rules by territory and segment.
- Context capture: a structured summary, the full transcript, rubric answers, and a fit score land in the CRM before the AE opens the record.
The model layer is a general-purpose LLM with a retrieval layer over approved content—product docs, pricing guardrails, competitive positioning. The agent does not improvise pricing. It is scoped to answer from retrieved content and escalate anything outside that scope. (Related: Designing a Lead Qualification Rubric That AI Can Follow)
The Handoff Logic
Handoff is rule-based, not vibes-based. The agent escalates to a human when any of the following fires:
- Fit score above threshold and the prospect requests a meeting—routed to the assigned AE with full context.
- High-value signals (enterprise headcount, named-account match) trigger immediate escalation to a senior rep rather than standard routing.
- Escalation triggers from the eval-tuned rule set: retrieval similarity score below threshold, an out-of-scope classifier firing, a frustration/negative-sentiment signal, or a repeated-question loop. There is no single "confidence" scalar—these are heuristics tuned against a labeled test set of past conversations, and they are re-tuned when the eval set shows drift.
- Explicit request: "I want to talk to a person" ends agent handling immediately.
Context transfer is the part the team underestimated. Early versions passed a transcript and a score. AEs ignored both. The version that worked passed a one-screen structured summary: rubric answers in plain language, the prospect's own words on pain, the fit score with the top two contributing factors, and a recommended next step. AEs read that. They did not read transcripts.
What Broke, What Held, and What They Learned
Failure Modes They Hit in the First 60 Days
The first two months were rough, and the team is specific about why.
- Over-qualification. The first rubric was tuned too tight. The agent disqualified prospects who were a strong fit but described their budget in non-standard terms ("we'd need to loop in finance" read as no budget). Fix: the rubric was rewritten to score signals rather than answers, and a human reviewer sampled 50 disqualified conversations per week for the first month. Disqualification precision went from roughly 60% to 88% over six weeks.
- The competitor hallucination. Asked to compare Northwind to a named competitor, the agent pulled an adjacent document from the retrieval index—an internal positioning memo that referenced an older pricing model—and synthesized a comparison that was partly wrong. The fix was structural, not a prompt tweak: competitive answers now require a retrieved passage that directly addresses the named competitor, the agent must cite which document it used, and if retrieval returns nothing above the similarity threshold it abstains and escalates. A regression eval of 120 scripted competitor questions runs on every content and model change.
- The week-three rep revolt. AEs had stopped trusting booked meetings after two bad ones landed on the same day. The team paused routing for 48 hours, put the AE acceptance definition in writing (accepted = first call held and not disqualified within five business days), and gave AEs a one-click "this wasn't qualified" flag that fed directly into the weekly QA review. The flag did more for trust than any accuracy improvement, because it gave AEs a channel that visibly changed the rubric.
What Actually Moved
Two numbers held up over two full quarters:
- Cost per qualified meeting fell from roughly $410 to under $190 fully loaded. That figure includes inference, telephony, retrieval infrastructure, and the two remaining SDRs' time spent on conversation design and QA—not just the model bill. The agent's raw inference and telephony cost per booked meeting was a small fraction of the delta; most of the savings came from removing SDR headcount and management overhead.
- AE acceptance of booked leads rose from a 54% baseline to roughly 68% across the two quarters. The definition is the one above, applied consistently to both periods.
Containment—the share of conversations the agent handled without a human joining—settled around 78%. The remaining 22% escalated for the reasons in the handoff logic, and the team treats that number as a health metric, not a target to minimize.
What They'd Tell the Next Team
- The rubric is the product. The model is a commodity; the weighted rubric, the retrieval corpus, and the eval set are the actual asset. Budget most of the work there.
- Measure acceptance the same way before and after. The team nearly fooled itself by changing the acceptance definition mid-rollout. Lock the definition first.
- Give AEs a veto that visibly changes behavior. Trust is built by showing reps that their feedback alters the rubric, not by improving accuracy in a dashboard.
- Keep a human on the eval loop. Two conversation designers running weekly QA on a sampled set of calls caught more regressions than any automated monitor.
The org that came out the other side is smaller at the top of the funnel and deeper in QA and conversation design. That is the structural change—not the agent itself, but where the humans went.