Can an AI judge grade a phone call? A calibration guide

Thousands of voice-agent calls are piling up, and the review queue grows faster than people can listen. An AI judge looks like the obvious answer: give a model the transcript and a rubric, then turn every call into a score.

That can help. It can also create a confident, automated version of one reviewer’s blind spots.

This guide is for voice-AI teams that want the scale of model-assisted review without pretending the model is ground truth. You will learn how to define a judge’s job, build a human-labeled calibration set, measure the errors that matter, handle uncertain cases, include audio evidence, and decide which judgments can affect a release.

The short answer

An AI judge can grade narrow, well-defined aspects of a phone call after it is calibrated against human decisions for that exact workflow. It should not be the only authority for critical policy, exact tool state, money movement, identity, or final business outcome.

Use deterministic checks for facts that already exist in structured data. Use a calibrated judge for meaning that needs interpretation. Route uncertain, high-risk, and disagreement cases to humans. Recalibrate whenever the model, prompt, rubric, evidence, language, or workflow changes.

Our opinion at Voxeval is blunt: an uncalibrated judge is a demo, not a control.

A judge needs a job description

“Grade this call” is not a job description. It asks one model to infer quality, policy, intent, evidence, and severity at the same time.

Split the review into named criteria. For a fictional insurance-status agent, those criteria might be:

• Did the agent explain the current claim status accurately?
• Did it avoid promising an approval date not present in the claim record?
• Did it ask the caller to repeat information unnecessarily?
• Did it explain the human handoff clearly?
• Did it interrupt the caller during a critical identifier?

This is a fictional example, not a Voxeval customer case.

Each criterion gets its own evidence, applicability rule, verdicts, and consequence. “No unnecessary repetition” may be a quality signal. “No invented approval date” may be a critical gate.

Scroll diagram horizontally on smaller screens.

Decide what should not use an AI judge

Before writing the prompt, remove checks that code or authoritative state can answer better.

Use deterministic evaluation for:

• Tool name, arguments, order, and call count
• Exact entities such as dates, IDs, amounts, and destinations
• Required event ordering
• Whether a disclosure token or approved phrase occurred when exact language is mandatory
• Final database, payment, scheduling, or ticket state
• Transfer destination and connection result
• Latency and interruption timing

An AI judge can explain those artifacts, but explanation is not verification. If the booking system says no appointment exists, a fluent judge summary should not overrule it.

Use model judgment for criteria such as clarity, relevance, whether a correction was acknowledged, whether a response was grounded in supplied evidence, or whether a repair forced the caller to restart needlessly.

Even then, keep the question narrow.

Write an evidence contract

The judge should receive only the evidence it is allowed to use.

For “the status explanation matched the claim record,” provide the relevant claim fields and the aligned agent statement. Do not provide a giant record and hope the model finds the right field.

For “the agent interrupted a critical identifier,” a cleaned transcript is insufficient. Provide aligned audio or timed voice-activity and playback events. A transcript may place the turns neatly one after another even when the caller heard overlap.

An evidence contract should name:

• The criterion
• When it applies
• Allowed evidence
• Required evidence that makes the item gradeable
• pass, fail, and not_applicable definitions
• An uncertain or insufficient_evidence outcome
• Severity and routing
• Examples near the boundary

Do not force a binary answer when the evidence is missing. Abstention is a feature.

Build a human-labeled calibration set

The calibration set teaches you what the judge actually does. It should not be a collection of obvious happy paths.

Include:

• Clear passes and clear failures
• Cases near each rubric boundary
• Short and long calls
• Corrections, interruptions, and failed tools
• Different caller speech patterns and supported languages
• Cases with missing or conflicting evidence
• Rare critical failures
• Calls likely to tempt style, verbosity, or position bias

Use real calls only when your organization has the right consent, purpose, access, and retention controls. Otherwise build approved fixtures that preserve the behavior without preserving the person’s data.

Have people label independently

At least two qualified reviewers should label a useful slice without seeing the model verdict. Ask them for the verdict, evidence span, and reason. Adjudicate disagreements with the domain owner.

Human disagreement is not noise to delete. It can reveal an unclear rubric, missing evidence, or a policy question the business has not answered.

The final calibration label should record the adjudication and source rule. A majority vote among reviewers who interpreted the rule differently is weak ground truth.

Measure errors by consequence

Overall agreement is easy to report and easy to misuse. If 95% of calls are compliant, a judge that says “pass” every time reaches 95% accuracy while catching none of the failures.

Build a confusion matrix for each criterion:

	Human pass	Human fail
Judge pass	True pass	Dangerous miss
Judge fail	False alarm	Correct catch

Then calculate the measures that match the job:

• Recall for failures: Of the human-labeled failures, how many did the judge catch?
• Precision for failures: Of the calls the judge flagged, how many were actual failures?
• False-negative rate: How often did it let a human-labeled failure pass?
• Abstention rate: How often did it send a case for review?
• Agreement by subgroup: Does performance change by language, call length, scenario, audio condition, or evidence quality?

Google’s classification guidance explains why precision, recall, and thresholds move together and why class imbalance makes plain accuracy weak. That guidance is general machine-learning education, not a voice-agent release standard. Apply the concepts with your own error costs.

For a critical disclosure judge, missing a real failure may be worse than reviewing extra false alarms. For a low-risk tone flag, the tradeoff can reverse.

Set the threshold from the error cost

Some judge outputs are categories. Others include scores or confidence. Do not accept the default cutoff without testing it.

Plot precision and recall across thresholds on a held-out set. Pick a threshold that satisfies the lane’s safety and review-capacity needs. Add a review band instead of forcing borderline scores into pass or fail.

For example:

• Score at or above 0.90: auto-pass for a low-risk clarity criterion
• Score from 0.65 to 0.89: human review
• Score below 0.65: fail or human confirmation, depending on consequence

Those numbers are an illustration, not a recommendation. Model confidence may not be calibrated probability. Test the behavior empirically.

Give the judge a rubric it cannot improvise

A strong judge prompt is compact and operational:

1. State the single criterion.
2. State the applicable policy or expected behavior.
3. Define every verdict.
4. List allowed evidence.
5. Tell the judge to abstain when required evidence is absent.
6. Require evidence references and a short reason.
7. Return a strict schema.

Use boundary examples from adjudicated cases. Do not stuff dozens of unrelated policies into one prompt.

LiveKit’s current test framework includes preset model judges for accuracy, coherence, conciseness, handoff, relevance, safety, task completion, and tool use. It also allows deterministic custom judges. These are useful building blocks, not calibrated proof for your domain. Measure their agreement with your reviewers before a preset verdict blocks a release.

Test the judge for known failure patterns

LLM judges can prefer longer answers, writing styles, familiar model outputs, or one position in a pairwise comparison. They can also rationalize a verdict after missing the key evidence.

G-Eval reported stronger alignment with human judgments than earlier automatic metrics for its summarization and dialogue tasks, while also flagging possible bias toward model-generated text. That 2023 result does not establish reliability for phone-call policy evaluation.

A 2025 preprint, Are We on the Right Way to Assessing LLM-as-a-Judge?, reports consistency problems for leading models on difficult judge cases under its Sage framework. It is a preprint with a specific benchmark, not a universal failure rate. The right takeaway is to test consistency rather than assume it.

Add adversarial calibration cases:

• Same answer, shorter and longer wording
• Same evidence, verdict options reversed in order
• Agent and human names swapped
• Irrelevant confident explanation added
• Missing evidence paired with a plausible conclusion
• Judge model grading output from its own model family
• Equivalent meaning with different dialect or grammar

Run the same item more than once when judge sampling or service behavior can vary.

Include audio when the criterion lives in audio

A phone call is not a transcript with a voice attached.

Audio review is needed for:

• Overlap and interruption
• Clipped words
• Long or awkward silence
• Pronunciation of names, amounts, and medical terms
• Prosody that turns a question into apparent confirmation
• Background speech or echo mistaken for the caller
• Whether the caller could hear a required statement

An audio-capable model can help triage these cases, but it needs its own calibration against human listening. Do not assume a multimodal judge understands telephony artifacts, mixed speakers, or timing because it accepts an audio file.

EVA-Bench separates accuracy measures from experience measures and includes turn-taking and speech dimensions. It is a 2026 research preprint and work in progress. Its helpful design cue is to keep task correctness and call experience visible as separate judgments.

Version the entire judge

A judge version is more than a model name. Store:

• Model and provider version
• Prompt and rubric
• Evidence-selection logic
• Tool or retrieval configuration
• Output schema
• Temperature and sampling settings
• Threshold and review band
• Calibration-set version
• Calibration results by criterion and subgroup
• Date and approver

Changing any of these can change the verdict. Run the held-out calibration set before promoting a new judge version.

Do not silently re-grade historical calls with a new judge and compare the trend to old scores. Either keep the old judge for the series or backfill and label the break.

Put humans where they add the most value

Human review should concentrate on:

• Critical failures and potential incidents
• Judge and deterministic-check disagreement
• Low-confidence or abstained cases
• New workflows and unsupported subgroups
• Random samples of auto-passed calls
• Drift in production distributions
• Appeals from operators or customers

Random review matters. If people inspect only judge failures, the team cannot estimate what the judge is missing.

Use reviewer feedback to update the rubric and calibration set before changing the agent. A judge bug can look like an agent regression.

A practical promotion rule

Before a judge can affect release status, require:

• Approved criterion and evidence contract
• Human-labeled development and held-out sets
• Acceptable failure recall and precision for the risk tier
• Subgroup results with no hidden critical gap
• Tested abstention and missing-evidence behavior
• Repeated-run consistency inside the approved budget
• Versioned prompt, model, threshold, and dataset
• Named human-review and rollback owners

Some criteria should remain advisory even after calibration. A tone judge can help sort calls without becoming a hard gate.

The voice-agent scorecard shows where deterministic, model, and human judgments fit across ten metric families. The SOP conversion guide helps define the policy source before calibration begins.

Where Voxeval fits

Voxeval is being built to store judge versions, human labels, evidence, disagreements, and calibration results beside the agent evaluation. That makes it possible to ask not only “did the call fail?” but “how trustworthy was the system that made that claim?”

The goal is not to remove humans from evaluation. It is to spend human attention on ambiguity, risk, and drift while machines handle repetitive, proven checks.

Common questions

How many human-labeled calls are needed to calibrate a judge?

There is no universal minimum. You need enough examples to cover each verdict, boundary, critical failure type, and supported subgroup, plus a held-out set large enough to estimate the error rate that drives your decision.

Should the judge see the full transcript?

Only when the criterion requires it. Narrow evidence reduces distraction and makes reasons auditable. Keep enough surrounding context to avoid changing the meaning.

Can the same judge grade every workflow?

The model can be shared, but the rubric, evidence, calibration data, thresholds, and performance claims should be workflow-specific. A judge calibrated for appointment clarity is not validated for payment authorization.

Can AI-judge scores be averaged into one quality number?

They can be summarized for exploration, but critical criteria should remain visible and may need hard gates. One average should not let strong tone hide a fabricated outcome or policy violation.