# SOP-to-voice-agent eval workshop template

Use this worksheet to turn one approved procedure into testable voice-agent release evidence. Copy it into your docs, invite the workflow owner, policy owner, engineering lead, and evaluator, then work from the source clause outward.

Aim for one workflow in a 60-minute session. Do not paste confidential production calls or personal data into the worksheet.

## 1. Name the release lane

- Workflow: `[workflow name]`
- Agent version: `[version or commit]`
- Channels and routes: `[phone, web, SIP route, region]`
- Languages: `[supported language lanes]`
- Workflow owner: `[name or role]`
- Policy owner: `[name or role]`
- Evaluation owner: `[name or role]`
- Decision date: `[date]`

## 2. Inventory the source material

Give every source a stable ID. Quote only the smallest clause needed to preserve meaning.

| Source ID | Approved source | Version/date | Clause or rule | Owner | Open question |
| --- | --- | --- | --- | --- | --- |
| SOP-01 | `[document or system rule]` | `[version]` | `[exact or tightly summarized rule]` | `[role]` | `[question or none]` |

Checklist:

- [ ] The source is approved for this release lane.
- [ ] Conflicting versions have been resolved.
- [ ] Required wording is marked as exact or meaning-equivalent.
- [ ] Permission boundaries and forbidden actions are explicit.
- [ ] Exceptions, hours, limits, and terminal states are present.

## 3. Extract the decision map

Write decisions before writing prompts or test utterances.

| Decision ID | Trigger | Required facts | Permitted action | Forbidden action | Next state | Source ID |
| --- | --- | --- | --- | --- | --- | --- |
| DEC-01 | `[caller intent or system event]` | `[facts that must be known]` | `[allowed action]` | `[action the agent must not take]` | `[state]` | `[SOP-01]` |

Ask these questions for every branch:

- What must be true before this branch can run?
- What can the caller correct or revoke?
- What happens when a tool is slow, fails, or returns an unexpected state?
- When must the agent stop, transfer, ask, or safely end?

## 4. Build the scenario matrix

Start with the riskiest boundary, not the happiest demo.

| Scenario ID | Decision ID | Caller goal | Variation | Audio condition | Tool condition | Expected terminal state | Severity |
| --- | --- | --- | --- | --- | --- | --- | --- |
| SCN-01 | DEC-01 | `[goal]` | `[boundary, correction, refusal, interruption, or failure]` | `[route/noise/device]` | `[success/delay/error/duplicate risk]` | `[durable state]` | `[critical/high/medium/low]` |

Cover at least:

- [ ] Happy path
- [ ] Missing prerequisite
- [ ] Caller correction
- [ ] Ambiguous critical entity
- [ ] Tool delay
- [ ] Tool failure
- [ ] Duplicate or retry risk
- [ ] Interruption at a consequential moment
- [ ] Required escalation
- [ ] Unsupported request or authority boundary

## 5. Write atomic assertions

Each assertion should answer one yes-or-no question. If the words “and” join two independent outcomes, split the assertion.

| Assertion ID | Scenario ID | Applies when | Assertion | Evidence needed | Severity | Pass rule | Source ID |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AST-01 | SCN-01 | `[condition]` | `[one observable claim]` | `[audio/event/tool/state]` | `[severity]` | `[exact threshold]` | `[SOP-01]` |

Use `not applicable` when the trigger never occurred. Do not turn a missing prerequisite into a false policy failure.

## 6. Lock the evidence contract

| Evidence layer | Required capture | Redaction or access rule | Retention | Evidence owner |
| --- | --- | --- | --- | --- |
| Audio | `[segments or full call]` | `[rule]` | `[duration]` | `[role]` |
| Transcript and turns | `[fields and timestamps]` | `[rule]` | `[duration]` | `[role]` |
| Tool events | `[request, response, idempotency, timing]` | `[rule]` | `[duration]` | `[role]` |
| Policy proof | `[source version and evidence span]` | `[rule]` | `[duration]` | `[role]` |
| Final state | `[before, expected, after]` | `[rule]` | `[duration]` | `[role]` |

## 7. Set the release rule before running tests

- Critical failure rule: `[example: any unauthorized side effect blocks release]`
- Minimum repeats per deterministic case: `[count]`
- Minimum repeats per variable voice/audio stratum: `[count]`
- Required strata: `[languages, routes, devices, speakers, workflows]`
- Known exclusions: `[what this suite does not prove]`
- Ready means: `[explicit condition]`
- Caution means: `[explicit condition and owner]`
- Blocked means: `[explicit condition and recovery action]`

## Worked fictional example

This example is invented to show the chain. It is not a real Voxeval customer or production incident.

- Source `SOP-07`: An appointment may be changed only after the caller confirms the existing appointment and selects an available replacement.
- Decision `DEC-12`: If the replacement slot is no longer available, do not cancel the existing appointment.
- Scenario `SCN-22`: Caller asks for Friday morning; the availability tool returns a slot, but booking later returns `conflict`.
- Assertion `AST-31`: The existing appointment remains active after the booking conflict.
- Evidence: Before-state snapshot, tool timeline, agent speech, and after-state snapshot.
- Severity: Critical.
- Release rule: Any run that removes the existing appointment without creating the replacement blocks the release.

## Final review

- [ ] Every scenario links to a decision and source.
- [ ] Every critical assertion links to evidence outside the agent's own words.
- [ ] Caller corrections and tool failures have explicit expected states.
- [ ] Privacy controls cover recordings, transcripts, tools, traces, review, and fixtures.
- [ ] A named owner must act on every failed release gate.
- [ ] The decision records what remains untested.

Voxeval recommends versioning this worksheet beside the eval suite so a source change can trigger the exact scenarios and assertions it affects.