The customer changed the order mid-sentence. Did your voice agent change the cart?

“No cheese.” Two words, expensive failure. The agent heard them and the transcript contains them, but the kitchen ticket still says extra cheese because the correction arrived while the agent was speaking and the old cart write had already started.

This is a release test for restaurant voice AI, not a menu demo. It shows QA and operations teams how to inject corrections at exact points, assert the cart diff, and verify the final POS ticket.

Modifier accuracy beats a pretty transcript

τ-Voice gives the broad evidence: voice systems can struggle when grounded tasks, realistic audio, and conversational timing collide. The benchmark reports task outcomes and conversation behavior together because a smooth call can still leave the wrong system state. For restaurants, the authoritative outcome is the committed order.

A May 2026 VoiceAutomationAI discussion offers practical corroboration, arguing that restaurant ordering lives or dies on modifiers and that production misses should become regression cases. It is an informal community post, not measured research.

Scroll diagram horizontally on smaller screens.

Build corrections as operations

Every test needs four groups of evidence:

• Setup: starting cart and operation preconditions.
• Stimulus: caller utterance and interruption point.
• Oracle: expected operation, forbidden operation, and expected spoken acknowledgement.
• Outcome: expected final cart and POS state.

For example, encode a “no onion” correction like this:

{
  "starting_cart": {
    "line_1": {
      "item": "cheeseburger",
      "modifiers": ["cheese", "onion"]
    }
  },
  "caller": "Actually, no onion",
  "expected_op": {
    "op": "REMOVE_MODIFIER",
    "line_id": "line_1",
    "modifier": "onion"
  },
  "forbidden_op": {
    "op": "REMOVE_ITEM",
    "line_id": "line_1"
  }
}

Do not grade this with “the assistant was helpful.” Compare the operation and the final record.

Keep operation scope in the oracle. A test should know whether one line, one modifier, or the whole order may change. It should also preserve fields that must remain untouched. When a size changes, quantity, unrelated modifiers, fulfillment mode, and customer details should not drift merely because the model rebuilt a larger object.

Record the precondition for each operation. A correction after confirmation is different from one after payment, and a modifier removal is different when the modifier is required by the catalog. The expected response may be an update, a clarification, a policy explanation, or a human handoff. The suite should judge the action against the actual cart state, not the words alone.

Run in three layers

State tests use no audio. They exercise deterministic operations and the final cart, which makes them the cheapest place to catch mutation scope, version, and idempotency bugs.

Text conversation tests vary phrasing, references, corrections, and tool failures while asserting the same typed operations. They show whether the language layer can produce the contract without introducing audio timing as another variable.

Audio tests add timing, accents, restaurant noise, phone transport, interruptions, and spoken output. They should still assert the same cart and POS state, then add delivery checks such as whether playback stopped and whether the corrected item was read back clearly.

At Voxeval, we run cheap layers often and audio layers on every meaningful voice or orchestration change. A failure should tell the team whether the bug lives in reference resolution, state mutation, timing, or POS reconciliation.

Four correction families worth running first

Narrow mutations. Start with direct replacement, negation, and reversal. “Make that medium, not large” should change size on the referenced line while quantity and modifiers stay put. “No mayo” should remove a modifier rather than the sandwich. “Actually, keep the mayo” should undo the prior removal once, without adding duplicate mayo or rebuilding the line.

These cases sound trivial, which makes them good unit tests for mutation scope. Run them against carts containing unrelated items and modifiers. A passing operation changes exactly what the caller corrected and leaves every protected field alone.

References among similar items. These cases need stable identities. “Add cheese to the second burger” should affect one line. “Remove the plain one” should produce a clarification when more than one item fits. Include provenance-based references such as “the pizza I changed” and relational phrases such as “the other large one.”

Position can move after an item is deleted or a cart is grouped for read-back. The test oracle should identify valid candidate lines from the same state the agent saw. If zero or several candidates remain, the correct result is not a low-confidence mutation. It is a short question.

Use a truth table to make the ambiguity explicit:

Line	Item	Size	Modifiers	Provenance
line_1	pizza	large	olives	first item
line_2	pizza	large	no olives	second item
line_3	pizza	medium	olives	changed from large

Probe that state with “Remove olives from the first,” “Delete the one I changed,” “Make the other large pizza medium,” and “Remove the medium.” The model can resolve some references from history, but the state engine must validate that exactly one line matches. Zero or several matches means clarify.

Add misleading but irrelevant history. Mention “the first pizza” earlier, reorder the summary, then ask about “the one without olives.” This checks whether resolution uses the current cart and recent conversational evidence rather than the first matching phrase in the transcript.

Numbers and self-correction. These cases expose unsafe intermediate work. “I said four, not fourteen” should end with quantity four and a confirmation when the price effect warrants one. “Three tacos, sorry, two tacos and one burrito” should create the corrected set without briefly committing the abandoned quantity. Add variants with leading speech disfluencies, a noisy digit, and a correction that arrives just after the model emits its first partial operation.

Check the operation sequence, not only the final total. A system that adds fourteen items and removes ten happens to reach four, but it can trigger inventory, pricing, or payment work that a direct correction would avoid.

Timing, availability, and locked orders. These conditions turn a simple correction into a race. Inject “No, pickup” while the agent says “I have this for delivery.” Playback should stop, fulfillment mode should change, address collection should be cancelled, and the workflow should resume from pickup requirements. Repeat the correction while POS validation is pending and while a write is in flight. The system should cancel where supported or reconcile and apply a versioned follow-up, never hide the race.

Then exercise unavailable substitutions, location changes, and corrections after commit. A sold-out side should produce approved alternatives rather than a silent swap. A new pickup location should trigger menu and price validation. A paid or locked order should follow restaurant and POS policy instead of pretending the draft is still mutable.

Move the interruption point

One utterance can expose different bugs depending on when it lands. Run “No onions” at each of these points and store the delivered assistant prefix for every run:

• 100 ms after the agent starts reading the item.
• Halfway through the item.
• During the next line.
• During POS validation.
• Immediately after the confirmation question.
• Immediately after commit.

IHBench is useful here because it preserves the delivered prefix of an interrupted turn and scores the next response. Its public set is not restaurant-specific, but the structure is easy to borrow.

Also store the workflow state and pending operation at the injection point. The same “no onions” utterance may safely update a proposed cart, require cancellation during validation, or need a post-commit exception flow. Timing without state cannot tell you which recovery was correct.

Run repeated trials for stochastic paths. Report whether the case passes consistently, not only whether one attempt eventually works. A restaurant cannot ask every fifth caller to repeat a modifier because the model happened to choose a different reference that time.

Test concurrency, not only language

Square’s order-update documentation uses an order version to prevent stale updates from overwriting newer changes. It also requires idempotency handling for intended updates.

Create a POS version conflict, an update that times out but later commits, a retry that reaches the POS twice, a staff edit during the call, and an out-of-order webhook. For every fault, assert that no item is duplicated, no correction is lost, the cart and POS converge, and the spoken confirmation matches the committed version.

This is where a voice-agent test becomes a distributed-system test, so capture durable identifiers in the evidence: order version, idempotency key, tool request ID, response status, and the final POS order ID. When a timeout later commits, those fields show whether the agent reconciled the original operation or created a second one. They also make a failure reproducible without relying on a screenshot of the kitchen ticket.

A fictional order that passes the transcript review

Consider an illustrative order that did not happen. A caller orders two paneer wraps, one spicy and one mild, then interrupts the summary: “Make the mild one spicy too.” The transcript is perfect, the final spoken summary says two spicy wraps, and the caller agrees.

The test should still fail because the agent updated both lines with SET paneer_wrap spice=spicy and no line ID. The broad operation happened to match the latest request, but on a later order with three wraps and one allergy-related modifier, the same mutation could be dangerous.

Create counterfactual siblings for cases like this. Keep the same utterance and add one extra similar item, one protected modifier, or a different catalog version. A broad mutation that happened to be correct in the original case will fail the sibling and reveal why the operation contract matters.

Turn production failures into permanent coverage

Promote production failures carefully. Recreate the menu and timing with synthetic or approved test data, retain the distinctive operation sequence, and label the source as a production-derived pattern rather than a copied customer call. Assign an owner and keep the case after the immediate bug is fixed. Restaurant menus and speech models change often enough that old failures return in new clothes.

Report results by location, menu version, fulfillment mode, correction type, interruption point, and severity. A blended modifier score can hide one store whose catalog maps “no cheese” to an unsupported note. Critical allergen, payment, and duplicate-order failures should remain hard gates rather than contributors to an average.

The tests need a typed oracle, which is why the order-state engine comes first. Keep the final state inside conversation-level evaluation so a lucky closing sentence cannot cover an unsafe mutation.

The restaurant correction regression pack includes fictional cases across these four families and their detailed variants. Replace menu IDs and policy rules with your own, then record actual caller audio under the right consent and retention controls. Run the pack before Friday’s dinner rush finds the next case for you.

Subscribe to Voxeval for more applied voice AI regression packs and release checks.