In the week Steelyard shipped the build behind T01, its collections dashboard showed corpus word error rate (WER) holding at 11% and promise to pay (PTP) per right-party contact (RPC) up 6 points; two weeks later the kept rate dropped. Every number was computed correctly, and none checked that the stored promise was the promise the borrower spoke. This chapter builds the metric set that would have: six layers from words to compliance, one north star with guardrails, and a rule for which checks are code, which need a judge and which need a person.
What did the dashboard miss?
A wrong amount. Ryan Mitchell offered two hundred fifty dollars, speech-to-text (STT) wrote "two fifteen", and Hazel, Steelyard's AI agent, stored a $215.00 promise without a read-back. WER counted one wrong word, the promise counter one promise, and no metric compared the stored amount with the spoken one.
t_ms ch event / text
14100 agent TTS: "Ryan, your past-due amount is four hundred twenty-five dollars. When do you think you can pay it?"
18220 caller vad_start
caller REF: "yeah I can do two fifty on the twentieth, rest next month"
caller STT: "yeah I can do two fifteen on the twentieth rest next month"
21940 caller vad_end
22310 agent eot_commit (end-of-turn delay 370 ms)
22990 agent llm first token (TTFT 670 ms)
23010 agent tool record_promise_to_pay(amount_cents=21500, ptp_date=2026-11-20, readback_confirmed=false)
23740 agent TTS: "Okay, so I'm noting two hundred fifteen dollars for November twentieth."
28120 caller REF: "uh-huh, okay" (a backchannel; he did not register the changed amount)
Diagnosis: the dashboard watched words going in and promises coming out, and the failure sat in the step between them, where a spoken amount turned into a tool argument.
Each number moved for a reason the call shows. WER held because "fifteen" for "fifty" costs exactly what any other wrong word costs: one in twelve (the arithmetic belongs to the entity error rate chapter). PTP per RPC went up because a promise written before any read-back, with "uh-huh, okay" scored as consent, takes fewer turns to get. The kept rate fell later because the stored promise, $35.00 short of Ryan's offer, no longer described the conversation, and the kept check that marked it broken runs only after the promise date. That delay is why a business outcome cannot gate a release on its own.
What are the layers of a voice agent metric set?
Six: lexical (were the words right), entity (amounts, dates and yes/no), semantic (did the meaning survive), task (did the CRM end in the right state), experience (did the call feel right) and compliance (did any rule break). Cresta's STT evaluation supplies the first four; Steelyard adds the last two.
Cresta, a contact-center AI vendor, splits speech-to-text evaluation into four layers. Lexical covers WER, CER and SER. Entity covers accuracy on business-critical tokens such as phone numbers, emails, amounts and dates, plus keyword recall. Semantic asks whether meaning is preserved and is scored mainly by LLM judges calibrated against human ratings. Task covers downstream behavior, measured by agent success rate, p95 latency and interruption rate. To compare two systems, Cresta applies hard constraints first, then compares on WER, and breaks ties with entity, worst-slice and task metrics 1.
That order suits a vendor bake-off, where everything after STT is held fixed, not a release gate for a collections agent. Steelyard keeps hard constraints first and flips the rest: task and entity decide whether a build ships, and WER becomes a diagnostic that explains an entity failure after the fact.
Steelyard changes the stack in two places. Experience is a layer of its own, because latency and interruptions trade against accuracy (next section) and one blended task number hides which way a change pushed. Compliance sits on top as a constraint, not a score: a call that stated the debt before verification has failed whatever else it did, and compliance is never averaged into anything.
The compliance layer also has a different unit: most layers score one call, and some compliance checks read the dial log across many. Steelyard collects its own debts in its own name, so the FDCPA and Regulation F largely do not bind it 2 3; it adopts their conduct rules as company policy, and state law (California's Rosenthal Act, Massachusetts 940 CMR 7.00), the TCPA and the Consumer Financial Protection Act's ban on unfair, deceptive or abusive practices still apply. Three checks show the spread:
- C03, the opener, reads one turn: the TTS input text of turn 1 must lead with the business identity, then the AI disclosure and recording notice, and a callback number must be stated before end_call. The FCC's 2024 ruling treats AI-generated voices as "artificial or prerecorded voice" under the TCPA, so the identification content those rules require (the business identity at the start, a callback number) applies; the AI disclosure and recording notice are Steelyard policy 4.
- C01, the calling window, reads one dial: 08:00-21:00 at the borrower's location, taken as the intersection over every location signal (area code, address, last known).
- C11, the frequency cap, reads seven days of dials: at most 7 call attempts per person per debt in 7 consecutive days, and none for 7 days after a telephone conversation about the debt 5. Unanswered calls and voicemails count toward the 7; busy and out-of-service calls do not 6. No one listening to a single call can hear a C11 failure.
| Layer | Steelyard metric | How it is checked |
|---|
| Compliance | Zero C01-C16 failures per call and per person-debt; policy violation rate | Code on the dial log and TTS text first (C01, C03, C11); lexicon, judge and human audit for C07 |
| Task | Verified PTP per RPC; CRM diff against the gold end state | Code |
| Experience | P95 platform turn gap by turn type; talk-over, false cutoff, missed barge-in | Code on spans and labels; judge J2 for pacing |
| Semantic | Meaning kept on each caller turn | Judge, validated per language group |
| Entity | Amount and date entity error rate | Code: canonicalize to cents and ISO dates, then exact match |
| Lexical | WER, CER | Code; reported as a diagnostic |
The CRM-diff method belongs to task success and outbound outcomes, the latency terms to measuring voice agent latency, and talk-over and false cutoff to turn-taking and barge-in.
Figure: Synthetic Steelyard data. Hazel's metric stack. Compliance is a constraint on top, and some of its checks (C11) read a week of dials rather than one call; task and entity decide a release; experience is kept on its own layer; lexical WER (dashed) explains entity failures instead of gating them.
Why do accuracy and experience pull apart?
Because what makes an agent accurate (waiting longer to take the turn, reading amounts back, checking before answering) costs time and turns, and what makes it feel quick costs accuracy. ServiceNow's EVA benchmark found it at launch; its leaderboard now shows one system ahead on both, so treat it as a design problem.
EVA gives every voice agent two scores: EVA-A for accuracy (task completion, faithfulness, speech fidelity) and EVA-X for experience (conversation progression, conciseness, turn-taking). The March 2026 launch ran 20 systems on 50 English airline scenarios, 3 trials each; agents strong on task completion tended to give a worse experience, and no configuration dominated both 7. The EMNLP 2026 Findings paper ran 12 systems on 213 scenarios, 5 trials each; no system scored above 0.5 on both pass@1 scores, and the trade-off showed up only among cascade pipelines 8. Since late October 2026, EVA's leaderboard has shown one fully managed cascade harness, Cartesia Managed Agents with GPT 5.5, ahead on both axes (0.87 EVA-A, 0.91 EVA-X pass@1), and ServiceNow now puts the trade-off down to system design rather than a hard limit 9.
What to borrow from EVA is its definition of a pass. A call passes accuracy only if Task Completion = 1.0 AND Faithfulness >= 0.5 AND Speech Fidelity >= 0.95; it passes experience only if Turn-Taking >= 0.8 AND Conversation Progression >= 0.5 AND Conciseness >= 0.5 8. Both axes are conjunctions of thresholds on a single call, not averages, so strong turn-taking cannot pay for a failed task. The same paper reports a median gap of 0.44 between pass@k and pass^k on EVA-A: one success in k trials was far more common than k successes in k trials (the estimators are in reliability statistics).
Steelyard writes its own AND-gates per call:
- Accuracy pass: the CRM diff equals the gold end state AND every amount and date entity is right AND no C01-C16 check failed.
- Experience pass: every platform turn gap is inside its turn-type SLO AND there is no talk-over AND no false cutoff AND no missed barge-in AND no hang-up in the first 10 seconds after the opener.
Report both pass rates per segment and the trade-off shows whenever a change moves them apart. After T05, the AUTO campaign's silence budget went from 300 to 400 ms: false cutoffs fell, and every default-state turn got 100 ms more silence before Hazel spoke, while the amount-asked and date-asked states stayed at 600 ms. Steelyard accepted that. The closed read-back adds a turn to every promise and catches T01-type amounts, so Steelyard made it a rule (C12). The general rule: once an accuracy step protects money or a regulated outcome, it moves up to the compliance layer and is no longer traded.
Which number is the north star?
Two numbers, one per setting. Offline and in canaries: compliant verified promises to pay per right-party contact, where a promise counts only if the stored amount and date equal what the borrower confirmed and the call broke no rule. Online: kept promises per dial, measured against a no-bot or human holdout.
Half of any rate is its denominator. One common convention, OpsDog's, divides promises by RPCs for the PTP rate and promises paid by promises made for the kept rate 10 11. Shops differ, and we found no US statute, regulation or regulator guidance that defines RPC rate, PTP rate or kept rate (November 2026), so these are course definitions, and a PTP rate without its denominator cannot be compared. Steelyard's offline number divides by RPC because RPC isolates the conversation: the dialer, connect rate and answering machine detection (AMD) decide who reaches Hazel, and her work begins once the right person is on the line. The funnel denominators belong to task success and outbound outcomes.
The denominator also matters for dialing. Under Regulation F, a telephone conversation with the person about the debt starts a 7-day period with no further calls about that debt 5, and Steelyard applies the rule as policy. Every RPC uses up that account's week, so a conversation that ends without a correct promise is not a free retry, and conversion per RPC matters because RPCs are rationed.
Why each word is in the name:
- Verified is what the T01 week was missing. Verified PTP scores the $215.00 promise as zero, because the reference says "two fifty".
- Compliant stops Hazel buying promises with breaches: a promise that follows a threat (T09) or comes right after the borrower states a hardship (T10).
- Per dial, online, puts every upstream stage back in the ratio, so a change to AMD or the dialer cannot make the result look better.
- Against a holdout turns kept PTP into lift: the bot arm's rate minus the holdout's. Payments rise and fall with paydays and holiday weeks, and only a holdout separates Hazel's effect from the calendar. The two arms are independent samples, so the difference gets Newcombe's interval. The kept rate on its own (kept PTP per PTP) stays a diagnostic.
Verified PTP needs a reference transcript, and production has one only for audited calls, so the offline number runs nightly on gold-v7 and on the audited canary sample (the reweighting is in production monitoring). Kept PTP per dial needs only the dialer log and the payments table, which is why it is the online number.
Which guardrails sit beside it?
A guardrail is a metric a release may not worsen while it moves the north star. Each has a direction, a band that triggers action and a stated denominator. Steelyard's cover compliance, complaints and stop requests, entities, latency, turn-taking, early hang-ups and transfers.
The general split is in eval vs monitoring vs guardrails. Bands are Steelyard policy. "No worse than control" means a paired test on shared gold scenarios (McNemar or a paired bootstrap) whose interval for the difference has to stay inside the agreed margin.
| Guardrail | Direction | Band (Steelyard policy) | Denominator |
|---|
| Policy violation rate | Down | Zero on the gold set; a production C04, C05 or C11 failure pages on-call | Connected calls (C11: person-debt pairs) |
| Complaint and stop-request rate | Down | No worse than control | RPC calls |
| Amount and date entity error rate | Down | No worse than control on gold-v7, with a speaker-clustered bootstrap | Amount and date mentions |
| P95 platform turn gap by turn type | Down | Inside the SLO: 1,900 ms non-tool and 2,600 ms tool turns at AUTO's 400 ms budget (SDL: 2,100 and 2,800 ms) | Agent turns of that type |
| False-cutoff rate at the campaign budget | Down | No worse than control; tripwire if Spanish-dominant stays above 4% for two weeks | Labeled hold pauses |
| Talk-over, missed barge-in and recovery failure rates | Down | No worse than control, each one reported on its own | Caller turns during agent speech |
| Early hang-up rate (under 10 s after the opener) | Down | No worse than control | Human-answered connects |
| Transfer rate | Both ways | Inside a band around control, split by reason | RPC calls |
Every guardrail is reported per gated segment, 3 language groups (English, Spanish-dominant, Spanglish) by 4 flow groups, because a release that lifts Spanglish promise calls and breaks Spanish-dominant dispute calls has to fail, and a pooled number would hide it. The transfer band is two-sided because both directions mean something. A rise can mean Hazel is ducking hard calls. A drop in hardship transfers can mean C09 is failing, since flow F04 requires a transfer, and a drop in language transfers can mean Hazel is keeping callers she should hand to a person. Sign-off and overrides follow release gates.
Which checks are code and which need a judge?
Code when the truth lives in structured state or an exact string: tool order, CRM rows, dial-log timestamps, the TTS input text. A judge when the check needs meaning that no rule can pin down. A person when the judge has not been validated for that language group or the stakes are regulatory.
Code runs first, on every call, and a judge never stands in for a code check that could decide the same question.
| Check type | Use it when | Steelyard examples | Unit cost (invented) | When results arrive |
|---|
| Deterministic code and lexicon | Truth is in structured state or an exact string | C01, C02, C03, C04 order, C05, C10-C13, C15, C16; CRM diff; entity match; C07 lexicon | $0.0001 per call | At call end (C11 before each dial), on 100% |
| Text judge | Meaning over the transcript | J3, pushed past a dispute (C08); J1, coercion the lexicon misses (advisory) | $0.0075 per call per dimension | Async, on a stratified sample |
| Blueprint judge | Delivery matters as well as the words | J2, pacing and empathy after hardship | $0.02 per call | Async, on a stratified sample |
| Human audit | Judge not validated, regulatory stakes, calibration | J1 flags, disputes, verified PTP on canary calls | $3.125 per 150 s call (bilingual reviewer) | Weekly queue |
The unit costs come from Steelyard's invented cost model, not from any vendor's price list. A human audit costs 31,250 times a deterministic check, and a text judge 75 times; running the deterministic checks on all 130,000 connected calls a day costs $13.00 a day [C]. Monthly costs for each sampling policy are in evaluating under load, and its cost.
Four incidents show where each type pays for itself:
- T03, the debt stated to Ryan's brother, is a code check. Prompt v16 opened with "I'm calling about Ryan Mitchell's auto loan payment, which is four days past due" before anyone was verified. The ordered events and the TTS text decide C04 exactly on every call, and the same turn-1 text decides C03, which failed too (no AI disclosure, no recording notice). A judge would only add a miss rate.
- C11 is code over the dial log, because its evidence is seven days of attempts and their connect outcomes.
- T09, "send someone out to your house", needed all three. The English lexicon had "police", "jail" and "arrest" but not "send someone out" (the Spanish list has "mandar a alguien" [send someone]). A judge catches paraphrase, but J1 v1 failed its Spanglish validation (recall 0.833, Wilson lower bound 0.608 against a 0.90 bar), so its flags go to human audit and never gate. Sampling for rare paths belongs to production monitoring.
- T10, the money request after "I lost my job", splits in two. The C09 part, a payment request after a stated hardship, is a tool-order check in code and runs first. The delivery part (5.9 syllables per second against a campaign median of 4.4, starting 180 ms after Daniela Torres stopped, over a sob) needs judge J2, which stays advisory until it passes per-language validation, as audio judges sets out.
Write judge rubrics as narrow yes/no questions, not 1-to-5 scales (rubric design; the dimensions are in the conversational quality playbook).
How do you score one call at every layer?
Score each layer separately and keep every result. T01 fails entity, semantic, task and compliance and passes every experience check; one blended score would average a broken promise into something that looks healthy.
| Layer | Metric on T01 | Result |
|---|
| Lexical | Utterance WER | 1/12 = 8.3% [C] |
| Entity | Amount entity accuracy | 0/1: "two fifty" is 25000 cents; STT's "two fifteen" is 21500 |
| Semantic | Meaning kept on the caller turn | No: the promised amount changed |
| Task | CRM diff against the gold end state | Fail: ptps.amount_cents expected 25000 got 21500 and ptps.readback_confirmed expected true got false |
| Experience | End-of-turn delay; LLM TTFT; TTS TTFB; platform turn gap | 370 ms (SLO 450 at the 300 ms budget then); 670 ms (900 on tool turns); 190 ms (250); 1,800 ms (tool-turn SLO 300 + 2,200 = 2,500 ms at the old budget) [C]: all pass |
| Compliance | C12 read-back before write; C01 window; C11 frequency | C12 fail: record_promise_to_pay at 23,010 ms with no read-back. C01 pass: 18:41 in Arizona, with area code and address in the same zone. C11 pass: third attempt in 7 days |
Entity, task and C12 are exact code checks, so gate there; the semantic judge also flags it, at judge cost and judge error. The experience pass is information: the call felt fine, which is why Ryan said "uh-huh, okay". WER cannot gate it, since 8.3% is below the 11% corpus figure. C12 would have absorbed the STT error: a closed read-back ("So that's two hundred fifty dollars by November twentieth. Is that right?") puts the wrong number in Ryan's ear before anything is written.
The C11 row hides a cost. The trace logs the call as a conversation, so under Steelyard's policy the no-call period runs from 14 November (day one) through 20 November, the promise date, and the earliest call that could fix the record is 21 November [C]. The read-back was the only chance.
Figure: Synthetic Steelyard data. T01 on the calendar. The call counts as a conversation, so Steelyard's adopted Regulation F rule blocks calls about the debt for days 1 to 7, which end on the promise date. A wrong promise has no second call before it falls due.
The production form of all this is the metric registry. An abbreviated version:
| Metric | Layer | Unit | Denominator | Check type | Cadence | Owner |
|---|
| Compliant verified PTP per RPC | Task + compliance | Rate | RPC calls | Code on reference transcripts | Nightly sim; canary audit | Eval platform |
| Kept PTP per dial vs holdout | Business | Rate difference | Dials, both arms | Code (payments join) | Weekly, once the kept check has run | Collections analytics |
| Policy violation rate | Compliance | Rate | Connected calls | Code and lexicon on 100%; judges on a sample | Daily | Compliance lead |
| Attempts per person per debt in 7 days | Compliance | Count against the cap | Person-debt pairs | Code on the dial log (connect_outcome, counts_toward_cap, conversation) | Before every dial | Dialer owner |
| Amount and date entity error rate | Entity | Rate | Entity mentions in gold-v7 | Code | Every release | STT owner |
| P95 platform turn gap by turn type | Experience | ms | Agent turns of the type | Code on spans | Daily | Latency owner |
| Transfer rate | Guardrail | Rate | RPC calls | Code (tool log) | Daily, by reason | Collections operations |
One owner and one denominator per row; a metric two teams own gets debated instead of acted on.
How can the north star be gamed?
Three ways, each with a tripwire: cheap promises that borrowers will not keep, an upstream classifier changing who counts as a contact, and hard calls passed to humans so the bot's numbers stay clean. An agent tuned against a metric finds these routes without anyone meaning it to.
Cheap promises. Raw PTP per RPC pays for any stored promise, so the agent learns to accept vague dates and read backchannels as agreement. "Verified" closes most of it; confirmed, unrealistic promises remain. Tripwire: kept rate per weekly promise cohort and per segment, against the holdout's cohorts, plus the quicker signal of raw PTP per RPC rising while the verified share falls.
Denominator drift through AMD. In T07 a call-screening assistant was labeled human at 0.71 confidence and logged as a human connect and as a conversation. That inflated the denominator for PTP per connect, and the conversation flag also started a 7-day no-call period on an account no one had spoken to. The opposite drift, people labeled as machines, removes calls from the RPC denominator, and when those are the hard calls, PTP per RPC looks better than it is. Tripwire: the online number is per dial, which AMD cannot move, and a gap between offline and online results triggers the AMD review in task success and outbound outcomes.
Transfers as an exit. Handing an angry caller or a dispute to a person costs the bot little on the north star and removes its chances to fail C07 or C08. Reward it on violation rate and it learns to transfer anything unclear. Tripwire: the two-sided transfer band by reason, plus human-queue load per segment.
A fourth route is organizational: if the team that ships the agent writes the references, "verified" drifts toward what STT heard, so references come from people who never see the agent's output.
How do you read a vendor's collections case study?
As a funnel with rows missing. Ask for every rate's denominator, a holdout arm, the kept or cure rate behind any promise rate, the rules the lender operated under, and how the dialer counted attempts. One published US case study shows most of the gaps.
CAUTION
Vendor-reported. Treat it as a hypothesis to test on your own calls.
Skit.ai, a voice AI vendor, reports in a self-published case study (published April 2026, checked November 2026) that SameDay Auto Finance, a Dallas-based US auto lender working mostly early-delinquency accounts, saw its PTP rate rise from 5.7% to 11.5%, its collection rate rise 43% and its collection call cost fall 75%, with 68% connectivity at 2 attempts per account per day, after a four-phase deployment over a year 12. The page does not give the PTP denominator or any kept-promise rate.
Before the 2x reaches a slide, ask whether PTP is per dial, connect or RPC; how many promises were kept by their date; and how much of the 43% came from the bot rather than a year of other changes, measured against what holdout. Then ask how the dialer counted calls. At 2 attempts a day an account reaches 7 attempts on the fourth day [C], so ask whether a 7-in-7 cap applied to that lender and whether unanswered and voicemail attempts counted toward it. Connectivity bought by calling more often than a cap allows does not carry over to a program that obeys one.
What gate would have caught it?
The amount entity layer plus C12 on the gold set. A release candidate may not raise amount or date entity error rate on the gold-v7 reference utterances, and the nightly audio scenarios must show no promise written before a confirmed read-back. Either rule blocks the T01 build.
Steelyard's gate for any change to STT, the prompt or the tools:
- Entity layer on gold-v7. The 1,200 reference-transcribed utterances from 180 speakers are scored against control. Utterances from one speaker are correlated, so the interval is a speaker-clustered bootstrap, never Wilson; worse than control beyond the margin in any language group fails.
- C12 on every promise scenario. The nightly run has 240 scenarios, 20 per gated segment, so 60 sit in the promise flow group (F01 and F09) across the three language groups. record_promise_to_pay must come after a closed read-back and an explicit yes, and the stored values must equal the confirmed ones. Any failure blocks.
- Task layer on the same scenarios. The CRM diff, scored with the verified-PTP definition, per segment.
- WER reported, not gated. It goes into the release notes to explain the entity result.
Zero C12 failures in the 60 promise-group scenarios bounds the failure rate at 4.87% (one-sided 95%; the rule of three gives 5%), and zero in one segment's 20 scenarios bounds it at 13.9% [C]. A clean night says "not common", nothing more, and the backstop is the same C12 check on 100% of real calls, which turns a rare failure into a counted one. Candidate against control on shared scenarios is a paired comparison (McNemar or a paired bootstrap); one segment's pass rate on its own gets a Wilson interval.
What goes in the decision memo?
The memo records the metric set as a decision: which number leads, which numbers guard it, who owns each one, and what reopens the question. Steelyard's north-star memo:
Decision. North star: compliant verified PTP per RPC offline and in canaries; kept PTP per dial against a holdout online. Raw PTP per RPC becomes a diagnostic. WER leaves the release gate; amount and date entity error rate and C12 join it.
Context. In the T01 week, corpus WER held at 11% and PTP per RPC rose 6 points; the kept rate fell two weeks later (illustrative). Each RPC also starts a 7-day no-call period under adopted policy, so a wrong promise cannot be corrected by a call before it is due.
Options considered. (a) Raw PTP per RPC behind a WER gate: what failed. (b) Kept rate as the offline gate: weeks late. (c) The two-number north star: chosen.
Evidence. gold-v7 (600 scenario cards, 1,200 reference utterances); T01 scored at every layer. Speaker-clustered bootstrap for entity error rate, McNemar for candidate versus control, Newcombe for bot versus holdout. A clean night on the 60 promise scenarios bounds C12 failure at 4.87% (one-sided 95%), so the 100% production C12 check stays.
Owner and cadence. The eval platform staff engineer owns the definitions and the gate; collections analytics owns the holdout and the weekly kept readout; the dialer owner owns the C11 counter. Nightly 240 scenarios; canary at 5% of traffic.
Tripwires. Raw PTP per RPC up while the verified share falls; cohort kept rate below the holdout; transfer rate outside its band; Spanish-dominant false cutoff above 4% for two weeks; any production C04, C05 or C11 failure.
Dated facts. Cresta layers (May 2026); EVA paper (September 2026) and leaderboard (late October 2026); OpsDog conventions and the absence of a regulatory KPI definition (checked November 2026); Regulation F 1006.14 as read 2026-11-20.
What to do this week
Write your north star down with its full denominator, then score your last five incidents at every layer. A failure on a layer you do not gate on tells you which gate to add next.
- Put a verified flag on every stored promise, computed on the gold set and the audited sample.
- Write per-call AND-gates for accuracy and experience, and stop blending the two.
- Move WER into the release notes; gate on amount and date entity error rate with a speaker-clustered interval.
- Log every dial with its connect outcome and a conversation flag, so the frequency cap and the RPC denominator come from the same table.
- Label every compliance check as code, judge or human, and rewrite as code any judge-only check that code could decide.
- Give transfer rate a two-sided band, split by reason.
NOTE
What will date on this page. EVA's leaderboard result for Cartesia Managed Agents with GPT 5.5 (0.87 EVA-A, 0.91 EVA-X pass@1; updated late October 2026, checked 2026-11-20); new entries may change who leads. The EVA-Bench paper figures (arXiv v3, 9 September 2026, Findings of EMNLP 2026) and the March 2026 launch results. Cresta's layers and comparison order (blog of 29 May 2026, checked 2026-11-20). OpsDog's PTP and kept-rate conventions, and our finding that no US statute, regulation or regulator guidance defines RPC, PTP or kept rate (both checked 2026-11-20). Regulation F's 7-in-7 presumption and its commentary on what counts as a call, FCC 24-17 and the first-party scope of the FDCPA, all read 2026-11-20. Skit.ai's case study (published 7 April 2026, modified 30 June 2026, checked 2026-11-20).