Steelyard ran a bake-off of four STT vendors on 16 kHz recordings from its mobile app, and vendor A came first. On mobile-to-trunk audio (AMR-WB to G.711), scored by amount entity error rate, vendor A came third of four. Whatever gave vendor A its lead did not survive the phone line. This is a known effect, not bad luck: in a Microsoft voice-search study, a wideband-only model went from 27.5% WER on 16 kHz test audio to 53.5% on the same audio downsampled to 8 kHz 1. This recipe builds the offline phone path, so every STT and TTS test in your suite runs on audio that sounds like a call.
The script and synthetic data for this recipe are also in the ai-evals-course repository on GitHub, in the folder voice-ai/labs/telephony-degradation-pipeline/.
What you build
One numpy script that takes 16 kHz audio (a synthetic chirp, or your own consented recording) and writes the source plus four phone-path conditions as WAV files, each one ready to send to your STT. It also prints the checks that show each stage did its job, and it writes a 36-row condition matrix that covers every pair of levels across five factors.
The stages:
| Function | What it simulates | Check it prints |
|---|
bandpass(x, sr, 300, 3400) | The telephone passband (an FFT mask, for teaching) | Share of energy above 3.4 kHz before and after |
resample_poly_np(x, 1, 2) | The 8 kHz media stream | Same check at 8 kHz |
mulaw_encode / mulaw_decode | G.711 companding, mu = 255 | Round-trip SNR at two levels, against 8-bit linear |
gilbert_elliott(n, p, r, seed) | Burst packet loss on 20 ms frames | Realized loss and mean burst length over 10 seeds |
apply_loss(x, mask, fill) | Lost frames played as silence or a repeated frame | Waveform SNR for each fill |
mix_snr(x, noise, snr) | Background noise at a fixed SNR | Realized SNR |
condition_matrix() | Which cells to run | Pair coverage |
The 20 ms frame is not arbitrary. Twilio Media Streams always delivers audio/x-mulaw at 8000 Hz, mono, and requires audio sent back in the same format 2; at 8,000 one-byte samples a second, 160 bytes is 20 ms, which is also RTP's default packetisation interval for G.711 3. Steelyard's CPaaS stream uses the same 160-byte frames, so the script drops loss in whole 20 ms frames.
Step 1: Create a fresh environment
The only dependency is numpy. The mu-law code is written out by hand because Python 3.13 removed the audioop module 4, so a recipe that imports it breaks on the next interpreter upgrade.
python3.11 -m venv .venv
.venv/bin/pip install numpy==2.1.3
Step 2: Save the script
Save this as telephony-degradation-pipeline.py. It starts with the same Steelyard constants block as the other voice recipes, so the capstone can tie them to one dataset name, and an --edition flag that defaults to global. The signal path is identical in every edition; only the seed, the dataset name and the carrier, codec chain and language levels of the condition matrix change. Each stage's check is an assert, so All checks passed. means every stage did its job on this run, and the synthetic run also asserts its own printed numbers, so a numpy upgrade that moves them fails loudly. The section comments in the code carry the same [1] to [6] labels as the printed output.
"""A phone-path degradation pipeline in numpy (aievals.co cookbook recipe).
Takes 16 kHz audio (a synthetic chirp by default, or your own consented WAV),
runs it through a teaching model of the phone path, and writes the source plus
four condition WAVs for your STT:
band-pass 300-3400 Hz -> resample to 8 kHz -> G.711-style mu-law round trip
-> Gilbert-Elliott burst loss on 20 ms frames -> optional noise at a fixed SNR
Run:
python3.11 -m venv .venv && .venv/bin/pip install numpy==2.1.3
.venv/bin/python telephony-degradation-pipeline.py
.venv/bin/python telephony-degradation-pipeline.py --wav my_consented_16k.wav
.venv/bin/python telephony-degradation-pipeline.py --edition india
--edition picks the world (default global). The signal path is the same in both;
only the seed, the dataset name and the condition-matrix levels change.
The filters and the companding here are for teaching. Gate on the ITU-T
Software Tool Library (G.191) or on real 8 kHz calls, not on this file.
"""
from __future__ import annotations
import argparse
import csv
import itertools
import os
import wave
import numpy as np
# --- GLOBAL constants (WORLD-GLOBAL world-global-v1.1) ---
GLOBAL = {
"lender_note": "Synthetic data.",
"dataset": "steelyard-calls (synthetic, seed 20261121)",
"seed": 20261121,
"clip_loss5_frozen": (6, [1, 1, 1, 3]), # [C] frames lost and bursts on the chirp at a 5% target
"products": {"AUTO": "auto loan", "PIL": "personal installment, digital",
"POS": "retail point-of-sale", "SBL": "small business, guarantor",
"SDL": "credit-builder and small-dollar"},
"calling_window_local": {"default": ("08:00", "21:00"), "SDL": ("09:00", "20:00")},
"campaigns": { # (book, silence budget ms, amount-asked / date-asked budget ms)
"AUTO-early-nov": ("AUTO", 400, 600),
"PIL-digital-nov": ("PIL", 400, 600),
"POS-short-nov": ("POS", 400, 600),
"SBL-guar-nov": ("SBL", 450, 650),
"SDL-biwk-nov": ("SDL", 600, 800),
},
"languages": {"English": 0.64, "Spanish-dominant": 0.18, "Spanglish": 0.14, "other": 0.04},
"customer_advocate": {"name": "Ms Dana Brooks", "email": "advocate@steelyard.example",
"phone": "(520) 555-0199",
"address": "1200 Example Avenue, Tucson, AZ 85701"},
"media": {"sample_rate_hz": 8000, "codec": "G.711 mu-law", "frame_ms": 20, "frame_bytes": 160},
"money_unit": "integer cents",
"usd_1500_forms": ["fifteen hundred", "one thousand five hundred", "1,500", "1500",
"$1,500.00", "mil quinientos", "mil quinientos dolares"], # 8th: "1.500" es-ES, R1
"usd_1500_cents": 150000,
"factors": { # condition matrix levels (WG19)
"carrier": ["carrier-a-mobile", "carrier-b-mobile", "carrier-c-mobile", "voip-leg"],
"codec_chain": ["amrnb->mulaw", "amrwb->mulaw", "opus->mulaw"],
"language_mix": ["English", "Spanish-dominant", "Spanglish"],
},
}
# --- end GLOBAL constants ---
EDITIONS = {"global": GLOBAL}
# The India edition's constants block goes here in the full file; it adds
# EDITIONS["india"]. Without it, --edition accepts only "global".
SR_WIDE = 16000
SR_PHONE = GLOBAL["media"]["sample_rate_hz"] # the same 8 kHz, 160-byte media in every edition
FRAME = GLOBAL["media"]["frame_bytes"] # 160 mu-law bytes = 160 samples = 20 ms at 8 kHz
assert all(w["media"] == GLOBAL["media"] for w in EDITIONS.values())
# ---------- [1] signals and the band-pass ----------
def synth_source(seconds: float = 4.0, sr: int = SR_WIDE) -> np.ndarray:
"""Linear chirp 100 Hz -> 7 kHz plus tones at 1 kHz and 5 kHz, peak about -3 dBFS."""
t = np.arange(int(seconds * sr)) / sr
f0, f1 = 100.0, 7000.0
chirp = np.sin(2 * np.pi * (f0 * t + (f1 - f0) * t**2 / (2 * seconds)))
x = 0.40 * chirp + 0.15 * np.sin(2 * np.pi * 1000 * t) + 0.15 * np.sin(2 * np.pi * 5000 * t)
return x / np.max(np.abs(x)) * 10 ** (-3 / 20)
def bandpass(x: np.ndarray, sr: int, lo: float = 300.0, hi: float = 3400.0) -> np.ndarray:
"""Brick-wall FFT mask. A teaching filter: it rings, and it is not G.712."""
spec = np.fft.rfft(x)
freqs = np.fft.rfftfreq(len(x), 1 / sr)
spec[(freqs < lo) | (freqs > hi)] = 0
return np.fft.irfft(spec, n=len(x))
def band_energy(x: np.ndarray, sr: int, lo: float, hi: float) -> float:
"""Share of the signal's energy above lo and up to hi Hz, in dB (0 dB = all of it)."""
power = np.abs(np.fft.rfft(x)) ** 2
freqs = np.fft.rfftfreq(len(x), 1 / sr)
share = power[(freqs > lo) & (freqs <= hi)].sum() / power.sum()
return 10 * np.log10(max(share, 1e-30))
def resample_poly_np(x: np.ndarray, up: int, down: int, taps: int = 161) -> np.ndarray:
"""Zero-stuff by `up`, Kaiser-windowed sinc low-pass, keep every `down`-th sample."""
cutoff = 0.5 / max(up, down) # cycles per sample at the upsampled rate
n = np.arange(taps) - (taps - 1) / 2
h = 2 * cutoff * np.sinc(2 * cutoff * n) * np.kaiser(taps, 8.6)
h *= up / h.sum()
stuffed = np.zeros(len(x) * up)
stuffed[::up] = x
y = np.convolve(stuffed, h)[(taps - 1) // 2:(taps - 1) // 2 + len(stuffed)]
return y[::down]
# ---------- [2] mu-law ----------
MU = 255.0
def mulaw_encode(x: np.ndarray) -> np.ndarray:
"""Continuous mu-law (mu = 255) to 8-bit codes. Not the bit-exact G.711 segment table."""
x = np.clip(x, -1.0, 1.0)
y = np.sign(x) * np.log1p(MU * np.abs(x)) / np.log1p(MU)
return np.round((y + 1) / 2 * 255).astype(np.uint8)
def mulaw_decode(codes: np.ndarray) -> np.ndarray:
y = codes.astype(np.float64) / 255 * 2 - 1
return np.sign(y) * np.expm1(np.abs(y) * np.log1p(MU)) / MU
def linear8_round_trip(x: np.ndarray) -> np.ndarray:
"""Plain 8-bit linear quantization, for contrast with companding."""
return np.round(np.clip(x, -1, 1) * 127) / 127
def snr_db(ref: np.ndarray, test: np.ndarray) -> float:
err = test - ref
return 10 * np.log10(np.sum(ref**2) / max(np.sum(err**2), 1e-30))
# ---------- [3] burst loss ----------
def gilbert_elliott(n_frames: int, p: float, r: float, seed: int) -> np.ndarray:
"""Two-state burst loss. p = P(good -> bad), r = P(bad -> good); every bad frame is lost.
Stationary loss rate p / (p + r); mean burst length 1 / r frames."""
rng = np.random.default_rng(seed)
u = rng.random(n_frames + 1)
lost = np.zeros(n_frames, dtype=bool)
bad = u[0] < p / (p + r) # start in the stationary distribution
for i in range(n_frames):
lost[i] = bad
bad = (u[i + 1] >= r) if bad else (u[i + 1] < p)
return lost
def burst_lengths(mask: np.ndarray) -> list[int]:
runs, run = [], 0
for lost in mask:
if lost:
run += 1
elif run:
runs.append(run)
run = 0
if run:
runs.append(run)
return runs
def apply_loss(x: np.ndarray, mask: np.ndarray, fill: str = "zero", frame: int = FRAME) -> np.ndarray:
"""Drop 20 ms frames. fill='zero' plays silence; fill='repeat' replays the last good
frame (a crude concealment, not G.711 Appendix I)."""
y = x.copy()
last_good = np.zeros(frame)
for i, lost in enumerate(mask):
seg = slice(i * frame, (i + 1) * frame)
if seg.start >= len(y):
break
if lost:
y[seg] = 0.0 if fill == "zero" else last_good[: len(y[seg])]
else:
last_good = x[seg].copy()
return y
def ge_params(target_loss: float, r: float) -> float:
"""p that gives the target stationary loss for a chosen r."""
return target_loss * r / (1 - target_loss)
# ---------- [4] noise ----------
def mix_snr(x: np.ndarray, noise: np.ndarray, snr: float) -> np.ndarray:
gain = np.sqrt(np.mean(x**2) / (np.mean(noise**2) * 10 ** (snr / 10)))
return x + gain * noise
# ---------- [5] WAV I/O ----------
def read_wav_16k(path: str) -> np.ndarray:
with wave.open(path, "rb") as w:
if (w.getnchannels(), w.getsampwidth(), w.getframerate()) != (1, 2, SR_WIDE):
raise SystemExit(f"{path}: need mono 16-bit PCM at 16 kHz")
pcm = np.frombuffer(w.readframes(w.getnframes()), dtype="<i2")
return pcm.astype(np.float64) / 32768
def write_wav(path: str, x: np.ndarray, sr: int) -> int:
pcm = (np.clip(x, -1, 1 - 1 / 32768) * 32768).astype("<i2")
with wave.open(path, "wb") as w:
w.setnchannels(1)
w.setsampwidth(2)
w.setframerate(sr)
w.writeframes(pcm.tobytes())
return os.path.getsize(path)
# ---------- [6] the condition matrix ----------
def matrix_factors(world: dict) -> dict:
"""Five factors in a fixed order; carrier, codec chain and language mix come from the edition."""
f = world["factors"]
return {"carrier": f["carrier"], "codec_chain": f["codec_chain"], "loss": ["1%", "5%", "10%"],
"snr": ["clean", "20 dB", "10 dB"], "language_mix": f["language_mix"]}
def condition_matrix(factors: dict) -> list[dict]:
"""Every carrier x codec chain x loss combination once; SNR and language mix assigned
with modular arithmetic so every pair of levels of every two factors appears."""
rows = []
for c, k, l in itertools.product(range(4), range(3), range(3)):
rows.append({
"carrier": factors["carrier"][c],
"codec_chain": factors["codec_chain"][k],
"loss": factors["loss"][l],
"snr": factors["snr"][(k + l) % 3],
"language_mix": factors["language_mix"][(k + 2 * l + c) % 3],
})
return rows
def pair_coverage(rows: list[dict], factors: dict) -> tuple[int, int]:
needed = covered = 0
for a, b in itertools.combinations(factors, 2):
seen = {(row[a], row[b]) for row in rows}
for pair in itertools.product(factors[a], factors[b]):
needed += 1
covered += pair in seen
return covered, needed
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--edition", choices=sorted(EDITIONS), default="global",
help="which world's seed and matrix levels to use (default global)")
ap.add_argument("--wav", help="your own consented mono 16-bit 16 kHz WAV")
ap.add_argument("--out", default="phone_path_out")
args = ap.parse_args()
world = EDITIONS[args.edition]
seed = world["seed"]
os.makedirs(args.out, exist_ok=True)
print(world["lender_note"])
print(f"Edition: {args.edition}. Dataset: {world['dataset']}")
src = read_wav_16k(args.wav) if args.wav else synth_source()
src = src[: len(src) - len(src) % (2 * FRAME)] # whole 20 ms frames after resampling
print(f"Source: {'your WAV' if args.wav else 'synthetic chirp 100 Hz -> 7 kHz + 1 kHz and 5 kHz tones'}, "
f"{len(src) / SR_WIDE:.2f} s at {SR_WIDE} Hz")
# [1] what the line removes
band = bandpass(src, SR_WIDE)
phone = resample_poly_np(band, up=1, down=2)
codes = mulaw_encode(phone)
phone_mu = mulaw_decode(codes)
print("\n[1] Energy above 3.4 kHz, as a share of each signal's total")
stages = [("source, 16 kHz", src, SR_WIDE), ("after 300-3400 Hz band-pass", band, SR_WIDE),
("after resample to 8 kHz", phone, SR_PHONE), ("after mu-law round trip", phone_mu, SR_PHONE)]
above = {}
for name, sig, sr in stages:
above[name] = band_energy(sig, sr, 3400, sr / 2)
shown = f"{above[name]:8.1f}" if above[name] > -200 else " < -200"
print(f" {name:30s} {shown} dB")
below = band_energy(src, SR_WIDE, 0, 300)
print(f" source energy below 300 Hz {below:8.1f} dB (also removed by the band-pass)")
drop_filter = above["source, 16 kHz"] - above["after resample to 8 kHz"]
drop_coded = above["source, 16 kHz"] - above["after mu-law round trip"]
print(f" drop through band-pass + 8 kHz: {drop_filter:.1f} dB; after mu-law: {drop_coded:.1f} dB "
f"(mu-law noise is broadband)")
assert above["after resample to 8 kHz"] < -40, "band-pass + 8 kHz must leave under -40 dB above 3.4 kHz"
if not args.wav: # chirp thresholds; speech has far less energy above 3.4 kHz to begin with
assert above["source, 16 kHz"] > -10, "source must carry energy above 3.4 kHz"
assert drop_filter > 40 and drop_coded > 40, "phone path must be more than 40 dB down above 3.4 kHz"
frozen = [round(above[name], 1) for name, _, _ in stages if name != "after 300-3400 Hz band-pass"]
assert frozen == [-2.9, -68.6, -46.0] and (round(drop_filter, 1), round(drop_coded, 1)) == (65.7, 43.1)
# [2] is mu-law the problem?
print("\n[2] Round-trip SNR on the 8 kHz phone-band signal")
print(f" {'level':>14s} {'mu-law 8-bit':>13s} {'linear 8-bit':>13s}")
snrs = {}
for level_db in (-3, -33):
x = phone / np.max(np.abs(phone)) * 10 ** (level_db / 20)
mu = snr_db(x, mulaw_decode(mulaw_encode(x)))
lin = snr_db(x, linear8_round_trip(x))
snrs[level_db] = (mu, lin)
print(f" {f'peak {level_db} dBFS':>14s} {mu:10.1f} dB {lin:10.1f} dB")
print(f" mu-law moves {snrs[-3][0] - snrs[-33][0]:.1f} dB over a 30 dB level drop; "
f"linear moves {snrs[-3][1] - snrs[-33][1]:.1f} dB")
assert snrs[-33][0] - snrs[-33][1] > 20, "at -33 dBFS mu-law should beat 8-bit linear by 20 dB"
if not args.wav: # chirp thresholds; speech's higher crest factor costs mu-law a few dB at -33
assert min(snrs[-3][0], snrs[-33][0]) > 30, "mu-law should stay above 30 dB SNR"
assert snrs[-3][0] - snrs[-33][0] < 5 < snrs[-3][1] - snrs[-33][1]
assert [round(v, 1) for v in (*snrs[-3], *snrs[-33])] == [37.7, 40.9, 34.8, 9.5], "frozen SNRs moved"
payload = codes.tobytes()
n_frames = len(payload) // FRAME
print(f" mu-law payload: {len(payload):,} bytes = {n_frames} frames of {FRAME} bytes (20 ms each)")
assert len(payload) % FRAME == 0 and n_frames * 20 == round(1000 * len(src) / SR_WIDE)
# [3] burst loss
r = 0.5
print(f"\n[3] Gilbert-Elliott loss on 20 ms frames, r = {r} (mean burst {1 / r:.1f} frames = "
f"{20 / r:.0f} ms), 50,000 frames x 10 seeds")
print(f" {'target':>6s} {'p':>8s} {'realized loss (min-max)':>27s} {'mean burst':>11s} netem equivalent")
for target in (0.01, 0.05, 0.10):
p = ge_params(target, r)
losses, bursts = [], []
for s in range(10):
mask = gilbert_elliott(50_000, p, r, seed=seed + s)
losses.append(mask.mean())
bursts.extend(burst_lengths(mask))
mean_loss, mean_burst = float(np.mean(losses)), float(np.mean(bursts))
print(f" {target:6.0%} {p:8.5f} {mean_loss:9.4f} ({min(losses):.4f}-{max(losses):.4f}) "
f"{mean_burst:9.2f} netem loss gemodel {100 * p:.2f}% {100 * r:.0f}% 100% 0%")
assert abs(mean_loss - target) / target < 0.05, "realized loss should match p/(p+r)"
assert abs(mean_burst - 1 / r) < 0.05, "mean burst should match 1/r"
mask5 = gilbert_elliott(n_frames, ge_params(0.05, r), r, seed=seed)
zero5 = apply_loss(phone_mu, mask5, "zero")
rep5 = apply_loss(phone_mu, mask5, "repeat")
print(f" this clip at 5%: {int(mask5.sum())} of {n_frames} frames lost, bursts {burst_lengths(mask5)}")
if not args.wav:
assert (int(mask5.sum()), burst_lengths(mask5)) == world["clip_loss5_frozen"], "seeded loss draw moved"
print(f" waveform SNR vs the unlost phone signal: zero fill {snr_db(phone_mu, zero5):.1f} dB, "
f"repeat fill {snr_db(phone_mu, rep5):.1f} dB")
# [4] noise at a fixed SNR
rng = np.random.default_rng(seed)
noise = resample_poly_np(bandpass(rng.standard_normal(len(src)), SR_WIDE), 1, 2)
noisy = mix_snr(phone, noise, 10.0)
realized = snr_db(phone, noisy)
print(f"\n[4] Noise mixed at a 10 dB target: realized {realized:.2f} dB")
assert abs(realized - 10.0) < 0.01
# [5] write the source and four conditions
noisy_mu = mulaw_decode(mulaw_encode(noisy))
conditions = [
("00_source_16k.wav", src, SR_WIDE),
("01_phone_mulaw.wav", phone_mu, SR_PHONE),
("02_phone_mulaw_loss5.wav", zero5, SR_PHONE),
("03_phone_mulaw_snr10.wav", noisy_mu, SR_PHONE),
("04_phone_mulaw_snr10_loss5.wav", apply_loss(noisy_mu, mask5, "zero"), SR_PHONE),
]
print(f"\n[5] Files in {args.out}/ (16-bit PCM WAV)")
for name, sig, sr in conditions:
size = write_wav(os.path.join(args.out, name), sig, sr)
print(f" {name:32s} {sr:5d} Hz {size:8,d} bytes")
if not args.wav:
assert size == 44 + 2 * len(sig) == 44 + 2 * 4 * sr
# [6] which conditions to run
factors = matrix_factors(world)
rows = condition_matrix(factors)
full = int(np.prod([len(v) for v in factors.values()]))
covered, needed = pair_coverage(rows, factors)
triples = {(row["carrier"], row["codec_chain"], row["loss"]) for row in rows}
with open(os.path.join(args.out, "conditions.csv"), "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=list(factors))
writer.writeheader()
writer.writerows(rows)
print(f"\n[6] Condition matrix: full factorial {full} cells; design {len(rows)} rows "
f"({len(triples)} distinct carrier x codec chain x loss cells)")
print(f" level pairs covered: {covered} of {needed} across all 10 factor pairs")
for row in rows[:3]:
print(" " + ", ".join(row.values()))
assert full == 324 and len(rows) == 36 and len(triples) == 36 and covered == needed
print("\nAll checks passed.")
if __name__ == "__main__":
main()
Step 3: Run it
.venv/bin/python telephony-degradation-pipeline.py
Running it with Python 3.11 prints:
Synthetic data.
Edition: global. Dataset: steelyard-calls (synthetic, seed 20261121)
Source: synthetic chirp 100 Hz -> 7 kHz + 1 kHz and 5 kHz tones, 4.00 s at 16000 Hz
[1] Energy above 3.4 kHz, as a share of each signal's total
source, 16 kHz -2.9 dB
after 300-3400 Hz band-pass < -200 dB
after resample to 8 kHz -68.6 dB
after mu-law round trip -46.0 dB
source energy below 300 Hz -16.4 dB (also removed by the band-pass)
drop through band-pass + 8 kHz: 65.7 dB; after mu-law: 43.1 dB (mu-law noise is broadband)
[2] Round-trip SNR on the 8 kHz phone-band signal
level mu-law 8-bit linear 8-bit
peak -3 dBFS 37.7 dB 40.9 dB
peak -33 dBFS 34.8 dB 9.5 dB
mu-law moves 2.9 dB over a 30 dB level drop; linear moves 31.4 dB
mu-law payload: 32,000 bytes = 200 frames of 160 bytes (20 ms each)
[3] Gilbert-Elliott loss on 20 ms frames, r = 0.5 (mean burst 2.0 frames = 40 ms), 50,000 frames x 10 seeds
target p realized loss (min-max) mean burst netem equivalent
1% 0.00505 0.0102 (0.0093-0.0122) 1.99 netem loss gemodel 0.51% 50% 100% 0%
5% 0.02632 0.0502 (0.0486-0.0526) 1.99 netem loss gemodel 2.63% 50% 100% 0%
10% 0.05556 0.1002 (0.0957-0.1046) 1.99 netem loss gemodel 5.56% 50% 100% 0%
this clip at 5%: 6 of 200 frames lost, bursts [1, 1, 1, 3]
waveform SNR vs the unlost phone signal: zero fill 16.1 dB, repeat fill 15.8 dB
[4] Noise mixed at a 10 dB target: realized 10.00 dB
[5] Files in phone_path_out/ (16-bit PCM WAV)
00_source_16k.wav 16000 Hz 128,044 bytes
01_phone_mulaw.wav 8000 Hz 64,044 bytes
02_phone_mulaw_loss5.wav 8000 Hz 64,044 bytes
03_phone_mulaw_snr10.wav 8000 Hz 64,044 bytes
04_phone_mulaw_snr10_loss5.wav 8000 Hz 64,044 bytes
[6] Condition matrix: full factorial 324 cells; design 36 rows (36 distinct carrier x codec chain x loss cells)
level pairs covered: 102 of 102 across all 10 factor pairs
carrier-a-mobile, amrnb->mulaw, 1%, clean, English
carrier-a-mobile, amrnb->mulaw, 5%, 20 dB, Spanglish
carrier-a-mobile, amrnb->mulaw, 10%, 10 dB, Spanish-dominant
All checks passed.
The run takes about a second and writes six files to phone_path_out/: the five WAVs and conditions.csv.
Step 4: Check what the line removed
Above 3.4 kHz the source carries -2.9 dB of its energy, about half of it, because the chirp spends half its sweep up there and the 5 kHz tone sits there the whole time. After the band-pass and the move to 8 kHz that share is -68.6 dB, a drop of 65.7 dB. The 1 kHz tone survives. The 5 kHz tone and the top half of the chirp are gone, and nothing downstream can bring them back.
The mu-law round trip lifts the share to -46.0 dB. That is not the filter failing: quantization noise is spread across the whole band, so some of it lands above 3.4 kHz. The path is still 43.1 dB down, more than the 40 dB the script asserts.
The FFT mask is a brick wall, and it rings in time around sharp onsets. That is fine for building intuition and test files. For numbers you will defend, use a reference filter: the ITU-T Software Tool Library (G.191) carries reference C code for G.711, its Appendix I concealment, an error insertion device and the FIR and IIR filters, and it is a more defensible phone-path simulator than ad hoc code 5.
Step 5: Read the mu-law numbers
On the phone-band signal at -3 dBFS peak, mu-law gives 37.7 dB round-trip SNR and plain 8-bit linear gives 40.9 dB. At -33 dBFS, mu-law still gives 34.8 dB and linear falls to 9.5 dB. Mu-law moved 2.9 dB over a 30 dB drop in level; linear moved 31.4 dB. That is the job companding does: a quiet borrower on a bad handset gets almost the same coding accuracy as a loud one.
So when a TTS voice sounds worse on calls, mu-law itself is the wrong suspect. ETSI TR 102 948 rates G.711 as natural sounding, with no perceptual degradation (an informal expert judgment); blame the bandwidth and any transcoding in the path 6. For STT, the 34.8 to 37.7 dB round-trip SNR above points the same way. The script's encoder uses the continuous mu = 255 curve, not G.711's segment table, so its SNR is close to a real codec's but not bit-exact. Swap in the STL's G.711 code when you need bit-exact output.
The payload line is the other check: 4 seconds at 8 kHz is 32,000 mu-law bytes, exactly 200 frames of 160 bytes. Generate outbound test audio in whole frames too.
Step 6: Check the burst loss, then set it on a live call
A two-state Gilbert-Elliott model sits in a good state (frames arrive) or a bad state (frames are lost). With p the chance of going bad and r the chance of recovering, the long-run loss rate is p/(p+r) and the mean burst is 1/r frames. The script fixes r = 0.5 (40 ms bursts on average) and solves for p. Over 10 seeds of 50,000 frames each, the realized loss is 0.0102, 0.0502 and 0.1002 for targets of 1%, 5% and 10%, and the mean burst is 1.99 frames at every target. The model does what the formula says.
The single clip does not. At a 5% target, this seed lost 6 of the clip's 200 frames, 3%, in bursts of 1, 1, 1 and 3 frames; the longest gap is 60 ms. Another seed on the same clip lost 20 frames, with one burst of 9 frames, 180 ms of silence in a 4-second utterance. A short clip draws its own loss rate. Log the realized mask with every file, and when you report STT accuracy under loss, report it against the realized rate per clip, not the target.
The two fill rules also say something. Against the unlost signal, zero fill scores 16.1 dB of waveform SNR and repeating the last good frame scores 15.8 dB. Waveform SNR punishes the repeat because a replayed frame is out of phase with the audio it replaces, while a listener may prefer it to a gap. Neither number tells you what your STT will do. In the ICASSP 2024 packet loss concealment challenge, the two co-winners raised overall quality from 2.37 for the unconcealed lossy audio to 3.49 and 3.44, but their word accuracy was 0.81 and 0.84 against 0.83 for the lossy input 7. Judge any concealment that feeds STT by word accuracy on your own audio.
Offline frame dropping tests the recognizer. It does not test the jitter buffer, the concealment in your media stack, voice activity detection or barge-in, which fail differently on a live call. Linux netem can impose the same burst model on a SIP test call's network path, using gemodel PERCENT [ R [ 1-H [ 1-K ]]]: the chance of entering the bad state, of leaving it, and the loss probabilities in the bad and good states 8. The last column of the output prints the matching settings, with every bad-state frame lost and no good-state loss. netem shapes only the packets leaving an interface, so set it on the box that sends the caller's audio toward your agent; on the agent's own box it impairs the TTS leg instead:
# on the caller-side test box's interface carrying the SIP media (5% target, 40 ms mean bursts)
sudo tc qdisc add dev eth0 root netem loss gemodel 2.63% 50% 100% 0%
sudo tc qdisc del dev eth0 root netem
Run both. The offline files tell you whether the model copes with missing audio; the live call tells you whether the rest of the stack does.
Step 7: Run your STT on the five files
Send each WAV in phone_path_out/ to the STT you are evaluating, in the same streaming mode and language setting you use in production, and score the transcripts with the entity error rate recipe, not WER alone. For Steelyard the number that matters is whether the amount survives each condition: "two fifty" must not come back as "two fifteen". A vendor that holds its amount entity error rate from 00_source_16k.wav to 04_phone_mulaw_snr10_loss5.wav is the one to shortlist; the bake-off ranked vendors on wideband app audio, the equivalent of the first file only.
Then run the matrix. conditions.csv holds 36 rows out of a full factorial of 4 carriers x 3 codec chains x 3 loss levels x 3 SNRs x 3 language mixes = 324 cells. Every carrier x codec chain x loss combination appears once, and SNR and language mix are assigned with modular arithmetic so that all 102 pairs of levels across the 10 factor pairs appear at least once. The script checks that by brute force. The global levels are three fictional US mobile carrier profiles and one VoIP leg, crossed with AMR-NB, AMR-WB and Opus, each transcoded to G.711 mu-law, and the English, Spanish-dominant and Spanglish language groups. This script implements only the mu-law hop. The AMR-NB, AMR-WB and Opus legs each need a licensed or reference codec build before those rows mean anything; until then, mark them "not run".
Two rules go with the matrix. STT and TTS gates run only on phone-path audio, and wideband results go in a separate column that never gates. And keep a real 8 kHz holdout from consented calls, because a simulator only covers the impairments you thought to model.
Step 8: Use your own recording
Synthetic chirps show that each stage works. To see what the path does to speech, record a consented volunteer reading Steelyard-style lines in English or Spanish (an amount such as "two fifty" or "mil quinientos" [fifteen hundred], a date, a loan tail) as mono 16-bit PCM at 16 kHz and pass it in:
.venv/bin/python telephony-degradation-pipeline.py --wav my_consented_16k.wav --out my_phone_path
The clip is trimmed to whole 20 ms frames and the stage checks still run, except the ones tuned to the chirp. The file-size check is skipped, because your clip length is your own, and so are the checks that freeze the chirp's printed numbers and its seeded loss draw. The two energy thresholds are tuned to the chirp: speech carries far less energy above 3.4 kHz to begin with, so after mu-law the drop is smaller, and the script asserts only that band-pass plus 8 kHz leaves less than -40 dB up there. The mu-law flatness threshold is also chirp-only, because speech's higher crest factor costs mu-law a few dB at -33 dBFS peak; the script still asserts that mu-law beats 8-bit linear by more than 20 dB at that level. Do not feed it audio from voices whose license forbids commercial use, and do not paste their output anywhere you publish.
The full file in the course repository also carries the India edition's constants block, which the copy above leaves out. Run it with --edition india for the India edition's data (its own seed, codec chains and language mixes); the India chapter explains that output. The energy, mu-law SNR, payload, file-size and matrix-count lines are identical in both editions; the burst-loss lines differ because the seed does.
What to do next
Score every file this script writes with the entity error rate recipe, slice the results by condition, and gate on the phone-path columns only. The telephony audio chapter covers which audio quality metric fits which direction of the call and why codec penalties differ by language and corpus. When the offline numbers hold, repeat the loss settings with netem on a live SIP test call before release.