Frontend engineers own the surface where model quality is actually felt: the streaming transcript, the retry button, the thumbs-down that nobody clicks. This track is for staff-plus frontend and full-stack engineers who ship conversational and AI-assisted product surfaces in TypeScript. The premise: perceived quality is measurable, the tooling can live in your existing Vitest setup, and the client is an instrument, not just a renderer. Two steps are hands-on recipes that take about an hour each.
The order is deliberate. Steps one and two build the mental model and the first artifact, in Python because that recipe is the shortest path to understanding the loop. Step three makes the loop native to your stack. Steps four and five are the quality-measurement craft: rubric-driven judges and the review UI you are better positioned to build than anyone else on the team. Steps six through eight cover the surfaces themselves: chat, multi-turn state, voice. Step nine moves the whole thing online.
The track
- Why evals are the moat (12 min). The three reasons LLM features rot (non-determinism, subjectivity, drift) and why ship-and-pray fails for AI. Hamel's argument is the canonical one 1.
- Your first eval in 30 minutes (30 min). Fifty labeled traces, a binary judge, a confusion matrix. The concepts transfer to TypeScript unchanged; do it once here first.
- TypeScript eval with Vercel AI SDK (60 min). Vitest as the eval runner, AI SDK calls under test, a zod-typed binary judge, and cost accounting that prints with the test summary 2 3. The recipe targets AI SDK 7 (released June 25, 2026) and Vitest 5 (September 3, 2026, Node 22.12 or later), so a project still on AI SDK 6 or Vitest 4 needs the version note at the top of the recipe 4 5.
- LLM-as-Judge: the practical introduction (10 min). Perceived quality needs a rubric, and a judge is how the rubric runs at scale. Binary plus critique beats 1-to-5 scales 6. Strong LLM judges matched human preferences with over 80% agreement in the MT-Bench study, the baseline case for using one at all 7.
- Build your trace viewer in an afternoon (12 min). The single highest-return tool in an eval program is a review UI, and that is frontend work. Streamlit and Next.js variants with the minimum feature set 8.
- Conversational quality (12 min). The task-type playbook for chat surfaces: persona consistency, context retention, recovery from misunderstanding, scored per turn 6.
- Multi-turn conversation eval (75 min). The hands-on version: per-turn binary rubrics plus a per-session state check, logged to Langfuse 9.
- Voice: STT, TTS, conversational (12 min). WER for transcription, MOS panels for synthesis, time-to-first-byte and barge-in for full duplex. P.808 (edition 06/21, still in force as of October 2026) is the crowdsourced listening-test standard 10.
- Online vs offline: where each fires (14 min). Offline evals on frozen datasets, online evals on sampled traffic, which Huyen frames as spot checks on a subset rather than exhaustive scoring of every request 11. The client's job is the join key: have the backend return the trace id to the browser, so a thumbs-down posts as a score on that trace 12.
What comes after the track
When your assistant starts calling tools, the multi-turn work extends into agentic evals: trajectory vs outcome scoring is the next concept up. On the judge side, biases and mitigations is the page to read before you trust a judge score in a launch review.
TIP
Step 5 pays for the whole track. Hamel's field guide says teams with thoughtfully designed data viewers iterate 10x faster than teams without them 8, and labeled data is the input to every other step here.