Reference
Evals Glossary
Every part of Evals from First Principles ends with a glossary that defines the terms it introduced. This page collects them in one place. Pick a part below and you land directly on its glossary, so you can look up a definition without re-reading the prose.
- Part 1: What Is a Score? Glossary
- Part 2: Building a Golden Set Glossary
- Part 3: Do Humans Even Agree? Glossary
- Part 4: LLM-as-Judge by Hand Glossary
- Part 5: Judging the Judge Glossary
- Part 6: Uncertainty Glossary
- Part 7: Is the Difference Real? Glossary
- Part 8: pass@k Glossary
- Part 9: Arenas Glossary
- Part 10: Calibration Glossary
- Part 11: Regression Gates Glossary
- Part 12: The Eval Flywheel Glossary
- Part 13: Online Evals Glossary
- Part 14: Agent-Trajectory Evals Glossary
- Part 15: A Tiny Eval Harness Glossary