One Axis, Three Benchmarks
26 min readA chart of Terminal-Bench 4.0 scores by reasoning effort looks like one benchmark run seventeen times. It is three different evaluations sharing a y-axis. What the chart supports, what it does not, and how to tell the difference from the numbers alone.
- evals
- llm
- agents
- benchmarks
Nine Models, Two Results
17 min readA ranking table with nine sorted rows looks like nine findings. This one supports two. How to read an eval leaderboard by its diagnostics instead of its order — and why the diagnostic that looks fatal is fine while the one that looks clean says almost nothing.
- evals
- llm
- statistics
- ranking
The Model Ranked Third Never Performs Third
10 min readTwo models finished 0.01 apart on the same leaderboard. One scored between 2 and 4 on eight of twelve prompts; the other was above 4 six times and below 2 four times. Why a mean over a prompt set destroys the property that decides whether you can ship.
- evals
- llm
- statistics
- deployment
Your Ten-Dimension Rubric Measures One Thing
11 min readA radar chart with ten axes promises ten independent signals about a model. In this study 97.6% of the variance sits on a single factor — and the rubric had explicitly instructed the judge not to let that happen.
- evals
- llm
- rubrics
- statistics
What Quantization Really Does to LLM Accuracy
31 min readFP8, Q4, GGUF — the bit count on the label tells you less than you think. What actually happens to model quality when you compress an LLM, worked through DeepSeek-V4-Flash-0731 and Llama 3.3 70B.
- llm
- quantization
- inference
- deepseek