Notes on building software, by Matthew Scott.https://www.mattmakes.io/bloghttps://www.mattmakes.io/blog/one-axis-three-benchmarkshttps://www.mattmakes.io/blog/one-axis-three-benchmarksA chart of Terminal-Bench 4.0 scores by reasoning effort looks like one benchmark run seventeen times. It is three different evaluations sharing a y-axis. What the chart supports, what it does not, and how to tell the difference from the numbers alone.Fri, 25 Sep 2026 00:00:00 GMThttps://www.mattmakes.io/blog/nine-models-two-resultshttps://www.mattmakes.io/blog/nine-models-two-resultsA ranking table with nine sorted rows looks like nine findings. This one supports two. How to read an eval leaderboard by its diagnostics instead of its order — and why the diagnostic that looks fatal is fine while the one that looks clean says almost nothing.Fri, 07 Aug 2026 00:00:00 GMThttps://www.mattmakes.io/blog/ranked-third-never-thirdhttps://www.mattmakes.io/blog/ranked-third-never-thirdTwo models finished 0.01 apart on the same leaderboard. One scored between 2 and 4 on eight of twelve prompts; the other was above 4 six times and below 2 four times. Why a mean over a prompt set destroys the property that decides whether you can ship.Fri, 07 Aug 2026 00:00:00 GMThttps://www.mattmakes.io/blog/ten-dimensions-one-factorhttps://www.mattmakes.io/blog/ten-dimensions-one-factorA radar chart with ten axes promises ten independent signals about a model. In this study 97.6% of the variance sits on a single factor — and the rubric had explicitly instructed the judge not to let that happen.Fri, 07 Aug 2026 00:00:00 GMThttps://www.mattmakes.io/blog/what-quantization-really-does-to-llm-accuracyhttps://www.mattmakes.io/blog/what-quantization-really-does-to-llm-accuracyFP8, Q4, GGUF — the bit count on the label tells you less than you think. What actually happens to model quality when you compress an LLM, worked through DeepSeek-V4-Flash-0731 and Llama 3.3 70B.Sun, 02 Aug 2026 00:00:00 GMT