When Your Model Is Wrong, Does It Crash or Does It Lie?

Every benchmark I have ever published, including my own, answers one question: how often is the model right?

Nobody publishes the other one, which for anyone actually shipping this stuff matters more: what happens when it's wrong?

Two models at 70% can be completely different products. One of them falls over loudly and a retry loop picks it up. The other hands you a clean, plausible, wrong answer and says nothing. Same score. Wildly different risk.

So I measured it.

Failures come in four kinds, and only one is expensive

The eval harness I've been building generates a Python script per task, runs it, and grades the file it produces. That gives me something a score alone throws away — I know how each failure failed:

	What happened	What catches it	Cost to survive
crash	the script raised	the traceback	a retry loop, ~40 lines
noop	exited 0, wrote nothing	path.exists()	one line
corrupt	exited 0, file won't parse	trying to open it	one line
lie	exited 0, file parses, content is wrong	nothing but a grader	a whole eval harness

The first three are all loud in the sense that something in your pipeline can notice them without knowing the right answer. Only the last one requires you to already know what the output should have been.

That's the number I care about: what share of a model's failures are the kind only a grader could catch? Call it the lie rate.

I'll admit that I built this metric wrong three times before it was right, and the third one is the interesting one.

The first version counted "exited 0 and wrote nothing" as a lie — but a one-line file-exists check catches that, no grader needed. The second counted a corrupt, unparseable file as a lie — same problem, you find that the instant you try to open it. Both inflated the number the whole article rests on.

The third was worse, because it would have produced a confident finding that was entirely false. I'd planned a third model — Phi-4-reasoning, same 14B as Phi-4, to test whether reasoning training moves failure character at fixed size. It came back at 47 crashes out of 48, which looked like a spectacular result about reasoning models.

It was my code extractor. Reasoning models emit a long preamble before the answer, so the extractor strips fences and, failing that, slices from the first line that looks like code. When neither matched, it returned the raw text — and I fed a paragraph of English to the Python interpreter, which raised SyntaxError, which my classifier faithfully recorded as the model crashing.

The model hadn't crashed. It had never produced a script at all. It spent its entire token budget looping on the same paragraph about where a table's boundaries might be, and got truncated mid-sentence.

A harness bug that produces zeroes is annoying. A harness bug that produces a plausible, publishable, wrong result is the thing this entire post is about — my own tooling failed in exactly the silent, confident way I was measuring models for. I caught it only because 47/48 was too clean a number to believe.

The fix is one line: when there's no fence and no code marker, return empty rather than prose. An honest empty is a signal. Fake code is a lie.

Which is the same lesson as the 19% that wasn't real, wearing a different hat: an eval you haven't validated is a random number generator with a spreadsheet attached, and your own new metric is an eval you haven't validated.

The setup

Phi, because Microsoft coined the "small language model" category and it's the closest thing this space has to a canonical family. Phi-4-mini-instruct (~3.8B) against Phi-4 (14B) — same trainer, same data recipe, roughly 3.7× the parameters.

One serving stack, deliberately. Both deployed to the same Azure AI Foundry resource. Last post I showed that the same model served by different providers is a different product — different quantization, different output caps, different honoured parameters. Running a size ladder across mixed providers would import exactly that noise into the experiment. So: one provider, one config, one variable.

36 tasks across Excel, Word and PowerPoint, three variants each, all must pass. 108 variants per model. Identical prompts, identical grader, identical everything except the deployment name.

The result

216 variants. 134 failures. Every one classified.

	pass	crash	noop	corrupt	lie	lie% of failures
Phi-4-mini (3.8B)	32 (30%)	56	1	0	19	25%
Phi-4 (14B)	50 (46%)	29	0	2	27	47%

The percentages are the headline, but the raw counts are the thing worth staring at:

total failures    76 → 58    down 24%
crashes           56 → 29    down 48%
silent lies       19 → 27    UP 42%

Scaling from 3.8B to 14B halved the crashes and increased the lies — in absolute numbers, not just as a share. χ² = 6.78, p = 0.009.

The better model was wrong less often, and more dangerously each time.

Why this happens

It's not mysterious once you see the failure text.

The small model's failures are overwhelmingly knowledge failures. It invents insert_cols(width=...), an openpyxl signature that doesn't exist. It adds an integer to a string. It calls a method with the wrong arity. Those are gaps in memorised API surface, and more parameters means more memorised API surface — so they're exactly what scaling fixes first.

What scaling doesn't fix at the same rate is judgment. Putting the right value in the wrong column. Swapping two headers. Writing the bullets into the title placeholder and leaving the title blank. Here's one from the 14B model, and it's a perfectly valid spreadsheet:

D1: got 'Revenue'   want 'UnitPrice'
E1: got 'UnitPrice' want 'Revenue'

Nothing raised. Nothing was missing. The file opens fine. Every number in it is a real number. It's just wrong, and the only thing in the universe that knows it's wrong is a grader holding the expected answer.

So the two failure curves come apart as capability rises. The loud ones fall away quickly. The quiet ones don't.

The consequence nobody prices

This inverts the usual procurement instinct, and it's the practical point of the whole exercise:

A better model doesn't reduce your need for an eval. It increases it.

Fewer failures, but a much higher proportion of them are invisible to everything except a grader. The cheap noisy model is the one your harness can rescue — hand it its own traceback and it fixes itself. The good quiet one will write a wrong invoice, save it successfully, exit zero, and never mention it.

If you are choosing a model on score alone, you are choosing on the wrong axis. The question isn't how often is it right. It's when it's wrong, does it crash or does it lie — because the first is an afternoon of harness work and the second is an ongoing obligation.

What I got wrong

Two things, in the interest of the numbers meaning something.

The per-format results contradict each other. I saw the Excel row first — 5% lie rate at 3.8B, 40% at 14B — and said the finding out loud before the other formats landed. Then Word came back inverted: 45% at 3.8B, 8% at 14B. Individual formats have only 11–25 failures in them, which is too few to read. The effect is real at the aggregate, where the samples are 76 and 58. It is not reliable cell by cell, and I'm not defending any single row.

This suite is noisier than it looks. Two identical runs of the same model scored 19% and 0% at task level, because a task passes only if all three variants pass — one flaky generation swings a whole task. At variant level those same runs were 21% and 17%. That all-or-nothing rule is right for catching a solver that memorised a cell position, which is what it was built for, and wrong as a unit for comparing models. Everything above is variant-level.

Caveats

Two points is not a ladder, and I tried twice to make it three. DeepSeek-V3.1 was meant to be the large control; it returned HTTP 410 model_deprecated on all 108 calls. Phi-4-reasoning was meant to separate capability from parameter count; once the extractor bug was fixed, the honest answer is that I can't measure it on this infrastructure. These deployments are capped at 20,000 tokens per minute, Azure reserves max_tokens against that budget rather than billing actual output, and a single reasoning call large enough to finish didn't return within five minutes. Not a result about the model — a result about my quota.

So the size question is answered with two points, and the more interesting question — is this about parameter count or about capability? — is left open. I'd rather leave it open than answer it with a number I had to torture.

One family, one serving stack, n=1. There's no cross-family control at the small end, so a surprising Phi-4-mini row can't be fully separated from a Phi quirk. Read it as "these two models," not "3.8B models."

The mechanism is a hypothesis, not a measurement. I'm inferring "scaling buys API knowledge faster than judgment" from the content of the failures. That's a plausible story consistent with the data. It isn't proven by it.

A free lesson from the control that died

The DeepSeek deployment deserves its own paragraph, because it taught me more than it would have as a data point.

It had been deployed for months. It still shows provisioningState: Succeeded in the Azure control plane today. Ask it a question and you get:

HTTP 410  {"code":"model_deprecated"}

Nobody told us. It was discovered because a job that depended on it silently produced nothing — and earlier that same day I'd asserted it "still serves," an assumption I never tested.

Here's the part that matters. My harness has a separate bucket for empty response versus model produced a wrong answer. Without that distinction, a dead endpoint would have been recorded as a model scoring zero, and I'd have published a chart showing DeepSeek-V3.1 at 0% against Phi-4 at 46%.

The deprecation didn't just remove a model. It tried to masquerade as a capability result. Instrumentation is the only reason it didn't.

So

A score is a summary of the times a model was right. Your risk lives entirely in the times it was wrong, and score says nothing about those.

Measure the shape of your failures, not just the count. Crashes are cheap — a retry loop and a traceback and you're most of the way home. Lies cost you an eval harness, forever, and the better your model gets, the larger the share of your remaining failures they become.

Build the ruler first. By now it's less a recommendation than a running joke in this series, but it keeps being the answer: it was the only thing that caught two bugs in my own metric, and the only thing standing between me and publishing a dead endpoint as a benchmark result.