gemma3:4b looked at a diff where I had renamed one file and changed nothing inside it, and wrote refactor: update authentication module. Nothing was refactored. Nothing was updated. But it read like a commit message — right register, right verb, right shape — and if I hadn't been watching it would have gone into the log and stayed there.
That's the failure mode I've spent the last year building against: a model that is wrong in a way the user can't detect. Not wrong loudly. Wrong smoothly, in well-formed language, in exactly the tone you were expecting. Three times, across three different projects, I hit that wall and made the same call — constrain the feature toward honesty, even when the output got less fluent, slower, or less fun to demo. Here's what each of those decisions looked like in code, and what it cost me.
The commit tool that got worse on purpose
Verbatim Commit is a small CLI that reads your staged diff and writes the commit message. The first version ran gemma3:4b through Ollama, because it's fast, it's local, and 4B is plenty for a sentence and a half.
It was plenty for a sentence and a half. It was not enough to be honest about a diff. It labeled renames as updates. It described the intent behind a change it had no way of knowing. Given a diff that touched three files, it would sometimes narrate a fourth. The messages were fluent, conventional-commits-shaped, and quietly fictional.
For most generated text you can shrug at that. Not here. A commit message is a permanent artifact that other people read as fact. Six months from now someone bisects a regression, reads fix: handle null user session, and trusts it. If the model made that up, I haven't saved anyone time — I've poisoned the record and made it harder to find the bug than if the message just said wip.
Two fixes. First, the prompt stopped asking for a good commit message and started asking for a constrained one: describe only what the diff shows, use the change type the diff actually supports, never state a rationale that isn't visible in the code. Second — the part I sat on for a while — I moved the default model to qwen2.5-coder:7b. And I kept the verify mode as a git hook, so the check runs whether or not you used the tool to generate the message.
I measured the cost rather than assert it, and the number I was ready to report is not the number I got. On speed, roughly what you'd expect: the 7b takes about 4.0 seconds at p50 against 2.5 for the 4b, and its slow tail is meaningfully worse — 38% of runs cross five seconds, where the 4b crosses it 9% of the time. Some of that gap is qwen writing more rather than computing slower; normalized per character it's still about 21% slower, so the direction holds.
The memory cost I was about to concede doesn't exist. As the CLI actually loads them, the 7b uses roughly a gigabyte less resident memory than the 4b, because gemma3:4b silently claims a 128k context window and qwen only asks for 32k. Force both to the same context and the 7b costs about 0.35 GB more — real, but a fraction of the figure I'd have written down from memory.
I'm leaving that in rather than quietly correcting the sentence, because it's the same failure this whole post is about. I had a confident, plausible, well-shaped claim about a tradeoff I'd made. I had no measurement behind it. And it was wrong in the direction that made the decision look more principled than it was. On a thin laptop the 4b is still the nicer thing to live with — for latency, which I'd have gotten right, and not for memory, which I'd have gotten backwards.
Worth separating two things I could easily let blur together: the default and what I actually run. Day to day I use gemma4:12b-mlx, on a machine that can afford it. That's not what ships. A default has to be right for someone I'll never meet, on hardware I can't see, who will never open the config file — which is a stricter question than what's best on my laptop, and the reason the answer isn't just "the biggest model I can run."
I took the trade anyway, because a slower tool that tells the truth about your diff beats a fast one whose output you have to read against the diff to trust. If I have to check it, it hasn't done the job.
The score I refused to let the model guess
Resuzen shows you a match score between your résumé and a job description. The easy implementation is four lines: paste both into the context, ask the model to rate the fit out of 100, render the number.
I built that version. It works, in the sense that it returns a number. It fails in the sense that the number is the product. Ask twice and you get 72 and 78. Ask nicely and you get a better score. And the direction of the drift is not random — a model asked to evaluate someone's work, with that person's work sitting in the context, flatters. That's not a scoring feature. That's a feature that talks somebody into applying for a job they're not ready for, and charges them for the privilege.
So I split it. The model does the part it's actually good at: reading a job description written in human prose and pulling out the skills, tools, and requirements it's really asking for — including the ones phrased three different ways in three different bullets. That's fuzzy linguistic work and code is bad at it. Then the coverage calculation happens in code. Deterministic set comparison, weighted, same inputs in and same number out, every single time.
The split is the whole insight, and it generalizes further than résumés: use the model for the fuzzy part, use code for the part that has to be reproducible. Anything a user will treat as a measurement should be computed, not generated.
The side effect showed up in interview prep. Because the pipeline knows exactly which requirements the résumé doesn't cover — not "the model felt it was a 78" but a concrete list — it can name the gaps and drill you on them. A model asked to score would have smoothed those gaps over. The deterministic version can't. It's the least flattering feature I've built and the most useful one.
The one I haven't fixed yet
The third project is where the rule stops being a story about good decisions.
Fixr answers vehicle maintenance questions. Users ask it when a part is due for replacement and what a repair should run them, and then they spend money based on the answer. Of the three projects here, it has the highest stakes and the weakest safeguards, and I'd rather say that in public than let a case study imply otherwise.
Here is what actually happens when you ask Fixy a question. The message goes to a single Gemini call, the response comes back, and it renders. Nothing sits between the model's text and your screen. No source is consulted, no claim is scored, and there's no branch anywhere that drops a number the system can't stand behind. The prompt asks for a typical cost and a recommended interval in every answer, which means the most checkable thing in the response is also the thing with nothing behind it.
What I do have is prompt instruction — lines telling the model to add verification language and not to guess at specs it isn't sure about. That's the model policing itself, and I want to be precise about what that's worth. An unenforced instruction is a preference. It is not a constraint, and I shouldn't have been counting it as one.
The part that's harder to write: I made this worse on purpose, once. An early version of the prompt said that when the model was unsure about vehicle-specific details, it should point the user to their owner's manual or a mechanic. I replaced that with an instruction to provide specific specs when confident, plus a hedge for critical values like torque and fluid capacities. The reasoning was real — generic advice is available anywhere, and vague answers were the top complaint. But look at the direction. I made the product more confident before I made it more verifiable, and I solved a vagueness problem by removing the caution rather than by adding a way to check.
There's a disclaimer, and it only appears on the empty state, before you've asked anything. The moment an answer shows up, it's gone. A disclaimer that isn't next to the claim isn't doing the job I told myself it was doing.
The frustrating part is that the checking source already exists in the repo. There's a maintenance schedule table with real intervals and cost ranges, and it works fine — it just feeds a separate, non-AI suggestions path that does plain arithmetic against your service records. The chat action never imports it.
Wiring it up isn't complicated, and I can describe the version I'd build. Fixy would return its numbers as structured data alongside the prose instead of burying them in a sentence. Code would compare those numbers against the table and render one of three states next to the claim: quiet when it's in range, both figures shown when it isn't, and marked unverified when the table has no entry for that service. The disclaimer would move from the empty state, where it currently sits, to sitting under every answer that carries a number.
I'm not building it this quarter. I want to be straight about that rather than let a description of the plan read like a description of the product. And even the version above is weaker than it sounds: the table is generic by service type, not per-vehicle, so it can catch a $600 oil change but it can't tell you whether an interval is right for your car. Actual per-vehicle verification needs manufacturer data I don't have access to. What's realistically on the table is visible uncertainty, not verified answers, and it's still sitting behind other work.
I've made this call once before, somewhere the stakes were public instead of personal. I built a public crime statistics page that pulled from two government sources. They used different schemas, and for the same reporting window they disagreed on the counts. The clean version picks the source you trust more, or splits the difference, and ships one confident number. What went out instead showed the disagreement — both figures, both sources, right there on the page. It reads worse. It's harder to screenshot. But a reader has no way to audit the underlying data, and the alternative was handing them my judgment call dressed up as a fact.
The rule I've been describing is easy to follow while you're building something. It's much harder once the thing has shipped, works, and nobody is complaining.
The pattern
Written out, the rule I keep landing on is boring, which is probably a good sign:
- Decide which part of the feature has to be reproducible, and move that part into code.
- Let the model do the linguistic work — parsing messy human text, summarizing, rephrasing — because that's what it's genuinely better at than code.
- If the model can be wrong in a way the user can't detect, add a verification layer or don't ship the feature.
- Given two defaults, prefer the one that fails visibly over the one that fails fluently.
Now the part that makes this a position rather than a brag: this rule costs me, every time, and the costs are not hypothetical.
The honest defaults are slower and hungrier. The honest score is lower than the flattering one, which is a worse first impression on a product where the first impression is most of the funnel. The verified answer sometimes comes back as "I can't confirm that," which is the least impressive thing a demo can do — and I have watched people's faces when it happens. A tool that says "I don't know" in a room full of tools that say something is not the one that gets remembered.
I should be clear about what I can't show you. In all three cases, I'm the one who decided the honest version was better. No user asked for a lower match score. Nobody has written in to say a commit message misled them six months later. I don't have a retention curve proving people prefer a tool that sometimes says it can't confirm something. If you came here for evidence that honest defaults win on the metrics, I don't have it, and I'd be a little suspicious of anyone who claims it this cleanly. What I have is a standard I'm not willing to trade away, and three repositories where you can check whether I actually held to it. Those are different claims, and I'd rather not blur them together.
I take the trade because of the asymmetry. When a fluent-but-wrong feature fails, it fails silently, at the exact moment the user has decided to trust it — the commit six months later, the application to the wrong job, the repair that costs more than the car is worth. When an honest feature fails, it fails in front of you, while you can still do something about it.
The standard
I'd rather ship something that makes me look slightly less clever and is right. That's the whole position. It's not a philosophy of AI, it's a build rule, and the only way to argue it is to show the code where it applied.
So: Verbatim Commit, Resuzen, and Fixr — two where I applied the rule and one where I'm still owing it. Go check my work. That's the point of writing it down.