Head-to-Head Value Test: GLM-5.3 Flash vs. Qwen3 Coder Next vs. DeepSeek V3.2
When you’re running an autonomous coding agent, the question that actually matters isn’t “which model writes the best code?” It’s “how much does it cost to get from task assigned to tests pass?” Those are very different questions, and optimizing for the wrong one is an easy way to burn money without making progress.
I put that idea to the test on a real project: fclpy, a Python-based ANSI Common Lisp interpreter I’ve been building. The task was ANSI-compliance work — fixing failing conformance tests against a fixed, unambiguous target. That makes it a great benchmark, because a “fix” that breaks other tests isn’t just wasted effort, it’s negative progress. There’s nowhere to hide a regression.
The lineup
As of late August 2026, the shortlist of cheap-but-capable coding models on OpenRouter looked something like this:
| Model | Input / M | Output / M | Notes |
|---|---|---|---|
| Qwen3 Coder Next | $0.12 | $0.80 | 262K context, built for agent workflows |
| DeepSeek V3.2 | ~$0.21 | ~$0.31 | Startlingly cheap output pricing |
| Qwen3 Coder Flash | $0.195 | $0.975 | 1M context |
| DeepSeek V3.1 | $0.25 | $0.95 | 671B/37B MoE, strong tool-use |
| GLM 4.5 Air | $0.13 | $0.85 | |
| GLM 5.3 (Flash) | $0.15 (on sale: $0.075) | $0.50 (on sale: $0.25) | Cache-read pricing is what actually matters here |
On paper, several of these look nearly interchangeable — fractions of a cent per thousand tokens apart. The real test was running them against the same messy, architecturally gnarly part of the codebase: the reader.
Note: prices above are OpenRouter’s standard listed rates. GLM-5.3 Flash is currently on a promotional OpenRouter rate of $0.075/M input and $0.25/M output (50% off the standard $0.15/$0.50) via select providers, which is part of why its cost-per-task numbers below look as good as they do.
The results were not close
| Model | Cost | Result |
|---|---|---|
| GLM-5.3 Flash | $0.80 | 220 reader tests repaired |
| Qwen3 Coder Next | $1.07 | ~13 tests |
| DeepSeek V3.2 | $1.12 | ~3 tests |
That’s roughly 0.36¢ per test fixed for GLM, versus 8.3¢ for Qwen and 37¢ for DeepSeek — and GLM’s number actually understates the gap, since the reader is one of the harder, more architecturally load-bearing parts of the project.
The full breakdown for the GLM run: 239 messages, 150K output tokens, 49M cached input tokens, for a total of $0.80. That’s an enormous amount of agent activity — investigation, planning, a substantial rewrite, and validation — for less than the price of a coffee.
Cheap isn’t the same as good: the Qwen regression story
The reader benchmark told part of the story, but a full test-suite run afterward told the rest — and it wasn’t flattering for Qwen. A full run confirmed real regressions:
numbers/number-comparison.lsp: 8 → 14 failures (+6)numbers/divide.lsp: 12 → 13 (+1)printer/format/format-e.lsp: 1 → 3 (+2)types-and-classes/types-and-class.lsp: 8 → 9 (+1)
Overall, the project went from 985 to 1,004 failing tests. Qwen wasn’t merely less productive than GLM — it made the codebase measurably worse. It did do real work in places (a legitimate 13 → 8 improvement on one file), so it’s not that the model is incapable. It’s that it’s action-oriented in a way that doesn’t reliably respect the constraints of the task. That’s a tolerable trait in a project with a forgiving CI loop; it’s a liability when you’re converging on a strict, semantically-defined compliance target.
GLM’s 220 fixes, by contrast, came with only a single regression — a genuinely different class of result. It wasn’t just “more fixes,” it was large, coherent, low-collateral-damage changes to a difficult subsystem.
The harness might matter more than the model
The most interesting part of this experiment wasn’t the model ranking — it was a side-by-side that emerged almost by accident, and it’s actually a cleaner comparison than it first appears: “Ox Alpha” is the same model as GLM-5.3 Flash, just under a different name. An earlier attempt at the same kind of ANSI-compliance work, using Ox Alpha inside a VS Code-based harness, had consumed roughly 350M tokens of input to fix about 100 tests, often getting stuck in what looked like research paralysis.
The GLM-5.3 Flash run, using a harness called OpenCode, used roughly 340K fresh input tokens (plus 49M cached) to fix 220 tests — for under a dollar. Same model, different harness, wildly different outcome.
Even comparing only fresh input tokens, that’s on the order of 700x less input consumed for roughly double the fixes — with the model held constant. That’s still not a fully controlled experiment (different repos states, different points in the project’s history), but holding the model fixed and swapping only the harness makes the case much harder to dismiss. The qualitative behavior changed too:
- Without the better harness: investigate → investigate → investigate → marginal progress
- With it: investigate → understand the architecture → make a major change → run the test gate → 220 tests fixed
That points to a conclusion worth sitting with: a coding agent isn’t just model + repository. It’s closer to model × context management × tool interface × execution loop × task state × feedback. The harness shapes the model’s action loop, not just its access to files. It may also explain part of why tightly integrated products (where the same team controls both the model and the harness) tend to feel more effective than the raw model benchmarks would predict.
Why cost-per-token is the wrong metric
One practical note that fell out of this: if your workload has a high cache-hit rate (mine runs around 98%, since the same repository context gets resent repeatedly), nominal input pricing becomes almost irrelevant. The economics are dominated by output tokens and cache-read cost instead. GLM-5.3 Flash’s cache-read price, for instance, is a small fraction of its output price — which is why a model that looks unremarkable on a straight input/output price sheet can still turn out to be the cheapest option in practice.
The metric that actually predicts value is dollars per completed task, not dollars per million tokens. And a slower, “worse” model that’s willing to iterate fifteen times can beat a stronger model that thinks for five minutes and makes one attempt.
Where this leaves the model ranking
Based on actual results rather than benchmark scores, for this specific workload:
- GLM-5.3 Flash — not just the highest raw fix count, but large, coherent, low-regression changes.
- DeepSeek V3.2 — the disciplined engineer. Good process adherence, real but modest progress, cheap.
- Qwen3 Coder Next — fine for tightly scoped, well-gated tasks; I wouldn’t give it broad, unsupervised write access to a repository again. It’s fast, but fast isn’t the same as forward progress.
The next experiment on deck: using GLM-5.3 Flash as a “lead” that plans and delegates, with cheaper (or free) GLM-5.2 workers handling bounded implementation tasks in parallel — trying to turn one model’s apparent knack for finding high-impact architectural fixes into a mostly-free, parallelized compliance machine.
This started as a cost comparison and ended up being a lesson about harnesses. If you’re evaluating “cheap” models for agentic coding work, don’t stop at the price sheet — run the same task through a full validation pass and count the regressions, not just the fixes.
