Why
I just gave Anthropic $6,000 in usage fees.
I’m on the $200/month Max subscription, but ever since the move to Opus 5.5 last week, I’ve been burning through credits at an alarming rate.
I hit my limit on Thursday, but I needed to finish a project, so I enabled pay-as-you-go usage. $6,000 later…
Anthropic’s biggest sin isn’t that it charges a lot. It’s the complete lack of transparency and predictability around those charges. I’m willing to pay for a great product. I’m not willing to write a blank check to use it.
So, I’m going to explore some alternatives.
The goal
I will need at least 50% of the performance of Claudio (our coding agent, running Claude Opus) on these four measures:
- Decode speed
- How fast the model writes its answer, in tokens per second, once it has started.
- Prefill speed
- How fast the model reads your prompt before it starts answering, also in tokens per second, and usually much higher than decode speed.
- Time to first token (TTFT)
- The wait between sending a request and seeing the first word of the answer, which is mostly prefill time.
- Context size
- The maximum amount of text, measured in tokens, that the model can hold in view at once, counting both your prompt and its reply.
The test bed
- claw-code. The UI and plumbing under test, with the underlying LLM swapped to see how each performs. Feedback says it is buggy; testing it anyway.
- Cursor. Added to the test bed after that feedback.
The models
- MiniMax M3 (minimax.io)
- GLM 5.3 Flash (docs.z.ai)
- DeepSeek V4 Flash (huggingface.co). Replaced Kimi, which wanted a lot of GPUs.
The infrastructure
| Option | What it is | Cost |
|---|---|---|
| Managed inference | Preconfigured cloud hosting for open-weight models. Most VPS providers now offer something along these lines. | Pay per use; the least expensive found so far is inference.net. |
| Rented GPUs | Dedicated GPU infrastructure; I handle the provisioning, configuration and optimisation myself. Needs at least 4× H200. | About $20/hour for 4× H200. |
| Local hardware | An Apple Mac Studio M5 Ultra with 512 GB of unified memory, expected at the end of this month.May be excluded: research shows it would be closer to 10% of the performance of Claudio (and I wouldn’t know where to borrow one anyway). | About $15,000. For comparison, buying 4× H200 would be about $200,000. |
The plan
I’ll benchmark the model / infrastructure combinations, focusing on coding performance, speed, reliability and, most importantly, total cost of ownership.
I’ll share the results here, including the configurations, benchmarks and actual costs.
Results
No results yet. The first measurements will appear here.
Feedback encouraged: comment on the LinkedIn post, or write to team@gyreresearch.com.