You don’t need the best model

When people work with AI, they always opt for “the best model” hen people work with AI, they always opt for “the best model” without having thought about it.
This is bad since it:

  1. Wastes a lot of resources (since those models are larger they require more energy).
  2. Wastes a lot of money (frontier models are usually 5x more expensive than their smaller siblings).
  3. Usually rate-limits your account a lot faster, which makes you less productive and forces you to upgrade, which brings me back to point 2.

Yes, the best models are “the best.” Which is wonderful, but using those models is not necessarily a requirement.
For less complex tasks in a team, do you let the best-in-class engineer (usually the most expensive one and usually a bit slower) work on those tasks, or do you let the most efficient person (time and cost wise) do it?


I think most people would opt for person two. Why don’t we think about this with AI models? Is it because someone else pays for the tokens (or the person who pays your subscription or the VC that invests in the AI companies)? Or is it because “it’s just easy to set and forget”?

Let’s take a look at the benchmarks from Artificial Analysis:

As you can see, GPT 5.2, Claude Opus 4.5, and Gemini 3 Pro are leading the benchmarks for coding. But when we look at the pricing, we see something different:

Claude Opus 4.5 is the most expensive, GPT 5.2 is already about 50% cheaper, and finally Gemini 3 Pro is even cheaper. So people are paying 50% more for a model that is just 2-3% better.

When we go back to the first chart, you can see Deepseek V3.2, GLM-4.7, and Kimi K2 Thinking are not far behind on intelligence (remember, they are almost as good as Sonnet 4.5, which most devs use at the company I work at).

But when you look at the second chart, you see something totally different. Kimi K2 Thinking is 90% cheaper than Opus, and the other models are even cheaper.

Even if you have to try it ten times with the cheaper models, which is rare because they are very good models if you use them properly, you are most likely paying less.

Then comes the question: How do I use them properly? First, you pick the correct provider, since open-weight models sometimes get deployed wrong, which causes tool-calling issues. This is something you just have to try and see what happens.

At the company I work for, I’ve done the testing and ensured the right providers are selected myself. What I personally did was just toggle one provider on in Kilo (one of the tools we use) and see if it has issues in a few turns, then move to the next, see what that does, etc.

We currently have Together, Novita, Nebius, Groq, Mistral, DeepInfra, AtlasCloud, and Parasail enabled for open-weight models, so just copy that list if you don’t want to tinker.

Then, with open-weight models I recommend always using an Architect/Plan mode (same for frontier models, but that can be expensive) so the model can do it’s research and make a plan before working. This ensures you, as the human in the loop, knows what the AI is up to and can steer it’s direction before taking off.

After that, closely monitor what it’s doing, just like you would do with frontier models, and that’s it!

But: when do you use frontier models? Personally, I use frontier models when I’m vibe coding UI, those models are really great for just generating new UI (and no, not based on a design, but just one shotting something new for a hobby project for example).

Or, when you need a proper architecture for something new. So when you are planning on doing a refactor, you let it write markdown file with guidelines. Then, when you start implementing you simply @mention that file with open-weight models to use that as a basis.

That’s it for now, if you have any questions drop them in the comments 🙂






Liking the microblog?

Sign up here so you are always up to date. I don’t spam and it’s completely free knowledge!


Posted

in

by

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.