AE43E69367F40FE1DB5F244507269D43

The Small Models Are the More Interesting Story

Frontier releases get the coverage. Models small enough to run on a laptop or a phone are changing more about what can actually be deployed.

Diagram illustrating the AI Proof-of-Concept chasm showing a feature succeeding in demo state but failing during production deployment.
A successful AI demo is a long way from production reality. Bridging the PoC chasm requires move-to-production evaluation, security alignment, and data engineering.

Attention follows the largest models, which is reasonable — they define what is possible. But the models that have changed what is practical over the past couple of years are the small ones, and they get a fraction of the coverage.

What "small" now means

Models in the range that fits comfortably on a consumer graphics card, or in quantised form on a recent phone, now handle summarisation, classification, extraction, translation and structured output well enough for production use. That was not true not long ago. The gains came from better training data and better training technique rather than from scale, which is why they arrived quietly.

Why it matters more than benchmark position

A model you can run yourself changes four things at once. Data never leaves your infrastructure, which resolves most privacy and residency questions before they are asked. Cost becomes fixed rather than per-request. Latency stops depending on somebody else's network. And the model cannot be deprecated, reprice, or change behaviour underneath you between one week and the next.

For anyone building where connectivity is unreliable or where regulation restricts sending personal data abroad, that combination is worth more than a few points of capability.

The honest trade-off

Small models are worse at long multi-step reasoning, at unusual tasks they were not tuned for, and at anything requiring broad world knowledge. The productive pattern is not choosing one but routing: a small local model handles the large volume of routine work, and the difficult minority is escalated to something larger. That is an engineering decision, and it is where most of the practical value currently sits.

Get the next one by email

AI News

Perplexity trusts GPT-6 Astra with end-to-end systems

Perplexity has deployed OpenAI GPT 6 Astra across its operational pipeline, granting the model direct execution authority. Here is how this shift to high autonomy AI systems changes software engineering and infrastructure management.

2 min read

AI News

Runway's Solaris Generates Apps as Video, No Code

Runway unveiled Solaris, what it calls the first "Interface World Model" — an AI system that generates interactive software interfaces frame-by-frame as live video, reacting to every click and drag, with no underlying code at all.

3 min read