Do you really need an expensive model for repetitive work?

Most of what an AI agent does all day is routine. Paying top-tier prices for every step is waste — here is how to right-size the model to the task, and how to judge the real cost.

The short version: AI agents call language models not only once, but several times for every job at hand — and most of those calls are routine. Routine work almost never requires the most powerful model on the market. The agent rather needs a model that does the same small job reliably, thousands of times a day. Build your agent so that small, inexpensive models handle the bulk and a top-tier model steps in only where the work is hard, and your running costs reliably fall to a fraction. Do not look at the price per call, rather look at the price per task.

Pricing mechanisms

When companies look at AI pricing, they usually look at a single question and a single answer. A few cents, sometimes less. That feels harmless.

An agent has a lot of work to do in the background, that makes a difference. Take an incoming customer email. Before anything reaches the customer, the agent reads the message, works out what it is about, pulls out the order number, looks up the customer, checks stock, drafts a reply, puts it into the right format and checks the result. That is easily ten model calls for one email. Add some hallucination checks and the workflow increases further. Multiply this by every email, every day, every working day of the year — and the harmless few cents become a real line in your budget.

So the question is not “what does the model cost?” It is “what does the model cost at the volume my process actually runs?”

What does the agent actually spend its day doing?

Look at the steps above again. Each one of them in isolation is not very demanding in any sense:

• Classify — is this an order, a complaint or a delivery question?
• Extract — find the order number, the date, the item.
• Route — send it to the right system or the right person.
• Format — turn the result into the structure your ERP expects.
• Call a tool — query a database, fetch a document.

These are manageable, repetitive jobs with straightforward answers. The key here is consistency: the same input handled the same way, every time. Frontier intelligence — the ability to reason through an open, unfamiliar problem — is only needed at a few steps in the agent’s workflow, if at all.

Paying top-tier prices for every one of those steps is like having your most senior engineer sort the incoming mail.

What is a “small model”, in plain terms?

A small language model is a model with far fewer internal parameters than the frontier models — typically a few billion instead of hundreds of billions. In practice that means three things:

1. It runs on modest hardware. A single server, sometimes a very good workstation — not a data centre.
2. It is fast and cheap to run. Less computation per answer means lower latency and a lower bill.
3. It does best on a narrow job. Trained or tuned for one task, it can match much larger models on that task. It will not match them on everything.

Think specialist, not generalist.

How much cheaper — and how good?

Researchers at NVIDIA put it bluntly in a position paper last year: serving a 7-billion-parameter model is 10 to 30 times cheaper — in latency, energy consumption and computation — than serving a model with 70 to 175 billion parameters (Belcák et al., 2025). The same paper looks at three open-source agent systems and estimates that roughly 40 to 70 percent of their model calls could be handled by much smaller models.

Does that still hold in late 2026? The models have changed, clearly; but the logic has not. A model that requires less computation per word operates way more economically. Many of today’s large models activate only part of themselves for each word, which narrows the gap somewhat — but anyhow, today’s price lists (below) show the obvious, it is still at least a factor of ten. And real-world use reflects it as well: in Hugging Face’s summer 2026 report, only 3 percent of this year’s downloads on the platform went to models above 70 billion parameters (Hugging Face, August 2026). Most AI that actually runs in production is small and highly specialized in its tasks.

Quality on narrow tasks holds up well. In a study revised in March 2026, a model with only 350(!) million parameters, trained specifically for tool calling, reached a 77.6 percent pass rate on the ToolBench benchmark — far ahead of the larger models the authors compared it with (Jhandi et al., rev. 2026). One caveat matters: those comparison models are older generations, and the authors say themselves that their model was optimised for this one benchmark. So read it as proof of a principle, not as a league table against today’s frontier. The principle is in the end what counts here though: a small model trained for one well-defined job can do that job very well — and by design, the nodes within an agent’s graph are very well defined and narrow.

The price gap is visible in public price lists, too. At the time of writing, one major provider lists its top-tier models at $10 per million input tokens and $50 per million output tokens, and its small tier at $1 and $5 — a factor of ten within the same product family (Anthropic pricing, September 2026). Open small models that you host yourself or rent from a European provider can be cheaper still.

Where does the right small model come from?

There are three routes, they build on each other and can be combined with ease.

1. Use the small tier of a large provider. The simplest start: no training, no own hardware, one contract. Good for proving that a process works at all.
2. Pick a ready-made specialist. Hugging Face, the largest public platform for open AI models, now hosts close to 3 million of them (Hugging Face, August 2026). Many are already trained for a single job — sorting documents, extracting fields, translating, finding similar texts. The catch: quality varies widely, and licences differ. Not every model may be used commercially, and not every model is well maintained. Choosing well is expert work.
3. Teach an open model your job. This is called fine-tuning — a form of transfer learning. You take a capable small model that already understands language and train it further on examples from your own process: your order emails, your product names, your answer formats. Modern techniques such as LoRA (Hu et al., 2021) only adjust a small part of the model, so this often runs on a single graphics card rather than a data centre. The training data can stay in-house — a real advantage under data-protection rules. (Sidenote: The writer of this article has had very good experience with this, for example in instance segmentation and detection in medical images; nevertheless, the labelling of images takes a while. Working with text should be by far simpler.)

Which route fits depends on volume and stability. Fine-tuning pays off when a task runs thousands of times a day and does not change every month. For a process that is still taking shape, start with route 1 and move down the list once the numbers justify it. What decides success on route 3 is less the technique than the examples: a model learns your job only as well as the examples show it.

A worked example

In the following, an example with a mid-sized distributor. The figures below are illustrative, not a client case — but the arithmetic is real and can be easily scaled to anyone’s business size.

• 2,000 incoming customer emails per working day
• about 10 model calls per email
• about 3,000 tokens in and 300 tokens out per call
• 250 working days per year

That is 20,000 model calls a day, or 60 million input tokens and 6 million output tokens.

• Every step on a top-tier model ($10 / $50): $900 per day, about $225,000 per year
• Every step on a small model ($1 / $5): $90 per day, about $22,500 per year
• Mixed — 9 of 10 steps small, 1 step top-tier: $171 per day, about $43,000 per year

The mixed setup costs less than a fifth of the all-top-tier setup. Caching and batch discounts shift the absolute numbers, but not the logic.

If you stopped reading here, you might conclude: “Just use the small model for everything.” But that would also be premature and even be a mistake.

The honest part: judge the cost per completed task

A cheap model that gets things wrong is not cheap. Every error has to be spotted and fixed by a person — and a person’s time costs far more than any model call.

So a more reasonable measure is the blended cost per completed task: this is the model costs plus the cost of fixing what went wrong. Let’s extend the example. Assume, again purely for illustration, that fixing a wrong result takes a clerk ten minutes at a fully loaded rate of $50 per hour — about $8.30 per error.

• All top-tier: $0.45 model cost, 1.5% errors → about $0.58 per completed email
• All small: $0.045 model cost, 8% errors → about $0.71 per completed email
• Mixed: $0.086 model cost, 2% errors → about $0.25 per completed email

Under these more realistic assumptions, the cheapest model is the most expensive way to get the job done — and the mixed design wins clearly. Your own error rates will differ. The point is the method: measure what a finished, correct task costs, not what a single call costs.

This is why well-built agents use what engineers call a heterogeneous architecture:

• Small models for the bulk — classifying, extracting, formatting, calling tools.
• A top-tier model in reserve — for the genuinely hard step: an ambiguous complaint, an unusual contract clause, a judgment call.
• A router or a check in between — deciding which model a request needs, or escalating when the small model is unsure.

The idea is well established in research. Stanford researchers showed as early as 2023 that chaining cheaper and more expensive models could match the best single model’s performance at up to 98 percent lower cost on their benchmarks (Chen, Zaharia & Zou, FrugalGPT). A 2024 study from Berkeley found that a learned router between a strong and a weaker model cut costs by more than half in some settings without lowering answer quality (Ong et al., RouteLLM).

There is a second benefit that matters in Germany: small open models can run on your own hardware or with a European host. That makes it easier to keep sensitive data where your data-protection obligations require it.

What does this mean for you?

If you are evaluating an AI agent — built in-house or bought in — ask these questions:

1. Which model handles which step? “We use the best model for everything” is not a quality statement. It is a cost statement.
2. What does one completed task cost — including the errors that need fixing?
3. Can you see those costs? Per process, per month, not just as one invoice from the model provider.
4. Can the model be swapped? Prices and models change every few months. A design that is locked to one model is locked to one price.

A good partner’s job is to right-size the model to the task, not to default to the biggest one. The biggest model is sometimes the right answer. It is rarely the right answer for every step.

How we build NeSt Agents

This is how we design NeSt Agents by default. We start from one concrete process and one measurable outcome. We map every step, assign the smallest model that does it reliably, and keep a top-tier model for the steps that genuinely need it. Where the volume justifies it, we select a specialist open model or train one on your own examples. Then we measure the cost per completed task — and adjust.

The result is an agent that fits into the tools you already use, keeps your data under control, and costs what the work is worth.

If you have a repetitive process in mind and want to know what an agent for it would really cost, book a free 30-minute call.

Related: What AI actually does for a business in 2026 · AI agents, explained for people who run a business

---

Sources

• Belcák, P. et al. (2025). Small Language Models are the Future of Agentic AI. NVIDIA Research. arXiv:2506.02153
• Jhandi, P., Kazi, O., Subramanian, S., Sendas, N. (2025, revised March 2026). Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning. arXiv:2512.15943
• Yakefu, A. et al. (August 2026). State of Open Models: Summer 2026. Hugging Face. huggingface.co
• Hu, E. J. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685
• Chen, L., Zaharia, M., Zou, J. (2023). FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv:2305.05176
• Ong, I. et al. (2024). RouteLLM: Learning to Route LLMs with Preference Data. arXiv:2406.18665
• Anthropic. Model pricing (accessed 16 September 2026). platform.claude.com