What open source means in AI, and what it doesn't
When people say open-source AI, they usually mean open-weight models: files you can download and run on your own hardware, from families such as Meta's Llama, Mistral, Google's Gemma, Alibaba's Qwen and DeepSeek. The model weights are free to obtain, but the licences vary considerably. Some are genuinely permissive; others, like Meta's community licence, carry conditions on commercial use. Reading the licence before building a business on a model is not optional.
The word free also needs unpacking. There is no per-token bill, but the model has to run somewhere, and that somewhere is a GPU: rented by the hour from a cloud provider, or bought outright. Free refers to the software, not the electricity, the hardware or the engineer who keeps it all running.
The real cost comparison: tokens versus GPUs
Commercial APIs are pure pay-per-use. If nobody uses your chatbot on a Sunday, you pay nothing on Sunday. Self-hosting inverts that: you pay for GPU capacity whether it is busy or idle, plus the engineering time to deploy, monitor, patch and upgrade the stack. A capable cloud GPU instance can easily cost more per month than many small firms' entire API bill.
The deciding variable is utilisation. A GPU that sits idle 90 per cent of the day is extremely expensive per request; the same GPU running near capacity around the clock can undercut API pricing substantially. That is why self-hosting tends to make sense for sustained, high-volume workloads and almost never for spiky, low-volume ones.
- Low or unpredictable volume: the API almost always wins on total cost
- Sustained high volume on a narrow task: self-hosting starts to compete
- Always include engineering salaries and on-call time in the comparison, not just hardware
Need a hand with this?
Our team delivers AI & Machine Learning for UK businesses — with a free initial consultation, transparent fixed quotes and no lock-in contracts. Tell us what you're working on →
Control and compliance: where open weights shine
Cost is only half the argument. With a self-hosted model, your data never leaves infrastructure you control, which matters enormously for firms handling sensitive material: legal practices bound by client confidentiality, healthcare providers, or suppliers with contractual data-residency clauses. Demonstrating to a regulator or an enterprise client that prompts and outputs stay on UK soil is far simpler when the model runs in your own environment.
Control also means stability. API providers deprecate models on their own schedule, and behaviour can shift between versions in ways that break carefully tuned workflows. A self-hosted model is pinned forever: the weights you validated are the weights you run. You can also fine-tune open models on your own data, creating a specialist the big labs do not sell.
Three scenarios where self-hosting genuinely wins
1. High-volume, narrow tasks
If you are classifying thousands of documents a day, extracting fields from invoices or tagging support tickets, a small open model fine-tuned for exactly that job can match a flagship API model on accuracy at a fraction of the running cost. The task is repetitive, the volume keeps the GPU busy, and the economics flip in your favour.
2. Hard data-residency and confidentiality requirements
Where contracts, regulation or professional duty forbid sending data to a third party, self-hosting is not a cost decision but an enabling one. An on-premises or UK-region deployment lets legal, medical and defence-adjacent businesses use modern AI at all.
3. AI baked into a product you sell
If AI features sit inside software you sell, per-token API fees eat directly into unit margins and expose you to a supplier's pricing decisions. Owning the model stabilises your cost of goods, and for edge or offline deployments (devices in the field, air-gapped installations) it is the only option.
Where the commercial APIs still win
For frontier capability, the big labs remain ahead. The hardest reasoning tasks, sophisticated tool use and long agentic workflows are where the flagship commercial models justify their price, and open models trail on the most demanding work even as they close the gap on routine tasks.
The operational argument is just as strong. An API gives you security patching, abuse filtering, uptime engineering and steady model improvements with zero staff cost. For most small businesses, the honest total cost of ownership calculation, including a share of a DevOps engineer's salary, favours the API well past the point where the raw compute numbers suggest otherwise.
Key Takeaway
Open-weight models remove the per-token bill but add GPU and engineering costs, so utilisation decides the economics: self-hosting wins for sustained high-volume tasks, hard data-residency requirements, and AI embedded in products you sell. For everything else, commercial APIs remain cheaper once staff time is counted. Start on an API, measure real volumes, and move specific workloads to open models only when the numbers clearly justify it.
A pragmatic middle path
This is not a binary choice. Managed hosting services, including Hugging Face endpoints and the model catalogues inside the major clouds, will run an open model for you by the hour, giving you model choice and data control without owning the infrastructure. Hybrid routing is another strong pattern: an open model handles the bulk of routine traffic while a commercial API takes the difficult queries.
- Start on a commercial API; it is the fastest way to prove the use case
- Instrument token usage from day one so you know your real volumes
- Revisit self-hosting when a single sustained workload dominates the bill
- Pilot on a managed open-model host before buying or renting your own GPUs
If you are weighing this decision for a specific workload, our team can help you run the numbers and pilot both routes.