How tokens work, and why every word costs money
AI providers such as Anthropic, OpenAI and Google do not charge per message, per user or per month. They charge per token. A token is a chunk of text the model processes, roughly four characters or three-quarters of a word in English, so a 1,000-word document is around 1,300 to 1,400 tokens. Every request you send is metered twice: once for input (everything you send the model, including the system prompt, any reference documents and the conversation so far) and once for output (the reply it generates).
Two details catch people out. First, output tokens usually cost around five times more than input tokens, so verbose replies are expensive; a sensible cap on response length is one of the cheapest optimisations available. Second, chat models are stateless: the full conversation history is resent with every turn, which means message twelve of a long conversation costs far more than message two, even if the user only typed one line.
- Rule of thumb: about four characters, or 0.75 words, per token in English
- Input and output are priced separately; output is typically around five times dearer
- Conversation history is resent every turn, so long chats get progressively pricier
- Setting a max output length caps the worst-case spend on any single reply
Model tiers: pay for the brain the task needs
Every major provider sells a tiered range, and the price gap between tiers is large. To take Anthropic's current line-up as an example, its small, fast Haiku-class model costs about $1 per million input tokens and $5 per million output tokens, while its flagship Opus tier costs $5 and $25 respectively (prices are quoted in US dollars across the industry). That is a fivefold difference for work the small model often handles perfectly well.
The discipline is called model tiering: route each task to the cheapest model that does it reliably, rather than sending everything to the flagship by default. A well-designed system might use a small model to classify an incoming query, answer straightforward FAQs with a mid-tier model, and only escalate genuinely complex reasoning to the top tier.
- Small tier: classification, routing, extraction, FAQ matching, formatting
- Mid tier: customer conversations, drafting, summarising documents
- Flagship tier: multi-step reasoning, complex analysis, agent workflows
- Review the routing quarterly; providers regularly release cheaper models that match last year's flagship
Need a hand with this?
Our team delivers AI Chatbots for UK businesses — with a free initial consultation, transparent fixed quotes and no lock-in contracts. Tell us what you're working on →
Prompt caching: the discount most businesses miss
A business chatbot resends the same material on every single call: the system prompt, the tone-of-voice rules, the FAQ content, the returns policy. Prompt caching lets the provider store that stable prefix and serve it back at a fraction of the price. On Anthropic's API, for instance, cached input tokens are billed at roughly a tenth of the normal input rate, in exchange for a modest one-off premium (around 25 per cent) when the cache is first written. OpenAI and Google offer comparable schemes.
Caching is a prefix match, so it only works if the repeated content is byte-for-byte identical and sits at the start of the request. A timestamp or user ID inserted at the top of the system prompt silently breaks the whole cache. Structure requests with stable content first and anything that changes per request at the end, then check the usage figures in the API response to confirm cached reads are actually happening.
- Put the system prompt, policies and FAQ content first and keep them identical between calls
- Keep anything volatile (dates, user names, the new question) at the end of the request
- Verify cache hits in the usage data; zero cached reads means something is invalidating the prefix
Batch processing and other easy wins
Not everything needs an instant answer. Most providers run a batch API that processes requests asynchronously, typically within a few hours, at half the standard price. Overnight report generation, summarising the day's support tickets, enriching a product catalogue or cleaning a mailing list are all ideal batch candidates. If the result is not needed while a human waits, it probably belongs in a batch.
Beyond batching, trim what you send. Shorten the system prompt to what actually changes behaviour, summarise or drop older conversation turns instead of resending them forever, and avoid pasting whole documents when a relevant extract will do. Input you never send is the cheapest input of all.
A worked example: what an SMB chatbot actually costs
Here is an illustrative monthly cost model for a small firm's customer-service chatbot. Assume 1,500 conversations a month, averaging six exchanges each, which is 9,000 model calls. Each call carries a 4,000-token cached block (system prompt plus FAQ content), around 500 tokens of fresh input, and produces a 250-token reply.
- Cached reads: 9,000 × 4,000 = 36M tokens at ~$0.30 per million ≈ $11
- Fresh input: 9,000 × 500 = 4.5M tokens at $3 per million ≈ $13.50
- Output: 9,000 × 250 = 2.25M tokens at $15 per million ≈ $34
- Total on a mid-tier model: roughly $58 a month, or about £45
Two comparisons make the levers obvious. Without caching, that 36M tokens of repeated context would bill at the full $3 rate, adding nearly $100 and roughly tripling the input cost. And if the workload suits a small-tier model at $1/$5, the same traffic comes to around $19 a month, about £15. The same chatbot can plausibly cost £15 or £150 a month depending entirely on how it is engineered.
Key Takeaway
You pay per token, and output tokens cost roughly five times more than input. Three levers control the bill: route each task to the cheapest model tier that handles it reliably, use prompt caching so repeated context bills at around a tenth of the normal rate, and push non-urgent work through batch APIs at half price. Engineered well, a typical small-business chatbot costs tens of pounds a month, not hundreds.
Habits that keep the bill down for good
Cost control is not a one-off exercise. Providers cut prices and release new tiers several times a year, and usage patterns drift as customers find new ways to use your bot. A short monthly review keeps spend proportionate to value.
- Set billing alerts and a hard monthly budget in the provider dashboard from day one
- Apply per-user and per-session rate limits so one abusive visitor cannot run up the bill
- Cap conversation length and summarise history beyond a set number of turns
- Log token usage per feature so you know which workflow is spending the money
- Re-check model choice and pricing quarterly; yesterday's flagship price often buys today's mid tier
If you would like help modelling costs for a specific use case or building a chatbot with tiering and caching designed in from the start, our team is happy to talk it through.