In plain English
You pay for tokens in and out, plus search and tools. Speed depends on answer length, model size, number of steps and slow tools. Send less, cache, route easy work to small models, batch, run in parallel and stream. Test every saving, and plan for ten times the usage.
- Token
- A small piece of text that is priced and counted.
- Prompt caching
- Reusing a repeated start of a prompt.
- Model routing
- Sending easy tasks to a cheaper model.
Picture it
Levers you can pull
Your AI feature
Cost and speed
Send less
Shorter prompts and history
Cache
Reuse repeated work