QWE AI Academy Cheap tokens don’t shrink the bill – routing does
Most write-ups cheer the price crash and stop. Wrong lever. When unit cost falls, volume jumps and waste becomes the line item that hurts. Winners flood low-stakes work with the cheapest capable model, then route and cap the expensive paths.
Concrete routing steps below. Method A vs Method B first – then a one-hour setup.
Method A vs Method B: blind flood or tiered routing?
Two camps.
Method A – Blind flood: Send every request to the strongest model. Tokens feel nearly free, so maximize quality everywhere. Sounds kind. It still trains sloppy habits and prints surprise invoices when agents loop or contexts balloon.
Method B – Tiered routing + value metering: Default to the cheapest model that can finish the job (Luna-class for summaries, drafts, classification). Escalate only for hard reasoning or high-stakes output. Meter completed work units or outcomes, not raw tokens. Hard budgets. Aggressive cache use.
Demos love A. Monthly statements and team sanity prefer B. An a16z survey of technical AI buyers (Aug 2026) leaned the same way: 27 preferred credits tied to recognizable work; only 14 wanted pure tokens. Price the highest measurable value layer you can defend.
Treat Luna like electricity for lights and Sol/Astra like electricity for the industrial mill. You don’t run the mill 24/7 just because kWh got cheaper.
Quick numbers (so the router isn’t vibes)
Sam Altman still reaches for “intelligence too cheap to meter” – the old nuclear line – and still admits customers buy it on a meter (Fortune coverage of his remarks). On OpenAI’s pricing page, as of late 2026 (standard short context): GPT-6 Luna $0.10 input / $0.50 output per 1M tokens; Sol $2 / $10; Astra $10 / $50. Cached input is typically ~10% of input. Batch is another -50%.
Unit prices fell hard – roughly tenfold in a year in some analyses – while enterprise generative AI spend more than tripled. Same Jevons pattern showed up in usage data: after discounts, cheap-tier volume multiplied (OpenRouter-style reports: Luna-class ~13.8×, other discounted tiers ~5.6×), mostly net-new work, not pure substitution.
Walkthrough: Method B in under an hour
Solo dev or small team version. No platform rewrite on day one.
- Open the OpenAI usage dashboard. Set a hard monthly budget and an email threshold. Docs warn enforcement can lag – overage is still yours.
- Add a one-line router before every completion:
def pick_model(task_type, stakes="low"):
if task_type in ["summarize", "classify", "draft", "extract"] and stakes == "low":
return "gpt-6-luna" # $0.10/$0.50 as of late 2026
elif task_type in ["code", "agent_plan"] and stakes == "med":
return "gpt-6-sol" # $2/$10
else:
return "gpt-6-astra" # only when quality really pays
Log model choice, input/output tokens, and one outcome tag (ticket_resolved, pr_merged, brief_delivered).
Prompt caching on stable system prompts or RAG prefixes next. That ~$0.01 Luna cached input rate is where bulk work stops feeling scary – if the prefix actually hits.
Agent runs need a per-session token or dollar ceiling. Hit the wall → kill, tighten the prompt, retry. Planning/learning-style agent stacks are the volume multipliers; Gartner-range figures put them around 5-30× tokens versus simple chat for similar tasks (and ~8-10× cost versus basic workflows in some notes). Cap the loop once here – don’t restate it everywhere.
Weekly review = dollars (or tokens) per successful outcome, not a raw token vanity chart. $4 of Luna that saves two hours? Keep flooding. Astra on trivia? Demote that task type only.
Day-one version needs no platform glue.
Edge cases that still bite
Method B fails closed if you skip these.
Agent loops and multi-step internal reasoning (often billed like output) still turn “pennies” into four-figure months when nobody set a per-run cap. Community single-user spikes into six figures exist; the mechanism is volume, not the sticker price.
Cache math is friendly until it isn’t. Misses on long or changing contexts – or first-time prefixes – silently pay full input. Long-context tiers add uplift on top. Combine cheap Luna volume with messy prefixes and the bill behaves like Sol. Measure hit rate on real traffic, not a demo prompt.
Promotional or short-context rates can end; >272K-style tiers can jump. Budget caps may enforce late. Secondary soft alert near 70% of budget beats discovering the invoice after a weekend job.
Sometimes the cheapest token is the one you never send – a tiny local model or a rules path clears the easy cases before the API ever sees them. How much of your traffic is actually “needs a frontier model” versus “needs a classifier”?
FAQ
Are tokens actually too cheap to meter right now?
Per-unit on high-volume Luna-class work: yes. Per bill: no. Meter volume and outcomes anyway.
Should I just use the cheapest model for everything?
No. A coding agent that fails twice and eats human fix-up time costs more than one clean Sol or Astra pass. Route by difficulty and stakes – bulk email classify → Luna; multi-file refactor with tests → Sol; novel architecture call → Astra – and count retries in the total.
How do I know if my routing is working?
After two weeks, park two numbers side by side: cost per successful outcome, and % of tasks that stayed on the cheap tier. First number falling while the cheap-tier share holds ~70-80% without quality tickets means the router is doing its job. Quality slip on one task type? Raise escalation for that type only. Global-reverting to Method A is how teams re-learn the surprise invoice. Don’t chase tokenmaxxing cosmetics when the outcome metric already moved.
Open the usage dashboard, set the budget alert, paste the three-line router into the next script. Do that while the phrase is still loud.