What your AI spend buys

Two dials decide whether AI pays off.

From the experiment: the same app cost A$0.52 to A$73 across models — a 143× spread, and price didn't predict quality. So the spend you control comes down to two dials: which model you route to, and whether you verify. Move them and watch the line item.

Dial 1 — Right-size the model

The cost of not routing by stakes: defaulting to a frontier model when a cheap one would do.

AI runs / month 10,000

Your average cost / run A$2.00

—

your spend / month

—

if routed to the 52¢ model

—

paid extra / year for no quality gain

—

Dial 2 — Verify with data

AI games its own metrics, so some outputs are wrong. What does catching them before they ship actually save?

Outputs that are wrong (and would ship unverified) 2%

Cost of one wrong output reaching production A$1,000

Verification cost / output (the second pass) A$0.20

—

exposure / month, unverified

—

verification cost / month

—

return on every $1 of verification

—

Assumes a verification pass catches ~90% of wrong outputs before they ship — the rest is residual risk. Cost-of-failure is yours to set: a dead UI is cheap; a miscalculated rate to 10,000 customers is not.

From the talk · the evidence: the 8-model experiment · how AI games its metrics · cost-controls playbook