AI Development — Kai Kalinowski

Every team is being told the same thing: use more AI.
So we are — and the bill is climbing.
Tokens. Seats. Models.

The question nobody's answering:
is it actually paying off?

SIX MONTHS OF WEEKEND R&D

I spent six months of weekends finding out — and built the systems to test it on.

Not slideware. Four working systems, real code, real costs, independently audited. Everything in this talk comes from one of them.

An AI software org

15 disciplined roles + a review panel take a one-line brief to a reviewed, tested, shipped product.

A home-lending platform

The banking product that org built — as a real GitHub repo: ADRs, six-lens reviews, a regulatory change request, money correct to the cent.

A trading research platform

666K lines of Python, 283 audit probes guarding every data path.

An automated newsroom

Research-to-publish pipeline with fact-checking and quality gates.

Two of these I audited independently this quarter. Both had quietly learned to game their own quality scores — and that finding is the heart of this talk.

A MENTAL MODEL WORTH KEEPING

An LLM is a brilliant, eager, overconfident intern who…

1 has read everything ever written

2 remembers nothing from yesterday

3 will never say "I don't know"

4 does exactly what you ask — including the dumb thing you didn't realise you asked

Every win — and every failure — I'll show you comes from one of these four.

For 30 years the scarce skill was:
can you build it?

AI is collapsing that.
The new scarce skill:

can you judge it?

Judgment is what converts AI spend into value. Without it, you're just paying for output you can't trust.

WHERE AI GAMES ITS OWN METRICS

Here's why spend doesn't automatically become value. I audited two of those systems this quarter — a trading platform and an automated newsroom. Both had quietly learned to game their own quality scores.

The trading system ranked money-losers as its “most robust” picks — every average looked flawless; only slicing the data by category exposed it. The newsroom slipped fabrications past its own truth-gates. This isn't a defect in those systems — it's what AI does.

AI optimises the number you gave it, not the goal you meant. So you verify — knowing where it games is the new expertise.

[This is the emotional centre of the talk — slow right down, let the examples land one at a time.] So why doesn't spend just become value? Because the AI will optimise whatever number you hand it. I audited two of these systems independently this quarter. Both had quietly learned to game their own quality scores. [beat] The trading platform was ranking money-losers as its most robust picks — the summary numbers looked perfect; only when I sliced by category did the losers surface at the top. The newsroom learned to "verify" a shaky claim by searching for it and counting its own echo back as proof. [land this] That is not a bug peculiar to my systems. It is what AI does — it optimises the metric, not the goal. So the discipline isn't out-coding the AI; it's becoming the person who knows where it games, and catches it. That discipline is what turns the spend into something you can trust.

LIVE · A REAL SOFTWARE ORG, ON A CHEAP MODEL

One sentence in.
A reviewed product out.

A 15-role AI software org — product owner, BA, architect, data · backend · frontend engineers, QA, security, a compliance & risk officer, performance, DevOps, tech lead, scrum master and a technical writer, plus a six-lens review panel — takes a one-line brief through inception, three sprints, a mid-sprint regulatory change request and a performance pass, to a reviewed, tested product committed to a real GitHub repo. Every role real. Every issue, commit and pull request real. Nothing scripted — and it runs on a cheap model.

Run the demo

[~10 minutes. Switch to the live demo — play the recording and narrate the org: in inception the PO frames the product and writes the backlog, the architect designs the system and records the decisions as ADRs, the data engineer sets the shared types, the designer sets the visual direction, DevOps wires up CI and the board. Then per feature: the BA writes numerically-anchored acceptance criteria, the engineer implements and self-heals to green, QA and the security engineer write failing tests first, a six-lens panel reviews the PR, three skeptics throw out the hallucinated findings so only real bugs get fixed, the compliance & risk officer signs off, the tech lead merges. Mid-sprint a regulatory change request lands ("APRA lifts the serviceability buffer") and the team re-opens, revises and re-reviews — the standout 'real team' beat. A performance pass makes a hot path many times faster, same answers proven by the tests; the technical writer closes the docs. The point to land: every role is real, it's a real GitHub repo, and it ran on a cheap model. Finish by opening the live product and clicking through it.]

INSIDE THE TEAM — HOW IT CHECKS ITSELF

Not one prompt — a disciplined pipeline that reviews its own work.

The two questions everyone asks after the demo: how do the agents hand off, and how do they catch each other's mistakes?

Brief

›

Plan
PO · BA · Architect

›

Build
Engineer ⟲ self-heal

›

Review
6 lenses → 3 skeptics

›

Compliance
risk sign-off

›

Tech Lead
approve PR

›

Merged
green & reviewed

Self-heal to green

The engineer writes code, runs the tests, and if any fail it regenerates and retries — up to six rounds — until the suite passes. "Done" means proven done, not claimed.

The AI checks the AI

Six reviewers read the same code through different lenses — including a compliance & risk officer. Every issue they raise is voted on by three independent skeptics — only confirmed bugs get fixed, hallucinated ones are thrown out.

You can trust the output because the verification is built into the team, not bolted on. It's the maker–checker–sign-off discipline a bank already applies to any high-stakes change.

[This answers the question people always ask after the demo — point at the flow, then the two loops. ~90 seconds.] Two things people always ask: how do the agents hand off, and how do they catch each other? Here it is. Work flows left to right — a brief becomes a plan, the plan becomes code, the code gets reviewed, the tech lead merges. Two things make it trustworthy. One: the engineer self-heals — it runs the tests and regenerates until they pass, so "done" means proven, not claimed. Two: the AI checks the AI — six reviewers read the code through different lenses, one of them a compliance and risk officer, and every issue is voted on by three skeptics, so a hallucinated "bug" gets thrown out and only real ones get fixed. [land it] That is maker, checker, sign-off — the same control discipline you already use for any high-stakes change. The model writes; the discipline is what makes it safe.

DIAL ONE — RIGHT-SIZE THE MODEL

Same brief. Same kind of app. Eight models — A$0.52 to A$73.

One identical brief through one 11-role pipeline, each model to a green, tested app. A 143× price spread — and dearer did not mean better.

MiMo

A$0.52 · the cheapest of all eight — and it shipped a working, tested app.

Gemini

A$0.93 · the cleanest reviewer — caught the most real bugs of any model.

Fable 5

A$73 · the frontier — the cleanest first-try draft, at 143× the price.

There's no "an AI" — dozens of models, priced 143× apart, and price doesn't predict quality. Pick wrong and you burn budget for no gain. Default to a cheap one; pay up only where a wrong answer is genuinely expensive. This is the first cost dial every team controls.

Costs in AUD ≈ USD list price × 1.42 (MiMo $0.36 · Gemini $0.66 · Fable $51.41). Full data: the 8-model experiment page.

I ran that same build — same brief, same 11-role pipeline — on eight different models. The cheapest, MiMo, did it for about fifty cents and shipped a working, tested app. The most capable, Fable, seventy-three dollars. That's a 143× spread for the same kind of app. And here's the kicker: paying more did not buy better. A ninety-three-cent model, Gemini, was the cleanest reviewer of the eight — it caught the most real bugs in code review. The frontier model's real edge was a clean first draft — getting it right first time — not across-the-board quality. [slow down — this is the number that matters] The takeaway: there is no "an AI" — there are dozens of models priced wildly apart, and price does not predict quality. Default to a cheap one and pay up only where a wrong answer is genuinely expensive. [Context for honesty: seven of eight shipped green; one, Kimi, never converged in this loop; and one, DeepSeek, passed every test but shipped a dead app — which is the next slide.] [Costs are AUD, roughly 1.42× the USD list price, stated on the slide.]

DIAL TWO — RIGHT-SIZE THE HARNESS

Same model. Two harnesses. Opposite outcome.

The model was rarely the hard part. How you wrap it moved cost and quality more than the model choice did.

THIN — a hand-rolled API loop

DeepSeek · A$5.50 · self-heals by regenerating the whole module — a million output tokens. Passed every test… and shipped a dead app. The tests checked the engine; nobody tested the UI.

→

THICK — the model's native agent

The same DeepSeek · A$0.48 · edits files instead of rewriting them, caches the big prompt. A working app, 39/39 tests. 11× cheaper — and the difference between broken and working.

The same model that thrashed in my loop built this entire orchestrator inside its agent — the scaffolding outweighs the model. But it cuts both ways: two dials, not one — right-size the model and the channel.

[This is the most counterintuitive finding in the talk — slow down.] Once you accept there's no "an AI", the next question is how you wrap it — the harness. Same model, two harnesses, opposite outcome. DeepSeek in my own hand-rolled API loop: it self-heals by regenerating the entire module every round — a million output tokens — cost about five dollars fifty, passed every automated test, and still shipped a completely dead app, because the tests checked the engine and nobody tested the UI. The exact same DeepSeek in its native agent: it edits files instead of rewriting them and caches the big system prompt, so it cost forty-eight cents — eleven times cheaper — and produced a working app with thirty-nine passing tests. [land it] The model that thrashed hardest in my loop is the one that built this entire fifteen-role orchestrator inside its agent. The scaffolding outweighs the model. But it genuinely cuts both ways: force a heavy multi-role automated pipeline through an interactive session and you'll burn hundreds of dollars and never finish — wrong tool for that job. So it's two dials, not one — the model and the channel — and you only find the right setting by measuring.

GOVERNANCE — CONTROLS YOU ALREADY TRUST

This isn't a new risk model. It's your control model, applied to AI.

Everything that makes the AI team trustworthy maps to a control a bank already runs. The platform didn't invent governance — it inherited it.

Maker · checker · sign-off

The engineer makes; a six-lens panel and three skeptics check; the compliance & risk officer and the tech lead sign off. Nothing merges unreviewed.

Fail-closed gates

"Done" means the tests are green and the gate is open. A red gate blocks the merge. The green checkmark is a claim — the gate is the control.

Auditability by default

Every decision is a real artifact — ADRs, issues, pull requests, review comments, the regulatory change-request thread. The whole build is inspectable after the fact.

A named human owns the call

The AI proposes; a person disposes. Model risk stays with an accountable owner, and the lessons log is the team's control memory of where it games.

You don't have to trust the AI. You verify it the way you verify any high-stakes change — and that discipline is exactly what a bank is already built to do.

[For the risk and governance people in the room — this is the slide that lets them say yes. Calm, plain.] If you take away one thing as a control person, take this: none of this asks you to trust a model. Everything that makes the AI team safe is a control you already run. Maker, checker, sign-off — the engineer makes, a six-lens panel and three skeptics check, the compliance officer and tech lead sign off, and nothing merges unreviewed. Fail-closed gates — "done" means the gate is open; a red test blocks the merge, because the green checkmark is only a claim. Auditability — every decision is a real artifact you can inspect: architecture decision records, issues, pull requests, the change-request thread. And a named human owns the judgment — the AI proposes, a person disposes, and the lessons log is your control memory. [land it] This isn't a new risk framework you have to invent. It's the control model you already have, pointed at AI.

WHAT THIS MEANS FOR YOUR TEAM

Nobody's left out. The skills that travel are thinking skills, not coding skills.

Engineers

AI is your pair-programmer. You own the design and review every line.

Verification is now the whole org's job. You're already the expert.

BA / PO

The requirements sparring partner. "Where is this ambiguous or self-contradictory?"

Domain & Risk Experts

Regulation, controls, customer context — your judgment is exactly what the AI doesn't have.

A PATH YOUR TEAM CAN START MONDAY

Rigor, economics and governance — turned into three moves.

This week — pick one task

One repetitive, low-risk workflow your team already does. Default to a cheap model in its native agent. One person owns the verification.

30 days — measure it

Run it daily, keep a lessons log, track cost against the human baseline. You walk out with a defensible answer to "is it paying off?"

90 days — wrap it in controls

Put the winners behind maker–checker–sign-off and fail-closed gates. Scale to the next workflow. The lessons log becomes team governance.

Every template for all three moves — the model/channel routing, the lessons log, the control checklist — is in the AI Fluency Kit.

[Practical and forward-looking — this is what a leader writes down and acts on. Brisk.] So what do you actually do with this? Three moves, and they map straight onto rigor, economics and governance. This week: pick one repetitive, low-risk task your team already does, run it with a cheap model in its native agent, and put one person on verification. Thirty days: run it daily, keep a lessons log, and measure the cost against what the human version costs — now you have a defensible answer to "is it paying off", not a vibe. Ninety days: take what worked, wrap it in the controls we just covered — maker-checker-sign-off, fail-closed gates — and scale to the next workflow, and that lessons log quietly becomes your team's governance. [gesture] Every template for all three is in the kit. You can start Monday.

"The most valuable person on an AI team isn't the one who trusts the AI most — it's the one who knows exactly when not to."

Questions?

The AI Fluency Kit — individual fluency and team governance: kaikalinowski.com/demos/kit/

The full 8-model cost experiment — every app, cost & quality read: kaikalinowski.com/demos/compare/

One-page executive summary: kaikalinowski.com/demos/onepager.html

[Slow, deliberate close — make eye contact, then deliver the last line and stop.] I'll leave you with the whole talk in one line: the most valuable person on an AI team isn't the one who trusts the AI most — it's the one who knows exactly when not to. [beat] That discipline is what turns AI spend into value. I've put it into a kit — one tier for individual fluency, a second for team governance and cost control — and the full eight-model experiment is there too, every cost and quality read in the open. Thank you. Q&A PREP (the likely hard questions): • "Does the trading system actually make money?" — It's a research platform I'm still building; today's lesson is the auditing discipline, not the P&L. I don't trade real size on it yet. • "Isn't '15 roles' just one model prompted 15 ways?" — Yes, and that's the point: one model wearing disciplined hats, each checking the others. The value is the process and the verification, not 15 separate AIs. • "Won't this take our jobs?" — It takes the typing, not the judgment. Everyone here becomes the director and reviewer — judging the output is the scarce skill, and it's what you already do. • "Interactive agent vs the API — which is cheaper?" — It depends on the job, and the answer surprised me. Forcing my heavy 11-role automated pipeline through an interactive Claude Code session burned ~$284 and couldn't finish — wrong tool. But a single focused build inside an agent is cheap, often cheaper than my API loop, because it caches the big system prompt and edits files instead of regenerating them. So: API for heavy automation, interactive agents for focused building — measure, because it cuts both ways. • "If the AI games its own metrics, why trust any output?" — You don't trust it, you verify it: human-anchored tests, a second skeptical pass, fail-closed gates. The green checkmark isn't proof.