Hiro Fukushima
Code That Checks What AI Writes
A language model does not check whether its source supports a claim. It predicts the most probable answer from everything it has read, and a probable answer can still be wrong, even when the source is part of what it reads.
A common fix is a second model that checks the text. But models learn from human ratings, and people rate agreement higher, so a second model tends to confirm what it is given.
In medicine, law, finance and compliance, people make decisions based on what a text says, so content written by AI has to be checked against its sources before it is published.
I built the Content Engine for this. The facts with an exact answer, such as numbers, dates, quotes and citations, are checked by code, which gives the same answer every time. A claim rated wrong or invented stops publication in a line of code that no model can skip. The same engine can check texts from any of these fields.
For each claim it finds in a text, the Content Engine returns one of five verdicts, confirmed, partially correct, wrong, invented or unverifiable. The text can be a draft or a page that is already live.
A claim check ends in a report that lists each claim with its verdict, plus a finding for each claim rated partially correct, wrong or invented, which says what is wrong with it.
The engine also picks out the claims that a person should decide, and hands them over together with the whole page they come from.
For content that has to be proven sentence by sentence, the engine runs a full read. Every sentence of the body text, the title and the search title and description gets its own decision, and a page counts as done only when no sentence is left without one.
The engine runs on a computer or on a server, and it checks a single article or a whole site, several pages at a time.
For sensitive documents, such as medical or legal records, the engine can work with a local AI model, so nothing leaves an organization's own computers, and a private document can also be checked against a person's own files with code alone.
And it can export the facts it has confirmed, each with the exact passage behind it, for an AI assistant to answer from.
In the first version of the engine, the only thing that kept a wrong claim from going out was the model deciding to stop. A verifier built on a model also judged two kinds of German quotation marks backwards in three of five files, and a short piece of code fixed it for good. So I rebuilt the engine so that code runs the engine and the stop itself, and the model answers the questions the code hands to it.
One field's rules had ended up inside the engine itself, and a later branch that wrote that field back into the engine was not merged. Everything specific to a field was moved out of the engine's code, and a test fails any change that writes a field's name into it, so a new field needs no change to the engine.
The checks were first tuned by reacting to live runs, and a fix in one round broke its own test in the next. I stopped that and made accuracy a measured number for each field, the share of wrong claims the engine catches and the share of correct claims it wrongly flags, and a change ships only when neither number gets worse.
A model that stopped answering could fail without anyone noticing, and each project had to catch that with its own check before a run. The engine now refuses to start a run when the model it needs cannot answer, so no project depends on remembering that check.
Every claim check leaves a record. For each claim it keeps the verdict and how it was reached, and it keeps a copy of each source the engine fetched, as it read at the moment of the check. The record is written before publication is stopped, so even a blocked text leaves its full evidence behind.
Every engine I have built since works on the same principle, each for its own kind of work.
The Design Engine connects an AI to a product's design system. The AI can still pick the wrong token or the wrong component, so the engine checks the design and the code against the system's rules, and a person approves every change.
The Conversation Engine decides what an AI assistant may say and do. It decides whether the assistant may answer, what it has to find out first and when it has to ask or stop, and the model is one governed part that does not set those rules.
The Verification Engine checks robotic simulation studies before they move on. Fixed checks catch the failures that are already known, a model weighs the rest, and an uncertain case goes to an engineer.
On a site about housing law, the full read checked every one of the 1,403 sentences on its 54 pages against the official text and corrected 273 of them.
The engine confirmed 1,243 claims across the same 54 pages. An independent check then read 150 claims from the same pages, drawn at random, against a fresh copy of the official text without the engine, and all 150 were correct.
In an encyclopedia about AI, the engine read all 1,269 sentences of seven pages against their sources. It found wrong sentences on every one of the seven pages. It corrected 418 sentences and removed the 72 that no source supported.
The engine confirmed 1,389 facts for an AI assistant that answers compliance questions.
The Content Engine grew with each of my first projects, because each one wrote its own rules into it, from a house style to an exception for one kind of text. When client work came, I had to start again from scratch, since a client needed an engine with nothing of my other projects in it. So I made the engine a tool of its own that stays the same for every project, and each project now plugs into it through a connector that holds its rules.
A client receives a fresh copy of the engine. Instructions inside the copy let an AI agent build the connectors for that client's content, so the client owns a version of the engine that grows with their work. For anyone building a tool for several clients, giving each of them a connector that plugs into the tool means every client starts from the same clean tool.
