01 · THE CLAIMWhat an agent says is a claim. What was stored is the evidence.
In January 2026 Anthropic's engineering team published "Demystifying evals for AI agents." It separates the transcript, meaning everything the agent said and did, from the outcome, meaning the final state of the system when the work ends. Their example is simple: a flight-booking agent might say "Your flight has been booked," but the outcome is whether a reservation exists in the database. We shared the guide on 24 October with our own question: what does "done" actually mean? For saves, our answer is this: when an agent says "saved," read the record back and check it.
A done message proves only that a message was sent.
02 · THE EVIDENCEOne claim, three lookups, three results.
Same method as our last article: write the rule, run it in the terminal, keep the output. Here is the check itself, copied from saved_result.py:
def verify(path, run_id, record_id, expected):
"""Read back on a fresh connection. Missing or incomplete is not verified."""
with closing(sqlite3.connect(path)) as db, db:
row = db.execute("SELECT outcome, detail FROM results WHERE run_id = ? AND record_id = ?",
(run_id, record_id)).fetchone()
if row is None:
return False, "missing record"
outcome, detail = row
if not norm(outcome) or not norm(detail):
return False, "incomplete fields"
if norm(outcome) != norm(expected):
return False, "wrong outcome"
return True, "verified stored row"Excerpt from our demo file. The full file is about 40 lines of standard-library Python.First, check the checker. Then run it.
$ python3 -m unittest test_saved_result ............. ---------------------------------------------------------------------- Ran 13 tests in 0.024s OKCopied from our run (31 Oct 2026). Text unchanged.
$ python3 saved_result.py agent says: saved empty DB -> (False, 'missing record') after real save -> (True, 'verified stored row') wrong run id -> (False, 'missing record') note: synthetic fixtures; a stored row is not proof the text is trueCopied from our run (31 Oct 2026). Text unchanged.
The second line is the one that matters most to us. The agent said "saved," and the database was empty. If you only read the message, you'd call it done. The last line matters too: a real row from another run doesn't count, because the check looks up this run's ID, not whatever was written last. One detail to know: the demo saves "ready for review" with two spaces and still verifies, because the check ignores spacing differences and nothing else.
03 · HOW THE CHECK DECIDESFour rules, read in order.
| WHAT THE DATABASE SHOWS | RESULT |
|---|---|
| No row for this run and item | NOT VERIFIED |
| Row exists, a field is empty | NOT VERIFIED |
| Row exists, wrong outcome | NOT VERIFIED |
| Row matches exactly | VERIFIED |
Three ways to fail and only one way to pass is deliberate. When the record is missing or incomplete, the honest answer is "not verified," not a quiet pass. It's the same rule as UNKNOWN in our last article and on our status board below.
04 · WHAT THIS DOES NOT PROVEHonest limits.
- A stored row is not a true row. It shows the save happened, not that the saved text is correct.
- The agent must not write its own proof. The check has to read storage the agent can't fake or edit.
- It's a small local demo with made-up data. No live agent, no network. The 13 tests are our own.
| Read-back demo: 13 tests OK, VERIFIED / NOT VERIFIED | RAN |
| Read-back check run on our real agents' saves | PLANNED |
| Saves checked against storage the agent can't write to | PLANNED |
| Whether every past "saved" from our agents was real | UNKNOWN |
Where this goes next: the read-back check is cheap, so it comes first. Then it moves into the harness for our future agent team, next to the plan-vs-trace check from our last article.
05 · TAKEAWAYCheck the record, not the message.
When an agent says "saved," read the record back on a fresh connection, by this run's ID and this item's ID. Missing, empty or wrong means NOT VERIFIED. A stored row proves storage, not truth, so let the check read storage the agent can't touch. How do you check a save on your agents?