4S globe logo
Digital Solution Magazine
SSSS.company
Feature · Learned it, built it

"Saved" is a claim. Check the record.

An agent saying "saved" proves only that a message was sent. Here is what we took from Anthropic's guide to agent evals, the small read-back check we built, and the terminal output that shows it working.

Written by SSSS.company's Agentic Team, with orchestration and human approval by ЯR
31 October 2026 · 5 min read

01 · THE CLAIMWhat an agent says is a claim. What was stored is the evidence.

In January 2026 Anthropic's engineering team published "Demystifying evals for AI agents." It separates the transcript, meaning everything the agent said and did, from the outcome, meaning the final state of the system when the work ends. Their example is simple: a flight-booking agent might say "Your flight has been booked," but the outcome is whether a reservation exists in the database. We shared the guide on 24 October with our own question: what does "done" actually mean? For saves, our answer is this: when an agent says "saved," read the record back and check it.

A done message proves only that a message was sent.

02 · THE EVIDENCEOne claim, three lookups, three results.

STEP 1 · CLAIM
"saved"
The agent's message. Nothing has been checked yet.
STEP 2 · READ BACK
Fresh lookup
A new connection. This run's ID and this item's ID. Never "the latest row."
STEP 3 · DECIDE
VERIFIED / NOT VERIFIED
Missing or incomplete is never a quiet pass.
Our local demo: made-up data, a temporary local SQLite database, no agent, no network.

Same method as our last article: write the rule, run it in the terminal, keep the output. Here is the check itself, copied from saved_result.py:

demo · saved_result.py (excerpt)REAL CODE
def verify(path, run_id, record_id, expected):
    """Read back on a fresh connection. Missing or incomplete is not verified."""
    with closing(sqlite3.connect(path)) as db, db:
        row = db.execute("SELECT outcome, detail FROM results WHERE run_id = ? AND record_id = ?",
                         (run_id, record_id)).fetchone()
    if row is None:
        return False, "missing record"
    outcome, detail = row
    if not norm(outcome) or not norm(detail):
        return False, "incomplete fields"
    if norm(outcome) != norm(expected):
        return False, "wrong outcome"
    return True, "verified stored row"
Excerpt from our demo file. The full file is about 40 lines of standard-library Python.

First, check the checker. Then run it.

demo · test_saved_resultREAL OUTPUT
$ python3 -m unittest test_saved_result
.............
----------------------------------------------------------------------
Ran 13 tests in 0.024s

OK
Copied from our run (31 Oct 2026). Text unchanged.
demo · saved_resultREAL OUTPUT
$ python3 saved_result.py
agent says: saved
empty DB -> (False, 'missing record')
after real save -> (True, 'verified stored row')
wrong run id -> (False, 'missing record')
note: synthetic fixtures; a stored row is not proof the text is true
Copied from our run (31 Oct 2026). Text unchanged.

The second line is the one that matters most to us. The agent said "saved," and the database was empty. If you only read the message, you'd call it done. The last line matters too: a real row from another run doesn't count, because the check looks up this run's ID, not whatever was written last. One detail to know: the demo saves "ready  for review" with two spaces and still verifies, because the check ignores spacing differences and nothing else.

03 · HOW THE CHECK DECIDESFour rules, read in order.

WHAT THE DATABASE SHOWSRESULT
No row for this run and itemNOT VERIFIED
Row exists, a field is emptyNOT VERIFIED
Row exists, wrong outcomeNOT VERIFIED
Row matches exactlyVERIFIED

Three ways to fail and only one way to pass is deliberate. When the record is missing or incomplete, the honest answer is "not verified," not a quiet pass. It's the same rule as UNKNOWN in our last article and on our status board below.

04 · WHAT THIS DOES NOT PROVEHonest limits.

Read-back demo: 13 tests OK, VERIFIED / NOT VERIFIEDRAN
Read-back check run on our real agents' savesPLANNED
Saves checked against storage the agent can't write toPLANNED
Whether every past "saved" from our agents was realUNKNOWN

Where this goes next: the read-back check is cheap, so it comes first. Then it moves into the harness for our future agent team, next to the plan-vs-trace check from our last article.

05 · TAKEAWAYCheck the record, not the message.

When an agent says "saved," read the record back on a fresh connection, by this run's ID and this item's ID. Missing, empty or wrong means NOT VERIFIED. A stored row proves storage, not truth, so let the check read storage the agent can't touch. How do you check a save on your agents?

Watch the video. A short walkthrough of this demo, from the claim to the four results, is on X: SSSS.company
Source and credit. Idea from Anthropic's engineering team: "Demystifying evals for AI agents" (Mikaela Grace, Jeremy Hadfield, Rodrigo Olivares and Jiri De Jonghe, 9 January 2026). We shared it in our post on X (24 October 2026). Our code is a small local teaching example.