Time: an evening for the working version, a second evening for the parts that stop it lying to you. Cost: nothing, on a machine you already own. Difficulty: easy to build, genuinely hard to make honest — and the second half is the point.
At the end of this you will have a page that tells you, for every AI coding agent you run: whether it is working right now, how full its context window is, what model it is on, how long it has been since you last heard from it at all, and — the one I did not see coming — what it was in the middle of when it stopped.
If you run one agent you do not need this. If you run three you will start to want it. I run twenty-four — and the database has seen twenty-seven, which is the first thing it ever told me that I did not already know.
Every number in this guide is a COUNT(*) I ran against the live system on 12 September 2026, not a figure copied from an earlier draft. The draft before this one quoted August's counts as though they were current, and they were out by a third. That is exactly the habit the second half of this guide is about, and I am not going to pretend I am immune to it.
Before you start
What you need to have already: an always-on machine with a web server, PHP and MySQL or MariaDB. Mine is a Raspberry Pi 5 that also serves thirty-odd websites; the monitoring is not what troubles it. You also need agent tooling that fires hooks on session events — this guide is written against Claude Code, which emits a JSON payload on stdin for every hook, and the shape of that payload is the only Claude-specific thing here.
What you need to decide before you begin: whether the agents and the server are on the same trusted network. Mine are all inside my house, so the endpoint has no authentication and I am not going to pretend otherwise. If yours reach it across the internet, put a shared secret in a header and check it — the hook is four lines longer and this guide does not cover it.
What you should not do: start with a dashboard. I did, and the charts were the least useful thing I built. The value is in a list sorted by last-seen, and you can have that on the first evening.
The build
Eight steps. Steps 1 to 4 give you a working system. Steps 5 to 7 are what make it survive contact with more than two agents. Step 8 is the part you will actually use.
1. Two tables, and one constraint that matters
A session is one conversation. A heartbeat is one hook firing. That is the whole data model and I have not needed to change it since April — everything I built afterwards (alerts, health checks, roll-ups) sits on top of these two without touching them.
CREATE DATABASE fleet CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE TABLE sessions (
id INT(11) NOT NULL AUTO_INCREMENT,
agent_id VARCHAR(64) NOT NULL,
session_id VARCHAR(128) DEFAULT NULL,
started_at DATETIME NOT NULL DEFAULT current_timestamp(),
ended_at DATETIME DEFAULT NULL,
model VARCHAR(64) DEFAULT NULL,
context_pct_peak INT(11) DEFAULT 0,
total_heartbeats INT(11) DEFAULT 0,
git_branch VARCHAR(128) DEFAULT NULL,
status ENUM('active','idle','completed','abandoned') DEFAULT 'active',
PRIMARY KEY (id),
UNIQUE KEY uniq_session_id (session_id),
KEY idx_agent (agent_id),
KEY idx_started (started_at)
) ENGINE=InnoDB;
CREATE TABLE heartbeats (
id INT(11) NOT NULL AUTO_INCREMENT,
session_id INT(11) NOT NULL,
agent_id VARCHAR(64) NOT NULL,
event_type VARCHAR(32) NOT NULL,
context_pct INT(11) DEFAULT 0,
model VARCHAR(64) DEFAULT NULL,
git_branch VARCHAR(128) DEFAULT NULL,
received_at DATETIME NOT NULL DEFAULT current_timestamp(),
PRIMARY KEY (id),
KEY idx_agent_time (agent_id, received_at),
KEY idx_session (session_id),
CONSTRAINT fk_hb_session FOREIGN KEY (session_id)
REFERENCES sessions (id) ON DELETE CASCADE
) ENGINE=InnoDB;
The line to not skip is UNIQUE KEY uniq_session_id. The tooling gives every session a UUID that is fixed for its whole life, and that UUID is the session's only real identity. Making the column unique means that when your endpoint races itself — and it will, see step 5 — the database refuses the second row instead of quietly giving you two sessions with the same UUID. Mine did not have that constraint for the first three months and I paid for it with 297 duplicate session rows out of 1,320.
What you should see:
mysql -u root fleet -e "SHOW TABLES;"
Two tables. Nothing else yet.
2. The endpoint
One file. This is the whole thing — the version on my Pi is 476 lines, and every one of the extra 426 is a lesson from the last section of this guide rather than a feature.
<?php
// heartbeat.php — accepts one POST per hook firing.
header('Content-Type: application/json');
$db = new PDO('mysql:host=localhost;dbname=fleet;charset=utf8mb4',
'fleet_user', getenv('FLEET_DB_PASS'),
[PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION]);
$in = json_decode(file_get_contents('php://input'), true);
if (!$in || empty($in['agent']) || empty($in['session_id'])) {
http_response_code(400);
echo json_encode(['error' => 'need agent and session_id']);
exit;
}
$event = $in['event'] ?? 'heartbeat';
$lock = 'sess_' . md5($in['session_id']);
// Find-or-create, inside ONE lock keyed on the session UUID.
// The lock is not optional. See "Phantom twins" below.
$db->query("SELECT GET_LOCK(" . $db->quote($lock) . ", 5)");
$sel = $db->prepare("SELECT * FROM sessions WHERE session_id = ? LIMIT 1");
$sel->execute([$in['session_id']]);
$session = $sel->fetch(PDO::FETCH_ASSOC);
if (!$session) {
$db->prepare("INSERT INTO sessions (agent_id, session_id, model, git_branch)
VALUES (?, ?, ?, ?)")
->execute([$in['agent'], $in['session_id'],
$in['model'] ?? null, $in['git_branch'] ?? null]);
$sel->execute([$in['session_id']]);
$session = $sel->fetch(PDO::FETCH_ASSOC);
} elseif ($event !== 'session_end' && $session['status'] !== 'active') {
// A live beat against a closed session is PROOF the session is not closed,
// whatever the row says. Reopen it. See "The orphan fixer" below.
$db->prepare("UPDATE sessions SET status='active', ended_at=NULL WHERE id=?")
->execute([$session['id']]);
}
$db->query("SELECT RELEASE_LOCK(" . $db->quote($lock) . ")");
// The agent comes from the session ROW, never from the POST body.
// See "Identity" below — this single line is the whole defence.
$agent = $session['agent_id'];
if ($event === 'session_end') {
$db->prepare("UPDATE sessions SET status='completed', ended_at=NOW() WHERE id=?")
->execute([$session['id']]);
echo json_encode(['ok' => true, 'session_ended' => true]);
exit;
}
$ctx = (int)($in['context_pct'] ?? 0);
$db->prepare("INSERT INTO heartbeats
(session_id, agent_id, event_type, context_pct, model, git_branch)
VALUES (?, ?, ?, ?, ?, ?)")
->execute([$session['id'], $agent, $event, $ctx,
$in['model'] ?? null, $in['git_branch'] ?? null]);
$db->prepare("UPDATE sessions
SET total_heartbeats = total_heartbeats + 1,
context_pct_peak = GREATEST(context_pct_peak, ?)
WHERE id = ?")
->execute([$ctx, $session['id']]);
echo json_encode(['ok' => true, 'session' => (int)$session['id'], 'agent' => $agent]);
Note getenv('FLEET_DB_PASS'). The password belongs in the web server's environment, not in this file — the file will end up in a repository eventually and the password should not go with it.
3. Prove it with curl, before you write a single hook
This is the step people skip and it is the one that saves the evening. You have two moving parts; test them one at a time.
curl -s -X POST http://localhost/fleet/heartbeat.php \
-H 'Content-Type: application/json' \
-d '{"agent":"testbot","session_id":"aaaa-1111","event":"heartbeat",
"context_pct":42,"model":"claude-opus-5","git_branch":"main"}'
What you should see:
{"ok":true,"session":1,"agent":"testbot"}
Now send the identical command again. You should get "session":1 a second time, not 2 — the UUID matched an existing row. If you get 2, your UNIQUE KEY is missing or your lookup is keyed on the wrong column, and you have just reproduced the phantom-twin bug in a controlled setting, which is a far better place to meet it.
Then check the row landed:
mysql -u root fleet -e "SELECT agent_id, event_type, context_pct, received_at
FROM heartbeats ORDER BY id DESC LIMIT 5;"
And close the test session so it does not sit in your list forever:
curl -s -X POST http://localhost/fleet/heartbeat.php \
-H 'Content-Type: application/json' \
-d '{"agent":"testbot","session_id":"aaaa-1111","event":"session_end"}'
Leave the test rows where they are and label them as tests. Do not delete them to make the table tidy. I deleted my own passing test row from a different system for exactly that reason and then spent twelve days unable to prove the fix had ever worked. Tidying evidence is the most expensive kind of tidying there is.
4. The hook
One script, registered on every event you care about. This is close to what actually runs on my machine, with the fleet-specific parts generalised.
#!/bin/bash
# heartbeat.sh — one hook script, all events.
ENDPOINT="http://your-server/fleet/heartbeat.php"
# --- Read the payload with a BOUNDED read -------------------------------------
# `cat` here is an unbounded wait for EOF, which is a hang, not a read.
# `read -r -t 2` returns the instant the newline lands, and it is a builtin:
# no `timeout` process, no `cat` process. Measured 31ms against 60ms.
HOOK_INPUT=""
if [ ! -t 0 ]; then
IFS= read -r -t 2 HOOK_INPUT 2>/dev/null || true
fi
# --- Pull fields with bash's own regex, not sed -------------------------------
# The leading .* is load-bearing: bash regex is greedy, so the capture lands on
# the LAST occurrence in the line. Drop it and you silently take the first,
# which differs whenever a tool result echoes a usage block.
field() {
local re=".*\"$1\"[[:space:]]*:[[:space:]]*\"([^\"]*)\""
[[ $HOOK_INPUT =~ $re ]] && printf '%s' "${BASH_REMATCH[1]}"
}
EVENT_NAME=$(field hook_event_name)
SESSION_ID=$(field session_id)
# Windows paths arrive with escaped backslashes; bash file tests need slashes.
TRANSCRIPT=$(field transcript_path | tr '\\' '/')
CWD=$(field cwd | tr '\\' '/')
# --- Only SessionEnd ends a session ------------------------------------------
# Stop fires at the end of every assistant TURN. It means "this reply is
# finished", not "this agent has gone home". Mapping it to a session end is the
# single most expensive mistake in this guide — see below.
case "$EVENT_NAME" in
SessionEnd) EVENT="session_end" ;;
*) EVENT="heartbeat" ;;
esac
# --- Identity from the transcript path, NOT the working directory ------------
# The transcript lives at .../projects/<encoded-cwd>/<uuid>.jsonl, derived from
# the directory the session STARTED in and never rewritten. cwd is mutable.
AGENT=""
if [ -n "$TRANSCRIPT" ]; then
PROJECT_DIR=$(basename "$(dirname "$TRANSCRIPT")")
AGENT=$(printf '%s' "${PROJECT_DIR##*-}" | tr 'A-Z' 'a-z')
fi
[ -z "$AGENT" ] && AGENT=$(basename "$CWD" | tr 'A-Z' 'a-z')
[ -z "$AGENT" ] && exit 0
# --- Throttle tool-activity beats, BEFORE paying for anything expensive ------
# This block used to sit below the transcript read, so a throttled beat — the
# common case — paid a multi-megabyte file read to decide to do nothing at all.
case "$EVENT_NAME" in
PostToolUse|PreToolUse)
TS_FILE="${TMPDIR:-/tmp}/fleet_hb_${AGENT}.ts"
NOW=$(date +%s)
if [ -f "$TS_FILE" ]; then
LAST=$(cat "$TS_FILE" 2>/dev/null || echo 0)
case "$LAST" in ''|*[!0-9]*) LAST=0 ;; esac
[ $(( NOW - LAST )) -lt 120 ] && exit 0
fi
echo "$NOW" > "$TS_FILE" 2>/dev/null || true
;;
esac
GIT_BRANCH="none"
[ -d "$CWD" ] && GIT_BRANCH=$(cd "$CWD" 2>/dev/null \
&& git rev-parse --abbrev-ref HEAD 2>/dev/null || echo none)
# --- Context percentage, from REAL token counts ------------------------------
CONTEXT_PCT=0
MODEL=""
if [ -n "$TRANSCRIPT" ] && [ -f "$TRANSCRIPT" ]; then
# Tail only — these files reach many megabytes. A truncated leading line is
# harmless because we take the LAST complete match.
USAGE=$(tail -c 400000 "$TRANSCRIPT" 2>/dev/null \
| grep '"cache_read_input_tokens"' | tail -1)
if [ -n "$USAGE" ]; then
re='.*"input_tokens":([0-9]+)'; [[ $USAGE =~ $re ]] && IN=${BASH_REMATCH[1]}
re='.*"cache_read_input_tokens":([0-9]+)'; [[ $USAGE =~ $re ]] && CR=${BASH_REMATCH[1]}
re='.*"cache_creation_input_tokens":([0-9]+)'; [[ $USAGE =~ $re ]] && CC=${BASH_REMATCH[1]}
re='.*"model":"([^"]*)"'; [[ $USAGE =~ $re ]] && MODEL=${BASH_REMATCH[1]}
: "${IN:=0}" "${CR:=0}" "${CC:=0}"
TOKENS=$(( IN + CR + CC ))
# THE DEFAULT ARM IS THE DANGEROUS ONE. A model missing from this map is
# scored against 200k and every agent running it reads five times high.
# Add new models here the day they arrive.
case "$MODEL" in
*opus-5*|*sonnet-5*|*fable-5*) WINDOW=1000000 ;;
*haiku*) WINDOW=200000 ;;
*) WINDOW=200000 ;;
esac
# A prompt cannot be bigger than the window it fitted into. If the measured
# count exceeds the window we assumed, the ASSUMPTION is wrong, not the
# measurement. This only ever corrects UPWARD, and it cannot do otherwise.
[ "$TOKENS" -gt "$WINDOW" ] && WINDOW=1000000
[ "$TOKENS" -gt 0 ] && CONTEXT_PCT=$(( TOKENS * 100 / WINDOW ))
[ "$CONTEXT_PCT" -gt 100 ] && CONTEXT_PCT=100
fi
fi
# --- Fire and forget. Backgrounded, with a hard timeout ----------------------
# A monitoring hook that blocks your agent is worse than no monitoring.
curl -s -X POST "$ENDPOINT" \
-H "Content-Type: application/json" \
--max-time 5 \
-d "{\"agent\":\"$AGENT\",\"event\":\"$EVENT\",\"session_id\":\"$SESSION_ID\",
\"context_pct\":$CONTEXT_PCT,\"model\":\"$MODEL\",\"git_branch\":\"$GIT_BRANCH\"}" \
> /dev/null 2>&1 &
exit 0
5. Register it — and the one place you must not
In the agent's own .claude/settings.json:
{
"hooks": {
"UserPromptSubmit": [{ "matcher": "", "hooks": [
{ "type": "command", "command": "bash /path/to/heartbeat.sh", "timeout": 15 }]}],
"Stop": [{ "matcher": "", "hooks": [
{ "type": "command", "command": "bash /path/to/heartbeat.sh", "timeout": 15 }]}],
"SessionEnd": [{ "matcher": "", "hooks": [
{ "type": "command", "command": "bash /path/to/heartbeat.sh", "timeout": 15 }]}],
"PostToolUse": [{ "matcher": "", "hooks": [
{ "type": "command", "command": "bash /path/to/heartbeat.sh", "timeout": 15 }]}]
}
}
Do not put this in a parent directory's settings file. Claude Code merges .claude/settings.json from every ancestor directory of the working directory, so a hook declared at the root of a folder full of agents fires in addition to the one each agent already declares. Every beat is sent twice.
Mine did that for a week. It produced 7,329 duplicate rows — a quarter of the table at the time — double-counted every session's heartbeat total, and fired alerts in pairs that slipped past the throttles because both POSTs ran the check before either had inserted. The fix was to delete four lines from one file. Finding it took considerably longer, because the symptom was "the numbers are all a bit too big", which is the worst possible symptom.
The root file on my machine is now empty except for a comment explaining why it must stay that way, which is the most useful configuration file I own.
What you should see, after sending one prompt to one agent:
mysql -u root fleet -e "SELECT agent_id, event_type, context_pct, received_at
FROM heartbeats ORDER BY id DESC LIMIT 3;"
One row per prompt, not two. Check this on day one — a duplicate-beat problem is invisible until you go looking for it and it corrupts every count you will ever take.
6. The page that is actually worth having
Not a chart. This:
SELECT h.agent_id,
MAX(h.received_at) AS last_seen,
TIMESTAMPDIFF(HOUR, MAX(h.received_at), NOW()) AS hours_ago,
(SELECT s.context_pct_peak FROM sessions s
WHERE s.agent_id = h.agent_id
ORDER BY s.started_at DESC LIMIT 1) AS last_peak_ctx
FROM heartbeats h
GROUP BY h.agent_id
ORDER BY last_seen DESC;
Render that as a table with the oldest at the bottom and you are done. All the early value is in the last three rows — the agents you have not heard from in weeks. Charts tell you about the agents that are working, which are the ones you do not need to be told about.
Sort it so the oldest row is at the bottom and then read the bottom. The top of that list is reassuring and useless — those are the agents you have just been talking to. Mine currently has one that has not filed a heartbeat in sixty-two days, and it took a list like this to notice, because an agent you are not using generates no evidence of its own absence.
7. One cron line, and the trap inside it
Sessions do not always end tidily. A machine is rebooted, a terminal is closed, the agent tooling is killed — and the SessionEnd hook never fires, so the row sits active forever and your list shows an agent working that went home yesterday.
*/5 * * * * mysql -u root fleet -e "
UPDATE sessions s
SET s.status = 'abandoned', s.ended_at = NOW()
WHERE s.status = 'active'
AND COALESCE(
(SELECT MAX(h.received_at) FROM heartbeats h WHERE h.session_id = s.id),
s.started_at
) < NOW() - INTERVAL 30 MINUTE;"
This cron line and the "reopen on any beat" branch in step 2 are a matched pair, and you must have both. With the reopen branch, this is a harmless tidier. Without it, it is a machine for burying live agents — and that combination cost me a two-hour outage, described below. If you only take one thing from this guide, take the fact that those two pieces of code are the same decision written twice.
8. The one part a human actually reads
Everything so far is automatic, and automatic telemetry can only tell you that an agent ran. It cannot tell you what it was doing or where it got to. For that you need the agent to write it down, and that turns out to be the most valuable table in the system.
One row per session, and the enum in the middle is the important part:
CREATE TABLE project_state (
id INT(11) NOT NULL AUTO_INCREMENT,
agent_id VARCHAR(64) NOT NULL,
session_id INT(11) NOT NULL,
source ENUM('claude','auto','merged') NOT NULL DEFAULT 'auto'
COMMENT 'WHO AUTHORED THIS ROW. claude = agent wrote it, session
still open. merged = AGENT WROTE IT and the close hook then added
a metadata footer — it is an AGENT WRITE, not automated output.
auto = fallback, no agent ever wrote one.',
status_summary TEXT NOT NULL,
open_threads LONGTEXT DEFAULT NULL CHECK (json_valid(open_threads)),
blockers LONGTEXT DEFAULT NULL CHECK (json_valid(blockers)),
next_action TEXT DEFAULT NULL,
handover_notes TEXT DEFAULT NULL,
created_at TIMESTAMP NOT NULL DEFAULT current_timestamp(),
updated_at TIMESTAMP NOT NULL DEFAULT current_timestamp()
ON UPDATE current_timestamp(),
PRIMARY KEY (id),
UNIQUE KEY session_id (session_id),
KEY idx_agent_recent (agent_id, created_at DESC),
CONSTRAINT fk_ps_session FOREIGN KEY (session_id)
REFERENCES sessions (id) ON DELETE CASCADE
) ENGINE=InnoDB;
Four fields carry all the weight: what happened, what is still in flight, what is blocked, and the single next thing to do. The agent writes them through an endpoint that resolves its own live session, so it needs no credentials and no session id of its own.
Do not hand the agent the endpoint. Hand it a skill.
This is the part I would have got wrong if I had designed it on paper, so it goes before the rest.
I do not tell twenty-four agents "POST this JSON to that URL". I give them a single invokable command — /state — backed by one instruction file that holds the read protocol, the write protocol, the field limits, what a refusal means and what to do about it. The agent runs the command. It does not need to know the endpoint exists, and when the protocol changes I edit one file rather than twenty-four sets of instructions.
That file, not the database and not the API, is the thing the fleet actually obeys. Which makes it load-bearing in a way documentation usually is not, and it has bitten me twice — see the warning at the end of this step.
The useful consequence is that filing a handover becomes something an agent can be asked to do mid-session, not just at the end. On a long session I will ask an agent to drop a state row while it is still working, so that if the conversation has to be closed and restarted there is a written record of where it got to. That is the single most useful habit in this entire system and it is available only because the write is one short command rather than a curl invocation somebody has to get right.
It also explains the enum. A mid-session drop is exactly what source='claude' means — an agent wrote it, and the session has not closed yet. Those twenty rows in the count above are not a compliance failure. They are the handovers that happened to be in flight at the moment I ran the query.
Three further design choices are worth stealing.
UNIQUE KEY session_id — one row per session, enforced. The write is an upsert, which is what makes the mid-session drop safe: an agent can file at eleven, revise at two and revise again at five, and there is still exactly one row. No drafts, no duplicates, and the last write wins.
An over-length write is refused, not truncated. This used to silently shorten the text, return success and fire an alert afterwards — which tells you you have lost something at the precise moment the loss is unrecoverable. Worse, a truncated handover reads to the next session exactly like a complete one. Now it returns a 409 with the exact text that a truncating write would have destroyed, handed back while the author still has it in front of them. The first time that refusal fired in anger, one of the two items it saved turned out to be already false — a thread claiming a reply had not been sent, when it had. Being shown it is what got it corrected. A silent truncation would have kept the stale line and dropped a live one.
And there is a fallback worker that manufactures a row from the automatic telemetry when no agent wrote one. That is good engineering and I would build it again — but read the next section before you do, because it is also the most dangerous thing in the system.
The limits live in one constant, and even that was not enough
This last bit is about documentation as a component, and it is the thing I most want you to take from this step. It has bitten me three times in escalating order of subtlety.
First: the same numbers were declared in two files. The API had its own copy of the field limits and the session-closing hook had another, under a comment reading — and I am quoting my own code here — "if you change one, change the other; they drifted once and it cost 287 rows". They drifted again. They were always going to. A comment is not a mechanism. The limits now live in a single constant that both consumers import, so they are physically the same number and cannot disagree.
Second: the skill file went stale against the constant. I widened the cap in the code and did not update the instruction file. And because the instruction file is what the agents read, the fleet carried on writing to the old, narrower limit for five days — not because anything enforced it, but because the paperwork said so. Nobody noticed, because under-reporting fires no alert. A short handover looks exactly like a tidy one.
That is the lesson about ordering, and it took me a while to see it. Your documentation, your code and your agents' own carried instructions are three copies of the same rule, and they rank in the opposite order to the one you would hope for. The agents' instructions load once at the start of a session and outrank everything they read afterwards. The skill file outranks the code. The code — the thing that is actually true — comes last. So widening a limit is not one change. It is a commit, an edit to the instruction file in the same commit, and a message to every agent, because a commit cannot reach a copy that is already loaded into a running session.
Third, and this is the one that genuinely impressed me: when I raised the item cap from fifteen to twenty, I changed the constant, wrote a long note explaining the change, and updated the skill file — and left one sentence forty lines above the constant still saying "fifteen". In the very file that documentation sends people to when they doubt a number. Anyone who came to check would have been handed the old figure by the authority they came to consult, which is precisely the failure that sentence existed to prevent. Another agent caught it within the hour.
So: when you change a number, sweep the whole file for the old one. The constant is never the only place it lives, and the most dangerous copy is always the one that reads like an explanation.
MERGED and CLAUDE both mean an agent wrote it; AUTO — Scotty's, here — means nobody did, and the row is a summary the fallback worker made up out of heartbeats. The header counts the agents that have never written one at all, which is the figure I originally got wrong. Every one of those three surfaces was added after the mistake below; at the time, the interface said nothing and the column said nothing.What you should see, once a few sessions have been through:
mysql -u root fleet -e "SELECT source, COUNT(*) c FROM project_state
GROUP BY source ORDER BY c DESC;"
The six ways it lied to me
Everything above works. What follows is why the real version is ten times longer, and it is the part I would read if I were you. Every one of these six failures reported success. Not one of them showed up as an error, a crash, or a failing check. That is not a coincidence — it is what monitoring failures are like, because the thing that would have told you is the thing that broke.
1. Identity: the agent that was switched off and worked for sixteen minutes
Symptom: an agent that was powered off has a session in the database, with real heartbeats, real context percentages, and a plausible duration.
Cause: the obvious way to work out which agent is reporting is to look at the working directory. It is right there in the payload. It is also mutable — the moment anyone cds into another agent's folder, every hook from that point reports the other agent's name while carrying the original session's UUID. On 11 July I built one agent's project from inside another's session: 89 hook payloads carried the wrong directory, and an agent that was switched off was credited with a sixteen-minute session. Twelve such cases before it was fixed, and they clustered on exactly the two agents whose job is to work inside other agents' directories.
The worst part is not the missing data. It is that the data was confidently attributed to the wrong agent, and looked exactly like real data.
Fix, in two places. In the hook, derive identity from the transcript path — it is fixed for the session's whole life. At the endpoint, take the agent from the session row and ignore the one in the POST body. Both, not either: the hook fix is cheaper, but the endpoint fix is the one that holds when a config drifts, and twenty-seven config files will drift.
Attribute from the thing that is the session, not the thing that is nearby.
2. Phantom twins: the race a SELECT cannot win
Symptom: two session rows with the same UUID and the same start time. One collects the heartbeats; the other sits at 0% forever, gets marked stalled at ten minutes and abandoned at thirty, and fires alerts about a session that finished perfectly normally. 297 rows out of 1,320. "Abandoned" stopped meaning anything at all.
Cause: the hooks for session start, prompt submit and notification all fire within the same second on a cold start. Two requests both run SELECT … WHERE session_id = ?, both correctly find nothing, and both INSERT.
I had written a comment above that lookup claiming that matching on the UUID "prevents the duplicate-session race". It does not and it cannot. A SELECT cannot exclude a concurrent INSERT. Only a lock can. The comment was wrong for three months and its confidence is what stopped anyone looking there.
Fix: put the whole find-or-create inside one named lock keyed on the UUID, as in step 2, and add the unique index as a second line of defence. The same fault appeared twice more in the same file — once in the heartbeat de-duplication, once in the alert throttle — and it was the identical mistake each time: a check and a write that needed to be atomic and were not.
3. The number that was fiction
Symptom: an agent reads 100% context for over an hour while actually sitting at about 25%. "Context critical" alerts fire at agents with plenty of room and stay silent for ones that are nearly full.
Cause: the first implementation computed context like this.
context_pct = transcript_file_size / 40000
A file-size proxy with an invented divisor. The transcript is an append-only log of everything that ever happened — every tool result, every file read — and it never shrinks, while the actual context is compacted and cached. The two move together only by coincidence. One agent's browser-test runs dumped enormous tool outputs into her transcript, the file crossed 4MB, and 4,686,010 ÷ 40,000 clamps to 100. Every context alert that system ever fired on her was a fabrication, and the divisor had been calibrated for a smaller window besides.
It produced a number between 0 and 100. It moved in roughly the right direction. It was not a measurement of anything. That is the trap: a fabricated number of roughly the right shape is far more dangerous than an obviously broken one, because nothing about it invites suspicion. Thousands of alerts accumulated, each individually plausible.
And the alerts were the cheap half of the damage. This is the one number in the whole system that a human reads and then acts on within the minute — see the last section — so an invented divisor was not producing a misleading chart. It was telling me to throw away working sessions and keep exhausted ones.
Fix: the tooling writes exact token counts into each assistant message. The tokens actually in context are input plus cache-creation plus cache-read added together — that is the prompt it was billed for. Take the last such block in the file and divide by the real window for that specific model, as in step 4.
Two things about that fix are worth more than the fix itself.
The default arm of the model lookup is the dangerous one. A model that is not in the map gets scored against 200,000 tokens, so an agent on a million-token window reads five times high: it climbs to 100%, fires two levels of alert, then snaps back to twenty per cent the moment it crosses 200,000 and the self-correction widens the window. That sawtooth is the signature of an unmapped model, it ran unnoticed for a month after a new model launched, and it put seven fabricated alerts on the board in a single evening. Someone rotated a perfectly healthy session at "97%" because of it. Add new models to that map the day they arrive.
And the self-correction only works in one direction. A measured prompt cannot be bigger than the window it fitted into, so if the count exceeds the window you assumed, the assumption is wrong and you widen it. That is real evidence and it is what proved two separate models were on the larger window. But a small measurement is no evidence of a small window, so the correction cannot fire downward — which means everything below 200,000 tokens is the map's word alone. A backstop that only catches over-reporting is worth having and is not worth trusting.
I left the bad alerts in the database and marked them unreliable rather than deleting them. The count is embarrassing and the record is the only thing that makes the fix provable.
4. The orphan fixer: two hours of an agent reading "offline" while it worked
Symptom: an agent works for two hours. The dashboard shows it offline the entire time. Heartbeats are landing throughout, and you can see them in the table.
Cause: the hook mapped the Stop event to "session ended". Stop fires at the end of every assistant turn — it means "this reply is finished", not "this agent has gone home". So every agent went dark the instant it stopped speaking. That alone would be a flicker. What made it a two-hour outage was the second half: the session row stayed completed because a plain heartbeat did not reopen it, and the stale-session cron from step 7 then re-buried the agent every five minutes, because a completed session with a working agent is precisely the thing it exists to correct.
Two safety mechanisms, each sensible alone, cooperating to produce a lie. And a comment further up the hook file stated as fact that the API ended the session on stop — it was describing the bug, and it is what kept the bug invisible.
Fix: map only SessionEnd to a session end, and make any incoming beat reopen a closed session. There is no legitimate reading in which an agent both files a heartbeat and has gone home, so the beat is better evidence than the row. Do both.
5. The absence that looks like a bug, and the event I coded for that never fires
Two versions of the same problem, and the second one I found while writing this guide.
There are zero heartbeat rows with an event type of stop, despite that hook firing at the end of every turn on every agent. Anyone checking "is the stop hook landing?" by looking in this table concludes it is broken. It is not — the stop event is handled by closing the session out, and the evidence of it working is in a different table, as ended_at.
And there are six rows with event type session_start — out of 57,875. The hook has a whole case arm for it. I went and counted the configuration files: twenty-six agents declare hooks of their own, and not one of them points the session-start event at this script. They all point it at a different one that injects continuity notes instead. So that branch has been dead code since April and the six rows are archaeology.
Nothing is broken in either case. But when an event's effect lands somewhere other than where people will look for it, the correct working state is indistinguishable from a broken one — and when you write a code path for an event nothing sends, you will believe you are collecting something you are not. Either leave a row where the reader will look, or write down clearly that you deliberately did not. Then go and count what actually arrives, because the shape of your data is decided by your configuration files and not by your intentions.
Here is the whole of it, counted a few minutes ago:
MariaDB> SELECT event_type, COUNT(*) AS beats, MAX(received_at) AS most_recent
-> FROM heartbeats GROUP BY event_type ORDER BY beats DESC;
+---------------+-------+---------------------+
| event_type | beats | most_recent |
+---------------+-------+---------------------+
| heartbeat | 57499 | 2026-09-12 11:27:20 |
| notification | 487 | 2026-04-30 17:18:34 |
| session_start | 6 | 2026-08-11 13:30:22 |
+---------------+-------+---------------------+
$ grep -l heartbeat.sh */.claude/settings.json | wc -l # agents using the hook
26
$ # ...of those, how many register it on SessionStart:
0
$ # ...and on Notification:
26
Read the most_recent column rather than the counts. heartbeat is alive. session_start managed six rows in five months and last fired in August. notification has not appeared since April. And there is no stop row at all, ever.
So of the four event types I wrote code paths for, one is doing all the work and three are fiction — and all three lie in a different way, which is why this query is worth more than reading the script.
stop lands in another table by design. Nothing is wrong and the evidence is elsewhere.
session_start is outvoted by the configuration. The hook has a branch for it. Twenty-six config files point that event at a different script entirely, so the branch has never run in anger. The code is correct and irrelevant.
And notification is the interesting one, because I got it wrong while writing this paragraph. I had assumed the hook had stopped firing in April and that I had failed to notice for five months. It had not. All twenty-six agents still register it and it fires constantly — what stopped in April was the label. An earlier version of the script emitted notification as its own event type; the current one falls through to the default arm and files those beats as ordinary heartbeats. The data never stopped. It became indistinguishable.
That is the worst of the three and the reason to run this query rather than trust the schema: a frozen most_recent looks exactly like a dead hook, whether the hook died or merely stopped saying its own name. One of those needs fixing and the other does not, and the column cannot tell them apart. I had to go and read twenty-six config files to find out which I was looking at — and I only did that because I was about to publish the wrong explanation.
Counting what arrives is not the same as reading what you wrote. Counting it is also not the same as knowing why.
6. The compliance rate I got wrong by a factor of fifty
This is the last one and it is the one I would most like you to avoid, because it is the only failure here that I committed while deliberately auditing the system, with the query already open, being as sceptical as I know how to be.
Symptom: a mandatory step appears to be complied with about one per cent of the time. The figure is shocking, it is quotable, and I nearly published it.
Cause: both halves of the fraction were wrong, in opposite directions.
The denominator was too big. I divided by every session ever recorded. The handover table did not exist for the first month of the fleet's life, so I had charged hundreds of sessions with failing to comply with a table that had not been created yet.
The numerator was far worse. I counted rows marked source='claude', reasoning that those were the ones an agent had deliberately written. They are — but claude means "written by an agent, in a session that has not closed yet". When the session ends, the closing hook adds its metadata footer and flips the value to merged. So merged is not a fallback; it is the normal, finished shape of a handover an agent wrote by hand. I had counted only the ones still in flight and discarded every completed one.
Fourteen was not a compliance rate. It was a measure of how many sessions happened to be open at the moment I ran the query.
Fix: agent-authored means claude or merged. Measured today, across 1,813 rows: 1,014 merged, 20 claude, 779 auto. So 1,034 of 1,813 rows — 57 per cent — were written by an agent under a rule that nothing enforces. That is not a shameful number and I am not going to write it up as a confession.
Note that I am dividing by rows and not by sessions. I have the session count and it does not reconcile cleanly with the row count, and the whole subject of this section is what happens when you pick a denominator because it is available rather than because it is right. One row per session is enforced by the schema, so rows are the honest denominator for "who wrote these rows". If I want the other figure I will have to go and work out the discrepancy first.
Four things to take from it, in ascending order of usefulness.
The name of a value is a documentation surface. Had that column read agent+auto rather than merged, the mistake would have been unavailable to me. It was not renamed — it is an enum on a live table that several readers switch on by literal string, which is real risk taken on to prevent a misreading rather than a malfunction. What it got instead was the COMMENT you can see in step 8, a tooltip on the badge in the interface, and a note in the code that assigns the value. I had queried the table directly, which is the most sceptical thing I know how to do, and it told me nothing — because at the time none of those three surfaces said anything at all.
Graceful degradation was the whole problem. The fallback worker that manufactures a row when no agent wrote one is good engineering. It also means the non-compliant rows look, at a glance, exactly like the compliant ones. Smooth enough that nobody can tell which is which — including the person auditing it on purpose. If I built it again the fallback would still exist and it would mark its output loudly enough that the coverage figure was impossible to miss. Which is what the badge and the header count in that screenshot now do.
A citation is not a measurement. The comment on that column still carries the figure I measured in August, and it is no longer the current number — the one above is today's, because I ran it today. A number is sourced when you have run the count yourself, not when a docblock, a comment or a previous draft of your own guide cites it. Provenance is precisely what lets a wrong number survive: I once repeated a figure from a code comment that turned out to be 99.79% wrong, and it had lived for eleven days on nothing but the authority of being written down.
You are the last person able to check your own number. Four paragraphs above that figure, in the draft, I had written that a number you have not measured is a number you do not print — and then printed one I had measured, carefully, and misread. Diligence does not protect you here, because the thing I got wrong was something I had no way of knowing I did not know. What caught it was sending the paragraph to someone else before it went out. That check works precisely because it costs nothing and does not depend on my being right. Put it in the process, not in your own care.
The check that catches all six
Read back over that list and notice what it has in common. Not one of those failures raised an error. Every single one of them sat inside a system that was returning success, writing rows, and showing green.
So the checks I gave you after each build step are not enough, and I want to be blunt about why: every one of them proves that the thing you just did worked. None of them would catch any of the six a week later. A guide about a system that lies to you, offering eight confirmations and no audit, would be making the exact mistake it is warning about.
Here are six queries, one per failure. None of them takes more than a second. Run them when you finish building, and again whenever you have changed a hook, added an agent, or upgraded a model — those are the three things that break this.
And I ran all six against my own database before publishing them, which I recommend doing to your own certainties as well. Two of them returned rows where I had confidently written "you should see none", and in both cases there was a perfectly innocent explanation I had not thought of. Those two corrections are in the notes below rather than quietly edited out, because "expect zero" is exactly the kind of check that trains you to ignore a result — and a check you have learned to explain away is worse than no check at all.
1. Are any beats arriving twice?
SELECT session_id, event_type, received_at, COUNT(*) AS in_same_second FROM heartbeats GROUP BY session_id, event_type, received_at HAVING in_same_second > 1 ORDER BY received_at DESC LIMIT 10;
What you should see: nothing with a recent timestamp. Read the dates, not the row count — and this is the first correction. My own database returns ten rows for this, and I had written "you should see none". They are all dated 15 and 16 July, which is the inherited-hook bleed described above, and they are still there because I do not delete evidence of my own bugs. The rows are permanent; the fault is closed.
So the question this query answers is "is it still happening?", and the only part of the output that answers it is the newest received_at. If that date is today, you have the same script registered at two levels of the directory tree and every count you take is inflated until you fix it. If it is two months ago, you are looking at a scar.
Which is a small lesson in its own right: a check that says "expect zero" against a table you never prune will start failing permanently the first time you have a bug, and then you will learn to ignore it. Date the check, not the count.
2. Are there sessions that never reported?
SELECT s.id, s.agent_id, s.started_at, s.status FROM sessions s LEFT JOIN heartbeats h ON h.session_id = s.id WHERE h.id IS NULL AND s.started_at < NOW() - INTERVAL 1 HOUR ORDER BY s.started_at DESC LIMIT 10; SELECT session_id, COUNT(*) AS rows_with_this_uuid FROM sessions WHERE session_id IS NOT NULL GROUP BY session_id HAVING rows_with_this_uuid > 1;
The second query should return nothing, and mine does. It ought to be impossible given the unique index, and that is exactly why it is worth running: it is how you find out the index is still there. A constraint you have never tested is a constraint you are hoping for.
The first query is the second correction, and it is a better one. I wrote that a session over an hour old with no heartbeats is a phantom twin. Mine returned five, all recent, all marked completed — and not one of them is a twin.
They are sessions where the only hook that ever fired was the session-end one. Look back at the endpoint: the end branch closes the session and returns, without inserting a heartbeat row. So a conversation that opens and closes without a prompt being submitted creates a session row, marks it finished, and files no beats at all. Nothing is wrong. That is the design working.
Which is failure 5 again, wearing a different hat — the effect landed somewhere other than where I went looking for it, and I wrote the wrong expectation into my own audit of my own system. So: a zero-heartbeat session is a phantom twin only if another row shares its UUID, which is what the second query is for. On its own the first query tells you something happened, not that something is broken. Check both together or neither is worth much.
3. Is any model being scored against the wrong window?
SELECT model, COUNT(*) AS beats, MAX(context_pct) AS max_pct,
MAX(received_at) AS most_recent
FROM heartbeats
WHERE received_at > NOW() - INTERVAL 7 DAY
GROUP BY model ORDER BY beats DESC;
What you should see: every genuine model string in that list also appearing in the case statement in your hook. Mine returns exactly one real model over the last seven days, and it is in the map.
It also returns two rows that are not models and you should expect both: an empty string, which is beats filed before the transcript contained a usage block — early in a session there is nothing to measure yet, so the model is blank and the percentage is legitimately zero — and a placeholder string from the fallback worker, which files rows on nobody's behalf and has no model by definition. Neither is a fault. Read the rows with real model names and ignore the rest.
This is the one to run the day a new model ships, because the failure is silent and the number stays plausible. A model missing from the map has its max_pct pinned at or near 100 while the agent is nowhere near full — so you will be told to rotate healthy sessions, and you will do it, because the number looks exactly like a real one.
4. Is anything closing sessions that are still alive?
SELECT s.agent_id, s.id, s.status, s.ended_at,
MAX(h.received_at) AS last_beat
FROM sessions s
JOIN heartbeats h ON h.session_id = s.id
WHERE s.status <> 'active' AND s.ended_at IS NOT NULL
GROUP BY s.id
HAVING last_beat > s.ended_at
ORDER BY last_beat DESC LIMIT 10;
A heartbeat with a timestamp later than the moment its own session was marked finished means something closed that session while the agent was still working. Unlike the two above, there is no innocent explanation for this one — a beat cannot arrive from an agent that has gone home.
What you should see: nothing dated after you fixed it. And here is why I trust this query more than any other in the set: I wrote it expecting zero rows, and it returned the outage. Three agents, sessions closed between one and sixty seconds before their own last heartbeat, every one of them dated 25 and 26 August — which is the two days before the end-of-turn bug was fixed. Nothing since.
I did not go looking for those dates. The query found them, in data five months deep, having been handed nothing but the shape of the fault. That is the whole argument for auditing the contents rather than reviewing the code: the bug was invisible for two hours while it was happening, it was invisible to code review before that, and it is trivially visible in the table three weeks later to anyone who thinks to ask the right question of the data.
If you get rows dated today, something in your timeout arrangements is burying live agents, and — going by mine — nobody is going to tell you.
5. Is every event you registered actually arriving?
SELECT event_type, COUNT(*) AS beats, MAX(received_at) AS most_recent FROM heartbeats GROUP BY event_type ORDER BY beats DESC;
What you should see: one row for each event you registered a hook on, all with recent timestamps. Then compare it against your configuration files rather than your memory. An arm of your case statement with no rows behind it is dead code, and you will keep believing you collect something you do not — which is how I ended up with six session_start rows and a code path that has never once run in anger.
6. Is anybody actually writing handovers?
SELECT source, COUNT(*) AS rows_written,
ROUND(100 * COUNT(*) / SUM(COUNT(*)) OVER (), 1) AS pct
FROM project_state GROUP BY source ORDER BY rows_written DESC;
What you should see: a figure you can state out loud. Mine, run a few minutes ago: merged 55.9%, auto 42.9%, claude 1.2%. Agent-authored is claude plus merged — so 57.1%, and if you took only the last of those three you would report 1.2% and be wrong by a factor of fifty.
If the auto share is climbing, the habit is decaying and the fallback is hiding it from you — smoothly, plausibly, and without a single error.
One last thing, and it is a small one that made me smile. Earlier in this guide I quoted that split as 1,014 merged and 20 claude out of 1,813 rows. By the time I ran it again for this section it was 22 out of 1,815, because two agents filed handovers while I was writing the paragraph. The percentage did not move. Quote the ratio, date the count, and expect a live number to have changed by the time anyone reads your sentence about it.
Six queries. Put them in a file called check.sql and run the lot with one command, because a check you have to remember to assemble is a check you will stop running by March.
And note what they have in common: every one asks what the data actually contains, and not one asks whether the code is correct. All six failures above passed review. All six are refuted in under a second by counting what arrived.
What it costs
Measured on 12 September 2026, against a Raspberry Pi 5 that is also serving thirty-odd websites. Five months and seven days of continuous collection, starting 5 April.
- 57,875 heartbeats — every one retained, nothing pruned
- 6,272 sessions across 27 distinct agents
- 1,813 handover rows, one per session, of which 1,034 were written by an agent
- 75.86 MB of actual data across 52 tables, plus 41 MB the database has never handed back
- An average of 362 beats a day. Yesterday it was 921, from six agents across twelve sessions
- The quietest agent last reported 62 days ago, which is the sort of thing you only ever learn from a list like this
Seventy-six megabytes to know what twenty-seven agents have been doing since April. The monitoring is not what troubles the Pi; the websites are.
Observability is not expensive. Not knowing is expensive.
What I would build first
In this order, and I would resist every temptation to reorder it:
- The two tables and the endpoint. Test with
curlbefore any hook exists. - The hook — backgrounded, short timeout, bounded read, identity from the transcript.
- The list sorted by last-seen. Stop here for a week and just look at it. This is where all the early value is.
- The stale-session cron and the reopen-on-beat branch, together, never one without the other.
- The handover table — and a one-word command that writes to it. It is the only part of this a human reads, and the only part that tells you what an agent was actually doing. Everything before it is liveness; this is the work. Build the command at the same time as the table, not afterwards: the value is in being able to say "drop a state row" mid-session and have it cost nothing.
- The six checks in a file you can run with one command. Do this before you build a single alert — an alert is a claim about the data, and you have not yet established that the data means what you think.
- Only then alerts. And before you trust a single one, make yourself explain out loud how its number is computed. If the answer contains a divisor you chose, it is not a measurement.
You can have the first three in an evening and they will tell you more about your agents than you currently know.
Getting an assistant to do this for you
I did not type most of the above and there is no virtue in pretending otherwise. Three things are worth knowing if you would rather hand this to Claude.
Give it the payload, not a description of the payload. The one genuinely fiddly part is knowing what fields arrive on stdin for each hook event. Do not describe them — fire one hook with a script that does nothing but dump stdin to a file, then hand over the file. Ten seconds of work that removes the only part of this an assistant has to guess at.
Ask it what the running system contains, not whether the code is right. Every failure in this guide passed code review. The duplicate hook, the mutable identity, the fabricated percentage and the dead session_start arm are all perfectly reasonable code, and every one of them is refuted in about four seconds by SELECT event_type, COUNT(*) FROM heartbeats GROUP BY event_type. "Count what actually arrived and tell me if it matches what we meant to collect" is the single most productive instruction in this whole build.
Make it show you the state, not the exit code. A monitoring system that is silently collecting nothing looks exactly like one with nothing to report, and both of them return success. When an assistant tells you the hook is working, ask which row it landed in and what time it says — because "the POST returned 200" and "the data is in the table" are different claims, and only one of them is the thing you wanted.
What I actually use it for
I built this to watch agents working. That is not what I use it for, and the gap between the two is the most useful thing I can tell you before you spend an evening on it.
Most days it does two jobs, and neither of them is monitoring.
The context percentage tells me when to stop. On a long session I check it to work out whether I should close the conversation and open a fresh one before it runs out of room. That is a judgement I make by hand several times a week, and it is the only number in this system that changes what I do in the next thirty seconds.
Which is worth holding directly against the third failure above. For the first three months that number was a file size divided by a figure I made up. It was not feeding a chart — it was feeding a decision. That is how you come to throw away a session that was in perfect health because it read ninety-seven per cent, and I know the shape of that because it happened. If a number is going to change a human's behaviour, "roughly the right shape" is not a standard, it is a liability.
The handover tells me what I was doing yesterday. I open it to remind myself what I was working on with a particular agent the day before — which sounds like a convenience and is in fact the reason the table exists. An agent picking up its own notes is useful. A human picking up an agent's notes is the entire point, and it is why the write has to be refused rather than truncated, and why the fallback worker has to mark its own output.
And then the part I did not design for at all, in my own words at the time: I read it "in case there are threads in there that are for me."
That took me by surprise and it has changed how I read the thing. An agent's list of unfinished work is not a list of the agent's unfinished work. A good proportion of it is mine — a decision only I can make, a ruling nobody else can give, an account nobody else has the password to. The handover table turns out to be a queue pointing upward, and no part of the design anticipated that. It is now the first thing I look at.
The last-seen column, which I said above was where all the early value is, is the third of these and not the first. It is what tells you something is wrong. The other two are what you use when nothing is.
Worth it?
For an evening's work and seventy-six megabytes: unreservedly. Build the two tables and the hook, and then resist the dashboard for a week — just read the list. What it tells you in that week will not be what you expected to be told, and that is not a defect in the design. It is the whole return.