Seeing What Happened

The agentic runtime writes every state change down as it happens. A run does not disappear when it finishes: the record stays, and you can open it again later.

This page is the tour of the windows onto that record. Most of it is read-only — you are looking at facts that were already committed. The last two sections (recording feedback, and re-running with a change) do write something, and they say so.

Nearly every command below talks to a running gateway. Start one first:

kx serve --dev-allow-local

Those commands default to http://127.0.0.1:50151; pass --endpoint <url> if yours listens elsewhere, and add --json for machine-readable output. The one exception is kx eval run, which runs entirely on your machine and needs no gateway at all.

Start from the list of runs

If you do not have an instance id in hand, list the runs the gateway has recorded. They come back newest first.

kx runs list
kx runs list --limit 20

To page further back, pass the lowest registered_seq from the page you already have:

kx runs list --limit 20 --before-seq 1042

The graph of one run

A run is a set of steps with edges between them. kx projection renders that graph: each step's state, its class, the pointer to its result, and the journal sequence number at which it committed.

kx projection --instance 00112233445566778899aabbccddeeff

The word projection is literal. The durable record is an append-only log of facts; the graph is computed by folding that log forward. That is what makes the next section possible.

Time travel

Because the graph is a fold over a log, you can stop the fold early and see the run as it looked at any earlier moment. Pass --at-seq with a journal sequence number:

# the run as it stands now
kx projection --instance 00112233445566778899aabbccddeeff

# the same run, folded only up to and including sequence 3
kx projection --instance 00112233445566778899aabbccddeeff --at-seq 3

Leaving --at-seq off means "fold to the current head". This is the honest kind of replay: you are not re-simulating anything, you are re-reading a prefix of what actually happened.

The live tail

kx events prints state changes as deltas. Two forms, and they are mutually exclusive:

kx events --instance 00112233445566778899aabbccddeeff --follow

kx events --all --follow

kx events --all --kind committed,failed --follow

kx events --all --kind committed,failed --json > feed.ndjson

Resume from where you stopped

--since takes a journal sequence number and streams the deltas after it, so a script can pick up without re-reading history:

kx events --all --since 512 --follow

Each JSON line carries the seq of the delta it describes. Record the last seq you processed and pass it back as --since next time. Without --json, the non-following form ends with a -- caught up at seq N -- line, which is the same cursor.

The reasoning trail

When a run reasons in a loop — think, act, observe, think again — each turn is committed as a durable fact, not just logged. Two read commands open that trail.

Think-and-act turns

kx react list --instance 00112233445566778899aabbccddeeff
kx react list --instance 00112233445566778899aabbccddeeff --limit 50

Each row shows the turn's mote id, which branch it settled on (pending, answer, tool, rejected, or dead_lettered), the tool it fired if it took the tool branch, and the run's durable budget caps. Rows come back newest first.

A single gateway journal can hold more than one reasoning chain per run. Scope to one with --chain:

kx react list --instance 00112233445566778899aabbccddeeff --chain <chain-hex64>

Re-ranking turns

When a retrieval step reorders its candidates with a model, that reordering is also a committed fact:

kx rerank list --instance 00112233445566778899aabbccddeeff

Each row shows the rerank step's mote id, the model that was resolved, the frozen outcome (pending, reranked, or failed_closed), how many candidates were in play, and — for a reranked outcome — the exact permutation the runtime enforced. It records the permutation, never a similarity score: the ordering that was actually applied is the fact, and the score is not.

Reading the actual output

The graph holds pointers, not bytes. When a step commits, its output is written once to a content store keyed by its own hash, and the journal records only that 32-byte address. That is why a run's graph stays small however large its outputs are.

Resolve a pointer back to bytes with kx content get:

kx content get --ref <result-ref-hex64> --instance 00112233445566778899aabbccddeeff
kx content get --ref <result-ref-hex64> --instance 00112233445566778899aabbccddeeff --out result.bin

The read is scoped. With --instance, you can read the refs that run actually committed; without it, you read the refs you uploaded yourself. Raw bytes go to stdout, --out writes a file, and --json prints the ref, the length, and the payload hex-encoded. A ref that is not in the scope you asked for is refused the same way whether it is missing or simply not yours, so this surface cannot be used to probe what exists.

Telemetry

Telemetry is the host's own measurement of how the steps ran: wall-clock time, which model was used, which tool fired. It lives in a rebuildable sidecar next to the rest of the run data. It is display-only — delete it and it comes back empty, and nothing about the run's truth or its digest changes.

# newest-first, across all runs
kx telemetry list

# one run
kx telemetry list --instance 00112233445566778899aabbccddeeff

# one step
kx telemetry list --mote <mote-hex64>

# page older rows: pass the lowest seq from the page you have
kx telemetry list --limit 200 --before-seq 4096

The rollup adds up the same numbers server-side, so a long run is totalled exactly rather than capped to whatever page you happen to have loaded:

kx telemetry summary
kx telemetry summary --instance 00112233445566778899aabbccddeeff
kx telemetry summary --json

You get, per model: a count, total output tokens, and total wall-clock milliseconds.

Metrics for a dashboard

If you want these signals in a monitoring stack, the serve can expose an opt-in Prometheus /metrics endpoint. It is off unless you ask for it:

kx serve --dev-allow-local --metrics-listen 127.0.0.1:9090
curl -s http://127.0.0.1:9090/metrics

The counters and gauges are folded from the durable journal — runs registered, steps proposed and committed, failures both in total and bucketed by reason, effects staged, the highest journal sequence folded, and build info — plus a recent-window block from the telemetry sidecar with wall-clock percentiles and summed output tokens. That latency block is omitted when no model step has run, rather than reporting a zero.

The endpoint is unauthenticated by design, the way scrape endpoints usually are. Bind it to loopback or a network you trust. This is a Prometheus text endpoint, not OpenTelemetry; OTLP export is on the roadmap.

The failure inbox

When a run gives up, that is a durable fact, and it lands somewhere you can find it. kx alerts list is a read-only inbox folded from the journal's terminal failures — dead-letters and worker-reported terminal failures. Terminal failures land in an inbox instead of vanishing.

kx alerts list
kx alerts list --instance 00112233445566778899aabbccddeeff --limit 50
kx alerts list --json

Rows come back newest first and carry the journal sequence of the failure fact, so you can jump straight to kx projection --at-seq and see the run at the moment it broke. Page older rows with --before-seq.

Retries that re-dispatch — a step that timed out or whose worker crashed and is being tried again — are deliberately left out. A row here means a run that is genuinely finished and finished badly.

Feedback on an answer

Reading a run often ends with a judgement: this answer was good, this one was not. You can record that so it is there next time. This one writes.

kx feedback submit --rating up --message-id <answer-id>
kx feedback submit --rating down --message-id <answer-id> \
  --instance 00112233445566778899aabbccddeeff \
  --comment "missed the second constraint"

--message-id is the stable key for one answer and is required; re-rating the same answer overwrites the old rating. Who submitted it is derived by the server, not claimed by the caller. Read it back with:

kx feedback list --instance 00112233445566778899aabbccddeeff

Like telemetry and alerts, feedback lives in a rebuildable sidecar and is advisory: it never becomes truth, identity, or part of the run's digest.

Turning quality into a number

Feedback is one person's opinion. Evaluation is a number you can gate on.

Score one run

kx eval score summarises a live run's trajectory: whether it reached a terminal, how many turns and tool calls it spent, how much of its budget it burned, and how many proposals were rejected.

kx eval score 00112233445566778899aabbccddeeff
kx eval score 00112233445566778899aabbccddeeff --json

from kortecx import KxClient

with KxClient("http://127.0.0.1:50151") as kx:
    q = kx.eval.score_run("00112233445566778899aabbccddeeff")
    print(q.terminal, q.reached_answer, q.turns_used, q.rejections)

import { KxClient } from "@kortecx/sdk";

const kx = new KxClient("http://127.0.0.1:50151");
const q = await kx.eval.scoreRun("00112233445566778899aabbccddeeff");
console.log(q.terminal, q.reachedAnswer, q.turnsUsed, q.rejections);

The regression gate

kx eval run scores a versioned golden suite against a committed baseline and exits non-zero on any regression. It runs entirely locally — no gateway, no model — so it cannot flake, which is what makes it usable in CI.

kx eval run
kx eval run --tolerance 20
kx eval run --json

A score is an integer per-mille, from 0 to 1000, and a pass/fail is an exact integer comparison. --tolerance is how far below the baseline a metric may drift before it counts as a regression. If the corpus itself changes, its digest changes and the gate fails closed until the baseline is deliberately re-captured — a change to what you are measuring is never silent.

Live-model numbers are advisory rather than a hard assertion, because local sampling is not bit-reproducible across machines. The golden gate is the ratchet.

Run it again with a change

Reading a run often ends with wanting to try it differently. kx runs rerun fetches the arguments a run was submitted with, overlays your edits, and invokes again — so this one starts a new run:

kx runs rerun 00112233445566778899aabbccddeeff --set topic=hello --wait
kx runs rerun 00112233445566778899aabbccddeeff --set count=3 --wait --out result.txt

A value that parses as JSON keeps its type (--set count=3 is the number 3); anything else is a string. Only the part of the graph your change actually affects recomputes — an unchanged re-run returns the existing result.

Where to go next