AgentTrace

A coding agent succeeds on a task. You change its system prompt and run the same task again. This time it fails.

You now have two long execution traces full of searches, file reads, tool calls, arguments, results, retries, and errors. The final outputs tell you the runs were different. They do not tell you where the execution behavior separated.

AgentTrace gives you a place to start looking.

AgentTrace aligns the two trajectories, collapses the behavior that stayed the same, and surfaces the first observable divergence. It compares tool calls, arguments, results, errors, and stopping behavior.

Agent runs do not line up event for event. One run searches twice where another searches once. A tool call gets inserted, skipped, or reordered. Matching event 12 in one trace to event 12 in the other stops meaning anything almost immediately.

Ids and timestamps differ on every run too, so a line diff marks nearly every line as changed even when the behavior is identical.

I modeled each execution as a sequence and built deterministic weighted Needleman-Wunsch alignment to pair behaviorally similar events while leaving gaps where one run inserted or skipped an action. No model in the loop, no embeddings, no randomness.

Testing on real Claude Code sessions exposed a flaw the synthetic fixtures had not. A model turn that only emits tool calls has no visible output, and every such turn carried the same signature, so a turn that ran one shell command compared as identical to a turn that ran eight searches.

I changed the behavioral signature to incorporate the tools a turn emits, and added regression tests. On the long validation pair:

Correct pairings, the first observable difference, and existing synthetic fixture behavior were unchanged.

AgentTrace identifies observable behavioral differences between executions. It does not determine why an agent behaved differently or decide which run is correct.