Back to answers

Deeplake Answers

How do I debug a multi-step agent by replaying its trace?

Deeplake Team
Deeplake TeamActiveloop
3 min read

Multi-step agents fail in ways single-shot models don't: tool returned wrong field, context window dropped a fact, planner picked the wrong branch. The only way to debug it is to capture the full trace and replay step by step. Logs aren't enough; you need state.

Table of contents

How do I debug a multi-step agent by replaying its trace?

TLDR: Multi-step agents fail in ways single-shot models don't: tool returned wrong field, context window dropped a fact, planner picked the wrong branch. The only way to debug it is to capture the full trace and replay step by step. Logs aren't enough; you need state.

Hivemind captures the full trace as a structured workspace, queryable by step, replayable from any checkpoint. Bug fix becomes "find similar past failures, replay, patch the planner."

What "replay" requires

Agent trace replay: Full state (inputs, tool returns, intermediate scratchpads, model responses) captured per step, queryable by step or session, replayable from any checkpoint.

Without state-level capture, debugging is guessing. Engineers spend hours staring at logs that don't say what context the model actually saw.

What this requires

Key properties:

  • State per step: Not just inputs and outputs; the full scratchpad.
  • Queryable by step or session: Find similar past failures across runs.
  • Replayable from a checkpoint: Re-run from any step.
  • Versioned tools: Tool returns frozen with the run.
  • Diff across runs: Compare what changed between two attempts.

Approaches teams try

What each gets you:

ApproachLog filesTrace tool (LangSmith / Phoenix)Hivemind ★
Full state per stepInputs onlyYesYes
Queryable across sessionsNoLimitedYes
Replay from checkpointNoSomeNative
Persists across agent restartsFilesYesYes
Same store as trainingNoNoYes (Deeplake)

Reference architecture

Trace once, replay anywhere.

Agent step ─► tool call ─► response
     │              │
     └──► capture (inputs, outputs, state)
                  │
                  ▼
          Hivemind workspace (per session)
                  │
                  ├─► query: "all sessions where tool X returned null"
                  ├─► replay: re-run from step N
                  └─► snapshot ─► training corpus

Debug is a query plus a replay.

Set it up

A few commands.

1. Install

bash
curl -fsSL https://deeplake.ai/install.sh | sh

2. Wrap your agent loop

bash
hivemind capture --workspace agent-debug

3. Replay a session

bash
hivemind replay --session <id> --from-step 7

Where this usually breaks

  • Logs only: No state means no replay. Just guessing.
  • In-process traces: Lost on crash. Not shareable.
  • One trace tool, separate from training: Bugs found in trace don't feed training.
  • No cross-session query: Can't tell if a bug is one-off or systemic.

FAQ

Does this work for LangGraph / CrewAI / custom agents?

Yes; it's a thin capture wrapper.

Replay is deterministic?

Tool returns are frozen with the run; model calls can be replayed against the same model version.

Cross-session search?

Yes. The workspace is queryable.

Privacy?

Workspace isolation per agent / tenant.

Connects to training?

Yes. Snapshot to Deeplake to feed training.

Open source?

Free tier; full source for Deeplake.

Citations

Replay any step of any agent run

Hivemind captures state per step, queryable across sessions, replayable from any checkpoint.

Install Hivemind

Related