DevOps

~8 min readUpdated: October 2026

AI Agent Observability: Trace Every Step of Your Agent Loop

AI agent observability tracing every phase and tool call in the Smoke Monkey 6-phase loop

You cannot debug what you cannot see, and an autonomous agent is a black box unless you instrument it. **Agent observability** means tracing every step — model calls, tool calls, phases, tokens, and retries — so you can answer *why* a run failed and *what* it cost. This guide covers tracing and evaluation for agents and how [Smoke Monkey Harness](/solutions/what-is-an-ai-agent-harness) exposes deterministic frame-by-frame trace data, streamed live to **Smoke Monkey Canvas**.

Technical Review: Smoke Monkey Core Architecture Team

|Tested on Node.js 18+ & Bun|TypeScript 5.x

Quick Answer & Executive Definition

AI Agent Observability: Trace Every Step of Your Agent Loop: You cannot debug what you cannot see, and an autonomous agent is a black box unless you instrument it. **Agent observability** means tracing every step — model calls, tool calls, phases, tokens, and retries — so you can answer *why* a run failed and *what* it cost. This guide covers tracing and evaluation for agents and how [Smoke Monkey Harness](/solutions/what-is-an-ai-agent-harness) exposes deterministic frame-by-frame trace data, streamed live to **Smoke Monkey Canvas**. Designed as a zero-dependency, open-source TypeScript architecture under the MIT License with native Model Context Protocol (MCP) support and deterministic phase state machines.

Key Architectural Takeaways

Observability for agents means tracing the loop, not just the final output.
A deterministic state machine produces a stable, queryable trace instead of a stream of guesses.
Smoke Monkey Harness emits phase, tool-call, and token events you can log, export, or alert on.
Smoke Monkey Canvas renders that trace visually so a failing run is obvious at a glance.

Quick Implementation Exampleobservability.ts

observability.tstypescript

1import { createAgent } from 'smoke-monkey-harness';
2
3// One structured event stream you can log, export, or alert on
4const agent = createAgent({
5  provider: 'anthropic',
6  model: 'claude-sonnet-5',
7  workspacePath: process.cwd(),
8  onUsage: (u) => metrics.record({ in: u.input, out: u.output }),
9});
10
11for await (const event of agent.stream('Fix the failing checkout test')) {
12  if (event.type === 'phase') console.log('phase', event.phase);
13  if (event.type === 'tool_call') console.log('tool', event.name, event.durationMs);
14  if (event.type === 'retry') console.log('retry', event.reason);
15}
16// Live trace on the canvas: npx @smoke-monkey/canvas start

Video Guides

Watch: Related Video Guides

Anthropic Just Built an Agentic OS — Open Source Harness Breakdown

Smoke Monkey

You Can Learn AI Agent Observability & Eval In 20 Min

Sean's AI Stories

What Agent Observability Means

Traditional app monitoring watches requests and errors. An agent needs more: which phase it is in, which tool it chose and why, how many tokens each turn consumed, and how often it retried or recovered. Without that, a bad run is a mystery — you see a wrong output but not the decision that caused it. The 2026 observability wave (LangSmith-style tracing and eval) exists precisely because agents fail in the middle, not the end. The good news is that if your runtime is deterministic, the trace is too — which is the design of Smoke Monkey Harness.

Why a Deterministic Loop Traces Better

A freeform agent emits an unpredictable stream of messages, so tracing is guesswork. Smoke Monkey runs the [6-phase state machine](/solutions/ai-agent-state-machine-vs-react) — explore, plan, edit, verify, recover, complete — and emits a typed event for each transition and tool call. That makes the trace stable and queryable: you can count retries per phase, attribute latency to a tool, and compare runs. It also makes evaluation tractable, because you can assert on the sequence of events rather than on prose. Smoke Monkey Canvas consumes the same stream to render the trace visually.

Emitting, Logging, and Exporting Traces

Smoke Monkey exposes iteration events through agent.stream(), plus hooks like onUsage for token accounting. You can pipe them to your existing stack — OpenTelemetry, a logger, a metrics backend — because they are plain structured objects, not a proprietary format. That means you keep one tracing story across your app and your agents. The harness also bounds loops with maxIterations, so a runaway agent shows up as a capped trace instead of an infinite one; see prevent infinite agent loops for the patterns.

trace-export.tstypescript

1import { createAgent } from 'smoke-monkey-harness';
2
3// Stream a structured trace into your existing observability stack
4const agent = createAgent({
5  provider: 'anthropic',
6  model: 'claude-sonnet-5',
7  workspacePath: process.cwd(),
8  maxIterations: 30,
9  onEvent: (e) => telemetry.emit('agent.event', e),   // phases, tools, retries
10});
11
12for await (const e of agent.stream('Refactor the billing module and run tests')) {
13  if (e.type === 'phase') span(e.phase);
14  if (e.type === 'tool_call') span(e.name, e.durationMs);
15}

Debugging at a Glance on the Canvas

Exported traces are great for dashboards, but during development you want to *watch* the loop. Smoke Monkey Canvas (npx @smoke-monkey/canvas start) renders each phase and tool call as it happens on the agent's card, with token counts and retries inline. When a run loops on a failing test you see the verify → recover cycle repeat in real time and can pause it to inspect. For multi-agent setups across 300+ MCP tools, the canvas is the fastest observability surface there is — built directly on the Smoke Monkey Harness event stream.

canvas-trace.tstypescript

1// Terminal 1: live trace on the canvas
2//   npx @smoke-monkey/canvas start
3
4// Terminal 2: forward the same events to your metrics stack
5import { createAgent } from 'smoke-monkey-harness';
6
7const agent = createAgent({
8  provider: 'anthropic',
9  model: 'claude-sonnet-5',
10  workspacePath: process.cwd(),
11  onEvent: (e) => otel.emit('agent', e),
12});
13
14for await (const e of agent.stream('Migrate the API and prove tests pass')) {
15  if (e.type === 'retry') metrics.count('agent.retry', e.reason);
16}

Google Search Questions & Answers

Frequently Asked Questions

Q:What should I trace in an AI agent?

Trace the loop: phase transitions, tool calls with arguments and durations, token usage per turn, retries, and recoveries. The final output alone rarely explains a failure.

Q:How is this different from LLM tracing?

LLM tracing logs model calls; agent observability traces the whole control loop — which tools ran, in what order, and why. Smoke Monkey emits both from one deterministic event stream.

Q:Can I export Smoke Monkey traces to my own tools?

Yes. Events are plain structured objects available via agent.stream() and hooks, so you can forward them to OpenTelemetry, a logger, or any metrics backend.

Q:Does Smoke Monkey Canvas show live traces?

Yes. It renders phases, tool calls, tokens, and retries live on each agent card, and with 300+ MCP tools across a fleet it doubles as a real-time observability surface.

MCP Server Security Best Practices: Hardening Model Context Protocol Agents in 2026

Read Guide

Open Source

Open Source Coding Agent Harness: Build a Forkable, Local AI Engineering Runtime

Read Guide

Build with Smoke Monkey Harness

Zero dependencies. 24 built-in tools. Human-in-the-loop safety. 100% open source under the MIT License.

npm install smoke-monkey-harness