Langfuse vs LangSmith vs Opik (and others): how to debug AI agents
Agents rarely crash. More often they "finish" with the wrong answer: they call the same tool three times, keep going after a 404, or make up a value and report success. Logs of the final output won't show you that. You need a trace of every step (each model call, tool call, and sub-agent, with its inputs, outputs, tokens, cost, and latency) and evals that check the results.
What to look for
- Nested traces: a single run should show as a tree of spans, including any sub-agents it hands work to, so you can see which step went wrong.
- Sessions: multi-turn chats and long jobs grouped together, not as hundreds of separate traces.
- Silent-failure detection: evaluators, scores, or automatic analysis that flag a "successful" run that actually failed.
- Evals: datasets built from production traces that you run again after every prompt or model change.
- OpenTelemetry: so you aren't locked in, and so you can send traces to more than one tool.
Langfuse: open source, with every feature on the free plan
Langfuse is open source and free to self-host. It offers agent traces and graphs, session and user tracking, token and cost tracking, OpenTelemetry ingestion, datasets, LLM-as-judge evaluators, and human annotation. The cloud Hobby plan is free, with 50k units a month and 30 days of data access. Core is $29/mo (100k units, then $8 per 100k), and Pro is $199/mo. A unit is any trace, observation, or score you send.
LangSmith: tracing and evals from the LangChain team
LangSmith handles tracing and evals for any framework, not only LangChain. The Developer plan is free for 1 seat with 5k base traces a month. Plus is $39/seat/mo with 10k base traces, then pay as you go. Base traces are kept for 14 days, and extended traces are kept for 180 days. LangSmith also sells Engine, which monitors traces, groups failures into issues, and proposes fixes along with evals. It's billed per run in LSUs (1 LSU = $1), and LangChain estimates about 7 to 45 LSUs a run.
Opik: open source from Comet, with test suites
Opik is open source, and Comet says the self-hosted version has the full feature set. It includes agent tracing, execution graphs, sessions, cost tracking, error surfacing, OpenTelemetry support, pass/fail test suites with assertions, and more than 30 built-in eval metrics. Free Cloud includes 25k spans a month with 60-day retention. Pro Cloud is $19/mo for 100k spans, then $5 per 100k.
Others worth a look
- Laminar is open source (Apache 2.0) and built for agents. It records LLM calls, tool calls, sub-agents, costs, and tokens as a transcript you can read. Its Signals feature analyzes runs to find and group failure modes you didn't define in advance, and it can turn those groups into eval datasets. Pricing is usage-based, with a free tier.
- Arize Phoenix is built on OpenTelemetry and accepts OTLP traces directly. It covers evals, datasets, experiments, and a playground. It's free to self-host with no features held back, but the license is Elastic License 2.0, not an OSI-approved open-source license. Arize AX is the managed version.
- Helicone is an open-source observability tool and gateway that logs requests with cost. It's free for 10,000 requests a month, and Pro is $79/mo.
- Braintrust is built around evals and scoring. The Starter plan is $0 with $10 of credits, 1 GB of data, and 10k scores. Pro is $249/mo.
OpenTelemetry export: keep your options open
Langfuse, Opik, and Phoenix all accept OpenTelemetry traces. If you build on Claude, the Claude Platform can now send OpenTelemetry traces (public beta, announced Oct 14, 2026) for Messages API requests, Managed Agents turns, model calls, and server tools to any OTLP/HTTP endpoint. An org admin sets this up as a destination in the Claude Console, and no code changes are needed. You choose whether it sends only metadata or full content. The Claude SDKs can also emit client-side spans.
How to catch silent failures
- Trace every tool call along with its result, not just the model calls.
- Turn failed and odd-looking production traces into a dataset, and run it again on every change.
- Score live traffic with LLM-as-judge or code checks. Langfuse, Opik, LangSmith, and Braintrust all support this.
- Set alerts for error spikes and cost spikes. A run whose cost suddenly jumps is often stuck in a loop (see tracking and capping agent spend).
- Don't log secrets in traces. Keep keys out of the agent entirely (see giving an agent API access without your keys).
Which one to pick
- Open source and self-hosted: Langfuse, Opik, or Laminar.
- Already using LangChain or LangGraph: LangSmith.
- Eval-heavy teams: Braintrust or Opik's test suites.
- You want a gateway that logs as well: Helicone.
See every option in Observability for AI agents.