LangSmith
A platform from the LangChain team to observe, evaluate and version the prompts of an LLM application: every step of a call or an agent is traced, outputs get scored against datasets or by a judge model, and prompts are tested in a playground. It does not require LangChain: the docs cite OpenAI, Anthropic, CrewAI, Vercel AI SDK or Pydantic AI, and it ingests OpenTelemetry traces from any application on its `/otel` endpoint. Free Developer plan (one seat, 5,000 traces a month), Plus plan at $39 per seat (10,000 traces), base traces kept for 14 days, self-hosting reserved to the Enterprise plan.
Strengths
- Traces of every step, evaluation and a prompt playground in the same product
- Independent from LangChain: OpenTelemetry traces accepted from any stack
- A free plan large enough to instrument a prototype (5,000 traces a month)
Limitations
- A hosted service: traces leave your machine, self-hosting only on Enterprise
- Base traces kept for 14 days, long retention costs extra
- Does not replace error tracking: it watches answer quality, not crashes
Best for
- Tracing a bad production answer back to the agent step that produced it
- Scoring a prompt’s outputs against a set of cases before and after a change
- Instrumenting an application that does not use LangChain, through OpenTelemetry