Opik
Observability platform for model-based systems, Apache 2.0 licensed, built by Comet. You instrument your calls and get full tracing of every request (prompt sent, answer, tokens, cost, latency), plus an evaluation layer that replays test sets against your traces. Self-hosting runs through Docker Compose or Kubernetes with no gated features; a managed cloud offering exists alongside with a free tier.
Strengths
- Apache 2.0 and self-hostable with no feature removed, unlike most competitors
- Tracing and evaluation in the same product: you replay tests directly against real traces
- Cost and latency visible per call, which makes the budget arguable with numbers
Limitations
- Self-hosting is a stack to maintain: it is free in license only, not in time
- Crowded and young category: the choice may need revisiting within a year
Best for
- Understanding after the fact why a production agent produced a given answer
- Tracking real cost per call before the bill becomes the topic