Product change·Oct 4 → Jan 1, 2026·Q4 2025·Cloud & DevOps
Datadog LLM Observability Unveils New Experiments Feature to Optimize Model Comparisons and Enhance Testing Capabilities
What changed
Feature added
New Experiments feature: Select best configurations with side-by-side experiment comparisons
Feature added
New Experiments feature: Generate versioned datasets from traces to standardize and reuse test cases
Feature added
New Experiments feature: Run experiments to compare models, prompts, and agent logic before release
Feature added
Enhanced evaluations: Added support for custom evaluations alongside out-of-the-box quality checks
Feature added
New Experiments feature: Debug traces in Playground by testing prompts, parameters, and models
Before and after
Oct 4, 2025 vs Jan 1, 2026Pulse.
100%
Change 1 of 5 · feature added
New Experiments feature: Select best configurations with side-by-side experiment comparisons
Sign in to view the full comparison
See the before/after screenshots and every marked-up change.
More from Datadog
Full history →- Week 40, 2026·improvedDatadog Llm Observability quadruples 30-day retention to $6.50 per 10K spans
- Q2 2026·packagingLLM Observability renamed Agent Observability, repriced per-span (Free+Pro)
- Week 25, 2026·packagingDatadog Llm Observability rebrands to Agent Observability
- Week 21, 2026·packagingDatadog Llm Observability adds tiered overage pricing for LLM spans
- Week 19, 2026·packagingDatadog Llm Observability launches Free tier and $160/mo Pro, ditches per-request
