AI Agents / Agent evaluation and observability
AI Agents: Recovery success rate
Inject transient failures and score whether the intended outcome still completes.
Environment and inputs: Framework-independent agent evaluation; OpenTelemetry illustrates trace correlation.
When to use it
Choose an observable task outcome before optimizing a model or prompt. Keep inputs and scoring rules stable enough to compare candidates, and inspect critical failures individually rather than hiding them inside an average quality score.
Check the result
Compare success and duplicate side effects across retries and lost responses.
Save and compare results
Save the test input, configuration and result. Fix the failing boundary, repeat the same test and compare it with the saved working configuration.
Scope and limits
These exercises have not been tested on your systems. Check feature support and configuration names against the installed release and linked documentation.