About this webinar
Once you can measure an agent, the next question is how to make it better. Most teams stop at evaluation and fix problems by hand, one prompt tweak at a time.
In Part 4, we built an evaluation harness with pass@k, trajectory checks, and full tracing. In Part 5, Kwasi Ankomah puts those traces to work. He analyzes the agent’s failing runs, diagnoses the recurring failure modes, and then updates the agent’s instructions, memory, and tools automatically – using off-the-shelf libraries rather than a custom pipeline. Because the test suite gives a hard signal, the improved agent is A/B tested against the baseline so you can watch pass@k climb.
What you will learn
- How to read and analyze an agent’s failing traces to find what’s really going wrong
- How to group individual failures into recurring failure modes
- How to automatically update an agent’s instructions, memory, and tools based on those failures
- How to do all of this with off-the-shelf libraries instead of building a bespoke pipeline
- How to A/B test the improved agent against the baseline and measure the gain in pass@k – no fine-tuning required
Why improving from traces matters
Every failed run contains information about how the agent could do better. Teams that ignore their traces keep fixing the same problems by hand. Teams that learn from them build agents that steadily improve. Because this approach changes the agent’s instructions, memory, and tools rather than the model’s weights, it’s faster, cheaper, and easier to roll back than fine-tuning.
Who should attend
AI and ML engineers running agents in development or production, data scientists working on LLM evaluation and optimization, and technical leads looking for a practical, repeatable way to improve agent performance. Following Part 4 is helpful but not required.