December 7–11

The LLM bootcamp is back — five days of hands-on training to build, deploy and secure LLM-powered applications.

Learn more →
WebinarUpcoming

Build self-improving AI agents: Turn failing traces into higher pass@k

This is the heart of the series — and the step most teams skip. Kwasi Ankomah shows how to read an agent's failing runs, find the patterns behind them, and automatically improve the agent without any fine-tuning.

Date
Wednesday, December 2, 2026
Time
11:00 AM PT
Format
Virtual

About this webinar

Once you can measure an agent, the next question is how to make it better. Most teams stop at evaluation and fix problems by hand, one prompt tweak at a time.

In Part 4, we built an evaluation harness with pass@k, trajectory checks, and full tracing. In Part 5, Kwasi Ankomah puts those traces to work. He analyzes the agent’s failing runs, diagnoses the recurring failure modes, and then updates the agent’s instructions, memory, and tools automatically – using off-the-shelf libraries rather than a custom pipeline. Because the test suite gives a hard signal, the improved agent is A/B tested against the baseline so you can watch pass@k climb.

What you will learn

  • How to read and analyze an agent’s failing traces to find what’s really going wrong
  • How to group individual failures into recurring failure modes
  • How to automatically update an agent’s instructions, memory, and tools based on those failures
  • How to do all of this with off-the-shelf libraries instead of building a bespoke pipeline
  • How to A/B test the improved agent against the baseline and measure the gain in pass@k – no fine-tuning required

Why improving from traces matters

Every failed run contains information about how the agent could do better. Teams that ignore their traces keep fixing the same problems by hand. Teams that learn from them build agents that steadily improve. Because this approach changes the agent’s instructions, memory, and tools rather than the model’s weights, it’s faster, cheaper, and easier to roll back than fine-tuning.

Who should attend

AI and ML engineers running agents in development or production, data scientists working on LLM evaluation and optimization, and technical leads looking for a practical, repeatable way to improve agent performance. Following Part 4 is helpful but not required.

Explore more events and resources

Browse our library of tutorials, webinars, and live sessions to keep building your AI skills.