About this webinar
Most agents are hard to evaluate because “correct” is subjective. Coding agents are different – a test suite gives a clear, objective answer. That makes them the perfect place to build a rigorous evaluation practice.
In Part 3, the agent gained a real environment and learned to ship a PR that passes CI. In Part 4, Kwasi Ankomah measures how well it actually performs. He builds the evaluation harness end to end – pass@k over a task set, trajectory evaluation, and full tracing – then uses SambaEval to compare SambaNova models and pick the best one for the executor role. Evaluation is the bridge to improvement, and this session sets up everything the rest of the series builds on.
What you will learn
- Why tests make coding agents uniquely measurable, and how to use them as ground truth
- How to measure pass@k across a task set to get a reliable view of agent performance
- How to run trajectory evaluation to check whether the agent took a sensible path, not just whether it got there
- How to add full tracing so every run can be inspected and debugged
- How to use SambaEval to select the best SambaNova model for the executor role
Why coding agent evaluation matters
Agents are non-deterministic – the same task can succeed once and fail the next time. A single passing demo says very little. Measuring pass@k across many tasks shows how reliable the agent really is, while trajectory evaluation catches agents that reach the right answer for the wrong reasons. Without this foundation, there’s no way to tell whether a prompt change, model swap, or new tool actually made things better.
Who should attend
AI and ML engineers building or shipping coding agents, data scientists responsible for evaluating LLM systems, and technical leads choosing models and evaluation tooling for production agents. Familiarity with earlier parts of the series is helpful but not required.