Your production agents already generate rich training data: every interaction is captured in your OTEL traces. This talk shows how to distill that telemetry into datasets to fine-tune Small Language Models (SLMs) and turn the process into a repeatable platform capability. Learn how to extract, filter, and train on production data, then roll out distilled models safely with evaluation gates, canary releases, and continuous improvement cycles to build faster, cheaper, and self-improving AI systems grounded in your observability pipeline.
Speaker
Ben O'Mahony
Principal AI Engineer @Thoughtworks, Startup Advisor, Speaker, Future Author and Time Magazine’s Person of the Year 2006
Ben is a highly accomplished Principal AI Engineer and Innovator who specializes in building AI Agent Products and Platforms that work. With deep expertise across the entire AI lifecycle, he excels at designing, deploying, and optimizing scalable, cost-effective solutions for enterprise clients. A recognized industry expert, Ben is also a sought-after conference speaker and author of the O'Reilly book: Building AI Agent Platforms (now in pre-release).