Recently we've been working on migrating our enterprise reporting application to a modern open-source metrics store. Historically, large-scale legacy migrations have been some of the most challenging projects in software. The weight of hundreds of thousands of lines of production code full of complexity and technical debt makes it extremely difficult to even evaluate new architectures, let alone deliver a fully rearchitected solution.
But this changed in early-to-mid 2025. We found that state-of-the-art coding LLMs are now powerful enough to deeply understand our legacy codebase. Although AI coding agents are not yet a push-button solution for problems of this magnitude, we found simple strategies for orchestrating AI agents effectively, enabling us to "move mountains" of technical debt and accomplish in days and weeks what used to take months and years.
Speaker
David Stein
Principal AI Engineer @ServiceTitan, Previously ML Infra Tech Lead @LinkedIn
David Stein is a Principal AI Engineer at ServiceTitan, leading work on agent evaluations — predicting performance before release and monitoring success in production to bring engineering rigor to AI deployments. He also works on upgrading ServiceTitan's data platform for use by both humans and AI agents.
Before ServiceTitan, David was a tech lead on LinkedIn's machine learning platform team, where he led pioneering work on feature stores for collaborative ML development. Throughout his career, David has focused on making it easier for teams to build intelligent applications reliably at massive scale.
