AI Works, Pull Requests Don’t: How AI Is Breaking the SDLC and What To Do About It

The presentation titled AI Works, Pull Requests Don’t: How AI Is Breaking the SDLC and What To Do About It by Michael Webster discusses the impact of AI on software development lifecycle (SDLC) and continuous integration processes at Circle CI.

Key Points Covered:

• AI Integration in Development: AI is being integrated into development workflows, from autocomplete and standard prompts to autonomous agents. This integration considerably impacts code output and organizational practices.

• Challenges with Pull Requests: Traditional pull requests (PRs) are becoming inefficient due to the increase in code volume produced by AI. Large PR sizes pose a review challenge, with AI often generating more code than is feasible to review manually.

• Strain on SDLC Process: AI's ability to produce large quantities of code rapidly clashes with existing linear software delivery models, causing bottlenecks and straining current best practices such as pull request reviews and compliance checks.

• Future Directions: The industry needs to move towards simplifying AI's complexity through better abstractions. Emphasis is needed on validating AI's outputs effectively rather than focusing solely on task completion and tool definitions.

Solutions and Suggestions:

• Develop strategies for managing AI-induced code complexity through effective abstractions and validation techniques.

• Embrace methodologies like trunk-based development and test-driven development to optimize and streamline the integration of AI in software pipelines.

• Focus more on testing and validation layers to ensure the reliability of AI-generated code before deployment.

Overall, the talk calls for a rethinking of traditional software engineering practices in the wake of AI advancements, proposing that the industry adopt new methods that accommodate the unprecedented scale and pace of AI development tools.

This is the end of the AI-generated content.

This talk will use CircleCI’s adoption of AI agents as a lens for both individual and organizational changes that AI brings. We will first discuss the realities of using AI as an individual engineer, where things work well today, and where tools are lacking. Next, we’ll take a systems thinking approach to explain how the increase in code output creates challenges for today’s “best practices”.

Finally, we’ll take a meta view and explore how changes to organizational practices are changing the discipline of software engineering itself. AI is just the latest iteration of a new capability obviating old approaches. We’ll look at how the tried and true principles of abstraction, encapsulation, and simplicity can be applied to navigate this latest cycle.

In this talk we will cover:
 

• How AI is changing the individual developer’s workflow:
  • The changes in how CircleCI uses AI, from autocomplete, to standard prompts, to fully autonomous agents: where AI works well and where it doesn’t
  • The realities of using non-deterministic tools as an engineer
• How individual changes are straining the SDLC process for larger organizations:
  • How development practices like test driven development, testing in production, and trunk based development are key to getting the most out of AI today.
• Where we think the industry needs to go:
  • Today we’re building layers of indirection to manage the complexity of AI. Instead, we need abstractions to obviate the complexity entirely.
  • Software Engineering is about creating a hypothesis in code and measuring the results. AI lets us test infinite hypotheses, so we need to adapt our measurement techniques and feedback loops accordingly. This should drive us towards simpler techniques.
  • We already know how to do this well, and the tools and techniques can be surprisingly simple.