How Warp ships 2,000 PRs a month with AI factories | Zach Lloyd (CEO, Warp)

Zach Lloyd introduces Warp as a platform that has evolved from a modern terminal into a comprehensive software factory, emphasizing it as a core product concept that includes repositories, servers, configurations, and code-defined agents. He critiques the dehumanizing tone often associated with the 'factory' metaphor, favoring a more whimsical comparison like Santa's workshop, where AI agents craft software from ideas. Zach describes how work is initiated publicly in Slack, tagged to a factory named Wilson, which then manages the process from triage to implementation through integrated tools like Linear and GitHub. This process includes automated issue creation, code PRs, and QA verification, even generating videos of the completed features to ensure quality without human review, although currently all PRs are reviewed by humans. He stresses the importance of the factory being a public, transparent system that enables multiple team members to contribute and observe, moving away from siloed, local workspaces.

Zach details how the factory automates many steps of software development, from triaging inputs in Slack to creating issues, merging pull requests, and verifying with computer use videos. The system fosters an environment where work is continuously happening in the open, supporting collaboration and contribution across teams. Beyond individual workflows, the factory consolidates data centrally in the cloud, providing managers with dashboards to track development velocity, automation level, human interactions per PR, and overall progress. Zach explains how these metrics serve as signals for operational efficiency, with human review remaining a bottleneck that they are exploring to further automate, such as risk scoring PRs with AI and using automated risk assessments to streamline approval processes.

Zach discusses tools for scoring agent runs, identifying failure modes, and implementing self-improvement loops where AI agents analyze failed runs to suggest updates to factory configurations. He introduces the idea of replaying past tasks with different model configurations to evaluate trade-offs between cost and quality, enabling evidence-based decisions for selecting the best models for specific tasks. This data-driven approach aims to optimize the entire development process, from cost management to quality assurance, encouraging continuous experimentation and refinement of AI models and processes. Zach also emphasizes the value of session-level telemetry for granular insights into productivity, code review, and resource usage, which when aggregated helps manage large-scale AI-driven development teams effectively.

Moving beyond engineering, Zach demonstrates how AI assists with non-technical tasks such as updating Figma slide decks, creating presentations, and conducting market research. Using various AI MCPs (Managed Chain Programs), he issues commands to generate new slides, refine designs, and summarize customer questions from sales calls. Zach highlights that AI tools enable him—the CEO—to perform complex, creative, and strategic work without deep technical skill, illustrating how AI can extend the capabilities of non-technical roles. He shares real-time examples of launching tasks across different tools, like Granola for meeting summaries and GOG CLI for outreach analysis, showing a workflow where AI acts like a personal assistant executing a range of business functions efficiently.

Zach addresses concerns about the human bottleneck in review processes, noting that all pull requests still undergo human review but emphasizing that the process is evolving towards more automation. He explains how current practices involve using AI for risk scoring PRs, allowing for automatic approval of low-risk changes, and reserving human review for more complex or high-risk code. Zach stresses the importance of transparency and constant use of their factory tools to monitor team activity, measure productivity ('human interactions per PR'), and ensure that the team focuses on the most impactful work. He also speaks about maintaining quality through traditional product development processes like user interviews and design jams, blending human insights with AI enhancements to sustain effective, high-quality software development at scale.