Doubling the productivity of your engineering team using AI (Brian Scanlan)

This section discusses Intercom's ambitious goal to double engineering throughput within nine months through AI adoption and organizational change. Brian Scanlan highlights how Intercom's evolution into an AI-first company has significantly impacted its growth rate, enabling it to surpass typical SaaS industry trends and even displace traditional products with AI-powered solutions like their customer support agent, FIN. The company’s strategic pivot towards AI involved standardizing on a platform (Cloud Code), investing in internal AI skills, and restructuring workflows around agents, culminating in measurable results such as increased productivity, reduced defect backlog, and enhanced code quality.

Here, Brian elaborates on Intercom’s platform strategy, emphasizing the importance of building a flexible, scalable environment centered on an agent-first approach. This involves standardizing tools like Cloud Code, creating domain-specific skills, and enabling agents to perform any technical task a senior engineer can. The approach aims to democratize access, reduce manual effort, and ensure consistent, high-quality outputs across teams. The emphasis on evergreen capabilities over custom tooling is to maintain agility and prevent stagnation in tool improvement.

This section covers the practical aspects of scaling AI within Intercom, including the development of hundreds of skilled AI plugins, continuous monitoring through session data, and setting robust safety and quality standards. The team employs rigorous testing, session analytics, and classification models to ensure AI tools deliver reliable and safe results. Brian also discusses AI automation in pull request approvals, moving towards full autonomous approval for high-quality, small PRs—highlighting ongoing efforts to optimize workflows and increase reliability.

Brian openly discusses the financial aspect of large-scale AI deployment, revealing weekly costs around $128,000, with plans to increase as use grows. The discussion emphasizes the importance of business continuity planning, including multi-provider fallback options and insurance policies such as open source models, to mitigate risks associated with reliance on proprietary AI services. He advocates for viewing AI costs as an investment that boosts productivity and reduces manual labor, rather than just expense.

Focusing on quality control, Brian details how Intercom manages skill reliability through testing, internal reviews, and continuous updates. They encourage experimentation and learning by allowing broader team contributions but maintain high standards for core skills. The team uses session data and classification to ensure skills are effective and safe. They also discuss the importance of avoiding overconfidence in AI, especially when dealing with less experienced users, and stress ongoing audits and improvements to maintain high ethical standards and accurate outputs.

In the final sections, Brian speculates on the future changes AI will bring to roles, workflows, and organizational structures. He mentions ongoing experiments, plans for remote agents, and the integration of AI across various functions, including product management. The overarching message is that AI adoption is inevitable and transformative, and companies must prepare by continuously testing, improving, and expanding their AI capabilities to stay competitive.