Engineering Enablement by DX
How Monzo runs data-driven AI experimentation
· Summarized
Key takeaways
- Monzo adopts AI cautiously, emphasizing structured experimentation to maintain compliance and innovation in a regulated industry.
- Data-driven evaluations are crucial for understanding the effectiveness of AI tools and guiding investment decisions.
- The introduction of AI tools has intensified existing bottlenecks, particularly in code reviews, prompting Monzo to develop solutions like preview environments.
- AI tools are being embraced across departments, not just engineering, improving collaboration and speeding up the prototyping process.
- The future of AI in engineering could involve orchestration of specialized AI agents, enhancing task management and productivity.
Summary
Introduction to Monzo's AI Journey
In this episode of Engineering Enablement, Laura Tacho speaks with Fabien Deshayes from Monzo Bank about how the bank is adopting AI in a regulated industry. Monzo, serving over 30 million customers, utilizes AI to enhance developer experience and productivity through structured experimentation.
Structured AI Experimentation at Monzo
Fabien elaborates on Monzo's careful approach to experimenting with AI tools. The bank emphasizes a structured rollout, beginning from GitHub Copilot to modern AI tools. They use data-driven evaluations to measure the effectiveness of tools like Cursor and Windsurf, ensuring they fit within the context of their organization.
Measuring AI Impact and Adoption
Monzo created specific criteria to assess the adoption and satisfaction of AI tools among engineers. They track metrics such as lines of code produced, tool retention rates, and user satisfaction. Fabien explains the importance of being data-driven to substantiate decisions related to continued investments in AI.
Addressing Bottlenecks and Improving Developer Experience
The introduction of AI tools has created new challenges for Monzo, particularly with code review processes. Fabien discusses how they are addressing bottlenecks, like enhancing review speed with temporary preview environments, allowing engineers to test and validate their code without overloading the development environment.
Broadening AI's Scope Within Teams
Monzo allows not just engineers but also product managers and designers to leverage AI tools for prototyping and validation, fostering cross-department collaboration. This shift is enhancing the quality and speed of the development process, significantly changing how features are developed.
Future of AI in Engineering at Monzo
As AI continues to evolve, Fabien shares insights on upcoming innovations, particularly in orchestration of AI agents. He envisions a scenario where engineers manage a team of specialized AI tools to streamline various tasks, thus enhancing overall productivity.
Sponsors
- Monzo · A digital bank in the UK, focusing on innovation and customer experience, serving over 30 million customers.
- GitHub Copilot in VS Code · A tool that integrates with Visual Studio Code to assist developers through AI-driven suggestions.
- Cursor · An AI-powered coding tool that was trialed at Monzo.
- Windsurf · An AI-powered development tool that was also part of Monzo's trials.
- Claude Code · Another AI tool that was trialed for its applicability in development tasks.
Show notes
In this episode of Engineering Enablement, host Laura Tacho talks with Fabien Deshayes, who leads multiple platform engineering teams at Monzo Bank. Fabien explains how Monzo is adopting AI responsibly within a highly regulated industry, balancing innovation with structure, control, and data-driven decision-making.
They discuss how Monzo runs structured AI trials, measures adoption and satisfaction, and uses metrics to guide investment and training. Fabien shares why the company moved from broad rollouts to small, focused cohorts, how they are addressing existing PR review bottlenecks that AI has intensified, and what they have learned from empowering product managers and designers to use AI tools directly.
He also offers insights into budgeting and experimentation, the results Monzo is seeing from AI-assisted engineering, and his outlook on what comes next, from agent orchestration to more seamless collaboration across roles.
Where to find Fabien Deshayes:
• LinkedIn: https://www.linkedin.com/in/fabiendeshayes
Where to find Laura Tacho:
• LinkedIn: https://www.linkedin.com/in/lauratacho/
• Website: https://lauratacho.com/
• Laura’s course (Measuring Engineering Performance and AI Impact): https://lauratacho.com/developer-productivity-metrics-course
In this episode, we cover:
(00:00) Intro
(01:01) An overview of Monzo bank and Fabien’s role
(02:05) Monzo’s careful, structured approach to AI experimentation
(05:30) How Monzo’s AI journey began
(06:26) Why Monzo chose a structured approach to experimentation and what criteria they used
(09:21) How Monzo selected AI tools for experimentation
(11:51) Why individual tool stipends don’t work for large, regulated organizations
(15:32) How Monzo measures the impact of AI tools and uses the data
(18:10) Why Monzo limits AI tool trials to small, focused cohorts
(20:54) The phases of Monzo’s AI rollout and how learnings are shared across the organization
(22:43) What Monzo’s data reveals about AI usage and spending
(24:30) How Monzo balances AI budgeting with innovation
(26:45) Results from DX’s spending poll and general advice on AI budgeting
(28:03) What Monzo’s data shows about AI’s impact on engineering performance
(29:50) The growing bottleneck in PR reviews and how Monzo is solving it with tenancies
(33:54) How product managers and designers are using AI at Monzo
(36:36) Fabien’s advice for moving the needle with AI adoption
(38:42) The biggest changes coming next in AI engineering
Referenced:
Follow this show on AudioBrevity and new episodes get summarized for you. Create a free account