How to measure AI developer productivity in 2025 | Nicole Forsgren · Lenny's Podcast: Product | Career | Growth summary

Key takeaways

Traditional productivity metrics like lines of code are misleading and can be easily gamed.
Developer Experience (DevEx) encompasses not only productivity but also developer happiness and cognitive load.
AI tools can enhance productivity but also introduce new challenges requiring developers to evaluate and trust AI-generated outputs.
Establishing a successful DevEx team involves listening to developer feedback, strategic prioritization, and effective communication.
Understanding leadership's priorities is essential for measuring the impact of DevEx improvements and securing buy-in.
Empathy and clear communication are vital elements in improving developer workflows and experiences.

Summary

The Challenge of Measuring Productivity

Nicole Forsgren discusses how traditional productivity metrics can be misleading. She explains how focusing on lines of code can create systems that are easily gamed, leading to poor assessments of developer performance. She emphasizes the importance of understanding the context of development work and raises concerns about the trustworthiness of AI-generated code in relation to productivity.

The Importance of Developer Experience (DevEx)

Forsgren highlights the concept of Developer Experience (DevEx), outlining that it's not just about productivity but also includes aspects like developer happiness and workflow optimization. She points to the need for understanding cognitive load and flow states in engineering, especially in an era where AI is transforming the field.

AI's Impact on Engineering Performance

The conversation delves into the dual impact of AI on engineering productivity. While AI tools have the potential to streamline processes and save time, they also introduce new challenges where developers may spend more time reviewing AI-generated code rather than writing it. Forsgren emphasizes a strategic approach in blending AI tools to enhance developer workflows effectively.

Building Effective Developer Experience Teams

Forsgren shares a seven-step framework for establishing a successful DevEx team. This includes listening to developer feedback, getting quick wins, using data to inform strategies, and effectively communicating those strategies throughout the organization. She discusses the importance of iterating and adapting as tools and workflow change.

Measuring Impact and Gaining Buy-in

The discussion also covers the nuances of measuring the impact of DevEx efforts and how to gain buy-in from leadership. Forsgren illustrates the necessity of aligning measurements with what matters to leadership, whether that’s velocity, market share, or cost savings. She stresses the need for a solid understanding of what's working and what's not.

The Human Element in Engineering

A significant theme is the importance of empathy and communication. Forsgren argues that engineering teams should not just focus on tools and processes but should actively engage with developers to understand their challenges, thereby leading to more meaningful and impactful changes to their work environment.

Show notes

Nicole Forsgren created the most widely used frameworks for measuring developer productivity—DORA and SPACE. She wrote the foundational book Accelerate and is about to release her newest book, Frictionless, a practical guide for helping teams move faster in the AI era. She’s currently Senior Director of Developer Intelligence at Google.

We discuss:

1. Why most productivity metrics are a lie

2. Signs that your engineering team could be moving much faster

3. Why AI accelerates coding but developers aren’t speeding up as much as you think

4. AI’s impact on engineers getting into “flow”

5. Her framework for building and scaling a developer experience team

6. The three components of developer experience: flow state, cognitive load, and feedback loops

—

Brought to you by:

Mercury—The art of simplified finances: https://mercury.com/

WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs: https://workos.com/lenny

Coda—The all-in-one collaborative workspace: https://coda.io/lenny

—

Where to find Nicole Forsgren:

• Twitter: https://twitter.com/nicolefv

• LinkedIn: https://www.linkedin.com/in/nicolefv/

• Website: https://nicolefv.com/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Nicole Forsgren

(05:09) The concept of developer experience (DevEx)

(08:33) Flow state and cognitive load in the age of AI

(12:02) Challenges in measuring productivity with AI

(21:19) The importance of developer experience for business value

(22:20) Common issues and solutions in developer experience

(26:49) Signs your eng team is moving too slow

(29:52) How AI is improving productivity

(33:32) Real examples of productivity improvements

(36:35) Introducing her new book, Frictionless

(43:40) How to get started building a DevEx team

(45:15) The impact of forming developer experience teams

(46:15) How to measure the impact of DevEx teams

(48:53) Measuring the impact of AI tools on productivity

(55:16) Survey design for developer experience

(57:59) Popular AI tools for developers

(59:08) Bringing a product mindset to DevEx improvements

(01:00:40) AI corner

(01:02:33) Lightning round and final thoughts

—

Referenced:

• How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google): https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer

• DORA: https://dora.dev/

• The SPACE framework: A comprehensive guide to developer productivity: https://getdx.com/blog/space-metrics/

• Measuring developer productivity with the DX Core 4: https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/

• Gloria Mark’s website: https://gloriamark.com/

• Taking Flight with Copilot: https://dl.acm.org/doi/10.1145/3589996

• DevEx in Action: https://spawn-queue.acm.org/doi/10.1145/3639443

• CodeX: https://openai.com/codex/

• Devin: https://devin.ai/

• Abi Noda on LinkedIn: https://www.linkedin.com/in/abinoda/

• DX is joining Atlassian: https://getdx.com/blog/dx-is-joining-atlassian/

• GitHub Copilot: https://github.com/features/copilot

• Cursor: https://cursor.com/

• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell

• Gemini Code Assist: https://codeassist.google/

• Claude Code: https://www.claude.com/product/claude-code

• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper

• Love Is Blind on Netflix: https://www.netflix.com/title/80996601

• Shrinking on AppleTV+: https://tv.apple.com/us/show/shrinking/umc.cmc.apzybj6eqf6pzccd97kev7bs

• Ninja Creami: https://www.amazon.com/Ninja-NC301-CREAMi-Containers-Bundle/dp/B0BLGR5JPV/

• Jura coffee maker: https://www.amazon.com/Jura-Nordic-Automatic-Coffee-Machine/dp/B0CF65BFZ1/

—

Recommended books:

• Frictionless: https://developerexperiencebook.com/

• DevEx Workbook: https://developerexperiencebook.com/#workbook

• Outlive: The Science and Art of Longevity: https://www.amazon.com/Outlive-Longevity-Peter-Attia-MD/dp/0593236599

• Back Mechanic: https://www.amazon.com/Back-Mechanic-Stuart-McGill-2015-09-30/dp/B01FKSGJYC

• How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything in Between: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512/

• The Undoing Project: A Friendship That Changed Our Minds: https://www.amazon.com/dp/B01KBM82M4/

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

Lenny may be an investor in the companies discussed.

To hear more, visit www.lennysnewsletter.com

How to measure AI developer productivity in 2025 | Nicole Forsgren