How to cut through the hype and measure AI’s real impact (Live from LeadDev London)

Key takeaways

Understanding the 'Disappointment Gap' is crucial for leaders to navigate the reality of AI in engineering.
Data should drive decisions about AI utilization rather than inflated media claims.
AI can enhance software development if coupled with strong engineering fundamentals.
The Core 4 framework provides a comprehensive way to measure performance across multiple dimensions.
The AI Measurement Framework offers specific metrics to assess the impact and utilization of AI tools.
Real-world case studies highlight the benefits of effective data measurement in improving organizational performance.

Summary

The Disappointment Gap in AI

Laura Tacho addresses the widening gap between the reality of AI usage in engineering teams and the exaggerated claims made in headlines. This gap can lead to a sense of disillusionment among developers who may feel that AI tools do not meet the expectations set by sensationalized media reports.

The Importance of Data Over Hype

To effectively utilize AI in organizations, leaders need to focus on data and real metrics rather than getting caught up in hype. Laura emphasizes that accurate data can guide leaders in making informed decisions about AI implementation and usage within their teams.

Building Better Software with AI

Laura discusses utilizing AI to enhance software delivery without sacrificing quality or reliability. She underscores that despite AI's potential, solid engineering fundamentals remain the foundation for successful software development.

The Core 4 Framework

Laura introduces the Core 4 framework, which helps organizations measure engineering performance across speed, effectiveness, quality, and impact. Each category holds significance and must be balanced to achieve optimal performance.

Measuring AI Impact: The AI Measurement Framework

This framework was developed to help organizations track the utilization, impact, and cost of AI tools. Understanding how AI tools are adopted and assessing their impact on productivity can help organizations make better strategic decisions regarding AI.

Real-World Examples and Case Studies

Laura shares insights from organizations like Block and others, illustrating how they effectively used the frameworks introduced to measure productivity and save significant time through AI adoption.

Show notes

In this special episode of the Engineering Enablement podcast, recorded live at LeadDev London, DX CTO Laura Tacho explores the growing gap between AI headlines and the reality inside engineering teams—and what leaders can do to close it.

Laura shares data from nearly 39,000 developers across 184 companies, highlights the Core 4 and introduces the AI Measurement Framework, and offers a practical playbook for using data to improve developer experience, measure AI’s true impact, and build better software without compromising long-term performance.

Where to find Laura Tacho:

• X: https://x.com/rhein_wein

• LinkedIn: https://www.linkedin.com/in/lauratacho/

• Website: https://lauratacho.com/

In this episode, we cover:

(00:00) Intro: Laura’s keynote from LDX3

(01:44) The problem with asking how much faster can we go with AI?

(03:02) How the disappointment gap creates barriers to AI adoption

(06:20) What AI adoption looks like at top-performing organizations

(07:53) What leaders must do to turn AI into meaningful impact

(10:50) Why building better software with AI still depends on fundamentals

(12:03) An overview of the DX Core 4 Framework

(13:22) Why developer experience is the biggest performance lever

(15:12) How Block used Core 4 and DXI to identify 500,000 hours in time savings

(16:08) How to get started with Core 4

(17:32) Measuring AI with the AI Measurement Framework

(21:45) Final takeaways and how to get started with confidence

Referenced: