Setting targets for developer productivity metrics

In the podcast, Abi Noda and Laura Tacho open the conversation with the importance of improving team systems without distorting data. They emphasize that simply setting targets like increasing innovation rates can lead teams to game the system, which is counterproductive. Instead, the focus should be on empowering teams to engage in the improvement process, where they feel supported and have the necessary time to enhance their systems.

The hosts introduce the Core 4 framework, which is designed for measuring developer productivity. This framework integrates concepts from DORA, SPACE, and FX, aimed at providing a comprehensive approach to metrics. Laura and Abi discuss how organizations often struggle with metrics comprehension, addressing the importance of starting with controllable input metrics instead of solely focusing on output metrics that lead to distortion.

Throughout the discussion, the hosts reference Goodhart’s Law, which warns that once a measure becomes a target, it ceases to be a good measure. They explain how organizations can sometimes distort outcomes to meet targets, which leads to lower quality and reliability in the overall system. This lays the groundwork for developing a healthier metrics system that truly drives improvement.

A critical part of the episode is the distinction between input metrics (controllable inputs that teams can influence) and output metrics (results produced by the system). The hosts provide a health analogy to illustrate this concept, emphasizing that the focus should be on controllable inputs that guide teams toward better outcomes. They stress that targeting outputs can lead to unintended consequences, such as reduced quality of work.

Abby and Laura touch on the topic of gamification within metrics, discussing how poorly designed systems can lead to inflated scores without genuine improvement. They recommend that metrics should be multi-dimensional to avoid one-dimensional fixes that overlook other critical aspects of productivity. Relying solely on specific targets can lead to dysfunctional behavior within teams.

The discussion also delves into strategies for gaining developer buy-in for metrics, emphasizing transparency, communication, and involvement in defining what metrics represent. The hosts stress the need to link developer metrics to broader organizational goals, bridging the gap between engineering efforts and business outcomes, ensuring all team members understand the metrics’ purpose and how they contribute to overall success.

In the concluding part of the episode, Abby and Laura stress the importance of setting realistic targets for metrics. They discuss the value of benchmarks, the nuance behind their application, and caution against making blanket statements about what success looks like. Each team or project may require a tailored approach based on its unique context and challenges.