I am often asked what Scrum Teams and organizations should measure. And while measurement is important, it’s not just about what we measure. It’s about understanding why we measure and how we use those measures. And well, that’s what actually helps you figure out what to measure. In this post, I will share some common patterns I have noticed that lead to using measures poorly.
If you start to recognize some of these patterns in your team or organization, well, that’s a good thing. Because seeing and understanding the problems (transparency) is the first step to improvement. And Evidence-Based Management (EBM) is a framework you can use to get back on track.
Yep, that’s right. Measures should NOT be used as a way to determine if a team or a leader is succeeding or failing.
Measures are a source of empirical information that helps guide decision-making. And the moment we make “achievement” of certain measures mean something about us in terms of our effort, our skills, or our worthiness, we risk losing the information we need to guide us in navigating complexity and unpredictability.
And then making the measures “look good” becomes the goal. And if teams are chasing measures, well… you can imagine the misalignment and friction this causes across the organization.
This leads us to the second common mistake.
Because we will want to measure only the things we can control, that likely means we are not measuring customer outcomes. And we are likely not creating goals that focus on customer outcomes.
When teams and organizations are focused mostly on measuring activities and outputs, they are missing key information. The number of milestones achieved or budget burned (activities) does not tell you much about your ability to deliver value. And even measures like revenue or features delivered (output) does not tell you much about the customer experience.
And this focus on measuring activities and outputs is likely to lead to “gaming” of the measures (whether conscious or unconscious).
Essentially, we forget that we are working in complex, unpredictable environments. We cannot control customer outcomes. But we can leverage empiricism to navigate if we have transparency to what’s really going on and frequently inspect and adapt our goals, our progress towards them, and our strategy.
This leads us to the third common mistake.
Measuring customer outcomes is a tricky business. How users respond to our products and services is influenced by many factors. We also have to consider both our short-term and long-term ability to deliver value, and quality and technical excellence come into play in this conversation.
This is why EBM introduces 4 Key Value Areas (Unrealized Value, Current Value, Time to Market, and the Ability to Innovate) that enable a more robust set of perspectives to understand both market value and organizational capability. Having multiple lenses helps balance our progress towards longer-term goals and our responsiveness and effectiveness as an organization.
When I facilitate classes and coach my clients on using measures effectively, we start with understanding why we measure and identify customer outcome-based goals. And then we determine what measures (i.e. empirical information) will help us understand progress towards our goals and help us validate (or invalidate) our assumptions. We also discuss that measures can and should change over time, so this needs to be part of our inspect and adapt feedback loops.
Do you want to learn how to leverage EBM to improve business agility?
Scrum.org has launched an official 1-day training for EBM. It is called Professional Agile Leadership – Evidence-Based Management (PAL-EBM). In this class, you will learn how goals, measures, and experiments fit together in order to create better alignment, empower teams, and enable innovation.
What I love about having a formal training opportunity is that it brings the concepts to life in a meaningful way. It allows participants to use a case study to practice applying concepts in real-time. And when you are grounded in the foundational knowledge, you can then apply the framework more effectively in your context.
AGILE SOCKS is a registered trademark of Agile Socks LLC. Other marks used herein are the property of their respective owners. For more information see Trademark Notice in Terms & Conditions.