Metrics Matter: How to rethink Measurement
Policymakers are working with increasing urgency to deliver on complex, interconnected priorities and are expected to deliver quantifiable results. Transparent measurement is fundamental to effective policy but it comes with significant dangers. Rethinking analytics is indispensable to sustainable strategy.
CHALLENGE
Transparent measurement has become the foundation of public policy. It is impossible to tackle the complexity of today’s interconnected challenges without a range of analytical tools.
So with ever more sophisticated tools available, why aren’t there more measurable signs of significant progress?
The challenge is rarely one of will. While the cultural sector more than most might still put a lot of store in gut feeling and inspiration, most still recognise the value of empirical analysis.
Most want to find standardised frameworks for accountability, comparison, and aggregation, often motivated by a desire for greater efficiency.
Empirical evidence not only informs actions, it also provides stakeholders and the public with a way to understand and evaluate progress.
But the challenge is to use metrics as an active tool in strategic thinking:
• As a foundation block for sustainable change.
• As a reliable measure of progress.
• As a transparent internal and external indicator of success.
• And as an agile way to build long-term development.
But clarity, accountability and complexity sit in constant tension.
Public bodies are rightly expected to demonstrate accountability. Metrics must therefore be clear, comparable, and robust enough to withstand scrutiny.
WORKING WITH A CHALLENGE PARTNER
Challenging established frameworks can be uncomfortable.
That’s where Rethink Creative comes in.
We work as a Challenge Partner, helping you examine assumptions, test measurement approaches and align policy more closely with real-world outcomes.
CONTACT US TODAY
CONTEXT
Measurement is an indispensable tool for policy-makers and business. For some it’s an obsession and that’s not surprising. The Age of Data has given us all so much to measure.
90% of all data in the history of the world was created in the last two years. The environmental impact of data storage is now roughly equal to that of the airline industry and AI is expected to drive annual growth of between 9 and 18%.
While Big Data was overwhelmingly advantageous to global giants, AI scales down. Even the smallest creative business will have access to unprecedented levels of digital insights.
Predictive analytics – mining data to forecast future trends – is growing at around 30% a year and is expected to be worth more than a trillion dollars by 2035.
There is plenty of convincing analysis of the failure of analysis.
Linda Chang, of the Toyota Research Institute, makes a compelling case for “quantification fixation” where faith in numbers dictates actions rather than guides them. “When we count, we change what counts.”
Data is also often used to justify pre-conceived ideas – as a numerical crutch for our confirmation bias. Sometimes this takes the form of what might be called fig-leaf consultancy, where an external agency is contracted to find evidence to backfill support for a policy already decided or implemented.
Perhaps the best-known problem is Analysis Paralysis, where an overwhelming amount of data can lead to indecision.
Sometimes the problem is a matter of internal culture, time and resources. We often end up measuring what is easier to measure, rather than what is most meaningful and insightful.
Measurement can begin to shape priorities in ways that are not always fully visible.
What is easy to measure can become what is prioritised.
What is prioritised can drift from what was originally intended, reducing the likelihood of achieving deeper intended outcomes.
The policy environment is increasingly complex. Expectations on public investment span economic, social, environmental, and technological outcomes, often simultaneously.
Yet measurement frameworks often rely on linear assumptions. Inputs lead to outputs, outputs indicate outcomes, and outcomes can be interpreted as proxies for impact.
In practice, more sophisticated thinking about systems change, behavioural effects and long-term impact already exist across parts of the ecosystem.
Outputs remain important. They tell us what has been delivered. However, they do not always explain what has changed as a result.
CONSEQUENCES
When KPIs are misaligned with intent, the effects are cumulative.
Incentives begin to shift –What is measured becomes what is prioritised. Activity that is easy to evidence can take precedence over activity that is harder to capture but more significant.
Impact can become uneven or incomplete – Participation may increase without corresponding progression. Investment may be distributed without equivalent strengthening of ecosystems. Access may improve without addressing underlying structural barriers.
Confidence in outcomes can outpace reality. Strong reporting can create a sense of progress that is not fully aligned with or reflective of underlying change.
One of the most significant consequences is the under-recognition of spillover effects.
Policy interventions often generate value beyond their immediate objectives. In the creative and cultural sectors in particular, innovation in one domain can transfer into others.
Technologies developed for creative applications may later be used in areas such as health, education or defence.
For example, developments in game engine technologies supported within creative sector contexts have increasingly been applied to simulation and training environments in defence and healthcare.
These cross-sector effects are often significant but are rarely built into measurement frameworks in a systematic way.
When spillover is not recognised across the system, opportunities to understand, connect and scale value can be missed across departments, sectors and places.
FROM MEASUREMENT TO MEANING
KEY QUESTIONS
What is actually changing as a result of this intervention, beyond delivery activity?
Who is benefiting and who is not yet seeing the intended outcomes?
Is change likely to persist beyond the life of the intervention?
What behaviours and incentives are being reinforced across the wider system?
Where can additional value be created beyond immediate objectives, and how can that value be recognised or shared.
Taken together, these questions help move measurement closer to impact while retaining clarity and accountability.
At the heart of these challenges is the Key Performance Indicator.
KPIs are often treated as fixed indicators of performance.
In practice, they could and should function as part of a learning system that shapes attention and behaviour.
In more complex policy environments, they are most useful when they evolve over time, combine quantitative and qualitative insight, allow for local interpretation where appropriate, and remain aligned with intended behavioural and system change.
