Measuring production alone doesn’t say much about the overall quality of a service. Local governments must focus on evaluating their impact to drive success.
Governments need quantitative metrics to gain a sense of how things are going. This makes government performance measurement an essential part of city leadership. But prioritizing the wrong metrics can lead decision-makers astray and cause people to lose sight of their goals. Leaders need to assess the impact of their actions as the true measure of success.
“You have to measure what people care about,” said Michelle Kobayashi, a leader in the survey science field and Principal Research Strategist at the community engagement and civic analytics company Polco. She’s spent decades helping governments hone in on measuring what matters.
“People care about affordable housing, they care about homelessness, they care about food security. They likely care less about your land use plan or how many days it takes to get a building permit” she said. “One of my goals is to switch the paradigm and get decision-makers to focus on outcomes.”
Government Performance Measurement: The Difference Between Outputs and Outcomes
Local governments usually demonstrate their performance with productivity numbers, called outputs. An output is a quantitative measure that often tells “how much.” The number of cops on the street, snowplow drivers on the road, and number of trees planted in the park are all outputs. Outputs do not express the overall impact or value of an action.
For instance, the average TikTok user spends 45.8 minutes per day on the app. The average museum-goer spends 17 seconds looking at a work of art. McDonald’s sells over 6 million burgers per day. The best restaurant in the world seats around 40 people per dinner service. What do these numbers say about the quality of the experience? Not a whole lot.
Yet, the majority of governments that measure performance use outputs alone because these data are easier to collect. And it takes more cognitive energy to pinpoint more meaningful indicators. The number of employees hired to mow park lawns is available and easy to report. The problem is that outputs alone lack context. That number of employees doesn’t tell you if the parks are well-groomed. So why does this happen?
“Some decision-makers haven’t thought through logically why they are providing the services and the desired results they are trying to achieve,” said Brent Stockwell, the Assistant City Manager of The City of Scottsdale, Arizona. The city has been a leader in performance management for many years.
When it’s time for decision-making, leaders should focus most on outcome data. Measuring outcomes, the effectiveness of an action, will help leaders think through what they are doing and why. Outcomes provide context, show efficacy, and reveal whether or not you are successfully achieving your goals.
“A focus on outcomes would be the percentage of workers who are unemployed, the percentage of people who feel safe, the fair market rent of housing units—things people actually care about,” Kobayashi said.
How To Avoid Making Bad Decisions with Data
All that said, outputs are still beneficial. Stockwell says Scottsdale uses a holistic approach to government performance measurement. He said they also include output measures in performance reports to help tell the full story of the data.
“You should choose some efficiency and effectiveness (outcome) measures to demonstrate how well you are achieving your desired results,” he said. “Including key output measures also gives a sense of the volume of the work that you are doing.”
For example, if your goal is to cut the amount of time it takes to get building permits, tracking output data is necessary. Outputs can be especially helpful when you track them over time. Output data may indicate how many books a library checks out per year. Any significant changes in such outputs can provide useful color when reported alongside outcomes.
But focusing on outputs alone can mislead people if they don’t establish their goals first.
“We often have good intentions, but if we don’t measure the outcomes our plan might not turn out the way we intended,” Kobayashi said.
The Drug Abuse Resistance Education (DARE) campaign is an example of one famous failure. Virtually every 5th-grade public school classroom in the 80s and 90s hosted the program to educate children about the dangers of drugs and alcohol. But, many studies now show that DARE students became more likely to use illicit substances. The exaggerated lessons inadvertently made drugs look cool. As a result, DARE fostered a sense of curiosity that may not have appeared otherwise.
This example shows why it’s important to establish goals and measure outcomes. Focusing on how many students completed the DARE program (output) missed the point when the overall goal was to lower the percentage of young people using drugs (outcome). Identifying outcomes gives decision-makers the ability to say whether a plan was successful. Then it’s easier to assess if the tactics are working or if it’s time to go back to the drawing board.
How To Identify Outcomes
After decades of helping cities work with data, Kobayashi understands why so many local governments forgo identifying outcomes. Outcomes require consolidating the most relevant inputs, analysis, and review alongside clearly established goals. Not every city has the resources in-house to take on the project.
“We know gathering, reporting and choosing the right outcome indicators are not simple tasks. But we want successful, data-based decision-making to be realistic for all cities,” Kobayashi said. “That’s why it’s so important to me to be able to provide governments access to resources that help them get the right output data and point out outcome measures based on their goals. ”
There are philosophies and best practices governments can use to come up with goal-based outcome measures on their own.
Stockwell admits finding the true goal takes intense inquiry. And many people get stuck trying to find the perfect indicator when no such thing exists.
“People can get overwhelmed by how many things there are to measure, and also they get overwhelmed by the problems associated with every measure. Because every measure has complexity and nuance and challenges associated with it,” Stockwell said.
Stockwell suggests that when evaluating your overall performance measurement strategy, make sure that your approach is consistent and intelligent.
Some outcome-based government performance measurement examples include:
- The Pavement Condition Index measures the relative quality of pavement conditions across municipalities or street segments
- Fires confined to the room of origin
- The number of human service clients who report improved financial condition within a certain time
Kobayashi suggests using a Complete Logic Model (CLM) to determine a valid output measurement. She wrote a handbook on the topic. A CLM illustrates the relationship between resources, activities, and intended effects. Similarly, Scottsdale provides training on a simple logic model that asks similar questions. What are the results you are trying to achieve? What is the benefit to the customer? What service are you delivering? What activities are you conducting? And what resources are you using? Each of these can be measured.
For simplicity, Kobayashi suggests that local government leaders use the Five-fold Why approach. This technique requires you to repeatedly ask yourself “why” when you first name a goal. The process leads you to the core reason for taking any action. The system teaches people how to understand what they actually want. Kobayashi says the Five-fold Why approach applies to local government planning too.
Let outcome-measurement serve as a North Star. Outcomes can guide you to your destination and help you make the decisions that support your end goal. Outputs should certainly feed into your outcomes. But if you go off track, reassess your actions and keep forging ahead until you reach your target.
Learn More
Polco gives city leaders outcome measures that accurately reveal the quality of government services. Learn more about Polco’s community engagement and civic analytics solutions.
About the Author
Jessie O’Brien serves as the head copywriter for Polco.
Artificial Intelligence Demystified- AI Toolkit for Municipalities
From the Event: Cybersecurity Trends: Keep Your Organization Prepared and Compliant
Project Yellow Light: Giving Youth Voices a Media Platform to Promote Road Safety
- NLC Partnerships
Are City Leaders Ready for AI?
Leveraging Video to Showcase Your Community
- CGI Digital
