posted on
One of the big advantages in developing written campaign plans is that they help you keep track of your progress through a campaign: where did you succeed, where did you fail, which assumptions proved correct and which didn’t. This is particularly true when you embrace an open approach to advocacy by sharing your goals, strategies and tactics with a wide variety of people who can help you judge how well you are doing and hold you to high standards. (We recently shared a midyear update on how our state campaigns are doing.)
But what happens when you simply can’t tell what is going on in your campaign, despite your best efforts? We call this problem “the missing middle.”
Takeoffs and landings
Pilots say that the most dangerous parts of flying are takeoffs and landings. But when it comes to tracking advocacy campaigns, advocacy leaders know these parts are by far the easiest.
What makes a campaign take off is the series of tactics you put in motion to advance along your strategic pathway towards your goal. As we saw in our tactics toolbox post, there are a lot of tactics to choose from. The good news is that once you have selected your tactics it’s usually pretty easy to set metrics for each tactic and track your process along the way. For example, if you are planning on producing a report on expanding pre-K, you can track that the report was created, how many people downloaded it from your website and what kind of feedback it received.
On the other end of your journey, having clarified your goals in your campaign plan, you will usually be in a pretty good position to assess whether that goal has been reached in the end. For example, at the end of the campaign you will know whether your state’s pre-K program was actually changed and with a little work will be able to say what percentage of your report’s recommendations made it into the final compromise that was signed into law.
The missing middle
Yet, even when you achieve your goal, it’s not always clear why it was reached. If your state’s pre-K program was changed, was it that research report that made the difference? Was it perhaps one of the other tactics in your campaign that was ultimately the cause? Or maybe it was going to happen anyway and your advocacy efforts just so happen to occur alongside these inevitable changes.
One way to think about this challenge is to plot out the ease with which you can measure the different aspects of your campaign against the timeline of the campaign itself. In the beginning of your campaign it’s relatively easy to measure your progress. For example, it’s simple to track whether the report you wanted to publish actually got produced on time.
However, as you proceed through the campaign timeline, it becomes progressively more difficult to measure your impact. After it’s published, you can look at downloads of the report and media mentions. With careful tracking you might be able to see if the report is mentioned in the legislative debate. By reviewing the various drafts of legislation, you might also be able to tease out some areas that closely map recommendations in the report.
The effort to collect metrics on the impact of your report gets harder and harder until finally you are almost completely in the dark. Did it influence the thinking of the small group of people charged with hammering out a last minute compromise? It will be difficult to tell from any public record. Often you will hear competing stories about what actually happened behind closed doors.
Then, right at the end when that final deal is publically announced, your task of collecting metrics becomes much easier again as you compare the policy in that deal to the original goal in your plan.
One reason that advocacy metrics have proven so difficult to track in the middle of a campaign is that unlike the example given above, tactics don’t exist in isolation. The true impact of your tactics is often the result of the way they interact with each other. To use a non-advocacy example, if your goal was to sweeten your coffee, you might deploy a tactic of dropping in a sugar cube and a second tactic of stirring the coffee with a spoon. They both contributed to the goal of sweeter coffee but stirring your coffee like crazy without the sugar cube wouldn’t accomplish anything, except making a big mess. The same is often true with advocacy tactics. The real power comes from the interaction effects between multiple tactics, which short of repeated advocacy experiments can be very difficult to independently quantify.
The challenge of measuring why a campaign succeeded or failed is compounded by how much of the activity in the middle of a campaign happens out of view. Why wasn’t that pre-K bill brought up for a vote in committee? Would a larger rally have made the difference? Could your report have been more compelling? Were your lobbying efforts ineffective? The answers to those questions are often found only in the heads of a small number of decision-makers who usually don’t have much interest in making the true reasons for their decision public.
Minding the gaps
In our campaigns, we have found three approaches that can help minimize the challenges of these missing metrics.
Stay humble. One of the reasons it’s so important to write things down and then track what actually happens in a campaign is that without these steps, we tend to fill in the gaps with recollections that are overly positive about what we accomplished in our work. The stories we are most likely to believe about what happened inside the missing middle of a campaign are the ones that are self-flattering. It’s important to constantly strive to bring humility to the examination of your campaigns. If you are telling yourself that the campaign you developed was flawless and you would have reached your goal except for this unexpected political development in the middle that will probably never happen again, it’s worth taking a step back and exploring a more humble interpretation of events.
Build insider relationships. Remember that there are people who do know what is happening in the middle of your campaign: elected officials, senior aides, trusted advisors and many more. They just aren’t the kind of people who tend to share that information widely. You can fill in the gaps in your own metrics by building trusting relationships with people in a position to see what you can’t see. The more of these relationships you have, the more successful you will be in developing an objective understanding of the impact of your campaign.
Think qualitatively. In his 1963 book Informal Sociology: A Casual Introduction to Sociological Thinking, William Bruce Cameron famously wrote “It would be nice if all of the data which sociologists require could be enumerated … However, not everything that can be counted counts, and not everything that counts can be counted.” More qualitative assessments can help you fill in the gaps in the missing middle. That could be everything from a formal focus group to a series of informal conversations. People’s impressions of your influence can often be more accurate than the hard numbers you get from the things you can count at the beginning and end of a campaign.
We hope this discussion of metrics was helpful as you work to measure the impact of your own campaigns. Please join us in the conversation about education advocacy on Twitter, Facebook and Instagram.
Email: marc.magee@50can.org
Twitter: @FiftyCAN
Facebook: 50CAN
Instagram: @fiftycan