Benchmarking: What is it?

The idea of "benchmarking" seems to a concept that people love to throw around at work, but very often is misunderstood. I'd like to help demystify it.  While I am no expert in benchmarking, almost a decade of doing benchmarking for a range of campaigning organisations has meant I have needed to research what it was, form a clear opinion on it and apply it in practice.

The idea (but not the application) of benchmarking is very simple: comparing common processes or metrics across different initiatives.

Benchmarking helps determine how good the results you are achieving and the process you are using are. If you achieve 25% on something, is that great, average or poor? You don't know until you identify what is great, average and poor results - and that is benchmarking.

A Benchmarking Example

A good example is how we each present ourselves.  There are elements of this we can choose (e.g. clothing, hair styles), elements we can not choose (e.g. genetics) and elements we can influence (body shape, how we speak, how we behave, lifestyle). As "social" animals we are constantly comparing others and ourselves with others.

This could be called social benchmarking. How we look is not benchmarking, but how we look compared to others is. We can measure some aspects and can't easily measure others. But it is not the results of individual comparisons that make the real different, but the package of comparisons and its impact on the end result.

What is NOT Benchmarking?

• Measuring results of any one initiative (email, action, campaign)
• Analysis of how a single email or campaigning action performed
• Evaluating the impact/success of a single action or campaign
• Reporting on how any one initiative performed
• Listing best practices used in any single initiative
• The results of a single survey of supporters or the public
• Producing a single plan or strategy
  

While these can contribute to a benchmarking effort, they are not in themselves benchmarking because they do not compare the results to anything.

What IS Benchmarking?

• Comparing the results of multiple initiatives (email, action, campaign, survey)
• Analysis of how multiple emails or campaigning actions performed
• Comparing the evaluation/success of multiple actions or campaigns
• Comparing best practices between actions, campaigns or organisations
• Comparing strategies or plans across organisations

The hardest part of benchmarking is really ensuring there is common criteria that is comparable. This means consistency of approach between multiple initiatives and multiple organisations. While the analysis of each single initiative is the most time consuming, ensuring the consistency of analysis is critical for insightful benchmarking.

Why Benchmark?

The basic reason for benchmarking is improvement. If you do regular benchmarking, it is thus for continuous improvement. The reason for improving is not only to be better in an area than others, but also to not get left behind and increase the benefits for a given cost and effort.

What this means in practice depends on what you benchmark. But for campaigning it usually means:

1. Having a greater campaigning impact (and ideally faster)
2. Recruiting more supporters (and not losing existing ones)
3. Cutting out ineffective activities (and the associated cost and effort)

Benchmarking Approaches

There are generally a two different styles of benchmarking:

1. Internal benchmarking: where results are compared internally over multiple different activities, time periods, geographical areas, etc.
2. Peer Benchmarking: where results are compared between organisations in the same sector
   

Internal benchmarking is relatively easy because the information required is readily available (if it exists).  Peer benchmarking is more difficult because it requires either use of publicly available data which is either incomplete or over-aggregated. Collaborative benchmarking occurs when multiple organisations each contribute data for the benchmarking exercise.

Furthermore, benchmarking can either be:

1. Quantitatively-oriented: where metrics are calculated and compared e.g. what is a good performance level and who was closest/furthest to it. This is generally data-right (more initiatives compared) but context-poor (less information about each initiative being compared).
2. Qualitatively-oriented: where processes and perception is critiqued and compared  e.g. what "best" looks and/or feel like and who was closest/furthest to it. This is generally data-poor (fewer initiatives compared) but context-rich (more information about each initiative being compared).
   

Benchmarking in Practice

Producing the 2009 eCampaigning Benchmarking Report is thus a qualitative and quantitative collaborative peer benchmarking initiative. 

The quantitative analysis requires four key steps:

1. Identifying the measures that are important and measurable
2. Collecting uniform input data in terms of what the data represents and how it is formatted
3. Analysing the input data in a consistent way (e.g. consistent formulas)
4. Comparing the results between emails, actions, organisations, themes, segments, countries, etc.
5. Review the results, interpret their meaning and make recommendations based on the findings

The qualitative analysis more of a evolving cycle:

1. Determine what "good" looks/feels like and how to recognise it
2. Design a way to record and report the findings
3. Apply the current methodology to a few real initiatives
4. Review if the methodology is suitable and refine it as necessary
5. Apply the refined methodology to a few new real initiatives (and refine further if necessary)
6. Review the results, interpret their meaning and make recommendations based on the findings

You may already use forms of internal benchmarking such as:

• Split (A/B) Testing of email and website performance
• Comparing results between email and actions
• Looking at peer organisations' websites and seeing what they do different/similar
• Sharing normal performance statistics with people in other organisations
  

Ideas for Benchmarking

Doing an eCampaigning Benchmarking Study is only one way you can use benchmarking. Other ways include:

• Surveying opinion (e.g. public, supporters, campaigning targets) before launching a campaign and then re-surveying them during and/or after the campaign and comparing the change in results
• Comparing your strategy with that of other organisations (or internally across departments, across time, etc.)
• Comparing campaigning communications (e.g. actions, emails, printed material, media coverage)

You probably have more ideas of what types of campaigning benchmarking could be done, so please suggest some others.