Lesson 10 - Invest in message testing – don’t rely only on instincts!

 

Essence

As there’s so much at stake in attitude change campaigning work, it’s important to invest in empirically testing what is triggered by new narratives and communications content before sharing widely, rather than just trusting your instinct about what you think will work with middle audiences.

 
Insight

When the goal of a campaign run by progressives is to change the attitudes of a more sceptical target audience, there is significant risk of getting it wrong or more worryingly, of even making the situation worse! If campaign messages and content trigger the so-called ‘backfire effect’1 , this leads to the response of hardening of attitudes. This is a lesson grounded in much first-hand experience of campaigners’ gut reaction of what they think will work with middle audiences proving wrong in focus group message testing. Testing is especially effective in identifying the elements that are NOT working or even backfiring. Based on this important feedback, campaigners can then make informed decisions on which campaign content to remove or adjust. On the more positive side, taking an iterative ‘test and learn’ approach helps in adjusting and finalising the messages and materials that are working and scale up from there to maximise impact2 .
 
So, as there’s so much at stake, it is important to invest in testing to understand exactly what is working (or not) in messaging and content. One of the Kazakh partners sums up this approach well: “Data is convincing and even just the idea of not making assumptions is very important”. There are many options for narrative testing to fit different needs and budgets – see our resource for more on this.
 


Cases

- Kazakhstan – The following were the stages of testing through the #Azamatbol (#GoodCitizens) campaign development and rollout period. 
 
Testing method
What was tested?
Stage of campaigning
1.
Focus groups 
The draft pitch, messages and early draft stories for the campaign were tested with groups from the target middle segments to see if the campaign concept works.
After initial planning but before investing in producing expensive content and videos.
2.
Randomised Controlled Trial 
The video materials for the campaign were tested to pick the winners, i.e. find the ones that did the most to shift attitudes positively.
Before the campaign rollout. Use the results to design the rollout.
3.
‘Test and Learn’ through the rollout
By monitoring performance and responses on social media in the early stages, campaigners worked out what was working and adjusted the messaging and materials to increase impact.
In the early stage (first month) of the campaign rollout

See the Advocacy Cases Section 6 for more details and background on the case.

 


 
Action

  • Any testing is better than no testing:  A wide range of material is commonly tested including pitches, top line narratives and messages, stories and protagonists, slogans and hashtags, visuals/memes, video material and website content. We have found it useful to plan testing at two stages in campaign development:

- Concept level – Having identified a target audience/segment of the public and developed a messaging and pitching approach, and even some draft content, it is useful to see if the strategy is working before spending large amounts of money on content production.

- Content level – This is the more traditional understanding of when to test, i.e. once images, video and campaign content have been developed and the aim is to identify which content and messaging works better.

 
But if you can’t be so strategic for whatever reason, it is still important to remember that “any testing is better than no testing”3 , and we outline a range of methods we’ve used specifically for testing for narrative change campaigning in a concise resource. 


 
What you can get wrong

  • Assuming that testing is only for well-funded organisations with high capacity: This is somehow a widely held view, but in fact, there are many cheaper and lower capacity options available to get feedback on ideas before launching a campaign. To make testing more accessible and feasible for all types of CSOs, we have outlined the costs of methods and capacity required for different testing options in our "Message Testing Methods for Narrative Change" resource.

 

<-- Lesson 9  |  Lesson Index -->