3.1 Test & adapt campaign elements

The third step of the narrative change campaign planning process – Prepare for responses & engagement - consists of three elements that give you the chance to test the campaign elements you prepared in the previous step and also guide you through preparing the team for the campaign roll out in the next. On this page, we focus on the first element – Test and adapt campaign elements.
 

PREPARE FOR RESPONSES & ENGAGEMENT
Elements
  1. Test & adapt campaign elements
  2. Develop talking points
  3. Prepare the team

 
Although the research base is a helpful starting point to gain an in-depth understanding of your target segment(s), you will also have made many assumptions regarding the audience in putting together your strategy and in the development of campaign elements. In a narrative change campaign, the assumption is that the campaign elements will be engaging and resonant at the right level and also that you will have introduced a ‘twist’ or challenge that would also lead these middle segment audiences to reconsider certain positions or stereotypes they hold. So, at this point you really need to test these assumptions before going public with the campaign. Otherwise, while well intended and carefully planned, it is still a big risk to launch the campaign without testing the actual resonance with the intended audience.
 
Two main tasks should be completed by your campaign team in this element, detailed in the bullets below.

 
Research to become informed about possible options for testing campaign messages. 

 There are many methods for testing messages and campaign elements that have different cost implications, and it’s worthwhile researching these to see what best fits both your needs and your budget. In the best-case scenario, you want to run a test that: 

  • gives you feedback from the specific target segment(s) you intend to engage;
  • allows you to test and get feedback on as many of the campaign elements as possible;
  • allows you to test options for various elements of the campaign, i.e. often using an A/B testing approach1 ;
  • allows you to see if there is some shifting of positions on the issue over the time of the test.

 
The most common methods used by campaigners are:
 

  • Focus groups – This entails putting together a series of groups from your target audience and have a moderator talk them through and give feedback on the campaign approach and elements. This is normally quite an expensive option, but provides very rich qualitative data on different levels and you normally have the chance to see opinion movement over time in the discussion. An added value of this method is that the campaign team can also play an active role as observers of focus groups and benefit from witnessing the emotional tone, as well as the discussion itself.
     
  • Social media tests – Many campaigners conduct A/B tests on Facebook, where you can test different messaging options, e.g. two variations of a mix of campaign messages, pictures and stories in a post that is fully targeting the segment you want, using the very high level of segmentation detail available through the Facebook advertising platform2 . This is a relatively cheap and effective method, but the feedback and data generated is more wide and thin in comparison to the deep nature of a focus group approach.
     
  • Email list test – This is a cheap option and could work if you have a diverse and large email list that covers your target segment. Those running the marriage equality campaign in Ireland in 2015 reported that this was a very effective method for them3 . You need to consider that while it is cost effective, there is a danger that you are only preaching to the converted, i.e. those who already signed up for your newsletter.
     
  • Online surveys – As with a large-scale email list, this method may be low cost option to consider. In many countries, there is a possibility to do inexpensive polling by adding questions to a very varied list of polling questions from a number of different customers. This could be a reasonable option with good reach to the middle. This approach provides good quantitative data, but is based on a random selection of the public rather than deliberately engaging your target segments and there is a risk of responses being influenced by the random questions that come before yours in such a low-cost polling approach. 

 
Equinet (a network of European state equality bodies) have helpfully shared other approaches and ways to do message testing on a tight budget4 .
 


Invest in testing assumptions and campaign elements based on your available budget.

As detailed above, there are many methods available for testing campaigns and all have different cost implications. A takeaway lesson from the experience of the campaigning organisation, Equinet is that “any testing is better than no testing”5 . To illustrate the value of such testing, Equinet reported on an environmental campaign that was designed to be humorous, but which the target audiences actually found offensive in tests. As a result, they had to go back to the drawing board and take the campaign in a different direction6 . Our partner, British Future also shared examples during an event in our Narrative Change Lab of messages they were sure would be effective with the middle, until they actually tested the messages in focus groups and found that they worked much better with the base compared to the intended audience of the middle! So, they responded by developing new messages for the middle, and keeping the original messages for the base. Your campaign elements need to be tested in a controlled way to ensure validity and from the feedback, you can (hopefully) make final adaptations to the approach and tweak campaign material. 
 
The following case is a further illustration of the value of message testing for the campaign development process for German campaigning teams we are supporting in their first engagement with the middle as the defined target audience.
 
 

CASE 3 – Narrative Change Lab – ICPA – Germany 
 
As part of our capacity building and campaign development programme in which we’re supporting the developing of two reframing campaigns, we ran a message testing event using a focus group approach. The test adopted the following approach
  • The focus groups were made up of the Economic Pragmatist and Moral Sceptics (Humanitarian Sceptics) segments of the middle in Germany7 .
  • We had four separate groups of eight people to ensure validity of the test, providing a sound evidence base on which to make decisions about the campaigns
  • A professional polling company conducted the screening to select the participants and moderated the groups
  • We tested two different messaging options for one of the campaigns, i.e. one a focus on traditional family values and the other celebrating the diversity of a city.
  • Campaign teams and our broader Lab participants and partners had the chance to observe all eight hours of the focus groups.

The lessons and learnings from the message testing experience were multi-faceted: Firstly, we got valuable feedback on campaign messages and elements that worked and didn’t work and the campaign teams were able to make adaptations on this basis. Secondly, the campaigners greatly benefitted from having the chance to observe the focus groups and this proved to be an important element in preparing to engage the middle audiences, i.e. literally seeing people respond to their campaign is the ultimate way to begin to make preparations for upcoming campaign discussions. Interestingly, the discussion became personalised as we took the middle segments off the research page and into real life, and now the campaign teams speak about specific people from the focus groups (e.g. Christian, Udo, Corinna) rather than abstract segments. They also became aware of the expectations and prejudices they brought into the focus groups about how the middle would respond to the campaign material – this was often debunked in the focus group discussions. 

 

PLANNING CHECKLIST
Step 3.1 Test for responses and adapt
 
Pre-testing:
  • Which campaign elements and/or approaches do you need to test?
  • What testing options are available and which are most suitable for you and your budget?
Post-testing:
  • How did the test go? What were the responses to the campaign elements?
  • What did you learn about your target segments?
  • How will you adapt your campaign after the feedback?

 

<< 3.0 - 3.2 >>