Chapter 3. Overview of the #KommMit pilot project

This chapter provides a short overview of the essentials of the #KommMit narrative change pilot project which is the foundation of this toolbox. Through this overview, you will get an introduction to:

• the targets and background of the pilot;
• the key features of the pilot approach;
• the results of the pilot.

We also include a checklist for you to start considering your experience with testing and evaluation of campaigning work, and more specifically to reflect on your thoughts on evaluating middle-orientated public communications work.

If you wish to see quick overview of the pilot in an infographic, click the link.

3.1 Background to the pilot

The #KommMit pilot project was developed by five members1 of the German nationwide CLAIM Allianz based on ICPA’s value-driven, narrative change approach (explained in Chapter 2). It was developed and tested within the RESET project, with the social media part of the pilot led by a coalition of three CLAIM members2 in April-May 2023. The aim of the pilot project was:

The motivations and ambitions of the coalition members for the #KommMit narrative change pilot were borne from increasing concern at the dominance and mainstreaming of anti-Muslim narratives in German society, which were also being used to serve a broader populist agenda, creating divisions within society3 and leading to an increase in attacks and hate speech on the Muslim community4 . See Chapter 1 for more on the problems and consequences that motivated both CLAIM and ICPA to invest in narrative change and strategic communications approaches as a lever to counter the dominance of anti-Muslim narratives.
 
Hence, starting from the title of the pilot project, #KommMit invites the target audience to get to know stories of everyday lives of Muslims living in Germany - which represent the experience of the vast majority of Muslims - as a means to reflect on and challenge the prevalent negative and dramatic stories peddled by the far right and much of the mainstream media. 
 
In addition to this topline aim, given the fact that this was the first time for the five organisations experimenting with a narrative change approach, the coalition wanted to provide the evidence that the approach worked and that the #KommMitpilot can be relatively easy and cheap to replicate. The long-term strategic communications strategy driving the scaling up ambitions of the #KommMit project is that repeated efforts to share and disseminate such value-based stories will build presence in the public discourse, socialise these new narratives in communities and wider networks and, eventually, build the broad support needed to shift norms and eventually policy. For more on this background and rationale, see Chapter 1 and see Chapter 2 for more on the narrative change advocacy approach.

3.2 Key features of the #KommMit pilot
 

‘The Established’ as the target group and unifying values

After a process of digging into opinion polling and segmentation research in Germany and weighing up the pros and cons, CLAIM members chose to target ‘The Established’5 segment of the German movable middle for the #KommMit pilot. The following table provides a profile of the demographics and attitudes of this middle segment relevant for the focus of the pilot project: 

 Figure 5: Profile of the chosen target middle segment (ICPA 2021)

 
CLAIM members identified the following set of unifying value appeals, shared between the 15 CLAIM members contributing to the strategy development process and the target middle segment:

• Interdependence and solidarity;
• Responsibility, contribution, participation;
• Stability and security; 
• Intergenerational futures.

These value appeals were initially tested in focus groups and verified in a national poll with middle groups, where they showed good resonance with the target segment. See Chapter 4 for much more on the value appeal, how and why ‘The Established’ were chosen, as well as the narrative development process. 

 
Three tradespeople as protagonists

The #KommMit pilot tested stories and social media content for three protagonists who were all tradespeople: Ayoub (baker), Yusuf (butchery business) and ‘Murat’ (carpenter). Tradespeople were chosen for their role as trusted, valued, down to earth, backbone community members and this tested well with middle audiences. The protagonists agreed to work with the pilot at different levels with Ayoub happy to feature in videos and share his story on social media; Yusuf consenting to share his story on the project website; and ‘Murat’ only agreeing to use of his stories (and not his identity) for testing purposes. See more on the decision to focus on tradespeople in Chapter 4 and working with protagonists in Chapter 5.

 
Social media pilot logic

The social media pilot was the main test of the #KommMit project, run as a series of paid ads on Facebook and Instagram6 between April 27 - May 31, 2023. The logic of the pilot, as outlined in the diagram below, was to reach the target audience with ads that hosted five videos about the protagonist, Ayoub, measure how they responded to the messages and material, and then see how many people were motivated to visit the project website to find out more about Ayoub, the other protagonist (Yusuf), and the overall project. See more about the social media design approach in Chapter 6.
 

Figure 6: The logic of the #KommMit social media pilot

 
3.3 The evaluation approach & key results

The pilot sought to test whether value-based messaging and storytelling could shift attitudes, reduce anxiety and open a constructive conversation around the issue of Muslims in Germany. As response-level testing was the focus of the pilot, the team did not invest in building partnerships with supporters, influencers and media to develop earned content or build their public support, i.e. on uptake targets. Given this scope, the evaluation focused on the targets of reach and response from ICPA’s evaluation framework for narrative change campaigns, as outlined in the following diagram: 
 

Figure 7: ICPA’s levels of evaluation for measuring narrative change effectiveness
 

The results achieved in the #KommMit pilot are detailed in the table below: 

Table 2: The detailed reach and response level results of the #KommMit pilot

 
The following are the six key results from the #KommMit pilot:

 
1. Value-based messaging with authentic storytelling positively shifted the attitudes of the target middle group on Muslims & integration.

Figure 8: Average attitude shift from three randomised controlled trials versus benchmark

 
Figure 8 above shows that the main aim of the pilot was proven with value-driven storytelling proving to be an approach that works to positively and significantly shift the attitudes of ‘The Established’ target segment. This result was generated from three separate tests called ‘Randomised Controlled Trials’ conducted with 4,400 people, i.e. tests on #KommMit videos and storyboards for the three protagonists in which a test group sees the campaign material and answers attitude questions. In parallel, a control group sees some neutral video on a random topic and answers the same questions. The results are then compared10 . A 6% shift may not sound like a lot, but the significance of even the benchmark of 2% can be illustrated as the difference between a progressive candidate in a local community getting elected or a proposal in a school board getting passed or not.
 

2. #KommMit reached twice as many of ‘The Established’ as expected (approximately 200,000), and at an affordable cost.

The challenge with reach is to develop content and copy that works within the limitations of the social media algorithm and also gets good engagement from the target audience. Investing in building a targeted audience profile was key, as was producing content that fit the social media mindset, i.e. short punchy videos that hold attention. See chapter 6 for more on audience profile building and video making. Figure 5 below shows that this strategy and investment paid off, with 200,000 ‘Established’ reached in the five-week period running the social media ads.

Figure 8: Percentage of target segment ‘The Established’ reached versus benchmark

 
Further, there is always a cost aspect to achieving reach and #KommMit reached the target audience at half the market rates, i.e. cost per thousand people reached was €1.44 versus a German market average of €2.8011 . These reach results show that it is possible and affordable to put more positive messaging about Muslims in the social media feeds of those groups who don’t often see such material, and who are indeed movable.

 
3. The campaign material triggered a high level of engagement, with over 80% positive responses and no outbursts of strong attacks or hate speech from the target middle group.

Getting the balance of resonance and dissonance right in the messaging and content is a challenge when dealing with divisive issues, and the #KommMit pilot succeeded in getting a very high level of engagement, as well as in getting the emotional balance right. This is evident from the fact that the vast majority of comments on both social media and in the Randomised Control Trial test had a positive focus or asked neutral questions and there were only a handful of hate speech-based responses from the target middle segment. This result is important as its shows that it is possible to have constructive, social cohesion-based conversations with middle groups without the aggression and hate speech too often experienced by activists in response to their social media content and campaigns. 
 

4. The values of ‘participation through professional lives’ and ‘care for intergenerational futures’ show particular promise in shifting attitudes.

The content that continually achieved the best results with ‘The Established’ target audience were stories of how the tradespeople protagonists enthusiastically engage their professions towards shared community goals and challenges, such as renovation work, and efforts to preserve traditions and key services, such as local bakeries. In addition, stories that focus on protagonists’ own learning and apprenticeship, as well as teaching the next generations, proved to be some of the best performing content (For example, see the Ayoub Apprentice & Mentorship video). See more on the messages in Chapter 4 and stories in Chapter 5.

 
5.     The #KommMit messaging and storytelling content is a demographic unifier.

Positive messaging on divisive societal issues such as migration often divides support, with multiple tests showing target groups from lower income and education brackets commonly less supportive. However, the results in Table 3 illustrate this was not the case with #KommMit. In fact, the content produced similar positive shifts across all income brackets and education levels. 
 

Table 3: Percentage attitude shift by income level in one randomised controlled trial.
 

Therefore, this type of messaging shows promise as a basis to build broad-scale public support using a campaigning approach. This result has extra significance in this time of inflation and economic downturn, as one would assume that such recent pressures would tend to harden attitudes for those with lower incomes, as the far right often scapegoats minority groups as the main cause of economic problems.
 

 
6. The material worked to positively shift the attitudes of the older people.

In the target ‘Established’ segment, around 85% are above 35 and over 50% are over 55, and it is interesting to see in Table 4 below that the material worked particularly well for these age brackets. Interestingly, for both of the age groups, the results tip the balance to make more positive attitudes on migration and Muslims into a majority, e.g. for those in the 55+ bracket, the control group score is 48% positive and the test group score is 55% positive (+ 7,1). Importantly, even for the younger age brackets within the segment, the material did not backfire. This data is particularly interesting as there is commonly held idea that it is extremely hard, if not impossible to shift the attitudes of these older age groups. This result proves the opposite, and it is welcome news that there is more potential than commonly thought.
 

Table 4: Percentage attitude shift by age group for in one randomised controlled trial.
 
 

3.4 Evaluation of your advocacy initiatives

<-- Chapter 2 | Chapter 4-1 --> 

• 1Arbeitsgemeinschaft der Evangelischen Jugend in Deutschland e. V. (aej) & Netzwerk gegen Islamfeindlichkeit und Rassismus Leipzig e.V. & Muslimische Jugendwerk & Multikulturelles Forum e.V. & Mosaik e.V.
• 2Arbeitsgemeinschaft der Evangelischen Jugend in Deutschland e. V. (aej) & Netzwerk gegen Islamfeindlichkeit und Rassismus Leipzig e.V. & Muslimische Jugendwerk
• 3Friedrich Ebert Stiftung (2019) Pragmatism, not Dogmatism: What Germans think about Migration ; More in Common (2017) Attitudes Towards National Identity, Immigration, and Refugees in Germany; Bakamo Social & Friedrich Ebert Stiftung (2019) Migration Narratives in Europe - A Social Media Intelligence Study: Country report for the Federal Republic of Germany.
• 4INSSAN (2020) Report on anti-Muslim prejudiced incidents.
• 5See a   short description of the Established und other segments in this debate. ICPA (2021)
• 6The platforms used most by ‘The Established’ as identified by More in Common (2022).
• 7This metric is called the Cost per Thousand (CPM) on Facebook Ad Manager. The market average of €2.80 was taken from this source in April 2023 .
• 8This metric is called the ThruPlay rate on Facebook Ad Manager. The ThruPLay rate of 5 to 10 cents was taken from this source in April 2023 .
• 9The benchmark comes from Swayable , who ran one of the randomised controlled trials for the #KommMit pilot and have run thousands of similar tests.
• 10See ICPA’s Testing Methods for Narrative Change resource ffor more detail on randomised controlled trials and other testing methods.
• 11The market average of €2.80 was taken from this source in April 2023 .