Countless organizations across the US are working to decrease toxic division by hosting events, leading programs and running campaigns to bridge divides between left & right and other groups of difference. Working alongside the Goals and Measures working group of BMAC in 2021, CHP developed a scalable survey and analysis tool designed to give bridging groups the reins to measure the outcomes they care about most.
The Social Cohesion Impact Measure (“SCIM”) is a composite of three parts:
A set of survey items identified and tested to measure outcomes bridging groups care about,
A pre and post survey tool, using Google Forms, that groups can easily administer to their participants,
An instant analysis tool that shows groups the the impact of each of their interventions.
SCIM’s survey measures
The most critical part of developing SCIM was identifying survey items that target the right outcomes (i.e. ones that bridgers care about) using the right survey questions (i.e. those demonstrating good measurement validity).
To get the outcomes right, we worked alongside BMAC’s Goals & Measures working group to identify the most salient outcomes the bridging field wished to measure. Building off that work and also reviewing the impact statements of bridging organizations, we developed and conducted a survey to home in on the outcomes that matter broadly across bridge building groups. We used the findings of that survey, combined with constructs that scholars use to gauge toxic polarization, to develop the full set of outcomes SCIM could measure.
Outcomes measured by SCIM: Affective polarization, Intergroup empathy, Intellectual humility, Support for pluralist norms, Respect & Understanding, Humanization, Value listening, Perceived morality, Perceived threat, Anger, Strength of group identity
To get the survey questions right, for each outcome we searched for measurement items used by social scientists and reputable survey organizations, selecting questions that scholars had tested for measurement validity or, when those did not exist, were broadly used by researchers
After collecting 39 questions for 13 outcomes, we conducted two sets of surveys with over 1,400 participants to identify the one to two strongest questions for each outcome. (Because SCIM surveys are completed voluntarily, we knew we had to keep them short by minimizing the number of questions for each outcome.) Finally, reviewing the results of those tests, we identified the set of outcomes and survey questions to include in SCIM, using multiple criteria including the chances of observing a change (i.e. sensitivity) and the distinctiveness of each outcome (i.e. discriminant validity).
SCIM’s survey and analysis tool
While getting the survey questions right is essential to making SCIM a valuable tool, they are only as good as an organization’s ability to easily conduct pre and post surveys – and get access to their results. Toward that end, we built SCIM’s survey and data analytics tools using Google Forms and Google Spreadsheets, which give organizations the ability to have full, free 24/7 access to their tools and data. Whereas more sophisticated survey and data analytic tools exist, all would require substantial ongoing technical maintenance, data analytics support, or financial costs. Google Forms and Spreadsheets, meanwhile, are able to meet the survey and data analytics needs of SCIM – and give organizations the reins to their own surveys, data and analysis.
SCIM’s adoption
As of December 2022, SCIM has been adopted and used by over a dozen bridge building groups, including our two pilot organizations The Village Square and BridgeUSA. Over 1400 participants across dozens of events have completed pre and post surveys, allowing us to see the impact these organizations’ work are having on those they come in contact with.To get a hands-on feel for how SCIM works, visit our SCIM Sandbox which shows you how one of our pilot organizations, The Village Square, uses SCIM to measure its impact.
If you are interested in learning about potentially adopting SCIM for your organization, please contact Sofia Politi at sofia@civichealthproject.org, so we can set up an introductory meeting. You can also learn more about how SCIM can support your work here.