We Support Insight Seekers

Civic Health Project is deeply committed to evaluating – and helping our academic, practitioner, and philanthropic partners to evaluate – the return on investment (ROI) of bridge-building and social cohesion interventions. To accelerate learnings on behalf of ourselves and other stakeholders, we’ve invested heavily in research, programs, and instrumentation designed to take the guesswork out of depolarization, social cohesion, and bridge-building

Grantee Spotlight

Strengthening Democracy Challenge

The Strengthening Democracy Challenge brought academics, practitioners, and industry experts together in a collective effort to identify effective interventions to improve Americans’ commitment to democratic principles of political engagement. With contributors’ help, the Challenge team identified promising, short interventions and scientifically evaluated them in one of the largest randomized experiments in the social sciences. While some existing interventions are in-person, time-intensive, and involve repeated exposure, the Challenge team focused on short, scalable interventions that have the potential to reach millions of people.

Ways to strengthen democracy, as determined by Stanford-led ‘mega study’

Published in Stanford News by Melissa De Witte

Statue of Liberty

Inside a ‘mega-study’ on election denial, polarization, and violence—and how to stop it

Published in Fast Company by Talib Visram

All Insight Seeker Grantees

Civic Health Project is proud to support the research endeavors of these capable and highly cited scholars, across multiple academic disciplines. Our work has been deeply informed and guided by their insights into the forces that deepen – and repair – political and social divisions in America.

Dr. David Broockman

Dr. David Broockman is an Associate Professor of Political Science at University of California, Berkeley.

Dr. Stewart Coles

Dr. Stewart Coles is an Assistant Professor in the Department of Communication at the University of Illinois at Urbana-Champaign.

Dr. Julia Minson

Dr. Julia Minson is an Associate Professor of Public Policy at the Harvard Kennedy School of Government.

Dr. Nick Rogers

Dr. Nick Rogers is a Visiting Assistant Professor of Sociology at the University of Pittsburgh.

Dr. Linda Tropp

Dr. Linda Tropp is Professor of Social Psychology in the Department of Psychological and Brain Sciences and Faculty Associate in the School of Public Policy at the University of Massachusetts Amherst.

Dr. Nathan Walter

Dr. Nathan Walter is an Assistant Professor in the Department of Communication Studies at Northwestern University.

Dr. Robb Willer

Dr. Robb Willer is a Professor of Sociology, Psychology and Organizational Behavior and the Director of the Polarization and Social Change Lab at Stanford University.

Strategic Projects

PQ: Measuring the (De)Polarizing Quotient of Interventions and Media

We live in a polarized time with multiple forces – in media and politics – amping up division between left and right. At the same time hundreds of organizations across the US are pushing back against toxic polarization by working to bridge divides in their communities, workplace and in civic institutions.

How can we measure what media content and which political actors are having the most toxic effect on civic society? How do civic groups know their work is having the depolarizing impact they intend? And is it possible to compare across bridging interventions and media content, including entertainment media, to see which forces are most effectively turning the dial up – or down – on toxic polarization?

While social scientists have tools to measure polarization and many bridge-building groups evaluate and report on the impact of their work, CHP recognized a need for better access to measurement tools among organizations with limited internal evaluation capacity, as well as for efficient methods to measure the impact of media on audiences and civic society. We also saw value in using standardized measurement items across tools to make it possible to identify which levers for change have the greatest impact.

In 2021 CHP started work to meet those needs, first developing an open-access tool in collaboration with bridge building organizations to measure the impact of their depolarizing interventions. That tool has now been adopted by a coalition of bridge-builders under the name “Social Cohesion Impact Measure” – or “SCIM”. CHP also began work to measure the depolarizing effect of entertainment media on audiences, piloting early stage studies to support the work of Bridging Entertainment Labs. We are now at the outset of developing tools to identify, at scale, online content that has a toxic – or positive – effect on civic society.

SCIM: Developing a PQ for the bridge building community

Countless organizations across the US are working to decrease toxic division by hosting events, leading programs and running campaigns to bridge divides between left & right and other groups of difference. Working alongside the Goals and Measures working group of BMAC in 2021, CHP developed a scalable survey and analysis tool designed to give bridging groups the reins to measure the outcomes they care about most.

The Social Cohesion Impact Measure (“SCIM”) is a composite of three parts:

A set of survey items identified and tested to measure outcomes bridging groups care about,
A pre and post survey tool, using Google Forms, that groups can easily administer to their participants,
An instant analysis tool that shows groups the the impact of each of their interventions.

SCIM’s survey measures

The most critical part of developing SCIM was identifying survey items that target the right outcomes (i.e. ones that bridgers care about) using the right survey questions (i.e. those demonstrating good measurement validity).

To get the outcomes right, we worked alongside BMAC’s Goals & Measures working group to identify the most salient outcomes the bridging field wished to measure. Building off that work and also reviewing the impact statements of bridging organizations, we developed and conducted a survey to home in on the outcomes that matter broadly across bridge building groups. We used the findings of that survey, combined with constructs that scholars use to gauge toxic polarization, to develop the full set of outcomes SCIM could measure.

Outcomes measured by SCIM: Affective polarization, Intergroup empathy, Intellectual humility, Support for pluralist norms, Respect & Understanding, Humanization, Value listening, Perceived morality, Perceived threat, Anger, Strength of group identity

To get the survey questions right, for each outcome we searched for measurement items used by social scientists and reputable survey organizations, selecting questions that scholars had tested for measurement validity or, when those did not exist, were broadly used by researchers

After collecting 39 questions for 13 outcomes, we conducted two sets of surveys with over 1,400 participants to identify the one to two strongest questions for each outcome. (Because SCIM surveys are completed voluntarily, we knew we had to keep them short by minimizing the number of questions for each outcome.) Finally, reviewing the results of those tests, we identified the set of outcomes and survey questions to include in SCIM, using multiple criteria including the chances of observing a change (i.e. sensitivity) and the distinctiveness of each outcome (i.e. discriminant validity).

SCIM’s survey and analysis tool

While getting the survey questions right is essential to making SCIM a valuable tool, they are only as good as an organization’s ability to easily conduct pre and post surveys – and get access to their results. Toward that end, we built SCIM’s survey and data analytics tools using Google Forms and Google Spreadsheets, which give organizations the ability to have full, free 24/7 access to their tools and data. Whereas more sophisticated survey and data analytic tools exist, all would require substantial ongoing technical maintenance, data analytics support, or financial costs. Google Forms and Spreadsheets, meanwhile, are able to meet the survey and data analytics needs of SCIM – and give organizations the reins to their own surveys, data and analysis.

SCIM’s adoption

As of December 2022, SCIM has been adopted and used by over a dozen bridge building groups, including our two pilot organizations The Village Square and BridgeUSA. Over 1400 participants across dozens of events have completed pre and post surveys, allowing us to see the impact these organizations’ work are having on those they come in contact with.To get a hands-on feel for how SCIM works, visit our SCIM Sandbox which shows you how one of our pilot organizations, The Village Square, uses SCIM to measure its impact.

If you are interested in learning about potentially adopting SCIM for your organization, please contact Sofia Politi at sofia@civichealthproject.org, so we can set up an introductory meeting. You can also learn more about how SCIM can support your work here.

Creating and Testing Classifiers for Civic Health (CaTCCH)

CHP is not the first to see the value in identifying toxic content at scale: social media platforms and civic organizations have developed numerous classifiers to identify, counter, and minimize content that is harmful to its users and that poses an immediate societal threat.

Creating and Testing Classifiers for Civic Health (CaTCCH) aims to build upon these efforts by supporting the development of classifiers which identify content that contributes to civic harm and health. Working with civic groups and technology teams, we aim to provide a public tool that can be adopted by platforms and civic groups to monitor and counter content that is demonstrated to have an impact on civic society.

Towards this effort, in 2022 CHP conducted interviews with stakeholders in civic groups across the political spectrum to define more precisely what ‘civic harm’ a classifier should be built to ward against. Those conversations identified two harms which groups broadly agreed were a danger to civic society: dehumanization and partisan violence. In 2023, CHP will work with social scientists and tech groups to develop a system to iteratively test classifiers to see which best identify content that leads to dehumanization and partisan violence. That system will have two central components: a platform to run iterative field experiments and a set of survey measures that predict civic harm.

This project will result in a refined and experimentally validated content classifier, research linking classified content to civic harm, and publicly available tools for content monitoring and future classifier improvement. Read here for more details on CaTCCH.

SCIM for Entertainment Media

Contact between partisans – face-to-face meetings like the ones conducted by bridge building groups discussed above – can have a powerful effect in reducing toxic polarization. But research tells us that “vicarious” contact with counter-partisans – through video and narrative storytelling – can also increase empathy and positive feelings towards political counterparts. CHP’s sister organization, Bridging Entertainment Labs, is built on just that theory and is working to promote bridging narrative content in the entertainment industry.

To support BEL’s work and provide evidence that entertainment content can, indeed, reduce toxic polarization and promote social cohesion, CHP conducted an internal study, using a 2017 episode of Black-ish, “Lemons”, as a test case. Black-ish’s producers were intentional in the production of “Lemons”, filmed after the contentious 2016 election, to create a narrative that promoted empathy across partisan divides. We wanted to see if they had the impact they intended.

In our study we recruited 123 HULU subscribers to watch a single sitcom episode. All subjects, who either identified as Democrats or Republicans, first completed a version of our SCIM survey including measures for affective polarization and intergroup empathy. Days later we invited them back to watch a sitcom, and randomly assigned each to watch either Lemons or one of six other sitcom episodes from early 2017 (with similar ratings on IMDB) on HULU. We used attention checks and monetary incentives to encourage participants to carefully watch their sitcom. After viewing, they answered the same set of survey questions from days before – and we compared how much subjects’ attitudes towards partisans changed depending on whether they watched Lemons or another sitcom episode.

We found – as Black-ish’s producers intended – that subjects who watched Lemons had statistically significantly more empathy toward counter partisans than those who watched another sitcom episode. Although not as strong, we also saw that Lemons viewers had more positive attitudes towards counter partisans across all other outcomes.

One thing to point out is that the effect was strongest among Democrats, and indeed we could not see a statistically significant effect among Republicans alone (not that such an effect does not exist, only that we did not observe one with this small sample). This may not be surprising however, since the protagonist who was advocating for empathy in the episode was a Democrat (the main character Dre) and research tells us that messages are more effective when they come from messengers one identifies with.

Given this is one study, as social scientists we should be cautious about overstating the potential of entertainment media to bridge divides; yet we find these results promising. In particular, we are struck by the fact that the Black-ish producers’ intent was to promote empathy – and that, indeed, is the dimension they moved their audiences most. As always, more research needs to be done, but this initial study makes us hopeful for the potential of media to bridge divides and build social cohesion.

Civic Health Logo