Chapter 6 Conclusion

In this project, we explored an open-access dataset that contains data related to misinformation, largely in relation to political affiliation and ideology. This data shows clear signs of proper experimental design: the sample is balanced in terms of political orientation and ideology, observations with a great number of missing values were omitted, and there is symmetry in fact-checking conditions administered to participants. However, the sample was not quite balanced in terms of gender and, besides the absence of non-gender conforming individuals, there were nearly double the number of women as men in the sample. Future work could explore the experimental nature of the data in more depth. Why were more women included than men? Was this intentional? Certainly gender plays a large role in politics and the argument could be made that this biases the data. If they had to include that many more women to get a balanced profile for political affiliation and ideology, you could say that many of the differences found in the study were related directly to different biases between men and women. Therefore, could this study be pushed further and include more male participants?

One of the most striking pieces of evidence found in analysis was the difference between Republicans’ and Democrats’ “actively open-minded thinking (AOT)” especially among male respondents. This AOT scale was originally created in the context of corporate decision making, but has become widespread in recent scholarly work on misinformation and fake news (e.g., Swire et al., 2017; Maertens et al., 2021). Our box plots showed that even the median AOT for democratic men was above the third quartile of Republican men. This means that around 50% of democratic men scored higher in open minded thinking than 75% of Republican men. This confirms speculation that many Republican men are very set in their political thinking. Though we find a similar pattern for women in these plots, it is not of the same scale.

Another remarkable feature of the sample was found with the boxplot for “confirmation bias”, a measure of individuals’ fact-checking disposition. It shows that Democrats and Republicans have similar scores for some quartiles, but not all. The two groups have similar third quartiles, but have a marked difference in medians. The median score for Republicans is similar to the first quartile of Democrats. So 50% of republicans score less than or equal to 25% of Democrats. This shows that Republicans in this sample were less likely to fact check headlines that were already concordant with their ideologies. They were willing to believe them to be true because they align with what they already believed to be true, resulting in confirmation bias. When the same box plots are split among ideologies, the most remarkable feature is the similarity among medians for the all groups (close to 0), except for the two most moderate segments among liberals and conservative. For these two groups, the medians are considerably above the others, indicating individuals who are not particularly politically affiliated were much more inclined to fact check articles when they were unsure they were accurate.

Limitations of our analysis include lack of access to some methodological decisions. For example, although the ideology scale had 7 items, conservative and liberals are asymmetrically distributed along those 7 values (liberals are those who choose 1 to 3; conservatives, 4 to 7). Due to the apparent robustness of the study design, we assume that the researchers had clear motivations for that asymmetrical split, but the dataset documentation does not cover that aspect. Having a thorough understanding of the underlying experimental design is crucial for a clearer analysis of this study. In the future, further analysis of statistically significant differences among groups can be conducted, following clues offered by the plots. For example, the difference in median AOT scores among male Democratic and Republian participants would help strengthen our claims. This can be done in addition to the aforementioned attention to the experimental nature of the dataset.