Ceren Budak: The Prosocial Ranking Challenge

Value of the Information & Strength of the Argument

The lecture provides valuable insights into the design and execution of a large-scale field experiment on social media. The argumentation is solid, grounded in the need for causal evidence and the challenges of conducting such research. The speaker clearly explains the methodology, including the technical infrastructure, recruitment difficulties, and the importance of pre-registration. The results are presented with appropriate caution, acknowledging the small effect sizes and the need for further research. The discussion of the interventions’ mechanisms adds depth, and the finding that prosocial algorithms do not harm engagement is a significant contribution to the debate.

Scientific Rigor, Source Quality, Title Accuracy

The lecture demonstrates scientific rigor through its reliance on pre-registration, randomized assignment, and validated measures. The speaker references a meta-analysis on social media effects and mentions the use of the Perspective API, a well-known tool. The title accurately reflects the content, which is a detailed account of the Prosocial Ranking Challenge. The talk is part of a series by the Summer Institute in Computational Social Science, adding to its credibility. However, as a lecture, it does not provide full citations for all claims, and the audience is not given access to the underlying data or code.

The title accurately reflects the content, which focuses on the Prosocial Ranking Challenge and its findings.

8/10

The lecture presents a well-structured overview of a large-scale, pre-registered field experiment, with clear methodology and transparent reporting of results. The speaker is a recognized researcher in computational social science, and the content is grounded in peer-reviewed literature. However, as a lecture, it lacks the full detail of a published paper, and some claims are presented without complete statistical context.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction to the Prosocial Ranking Challenge and its goals.
• 05:00Discussion of the challenges of running large-scale experiments on social media.
• 10:00Overview of the system design: Chrome extension and ranking server.
• 15:00Description of the five interventions tested.
• 20:00Explanation of outcome measures: affective polarization, user experience, and engagement.
• 25:00Presentation of main results: reduction in polarization.
• 30:00Analysis of how the intervention worked: out-party hostility and in-party warmth.
• 35:00Discussion of user engagement and implications for platform incentives.
• 40:00Conclusion and future directions.

• Tech for Open Minds series — The lecture is part of this series, providing context for the talk.

• Meta-analysis of social media effects — Referenced in the lecture as evidence of negative associations between social media and polarization.