Are Perverse Incentives Responsible for the Replication Crisis?

Value of the Information & Strength of the Argument

The lecture provides a valuable critical perspective on a widely accepted narrative in meta-science. Machery’s argument is well-structured: he first establishes the existence of the replication crisis with concrete examples and data, then distinguishes proximate and distal causes, and finally challenges the incentive-based explanation. He uses specific studies (Fanelli 2010, Open Science Collaboration 2015, Begley & Ellis 2012) to support his claims, and he introduces the concept of ‘ghost literatures’ to illustrate the broader impact of replication failures. The argument is persuasive in its call for more rigorous empirical investigation into the causes of QRPs, rather than relying on intuitive assumptions about incentives. However, the talk is an opinion piece, and Machery does not provide a fully developed alternative explanation, leaving the audience with a critical but incomplete picture. The reasoning is sound but could benefit from more direct engagement with counterarguments or empirical evidence on incentives.

Scientific Rigor, Source Quality, Title Accuracy

Machery demonstrates scientific rigor by referencing key meta-scientific studies and clearly distinguishing between proximate and distal causes. He cites specific papers (e.g., Fanelli 2010, Open Science Collaboration 2015, Begley & Ellis 2012) and discusses their findings accurately. The title accurately reflects the content, as the lecture directly addresses the question of whether perverse incentives are responsible for the replication crisis. The talk is well-organized and the argument is coherent. However, as a lecture, it does not provide a systematic review of all relevant literature, and some claims (e.g., the lack of evidence for incentives) are asserted rather than thoroughly demonstrated. The sources cited are credible and relevant, but the talk would benefit from more explicit discussion of the limitations of the evidence.

The title accurately reflects the central question addressed: whether perverse incentives are responsible for the replication crisis. The talk directly engages with this question and argues against the common assumption.

8/10

The talk is delivered by a distinguished professor in philosophy of science, drawing on well-established meta-scientific literature (e.g., Fanelli 2010, Open Science Collaboration 2015, Begley & Ellis 2012). The argument is structured and references key studies, though it remains an opinion piece with selective use of evidence.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction and announcements about upcoming events.
• 05:00Introduction to the concept of 'fake science' and examples.
• 10:00Discussion of the replication crisis and evidence of replication failures.
• 15:00Introduction of proximate and distal causes.
• 20:00Explanation of questionable research practices (QRPs) and their role.
• 25:00Discussion of the usual story: perverse incentives as distal cause.
• 30:00Critique of the incentive narrative and call for more empirical evidence.
• 35:00Introduction of 'ghost literatures' with examples (power pose, willpower).
• 40:00Further discussion on the role of incentives and alternative explanations.
• 45:00Conclusion and Q&A session.

• Fanelli (2010) - Positive results in science — Cited as evidence of high proportion of positive results across scientific fields.
• Open Science Collaboration (2015) - Estimating the reproducibility of psychological science — Cited as a major replication study showing low replication rates in psychology.
• Begley & Ellis (2012) - Drug development: Raise standards for preclinical cancer research — Cited as evidence of low replication rates in oncology studies.
• Stuart Ritchie - Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth — Quoted as a representative claim that perverse incentives are responsible for questionable research practices.

• Fanelli (2010) - Positive results in science — Supports the claim that many scientific fields report an implausibly high proportion of positive results.
• Open Science Collaboration (2015) - Estimating the reproducibility of psychological science — Provides empirical evidence of low replication rates in psychology, consistent with the replication crisis.
• Begley & Ellis (2012) - Drug development: Raise standards for preclinical cancer research — Shows low replication rates in oncology, supporting the idea that replication failures extend beyond psychology.

• Stuart Ritchie - Science Fictions — Ritchie argues that perverse incentives are a major cause of questionable research practices, which Machery challenges.