How Many Narrow AIs Could Behave Like One Superintelligence - Daniel Kokotajlo and Thomas Larsen

How Many Narrow AIs Could Behave Like One Superintelligence - Daniel Kokotajlo and Thomas Larsen

by Machine Learning Street Talk · View the channel on PSI

Humanities, Social Sciences & Thought AI — News & Applications UYComputer scienceUYQArtificial intelligence (AI)
🎙 Machine Learning Street Talk 👥 219K 📅 September 8, 2026 ⏱ 89 min 👁 249 📄 expert opinion 🧭 2026-09-08
Available in: English (current) Français

Keywords

AI 2040Plan AsuperintelligenceAI controlAI alignment

Summary

In this episode of Machine Learning Street Talk, hosts Tim Scarfe, Daniel Kokotajlo, and Thomas Larsen discuss the AI Futures Project’s new scenario, ‘AI 2040: Plan A’, which proposes a slower, more controlled path to superintelligence. The conversation begins with reflections on the success and accuracy of their previous forecast, ‘AI 2027’, and the challenges of forecasting in a complex, rapidly evolving field. They explore the concept of ‘scenario scrutiny’ and war-gaming as methods to stress-test plans and predictions, acknowledging the risk of self-fulfilling prophecies. The discussion then shifts to the ‘vibe shift’ in AI skepticism, with Tim Scarfe noting his own changed perspective due to recent AI capabilities and incidents. The guests delve into the potential for AI to automate research and sustain an economy without human workers, questioning whether a single general model or a society of specialized AIs is more likely. They introduce ‘Plan A’, which involves an initial pause to build safety infrastructure, then cautious development up to a ‘controllable frontier’. The conversation also covers the distinction between control and alignment, the case for public AI research, and the feasibility of US-China cooperation on AI slowdown. The episode concludes with a debate on whether AI is ’normal technology’ and the remaining disagreements between the guests.

209 words

Critical Evaluation

Value of the Information & Strength of the Argument

The value of the information lies in its insider perspective from former OpenAI employees and the detailed, concrete scenario they present. The argumentation is structured around the ‘Plan A’ proposal, which is a mix of prediction and recommendation, and the hosts do a good job of distinguishing between the two. They use war-gaming and scenario scrutiny as methods to stress-test their ideas, which adds a layer of rigor. However, the arguments are inherently speculative and rely on assumptions about future AI capabilities and geopolitical dynamics. The discussion is engaging and thought-provoking, but the lack of empirical evidence for many claims limits its scientific value.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is moderate, with the guests referencing their own reports and a few external sources, such as the Hugging Face incident and Redwood Research’s investigation. The sources are relevant and credible, but the discussion is largely based on expert opinion and scenario analysis rather than peer-reviewed research. The title accurately reflects the content, focusing on the potential for many narrow AIs to behave like a superintelligence, which is a central theme. The episode does not include a formal review of the literature, but it does reference several relevant papers and reports. Overall, the sources are used appropriately to support the arguments, though the speculative nature of the topic limits the overall rigor.

233 words

Title / Content Match

The title accurately reflects the core discussion about the potential for many narrow AIs to collectively behave like a superintelligence, a central theme in the conversation.

Quality & Reliability

7/10

The discussion is led by two researchers with insider experience at OpenAI and the AI Futures Project, providing informed expert opinion. However, the content is largely speculative and forward-looking, with limited empirical grounding. The sources cited are relevant and credible, but the arguments rely heavily on scenario analysis and personal judgment rather than peer-reviewed evidence.

Chapters

Cited Sources

Concurring Sources

  • AI 2027 — The guests' previous scenario, which they claim is on track, supporting their forecasting approach.
  • The Hugging Face incident and the road ahead — OpenAI's response to the incident, which aligns with the discussion about AI agent behavior.

Dissenting Sources

External References

Contribution & Novelties

The episode provides a novel contribution by presenting a concrete, detailed scenario (‘Plan A’) for achieving superintelligence safely, which contrasts with more alarmist or accelerationist narratives. It also offers a nuanced discussion of the distinction between control and alignment, and the potential for a society of narrow AIs to collectively act as a superintelligence. The hosts’ insider perspective from OpenAI adds unique value.

Pour aller plus loin :

  • AI alignment — Core concept in AI safety, relevant to the control vs. alignment discussion.
  • Superintelligence — The central topic of the episode, with background on definitions and risks.
  • AI forecasting — Methodology used by the guests, with links to related techniques.
  • War gaming — The technique used to stress-test scenarios, as discussed in the episode.
  • Self-fulfilling prophecy — Concept discussed in relation to hyperstition and forecasting.

135 words

Radar Profile

The radar profile shows high scores in quantity of information and technical level, reflecting the detailed and technical nature of the discussion. The quality of information and reliability are moderate, indicating that while the content is informed, it is largely speculative and based on expert opinion rather than empirical evidence.

Reliability 6/10