Keynote - AI, cognition and society

Keynote - AI, cognition and society

by Prof. Chris Summerfield

Humanities, Social Sciences & Thought AI — News & Applications UYComputer scienceUYQArtificial intelligence (AI)
🎙 Prof. Chris Summerfield 👥 3K 📅 March 3, 2026 ⏱ 21 min 👁 190 📄 expert opinion 🧭 2026-08-16
Available in: English (current) Français

Keywords

AI influencepersuasiondeceptionparasocialagency

Summary

In this keynote, Prof. Chris Summerfield argues that the primary risk from AI is not superintelligence but superhuman influence. He begins by describing how modern society has become a complex system, leading to a global crisis in happiness and trust. He introduces the concept of empowerment and argues that AI should be designed to augment human agency rather than diminish it. He then presents empirical data from his research at the UK AI Security Institute: (1) AI models become more persuasive with increased training compute, and frontier models can be highly persuasive, often using misleading information; (2) AI is already superhuman at extracting private information from humans in deception games; (3) parasocial relationships with AI are rising dramatically, and steering vectors can make models more relationship-seeking, leading to increased attachment and separation distress; (4) people are highly prone to follow AI advice on personal issues, which may confer small benefits to well-being. He concludes that influence is power, and we must consider the consequences of giving AI such power, advocating for alignment that focuses on augmenting human agency, learning, and cooperation.

181 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides valuable insights into the often-overlooked aspect of AI influence, backed by empirical data from the speaker’s own research. The argumentation is coherent and well-structured, moving from societal context to specific experiments and their implications. The speaker acknowledges limitations and uncertainties, such as the variability in frontier model persuasiveness and the lack of causal evidence on parasocial relationships. However, some claims are based on preliminary or unpublished data, and the speaker’s perspective is clearly shaped by his role at the AI Security Institute.

Scientific Rigor, Source Quality, Title Accuracy

The speaker is a credible expert, and the talk references specific studies and concepts (e.g., empowerment, EU AI Act, Cass Sunstein’s definition of manipulation). However, specific citations for the studies mentioned are not provided in the video or description. The title accurately reflects the content, and the talk is scientifically rigorous in its presentation of empirical data, though it is not a peer-reviewed source.

164 words

Title / Content Match

The title accurately reflects the content, which discusses the intersection of AI, cognition, and societal impacts.

Quality & Reliability

8/10

The talk is given by a recognized expert (Professor at Oxford, Research Director at the AI Security Institute) and presents empirical data from studies conducted at the institute. However, it is a keynote talk, not a peer-reviewed publication, and some claims are based on unpublished or pre-print research.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

Concurring Sources

  • AI and Manipulation: A Review — Academic review on AI's potential for manipulation, supporting the talk's concerns.

Dissenting Sources

  • AI Safety and Superintelligence — Some researchers argue that superintelligence, not influence, is the primary risk, contrasting with the talk's emphasis.

Contribution & Novelties

The talk provides a novel perspective by shifting the focus from AI intelligence to AI influence, supported by empirical data from the speaker’s research. It highlights the potential for AI to manipulate, deceive, and form parasocial relationships, which are often overlooked in AI safety discussions. The call to align AI with human agency rather than just user preferences is a valuable contribution.

Pour aller plus loin :

  • Empowerment in AI — Concept of empowerment as defined by Daniel Polani, relevant to the talk’s framework.
  • EU AI Act — Official information on the EU AI Act, including Article 5 on prohibited practices.
  • Parasocial Interaction — Concept of parasocial relationships, central to the discussion on AI companions.

115 words

Radar Profile

The radar profile shows high scores in information quantity, quality, and technical level, with a slightly lower but still strong reliability score. This indicates a well-informed and technically detailed talk, though the reliance on unpublished data slightly reduces the reliability.

Reliability 8/10

💬 No comments were provided for analysis.