Keynote Lecture: Ellie Pavlick - CCN 2025

Value of the Information & Strength of the Argument

The talk provides valuable insights into the empirical study of compositionality in neural networks, moving beyond theoretical debates. Pavlick presents concrete evidence from her lab’s research, including visual odd-one-out tasks and circuit analysis in LLMs, to support her claims. The argumentation is solid, with clear examples and a logical progression from structural to functional to context-sensitive aspects. She acknowledges limitations and ongoing work, which adds credibility. The talk is well-suited for an expert audience, offering a nuanced perspective on a central question in cognitive science and AI.

Scientific Rigor, Source Quality, Title Accuracy

Pavlick references several studies, including work by Wang et al. on circuits in language models, and her own lab’s research. She does not provide explicit citations during the talk, but the description likely contains links to relevant papers. The title accurately reflects the content, and the talk is well-structured. The presentation is scientifically rigorous, with careful explanations of methods and findings. The adéquation between title and content is excellent.

The title accurately reflects the content: a keynote lecture by Ellie Pavlick on emergent compositionality in neural networks, presented at CCN 2025.

8/10

The talk is delivered by a leading researcher in NLP and cognitive science, presenting empirical findings from her lab and others. The content is well-structured, with clear examples and references to published work. However, as a keynote, it is an expert opinion and synthesis rather than a peer-reviewed study, and some claims are based on ongoing research.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction to compositionality and its importance in cognitive science and AI.
• 10:00Discussion of definitions of compositionality, including Fodor and Pylyshyn's view.
• 20:00Introduction to structural compositionality and evidence from visual odd-one-out tasks.
• 30:00Circuit analysis in LLMs: example of duplicate, inhibit, and copy components.
• 40:00Functional compositionality: logit lens analysis showing intermediate computations.
• 50:00Context-sensitivity: when compositional behavior is inconsistent and the role of inhibition.
• 1:00:00Conclusion and future directions for integrating structural, functional, and context-sensitive aspects.

• Wang et al. (2022) - Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small — Referenced as influential work on circuits in language models.

• Wang et al. (2022) - Interpretability in the Wild — Supports the existence of circuits in LLMs.

• Fodor & Pylyshyn (1988) - Connectionism and cognitive architecture — Argues that neural networks lack compositionality, contrasting with the empirical findings presented.