Meaning in Large Language Models: Bridging Formal Semantics, Pragmatics, and Learned Representations

Value of the Information & Strength of the Argument

The talk provides a valuable historical overview and a thought-provoking argument for rethinking semantics and pragmatics in light of connectionist models. Potts effectively argues that the current symbolic framework is contingent and may limit the scope of semantic theory. He supports his claims with references to key figures and works, and he engages with counterarguments. The argumentation is solid, though some historical interpretations are open to debate.

Scientific Rigor, Source Quality, Title Accuracy

The talk demonstrates scientific rigor through its detailed historical analysis and references to foundational works. Potts cites specific papers and authors, and he acknowledges the limitations of his historical account. The title accurately reflects the content, and the presentation is well-organized. The sources are credible, and the speaker’s expertise adds to the reliability.

The title accurately reflects the content, which bridges formal semantics, pragmatics, and learned representations in LLMs.

8/10

The speaker is a renowned professor of linguistics and computer science at Stanford, with extensive publications. The talk is well-structured, historically grounded, and critically engages with the field. However, it is a seminar presentation, not peer-reviewed, and some historical claims are debatable.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and personal motivation
• 05:00Historical timeline from logical positivism to modern AI
• 10:00Influences on Montague semantics and the central dogmas
• 15:00Discussion of the Turing test and behaviorism
• 20:00Question and answer session on history and precursors
• 25:00Central dogmas of semantic theory and commentary
• 30:00Introduction to connectionist semantics and its implications
• 35:00Discussion of LLMs as tools for semantic investigation
• 40:00Potential future of semantics and pragmatics with AI

• CausalGym: Benchmarking causal interpretability methods on linguistic tasks — Referenced as a work by Arora, Jurafsky, and Potts on interpretability.
• Mission: Impossible Language Models — Referenced as a work by Kallini et al. on language models.
• RAVEL: Evaluating interpretability methods on disentangling language model representations — Referenced as a work by Huang et al. on interpretability.
• ARES: An automated evaluation framework for retrieval-augmented generation systems — Referenced as a work by Saad-Falcon et al. on evaluation.

• Montague grammar — Supports the discussion of formal semantics.
• Connectionism — Supports the discussion of connectionist semantics.

• Chomsky's review of Skinner's Verbal Behavior — Potts discusses the Chomskyan rejection of behaviorism, which contrasts with Turing's behaviorist criteria.