Systematicity in language models' knowledge and self-knowledge

Value of the Information & Strength of the Argument

The talk provides valuable insights into the debate on whether language models possess knowledge and beliefs, and proposes a concrete research direction. The argumentation is solid, building from observed failures to a philosophical framework and then to empirical evidence and proposed solutions. The speaker acknowledges limitations and alternative views, making the argument nuanced.

Scientific Rigor, Source Quality, Title Accuracy

The talk references several papers, including ‘Deductive closure training of language models’ (Akyürek et al., 2024) and ‘Training Language Models to Explain Their Own Computations’ (Li et al., 2025), which are relevant and credible. The title accurately reflects the content, focusing on systematicity in knowledge and self-knowledge. The presentation is rigorous, with clear methodology and interpretation of results.

The title accurately reflects the content, which explores systematicity in language models' knowledge and self-knowledge.

8/10

The talk is given by a leading researcher (MIT professor) and presents a coherent research agenda with references to published papers. However, it is a seminar presentation, not a peer-reviewed publication, and some claims are based on ongoing work.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction: example of a language model giving a correct answer with confidence estimate, but failing on a related question.
• 10:00Discussion of interpretationist vs representationalist views of internal world models.
• 20:00Presentation of behavioral benchmarks for systematicity, including physical reasoning and counterfactual tasks.
• 30:00Introduction of probing methodology to test for linear encodings of world state in LM representations.
• 40:00Results showing that world state is linearly decodable from entity mentions, and that this capability arises from pre-training.
• 50:00Discussion of training objectives for internal systematicity, including deductive closure training.
• 1:00:00Examples of improvements in generalization and self-explanation reliability from these objectives.
• 1:10:00Conclusion and open questions about the nature of LM knowledge and implications for AI trustworthiness.

• Deductive closure training of language models for coherence, accuracy, and updatability — Cited as a method for training LMs to improve coherence and accuracy.
• Training Language Models to Explain Their Own Computations — Cited as a method for improving self-explanation reliability.
• Beyond binary rewards: Training lms to reason about their uncertainty — Cited as a method for training LMs to reason about uncertainty.

• Language Models as Knowledge Bases? — Supports the idea that LMs encode factual knowledge.
• Emergent Abilities of Large Language Models — Discusses emergent capabilities that may indicate internal world models.

• On the Dangers of Stochastic Parrots — Argues that LMs are just stochastic parrots without true understanding, contrasting with the speaker's view.