Born a Transformer, Always a Transformer? On the Effect of Pretraining

Born a Transformer, Always a Transformer? On the Effect of Pretraining

by Mayank Jobanputra

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQNNeural networks and fuzzy systems
🎙 Mayank Jobanputra 👥 3K 📅 August 31, 2026 ⏱ 19 min 👁 2 📄 literature review 🧭 2026-08-31
Available in: English (current) Français

Keywords

retrievalcopyingdirectional biasinduction headspretraining

Summary

The talk, given by Mayank Jobanputra, presents his paper on the effects of pretraining on the architectural abilities of transformers, specifically focusing on retrieval and copying tasks. It begins by referencing theoretical results showing that vanilla transformers struggle with length generalization on tasks involving repeated tokens, such as non-unique retrieval and copying. The talk formally defines retrieval and copying tasks, distinguishing between unique and non-unique variants, and notes that theory predicts transformers can solve unique tasks but not non-unique ones. The speaker then presents empirical results from pretrained LLMs (Llama 3 70B and Qwen 2.5 32B) on these tasks, revealing a directional bias: models perform better on forward/rightward tasks than backward/leftward ones, which theory does not predict. This bias is attributed to autoregressive pretraining, which rarely requires backward copying. The talk also shows that fine-tuning can eliminate this bias, and that induction heads are responsible for forward copying while anti-induction heads are needed for backward copying. Finally, the speaker discusses real-world implications, such as in agentic coding tasks, and mentions that results extend to word-level vocabularies and other models.

179 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides valuable insights into the practical behavior of pretrained transformers on formal tasks, bridging theoretical expressivity results with empirical observations. The argumentation is solid: it clearly defines tasks, presents empirical evidence, and supports claims with ablation studies (removing induction/anti-induction heads) and fine-tuning experiments. The directional bias finding is novel and well-motivated, with a plausible explanation rooted in pretraining distribution. However, the talk is relatively brief and does not delve deeply into the theoretical proofs or the full experimental setup, which limits the depth of the argumentation.

Scientific Rigor, Source Quality, Title Accuracy

The talk references relevant literature, including the RASP paper and the flip-flop glitches paper, and mentions the Anthropic induction circuits work. The sources are appropriate for the topic. The title accurately reflects the content, which investigates whether pretraining overcomes theoretical limitations. The talk does not provide detailed citations or URLs, but the context suggests a rigorous research paper. The presentation is clear and well-structured, though the lack of visual aids in the transcript makes it harder to assess the full rigor.

184 words

Title / Content Match

The title accurately reflects the content, which investigates whether pretraining overcomes theoretical limitations of transformers on retrieval and copying tasks.

Quality & Reliability

7/10

The talk presents a clear research question, defines formal tasks, and supports claims with empirical results and theoretical proofs. However, the presentation is concise and lacks detailed methodological exposition, and the video has minimal engagement (2 views, 0 likes), limiting external validation.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

  • RASP paper (2023) — Discussed as showing inherent limitations of transformers on retrieval and copying tasks.
  • Flip-flop glitches paper (Bingman Leu) — Discussed as showing transformers struggle with repeated tokens in retrieval tasks.
  • Anthropic induction circuits blog/paper — Mentioned as showing attention heads can learn to copy patterns.

Concurring Sources

  • RASP paper — Theoretical limitations on retrieval and copying align with the paper's findings.
  • Anthropic induction circuits — Supports the role of induction heads in copying tasks.

Dissenting Sources

  • Theoretical predictions of no directional bias — The paper's finding of a directional bias contradicts the theoretical expectation that transformers should handle forward and backward tasks equally.

Contribution & Novelties

The talk contributes a systematic empirical evaluation of pretrained LLMs on formal retrieval and copying tasks, revealing a directional bias not predicted by theory. It provides evidence that this bias stems from autoregressive pretraining and can be mitigated by fine-tuning. The identification of anti-induction heads as responsible for backward copying is a novel mechanistic insight.

Pour aller plus loin :

88 words

Radar Profile

The radar profile shows high scores in quality and technical level, with slightly lower scores in quantity and reliability. This indicates a technically sound presentation with a moderate amount of information, but the reliability is somewhat limited by the lack of detailed citations and the small audience.

Reliability 7/10

💬 No comments were provided for analysis.