Like Having an Intern? The Impact of LLMs on Software Engineering Careers

Value of the Information & Strength of the Argument

The talk provides valuable insights into the real-world impact of LLMs on software engineering, based on a preliminary survey and a review of recent research. The argumentation is solid, combining quantitative data (survey results, DORA report) with qualitative sociological perspectives (Dreyfus model, Bloom’s taxonomy). The speakers effectively highlight the paradox between high adoption and low trust, and they connect this to the potential negative effects on skill development. However, the survey methodology is not fully disclosed, and the sample size is unknown, which limits the generalizability of the findings. The argumentation is persuasive but relies heavily on anecdotal evidence and the speakers’ interpretations.

Scientific Rigor, Source Quality, Title Accuracy

The talk references several credible sources, including the DORA report (Google Cloud), a study by METR, and a study by He et al. on GitHub productivity. The speakers also cite the work of Andre Alexandrescu and Dr. Daisy Hullman, though these are more anecdotal. The title accurately reflects the content, focusing on the impact of LLMs on software engineering careers. The talk is well-structured and the sources are relevant, but the lack of detailed citations for the survey and the reliance on personal anecdotes slightly reduce the overall rigor.

The title accurately reflects the content, which focuses on the impact of LLMs on software engineering careers, particularly the junior-to-senior progression.

7/10

The talk presents preliminary survey results and references peer-reviewed studies, but the survey methodology is not detailed and the sample size is not disclosed. The speakers are credible (a software engineering manager and a sociologist), but the content is largely anecdotal and opinion-based.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction by Tom Sellek and Netta Avnoon, explaining their backgrounds and the motivation for the talk.
• 05:00Discussion of Andre Alexandrescu's talk and the 'intern' metaphor for LLMs.
• 10:00Presentation of survey results on sentiment and usage of LLMs.
• 15:00Review of DORA report and other studies on productivity and code quality.
• 20:00Discussion of the trust paradox: high adoption but low trust in LLM-generated code.
• 25:00Introduction to Bloom's taxonomy and the Dreyfus model of expertise development.
• 30:00Explanation of the Dreyfus model and how expertise is embodied and intuitive.
• 35:00Discussion of the impact of LLMs on skill development and the junior-to-senior progression.
• 40:00Sociological perspective on AI's jurisdiction and potential expansion into software engineering.
• 45:00Conclusion and call for more research on the long-term effects of LLMs on the profession.

• NDC Conferences — Conference website where the talk was recorded.
• NDC Toronto — Specific conference event page.

• DORA Report 2025 — Report cited for LLM adoption and productivity gains.
• METR Study on Developer Productivity — Study cited for time reduction in developer tasks.

• Study on GitHub Productivity — While the talk cites this study as showing 3x-5x increase in lines of code, some critics argue that lines of code is not a good measure of productivity and may not reflect actual quality or efficiency.