Symposium, Cognitive aspects of trust in human AI teams

Value of the Information & Strength of the Argument

The value of the information lies in its synthesis of current research on trust in human-AI teams, drawing on multiple disciplines. The argumentation is solid, with speakers referencing empirical studies and theoretical frameworks. However, some claims are based on anecdotal evidence or ongoing research, and the symposium format limits depth. The discussion of XAI methods and their limitations is particularly valuable, as is the emphasis on trust calibration rather than simply increasing trust.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is generally high, with speakers citing relevant literature and ongoing projects. The quality of sources is good, including references to meta-analyses and influential papers. The title accurately reflects the content. The symposium is part of a funded research project, lending credibility. However, some sources are not explicitly cited in the transcript, and the reliance on unpublished work in some cases reduces the overall rigor.

The title accurately reflects the content, which focuses on cognitive aspects of trust in human-AI teams.

7/10

The symposium features expert speakers from cognitive science, AI, and philosophy, discussing established research and ongoing projects. Claims are supported by references to studies and papers, though some statements are anecdotal or based on unpublished work.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction by Eda Ismail-Tsaous, outlining the symposium's goals and the Ethyde project.
• 10:00Ute Schmid discusses the black-box nature of ML models and the need for explainable AI.
• 20:00Schmid explains the EU AI Act's requirements for trustworthy AI, emphasizing human agency and oversight.
• 30:00Schmid contrasts 'explain to understand' and 'explain to revise', introducing concept-based and near-miss explanations.
• 40:00Schmid discusses the limitations of LIME and the importance of faithful explanations.
• 50:00Johannes Hühn begins his talk on interpretability and biases in machine learning.
• 1:00:00Hühn explains the concept of bias in ML, referencing Tom Mitchell's work and Occam's razor.
• 1:10:00Discussion on the importance of simplicity and generality in rule-based models.
• 1:20:00Fritz Becker presents psychological perspectives on trust and expertise.
• 1:30:00Final talk on decision-making contexts and ethical issues, followed by Q&A.

• Vaccaro et al., 2024 — Referenced in the abstract regarding human-AI team performance.
• Papenmeier et al., 2022 — Referenced in the abstract regarding the relationship between explanations, trust, and performance.
• Longo et al., 2024 — Referenced in the abstract regarding faithful XAI methods.
• LIME paper (Ribeiro et al.) — Mentioned by Ute Schmid as a popular XAI method with faithfulness issues.
• Layer-wise Relevance Propagation (LRP) — Mentioned as a feature relevance method from the HHAI group in Berlin.
• Teso et al. - CAIP — Referenced as a framework for explanatory interactive machine learning.
• Nature Human Behaviour meta-analysis — Referenced by Schmid regarding human-AI team performance.
• Tom Mitchell's 1980 technical report — Referenced by Hühn for the definition of bias in machine learning.

• Vaccaro et al., 2024 — Supports the claim that human-AI teams often underperform.
• Papenmeier et al., 2022 — Supports the complexity of explanations and trust.
• Longo et al., 2024 — Supports the need for faithful XAI methods.