École d'été | 8 juin 2026 : Panel de la journée

This panel discussion, part of the UQAM summer school on knowledge, reasoning, and decision-making, brings together experts to discuss the intersection of large language models (LLMs) and formal logic. The conversation begins with a question about the reliability of LLMs when generating formal expressions, highlighting the phenomenon of ‘data artifacts’ where models may produce correct answers for superficial reasons, such as relying on punctuation or incidental dataset features. The panelists discuss examples from natural language inference, where models perform well even when premises are blanked out, due to statistical regularities in conclusions. They emphasize the importance of focusing on hard regions of the problem space to avoid trivial solutions. A key theme is the need for deterministic validation steps, such as graph comparison or theorem provers, to ensure correctness independent of the LLM. The discussion also touches on the ease of using LLMs for SPARQL queries, the governance of AI systems, and the potential for combining neural and symbolic approaches. The panelists share practical experiences, including the use of LLMs for ontology-based data integration and the importance of human oversight in certifying queries. The conversation concludes with reflections on the limitations of current architectures and the need for broader thinking beyond question answering.