Ramani Duraiswami, Towards AGI: Building, Benchmarking and Improving Large Audio-Language Models

Value of the Information & Strength of the Argument

The talk provides a high-value overview of the state of the art in LALMs, grounded in the speaker’s extensive research. The argumentation is solid, tracing the evolution of the Audio Flamingo family and linking design choices to empirical results. The speaker is candid about limitations, such as the challenge of diarization and the risk of catastrophic forgetting, which strengthens the credibility of the presentation. The emphasis on open-source models and benchmarks is a significant contribution to the field.

Scientific Rigor, Source Quality, Title Accuracy

The talk is scientifically rigorous, referencing a series of peer-reviewed publications (ICLR, EMNLP, ICML, NeurIPS) and benchmarks (MMAU, MMAU-Pro). The sources are credible and directly relevant. The title accurately reflects the content, which is a detailed account of building, benchmarking, and improving LALMs. The speaker’s expertise and the depth of technical detail support the high quality of the information presented.

The title accurately reflects the content: the talk covers the building, benchmarking, and improvement of large audio-language models, with a clear focus on the path towards auditory general intelligence.

8/10

The talk is given by a leading researcher in the field, presenting a coherent overview of a series of peer-reviewed publications (ICLR, EMNLP, ICML, NeurIPS). The claims are supported by references to specific models and benchmarks, and the speaker acknowledges limitations and open questions. However, as a conference talk, it is an expert opinion and not a peer-reviewed meta-analysis, and some claims are presented without full experimental detail.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction and speaker background
• 10:00Motivation for LALMs and the promise of LLMs
• 20:00Core recipe for building LALMs: architecture and design choices
• 30:00Evolution of the Audio Flamingo family of models
• 40:00Benchmarking: creation of MMAU and MMAU-Pro
• 50:00Improvements in Audio Flamingo Next: temporal reasoning and diarization
• 1:00:00Applications to genomics and scientific computing
• 1:10:00Q&A session and discussion

• MMAU: A Holistic Benchmark of Agent Capabilities for Diverse Audio Understanding — Introduced as the first comprehensive benchmark for audio general intelligence, published at ICLR 2025.
• Audio Flamingo 2: An Audio-Language Model with Long Audio Understanding — One of the models in the Audio Flamingo family, presented as a step towards long audio understanding.
• Audio Flamingo 3: An Audio-Language Model with Long Audio Understanding — The most capable model in the family, trained on 50 million data pieces and capable of understanding music, speech, and events.
• Audio Flamingo Next: Towards Auditory General Intelligence — The latest model, extending to 30 minutes of audio and incorporating temporal reasoning.
• MMAU-Pro: A Benchmark for Audio Understanding — Successor to MMAU, developed during JSALT 2025, presented at AAAI 2026.

• Audio Flamingo 2 — The paper describing the model, which supports the claims about its capabilities.
• MMAU — The benchmark paper, which validates the evaluation methodology.

• Gemini 1.5 Pro — While the speaker claims their models outperform Gemini on some benchmarks, Gemini is a proprietary model with different training data and objectives, making direct comparison complex.