Emulating Real-World PII with a Large-Scale Synthetic Dataset to Audit LLM Memorization

Value of the Information & Strength of the Argument

The talk provides valuable contributions by addressing a critical gap in privacy research: the lack of realistic datasets with PII for studying LLM memorization. The PANORAMA dataset is a significant resource, being the largest of its kind at release. The argumentation is solid, with clear motivation, methodology, and experimental results. The speakers effectively demonstrate the dataset’s utility through a concrete audit study, showing how it can be used to measure memorization and PII leakage. They also discuss the importance of internal consistency in synthetic profiles and the use of Wikipedia scaffolding to enhance realism. The findings on content-type-specific memorization are novel and provide insights for future research. The presentation is well-structured and persuasive, though it could benefit from more details on the generation pipeline and potential limitations.

Scientific Rigor, Source Quality, Title Accuracy

The talk demonstrates scientific rigor through a systematic approach to dataset creation and evaluation. The speakers mention using entity recognition filters to remove profiles with Wikipedia contamination, ensuring the synthetic nature of the data. They also compare different LLMs for generation, selecting O3 for its balance of originality and realism. The audit study uses established metrics (ROUGE-L F1, soft match rate) and bootstrap resampling for confidence intervals. The title accurately reflects the content, and the presentation is well-aligned with the abstract. The speakers cite prior work on memorization (e.g., Nicholas Carlini’s divergence attack) and mention the lack of PII in existing datasets like The Pile and RedPajama. However, they do not provide specific citations for these claims, and the talk is not a peer-reviewed publication, so the rigor is high but not at the level of a formal paper.

The title accurately reflects the content: the talk introduces a synthetic dataset to emulate real-world PII and uses it to audit LLM memorization.

8/10

The talk presents a novel synthetic dataset (PANORAMA) and a rigorous audit methodology, with clear experimental design and results. The speakers are from Microsoft AI, and the work is open-source. However, the presentation is a conference talk, not a peer-reviewed paper, and some details (e.g., exact training hyperparameters) are omitted.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction to the problem of LLM memorization and the need for realistic PII datasets.
• 02:00Example of a divergence attack on GPT-3.5 extracting personal information.
• 05:00Challenges in measuring memorization due to lack of PII in existing datasets.
• 08:00Introduction to PANORAMA dataset: 9,674 profiles, 384,789 samples, six content types.
• 10:00Example of a synthetic profile (Karen Smith) and how information is distributed across online modalities.
• 13:00Data generation pipeline: using Wikipedia articles as scaffolding for realism.
• 16:00Constraints in profile generation: age-appropriate education, marital status, etc.
• 19:00Comparison of LLMs for generation: Gemma, GPT-4, O3; selection of O3.
• 22:00Content type generation rules: social media, forum comments, reviews, marketplace ads.
• 25:00Example of Emily Hayes' profile and generated content across platforms.
• 28:00Audit study: continued pre-training Mistral 7B on PANORAMA with varying repetition.
• 31:00Results: memorization increases with repetition, online ads highest, social media lowest.
• 34:00PII leakage rises from 2.4% baseline to 44% after training on PANORAMA.

• PANORAMA dataset on Hugging Face — The dataset is open-source and available for download.
• Carlini et al. on extracting training data from LLMs — Referenced as prior work on memorization and divergence attacks.

• Carlini et al. 2021, Extracting Training Data from Large Language Models — Supports the claim that LLMs memorize and can be attacked to extract private information.

• No discordant sources found — The talk does not present conflicting evidence; it builds on prior work.