Florent Draye

Florent Draye is a PhD Student in Machine Learning at the Max Planck Institute for Intelligent Systems (MPI-IS), Germany. He is affiliated with the ELLIS PhD program and supported by the Hector Fellow Academy. Advised by Prof. Bernhard Schölkopf, his work focuses on mechanistic interpretability of large language models (LLMs), particularly their 'world models' and related techniques like cross-layer transcoders and attribution graphs.

Education:

• PhD, Machine Learning (Max Planck Institute, 2024–Present)
• MSc, Neural Systems and Computation (ETH Zürich, 2022–2024; GPA: 5.9/6)
• Exchange Program, Mathematics (École Normale Supérieure (Ulm), 2021–2022; GPA: 18.3/20)
• BSc, Mathematics (EPFL, 2019–2022; GPA: 5.88/6)

Research Interests: Florent’s research explores the internal mechanisms of LLMs, aiming to uncover their structured representations (world models) and computational processes through mechanistic interpretability. He investigates cross-layer transcoders and attribution graphs to dissect these systems, addressing their behaviors and limitations.

Article Trends: His early work on the Universal Approximation Theorem (2022) highlights foundational theoretical contributions to neural networks, while his current research emphasizes practical interpretability frameworks for LLMs.

Scientific Awards:

• Best bachelor average grade at EPFL (2019–2022), ranking among 1200+ students

Professional Experiences:

• Machine Learning Research Intern (Gatsby Unit UCL, 09/2023–02/2023)
• Data Science Intern (Mindmaze, Summer 2023)

Labs & Teams: Florent works at the MPI-IS under Prof. Bernhard Schölkopf, with prior affiliations at the Gatsby Unit (UCL) and Mindmaze. He contributes to open-source projects like hdelphi and SAELens, extending tools for language model interpretability.