Antonio Orvieto

Antonio Orvieto is a Professor and Principal Researcher at the Max Planck Institute for Intelligent Systems and ELLIS Institute Tübingen, where he leads the Deep Models and Optimization research group. He is also a lecturer at the University of Tübingen and faculty for the CLS, ELLIS, and IMPRS-IS PhD Programs. His research focuses on improving the efficiency of deep learning technologies through theoretical understanding of optimization dynamics and innovative neural network architectures.

Dr. Orvieto earned his PhD from ETH Zürich under the supervision of Prof. Dr. Thomas Hofmann and Dr. Aurelien Lucchi. Prior to his PhD, he obtained his master's degree in Robotics, Systems, and Control from ETH. His educational background also includes undergraduate studies at Universita' degli studi di Padova in Italy. During his academic journey, he gained research experience at DeepMind (UK), Meta (US), MILA (CA), INRIA (FR), and HILTI (LI).

Dr. Orvieto's research spans two main areas: understanding the intricacies of large-scale optimization dynamics and designing innovative architectures and powerful optimizers capable of handling complex data. His work particularly focuses on decoding patterns in sequential data, with applications in biology, neuroscience, natural language processing, and music generation. His theoretical approach to deep learning has led to significant contributions in understanding recurrent neural networks, transformers, and optimization methods. His recent publications reveal a strong focus on sequence modeling, optimization theory, and the theoretical foundations of deep learning, with particular emphasis on improving training efficiency and model performance.

• Schmidt Sciences AI2050 Early Career Fellow

Dr. Orvieto actively mentors PhD students and leads a vibrant research group at the Max Planck Institute for Intelligent Systems. His Deep Models and Optimization group includes PhD students Destiny Okpekpe, Felix Sarnthein, Diganta Misra, Sajad Movahedi, and Wenjie Fan, among others. He is deeply involved in doctoral education as faculty for multiple PhD programs including CLS, ELLIS, and IMPRS-IS. His teaching includes the course "Nonconvex Optimization for Deep Learning" at the University of Tübingen.

The Deep Models and Optimization research group investigates the interplay between optimizers and architectures in deep learning, with a focus on developing new networks for long-range reasoning. The group's mission is to design new optimizers and neural networks to accelerate technology and scientific discovery, with a strong theoretical foundation in optimization theory. They strongly believe that deep learning will revolutionize science and technology, and they aim to make powerful deep learning solutions accessible to scientists and engineers regardless of resource limitations.