Aaron Defazio
Aaron Defazio is a machine-learning researcher whose optimization methods reduce learning-rate tuning and training-schedule choices for AI. At the Australian National University, he completed an honours thesis on network topology in 2010, published a paper on scale-free networks in 2012, and introduced Finito and SAGA in his 2014 PhD thesis. He coauthored MADGRAD in 2022 and Schedule-Free in 2024; D-Adaptation won the 2023 ICML Outstanding Paper Award. At the University of Georgia's AI Research Day in 2025, he gave a keynote as Senior Staff AI Research Scientist at Meta. He published ScheduleFree+ in May 2026 and is a Member of Technical Staff at OpenAI as of October 2026.
Timeline
-
2010
Network Topology Tomography
At the Australian National University, he completes the honours thesis Network Topology Tomography in 2010 under Tiberio Caetano's supervision.
-
2012
Learning scale-free networks
With Tiberio Caetano, he presents a convex formulation for learning scale-free networks in a 2012 NIPS paper.
-
2014
A PhD thesis on optimization
He completes a PhD at the Australian National University in 2014 under Tiberio Caetano; his thesis introduces Finito and SAGA for machine-learning optimization.
-
2022
The MADGRAD optimizer
With Samy Jelassi, he introduces MADGRAD, an AdaGrad-family method for adaptive stochastic optimization, in 2022.
-
2023
D-Adaptation wins an ICML award
With Konstantin Mishchenko, he presents D-Adaptation, a single-loop method that sets the learning rate without knowing the distance from the initial point to the solution set. The work receives the 2023 ICML Outstanding Paper Award.
-
2024
Schedule-Free learning
With five coauthors, he proposes Schedule-Free in 2024, a method that avoids choosing a training stop step and adds no hyperparameters beyond standard optimizers with momentum.
-
2025
Theory to Practice at FAIR
At the University of Georgia's AI Research Day in 2025, he gives a keynote titled ‘Theory to Practice at FAIR: AI training algorithms’ as Senior Staff AI Research Scientist at Meta.
-
As of October 2026
OpenAI and ScheduleFree+
In May 2026, he introduces ScheduleFree+, a learning-rate-free and schedule-free method for training large language models. As of October 2026, he is a Member of Technical Staff at OpenAI.
Career
Education
- 2014Australian National UniversityPhD
- 2010Australian National UniversityHonours
Sources and credits
Photo: We have no published photo of Aaron Defazio yet. Send us a free-licensed photo.
