George E. Dahl

Timeline

1. 2009–2010

   Deep networks for speech

   In 2009, he begins collaborating with Abdel-rahman Mohamed on deep neural networks for speech recognition. In Geoffrey Hinton’s Toronto lab, he trains the models on high-end graphics cards; they outperform earlier methods on a standard three-hour benchmark. In 2010, Mohamed and Dahl intern at Microsoft Research in Redmond and apply their methods to larger speech tasks.

2. 2013

   Initialization and momentum

   At ICML in 2013, he coauthors “On the importance of initialization and momentum in deep learning” with Ilya Sutskever, James Martens and Geoffrey Hinton. The paper shows that well-designed initialization and a gradual momentum schedule let stochastic gradient descent train deep and recurrent networks.

3. 2014

   QSAR and the Merck challenge

   In 2014, he coauthors “Multi-task Neural Networks for QSAR Predictions,” a public-data study that uses methods similar to those his team uses to win Kaggle’s Merck molecular activity challenge. He leads the team that wins the contest.

4. 2015

   A Toronto PhD and Google Brain

   He completes a PhD in the University of Toronto’s Machine Learning Group under Geoffrey Hinton in 2015, and joins Google’s Brain team as a research scientist. His 2015 journal paper, “Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships,” uses his Merck contest code.

5. 2019

   Data-parallel training limits

   In 2019, he coauthors a Google AI study of batch size and training time across six neural-network types and seven datasets. The study identifies perfect scaling, diminishing returns and maximal data parallelism, where larger batches stop reducing training time.

6. 2023–2025

   The AlgoPerf benchmark

   He coauthors the AlgoPerf benchmark in 2023, measuring neural-network training algorithms by time-to-result across workloads on fixed hardware. In 2025, he coauthors the inaugural competition report: Distributed Shampoo wins the external-tuning ruleset, and Schedule Free AdamW wins the self-tuning ruleset.

7. 2026

   Learning-rate schedules

   In 2026, he coauthors “What do near-optimal learning rate schedules look like?” The study tests schedule families on three workloads and finds that warmup and decay are robust features of good schedules; weight decay can strongly affect the best schedule shape.

8. As of October 2026

   Research scientist and AlgoPerf co-chair

   He is a research scientist at Google, with current research on neural-network training algorithms. In 2024, he becomes chair of MLCommons’ Algorithms Working Group for the 2024–2026 term. In June 2026, he co-chairs AlgoPerf at Google DeepMind with Frank Schneider.