Nicolas Le Roux

Nicolas Le Roux is a machine learning researcher who works on optimization and reinforcement learning, from training neural networks to fine-tuning language models. After a 2008 PhD with Yoshua Bengio at Université de Montréal, he developed the stochastic average gradient method with Mark Schmidt and Francis Bach, and the three shared the 2018 Lagrange prize. At Google Brain in Montréal he worked on policy optimization, and his paper on surrogate functions for reinforcement learning, written with Sharan Vaswani and colleagues, received an honorable mention at AISTATS 2022. In 2025 he posted Tapered Off-Policy REINFORCE, an algorithm for fine-tuning large language models with reinforcement learning. He is an adjunct professor at McGill University and Université de Montréal and is taking a break at the moment.

Now at: McGill University and Université de Montréal
Badges: Bridge
Boomerang

Timeline

2007–2010

Doctorate under Yoshua Bengio

He earns a PhD at Université de Montréal in 2008, working with Yoshua Bengio on neural networks and their optimisation. With Pierre-Antoine Manzagol and Bengio, he introduces the TONGA natural gradient algorithm at NIPS 2007, and a 2008 Neural Computation paper with Bengio shows that restricted Boltzmann machines are universal approximators of discrete distributions. In 2010, a Microsoft Research report with Nicolas Heess, Jamie Shotton and John Winn introduces the masked RBM, which models occlusion boundaries in image patches.
2012–2013

The stochastic average gradient

In 2012, he posts a stochastic gradient method with Mark Schmidt and Francis Bach of INRIA Paris that keeps a memory of previous gradient values to reach a linear convergence rate on strongly convex sums. The 2013 paper Minimizing Finite Sums with the Stochastic Average Gradient names the SAG method and improves the general convergence rate from O(1/k^{1/2}) to O(1/k).
2018

Sharing the Lagrange prize

He is a co-recipient of the 2018 Lagrange prize with Francis Bach and Mark Schmidt, awarded for work on stochastic variance reduction methods.
2019–2020

Google Brain in Montréal

As a research scientist at Google Brain in Montréal, he is a mentor or expert at the AI4Good Lab in 2019 and 2020. With Zafarali Ahmed, Mohammad Norouzi and Dale Schuurmans, he coauthors Understanding the impact of entropy on policy optimization, accepted to ICML 2019, which examines whether entropy regularization helps exploration in reinforcement learning.
2022

An AISTATS honorable mention

His paper A general class of surrogate functions for stable and efficient reinforcement learning, written with Sharan Vaswani, Olivier Bachem, Pablo Samuel Castro and others, receives an honorable mention at AISTATS 2022. It proposes FMA-PG, a framework based on functional mirror ascent that yields a family of surrogate functions with policy improvement guarantees.
August 2023

Optimization in function space

In August 2023, he gives a talk at GERAD in Montréal on recent advances in functional optimization, arguing that in several cases reinforcement learning and supervised learning problems are best analyzed in function space. He has managed research teams at Criteo, Google and Microsoft, holds a CIFAR AI Chair, and is an adjunct professor at Université de Montréal and McGill.
March 2025

Tapered Off-Policy REINFORCE

In March 2025, he posts Tapered Off-Policy REINFORCE (TOPR), written with Marc G. Bellemare and eight other authors, an algorithm for fine-tuning large language models with reinforcement learning. TOPR uses a tapered, asymmetric variant of importance sampling, and on the GSM8K and MATH reasoning benchmarks it helps 8B-parameter models match the performance of 70B-parameter models.
As of October 2026

Adjunct faculty in Montréal

He is an adjunct professor at McGill University's School of Computer Science and at Université de Montréal's Department of Computer Science and Operations Research. He describes himself as taking a break at the moment, and he remains adjunct faculty at both universities.