Pablo Samuel Castro

Pablo Samuel Castro is a reinforcement learning researcher at Google DeepMind in Montreal whose work covers open-source research tools, real-world control and the way deep RL results are measured. Born and raised in Quito, Ecuador, he moved to Montreal after high school and earned his master's and PhD in reinforcement learning at McGill with Doina Precup and Prakash Panangaden. At Google since 2012, he introduced a TensorFlow-based RL research framework with Marc Bellemare in 2018, coauthored a 2020 Nature paper on a flight controller for Loon balloons, and shared a NeurIPS Outstanding Paper Award in 2021 for "Deep Reinforcement Learning at the Edge of the Statistical Precipice". In 2023 he coauthored BBF, a value-based agent for the Atari 100K benchmark, and in 2024 a paper on mixtures of experts for deep RL. He is a senior staff research scientist at Google DeepMind, a Core Industry Member at Mila and an adjunct professor at Université de Montréal.

Now at: Google DeepMind, Mila and Université de Montréal
Badges: Bridge

Timeline

By 2012

Reinforcement learning PhD at McGill

Born and raised in Quito, Ecuador, he moves to Montreal after high school to study at McGill. After finishing his bachelor's degree and working at a flight simulator company, he earns a master's and a PhD at McGill in reinforcement learning, supervised by Doina Precup and Prakash Panangaden.
2012

Joining Google in Pittsburgh

After a 10-month postdoc in Paris, he moves to Pittsburgh to join Google, where he has worked since 2012.
August 2018

The Dopamine research framework

In August 2018, as a research software developer on the Google Brain team, he and Marc G. Bellemare introduce a TensorFlow-based framework for flexible and reproducible reinforcement learning research. The 2018 preprint, Dopamine, A Research Framework for Deep Reinforcement Learning, adds Subhodeep Moitra, Carles Gelada and Saurabh Kumar as coauthors.
December 2020

Steering Loon balloons with RL

In December 2020, he coauthors a Nature paper with Marc G. Bellemare, Salvatore Candido and others on a reinforcement learning flight controller for Loon superpressure balloons. The controller is deployed in a 39-day controlled experiment over the Pacific Ocean and outperforms Loon's previous algorithm.
2021

A NeurIPS outstanding paper

In 2021, he coauthors "Deep Reinforcement Learning at the Edge of the Statistical Precipice" with Rishabh Agarwal, Max Schwarzer, Aaron Courville and Marc G. Bellemare, and it receives a NeurIPS Outstanding Paper Award. The paper advocates interval estimates, performance profiles and interquartile mean scores for evaluating deep RL, and comes with the open-source library rliable.
2023

The BBF Atari agent

In 2023, he coauthors "Bigger, Better, Faster" at ICML with Max Schwarzer, Johan Obando-Ceron, Aaron Courville, Marc Bellemare and Rishabh Agarwal. The value-based agent BBF scales up its networks for value estimation and reaches super-human performance on the Atari 100K benchmark.
2024

Mixtures of experts for deep RL

In 2024, he coauthors "Mixtures of Experts Unlock Parameter Scaling for Deep RL" with Johan Obando-Ceron, Doina Precup and others. Adding Soft MoE modules to value-based networks gives models that scale with parameter count, which provides empirical evidence toward scaling laws for reinforcement learning.
As of October 2026

Google DeepMind in Montreal

He is a senior staff research scientist at Google DeepMind in Montreal, working on fundamental reinforcement learning research and advocating for more LatinX representation in the research community. He is also a Core Industry Member at Mila and an adjunct professor in the Department of Computer Science and Operations Research at Université de Montréal.