Kamal Ndousse

Kamal Ndousse is a machine learning researcher at Anthropic who works on reinforcement learning for large language models. Trained in mathematics and physics at MIT, he worked at Fitbit and Coinbase before he joined the OpenAI Scholars program in February 2020 to study social multi-agent reinforcement learning with Natasha Jaques. Their paper on social learning, written with Douglas Eck and Sergey Levine, won a best paper award at the NeurIPS 2020 Cooperative AI Workshop, and a fellowship on OpenAI's open-endedness team began in September 2020. At Anthropic, which he joined in April 2021, he co-authored papers on a helpful and harmless assistant trained with RLHF, on Constitutional AI and on Sleeper Agents. In October 2026, he co-authored BIRD, a benchmark for systems that use language models to write reward code.

Now at: Anthropic

Timeline

By 2020

MIT, Fitbit and Coinbase

He earns a B.S. in mathematics and physics from MIT. Before 2020, he works as an algorithms research scientist at Fitbit and as a machine learning engineer at Coinbase.
2020–2021

Social learning as OpenAI Scholar

In February 2020, he joins the OpenAI Scholars program to study social multi-agent reinforcement learning with Natasha Jaques. "Learning Social Learning", written with Douglas Eck, Sergey Levine and Jaques, wins a best paper award at the NeurIPS 2020 Cooperative AI Workshop. Its ICML 2021 version, "Emergent Social Learning via Multi-agent Reinforcement Learning", asks whether reinforcement learning agents can learn to use cues from experts in a shared environment.
2020–2022

OpenAI open-endedness fellowship

In September 2020, he starts a fellowship on OpenAI's open-endedness team. With Joel Lehman, Kenneth O. Stanley and others, he co-authors "Evolution through Large Models" in June 2022, which uses code-generating language models to improve mutation in genetic programming.
2021

Reinforcement learning at Anthropic

He starts at Anthropic in April 2021, working on systems that train large language models with reinforcement learning. In December 2021, he co-authors "A General Language Assistant as a Laboratory for Alignment", which finds that ranked preference modeling performs much better than imitation learning.
April 2022

A helpful and harmless assistant

In April 2022, he is the third author, after Yuntao Bai and Andy Jones, of "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback". The paper finetunes language models with preference modeling and RLHF, updating preference models and policies weekly with fresh human feedback.
December 2022

Constitutional AI

Anthropic's "Constitutional AI: Harmlessness from AI Feedback", which he co-authors in December 2022, trains a harmless AI assistant whose only human oversight is a list of rules or principles. Its reinforcement learning phase uses a preference model trained on AI judgments as the reward signal, a method called RL from AI Feedback (RLAIF).
January 2024

Sleeper Agents

In January 2024, he co-authors "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training". The study builds language models with backdoor behavior and finds that supervised fine-tuning, reinforcement learning and adversarial training leave the backdoors in place.
October 2026

The BIRD reward-design benchmark

As an Anthropic researcher, he co-authors "A Bird's-Eye View of Iterative Reward Design" with members of the Anthropic Fellows Program in October 2026. The paper introduces BIRD, a benchmark for LLM-based systems that write and refine reward code, and finds a small set of simple design choices that improve performance across MuJoCo, Meta-World, Assistax and HumanoidBench.