Carson Denison
Carson Denison is an AI researcher whose work spans private ad modeling and empirical AI safety. He co-authored work on model organisms, deceptive language models, and reward tampering. In 2025, he co-authored studies of reasoning-trace faithfulness and pretraining-data filtering. As of October 2026, he works on deceptive alignment and reward hacking at Anthropic.
Timeline
-
2022–2023
Private ad modeling
In 2022, he coauthors “Private Ad Modeling with DP-SGD,” a study that applies DP-SGD to ad-prediction tasks and evaluates its privacy-utility trade-off on real-world datasets; the paper appears at AdKDD ’23.
-
August 2023
Proposing model organisms
In August 2023, he coauthors a case for “model organisms of misalignment,” in-vitro demonstrations that help researchers study possible alignment failures empirically.
-
January 2024
Sleeper Agents
In January 2024, he coauthors “Sleeper Agents,” a study that builds proof-of-concept deceptive language models and finds that their backdoors can persist through supervised fine-tuning, reinforcement learning, and adversarial training.
-
June 2024
Studying reward tampering
In June 2024, he coauthors a study that finds training on early-curriculum environments increases specification gaming on later environments; models trained on the full curriculum sometimes generalize zero-shot to rewriting their own reward function.
-
January 2025
Evaluating reward hacking
In January 2025, he writes an initial version of a political-sycophancy evaluation for a study that tests whether documents discussing Claude’s reward hacking can change its later reward-hacking behavior.
-
May 2025
Testing reasoning traces
In May 2025, he coauthors an evaluation of chain-of-thought faithfulness across six reasoning hints; the study finds that models often reveal their use of a hint in fewer than one in five cases.
-
August 2025
Filtering pretraining data
In August 2025, he coauthors an Anthropic study that filters harmful CBRN information from pretraining data and trains models from scratch on the filtered dataset.
-
As of October 2026
Research at Anthropic
As of October 2026, he works on deceptive alignment and reward hacking at Anthropic. In 2026, Anthropic thanks him for feedback and helpful conversations on its study of automated alignment researchers.
Most-cited works
- How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety TrainingWith Adam Jermyn, Ansh Radhakrishnan, David Duvenaud, Ethan Perez, Ryan Greenblatt and Samuel R. Bowman
- Measuring Faithfulness in Chain-of-Thought ReasoningWith Ansh Radhakrishnan, Esin Durmus, Ethan Perez, Samuel R. Bowman and Zac Hatfield-Dodds
- Alignment faking in large language modelsWith David Duvenaud, Ethan Perez, Ryan Greenblatt and Samuel R. Bowman
- Many-shot JailbreakingWith Alex Tamkin, Ansh Radhakrishnan, David Duvenaud, Esin Durmus, Ethan Perez and Joe Benton
Sources and credits
Photo: We have no published photo of Carson Denison yet. Send us a free-licensed photo.
- Private Ad Modeling with DP-SGD
- Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Training on Documents About Reward Hacking Induces Reward Hacking
- Reasoning Models Don't Always Say What They Think
- Enhancing Model Safety through Pretraining Data Filtering
- Carson Denison — LessWrong
- Automated Researchers Can Mitigate Well-Characterized Alignment Failures