PeopleofSuperintelligence

Researcher

Rafael Rafailov

Rafael Rafailov is a reinforcement learning researcher and the first author of Direct Preference Optimization, a method that aligns language models with human preferences using a simple classification loss. A Berkeley graduate who worked as a junior portfolio manager at Goldman Sachs, he earned master's degrees at Stanford and became a PhD student in the Stanford Artificial Intelligence Laboratory, where he worked on offline reinforcement learning for robots. The 2023 paper, written with Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher Manning and Chelsea Finn, was a NeurIPS 2023 Outstanding Main Track Runner-up. He coauthored OpenVLA and Agent Q in 2024, and in 2025 he coauthored a NeurIPS paper on adaptive length penalties for reasoning models and gave a TED AI talk arguing that the first superintelligence will be a superhuman learner. He is a reinforcement learning researcher at Thinking Machines Lab.

Timeline

  1. By 2021

    From Goldman Sachs to Stanford

    He graduates from UC Berkeley with highest honors in Applied Mathematics, Statistics and Economics, then works as a junior portfolio manager in the Quantitative Investment Strategies unit at Goldman Sachs. He earns master's degrees in Statistics and Computer Science at Stanford and, by 2021, is a PhD student in computer science there, part of the Stanford Artificial Intelligence Laboratory.

  2. 2021–2022

    Offline reinforcement learning for robots

    In 2021, he coauthors two NeurIPS papers with Chelsea Finn: COMBO, a model-based offline reinforcement learning algorithm that regularizes the value function on out-of-support state-action tuples, and V-MAIL, a variational model-based adversarial imitation learning method. His 2022 ICLR paper "Vision-Based Manipulators Need to Also See from Their Hands," written with Kyle Hsu, Moo Jin Kim, Jiajun Wu and Finn, is an oral presentation.

  3. May 2023

    Direct Preference Optimization

    In May 2023, he is first author of Direct Preference Optimization: Your Language Model is Secretly a Reward Model, written with Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning and Chelsea Finn. DPO solves the standard reinforcement learning from human feedback problem with only a simple classification loss, and fine-tuning with it exceeds PPO-based RLHF in controlling the sentiment of generations.

  4. 2023

    A NeurIPS runner-up award

    NeurIPS 2023 names Direct Preference Optimization one of its two Outstanding Main Track Runner-Ups. The conference accepts 3,540 of 13,300 submitted papers.

  5. June–August 2024

    OpenVLA and Agent Q

    In June 2024, he is one of the 18 authors of OpenVLA, a 7-billion-parameter open-source vision-language-action model trained on 970,000 real-world robot demonstrations. In August 2024, he is the last author of Agent Q, which combines guided Monte Carlo Tree Search, self-critique and an off-policy variant of DPO, and lifts Llama-3 70B's success rate in real-world booking scenarios from 18.6% to 81.7%.

  6. 2025

    Adaptive length penalties for reasoning

    In 2025, he coauthors Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning with Violet Xiang, Chase Blagden, Nathan Lile, Sang Truong, Chelsea Finn and Nick Haber, a NeurIPS 2025 paper. Its Adaptive Length Penalty cuts average token usage by 50% when post-training DeepScaleR-1.5B, without significantly dropping performance.

  7. October 2025

    A superhuman learner

    At TED AI San Francisco in October 2025, he speaks as a reinforcement learning researcher at Thinking Machines Lab and argues that "the first superintelligence will be a superhuman learner." He describes a system that adapts efficiently, proposes its own theories and experiments, and uses the environment to verify them.

  8. As of October 2026

    Researcher at Thinking Machines Lab

    He is a reinforcement learning researcher at Thinking Machines Lab, the startup co-founded by former OpenAI CTO Mira Murati. His research areas are reinforcement learning, agents and foundation models.

Career

  1. Since 2025Thinking Machines LabReinforcement learning researcher
  2. Goldman SachsJunior portfolio manager

Education

  1. University of California, BerkeleyDegree in Applied Mathematics, Statistics and Economics
  2. Stanford UniversityMaster's degrees in Statistics and Computer Science

From the biography.

  1. The median h-index is 48 at universities and 14 at frontier AI labsIncludes Thinking Machines Lab. The median h-index is 48 at universities and 14 at frontier AI labs. We compare publishing careers, recent citations, dual appointments and job moves across 2,420 university researchers and 308 frontier-lab researchers.
  2. Companies supply the most prior-work hires at 7 of the 9 frontier AI labsIncludes Thinking Machines Lab. Our estimates of the organizations where 550 people worked directly before joining 9 frontier AI labs across 620 career transitions, comparing the main feeder organizations per lab.
  3. 22 of 133 people shaping AI with a former OpenAI role now work at AnthropicIncludes Thinking Machines Lab. Our estimates of where OpenAI recruits came from and where 133 people with former OpenAI roles work now, including the companies and labs they founded.
  4. 5,000 frontier users would be 38.2% of the core labs' 13,100 reported staffIncludes Thinking Machines Lab. 5,000 frontier users would be 38.2% of the core labs' 13,100 reported staff. We test the estimate against 3,913 named model contributors and dated all-staff records across Google DeepMind, OpenAI and Anthropic, plus six additional frontier-grade labs.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Rafael Rafailov yet. Send us a free-licensed photo.

  1. Rafael Rafailov rmrafailov.github.io. Checked 22 October 2026.
  2. Direct Preference Optimization: Your Language Model is Secretly a Reward Model arxiv.org. Checked 22 October 2026.
  3. Announcing the NeurIPS 2023 Paper Awards blog.neurips.cc. Checked 22 October 2026.
  4. OpenVLA: An Open-Source Vision-Language-Action Model arxiv.org. Checked 22 October 2026.
  5. Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents arxiv.org. Checked 22 October 2026.
  6. Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning arxiv.org. Checked 22 October 2026.
  7. Publications - Stanford IRIS Lab irislab.stanford.edu. Checked 22 October 2026.
  8. Thinking Machines Rethinking AGI newsletter.opentools.ai. Checked 22 October 2026.
  9. Rafael Rafailov - Google Scholar scholar.google.com. Checked 22 October 2026.