Jonathan Tompson

Jonathan Tompson is a robotics and computer vision researcher whose work runs from convolutional-network hand tracking to embodied language models for robots. He studied electrical engineering at Harvard and Columbia, designed mixed-signal radio-frequency chips at Epoch Microelectronics from 2008 to 2011, and earned a computer science PhD at New York University in 2015. He led a hand-tracking project for Google Daydream in 2016, then coauthored Transporter Networks in 2020 and PaLM-E in 2023. Learning Interactive Real-World Simulators, a paper he coauthored, was named an Outstanding Paper at ICLR 2024, and he was among the authors of the Gemini Robotics report in March 2025. He is a Principal Research Scientist (Director) at Meta Reality Labs and ML Lead for Meta Robotics Studio.

Now at: Meta
Badges: Bridge

Timeline

2006–2011

Chip design at Epoch Microelectronics

He earns a BA in electrical engineering at Harvard, where his 2006 undergraduate honors thesis with Gu-Yeon Wei designs a transformer-based communication system for high-speed digital systems, and a master's in electrical engineering at Columbia. From 2008 to 2011, he works as a hardware design engineer at Epoch Microelectronics, designing mixed-signal radio-frequency chips.
2014–2015

Hand and body tracking at NYU

In 2014, he coauthors "Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks" for SIGGRAPH with Murphy Stein, Yann LeCun and Ken Perlin, and releases the NYU Hand Pose Dataset. In 2015, he completes a computer science PhD at New York University with the thesis "Localization of Humans in Images Using Convolutional Networks". His CVPR 2015 paper with Ross Goroshin, Arjun Jain, Yann LeCun and Christoph Bregler adds a position-refinement model that outperforms all existing approaches on the MPII human pose dataset.
2016

Hand tracking for Google Daydream

At Google in 2016, he leads a project on Daydream to track hands from a head-mounted camera. The ConvNet-based system runs in real time on embedded hardware and shows robustness to occlusion and hand-shape variation.
2020

Transporter Networks for manipulation

In 2020, he coauthors Transporter Networks with Andy Zeng, Pete Florence and colleagues, presented at CoRL 2020. The model architecture rearranges deep visual features to infer spatial displacements for pick-and-place, and it is orders of magnitude more sample efficient than the benchmarked alternatives.
2023

PaLM-E, an embodied language model

In 2023, he is among the 22 authors of PaLM-E, an embodied multimodal language model that feeds continuous sensor inputs into a pretrained language model. A single PaLM-E model handles robotic manipulation planning, visual question answering and captioning across multiple embodiments.
2024

An ICLR outstanding paper

Learning Interactive Real-World Simulators, coauthored with Sherry Yang, Yilun Du, Seyed Ghasemipour, Leslie Kaelbling, Dale Schuurmans and Pieter Abbeel, is named an Outstanding Paper at ICLR 2024. It introduces UniSim, a universal simulator of real-world interaction built through generative modeling, which trains vision-language policies and reinforcement learning policies that deploy in the real world in zero shot.
March 2025

The Gemini Robotics report

In March 2025, he is among the authors of the Gemini Robotics report. It introduces Gemini Robotics, a Vision-Language-Action model built on Gemini 2.0 that controls robots directly, and Gemini Robotics-ER for embodied reasoning.
As of October 2026

Robotics at Meta Reality Labs

He is a Principal Research Scientist (Director) at Meta Reality Labs in California, where he is ML Lead for Meta Robotics Studio, after a Senior Staff Research Scientist role at Google DeepMind.