PeopleofSuperintelligence

Researcher

Toby Sharp

Toby Sharp is a computer-vision researcher whose work spans GPU decision forests, Kinect body tracking, and real-time hand tracking. He studied mathematics, built a career in software engineering and scientific computing, and published a steganography paper in 2001. At Microsoft Research, he coauthored work on geodesic image and video editing, contributed high-speed code to Kinect's machine-learning body tracker, and first-authored a real-time hand tracker. He shared the MacRobert Award for Kinect's machine-learning contribution in 2011 and the Longuet-Higgins Prize for the pose-recognition paper in 2021. After roles in Microsoft's AI and HoloLens work and as Google's lead architect of augmented reality, he works at Arm as of October 2026.

Now at
Arm

Timeline

  1. 2001

    Mathematics and a steganography paper

    From an early age, he immerses himself in computing, studies mathematics, and builds a software career in scientific computing. In 2001, he writes “An implementation of key-based digital signal steganography” for the International Workshop on Information Hiding.

  2. 2008

    Decision forests on a GPU

    In 2008, he is sole author of “Implementing Decision Trees and Forests on a GPU,” published at ECCV. The method trains and evaluates decision forests entirely on a GPU, with object-recognition results matching the CPU in about 1% of the time.

  3. November 2010

    Geodesic image and video editing

    In November 2010, he coauthors “Geodesic image and video editing” with Antonio Criminisi, Carsten Rother, and Patrick Perez. The technique applies a generalized geodesic distance transform to edge-sensitive editing across images and videos.

  4. 2011

    Body tracking for Kinect

    In 2011, he coauthors “Real-Time Human Pose Recognition in Parts from a Single Depth Image,” which predicts 3D body-joint positions from one depth image and wins CVPR's Best Paper Award. At Microsoft Research Cambridge, he works on high-speed implementations of Jamie Shotton's body-part algorithm for Kinect.

  5. June 2011

    A MacRobert Award

    In June 2011, he shares the Royal Academy of Engineering's MacRobert Award with Andrew Blake, Andrew Fitzgibbon, Jamie Shotton, and Mat Cook. The award recognizes the machine learning behind Kinect's human-motion capture.

  6. April 2015

    Real-time hand tracking

    In April 2015, he is first author of “Accurate, Robust, and Flexible Real-time Hand Tracking,” presented at CHI with fourteen coauthors, including Jonathan Taylor, Jamie Shotton, Andrew Fitzgibbon, Shahram Izadi, and Pushmeet Kohli. The system tracks detailed hand motion from one depth camera and receives a Best of CHI Honorable Mention.

  7. June 2021

    The Longuet-Higgins prize

    In June 2021, he receives the Longuet-Higgins Prize with Jamie Shotton, Andrew Fitzgibbon, Mark Finocchio, Alex Kipman, and Andrew Blake for “Real-Time Human Pose Recognition in Parts from Single Depth Images.”

  8. As of October 2026

    Later roles at Google and Arm

    After serving as principal software scientist for Microsoft AI and HoloLens, he becomes Google's lead architect of augmented reality. As of October 2026, he works at Arm.

Career

  1. NowArm
  2. 2011Microsoft ResearchSenior Research Software-Development Engineer
  3. GoogleLead Architect of Augmented Reality
  4. MicrosoftPrincipal Software Scientist

From the biography.

  1. Among employers with 30+ full-time cases, OpenAI leads primary technical role H-1B pay at $310,000, and Amazon, Google and Microsoft file 57% of all casesIncludes Microsoft. Across 440,669 certified US H-1B filings from 44 employers (FY2020–FY2026), OpenAI leads primary technical role base pay at $310,000 and xAI leads overall base pay at $290,000.
  2. Google had the most people shaping AI every year from 2014 to 2025Includes Google. Our estimates of which organizations employed the most people shaping AI each year from 1955 to 2025, tracing the shift from university labs to companies.
  3. Google has the highest combined h-index among AI organizationsIncludes Google. We estimate that Google holds the highest combined h-index at 8,016, while universities account for 73% of the deduplicated total across AI organizations.
  4. 20% of people at companies who shape AI also hold a university role, led by Google and Stanford UniversityIncludes Google. We estimate that 539 of the 6,766 people with a known current organization (8%) hold both a university role and a company role, including 20% of people at companies and 13% of people at universities.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Toby Sharp yet. Send us a free-licensed photo.

  1. Toby Sharp - Google Scholar scholar.google.co.uk. Checked 21 October 2026.
  2. THE AI CHICKS — What's Next For Augmented Reality & AI with Google's AR Guru Toby Sharp theaichicks.com. Checked 21 October 2026.
  3. Implementing Decision Trees and Forests on a GPU - Microsoft Research microsoft.com. Checked 21 October 2026.
  4. Geodesic image and video editing - Microsoft Research microsoft.com. Checked 21 October 2026.
  5. Real-Time Human Pose Recognition in Parts from a Single Depth Image - Microsoft Research microsoft.com. Checked 21 October 2026.
  6. Kinect Body Tracking Reaps Renown - Microsoft Research microsoft.com. Checked 21 October 2026.
  7. Microsoft Research Cambridge: Year in Review - Microsoft Research microsoft.com. Checked 21 October 2026.
  8. Accurate, Robust, and Flexible Real-time Hand Tracking - Microsoft Research microsoft.com. Checked 21 October 2026.
  9. Mixed Reality & AI – Cambridge - Microsoft Research: News And Awards microsoft.com. Checked 21 October 2026.
  10. Real-Time Human Pose Recognition in Parts from Single Depth Images [Video] - Microsoft Research microsoft.com. Checked 21 October 2026.