PeopleofSuperintelligence

Researcher

Harsh Trivedi

Harsh Trivedi is a research scientist at the Allen Institute for Artificial Intelligence (Ai2) who builds and evaluates AI agents for tasks involving tool use, coding and UI interactions. As a PhD student at Stony Brook University advised by Niranjan Balasubramanian, he worked on reliable multi-step reasoning, publishing the MuSiQue dataset in 2022 and IRCoT, a method that interleaves retrieval with chain-of-thought reasoning, in 2023. His AppWorld benchmark, a simulated world of 9 apps and 457 APIs, won the Best Resource Paper Award at ACL 2024, and in October 2024 he and Balasubramanian were invited to a White House event marking the first allocation of the NAIRR Pilot. Between 2025 and 2026 he coauthored the Olmo 3 open language models, the Terminal-Bench 2.0 benchmark and the MolmoWeb web agents.

Timeline

  1. 2019

    Multee at Stony Brook

    As a PhD student at Stony Brook University advised by Niranjan Balasubramanian, he works on how to evaluate and teach reliable multi-step reasoning. In June 2019, he and his coauthors introduce Multee at NAACL, an architecture that uses pre-trained entailment models for multi-hop question answering.

  2. 2020–2022

    DiRe and the MuSiQue dataset

    In November 2020, he, Niranjan Balasubramanian, Tushar Khot and Ashish Sabharwal publish the EMNLP paper "Is Multihop QA in DiRe Condition?", which formalizes disconnected reasoning and offers a model-agnostic probe for measuring how much a model can cheat through it. In 2022, the four publish MuSiQue in TACL, a multihop dataset of 25K questions with 2 to 4 hops built by composing connected single-hop questions.

  3. July 2023

    IRCoT at ACL 2023

    At ACL in Toronto in July 2023, he introduces IRCoT, a method for multi-step questions that interleaves retrieval with the steps of a chain of thought. With GPT3, IRCoT improves retrieval by up to 21 points and question answering by up to 15 points, and it reduces hallucination.

  4. August 2024

    A Best Resource Paper Award

    AppWorld, his benchmark for interactive coding agents, simulates 9 day-to-day apps operable through 457 APIs and the digital lives of about 100 fictitious users, with 750 agent tasks; GPT4O solves only about 49% of its normal tasks. In August 2024, the paper wins the Best Resource Paper Award at ACL 2024 in Bangkok.

  5. October 2024

    AppWorld at the White House

    In October 2024, he and Balasubramanian are invited to represent Stony Brook at the White House Office of Science and Technology Policy launch event for the first allocation of the NAIRR Pilot. AppWorld is one of 35 projects nationally recognized by the National Science Foundation and the Department of Energy through the pilot.

  6. October–December 2025

    HAL and Olmo 3

    In October 2025, he coauthors the Holistic Agent Leaderboard (HAL) with Sayash Kapoor, Percy Liang, Arvind Narayanan and others, a standardized evaluation harness validated with 21,730 agent rollouts across 9 models and 9 benchmarks. In December 2025, he is a coauthor of Olmo 3, a family of fully open language models at the 7B and 32B parameter scales.

  7. January–July 2026

    Terminal-Bench, MolmoWeb and AppWorld-UL

    In January 2026, he coauthors Terminal-Bench 2.0, a benchmark of 89 hard tasks in computer terminal environments. His April 2026 work on MolmoWeb yields a family of fully open multimodal web agents in 4B and 8B sizes that predict browser actions from webpage screenshots. In July 2026, he coauthors AppWorld-UL with Junzhi Chen and others, a benchmark of 516 tasks built on AppWorld in which agents must interact with users, for example to ask clarification questions.

  8. As of October 2026

    Research scientist at Ai2

    In 2026, he is a research scientist at the Allen Institute for Artificial Intelligence (Ai2), where he builds and evaluates AI agents for complex tasks involving tool use, coding and UI interactions.

  1. Google had the most people shaping AI every year from 2014 to 2025Our estimates of which organizations employed the most people shaping AI from 1961 to 2025, with a year-by-year chart.
  2. The United States employs an estimated 53% of people in our data with a known employment countryOur estimates of where AI careers are concentrated now and how countries’ shares of dated positions changed over time.
  3. 46% of living people in our data with known birth and employment countries work outside their birth countryOur estimate of cross-border careers in AI, from country of birth and the country of each current organization.
  4. More people shaping AI studied at Massachusetts Institute of Technology than at any other institutionOur estimates of where the people shaping AI studied and where their careers led, from education records in cited biographies.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Harsh Trivedi yet. Send us a free-licensed photo.

  1. Harsh Trivedi harshtrivedi.me. Checked 19 October 2026.
  2. Repurposing Entailment for Multi-Hop Question Answering Tasks - ACL Anthology aclanthology.org. Checked 19 October 2026.
  3. Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning - ACL Anthology aclanthology.org. Checked 19 October 2026.
  4. MuSiQue: Multihop Questions via Single-hop Question Composition - ACL Anthology aclanthology.org. Checked 19 October 2026.
  5. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions - ACL Anthology aclanthology.org. Checked 19 October 2026.
  6. AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents - ACL Anthology aclanthology.org. Checked 19 October 2026.
  7. ACL 2024 Awards for LST Contributions! | Universität des Saarlandes uni-saarland.de. Checked 19 October 2026.
  8. Stony Brook Researcher Develops Groundbreaking AI Platform, Earns National Recognition | AI Innovation Institute ai.stonybrook.edu. Checked 19 October 2026.
  9. Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation arxiv.org. Checked 19 October 2026.
  10. Olmo 3 arxiv.org. Checked 19 October 2026.
  11. Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces arxiv.org. Checked 19 October 2026.
  12. MolmoWeb: Open Visual Web Agent and Open Data for the Open Web arxiv.org. Checked 19 October 2026.
  13. AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use arxiv.org. Checked 19 October 2026.