PeopleofSuperintelligence

Researcher

Jiasen Lu

Jiasen Lu is a computer vision researcher whose work links images and language, from visual question answering to open multimodal models. With Devi Parikh and Dhruv Batra, he coauthored the 2015 VQA paper, and in 2019 he introduced ViLBERT, a model that extended BERT to learn joint representations of images and text. After a computer science PhD at Georgia Tech, he joined the Allen Institute for AI in February 2020, where he led Unified-IO, Unified-IO 2 and Molmo. Those models grew from vision and language tasks to audio and action, and Molmo's open datasets were collected without help from other vision-language models. At Apple, where he is a staff research scientist, he coauthored the SlowFast-LLaVA-1.5 video language models in 2025.

Now at
Apple

Timeline

  1. 2015

    The VQA dataset

    He is one of three equal-contribution lead authors of "VQA: Visual Question Answering", presented at ICCV 2015 with Aishwarya Agrawal, Stanislaw Antol, Margaret Mitchell, C. Lawrence Zitnick, Dhruv Batra and Devi Parikh. The paper proposes free-form, open-ended questions about images and releases a dataset of about 250,000 images, 760,000 questions and 10 million answers.

  2. 2016–2018

    Co-attention and grounded captions

    At NIPS 2016, he presents a hierarchical co-attention model for visual question answering that reasons jointly about image regions and question words. With Caiming Xiong, Devi Parikh and Richard Socher, he introduces an adaptive attention model with a visual sentinel for image captioning at CVPR 2017. His Neural Baby Talk model, shown at CVPR 2018, writes captions explicitly grounded in entities that object detectors find in the image.

  3. 2019

    ViLBERT and 12-in-1

    With Dhruv Batra, Devi Parikh and Stefan Lee, he presents ViLBERT at NeurIPS 2019, extending BERT into a two-stream model that learns joint image and text representations and transfers them to four vision-and-language tasks. In December 2019, he and Vedanuj Goswami post 12-in-1, a single model trained on 12 datasets that cuts about 3 billion parameters of separate single-task models to 270 million.

  4. February 2020

    Georgia Tech PhD and Ai2

    He earns a PhD in computer science at Georgia Tech's School of Interactive Computing with the thesis "Visually Grounded Language Understanding and Generation", supervised by Devi Parikh and working closely with Dhruv Batra. In February 2020, he joins the Allen Institute for AI in Seattle.

  5. 2022

    Leading Unified-IO at Ai2

    On Ai2's Prior team, he leads Unified-IO, posted in June 2022 with Christopher Clark, Rowan Zellers, Roozbeh Mottaghi and Aniruddha Kembhavi. The model turns every input and output into discrete tokens so that one transformer trains on over 90 datasets, and it becomes the first model to perform all 7 tasks on the GRIT benchmark.

  6. 2023–2024

    Unified-IO 2

    In 2023, he, Christopher Clark, Sangho Lee and Zichen Zhang lead Unified-IO 2 at the Allen Institute for AI, the first autoregressive multimodal model that understands and generates images, text, audio and action. The team releases all its models, and the paper appears at CVPR 2024 in Seattle.

  7. September 2024

    Molmo and PixMo

    In September 2024, he coauthors Molmo, a family of open vision-language models that he helps lead at Ai2. Its PixMo datasets of detailed captions, free-form image questions and 2D pointing are collected without external vision-language models, and the 72B model outperforms Claude 3.5 Sonnet and Gemini 1.5 Pro.

  8. As of October 2026

    Staff research scientist at Apple

    He is a staff research scientist at Apple. In August 2025, Apple Machine Learning Research publishes SlowFast-LLaVA-1.5, a COLM paper he coauthors with Mingze Xu, Mingfei Gao and others on token-efficient video language models built mainly at 1B and 3B scales for mobile use.

Career

  1. NowAppleStaff research scientist
  2. From 2020Allen Institute for AI (Ai2)

Education

  1. 2020Georgia Institute of TechnologyPhD

From the biography.

  1. Of 943 people shaping AI who worked at Google, 446 work there nowIncludes Apple. Our estimates of where the 943 people shaping AI who worked at Google and Google DeepMind work now: 446 at Google and 452 across 278 other organizations.
  2. Google alumni have founded or led 133 other organizations, more than any other AI labIncludes Apple. We estimate that alumni of 17 major AI labs went on to found or lead 350 other organizations, including 226 they founded, and we trace 472 career moves directly between 23 labs since 2010.
  3. Among employers with 30+ full-time cases, OpenAI leads primary technical role H-1B pay at $310,000, and Amazon, Google and Microsoft file 57% of all casesIncludes Apple. Across 440,669 certified US H-1B filings from 44 employers (FY2020–FY2026), OpenAI leads primary technical role base pay at $310,000 and xAI leads overall base pay at $290,000.
  4. The top 10 organizations employed 22% of people shaping AI in 2025: 51% of company researchers and 21% of university researchersIncludes Apple. We estimate the share of people shaping AI who worked at the top 10 organizations each year from 1955 to 2025, comparing concentration across companies, universities, and institutes and labs.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Jiasen Lu yet. Send us a free-licensed photo.

  1. [1505.00468] VQA: Visual Question Answering arxiv.org. Checked 20 October 2026.
  2. [1606.00061] Hierarchical Question-Image Co-Attention for Visual Question Answering arxiv.org. Checked 20 October 2026.
  3. [1612.01887] Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning arxiv.org. Checked 20 October 2026.
  4. [1803.09845] Neural Baby Talk arxiv.org. Checked 20 October 2026.
  5. ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks proceedings.neurips.cc. Checked 20 October 2026.
  6. [1912.02315] 12-in-1: Multi-Task Vision and Language Representation Learning arxiv.org. Checked 20 October 2026.
  7. Welcome to My Homepage! - Jiasen Lu's Homepage jiasenlu.github.io. Checked 20 October 2026.
  8. [2206.08916] Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks arxiv.org. Checked 20 October 2026.
  9. Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action unified-io-2.allenai.org. Checked 20 October 2026.
  10. Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action - Illinois Experts experts.illinois.edu. Checked 20 October 2026.
  11. [2409.17146] Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models arxiv.org. Checked 20 October 2026.
  12. SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understanding - Apple Machine Learning Research machinelearning.apple.com. Checked 20 October 2026.