PeopleofSuperintelligence

Researcher

Nicole Meister

Nicole Meister is a PhD student at Stanford whose research centers on designing fair evaluations of AI systems, from gender bias in image datasets to contamination in language models. At Princeton she earned a BSE in electrical and computer engineering in 2022, and her senior thesis on gender bias in computer vision datasets won the Calvin Dodd MacCracken Senior Thesis/Project Award. She coauthored Gender Artifacts in Visual Datasets at ICCV 2023, and Proving Test Set Contamination in Black-Box Language Models, written with Yonatan Oren and others, received an honorable mention at ICLR 2024. Her NAACL 2025 paper benchmarked how closely language models match the opinions of demographic groups, and a 2026 NeurIPS oral showed that design choices can flip the conclusions of resume-screening bias studies. She is a final-year PhD student and NSF graduate research fellow at Stanford, advised by Tatsunori Hashimoto and Carlos Guestrin in the Stanford NLP group.

Badges

Timeline

  1. 2021

    Adobe Research internship

    In 2021, she interns at Adobe Research, where she develops a reading tool in Adobe Acrobat for acronym definitions and builds an acronym disambiguation dataset.

  2. 2022

    MacCracken Award at Princeton

    She graduates from Princeton in 2022 with a BSE in electrical and computer engineering, advised by Olga Russakovsky. Her senior thesis on gender bias in computer vision datasets earns the Calvin Dodd MacCracken Senior Thesis/Project Award, and she heads to Stanford for a computer science PhD with a National Science Foundation fellowship.

  3. 2023

    Gender artifacts in visual datasets

    Gender Artifacts in Visual Datasets, which she coauthors with Dora Zhao, Angelina Wang, Vikram V. Ramaswamy, Ruth Fong and Olga Russakovsky, appears at ICCV in 2023. The paper finds that gender artifacts are ubiquitous in the COCO and OpenImages datasets, and that bias mitigation methods that attempt to remove gender remove more information from the scene than the person.

  4. 2024

    An ICLR honorable mention

    Proving Test Set Contamination in Black-Box Language Models, written with Yonatan Oren, Niladri Chatterji, Faisal Ladhak and Tatsunori Hashimoto, is one of the honorable mentions among the ICLR 2024 outstanding papers. Posted to arXiv in October 2023, the paper shows that test set contamination can be proven without access to pretraining data or model weights.

  5. April 2025

    Benchmarking distributional alignment

    In April 2025, her paper Benchmarking Distributional Alignment of Large Language Models, written with Carlos Guestrin and Tatsunori Hashimoto, appears at NAACL in Albuquerque. The paper builds a dataset that goes beyond political values, sets human baselines, and finds that language models can more accurately describe an opinion distribution than simulate it.

  6. 2026

    A NeurIPS oral on evaluation design

    In 2026, she coauthors the NeurIPS oral GenAI Evaluation Results are Largely Artifacts of Evaluation Design Choices with Hanna Wallach, Solon Barocas, Carlos Guestrin, Tatsunori Hashimoto and others. The study analyzes 36 papers on gender and racial bias in generative AI resume screening and runs 1,134 counterfactual experiments.

  7. As of October 2026

    Final-year PhD at Stanford

    She is a final-year PhD student and NSF graduate research fellow at Stanford University, advised by Tatsunori Hashimoto and Carlos Guestrin in the Stanford NLP group. Her research interests are in designing fair evaluations.

Career

  1. 2021AdobeResearch Intern

Education

  1. 2022Princeton UniversityBSE in Electrical and Computer Engineering
  2. From 2022Stanford UniversityPhD in Computer Science

From the biography.

  1. Among employers with 30+ full-time cases, OpenAI leads primary technical role H-1B pay at $310,000, and Amazon, Google and Microsoft file 57% of all casesIncludes Adobe. Across 440,669 certified US H-1B filings from 44 employers (FY2020–FY2026), OpenAI leads primary technical role base pay at $310,000 and xAI leads overall base pay at $290,000.
  2. The company share of AI researchers’ top papers was 31% in 2020–2024 and 8% in 1970–1974Includes Adobe. We estimate where the people shaping AI worked when their most-cited papers appeared: the company share was 31% in 2020–2024 and 8% in 1970–1974, and Google led all organizations in 2020–2024.
  3. Google has the highest combined h-index among AI organizationsIncludes Adobe. We estimate that Google holds the highest combined h-index at 8,016, while universities account for 73% of the deduplicated total across AI organizations.
  4. 20% of people at companies who shape AI also hold a university role, led by Google and Stanford UniversityIncludes Stanford University. We estimate that 539 of the 6,766 people with a known current organization (8%) hold both a university role and a company role, including 20% of people at companies and 13% of people at universities.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Nicole Meister yet. Send us a free-licensed photo.

  1. Nicole Meister nicolemeister.github.io. Checked 22 October 2026.
  2. On Class Day, graduates were told their dedication would be the foundation of future success - Princeton Engineering engineering.princeton.edu. Checked 22 October 2026.
  3. Gender Artifacts in Visual Datasets - ICCV 2023 Open Access Repository openaccess.thecvf.com. Checked 22 October 2026.
  4. ICLR 2024 Outstanding Paper Awards - ICLR Blog blog.iclr.cc. Checked 22 October 2026.
  5. [2310.17623] Proving Test Set Contamination in Black Box Language Models arxiv.org. Checked 22 October 2026.
  6. Benchmarking Distributional Alignment of Large Language Models - ACL Anthology aclanthology.org. Checked 22 October 2026.
  7. NeurIPS Oral GenAI Evaluation Results are Largely Artifacts of Evaluation Design Choices: Evidence from Audit Studies of Resume Screening neurips.cc. Checked 22 October 2026.