PeopleofSuperintelligence

Researcher

Jeffrey Pennington

Jeffrey Pennington is a physicist turned machine learning researcher who coauthored GloVe, an unsupervised algorithm for word vectors, and who applies random matrix theory to deep neural networks. After a 2013 Stanford PhD in theoretical particle physics with Lance Dixon at SLAC, he joined Stanford's Natural Language Processing Group as a postdoctoral fellow and published GloVe with Richard Socher and Christopher Manning in 2014. At Google Brain in New York City, his papers from 2017 to 2019 showed how vanilla convolutional networks can train at ten thousand layers and how very wide networks evolve as linear models under gradient descent. GloVe won ACL's Test-of-Time Paper Award in 2024. He works at Google DeepMind on compute-optimal neural scaling laws.

Works
54
Citations
37,423
h-index
21
Badges

Publication numbers from OpenAlex, as of 22 October 2026.

Timeline

  1. 2013

    A physics PhD at Stanford

    He completes a Stanford PhD in theoretical particle physics in 2013, working at the SLAC National Accelerator Laboratory with Lance Dixon in the theoretical physics group. His dissertation, Generalized polylogarithms in perturbative quantum field theory, presents a method for calculating scattering amplitudes from their analytic structure.

  2. 2014

    GloVe word vectors

    As a postdoctoral research fellow in the Stanford Artificial Intelligence Laboratory, he works with Chris Manning in the Natural Language Processing Group on deep learning for language. With Richard Socher and Christopher Manning, he coauthors GloVe: Global Vectors for Word Representation, an unsupervised algorithm for word vectors published at EMNLP in 2014. He prepares the original GloVe release in August 2014.

  3. 2017

    Random matrices at Google Brain

    In 2017, he is a research scientist at Google Brain in New York City and lectures on random matrix theory in Stanford's Theories of Deep Learning course. With Yasaman Bahri, he uses random matrix theory to study the geometry of neural network loss surfaces at ICML 2017. A NIPS 2017 paper with Samuel Schoenholz and Surya Ganguli applies free probability to compute the singular value distribution of a deep network's input-output Jacobian.

  4. 2018

    Training 10,000-layer CNNs

    With Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein and Samuel Schoenholz, he shows in 2018 that vanilla convolutional networks with ten thousand layers or more can train with an appropriate initialization scheme. The paper appears at ICML 2018.

  5. 2019

    Wide networks as linear models

    In 2019, he coauthors Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent, accepted to NeurIPS 2019. The paper shows that in the infinite-width limit, training follows a linear model obtained from the network's first-order Taylor expansion around its initial parameters.

  6. February 2021

    Staff research scientist at Google Brain

    In February 2021, as a Staff Research Scientist at Google Brain, he gives an applied mathematics colloquium at the University of Colorado Boulder on demystifying deep learning through high-dimensional statistics. The talk links an analysis of bias and variance in random feature kernel regression to neural network training through the Neural Tangent Kernel.

  7. 2024

    ACL Test-of-Time award for GloVe

    GloVe, his 2014 paper with Richard Socher and Christopher Manning, receives the Association for Computational Linguistics Test-of-Time Paper Award for 2014 papers in 2024.

  8. By 2025

    Scaling laws at Google DeepMind

    By 2025, he works at Google DeepMind on compute-optimal neural scaling laws with Courtney Paquette, Elliot Paquette and Lechao Xiao, his coauthors on the 2024 paper 4+3 Phases of Compute-Optimal Neural Scaling Laws. In July 2025, his paper with Shikai Qiu, Lechao Xiao, Andrew Gordon Wilson and Atish Agarwala on scaling collapse in compute-optimally trained networks is an oral presentation at ICML 2025.

Career

  1. From 2017Google BrainResearch Scientist, Staff Research Scientist
  2. 2015–2022Google
  3. 2014–2013Stanford UniversityResearcher, Postdoctoral research fellow

Education

  1. 2013Stanford UniversityPhD

From the biography and OpenAlex affiliations (the years of papers that list the organization), so some years are approximate.

Most-cited works

  1. Glove: Global Vectors for Word Representation2014 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 33,575 citationsWith Christopher D. Manning and Richard Socher
  2. Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions2011 · 1,186 citationsWith Andrew Ng, Christopher D. Manning and Richard Socher
  3. Deep Neural Networks as Gaussian Processes2017 · 602 citations
  4. Sensitivity and Generalization in Neural Networks: an Empirical Study2018 · 297 citations
  5. Statistical Mechanics of Deep Learning2019 · Annual Review of Condensed Matter Physics · 237 citations

From OpenAlex, as of 21 October 2026.

  1. Of 943 people shaping AI who worked at Google, 446 work there nowIncludes Jeffrey Pennington. Our estimates of where the 943 people shaping AI who worked at Google and Google DeepMind work now: 446 at Google and 452 across 278 other organizations.
  2. The median h-index is 48 at universities and 14 at frontier AI labsIncludes Jeffrey Pennington. The median h-index is 48 at universities and 14 at frontier AI labs. We compare publishing careers, recent citations, dual appointments and job moves across 2,420 university researchers and 308 frontier-lab researchers.
  3. Google had the most people shaping AI every year from 2014 to 2025Includes Google. Our estimates of which organizations employed the most people shaping AI each year from 1955 to 2025, tracing the shift from university labs to companies.
  4. Google has the highest combined h-index among AI organizationsIncludes Google. We estimate that Google holds the highest combined h-index at 8,016, while universities account for 73% of the deduplicated total across AI organizations.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Jeffrey Pennington yet. Send us a free-licensed photo.

  1. Jeffrey Pennington nlp.stanford.edu. Checked 20 October 2026.
  2. Jeffrey Pennington - The Mathematics Genealogy Project mathgenealogy.org. Checked 20 October 2026.
  3. GloVe: Global Vectors for Word Representation nlp.stanford.edu. Checked 20 October 2026.
  4. Theories of Deep Learning (STATS 385) by stats385 stats385.github.io. Checked 20 October 2026.
  5. Geometry of Neural Network Loss Surfaces via Random Matrix Theory proceedings.mlr.press. Checked 20 October 2026.
  6. [1711.04735] Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice arxiv.org. Checked 20 October 2026.
  7. [1806.05393] Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks arxiv.org. Checked 20 October 2026.
  8. [1902.06720] Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent arxiv.org. Checked 20 October 2026.
  9. APPM Department Colloquium - Jeffrey Pennington | Applied Mathematics | University of Colorado Boulder colorado.edu. Checked 20 October 2026.
  10. ACL Test-of-Time Papers Award Recipients - Admin Wiki aclweb.org. Checked 20 October 2026.
  11. FDS Colloquium: Elliot Paquette (McGill), "High-dimensional Optimization with Applications to Compute-Optimal Neural Scaling Laws" - Yale FDS fds.yale.edu. Checked 20 October 2026.
  12. [2405.15074] 4+3 Phases of Compute-Optimal Neural Scaling Laws arxiv.org. Checked 20 October 2026.
  13. Google at ICML 2025 research.google. Checked 20 October 2026.
  14. [2507.02119] Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks arxiv.org. Checked 20 October 2026.