Zachary DeVito

Timeline

1. 2011–2014

   Liszt, Terra and a Stanford PhD

   He is first author of the 2011 SC paper on Liszt, a domain-specific language for portable mesh-based PDE solvers written with Pat Hanrahan and others. In 2013, he is first author of the PLDI paper on Terra, a low-level language meta-programmed using Lua. His 2014 Stanford computer science PhD thesis on Terra lists Pat Hanrahan as primary advisor.

2. By 2016

   Postdoc with Pat Hanrahan

   With a recent Stanford PhD and an undergraduate degree from Princeton, he is a postdoc in computer science at Stanford, working with Pat Hanrahan. He builds high-performance domain-specific languages, among them Opt, which turns energy specifications for non-linear least-squares optimizers into GPU implementations, and Ebb, the second generation of Liszt, for physical simulation.

3. 2019

   The PyTorch paper and TorchScript

   In 2019, he is one of the authors of "PyTorch: An Imperative Style, High-Performance Deep Learning Library", published in the NeurIPS 2019 proceedings with Adam Paszke, Sam Gross, Soumith Chintala and others. At the NeurIPS 2019 workshop Program Transformations for ML, he also gives the talk "Optimized execution of PyTorch programs with TorchScript".

4. 2021–2022

   torch.fx program capture

   In 2021, he co-authors torch.fx with James Reed, Horace He, Ansley Ussery and Jason Ansel, all listed with Facebook AI. The paper describes a program capture and transformation library for PyTorch written entirely in Python, and it appears at MLSys in 2022.

5. December 2023

   Explaining PyTorch memory snapshots

   In December 2023, he and Aaron Shi publish "Understanding GPU Memory 1: Visualizing All Allocations over Time" on the PyTorch blog. The post describes the Memory Snapshot tool, which gives a fine-grained GPU memory visualization for debugging GPU out-of-memory errors.

6. July 2024

   The Llama 3 paper

   In July 2024, he is among the authors of "The Llama 3 Herd of Models", which presents language models that natively support multilinguality, coding, reasoning and tool usage. The largest is a dense Transformer with 405B parameters and a context window of up to 128K tokens.

7. 2025

   Reliability of large ML clusters

   In 2025, the HPCA conference includes "Revisiting Reliability in Large-Scale Machine Learning Research Clusters", which he co-authors with colleagues at FAIR at Meta. The paper presents quantitative analysis and operational experience from managing two large, multi-tenant ML clusters.

8. As of October 2026

   WeightBridge for reinforcement learning

   He is at FAIR at Meta, where in September 2026 he co-authors WeightBridge with Xuanlin Jiang, Samuel Hsia, Michael Kuchnik, Minlan Yu and Carole-Jean Wu. The library moves updated weights from trainers to rollout generators in reinforcement learning for LLMs and reduces average GPU stall time by up to 42 times over the state-of-the-art open-source RL framework.