Che Zheng
Che Zheng is a researcher whose work spans neural text generation, transformer design, syntax, and multimodal AI. He completed an MS in computational data science at Carnegie Mellon in 2015, with Eric Nyberg as his advisor, and went to Google after graduation. His papers explored how to identify generated text, rethink transformer attention, and infer grammatical structure. In 2024, he coauthored Reka's report on the Core, Flash, and Edge models and Vibe-Eval, a benchmark for multimodal chat.
Timeline
-
2015
A Carnegie Mellon master's
In 2015, he completes an MS in Computational Data Science at Carnegie Mellon under advisor Eric Nyberg, then moves to Google after graduation.
-
2020
Testing generated-text fingerprints
In 2020, he and his coauthors introduce a task for identifying which variant of a neural text-generation model produced a passage.
-
2021
Rethinking transformer attention
In 2021, he coauthors Synthesizer, which tests how much dot-product self-attention contributes to Transformer performance.
-
2021
Inducing syntactic structure
In 2021, he coauthors StructFormer, a model that induces dependency and constituency structure at the same time.
-
By 2024
Coauthoring Reka reports
In 2024, he coauthors Reka's technical report introducing the Core, Flash, and Edge multimodal models. He also coauthors Vibe-Eval, a multimodal chat benchmark with 269 visual-understanding prompts.
Career
- 2015–2023Google
Education
- 2015Carnegie Mellon UniversityMS in Computational Data Science
Most-cited works
- Synthesizer: Rethinking Self-Attention in Transformer Models
- StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language ModelingWith Aaron C. Courville
- Choppy: Cut Transformer for Ranked List Truncation
- Reverse Engineering Configurations of Neural Text Generation Models
- Synthesizer: Rethinking Self-Attention for Transformer Models
Sources and credits
Photo: We have no published photo of Che Zheng yet. Send us a free-licensed photo.
- Che Zheng - Language Technologies Institute - School of Computer Science - Carnegie Mellon University
- Reverse Engineering Configurations of Neural Text Generation Models - ACL Anthology
- Synthesizer: Rethinking Self-Attention for Transformer Models - Proceedings of Machine Learning Research
- StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling - ACL Anthology
- [2404.12387] Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models
- [2405.02287] Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models


