PeopleofSuperintelligence

Researcher

Lvmin Zhang

Also known as Lyumin Zhang, lllyasviel

Lvmin Zhang is a computer scientist whose 2017 paper transferred painting styles to anime sketches; his two-stage colorization method appeared at SIGGRAPH Asia in 2018, and he founded Style2Paints Research in 2018. He earned a B.Eng. from Soochow University in 2021, joined Tien-Tsin Wong's lab at CUHK in 2021, and began a computer science Ph.D. with Maneesh Agrawala at Stanford in 2022. ControlNet added spatial guidance to text-to-image diffusion and won ICCV's 2023 Marr Prize; Fooocus streamlined image generation with Stable Diffusion XL. LayerDiffuse enabled transparent images and layers; IC-Light applied diffusion to relighting and was selected for an ICLR 2025 oral, while FramePack packed frame context for long videos and became a NeurIPS 2025 Spotlight. StreamDiffusionV2 won a Best Research Paper Award at MLSys 2026, and as of October 2026 Zhang researches AI-driven computational art in Stanford's computer science program.

Works
43
Citations
5,265
h-index
10
Badges

Publication numbers from OpenAlex, as of 22 October 2026.

Timeline

  1. 2017

    Anime sketch style transfer

    In June 2017, he coauthors a paper with Yi Ji and Xin Lin that applies a painting's style to anime sketches with a residual U-net and an auxiliary classifier GAN.

  2. 2018

    Sketch colorization and Style2Paints

    In 2018, his SIGGRAPH Asia paper with Chengze Li, Tien-Tsin Wong, Yi Ji and Chunping Liu presents a two-stage method for coloring line art. He collaborates with digital painting artists over many years and founds Style2Paints Research in 2018.

  3. 2021–2022

    From Soochow to Stanford

    He receives a B.Eng. from Soochow University in 2021 and joins Tien-Tsin Wong's lab as a research assistant at the Chinese University of Hong Kong in 2021. In 2022, he begins a computer science Ph.D. at Stanford, advised by Maneesh Agrawala.

  4. 2023

    ControlNet and Fooocus

    In February 2023, he posts ControlNet with Anyi Rao and Maneesh Agrawala, an architecture that adds spatial controls such as edges, depth, segmentation and human pose to large pretrained text-to-image diffusion models. Stable Diffusion interfaces such as AUTOMATIC1111 and ComfyUI adopt it, and the paper wins ICCV’s 2023 Marr Prize. In August 2023, he releases Fooocus, an open-source text-to-image program built on Stable Diffusion XL with default settings and prompts that streamline generation.

  5. 2024

    Transparent layers with LayerDiffuse

    In February 2024, he and Maneesh Agrawala post LayerDiffuse, which lets large pretrained latent diffusion models generate transparent images and layers.

  6. 2025

    IC-Light and FramePack

    With Anyi Rao and Maneesh Agrawala, he presents IC-Light, a diffusion-based training method for image relighting; IC-Light is selected for an oral presentation at ICLR 2025. In April 2025, he and Shengqu Cai, Muyang Li, Gordon Wetzstein and Agrawala post FramePack, a next-frame prediction structure that compresses input-frame context by importance so more frames fit a fixed context length. It enables video generation with thousands of frames on a laptop GPU with 6GB of memory, and its NeurIPS 2025 paper receives a Spotlight.

  7. 2026

    StreamDiffusionV2 at MLSys

    During Summer 2026, he serves as a course assistant for Stanford’s CS148. StreamDiffusionV2, a training-free pipeline for interactive video generation that he coauthors with Chenfeng Xu, Song Han, Maneesh Agrawala and others, wins a Best Research Paper Award at MLSys 2026.

  8. As of October 2026

    AI research at Stanford

    As of October 2026, he is a Ph.D. candidate in Stanford’s computer science program and researches AI-driven computational art.

Career

  1. 2026Stanford UniversityCourse Assistant
  2. 2022Chinese University of Hong KongResearch Assistant
  3. Style2Paints ResearchFounder

Education

  1. From 2022Stanford UniversityPh.D.
  2. 2021Soochow UniversityB.Eng.

From the biography and OpenAlex affiliations (the years of papers that list the organization), so some years are approximate.

Most-cited works

  1. Adding Conditional Control to Text-to-Image Diffusion Models2023 · IEEE/CVF International Conference on Computer Vision (ICCV) · 4,487 citationsWith Maneesh Agrawala
  2. Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN2017 · IAPR Asian Conference on Pattern Recognition (ACPR) · 180 citations
  3. Two-stage sketch colorization2018 · ACM Transactions on Graphics · 175 citations
  4. User-Guided Line Art Flat Filling with Split Filling Mechanism2021 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition · 68 citations
  5. Transparent Image Layer Diffusion using Latent Transparency2024 · ACM Transactions on Graphics · 47 citationsWith Maneesh Agrawala

From OpenAlex, as of 21 October 2026.

  1. The Massachusetts Institute of Technology trained or employed 44 of the 187 AI major-prize winners, more than any other universityIncludes Lvmin Zhang. The Massachusetts Institute of Technology trained or employed 44 of the 187 AI major-prize winners, more than any other university. We compare doctorate and 5+ year employment links, career timing and current workplaces for 187 major-prize and 1,111 best-paper winners.
  2. 20% of people at companies who shape AI also hold a university role, led by Google and Stanford UniversityIncludes Stanford University. We estimate that 539 of the 6,766 people with a known current organization (8%) hold both a university role and a company role, including 20% of people at companies and 13% of people at universities.
  3. Google had the most people shaping AI every year from 2014 to 2025Includes Stanford University. Our estimates of which organizations employed the most people shaping AI each year from 1955 to 2025, tracing the shift from university labs to companies.
  4. More people shaping AI studied at Massachusetts Institute of Technology than at any other institutionIncludes Stanford University. We estimate where the people shaping AI studied and where their careers led. Massachusetts Institute of Technology leads with 467 former students, including 291 recorded doctorates.

Colleagues

People named in this biography, and people with dated positions at the same organization in the same years.

Sources and credits

Photo: We have no published photo of Lvmin Zhang yet. Send us a free-licensed photo.

  1. [1706.03319] Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN arxiv.org. Checked 21 October 2026.
  2. Two-stage Sketch Colorization ttwong12.github.io. Checked 21 October 2026.
  3. Style2Paints Research (S2PR) lllyasviel.github.io. Checked 21 October 2026.
  4. Lvmin Zhang lllyasviel.github.io. Checked 21 October 2026.
  5. [2302.05543] Adding Conditional Control to Text-to-Image Diffusion Models arxiv.org. Checked 21 October 2026.
  6. Fooocus - Wikipedia en.wikipedia.org. Checked 21 October 2026.
  7. [2402.17113] Transparent Image Layer Diffusion using Latent Transparency arxiv.org. Checked 21 October 2026.
  8. Scaling In-the-Wild Training for Diffusion-based Illumination Harmonization and Editing by Imposing Consistent Light Transport proceedings.iclr.cc. Checked 21 October 2026.
  9. Anyi Rao anyirao.com. Checked 21 October 2026.
  10. [2504.12626] Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models arxiv.org. Checked 21 October 2026.
  11. FramePack lllyasviel.github.io. Checked 21 October 2026.
  12. Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models proceedings.neurips.cc. Checked 21 October 2026.
  13. CS 148: Introduction to Computer Graphics and Imaging www-leland.stanford.edu. Checked 21 October 2026.
  14. GitHub - chenfengxu714/StreamDiffusionV2: StreamDiffusion, Live Stream APP github.com. Checked 21 October 2026.
  15. StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation proceedings.mlsys.org. Checked 21 October 2026.
  16. Spotlight: Lvmin Zhang | Stanford University School of Engineering engineering.stanford.edu. Checked 21 October 2026.