Johnny Tian-Zheng Wei | 魏天正

Johnny Tian-Zheng Wei
Johnny Tian-Zheng Wei

Pronouns: he/him

Hi! I am an NRC Postdoctoral Research Associate at NIST CAISSI, working on the Applied Systems team led by Stevie Bergman. My interest is in AI measurement, and my work often takes basic concepts from inferential statistics. Previously, I earned my PhD at the University of Southern California, where I was advised by Robin Jia. I earned my B.S. in Mathematics at the University of Massachusetts Amherst. During the slower weeks, you might find me lifting (myself), biking, hiking, reading, writing, and/or learning. For all that I do, I want to do with love.

A recording of my thesis defense is available here.

Email: jtwei [strudel] usc.edu

Links: [Twitter] [Github] [Google Scholar]


Selected publications

Junior collaborators are denoted with ☆, and my mentorship style is described here.
  • Statistically principled measurement of large language models by spiking the training data
    Johnny Tian-Zheng Wei.
    PhD thesis.
    [abstract] [video]

  • Correcting test set contamination by spiking the training data
    Johnny Tian-Zheng Wei*, Jerry Li*☆, Ameya Godbole, and Robin Jia.
    AIMS workshop at COLM 2026.
    [abstract]

  • Hubble: a model suite to advance the study of LLM memorization
    Johnny Tian-Zheng Wei*, Ameya Godbole*, Mohammad Aflah Khan*, Ryan Wang☆, Xiaoyuan Zhu☆, James Flemings, Nitya Kashyap☆, Krishna P. Gummadi, Willie Neiswanger, and Robin Jia.
    ICLR 2026 (Oral).
    [abstract] [bib] [github] [website] [video] [poster] [proposal] [press]

  • Interrogating LLM design under a fair learning doctrine
    Johnny Tian-Zheng Wei*, Maggie Wang*☆, Ameya Godbole, Jonathan H. Choi, and Robin Jia.
    FAccT 2025.
    [abstract] [bib] [video]

  • Proving membership in LLM pretraining data via data watermarks
    Johnny Tian-Zheng Wei*, Ryan Wang*☆, and Robin Jia.
    Findings of ACL 2024.
    [abstract] [github] [bib]

  • Operationalizing content moderation "accuracy" in the Digital Services Act
    Johnny Tian-Zheng Wei, Frederike Zufall, and Robin Jia.
    AIES 2024.
    [abstract] [bib] [github] [impact 1|2] [press]


Miscellany

  • Two books have greatly influenced my perspective on research. They are Geek Heresy by Kentaro Toyama, and The Worlds I See by Li Fei-Fei. Thank you both for sharing your stories.
  • I recommend How to Avoid a Climate Disaster for everyone, especially if you're an undergraduate or younger.
  • This webpage was adapted from Nelson's. Thanks!