Hao Peng

3314 SC

201 North Goodwin Avenue

Urbana, IL 61801

I am an Assistant Professor at the Siebel School of Computing and Data Science of the University of Illinois at Urbana-Champaign (UIUC). I received my Ph.D. from the University of Washington, with Noah Smith, and my Bachelors Degree from Peking University.

I build intelligent systems that help make decisions under the uncertainty inherent in real-world problems, especially in scientific discovery and healthcare. Towards that, I currently study LLM post-training, causal reasoning, and agents that learn from accumulated experience.

Outside work, I cater to the whims of a quartet of furry overlords: Meera, Loki, Sylvie, and Kea. When they release me from their service, I cycle in the summer, and (backcountry) ski in the winter.

Undergraduate and master’s students: We love hearing from motivated undergraduate and master’s students! If you’d like to collaborate with us, please start by filling out this form.

news

Sep 30, 2026	Our theoretical characterization of RoPE’s semantic and positional failures leads to RoPE Profiler, which quantifies these weaknesses with little overhead and guides targeted interventions that improve long-context performance without additional training.
Sep 30, 2026	Covert Assistance: helpful agents can share protected credentials while evading oversight, even without adversarial incentives.
Sep 28, 2026	We offer a theoretical explanation for the sparse learning observed in our recent work: on-policy sampling leads to small diagonal Fisher and small expected gradients, connecting sparsity across RL and on-policy distillation.
Sep 28, 2026	Learning from Teacher Continuations at Student States offers an efficient online distillation paradigm that addresses key limitations of and outperforms existing distillation methods.

recent publications

RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures

Yuyang Wu, Yufeng Du, and Hao Peng

2026

paper
Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems

Deema Alnuhait, Gengyu Wang, Muhammad Khalifa, and Hao Peng

2026

paper
Understanding LLM Parameter Update Sparsity through the Lens of Fisher

Yufan Zhang, Sagnik Mukherjee, and Hao Peng

2026

paper
Learning from Teacher Continuations at Student States

Haojin Wang, Dylan Zhang, Huaibo Chen, Suhao Yu, Yihang Sun, Zhanyang Jin, Jiaying Ye, Dianqi Li, Prasanna Sattigeri, Kamal Youcef-Toumi, and 1 more author

2026

paper website
spotlight

Towards a Universal Causal Reasoner

Qirun Dai, Xiao Liu, Jiawei Zhang, Dylan Zhang, Hao Peng, and Chenhao Tan

In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2026

paper
Useful Memories Become Faulty When Continuously Updated by LLMs

Dylan Zhang, Yanshan Lin, Zhengkun Wu, Yihang Sun, Bingxuan Li, Dianqi Li, and Hao Peng

In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2026

paper