Hao Peng
3314 SC
201 North Goodwin Avenue
Urbana, IL 61801
I am an Assistant Professor at the Siebel School of Computing and Data Science of the University of Illinois at Urbana-Champaign (UIUC). I received my Ph.D. from the University of Washington, with Noah Smith, and my Bachelors Degree from Peking University.
I’m broadly interested in large language models (LLMs). Recently, I focus on AI towards:
- Solving complex reasoning problems in a generalizable and data-efficient way; I believe that learning from experience (e.g., through reinforcement learning) and insights from human cognition are crucial towards this goal;
- Understanding and reasoning about the world causally;
- Positively impacting society;
- Advancing the frontier of human knowledge and contributing to scientific discovery, which I see as the ultimate demonstration of true generalization beyond the human knowledge they have been trained on.
Outside work, I cater to the whims of a quartet of furry overlords: Meera, Loki, Sylvie, and Kea. When they release me from their service, I cycle in the summer, and (backcountry) ski in the winter.
Undergraduate and master’s students: We love hearing from motivated undergraduate and master’s students! If you’d like to collaborate with us, please start by filling out this form.
news
| May 15, 2026 | In long contexts, RoPE provably loses both its locality bias and consistent token relevance, while increasing its base trades off distinguishing positions against distinguishing tokens—see our new preprint RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably. |
|---|---|
| May 13, 2026 | Continuously consolidating useful experiences into LLM-written memory can eventually make agents worse than using no memory; preserving raw episodes and gating consolidation is more reliable—see our new preprint Useful Memories Become Faulty When Continuously Updated by LLMs. |
| Feb 17, 2026 | Stronger SFT can hurt downstream RL; our PEAR reweighting makes SFT checkpoints better starters for RL and improves post-RL performance—new preprint Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learning. |
| Feb 17, 2026 | Surprisingly, plain SGD matches (or beats) AdamW for RL in LLMs while updating <0.02% of parameters—see our new preprint Do We Need Adam?. |
recent publications
-
- oralDo We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMsIn Proceedings of the International Conference on Machine Learning (ICML), 2026
- Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement LearningIn Proceedings of the International Conference on Machine Learning (ICML), 2026
- RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable EnvironmentsIn Proceedings of the International Conference on Machine Learning (ICML), 2026