Hao Peng

hao/headshot.jpg

3314 SC

201 North Goodwin Avenue

Urbana, IL 61801

I am an Assistant Professor at the Department of Computer Science of the University of Illinois at Urbana-Champaign (UIUC).

I received my Ph.D. from the University of Washington, with Noah Smith, and my Bachelors Degree from Peking University. I spent one year at the Allen Institute for Artificial Intelligence as a Young Investigator, and time at Microsoft Research, Google, and DeepMind as an intern.

My research interest broadly spans natural language processing and machine learning. My current interests primarily include making language AI more efficient and accessible, and evaluating and improving large language models’ reasoning capabilities, factuality, and trustworthiness, and their applications in the scientific domain.

Outside of work, I cater to the whims of a trio of furry overlords: Meera, Loki, and Sylvie. When they release me from their service, I cycle in the summer, and (backcountry) ski in the winter.

I’m looking for students and interns that are familiar with and interested in building LLMs. If Megatron, DeepSpeed, and training LLMs with hundreds GPUs sound exciting to you, please drop me an email and include Meera+Loki+Sylvie in the subject!

news

Apr 16, 2024 I will give a talk at the Midwest Speech and Language Days at UMich.
Apr 12, 2024 I will give a talk at UChicago and TTIC.
Apr 11, 2024 I will give a talk at the Argonne National Laboratory.
Apr 2, 2024 Check out Eurus, our state-of-the-art opensource LLMs!
Feb 15, 2024 Pretrained LLMs can be adapted to handle 128K-long context with surprisingly small amount of continual pretraining. Check out our new preprint!

recent publications

  1. Advancing LLM Reasoning Generalists with Preference Trees
    Lifan Yuan, Ganqu Cui, Hanbin Wang, Ning Ding, Xingyao Wang, Jia Deng, Boji Shan, Huimin Chen, Ruobing Xie, Yankai Lin, and 5 more authors
    arXiv preprint, 2024
  2. Source-Aware Training Enables Knowledge Attribution in Language Models
    Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, and Hao Peng
    arXiv preprint, 2024
  3. Language Models Hallucinate, but May Excel at Fact Verification
    Jian Guan, Jesse Dodge, David Wadden, Minlie Huang, and Hao Peng
    In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024
  4. LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models
    Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang
    In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024
  5. Executable Code Actions Elicit Better LLM Agents
    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji
    arXiv preprint, 2024
  6. Data Engineering for Scaling Language Models to 128K Context
    Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng
    arXiv preprint, 2024