Lifan Yuan 袁立凡

Lifan Yuan 袁立凡

lifan4@illinois.edu

I am a first-year PhD student at University of Illinois Urbana-Champaign, advised by Prof. Hao Peng. Previously, I was fortunate to work with Prof. Zhiyuan Liu at THUNLP and Prof. Heng Ji at UIUC.

My research pursues scalable oversight for LLM (self-)evolution. To this end, I work on:

  • Scalable data synthesis and training algorithms that directly improve LLMs [UltraFeedback, UltraInteract/Eurus].
  • Scalable feedback development that unlocks the ability of LLMs to provide supervision signals for training or guidance at inference [Implicit PRM].
  • Scalable systems that incorporate such feedback to enhance LLMs and, in return, help improve feedback quality [PRIME/Eurus-2].

Currently, I am specifically focused on reasoning as it remains a challenge for both LLMs and humans, thus presenting an opportunity to better unleash the power of scalable oversight.

🔥 News

  • 2025.01: Eurus has been accepted to ICLR.
  • 2025.01: We introduce PRIME, a scalable RL solution for advanced reasoning through implicit process rewards! We also release Eurus-2, which is trained from Qwen2.5-Math-Base to surpass Qwen2.5-Math-Instruct using only 1/10 of the data.
  • 2024.12: We release Implicit PRM, get your model free process rewards without process labels! Together, we also release the SOTA Llama-3.1-8B-based PRMs!
  • 2024.09: NCA has been accepted to NeurIPS and CPO has been accepted to EMNLP.
  • 2024.05: UltraFeedback and CodeAct have been accepted to ICML. UltraFeedback has been perhaps the most popular preference dataset since its release – it has powered 1k+ models on HuggingFace, ranking in the top 5 on HF! CodeAct has also been included as a major part of the star project OpenHands!
  • 2024.04: We release Eurus, a suite of open-source LLM reasoning generalists! Eurus models are powered by UltraInteract preference trees, please check out!

📝 Publications

  • Selected
  • All

* denotes equal contribution


  • Process Reinforcement through Implicit Rewards [Paper][Blog]
    Ganqu Cui*+, Lifan Yuan*+, Zefan Wang+, Hanbin Wang+, Wendi Li+, Bingxiang He+, Yuchen Fan+, Tianyu Yu+, Qixin Xu+, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, Ning Ding*.
    (* denotes project leads; + denotes core authors)
    Preprint
  • Free Process Rewards without Process Labels [Paper]
    Lifan Yuan*, Wendi Li*, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, Hao Peng.
    Preprint
  • Advancing LLM Reasoning Generalists with Preference Trees [Paper]
    Lifan Yuan*, Ganqu Cui*, Hanbin Wang*, Ning Ding, Xingyao Wang, Jia Deng, Boji Shan, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyuan Liu, Maosong Sun.
    ICLR 2025; ICML 2024 Workshop On AI4Math
  • UltraFeedback: Boosting Language Models with High-quality Feedback [Paper]
    Ganqu Cui*, Lifan Yuan*, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, Maosong Sun.
    ICML 2024
  • Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations [Paper]
    Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun.
    NeurIPS 2023 (Datasets and Benchmarks Track)

    Preprints


  • Process Reinforcement through Implicit Rewards [Paper][Blog]
    Ganqu Cui*+, Lifan Yuan*+, Zefan Wang+, Hanbin Wang+, Wendi Li+, Bingxiang He+, Yuchen Fan+, Tianyu Yu+, Qixin Xu+, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, Ning Ding*.
    (* denotes project leads; + denotes core authors)
  • Free Process Rewards without Process Labels [Paper]
    Lifan Yuan*, Wendi Li*, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, Hao Peng.
  • Prudent Silence or Foolish Babble? Examining Large Language Models' Responses to the Unknown [Paper]
    Genglin Liu, Xingyao Wang, Lifan Yuan, Yangyi Chen, Hao Peng.
  • Zero-Shot Generalization during Instruction Tuning: Insights from Similarity and Granularity [Paper]
    Bingxiang He*, Ning Ding*, Cheng Qian*, Jia Deng, Ganqu Cui, Lifan Yuan, Huan-ang Gao, Huimin Chen, Zhiyuan Liu, Maosong Sun.

    • 2025


  • Advancing LLM Reasoning Generalists with Preference Trees [Paper]
    Lifan Yuan*, Ganqu Cui*, Hanbin Wang*, Ning Ding, Xingyao Wang, Jia Deng, Boji Shan, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyuan Liu, Maosong Sun.
    ICLR

    • 2024


  • Noise Contrastive Alignment of Language Models with Explicit Rewards [Paper]
    Huayu Chen, Guande He, Lifan Yuan, Ganqu Cui, Hang Su, Jun Zhu.
    NeurIPS
  • Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [Paper]
    Yiju Guo*, Ganqu Cui*, Lifan Yuan, Ning Ding, Jiexin Wang, Huimin Chen, Bowen Sun, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun.
    EMNLP
  • UltraFeedback: Boosting Language Models with High-quality Feedback [Paper]
    Ganqu Cui*, Lifan Yuan*, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, Maosong Sun.
    ICML
  • Executable Code Actions Elicit Better LLM Agents [Paper]
    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji.
    ICML
  • CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets [Paper]
    Lifan Yuan*, Yangyi Chen*, Xingyao Wang, Yi R. Fung, Hao Peng, Heng Ji.
    ICLR
  • MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback [Paper]
    Xingyao Wang*, Zihan Wang*, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, Heng Ji.
    ICLR

    • 2023


  • Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT [Paper]
    Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu.
    EMNLP
  • Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations [Paper]
    Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun.
    NeurIPS (Datasets and Benchmarks Track)
  • Removing Backdoors in Pre-trained Models by Regularized Continual Pre-training [Paper]
    Biru Zhu*, Ganqu Cui*, Yangyi Chen, Yujia Qin, Lifan Yuan, Chong Fu, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu.
    TACL
  • A Close Look into the Calibration of Pre-trained Language Models [Paper]
    Yangyi Chen*, Lifan Yuan*, Ganqu Cui, Zhiyuan Liu, Heng Ji.
    ACL
  • Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework [Paper]
    Lifan Yuan*, Yichi Zhang*, Yangyi Chen, Wei Wei.
    ACL (Findings)
  • From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework [Paper]
    Yangyi Chen*, Hongcheng Gao*, Ganqu Cui*, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji.
    ACL (Findings)

    • 2022


  • A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks [Paper]
    Ganqu Cui*, Lifan Yuan*, Bingxiang He, Yangyi Chen, Zhiyuan Liu, Maosong Sun.
    NeurIPS (Datasets and Benchmarks Track) (Spotlight)
  • FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition [Paper]
    Lifan Yuan*, Linyi Yang*, Leyang Cui, Wenyang Gao, Yue Zhang.
    COLING (Oral)
  • Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis [Paper]
    Lirong Wu*, Lifan Yuan*, Guojiang Zhao, Haitao Lin, Stan Z. Li.
    IEEE Transactions on Neural Networks and Learning Systems (TNNLS)

📄 Academic Services

2025: ICLR, ICML.

2024: ICLR, ICML, NeurIPS, ARR.

2023: ACL, NeurIPS, EMNLP, ARR.

2022: NeurIPS, EMNLP, ARR.

💬 Invited Talks

  • 2025.02, Process Reinforcement through Implicit Rewards, Google DeepMind.