Publications
2026
- EMNLPKun Liang, Clive Bai, Xin Xu, Chenming Tang, Sanwoo Lee, Weijie Liu, Saiyong Yang, and Yunfang WuFindings of EMNLP, Oct 2026
Recent Large Reasoning Models (LRMs) achieve strong performance by leveraging long-form Chain-of-Thought (CoT) reasoning, but uniformly applying overlong reasoning at inference time incurs substantial and often unnecessary computational cost. To address this, prior work explores various strategies to infer an appropriate reasoning budget from the input. However, such approaches are unreliable in the worst case, as estimating the minimal required reasoning effort is fundamentally difficult, and they implicitly fix the trade-off between reasoning cost and accuracy during training, limiting flexibility under varying deployment scenarios. Motivated by these limitations, we propose ORBIT, a controllable multi-budget reasoning framework with well-separated reasoning modes triggered by input. ORBIT employs multi-stage reinforcement learning to discover Pareto-optimal reasoning behaviors at each effort, followed by on-policy distillation to fuse these behaviors into a single unified model. Experiments show that ORBIT achieves (1) controllable reasoning behavior over multiple modes, (2) competitive reasoning density within each mode, and (3) integration of these frontier policies into a single unified student model while preserving clear mode separation and high per-mode performance.
@inproceedings{liang2026orbit, title = {ORBIT: On-policy Exploration-Exploitation for Controllable Multi-Budget Reasoning}, author = {Liang, Kun and Bai, Clive and Xu, Xin and Tang, Chenming and Lee, Sanwoo and Liu, Weijie and Yang, Saiyong and Wu, Yunfang}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026}, year = {2026}, month = oct, } - EMNLPKun Liang, Chenming Tang, Clive Bai, Weijie Liu, Saiyong Yang, and Yunfang WuEMNLP, Oct 2026
On-policy distillation (OPD) transfers reasoning behavior by training a student on teacher feedback along student-generated trajectories, but standard full-rollout training ties every update to a costly completion and can over-allocate supervision to late positions with low marginal value for the current student. We revisit this assumption through the useful supervision horizon: student-induced rollouts can drift from teacher-preferred continuations, while aligned prefixes may already preserve the long-horizon OPD update direction. We propose ADWIN, an adaptive-window framework for OPD that treats rollout length as an online admissibility decision, training on short teacher-anchored prefixes while using delayed full-rollout probes to audit prefix–full alignment and adapt the next horizon with staleness control. Across math and code reasoning benchmarks in single-task, multi-task, and strong-to-weak settings, ADWIN improves the accuracy–compute trade-off over full-rollout OPD and prefix-based baselines, reducing end-to-end training cost by up to 4.1 times while achieving comparable or better accuracy.
@inproceedings{liang2026adwin, title = {ADWIN: Adaptive Windows for Horizon-Aware On-Policy Distillation}, author = {Liang, Kun and Tang, Chenming and Bai, Clive and Liu, Weijie and Yang, Saiyong and Wu, Yunfang}, booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing}, year = {2026}, month = oct, } - PreprintKun Liang, Chenming Tang, Clive Bai, Weijie Liu, Zeyuan Liu, Qingyang Zhang, Saiyong Yang, and Yunfang WuπPPO · arXiv preprint, Sep 2026
Assigning credit to intermediate steps remains a central challenge in training Large Language Models (LLMs) on multi-step reasoning tasks with sparse terminal rewards, and actor-critic methods such as PPO address this by learning value functions to construct token-level advantages. Their effectiveness, however, hinges on reliable value estimation, a difficult task requiring the critic to both assess progress toward a correct solution and anticipate an evolving policy’s future behavior; errors in either can compromise credit assignment and destabilize online training. In this paper, we revisit the standard state-only formulation of value estimation and propose πPPO, a self-privileged actor-critic framework. By reusing verified same-prompt rollouts as contrastive evidence, πPPO helps the critic assess intermediate reasoning against successful and failed attempts, while preserving standard policy optimization and the deployment interface. Experiments show that πPPO consistently improves value-estimation quality by a substantial margin and outperforms representative actor-critic and critic-free RLVR baselines on challenging mathematical reasoning benchmarks, while remaining effective even when paired with substantially smaller asymmetric critics.
@misc{liang2026pippo, title = {Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR}, author = {Liang, Kun and Tang, Chenming and Bai, Clive and Liu, Weijie and Liu, Zeyuan and Zhang, Qingyang and Yang, Saiyong and Wu, Yunfang}, month = sep, year = {2026}, archiveprefix = {arXiv}, primaryclass = {cs.LG}, url = {https://arxiv.org/abs/2609.37825}, } - ICMLZhaoxi Zhang, Yitong Duan, Yanzhi Zhang, Yiming Xu, Zhixiang Wang, Kun Liang, Weikang Li, Jiahui Liang, Deguo Xia, Jizhou Huang, Jiyan He, Shuxin Zheng, and Yunfang WuICML, Jul 2026
Locating files and functions requiring modification in large software repositories is challenging due to their scale and structural complexity. Existing LLM-based methods typically treat this as a repository-level retrieval task and rely on multiple auxiliary tools, which often overlook code execution logic and complicate model control. We propose RepoNavigator, an LLM agent equipped with a single execution-aware tool: jumping to the definition of an invoked symbol. This unified design reflects the actual flow of code execution while simplifying tool manipulation. RepoNavigator is trained end-to-end via Reinforcement Learning (RL) directly from a base pretrained model, without relying on closed-source distillation. Experiments demonstrate that RL-trained RepoNavigator achieves state-of-the-art performance, with the 7B model outperforming 14B baselines, the 14B model surpassing 32B competitors, and the 32B model exceeding closed-source models such as GPT-5 on most metrics. These results confirm that integrating a single, structurally grounded tool with RL training provides an efficient and scalable solution for repository-level issue localization.
@inproceedings{pmlr-v306-zhang26an, title = {One Tool Is Enough: Reinforcement Learning of LLM Agents for Repository-Level Code Navigation}, author = {Zhang, Zhaoxi and Duan, Yitong and Zhang, Yanzhi and Xu, Yiming and Wang, Zhixiang and Liang, Kun and Li, Weikang and Liang, Jiahui and Xia, Deguo and Huang, Jizhou and He, Jiyan and Zheng, Shuxin and Wu, Yunfang}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {155168--155194}, year = {2026}, month = jul, volume = {306}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, url = {https://proceedings.mlr.press/v306/zhang26an.html}, }
2025
- EMNLPSanwoo Lee, Kun Liang, and Yunfang WuEMNLP, Nov 2025 (Oral presentation)
Recent advances in cross-prompt automated essay scoring typically train models jointly on all available source domains, often requiring simultaneous access to unlabeled target domain samples. However, using all sources can lead to suboptimal transfer and high computational cost. Moreover, repeatedly accessing the source essays for continual adaptation raises privacy concerns. We propose a source-free adaptation approach that selectively merges the parameters of individually trained source models without further access to the source datasets. In particular, we mix the task vectors—the parameter updates from fine-tuning—via a weighted sum to efficiently simulate selective joint-training. We use Bayesian optimization to determine the mixing weights using our proposed Prior-encoded Information Maximization (PIM), an unsupervised objective which promotes score discriminability by leveraging useful priors pre-computed from the sources. Experimental results with LLMs on in-dataset and cross-dataset adaptation show that our method (1) consistently outperforms joint-training on all sources, (2) maintains superior robustness compared to other merging methods, (3) excels under severe distribution shifts where recent leading cross-prompt methods struggle, all while retaining computational efficiency.
@inproceedings{lee-etal-2025-composable, title = {Composable Cross-prompt Essay Scoring by Merging Models}, author = {Lee, Sanwoo and Liang, Kun and Wu, Yunfang}, booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing}, month = nov, year = {2025}, address = {Suzhou, China}, publisher = {Association for Computational Linguistics}, pages = {24386--24400}, doi = {10.18653/v1/2025.emnlp-main.1240}, url = {https://aclanthology.org/2025.emnlp-main.1240/}, }