Test-Time Deep Thinking to Explore Implicit Rules
Aug 1, 2026·,,,,,,
,·
0 min read
Wentong Chen
Xin Cong
Zhong Zhang
Yaxi Lu
Siyuan Zhao
Yesai Wu
Qinyu Luo
Haotian Chen
Corresponding author
,Yankai Lin
Corresponding author
,Zhiyuan Liu
Maosong Sun
Abstract
LLM agents often fail in environments whose rules are hidden and must be inferred through interaction, leading to repetitive trial and error. We propose TTExplore, in which a thinker model analyzes the interaction history to infer implicit rules and guide an actor, and train a 7B model, Exp-Thinker, with a stable reinforcement-learning pipeline, yielding a 14–19 point average improvement across five text-based embodied tasks.
Type
Publication
In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026)
Authors
Haotian Chen
(he/him)
Research Assistant Professor · LLM Post-Training / AI4AI
I am a Research Assistant Professor at the School of Artificial Intelligence, Shanghai Jiao Tong University, where I work with Prof. Junchi Yan at RethinkLab. I study how to build AI systems that can automate long-horizon, effort-intensive, and creativity-demanding tasks such as research, engineering, and development. My current work focuses on LLM post-training for autonomous agents, agent memory and continual learning, and AI4Research / AI4AI. Before joining SJTU, I received my PhD in Data Science from Fudan University, advised by Prof. Xiangdong Zhou, and completed postdoctoral research at Tsinghua University (THUNLP), working with Prof. Zhiyuan Liu and Prof. Maosong Sun. I was also a research intern at the Machine Learning Research Group of Microsoft Research Asia, mentored by Xiao Yang, and at the Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, working with Prof. Yang Yu.