Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction
Abstract
We take the first step toward understanding model decision rules in DocRE. Through annotations and RE-specific attacks, we reveal that SOTA models exhibit different decision rules from humans, severely damaging robustness. We introduce MAP to evaluate understanding and reasoning capabilities.
Type
Publication
In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)
Authors
Haotian Chen
(he/him)
Research Assistant Professor
I am a Research Assistant Professor at the School of Artificial Intelligence, Shanghai Jiao Tong University, where I work with Prof. Junchi Yan at RethinkLab. I study how to build AI systems that can automate long-horizon, effort-intensive, and creativity-demanding tasks such as research, engineering, and development. My current work focuses on autonomous agents, large language models, and AI4Research. Before joining SJTU, I received my PhD in Data Science from Fudan University, advised by Prof. Xiangdong Zhou, and completed postdoctoral research at Tsinghua University (THUNLP), working with Prof. Zhiyuan Liu and Prof. Maosong Sun. I was also a research intern at the Machine Learning Research Group of Microsoft Research Asia, mentored by Xiao Yang, and at the Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, working with Prof. Yang Yu.