General Greedy De-Bias Learning

doi:10.1109/TPAMI.2023.3240337

	General Greedy De-Bias Learning
	Han, Xinzhe 1,2; Wang, Shuhui 2,3; Su, Chi 4; Huang, Qingming 1,2; Tian, Qi 5
	2023-08-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	45 期号:8 页码:9789-9805
摘要	Neural networks often make predictions relying on the spurious correlations from the datasets rather than the intrinsic properties of the task of interest, facing with sharp degradation on out-of-distribution (OOD) test data. Existing de-bias learning frameworks try to capture specific dataset bias by annotations but they fail to handle complicated OOD scenarios. Others implicitly identify the dataset bias by special design low capability biased models or losses, but they degrade when the training and testing data are from the same distribution. In this paper, we propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and base model. The base model is encouraged to focus on examples that are hard to solve with biased models, thus remaining robust against spurious correlations in the test stage. GGD largely improves models' OOD generalization ability on various tasks, but sometimes over-estimates the bias level and degrades on the in-distribution test. We further re-analyze the ensemble process of GGD and introduce the Curriculum Regularization inspired by curriculum learning, which achieves a good trade-off between in-distribution (ID) and out-of-distribution performance. Extensive experiments on image classification, adversarial question answering, and visual question answering demonstrate the effectiveness of our method. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge. Codes are available at https://github.com/GeraldHan/GGD.
关键词	Task analysis Correlation Training Data models Question answering (information retrieval) Visualization Image classification Curriculum learning dataset biases greedy strategy robust learning
DOI	10.1109/TPAMI.2023.3240337
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2018AAA0102000] ; National Natural Science Foundation of China[62022083] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[61931008] ; National Natural Science Foundation of China[U21B2038] ; Beijing Nova Program[Z201100006820023]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001022958600034
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21320
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Shuhui
作者单位	1.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Peng Cheng Lab, Shenzhen 518066, Peoples R China 4.SmartMore, Beijing 100085, Peoples R China 5.Huawei Cloud & AI, Shenzhen 518129, Guangdong, Peoples R China
推荐引用方式 GB/T 7714	Han, Xinzhe,Wang, Shuhui,Su, Chi,et al. General Greedy De-Bias Learning[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(8):9789-9805.
APA	Han, Xinzhe,Wang, Shuhui,Su, Chi,Huang, Qingming,&Tian, Qi.(2023).General Greedy De-Bias Learning.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(8),9789-9805.
MLA	Han, Xinzhe,et al."General Greedy De-Bias Learning".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.8(2023):9789-9805.