Hierarchical image-to-image translation with nested distributions modeling

doi:10.1016/j.patcog.2023.110058

	Hierarchical image-to-image translation with nested distributions modeling
	Qiao, Shishi 1,2,3; Wang, Ruiping 2,3; Shan, Shiguang 2,3; Chen, Xilin 2,3
	2024-02-01
发表期刊	PATTERN RECOGNITION
ISSN	0031-3203
卷号	146 页码:12
摘要	Unpaired image-to-image translation among category domains has achieved remarkable success in past decades. Recent studies mainly focus on two challenges. For one thing, such translation is inherently multi-modal (i.e. many-to-many mapping) due to variations of domain-specific information (e.g., the domain of house cat contains multiple sub-modes), which is usually addressed by predefined distribution sampling. For another, most existing multi-modal approaches have limits in handling more than two domains with one model, i.e. they have to independently build two distributions to capture variations for every pair of domains. To address these problems, we propose a Hierarchical Image-to-image Translation (HIT) method which jointly formulates the multi-domain and multi-modal problem in a semantic hierarchy structure by modeling a common and nested distribution space. Specifically, domains have inclusion relationships under a particular hierarchy structure. With the assumption of Gaussian prior for domains, distributions of domains at lower levels capture the local variations of their ancestors at higher levels, leading to the so-called nested distributions. To this end, we propose a nested distribution loss in light of the distribution divergence measurement and information entropy theory to characterize the aforementioned inclusion relations among domain distributions. Experiments on ImageNet, ShapeNet, and CelebA datasets validate the promising results of our HIT against state-of-the-arts, and as additional benefits of nested modeling, one can even control the uncertainty of multi-modal translations at different hierarchy levels.
关键词	Image-to-image translation Distribution modeling Information entropy Generative adversarial network
DOI	10.1016/j.patcog.2023.110058
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2021ZD0111901] ; Natural Science Foundation of China[U21B2025] ; Natural Science Foundation of China[U19B2036] ; Natural Science Foundation of China[62206260]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001102929200001
出版者	ELSEVIER SCI LTD
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38090
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ruiping
作者单位	1.Ocean Univ China, Coll Informat Sci & Engn, Qingdao 266100, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Qiao, Shishi,Wang, Ruiping,Shan, Shiguang,et al. Hierarchical image-to-image translation with nested distributions modeling[J]. PATTERN RECOGNITION,2024,146:12.
APA	Qiao, Shishi,Wang, Ruiping,Shan, Shiguang,&Chen, Xilin.(2024).Hierarchical image-to-image translation with nested distributions modeling.PATTERN RECOGNITION,146,12.
MLA	Qiao, Shishi,et al."Hierarchical image-to-image translation with nested distributions modeling".PATTERN RECOGNITION 146(2024):12.