A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning

doi:10.1145/3605548

	A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning
	Zhang, Yuxin 1,2; Tang, Fan 3; Dong, Weiming 1,2; Huang, Haibin 4; Ma, Chongyang 4; Lee, Tong-Yee 5; Xu, Changsheng 1,2
	2023-10-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS
ISSN	0730-0301
卷号	42 期号:5 页码:16
摘要	This work presents Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, that can fit in most existing arbitrary image style transfer models, such as CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches based on deep neural networks typically use second-order statistics to generate the output. However, these hand-crafted features computed from a single image cannot leverage style information sufficiently, which leads to artifacts such as local distortions and style inconsistency. To address these issues, we learn style representation directly from a large number of images based on contrastive learning by considering the relationships between specific styles and the holistic style distribution. Specifically, we present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components: a parallel contrastive learning scheme for style representation and transfer, a domain enhancement (DE) module for effective learning of style distribution, and a generative network for style transfer. Qualitative and quantitative evaluations showthe results of our approach are superior to those obtained via state-of-the-art methods. The code is available at https://github.com/ zyxElsa/CAST_pytorch.
关键词	Arbitrary style transfer contrastive learning style encoding
DOI	10.1145/3605548
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2020AAA0106200] ; National Natural Science Foundation of China[61832016] ; National Natural Science Foundation of China[62102162] ; National Natural Science Foundation of China[U20B2070] ; Beijing Natural Science Foundation[L221013] ; National Science and Technology Council[111-2221-E-006-112-MY3]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001086833300008
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：14[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21096
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Yuxin
作者单位	1.Chinese Acad Sci, Inst Automat, MAIS, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, 6 Kexueyuan South Rd Zhongguancun, Beijing 100190, Peoples R China 4.Kuaishou Technol, 6 Shangdi West Rd, Beijing 100085, Peoples R China 5.Natl Cheng Kung Univ, Dept Comp Sci & Informat Engn, 1 Univ Rd, Tainan 701, Taiwan
推荐引用方式 GB/T 7714	Zhang, Yuxin,Tang, Fan,Dong, Weiming,et al. A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning[J]. ACM TRANSACTIONS ON GRAPHICS,2023,42(5):16.
APA	Zhang, Yuxin.,Tang, Fan.,Dong, Weiming.,Huang, Haibin.,Ma, Chongyang.,...&Xu, Changsheng.(2023).A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning.ACM TRANSACTIONS ON GRAPHICS,42(5),16.
MLA	Zhang, Yuxin,et al."A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning".ACM TRANSACTIONS ON GRAPHICS 42.5(2023):16.