Co-ViSu: Accelerating Video Super-Resolution With Codec Information Reuse

doi:10.1109/TCAD.2025.3543428

	Co-ViSu: Accelerating Video Super-Resolution With Codec Information Reuse
	Fan, Haishuang 1,2; Sun, Qichu 1,2; Wu, Jingya 1,3; Lu, Wenyan 1,3; Li, Xiaowei 1,2; Yan, Guihai 1,3
	2025-09-01
发表期刊	IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
ISSN	0278-0070
卷号	44 期号:9 页码:3451-3464
摘要	High-resolution (HR) videos have gained popularity with the widespread adoption of high-definition displays. Super-resolution (SR) techniques aim to recover HR frames from low-resolution (LR) frames. While deep neural network (DNN)-based SR methods have outperformed traditional techniques in quality, they face performance challenges. FPGA-based SR accelerators have been developed to optimize the performance and power efficiency. However, most of these accelerators process only uncompressed video frames and perform per-frame DNN inference, overlooking the temporal-spatial information inherent in compressed video bitstreams. We propose a novel compressed video SR workflow that includes a codec information reuse algorithm and a dedicated FPGA accelerator named Co-ViSu. Our approach leverages the observation that non-key frames can be reconstructed using codec information and HR key-frames, significantly reducing DNN computations. The Co-ViSu algorithm employs subpixel interpolation to enhance high-frequency details and an MV-aware method to improve SR reconstruction quality. The Co-ViSu hardware integrates decoder, SR, and encoder engines within a parallel pipeline architecture, utilizing codec information reuse to bypass non-key frame decoding, eliminate complex DNN computations, and accelerate encoding processes. Experimental results demonstrate that Co-ViSu achieves performance improvements ranging from 3.6x to 9.4x and a 4.2x gain in energy efficiency with minimal quality loss compared to traditional flow. Additionally, Co-ViSu offers a 2.1x increase in throughput compared to state-of-the-art solutions.
关键词	Binary sequences Streaming media Decoding Artificial neural networks Superresolution Kernel Engines Design automation Video codecs Throughput Accelerator codec FPGA super-resolution (SR)
DOI	10.1109/TCAD.2025.3543428
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China (NSFC)[62090020] ; National Natural Science Foundation of China (NSFC)[92373206] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB44030100] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB0660100] ; Internship Program of YUSUR Technology Company Ltd
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Interdisciplinary Applications ; Engineering, Electrical & Electronic
WOS记录号	WOS:001563972400030
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41743
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wu, Jingya; Yan, Guihai
作者单位	1.Chinese Acad Sci, Inst Comp Technol, SKLP, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 101408, Peoples R China 3.YUSUR Technol Co Ltd, Beijing 100094, Peoples R China
推荐引用方式 GB/T 7714	Fan, Haishuang,Sun, Qichu,Wu, Jingya,et al. Co-ViSu: Accelerating Video Super-Resolution With Codec Information Reuse[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2025,44(9):3451-3464.
APA	Fan, Haishuang,Sun, Qichu,Wu, Jingya,Lu, Wenyan,Li, Xiaowei,&Yan, Guihai.(2025).Co-ViSu: Accelerating Video Super-Resolution With Codec Information Reuse.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,44(9),3451-3464.
MLA	Fan, Haishuang,et al."Co-ViSu: Accelerating Video Super-Resolution With Codec Information Reuse".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 44.9(2025):3451-3464.