RePC: A Novel Neural Video Quality Enhancement System Framework for ABR Streaming of VBR-encoded Videos

doi:10.1145/3727879

	RePC: A Novel Neural Video Quality Enhancement System Framework for ABR Streaming of VBR-encoded Videos
	Shi, Mengyu 1,2; Wang, Miao 1,2; Zhang, Yujun 1,2,3
	2025-05-01
发表期刊	ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
ISSN	1551-6857
卷号	21 期号:5 页码:22
摘要	With the emergence of next-generation video applications and increasing spatial resolutions, delivering highquality video is still limited by network bandwidth. Adaptive bitrate (ABR) can select the appropriate bitrate for video streaming based on bandwidth, which can mitigate rebuffering caused by insufficient bandwidth. In comparison to Constant Bitrate (CBR), Variable Bitrate's (VBR) encoding scheme can achieve the same quality with less bandwidth consumption and is gradually being widely used in ABR streaming. However, the quality of the video is still degraded due to a poor network. Recent research utilizes Super-resolution (SR) in ABR streaming to construct neural Video Quality Enhancement (VQE) systems, thereby improving the quality of video segments downloaded due to insufficient bandwidth. However, SR cannot participate in the downsampling encoding process of videos, which results in the effectiveness of existing SR-based VQE systems being inherently limited due to unavoidable information loss during downsampling encoding. Concurrently, SR's high computational cost restricts neural VQE systems' deployment on clients without GPUs. In contrast to the unidirectional workflow of SR, Rescaling can be integrated into the downsampling encoding process of videos, allowing favorable information to be retained for VQE. To implement high-quality real-time VQE for ABR streaming of VBR-encoded videos on CPUs, we propose RePC, a novel neural VQE system framework for optimizing existing neural VQE systems based on Rescaling (Re) for the first time, and Patch Content-awareness (PC). In detail, RePC uses Rescaling instead of SR to achieve better VQE by participating in the video downsampling. We also propose a Video Single-Image Rescaling model, VSIR, to indicate the effectiveness of RePC in quality enhancement. To speed up VQE, RePC designs a PC algorithm to mix interpolation and neural computation based on the practical upsampling ability. Our evaluation results demonstrate quality gains of 0.55-2.96 dB in PSNR and 1.79-3.18 in VMAF with fewer parameters, a speed-up of 15x-286x well up to real-time requirements on CPUs, and Quality of Experience (QoE) improvements of 16.58-26.65 are also achieved in an ABR system under various networking conditions.
关键词	Video Streaming Adaptive Bitrate Variable Bitrate Video Quality Enhancement Super-resolution Rescaling Content-awareness
DOI	10.1145/3727879
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[U24B6012] ; National Natural Science Foundation of China[U2333201] ; National Natural Science Foundation of China[62372429] ; Innovation Funding of ICT, CAS[E461040] ; Pilot for Major Scientific Research Facility of Jiangsu Province of China[BM2021800]
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS记录号	WOS:001520936400001
出版者	ASSOC COMPUTING MACHINERY
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42298
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Yujun
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Nanjing Inst InforSuperBahn, Nanjing, Peoples R China
推荐引用方式 GB/T 7714	Shi, Mengyu,Wang, Miao,Zhang, Yujun. RePC: A Novel Neural Video Quality Enhancement System Framework for ABR Streaming of VBR-encoded Videos[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2025,21(5):22.
APA	Shi, Mengyu,Wang, Miao,&Zhang, Yujun.(2025).RePC: A Novel Neural Video Quality Enhancement System Framework for ABR Streaming of VBR-encoded Videos.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,21(5),22.
MLA	Shi, Mengyu,et al."RePC: A Novel Neural Video Quality Enhancement System Framework for ABR Streaming of VBR-encoded Videos".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 21.5(2025):22.