BiVM : Accurate Binarized Neural Network for Efficient Video Matting

Deep neural networks for real-time video matting suffer significant computational limitations on edge devices, hindering their adoption in widespread applications such as online conferences and short-form video production. Binarization emerges as one of the most common compression approaches with co...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 47(2025), 10 vom: 03. Sept., Seite 9250-9265
1. Verfasser: Qin, Haotong (VerfasserIn)
Weitere Verfasser: Liu, Xianglong, Ma, Xudong, Ke, Lei, Zhang, Yulun, Luo, Jie, Magno, Michele
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2025
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000caa a22002652c 4500
001 NLM392372568
003 DE-627
005 20250918232117.0
007 cr uuu---uuuuu
008 250912s2025 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2025.3584928  |2 doi 
028 5 2 |a pubmed25n1573.xml 
035 |a (DE-627)NLM392372568 
035 |a (NLM)40601443 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Qin, Haotong  |e verfasserin  |4 aut 
245 1 0 |a BiVM  |b Accurate Binarized Neural Network for Efficient Video Matting 
264 1 |c 2025 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 18.09.2025 
500 |a published: Print 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Deep neural networks for real-time video matting suffer significant computational limitations on edge devices, hindering their adoption in widespread applications such as online conferences and short-form video production. Binarization emerges as one of the most common compression approaches with compact 1-bit parameters and efficient bitwise operations. However, accuracy and efficiency limitations exist in the binarized video matting network due to its degenerated encoder and redundant decoder. Following a theoretical analysis based on the information bottleneck principle, the limitations are mainly caused by the degradation of prediction-relevant information in the intermediate features and the redundant computation in prediction-irrelevant areas. We present BiVM, an accurate and resource-efficient Binarized neural network for Video Matting. First, we present a series of binarized computation structures with elastic shortcuts and evolvable topologies, enabling the constructed encoder backbone to extract high-quality representations from input videos for accurate prediction. Second, we sparse the intermediate feature of the binarized decoder by masking homogeneous parts, allowing the decoder to focus on representation with diverse details while alleviating the computation burden for efficient inference. Furthermore, we construct a localized binarization-aware mimicking framework with the information-guided strategy, prompting matting-related representation in fullprecision counterparts to be accurately and fully utilized. Comprehensive experiments show that the proposed BiVM surpasses alternative binarized video matting networks, including state-of-the-art (SOTA) binarization methods, by a substantial margin. Moreover, our BiVM achieves significant savings of 14.3x and 21.6x in computation and storage costs, respectively. We also evaluate BiVM on ARM CPU hardware 
650 4 |a Journal Article 
700 1 |a Liu, Xianglong  |e verfasserin  |4 aut 
700 1 |a Ma, Xudong  |e verfasserin  |4 aut 
700 1 |a Ke, Lei  |e verfasserin  |4 aut 
700 1 |a Zhang, Yulun  |e verfasserin  |4 aut 
700 1 |a Luo, Jie  |e verfasserin  |4 aut 
700 1 |a Magno, Michele  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 47(2025), 10 vom: 03. Sept., Seite 9250-9265  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnas 
773 1 8 |g volume:47  |g year:2025  |g number:10  |g day:03  |g month:09  |g pages:9250-9265 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2025.3584928  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 47  |j 2025  |e 10  |b 03  |c 09  |h 9250-9265