Test-Time Adaptation for Video Frame Interpolation via Meta-Learning

Video frame interpolation is a challenging problem that involves various scenarios depending on the variety of foreground and background motions, frame rate, and occlusion. Therefore, generalizing across different scenes is difficult for a single network with fixed parameters. Ideally, one could hav...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 44(2022), 12 vom: 01. Dez., Seite 9615-9628
1. Verfasser: Choi, Myungsub (VerfasserIn)
Weitere Verfasser: Choi, Janghoon, Baik, Sungyong, Kim, Tae Hyun, Lee, Kyoung Mu
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2022
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM333507150
003 DE-627
005 20231225221805.0
007 cr uuu---uuuuu
008 231225s2022 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2021.3129819  |2 doi 
028 5 2 |a pubmed24n1111.xml 
035 |a (DE-627)NLM333507150 
035 |a (NLM)34813468 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Choi, Myungsub  |e verfasserin  |4 aut 
245 1 0 |a Test-Time Adaptation for Video Frame Interpolation via Meta-Learning 
264 1 |c 2022 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 08.11.2022 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Video frame interpolation is a challenging problem that involves various scenarios depending on the variety of foreground and background motions, frame rate, and occlusion. Therefore, generalizing across different scenes is difficult for a single network with fixed parameters. Ideally, one could have a different network for each scenario, but this will be computationally infeasible for practical applications. In this work, we propose MetaVFI, an adaptive video frame interpolation algorithm that uses additional information readily available at test time but has not been exploited in previous works. We initially show the benefits of test-time adaptation through simple fine-tuning of a network and then greatly improve its efficiency by incorporating meta-learning. Thus, we obtain significant performance gains with only a single gradient update without introducing any additional parameters. Moreover, the proposed MetaVFI algorithm is model-agnostic which can be easily combined with any video frame interpolation network. We show that our adaptive framework greatly improves the performance of baseline video frame interpolation networks on multiple benchmark datasets 
650 4 |a Journal Article 
700 1 |a Choi, Janghoon  |e verfasserin  |4 aut 
700 1 |a Baik, Sungyong  |e verfasserin  |4 aut 
700 1 |a Kim, Tae Hyun  |e verfasserin  |4 aut 
700 1 |a Lee, Kyoung Mu  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 44(2022), 12 vom: 01. Dez., Seite 9615-9628  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:44  |g year:2022  |g number:12  |g day:01  |g month:12  |g pages:9615-9628 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2021.3129819  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 44  |j 2022  |e 12  |b 01  |c 12  |h 9615-9628