|
|
|
|
| LEADER |
01000naa a22002652 4500 |
| 001 |
NLM359033725 |
| 003 |
DE-627 |
| 005 |
20231226080014.0 |
| 007 |
cr uuu---uuuuu |
| 008 |
231226s2023 xx |||||o 00| ||eng c |
| 024 |
7 |
|
|a 10.1109/TPAMI.2023.3292266
|2 doi
|
| 028 |
5 |
2 |
|a pubmed24n1196.xml
|
| 035 |
|
|
|a (DE-627)NLM359033725
|
| 035 |
|
|
|a (NLM)37402185
|
| 040 |
|
|
|a DE-627
|b ger
|c DE-627
|e rakwb
|
| 041 |
|
|
|a eng
|
| 100 |
1 |
|
|a Xiao, Junbin
|e verfasserin
|4 aut
|
| 245 |
1 |
0 |
|a Contrastive Video Question Answering via Video Graph Transformer
|
| 264 |
|
1 |
|c 2023
|
| 336 |
|
|
|a Text
|b txt
|2 rdacontent
|
| 337 |
|
|
|a ƒaComputermedien
|b c
|2 rdamedia
|
| 338 |
|
|
|a ƒa Online-Ressource
|b cr
|2 rdacarrier
|
| 500 |
|
|
|a Date Revised 04.10.2023
|
| 500 |
|
|
|a published: Print-Electronic
|
| 500 |
|
|
|a Citation Status PubMed-not-MEDLINE
|
| 520 |
|
|
|a We propose to perform video question answering (VideoQA) in a Contrastive manner via a Video Graph Transformer model (CoVGT). CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning. 2) It designs separate video and text transformers for contrastive learning between the video and text to perform QA, instead of multi-modal transformer for answer classification. Fine-grained video-text communication is done by additional cross-modal interaction modules. 3) It is optimized by the joint fully- and self-supervised contrastive objectives between the correct and incorrect answers, as well as the relevant and irrelevant questions respectively. With superior video encoding and QA solution, we show that CoVGT can achieve much better performances than previous arts on video reasoning tasks. Its performances even surpass those models that are pretrained with millions of external data. We further show that CoVGT can also benefit from cross-modal pretraining, yet with orders of magnitude smaller data. The results demonstrate the effectiveness and superiority of CoVGT, and additionally reveal its potential for more data-efficient pretraining
|
| 650 |
|
4 |
|a Journal Article
|
| 700 |
1 |
|
|a Zhou, Pan
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Yao, Angela
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Li, Yicong
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Hong, Richang
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Yan, Shuicheng
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Chua, Tat-Seng
|e verfasserin
|4 aut
|
| 773 |
0 |
8 |
|i Enthalten in
|t IEEE transactions on pattern analysis and machine intelligence
|d 1979
|g 45(2023), 11 vom: 04. Nov., Seite 13265-13280
|w (DE-627)NLM098212257
|x 1939-3539
|7 nnns
|
| 773 |
1 |
8 |
|g volume:45
|g year:2023
|g number:11
|g day:04
|g month:11
|g pages:13265-13280
|
| 856 |
4 |
0 |
|u http://dx.doi.org/10.1109/TPAMI.2023.3292266
|3 Volltext
|
| 912 |
|
|
|a GBV_USEFLAG_A
|
| 912 |
|
|
|a SYSFLAG_A
|
| 912 |
|
|
|a GBV_NLM
|
| 912 |
|
|
|a GBV_ILN_350
|
| 951 |
|
|
|a AR
|
| 952 |
|
|
|d 45
|j 2023
|e 11
|b 04
|c 11
|h 13265-13280
|