Local Correspondence Network for Weakly Supervised Temporal Sentence Grounding
Weakly supervised temporal sentence grounding has better scalability and practicability than fully supervised methods in real-world application scenarios. However, most of existing methods cannot model the fine-grained video-text local correspondences well and do not have effective supervision infor...
| Publié dans: | IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. - 1992. - 30(2021) vom: 01., Seite 3252-3262 |
|---|---|
| Auteur principal: | |
| Autres auteurs: | , , |
| Format: | Article en ligne |
| Langue: | English |
| Publié: |
2021
|
| Accès à la collection: | IEEE transactions on image processing : a publication of the IEEE Signal Processing Society |
| Sujets: | Journal Article |
| Résumé: | Weakly supervised temporal sentence grounding has better scalability and practicability than fully supervised methods in real-world application scenarios. However, most of existing methods cannot model the fine-grained video-text local correspondences well and do not have effective supervision information for correspondence learning, thus yielding unsatisfying performance. To address the above issues, we propose an end-to-end Local Correspondence Network (LCNet) for weakly supervised temporal sentence grounding. The proposed LCNet enjoys several merits. First, we represent video and text features in a hierarchical manner to model the fine-grained video-text correspondences. Second, we design a self-supervised cycle-consistent loss as a learning guidance for video and text matching. To the best of our knowledge, this is the first work to fully explore the fine-grained correspondences between video and text for temporal sentence grounding by using self-supervised learning. Extensive experimental results on two benchmark datasets demonstrate that the proposed LCNet significantly outperforms existing weakly supervised methods |
|---|---|
| Description: | Date Revised 03.03.2021 published: Print-Electronic Citation Status PubMed-not-MEDLINE |
| ISSN: | 1941-0042 |
| DOI: | 10.1109/TIP.2021.3058614 |