Semi-Dense Feature Matching With Transformers and its Applications in Multiple-View Geometry

We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 45(2023), 6 vom: 21. Juni, Seite 7726-7738
1. Verfasser: Shen, Zehong (VerfasserIn)
Weitere Verfasser: Sun, Jiaming, Wang, Yuang, He, Xingyi, Bao, Hujun, Zhou, Xiaowei
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM349229910
003 DE-627
005 20231226042129.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2022.3223530  |2 doi 
028 5 2 |a pubmed24n1164.xml 
035 |a (DE-627)NLM349229910 
035 |a (NLM)36409815 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Shen, Zehong  |e verfasserin  |4 aut 
245 1 0 |a Semi-Dense Feature Matching With Transformers and its Applications in Multiple-View Geometry 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 07.05.2023 
500 |a Date Revised 07.05.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The global receptive field provided by Transformer enables our method to produce dense matches in low-texture areas, where feature detectors usually struggle to produce repeatable interest points. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin. We further adapt LoFTR to modern SfM systems and illustrate its application in multiple-view geometry. The proposed method demonstrates superior performance in Image Matching Challenge 2021 and ranks first on two public benchmarks of visual localization among the published methods. The code is available at https://zju3dv.github.io/loftr 
650 4 |a Journal Article 
700 1 |a Sun, Jiaming  |e verfasserin  |4 aut 
700 1 |a Wang, Yuang  |e verfasserin  |4 aut 
700 1 |a He, Xingyi  |e verfasserin  |4 aut 
700 1 |a Bao, Hujun  |e verfasserin  |4 aut 
700 1 |a Zhou, Xiaowei  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 45(2023), 6 vom: 21. Juni, Seite 7726-7738  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:45  |g year:2023  |g number:6  |g day:21  |g month:06  |g pages:7726-7738 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2022.3223530  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 45  |j 2023  |e 6  |b 21  |c 06  |h 7726-7738