Leveraging Commonsense for Object Localisation in Partial Scenes

Leveraging Commonsense for Object Localisation in Partial Scenes

We propose an end-to-end solution to address the problem of object localisation in partial scenes, where we aim to estimate the position of an object in an unknown area given only a partial 3D scan of the scene. We propose a novel scene representation to facilitate the geometric reasoning, Directed...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 45(2023), 10 vom: 03. Okt., Seite 12038-12049
1. Verfasser: Giuliari, Francesco (VerfasserIn)
Weitere Verfasser: Skenderi, Geri, Cristani, Marco, Bue, Alessio Del, Wang, Yiming
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM356373223
003 DE-627
005 20231226070323.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2023.3272523  |2 doi 
028 5 2 |a pubmed24n1187.xml 
035 |a (DE-627)NLM356373223 
035 |a (NLM)37134033 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Giuliari, Francesco  |e verfasserin  |4 aut 
245 1 0 |a Leveraging Commonsense for Object Localisation in Partial Scenes 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 06.09.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a We propose an end-to-end solution to address the problem of object localisation in partial scenes, where we aim to estimate the position of an object in an unknown area given only a partial 3D scan of the scene. We propose a novel scene representation to facilitate the geometric reasoning, Directed Spatial Commonsense Graph (D-SCG), a spatial scene graph that is enriched with additional concept nodes from a commonsense knowledge base. Specifically, the nodes of D-SCG represent the scene objects and the edges are their relative positions. Each object node is then connected via different commonsense relationships to a set of concept nodes. With the proposed graph-based scene representation, we estimate the unknown position of the target object using a Graph Neural Network that implements a sparse attentional message passing mechanism. The network first predicts the relative positions between the target object and each visible object by learning a rich representation of the objects via aggregating both the object nodes and the concept nodes in D-SCG. These relative positions then are merged to obtain the final position. We evaluate our method using Partial ScanNet, improving the state-of-the-art by 5.9% in terms of the localisation accuracy at a 8x faster training speed 
650 4 |a Journal Article 
700 1 |a Skenderi, Geri  |e verfasserin  |4 aut 
700 1 |a Cristani, Marco  |e verfasserin  |4 aut 
700 1 |a Bue, Alessio Del  |e verfasserin  |4 aut 
700 1 |a Wang, Yiming  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 45(2023), 10 vom: 03. Okt., Seite 12038-12049  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:45  |g year:2023  |g number:10  |g day:03  |g month:10  |g pages:12038-12049 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2023.3272523  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 45  |j 2023  |e 10  |b 03  |c 10  |h 12038-12049