Conditional Feature Learning Based Transformer for Text-Based Person Search

Text-based person search aims at retrieving the target person in an image gallery using a descriptive sentence of that person. The core of this task is to calculate a similarity score between the pedestrian image and description, which requires inferring the complex latent correspondence between ima...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. - 1992. - 31(2022) vom: 14., Seite 6097-6108
1. Verfasser: Gao, Chenyang (VerfasserIn)
Weitere Verfasser: Cai, Guanyu, Jiang, Xinyang, Zheng, Feng, Zhang, Jun, Gong, Yifei, Lin, Fangzhou, Sun, Xing, Bai, Xiang
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2022
Zugriff auf das übergeordnete Werk:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM346202574
003 DE-627
005 20231226030844.0
007 cr uuu---uuuuu
008 231226s2022 xx |||||o 00| ||eng c
024 7 |a 10.1109/TIP.2022.3205216  |2 doi 
028 5 2 |a pubmed24n1153.xml 
035 |a (DE-627)NLM346202574 
035 |a (NLM)36103442 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Gao, Chenyang  |e verfasserin  |4 aut 
245 1 0 |a Conditional Feature Learning Based Transformer for Text-Based Person Search 
264 1 |c 2022 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 26.09.2022 
500 |a Date Revised 26.09.2022 
500 |a published: Print-Electronic 
500 |a Citation Status MEDLINE 
520 |a Text-based person search aims at retrieving the target person in an image gallery using a descriptive sentence of that person. The core of this task is to calculate a similarity score between the pedestrian image and description, which requires inferring the complex latent correspondence between image sub-regions and textual phrases at different scales. Transformer is an intuitive way to model the complex alignment by its self-attention mechanism. Most previous Transformer-based methods simply concatenate image region features and text features as input and learn a cross-modal representation in a brute force manner. Such weakly supervised learning approaches fail to explicitly build alignment between image region features and text features, causing an inferior feature distribution. In this paper, we present CFLT, Conditional Feature Learning based Transformer. It maps the sub-regions and phrases into a unified latent space and explicitly aligns them by constructing conditional embeddings where the feature of data from one modality is dynamically adjusted based on the data from the other modality. The output of our CFLT is a set of similarity scores for each sub-region or phrase rather than a cross-modal representation. Furthermore, we propose a simple and effective multi-modal re-ranking method named Re-ranking scheme by Visual Conditional Feature (RVCF). Benefit from the visual conditional feature and better feature distribution in our CFLT, the proposed RVCF achieves significant performance improvement. Experimental results show that our CFLT outperforms the state-of-the-art methods by 7.03% in terms of top-1 accuracy and 5.01% in terms of top-5 accuracy on the text-based person search dataset 
650 4 |a Journal Article 
700 1 |a Cai, Guanyu  |e verfasserin  |4 aut 
700 1 |a Jiang, Xinyang  |e verfasserin  |4 aut 
700 1 |a Zheng, Feng  |e verfasserin  |4 aut 
700 1 |a Zhang, Jun  |e verfasserin  |4 aut 
700 1 |a Gong, Yifei  |e verfasserin  |4 aut 
700 1 |a Lin, Fangzhou  |e verfasserin  |4 aut 
700 1 |a Sun, Xing  |e verfasserin  |4 aut 
700 1 |a Bai, Xiang  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on image processing : a publication of the IEEE Signal Processing Society  |d 1992  |g 31(2022) vom: 14., Seite 6097-6108  |w (DE-627)NLM09821456X  |x 1941-0042  |7 nnns 
773 1 8 |g volume:31  |g year:2022  |g day:14  |g pages:6097-6108 
856 4 0 |u http://dx.doi.org/10.1109/TIP.2022.3205216  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 31  |j 2022  |b 14  |h 6097-6108