Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes

In this paper, we present a label transfer model from texts to images for image classification tasks. The problem of image classification is often much more challenging than text classification. On one hand, labeled text data is more widely available than the labeled images for classification tasks....

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 39(2017), 7 vom: 01. Juli, Seite 1360-1373
1. Verfasser: Guo-Jun Qi (VerfasserIn)
Weitere Verfasser: Wei Liu, Aggarwal, Charu, Huang, Thomas
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2017
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article Research Support, U.S. Gov't, Non-P.H.S.
LEADER 01000naa a22002652 4500
001 NLM262196360
003 DE-627
005 20231224201229.0
007 cr uuu---uuuuu
008 231224s2017 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2016.2587643  |2 doi 
028 5 2 |a pubmed24n0874.xml 
035 |a (DE-627)NLM262196360 
035 |a (NLM)27392343 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Guo-Jun Qi  |e verfasserin  |4 aut 
245 1 0 |a Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes 
264 1 |c 2017 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 01.11.2018 
500 |a Date Revised 01.11.2018 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a In this paper, we present a label transfer model from texts to images for image classification tasks. The problem of image classification is often much more challenging than text classification. On one hand, labeled text data is more widely available than the labeled images for classification tasks. On the other hand, text data tends to have natural semantic interpretability, and they are often more directly related to class labels. On the contrary, the image features are not directly related to concepts inherent in class labels. One of our goals in this paper is to develop a model for revealing the functional relationships between text and image features as to directly transfer intermodal and intramodal labels to annotate the images. This is implemented by learning a transfer function as a bridge to propagate the labels between two multimodal spaces. However, the intermodal label transfers could be undermined by blindly transferring the labels of noisy texts to annotate images. To mitigate this problem, we present an intramodal label transfer process, which complements the intermodal label transfer by transferring the image labels instead when relevant text is absent from the source corpus. In addition, we generalize the inter-modal label transfer to zero-shot learning scenario where there are only text examples available to label unseen classes of images without any positive image examples. We evaluate our algorithm on an image classification task and show the effectiveness with respect to the other compared algorithms 
650 4 |a Journal Article 
650 4 |a Research Support, U.S. Gov't, Non-P.H.S. 
700 1 |a Wei Liu  |e verfasserin  |4 aut 
700 1 |a Aggarwal, Charu  |e verfasserin  |4 aut 
700 1 |a Huang, Thomas  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 39(2017), 7 vom: 01. Juli, Seite 1360-1373  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:39  |g year:2017  |g number:7  |g day:01  |g month:07  |g pages:1360-1373 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2016.2587643  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 39  |j 2017  |e 7  |b 01  |c 07  |h 1360-1373