Retrieval of Sentence Sequences for an Image Stream via Coherence Recurrent Convolutional Networks

We propose an approach for retrieving a sequence of natural sentences for an image stream. Since general users often take a series of pictures on their experiences, much online visual information exists in the form of image streams, for which it would better take into consideration of the whole imag...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 40(2018), 4 vom: 05. Apr., Seite 945-957
1. Verfasser: Park, Cesc Chunseong (VerfasserIn)
Weitere Verfasser: Kim, Youngjin, Kim, Gunhee
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2018
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article Research Support, Non-U.S. Gov't
LEADER 01000caa a22002652 4500
001 NLM271604557
003 DE-627
005 20250221144356.0
007 cr uuu---uuuuu
008 231224s2018 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2017.2700381  |2 doi 
028 5 2 |a pubmed25n0905.xml 
035 |a (DE-627)NLM271604557 
035 |a (NLM)28475047 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Park, Cesc Chunseong  |e verfasserin  |4 aut 
245 1 0 |a Retrieval of Sentence Sequences for an Image Stream via Coherence Recurrent Convolutional Networks 
264 1 |c 2018 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 04.03.2019 
500 |a Date Revised 04.03.2019 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a We propose an approach for retrieving a sequence of natural sentences for an image stream. Since general users often take a series of pictures on their experiences, much online visual information exists in the form of image streams, for which it would better take into consideration of the whole image stream to produce natural language descriptions. While almost all previous studies have dealt with the relation between a single image and a single natural sentence, our work extends both input and output dimension to a sequence of images and a sequence of sentences. For retrieving a coherent flow of multiple sentences for a photo stream, we propose a multimodal neural architecture called coherence recurrent convolutional network (CRCN), which consists of convolutional neural networks, bidirectional long short-term memory (LSTM) networks, and an entity-based local coherence model. Our approach directly learns from vast user-generated resource of blog posts as text-image parallel training data. We collect more than 22 K unique blog posts with 170 K associated images for the travel topics of NYC, Disneyland , Australia, and Hawaii. We demonstrate that our approach outperforms other state-of-the-art image captioning methods for text sequence generation, using both quantitative measures and user studies via Amazon Mechanical Turk 
650 4 |a Journal Article 
650 4 |a Research Support, Non-U.S. Gov't 
700 1 |a Kim, Youngjin  |e verfasserin  |4 aut 
700 1 |a Kim, Gunhee  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 40(2018), 4 vom: 05. Apr., Seite 945-957  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:40  |g year:2018  |g number:4  |g day:05  |g month:04  |g pages:945-957 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2017.2700381  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 40  |j 2018  |e 4  |b 05  |c 04  |h 945-957