Synthesizing Knowledge-Enhanced Features for Real-World Zero-Shot Food Detection

Food computing brings various perspectives to computer vision like vision-based food analysis for nutrition and health. As a fundamental task in food computing, food detection needs Zero-Shot Detection (ZSD) on novel unseen food objects to support real-world scenarios, such as intelligent kitchens a...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. - 1992. - 33(2024) vom: 06., Seite 1285-1298
1. Verfasser: Zhou, Pengfei (VerfasserIn)
Weitere Verfasser: Min, Weiqing, Song, Jiajun, Zhang, Yang, Jiang, Shuqiang
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM368091953
003 DE-627
005 20240214233230.0
007 cr uuu---uuuuu
008 240207s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TIP.2024.3360899  |2 doi 
028 5 2 |a pubmed24n1293.xml 
035 |a (DE-627)NLM368091953 
035 |a (NLM)38319769 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Zhou, Pengfei  |e verfasserin  |4 aut 
245 1 0 |a Synthesizing Knowledge-Enhanced Features for Real-World Zero-Shot Food Detection 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 14.02.2024 
500 |a Date Revised 14.02.2024 
500 |a published: Print-Electronic 
500 |a Citation Status MEDLINE 
520 |a Food computing brings various perspectives to computer vision like vision-based food analysis for nutrition and health. As a fundamental task in food computing, food detection needs Zero-Shot Detection (ZSD) on novel unseen food objects to support real-world scenarios, such as intelligent kitchens and smart restaurants. Therefore, we first benchmark the task of Zero-Shot Food Detection (ZSFD) by introducing FOWA dataset with rich attribute annotations. Unlike ZSD, fine-grained problems in ZSFD like inter-class similarity make synthesized features inseparable. The complexity of food semantic attributes further makes it more difficult for current ZSD methods to distinguish various food categories. To address these problems, we propose a novel framework ZSFDet to tackle fine-grained problems by exploiting the interaction between complex attributes. Specifically, we model the correlation between food categories and attributes in ZSFDet by multi-source graphs to provide prior knowledge for distinguishing fine-grained features. Within ZSFDet, Knowledge-Enhanced Feature Synthesizer (KEFS) learns knowledge representation from multiple sources (e.g., ingredients correlation from knowledge graph) via the multi-source graph fusion. Conditioned on the fusion of semantic knowledge representation, the region feature diffusion model in KEFS can generate fine-grained features for training the effective zero-shot detector. Extensive evaluations demonstrate the superior performance of our method ZSFDet on FOWA and the widely-used food dataset UECFOOD-256, with significant improvements by 1.8% and 3.7% ZSD mAP compared with the strong baseline RRFS. Further experiments on PASCAL VOC and MS COCO prove that enhancement of the semantic knowledge can also improve the performance on general ZSD. Code and dataset are available at https://github.com/LanceZPF/KEFS 
650 4 |a Journal Article 
700 1 |a Min, Weiqing  |e verfasserin  |4 aut 
700 1 |a Song, Jiajun  |e verfasserin  |4 aut 
700 1 |a Zhang, Yang  |e verfasserin  |4 aut 
700 1 |a Jiang, Shuqiang  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on image processing : a publication of the IEEE Signal Processing Society  |d 1992  |g 33(2024) vom: 06., Seite 1285-1298  |w (DE-627)NLM09821456X  |x 1941-0042  |7 nnns 
773 1 8 |g volume:33  |g year:2024  |g day:06  |g pages:1285-1298 
856 4 0 |u http://dx.doi.org/10.1109/TIP.2024.3360899  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 33  |j 2024  |b 06  |h 1285-1298