Joint Patch and Multi-label Learning for Facial Action Unit and Holistic Expression Recognition

Most action unit (AU) detection methods use one-versus-all classifiers without considering dependences between features or AUs. In this paper, we introduce a joint patch and multi-label learning (JPML) framework that models the structured joint dependence behind features, AUs, and their interplay. I...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. - 1992. - 25(2016), 8 vom: 15. Aug., Seite 3931-3946
1. Verfasser: Kaili Zhao (VerfasserIn)
Weitere Verfasser: Wen-Sheng Chu, De la Torre, Fernando, Cohn, Jeffrey F, Honggang Zhang
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2016
Zugriff auf das übergeordnete Werk:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM268250154
003 DE-627
005 20231224222315.0
007 cr uuu---uuuuu
008 231224s2016 xx |||||o 00| ||eng c
024 7 |a 10.1109/TIP.2016.2570550  |2 doi 
028 5 2 |a pubmed24n0894.xml 
035 |a (DE-627)NLM268250154 
035 |a (NLM)28113424 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Kaili Zhao  |e verfasserin  |4 aut 
245 1 0 |a Joint Patch and Multi-label Learning for Facial Action Unit and Holistic Expression Recognition 
264 1 |c 2016 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 20.11.2019 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Most action unit (AU) detection methods use one-versus-all classifiers without considering dependences between features or AUs. In this paper, we introduce a joint patch and multi-label learning (JPML) framework that models the structured joint dependence behind features, AUs, and their interplay. In particular, JPML leverages group sparsity to identify important facial patches, and learns a multi-label classifier constrained by the likelihood of co-occurring AUs. To describe such likelihood, we derive two AU relations, positive correlation and negative competition, by statistically analyzing more than 350,000 video frames annotated with multiple AUs. To the best of our knowledge, this is the first work that jointly addresses patch learning and multi-label learning for AU detection. In addition, we show that JPML can be extended to recognize holistic expressions by learning common and specific patches, which afford a more compact representation than the standard expression recognition methods. We evaluate JPML on three benchmark datasets CK+, BP4D, and GFT, using within-and cross-dataset scenarios. In four of five experiments, JPML achieved the highest averaged F1 scores in comparison with baseline and alternative methods that use either patch learning or multi-label learning alone 
650 4 |a Journal Article 
700 1 |a Wen-Sheng Chu  |e verfasserin  |4 aut 
700 1 |a De la Torre, Fernando  |e verfasserin  |4 aut 
700 1 |a Cohn, Jeffrey F  |e verfasserin  |4 aut 
700 1 |a Honggang Zhang  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on image processing : a publication of the IEEE Signal Processing Society  |d 1992  |g 25(2016), 8 vom: 15. Aug., Seite 3931-3946  |w (DE-627)NLM09821456X  |x 1941-0042  |7 nnns 
773 1 8 |g volume:25  |g year:2016  |g number:8  |g day:15  |g month:08  |g pages:3931-3946 
856 4 0 |u http://dx.doi.org/10.1109/TIP.2016.2570550  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 25  |j 2016  |e 8  |b 15  |c 08  |h 3931-3946