Time to retire F1-binary score for action unit detection

Detecting action units is an important task in face analysis, especially in facial expression recognition. This is due, in part, to the idea that expressions can be decomposed into multiple action units. To evaluate systems that detect action units, F1-binary score is often used as the evaluation me...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:Pattern recognition letters. - 1998. - 182(2024) vom: 01. Juni, Seite 111-117
1. Verfasser: Hinduja, Saurabh (VerfasserIn)
Weitere Verfasser: Nourivandi, Tara, Cohn, Jeffrey F, Canavan, Shaun
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:Pattern recognition letters
Schlagworte:Journal Article Action units Data imbalance F1 score Machine learning
LEADER 01000caa a22002652 4500
001 NLM375739513
003 DE-627
005 20240803233015.0
007 cr uuu---uuuuu
008 240801s2024 xx |||||o 00| ||eng c
024 7 |a 10.1016/j.patrec.2024.04.016  |2 doi 
028 5 2 |a pubmed24n1490.xml 
035 |a (DE-627)NLM375739513 
035 |a (NLM)39086494 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Hinduja, Saurabh  |e verfasserin  |4 aut 
245 1 0 |a Time to retire F1-binary score for action unit detection 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 03.08.2024 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Detecting action units is an important task in face analysis, especially in facial expression recognition. This is due, in part, to the idea that expressions can be decomposed into multiple action units. To evaluate systems that detect action units, F1-binary score is often used as the evaluation metric. In this paper, we argue that F1-binary score does not reliably evaluate these models due largely to class imbalance. Because of this, F1-binary score should be retired and a suitable replacement should be used. We justify this argument through a detailed evaluation of the negative influence of class imbalance on action unit detection. This includes an investigation into the influence of class imbalance in train and test sets and in new data (i.e., generalizability). We empirically show that F1-micro should be used as the replacement for F1-binary 
650 4 |a Journal Article 
650 4 |a Action units 
650 4 |a Data imbalance 
650 4 |a F1 score 
650 4 |a Machine learning 
700 1 |a Nourivandi, Tara  |e verfasserin  |4 aut 
700 1 |a Cohn, Jeffrey F  |e verfasserin  |4 aut 
700 1 |a Canavan, Shaun  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t Pattern recognition letters  |d 1998  |g 182(2024) vom: 01. Juni, Seite 111-117  |w (DE-627)NLM098154265  |x 0167-8655  |7 nnns 
773 1 8 |g volume:182  |g year:2024  |g day:01  |g month:06  |g pages:111-117 
856 4 0 |u http://dx.doi.org/10.1016/j.patrec.2024.04.016  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 182  |j 2024  |b 01  |c 06  |h 111-117