Time to retire F1-binary score for action unit detection
Detecting action units is an important task in face analysis, especially in facial expression recognition. This is due, in part, to the idea that expressions can be decomposed into multiple action units. To evaluate systems that detect action units, F1-binary score is often used as the evaluation me...
| Veröffentlicht in: | Pattern recognition letters. - 1998. - 182(2024) vom: 01. Juni, Seite 111-117 |
|---|---|
| 1. Verfasser: | |
| Weitere Verfasser: | , , |
| Format: | Online-Aufsatz |
| Sprache: | English |
| Veröffentlicht: |
2024
|
| Zugriff auf das übergeordnete Werk: | Pattern recognition letters |
| Schlagworte: | Journal Article Action units Data imbalance F1 score Machine learning |
| Zusammenfassung: | Detecting action units is an important task in face analysis, especially in facial expression recognition. This is due, in part, to the idea that expressions can be decomposed into multiple action units. To evaluate systems that detect action units, F1-binary score is often used as the evaluation metric. In this paper, we argue that F1-binary score does not reliably evaluate these models due largely to class imbalance. Because of this, F1-binary score should be retired and a suitable replacement should be used. We justify this argument through a detailed evaluation of the negative influence of class imbalance on action unit detection. This includes an investigation into the influence of class imbalance in train and test sets and in new data (i.e., generalizability). We empirically show that F1-micro should be used as the replacement for F1-binary |
|---|---|
| Beschreibung: | Date Revised 03.08.2024 published: Print-Electronic Citation Status PubMed-not-MEDLINE |
| ISSN: | 0167-8655 |
| DOI: | 10.1016/j.patrec.2024.04.016 |