Winning Solutions and Post-Challenge Analyses of the ChaLearn AutoDL Challenge 2019

Winning Solutions and Post-Challenge Analyses of the ChaLearn AutoDL Challenge 2019

This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) that had been introduced in a variety of settings, but lacked fair comparisons. All input data modalities (time serie...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 43(2021), 9 vom: 23. Sept., Seite 3108-3125
1. Verfasser: Liu, Zhengying (VerfasserIn)
Weitere Verfasser: Pavao, Adrien, Xu, Zhen, Escalera, Sergio, Ferreira, Fabio, Guyon, Isabelle, Hong, Sirui, Hutter, Frank, Ji, Rongrong, Junior, Julio C S Jacques, Li, Ge, Lindauer, Marius, Luo, Zhipeng, Madadi, Meysam, Nierhoff, Thomas, Niu, Kangning, Pan, Chunguang, Stoll, Danny, Treguer, Sebastien, Wang, Jin, Wang, Peng, Wu, Chenglin, Xiong, Youcheng, Zela, Arber, Zhang, Yang
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2021
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article Research Support, Non-U.S. Gov't
LEADER 01000naa a22002652 4500
001 NLM324451938
003 DE-627
005 20231225190343.0
007 cr uuu---uuuuu
008 231225s2021 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2021.3075372  |2 doi 
028 5 2 |a pubmed24n1081.xml 
035 |a (DE-627)NLM324451938 
035 |a (NLM)33891549 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Liu, Zhengying  |e verfasserin  |4 aut 
245 1 0 |a Winning Solutions and Post-Challenge Analyses of the ChaLearn AutoDL Challenge 2019 
264 1 |c 2021 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 29.09.2021 
500 |a Date Revised 29.09.2021 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) that had been introduced in a variety of settings, but lacked fair comparisons. All input data modalities (time series, images, videos, text, tabular) were formatted as tensors and all tasks were multi-label classification problems. Code submissions were executed on hidden tasks, with limited time and computational resources, pushing solutions that get results quickly. In this setting, DL methods dominated, though popular Neural Architecture Search (NAS) was impractical. Solutions relied on fine-tuned pre-trained networks, with architectures matching data modality. Post-challenge tests did not reveal improvements beyond the imposed time limit. While no component is particularly original or novel, a high level modular organization emerged featuring a "meta-learner", "data ingestor", "model selector", "model/learner", and "evaluator". This modularity enabled ablation studies, which revealed the importance of (off-platform) meta-learning, ensembling, and efficient data management. Experiments on heterogeneous module combinations further confirm the (local) optimality of the winning solutions. Our challenge legacy includes an ever-lasting benchmark (http://autodl.chalearn.org), the open-sourced code of the winners, and a free "AutoDL self-service." 
650 4 |a Journal Article 
650 4 |a Research Support, Non-U.S. Gov't 
700 1 |a Pavao, Adrien  |e verfasserin  |4 aut 
700 1 |a Xu, Zhen  |e verfasserin  |4 aut 
700 1 |a Escalera, Sergio  |e verfasserin  |4 aut 
700 1 |a Ferreira, Fabio  |e verfasserin  |4 aut 
700 1 |a Guyon, Isabelle  |e verfasserin  |4 aut 
700 1 |a Hong, Sirui  |e verfasserin  |4 aut 
700 1 |a Hutter, Frank  |e verfasserin  |4 aut 
700 1 |a Ji, Rongrong  |e verfasserin  |4 aut 
700 1 |a Junior, Julio C S Jacques  |e verfasserin  |4 aut 
700 1 |a Li, Ge  |e verfasserin  |4 aut 
700 1 |a Lindauer, Marius  |e verfasserin  |4 aut 
700 1 |a Luo, Zhipeng  |e verfasserin  |4 aut 
700 1 |a Madadi, Meysam  |e verfasserin  |4 aut 
700 1 |a Nierhoff, Thomas  |e verfasserin  |4 aut 
700 1 |a Niu, Kangning  |e verfasserin  |4 aut 
700 1 |a Pan, Chunguang  |e verfasserin  |4 aut 
700 1 |a Stoll, Danny  |e verfasserin  |4 aut 
700 1 |a Treguer, Sebastien  |e verfasserin  |4 aut 
700 1 |a Wang, Jin  |e verfasserin  |4 aut 
700 1 |a Wang, Peng  |e verfasserin  |4 aut 
700 1 |a Wu, Chenglin  |e verfasserin  |4 aut 
700 1 |a Xiong, Youcheng  |e verfasserin  |4 aut 
700 1 |a Zela, Arber  |e verfasserin  |4 aut 
700 1 |a Zhang, Yang  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 43(2021), 9 vom: 23. Sept., Seite 3108-3125  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:43  |g year:2021  |g number:9  |g day:23  |g month:09  |g pages:3108-3125 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2021.3075372  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 43  |j 2021  |e 9  |b 23  |c 09  |h 3108-3125