Depth Selection for Deep ReLU Nets in Feature Extraction and Generalization

Deep learning is recognized to be capable of discovering deep features for representation learning and pattern recognition without requiring elegant feature engineering techniques by taking advantages of human ingenuity and prior knowledge. Thus it has triggered enormous research activities in machi...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 44(2022), 4 vom: 02. Apr., Seite 1853-1868
1. Verfasser: Han, Zhi (VerfasserIn)
Weitere Verfasser: Yu, Siquan, Lin, Shao-Bo, Zhou, Ding-Xuan
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2022
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article Research Support, Non-U.S. Gov't
LEADER 01000caa a22002652c 4500
001 NLM31648928X
003 DE-627
005 20250228055059.0
007 cr uuu---uuuuu
008 231225s2022 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2020.3032422  |2 doi 
028 5 2 |a pubmed25n1054.xml 
035 |a (DE-627)NLM31648928X 
035 |a (NLM)33079656 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Han, Zhi  |e verfasserin  |4 aut 
245 1 0 |a Depth Selection for Deep ReLU Nets in Feature Extraction and Generalization 
264 1 |c 2022 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 28.03.2022 
500 |a Date Revised 01.04.2022 
500 |a published: Print-Electronic 
500 |a Citation Status MEDLINE 
520 |a Deep learning is recognized to be capable of discovering deep features for representation learning and pattern recognition without requiring elegant feature engineering techniques by taking advantages of human ingenuity and prior knowledge. Thus it has triggered enormous research activities in machine learning and pattern recognition. One of the most important challenges of deep learning is to figure out relations between a feature and the depth of deep neural networks (deep nets for short) to reflect the necessity of depth. Our purpose is to quantify this feature-depth correspondence in feature extraction and generalization. We present the adaptivity of features to depths and vice-verse via showing a depth-parameter trade-off in extracting both single feature and composite features. Based on these results, we prove that implementing the classical empirical risk minimization on deep nets can achieve the optimal generalization performance for numerous learning tasks. Our theoretical results are verified by a series of numerical experiments including toy simulations and a real application of earthquake seismic intensity prediction 
650 4 |a Journal Article 
650 4 |a Research Support, Non-U.S. Gov't 
700 1 |a Yu, Siquan  |e verfasserin  |4 aut 
700 1 |a Lin, Shao-Bo  |e verfasserin  |4 aut 
700 1 |a Zhou, Ding-Xuan  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 44(2022), 4 vom: 02. Apr., Seite 1853-1868  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnas 
773 1 8 |g volume:44  |g year:2022  |g number:4  |g day:02  |g month:04  |g pages:1853-1868 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2020.3032422  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 44  |j 2022  |e 4  |b 02  |c 04  |h 1853-1868