|
|
|
|
| LEADER |
01000naa a22002652 4500 |
| 001 |
NLM341395501 |
| 003 |
DE-627 |
| 005 |
20231226011658.0 |
| 007 |
cr uuu---uuuuu |
| 008 |
231226s2023 xx |||||o 00| ||eng c |
| 024 |
7 |
|
|a 10.1109/TPAMI.2022.3178101
|2 doi
|
| 028 |
5 |
2 |
|a pubmed24n1137.xml
|
| 035 |
|
|
|a (DE-627)NLM341395501
|
| 035 |
|
|
|a (NLM)35617189
|
| 040 |
|
|
|a DE-627
|b ger
|c DE-627
|e rakwb
|
| 041 |
|
|
|a eng
|
| 100 |
1 |
|
|a Li, Tao
|e verfasserin
|4 aut
|
| 245 |
1 |
0 |
|a Low Dimensional Trajectory Hypothesis is True
|b DNNs Can Be Trained in Tiny Subspaces
|
| 264 |
|
1 |
|c 2023
|
| 336 |
|
|
|a Text
|b txt
|2 rdacontent
|
| 337 |
|
|
|a ƒaComputermedien
|b c
|2 rdamedia
|
| 338 |
|
|
|a ƒa Online-Ressource
|b cr
|2 rdacarrier
|
| 500 |
|
|
|a Date Completed 07.04.2023
|
| 500 |
|
|
|a Date Revised 07.04.2023
|
| 500 |
|
|
|a published: Print-Electronic
|
| 500 |
|
|
|a Citation Status PubMed-not-MEDLINE
|
| 520 |
|
|
|a Deep neural networks (DNNs) usually contain massive parameters, but there is redundancy such that it is guessed that they could be trained in low-dimensional subspaces. In this paper, we propose a Dynamic Linear Dimensionality Reduction (DLDR) based on the low-dimensional properties of the training trajectory. The reduction method is efficient, supported by comprehensive experiments: optimizing DNNs in 40-dimensional spaces can achieve comparable performance as regular training over thousands or even millions of parameters. Since there are only a few variables to optimize, we develop an efficient quasi-Newton-based algorithm, obtain robustness to label noise, and improve the performance of well-trained models, which are three follow-up experiments that can show the advantages of finding such low-dimensional subspaces. The code is released (Pytorch: https://github.com/nblt/DLDR and Mindspore: https://gitee.com/mindspore/docs/tree/r1.6/docs/sample_code/dimension_reduce_training)
|
| 650 |
|
4 |
|a Journal Article
|
| 700 |
1 |
|
|a Tan, Lei
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Huang, Zhehao
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Tao, Qinghua
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Liu, Yipeng
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Huang, Xiaolin
|e verfasserin
|4 aut
|
| 773 |
0 |
8 |
|i Enthalten in
|t IEEE transactions on pattern analysis and machine intelligence
|d 1979
|g 45(2023), 3 vom: 26. März, Seite 3411-3420
|w (DE-627)NLM098212257
|x 1939-3539
|7 nnns
|
| 773 |
1 |
8 |
|g volume:45
|g year:2023
|g number:3
|g day:26
|g month:03
|g pages:3411-3420
|
| 856 |
4 |
0 |
|u http://dx.doi.org/10.1109/TPAMI.2022.3178101
|3 Volltext
|
| 912 |
|
|
|a GBV_USEFLAG_A
|
| 912 |
|
|
|a SYSFLAG_A
|
| 912 |
|
|
|a GBV_NLM
|
| 912 |
|
|
|a GBV_ILN_350
|
| 951 |
|
|
|a AR
|
| 952 |
|
|
|d 45
|j 2023
|e 3
|b 26
|c 03
|h 3411-3420
|