|
|
|
|
| LEADER |
01000caa a22002652c 4500 |
| 001 |
NLM393512576 |
| 003 |
DE-627 |
| 005 |
20251007231851.0 |
| 007 |
cr uuu---uuuuu |
| 008 |
251003s2025 xx |||||o 00| ||eng c |
| 024 |
7 |
|
|a 10.1109/TPAMI.2025.3593657
|2 doi
|
| 028 |
5 |
2 |
|a pubmed25n1591.xml
|
| 035 |
|
|
|a (DE-627)NLM393512576
|
| 035 |
|
|
|a (NLM)40729723
|
| 040 |
|
|
|a DE-627
|b ger
|c DE-627
|e rakwb
|
| 041 |
|
|
|a eng
|
| 100 |
1 |
|
|a Lee, Wu
|e verfasserin
|4 aut
|
| 245 |
1 |
0 |
|a HPformer
|b Low-Parameter Transformer With Temporal Dependency Hierarchical Propagation for Health Informatics
|
| 264 |
|
1 |
|c 2025
|
| 336 |
|
|
|a Text
|b txt
|2 rdacontent
|
| 337 |
|
|
|a ƒaComputermedien
|b c
|2 rdamedia
|
| 338 |
|
|
|a ƒa Online-Ressource
|b cr
|2 rdacarrier
|
| 500 |
|
|
|a Date Revised 06.10.2025
|
| 500 |
|
|
|a published: Print
|
| 500 |
|
|
|a Citation Status PubMed-not-MEDLINE
|
| 520 |
|
|
|a Transformers based on Self-Attention (SA) mechanism have demonstrated unrivaled superiority in numerous areas. Compared to RNN-based networks, Transformers can learn the temporal dependency representation of an entire sequence in parallel, while efficiently dealing with long-range dependencies. However, the $\mathcal {O}(L^{2})$O(L2) ($L$L denotes the length of the sequence) computational complexity of the SA mechanism and the high memory usage make the construction cost of the Transformer-based model prohibitively expensive. To address these challenges, we propose a Transformer-like model, HPformer: Low-Parameter Transformer with Temporal Dependency Hierarchical Propagation. HPformer first chunks the sequence into $K$K ($K = \left\lceil \log {L} \right\rceil + 1$K=logL+1, $\left\lceil \cdot \right\rceil$· denotes ceiling operation) sequence segments, then leverages the hierarchical propagation mechanism with $\mathcal {O}(L)$O(L) computational complexity to learn the temporal dependencies between the segments and within the segments, and ultimately generates $K$K vectors as $Key$Key matrices. This reduces the complexity of the SA mechanism from $\mathcal {O}(L^{2})$O(L2) to $\mathcal {O}(L\log {L})$O(LlogL). In addition, we employ a strategy of sharing $Key$Key and $Value$Value matrices between layers to build the HPformer, thus reducing memory usage. Extensive experiments based on public health informatics benchmark and Long-Range Arena (LRA) benchmark have demonstrated that HPformer has advantages over Transformer-based models in terms of memory usage and efficiency
|
| 650 |
|
4 |
|a Journal Article
|
| 700 |
1 |
|
|a Shi, Yuliang
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Yu, Han
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Cheng, Lin
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Wang, Xinjun
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Yan, Zhongmin
|e verfasserin
|4 aut
|
| 700 |
1 |
|
|a Kong, Fanyu
|e verfasserin
|4 aut
|
| 773 |
0 |
8 |
|i Enthalten in
|t IEEE transactions on pattern analysis and machine intelligence
|d 1979
|g 47(2025), 11 vom: 28. Okt., Seite 10770-10786
|w (DE-627)NLM098212257
|x 1939-3539
|7 nnas
|
| 773 |
1 |
8 |
|g volume:47
|g year:2025
|g number:11
|g day:28
|g month:10
|g pages:10770-10786
|
| 856 |
4 |
0 |
|u http://dx.doi.org/10.1109/TPAMI.2025.3593657
|3 Volltext
|
| 912 |
|
|
|a GBV_USEFLAG_A
|
| 912 |
|
|
|a SYSFLAG_A
|
| 912 |
|
|
|a GBV_NLM
|
| 912 |
|
|
|a GBV_ILN_350
|
| 951 |
|
|
|a AR
|
| 952 |
|
|
|d 47
|j 2025
|e 11
|b 28
|c 10
|h 10770-10786
|