Learning an Adaptive Sparse Transformer for Efficient Image Restoration
Transformer-based approaches have shown promising performance in image restoration tasks due to their ability to model long-range dependencies, which are essential for recovering clear images. Although various efficient attention mechanisms have been proposed to address the intensive computational l...
| Veröffentlicht in: | IEEE transactions on pattern analysis and machine intelligence. - 1979. - 47(2025), 11 vom: 01. Okt., Seite 10344-10360 |
|---|---|
| 1. Verfasser: | |
| Weitere Verfasser: | , |
| Format: | Online-Aufsatz |
| Sprache: | English |
| Veröffentlicht: |
2025
|
| Zugriff auf das übergeordnete Werk: | IEEE transactions on pattern analysis and machine intelligence |
| Schlagworte: | Journal Article |
| Zusammenfassung: | Transformer-based approaches have shown promising performance in image restoration tasks due to their ability to model long-range dependencies, which are essential for recovering clear images. Although various efficient attention mechanisms have been proposed to address the intensive computational loads of transformers, they often suffer from redundant information and noisy interactions from irrelevant regions, as they consider all available tokens. In this work, we propose an Adaptive Sparse Transformer (AST-v2) to mitigate these issues by reducing noisy interactions in irrelevant areas and removing feature redundancy along channel dimension. AST-v2 incorporates two core components: an Adaptive Sparse Self-Attention (ASSA) block and a Feature Refinement Feed-forward Network (FRFN). ASSA adopts a dual-branch design, where the sparse branch guides the modulation of standard dense attention weights. This paradigm reduces the negative impact of irrelevant token interactions while preserving the important ones. Meanwhile, FRFN utilizes an enhance-and-ease scheme to eliminate feature redundancy across channels, enhancing the restoration of clear images. Experimental results on commonly used benchmarks show the competitive performance of our method for 6 restoration tasks, including rain streak removal, haze removal, shadow removal, snow removal, blur removal, and low-light enhancement. The code is available in the supplementary materials |
|---|---|
| Beschreibung: | Date Revised 06.10.2025 published: Print Citation Status PubMed-not-MEDLINE |
| ISSN: | 1939-3539 |
| DOI: | 10.1109/TPAMI.2025.3594910 |