NER-Net+ : Seeing Motion at Nighttime With an Event Camera
We focus on a very challenging task: imaging at nighttime dynamic scenes. Conventional RGB cameras struggle with the trade-off between long exposure for low-light imaging and short exposure for capturing dynamic scenes. Event cameras react to dynamic changes, with their high temporal resolution (mic...
| Publié dans: | IEEE transactions on pattern analysis and machine intelligence. - 1979. - 47(2025), 6 vom: 03. Mai, Seite 4768-4786 |
|---|---|
| Auteur principal: | |
| Autres auteurs: | , , , , , , , |
| Format: | Article en ligne |
| Langue: | English |
| Publié: |
2025
|
| Accès à la collection: | IEEE transactions on pattern analysis and machine intelligence |
| Sujets: | Journal Article |
| Résumé: | We focus on a very challenging task: imaging at nighttime dynamic scenes. Conventional RGB cameras struggle with the trade-off between long exposure for low-light imaging and short exposure for capturing dynamic scenes. Event cameras react to dynamic changes, with their high temporal resolution (microsecond) and dynamic range (120 dB), and thus offer a promising alternative. However, existing methods are mostly based on simulated datasets due to the lack of paired event-clean image data for nighttime conditions, where the domain gap leads to performance limitations in real-world scenarios. Moreover, most existing event reconstruction methods are tailored for daytime data, overlooking issues unique to low-light events at night, such as strong noise, temporal trailing, and spatial non-uniformity, resulting in unsatisfactory reconstruction results. To address these challenges, we construct the first real paired low-light event dataset (RLED) through a co-axial imaging system, comprising 80,400 spatially and temporally aligned image GTs and low-light events, which provides a unified training and evaluation dataset for existing methods. We further conduct a comprehensive analysis of the causes and characteristics of strong noise, temporal trailing, and spatial non-uniformity in nighttime events, and propose a nighttime event reconstruction network (NER-Net+). It includes a learnable event timestamps calibration module (LETC) to correct the temporal trailing events and a non-stationary spatio-temporal information enhancement module (NSIE) to suppress sensor noise and spatial non-uniformity. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods in visual quality and generalization on real-world nighttime datasets |
|---|---|
| Description: | Date Revised 07.05.2025 published: Print-Electronic Citation Status PubMed-not-MEDLINE |
| ISSN: | 1939-3539 |
| DOI: | 10.1109/TPAMI.2025.3545936 |