Multimodal Contrastive Learning for Cybersickness Recognition Using Brain Connectivity Graph Representation

Cybersickness significantly impairs user comfort and immersion in virtual reality (VR). Effective identification of cybersickness leveraging physiological, visual, and motion data is a critical prerequisite for its mitigation. However, current methods primarily employ direct feature fusion across mo...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - PP(2025) vom: 06. Okt.
1. Verfasser: Wang, Peike (VerfasserIn)
Weitere Verfasser: Li, Ming, Wang, Ziteng, Liu, Yong-Jin, Wang, Lili
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2025
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000naa a22002652c 4500
001 NLM393680479
003 DE-627
005 20251007232633.0
007 cr uuu---uuuuu
008 251007s2025 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2025.3616797  |2 doi 
028 5 2 |a pubmed25n1592.xml 
035 |a (DE-627)NLM393680479 
035 |a (NLM)41052115 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Wang, Peike  |e verfasserin  |4 aut 
245 1 0 |a Multimodal Contrastive Learning for Cybersickness Recognition Using Brain Connectivity Graph Representation 
264 1 |c 2025 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 06.10.2025 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a Cybersickness significantly impairs user comfort and immersion in virtual reality (VR). Effective identification of cybersickness leveraging physiological, visual, and motion data is a critical prerequisite for its mitigation. However, current methods primarily employ direct feature fusion across modalities, which often leads to limited accuracy due to inadequate modeling of inter-modal relationships. In this paper, we propose a multimodal contrastive learning method for cybersickness recognition. First, we introduce Brain Connectivity Graph Representation (BCGR), an innovative graph-based representation that captures cybersickness-related connectivity patterns across modalities. We further develop three BCGR instances: E-BCGR, constructed based on EEG signals; MV-BCGR, constructed based on video and motion data; and S-BCGR, obtained through our proposed standardized decomposition algorithm. Then, we propose a connectivity-constrained contrastive fusion module, which aligns E-BCGR and MV-BCGR into a shared latent space via graph contrastive learning while utilizing S-BCGR as a connectivity constraint to enhance representation quality. Moreover, we construct a multimodal cybersickness dataset comprising synchronized EEG, video, and motion data collected in VR environments to promote further research in this domain. Experimental results demonstrate that our method outperforms existing state-of-the-art methods across four critical evaluation metrics: accuracy, sensitivity, specificity, and the area under the curve. Source code: https://github.com/PEKEW/cybersickness-bcgr 
650 4 |a Journal Article 
700 1 |a Li, Ming  |e verfasserin  |4 aut 
700 1 |a Wang, Ziteng  |e verfasserin  |4 aut 
700 1 |a Liu, Yong-Jin  |e verfasserin  |4 aut 
700 1 |a Wang, Lili  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g PP(2025) vom: 06. Okt.  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnas 
773 1 8 |g volume:PP  |g year:2025  |g day:06  |g month:10 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2025.3616797  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d PP  |j 2025  |b 06  |c 10