A high-throughput big-data orchestration and processing system for the High Energy Photon Source

open access.

Bibliographische Detailangaben
Veröffentlicht in:Journal of synchrotron radiation. - 1994. - 30(2023), Pt 6 vom: 01. Nov., Seite 1086-1091
1. Verfasser: Li, Xiang (VerfasserIn)
Weitere Verfasser: Zhang, Yi, Liu, Yu, Li, Pengcheng, Hu, Hao, Wang, Liwen, He, Ping, Dong, Yuhui, Zhang, Chenglong
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:Journal of synchrotron radiation
Schlagworte:Journal Article beamline software data acquisition data management high-throughput experiment multimodal experiments
LEADER 01000caa a22002652c 4500
001 NLM362258651
003 DE-627
005 20250305064018.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1107/S1600577523006951  |2 doi 
028 5 2 |a pubmed25n1207.xml 
035 |a (DE-627)NLM362258651 
035 |a (NLM)37729071 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Li, Xiang  |e verfasserin  |4 aut 
245 1 2 |a A high-throughput big-data orchestration and processing system for the High Energy Photon Source 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 05.11.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a open access. 
520 |a High-data-throughput and multimodal-acquisition experiments will prevail in next-generation synchrotron beamlines. Orchestrating dataflow pipelines connecting the data acquisition, processing, visualization and storage ends are becoming increasingly complex and essential for enhancing beamline performance. Mamba Data Worker (MDW) has been developed to address the data challenges for the forthcoming High Energy Photon Source (HEPS). It is an important component of the Mamba experimental control and data acquisition software ecosystem, which enables fast data acquisition and transmission, dynamic configuration of data processing pipelines, data multiplex in streaming, and customized data and metadata assembly. This paper presents the architecture and development plan of MDW, outlines the essential technologies involved, and illustrates its current application at the Beijing Synchrotron Radiation Facility (BSRF) 
650 4 |a Journal Article 
650 4 |a beamline software 
650 4 |a data acquisition 
650 4 |a data management 
650 4 |a high-throughput experiment 
650 4 |a multimodal experiments 
700 1 |a Zhang, Yi  |e verfasserin  |4 aut 
700 1 |a Liu, Yu  |e verfasserin  |4 aut 
700 1 |a Li, Pengcheng  |e verfasserin  |4 aut 
700 1 |a Hu, Hao  |e verfasserin  |4 aut 
700 1 |a Wang, Liwen  |e verfasserin  |4 aut 
700 1 |a He, Ping  |e verfasserin  |4 aut 
700 1 |a Dong, Yuhui  |e verfasserin  |4 aut 
700 1 |a Zhang, Chenglong  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t Journal of synchrotron radiation  |d 1994  |g 30(2023), Pt 6 vom: 01. Nov., Seite 1086-1091  |w (DE-627)NLM09824129X  |x 1600-5775  |7 nnas 
773 1 8 |g volume:30  |g year:2023  |g number:Pt 6  |g day:01  |g month:11  |g pages:1086-1091 
856 4 0 |u http://dx.doi.org/10.1107/S1600577523006951  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_40 
912 |a GBV_ILN_350 
912 |a GBV_ILN_2005 
951 |a AR 
952 |d 30  |j 2023  |e Pt 6  |b 01  |c 11  |h 1086-1091