YNAO OpenIR  > 其他
Alternative TitleThe Study and Application of a High Performance UVFITS Assembly System Based on MPI
陈泰燃1; 王威3; 王锋1,2; 邓辉1; 刘应波1; 梅盈1
Source Publication天文研究与技术(Astronomical Research & Technology)
Contribution Rank第2完成单位
Indexed ByCSCD
KeywordUvfits 海量数据 Mpi 并行计算 高性能计算

中国明安图超宽频谱射电日像仪(Mingantu Ultrawide Spectral Radioheliograph,MUSER)进入实际观测后,每3 ms产生一帧100 k B左右的数据,每天的原始观测数据约3.5 TB。由于射电日像仪的原始数据采用自定义格式,为了后续数据分析和共享的需要,有必要根据数据存储需求把这些原始数据转换成天文常用的文件格式。在前期工作中已经实现了原始数据格式到UVFITS文件的转换,在此基础上研究了基于MPI的集群并行环境下UVFITS合成系统的性能优化。通过实验验证,在改进后的并行环境下,UVFITS合成系统的性能达到了需求的2.5倍,可以有效处理当前及未来一定时间内射电日像仪的海量观测数据。同时,改进后的系统具有良好的横向扩展能力,能够为相关项目的数据处理提供借鉴和参考。

Other Abstract

Mingantu Ultrawide Spectral Radioheliograph (MUSER) generates 100 kilobytes raw observational data in every 3 milliseconds and more than 3. 5 Terabytes data per day. For further data analysis and sharing, it is necessary to convert the raw data stored in self-defined format to standard format usually used in the field of radio astronomy. In previous work,we have analyzed the format of UVFITS and converted the raw data to UVFITS file successfully. However, the efficiency of the format converting system needs to be further improved. This paper presents a parallel UVFITS file assembly system based on cluster parallel environment. Experiments show that the system can reduce the execution time of assembling a UVFITS file to about 1.2 milliseconds, 2.5 times faster than that of the data acquisition, which is very promising to meet the data processing requirements in the project. Moreover, the parallel system can be used for reference in other systems. The implementation of this parallel data format converting system can also provide a good reference to similar data processing systems.

Funding Project国家自然科学天文联合基金[U1231205] ; 国家自然科学基金[11263004] ; 国家自然科学基金[11203011] ; 国家自然科学基金[11163004] ; 国家自然科学基金[11103005] ; 云南省应用基础基金重点项目[2013FA013] ; 云南省应用基础基金重点项目[2013FA032] ; 云南省应用基础基金重点项目[2013FZ018]
Funding Organization国家自然科学天文联合基金[U1231205] ; 国家自然科学基金[11263004, 11203011, 11163004, 11103005] ; 云南省应用基础基金重点项目[2013FA013, 2013FA032, 2013FZ018]
Subject Area计算机科学技术 ; 计算机软件 ; 计算机应用
MOST Discipline Catalogue工学 ; 工学::计算机科学与技术(可授工学、理学学位)
Citation statistics
Cited Times:5[CSCD]   [CSCD Record]
Document Type期刊论文
Corresponding Author王锋
2.中国科学院云南天文台, 云南,昆明,650012
Corresponding Author AffilicationYunnan Observatories, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
陈泰燃,王威,王锋,等. 基于MPI的高性能UVFITS数据合成研究与应用[J]. 天文研究与技术(Astronomical Research & Technology),2016,13(2):184-189.
APA 陈泰燃,王威,王锋,邓辉,刘应波,&梅盈.(2016).基于MPI的高性能UVFITS数据合成研究与应用.天文研究与技术(Astronomical Research & Technology),13(2),184-189.
MLA 陈泰燃,et al."基于MPI的高性能UVFITS数据合成研究与应用".天文研究与技术(Astronomical Research & Technology) 13.2(2016):184-189.
Files in This Item:
File Name/Size DocType Version Access License
基于MPI的高性能UVFITS数据合成研(272KB)期刊论文出版稿开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[陈泰燃]'s Articles
[王威]'s Articles
[王锋]'s Articles
Baidu academic
Similar articles in Baidu academic
[陈泰燃]'s Articles
[王威]'s Articles
[王锋]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[陈泰燃]'s Articles
[王威]'s Articles
[王锋]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 基于MPI的高性能UVFITS数据合成研究与应用_陈泰燃.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.