YNAO OpenIR  > 其他
OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing
Wei SL(卫守林)1,2,3; Wang F(王锋)1,2,3; Deng, Hui2; Liu, Cuiyin2; Dai W(戴伟)1,2,3; Liang, Bo2; Mei Y(梅盈)1,2,3; Shi, Congming2; Liu, Yingbo4; Wu, Jingping2; Wang, F
发表期刊PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC
2017-02-01
卷号129期号:972
DOI10.1088/1538-3873/129/972/024001
产权排序第1完成单位
收录类别SCI
关键词Instrumentation: Interferometers Methods: Data Analysis Methods: Numerical Techniques: Image Processing Techniques: Interferometric
摘要The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their limitations and programming difficulties. In this paper, we therefore present OpenCluster, an open-source distributed computing framework to support rapidly developing high-performance processing pipelines of astronomical big data. We first detail the OpenCluster design principles and implementations and present the APIs facilitated by the framework. We then demonstrate a case in which OpenCluster is used to resolve complex data processing problems for developing a pipeline for the Mingantu Ultrawide Spectral Radioheliograph. Finally, we present our OpenCluster performance evaluation. Overall, OpenCluster provides not only high fault tolerance and simple programming interfaces, but also a flexible means of scaling up the number of interacting entities. OpenCluster thereby provides an easily integrated distributed computing framework for quickly developing a high-performance data processing system of astronomical telescopes and for significantly reducing software development expenses.
项目资助者Kunming University of Science and Technology ; Yunnan Observatories of Chinese Academy of Sciences ; National Natural Science Foundation of China(U1231205, 11403009) ; Applied Basic Research Foundation of Yunnan Province(2013FA013, 2013FA032, 2013FZ018)
语种英语
学科领域天文学
文章类型Article
ISSN0004-6280
URL查看原文
WOS记录号WOS:000391578300001
WOS研究方向Astronomy & Astrophysics
WOS类目Astronomy & Astrophysics
关键词[WOS]MAPREDUCE ; COMPUTATIONS
引用统计
文献类型期刊论文
条目标识符http://ir.ynao.ac.cn/handle/114a53/9874
专题其他
通讯作者Wang, F
作者单位1.Yunnan Observatories, Chinese Academy of Sciences, China
2.Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, China
3.University of Chinese Academy of Sciences, China
4.Yunnan academy of scientific & technical information, China
推荐引用方式
GB/T 7714
Wei SL,Wang F,Deng, Hui,et al. OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing[J]. PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC,2017,129(972).
APA Wei SL.,Wang F.,Deng, Hui.,Liu, Cuiyin.,Dai W.,...&Wang, F.(2017).OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing.PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC,129(972).
MLA Wei SL,et al."OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing".PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC 129.972(2017).
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
OpenCluster_ A Flexi(1736KB)期刊论文作者接受稿开放获取CC BY-NC-SA浏览 请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Wei SL(卫守林)]的文章
[Wang F(王锋)]的文章
[Deng, Hui]的文章
百度学术
百度学术中相似的文章
[Wei SL(卫守林)]的文章
[Wang F(王锋)]的文章
[Deng, Hui]的文章
必应学术
必应学术中相似的文章
[Wei SL(卫守林)]的文章
[Wang F(王锋)]的文章
[Deng, Hui]的文章
相关权益政策
暂无数据
收藏/分享
文件名: OpenCluster_ A Flexible Distributed Computing Framework for Astronomical Data Processing.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。