YNAO OpenIR  > 其他
Alternative TitleThe Application of ElasticSearch in the Massive Astronomical Data Retrieval
陈亚杰1; 王锋1,2; 邓辉1; 刘应波1
Source Publication天文学报(Acta Astronomica Sinica)
Contribution Rank第2完成单位
Indexed ByCSCD ; 核心
Keyword方法 数值 解析 天文数据库 其他诸多方面

天文观测数据是天文研究的基础,但传统的集中式数据检索方法已难以满足日益增长的海量天文数据的高性能检索和查询需求.提出了一种基于Elastic Search分布式搜索引擎,通过River机制对现有的海量FITS(Flexible Image Transport System)数据进行索引构建,从而实现海量FITS数据高效检索的方法,并讨论了其中的近实时检索和查询的关键技术.实测结果表明,在百万到千万级的天文数据量下,该方法可获得极高的检索性能,并能够很方便地集成到现有的天文数据归档系统中,完全可以满足当前国内各类望远镜系统天文数据的归档要求.

Other Abstract

Astronomical observational data are the fundamental element for modern astronomical researches. However, with the rapid increase of astronomical data, the traditional centralized retrieval methods are hard to meet the requirements of highperformance data retrieval. In the study, we present a novel method which is based on the Elastic Search distributed retrieval engine and River mechanism to create data indexes, and provide high performance data retrieval for massive FITS(Flexible Image Transport System) data. We discuss the key technologies of the nearly real-time retrieval and query. The experimental results show that the method is capable of obtaining high retrieval performance especially for the cases in which the number of the FITS data exceeds millions or even tens of millions. Meanwhile, the method can be easily integrated into the current astronomical data archiving systems, and completely meet the archive requirements of all kinds of astronomical telescope systems.

Funding Project国家自然科学基金项目[U1231205] ; 国家自然科学基金项目[11303011] ; 国家自然科学基金项目[11263004] ; 国家自然科学基金项目[11463003] ; 国家自然科学基金项目[11163004] ; 云南省应用基础基金重点项目[2013FA013] ; 云南省应用基础基金重点项目[2013FA032]
Funding Organization国家自然科学基金项目[U1231205, 11303011, 11263004, 11463003, 11163004] ; 云南省应用基础基金重点项目[2013FA013, 2013FA032]
Subject Area天文学 ; 天文学其他学科 ; 计算机科学技术 ; 计算机软件 ; 计算机应用
MOST Discipline Catalogue理学 ; 理学::天文学 ; 工学 ; 工学::计算机科学与技术(可授工学、理学学位)
Citation statistics
Cited Times:1[CSCD]   [CSCD Record]
Document Type期刊论文
Recommended Citation
GB/T 7714
陈亚杰,王锋,邓辉,等. ElasticSearch分布式搜索引擎在天文大数据检索中的应用研究[J]. 天文学报(Acta Astronomica Sinica),2016,57(2):241-251.
APA 陈亚杰,王锋,邓辉,&刘应波.(2016).ElasticSearch分布式搜索引擎在天文大数据检索中的应用研究.天文学报(Acta Astronomica Sinica),57(2),241-251.
MLA 陈亚杰,et al."ElasticSearch分布式搜索引擎在天文大数据检索中的应用研究".天文学报(Acta Astronomica Sinica) 57.2(2016):241-251.
Files in This Item:
File Name/Size DocType Version Access License
ElasticSearch分布式_省略_(869KB)期刊论文出版稿开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[陈亚杰]'s Articles
[王锋]'s Articles
[邓辉]'s Articles
Baidu academic
Similar articles in Baidu academic
[陈亚杰]'s Articles
[王锋]'s Articles
[邓辉]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[陈亚杰]'s Articles
[王锋]'s Articles
[邓辉]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: ElasticSearch分布式_省略_引擎在天文大数据检索中的应用研究_陈亚杰.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.