YNAO OpenIR  > 其他
射电天文数据实时计算的关键技术研究
其他题名Research on Key Technologies in Radio Astronomical Data Realtime Processing
戴伟
学位类型博士
导师王锋
2019-07-01
学位授予单位中国科学院大学
学位授予地点北京
学位专业天文技术与方法
关键词海量数据 实时处理 实时采集 相干消色散 射电干涉成像
摘要射电数据是射电天文观测的结果和进行射电天文学研究的核心基础。随着射电设备和观测技术的发展,新一代射电望远镜的涌现,射电天文学家获取天文观测数据的能力得到了空前的加强。射电天文的数据已经以一种实时、顺序、海量和无限的方式到达。如何通对观测数据进行实时处理使得射电天文学家面对着前所未有的挑战。本文针对云南天文台40米射电望远镜脉冲星数字终端以及中国新一代厘米-分米波综合孔径望远镜即明安图射电频谱日像仪(MingantU SpEctral Radioheliograph, MUSER)的实时数据处理面临的问题开展研究,具体工作说明如下:(1) 实时数据采集是大多数天文数据实时处理流水线的起点,本研究分析了当前射电天文数据流传输的特点,针对传统Linux操作系统网络协议栈性能较低的问题,采用以内核旁路、零拷贝的用户态空间网络加速技术,实现了万兆以太网络下线速的无丢失的数据采集,以此技术为基础的数据采集框架,已经成功应用于云台40米脉冲星的数字后端的数据采集。(2) 近年来,图像处理器(Graphics Processing Unit,GPU)技术得到了快速的发展,由于优异的浮点运算性能和较好的性价比,天文海量数据的实时数据处理除了采用传统的专用硬件芯片、中央处理器解决方案外,采用GPU用于实时数据处理已经成为必然的趋势。本文初步设计了一个基于GPU的天文海量数据处理的实时计算框架。并将该计算框架应用于相干消色散的实时数据处理中,实现了从解码、转置、傅里叶变换、消色散、分通道、逆傅里叶变换、偏振检测、折叠、消噪、归档输出等数据处理模块,并在40米望远镜上进行了试观测。此外,针对40米望远镜脉冲星观测面临的射频干扰问题,利用独立成分分析对混合信号进行分解,根据观测信号中脉冲星信号和干扰信号的分布特点,分解出独立的射频干扰信号和脉冲星信号,消除射频干扰信号。(3) 实时处理需要充分利用所有的计算资源,在异构环境下实现天文计算软件的无缝运行和快速移植,能够提高计算资源的利用效率。对OpenCL技术实现异构环境下实时天文数据处理的可用性进行了研究,并在此基础上实现了MUSER的射电干涉阵成像网格化算法和洁化算法,在保证了算法运行效率基本不变的基础上,其硬件不再限制于NVIDIA的GPU环境,为异构环境下实时数据并行处理进一步提供了扩展性。(4) 采用分布式系统在集群上实时处理数据处理水平扩展计算能力也是一个主要研究内容。为了提高分布式实时计算环境的弹性部署和自动扩展的效率,研究了基于轻量级容器Docker技术的天文实时处理集群敏捷化构建与部署,并使用容器技术封装MUSER现有系统,并在不同硬件模式下与物理机和其他虚拟机技术上的封装进行性能测试比较。论文研究了目前海量天文数据实时处理的若干问题,实际应用性较强。基于用户空间态的数据采集技术可应用数字终端前后端的超高IO的数据通讯。利用GPU和OpenCL构建的异构计算平台下可为实时数据处理提供加速。而轻量级容器的虚拟化,方便构建实时计算集群。研究内容为天文数据的实时处理提供了参考,为下一步相关工作打下了良好的基础。
其他摘要Data is the result of astronomical observations and the core basis for conducting astronomical research. With the development of astronomical equipment and observation technology, a new generation of astronomical telescopes has emerged, and the ability of astronomers to acquire astronomical observation data has been unprecedentedly strengthened. Astronomical data has arrived in a real-time, sequential, massive and infinite way. How to process observation data in real time through distribution and parallel means that astronomers face unprecedented challenges.This paper studies the problems faced by the Yunnan Observatory's 40-meter radio telescope pulsar digital terminal and the real-time data processing of new generation cm-decimeter aperture telescope, MingantU SpEctral Radioheliograph (MUSER). The detailed work description is as follows:(1) Real-time data acquisition is the starting point of most real-time processing pipelines for astronomical data. This study analyzes the characteristics of current radio astronomy data stream transmission, and uses kernel bypass and zero-copy user-space network acceleration technology to solve the performance of traditional Linux operating system network protocol stack. This paper realizes the lossless data acquisition of line rate under 10 Gigabit Ethernet. The data acquisition framework based on this technology has been successfully applied to the data acquisition of the digital back end of the YNAO 40-meter pulsar.(2) In addition to the use of dedicated hardware chips, the central processor's traditional solution to deal with astronomical massive data, due to excellent floating-point performance and better cost performance, the use of graphics processing units (GPU) for real-time data processing has become an inevitable trend. A real-time massive data processing framework based on GPU ss designed. The framework is applied to the real-time data processing of coherent dedispersion, which realizes decoding, transposition, Fourier transform, dedispersion, subchannel, inverse Fourier transform, polarization detection, folding, denoising, and archiving. The data processing module was tested on YNAO 40-meter telescope.(3) Real-time processing needs to make full use of all computing resources to achieve seamless operation and fast migration of astronomical computing software in a heterogeneous environment, which can improve the utilization efficiency of computing resources. We have studied the application of OpenCL technology in real-time astronomical data processing in heterogeneous environments. Based on this, MUSER's imaging gridding algorithm and cleaning algorithm are implemented, which guarantees that the algorithm's operating efficiency is basically unchanged. It is no longer limited to NVIDIA's GPU environment, providing further scalability for real-time data parallel processing in heterogeneous environments.(4) Real-time processing of data processing horizontal expansion computing capability on a cluster using a distributed system is also a major research content. In order to improve the efficiency of flexible deployment and automatic expansion of distributed real-time computing environment, the astronomical real-time processing cluster agile construction and deployment based on lightweight container Docker technology is studied, and the MUSER existing system is encapsulated by container technology, and in different hardware. The performance is compared with the physical machine and other virtual machine technology.The paper studies several problems in the real-time processing of massive astronomical data. The data acquisition technology based on the user space can apply the data communication of the super high IO at the front and rear ends of the digital terminal. Data processing based on GPUs and OpenCL with heterogeneous computing platforms can greatly help accelerate computati and can enable better, faster science. The virtualization of lightweight containers makes it easy to build real-time computing clusters. The research content provides a reference for the real-time processing of astronomical data, laying a good foundation for the next step of related work.
学科领域天文学 ; 射电天文学 ; 射电天文方法 ; 计算机科学技术 ; 计算机应用
学科门类理学 ; 理学::天文学 ; 工学 ; 工学::计算机科学与技术(可授工学、理学学位)
页数101
语种中文
文献类型学位论文
条目标识符http://ir.ynao.ac.cn/handle/114a53/25458
专题其他
作者单位中国科学院云南天文台
第一作者单位中国科学院云南天文台
推荐引用方式
GB/T 7714
戴伟. 射电天文数据实时计算的关键技术研究[D]. 北京. 中国科学院大学,2019.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
射电天文数据实时计算的关键技术研究.pd(3568KB)学位论文 开放获取CC BY-NC-SA浏览 请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[戴伟]的文章
百度学术
百度学术中相似的文章
[戴伟]的文章
必应学术
必应学术中相似的文章
[戴伟]的文章
相关权益政策
暂无数据
收藏/分享
文件名: 射电天文数据实时计算的关键技术研究.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。