mcatCS: A Highly Efficient Cross-matching Scheme for Multi-band Astronomical Catalogs

doi:10.1088/1538-3873/ab024c

CORC > 国家天文台 > 中国科学院国家天文台

	mcatCS: A Highly Efficient Cross-matching Scheme for Multi-band Astronomical Catalogs
	Li,Bingyao 1; Yu,Ce 1; Li,Chen 1; Hu,Xiaoteng 1; Xiao,Jian 1; Tang,Shanjiang 1; Cui,Chenzhou 2; Fan,Dongwei 2
刊名	Publications of the Astronomical Society of the Pacific
	2019-03-19
卷号	131 期号:999
关键词	methods: data analysis techniques: miscellaneous catalogs surveys
ISSN号	0004-6280
DOI	10.1088/1538-3873/ab024c
英文摘要	Abstract Multi-band astronomical catalog cross-matching has always been, and will continue to be, indispensable to astronomy research. However, the archived data volume in different wavebands is extremely huge, which results in the cross-matching process having high computational consumption and slow response. The complexity will also be augmented by the continuous growth of observational data. In this paper, we present mcatCS (multi-band catalog Cross-matching Scheme), a distributed cross-matching scheme to efficiently integrate celestial object data from billion-row multi-band astronomical catalogs. It is deployed on a cluster of commodity machines and provides a command-line-based interface to the end user. To allow fast cross-matching, the data in catalogs are reformatted into the Grouped Spatial Index File, which is a specially designed multi-band catalog uniform format. Furthermore, a min-conflicts data layout strategy is utilized to maximize the parallelization of cross-matching. Using real data, archived in the National Astronomical Observatories of China, we verify that mcatCS has good capabilities for performing efficient and reliable cross-matching between billion-row multi-band catalogs, and experimental results show that the query response speed is 38% to 45% greater than that of MongoDB and 21% to 32% greater than that of PostgreSQL with the HEALPix B-tree index. Moreover, although Q3C and H3C—the extension index packages for PostgreSQL—offer faster query response speed for less than 85 million sources, mcatCS proves to be advantageous after sources scale up to 100 million, and achieves a time reduction of 30.3% and 30.7% compared to Q3C and H3C for 200 million sources.
语种	英语
出版者	The Astronomical Society of the Pacific
WOS记录号	IOP:0004-6280-131-999-AB024C
内容类型	期刊论文
源URL	[http://ir.bao.ac.cn/handle/114a11/25324]
专题	中国科学院国家天文台
作者单位	1.College of Intelligence and Computing, Tianjin University, Tianjin 300350, People’s Republic of China; yuce@tju.edu.cn 2.National Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, People’s Republic of China
推荐引用方式 GB/T 7714	Li,Bingyao,Yu,Ce,Li,Chen,et al. mcatCS: A Highly Efficient Cross-matching Scheme for Multi-band Astronomical Catalogs[J]. Publications of the Astronomical Society of the Pacific,2019,131(999).
APA	Li,Bingyao.,Yu,Ce.,Li,Chen.,Hu,Xiaoteng.,Xiao,Jian.,...&Fan,Dongwei.(2019).mcatCS: A Highly Efficient Cross-matching Scheme for Multi-band Astronomical Catalogs.Publications of the Astronomical Society of the Pacific,131(999).
MLA	Li,Bingyao,et al."mcatCS: A Highly Efficient Cross-matching Scheme for Multi-band Astronomical Catalogs".Publications of the Astronomical Society of the Pacific 131.999(2019).