Quantile regression in big data: A divide and conquer based strategy

doi:10.1016/j.csda.2019.106892

CORC > 数学与系统科学研究院 > 中国科学院数学与系统科学研究院 > 应用数学研究所

	Quantile regression in big data: A divide and conquer based strategy
	Chen, Lanjue 1,2,3; Zhou, Yong 3,4
刊名	COMPUTATIONAL STATISTICS & DATA ANALYSIS
	2020-04-01
卷号	144 页码:17
关键词	Data stream Divide and conquer Estimating equation Massive data sets Quantile regression
ISSN号	0167-9473
DOI	10.1016/j.csda.2019.106892
英文摘要	Quantile regression, which analyzes the conditional distribution of outcomes given a set of covariates, has been widely used in many fields. However, the volume and velocity of big data make the estimation of quantile regression model extremely difficult due to the intensive computation and the limited storage. Based on divide and conquer strategy, a simple and efficient method is proposed to address this problem. The proposed approach only keeps summary statistics of each data block and then can use them to reconstruct the estimator of the entire data with asymptotically negligible approximation error. This property makes the proposed method particularly appealing when data blocks are retained in multiple servers or come in the form of data stream. Furthermore, the proposed estimator is shown to be consistent and asymptotically as efficient as the estimating equation estimator calculated using the entire data together when certain conditions hold. The merits of the proposed method are illustrated using both simulation studies and real data analysis. (C) 2019 Elsevier B.V. All rights reserved.
资助项目	State Key Program in the Major Research Plan of National Natural Science Foundation of China[91546202] ; State Key Program of National Natural Science Foundation of China[71931004]
WOS研究方向	Computer Science ; Mathematics
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:000515446200029
内容类型	期刊论文
源URL	[http://ir.amss.ac.cn/handle/2S8OKBNM/50840]
专题	应用数学研究所
通讯作者	Zhou, Yong
作者单位	1.City Univ Hong Kong, Dept Management Sci, Kowloon, Hong Kong, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China 4.East China Normal Univ, Key Lab Adv Theory & Applicat Stat & Data Sci, MOE, Acad Stat & Interdisciplinary Sci, Shanghai 200062, Peoples R China
推荐引用方式 GB/T 7714	Chen, Lanjue,Zhou, Yong. Quantile regression in big data: A divide and conquer based strategy[J]. COMPUTATIONAL STATISTICS & DATA ANALYSIS,2020,144:17.
APA	Chen, Lanjue,&Zhou, Yong.(2020).Quantile regression in big data: A divide and conquer based strategy.COMPUTATIONAL STATISTICS & DATA ANALYSIS,144,17.
MLA	Chen, Lanjue,et al."Quantile regression in big data: A divide and conquer based strategy".COMPUTATIONAL STATISTICS & DATA ANALYSIS 144(2020):17.