Monitoring the data quality of data streams using a two-step control scheme

doi:10.1080/24725854.2018.1530487

CORC > 上海财经大学 > 上海财经大学

	Monitoring the data quality of data streams using a two-step control scheme
	Yu, Miaomiao 1; Wu, Chunjie 1; Tsung, Fugee 2
刊名	IISE TRANSACTIONS
	2019-09-02
卷号	51 期号:9 页码:985-998
关键词	Data quality two-step control scheme weighted-sum statistic data streams Mahalanobis distance
ISSN号	2472-5854
DOI	10.1080/24725854.2018.1530487
英文摘要	Data-rich environments provide unprecedented opportunities for monitoring data quality. This article focuses on the quality of data streams. We use indicator variables to measure the six dimensions of data quality and a glitch index to indicate the poor level of quality. A two-step control scheme is proposed considering two relationships: the inter- and intra-correlation. In the first step, the Mahalanobis distance is applied to an chi(2)-type control chart to monitor the quality of a data stream. In the second step, a Shewhart control chart is built based on a weighted-sum statistic, which measures the quality of the whole process. The feasibility and effectiveness of the control scheme are illustrated through detailed simulation studies and one landslide example. The simulated results, considering the three cases of no correlation, low correlation, and high correlation, show that the proposed approach can detect the mean shift in multi-attribute data sensitively and robustly. The example, in which sensors are used to collect data on accelerations in Taiwan, demonstrates the superiority of our design over four traditional control charts, producing the closest type-I error to the given level and the highest power under the same type-I error.
WOS研究方向	Engineering ; Operations Research & Management Science
语种	英语
出版者	TAYLOR & FRANCIS INC
WOS记录号	WOS:000472189100005
内容类型	期刊论文
源URL	[http://10.2.47.112/handle/2XS4QKH4/193]
专题	上海财经大学
通讯作者	Tsung, Fugee
作者单位	1.Shanghai Univ Finance & Econ, Sch Stat & Management, Shanghai, Peoples R China; 2.Hong Kong Univ Sci & Technol, Dept Ind Engn & Decis Analyt, Kowloon, Hong Kong, Peoples R China
推荐引用方式 GB/T 7714	Yu, Miaomiao,Wu, Chunjie,Tsung, Fugee. Monitoring the data quality of data streams using a two-step control scheme[J]. IISE TRANSACTIONS,2019,51(9):985-998.
APA	Yu, Miaomiao,Wu, Chunjie,&Tsung, Fugee.(2019).Monitoring the data quality of data streams using a two-step control scheme.IISE TRANSACTIONS,51(9),985-998.
MLA	Yu, Miaomiao,et al."Monitoring the data quality of data streams using a two-step control scheme".IISE TRANSACTIONS 51.9(2019):985-998.