Aggregating Crowdsourced Quantitative Claims: Additive and Multiplicative Models

doi:10.1109/TKDE.2016.2535383

	Aggregating Crowdsourced Quantitative Claims: Additive and Multiplicative Models
	Ouyang, Robin Wentao 1; Kaplan, Lance M.2; Toniolo, Alice 3; Srivastava, Mani 4,5; Norman, Timothy J.3
	2016-07-01
发表期刊	IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
ISSN	1041-4347
卷号	28 期号:7 页码:1621-1634
摘要	Truth discovery is an important technique for enabling reliable crowdsourcing applications. It aims to automatically discover the truths from possibly conflicting crowdsourced claims. Most existing truth discovery approaches focus on categorical applications, such as image classification. They use the accuracy, i.e., rate of exactly correct claims, to capture the reliability of participants. As a consequence, they are not effective for truth discovery in quantitative applications, such as percentage annotation and object counting, where similarity rather than exact matching between crowdsourced claims and latent truths should be considered. In this paper, we propose two unsupervised Quantitative Truth Finders (QTFs) for truth discovery in quantitative crowdsourcing applications. One QTF explores an additive model and the other explores a multiplicative model to capture different relationships between crowdsourced claims and latent truths in different classes of quantitative tasks. These QTFs naturally incorporate the similarity between variables. Moreover, they use the bias and the confidence instead of the accuracy to capture participants' abilities in quantity estimation. These QTFs are thus capable of accurately discovering quantitative truths in particular domains. Through extensive experiments, we demonstrate that these QTFs outperform other state-of-the-art approaches for truth discovery in quantitative crowdsourcing applications and they are also quite efficient.
关键词	Crowdsourcing truth discovery quantitative task probabilistic graphical model
DOI	10.1109/TKDE.2016.2535383
收录类别	SCI
语种	英语
资助项目	U.S. ARL ; U.K. Ministry of Defense[W911NF-06-3-0001] ; U.S. NSF[CNS-1213140]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical & Electronic
WOS记录号	WOS:000380117500002
出版者	IEEE COMPUTER SOC
引用统计	被引频次：15[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/8261
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Ouyang, Robin Wentao
作者单位	1.Chinese Acad Sci, Inst Comp Technol, CAS Key Lab Network Data Sci & Technol, Beijing, Peoples R China 2.US Army Res Lab, Networked Sensing & Fus Branch, Adelphi, MD 20783 USA 3.Univ Aberdeen, Dept Comp Sci, Aberdeen, Scotland 4.Univ Calif Los Angeles, Dept Elect Engn, Los Angeles, CA 90095 USA 5.Univ Calif Los Angeles, Dept Comp Sci, Los Angeles, CA 90095 USA
推荐引用方式 GB/T 7714	Ouyang, Robin Wentao,Kaplan, Lance M.,Toniolo, Alice,et al. Aggregating Crowdsourced Quantitative Claims: Additive and Multiplicative Models[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2016,28(7):1621-1634.
APA	Ouyang, Robin Wentao,Kaplan, Lance M.,Toniolo, Alice,Srivastava, Mani,&Norman, Timothy J..(2016).Aggregating Crowdsourced Quantitative Claims: Additive and Multiplicative Models.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,28(7),1621-1634.
MLA	Ouyang, Robin Wentao,et al."Aggregating Crowdsourced Quantitative Claims: Additive and Multiplicative Models".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 28.7(2016):1621-1634.