An Empirical Study of Workers’ Behavior in Spatial Crowdsourcing

We performed a spatial crowdsourcing experiment with more than 238 real users in two separate campaigns (one with fixed rewards and one with increasing rewards). Particularly, we designed a Genkii app that allows users to obtain a monetary reward for reporting their mood / sentiment and rewarded taskers through the Yahoo Japan Crowdsourcing payment platform. We then analyzed the responses to identify the effects fixed vs. increasing rewards, the effects of mobility, temporal patterns, retainment analysis, etc. Our findings in this study are three-fold.

We first report the worker performance during the two campaigns. We obtained a total of 1059 reports from both campaigns, out of which 436 reports were from the first campaign and 623 reports from the second. We observe an “on-boarding effect” in both campaigns in which 40% of the users (with at least one report) made only one report. At the same time, 24% of the users are considered active, who made at least 10 reports. We also observed peak numbers of reports during pastimes in Japan (4, 12 and 20) and the least number of reports during commute times (1, 9 and 17).

Second, we compared user participation in the two reward strategies (i.e., how well a user is retained in our 10-task campaign). Our analysis shows that the overall user participation decreases significantly in both campaigns, among which the drop rate is less in the increasing reward campaign. Particularly, the largest drop rate is between the first and the second reports. This result shows that workers are motivated by growing incentives to stay in the campaign. In addition, with the increasing reward campaign, 17% of the users finish the 10-task campaign while this number is only 11% with the fixed reward campaign.

Third, we studied worker mobility from the reporting locations. We categorized Genkii users with at least six reports based on their mobility. Each worker has a certain degree of mobility defined as the area of the minimum bounding rectangle that encloses all the reporting locations, which is highly correlated with his/her commuting pattern. We observed that 75% of the workers travel within 500 square km. This result suggests that users tend to contribute data in the proximity of their homes. In addition, users are more likely to report Happy mood if they commute long distances named “Commuter”, while the ones who travel short distances, the so-called “House Dweller” have a large fraction of Dull reports.

Hien To, Ruben Geraldes, Cyrus Shahabi, Seon Ho Kim, and Helmut Prendinger, An Empirical Study of Workers’ Behavior in Spatial Crowdsourcing, Third International ACM SIGMOD Workshop on Managing and Mining Enriched Geo-Spatial Data, San Francisco, CA, USA, June 26 – July 1, 2016