Analyzing the quality of information solicited from targeted strangers on social media

Abstract

The emergence of social media creates a unique opportunity for developing a new class of crowd-powered information collection systems. Such systems actively identify potential users based on their public social media posts and solicit them directly for information. While studies have shown that users will respond to solicitations in a few domains, there is little analysis of the quality of information received. Here we explore the quality of information solicited from Twitter users in the domain of product reviews, specifically reviews for a popular tablet computer and L.A.-based food trucks. Our results show that the majority of responses to our questions (>70%) contained relevant information and often provided additional details (>37%) beyond the topic of the question. We compare the solicited Twitter reviews to other user-generated reviews from Amazon and Yelp, and found that the Twitter answers provided similar information when controlling for the questions asked. Our results also reveal limitations of this new information collection method, including its suitability in certain domains and potential technical barriers to its implementation. Our work provides strong evidence for the potential of this new class of information collection systems and design implications for their future use. Copyright 2013 ACM.