In this section, we turn to the second facet of computational social research—research design utilizing crowdsourcing technology to extract collective intelligence. We focus on one stage in a research project, data generation, including data collection and data production processes such as text annotation.
We discuss three such types of research designs: 1) an open survey that evolves over time based on the ideas of its participants; 2) a system that distributes microtasks in the crowd, whose outputs are as reliable and valid as those from expert human readers; and 3) a software application that interfaces with crowdsourcing technology, and that automates recruiting, collecting both behavior and survey data, and providing incentives to generate responses all in one stop.
These kinds of research design have the potential to improve the scope, efficiency, cost, scalability, sampling, response rates, and convenience of social scientific projects, compared to research in the analog age. Challenges of data quality control, assessing response biases, or adjusting sampling biases can be addressed in the design phase or analysis of data.