18 Cultivating Social Science Data in the Age of Agentic AI
Pratik Sachdeva
University of California, Berkeley
Dennis M. Feehan
University of California, Berkeley
feehan@berkeley.edu
Abstract: Much excitement about the future of quantitative social science surrounds how it can make what we already do faster and more efficient: for example, agentic AI can accelerate the time from idea to initial results,1 it promises faster and more comprehensive literature reviews,2 it can help develop and check mathematical derivations,3 and it can give high-quality feedback4 on draft manuscripts. These parts of “future” have arrived first. But once researchers figure out how to work with these new tools, there will also be changes to what quantitative social scientists do. We will examine these changes by focusing on the ways that agentic AI will enable the production of new data. We argue that social science data is generated through one of three mechanisms: 1) foraged data, which the researcher gathers but which are created by people completely outside of the researcher’s control; 2) farmed data, which the researcher designs and which is created as the result of interactions between the researcher and people; and 3) lab-grown data, which is created by the researcher in isolation, without any interaction with people. Agentic AI will affect how researchers produce all three types of data. These changes will arrive as technological and methodological innovations, but we expect their ultimate impact to be greatest on theory: new forms of observations will enrich existing theories and propel the creation of new ones.
AI usage statement: AI tools (Claude Opus 4.8 and GPT 5.5 Pro) were used to (i) support brainstorming, particularly in the development of the overarching metaphor; (ii) conduct literature review; (iii) provide feedback on clarity, tone, and grammar of the text; and to (iv) copyedit.
18.1 Introduction
Much excitement about the future of quantitative social science surrounds how it can make what we already do faster and more efficient: for example, agentic AI can accelerate the time from idea to initial results,5 it promises faster and more comprehensive literature reviews,6 it can help develop and check mathematical derivations,7 and it can give high-quality feedback8 on draft manuscripts. These parts of the “future” have arrived first. But once researchers figure out how to work with these new tools, there will also be changes to what quantitative social scientists do. We will examine these changes by focusing on the ways that agentic AI will enable the production of new data. We argue that social science data is generated through one of three mechanisms: 1) foraged data, which the researcher gathers but which are created by people completely outside of the researcher’s control; 2) farmed data, which the researcher designs and which is created as the result of interactions between the researcher and people; and 3) lab-grown data, which is created by the researcher in isolation, without any interaction with people. Agentic AI will affect how researchers produce all three types of data. These changes will arrive as technological and methodological innovations, but we expect their ultimate impact to be greatest on theory: new forms of observations will enrich existing theories and propel the creation of new ones.
18.2 Foraged Data
Many research projects have been frustratingly stalled because the right data needed to answer a research question are large, unstructured, or disjointed in a way that makes them inaccessible to social scientists. Perhaps the data exist in scattered databases and needs heavy cleaning and linking, or maybe the data are in a format that requires advanced technical skills to work with. The throughline is that the data exist, but due to lack of resources, time, expertise, or the right collaborator, it remains frozen, effectively inaccessible to help answer the research question at hand.
The promise of agentic AI means that this data can now be foraged. Foraged data are created as the result of a social system that is not controlled or designed by the researcher, such as data from historical archives, administrative records, and most digital trace data. The researcher studies this system by observing and collecting measurements that can then be analyzed. Because the social system is not controlled by the researcher, the data of interest are also not controlled, and so it may be challenging to collect or transform it into a format suitable for the research question.
Agentic AI promises to support the production of foraged data because it will lower the barriers for researchers to work with the tools needed to collect, organize, assemble, and analyze these data. We articulate two dimensions on which agentic AI can help researchers forage data. First, it can support assembly of data: creating the dataset needed for a specific purpose by assembling and connecting existing data sources. Second, they can support analysis of data: streamlining the process of analyzing foraged data to support research development. We discuss each in turn.
18.2.1 Assembly
Scouring for the right data – whether that be trudging through web APIs to query the data one needs or converting scanned PDFs into usable text data – is one of the most time consuming parts of computationally driven work. The payoff is evident: among the most useful and important datasets in social science are those in which assembly was the crowning achievement, such as the historical census microdata assembled by IPUMS.9 In these cases, the work needed to forage the data was so great that it was considered a scholarly contribution itself.
Assembly is difficult partially because every pipeline will be different based on the domain and needs of the research question. There are a common set of data analysis operations that will be invoked repeatedly, such as identifying the correct data sources; acquiring the data from these sources using a combination of web APIs, web scraping, or scanning documents; wrangling the data in all its messiness into a well structured schema; and conducting record linkage across multiple datasets, if needed.
Nonetheless, agentic AI is increasingly capable at all of these intermediate steps. Agentic AI models, via “tool use” can autonomously access web APIs (Schick et al. 2023); they are also highly capable at web scraping and computer use, when permitted (Zhou et al. 2024). AI models – with their ability to leverage both world knowledge and pattern matching – have been repeatedly demonstrated to exhibit state-of-the-art data structuring, schema design, and entity linkage (Narayan et al. 2022; Peeters et al. 2024; Parciak et al. 2024). The effectiveness and impacts of these capabilities have only increased with the product developments that have accompanied the age of agentic AI. There remain uncertainties here about to what degree agentic AI can autonomously conduct an entire pipeline given a human-driven specification. But what is clear is that there are now datasets that can be foraged by researchers without the researchers needing either highly technical knowledge or having to write a single line of code.
One possible consequence of agentic AI lowering the barriers to foraging data is that it may become easier to create bespoke datasets. The time intensiveness and cost of data assembly often meant that it was more efficient for it to be carried out by centralized institutions. Thus, research designs might conform to pre-assembled, general-purpose data. If, however, data can be easily foraged, then it is possible that researchers can assemble datasets on-the-fly for narrowly specified research questions.
18.2.2 Analysis
The age of big data brought about the development of computational social science (Salganik 2019), where many social science questions could be answered using computational, statistical, or machine learning methods. The result is one of vibrant research development, opening many possibilities with respect to the types of researcher questions that could be asked and answered. The fruits of these methods could only be realized, however, provided that the researcher had access to the technical skills and know-how to apply them to the problem.
For example, until recently, a social scientist with a large corpus who needed to label the data according to some schema has had to:
- obtain training in machine learning methodology;
- gather the requisite data;
- label the dataset, often with research assistants to obtain high quality, “ground truth” labels;
- train and evaluate the machine learning model;
- deploy the model; and
- adequately analyze the model’s outputs and contextualize them for the scientific problem.
Often, completing these steps required months of technical training, computational resources, or a knowledgeable collaborator. Now, it is completely feasible to write up a prompt and have Claude Code or Codex simply do all of the above. As these tools improve, it seems likely that social scientists will no longer need to really be concerned with the details of training an ML model, or worrying about validation loss, etc.; these are just technical matters to be sorted out by the agent. One only needs a well-posed research question. Of course, models can make mistakes or make bad assumptions – in the short term, technical expertise is still required to ensure that the output of agentic AI tools is correct. But the increasing capabilities of agentic tooling, together with improved training around using agentic AI tools, means that the cautious researcher can expect to make much further progress than ever before.
To be clear, we do not mean to suggest that agentic tooling can autonomously conduct any social science analysis pipeline (yet). But there are a wide array of notable analysis techniques, commonly encountered in the analysis of social science data, that can be streamlined with agentic AI, such as:
- Data annotation: The “LLM-as-rater” or “LLM-as-judge” paradigms are increasingly used to flexibly apply labels or code foraged data, often at an accuracy and speed that exceeds that of humans (Gilardi et al. 2023; Ziems et al. 2024). The flexibility is important: any specification of a labeling scheme in natural language, sufficiently detailed, can be handled by AI models.
- Optical character recognition (OCR): models are approaching and in some cases surpassing human performance on OCR, especially on difficult to read handwritten documents (Humphries et al. 2024).
- Image classification: AI models are increasingly being integrated with vision models, and so can flexibly classify images or even autonomously train and deploy other models to classify images (Achmann-Denkler et al. 2025).
- Video/audio classification and analysis: Like image classification, AI models are gaining additional modalities in video and audio. Their increasing capability and scale facilitates general computation across these new modalities, opening the doors to conducting tasks such as audio transcription, extraction of relevant features (e.g., tone and pitch), and automated coding of audio-visual content in videos (Tarr et al. 2022; Chan et al. 2025).
Notably, some of these analysis techniques are not as widely applied in social sciences (but could be – like video analysis). The advancement of AI capabilities may allow researchers to ask new questions due to foraging of data being easier.
18.3 Farmed Data
Of course, many research questions cannot be answered using foraged data: they require data that does not yet exist. For example, one researcher may find that they must collect their own semi-structured interview data; another may need to develop a survey instrument according to a construct specific to their research question. In these settings it is unlikely that the exact data the researcher needs is available to be foraged, with or without agentic AI. Instead, the researcher must farm their own data, “growing it” with care by developing instruments that interact with the real world – human subjects – and eliciting the data needed for the question at hand. Agentic AI tools present interesting opportunities to support researchers in farming their own data.
Currently, AI tools have already demonstrated their usefulness in survey development. AI models (and not necessarily agentic tools) can be used to support the development of survey items, whether writing them entirely or evaluating human-written questions to ensure they follow best practices. Early work considered this in a fairly narrow scope: AI (specifically, LLMs) was seen as a tool that could help craft items for surveys (Hommel et al. 2022). Indeed, SurveyMonkey released work demonstrating that generative AI tools could be used to craft survey items on its platform (Maiorino et al. 2023). The quality of survey items produced by generative AI can vary, with some works finding they have more clarity than human-generated items, and others finding they lack nuance. But the quality of survey items produced by generative AI has greatly improved with model capability, and we can expect these trends to continue.
These improvements are important and offer the potential to make survey research faster and more efficient. But they are focused on the paradigm of the classical structured survey. In many ways, this paradigm is the product of compromises that were needed to make surveys feasible. In the medium term, we can envision ways in which agentic AI may improve upon this classical paradigm to more fundamentally change the practice of survey research.
For example, it is rare for surveys to ask completely open-ended questions, because it has traditionally been difficult to record and analyze the results of those questions: they would require transcription or audio recording, and then an iterative process of manual coding. The resources and time required make this kind of question infeasible for most surveys. Instead, most surveys use alternatives that are practical but imperfect; for example, questions that have categorical response options. But the set of response options researchers come up with when designing the instrument may not do a good job (Salganik and Levy 2015) of representing the preferences of the survey respondents.
In the long term, agentic AI may be able to help researchers improve data collection by starting to break down the boundary between qualitative methods like in-depth interviews and structured surveys. Qualitative methods have many advantages: they are rich and deep, and they can produce insights that the researcher may want to incorporate in their theory building. Scalable qualitative methods using LLMs interviewing according to a semi-structured protocol, such as Anthropic’s Interviewer10, have already begun to be implemented in dataset construction. While there are critiques of these scalable qualitative approaches, their ability to flexibly probe and follow up on respondents captures the depth of an interview and the scale of a survey. The classical paradigm could not do both: much of the modern survey is the product of compromises that were needed to be able to collect and analyze data in a consistent way across many different interviewers.
This compromise could be pushed to its limits. One could imagine complex, autonomous pipelines that weave together both open-ended and dynamic closed-response engagements. Survey and interview deployments could be iterated on quickly, with humans acting in-the-loop to ensure that the overall structure accurately reflects the constructs being tested. The potential of agentic AI means that farmed data could be obtained in novel, sophisticated ways that are precise, robust, and nuanced.
18.4 Lab-grown Data
Farmed data relies upon the researcher being able to recruit or enroll people to participate in a study. Unfortunately, it is increasingly difficult to get people to participate in many types of social science research; for example, survey response rates (Tourangeau and Plewes 2013) have been steadily falling (Czajka and Beyler 2016) and have reached worryingly low levels (Meyer et al. 2015).
In these early days of agentic AI, one solution has been proposed: silicon samples; that is, survey data generated by AIs rather than from interviews with real people (Argyle et al. 2023). We are highly skeptical (O’Grady 2025) that AI-generated responses will serve as an adequate substitute for interactions with real people. Ultimately, social science data should authentically reflect the population it aims to represent. Even if current social science methodologies, like surveys, have their own issues, those issues are well-specified and can be accounted for in precise ways. AI models, though, carry their own issues, which are least predictable in exactly the novel contexts research often targets. They are beholden to their training data, they tend toward modal or sometimes stereotyped responses, and most importantly, the largest and most powerful models are opaque in their design and training processes, preventing thorough diagnostics of when incorrect conclusions are drawn.
Nonetheless, we do see lab-grown data serving a specific purpose to improve social science processes: reducing the infrastructural friction in established methodologies. Specifically, lab-grown data might be able to help researchers maximize the effectiveness of the rare and valuable opportunities they have to interact with actual people. Three examples include:
- Power Analysis and Study Design: Researchers often use highly simplified simulations in power analyses. Lab-grown data may be able to improve on this by making it easy to model a range of more complex data sets to help ensure that a planned study has sufficient statistical power for its goals.
- Replicability: Lab-grown data may be useful for developing code and other tools that can be part of a pre-analysis plan that can be pre-registered before farmed data is collected and analyzed.
- Sensitive Data: Realistic synthetic data may be useful in facilitating the development of code to be run in walled off settings where data should not be directly accessed.
Finally, lab-grown data will be important for understanding the agentic AIs themselves. AI models are so large and complex that there is currently no framework for really understanding how they work; they are black boxes. To better understand agentic AI, social scientists have started to study them using methods similar to how they study people, by eliciting responses from previously used social science instruments and leveraging existing theories to characterize model behavior. We can expect this line of research to only expand with increasing model capabilities.
18.5 Bearing the fruits of new data
The production of new data via the above processes will shape social science. While the data is important, the heart of social science is its theory. Bearing the fruits of foraged, farmed, and lab-grown data means that new questions can be asked (Breen and Feehan 2024) and answered. Social science has seen this pattern before, where new data made new theory possible. For example, the arrival of large-scale relational data firmly established network thinking as a quantitative science; machine-readable digitized text reshaped the study of culture and political attitudes; the linking and organization of administrative and panel data transformed empirical economics; and the proliferation and deployment of cross-national surveys allowed political science to test theories. In each case, new theory was provoked by new data. Agentic AI – promising to lower the barriers to foraging, farming, and growing data – will allow researchers to test existing theories and to create new ones on a scale and speed that was not previously possible.