Can AI replace surveys? What behavioral data shows, where synthetic respondents fall short and when you need to ask real people yourself. With a checklist.
Yes, but not for every question: behavioral data shows what people do to the extent that their actions are recorded, and AI models predict which answer would be typical for a group. For reasons, attitudes and intentions, you need to ask people themselves. AI can't replace surveys here: in the comparison studies published so far, synthetic respondents reproduced real answers only under narrow conditions and did not do so reliably (Argyle et al. 2023, p. 341; Bisbee et al. 2024, p. 401).
Let's go through it step by step. You'll learn:
- what you only find out by asking,
- what behavioral data and big data do better and where they mislead,
- whether AI can replace surveys, that is, whether synthetic respondents can stand in for real ones, and what the research on "silicon samples" shows,
- where AI really does improve your survey,
- how to decide which source fits your question: existing data, an AI model or a survey.
The question comes in two versions. The older one goes: "We already have the data, so why ask?" For customers, this means data from CRM systems, online stores and web analytics. For a company's own employees, it means HR records. The newer one is only a few years old: "Can't an AI simply simulate the answers?" Both questions are fair. If you have data, you should use it before you bother people with a questionnaire. And if you can ask an AI, you should know what it's actually answering. You learn the most when you combine data, AI and a survey.
What do you only find out by asking?
In my course, I define a survey as "one of the most popular methods of collecting primary data, in which the researcher interacts with respondents to obtain information about people's attitudes, opinions, knowledge and behaviors" (Marx 2026, slide 6). Primary means that the data doesn't exist yet. You generate it for your question. Secondary data, by contrast, was collected at an earlier point in time (Marx 2026, slide 14), usually for a different purpose (more on this in Principles of survey research). This article asks a narrower question: how much of that can you get without a survey?
Of the four things in the definition, one can often be captured in other ways, namely behavior. The other three are harder to get at. Jacob et al. put it this way: surveys try to capture latent phenomena that can't be observed directly, that is, attitudes, values, wishes, hopes, knowledge and behavioral dispositions. All of this, they say, can only be learned indirectly, by asking (2019, p. 5).
Malhotra and Birks show the difference with an everyday example. The biggest disadvantage of observation, they write, is that "the reasons for the observed behaviour may be difficult to determine because little is known about the underlying motives, beliefs, attitudes and preferences". Someone observed buying a brand of cereal may not even like it and may be buying it for someone else in the household (2007, p. 290). The same entry in the checkout data can stand for enthusiasm, for habit or for a favor.
This leaves five kinds of questions where you can hardly avoid a survey:
| What you want to know | Example | Why data alone isn't enough |
|---|---|---|
| What you want to knowReasons and motives | Why do customers cancel? | The cancellation is recorded, but the reason isn't |
| What you want to knowAttitudes and evaluations | How satisfied are employees with their manager? | Satisfaction leaves no unambiguous trace |
| What you want to knowIntentions and expectations | Would you use the new offer? | There's no behavior yet for something that doesn't exist |
| What you want to knowKnowledge and understanding | Do users understand what the plan includes? | A click doesn't show whether someone understood |
| What you want to knowPeople outside your data | Why do prospects buy from a competitor? | Someone who was never a customer isn't in your CRM |
The last row is often overlooked. Your own data describes the people who are already there. It says little about those who never came or who left quietly.
A carefully designed survey can do one more thing: it tells you how widespread an attitude is in a particular group. Whether an attitude is typical, for whom, and whether it changes over time can, according to Jacob et al., only be established with quantitative, representative surveys (2019, p. 5). What "representative" requires is covered in the article Representative sample.
Big data vs surveys: what does behavioral data do better?
Now the other side, and it's a strong one. For questions about recorded behavior, behavioral data is often more accurate than self-reports. Observation permits "measurement of actual behaviour rather than reports of intended or preferred behaviour", with no reporting bias and with little or no influence from interviewers (Malhotra and Birks 2007, p. 290). Some things can only be observed: behavior patterns that respondents aren't aware of or can't communicate (ibid.), and everything respondents can't or won't put into words, for instance because it's unconscious or too long ago (Möhring and Schlütz 2019, p. 9).
In practice, this means: if you want to know how often customers open your app, which page they leave from or how long they wait in the queue, don't ask them. Look at the data. People remember things like that poorly. The limits of memory are limits of every survey (Schnell 2019, p. 7), and for questions about actual behavior, standardized observation is often a sensible alternative (ibid., p. 8).
Often the answer is already there. For many research questions, individual-level data already sits in databases, and where you can draw on such process-generated data, it frequently has advantages over a survey. Otherwise, it's worth checking whether the question can be answered by a secondary analysis of survey data that has already been collected, for example from academic data archives such as the GESIS Data Archive (Schnell 2019, pp. 8-9). In my course, the same idea appears as a rule: don't start a market research project "when decision-relevant information is already available" (Marx 2026, slide 10). A survey whose answer is already in your database costs your respondents time and costs you their trust.
Where data misleads. The strength of behavioral data has three downsides.
First, it shows only what gets recorded, and only for those who get recorded. In their overview of syndicated data sources, Malhotra and Birks note that scanner data reflects actual purchases, is timely and comparatively inexpensive, but may not be representative. In the same table, they describe surveys as the most flexible way of obtaining data, including information on underlying motives (2007, p. 107).
Second, the data was produced for a different purpose. A search log, a ticket system or a checkout database measures what the system needs, not what you want to know. Lazer et al. make the point in general terms: most of the big data sources that have received public attention don't come from instruments designed to produce valid and reliable data (2014, p. 1203).
Third, sheer volume tempts people into overconfidence. The best-known example is Google Flu Trends, a service that was meant to predict flu outbreaks from search queries. In February 2013, it became known that it was predicting more than double the proportion of doctor visits for influenza-like illness that the US Centers for Disease Control and Prevention (CDC) reported, even though it had been built to predict exactly those figures (Lazer et al. 2014, p. 1203). Between August 2011 and September 2013, its estimate was too high in 100 out of 108 weeks (ibid., p. 1204). The authors name one cause "big data hubris": "the often implicit assumption that big data are a substitute for, rather than a supplement to, traditional data collection and analysis" (ibid., p. 1203). Their conclusion isn't a case against big data. Traditional "small data" often contains information that big data doesn't, they argue, and what's needed is an analysis of all data sources, old and new (ibid., p. 1205).
The what and the why. For recorded behavior, behavioral data is often more accurate than self-reports. Why people act that way, what they think of it and what they plan to do next can usually only be inferred from it indirectly. A survey adds the reasons, evaluations and intentions that people state themselves.
Can AI replace surveys? Can synthetic respondents stand in for real ones?
Since large language models arrived, a new shortcut has suggested itself. You describe a person to the model, say "42 years old, teacher, lives in Manchester, leans left", ask it the questions from the questionnaire and repeat this for hundreds of such profiles. The result is a sample with no fieldwork period and no response rate problem. In research, such artificial samples are called "silicon samples" or synthetic respondents. When the model imitates specific individuals, for example a company's own customers or employees, people also speak of "digital twins".
The case for. The term "silicon sample" goes back to a political science study. Argyle et al. fed the language model GPT-3 thousands of sociodemographic profiles of real respondents from large US surveys and showed that the model reproduces the response distributions of many subgroups remarkably well. They call this property "algorithmic fidelity" (2023, p. 337). The authors word their claim carefully, though. This fidelity first has to be verified for the topic area and the groups of interest (ibid., p. 341). They don't expect agreement at the level of individual people, only in the overall distribution (ibid., p. 348), and their studies deal with US politics only.
The case against. Later studies looked more closely, and the picture is sobering.
Bisbee et al. had ChatGPT (version 3.5) adopt various personas and give feeling thermometer ratings for eleven social and political groups, then compared them with the real answers from the American National Election Studies (ANES) of 2016 to 2020. The averages matched well. That holds for the overall sample: in subgroups by party and race, the synthetic values were off by roughly 10 to 20 points on the 100-point scale (2024, p. 406). The synthetic data still wasn't fit for statistical inference. The answers varied less than those of real respondents, regression coefficients often differed significantly, small changes in the wording of the prompt changed the distribution, and the same prompt produced significantly different results within three months (ibid., p. 401). In all, 48% of the coefficients from the ChatGPT data differed significantly from those from the real survey, and in 32% of these cases even the sign flipped (ibid., pp. 402-403). The variation was especially low for questions about ethnic and religious groups (ibid., p. 406).
For Germany, von der Heyde et al. tested whether GPT-3.5 can predict vote choice. They recreated the respondents of the German Longitudinal Election Study 2017 as a synthetic sample. The model didn't predict vote choice accurately and was biased toward the Green and Left parties. It captured the tendencies of "typical" voter groups, such as people with a firm party attachment, but missed more complex influences on vote choice (2026). The freely available preprint gives the scale: 1,905 recreated respondents with 13 characteristics each, and on average only 46% of the predictions matched what the respective person had reported. The Greens, the Left and non-voters were overestimated, while the FDP and the AfD were underestimated (preprint 2024, pp. 13 and 18-19).
Wang et al. describe two limitations that, according to their analysis, lie in the way today's language models are trained. First, the models misportray groups: they "more likely represent what out-group members think of that group, than what in-group members think of themselves". Second, they flatten the diversity within a group, because they're trained to give the most likely answer (2025, p. 2 of the arXiv version). They tested this on four language models and with 3,200 participants from 16 demographic groups in the US (ibid., p. 1). GPT-3.5 and GPT-4 mostly used only three of five possible response options in each set of 100 answers (ibid., p. 6). Remedies such as a higher "temperature", that is, more randomness in the output, reduce the problems but don't remove them (ibid., pp. 2 and 7-8).
Even earlier, Santurkar et al. had tested nine language models with 1,498 questions from Pew Research Center surveys (2023, p. 29972). The models' opinions differed substantially from those of 60 US demographic groups, even when the models were steered toward a specific group (ibid., pp. 29971 and 29977). Among the poorly represented groups were people aged 65 and over and widowed people (ibid., p. 29976).
Two targets. On the left, real respondents with points spread widely around the center. On the right, synthetic respondents with points crowded tightly together. Both have a similar center in the overall group, but the spread differs clearly.
Schematic illustration of the finding in Bisbee et al. 2024, pp. 401 and 406: similar means in the overall group, but less variation than among real respondents. If you compare only the mean, you miss this.
| Study | What was tested | Result |
|---|---|---|
| StudyArgyle et al. 2023 | GPT-3 with profiles of real US respondents, political attitudes and voting behavior | Distributions of many subgroups matched well. Transferability to other topics and groups has to be verified |
| StudyBisbee et al. 2024 | ChatGPT personas, feeling thermometer ratings for eleven groups, comparison with ANES | Means good, variation too small, relationships often wrong, results unstable over time and across prompt wordings |
| Studyvon der Heyde et al. 2026 | GPT-3.5, vote choice in Germany, comparison with GLES 2017 | Bias toward the Greens and the Left, "typical" voter groups captured, more complex influences missed |
| StudySanturkar et al. 2023 | Nine language models, 1,498 questions from US opinion surveys, 60 groups | Substantial misalignment, even after steering toward a specific group |
| StudyWang et al. 2025 | Four language models, including GPT-3.5 and GPT-4, 3,200 participants, 16 groups | Groups portrayed from an outside view and flattened, remedies help only partly |
Why is that? A language model has learned from texts that have already been written. It's good at reproducing what is typically said about a group. General training texts don't let you reliably infer how individual people's views differ, how they judge a product that doesn't exist yet or what your workforce thinks about the new shift schedule this fall. That's my simplified reading. Wang et al. also attribute the flattening to the way the models are trained: what gets rewarded is the most likely answer (2025, p. 2 of the arXiv version). This explains why the average often fits and the variation doesn't: a model that predicts what's typical produces too few atypical answers.
The state of research. The studies in the table work with language models from 2020 to 2023, from GPT-3 to GPT-4. Newer models and methods can reduce individual weaknesses. One example is a preprint by Park et al. The team had 1,052 people in the US interviewed for about two hours each and built an AI agent from each interview on the basis of GPT-4o (2024, pp. 3 and 5). These agents predicted their human counterparts' answers to questions from the General Social Survey 83% as accurately as the people themselves repeated their own answers two weeks later. Agents that knew only age, gender, race and political ideology reached 74% (ibid., pp. 6-7). That's a notable result, and it supports the principle more than it overturns it: the better simulation worked because the people had been interviewed at length beforehand. The authors themselves note that their evaluation measures accuracy for individual people and doesn't show whether the agents also reproduce the relationships between attitudes within a sample correctly (ibid., p. 11).
In practice. A silicon sample consists of answers that a model has generated. It isn't a sample of people who were surveyed but a prediction of what such a sample would say. That's true of better models as well. You only know whether the prediction is right once you've asked people.
Where do AI and synthetic data really help?
None of this means that AI has no place in survey research. It just belongs somewhere other than in the respondents' seat.
Sarstedt et al. reviewed the studies on silicon samples for consumer and marketing research. The results differ considerably from one domain to another. Silicon samples are particularly promising, they find, in the upstream steps of the research process, such as qualitative pretesting and pilot studies (2024, p. 1254). Their example: GPT-4 recognized that the statement "I'm satisfied with the products and services of the company" is double-barreled, asking about two things at once, products and services (ibid., p. 1256). Argyle et al. also see the benefit before the actual data collection: to pilot question wordings, screen measures, select key relationships and draft analysis plans before collecting data from people (2023, p. 349).
This points to four sensible uses:
Drafting and checking the questionnaire. An AI assistant can write a first draft and go through a finished draft looking for double-barreled, leading or unclear questions and unbalanced scales. The decisions are yours. A prompt for this could look like this: "Check this questionnaire for double-barreled, leading and unclear questions and for unbalanced scales. For each issue, identify its location, explain the problem and suggest a revision. Don't change anything yourself." How a good questionnaire is put together is covered in the article Questionnaire design, and how to test it with real people in Questionnaire pretest.
Hypotheses before fieldwork. What might the reasons for cancellations be, and which response options are missing from a list? A model supplies ideas you might not have come up with on your own. Whether they hold is for the survey to test.
Test data, not result data. Made-up, that is, synthetic answers are suitable for checking selected skip logic paths, the export and the analysis before launch. Test responses say nothing about the views of real respondents. Delete such test responses before launch so that they don't end up in the results. In QUESTIONSTAR you can generate such test data directly in the survey and delete it again afterwards. If an AI assistant is connected through the MCP interface, it can create and delete test responses as well.
Analyzing open-ended answers. An AI assistant can pre-sort hundreds of free-text answers and suggest themes, frequencies and relevant verbatim quotes for you to check. The answers still come from people. The AI only helps with the reading. Check samples of the assignments yourself. Pass free text only to an assistant whose data processing you've checked, because free text often contains names and other personal details.
And synthetic answers as results? Wang et al. themselves name cases in which the benefits can outweigh the harms: when synthetic answers "supplement rather than replace human participants (e.g., pilot studies), when study costs are otherwise prohibitive, or when directly involving participants risks exposing them to distressing content" (2025, pp. 2 and 10 of the arXiv version). For these cases, Sarstedt et al. offer two clear rules. Always compare silicon samples with a human benchmark sample, even if that eats up part of the time and cost advantage (2024, p. 1263). And discard synthetic data that shows no variation, only a single "right answer" (ibid., p. 1264). Researchers who use language models without sufficient reflection, or for tasks they weren't built for, will likely be misinformed, they conclude (ibid., p. 1266).
Five questions to ask providers of synthetic respondents
1. Which human sample was it compared with, on my topic and in my target group? (Argyle et al. 2023, p. 341; Sarstedt et al. 2024, p. 1263)
2. Does the variation match too, not just the mean? (Sarstedt et al. 2024, p. 1264)
3. Do relationships and subgroups match as well? (Bisbee et al. 2024, pp. 401 and 406)
4. Which model, which version and which date, and how stable are the results when the run is repeated? (Bisbee et al. 2024, p. 401)
5. How is the model supposed to know anything about my new product or my workforce?
Survey, data or AI: which should you use when?
Schnell places a series of checks before every survey. Is there a precise research question? Are there alternatives to a survey? Do psychological or social obstacles stand in the way of data collection? Are time and resources sufficient? If one of these checks fails, his flowchart says: stop (2019, pp. 5-6). The following flowchart applies this idea to today's situation with behavioral data and AI.
Flowchart with three check questions. If the answer already exists, analyze the existing data. If it's about recorded behavior, analyze the behavioral data. If it's about reasons, attitudes, intentions or knowledge, run a survey, with AI helping to draft and check the questionnaire.
Based on the checks before every survey in Schnell 2019, pp. 5-6, extended to include behavioral data and AI. Many projects end up with a combination.
In practice, most projects don't end up with a single source but with a combination. This is how the work is divided:
| Your question | Behavioral data | AI model | Survey |
|---|---|---|---|
| Your questionHow many customers canceled last quarter? | answers it | not needed | not needed |
| Your questionWhy did they cancel? | gives clues (timing, plan) | supplies hypotheses | captures the reasons respondents give |
| Your questionHow satisfied are employees with their managers? | hardly | can guess, not measure | captures the satisfaction respondents report |
| Your questionWould a new offer be used? | doesn't exist yet | supplies ideas for options | measures the intention, not the later behavior |
| Your questionIs the question wording understandable? | no | helps with checking | a pretest with real people reveals comprehension problems |
| Your questionHow many pages does a user visit on average? | answers it more accurately | no | inaccurate, relies on memory |
The most important sentence in the flowchart is at the very top: what do you want to know, and what for? Without a clear research question or decision, one more survey won't help either. Schnell calls it largely pointless to collect data that merely seems interesting (2019, p. 7).
Strongest together: the what from data, the why from the survey
In research methods, combining several methods on the same topic is called triangulation (Marx 2026, slide 16). Malhotra and Birks note that, other than the use of scanner data, few projects rely solely on observation, and they call the use of observation in conjunction with surveys "quite fruitful" (2007, p. 290). Schnell stresses that for valid data on behavior, you have to observe the behavior over time rather than infer it from a survey. At the same time, he expects surveys to remain important mainly because survey data can be linked with data from other sources, such as transaction, mobility and sensor data or administrative data (2019, p. 377). He goes further here than a survey provider might like: in such projects, he writes, survey data are only a small part of the data collection (ibid.).
Here's what that looks like in three typical cases:
Online store. Web analytics shows that many visitors drop off on the shipping page. A short survey that appears at exactly this point asks for the reason: shipping costs, delivery time, a missing payment method or just a price comparison? With the answers, you narrow down the causes and know which change to test first.
Workforce. HR records show that an unusually high number of people are quitting in one department. Whether it's the management, the pay, the workload or a competitor down the road isn't in the records. You ask about that in an employee survey, and in a way that makes honest answers safe.
Thesis. Before you run your own survey, check whether data on your question already exists: earlier studies, public statistics, data sets in archives. Your own survey then contributes the piece that's missing from the existing data. How to plan your own survey as part of a thesis is covered in the article How to write your thesis.
Four steps: review the data, form hypotheses, run the survey, interpret the results together.
- Review the dataWhat do behavioral data and existing studies show? Where does something stand out?
- Form hypothesesGather possible reasons with help from AI, then use them to develop response options
- Run the surveyDraft the questionnaire, check it with AI, test it with people, invite real respondents
- Interpret the results togetherThe what from the data, the why from the survey
Triangulation in practice: several sources on the same topic (Marx 2026, slide 16).
A note on linking: if you combine survey answers with customer data or HR data about a specific person, you're processing personal data and have to tell respondents so openly. Such a survey is then no longer anonymous. What that means legally is covered in the article Data protection in online surveys. Often it's enough to compare at group level, for example by department or plan, without linking individuals.
What can't surveys do?
Anyone who defends surveys should also name their weaknesses. The literature is clear on this.
Respondents don't know everything about themselves. Malhotra and Birks point out that respondents are often not consciously aware of their motives and therefore can't give accurate answers, and that fixed response options can limit how validly beliefs and feelings are measured (2007, p. 266). In addition, "the survey imposes the language and logic of the researcher on the questionnaire respondents" (ibid.).
Memory has limits. If you want to know actual behavior, don't ask about details that lie decades in the past (Schnell 2019, p. 7).
Asking changes what it measures. Möhring and Schlütz call the survey highly reactive: every measurement influences the social reality it's supposed to capture. One example is social desirability (2019, p. 7). When respondents fear employment-related or criminal consequences, it's even in their rational interest not to answer truthfully (Schnell 2019, p. 8). How to deal with this is covered in the article Social desirability bias.
An answer costs nothing. Jacob et al. remind us that giving a survey answer carries little psychological cost: someone who gives a substantive answer without having an opinion or any knowledge suffers no disadvantage and appears informed. They don't conclude from this that surveys should be abandoned, though, but that the measurement instrument deserves particular care (2019, p. 5).
Intention isn't behavior yet. Someone who says in a survey that they would use the new offer hasn't bought it yet. Observation measures actual behavior rather than reported intentions (Malhotra and Birks 2007, p. 290), and for valid data on behavior, according to Schnell, you have to observe the behavior itself (2019, p. 377). So read answers to intention questions as an expression of interest, and check them as soon as behavioral data is available.
Who answers determines the result. A survey describes the people who answered. Whether those are the people you want to say something about depends on sampling and response. More on this in the articles Sampling methods, Survey response rate and Representative sample.
None of these weaknesses is an argument for silicon samples. A language model doesn't know your specific respondents, and it brings biases of its own. They're arguments for a well-designed survey and for interpreting survey results alongside other data.
AI and QUESTIONSTAR: an assistant for the survey, not a replacement for respondents
A word on our own product, with a clear line. QUESTIONSTAR generates no synthetic respondents and no AI answers as result data. The AI helps you with the survey. The answers come from people.
Through the MCP interface. With the MCP interface, you connect AI assistants such as Claude or ChatGPT directly to your QUESTIONSTAR surveys. The assistant can then, on your behalf, create a survey and write questions, go through the draft with QUESTIONSTAR's checklist and list the places it considers problematic. After data collection, it reads the responses, groups open-ended answers into themes, builds crosstabs and calculates correlations, t-tests and regressions. Write calls are real changes in your account, and good AI clients ask for your permission first.
What the assistant sees and what it doesn't. When the assistant reads or exports responses, they contain no names, email addresses or other contact data of the respondents. Questions you mark as personal data aren't passed on either when the setting is on. Unmarked free text can still contain personal information. And check results that an assistant calculates the way you would check the work of a new colleague.
The survey itself. The real work is still surveying real people. You invite your respondents personally. That works in the free plan too. You remind only those who haven't responded yet, and you analyze the results in QUESTIONSTAR or export the raw data as XLSX, CSV or SPSS.
Checklist: do I need a survey?
- Write down the research question you want to answer or the decision you need to inform.
- Check whether the answer already exists: in your own data, earlier studies or public data archives.
- Is it about recorded behavior (purchases, clicks, usage, times)? Then analyze the data first.
- Is it about reasons, attitudes, intentions, knowledge or people outside your data? Then run a survey.
- Use AI for drafting, checking, hypotheses and test data, not as a replacement for respondents.
- If you do create a silicon sample, compare it with real answers and check the variation, not just the mean.
- Interpret survey results alongside other data, and tell respondents openly if you link their answers with other data.
Frequently asked questions: surveys, big data and AI
Are surveys still relevant?
Yes, for questions about reasons, attitudes, intentions and knowledge. These things can't be observed directly and can only be learned indirectly, by asking (Jacob et al. 2019, p. 5). For actual, recorded behavior, existing data is often the better source (Schnell 2019, pp. 8-9).
Can AI replace surveys and survey respondents?
So far, AI has not proved to be a reliable replacement. With ChatGPT 3.5, Bisbee et al. found means similar to those in the real survey, but too little variation, diverging relationships and unstable results (2024, p. 401). A preprint by Park et al. shows better predictions for individual people with GPT-4o after these people had been interviewed at length, but it doesn't test the relationships between attitudes within a sample (2024, pp. 6-7 and 11). AI is well suited to drafting, checking and pretesting a questionnaire (Sarstedt et al. 2024, p. 1254).
What are synthetic respondents or silicon samples?
A silicon sample is an artificial sample: a language model is given profiles of people, such as age, education and political leaning, and generates matching answers to the questions (Argyle et al. 2023, p. 337). The result is a prediction of what people would say, not a measurement.
What is the difference between big data and a survey?
Big data refers to large data sets that mostly arise as a by-product, whereas a survey is a method of collecting data on purpose. The behavioral data discussed here often shows recorded behavior more accurately than self-reports, but reveals motives, attitudes and preferences only indirectly (Malhotra and Birks 2007, p. 290). A survey collects answers to your questions and can also reach people who don't appear in your own data.
When is observation or data analysis better than a survey?
When it's about behavior that people remember poorly, aren't aware of or don't want to talk about (Möhring and Schlütz 2019, p. 9). And when the information already sits in databases: such process-generated data frequently has advantages (Schnell 2019, p. 9).
How do you combine behavioral data and a survey?
The data shows where something stands out, and the survey adds the reasons people report. An example: web analytics shows drop-offs on the shipping page, and a short survey at that point asks for the reason. Combining several methods on the same topic is called triangulation (Marx 2026, slide 16).
Sources
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C. and Wingate, D. (2023). Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis, 31(3), 337-351. DOI 10.1017/pan.2023.2.
Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B. and Larson, J. M. (2024). Synthetic Replacements for Human Survey Data? The Perils of Large Language Models. Political Analysis, 32(4), 401-416. DOI 10.1017/pan.2024.5.
Jacob, R., Heinz, A. and Décieux, J. P. (2019). Umfrage. Einführung in die Methoden der Umfrageforschung [Surveys: an introduction to the methods of survey research] (4th ed.). Berlin/Boston: De Gruyter Oldenbourg. In German.
Lazer, D., Kennedy, R., King, G. and Vespignani, A. (2014). The Parable of Google Flu: Traps in Big Data Analysis. Science, 343(6176), 1203-1205. DOI 10.1126/science.1248506.
Malhotra, N. K. and Birks, D. F. (2007). Marketing Research. An Applied Approach (3rd European edition). Harlow: Pearson Education.
Marx, P. (2026). Principles of Survey Research. Online course, slides 6, 10, 14 and 16 (chapter 1). QUESTIONSTAR. Course, chapter 1.
Möhring, W. and Schlütz, D. (2019). Die Befragung in der Medien- und Kommunikationswissenschaft. Eine praxisorientierte Einführung [Surveys in media and communication studies: a practical introduction] (3rd ed.). Wiesbaden: Springer VS. In German.
Park, J. S., Zou, C. Q., Kamphorst, J., Egan, N., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., Liang, P., Willer, R. and Bernstein, M. S. (2024). LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals. Preprint, arXiv:2411.10109. Page numbers follow version 3 of June 28, 2026. First version 2024 under the title "Generative Agent Simulations of 1,000 People".
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P. and Hashimoto, T. (2023). Whose Opinions Do Language Models Reflect? Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 29971-30004. proceedings.mlr.press/v202/santurkar23a.html.
Sarstedt, M., Adler, S. J., Rau, L. and Schmitt, B. (2024). Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines. Psychology & Marketing, 41(6), 1254-1270. DOI 10.1002/mar.21982.
Schnell, R. (2019). Survey-Interviews. Methoden standardisierter Befragungen [Survey interviews: methods of standardized surveys] (2nd ed.). Wiesbaden: Springer VS. In German. DOI 10.1007/978-3-531-19901-6.
von der Heyde, L., Haensch, A.-C. and Wenz, A. (2026). Vox Populi, Vox AI? Using Large Language Models to Estimate German Vote Choice. Social Science Computer Review, 44(3), 549-571 (first published online 2025). DOI 10.1177/08944393251337014. Preprint 2024 under the title "Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion", arXiv:2407.08563.
Wang, A., Morgenstern, J. and Dickerson, J. P. (2025). Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence, 7(3), 400-411. DOI 10.1038/s42256-025-00986-z. Page numbers follow the freely available version arXiv:2402.01908, version 3 of February 3, 2025.
I wish you good data, a smart AI at your side and respondents who tell you what isn't in any database.
Related articles:
- Principles of survey research
- Representative sample: what it is and how to check yours
- Social desirability bias: how to ask sensitive questions
Dr. Paul Marx
Author: Dr. Paul Marx
Dr. Paul Marx is the founder and Managing Director of QUESTIONSTAR and was a professor of marketing at the University of Siegen.