Survey methods: online, phone, paper or mixed-mode?

Online, phone, mail or face-to-face: what each survey method reaches and costs, when online alone is enough, and how to combine modes in a mixed-mode survey.

In short: the survey method is the way your questions reach people and their answers come back, and it helps decide whom you reach, how many take part, what the survey costs and how people answer. An online survey alone is enough if almost everyone in your target group can be reached online and you can invite them directly. A second mode is worth it if an identifiable part of the group is missing online and probably answers differently on your topic. Often a second way of inviting is enough, for example a letter with a QR code that leads to the same online questionnaire. Only when people can't answer on a screen do you need a second response mode such as paper. In that case, ask the same questions in the same form everywhere and record which mode each answer came through.

We'll go through the decision in five questions:

1. Which survey methods are there, and what does the choice change?
2. When is an online survey alone enough?
3. When is a second mode worth it, and which one?
4. Why can't you simply compare the answers from two modes?
5. How do you combine modes in a mixed-mode survey?

At the end you'll find what QUESTIONSTAR offers for this, a checklist and answers to frequently asked questions.

The question comes up in many forms. An HR manager is planning an employee survey, but the warehouse staff have no work email address. A town wants to hear from its eighty-year-olds, too, about the new town center. A student would like to add twenty telephone interviews to his online survey and wonders whether he may count the two together. All three cases are about the same thing: the way you ask isn't neutral. What the three can do is covered further down.

The decision in three questions

If you're short on time, three questions will get you to a result. The sections that follow give the reasons for each of them.

A simple multiplication answers the second question. The third decides between a second invitation channel and a second response mode.

Which survey methods are there?

If you want to know something from someone, you can call, send a letter, ring the doorbell or send a link. In survey research, these ways are called survey methods, data collection methods or survey modes. In this article, "mode" and "survey method" mean the same thing.

Survey method (mode): the way in which the questions of a standardized survey reach the respondents and in which their answers are recorded.

The five common modes at a glance:

The assessments follow Jacob et al. (2019, pp. 107-122). The figures for maximum survey length and fieldwork duration come from their comparison table and are explicitly only rough guidelines (ibid., pp. 128-129). There's no winner in this table: there is no single best survey mode (Möhring and Schlütz 2019, p. 150). A standard English textbook says the same: "no survey method is superior in all situations" (Malhotra and Birks 2007, p. 283). Every mode is a trade-off.

Behind the table are two questions that textbooks use to sort the modes. First: is the survey interviewer-administered, with someone asking the questions, or self-administered, with respondents completing the questionnaire themselves? Second: do they hear the questions, or do they see them in front of them? Accordingly, Jacob et al. distinguish oral interviews, face-to-face or by telephone, from written surveys: by mail, online or as a group survey, in which a group that's present fills in the questionnaire in one room (2019, pp. 106 and 115). The section on answers from two modes comes back to both differences.

This article deals with standardized surveys, in which everyone gets the same questions in the same form. Open conversations with an interview guide and group discussions are a different tool for a different kind of question. More on that in the article on qualitative methods.

What does the choice of method change?

The survey method isn't a shipping route where the same parcel arrives in the end, whether the postal service brings it or a courier. Your choice of mode changes whom you reach, because different modes have different gaps in their coverage of the target group (Schnell 2019, p. 308; for the term, see sampling methods). It changes how many take part: many people find it harder to turn down an interviewer at the front door than to put a questionnaire in the recycling, hang up the phone or ignore a link (Jacob et al. 2019, p. 113; more under survey response rate). Costs and duration change, although the advantage of the cheap mode quickly becomes meaningless if too few people, or the wrong people, answer through it (Schnell 2019, pp. 307-308). And it changes how people answer. That's the hardest of the four topics, and it gets a section of its own below. Keep this principle in mind when deciding: the topic must determine the choice of mode, not the other way around (Möhring and Schlütz 2019, p. 149).

When is an online survey alone enough?

An online survey alone is enough if three conditions are met.

The target group is online. For most target groups, that's true today. To take Germany as an example: according to the Federal Statistical Office, just over 3% of people aged 16 to 74 had never used the internet in 2025. Anyone who's rarely online already counts as online in these statistics. Among those aged 65 to 74, the figure was 10%, and people aged 75 and over aren't surveyed at all in this data collection (Statistisches Bundesamt 2026). So the number says nothing about the very group that matters in a residents' survey that includes the very old.

You can invite people directly. "Online" doesn't yet mean "reached". Online surveys are particularly well suited when they're based on an address list (Jacob et al. 2019, p. 126). If you distribute an open link instead, you reach the people who happen to see it (see representative sample and finding survey respondents).

The questions work without someone to explain them. On a screen, nobody is there to explain an unclear question or to probe (Jacob et al. 2019, p. 122). People are reluctant to answer open-ended questions at length in writing, somewhat less so online than on paper (Möhring and Schlütz 2019, p. 152). If your topic needs long explanations or depends on detailed answers in people's own words, that's a reason for a conversation and not for a second questionnaire mode.

The pretest settles the third condition. With the first two, something is often left over: part of the target group can't be reached online. How large can that part be?

How many missing people are too many?

There's no fixed limit for this, but there is a simple calculation. How far an online-only survey deviates from the result for everyone because of missing coverage depends on two quantities: how large the part is that it doesn't reach, and how differently this part answers the question at hand. Multiplying the two gives the deviation:

So it becomes a problem only when both come together, a noticeable share and a noticeable difference, and that depends on the individual question. The formula looks only at the gap in coverage. If certain people among those reached answer more often than others, a further deviation can be added.

Let's take a housing cooperative that will accompany us through the rest of the article. It wants to survey its 2,000 members. For 1,400 members it has an email address, for 600 only a postal address. The numbers are invented.

For your own planning, a rough estimate of both quantities is enough:

Deviation of the online figure from the result for everyone, in percentage points.

You can often estimate the share roughly from your contact lists: it's the customers without an email address, the employees who don't work at a screen, the members who can be reached only by mail. You don't know the difference, but you can judge it. Ask yourself: does my topic have to do with what distinguishes the missing people from the others, that is, with age, technology, workplace or life situation? If yes, expect a large difference. If no, a small one.

You don't need an exact number for this. As a test, calculate with two values from the table, a small one and a large one: with 30% missing, a difference of 10 points gives a deviation of 3 points, and a difference of 50 points gives one of 15. If neither value would change your decision, the online survey is enough. If the large one would change it, and your topic has to do with what distinguishes the missing people, a second mode is worth it.

One thing doesn't help against this deviation: more responses. Whether 400 or 1,200 members answer online, the 600 without an email address aren't among them in either case. The same pattern applies to those who were invited and don't respond. More on that under survey response rate.

When is a second mode worth it, and which one?

A second mode is worth it if the calculation above gives a deviation that could change your decision. But before you print a second questionnaire, one distinction can save you a lot of work.

Check first: is a second way of inviting enough?

The way you invite someone and the way this person answers are two different things.

That's why Schnell recommends separating the contact phase from the data collection phase. He uses the term mixed-mode survey only when different groups of respondents answer in different modes during data collection itself (2019, p. 310).

In practice, this is good news. Often people have internet access but never receive an invitation because you don't have their email address. In that case, a second invitation channel is enough, and the questionnaire stays the same:

• a letter with a QR code to everyone for whom you have only a postal address, if you like with a short link next to it for anyone who can't scan a code,
• a notice or flyer with a QR code where people pass by anyway: on the bulletin board, at the checkout, in the waiting room,
• a tablet on site on which the survey runs: in the break room, at reception, at a trade fair booth.

An example from Jacob et al. shows that a letter can bring people to the screen: for one study, 45,000 Germans living abroad were invited by mail to an online survey, with an invitation letter and two reminders. More than 11,000 took part online, about one in four of the people contacted (2019, p. 132).

The gain: one questionnaire, one data set, no data entry. Mixing contact modes has no consequences for measurement as long as everyone answers in the same mode (de Leeuw 2005, pp. 239 and 247). The limit: the letter with a QR code, too, reaches only people who are able and willing to answer online. And a tablet in the break room isn't the same as your own smartphone on the sofa, because colleagues can watch there. So even when the questionnaire is the same, record the invitation channel for each response.

A real second response mode: paper or telephone

You need a second response mode for people who can't or don't want to answer on a screen.

Paper is the closest option. For people who are rarely online, it's the more familiar medium (Weinreich and von Lindern 2008, p. 149), and respondents complete the questionnaire themselves, just as they do online. The price is printing, postage and entering every questionnaire by hand.

The telephone fits when reading itself is the hurdle, or when you can't get people to answer in any other way. But it forces you into a short questionnaire with simple questions (Jacob et al. 2019, p. 111), and it brings in a person the respondent is answering to.

Offered as a choice, assigned or used as a follow-up?

Two response modes can be distributed in three ways, and the literature judges each of them differently.

Schnell describes assignment by reachability (2019, p. 311), and Jacob et al. call it a respondent-specific design (2019, p. 130). The follow-up is the classic sequential design (de Leeuw 2005, p. 240). The reasons behind the two right-hand columns follow.

The evidence generally weighs against letting respondents choose their response mode. There's no firm evidence that offering a choice reduces nonresponse (de Leeuw 2005, p. 240). In nineteen experimental comparisons, mail surveys that additionally offered a web option actually had lower response rates than mail surveys without it (Medway and Fulton 2012, according to the abstract of the article; likewise Schnell 2019, p. 309, fn. 7).

A survey of physicians in the German state of Rhineland-Palatinate shows what can happen with a free choice. A link to the online version was enclosed with the paper questionnaire. Of 1,648 participants, 1,261 answered on paper, which is 77%, and younger men were overrepresented among those who took part online (Jacob et al. 2019, p. 131). If you offer a choice like this, you have to expect to pay for a lot of paper and to get two groups that differ from the outset. An often-cited counterexample from market research has no comparison group and is therefore not sufficient as evidence (Malhotra and Birks 2007, p. 265).

The follow-up is better supported, but it has its price. According to de Leeuw, sequential designs are common and effective for reducing nonresponse: first an inexpensive mode for everyone, then a more expensive one for those who haven't responded (2005, pp. 240-241). Schnell, too, calls renewed contact in a different mode common practice (2019, p. 311). But de Leeuw warns: as soon as answers are collected in several modes, whether one after the other or at the same time, data quality can suffer, because the modes can measure differently (de Leeuw 2005, pp. 241 and 247). And the follow-up costs money, as the calculation below shows.

From these findings, a rule in two parts can be derived for a small project. It's an inference, not the result of a single study:

Assign first, then decide about the follow-up. (a) Assign the response mode by reachability and don't offer it as a choice: online for everyone you can invite online, the second mode only for the rest. (b) Whether you then follow up with those who haven't responded is a separate decision about cost and comparability.

With part (a), the expensive mode stays small, and for every answer you know why it came in this mode. With part (b), the respondents have helped decide who answers in which mode, and the comparison between the modes becomes even more uncertain.

Worked example: paper questionnaire or letter with a QR code?

The second invitation channel costs less work. The second response mode also brings in the people who can't answer online at all. What that means in euros and hours depends on your prices. The calculation itself is always the same, and it's complete only when the hours of data entry are in it.

The third question from the beginning decides which option fits: what are the people you don't reach online missing? If they're missing only the invitation, the letter is enough. If they lack internet access or experience using a screen, only paper or a conversation helps. If you also want to follow up on paper with the 980 people invited online who didn't respond, in this example you'd expect a further 980 × 1.50 euros = 1,470 euros for the mailing at 1.50 euros per paper mailing, plus return postage and data entry.

Why can't you simply compare the answers from two modes?

Let's assume the cooperative has decided on the paper questionnaire. The question "How satisfied are you with the property management?" is answered on a scale from 1 (very dissatisfied) to 5 (very satisfied). Online, the mean is 3.6. On paper, it's 4.1. At the board meeting, someone says right away: "People who answer by mail are more satisfied."

The sentence sounds like a simple reading of the numbers. But it contains an assumption that nobody has checked. A picture makes it visible.

Imagine two bathroom scales. On the first, you weigh group A. On the second, group B. For group B, the display shows 2 kilograms more on average. Are the people in group B heavier? That's the obvious reading. But there's a second one: the second scale shows more. And a third: a bit of both. From the two averages, you can't tell which reading is right. That's because the measurements differ in two things at once: different people are standing on the scale, and it's a different scale. Either of the two causes alone could produce the same 2 kilograms on the display.

You may know the scale that shows 2 kilos too much at every weighing from the article on reliability and validity. There, one person stood on one scale, and the question was whether you can trust the display. Here, one more question is added: what if two groups stand on two different scales?

How would you find out what's behind it? There are two ways. You can put the same people on both scales, one after the other. If the second scale shows more then as well, it's the scale. Or you let a random draw decide who steps on which scale. Then both groups weigh about the same on average, and a difference that goes beyond the chance variation of the draw comes from the scale. What doesn't help: letting people choose for themselves, or distributing them by a characteristic that's related to weight, for example all the tall people on the second scale. Then people and scale are tied together again, and you're none the wiser.

A survey in two modes is similar. The scale is the questionnaire together with the mode in which it arrives: the same questionnaire in a different mode is a different scale. And almost always, different people are standing on the two scales, because nobody was assigned by a random draw. Even after a random draw, different people can end up taking part in the two modes, because nobody can be forced to answer. In our example, the paper questionnaire went to exactly those members for whom there's no email address, and those are mostly the older ones. So the difference of 0.5 points can have two causes:

• Different people: the selection effect. That's the part of a difference that arises because different people answer in this mode, here for example more older people who really are more satisfied. Jacob et al. speak of effects of method selection (Décieux 2012; Décieux and Hoffmann 2014, cited in Jacob et al. 2019, p. 132).
• A different mode: the mode effect. That's the part that arises solely because the question was asked in a different mode: the same person would have answered differently on paper than on a screen. It's also called a method effect (ibid.).

For Schnell, this is the central problem of all surveys with several modes: without true random assignment, differences between the groups can be explained by the way of surveying or by real differences between the respondents (2019, p. 313; likewise Möhring and Schlütz 2019, p. 149, and de Leeuw 2005, p. 249).

What does this mean for your survey? Three things.

First: separate observation and explanation. In the answers you have, the satisfaction score is higher on paper than online. You may report that. But it doesn't follow that the people behind the paper answers really are more satisfied: the difference can also have arisen from the response mode. Statements such as "People who answer by mail are more satisfied" or "Respondents interviewed by telephone are more critical" turn an observation into a characteristic of the respondents, and the data don't support that.

Second: the two effects aren't equally unwelcome. That different people answer in the second mode was the purpose: two modes can make up for each other's gaps in coverage (Jacob et al. 2019, p. 132). Only the influence of the mode on the answer is unwelcome. You want to keep that as small as possible.

Third: you can hardly check this afterward. This is where the picture stops working, in two respects. You can check a scale by putting a weight on it whose mass you know. For satisfaction, there's no such test weight. And unlike a miscalibrated scale, the mode doesn't shift every answer by the same amount: more for sensitive questions, hardly at all for questions about facts. The two clean procedures from the picture are rarely available either: surveying the same people twice usually fails because of the effort, and the random draw, as a randomized design, mainly serves methods research (Jacob et al. 2019, p. 130). For an ordinary survey, this means: whatever you can do for comparability, you have to do beforehand. You build the two scales to be as similar as possible. How to do that is shown in the next section.

How large the influence of the mode is depends on what the modes differ in. Mode effects are described as mostly small, but detectable across the whole survey (Décieux 2012; Décieux and Hoffmann 2014, cited in Jacob et al. 2019, p. 132). Schnell also expects rather small differences and finds the strongest ones when comparing modes with and without an interviewer (2019, p. 308). An English review of mode comparisons comes to the same result: "clear but usually small differences between methods", with the dividing line between interviewer modes and the others (de Leeuw 2005, p. 245). What matters, the argument goes, isn't whether there are differences, but whether they're meaningful for the purpose at hand (Jäckle et al. 2010, cited in Schnell 2019, p. 313).

Online and paper are comparatively similar to each other: respondents read the questions and fill in the questionnaire themselves. Differences remain in skip logic, overview, presentation and the situation when answering. Between online and telephone, hearing and the influence of the interviewer are added.

Worked example: a 0.5-point difference between online and paper

The two causes can't be separated for certain after the fact, least of all with the means of an ordinary survey. They can be narrowed down: by comparing like with like, that is, the answers from both modes within groups that share important characteristics such as age or education (de Leeuw 2005, p. 249). In the picture, that would mean looking only at people of similar build on both scales.

The comparison within like groups narrows things down. It proves nothing. It balances only the characteristics you know and have collected. The same goes for weighting: it can balance differences in the characteristics taken into account, but it doesn't remove the influence of the mode on the answer. More on weighting in the article on the representative sample.

In short: if two modes deliver different results, then different people answered in them, or the mode changed the answers, or both. As with two scales used for different groups, the means alone don't tell you which explanation applies.

Mixed-mode surveys: how do you combine modes without comparing apples and oranges?

Apples and oranges can go into the same basket if you know how many of each kind are in it, and if all of them were weighed on the same scale. For a mixed-mode survey, that is, a survey with several response modes, that comes down to six steps.

1. Name the reason. Write down in one sentence who's missing online and how many people that is. "More responses" isn't a reason. If you can't write that sentence, stay with one mode. From a methodological point of view, a single mode is the better choice anyway (Möhring and Schlütz 2019, p. 152).

2. Choose modes that are similar to each other. The differences are largest between modes with and without an interviewer (Schnell 2019, p. 308). Online, paper and a tablet on site belong to the same family: the person reads and fills in the questionnaire alone. Add the telephone only if there's no other way.

3. Design the questions for the most restrictive mode. This is the mode with the fewest options for presenting questions and guiding respondents. On paper, there's no automatic skip logic. On the telephone, there are no images and no long lists. Dillman calls the principle unimode design: word and present questions so that they're the same stimulus in every mode (Dillman 2000, cited in Schnell 2019, p. 313). De Leeuw also refers to Dillman's unimode principles as helpful for minimizing differences between modes (2005, p. 248). From his list, these rules are the most useful for small projects (Schnell 2019, p. 314):

• use the same answer options in all modes and include them in the question text,
• don't change the structure of a question between the modes,
• prefer few answer categories to many,
• write explanations that an interviewer would otherwise give into the question itself.

This is what the same question looks like in three modes:

If you follow Jacob et al. strictly, even these five categories would be one too many on the telephone: they advise avoiding questions with more than four answer alternatives there (2019, p. 111). The course slides say the same about the number of answer categories: "fewer in telephone interviews, where scales are harder to keep track of" (Marx 2026, slide 58). So if you add the telephone, decide beforehand: four categories in all modes, or five categories and reading especially slowly on the telephone. A scale with eleven points online and one with four categories on the telephone would be two different scales. How well such rules actually reduce the influence of the mode has hardly been studied so far (Schnell 2019, p. 314). They're the obvious precaution, not a guarantee.

4. Define who gets which mode and when. The rule from above applies here: Assign first, then decide about the follow-up. In the example, that means online for everyone with an email address and paper for the rest. And make sure nobody counts twice. Personal links or numbered paper questionnaires help you detect duplicate participation. Decide in advance how you'll match online and paper answers from the same person and which answer stays in the data set if someone takes part twice. Tell respondents openly what the link or number is for (see anonymous or confidential).

5. Record the mode. Two pieces of information belong to every answer: the invitation channel and the response mode. Flag paper questionnaires that someone enters by hand, for example with an additional question "Mode of entry" that the person entering the data fills in. Otherwise they look like online answers in the data set. The device is a small mode of its own, too: on a smartphone, a question looks different than on a large screen. When paper questionnaires are entered, make sure every answer is recorded with the same code as online, so "very satisfied" is 5 in both modes. For questions that someone didn't get because of skip logic, use the same code in all modes as well (see questionnaire design). And have a second person check about every tenth questionnaire that was entered.

6. Analyze together, check separately, report openly. Report the overall result of all modes. For the most important questions, also show the breakdown by mode, provided the groups are large enough, and don't interpret differences between the modes as a characteristic of the people. There's no fixed limit for "large enough". In the example above, the comparison was already uncertain with 40 responses in the smallest group. In the methods section or in the report, it can read like this:

Before the start, test each mode separately: the paper questionnaire with a pen in your hand, the telephone version read out loud (see pretest).

The three cases from the beginning

Two things don't fit into the table. The town's age limit is its assumption about where access is missing, and it belongs in the report. And if the student's conversations are meant to be open and go into depth, they aren't a second response mode but a qualitative study of their own, which he analyzes separately (see qualitative methods).

Survey modes with QUESTIONSTAR

QUESTIONSTAR is a tool for online surveys. For the modes in this article, that means:

Several invitation channels, one questionnaire. Your respondents reach a survey through a public link, through personal links or through a QR code that you print on a letter, a notice or a flyer. The email invitation inserts the personal links automatically, or you generate them in the "Personal Links" section of the "Distribute Survey" view (recognizing returning respondents). You define who may take part in the access settings: everyone with the link, only with a password or only invited people. Personal invitations also work in the free plan and also in anonymous surveys. In that case, reminders go only to those who haven't responded yet.

A letter with a QR code and a personal password. With the "Invited only" setting, the public link, that is, the link behind the QR code, also asks for a password, and this password is a separate one for each invited person. It's in the "PersonalizedID" field of the contact list. Anyone who isn't in the contact list can't take part. So a letter can carry the QR code for everyone and the personal password for the individual person.

Personal links without an email address. The contact list doesn't need email addresses or any other personal data. You create one row for each person, and QUESTIONSTAR generates a personal link for each row. You then send the links yourself: by email from your own system, or you convert them into QR codes and print those in your letters. That also works for very large lists. The addresses stay with you and don't reach QUESTIONSTAR.

See which channel the answers come through. The completions widget shows how many times people took part through the public link, the email invitation, social media and the QR code, and which devices they used: desktop, tablet or phone. In addition, the export of the responses shows for every single answer which channel it came through.

A tablet on site. For the break room, reception, the waiting room or a trade fair booth, there's a guide to kiosk mode on the iPad. After completion, or after a period without input, the survey starts again from the beginning, so the next person finds an empty questionnaire. More on that in the article on the employee survey.

The paper version. You export the finished questionnaire as a Word or PDF file. That way, the paper questionnaire comes from the same source as the online version. In the printout, check the places with skip logic: on paper, you need an instruction such as "continue with question 12".

Answers from paper and from the telephone. Whatever is produced on paper or in a telephone conversation has to be entered by someone. The help article on the access settings describes the mechanics for this, using the example of an interviewer who surveys several people with the questionnaire on a tablet: if multiple completions aren't prevented, one person can complete the questionnaire repeatedly, and each completion creates a new record. In the same way, a staff member enters the paper questionnaires one after the other. Decide before the start who does this and how long it takes. And flag these answers: entered through the normal link, paper answers would look like online answers in the data set, and two modes can be checked and compared only if every answer carries its mode. There are two ways to do this. Either you include a separate question in the questionnaire, such as "Mode of entry", which the person entering the data fills in. Or you append a parameter to the link used for data entry, and it's saved with every answer.

Duplicate participation. QUESTIONSTAR recognizes returning people through the personal link or, with the public link, through a cookie in the same browser on the same device (recognizing returning respondents). Preventing someone from being counted twice after also returning a paper questionnaire requires your own rule from step 4.

Checklist: choosing a survey method and combining modes

1. Write down the target group: who can be reached and invited online, and who can't?
2. Check the topic and the questionnaire: length, images, skip logic, open-ended questions. Does that fit the planned mode?
3. Do the calculation for the missing part: share times estimated difference gives the deviation due to missing coverage.
4. Small deviation: stay online and name the gap in the report.
5. Large deviation: first check a second invitation channel (letter with a QR code, notice, tablet on site).
6. If that isn't enough: choose a second response mode from the same family, paper before telephone, and include the hours of data entry in the calculation.
7. Assign first, then decide about the follow-up. Assign the response mode by reachability and don't offer it as a choice, and rule out duplicate participation.
8. Keep questions and answer options the same in all modes, design them for the most restrictive mode and test each mode separately.
9. Record the invitation channel and the response mode for every answer, and flag paper questionnaires that were entered by hand.
10. Report the overall result, break it down by mode, and don't interpret differences between modes as a characteristic of the people.

Frequently asked questions about survey methods

What are the advantages and disadvantages of an online survey?

An online survey is fast and inexpensive because there's no postage, printing, interviewer or data entry, and the questionnaire automatically skips the questions that don't apply to someone (Jacob et al. 2019, pp. 120-121). Its weakness is coverage: it reaches only people who are online and receive an invitation. It's well suited when it's based on an address list, such as a customer database or an email directory (ibid., p. 126).

When does a telephone survey make sense?

A telephone survey makes sense when the target group is hard to reach in writing or when follow-up questions are needed, because only modes with an interviewer allow meaningful follow-up questions (Schnell 2019, p. 307). It requires short questions with few answer options, because respondents hear the questions and options rather than seeing them (Jacob et al. 2019, p. 111). As a rough guide, the interview should take no longer than 20 minutes (ibid., p. 129). The willingness to take part in telephone interviews has fallen considerably (ibid., pp. 109-110).

What is a mixed-mode survey?

A mixed-mode survey combines several survey methods in order to make up for the disadvantages of individual modes (Jacob et al. 2019, p. 130). In the narrower sense, it exists only when different groups of respondents answer in different modes, for example online and on paper. An invitation by letter to an online survey changes only the contact mode (Schnell 2019, p. 310).

What is a mode effect?

A mode effect is a difference in the answers that arises solely because the question was asked in a different mode, for example on the telephone instead of on a screen. It has to be distinguished from the selection effect: in different modes, different people usually answer, too (Jacob et al. 2019, p. 132). Without random assignment, the two causes can't be separated for certain (Schnell 2019, p. 313).

Can I analyze online and paper answers together?

Yes, if the same questions with the same answer options were asked in both modes and the mode is recorded for every answer. Report the overall result and, for important questions, the breakdown by mode. Differences between the modes can be due to the composition of the groups or to the mode, and they shouldn't be interpreted as a characteristic of the respondents (Schnell 2019, p. 313).

Should respondents be able to choose the mode themselves?

Generally not. In nineteen experimental comparisons, mail surveys that additionally offered a web option had significantly lower response rates than mail surveys without it (Medway and Fulton 2012, according to the abstract of the article; likewise Schnell 2019, p. 309, fn. 7). An often-cited counterexample from market research has no comparison group (Malhotra and Birks 2007, p. 265). From this follows the rule of this article: Assign first, then decide about the follow-up. (a) Assign the response mode by reachability and don't offer it as a choice: online for everyone you can invite online, the second mode only for the rest. (b) Whether you then follow up with those who haven't responded is a separate decision about cost and comparability.

Sources

Bowling, A. (2005). Mode of Questionnaire Administration Can Have Serious Effects on Data Quality. Journal of Public Health, 27(3), 281-291. Cited in Schnell (2019), p. 309.

Décieux, J. P. (2012). Modeeffekte bei Onlineumfragen. Ein multivariater Methodenvergleich unter Zuhilfenahme eines Propensity Score Matchings [Mode effects in online surveys: a multivariate comparison of methods using propensity score matching]. München. In German. Cited in Jacob et al. (2019), p. 132.

Décieux, J. P. and Hoffmann, M. (2014). Antwortdifferenzen im Junk und Crime Survey: Ein Methodenvergleich mit goffmanscher Interpretation [Differences in answers in the Junk and Crime Survey: a comparison of methods with a Goffmanian interpretation]. In M. Löw (ed.), Vielfalt und Zusammenhalt. Verhandlungen des 36. Kongresses der Deutschen Gesellschaft für Soziologie in Bochum und Dortmund 2012. Frankfurt a. M. In German. Cited in Jacob et al. (2019), p. 132.

de Leeuw, E. D. (2005). To Mix or Not to Mix Data Collection Modes in Surveys. Journal of Official Statistics, 21(2), 233-255. scb.se, accessed November 12, 2026.

DHL Group (2026). Deutsche Post AG passt Briefporto und Produkte zum 1. Januar 2027 an [Deutsche Post AG adjusts letter postage and products as of January 1, 2027]. Press release of October 20, 2026. In German. group.dhl.com, accessed November 12, 2026.

Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method (2nd ed.). New York: Wiley. Cited in Schnell (2019), pp. 313-314, and in de Leeuw (2005), p. 248.

Holbrook, A. (2008). Response Order Effects. In P. Lavrakas (ed.), Encyclopedia of Survey Research Methods, vol. 2 (pp. 754-755). Thousand Oaks: Sage. Cited in Schnell (2019), p. 309, fn. 9.

Jäckle, A., Roberts, C. and Lynn, P. (2010). Assessing the Effect of Data Collection Mode on Measurement. International Statistical Review, 78(1), 3-20. Cited in Schnell (2019), p. 313.

Jacob, R., Heinz, A. and Décieux, J. P. (2019). Umfrage. Einführung in die Methoden der Umfrageforschung [Surveys: an introduction to the methods of survey research] (4th ed.). Berlin/Boston: De Gruyter Oldenbourg. In German.

Malhotra, N. K. and Birks, D. F. (2007). Marketing Research. An Applied Approach (3rd European edition). Harlow: Pearson Education.

Marx, P. (2026). Principles of Survey Research. Online course, chapter 2, slide 58. QUESTIONSTAR. Slide 58.

Medway, R. L. and Fulton, J. (2012). When More Gets You Less: A Meta-analysis of the Effect of Concurrent Web Options on Mail Survey Response Rates. Public Opinion Quarterly, 76(4), 733-746. DOI 10.1093/poq/nfs047. Only the abstract of the article was read, not the full text.

Möhring, W. and Schlütz, D. (2019). Die Befragung in der Medien- und Kommunikationswissenschaft. Eine praxisorientierte Einführung [The survey in media and communication studies: a practice-oriented introduction] (3rd ed.). Wiesbaden: Springer VS. In German.

Schnell, R. (2019). Survey-Interviews. Methoden standardisierter Befragungen [Survey interviews: methods of standardized surveys] (2nd ed.). Wiesbaden: Springer VS. In German. DOI 10.1007/978-3-531-19901-6.

Statistisches Bundesamt (2026). Gut 3 % der Bevölkerung im Alter von 16 bis 74 Jahren in Deutschland sind offline [Just over 3% of the population aged 16 to 74 in Germany are offline]. Zahl der Woche no. 09 of February 24, 2026. In German. destatis.de, accessed November 12, 2026.

Weinreich, U. and von Lindern, E. (2008). Praxisbuch Kundenbefragungen [Customer surveys: a practical handbook]. München: mi-Fachverlag, FinanzBuch Verlag. In German.

I hope your chosen mode reaches everyone you want to hear from, and that the responses can be analyzed together.

Related articles:

• Sampling methods: random, quota and cluster sampling
• Survey response rate: calculate, assess and increase it
• Social desirability bias: how to ask sensitive questions

Dr. Paul Marx

Author: Dr. Paul Marx
Dr. Paul Marx is the founder and Managing Director of QUESTIONSTAR and was a professor of marketing at the University of Siegen.