Questionnaire design: structure, question types and checklist

Questionnaire design step by step: question order, survey question types, filter questions, survey length, required questions and layout. With a checklist.

Good questionnaire design starts before the first question: decide what information you need, and cut every question whose answer you won't use. Then follow a proven order: a short introduction, an easy opening question, topic blocks from general to specific, demographics at the end, and a thank-you. Choose each question type by what you want to measure, use filter questions sparingly, force a response only where it's essential, and test everything before you launch.

Why is it worth the effort? A textbook sums it up in one thought: mistakes in the analysis can almost always be repaired, mistakes in data collection almost never (Jacob et al. 2019, p. V). Once the questionnaire is in the field, you get the answers it produces. Not the ones you would have needed.

This article is about the questionnaire as a whole: question order, question types, filters, length, required questions, languages and layout. How to word a single question clearly and neutrally is covered in my article How to write good survey questions.

What a questionnaire has to do

A questionnaire has three jobs (Malhotra and Birks 2007, pp. 371-372; Marx 2026, slide 80). It translates your information needs into questions that people are able and willing to answer. It motivates them to take part and to stay until the end. And it keeps response error small.

Are there fixed rules for this? According to Malhotra and Birks, no scientific principles guarantee an optimal questionnaire, so questionnaire design is "a skill acquired through experience" (Malhotra and Birks 2007, p. 372). Jacob et al. counter that cognitive research has made many design questions empirically testable (Jacob et al. 2019, p. 133). I see both. For much of this article there's evidence, for some of it only good reasons and experience. Where the literature disagrees, I'll say so.

First the table, then the question

One tool I recommend to everyone is the dummy table: a blank analysis table that you set up before you write the first question (Malhotra and Birks 2007, p. 374). Say an online store wants to know whether regular customers are more satisfied than new customers. The blank table has two rows, "regular customer" and "new customer", and one column, "share satisfied". You can already see that you need two questions, one on satisfaction and one on the customer group. And you notice what's missing, for example a clear definition of when someone counts as a regular customer.

The table also helps you decide what to cut. If you have no use for an answer, delete the question (Malhotra and Birks 2007, p. 377). Jacob et al. suggest asking a basic question: why do I want to know this? (Jacob et al. 2019, p. 275). In my course, I ask about every piece of information whether I really need to know it and whether I know what I'll do with it (Marx 2026, slide 116). The first draft is almost always too long, and it usually needs substantial cuts before testing (Schnell 2019, p. 105).

Questionnaire structure step by step

Broadly, a questionnaire consists of an introduction, opening questions, blocks of questions, sociodemographic questions and a closing, and any deviation needs a reason (Jacob et al. 2019, p. 200). A practitioner's book compares this to building a house: first the shell, then the details (Weinreich and von Lindern 2008, p. 50).

Let's look at an example everyone knows: a conversation with a stranger. You don't start by asking about their income. You begin with something easy, then get to the topic, and personal matters come only once you know each other a little. That's exactly how Jacob et al. justify putting demographics at the end (Jacob et al. 2019, p. 207), and Schumann describes the survey as a kind of conversation (Schumann 2019, p. 75). If you're unsure, ask yourself: would I do it this way in a conversation?

The introduction: short and honest

The introduction states the topic, who is asking, how the person was selected, how long it takes, that taking part is voluntary and that the answers are analyzed confidentially (Jacob et al. 2019, p. 201; Marx 2026, slide 114). Online, the very first screen should motivate people to take part, but stay brief (Schnell 2019, p. 294).

Two things easily go wrong. Long explanations about anonymization tend to raise concerns, and describing the goals in too much detail can steer the answers (Jacob et al. 2019, p. 201). So when my course says "Emphasize anonymity" and "Explain the reasons and goals", I mean one clear sentence on anonymity, no explanation of the procedure, and the purpose rather than the hypothesis. "We'd like to improve our customer service" is good. "We suspect that the waiting time on the phone is too long" is too much.

The opening question: easy and relevant to everyone

Opening questions should be interesting, simple and non-threatening, and questions that ask for opinions work well (Malhotra and Birks 2007, p. 388).

It's important that the opening question applies to everyone. Anyone who has to answer "no" or "doesn't apply" right away may easily conclude that the survey has nothing to do with them (Jacob et al. 2019, p. 149; Schnell 2019, p. 108). Questions you might also use to start an everyday conversation work well, for example: "Where have you spent most of your life?" (adapted from Jacob et al. 2019, p. 149).

Can the first question be open-ended? Möhring and Schlütz consider it possible (Möhring and Schlütz 2019, p. 112). For online surveys, others advise against it, because experience shows that many open questions at the start increase the break-off rate (Thielsch et al. 2012, p. 227; Schnell 2019, p. 294). For online surveys, I therefore recommend a closed, easy opening question.

One exception: if you first have to check whether someone belongs to the target group, these qualifying questions are the opening questions, even if they include age (Malhotra and Birks 2007, p. 388).

Topic blocks: from general to specific

Finish one topic before you start the next, and use a short sentence as a transition (Malhotra and Birks 2007, p. 389), because constant jumping around is confusing (Jacob et al. 2019, p. 204). A transition can be as simple as "Now we come to a completely different topic ..." or "We only have a few more questions about ..." (adapted from Möhring and Schlütz 2019, p. 117).

Within a block, the funnel helps: the general question first, then the specific ones, so that they don't influence the general one (Malhotra and Birks 2007, pp. 388-389). So first ask "Which aspects play an important role for you when choosing a department store?" and only then how important the convenience of the location is (Marx 2026, slide 111). The other way around, you would already have brought up the location, and many people would mention it simply because it's on their mind.

When you ask about behavior and attitudes on the same topic, you usually start with behavior. It's easier to answer, and otherwise people tend to adjust the behavior they report to the attitude they've just stated (Brace 2018, p. 47, cited in Schnell 2019, p. 109).

After a demanding block of recall questions, knowledge questions or sensitive questions, give respondents a break with a block of easier questions (Jacob et al. 2019, p. 204; Schumann 2019, p. 75). The order of the blocks matters too: questions about complaints at the start of a customer questionnaire can color the satisfaction questions that follow (Weinreich and von Lindern 2008, pp. 50-51). There's more on this in the article Order effects in surveys.

Sensitive questions: not at the start

Where do sensitive questions go? Malhotra and Birks as well as Schnell advise placing them as late as possible, with income as the last of the personal questions (Malhotra and Birks 2007, p. 388; Schnell 2019, p. 109; Marx 2026, slide 110). By then, trust has developed, and if someone drops out, you already have their other answers (Malhotra and Birks 2007, p. 380; Schnell 2019, p. 109).

Möhring and Schlütz see it differently. They put sensitive topics roughly in the middle, because people are less likely to drop out there, and follow them with easy buffer questions (Möhring and Schlütz 2019, p. 115).

Here's how I handle it: sensitive survey topics come only once trust has developed, followed by an easy block. Personal details such as income go at the very end. The literature doesn't give you one rule for every questionnaire here.

Demographics at the end, then a thank-you

Demographic questions at the start are uninteresting, leave open what the survey is about and can come across as intrusive (Jacob et al. 2019, p. 207). At the end, people are more likely to answer them (Möhring and Schlütz 2019, p. 115). Name or address, if you ask for them at all, come last (Malhotra and Birks 2007, p. 388).

Besides questions about the target group, justified exceptions are quotas, where the interview should end quickly if someone doesn't fit (Weinreich and von Lindern 2008, p. 86). And if you want to check whether those who drop out differ from the others, you need the demographics at the start (Thielsch et al. 2012, p. 227).

Announce the block and say specifically why you need the information. That makes the request understandable (Malhotra and Birks 2007, p. 380). Jacob et al. and Schumann suggest a stock phrase along the lines of "information for statistical purposes", which Schnell considers pointless (Jacob et al. 2019, p. 207; Schumann 2019, p. 75; Schnell 2019, p. 109). I agree with Schnell: "Finally, a few questions about you. We need them to check whether younger and older customers differ in how satisfied they are."

An easy summary question works well at the end (Jacob et al. 2019, p. 150), for example: "All in all, how satisfied are you with our service?" with the points "very dissatisfied / somewhat dissatisfied / neither satisfied nor dissatisfied / somewhat satisfied / very satisfied". After that comes an open question for anything that was missing (Malhotra and Birks 2007, p. 381; Weinreich and von Lindern 2008, p. 52), and the thank-you. Jacob et al. are brief and firm on this: you must always do it (Jacob et al. 2019, p. 207).

Survey question types at a glance

There's no single agreed classification of question types (Malhotra and Birks 2007, pp. 381-382; Möhring and Schlütz 2019, pp. 80-81). In quantitative surveys, closed questions usually take priority (Jacob et al. 2019, p. 105), but they require you to know the possible answers (Schumann 2019, p. 59). The names in the first column of the overview match the question types in QUESTIONSTAR. The advice applies to any tool.

Open or closed?

Open questions influence the answer less and sometimes bring unexpected insights. But they're time-consuming to analyze and favor articulate respondents (Malhotra and Birks 2007, p. 381; Jacob et al. 2019, p. 105). Choose the highest level of measurement you can: age as a number can later be grouped into age bands, but not the other way around (Möhring and Schlütz 2019, p. 80).

In lists, respondents tend to tick the first or the last answer, "particularly the first". Malhotra and Birks therefore advise varying the order of the answers (Malhotra and Birks 2007, p. 382). In QUESTIONSTAR, you do this with random order of the answer options. That's suitable for unordered lists, such as brands or channels used. The points of a rating scale, on the other hand, stay in their logical order (Pew Research Center n.d.).

Rating or ranking?

In a rating, respondents assess each item on the same scale. In a ranking, they put the items in order. A textbook shows the difference with ice cream flavors: third place can mean "inedible" or "very good", and the ranking doesn't tell you which (Jacob et al. 2019, pp. 184-185). So for absolute importance, use a rating, and for the question of what matters more than something else, use a ranking (Schumann 2019, pp. 72-73).

Matrix: useful or harmful?

On matrix questions, the sources openly contradict each other. Malhotra and Birks consider them useful when related questions share the same response options (Malhotra and Birks 2007, p. 394). Schnell advises against them, because they put people off, make it easy to slip into the wrong row and invite mechanical answering. In web surveys, he wants to avoid them entirely (Schnell 2019, pp. 114 and 294). Möhring and Schlütz weigh it up: a whole matrix on one screen reinforces response patterns, while splitting it up makes the questionnaire longer and increases missing answers (Möhring and Schlütz 2019, p. 180).

My advice lies in between. A short matrix with a few statements on the same topic is fine, page-long batteries are not. On a smartphone, the matrix looks different anyway, more on that in the section Layout.

Scales: just an overview

For the rating scales in this article, I recommend the same number of positive and negative points and a neutral midpoint. Malhotra and Birks explain the distinction between balanced and unbalanced scales and between an even and an odd number of points (Malhotra and Birks 2007, p. 352). A proven example is "very good / good / partly good, partly bad / bad / very bad" (Schumann 2019, p. 71, translated). You'll find more in A list of common Likert scales, in the article Likert scales: everything you ever wanted to know, but never dared to ask and in Likert scales: five aspects to consider when creating them.

"Don't know" or "can't judge" isn't the midpoint of the scale. Offer it only for factual and knowledge questions, not for attitudes and opinions (Marx 2026, slide 61). This also covers things that not everyone has experienced or observed. Only someone who has already submitted a vacation request can judge "How quickly does HR process vacation requests?". Anyone can answer "How satisfied are you with your work?". Where you offer "don't know", set it apart from the scale points with a line or a gap (Schumann 2019, pp. 69-70; Schnell 2019, p. 112). Otherwise it looks like a midpoint, because respondents also read meaning from the layout, for example "the middle means typical" (Tourangeau et al. 2004, cited in Möhring and Schlütz 2019, p. 179).

You'll find suitable starting points under Templates.

Filter questions: the right path for everyone

Let's look at a simple example. Anyone who is retired and no longer working doesn't get questions about satisfaction with their current job. Those questions are skipped for them (adapted from Schumann 2019, p. 54). Or: before you ask how satisfied someone is with their car insurance, ask whether they have car insurance at all (Marx 2026, slide 88).

A filter consists of the filter question, the skip instruction, the filtered questions and the skip target. The filter question must allow exactly one answer for everyone, and you ask about complicated conditions one at a time (Schnell 2019, p. 116).

Online, filtering is a big advantage: irrelevant questions don't even appear. On paper, by contrast, filters can provoke break-offs (Jacob et al. 2019, p. 121). Still, Schnell advises using filters sparingly online too, because anyone who changes an earlier answer can run into problems, and testing every path is expensive (Schnell 2019, p. 295).

Draw the flow

Draw a flowchart of the paths for every case and derive the filters from it (Malhotra and Birks 2007, pp. 389-390). One example is the flowchart of a questionnaire for a department store in my course (Marx 2026, slide 112). Even a sketch on paper with boxes and arrows shows you dead ends that you won't see in the questionnaire editor.

Two rules help with the arrangement. Keep the destination question close to the question that triggers the branch, and arrange the filters so that respondents can't anticipate that "yes" brings more questions (Malhotra and Birks 2007, p. 389; Marx 2026, slide 111). Respondents who learn that tend to answer "no" later on (Schnell 2019, pp. 116-117).

Not asked isn't the same as left unanswered

The most important point comes from Schnell: record explicitly whether someone met the filter condition. Otherwise you can't tell later whether a question was correctly skipped or simply not answered, and that's one of the main causes of unexplained missing values in surveys (Schnell 2019, p. 116).

Worked example: the right base

The numbers in this example are made up, but the principle is backed by the literature. A company surveys its customers. 400 people reach the end of the questionnaire, and they may leave individual questions unanswered along the way.

The filter question is: "Have you contacted our customer service in the last 12 months?" 160 answer "yes" and 240 answer "no". Together that's 160 + 240 = 400.

Only the 160 who said "yes" get the follow-up question "How satisfied were you with your most recent contact?" with the points "very dissatisfied / somewhat dissatisfied / neither satisfied nor dissatisfied / somewhat satisfied / very satisfied". 150 answer, and 10 leave the question unanswered. The 150 answers are distributed like this:

How many are satisfied? Here, satisfied means "somewhat satisfied" or "very satisfied", that's 60 + 30 = 90 people. Based on everyone who answered the question, that's 90 / 150 = 60%.

Now the common mistake. If you can't see in the data set who never got the question, it's easy to divide by all participants: 90 / 400 = 22.5%. This calculation includes the 240 people who were never asked the question in the denominator for satisfaction with a service contact. But they simply had no contact with customer service.

Keeping the two categories separate also lets you calculate the missing-answer rate among those who were asked: 10 / 160 = 6.25% ≈ 6.3%. For this, "not asked" (240 people) and "asked but not answered" (10 people) must be coded differently in the data set (Schnell 2019, p. 116).

A required question would have closed the 10 gaps, but wouldn't necessarily have produced better data. There's more on this in the section Required questions.

By the way, in QUESTIONSTAR you set up filters with skip logic. It takes effect when respondents move to the next page, so the filter question goes on an earlier page than the filtered questions. For a follow-up question on the same page, use display logic.

Electronic questionnaires often contain filter errors (Schnell 2019, p. 118). So test every path, as described in the section Testing.

How long should a survey be?

I can't give you a fixed number, and neither can the literature. How long a survey can be depends on the topic, the target group and the survey mode (Schnell 2019, p. 105). Schnell puts it plainly: the only people who can decide how long a survey may take are the respondents (Schnell 2019, p. 108).

In their comparison table of survey modes, Jacob et al. give the following maximum durations:

Source: Jacob et al. 2019, p. 129, Table 4.1. Treat the values as rough guidance, not as a fixed limit.

For online surveys, other sources are stricter. Brace suspects that fatigue can set in after just 15 minutes and lead to break-off (Brace 2018, p. 17, cited in Schnell 2019, p. 105). For academic online surveys, Möhring and Schlütz recommend ideally staying under 15 to 20 minutes (Möhring and Schlütz 2019, p. 147). For smartphone surveys, Jacob et al. cite 10 to 15 minutes as the current recommendation (Mergener and Décieux 2018, cited in Jacob et al. 2019, p. 128). According to a Canadian industry guideline, questionnaires that take more than 30 minutes are generally considered overly long, except for personal interviews at home (Malhotra and Birks 2007, p. 396).

Length is more than the number of questions

Twenty yes/no questions aren't the same as twenty open questions. The burden depends on the topic, the effort per question, the number of open questions, complex scales and the survey mode (Malhotra and Birks 2007, p. 396). A ranking or distributing 100 points takes more effort than simply ticking a box (Malhotra and Birks 2007, p. 374), and cognitively demanding questions make break-offs more likely (Fuchs and Lamnek 1990 and Peytchev 2009, cited in Möhring and Schlütz 2019, pp. 111-112). Rules of thumb that calculate a duration from the number of questions only give rough guidance. It's better to measure the duration in your test.

You can often reduce the effort without losing information, for example with a list of department store departments to tick instead of free recall (Marx 2026, slide 104). Squeezing questions together so the questionnaire looks shorter, on the other hand, leads to errors and lowers people's willingness to take part (Malhotra and Birks 2007, p. 391).

Measuring the duration correctly

Measure the duration with the median, not the mean, because anyone who makes a coffee halfway through produces outliers at the top end. Also look at which questions are followed by especially many break-offs (Jacob et al. 2019, p. 122).

If you run surveys at a university, keep this in mind: school classes or students in a lecture hall will fill in even long questionnaires, and that can easily distort your sense of what you can ask of other people (Schnell 2019, p. 105, n. 2).

Required questions and forced response: why a full table can mislead

Many tools let you make questions required: respondents can only move on once they've answered. The motive is understandable. You don't want gaps in your data.

Even so, most of the literature advises against it. Jacob et al. generally recommend doing without: participation must be voluntary for each individual question too, coercion triggers defiance and rejection (reactance), and you trade missing answers to individual questions for the more serious loss of participants altogether, in their terms item nonresponse for unit nonresponse (Jacob et al. 2019, p. 123). In their experiments, respondents lied more often when forced to answer (Décieux et al. 2015, cited in Jacob et al. 2019, p. 123) and dropped out earlier and more often, even with an incentive (Décieux et al. 2015 and Mergener et al. 2015, cited in Jacob et al. 2019, p. 123). Schnell is just as clear: under no circumstances should you try to force respondents to answer a question (Schnell 2019, p. 295).

The middle way: remind, don't force

Möhring and Schlütz are somewhat softer. For key variables, a required answer may be advisable. Besides requiring an answer or allowing respondents to move on without a reminder, they describe a third option: a reminder that respondents can dismiss without answering. In their view, research ethics speak strongly in favor of this middle option (Möhring and Schlütz 2019, p. 180). Motivating messages when a question is skipped are also meant to help keep respondents until the end (Al Baghal and Lynn 2015, cited in Möhring and Schlütz 2019, p. 111).

Here's how I sum it up: force an answer at most where a question is essential for the analysis, for example a filter question. And say at the start that nobody has to answer a question they find uncomfortable (Malhotra and Birks 2007, p. 396).

By the way, in QUESTIONSTAR you set this in the Common Question Settings. A question can be set to "Not required", "Require: Strict Mode" or "Require: Soft Mode". With Soft Mode, a friendly request to answer appears the first time someone skips the question, and the second time they can skip it. That corresponds to the reminder option described above. For question types with several items, such as Matrix, Form or Sliders, you can also mark individual items as optional, for example an "Other" field.

The literature cited here hasn't tested Soft Mode, so whether it produces better data than Strict Mode is an open question. I still consider it the better default. That's my conclusion from the findings, not a test of my own: people who aren't forced have less reason to answer dishonestly or to drop out.

Three things called "forced"

In English, "forced" names three different things. A required question is called "forced answering" (Jacob et al. 2019, p. 123) or "forced response" (Mergener et al. 2015, cited in Jacob et al. 2019, p. 123). A scale without a "no opinion" option is a "forced rating scale" (Malhotra and Birks 2007, p. 353). And ranking is a "forced-choice format" because it forces differences (Möhring and Schlütz 2019, p. 82). My course uses "forced vs. non-forced response" in the second sense (Marx 2026, slide 61). These are three different decisions. That's why in this article I say "required question" for the first, speak of a scale with or without "don't know" for the second, and simply say "ranking" for the third.

Multilingual questionnaires

Let's look at an example. When the Eurobarometer was translated, it turned out that in France "mobile phone" is rendered as "téléphone mobile", while in French-speaking Belgium the same words would mean a "wireless handset" (Malhotra and Birks 2007, p. 760). Same language, different question. Or take a German statement from employee surveys, "In meinem Team ziehen alle an einem Strang", which means that everyone in the team pulls together. Translated word for word, it becomes "In my team everybody pulls at a rope". Translated back, the original German sentence comes out again, and nobody notices that the English version makes no sense (Linke 2018, p. 43).

A few rules follow from this. Even in the source questionnaire, avoid idioms, colloquial language and technical terms that are hard to translate, and ask two simple questions rather than one complex one (Linke 2018, p. 43; Malhotra and Birks 2007, p. 395). The goal isn't the same wording but the same meaning, especially for evaluative terms and for demographics such as educational qualifications (Schnell 2019, p. 178; Möhring and Schlütz 2019, p. 194; Malhotra and Birks 2007, pp. 394-395).

Translating a questionnaire and then translating it back can reveal ambiguities (Möhring and Schlütz 2019, p. 195), but this process alone isn't enough, as the rope example shows. Schnell describes the standard as a translation team with preparation, translation, pretest, review and documentation (Schnell 2019, pp. 178-179). It isn't enough that someone speaks the language fluently. As a practitioner quoted by Malhotra and Birks puts it, "current idiomatic knowledge" is required (Malhotra and Birks 2007, p. 760). Test the translation with monolingual native speakers, and compare it with bilingual respondents who answer the original (Malhotra and Birks 2007, p. 395).

Resist the temptation to build a separate scale for each country. Even though response styles differ, Linke recommends questions that are as culture-independent as possible (Linke 2018, p. 44). And if you survey in only one language, you won't reach whole parts of the population, which, depending on the topic, can bias the results (Schnell 2019, p. 178).

In QUESTIONSTAR, you can set up one questionnaire in several languages, see Translations and language settings. The translation itself and checking it remain your job.

Layout on screen and on smartphones

Let's start with a cautionary example. A questionnaire had two short blocks of statements one after the other. In the first block, the box for "agree" was on the left, in the second, the box for "disagree". Each block was fine on its own. But the analysis showed that a large share of respondents had apparently missed the switch and ticked the opposite of what they meant (Jacob et al. 2019, pp. 204-205). This happens easily when you put together questions from different questionnaires (Jacob et al. 2019, p. 204).

The lesson is a consistent layout. All questions look the same, instructions are formatted differently from questions and sit right next to the question, no question is separated from its answers, and answers are arranged vertically and, online, all visible at the same time (Schnell 2019, pp. 111-112 and 294; Malhotra and Birks 2007, p. 391).

Scrolling or pages?

For short questionnaires, scrolling is usually easier (Schnell 2019, p. 295). Möhring and Schlütz, by contrast, recommend at most one longer question or two to three shorter ones per screen page (Möhring and Schlütz 2019, p. 177). One question per page helps respondents concentrate in complex questionnaires, but takes longer and loses the context (Möhring and Schlütz 2019, p. 180). My advice: let short questionnaires scroll, and split longer ones into pages by topic.

Show respondents where they are, most simply with something like "Question 12 of 30" (Schnell 2019, p. 295). According to the cited research, a graphical progress indicator has been shown to reduce survey break-offs only in very short surveys (Dillman et al. 2014, cited in Schnell 2019, p. 295, and in Möhring and Schlütz 2019, p. 181). With filters it's inaccurate, because it's based on all possible questions, which can be irritating (Möhring and Schlütz 2019, p. 181). Practitioner books still recommend it for orientation (Weinreich and von Lindern 2008, p. 51; Linke 2018, p. 66). For the same reason, a fixed total such as "of 30" isn't right for everyone when you use filters, and skipped numbers can easily look like a programming error. Möhring and Schlütz see reasons to do without question numbers online in that case (Dillman et al. 2014, cited in Möhring and Schlütz 2019, p. 178).

In QUESTIONSTAR, the progress indicator is usually switched on and can be shown as a bar, as a percentage or as a counter such as 6/41. Where you use skip logic or display logic, I advise against the counter, because the total then isn't right for everyone.

On smartphones

Some respondents answer on a smartphone or tablet. In five access panels, the share was already between 7% and 29% in 2016 (Schnell 2019, p. 302, n. 49).

The good news: overall, data quality hardly differs by device. But sliders or date pickers are harder to use on a smartphone, and filling in the questionnaire takes a little longer (Antoun et al. 2017 and Tourangeau et al. 2017, cited in Möhring and Schlütz 2019, pp. 147-148). Use such formats sparingly and explain them briefly (Möhring and Schlütz 2019, pp. 180-181).

For small screens, Linke advises splitting long blocks of statements, placing the question and the scale one below the other, making answer areas generous enough for a thumb, keeping texts plain and short, and building nothing that requires zooming (Linke 2018, pp. 48-49).

The most important thing is to look at real devices. A questionnaire that looks perfect on your own computer can look terrible in another browser or on a smartphone, so the technical side definitely belongs in the pretest (Möhring and Schlütz 2019, p. 147). In QUESTIONSTAR, the preview shows you the questionnaire in the desktop, tablet and phone views. It doesn't replace a real smartphone, but it helps you find many problems beforehand.

Final step: testing

No questionnaire should go into the field without adequate testing, and even the best one can be improved by a pilot test (Malhotra and Birks 2007, p. 391). Test every aspect on a small sample: content, wording, order, form and layout, question difficulty and instructions (Marx 2026, slide 115). Test respondents should come from the population you intend to survey (Jacob et al. 2019, pp. 214-215).

Test proven questions too, because, as the switched columns show, questions that are good on their own can fail in combination (Jacob et al. 2019, pp. 204-205). That includes every filter path and different devices and browsers. And put the test data next to your dummy tables. If part of a table stays empty, check whether a necessary question is missing (Malhotra and Birks 2007, p. 392).

By the way, in the QUESTIONSTAR preview, the preview mode settings let you switch on "Ignore skip logic" and "Ignore validation". That way you can reach every question, including those behind filters, without following each route separately. To check the filters, switch skip logic back on and go through each path once.

Questionnaire design checklist

You can go through these points before you launch:

• For every question, it's clear which analysis it feeds into. Everything else has been cut.
• The introduction states the topic, who is asking, the duration, that taking part is voluntary and that answers are analyzed confidentially, briefly and without hypotheses.
• The opening question is easy, interesting and relevant to everyone.
• Topics are grouped in blocks, from general to specific, with short transitions and an easy block after demanding questions.
• Sensitive topics aren't at the start. Demographics are at the end, with a sentence on why you need the information.
• Every question type fits what you want to measure, and scales are symmetric with a neutral midpoint.
• "Don't know" appears only for questions that not everyone can judge, and it's set apart from the scale.
• Every filter path has been drawn and tested, and "not asked" is distinguished from "asked but not answered".
• Answers are forced only where it's essential.
• The duration has been measured in the test, using the median.
• The questionnaire has been checked on a smartphone, tablet and desktop.
• At the end there's an open question and a thank-you.

Frequently asked questions about questionnaire design

How is a questionnaire structured?

A questionnaire starts with a short introduction, followed by an easy opening question, topic blocks from general to specific, the sociodemographic questions and a thank-you (Jacob et al. 2019, p. 200). Don't put sensitive topics at the start. More under Structure.

What types of survey questions are there?

Basically, open and closed questions. Closed questions include single choice, multiple choice, yes/no questions, rankings and scales (Malhotra and Birks 2007, pp. 381-382), plus matrix questions and, especially online, formats such as dropdowns and sliders. You'll find an overview under Question types.

How long should an online survey be?

There's no fixed limit. For online surveys, Jacob et al. give a maximum of 20 minutes (Jacob et al. 2019, p. 129), and some sources recommend less. Measure the duration in your test, using the median.

Should I use required questions (forced response)?

Only sparingly, for example for a filter question that's essential for the rest of the questionnaire, because forcing answers can lead to more break-offs and dishonest answers (Jacob et al. 2019, p. 123). On research ethics grounds, there is a strong case for a reminder that respondents can dismiss without answering (Möhring and Schlütz 2019, p. 180). More under Required questions.

Where do demographic questions go?

As a rule, at the end, with a short sentence on why you need the information (Malhotra and Birks 2007, p. 380; Schnell 2019, p. 109). They belong at the start only for questions about the target group, for quotas or if you want to compare those who drop out with the others.

What is a filter question?

A question whose answer determines where the questionnaire continues, so that nobody gets questions that don't apply to them. Record separately who was not shown a question and who saw it but left it unanswered (Schnell 2019, p. 116). More under Filter questions.

Sources

Al Baghal, T. and Lynn, P. (2015). Using motivational statements in web-instrument design to reduce item-missing rates in a mixed-mode context. Public Opinion Quarterly, 79(2), 568-579. Cited in Möhring and Schlütz (2019), p. 111.

Antoun, C., Couper, M. P. and Conrad, F. G. (2017). Effects of Mobile versus PC Web on Survey Response Quality. Public Opinion Quarterly, 81(S1), 280-306. Cited in Möhring and Schlütz (2019), pp. 147-148.

Brace, I. (2018). Questionnaire Design. How to Plan, Structure and Write Survey Material for Effective Market Research (4th ed.). London: Kogan Page. Cited in Schnell (2019), pp. 105 and 109.

Décieux, J. P., Mergener, A., Neufang, K. and Sischka, P. (2015). Implementation of the Forced Answering Option within Online Surveys: Do Higher Item Response Rates Come at the Expense of Participation and Answer Quality? Psihologija, 48, 311-326. Cited in Jacob et al. (2019), p. 123.

Dillman, D. A., Smyth, J. D. and Christian, L. M. (2014). Internet, Phone, Mail, and Mixed-Mode Surveys. The Tailored Design Method (4th ed.). Hoboken, NJ: Wiley. Cited in Schnell (2019), p. 295, and Möhring and Schlütz (2019), pp. 178 and 181.

Fuchs, M. and Lamnek, S. (1990). Fragebogendramaturgie und Abbruchverhalten [Questionnaire dramaturgy and break-off behavior]. planung & analyse, (3), 101-107. In German. Cited in Möhring and Schlütz (2019), pp. 111-112.

Jacob, R., Heinz, A. and Décieux, J. P. (2019). Umfrage. Einführung in die Methoden der Umfrageforschung [Surveys: an introduction to survey research methods] (4th ed.). Berlin/Boston: De Gruyter Oldenbourg. In German.

Linke, R. (2018). Mitarbeiterbefragungen optimieren. Von der Befragung zum wirksamen Management-Instrument [Optimizing employee surveys: from the survey to an effective management tool]. Wiesbaden: Springer Gabler. In German.

Malhotra, N. K. and Birks, D. F. (2007). Marketing Research. An Applied Approach (3rd European ed.). Harlow: Pearson Education.

Marx, P. (2026). Principles of Survey Research. Online course, Chapter 2 "Survey: Measurement and Scaling" and Chapter 3 "Questionnaire". QUESTIONSTAR. Course, chapter 3.

Mergener, A. and Décieux, J. P. (2018). Die "Kunst" des Fragenstellens. Praxisorientierte Hinweise zur Optimierung der Fragebogenqualität [The "art" of asking questions: practical advice on improving questionnaire quality]. In B. Keller, H.-W. Klein and T. Wirth (eds.), Qualität und Data Science in der Marktforschung. Prozesse, Daten und Modelle der Zukunft [Quality and data science in market research: processes, data and models of the future] (pp. 81-97). Wiesbaden. In German. Cited in Jacob et al. (2019), p. 128.

Mergener, A., Sischka, P. and Décieux, J. P. (2015). "To force or not to force. That is the question!": Die Auswirkungen des Einsatzes von Forced-Response-Fragen auf die Qualität der Befragungsergebnisse [The effects of forced-response questions on the quality of survey results]. In S. Lessenich (ed.), Routinen der Krise, Krise der Routinen [Routines of crisis, crises of routine]. Verhandlungen des 37. Kongresses der Deutschen Gesellschaft für Soziologie in Trier 2014 [Proceedings of the 37th Congress of the German Sociological Association in Trier 2014]. In German. Cited in Jacob et al. (2019), p. 123.

Möhring, W. and Schlütz, D. (2019). Die Befragung in der Medien- und Kommunikationswissenschaft. Eine praxisorientierte Einführung [Surveys in media and communication studies: a practical introduction] (3rd ed.). Wiesbaden: Springer VS. In German. DOI 10.1007/978-3-658-25865-8.

Pew Research Center (n.d.). Writing Survey Questions. Online, accessed November 9, 2026.

Peytchev, A. (2009). Survey breakoff. Public Opinion Quarterly, 73, 74-97. Cited in Möhring and Schlütz (2019), p. 112.

Schnell, R. (2019). Survey-Interviews. Methoden standardisierter Befragungen [Survey interviews: methods of standardized questioning] (2nd ed.). Wiesbaden: Springer VS. In German. DOI 10.1007/978-3-531-19901-6.

Schumann, S. (2019). Repräsentative Umfrage. Praxisorientierte Einführung in empirische Methoden und statistische Analyseverfahren [Representative surveys: a practical introduction to empirical methods and statistical analysis] (7th ed.). Berlin/Boston: De Gruyter Oldenbourg. In German.

Thielsch, M. T., Lenzner, T. and Melles, T. (2012). Wie gestalte ich gute Items und Interviewfragen? [How do I design good items and interview questions?] In M. T. Thielsch and T. Brandenburg (eds.), Praxis der Wirtschaftspsychologie. Bd. II, Themen und Fallbeispiele für Studium und Anwendung [Business psychology in practice, vol. II: topics and case studies for study and application] (pp. 221-240). Münster: Monsenstein und Vannerdat. In German.

Tourangeau, R., Couper, M. P. and Conrad, F. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368-393. Cited in Möhring and Schlütz (2019), p. 179.

Tourangeau, R., Maitland, A., Rivero, G., Sun, H., Williams, D. and Yan, T. (2017). Web Surveys by Smartphone and Tablets. Effects on Survey Responses. Public Opinion Quarterly, 81(4), 896-929. Cited in Möhring and Schlütz (2019), pp. 147-148.

Weinreich, U. and von Lindern, E. (2008). Praxisbuch Kundenbefragungen [Customer surveys: a practical handbook]. München: mi-Fachverlag, FinanzBuch Verlag. In German.

I wish you a questionnaire that your respondents are happy to complete to the end, and data you can really work with.

Related articles:

• Writing questions: how an idea becomes a robust question
• Order effects in surveys: what they distort and how to avoid them
• Employee survey: process, questions and analysis

Dr. Paul Marx

Author: Dr. Paul Marx
Dr. Paul Marx is the founder and Managing Director of QUESTIONSTAR and was a professor of marketing at the University of Siegen.