Panel survey: measuring change in a longitudinal study

Panel or trend study? What a panel survey shows, how to link waves by ID and how to deal with panel attrition. With worked examples and a planning checklist.

A panel survey asks the same people several times and repeats at least a fixed core of questions each time. That's why it shows who has changed and in which direction. If you instead survey a new sample each time (a trend study), or the same people without being able to link their answers, you only see whether the overall figure has shifted: the net change. In return, the panel asks more of you: an identifier that lets you link one person's answers, unchanged questions and a plan for the participants who drop out between surveys.

This article answers five questions in turn:

1. What is a panel survey, and how does it differ from a trend study?
2. Why do two surveys of different people show only the net change?
3. When do you need a panel, and when is a trend study enough?
4. How do you link the survey waves, and how far apart should they be?
5. What do you do when participants drop out of the panel, and what does repeated questioning change?

At the end, you'll find how to set it up in QUESTIONSTAR, from the contact list to the table that shows who switched, and a checklist for planning.

Two worked examples show what's at stake. In the first, 60.0% of customers are satisfied in each of two surveys, and yet 120 of 400 people have changed their verdict. In the second, the comparison suggests an increase of 5.0 percentage points, because the people who are missing are mainly those who weren't among the satisfied in spring. Among the same people, the share has actually fallen by 7.9 percentage points.

Panel here means your own repeated survey (employees, customers, a cohort of students), and not the purchased pool of participants that a provider offers and that is also called a panel, or more precisely an access panel. How to vet such providers is covered in Finding survey respondents.

What is a panel survey, and what is a trend study?

If you want to measure a change, you have to measure at least twice. A one-off survey, or cross-sectional survey, shows the situation at one point in time. To study change, you need several surveys conducted at different points in time, called waves (Jacob et al. 2019, p. 94). There are two ways to repeat a survey, and they differ in a single point: who is surveyed the second time.

Trend study: different people each time. A trend study consists of surveys conducted at different times with comparable content and independent samples, because different people are surveyed each time (Schnell 2019, p. 59). When the questionnaire is very similar, the sampling is the same and the interval is short, Schnell also speaks of repeated cross-sections (ibid.). An example: an online retailer draws a new random sample from its customer database every year and asks it the same questions.

A further case: the same people, but not linked. The anonymous employee survey that goes out to the whole workforce every year is, strictly speaking, not a trend study, because largely the same people answer. But since their answers can't be linked, you can only analyze the survey like a trend study: you see the overall figures for both years and not the path of the individual person.

Panel survey: the same people again. The definition in one sentence:

"A panel is a research design in which the same people are surveyed several times with (almost) the same instrument. A panel is a longitudinal design." (Möhring and Schlütz 2019, p. 158, my translation)

An English textbook puts it the same way: in a longitudinal design, the sample remains the same over time, so "the same people are studied over time" (Malhotra and Birks 2007, p. 76).

Many people know the pattern from home, from the pencil marks on the door frame with which parents record how their child is growing: the same child, the same tape measure, a new mark with the date.

Longitudinal study is the umbrella term for studies with several survey waves (Jacob et al. 2019, p. 94). Often it refers to the panel in the narrower sense, and the two words are frequently used interchangeably (Malhotra and Birks 2007, p. 76).

Why do two cross-sections show only the net change?

A service company asks its customers in spring and in fall how satisfied they are. Both times, 60% are satisfied. The obvious reading is: nothing has happened. But the two numbers alone can't tell you whether that's true. An everyday picture shows why not.

A city bus is making its way through town. At the City Hall stop there are 24 passengers on board, and a few stops later, at Central Station, there are 24 again. Did nobody get off in between, and did nobody get on? You can't read that from the two numbers. It's possible that the same 24 people rode all the way. It's just as possible that six got off along the way and six others got on. And if there are 26 passengers on the bus at Central Station, all you know is that two more people got on than got off. Whether it was two and none or eight and six, you don't know. Anyone who counts heads at two stops knows two states and the difference between them, but not the movement in between.

This matters to the transport planner. If she only wants to know whether the bus is big enough, counting is all she needs. If she wants to know whether people ride all the way from City Hall to Central Station or change along the way, counting doesn't get her any further. For that, she would have to recognize the passengers: know who is on the bus at Central Station and was already on it at City Hall. That takes more effort than counting, because someone has to keep track of the individual passengers.

It's similar with the satisfied respondents in a survey. The group of satisfied people is the bus. Between spring and fall, people leave this group: they were satisfied and no longer are. Others join. Two surveys whose answers can't be linked count heads at two stops: 60% in spring, 60% in fall. That applies to two new samples just as it does to the anonymous survey of the same workforce, where the same bus is counted but nobody recognizes the passengers. What you can see is the state at two points in time and the difference between the two states. This difference is called the net change: what's left after the switches in both directions have canceled each other out. It can be zero, as with 60% twice, or two points, as with 60% and 62%. In both cases, how many people switched in total stays hidden. That number is often called the gross change.

The movement only becomes visible once you recognize the respondents, that is, once you know which answer from fall belongs to which answer from spring. That's what the panel does: it surveys the same people and links their answers. Cross-sections, by contrast, are mainly suited to detecting net changes, such as a rise in an unemployment rate from 10% to 12%. They don't show how many people were unemployed at both points in time or how many became unemployed in between (Schnell 2019, p. 62).

The picture has two limits. Whoever gets off the bus is gone. Whoever leaves the group of satisfied people in the survey is still surveyed and now belongs to the other group. And on the bus the count is exact, whereas a survey measures less precisely: some people tick something different the second time without having changed their mind. Part of the observed switching is therefore measurement error and not real change (Schumann 2019, p. 112, n. 13).

The rule behind the picture: very different movements fit the same two states. How far that can go is shown by the following worked example, in which the bus is ten times as big: 240 satisfied people instead of 24 passengers, 60 switches in each direction instead of six.

Worked example: 60% satisfied twice, 120 of 400 have switched

So the same marginal totals say nothing about the inside of the table. With 240 satisfied people twice, anywhere between 0 and 320 of the 400 people may have switched: 0 if everyone sticks to their verdict, and 320 if all 160 who were not satisfied become satisfied and, in return, 160 of the 240 satisfied leave the group. Only linking the answers will show how many people actually switched.

A table that sets the answers of the first wave against those of the second is called a turnover table (Malhotra and Birks 2007, p. 77), or a transition table. Textbooks show the same idea with brands and parties: Brand A is bought by 200 of the 1,000 respondents in each of two periods, but only 100 of them buy it both times (Malhotra and Birks 2007, pp. 77-78), and a party that reaches 40% in each of two polls may have the same voters both times or completely different ones (Schumann 2019, p. 112, n. 12).

What you do with a turnover table: you now know two groups that two percentages say nothing about. The first were satisfied and no longer are. What happened to them? The others have become satisfied. What was different for them? Because you also have the other answers of both groups, you can look at how they differ from everyone else. Read the result as an indication and not as proof. Measurement error can make individual people appear to have improved although their attitude hasn't changed. Many analyses don't use this information at all and treat panel data like two cross-sections (Schnell 2019, p. 345).

In short: two surveys whose answers can't be linked show two states and the difference between them, the net change. How many people switched in which direction is shown only by the panel, because it recognizes the respondents, and even there part of the switching is measurement error.

Panel or trend study: which fits your question?

Your question decides which design fits. If you want to know how an overall figure develops, a trend study or a repeated survey without an identifier is enough. If you want to know who has changed, you need a panel.

Six considerations help you weigh the choice.

Precision. Two new samples differ from each other simply because of random selection. In a trend study, you therefore have to tell real changes apart from those that arise only from sampling error (Schumann 2019, p. 111, n. 11). To detect an actual change, a panel needs fewer cases than repeated cross-sections, and the gain is larger the more closely the answers of the two waves are related (Schnell 2019, p. 63 and n. 32). They're closely related when most people answer the same the second time as the first, that is, when few people switch. If they aren't related at all, the advantage disappears.

Anonymity. A panel has to be able to link one person's answers, at least through an identifier. A survey without any identifier can therefore only be analyzed like a trend study. That applies to the annual anonymous employee survey just as it does to the short pulse surveys in between.

Effort. With the same number of cases and the same mode of data collection, a trend study is cheaper, mainly because there's no panel maintenance (Schnell 2019, p. 60). A panel needs more lead time, a data protection concept and more careful data management, and it places a heavier burden on respondents (Schnell 2019, p. 63).

The target population changes. A panel reflects the people who were selected at the start. New employees or new customers aren't included unless you add them specifically. Repeated cross-sections are easier to adapt to a changed population (Schnell 2019, p. 62).

Asking in retrospect is no substitute. The question "Were you more satisfied a year ago?" saves you the second wave but doesn't deliver what it promises. Attitudes collected in retrospect are considered worthless for measuring earlier attitudes: what you collect is respondents' ideas about their earlier state, and not the state itself (Schnell 2019, p. 59).

Causes. A panel shows that something changed in the same people after an intervention. Textbooks therefore describe panels as supporting conclusions about causes (Jacob et al. 2019, p. 94; Möhring and Schlütz 2019, p. 159). More precisely: to conclude cause and effect, three conditions have to be met, namely an association, the right order in time and the exclusion of other explanations (Malhotra and Birks 2007, p. 303). The panel helps to settle the second. It doesn't settle the third: without a comparison group, it remains open whether the same people would have changed without the intervention as well. More on this under Correlation vs. causation.

How do you link the waves?

A survey becomes a panel only when one person's answers from different waves can be matched to each other and when the same thing was asked in every wave. You have to settle both before the first wave, because neither can be created after the fact.

The same person: a stable identifier

The transport planner would have to remember the faces on the bus. In a survey, an identifier does that job: one code per person that stays the same in all waves and appears in every data set. There are two ways to get one. For comparison, the third row of the table shows the survey without an identifier.

The self-generated code is the usual way out when you need individual trajectories but don't want to store names or addresses, for example with sensitive topics. The code questions should ask for characteristics that don't change and have a personal connection, for example letters from your own name or from relatives' names, or digits of a date of birth (Grube et al. 1989, cited in Pöge 2008, p. 60). Six to ten characters are usual (Schnell et al. 2006, cited in Pöge 2008, p. 60). Instructions for six characters could look like this:

Put together, the code in this example is AAK07X.

The instructions deliberately ask for nothing that's in the personnel file or the customer database, so neither the person's own date of birth nor their last name. Otherwise anyone who has these records could reconstruct the codes. That can never be ruled out completely, least of all in small groups. Even with a code, the data therefore remains pseudonymous and not anonymous.

Also expect that not every code will come out the same the second time. In a panel study with school students who formed the code anew on paper every year, it matched exactly between two waves in only about 60% of the matches found. The remaining pairs were only found by an error-tolerant procedure (Pöge 2008, p. 64). For your survey, that means three things:

1. Ask only for what everyone knows for certain and always writes the same way.
2. Before the analysis, check whether any code occurs twice.
3. Treat codes that differ in only one position as possible matches: check such cases against additional details that are as stable over time as possible, such as the year of birth. If the match remains ambiguous, don't analyze the answers as a linked pair.

Don't pass on personal links. In written and online surveys, there's no certainty about who actually fills in the questionnaire (Möhring and Schlütz 2019, p. 162). The authors therefore recommend online panels only with reservations (ibid.). With personal links sent to work or customer addresses the risk is smaller, but it isn't ruled out. Ask explicitly that the link not be passed on.

Linked data isn't anonymous. As long as answers can be matched to a person through an identifier, the data is pseudonymous and remains personal data. Say in the invitation that the waves will be linked and why, who can see the matching and when it will be deleted. What distinguishes anonymous from pseudonymous is covered in Data protection in online surveys. I'm not a lawyer and can't tell you what applies legally in your organization. So before the first wave, clarify with the works council or staff council and those responsible for data protection which data you link, who can see the matching and when it will be deleted.

The same instrument: wording and scale stay the same

Back at the door frame from the beginning: if you measure the child once with shoes on and once barefoot, you'll find a difference between two marks that isn't growth. Changing a survey question can create the same problem. When there's a change between two points in time, it usually can't be settled unambiguously whether it reflects real change or differences in how the data were collected, for example changed measurement instruments (Schnell 2019, p. 60). That applies to trend studies and panels alike.

Four rules follow from this:

• Core questions keep exactly the same wording. The same wording, the same number of scale points, the same labels, the same direction of the scale.
• A fixed core, changing additions. It's usual to repeat part of the questions in every wave and to ask some further questions only once per wave (Schnell 2019, p. 60). Put the core questions at the beginning so that changing additional questions don't influence them.
• Same mode, same circumstances. Don't switch between paper and online or between a survey in the seminar room and at home if you can avoid it.
• Mark changes as a break. If a question has to be changed, note it and don't compare this question across the break.

Even an unchanged question can come to measure something different over time. In a youth study, 35% suddenly said in 2016 that they were "drawing, painting, taking photos and filming" several times a week: young people now counted selfies as well (Jacob et al. 2019, p. 94). There's no universal solution to this (ibid., pp. 94-95). So before every wave, check whether your core questions are still understood the way they were the first time.

How far apart should the waves be?

The interval depends on the change you expect: it has to be long enough for the change to occur and short enough that you don't lose your participants. The literature doesn't give a fixed rule. The interval between the waves and their number are among the questions to be settled before you start (Schnell 2019, p. 325).

Three typical cases:

The considerations behind how often to run employee surveys are covered under How often should you run an employee survey?

Both directions have their price: more waves and a longer duration cost you participants (Jacob et al. 2019, p. 95), and longer intervals let addresses go out of date (Schumann 2019, p. 112, n. 15).

What do you do when participants drop out of the panel?

Expect that in every wave some people who answered before will be missing, and check whether those who remain still represent the same group. Dropping out means something different here than switching in the bus picture: there, someone left the group of satisfied people and was still surveyed. Here, someone leaves the survey itself. This loss is called panel attrition, in older textbooks also panel mortality: some of the respondents of the first wave are no longer available for the second, and as a rule this continues from wave to wave (Schumann 2019, p. 112; Schnell 2019, p. 329).

How large the loss is depends on the panel. Large population panels lose at least 3 to 5% per wave even under ideal conditions (Schnell 2019, p. 331), and for consumer panels up to 20% a year is cited (Malhotra and Birks 2007, p. 78). For your own online surveys of customers, employees or students, the sources evaluated here give no figures. Measure your own rate from the second wave on.

The reasons can be grouped into two: people can't be reached, or they lack motivation (Scherer 2000, p. 217, cited in Möhring and Schlütz 2019, p. 160). Malhotra and Birks name the same two: panel members drop out because they move away or lose interest (2007, p. 78). In your own panel, that means: the email address is no longer valid, someone has left the company or canceled the contract, or the person simply doesn't feel like it anymore.

Attrition becomes a problem when it isn't random, and as a rule it isn't (Schumann 2019, p. 112, n. 16). If the people who drop out are mainly people who resemble each other, that distorts the sample (Möhring and Schlütz 2019, p. 160). The second worked example shows what this can do.

Worked example: how attrition creates an apparent improvement

Whether a change in the same people is compatible with chance is checked by tests for paired samples: the McNemar test for yes/no proportions, the paired t-test for means. Which one fits when is covered under Which statistical test should I use? Where you can run the McNemar test is covered in the section on QUESTIONSTAR.

Checking and reporting attrition

Here a panel has an advantage over any one-off survey: for those who dropped out, you know the answers from the earlier waves (Schnell 2019, p. 337). Use that in four steps.

1. Report retention for each wave. How many answered in wave 1, how many of them in wave 2, how many in both?
2. Compare those who stayed with those who dropped out. Set the first-wave answers of the two groups side by side, for the core questions and for characteristics such as department, customer group or degree program. If they differ clearly, the measured change can't simply be transferred to all respondents of the first wave.
3. Measure changes only in the same people. For the statement "risen" or "fallen," what counts are the people who answered in both waves. The figures per wave for everyone who answered in that wave are reported separately, each with its own number of cases.
4. Report openly. A short paragraph in the methods section is enough, as in the example below.

"In the first wave, 400 people answered, and 280 of them (70.0%) also answered in the second. Those who dropped out were less often satisfied in the first wave than those who stayed (30.0% compared with 72.9%). Changes are therefore reported only for the 280 people who took part in both waves. They can't simply be applied to all respondents of the first wave."

Two limits remain. The comparison only uncovers differences that you measured in the first wave. And the loss can hardly be repaired by calculation: replacing dropouts with similar new people isn't advisable, because you often don't know which characteristics are related to the topic (Möhring and Schlütz 2019, p. 160, n. 75), and weighting rests on substantive assumptions and is no trivial task (Schnell 2019, p. 345). How to report exclusions and numbers of cases in general is covered under Survey data cleaning.

Planning ahead: start larger

Plan the first wave large enough that enough people are left in the last one. A rule of thumb for panels says: the first selection should include twice as many people as should still be in the panel after five repetitions (Möhring and Schlütz 2019, pp. 160-161). In other words, about 87% answer again from one round to the next. With an assumption of your own that's more cautious, you calculate like this: if 80% answer again from wave to wave, then of 400 people, 320 remain in the second wave, 256 in the third and 205 in the fourth. If you want 200 people to be left in the third wave by this calculation, you need at least 313 answers in the first wave under this assumption: 200 / (0.8 × 0.8) = 312.5, rounded up to 313. How many invitations that takes, you calculate with your response rate.

How many people should be left in the last wave depends on how small a change you still want to tell apart from chance. As a guide, calculated as in the example on precision under Panel or trend study: if 30% of those who stayed change their verdict and the two directions roughly balance out, the chance range is about ±10.7 percentage points with 100 people, about ±7.6 with 200 and about ±5.4 with 400. A measured change that's smaller than this range may be chance.

Does repeated surveying change the answers?

Yes, that can happen, though usually only to a small degree. The technical term is panel conditioning, also called the panel effect: the effect that a survey within a panel has on response behavior in later waves (Schnell 2019, p. 327). Anyone who's asked about a topic several times pays more attention to it, forms an opinion or sticks to an opinion once expressed. Motivation can also rise or fall (Waterton and Lievesley 1989, cited in Schnell 2019, p. 327). Malhotra and Birks describe it as response bias in panels: new members are often biased in their first answers, and long-standing members may give biased answers too, for example out of boredom or fatigue (2007, p. 78). Topics on which respondents didn't yet have much of an opinion in the first wave are particularly susceptible (Möhring and Schlütz 2019, p. 162).

How strong is the effect? It can hardly be avoided, but for most research questions it seems to be rather small (Schnell 2019, p. 327). One literature review reports deviations of panel estimates from cross-sectional estimates of 5 to 15%, with wide variation even for the same research question (Cantor 2007, p. 132, cited in Schnell 2019, p. 327, n. 8).

The effect can be checked with a comparison: alongside the panel wave, survey a new sample that has never taken part. If the results differ more than the chance variation of the two samples explains, that may be due to the repeated questioning or to attrition (Schnell 2019, p. 64). For small projects, this is rarely feasible. In that case, mention the possibility in the report, especially for topics your respondents had hardly thought about before the first wave.

Panel maintenance: consent, contact details, reminders

Panel maintenance means everything that makes sure participants stay reachable and want to keep taking part (Schnell 2019, p. 338). What works best is what's in your own hands: survey incentives and, above all, how pleasant respondents found the previous waves (ibid.). Anyone who experiences the first wave as boring or time-consuming will hardly agree to a repetition (Schnell 2019, p. 331).

Two points need a further note. Consent is itself a selection stage: those who agree may differ from those who decline (Schnell 2019, p. 330). So count how many of the first-wave respondents agreed. And with contact details, what counts is how long the address will last: a university email address often stops working after graduation, and a work email address when someone leaves the employer. If you want to reach graduates or former customers again, you're better off asking for a private address.

Panel surveys with QUESTIONSTAR

In QUESTIONSTAR, you put together a panel survey of your own from ordinary building blocks. You survey your own contacts. QUESTIONSTAR doesn't provide a panel of participants. The overview shows what you set up. Below it are the details and the steps from two exports to the turnover table.

The personal link contains a randomly generated identifier and therefore must not be passed on (Recognizing returning respondents).

One key that appears in both waves. Give each person in the contact list a personal ID and keep the same ID in all waves. It appears in the answers of every wave and serves as the key for linking. You can equally use a participant number of your own or the email address as the key. The identifier of the individual response, by contrast, is new in every wave and isn't suitable for linking. Before sending, check with a test contact that the key is output identically in the exports of both waves.

Set anonymity deliberately. The anonymity setting has two levels:

For a panel, at the stricter level only the self-generated code remains, as a question in the questionnaire, and the answers linked this way are then no longer anonymous in the strict sense. At that level, reports only appear once ten people have completed the questionnaire in full. Neither setting can be undone (Privacy settings), so decide before the first wave. You also determine which users of your account have access to the results or to the personal details.

Consent via double opt-in. If the first wave starts with an open link, ask for the email address for the next wave at the end. With double opt-in, QUESTIONSTAR sends a confirmation email after the questionnaire has been completed. Consent only takes effect with the click on the link in it, and the address, together with the confirmation, is stored in a contact list that you can download later and use to get in touch. The key here is the email address: it's stored with the answers of the first wave (in the second export file if personal details are stored separately) and in the contact list. Or you ask for the self-generated code as early as the first wave. It's an ordinary text question, works in every wave with an open link as well, and can be carried over into the contact list as a further field via double opt-in.

From two exports to the turnover table

The reports work within one survey. They don't compare different surveys, and so not two waves either. You therefore calculate the comparison of the waves outside the reports: in Excel or with an AI assistant.

In Excel, it takes five steps:

1. Export both waves, as an XLSX or CSV file (for statistics programs also as an SPSS file). In both files, the personal ID is stored with each response.
2. Put each person's answers side by side. Add a new column to the table of the first wave and use the Excel function XLOOKUP to fetch the answer of the second wave via the personal ID. The formula is in the example below this list. Each row is then one person with both answers.
3. Mark the dropouts. Where the function shows the error value #N/A, the personal ID wasn't found in the second file. First check whether the IDs are written correctly and identically in both files. If the person really is missing in wave 2, mark them as dropped out. You need these rows for comparing those who stayed with those who dropped out. For the turnover table, you filter them out.
4. Group the answers if you form two groups as in the worked examples: in one further column each, 4 and 5 as satisfied, 1 to 3 as not satisfied. Missing answers and "no answer" count toward neither group. Only people who answered this question in both waves belong in the turnover table.
5. Count with a pivot table. That's the Excel feature that counts how often each combination occurs (Insert tab, PivotTable). Drag the answer from wave 1 into the rows, the answer from wave 2 into the columns and the personal ID as a count into the values. The result is your turnover table.

Step 2 with a small example (made-up rows). Copy the table of the second wave into the file of the first wave as a second sheet and name the two sheets "Wave1" and "Wave2". In the example, column A contains the personal ID and column B the answer to the satisfaction question (1 to 5).

Sheet "Wave1":

Sheet "Wave2" (the order of the people may be different, and person 1004 didn't answer again):

In the sheet "Wave1", enter this in cell C2:

=XLOOKUP(A2,Wave2!A:A,Wave2!B:B)

In words: take the ID from A2, look for it in column A of the sheet "Wave2" and return the value from column B of the same row. In the English version of Excel, the three arguments are separated by commas. For person 1001, the formula returns 5. Copy the formula down into cells C3 to C5: there you'll see 3, 4 and #N/A, because the ID 1004 doesn't occur in the sheet "Wave2". In older Excel versions without XLOOKUP, =VLOOKUP(A2,Wave2!A:B,2,FALSE) does the same, with FALSE as the last argument so that only exact matches count. In your file, replace A and B with the columns in which the personal ID and the answer are actually stored.

This is what the linked table looks like after step 4 (the same four people):

The pivot table counts the combinations of the last two columns: person 1001 lands in the cell "stayed satisfied", 1002 and 1003 are switchers, and 1004 is missing in wave 2 and isn't counted in the turnover table.

The second approach doesn't require Excel. You connect QUESTIONSTAR to an AI assistant such as ChatGPT or Claude via the MCP interface and ask it, for example: "Link wave 1 and wave 2 using the respondents' personal ID. Create a turnover table for the overall satisfaction question (4 and 5 against the rest) and compare the first-wave answers of those who answered again in wave 2 with those who are missing." The calculation is done by the assistant and not by QUESTIONSTAR. Check one result by hand.

• McNemar test. The QUESTIONSTAR reports don't calculate a McNemar test. You calculate it with the exported data in a statistics program or have the assistant calculate it. All it needs are the two switcher counts from the turnover table, 40 and 18 in the second example above.
• Protection of personal data. Questions marked as personal data aren't passed to the assistant when the protection setting is switched on. Free-text answers can still contain names, so check beforehand what is passed on.

By the way: the least work comes with a panel that runs through a contact list with personal links from the first wave on. Then you see in every wave who's missing, you remind only those people, and the personal ID links the answers across all waves.

Checklist: planning a panel survey

• Question settled: is it about how an overall figure develops (a trend study or a repeated survey without an identifier is enough) or about who has changed (panel)?
• Number of waves and interval set to fit the expected change.
• Way of linking chosen: the same personal ID for each person in all waves, or a self-generated code. Checked with a test contact that the same key appears in the data of every wave.
• Core questions defined that are asked in every wave word for word and with the same scale.
• Said in the invitation that there are several waves, why the answers are linked and who can see the matching. Involved the works council or staff council and those responsible for data protection, where applicable.
• Consent to being contacted again obtained, lasting contact address asked for.
• First wave planned larger than the last one should be.
• Results reported back between the waves and the next wave announced.
• After every wave: retention reported, those who stayed and those who dropped out compared on the basis of the first wave.
• Changes reported only for people who answered in both waves, together with the turnover table.

Frequently asked questions about panel surveys

What is a panel survey?

A panel survey asks the same people several times with (almost) the same questionnaire (Möhring and Schlütz 2019, p. 158). The individual surveys are called waves. Because one person's answers can be linked across the waves, a panel shows not only whether an overall figure has changed but also who has changed.

What is the difference between a panel and a trend study?

Both repeat a survey. The trend study draws a new sample each time, while the panel surveys the same people again and links their answers. A trend study therefore shows only changes in overall figures, such as proportions or means. Changes in individual people can only be identified with a panel (Schumann 2019, pp. 111-112). An anonymous repeated survey of the same people can only be analyzed like a trend study.

What is a longitudinal study?

A longitudinal study examines the same question at several points in time, in contrast to a cross-sectional study with only one survey (Jacob et al. 2019, p. 94). Longitudinal studies include the panel (the same people), the trend study (new people each time) and the cohort study, a panel of people who experienced the same event at the same time, such as starting university in the same semester (Schnell 2019, p. 325).

What is panel attrition?

Panel attrition, also called panel mortality, is the loss of participants from wave to wave: some of those who answered in the first wave no longer answer later (Schumann 2019, p. 112). This is a problem above all when certain groups drop out more often than others, because those who remain then no longer represent the same group.

What is panel conditioning?

Panel conditioning (also called the panel effect) occurs when the repeated survey itself changes the answers, for example because respondents become more attentive to the topic (Schnell 2019, p. 327). For most research questions, the effect seems to be rather small (ibid.). It can be checked with a new sample surveyed in parallel.

Can a panel survey be anonymous?

Not in the strict sense, because one person's answers have to be linked through an identifier. What is possible is a panel without names and addresses: in every wave, respondents form a personal code by following the same instructions. The data remains pseudonymous, and some codes are formed differently the second time: in a study of school students, only about 60% of the matches found were exact (Pöge 2008, p. 64). A survey with no identifier at all can only be analyzed over time like a trend study.

How many people do I need for a panel?

Enough that enough are left in the last wave. One rule of thumb says: twice as many as should still be in the panel after five repetitions (Möhring and Schlütz 2019, pp. 160-161). It's better to calculate with your own retention: at 80% from wave to wave, 256 of 400 people remain in the third wave.

Sources

Cantor, D. (2007). A Review and Summary of Studies on Panel Conditioning. In S. Menard (Ed.), Handbook of Longitudinal Research: Design, Measurement, and Analysis (pp. 123-138). Amsterdam: Academic Press. Cited in Schnell (2019), p. 327, n. 8.

Grube, J. W., Morgan, M. and Kearney, K. A. (1989). Using Self-Generated Identification Codes to Match Questionnaires in Panel Studies of Adolescent Substance Use. Addictive Behaviors, 14, 159-171. Cited in Pöge (2008), p. 60.

Jacob, R., Heinz, A. and Décieux, J. P. (2019). Umfrage. Einführung in die Methoden der Umfrageforschung [Surveys: an introduction to the methods of survey research] (4th ed.). Berlin/Boston: De Gruyter Oldenbourg. In German.

Malhotra, N. K. and Birks, D. F. (2007). Marketing Research: An Applied Approach (3rd European ed.). Harlow: Pearson. Chapter 3, Research design, pp. 74-78 (longitudinal designs and panels, turnover table, mortality and response bias in panels), and p. 303 (conditions for causality).

Möhring, W. and Schlütz, D. (2019). Die Befragung in der Medien- und Kommunikationswissenschaft. Eine praxisorientierte Einführung [The survey in media and communication studies: a practical introduction] (3rd ed.). Wiesbaden: Springer VS. In German. DOI 10.1007/978-3-658-25865-8.

Pöge, A. (2008). Persönliche Codes "reloaded" [Personal codes "reloaded"]. Methoden, Daten, Analysen (mda), 2(1), 59-70. In German. Full text at SSOAR.

Scherer, H. (2000). Wählt das Panel anders? [Does the panel vote differently?] In H. Bohrmann, O. Jarren, G. Melischek and J. Seethaler (Eds.), Wahlen und Politikvermittlung durch Massenmedien [Elections and political communication through mass media] (pp. 213-234). Opladen: Westdeutscher Verlag. In German. Cited in Möhring and Schlütz (2019), p. 160.

Schnell, R. (2019). Survey-Interviews. Methoden standardisierter Befragungen [Survey interviews: methods of standardized surveys] (2nd ed.). Wiesbaden: Springer VS. In German. DOI 10.1007/978-3-531-19901-6.

Schnell, R., Bachteler, T. and Reiher, J. (2006). Die Anwendung statistischer Record-Linkage-Methoden auf selbstgenerierte Codes bei Längsschnitterhebungen [Applying statistical record linkage methods to self-generated codes in longitudinal surveys]. ZA-Information, 59, 128-142. In German. Cited in Pöge (2008), p. 60.

Schumann, S. (2019). Repräsentative Umfrage. Praxisorientierte Einführung in empirische Methoden und statistische Analyseverfahren [Representative surveys: a practical introduction to empirical methods and statistical analysis] (7th ed.). Berlin/Boston: De Gruyter Oldenbourg. In German.

Waterton, J. and Lievesley, D. (1989). Evidence of Conditioning Effects in the British Social Attitudes Panel Survey. In D. Kasprzyk, G. Duncan, G. Kalton and M. P. Singh (Eds.), Panel Surveys (pp. 319-339). New York: Wiley. Cited in Schnell (2019), p. 327.

I hope most people in your panels stay until the last wave, and that your comparisons show who has moved.

Related articles:

• Employee survey: process, questions and analysis
• Survey response rate: calculate, assess and increase it
• Which statistical test should I use? A guide with examples

Dr. Paul Marx

Author: Dr. Paul Marx
Dr. Paul Marx is the founder and Managing Director of QUESTIONSTAR and was a professor of marketing at the University of Siegen.