Representative sample: what it is and how to check yours

More responses don't make a survey representative. Here's what really matters, how to check your sample and how to report its limits honestly.

"Is your survey representative?" Students hear this question at their thesis defense, if not sooner. HR and CX managers hear it when management asks for "solid numbers". The short answer: a representative sample is one that mirrors the population in the characteristics that matter for your question, including the ones you can't check. The only reliable way to get there is random selection, and even a random sample gives you only an approximately accurate picture. That's why, strictly speaking, survey methodology reserves the word for random samples. The number of responses alone doesn't make a sample representative.

In this article I'll show you why that is, how to check your own sample, and how to describe the result so that your supervisor or your manager can work with it. For how many people to survey, use our calculator: calculate your sample size. Here we deal with the other half of the question: who those people are.

Representative sample: definition and an everyday example

A representative sample is a small group that reflects the whole group you care about, so that what you find in the sample holds, roughly, for everyone. Think of a pot of soup. You've cooked a big pot and want to know whether it has enough salt. You don't have to eat the whole pot for that. One spoonful is enough, provided you've stirred well first. Then every spoonful tastes roughly like the whole soup. Whether the pot holds five liters or fifty hardly matters.

If the salt has settled at the bottom, though, a spoonful from the surface tells you very little. A bigger spoon from the surface doesn't help either. All you get is a very precise taste of the top layer.

A sample works exactly the same way. The pot is the population, that is, everyone you want to say something about. The spoon is the sample. Stirring corresponds to random selection: every person in the population has a known chance, greater than zero, of ending up in the sample. The baker in our article on calculating sample size pierces the bread in several spots. The soup shows what happens when you always taste from the same spot. A sample like that spoonful from the top is called non-representative, or biased: it differs from the population in a systematic way, not by chance.

By the way, "representative" isn't a technical term in statistics. Schumann (2019, p. 84), author of a German textbook on representative surveys, notes that the literature defines the word in different ways. For him, a "representative survey" is simply shorthand for a survey based on a properly designed random sample. Jacob, Heinz and Décieux (2019, p. 71) take the same view.

Representative of whom?

Schumann (2019, pp. 84-85) distinguishes three groups.

The target population is everyone you want to make statements about. Take a student who, for her bachelor's thesis, wants to know where students in her department go to study during exam season.

The frame population is everyone who has any chance at all of being invited. If the student shares her link in the WhatsApp groups of her year, that's the members of those groups. Anyone who isn't in one of the groups can't take part.

The inference population is the group you can actually draw conclusions about in the end. If, for example, students who study part-time alongside a job don't answer because they're short on time, you can't say anything about them, even though they were invited.

The goal is for these three groups to match as closely as possible. That's why you decide whom your results should apply to before the survey, not when you analyze the data.

Representative in which characteristics?

No sample reproduces every characteristic of a population. It's finite, so it can't (Schumann 2019, p. 84, footnote 7, citing Diekmann). The practical question is: which characteristics are related to what you're measuring? For employee satisfaction, that might be department, shift work and length of service. Hair color isn't one of them. For our student, who asks about places to study, what counts is whether someone studies full-time or part-time.

Why a bigger sample isn't more representative: sampling bias

The most famous proof of this is 90 years old. Before the 1936 US presidential election, the magazine Literary Digest wrote to 10 million Americans. The addresses came from subscriber lists, telephone directories and club membership lists. 2.4 million questionnaires came back. The forecast: Republican Alf Landon would win clearly, and Franklin D. Roosevelt would get only 43% of the vote. In fact, Roosevelt won by a landslide, with about 61% of all votes and 62% of the votes for the two major candidates. George Gallup correctly predicted Roosevelt's victory with a quota sample that was only a fraction of that size (Jacob, Heinz and Décieux 2019, pp. 71-72).

The authors give two reasons for the failure. The first was the selection: in 1936, people who owned a telephone or belonged to a club were disproportionately middle and upper class and tended to vote Republican. The second was the response: fewer people from lower-income groups sent the questionnaire back. Textbooks call the first problem selection bias (often also called sampling bias) and the second nonresponse bias (Malhotra and Birks 2007, pp. 412 and 488). If the selection is flawed from the start, the authors write, no sample size, however large, can fix it. The error simply repeats itself on a large scale.

Worked example: an employee survey

A company has 1,000 employees, 600 in the office and 400 in production. Let's assume we know the truth: 70% are satisfied in the office and 40% in production. Overall, then,

0.6 × 70% + 0.4 × 40% = 58% are satisfied.

The survey runs via a link on the intranet. In the office, everyone sits at a computer. In production, hardly anyone does. 350 people from the office answer and 50 from production, 400 in total. A response rate of 40%, which many would be proud of. The result:

(350 × 70% + 50 × 40%) / 400 = (245 + 20) / 400 = 66% satisfied.

The survey is 8 percentage points too high. If you calculate the usual margin of error for 400 responses (95% confidence level, with the correction for a finite population), you get about ±3.6 percentage points. That sounds precise. But the range from roughly 63% to 70% doesn't contain the true value of 58%.

Now HR sends two reminders. In the end, all 600 office employees have answered, and still only 50 from production. Result: (420 + 20) / 650 = 68%. More responses, bigger error.

The margin of error only measures random error, the fluctuation that comes from not surveying everyone. It's blind to a skewed selection. That's why the American Association for Public Opinion Research (AAPOR) makes clear that a margin of error applies only to samples with known selection probabilities. For samples that people opt into themselves, it shouldn't be reported (AAPOR statement on the margin of error, and Baker et al. 2013).

How to get a representative sample

There are essentially two routes to representative sampling.

The first is random sampling. For that you need a list that contains your target population as completely as possible, such as the staff list or the customer database. You draw from it at random, and with small groups you often simply invite everyone. Then the problem shifts to the people who don't respond. Whether these nonrespondents differ on your topic can rarely be checked, because you'd need exactly the data that are missing (Schumann 2019, p. 105).

The second is quota sampling. You specify how many people with certain characteristics should be in the sample, for example 52% women and 48% men, as in the population. Who gets surveyed within these quotas is left open. On the street, interviewers tend to approach people who look friendly and are easy to reach (Malhotra and Birks 2007, pp. 413-414). Online, the quota fills up with whoever sees the link first.

Ironically, it was Gallup who showed that quotas can miss too. In 1948, his institute used a quota sample to predict that Thomas Dewey would win. The winner was Harry Truman (Jacob, Heinz and Décieux 2019, p. 17). The Chicago Daily Tribune headline "Dewey Defeats Truman" opens the sampling chapter of my course Principles of Survey Research (slide 118).

Quotas have a second catch. If gender and field of study are set as separate quotas, the sample can be right on both characteristics and still be skewed. Say that out of 100 respondents, 50 should be women and 50 men, and also 50 from business administration and 50 from economics. These targets are met even if all the women study business administration and all the men study economics. Jacob, Heinz and Décieux (2019, pp. 88-89) show this with real student numbers.

Experts disagree on how good quota samples are (Jacob, Heinz and Décieux 2019, p. 90, and Malhotra and Birks 2007, p. 414). Everyone agrees on one thing: a quota sample matches on the quota characteristics. For all other characteristics, that's a hope, not a certainty (Schumann 2019, p. 98).

You'll find a compact overview of strengths and weaknesses in chapter 4, "Sampling", of my course Principles of Survey Research (slide 134).

Online surveys: when they can be representative

Whether an online survey can be representative depends above all on how people get to your questionnaire.

The open link. You post the link on Instagram or in your newsletter, and whoever feels like it takes part. That's a convenience sample with self-selection. In the words of Malhotra and Birks (2007, p. 412), convenience samples "are not representative of any definable population". They're still useful for exploratory studies that first gather ideas. Many good bachelor's theses work exactly this way. Just don't sell the result as representative.

The online panel. If a random sample is drawn from a panel, it's representative only of the panel to begin with. Whether the panel itself reflects the population depends on how it was recruited. Many online panels are built through self-registration, and it's the motivated people who tend to stay (Jacob, Heinz and Décieux 2019, pp. 92-93). So ask the provider how the panel was recruited, not just about the quotas.

The complete list. This is where online surveys are strong. Jacob, Heinz and Décieux (2019, p. 126) consider them particularly well suited when they're based on an address list, such as a customer file or the email directory of a university or a company. For HR and CX, that's the normal case. You can invite everyone or draw at random, send targeted follow-ups and check afterward who answered. Check whether everyone really is on the list. Employees without a work email address, as in our production example, need their own way in, for instance a QR code on the notice board or a tablet in the break room.

In QUESTIONSTAR you can set quotas per answer option or respondent profile, that is, a maximum per group, and decide what happens when a quota is full: continue to the end of the survey, branch to another path, or politely decline. That way you prevent one group from flooding the sample. With screening questions you keep out people who don't belong to your population. Email reminders go only to those who haven't answered yet. A quota doesn't bring in missing groups, though. In our example, a quota would cap the office, and it wouldn't produce a single additional response from production. None of this guarantees representativeness, because even a full quota says nothing about who answered within the quota.

Weighting: when it helps and when it doesn't

If the sample is skewed, the obvious idea is to correct it mathematically. That's called weighting. Each group gets the weight "share in the population (target) divided by share in the sample (actual)". Schumann (2019, p. 106) works through an example: women aged 50 to 60 make up 8.3% of eligible voters in Germany, but only 7.1% of the sample. Each of these women then counts with a weight of 8.3 / 7.1 = 1.17.

Let's apply this to our employee survey. The office makes up 60% of the workforce but 87.5% of the responses. Weight: 0.6 / 0.875 = 0.69. Production makes up 40% of the workforce but only 12.5% of the responses. Weight: 0.4 / 0.125 = 3.2. The 350 office responses then count together like roughly 240 people (350 × 0.69), and the 50 from production like 160 (50 × 3.2). Weighted, you get (240 × 70% + 160 × 40%) / 400 = (168 + 64) / 400 = 58%, the true value again.

But this only works if the 50 production employees who answered think like the 350 who didn't. And that's exactly what you don't know. Suppose that mostly shift supervisors answered, because they sit at a computer more often. They're more satisfied, say 60% instead of 40%. Then weighting gives 0.6 × 70% + 0.4 × 60% = 66%. Unweighted, it would be (245 + 30) / 400 = 69% in this case. So weighting recovers only 3 of 11 percentage points. It can't do more, because all it did was multiply the opinions of 50 people by 3.2.

There's also a price people often overlook: unequal weights make the result less certain. In our example, 50 people carry 40% of the result. Each single response from production weighs almost five times as much as one from the office. By a common approximation from the statistician Leslie Kish, the 400 weighted responses count about as much as 240 unweighted ones.

Jacob, Heinz and Décieux (2019, p. 93) call weighting by age and gender an untestable hope that all other characteristics will fall into line as well. If you weight, the procedure belongs in the report (Malhotra and Birks 2007, p. 488). And weights that make certain customers count more because of their revenue answer a different question. Report such results separately.

If you need to weight, export the raw data as an SPSS, CSV or Excel file and do the weighting in SPSS, Stata or R (for example with the "survey" package). My advice for most theses and internal surveys: check first and report openly. Weight only if you have good reference data and can justify why the respondents in each group resemble the nonrespondents.

Is my sample representative? How to check and report it

With data on the population, you can check where your sample demonstrably deviates on known characteristics. That doesn't prove representativeness. Preparation starts before data collection, because you can only compare what you've asked.

1. Get reference data: your university's student statistics or registrar's office, the HR system, the customer database, or, for the general population, your national statistics office.
2. Ask about exactly these characteristics, in the same categories. If you ask "under 30", you can't compare with "18 to 24".
3. Put both side by side in a table.
4. Assess each deviation by whether the characteristic is related to your research question.

Let's look at an example with made-up numbers. Our student has collected 212 responses via WhatsApp and Instagram groups. Topic: places to study during exam season.

The sample contains clearly more women, more business administration students and more bachelor's students than the department, mostly people from the student's own circle. That's normal for convenience samples. The last row matters most. Part-time students probably study differently, often in the evening and at home. With 6 responses, nothing can be said about them. An overall figure like "62% study in the library during exam season" therefore describes mainly full-time bachelor's students.

And even if every row had matched: people who are active in WhatsApp groups and fill out their classmates' surveys may also be more diligent than average. No comparison table shows that.

Wording for your methods section

You can describe the limitations in your thesis like this, or along similar lines:

"The sample was obtained as a convenience sample via student groups on social media. It is therefore not representative of the students in the department. A comparison with university statistics shows an overrepresentation of women and business administration students and a marked underrepresentation of part-time students (Table X)."

"The results should be read as indicative of the surveyed group only. Margins of error are not reported, as they require random selection."

"Too few responses are available for part-time students. A targeted survey of this group would be a sensible next step."

The same belongs in a report to management: who was invited, who answered, which groups are missing and what that means for the numbers. For our employee example, something like this:

"All 1,000 employees were invited. 350 of 600 in the office and 50 of 400 in production answered. The overall figure of 66% satisfaction therefore mainly reflects the office. For production, the numbers are not reliable."

Indicative results instead of representativeness

"Indicative, not representative" describes exactly what your data can do. Many theses don't try to estimate percentages at all. They test whether two things are related, for example place of study and exam anxiety. For nonresponse in random samples, Schumann (2019, p. 106, footnote 56) points out that shares and means are usually more affected than relationships. For convenience samples, that's only a pointer. If participation itself is related to both characteristics, a relationship can be distorted too, for instance when students with severe exam anxiety don't even open a survey about exam stress. How to structure the methods section as a whole is covered in our article How to write my thesis.

Frequently asked questions about representative samples

How many respondents make a survey representative?

There's no number at which a survey becomes representative. Size determines how precise your result is, provided the selection was random. How many responses you need for a given precision is shown by our sample size calculator. If the selection is skewed, even 2.4 million responses won't make it better.

Can an online survey be representative?

Not through an open link on social media, because the participants select themselves. Based on a complete list, such as all employees or all customers, it has a good starting point. Then it comes down to who answers and who doesn't.

Is a quota sample good enough?

For many practical purposes it's a reasonable compromise. But it only ensures that the quota characteristics match. So write "quota sample by age and gender" and not simply "representative".

Does my bachelor's thesis need a representative sample?

Usually not. Most bachelor's theses work with convenience samples, and supervisors know that. What's expected is that you know the limitations and name them. If in doubt, clarify with your supervisor whether your research question aims to estimate shares or to examine relationships.

What if my sample isn't representative?

Describe openly which groups are missing, as shown in the section Is my sample representative? How to check and report it. If there's still time, reach out specifically to the missing groups. You'll find ideas for that in our article Where do I find respondents for my survey.

Sources

Jacob, R., Heinz, A. and Décieux, J. P. (2019). Umfrage. Einführung in die Methoden der Umfrageforschung [Surveys: an introduction to survey research methods] (4th ed.). Berlin/Boston: De Gruyter. In German. pp. 16-17 (Literary Digest 1936, Gallup 1948), pp. 71-72 (population and sample, Literary Digest), pp. 88-90 (quota sampling), pp. 92-93 (access panels), p. 126 (online surveys).

Schumann, S. (2019). Repräsentative Umfrage. Praxisorientierte Einführung in empirische Methoden und statistische Analyseverfahren [Representative surveys: a practical introduction to empirical methods and statistical analysis] (7th ed.). Berlin/Boston: De Gruyter. In German. pp. 84-85 (representativeness, target, frame and inference population), p. 98 (quota sampling), pp. 105-106 (nonresponse, weighting).

Malhotra, N. K. and Birks, D. F. (2007). Marketing Research: An Applied Approach (3rd European ed.). Harlow: Pearson. pp. 412-414 (convenience and quota sampling, selection bias), p. 488 (weighting, nonresponse bias).

Marx, P. Principles of Survey Research (slide deck, 2026), chapter 4 "Sampling", slides 118-134.

Baker, R., Brick, J. M., Bates, N. A. et al. (2013). Report of the AAPOR Task Force on Non-Probability Sampling. American Association for Public Opinion Research. Short version: Summary Report of the AAPOR Task Force on Non-probability Sampling. Journal of Survey Statistics and Methodology, 1(2), 90-143.

AAPOR (n.d.). Margin of Sampling Error/Credibility Interval (statement). aapor.org.

Found a correction or addition? Write to editors@questionstar.com. We read everything and reply personally.

I wish you every success with your survey!

Related articles:

• Sample size calculator: calculate your sample size the easy way
• Where do I find respondents for my survey
• Calculating sample size: technical details explained simply

Date: November 3, 2026
Author: Dr. Paul Marx