How many responses you need
A survey asks a sample instead of everyone, so its result is an estimate. The sample size decides how close that estimate is likely to be. Three numbers set it: the confidence level (how sure you want to be, usually 95%), the margin of error (the plus or minus range you accept, such as ±5 percentage points) and the expected proportion (your best guess of the result, 50% if you have none). A fourth, the population size, only matters when the sample is a noticeable share of it.
Here z is the critical value of the normal distribution for your confidence level, p the expected proportion and E the margin of error, both as fractions. The second formula is the finite population correction. It is the same as the form in the Penn State STAT 506 lesson on sample size for a proportion, n = N × p(1 − p) ÷ ((N − 1) × E² ÷ z² + p(1 − p)), rearranged. The result is always rounded up, because a sample size of 384.1 needs a 385th response to reach the margin. For a very large population the last factor of the margin of error is close to 1 and drops out, which leaves the normal-approximation interval p ± z × √(p(1 − p) ÷ n) in the NIST/SEMATECH e-Handbook.
The calculator uses exact critical values, such as 1.95996 for 95%, rather than the rounded 1.96. The difference almost never changes the rounded sample size.
| Confidence level | z (two-sided) |
|---|---|
| 80% | 1.282 |
| 85% | 1.440 |
| 90% | 1.645 |
| 95% | 1.960 |
| 98% | 2.326 |
| 99% | 2.576 |
| 99.9% | 3.291 |
Worked example
You want to survey customers at 95% confidence with a margin of error of ±5 points and no earlier estimate, so p = 50%. Then n₀ = 1.95996² × 0.5 × 0.5 ÷ 0.05² = 384.15, rounded up to 385 responses. If you have only 2,000 customers, the correction gives 322.39, so 323 responses are enough. The other way round: 400 responses from a large population give a margin of error of ±4.90 points at 95%.
Penn State's lesson works through an approval rating with a margin of error of 3 points at 95%: with an earlier estimate of 22% it needs 733 people, and with the conservative 50% it needs 1,068. The calculator gives 733 and 1,068 for the same inputs.
Sample sizes at a glance
Responses needed for a very large population with p = 50%:
| Margin of error | 90% confidence | 95% confidence | 99% confidence |
|---|---|---|---|
| ±1% | 6,764 | 9,604 | 16,588 |
| ±2% | 1,691 | 2,401 | 4,147 |
| ±3% | 752 | 1,068 | 1,844 |
| ±5% | 271 | 385 | 664 |
| ±10% | 68 | 97 | 166 |
Halving the margin of error roughly quadruples the sample, because E is squared in the formula. And the population size, at 95% confidence with p = 50%:
| Population | ±5% margin | ±3% margin |
|---|---|---|
| 100 | 80 | 92 |
| 500 | 218 | 341 |
| 1,000 | 278 | 517 |
| 5,000 | 357 | 880 |
| 10,000 | 370 | 965 |
| 100,000 | 383 | 1,056 |
| 1,000,000 | 384 | 1,066 |
Above a few tens of thousands the population barely changes the answer. That is why national polls and a survey of one large city can use similar sample sizes.
Why 50% is the safe choice
The product p × (1 − p) is 0.25 at p = 50% and smaller for every other value, so 50% gives the largest sample. Penn State's lesson calls this the conservative sample size. If an earlier survey or a pilot found a share far from 50%, entering it lowers the sample, at the risk that a new result closer to 50% comes with a wider margin than planned. At 95% and ±5 points:
| Expected proportion | Responses |
|---|---|
| 50% | 385 |
| 30% | 323 |
| 20% | 246 |
| 10% | 139 |
| 5% | 73 |
A proportion of 10% needs the same sample as 90%, because p × (1 − p) is symmetric.
Sample size for an A/B test
An A/B test compares the conversion rates of two variants. Its sample size needs the baseline rate, the smallest relative lift worth detecting, a confidence level and a power, the chance of detecting a real lift of that size (80% is the usual default). The A/B mode uses the same function as the A/B test significance calculator, so both give the same number:
Here p₁ is the baseline rate, p₂ the rate after the lift, p̄ their average, zα the critical value for the confidence level (one- or two-sided) and zβ the one for the power. It is built like the one-sample formula in the NIST/SEMATECH e-Handbook, section 7.2.4.2: the confidence term uses the spread with no difference, the power term the spread with the difference. At a 5% baseline, a 10% relative lift, 95% confidence and 80% power you need 31,234 visitors per variant, 62,468 in total.
What the formula assumes
- A random sample. The formulas describe simple random sampling, where every member of the population has the same chance to be picked. An opt-in poll on a website is not random, and a large sample does not fix that.
- One question at a time. The margin of error applies to the whole sample. A subgroup, say only customers under 30, is a smaller sample with a wider margin. Plan the size for the smallest group you want to report on.
- Usable answers. The sample size counts completed responses. Use the response rate field to see how many people to invite, and the z-score calculator to look up a critical value for any other confidence level.
How to use the sample size calculator
- 1Choose what to find: the sample size for a survey, the margin of error of a sample you already have, or the visitors for an A/B test.
- 2Set the confidence level. 95% is the common choice.
- 3For a survey, enter the margin of error, keep 50% as the proportion unless you have an earlier estimate, and add the population size if it is small. For an A/B test, enter the baseline conversion rate and the lift to detect.
- 4Read the result, add an expected response rate to see how many people to invite, and copy the result or a link with your numbers.
Sources
- Penn State STAT 506, Lesson 2: Confidence Intervals and Sample Size, section 2.3, sample size for estimating a proportion with and without the finite population correction, and the approval rating example
- NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.4.1 Confidence intervals, the normal-approximation interval for a proportion
- NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.4.2 Sample sizes required, the sample size for a test about a proportion with a given power
Frequently asked questions
What sample size do I need for a survey?
For a large population, a 95% confidence level and a margin of error of ±5 percentage points you need 385 responses, using the safe assumption of a 50% proportion. For ±3 points it is 1,068. A smaller population needs fewer: 323 for a population of 2,000 at ±5 points.
What is the margin of error?
It is the plus or minus range around a survey result. If 60% of your sample says yes and the margin of error is ±5 points at 95% confidence, the true share in the population is likely between 55% and 65%. "Likely" means that 95% of samples drawn the same way would give an interval that contains the true value.
Why does the calculator use 50% as the proportion?
The sample size grows with p × (1 − p), and that product is largest at p = 50%. If you do not know the answer in advance, 50% gives a sample that is big enough for any result. If an earlier survey found about 20%, entering 20% gives a smaller sample, but the margin of error is only met if the new result is close to that guess.
Does the population size matter?
Only when the sample is a noticeable share of the population. At 95% and ±5 points, a population of 100,000 needs 383 responses and an unlimited one 385. For a population of 2,000 the finite population correction brings it down to 323. Leave the field empty when the population is very large or unknown.
Which confidence level should I use?
95% is the most common choice. 90% needs a smaller sample but gives a higher chance that the true value lies outside your margin; 99% needs a larger sample. Choose the level before you collect data and report it together with the margin of error.
How do I account for people who do not respond?
Enter your expected response rate. The calculator divides the sample size by it to show how many people to invite: for 385 responses at a 20% response rate, invite 1,925. Keep in mind that people who answer can differ from those who do not, which no sample size fixes.
How is the sample size for an A/B test different?
An A/B test compares two conversion rates, so it needs a minimum detectable lift and a power as well as a confidence level. At a 5% baseline rate, detecting a 10% relative lift with 95% confidence and 80% power takes 31,234 visitors per variant. The A/B mode here uses the same formula as the A/B test significance calculator.