The sample size for a media survey is the number of interviews that makes your smallest reported group precise enough for the decision: about 400 gives ±4.9 points and about 1,000 gives ±3.1 points, at 95% confidence and before weighting.
Key points:
- Margin of error depends on the number of interviews and hardly at all on the size of the population.
- Set the sample from your smallest reported group, since each group has its own margin.
- Weighting and clustering widen the margin. Quote it after both.
- A three-point rise between waves of 1,000 sits inside the noise.
- A larger sample shrinks random error only. It does nothing for a biased sample.
How do you calculate sample size for an audience survey?
The margin of error at 95% confidence is 1.96 × √(p × (1 - p) / n). Here p is the result as a proportion and n is the number of interviews.
Take 400 interviews and a result of 50%. The sum inside the root is 0.25 ÷ 400 = 0.000625, whose square root is 0.025. Multiply by 1.96 and you get 0.049, or ±4.9 points.
To go the other way, n = 1.96² × p × (1 - p) ÷ m², where m is the margin you want. For ±5 points at 50% that is 3.8416 × 0.25 ÷ 0.0025, or about 384 interviews. Pew Research Center gives the same logic at a larger scale: a simple random sample of 1,067 yields about ±3 points.
What margin of error does each sample give?
The table shows margins in percentage points at 95% confidence, for a simple random sample before weighting.
| Interviews | Result near 50% | Result near 20% | Result near 10% |
|---|---|---|---|
| 100 | ±9.8 | ±7.8 | ±5.9 |
| 200 | ±6.9 | ±5.5 | ±4.2 |
| 400 | ±4.9 | ±3.9 | ±2.9 |
| 600 | ±4.0 | ±3.2 | ±2.4 |
| 1,000 | ±3.1 | ±2.5 | ±1.9 |
| 1,500 | ±2.5 | ±2.0 | ±1.5 |
| 2,000 | ±2.2 | ±1.8 | ±1.3 |
| 3,000 | ±1.8 | ±1.4 | ±1.1 |
Precision gets expensive fast. Halving the margin takes four times the interviews, so moving from ±4.9 to about ±2.5 means going from 400 to about 1,500.
Why does population size hardly matter?
A county of two million needs about the same sample as a town of 100,000. With 400 interviews, the margin near 50% is ±4.9 in both.
The correction only bites when you interview a large slice of the population. In a place of 5,000 people, 400 interviews give ±4.7. That is a small gain for sampling one person in twelve.
So a small county deserves as many interviews as a large one if you will report it on its own. A big footprint does not need a big sample for one overall figure.
What should set the number of interviews?
Your smallest reported group sets the number. Each group has its own margin, and Pew notes that subgroup margins are larger, sometimes much larger.
Here is an invented example. In a sample of 1,000, the 15% aged 18 to 24 come to 150 interviews, and a result near 50% in that group carries about ±8.0 points. A breakfast decision aimed at young listeners cannot rest on that.
List the groups first, then multiply. As a working rule, aim for about 400 in any group you will make a decision on, and do not print a percentage from fewer than 100. The Media Rating Council's standards ask a service that sets such minimums to state them.
How much does weighting cost?
Weighting improves balance and widens the margin. Pew calls this increase in variability the design effect and says pollsters should account for it.
Divide your sample by the design effect to get the effective sample. A survey of 1,000 with a design effect of 1.5 behaves like 667 interviews, so ±3.1 becomes about ±3.8. A design effect of 2 behaves like 500 interviews, or about ±4.4.
Ask suppliers for the weighted margin. Our page on how to weight a survey shows how to calculate it.
How much does clustering cost?
Interviewing in clusters saves travel and widens the margin, because neighbours resemble each other. A sample of 20 interviews on one street holds less information than 20 across a county.
Afrobarometer clusters eight interviews in each primary sampling unit to control cost. Its samples usually include 1,200 or 2,400 cases, and it states margins of ±2.8 and ±2.0 points at 95% confidence.
Keep clusters small and numerous. Ten interviews in each of 100 places beat 50 interviews in each of 20.
How many interviews for county results?
Give each county you will report a usable sample, then weight back for the combined figure. Here is a worked case using county totals from the 2019 census. They are all-age totals, used here only to show the arithmetic.
| County | 2019 population | Share of five counties | Interviews | Weight |
|---|---|---|---|---|
| Nairobi City | 4,397,073 | 38.8% | 400 | 1.94 |
| Kiambu | 2,417,735 | 21.3% | 400 | 1.07 |
| Nakuru | 2,162,202 | 19.1% | 400 | 0.95 |
| Mombasa | 1,208,333 | 10.7% | 400 | 0.53 |
| Kisumu | 1,155,574 | 10.2% | 400 | 0.51 |
Each county gets ±4.9 points on its own. Combined, 2,000 interviews carry a design effect of about 1.27 from these weights, which leaves an effective sample of about 1,576 and a margin of about ±2.5 points. Equal allocation buys county results at a small cost to the total.
How big a change between waves is real?
The margin on a change is wider than the margin on either figure. Pew's worked case shows how quickly single-poll movements turn out to be noise.
Here is the sum, with invented figures. Reach rises from 20% to 23% between two waves of 1,000. The margin on that change is 1.96 × √(0.2 × 0.8 ÷ 1,000 + 0.23 × 0.77 ÷ 1,000), or about ±3.6 points.
A three-point rise is inside that. At 2,000 per wave the margin falls to about ±2.5, and the rise is clear. The break-even is about 1,440 per wave.
So decide what size of change matters before you buy the sample. Then say "no clear change" in the report when that is the finding.
How close can two stations be?
Two stations in one survey share a sample, and a lead has its own margin. Pew puts the margin on a lead at about twice that of a single figure when two contenders hold most of the sample. With many stations the sum differs, but the lesson holds.
Say Station A has 21% and Station B has 18% in a sample of 1,000, in an invented case. The margin on that three-point gap is about ±3.9 points. That ranking is unproven. The same goes for many entries in any list of top radio stations and for small gaps in share of audience.
What samples do the big systems use?
Compare with care, since the populations and methods differ. These are each body's own published figures.
| System | Stated sample | Where |
|---|---|---|
| RAJAR | Over 22,000 adults per quarter | UK |
| Nielsen Audio | About 1,000 to 6,000 people per market | US |
| KARF tracking survey for the CA, 2019 | Target of 3,000 daily, aged 15 and over | Kenya |
| Afrobarometer | 1,200 or 2,400 per country, not a media survey | Africa |
A station does not need a national currency's sample. It needs enough for its own area and its own decision.
Does a bigger sample fix a biased one?
A big sample does not cure bias. Pew tested online opt-in samples and found virtually no difference in accuracy between 8,000 interviews and 2,000: the leading procedure at 8,000 was only 0.2 points better.
The modelled margin fell from ±1.8 to ±0.5 points while average bias stayed at 6.3 points. As Pew puts it, estimates become more tightly clustered, often around the wrong value.
This is why 20,000 text replies to a competition tell you little about reach. The count is impressive, and the people chose themselves. See on-air responses and reach.
What sample should a small station use?
Sample your footprint and report a range. For a community radio station, about 400 interviews in the area it serves is a workable start.
Small audiences need care. With 400 interviews, an invented reach of 10% carries about ±2.9 points, or roughly 7% to 13%. A reach of 5% carries about ±2.1 points, or roughly 3% to 7%.
Say it that way. A range protects you when a later wave lands elsewhere inside it.
What are the common mistakes?
Most sample errors repeat:
- Quoting the margin for the whole sample beside a subgroup figure.
- Quoting a margin for the result near 50% on every line, or leaving it out.
- Ignoring the design effect after weighting.
- Calling a move a trend when it sits inside the margin on the change.
- Buying a larger sample in place of a better one.
The audience research report page shows how to present all this on the page, and day after recall shows how to spread interviews across the week.
Read next
- Audience measurement software for Kenyan media: the TI pack page for this work
- How to run an audience survey in Kenya: all the pages in this section
- How to weight a survey
- How to report an audience wave
- Audience metrics explained
- Takwimu Intelligence (TI) home page
- Audience measurement in Kenya: the guide to this library
- The TI platform and how we check our numbers
Where TI fits
The margin belongs on the page beside the number, where the buyer can see what the sample can bear. Takwimu Intelligence (TI) weights answers to KNBS census totals at county level, applies cross-tabs, and shows a margin of error with each figure.
To see it on your own station or title, book a walk-through.