Skip to content

guide

Weighting an audience survey to the population

Weighting corrects who answered. It costs precision, it cannot find who you missed, and every step should be written down.

Takwimu editorial team, Editorial team8 min read

To weight a survey, you compare your sample with known population totals, give each respondent a weight equal to their group's share of the population divided by its share of the sample, and report the result with the precision the weighting cost.

Key points:

  • A weight is population share divided by sample share.
  • Weight only on variables with a trusted total. In Kenya that means KNBS census figures, in practice at county level.
  • Which variables you choose matters more than which statistical method you use.
  • Weighting always costs precision. Report the effective sample.
  • Weighting cannot repair a self-selected sample.

What does weighting a survey do?

Weighting makes a sample look like its population on the variables you can check. If women are 50% of the population and 40% of your interviews, each woman counts for a little more and each man for a little less.

No sample comes back perfect. Interviewers find some people more easily than others, and young men are often out when the call comes. Weights correct that imbalance after the fact.

How do you calculate a survey weight?

Divide the population share by the sample share. Here is an invented example with 1,000 interviews, 600 with men and 400 with women, in a population split evenly.

Group Population share Sample share Weight
Men 50% 60% 0.50 ÷ 0.60 = 0.83
Women 50% 40% 0.50 ÷ 0.40 = 1.25

Check the sum: 600 × 0.83 + 400 × 1.25 comes to about 1,000. Weighted totals should match the sample size.

Now see the effect. Say station reach is 40% among men and 25% among women. Unweighted, reach is 34%, because men are over-represented. Weighted, it is 32.5%.

How do you weight by county?

The same sum works across places, and this is where weighting does the most for a regional station. This table uses 2019 census totals with an equal 400 interviews in each county. The reach column is invented.

County 2019 population Share Sample share Weight Reach
Nairobi City 4,397,073 38.8% 20% 1.94 10%
Kiambu 2,417,735 21.3% 20% 1.07 30%
Nakuru 2,162,202 19.1% 20% 0.95 40%
Mombasa 1,208,333 10.7% 20% 0.53 5%
Kisumu 1,155,574 10.2% 20% 0.51 5%

The plain average of the reach column is 18.0%. Weighted by population it is 18.9%. The gap could run either way, and in a report it could decide a ranking.

For a real wave, use adult totals. Those census figures cover all ages, and your survey does not.

Where do the population totals come from?

Use KNBS. The 2019 census counted 47,564,296 people: 23,548,056 male, 24,014,716 female and 1,524 intersex, with county and sub-county tables. Its reference point was the night of 24th/25th August 2019.

The question is how low you can go. Census totals exist for small areas, but your sample does not. County is the practical floor for most audience work, because that is where a sample of a few hundred meets a published total.

A target is only as useful as the interviews beneath it. A weighting cell with five people in it gives a weight that swings the whole result.

What is raking?

Raking adjusts to several variables at once when you know each total separately. Pew Research Center describes it as adjusting one variable, then the next, then repeating, because correcting one can push another out of line. The cycle runs until the sample lines up with the population on all of them.

Its appeal is that it only requires the marginal proportions. You need the county totals, the sex split and the age split, and you do not need a table of every combination. Most audience surveys should start here.

Run it like this:

  1. Weight to county.
  2. Adjust those weights to sex.
  3. Adjust those to age group.
  4. Repeat until the totals stop moving.

Which variables should you weight on?

Choose variables linked to both listening and to who answers. Pew found that choosing the right variables matters more than choosing the right statistical method: adding political variables to demographics cut bias by a further 1.4 points, while complex methods added no more than 0.3 points beyond raking.

For Kenyan audience work the natural set is county, sex, age and urban or rural. Each has a census total and each shapes media use. Add education where you have a total for the same population.

Be careful with LSM. It is common in media reporting, and a weight needs a population total you can defend. Never weight on station listening itself, since that is the thing you came to measure.

What is the design effect?

Weighting gives some people more say and so reduces the information in the sample. The design effect measures that cost. The usual approximation is n × Σw² ÷ (Σw)², where w is each person's weight and n the number of interviews.

For the men and women above, Σw is 1,000 and Σw² is about 1,042. The design effect is 1.04, for an effective sample of 960. That is a cheap correction.

The county table costs more, at about 1.27, and turns 2,000 interviews into about 1,576. Pew says pollsters should account for this in the margin they report. Our page on sample size for a media survey shows what it does to the margin.

How large should a weight get?

Set a cap before you look at results. A person with a weight of 8 is standing in for eight others, and if that person happens to love your station, your figures jump.

The real fix is in the field. If young men are short, call back and recruit more of them while the survey is open. A weight should finish the job that quotas and callbacks began.

Look at the distribution of weights in each wave. A long tail tells you where fieldwork failed.

Can weighting fix a bad sample?

Weighting fixes imbalance among people you reached. It has little power over people who were never in the frame or who chose themselves.

Pew's test of online opt-in samples is the sobering evidence. Even the most effective procedures could not remove most of the bias: no procedure brought average bias below 6 points, from 8.4 unweighted, which removes about 30%.

The same held in Nigeria, where weighting SMS and IVR data for demographics did not reduce bias. So do not weight a text-in to the census and call it reach. Replies are self-selected, as our page on on-air responses and reach sets out.

The mode you use decides how much repair you will need. See phone survey vs face to face.

What do the standards ask for?

They ask you to show your working. The Media Rating Council's minimum standards say weighting must rest on systematic, logical procedures, consistently applied and defensible by empirical analysis.

Reports must state whether weighting was used, and the service must disclose its procedures and the reasons for them. Good reasons include a known imbalance. Bad reasons include a result the sales team disliked.

Should you show unweighted numbers?

Show both, in the right place. Percentages should be weighted, and the base beneath each should be the real number of interviews.

A weighted base of 400 built on 90 real interviews misleads a reader about precision. Print the unweighted count beside the margin. That one habit prevents most overreading of small groups in the audience profile.

How do you keep weights consistent between waves?

Use the same targets, variables and cap in every wave. A change of target can look like a change in audience, so record each one.

If you adopt better totals, rerun past waves on them and publish both series for one wave. Otherwise your trend will have a step in it that nobody can explain, and share of audience comparisons will suffer.

What do practitioners get wrong?

These errors come up again and again:

  • Weighting to all-age census totals when the survey covers adults.
  • Weighting on a variable with no published total.
  • Weighting tiny cells, which gives one person huge influence.
  • Changing targets after seeing results.
  • Quoting the margin from the raw sample size.
  • Weighting self-selected replies and calling the result representative.

The audience research report page shows how to set out your weighting in a wave report.

Where TI fits

Weighting should stop where trusted population totals stop. Takwimu Intelligence (TI) weights answers to KNBS census totals at the level where those totals exist, in practice the county, and each figure shows its margin of error. Replies to an on-air call stay out of that work, because they are self-selected.

To see it on your own station or title, book a walk-through.

Takwimu Intelligence (TI) is audience measurement software for Kenyan media.

Sources

  1. 1

    2019 Kenya Population and Housing Census, Volume I: Population by County and Sub-County · Kenya National Bureau of Statistics, November 2019National and county totals by sex. Opened 5 November 2026.

  2. 2

    How different weighting methods work · Pew Research Center, 26 January 2018How raking works. Opened 5 November 2026.

  3. 3

    For Weighting Online Opt-In Samples, What Matters Most? · Pew Research Center, 26 January 2018Variables against methods, remaining bias, and 8,000 against 2,000 interviews. Opened 5 November 2026.

  4. 4

    5 key things to know about the margin of error in election polls · Pew Research Center, 8 September 2016Margins, subgroups, changes between polls and design effect. Opened 5 November 2026.

  5. 5

    In Search of the Optimal Mode for Mobile Phone Surveys in Developing Countries · Lau, Cronberg, Marks and Amaya, Survey Research Methods 13(3), 2019Response, bias and relative cost of CATI, IVR, SMS and face-to-face in Nigeria. Abstract read. Opened 5 November 2026.

  6. 6

    Minimum Standards for Media Rating Research · Media Rating Council, December 2011Ethical, operational and disclosure standards for rating services. Opened 5 November 2026.

More from the library