Phone survey vs face to face comes down to who you cannot afford to miss: face-to-face reaches people without a phone and gives the stronger benchmark, while phone is quicker and cheaper to repeat. A sound audience programme uses the first to set the base and the second to track it.
Key points:
- In 2023/24, 53.7% of people aged 3 and above owned a mobile phone, against 93.8% of households.
- Phone respondents give more shortcut answers and more socially desirable ones than face-to-face respondents.
- Questions about yesterday hold up better across modes than opinion questions.
- SMS, voice and web surveys cost less and reach a narrower slice of the population.
- Changing mode between waves can produce a jump that has nothing to do with the audience.
Who can a phone survey reach in Kenya?
A phone survey reaches phone owners and those who share one. KNBS and the Communications Authority's analysis of the 2023/24 Kenya Housing Survey gives the clearest picture, for people aged 3 and above.
| Group | Owned a mobile phone |
|---|---|
| All people | 53.7% |
| Urban | 64.6% |
| Rural | 48.6% |
| Male | 54.5% |
| Female | 52.9% |
Ownership rose from 47.3% in the 2019 census. Use stood at 64.9%, with 11.3% using a phone they did not own. At household level, 93.8% had a mobile phone.
Read the household and individual lines together. Nearly every home can be reached, but not everyone in it. When a shared phone rings, the owner tends to answer, so ask for the person you selected and call back if they are out.
What does face-to-face do that phone cannot?
Face-to-face starts from places. You can draw areas, then households, then a person, and nobody needs a phone or a place on a list.
Afrobarometer's design shows the method: primary sampling units are chosen at random, then start points, then households, and interviewers select one person at random and alternate between men and women.
Response is the other gain. In a 2019 study in Nigeria, face-to-face reached a 99% response rate against 15% for CATI. An interviewer can also show a card of station logos, which a phone call cannot do.
What does phone do better?
Phone wins on speed and spread. A call centre can cover 47 counties in a week with no travel, which matters for a daily or weekly tracker.
Supervision is simpler too. Calls run from one room and a supervisor can listen in. Clusters disappear, as each call can go anywhere.
This is why the KARF tracking survey used computer-aided telephone interviews for daily data. Note what came first, though: panellists were recruited face to face.
Does the quality of answers differ?
The mode changes how people answer. Holbrook, Green and Krosnick tested this on three national experiments in the US, with long questionnaires.
Telephone respondents were more likely to take shortcuts, shown by more no-opinion answers, less variation across items and more agreement. They were less engaged, more suspicious of the interview and more likely to give socially desirable answers, even though their interviews were shorter.
Not every question suffers equally. Pew Research Center's test of phone against web found an average gap of 5.5 points on 60 questions, ranging from 0 to 18. Activities "yesterday" showed little or no difference, while ratings and satisfaction showed the most. People were much more likely to say they were very satisfied with family life on the phone than on the web: 62% against 44%.
For a station, that sorts the questionnaire. Recall of yesterday's listening is the sturdy part. Appreciation scores are the part that moves with the mode.
How do SMS, IVR and web compare?
These cost less and reach fewer kinds of people. The Nigerian study compared them directly, and its response rates were CATI 15%, IVR 3% and SMS 0.2%.
All the mobile modes under-represented women, older people, less educated people and rural residents. Weighting SMS and IVR data for demographics did not reduce bias. For a survey of 3,000 completes, IVR cost 43% of CATI and SMS 24%.
Those are Nigerian figures from one study. Treat them as a guide. Use these modes for short questions, for people you already know how to reach, or as a panel with known make-up.
GeoPoll describes recruiting nationally representative panellists for regular surveys sent through several mobile channels. A panel like that differs from an open call on air, where people choose themselves. Replies to an on-air call are feedback, and on-air responses do not measure reach.
Where does a phone sample come from?
The source of the numbers decides what you can claim. There are three common sources:
- Random numbers generated from valid prefixes, which reach anyone with a working line.
- A panel recruited by a probability method, as KARF described.
- A list your station already holds, such as competition entries or a call log.
The third is a list of fans. It is useful for programme feedback and useless for reach. It also carries legal limits, since those people gave you their numbers for another purpose, as our page on data protection for surveys in Kenya explains.
Can you combine phone and face-to-face?
You can, and the strongest designs do. Recruit in person, then interview by phone, which brings the reach of one and the speed of the other.
Another pattern is one large face-to-face benchmark a year, with phone waves between. The benchmark also gives you a fresh picture of who phone misses, which feeds your survey weighting.
Report each mode's share of interviews. A buyer should be able to see the mix.
What happens if you change mode between waves?
A change of mode can look like a change in audience. Here is an invented example: reach of 24% in a face-to-face wave, then 29% in the next wave by phone.
Your sales team will call that growth. It may be the mode. The phone sample may hold more urban and younger people, and the two modes may draw out different answers.
If you must switch, build a bridge. Run both modes side by side for one wave and measure the gap. Then report the old series and the new one as two lines, with the bridge wave shown.
How does mode change a prompted list?
A list works differently by ear and by eye. Pew reports that telephone respondents more often choose items heard later in a list, and people completing a survey themselves tend to choose items at the top.
On the phone, keep lists short and rotate them, and filter by county first so that nobody hears thirty names. Face-to-face, a show card lets people scan, so more names are workable. Our page on prompted recall has the detail.
Which should a Kenyan station choose?
Choose by the decision and the area. This table is a starting point.
| Situation | Sensible mode |
|---|---|
| First wave, no benchmark | Face-to-face, or a phone sample checked against one |
| Regular tracking in urban counties | Phone |
| Rural area with low phone ownership | Face-to-face |
| Community radio station in a small footprint | Face-to-face, in clusters |
| Quick reaction to a new programme | Phone or an existing panel |
| Listener feedback from an on-air call | Any channel, labelled as self-selected |
What do buyers get wrong?
Buyers often compare prices per interview and stop there. A cheap interview that misses half the population costs more than it saves.
Ask suppliers four questions. Who was left out, and how many? What was the response rate? What was the mode of each interview? Which answers were read from a card? A supplier listed under who measures media audiences in Kenya should answer these readily.
Read next
- Audience measurement software for Kenyan media: the TI pack page for this work
- How to run an audience survey in Kenya: all the pages in this section
- How to run an audience survey in Kenya
- Sample size for an audience survey
- Weighting an audience survey
- Takwimu Intelligence (TI) home page
- Audience measurement in Kenya: the guide to this library
- The TI platform and how we check our numbers
Where TI fits
However a study is collected, a reader should be able to tell who was reached and who was left out. Takwimu Intelligence (TI) weights answers to KNBS census totals at county level and shows a margin of error with each figure. A study can also be announced on air, with replies by USSD, SMS, voice or web through the outlet's own gateway account and a code per station, and those replies are self-selected, so they never measure reach.
To see it on your own station or title, book a walk-through.