Portable People Meter
EXCLUSIVE UPDATE REPORT FROM HARRY VALENTINE, EVP
Arbitron hosted a special Consultant's Fly-In at their Columbia, MD headquarters to update the status of their Portable People Meter testing now in progress in the Philadelphia DMA. This is the last major phase of the PPM research program. Earlier PPM tests were conducted in Manchester, England and Wilmington, DE.
If you're not already familiar with the Portable People Meter (PPM), it is a pager-size device worn by radio survey participants that measures their exposure to encoded signals. PPM audience measurement is not limited to radio. The PPM can measure usage of any medium with an audio signal including television, cable TV, Internet streaming and satellite broadcasts. The only caveat is that in order to be measured, a signal must include special inaudible encoding.
The Portable People Meter solves problems with diary-based measurement that have bugged broadcasters for years. Diaries measure recall, biasing the results toward top-of-mind stations. The PPM measures actual exposure and captures instances of listenership that the diary misses. So the PPM uncovers a bigger cume for radio. With diaries, only a small number of respondents are measured during any given week. The PPM measures the complete sample every week. This adds stability to the results and allows broadcasters the ability to check ratings for specific one-time events. In short, the PPM gives a much more accurate and believable picture of people's radio usage.
Here are some highlights from this week's Arbitron PPM Consultant Fly-In:
How the PPM Works
Broadcasters encode their signals with codes unique to each station or media outlet. The Portable People Meter, worn by daily respondents, picks up that code wherever the participant is exposed to an encoded signal that would be audible to the human ear. Every night, participants dock their PPM units in PPM docking station where the media exposure data is uploaded to Arbitron and the PPM unit is recharged for the next day.
The battery in the PPM is good for 27 hours and the unit will hold a week's worth of data. The PPM is resilient; the medium from which it is recording exposure doesn't matter as long as the signal is encoded and the programming is audible to the participant. The PPM has a motion detector to let Arbitron know if the participant is wearing the unit. According to one Arbitron representative, the motion detector is sensitive enough to detect breathing. Arbitron reports the equipment to be over 99 percent problem free.
PPM Sample Design
One of the big differences between diary and Portable People Meter audience measurement is that the PPM uses a panel sample. With diaries, the sample changes every week. With a panel, the same participants continue to be measured week-to-week over a period of time. With diaries, only roughly one-twelfth of the quarterly sample participants are measured on any given week. In contrast, the panel measures all quarterly respondents every week. With PPM measurement, the need to extrapolate has been completely eliminated.
To recruit the PPM panel, Arbitron starts the same way they do to recruit diary holders; with a random sample of telephone numbers. Panel sampling is two-phased; Arbitron conducts a telephone enumeration study to discover household characteristics including county, household size, presence of children, employment status, race or ethnic background, language preference, number of television sets and the presence of cable or satellite TV in the household.
Using household characteristics uncovered in the enumeration study, Arbitron stratifies the sample to keep it in line with the general population. The proportion of households with a certain characteristic in the sample is roughly the same as the proportion of households with that characteristic in the universe. So panel methodology solves the problem of getting a "funny" sample from time-to-time.
When PPM measurement is ongoing, Arbitron will maintain a pool of enumerated respondents to replace panelists who churn. This will enable Arbitron to carefully control the panel makeup to be a socio-economic mirror of the population at all times.
The PPM radio data is this study is based on 47 encoded Philadelphia stations. This represents around 90 percent of radio listening in the market. Comparisons were made between the Spring Phase II Philadelphia Arbitron (March-April-May) for the diary and April 25 through May 22 for the PPM. Note that the time span had to be three months for the diary compared to only one month for the PPM in order to have comparable samples.
The total of in-tab diaries for the 12 weeks covered by the old methodology was 4,644 compared to 1,500 installed panelists in the DMA and 950 installed panelists in the Metro for the PPM. Arbitron stressed that the 1,500 was only the test panel size; their goal for ongoing measurement is a larger panel. Nevertheless, the smaller panel consistently yielded more stable results.
One reason the panel is so much better becomes evident when you do the math. 4,600 diary-keepers divided by twelve weeks in the survey results in an average number of diary-keepers of 387 per week. In contrast, with an average daily in-tab of 663 panelists, the PPM delivered more sample size in one day that the diary did in an entire week! With that sample size, and remember that Arbitron's goal is to make it bigger, it will be possible to isolate audience levels of one-time events on a station.
To illustrate the value of a big daily sample, a graph of WIP's listenership was shown for the Monday and Tuesday following the Super Bowl. The audience of the sports-formatted station had a huge spike on those days. Information like that is obviously useful to programming and can also help sales get premium rates for special events that have been proven to increase listening levels.
The PPM is different methodology and will obviously yield different results than diary measurement. Here are some examples of how the PPM and diary results compare:
PPM closely track diaries in terms of total radio audience ratings. Where they vary, the PPM more often shows slightly higher ratings than the diary, especially, as illustrated by this chart, in middle demos in greater demand by advertisers. All ratings are for Monday through Sunday, 6:00 AM to 12:00 Midnight:
Market AQH Ratings
The dramatically higher cumes the PPM shows can put radio in a much better neighborhood with mass media like television and newspaper. We have always maintained that radio needs to start selling the big number. The PPM certainly gives radio the tools. Arbitron compared the cume ratings of Philadelphia radio clusters with the television cume ratings of the bread-and-butter 6:00 PM and 11:00 PM local TV newscasts. The results speak for themselves.
Another point about cume, according to Arbitron, is that the PPM shows it to grow over time. For example, WBEB's weekly cume shown by the PPM is over 50 percent of the market. The PPM revealed a monthly cume for the station of around 70 percent.
PPM Panel Maintenance
Arbitron did a good job demonstrating the advantages of panel methodology over our old friend, the diary. Beyond the advantages already outlined in this update, it needs to be pointed out that Arbitron has a comprehensive program in place for panel maintenance. This program includes:
One of the biggest concerns broadcasters have expressed with respect to panel methodology is slow turnover of respondents. Arbitron's present plan is to keep a panelist for up to two years. Television wants a two-year period so they can look at year-to-year trends while radio feedback has been for a shorter term. Arbitron is considering a shorter maximum term for panelists, perhaps a year or 18 months. We believe this issue to be moot and of no concern for two reasons:
In practice, panel compliance has been good and consistent over five different studies; by sex, age, race, and over time. Panelists carry their PPM units an average of over 15 hours a day, undocking them early and docking them late. Average daily in-tab runs about 80 percent of the installed panel. Most of the undocked time, PPM's are being carried; 15:10 out of 15:45 daily. Males and females comply equally well. There is no evidence of fatigue.
According to Arbitron, the Wilmington, DE study found PPM ratings to be about three times more reliable that diary estimates overall. That is to say that 1,000 PPM panelists in-tab has the same statistical power as a quarterly in-tab diary sample of 3,000.
The reason for this has to do with the longer time frame panelists are in place. In a twelve-week rating period, one diary holder yields seven days of behavior observations while one PPM panelist yields 84 days of behavior observations. The more days that are measured, the better picture you get of a respondent's behavior. Graph after graph of week-by-week data consistently demonstrated flatter results from PPM measurement while diaries exhibited their familiar heart-attack fluctuations. Arbitron plans to repeat this study for the full Philadelphia test but we believe they will find a similar result.
Another benefit of panel measurement is that it better accounts for changes in behavior than the diary. Since a panel is made up of the substantially the same people over time, a change in the result is more likely to indicate a change in behavior. With diary measurement you can never really be sure if a change in result was due to a change in behavior or a change in the sample.
What does this mean for the ultimate panel sample sizes? Arbitron can't say because the testing is not complete, but here's what we think. Arbitron could cut present diary samples by two-thirds and have the same statistical reliability, but they consistently say they want a bigger sample than that. But financial pressure (PPM hardware costs more than diaries) will probably result in PPM samples being reduced from the present diary sample sizes. Look for the end result to be somewhere in the middle.
Don't Get Freaked Out About Time-Spent Listening
"It's important not to get freaked out about it." That's what Bob Michaels, VP of Arbitron Radio Programming Services, said about the PPM time-spent-listening results. We wholeheartedly agree. Cume is the most important number and the PPM shows an abundance of it. And as long as a station has cume, there are techniques that can be used to increase TSL and AQH.
Total time-spent-listening (TSL) to radio was pretty much the same with the PPM as with the diary. But while cume has doubled, the new listeners revealed by the PPM are more casual, driving down TSL for individual stations. The net of all this is a slight increase in average quarter-hour.
There are three obvious reasons for reduced individual station TSL. First, the "new" listeners revealed by PPM measurement are casual listeners that don't listen to the station as much. When their lower TSL is factored in, it reduces the overall TSL.
Second, PPM measurement eliminates rounding that occurred in diary entries. Anyone who has reviewed diaries has seen the pattern where a listener would write down a start time and draw an arrow down through the day. The PPM only records when the listener is exposed to the encoded signal.
Finally, the PPM reveals listeners listening to more stations. According to Bob Michaels, in May the average PPM panelist listened to ten different stations. So individual listener TSL to radio is spread thinner.
There is one other interesting thing about TSL that broadcasters will have to get used to; new terms for it. The new terms are "Average Time Exposed" (ATE) for daily TSL, and "Average Weekly Time Exposed" (AWTE) for weekly TSL.
When People Begin Listening to Radio
Diaries have consistently shown more tune-in in the first and third quarter-hours and less in the second and fourth quarter-hours. Specifically, Arbitron reports that diaries show 48 percent of tune-in in the first quarter-hour, 13 percent in the second quarter-hour, 28 percent of tune-in in the third quarter-hour and 11 percent in the last quarter-hour.
The PPM shows a much more uniform pattern of tune in of 27 percent, 24 percent, 25 percent and 24 percent in the first through fourth quarter-hours respectively. Since PPM measures actual exposure rather than recall, it now appears that variable tune-in rates by quarter-hour may be more related to diary methodology than to listenership.
One of the most exciting aspects of the Portable People Meter is the multi-media data it will generate. According to Arbitron, 11 broadcast TV stations are now encoding for the Philadelphia market test, three of which are newly encoding with phase two of the study. While multi-media data was not the main focus of this week's consultant fly-in, PPM measurement will allow stations for the first time to learn the consumption habits of their precise panel members in other media.
Arbitron has said all along that the expense of PPM measurement would require participation by other media, specifically television and/or cable. To that extent, Arbitron says that Nielsen Media Research has an option to participate in a joint venture for the Portable People Meter. Nielsen is contributing to the expertise and funding of a study and negotiations have started. Arbitron expects to have a decision on the joint venture is expected by the end of this year.
If there is no joint venture with Nielsen, Arbitron will accelerate efforts on other PPM initiatives internationally and for other PPM applications. They would also look ways to reduce costs of the PPM. However, under this scenario, they say that the PPM would not be launched in the foreseeable future.
If there is a joint venture between Arbitron and Nielsen, they would likely form a jointly owned company with the PPM as its sole focus. Data from this company would be fed back to the parent companies, which would in turn service their own clients. The source for the multi media data the PPM will generate is yet to be negotiated.
Assuming that the joint venture happens between Arbitron and Nielsen Media Research and that there is customer support, Arbitron is tentatively planning to launch Philadelphia by mid-2003, an additional four top 10 markets in the next 12 to 18 months and ultimately the top 100 DMAs by the end of 2008. The top 100 DMAs cover 170 radio markets.