Station to Stationpanel is accepted into production. The Québec PPM panel is a French- ... The standard measurement instrument for radio in Canada is a personal diary. The diary is

© Copyright by ESOMAR® / The ARF

The authors report findings from a comprehensive analysis of radioaudience data captured by Arbitron’s Portable People Meter (PPM). Thepaper compares the diary and PPM results for a common area and timeperiod and list of stations. Comparisons, not unexpectedly, showdifferences in overall amount of radio use, the distribution of radio use bytime periods, the relative importance of radio listening for differentsubgroups of the population, the reach of radio, and shares and reach ofthe different stations. There are also indications that the structure ofradio listening (e.g., relative weight of heavy-light listening, demographicprofiles, etc.) is different. The reasons for these substantive differences areexplored. It becomes evident that the tasks of writing a daily diary andcarrying a PPM attract different populations, as measured by standarddemographics, socio-economic and lifestyle variables, but account forrelatively small differences between the methodologies. The paper alsoexamines differences in the definition and measurement of radio listeningas the reason for differences between diary and PPM measurement. Thisanalysis points to the definition of radio listening as the predominantreason for the different radio audiences observed.

STATION TO STATION

Measuring radio audienceswith a PPM panel in Québec

Pasquale (Pat) A. PellegriniKen Purdye

Pasquale (Pat) A. Pellegrini, Ken Purdye


2

INTRODUCTION

BBM has been operating a commercial Personal People Meter (PPM) panel forthe measurement of French-speaking television audiences in Québec foralmost two years now. Our extensive experience in setting up and enhancingthis panel is covered fully in an earlier paper (Pellegrini and Purdye, 2004). Inaddition to measuring TV audiences, 22 radio stations (both English andFrench, Canadian and U.S.) have allowed their signals to be encoded tofacilitate the testing of Radio measurement by this same PPM panel. The radioPPM data for these stations has been produced and analysed, and are thesubject of this paper.

BackgroundThe Personal People Meter is an audience measurement system developed bythe Arbitron Company in the United States. Co-operating broadcasters placean identifying audio code and a time stamp in their output using a specialencoder. A sample of the population is equipped with a small carry-arounddevice similar to a pager, which “listens” for, detects and stores these codes.The PPM therefore provides a near-passive record of the sample’s exposure toencoded media (Patchen and Webb, 2002; Pellegrini and Purdye, 2004).The BBM Canada PPM Television panel in Québec currently has an installedbase of about 550 homes, of which about 375 are in the Montreal ExtendedMarket. Data from the panel have been used for two years as the basis oftrading television time in Quebec. The panel is selected in a two-stepprocedure. A large Establishment Survey is used as the master sample fromwhich the actual audience panel is selected, using stratified randomprocedures. Viewing data from respondents is collected and edited on a dailybasis: respondents are accepted into tabulation provided they meet fixedqualification rules, the most important of which is that the PPM has shown atleast four hours of carry time during the preceding 24 hour period, as indicatedby a built-in motion detector. On an average day, about 90% of the installedpanel is accepted into production. The Québec PPM panel is a French-language panel, comprising those who speak French most often at home.Although now operating solely as a television measurement panel, the majorityof radio stations in Montreal have also allowed their signals to be encoded.Currently, the encoded stations account for over 92% of all radio listening inthe Montreal extended market. The data is currently processed and edited,using the television compliance and qualification rules, although it is notreleased to BBM members so as not to disrupt the market which obtains itscurrency through a diary methodology.

Station to station


3

The standard measurement instrument for radio in Canada is a personal diary.The diary is a one-week record of radio listening with a fresh sample selectedfor each week of an eight-week “sweep” period. There are two such sweepperiods per year. The sample of homes is selected at random and contacted bytelephone. Those homes co-operating in the telephone survey are sent radiodiaries to complete. All members of contacted households are sent a personaldiary covering a one-week period (Monday through Sunday). Diaries areedited on a weekly basis – the diary must be acceptably completed for each ofits seven days in order to be accepted into tabulation.The diary has a time grid for each day, marked into quarter hours. Respondentsare asked to keep a record of their radio listening in this diary by entering intothe grid the call letters or frequencies of stations listened to, together with thelocation of listening. It is important to note that this diary format issignificantly different from that used by Arbitron (which is much more open-ended), or indeed the RAJAR diary in the U.K. (which is more closed-ended).As the diary is the currency for radio audience measurement, any movement toelectronic measurement necessitates a careful comparison and analysis of thecompeting methodologies. We now have two diary sweep periods during thelife of the TV PPM panel within which we can compare radio audiencemeasurements. The Spring 2004 sweep extended from February 16 throughApril 11, 2004 while the Fall 2004 sweep was from September 6 throughOctober 31, 2004.The standard geographic area used for analysis by the radio industry in Canadais the Montréal Census Metropolitan Area (CMA) as defined by StatisticsCanada. However, the comparisons in this paper are made on a somewhatlarger area, the Montreal Extended Television Market (EM). This was dictatedby the sampling and weighting structure of the PPM panel, which, as notedearlier, is specifically designed for television measurement. Diary data werealso processed for this larger area. The Montréal CMA has about 80% of thepopulation of the EM.For purposes of the comparisons, “Radio Listening” is defined as the sum oflistening to the 20 (Spring) or 22 (Fall) encoded radio stations, for both PPMand the diary survey. For the Spring 2004 survey, the weekly diary sample wason average 576 people aged 12 years and over, for a total of 4,609 respondentsover the eight weeks of the survey. The PPM panel averaged around 548people aged 12+ years per week. It would be easy to make the comparison of4609 vs. 548 diaries and dismiss the PPM sample as “too small”. However, weneed to keep in mind that the PPM sample is a panel: theoretically, the samesample is used for eight weeks. This does not mean that it is worth 8 x 548 =4384, but the “re-use” factor means that the effective base is considerably



4

more than 548. For the Fall 2004 sample, equivalent figures were 794 for thediary and 641 for the PPM panel

COMPARATIVE ANALYSIS RESULTS

During the Fall 2004 sweep survey, the average diary respondent listened tothe radio for 2.6 hours per day. The equivalent figure for the PPM panel at thesame time was 2.0 hours per day. On the face of it, PPM shows about 20% lessradio listening in total than the diary. However, this is to ignore an importantdifference in definition. The diary collects information for each quarter hourduring the week. Respondents are asked to count themselves as listening toradio for a specific quarter hour if they listen to at least five minutes within it.They are then counted as having listened for the full fifteen minutes. Radiolistening, according to the diary, is essentially an agglomeration of quarter-hour cumulative audience estimates.The PPM rules used for these comparisons are identical to those used for thecommercial TV panel: exposure is recorded on a continuous basis and thensummarized by minute. A minute is counted as listening if at least 31 secondsof exposure occurred. Each minute is counted as a full minute of listening.We can simulate the diary definition of listening with the PPM data bycounting a quarter hour as “listened to” if the respondent listened to at leastfive minutes of radio within it. In all such cases, the full quarter hour is thencredited as listening, as with the diary. When this is done, the PPM and diarydata become much closer: The average PPM panel respondent listened to 2.6hours of radio per day, about the same level as the diary. Our comparison ofBBM data for the Spring 2004 sweep period shows similar results. Thelistening estimates were 2.6 hours per day for the diary and 2.5 hours per dayfor PPM. Arbitron’s comparison of diary and PPM in the first year of thePhiladelphia test showed almost equal quarter hour ratings for the twomethods, when adjustments were made for the definition of radio listening(Patchen and Webb, 2002).However, the two measurement systems (PPM and diary) differ in how theyarrive at the common 2.6 hours of listening per capita, per day. With the diarysystem, 69% of respondents listened to radio at all on an average day, and theylistened for 3.7 hours each. PPM shows more people listening to radio, butthey listen less. According to PPM, more people listened to radio on anaverage day (81% of respondents) but they listened for a shorter period of time(3.1 hours).

Station to station


5

Differences by Age and GenderThere are significant differences between the Diary and PPM with respect tothe amount of radio listening reported by gender and age group. There is aconsistent tendency for PPM to estimate larger radio audiences for men thanthe diary (2.9 hours per day per capita for the PPM, 2.7 hours per day percapita for the diary). Conversely, there is a tendency for the diary to estimatelarger radio audiences for women than the PPM (2.8 vs. 2.4 hours,respectively).There is also a tendency for the diary to show larger audiences than PPM forolder respondents, while the reverse is true for younger respondents. Figure 1shows the average hours of radio listening per day per capita for different agegroups for the two measurement systems. Among teens, the PPM shows radiolistening about 50% higher than the diary. The difference decreases as the ageof the respondent increases. Among those 65 years and over, the PPM showsradio listening at about 80% of the diary levels.

Figure 1

1 .0

1 .7

2.6

3.02.9

2 .8

1 .5

1.8

2.7 2 .7

2 .2 2.3

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

12 -1 7 1 8-2 4 2 5-34 35-49 50 -64 65+

AGE GR OU P

AVE

RA

GE

HO

UR

S O

F L

ISTE

NIN

G

DIAR Y

PPM



6

Radio with the PPM is a younger and more masculine medium than it is withthe diary. These findings are consistent with Arbitron’s findings inPhiladelphia (Patchen and Webb, 2000). Given the importance of the age/sexvariables in both programming and the trading of radio time, this difference isan important one.

Differences by DaypartFigure 2 shows the audience to radio on a quarter-hour basis, throughout theaverage weekday (Monday to Friday). The variables are time (x axis) and thepercent of the population using radio (y axis). The data is from the Fall 2004comparison and is quite similar to the findings for the Spring of 2004. Mondayto Friday has been chosen for the comparison, since listening patterns aresomewhat different from this on the weekends for both the PPM and the Diary.

Figure 2

0 .0

5 .0

10 .0

15 .0

20 .0

25 .0

30 .0

05:0

0 -

05:1

5

05:4

5 -

06:

00

06:3

0 -

06:

45

07:1

5 -

07:3

0

08:

00 -

08:

15

08:4

5 -

09:0

0

09:

30 -

09:

45

10:1

5 -

10:3

0

11:

00 -

11:

15

11:

45 -

12:

00

12:3

0 -

12:4

5

13:

15 -

13:

30

14:0

0 -

14:1

5

14:

45 -

15:

00

15:3

0 -

15:4

5

16:

15 -

16:

30

17:0

0 -

17:

15

17:4

5 -

18:0

0

18:

30 -

18:

45

19:1

5 -

19:3

0

20:

00 -

20:

15

20:4

5 -

21:

00

21:3

0 -

21:

45

22:1

5 -

22:

30

23:0

0 -

23:1

5

23:

45 -

24:

00

24:3

0 -

24:4

5

PE

OP

LE U

SIN

G R

AD

IO (

%)

DIAR Y

PPM

The two series show similar audience levels in the early morning hours, butthen diverge sharply, with the maximum difference appearing between 7:30am and 8:00 am. The difference then declines until the two series converge ataround 10:30-11:00 am. The series follow each other closely for the rest of theday, although there is an increase in audience for the diary, unmatched by thePPM, between 4:30 and 5:30 pm. (This blip was more pronounced during the

Station to station


7

Spring 2004 survey). During the evening hours, the PPM curve is consistentlyhigher than the diary curve.Similar patterns exist for adults only (18+ years), for adult men and womenand for different age groups. There are differences between the diary and thePPM with respect to the overall amount of radio listening, as we have seen.However, the patterns shown in figure 2 persist for each of the groups: there isa gap during the morning drive period, with convergence by 10:30-11:00 am,after which time the two series parallel each other closely.As a consequence, the morning drive period is relatively less important in aPPM world than in a diary world (Poesmans, 2004; Futsaeter, 2004).Conversely, daytime and evening hours are relatively more important. This isillustrated in figure 3, which shows the distribution of weekday radio listeningbetween the four broad time periods of morning drive, daytime, afternoondrive and evening, for the two systems. While daytime hours (Mo-Fr 10:00 am- 4:00 pm) are the most important according to both measurement systems,they are more important for the PPM than for the diary. The morning driveperiod is relatively more important for the diary; evening hours are relativelymore important for PPM.

Figure 3

32.527.2

40.8

42.9

17.017.3

9.6 12.6

0%

20%

40%

60%

80%

100%

DIARY PPM

PER

CE

NT O

F W

EEK

DA

Y LI

STEN

ING

M F7-1

M F4-7

M F10-4

M F 5-10



8

Recall from above that the difference between the Diary and PPM results areless pronounced for men than for women. This observation, combined with thegreater importance of the daytime time block in the PPM data, produces aninteresting result: in the PPM data, male listeners are more numerous thanfemale listeners during the daytime period. With the PPM, the average PeopleUsing Radio (PUR) for men is 23% and for women 19%. The equivalent diaryestimates are 20% for men and 23% for women.The most important difference between the two time series in figure 2 is thediscrepancy in the morning drive period, and to a much lesser extent in theafternoon drive period. Why does the diary produce relatively higher estimatesof radio listening in these periods? We know that there is no such issue withtelevision. Until Fall 2004, BBM operated a push button meter panel inQuébec, in parallel with the PPM panel. The two panels showed very similaramounts of television viewing in the early morning hours with parallel viewingcurves. In addition, we know that by 7.30 am, half of the PPMs in the samplehave been undocked and are showing motion (Pellegrini and Purdye, 2004). Ifmorning television produces similar figures, why is this not the same formorning radio?The morning drive period is radio’s prime time, the equivalent of the 8:00-10:00 pm period for television. When BBM replaced a television diary with apush button meter system, the largest differences between the two systemswere found in prime time (BBM Canada, 1999). The diary had relativelylarger audiences for prime time programming, the meter for off-prime periods;the meter showed smaller variance across time. The same effect is evident herefor radio’s prime time (morning drive) compared to its off-prime periods(evening and weekend). To some extent, we may be observing here the samephenomenon – the tendency of a diary to accentuate the positive and viceversa.One tentative hypothesis is that this difference in listening estimates is relatedto the capture of out of home listening, and specifically in car radio listening.The version of the PPM used in the Québec PPM panel does not provideinformation on the location of the radio listening detected (in home, at work, ina car or other). However, the diary does collect such information on a quarter-hour by quarter-hour basis. From this diary research we know that theeponymous morning and afternoon drive periods have a larger percentage ofthe total radio audience in cars than other time periods. Is there an associationbetween the percent of the total radio audience that is in cars and therelationship between the diary and PPM estimates of total radio listening? Wehave tried to test this by correlating the difference between the diary and PPMestimates with the percentage of the radio audience that is in car (according tothe diary).

Station to station


9

It is difficult to specify a simple regression model of the type Yt = a + bXt + et,where Y is the relationship between PPM and the Diary, X is in-car radiolistening and e is the error term. The error term is likely to be highlyautocorrelated, violating one of the preconditions for a simple regressionanalysis. We therefore took first differences and estimated

Yt - Yt-1 = b(Xt - Xt-1)where Y is the difference between the two estimates of PUR and X is thepercent of the diary audience located in cars. The results of this analysis aremildly supportive of the hypothesis that in-car listening may be (to someextent) the culprit.The regression is “significant” beyond the 99% level, as measured by an Ftest) and the value for b is significant beyond the 99% level (t = 6.3). The fit ispoor, with an R2 of just .34 and there is some evidence that autocorrelation inthe error term persists in spite of the differencing. (There seems to be atendency to underestimate morning drive and overestimate evening drive andthe Durbin-Watson statistic is uncomfortably high). Perhaps this result issimply due to the wide array of unmeasured variables in the model. Clearly,the difference between the two series is due to many more things than thesimple percentage of the audience listening in cars.Figure 4 shows the observed difference in PUR (Diary minus PPM) plottedagainst the predicted difference, by quarter hour. It shows the actual differenceand the difference that is predicted with knowledge only of the percentage ofaudience in the car. Knowing that indexes are the preferred tool of mediaanalysts, we have repeated this analysis using relative differences (Diary PUR:PPM PUR) as the dependent variable. The fit is still a rough one with a smallR2 and, again, there is some evidence of autocorrelation in the error term.



10

Figure 4

-4

-2

0

2

4

6

8

10

05:0

0 - 0

5:15

05:4

5 -

06:0

0

06:3

0 - 0

6:45

07:1

5 - 0

7:30

08:0

0 - 0

8:15

08:4

5 - 0

9:00

09:3

0 -

09:4

5

10:1

5 - 1

0:30

11:0

0 - 1

1:15

11:4

5 - 1

2:00

12:3

0 - 1

2:45

13:1

5 -

13:3

0

14:0

0 - 1

4:15

14:4

5 - 1

5:00

15:3

0 - 1

5:45

16:1

5 - 1

6:30

17:0

0 - 1

7:15

17:4

5 -

18:0

0

18:3

0 - 1

8:45

19:1

5 - 1

9:30

20:0

0 - 2

0:15

20:4

5 -

21:0

0

21:3

0 - 2

1:45

22:1

5 - 2

2:30

23:0

0 - 2

3:15

23:4

5 - 2

4:00

24:3

0 - 2

4:45

M ONDAY-FRIDAY 5:00 AM - 1:00 AM 12+

DIF

FER

ENC

E IN

PU

R (P

CT P

TS)

OBSERVED

PREDICTED

Nonetheless, this analysis does give some food for thought. There is obviouslysome relationship here through the fog of unmeasured variables andautocorrelated errors. One can speculate on why there may be such an issuewith PPM measurement in cars. There may be a technical issue with capturingthe encoded signals if the PPM is carried in a pocket under several layers ofclothing during a typical Montreal winter. There may be a compliance issuewith those who do not habitually listen to radio in the work place and thereforedo not see the need to take the PPM with them, although compliance statisticsargue against this. One final consideration is that the difference between thediary and the PPM in the morning (and evening) drive periods is greater forthe Spring survey (February - March when the weather is cooler) than in theFall (September - October, when the weather is warmer).Alternatively, there may be an issue with the way in which recording of in-carradio listening takes place with the diary. Obviously such listening is notrecorded contemporaneously, but rather on a recall basis, long after the event.There is scope here for over-estimation of actual listening – a point we returnto later in this paper. To illustrate how this applies to the morning drive period,we can consider the reach differences between diary and PPM. For most time

Station to station


11

periods, PPM is considerably higher than the diary. However, this is not thecase during the morning drive period. Here, the diary reports average dailyreach of 60 vs. 44 (morning drive versus day), while PPM reports 57 and 63,respectively. There is certainly an issue here that is associated with in-carlistening that merits further investigation.

Differences by Day of the WeekWhen we look at radio listening by day, the diary shows a consistent pattern:Monday levels are greater than Tuesday levels; Tuesday levels are greater thanWednesday levels, and so on throughout the week. People listen to less radioas the week progresses. This is true for both average PUR and for daily reachof radio and for different age groups and men and women. However, it is notthe case for the PPM data. Listening is at about the same level for eachweekday; if anything there seems to be a small increase at the end of the weekon Thursday and Friday. It has been known for some time that the drop off inrecorded listening from day to day in the diary is a consequence of respondentfatigue. (Experiments starting the diary with a different day show that the newstart day invariably becomes the highest listening day. For this reason, BBMpublishes only average Monday-Friday figures). We have been unable todetect fatigue in the PPM panel.Both the Diary and the PPM data show weekend listening less than weekdaylistening, although the drop-off is markedly less with PPM. The average diaryrespondent listens to the radio for 14.6 hours during the Monday-Friday week;the average PPM respondent listens for an hour less (13.6 hours). The averagediary respondent listens to the radio for 3.4 hours on the weekend (Saturdayand Sunday). The average PPM respondent listens to an hour more (4.4 hours).Again, there may be a relationship here with diary fatigue.

Station Shares of Audience and ReachAudience shares for the different stations encoded are quite similar for theDiary and for PPM (see table 1). The correlation coefficient is .95 (Spring) and.98 (Fall), which clearly shows significant agreement. The rank order ofstations is also well preserved. Within this overall high level of agreement, it isclear that AM stations as a group tend to have slightly lower audience shareswith the PPM system than with the diary system. (A difference of 5 percentagepoints in the Spring and 2 percentage points in the Fall). There seems also tobe a slight tendency for English language stations to have a higher share ofaudience, as a group, with PPM than with the diary. In addition, it wouldappear that the talk stations, as a group, lose audience with PPM (down 7percentage points in the Spring and 2 percentage points in the Fall). The twoclassical music stations consistently lose audience share with PPM, which



12

leaves the large group of “adult contemporary, hits, and rock” to pick up thedifference. These stations are oriented to younger people and have slightlyhigher audience shares with PPM than with the diary. This is consistent withthe change in the audience composition of radio as a whole, noted earlier.

Table 1STATION AUDIENCE SHARES, MO-SU, 5:00 AM – 1:00 AM, 12+

PPM Diary

CBF-FM 10.1 11.3

CBFX-FM 1.2 2.3

CBME-FM 0.0 0.3

CBM-FM 0.0 0.2

CFGL-FM 15.0 12.0

CFQR-FM 2.5 2.8

CHMP-FM 5.0 5.7

CHOM-FM 6.0 5.3

CIME-FM 0.9 1.2

CINF 1.0 1.3

CINW 0.1 0.1

CITE-FM 13.2 12.5

CJAD 0.1 0.2

CJFM-FM 4.3 3.4

CJMS 0.6 1.0

CJPX-FM 2.3 4.6

CJWI 0.4 0.4

CKAC 6.7 7.7

CKGM 0.3 0.1

CKMF-FM 15.9 13.3

CKOI-FM 14.2 13.4

WVNV-FM 0.0 0.3

WYUL-FM 0.2 0.9

100.00 100.00

Station to station


13

The daily reach levels for individual radio stations produced by the PPMsample are, in general, higher than those of the diary sample. In some cases,the difference is quite pronounced, especially for the adult contemporarymusic stations, where PPM daily reach levels are consistently double thoseshown by the diary.Weekly reach levels by station are also much higher with PPM than with thediary and, again, it is generally the higher audience stations (according to thediary) that show the most increase in weekly reach (according to PPM). Infigure 5, the weekly reach levels have been converted to logarithms to simplifythe visual comparison and to adjust for heteroscedasticity in the error term. Ascan be seen, the larger the weekly reach, according to the diary, the larger theincrease in reach shown by PPM.

Figure 5

y = 1.1218x + 0.4155R2 = 0.9111

-2

-1

0

1

2

3

4

5

-1.5 -1 -0.5 0 0.5 1 1 .5 2 2.5 3 3.5 4

LN DIARY W EE KLY REACH

LN P

PM W

EEK

LY R

EA

CH



14

The higher reach levels with PPM have the same implication as the higherreach levels for radio listening as a whole, discussed earlier. For most stations,PPM discovers more listeners than the diary, but they listen less to the stationthan indicated by the diary. Restricting the analysis to the twelve radio stationswith significant audiences, the average listener spends 9.4 hours with thestation according to the diary and 3.6 hours according to the PPM. A corollaryof this is that PPM listeners listen to more stations per week (4.7 on average)than diary listeners (2.2 on average).In our comparisons of a push-button television and a PPM television panel, wenoticed a tendency for the audience composition of certain television programsto lose some of their age skew with PPM (Pellegrini and Purdye, 2004). Thesame tendency appears to take place with radio, although it is much lesspronounced. Here is one example, for CFGL-FM, an easy-listening musicstation, which the diary shows as appealing predominantly to a 35-54 agegroup with 78% of its audience in this category. However, according to thePPM, 63% of the audience is in this same group. The difference is dispersedamong younger and older age groups.

Light Listeners and Heavy ListenersWe have seen that, compared to the diary, PPM shows radio (and individualstations) with more reach, but less average hours of listening per listener.There is a related difference in the structure of radio listening in the twomeasurement systems. The Diary system classifies all listeners into quintilesand we have done the same thing with the PPM sample for the average weekof the eight-week period. In this case, the analysis uses average minuteaudiences. Each sample clearly has its own definition of the limits for eachquintile, but we can compare the two.

Station to station


15

Figure 6

1 36

9

14

16

25

26

5446

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

DIARY PPM

PERCENT OF TOTAL LISTENING (MONDAY-SUNDAY 5:00 AM-1:00 AM)

Q5-HeavyQ4-Med/Heavy

Q3-MediumQ2-Light/MedQ1-Light

The distribution of total radio listening hours between the five quintiles isshown in figure 6 for the diary and PPM data. Clearly the heavy-listeningquintile (5) is of paramount importance for radio listening in both systems.However, it is not quite as important for PPM as for the diary. Quintile 5accounts for 54% of all diary radio listening and 46% of all PPM radiolistening. With PPM, heavy listeners listen less than with the diary, lightlisteners listen more.



16

DISCUSSION

We have, of course, speculated on possible reasons for the differencesdiscussed above. Clearly, there are two possible sources: the sample and themeasurement instrument. Are different types of people attracted to complete aradio diary compared to those who find carrying around a PPM appealing? Arethere inherent properties of the diary and the PPM which contribute to theobserved differences? The speculation and research is ongoing at BBM but wepresent here our current findings and thinking.

Differences in Sample CompositionThe comparisons in this paper used the (BBM-defined) “francophone” radiodiaries from the Spring 2004 and Fall 2004 radio sweep surveys in theMontreal Extended Television Market. They also used the Montrealfrancophone TV panel as it existed during the weeks of the radio sweeps.Although the surveys measure the same population, the same geographicalarea, the same radio stations and are conducted at the same time, there aresignificant differences in sampling and survey methodology. The question is:are the resulting samples similar?Each methodology collects “background” demographic, socio-economic andother data on participants using questionnaires. The two samples werecompared and 25 items identified where the questions were either identical orroughly equivalent in wording. The profiles of the “weighted” samples werethen compared for each of these 25 items. Weighted samples were used, sincethey were also used in the initial comparison of radio listening. The radio diarysample is weighted by geography and age/sex. The PPM sample is weightedby geography, age/sex, household size and TV reception type. The profilecomparisons show differences that remain between the two samples after theseweighting processes have been applied. As with all surveys, some of thequestions contained “no answers”. In these cases, the profiles werereprocessed excluding the no answers. The incidence of “no answers” variedby question and was roughly the same between the two surveys.The Diary and PPM results were then compared for each of the 25 items, usingeither a chi-square or a t-test. This procedure was followed for both the Springand the Fall survey comparisons. The results of the comparisons are shown intable 2. Of the 25 items listed in the table, 11 showed a “significant” differenceat the 5% level or less in both of the comparisons. Three more showed a“significant” difference in one of the two comparisons. The other 11 variablesshowed roughly similar distributions in the two samples.

Station to station


17

Table 2COMPARISON OF DIARY AND PPM SAMPLES ON COMPARABLE VARIABLES

Variable Spring Fall Direction of difference

Official Language p<.001 p<.001 Diary has more bilinguals.

Visit Museum/art gallery p<.001 p<.001 Diary has more who visit.

Go to Movie in past year p<.001 p<.001 Diary has more who go to movies.

Sporting events in past year p<.001 p<.001 Diary has more who attend.

Access internet in past week p<.001 p<.01 Diary has more internauts.

Education p<.01 p<.001 Diary has more graduates, fewerCEGEP graduates.

Household Income p<.01 p<.001 Diary has higher household income.

Household Size p<.05 p<.001 Diary has fewer one-person hholds.

GICs/Term deposits p<.05 p<.05 Diary has more who own.

Mutual funds p<.05 p<.05 Diary has more who own.

Visit Bar/pub p<.05 NS Diary has fewer who patronize.

Gardening p<.05 p<.05 Diary has more who garden.

Principal shopper p<.05 p<.05 Diary has more sole principalshoppers.

Occupation, (Full time) p<.1 NS Diary has more MOPS, fewer bluecollar

Computer at home p<.1 NS Diary has more who own

Age Group NS NS (Weighting Variable for both)

Visit Coffee/donut shop NS NS

Fine dining restaurant NS NS

Hours Worked NS NS

Kids<12 in home NS p<.05 Diary has fewer homes with kids<12

Loan NS NS

Mortgage NS NS

Mother Tongue NS NS

RSPs NS NS

Sex NS NS (Weighting variable for both)



18

Anyone who looks at this list could find commonalties and multivariatestatistical analysis might well shed further insight. However, it is clear (at leastto the authors) that several threads do indeed exist. Here are four.As might be expected, given the universe definition, over 95% of each samplehave a mother tongue French. (Mother tongue is defined as the language firstlearned in childhood and still understood.) However, there is a significantdifference between the two in their knowledge of Canada's other officiallanguage, English. The diary has a consistently greater percentage of bilingualfrancophones than the PPM.Secondly, the diary sample is significantly more “upscale” than the PPM. Thediary sample has a higher percentage of university graduates and high-incomehouseholds. In one of the comparisons, it also had a higher percentage ofMOPs (Managers, Owners, Professionals), although not in the other. The diarysample is more likely to own Guaranteed Investment Certificates or termdeposits and mutual funds. It is (marginally) more likely to have a computer athome, and has a higher proportion of internet users. This picture is notcompletely one sided. Several items that would be a priori expected to belongto this cluster of items do not show a difference between the two samples: finedining, for example.Thirdly, the diary sample has fewer people living in one-person householdsthan the PPM panel. This is not surprising, since it is recruited via a telephonesurvey and any adult at home is an eligible respondent. Unless special effortsare made, a bias in favour of larger households is almost inevitable. Note thatthe PPM panel uses household size as a weighting rim, while the diary sampledoes not. The difference may be, to some extent, a simple artifact of this.Nonetheless, the difference in sample profiles, whether natural or forced, isevident in the comparison and could have an impact on the listening data.Finally, the diary sample is more “active” than the PPM sample. Diarists go tothe movies, to art galleries and museums, to professional sporting events andspend more time on the internet than do PPM carriers. They even do moregardening. (Or, as an alternative explanation, they are more prone to “yea-saying” when answering long questionnaires).So, there do appear to be at least these four differences in the profiles of thetwo samples. The logical next question is: are the differences associated withdifferences in radio listening? For example, do “bilinguals” (represented morein the diary sample than in the PPM sample) listen to more radio, to morebreakfast radio, choose “talk” radio stations like Radio-Canada and CKACmore than “unilinguals” (represented more in the PPM sample than in thediary sample)? In other words, can these differences in sample characteristics

Station to station


19

explain to at least some extent the behavioural differences noted abovebetween the two samples?To test this, we constructed four summary measures of radio listening, andtested whether the measures were correlated with the groups of significantvariables. Here are the measures of radio listening that have been used for eachof the sample subgroups and a summary of the results.1. Total amount of radio listening. Measure: The average use of radio per

quarter hour for the sample subgroups, compared to the average use ofradio for all adults 18+ years (Monday-Friday 5:00 am to 1:00 am). Theaverage for all adults is 15.6%. If differences in sample composition canbe used to explain differences in radio listening between the diary andPPM, then we should expect average listening for the over-representedgroups to be higher than this.In fact, they are not. Bilingual respondents listen to roughly the sameamount of radio as the total sample. “Up-scale” respondents (graduates,MOPs, high income) listen, if anything, to less radio than the total sample.Those living in households of one person or more listen to about the sameextent as the total sample.Respondents taking part in various activities (museum visits, etc.) listen toabout the same amount of radio as the total sample.

2. The relative importance of “morning drive listening”. We have used thepercentage of all radio listening (Mo-Fr 5:00 am - 1: 00 am) that isaccounted for by the breakfast period (Mo-Fr 5:00 am - 1: 00 am). For alladults, this is 32%. If differences in sample composition can explain thedifference between the diary and the PPM with respect to the relativeimportance of morning drive radio listening, then we should expect to seehigher figures for the over-represented groups.The results are mixed. Bilingual respondents show no abnormal tendencyto listen more in breakfast than the rest of the day. The same is true forthose living in households consisting of more than one person, and for thevarious measures of participation in activities. There is nothing here tosuggest that their over-representation in the sample compared to PPM isresponsible for the observed difference.However, for the “upscale” groups (MOPs, Graduates and high-incomegroups), this tendency does exist, although much more in the Spring thanin the Fall survey. This group does tend to listen relatively more in themorning drive than the rest of the day, compared to the total sample ofadults. Therefore, their over-representation in the diary may possibly becontributing to the observed tendency for the diary to show relatively more



20

listening in the morning drive than the PPM. Quantifying this factor is notpossible from this analysis but in all likelihood it is not enough to accountfor the size of the observed difference between the two systems.

3. The relative importance of weekend, compared to weekday, listening. Themeasure used is similar to that used in the previous test: the percentage ofall radio listening (Mo-Su 5:00 am – 1:00 am) that is accounted for byweekend listening (Saturday and Sunday). For all adults, this is about 19%.If differences in sample composition can explain the difference betweenthe diary and the PPM with respect to the relative importance of weekendradio listening, then we should expect to see a higher ratio for the groupsover-represented in the diary survey.There are no such differences. The proportion of radio listening accountedfor by the weekend is about 19% for all the sample subgroups where thediary sample is over-represented compared to PPM.

4. Differences in audience share. We have compared the audience share forall adults 18+ years to the audience share for each of the groups over-represented in the radio diary survey. If there are differences in share in thepredicted direction, we can take this as evidence that sample compositionis affecting audience shares. The “predicted direction” is higher for “talk”stations and lower for “music stations”: PPM yields lower audience sharesfor “talk” stations: and shows higher audience shares for stations that maybe termed “music” stations.If the observed differences in sample composition are influencing thisresult, then we should expect sample groups over-represented in the diarysample to show higher audience shares for the two talk stations and loweraudience shares for the four music stations. Is this the case?The results for talk stations are mixed, but tend to support the hypothesis.MOPs, University graduates, high-income households, those who go togalleries and museums, and to a lesser extent bilingual respondents, are allgroups that are over-represented (relatively) in the diary survey. Theaudience share for the talk stations taken as a group is also higher amongthese audience categories, although not consistently so for both surveys orfor all stations in the group. For the other over-represented groups, theshare of talk stations is about on a par with the average for all respondents18+.Results are also mixed for the music stations taken as a group. For MOPS,university graduates and art gallery hounds, the audience share of thesefour stations is lower than the average for 18+ years. However, the sharefor the other groups is either essentially the same as or higher thanaverage.

Station to station


21

In sum, there are significant differences in the composition of the PPM and theDiary samples on a wide range of variables. The diary sample tends to have arelatively greater proportion of bilingual respondents, “upscale” respondents(as measured by occupation, education and household income, largerhouseholds (i.e. households of more than one person) and more “active”respondents (as measured by a small list of self-reported activities).However, the practical importance of these differences as an explanation ofdifferences in estimates of radio listening between PPM and the diary ismodest. This was tested by comparing groups relatively over-represented inthe diary sample with the total for various measures of radio listening. Thediary shows slightly more listening to radio than PPM, even when correctionsare made for differences in the strict definition of listening. However, samplesubgroups over-represented in the diary sample do not show more radiolistening than average – indeed for some of them the reverse is the case. Onthis evidence, differences in sample cannot explain the small difference inreported listening between PPM and the diary.The diary shows significantly greater listening to radio in the breakfast, ormorning drive period, than PPM. “Upscale” respondents (MOPS, graduatesand high-income homes) also show relatively more radio listening in thisperiod. This is not true for other groups over-represented in the diary survey.On this evidence, differences in sample could explain part of the difference inthe importance of the morning drive period between the diary and the PPM.However, there is no evidence that differences in sample composition accountfor the relatively greater importance of weekend listening in the PPM results.Audience shares are different for PPM and the diary. The diary showsrelatively higher audience shares for “talk” stations (taken as a group) andrelatively lower audience shares for background music stations (taken as agroup). Again, evidence for the possible impact of differences in samplecomposition on this result is mixed. It is possible that the relative over-representation of “upscale” respondents in the diary sample is contributing tothis difference.Taken together, the results suggest that the relative skew to bilingualrespondents, larger households and more active respondents in the diarysurvey seems to have no effect on the results. However, the “upscale” skewmay be contributing to differences in estimates for the importance of morningdrive, and in audience shares.



22

Differences in the Definition and Measurement of Radio ListeningWe have already discussed differences in the formal definition of radiolistening between the diary and PPM. We noted that the diary counts a listenerfor a whole quarter hour if he/she listened to at least five minutes within it.This is different from the normal unit of measure for TV electronic meterpanels, like the PPM, which use units of one minute. In the comparisons madeabove, the PPM data was recalculated using the diary definition of listening,which approximates a 15-minute reach measure with a 5-minute non-consecutive minimum. However, the measurement issue is much morecomplex than this (Pellegrini, 2005).It is relatively clear what the PPM measures: exposure to an encoded radiosignal. If the PPM is being carried or worn by the respondent, it will pick upany codes to which it is exposed, with known tolerances. “Listening” equalsexposure, and the recording of exposure is immediate, with no recall required.Exposure is recorded in home and out of home quite automatically.This exposure is verified by a motion detector – an integral feature of thePPM. Motion data is downloaded nightly with the exposure data, and is usedfor daily qualification of respondents. Currently, respondents are rejected for aday if there is less than four hours of motion. The great majority of panelmembers have more motion than this: the PPM is out of its dock for a median15.3 hours per day and motion is detected for a median 14.4 hours per day orabout 95% of the time it is out of the dock. These results are comparable withthose reported by Arbitron during its Philadelphia test (Patchen and Webb,2002) as well as other tests around the world (Pellegrini, 2004; Pellegrini andPurdye, 2004; Futsaeter, 2004; Poesmans, 2004).Although there is a high degree of carry-around compliance, it does show a(weak) association with radio listening: the more somebody carries around aPPM, the more they are exposed to radio. This could expose the PPM listeningestimates to the possibility of an upward bias. This is illustrated in figure 8which shows average hours of radio listening (y-axis) for different ranges ofcarry-around time. Ignoring the first two data points which are based on verysmall sample sizes (and are non-compliant, out of tab respondents), there is aslight positive slope to the curve. With this small caveat, the definition of radiolistening inherent in the PPM is clean and clear: exposure to a radio station,verified by data from a motion detector.

Station to station


23

Carry Around vs. ListeningThe diary is a different instrument, however. The written instructions given tothe respondent ask him/her to record his/her “listening” without attempting todefine what is meant more closely than the “five minute rule” discussedearlier. Other than that, the definition is left to the respondent (some examplesinclude: paying attention, radio on or off, radio in the background, vaguelyaware, etc.). It is evident from the structure of the diary that the respondentmust know the name of the station, its dial position, a slogan, or some otheridentifying information. The only way to indicate listening is to write downthat a station was listened to within the quarter hour. The respondent must alsohave some external time reference; he/she must know the particular quarterhours within which listening took place. To recapitulate, to record his/her“listening” the respondent must know the quarter hour, the identity of thestation listened to, and whether the listening was for at least five minuteswithin the quarter hour.Respondents are exhorted to “carry your diary with you at all times, so you canrecord your listening wherever it occurs during the survey week”. However, itis clear from debriefing interviews that this is more honoured in the breachthan in the observance. Most respondents complete their diaries a couple oftimes a day or once a day. Problems of recall and memory cast a fog over therespondent’s ability to answer the three questions: which quarter hour, whichstation, at least five minutes? And this problem is compounded by notknowing exactly what personal definition respondents are applying to“listening”.We can illustrate the problem by looking at the PPM exposure data. In thecomparisons provided earlier, we paralleled the formal diary definition byqualifying PPM listening on a quarter hour basis using a five-minuteminimum. For example, in figure 2 we showed quarter-hour by quarter-hourestimates of PUR for the diary and the PPM for an average weekday. Howdoes the PPM data change if we use a qualification other than five minutes?The answer is in figure 7 which shows the original listening curves from figure2 (Diary and 5 minute PPM) plus additional curves from PPM for 1, 3 and 7minutes). All the PPM curves move in parallel, but there is a wide spreadbetween them (the stricter the qualification rule, the lower the curve). Howaccurately diary respondents are interpreting the 5-minute rule is clearlycrucial in any comparison with the PPM. (If they are actually using a 1-minuterule, for example, the gap between diary and PPM in the morning drive periodis attenuated, but the difference increases for the rest of the day).



24

Figure 7

0

5

10

15

20

25

30

05:0

0 - 0

5:1

5

05:4

5 -

06:0

0

06:3

0 - 0

6:45

07:1

5 - 0

7:30

08:0

0 - 0

8:1

5

08:4

5 - 0

9:00

09:3

0 - 0

9:45

10:1

5 - 1

0:30

11:0

0 - 1

1:1

5

11:4

5 - 1

2:00

12:3

0 - 1

2:45

13:1

5 - 1

3:30

14:0

0 - 1

4:1

5

14:4

5 - 1

5:0

0

15:3

0 -

15:4

5

16:1

5 - 1

6:30

17:0

0 - 1

7:15

17:4

5 - 1

8:0

0

18:3

0 - 1

8:4

5

19:1

5 -

19:3

0

20:0

0 - 2

0:15

20:4

5 - 2

1:0

0

21:3

0 - 2

1:4

5

22:1

5 -

22:3

0

23:0

0 - 2

3:15

23:4

5 - 2

4:00

24:3

0 - 2

4:4

5

M O-FR 5 :00 AM - 1:00 AM

PU

R (%

)

1MIN

3 MINS

5 MINS

7 MINS

DIARY

Recording of out-of-home radio is known to present special problems for theradio diary since entries are made ex post facto. Even if the diary is carriedaround, it is difficult to believe that entries are made while driving, or at theoffice. Not surprisingly, the PPM shows relatively more radio listening forgroups for whom out-of-home listening is important. For example, PPM showsmore listening for men (60% of listening away from home) than for women(44% of listening away from home). Similar correlations apply to different agegroups: younger respondents show more listening with PPM than the diary.They also do proportionately more of their listening out of home.The problem of interpreting exactly what respondents have in their heads whenthey make an entry in a radio is exacerbated by a further problem. As indicatedearlier, BBM uses a “household-flooding” approach to sampling: samples ofhouseholds are drawn and then each member of selected households is askedto complete a personal radio listening diary. It is known that a certainproportion of returned diaries are “kept diaries”, that is, they are completed byone household member on behalf of another. In most cases, this is done on aco-operative basis. (The diary minder asks the diary recipient what he or shelistened to during the day). Nonetheless, this adds to the problems ofdeciphering what “listening” is being reported and how it is being filtered andrecalled.

Station to station


25

As indicated earlier, PPM estimates radio listening with more listeners and lessaverage time spent listening than the diary. Radio reaches more people, butthey listen less, thus standing conventional wisdom on its head: radio is, afterall, supposed to be a “frequency” medium. Again, this appears to be a logicalconsequence of the different definitions of listening used by the twomethodologies – passive measurement of exposure and conscious recall of“listening.” The diary has fewer listening events, but they are longer; the PPMhas more, but shorter listening events (Patchen and Webb, 2002).

SUMMARY AND CONCLUSIONS

We have compared parallel systems for the measurement of radio inFrancophone Montréal. The comparisons use the same time, the sameuniverse, the same list of radio stations and the same geographical area. Theonly differences are the system of measurement (diary vs. PPM) and thesample.Once corrected for a difference in the definition of radio listening, the twosystems yield almost identical estimates of the total amount of radio listening.However, the two do produce different pictures of the distribution andstructure of radio listening. Compared to the diary, PPM shows radio with ahigher daily reach and less listening per listener; the two drive periods are ofless importance; evening and weekend radio are of greater importance. In theworld of PPM, radio is more masculine and younger. Its heavy listeners listenless while its light listeners listen more. Although the shares of audienceproduced by the different systems are very similar, it seems that PPM yieldsslightly larger shares of listening for popular music stations, slightly smallershares of listening for talk radio stations.These differences can trace to differences in sample composition or todifferences inherent to the two methodologies. We have demonstrated that,although the diary and the PPM attract different types of people to theirrespective samples, they are generally uncorrelated with the substantivedifferences found in the comparisons. Our discussion on the definitions ofradio listening used by the PPM and the diary leads us to believe that they aremeasuring, at times, different things and that this is the predominant reason forthe two different pictures of radio listening that we outlined earlier.To test this, we are currently conducting two simple but valuable experiments.Respondents to a radio diary survey are being asked to carry around a PPM fora few weeks. In addition, rotated members of the PPM panel are being askedto complete a one-week diary of their radio listening. Our prediction is thateach of these paired comparisons will show differences in the directionsindicated in this paper. The PPM will show less listening to radio in the



26

morning drive period, more listening in the evenings and on weekends, morereach and less average listening per listener. We await the results anxiously.

REFERENCES

BBM Canada (1999). Fall ’98 Meter vs. Diary Data: A comparison of apples andoranges. BBM Canada.

Futsaeter, Knut-Arne. (2004). Testing of PPM for Radio in Norway, The 2004 EuropeanRadio Symposium, Berlin.

Patchen, Robert H. and Beth M. Webb. (2000). A Full Year of Audience Research withPPM. What we’ve learned so far. ARF-ESOMAR WAM Proceedings, Los Angeles.

Patchen, Robert H. and Beth M. Webb. (2002). The Future is Now: The Very LatestFinings from the US Market Launch of the Portable People Meter in Philadelphia. ARF-ESOMAR WAM Proceedings. Cannes.

Pellegrini, P.A. (2005). Listen without Prejudice: Passive measurement for Radio, TVand beyond. VUE: the magazine of the Marketing Research and Intelligence Association(forthcoming).

Pellegrini, P.A. (2004). Personal Meters: Tests, Trials and Comparative Studies. The2004 European Radio Symposium, Berlin. ASI, UK.

Pellegrini, P.A. and Ken Purdye (2004). Passive vs. Button Pushing: A ComprehensiveComparison from Parallel TV Meter Panels in Quebec. ARF-ESOMAR WAMProceedings, Geneva.Poesmans, Daniel. (2004). One year PPM Panel – a confrontation between expectationand reality. The 2004 European Radio Symposium, Berlin.

THE AUTHORS

Pasquale (Pat) A. Pellegrini is Vice President Research, BBM Canada, Canada.

Ken Purdye is Meter Consultant, BBM Canada, Canada.

Documents

Station to Stationpanel is accepted into production. The Québec PPM panel is a French- ... The standard measurement instrument for radio in Canada is a personal diary. The diary is