terry,

you seem to have direct knowledge of arbitron's methodology. have you worked for them?

you recently stated that arbitron always achieves a truly random sample while never returning to the same well of willing participants. i have had some statistical training, but i gladly defer to your command of the subject.

it just seems unlikely that arbitron, who i assume gets the usual "one in ten affirmative" rate, would be able to maintain pure random samples as they claim. i always assumed that this, along with self report bias (which the ppm will correct?), were the two big weaknesses with the diary method.

i also have a ppm question. do they record data while docked? if not, then how will morning numbers be affected by this? most morning shows begin at six, but i read that the average ppm participant didn't undock until eight am. on a four hour show, that's a huge loss in credit.

Never worked for Arbitron, but served for years on the RAB/Arbitron GOALS committee for years as VP of Research for RKO.

Arbitron gets about a 60% cooperation rate once they actually get a live contact, and about 60% of those actually return the diaries. Pure random simply means that each person in the defined universe of the study have an equal chance of being selected. (Arbitron's defined universe for diaries is persons 12+ living in a household with a telephone, but not in military, educational, or other group quarters. (Frat houses and prisons!))

I believe the ppm records listening captured while docked, as long as there as been movement on the meter in the past 10 or 12 hours. (I'd have to check exact time frame.) After a certain time without movement, the meter is dropped from the sample until movement begins again.

Although it is purely an opinion, I suspect the bias caused by the self selection of those ppm users willing to keep and correctly use the meter for years at a time will be far more dramatic than by the self-reporting bias.

Now that we're seeing PPM data, it is clear that the diary failed to capture the stray, out-of-the ordinary short bursts of unusual listening, slightly decreasing cume. The diarykeeper also often summarized (9-5) longer spans of listening, omitting trips to the toilet and copier which moved them away from the radio. So TSL's were inflated.

With the meter we are seeing more occasions of listening with much shorter times per occasion.

>> but not in military, educational, or other group quarters. (Frat houses and prisons!))

I can understand that prisoners don't patronize the advertisers, but why not frat houses? Wouldn't "educational" include college dorms? That's a substantial number of the 18-34 demographic excluded.

Eugene's KNRQ (alt rock) swapped to a more powerful stick in part to get a signal into the OSU dorms. After the move the 12+ share was about the same as it was with the lower power. Could the frat house/educational quarters issue have been a factor?

BTW, I too have been contacted twice by Arbitron and I declined both times -- too many connections in the biz at the time.

The no-dorms-or-barracks rule has always really angered young male stations from time to time.

I did work for an oldies station and an urban station in Fayetteville NC. HUGE military town. The oldies listeners lived off base, making it easy to recruit and/or phone-interview them. A large number of the young male African Americans lived on base, and being of no interest to Arbitron, they were of "no interest" to our research. Made it damn hard to recruit for a music test.

And when working for a classic rock station in Virginia Beach, we watched as the station's entire (virtually) male cume (and 80% of those recruited for an Auditorium Test) set sale in the dead of night for the Gulf War I.

The group quarters exclusion may dissappear now that cell phones are the norm. The rule was originially put in place due to "group phones" isssues, among others.

interesting. i see you point about the same sampling frame over the course of a year or more. better hope your station is favored heavily in that particular frame.

i am guessing that arbitron probably uses probability sampling techniques like stratified or cluster sampling to identify the specific populations in each cell (all the age and gender breakdowns)and then randomizes the participants from each cluster? that makes more sense to me because you don't need that many people to get a good confidence interval per cluster, right?

do you consider 60% to be a good return? after all, it is just over half and the lower the rate of return the more the sample can be biased.

for example, i have always suspected that the kind of people who say yes to telemarketers and surveys are most likely to be those that only represent a certain aspect of the population
(lower socioeconomic status, etc.), which may not have as much going on in their lives.

i wonder if we are not then just randomizing the one particular population for the survey.
that sounds kind of harsh and may not be true, but has there ever been a study of that part of the population that says yes?

No, Arbitron does not used stratified or cluster sampling. Such techniques are barred from receiving accreditation from the Media Ratings Council. It is purely random at a 12+ (6+ with ppm) level.

The sample is then weighted to match the population. This can lead to situations where a 23 year old female from one county can be worth (in Persons-per-diary value weighting)many times more than a female from another county.

I often use an example of one summer in NYC when only 2 black male 45-54 diaries were returned from the Bronx High Density Black Area. Each diary was worth more than 20000 people, compared to an average of about 1800 ppdv. Both diaries were from the same household, and were avid Mets fans. I had to keep reminding the sales staff at country+baseball WHN that our 45-54 numbers, particularly our black 45-54 numbers, would disappear with the next book.

60% is a fantastic return rate in the real world. Arbitron has done more work and study on return and response rates than almost any survey company on earth. It still isn't perfect, but it is a remarkable number.

Arbitron has also done massive and highly respected non-response bias studies. The results have repeatedly shown that those who do not participate in a study such as Arbitron's have virtually identical listening patterns and behaviours as those who do participate.

One of the reasons I have such great respect for Arbitron is because of the money they have spent researching their own research (using both internal and external sources!) The biggest problem with Arbitron is manner radio chooses to use the data --- in levels of detail which are beyond the scope of the study's designs, and refusing to pay for a study that could provide greater confidence at the levels of detail radio insists upon using. Radio tends to totally ignore the word in the second largest font in each study..... the word "Estimates!" Radio pays for a 12+ cume study, and then tries to use the data to estimate women 18-24 who listen to hockey on Tuesday nights between 10 and 10:30 pm.

thanks for the info, terry. (i am curious how they managed to study those who did not participate in studies.) i hope i can pick your brain again sometime in the future...mike trunnell

A few years ago I attended an Arbitron presentation, and remember being stunned by the fact that they don't have a method for contacting people via cell phone...being a CHR PD at the time, it concerned me because a large segment of my listening audience - and even myself for that matter - had gone "wireless" - cell phone only.

Has Arbitron addressed this at all, or is there simply nothing they can do?

From the minutes of the Dec. 3-5 2007 Arbitron Advisory Meeting:

"Mr. Cohen stated that the most recent cell-phone-only statistics from the government indicate that the cell-only percentage for the second half of 2006 was 11.8% overall and 25% among persons 18–24. He said that while the coverage of the landline telephone frame is deteriorating, the costs to recruit cell-only households using traditional telephone frame random-digit dialing (RDD) techniques is several orders of magnitude higher than the cost of recruiting landline households. Some reasons for the higher cost include the higher no-answer rate, lower cooperation rates and the fact that current laws and regulations prohibit the use of auto-dialers, i.e., cell-only household recruitment calls must be hand-dialed.

Arbitron is researching alternative means to bring cell-only households into the sample. The company currently is conducting a field test using an address frame to recruit households. Pending success of the tests, implementation could occur in some markets in 2009.

The Council expressed concern about Arbitron’s current practice of capping cell-phone-only households in its PPM New York service (and other radio first markets beyond Philadelphia) at 5%. Arbitron indicated that it was prepared to use the address-frame sampling method being tested in the Diary service to increase the cell-phone-only portion of the PPM sample if testing proves successful."

Why not text them?

Oh yeah...

Arby: What are you listening to right now?

Celler: Wup dat Jam b da szit!

Arby: !?!

Back to the landlines.