What could comScore learn from Domino's Pizza?

Last week, The Wall Street Journal's AllThingsD ran an article about comScore, the Internet website traffic measuring service. Yesterday, Jason Calacanis chimed in with his own heated remarks about comScore and one of its investors, and today, Michael Arrington and TechCrunch got sucked into the debate. I've linked to the articles on all three sites, so I won't rehash their arguments. However, let me summarize what I understand as the facts, and then make a few suggestions.

For more than ten years, comScore has used a diary approach for measuring website traffic, the same basic approach used for decades by Nielsen and Arbitron to report broadcast ratings. comScore's users manually reported the sites that they visited, and then their reports were aggregated and extrapolated to come up with gross visit counts. The problem was that almost since comScore's inception, many website operators claimed that their own reports showed much higher traffic than comScore, but comScore defended its methodology and claimed that it was accurate.

Radio and television broadcasters and their advertisers knew for decades that the diary method was inaccurate. People forgot to report programs that they watched or listened to. African-American and Hispanic households were underrepresented in diary samples, as were low-income households in general. Everyone knew that the methodology had serious problems. With the implementation of people meters that automatically track television viewership, the industry saw just how inaccurate the old diary-based system was.

The comScore diary system had similar deficiencies to the broadcast systems, but the company adamantly denied any flaws. Then, a few years ago, Google and Quantcast, among others, introduced "beacon" technologies that would report every time a webpage was viewed, bringing people meter-like technology to the Internet. These systems provided results that were generally much closer to the publishers' own logs than comScore.

Last week, comScore announced its own beacon-based system and stated that it will be significantly more accurate than its diaries. However, for a website operator to get comScore beacons, it either has to pay $10,000 per year to subscribe to comScore's report service or pay a one-time $5,000 fee for the company to "audit" the placement of the website's beacons to insure that one and only one beacon is on every page. If the website operator doesn't want to pay, comScore will continue to use its diary method, which it admits is inaccurate, to measure that site's traffic. It was this "pay for accurate numbers" approach that AllThingsD reported and that made Calacanis go ballistic.

comScore has the right to charge whatever it wants for its service, although $5,000 to "audit" beacon placement seems like a lot of money, given that it's likely to be a largely automated service. What I have problems with is comScore continuing to offer the diary service at all, and that's where my gratuitous Domino's Pizza reference comes in.

Recently, Domino's began running television ads taken from focus groups, where participants said things like "the crust is rubbery" and "the box tastes better than the pizza." Domino's ads say that it has taken the criticisms to heart and has changed its pizzas so they taste better. Domino's no longer sells its old, crappy pizzas, only the new ones. comScore, however, continues to offer the old, crappy diary measurements, right alongside the new, superior, beacon-based ones. If the old methodology doesn'r work very well, STOP USING IT!

If I was an advertiser, I'd now take comScore's diary ratings with a grain of salt. Mixing the two methodologies would actually make it more likely that I'd look at a competitive rating system, not less. I'm sure that comScore is afraid that dropping the diaries before it has a critical mass of beacon-equipped sites will make its overall service much less valuable, but there's an alternative approach it could have taken that would allow them to phase out diaries quickly and still generate revenue:

1. Make the beacons available to website operators free, just like Quantcast and Google do.
2. Provide the auditing service for the first year at a much lower price, and make it an annually renewable service.
3. Clearly distinguish in ratings reports which sites are audited and which are not. This will alert advertisers to sites that may be "gaming the system" by placing multiple beacons on a single page.

It's in comScore's best interest to get rid of the diaries and go to the audited system it's pursuing. It should do everything it can to make the transition as quickly as possible, even if it means leaving some money on the table in the short term.