What could comScore learn from Domino's Pizza?

Last week, The Wall Street Journal's AllThingsD ran an article about comScore, the Internet website traffic measuring service. Yesterday, Jason Calacanis chimed in with his own heated remarks about comScore and one of its investors, and today, Michael Arrington and TechCrunch got sucked into the debate. I've linked to the articles on all three sites, so I won't rehash their arguments. However, let me summarize what I understand as the facts, and then make a few suggestions.

For more than ten years, comScore has used a diary approach for measuring website traffic, the same basic approach used for decades by Nielsen and Arbitron to report broadcast ratings. comScore's users manually reported the sites that they visited, and then their reports were aggregated and extrapolated to come up with gross visit counts. The problem was that almost since comScore's inception, many website operators claimed that their own reports showed much higher traffic than comScore, but comScore defended its methodology and claimed that it was accurate.

Radio and television broadcasters and their advertisers knew for decades that the diary method was inaccurate. People forgot to report programs that they watched or listened to. African-American and Hispanic households were underrepresented in diary samples, as were low-income households in general. Everyone knew that the methodology had serious problems. With the implementation of people meters that automatically track television viewership, the industry saw just how inaccurate the old diary-based system was.

The comScore diary system had similar deficiencies to the broadcast systems, but the company adamantly denied any flaws. Then, a few years ago, Google and Quantcast, among others, introduced "beacon" technologies that would report every time a webpage was viewed, bringing people meter-like technology to the Internet. These systems provided results that were generally much closer to the publishers' own logs than comScore.

Last week, comScore announced its own beacon-based system and stated that it will be significantly more accurate than its diaries. However, for a website operator to get comScore beacons, it either has to pay $10,000 per year to subscribe to comScore's report service or pay a one-time $5,000 fee for the company to "audit" the placement of the website's beacons to insure that one and only one beacon is on every page. If the website operator doesn't want to pay, comScore will continue to use its diary method, which it admits is inaccurate, to measure that site's traffic. It was this "pay for accurate numbers" approach that AllThingsD reported and that made Calacanis go ballistic.

comScore has the right to charge whatever it wants for its service, although $5,000 to "audit" beacon placement seems like a lot of money, given that it's likely to be a largely automated service. What I have problems with is comScore continuing to offer the diary service at all, and that's where my gratuitous Domino's Pizza reference comes in.

Recently, Domino's began running television ads taken from focus groups, where participants said things like "the crust is rubbery" and "the box tastes better than the pizza." Domino's ads say that it has taken the criticisms to heart and has changed its pizzas so they taste better. Domino's no longer sells its old, crappy pizzas, only the new ones. comScore, however, continues to offer the old, crappy diary measurements, right alongside the new, superior, beacon-based ones. If the old methodology doesn'r work very well, STOP USING IT!

If I was an advertiser, I'd now take comScore's diary ratings with a grain of salt. Mixing the two methodologies would actually make it more likely that I'd look at a competitive rating system, not less. I'm sure that comScore is afraid that dropping the diaries before it has a critical mass of beacon-equipped sites will make its overall service much less valuable, but there's an alternative approach it could have taken that would allow them to phase out diaries quickly and still generate revenue:

1. Make the beacons available to website operators free, just like Quantcast and Google do.
2. Provide the auditing service for the first year at a much lower price, and make it an annually renewable service.
3. Clearly distinguish in ratings reports which sites are audited and which are not. This will alert advertisers to sites that may be "gaming the system" by placing multiple beacons on a single page.

It's in comScore's best interest to get rid of the diaries and go to the audited system it's pursuing. It should do everything it can to make the transition as quickly as possible, even if it means leaving some money on the table in the short term.

Measuring Traffic

I'm experimenting with multiple ways of measuring this blog's traffic (although that could be compared to hooking an EKG up to a dead body.) I've used Google Analytics for some time, as well as FeedBurner's stats. They're both Google services, but they measure different things: Google Analytics measures traffic to the site, while FeedBurner measures subscribers to my RSS feeds. Both are important in figuring out who's using the site and what articles get the most interest.

I've recently added IZEARanks.com and Quantcast. Both services gather usage statistics directly from the sites themselves, rather than from a sampling of Internet users. I've placed a bit of JavaScript into the right-hand column of my site for each service (if you're reading this as a webpage, just scroll down and you'll see them.) IZEARanks focuses exclusively on blogs, while Quantcast covers websites, blogs and other services.

IZEARanks.com currently offers three metrics: RealRank, Page Views and Unique Visitors. According to IZEARanks.com, RealRank is calculated by comparing all blogs in their system 70% on daily unique visitors, 20% by daily active inbound links and 10% by daily page views. Quantcast tracks number of monthly unique visitors and frequency of visits, but it also attempts to identify the demographics of visitors. Quantcast also has an interesting media planner feature that enables advertising media buyers to select sites based on their demographics, reach and other characteristics, and to compare sites on those same elements.

For now IZEARanks.com seems to be better for "bragging rights" for blogs, while Quantcast is setting itself up to be a real challenger to Nielsen//NetRatings and comScore. Bloggers don't have to pay a dime to use any of these services, so it's worth experimenting with them to find out which ones get you the best information and sales tools.