Who's right? Panel-based measurement versus raw server logs

Good summary in the NYT today about how hard it is to get good numbers in online advertising: How Many Site Hits? Depends Who’s Counting - New York Times.

How many people visited Style.com, the online home of Vogue and W magazines, last month? Was it 421,000, or, more optimistically, 497,000? Or was the real number more than three times higher, perhaps 1.8 million?

The main discussion ends up being around comparing the panel-based numbers (Nielsen, ComScore, and others) versus the server logs that people have.

In truth, the panel-based folks should probably offer a pixel to publishers to get even deeper counts - panels can be very off, which works in TV because there's no other options, but not in online advertising.

On another note, I'm getting my wisdom teeth out this afternoon so I won't be blogging for the next week or so. Wish me luck!

Comments

[David]

Good luck on getting your wisdom teeth out. you'll need it :)

[Andrew]

I agree in part. Panels should offer a pixel option in order to enable publishers to help the panels improve their reporting.

However, I think it's more important for a stats package to be internally consistent than to be partially accurate. For example, lets say I'm comparing Google.com and Yahoo.com in comScore. Hypothetically, comScore has a pixel on Google's page, but not one on Yahoo's page. I'd rather comScore report to me the panel-extrapolated data for both Google and Yahoo than the pixel-logs for Google and the panel-extrapolated data from Yahoo. Any errors in methodology will come out in the wash in a side-by-side comparison as long as the methodology for report both stats are internally consistent.

But, it would be create if comScore created some kind of "pixel-outreach" program to improve their ability to extrapolate from a panel. But, my gut says they're probably already doing this.