Measuring Internet traffic: where are the biases?

There have been quite a few discussions on traffic measurement lately. The general consensus is that all of them have some sort of problems. It would be an interesting exercise to see where are the biases and how we may be able to compensate for them.

ComScore and Hitwise are two leading paid services. They use two different approaches: ComScore is “Panel based” and Hitwise is “ISP based”.

1. ComScore

ComScore has over 2 million users who have installed ComSore’s data collection software on their computers (although their US panel sample is 120K in the US and global panel is 500K outside the US). Their users are randomly selected. ComScore recruits them over the web by offering virus protection scanning, web acceleration or sweepstakes prizes under a number of channels (e.g., PermissionResearch, OpinionSquare and Marketscore).

ComScore’s demographic tends to be skewed toward naive Internet users as more sophisticated users are less likely to install ComScore’s toolbar. Serious security issues have been raised with their software. If you are interested in the details of how ComScore collects user data and the security implications, I would recommend you to read the articles by Stanford, Cornell and Forbes.

2. Hitwise

Hitwise gets its user data from ISPs that it has partnered with. According to Hitwise it have over 10 million US and 25 million worldwide users.

While Hitwise has a much larger and diverse pool of sample users, its data partners are mostly small ISPs and has much more dial-up users in the data set.

In general, Hitiwse’s data tends to be more skewed towards home use and underestimates broadband or work use.

3. Alexa

Alexa offers a free traffic data service and is a subsidiary of Amazon.com. Alexa collects information from over 20 million users who have installed the “Alexa Toolbar”. The Alexa toolbar is available on Internet Explorer and an extension (Status Bar) can be used for Firefox.

Alexa toolbar is offered as a webmaster tool and its user panel is biased towards techies/geeks and webmasters in particular. Alexa’s number can not be used to compare two sites with very different demographic.

Another problem with Alexa is that its numbers are relative shares (percentage of the total population). Because Alexa’s international base is growing much more rapidly, US websites’ Alexa numbers tend to increase slower than their internal stats or even show some decline.

4. Compete and Quantcast

Compete and Quantcast are two smaller free services. Compete.com tries to combine toolbar panel and ISP data whereas Quantcast requires websites to install a tracking pixel. For some websites, they offer good numbers whiles for others, their numbers can be way off. It is still unproven that their approaches offer better results. You can read detailed discussions from Venturebeat, Matt Cutts and Traffick.

Advertisements

2 comments so far

  1. […] sampling based panel has its biases too. For example, the way ComScore gets their user panel does not give me much confidence either although companies pay hundred of thousand a year for the […]

  2. […] compara el tráfico de una página con otra. Este análisis está basado en el popular artículo de life is an adventure. Los servicios más populares […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: