Direct ‘Spam’ in Google Analytics

A number of my clients have seen unexplained direct traffic in their Google Analytics accounts. There is no apparent purpose or reason behind this traffic — it just exists; sometimes for a short period of time, and sometimes for months. A number of theories have been thrown around, including ad click fraud, affiliation fraud, purpose-built crawlers that hide their identity, and more. Whatever the reason, people need to be able to see the non-bot traffic to their websites.

What can you do if you are affected?

Check your Referral Exclusion List

The Referral Exclusion List prevents traffic from the listed domains from starting a new referral session. If your site links to a partner site (like a payment gateway), this would join an initial session on your site with a returning visit from the partner site.

If you list other domains (like spam referrals), it will strip the referrals off the session, but the sessions remain — as direct visits. Use exclude filters in your view for spam referrals.

Check City, Service Provider and Network Domain

Sometimes there is a single crawler, bot or service causing the whole thing. Maybe it is an ‘uptime’ monitor that you initiated. If it all comes from one place, you can easily build a filter to eliminate it.

 

Check Browser and Operating System Versions

If the traffic comes from various locations, it may be generated from one of several bots that have been propagated around the internet. Look for a spike in direct traffic from old browser versions like Firefox 11.0, Internet Explorer 7.0, or Chrome 18.0.1015.168. Programmers sometimes use code they found on the ‘net and it uses embedded versions of these browsers. Real people tend to upgrade their devices; bots tend to stay the same. If you aren’t sure about a particular version, check the traffic using that version over the past 6 months and see if it is all direct, or if it includes any organic search visits.

 

Check Browser Size

When programmer’s make a bot, the code can spread across browser versions, and maybe even operating systems, but if they open a hidden browser window to visit your page, it is probably the same size, every time. In particular, look for sizes like 620×480 or 610×480 that have a lot of direct visits but no other sources.

 

Helpful Filters I Have Used

After performing a significant analysis on several client sites, I came up with this series of filters, which must be implemented IN ORDER. It works on the theory that most of the direct spam was from old browsers or specific browser sizes, and it landed only on the home page of the web site. Note that there were always a few people running really old computers that happen to match the profile, and they visit other pages of the site as well. I did not want to filter out this other traffic, especially if it included ecommerce transactions!

Filtering out visits from a  specific service provider is fairly simple, but how do you eliminate visits based on multiple criteria, like visits to the home page from a specific browser and version? Google Analytics Advanced Filters offers a solution.

A note of caution: this is not a comprehensive guide to advanced filters — that really is an advanced topic. I offer it as an example of what I have used for my own clients so that others can benefit. Also, it worked for me in the past…it may not work for you in the future.

Capture the Browser and Browser Version

The first filter gets the browser (Firefox, Internet Explorer, Chrome, etc) and the browser version (11.0, 43.0.2357.130, etc) into a single blob that I can work with — I used a field called “User Defined”. After the filter runs, the User Defined field will contain things like “Firefox 11.0”.

Tag Unwanted Traffic

Next comes a series of filters that change the User Defined value to “Direct Spam” if it happens to use a specific browser version AND is a hit to the home page (in this case, “/Home”; yours might be “/” in which case use the expression “^/$“).

The list of browser versions I have used includes (verify on your own site before you adopt these):

Firefox 11.0
Internet Explorer 7.0
Chrome 18.0.1025.168
Chrome 39.0.2171.95
Safari 5.1
Chrome 43.0.2357.130
Internet Explorer 9.0
Internet Explorer 8.0
Firefox 3.6.28
Firefox 12.0

Tag Other Unwanted Traffic

You may have other criteria, like browser size, that you want to use – include similar filters for them as well.

Exclude Tagged Traffic

Finally, look at the value in User Defined, and if it has been tagged as “Direct Spam”, then exclude it.

 

Put The Filters In Order!

This is critical! The first filter (capturing the browser and version) MUST be first, and the last filter (excluding the spam) MUST be last.  Google will execute the filters in order, and that is the only way they will accomplish the task.

 

Enjoy It While It Lasts

You should now see a drop in direct spam visits as the filters take effect (use a segment for historical reporting). Just remember that things change and you will need to revisit the effectiveness of the filters as time passes, especially if you see a jump in direct visits again.

This article describes a technique I used that was effective for me. Every website is different– you need to do your own analysis before implementing a solution.