Category Archives: Spam Filter

Removing Spam Filters

To remove old spam filters from your account, open Google Analytics and click the link for the Admin panel (bottom left). Make sure your Account is selected, and click the All Filters link in the Account (left) column:

For each spam filter, click the ‘delete’ link beside it to remove it from your account. Analytics Edge spam filters can be identified with names starting with ‘Valid Hostname” and ‘Spam Crawlers’:

Direct ‘Spam’ in Google Analytics

A number of my clients have seen unexplained direct traffic in their Google Analytics accounts. There is no apparent purpose or reason behind this traffic — it just exists; sometimes for a short period of time, and sometimes for months. A number of theories have been thrown around, including ad click fraud, affiliation fraud, purpose-built crawlers that hide their identity, and more. Whatever the reason, people need to be able to see the non-bot traffic to their websites.

What can you do if you are affected?

Check your Referral Exclusion List

The Referral Exclusion List prevents traffic from the listed domains from starting a new referral session. If your site links to a partner site (like a payment gateway), this would join an initial session on your site with a returning visit from the partner site.

If you list other domains (like spam referrals), it will strip the referrals off the session, but the sessions remain — as direct visits. Use exclude filters in your view for spam referrals.

 

Check City, Service Provider and Network Domain

Sometimes there is a single crawler, bot or service causing the whole thing. Maybe it is an ‘uptime’ monitor that you initiated. If it all comes from one place, you can easily build a filter to eliminate it.

 

Check Browser and Operating System Versions

If the traffic comes from various locations, it may be generated from one of several bots that have been propagated around the internet. Look for a spike in direct traffic from old browser versions like Firefox 11.0, Internet Explorer 7.0, or Chrome 18.0.1015.168. Programmers sometimes use code they found on the ‘net and it uses embedded versions of these browsers. Real people tend to upgrade their devices; bots tend to stay the same. If you aren’t sure about a particular version, check the traffic using that version over the past 6 months and see if it is all direct, or if it includes any organic search visits.

 

Check Browser Size

When programmer’s make a bot, the code can spread across browser versions, and maybe even operating systems, but if they open a hidden browser window to visit your page, it is probably the same size, every time. In particular, look for sizes like 620×480 or 610×480 that have a lot of direct visits but no other sources.

 

Helpful Filters I Have Used

After performing a significant analysis on several client sites, I came up with this series of filters, which must be implemented IN ORDER. It works on the theory that most of the direct spam was from old browsers or specific browser sizes, and it landed only on the home page of the web site. Note that there were always a few people running really old computers that happen to match the profile, and they visit other pages of the site as well. I did not want to filter out this other traffic, especially if it included ecommerce transactions!

Filtering out visits from a  specific service provider is fairly simple, but how do you eliminate visits based on multiple criteria, like visits to the home page from a specific browser and version? Google Analytics Advanced Filters offers a solution.

A note of caution: this is not a comprehensive guide to advanced filters — that really is an advanced topic. I offer it as an example of what I have used for my own clients so that others can benefit. Also, it worked for me in the past…it may not work for you in the future.

Capture the Browser and Browser Version

The first filter gets the browser (Firefox, Internet Explorer, Chrome, etc) and the browser version (11.0, 43.0.2357.130, etc) into a single blob that I can work with — I used a field called “User Defined”. After the filter runs, the User Defined field will contain things like “Firefox 11.0”.

Tag Unwanted Traffic

Next comes a series of filters that change the User Defined value to “Direct Spam” if it happens to use a specific browser version AND is a hit to the home page (in this case, “/Home”; yours might be “/” in which case use the expression “^/$“).

The list of browser versions I have used includes (verify on your own site before you adopt these):

Firefox 11.0
Internet Explorer 7.0
Chrome 18.0.1025.168
Chrome 39.0.2171.95
Safari 5.1
Chrome 43.0.2357.130
Internet Explorer 9.0
Internet Explorer 8.0
Firefox 3.6.28
Firefox 12.0

Tag Other Unwanted Traffic

You may have other criteria, like browser size, that you want to use – include similar filters for them as well.

Exclude Tagged Traffic

Finally, look at the value in User Defined, and if it has been tagged as “Direct Spam”, then exclude it.

 

Put The Filters In Order!

This is critical! The first filter (capturing the browser and version) MUST be first, and the last filter (excluding the spam) MUST be last.  Google will execute the filters in order, and that is the only way they will accomplish the task.

 

Enjoy It While It Lasts

You should now see a drop in direct spam visits as the filters take effect (use a segment for historical reporting). Just remember that things change and you will need to revisit the effectiveness of the filters as time passes, especially if you see a jump in direct visits again.

This article describes a technique I used that was effective for me. Every website is different– you need to do your own analysis before implementing a solution.

Spam in Google Analytics?

Yes, the popularity of Google Analytics has gotten to the point that it has become a popular target for spammer. They have discovered that they can spoof the system into thinking that real traffic went to your website from fake referrals, searches and events.

Their objective: make you go to their website to see what is going on. Don’t do it! 

What is a ghost referral?

A ghost referral is a fake visit created by spammers spoofing the Google Analytics tracking servers into thinking that your website had referral traffic from their site. In reality, no one ever visited your site! They don’t even know what it is – they randomly select tracking id numbers. A Valid Hostname filter is the best defense.

What is a ghost visit?

A ghost referral is a fake visit created by spammers spoofing the Google Analytics tracking servers into thinking that your website had real traffic, and it could appear as a referral, as an organic search with fake keywords, as direct visits with fake landing pages and even as fake events. In reality, no one ever visited your site! They don’t even know what it is – they randomly select tracking id numbers. A Valid Hostname filter is the best defense.

What is a valid hostname filter?

When traffic hits your web server, it records the server name in the Google Analytics ‘hostname‘ field. Since spoofed traffic doesn’t actually know what your website is, it contains fake hostname values. Your can easily create a filter to let in only traffic that contains your valid hostname value. See the Definitive Guide for detailed instructions.

Definitive Guide to Removing All Google Analytics Spam

This is a PROVEN WORKING SOLUTION with complete filter expressions.

2017-09-13: the need for this solution has dramatically dropped over the past year as Google’s defenses have gotten significantly better. Referral spam attacks are routinely stopped within a day, so creating a filter the next day is a waste of time.  If you want to get rid of the remaining spam from your reports, you will need to use a Segment (step 4) that includes all of the filter expressions listed below. — Mike Sullivan

How to Prevent and Remove Spam:

  1. BEFORE YOU START: Make an Unfiltered View!
  2. Implement a Valid Hostname Filter to eliminate ghost visits (like sell-fb-group-here.com)
  3. Implement Spam Crawler Filters to eliminate the targeted spam visits (like ḷ.com and foxweber.com)
  4. Create a Custom Segment with these filters to use for reporting

All the information you need (and more!) is provided below in this step-by-step guide. Filter expressions are updated within a day or two as needed.

Continue reading

Why You Should Not Use the Referral Exclusion List for Spam

While all of us fight to get Google Analytics referral spam under control, one piece of well-intentioned, but bad advice comes up over and over again: use the Referral Exclusion List. Some people (myself included) have stated that is a bad idea, but no one has taken the time to really explain why. Here is the Why

[if you want to know what you should do, read the Definitive Guide to Removing Referral Spam]

Continue reading