I rarely write tool specific posts on this blog but since I have recently been asked by a few people about this issue and it affects every web analytics tool, I decided to post it here.
A few months ago, Google, the search engine, started encrypting searches for user who are logged into their Google account while conducting the search. As a result of this encryption, the keyword that the visitors search to arrive to your site is not passed in the referring URL. Web Analytics tools rely on the keywords passed in the referring URL to build the search engine traffic report and in the absence of the keywords there is nothing to report, though they still see that the visits came from Google search. So Google Analytics now marks those visits that do not have a keyword but come from Google with “(not provided)” keyword instead of the actual keyword.
Finding those keywords
Google still tracks all the keywords search by logged in users but just does not pass it in the referrer to the site that the user clicks through to. These keywords are available in the Google Webmaster Tools. To see the report you will have to register your sites in Google Webmaster Tool. Google Webmaster tools will allows you to see all the keywords that were searched, the number of clicks your site got, the average position of your site for those keywords and the landing pages.
If you are not using Google Analytics on your site then you will have to login to Webmaster Tools anytime you want to see those reports.
(All images provided by GAPremium.com – Google Analytics Tips and Tricks)
If you are using Google Analytics then you can connect Google Analytics reports and Google Webmaster tools to get Webmaster reporting within the Google Analytics interface.
However there are three issues with this report when used with Google Analytics (or any another web analytics tool)
- You don’t get other metric (e.g. goal conversion) about the visits that arrived from the keywords.
- This list of keywords includes not only the keywords marked with “(not provided)” but also the other keywords that you see in Organic traffic report. So you will have to do extra analysis to see which keywords are hidden under “(not provided)”.
- If you look at Google Webmaster tool report then you will notice that there are a lot more impressions and clicks than those displayed in the Webmaster report and the Google Analytics report (see below). I was not able to find a reason why Google is only displaying the partial number of keywords, if you know the reason then please let me know.
Digital Analytics & Marketing Jobs
- Project Manager, Research at Turner Broadcasting Systems (Atlanta, GA)
- Web Marketing Analyst-FLU000617 at Fluke (Everett, WA)
- Research Project Manager, PBS KIDS Sprout at NBC (Philadelphia, PA)
- Web Analytics Manager at Tripadvisor (Newton Upper Falls, MA)
- Digital Customer Analytics Sr Consultant at American Cancer Society (Atlanta, GA)
- Web/Digital Analytics Auditing at Audit Bureau of Circulations (Arlington Heights, IL)
- Senior Web Analyst at Pop (Seattle, WA)
- Online Analytics Specialist, Communications, Brookings at Brookings (Washington, DC)
- Director, Brand & Multiplatform Research at Turner Broadcasting System (Atlanta, GA)
- Web Analytics Analyst – Business Intelligence Group at Cmn.com (Houston, TX)
- Senior Manager, Digital Research at NBC (New York, NY)
- Senior Analyst, Modeling, Segmentation, and Forecasting at ARC Worldwide, An Affiliate of Leo Burnett (Chicago, IL)
- Senior Analyst, Digital Measurement | Optimization at ARC Worldwide, An Affiliate of Leo Burnett (Chicago, IL)
- Senior database developer (Sr. SDE) at Optizent (Redmond, WA)
- Associate Games Analyst at Arkadium (New York, NY)
- Director of Analytics at Arkadium (New York, NY)
- Web Analytics Engineer – Contract (POP) at Pop (Seattle, WA)
- Website Analyst at Simplexity (Reston, VA)
While I am all about security and like to see Google forcing SSL as much as possible, this is a pretty big deal for SEO folks. It’s one thing if you have to go through linking in Webmaster Tools to your Analytics account to get these words to show up. But if you are unable to track conversions and still have to explain “not provided” to your clients, it’s nothing short of broken.
I always wondered why there were keywords I would see in webmaster tools and those same keywords were not reflected in analytics. Do you ever wonder what the read big difference is between analytics and webmaster tools? Why can’t they just combine the two somehow? Seems redundant to me.
Thanks for the informative post!
Heather, they can combine both easy but they won’t.
This would give SEOs the whole picture, from SERPs to goals (impressions, clicks, visits, conversions), very far away from what Google is got in mind. The privacy thing is cheap excuse.
In any case there are ways to combine data from both sources but this requires some workaround.
Thanks for posting this sir. I was facing similar issue with my blog analytics. I can now find all the missing keywords in Webmaster. But truly a strange & fascinating move by Google.
The data displayed in Webmaster Tools may differ from the data displayed in other tools, such as Google Analytics. Possible reasons for this include:
We may not have crawled your site since the changes were last made.
Webmaster Tools does some additional data processing – for example, to eliminate duplicates and visits from robots – that may cause your stats to differ from stats listed in other sources.
Some tools, such as Google Analytics, track traffic only from users who have enabled JavaScript in their browser.
Hi
That’s a good point Heather.
From a commerce perspective you want to easily find the keywords which lead to conversions, and with the data encryption shielding a high percentage of the results, this has become 90% guess work.
It seems odd they shield the results for one reporting tool but not the other.
Very good article Mr Batra thanks for posting
Richard
There are many reasons why there are discrepancies in the data between WMT and GA. It’s not easy to find but Google’s official reasons are listed here at the bottom;
https://support.google.com/webmasters/bin/answer.py?hl=en&answer=35252
+++
Some processing of our source data—for example, to eliminate duplicates and visits from robots—may cause your stats to differ from stats listed in other sources. However, these changes should not be significant.
Some tools, such as Google Analytics, track traffic only from users who have enabled JavaScript in their browser.
Some tools define “keywords” differently. For example, the Keywords tool in Google AdWords displays the total number of Google searches for that keyword across the web. The of user queries for that keyword across the web. The Webmaster Tools Search Queries page, however, shows how many of those keyword searches returned your pages in Google search results, and this is a smaller number.
There may be a lag between when the numbers are calculated and when they are visible to webmasters—although data gets published in intervals, we are continually collecting it. Normally, however, data should be available in 2-3 days.
Time zones matter. Search Queries tracks daily data according to Pacific Daylight Time (PDT). If your other systems use different time zones, your daily views may not match exactly.
To protect user privacy, Google doesn’t aggregate all data. For example, we may not track some queries are made a very small number of times or that contain personal or sensitive information.
+++
The main difference is of course due to WMT data being from the server side and the GA being from the browser side.
Good post thanks.
Steve
Great article. Thanks for answering a question thats been on my mind for weeks….and answering it thoroughly!!
This is a recent change i.e. since they switched SEO query reporting from Analytics to Webmaster tools. Now if I want to find out what search keywords are bringing visitors to a particular page all I get is “not provided”. It’s a real PITA.
Excellent Post! About 33% of our site’s traffic is now falling into the keywords “not provided.” The explanation for the differences you are seeing between GWT and GA probably has to do with the fact that GWT “buckets” clicks. This post in Search Engine Watch does a good job of explaining what that means and also shows you how to build a custom report in Google Analytics that incorporates keywords from GWT so that it becomes possible to retrieve some of the keywords “not provided.”
http://searchenginewatch.com/article/2158540/Recovering-Not-Provided-Keyword-Data
Thanks for the valuable insights Anil. I have about half of my traffic from sources ‘not provided’!! Very difficult to manage and this is growing steadily. At least now I know why this is happening. Cheers mate.