Something’s Fishy in the State of Google’s New Keyword Tool [Data]

With the launch of Google’s New Keyword Tool, they are now taking mobile device searches into account when calculating search volume, indicating search volume should be increasing for virtually all queries. Conductor’s research found that although the majority of keywords increased in search volume, 8% actually decreased, some by some substantial amounts, suggesting there is more going on than meets the eye.

Earlier this week Google completely turned off the old Adwords Keywords Tool as part of their migration to the new AdWords Keyword Planner. The Keyword Tool was a primary tool SEOs used to view search volumes for their keywords and discover additional keywords to optimize in the search engines.

Google’s statement on the change made it pretty clear that the new tool was about adding search volume from mobile devices, whereas before the switch, search volume was calculated only from desktop and laptops (emphasis added):

In general, you’ll notice that the average search volume data is higher in Keyword Planner as compared to the exact match search volume data you got with Keyword Tool. That’s because we’ll show you the average number of searches for a keyword idea on all devices (desktop and laptop computers, tablets, and mobile phones). With Keyword Tool, we showed you average search volume for desktop and laptop computers by default.

Logic would seem to dictate that the addition of a data source to an existing pool of data would result in that pool getting larger. To test this hypothesis and to measure the extent to which the pool did or did not grow, we analyzed more than 550,000 keywords, a representative sample from our database in our Enterprise SEO Platform.

We’re going to walk you through a number of charts, looking at the change from a number of different angles. Hopefully, by the time you are done you will have a good sense of what happened. And, based on the resulting data we’ll give you a number of takeaways you should be thinking about in your own keyword strategy.

25% of Keywords Stayed the Same or Decreased

First, the high level view. Google has stated that the change is about accounting for searches across mobile platforms that were not previously accounted for. That should mean that there are few if any keywords whose search volume decreases.

For the most part the data shows that that is fact the case with 3/4’s (74%) of keywords increasing in search volume.

Overall, individual keywords saw a 52% increase on average in search volume

Click to tweet

But, that leaves 17% of keywords that somehow stayed the same, and oddly, a full 8% of keywords whose search volume actually decreased.

pre vs post keyword tool performance

Looking at this in a slightly different way, while the pie chart above shows the distribution of ‘up vs. down vs. stayed the same’, the scatter plot below nicely illustrates the movement of keywords pre and post change with the vertical axis showing the search volume prior to the change and the horizontal axis showing the search volume post change.

This view shows the 75% of keywords on the ‘increase’ side of the line, but it also shows that there are a cluster of keywords that decreased, many in a substantial way.

find high msv queries to drive more traffic with keyword planner

Next, we look at a distribution of the keywords by ‘bucket’ of movement. This view will tell us how much the keywords’ search volume changed. That is—how many increased by 21%-40% etc. vs. how many decreased by 21%-40% etc.
(Read the chart as the vertical axis shows the percent of keywords and the horizontal axis shows their range of movement. So, for example, 4.22% of keyword search volumes increased by 1%-20%).

Looking closely, two observations that have the potential to impact the day to day life of the SEO stand out:

  • Fully a quarter (26%) of keywords increased by more than 60%
    While I’m sure Google has no, *ahem*, objection, to advertisers seeing a higher (read: much higher) search volume in their ad tool for a large percentage of keywords, with such a large number of keywords increasing, it pays for SEOs to download updated search volumes and compare to old numbers to get a handle on what has changed in their own individual landscape.
  • 8% of keywords went from zero search volume to a search volume of some kind
    Frequent users of Google’s Adword Tool are used to seeing the character ‘-‘ a lot—an indication that Google has no search volume for their keyword. Of the total bucket of keywords we analyzed, 8% of the keywords previously had no search volume and with the new tool now have a search volume of some kind. This is another reason for SEOs to take a close look at how the change has affected them.

Another way to say this that will emphasize the degree to which things have changed is while the 8% metric above points out that of the total bucket of keywords–some of which change by x or by y–8% went from 0 to some search volume. Looking at the bucket of keywords that started out pre-change at 0, 35% now have a search volume of some kind.

35% of keywords that previously reported ‘zero’ for search volume, now have a search volume of some kind

Click to Tweet

percent change of search volume using keyword planning tool

High vs. Low: Keywords with Already High Search Volumes Increased at Greater Rate than Low Volume

Next, we segmented the data by high vs. low volume keywords to see if there were differences in how search volumes changed based on the starting search volume of the keyword. Although there are different ways to define “high vs. low volume”, and the scale can differ significantly by vertical, we set the threshold for high-volume as 10,000 monthly searches.

Looking at the data this way, we found that the rich very much got richer. That is, those keywords that already had high search volumes increased at rates significantly greater than low volume queries. Nearly half (48%) of high volume keywords increased by more than 60% compared to just 24% of low volume keywords increasing that much. And, many more low volume keywords remained low volume keywords, with no change at all in search volume.

keyword search volume increases

Head vs. Long Tail: Head Terms Increased at Greater Rate than Long Tail

Next we looked at head terms (queries with 1 or two words in the query) vs. long tail (queries with 3 or more words).

Here again we saw that head terms (which generally have higher search volume than long tail terms) had a higher proportion of terms that increased by more than 40% (48% to 40%) – although not nearly as wide a gap as high vs. low–suggesting keywords with more words in the term is not as absolute an indicator of search volume as the industry would sometimes have us think.

head terms of one or two words and their growth

long tail terms of three words or more and their growth


Earlier we mentioned that we saw a number of keywords search volume drop significantly—which poses the question if the change is strictly the addition of mobile device search volumes as Google states. Here are several examples of keywords whose search volume decreased significantly:

pre versus post keyword tool table of pre and post search volume

Could the huge drop in ‘facebook’ search volume be explained by mobile users increasingly turning to the mobile app vs. mobile search? And, the Oakland A’s are in first place in their division this year—could the search volume for their team have dropped by 113,000?

More likely the drops are indicators that there are still gremlins to be worked out in the system.

Likewise, there are several anomalies of disproportionate increases in search volume. Here are several examples of keywords whose search volume increased by orders of magnitude that seem pretty impossible, unless there is a sudden rush to search for coffee and office equipment related products on a mobile phone:

second example of pre versus post keyword tool table of pre and post search volume


When we consider what to make of all this, a cynical view would point out that if a key Google objective in rolling out a new keyword tool is increasing revenue by driving up search volume so advertisers would be compelled to spend more and CPC increases are more easily justified, then, mission accomplished.

The cynic might further point out that one way of viewing the skew in % increase of search volume to high volume vs. low volume terms is that if Google set out to increase revenue, they likely focused on increasing those terms that would have the highest return on investment while paying less attention to lower ROI terms. A purely cynical take on things, but worth pointing out.

Even if we are not quite that cynical, given the percentage of keywords going down or staying the same rather than going up as we’d expect, together with the anomalies of keywords whose search volumes changed significantly, (a small sample of which we showed you) it certainly seems like there is more going on than just Google adding mobile search volume.

Taken together, the bucket of keywords going down or staying the same and the keyword anomalies, it certainly seems like there is more going on than just Google adding mobile search volume

Click to Tweet

Rather, it seems more likely that Google has taken the opportunity of rolling out a new tool to tweak the way they calculate search volume, adjusting more inputs than just adding mobile searches. And, the data and anomalies suggest that there is still more tweaking to do. Count on additional adjustments occurring over the coming weeks.

So what does all this mean for you?

As we mentioned above, take the opportunity now to evaluate for yourself what has changed in your own landscape. Look at your keywords by sorting on largest percent change so as to extricate those keywords with the largest before-after delta.

Users of Conductor’s platform can easily access historical Google Monthly Search Volume and determine which keywords have experienced the greatest percent change. And, Searchlight’s advanced filtering functionality provides a deeper level of search volume analysis.

image of keyword planning tool

Once that’s done, some may have some decisions to make.  Say you’ve been focusing on keywords x, y and z. Now your analysis shows that keywords a, b, and c have a higher search volume than x, y and z.  What to do?

First, I wouldn’t make any hasty changes in strategy. As we pointed out, we expect Google to make continued adjustments in the post launch wake. Second, if over time you do find that the new search volumes do hold steady, avoid tunnel-visioning on search volume as a sole determinant.  Instead, look closely at metrics like conversions, and on-page time to determine which keywords give you the best bang for your buck and focus there.

At the end of the day, it’s Google’s world. We just play in it.

31 Responses to Something’s Fishy in the State of Google’s New Keyword Tool [Data]

  1. Just did a search on my phone for “printers and copiers” and got a bunch of shopping results. Not something I’d think people would buy on the go? “Coffee” of course makes sense.

    Thanks for the analysis. Good data to keep in mind during the transition.

    • yea, coffee makes a bit more sense than the others, although even with adding mobile searches the % change is still disproportionate to other searches we saw that would also seem to go up significantly with the addition of mobile.

    • Anonymous says:

      Printers and copiers makes sense if people are looking for printers to print documents on the go. Between autocorrect and predictive search results, this one seems to get garbled for me more than any other mobile search either, so I actually feel pretty comfortable saying that this is largely people looking for a place to print documents on the go that gets horribly “corrected” either by the phone itself, or by google search.

  2. Adam Dince says:

    Once again, Nathan Safran proves why he’s the bee’s knees of market research

  3. MJField says:

    Great post Nathan, your last point is the main takeaway from this. Even if search volumes are more accurate, the key decision is still understanding which ones work best for you.

  4. Jake says:

    Keyword Planner, Enhanced Campaigns and their escalating attack on organic SEO are all revenue plays, plain and simple. It’s all about squeezing every last penny out of PPC spenders.

    Great read here drilling down into some relevant data. I miss the old keyword tool already.

  5. Arnie says:

    This is going to seem like a very basic question, but when you ran your research on the 550K keywords, was it always for the exact match of those words?

    • John says:

      I don’t think you can do an exact match search anymore. When you choose exact match it has no affect on the search volume compared to broad or phrase. None that I can figure out anyway.

      • bob says:

        i thought it defaulted to exact match

        • Arnie says:

          Yes – I believe it does default to exact match.

          • It only supports Exact Match. Previous KW tool supported all three but the new one only gives volume estimates using Exact Match. That is why volumes don’t change when you change the match type of the keywords. Their help page explicitly states that they only do exact.

            From what I understand, while the KW tool is dead, the AdWords API still uses the “old” metrics. However that will be cut over in the very near future to the new system of calculating volumes.

  6. Rick Noel says:

    “At the end of the day, it’s Google’s world. We just play in it.”

    So true. I really appreciate your thoughtful analysis and insights Nathan. Your advice to not make major strategy shifts, especially in SEO, is a good one.

    The actual search volume has not changed since the external keyword tool was recently retired, just the Google window into the data and the processing of that data (obviously black box) has changed.

    We recommend using data from other sources, like Bing, Wordstream, Word Tracker, etc. to help triangulate the data and see if the Keyword Planner Number is more believable than Keyword Tool data was.

    Mobile is now 17% of web traffic (according to StatCounter) which can be used to swag how much is mobile and what the volume uptick should look like. though mobile search traffic % will vary by search. Some traffic is surely being lost to mobile apps which will continue as smartphone penetration continues to grow.

    The Gremlins will be worked out in time. Thanks for sharing.

  7. Adegboye says:

    Thanks. Wonderful job. At the end of the day,its really Google’s world,we only learn to play in it.

  8. HP says:

    The tool always makes me weary if big G is controlling which keywords I market for. It seems like everytime I find a keyword not being targeted by my competitors does not take but a couple of days and the CPC skyrockets.

    I am in the in-house webmaster for our company and know for a fact that our competitors do not have a dedicated IT person and may not even be using a marketing service. Big G is Fishy why would’nt PPC is their bread and butter in the end.

  9. The referenced Google support page actually explains the 8% of keywords decreasing:
    “There’s quite a bit of overlap between broad and phrase match keywords, and the search volume statistic in Keyword Tool didn’t take that overlap into account.”
    Meaning they removed the overlapping numbers.

  10. Jonathan says:

    Great post and a really good look at the data. The anomaly of 1000+ search terms dropping to zero simply backs up your last point – there is no point in changing your SEO strategy based on this as the numbers didn’t suddenly fall to zero.

    We’ve always explained to clients that ‘-‘ doesn’t mean that no one ever searches for that term, it’s simply that Google is reflecting the numbers it wants to show in the tool. Now the tool has changed (are we the only ones’ to find it really hard to use???) we’re back to square one which is ‘how believable are the numbers’? We’ve never trusted them fully and still don’t.

    So as Rick says above, use a range of tools and make your own decisions on which keywords matter. Above all Nathan nailed it with “look closely at metrics like conversions, and on-page time to determine which keywords give you the best bang for your buck and focus there”

    Who ever said this was an easy job!!

    • Hey Jonathan–thanks for the comment. From what I’ve seen, it seems like you are not alone in not caring for the new tool.

      • Jonathan says:

        Hi Nathan,

        As far as we can see the new tool is designed to make sure you research and build for PPC and it links nicely with recent changes on the SERPS where more of the prime real estate is taken up with paid channels. It is getting harder to compete organically in this environment and with Google seemingly becoming more and more expensive a lot of our clients are looking for alternatives.

        Facebook advertising anyone? 😉

  11. Wolf Toni says:

    The old tool was easier to use the newone. I hope the will develop it furthermore.

    • Michelle says:

      You’re just used to the old tool. Once you use the new tool for a few days or so, you’ll notice it’s actually better and easier to use than the old version.

  12. mvarga says:

    There is no doubt in your analysis. But my experience is:

    1. Mobile users use more general and head keywords. Therefore the increase of head keywords can be explained by this behaivour
    2. Keyword planner offer different match types – have you added all the kewords in all matchtypes?
    3. Suggest and preview offer the possibility to click on the result by typing only a partl of the keyword – e.g. you type ‘faceb’ and get the results you are able to click. Therfore the keyword ‘facebook’ should be combined with others like face, faceb, facebo, faceboo, facebook.
    4. Keyword planner is a tool that use 2 old tools – keyword tools and traffic estimator
    5. There is a strong correlation with the bid and budget – have you test this?

    Very intersting post and I red it thoroughly twice. Maybe once again 🙂

  13. Great article, Nathan! And released at just the right time.

    A fellow copywriter and I were just scratching our heads wondering how the monthly searches for a target phrase could have dropped so significantly in less than a month (from 8,100 monthly searches in the Keyword Tool to 1,000 queries in the Keyword Planner).

    Your article has given us some peace of mind that this drop may be a tool glitch… and not us losing our minds.

    Really good word.

  14. James Abbott says:

    Thank you for publishing this detailed analysis Nathan – very valuable. Now I’m wondering: what would be my go-to source for honest, non-skewed keyword research? Thanks again.

  15. Spook SEO says:

    It’s far from perfect but google’s keywords tool is still the best available option, so thanks for the detailed review Nathan, at least we have a better idea what kind of bias we will run into when using this tool

    • Nathan Safran says:

      I agree that it is far from perfect, but knowing what we have going in is valuable and it helps to use multiple data sources too.

  16. Abdul Rehman says:

    glad to learn from here.

  17. chimezie nwatarali says:

    I agree that this is odd. Especially the coffee searches skyrocketing, as well as espresso maker and printers and copiers. I am wasn’t sure how relevant this article still is, since Google must have done some updates, so I checked out the search volumes for the keywords that increased in search volume in this article.

    Coffee: 550,000
    Espresso Maker: 12,100
    printers and copiers: 1,600

    The numbers are reduced but it looks like it is still a significant jump in the last 2 years.