Better Metrics For Better Results: Keyword Research 2.0™

This page describes our keyword metrics in great detail - a brief description of our keyword analysis reports is also available.

For a website, the first stage in the sales cycle is bringing in new visitors. Whether your web site is designed to communicate, persuade, or sell, success depends on bringing in the right visitors at the right time. Even a small increase in the quality of traffic will pay off at every subsequent stage in the sales cycle.

When it comes to traffic, quality counts. The better you target your audience, the more successful your site will be. If the quantity of traffic were all that mattered, nobody would bother with search engine marketing. If you don't believe this, go to "BuyHitsCheap dot com," buy 10,000 website visitors for $25, and see how many sales you make. Didn't sell anything? Of course not!

Successful online marketing is all about targeting, or in search engine terms, keyword selection. With enough resources, you can compete for any search term, or at least buy paid listings. Successful search engine marketing means making the right selections, based on three factors: popularity, relevance, and competition.

Our keyword research reports are designed to give you the best possible data on all three of these critical factors. We are always looking for ways to improve, of course, but we believe we provide the best keyword metrics available today.

Measuring Search Term Popularity

There are four main sources that can give us information on the popularity of different search terms. None of them is perfect, because all of them are skewed in some way. Skewed data is caused by different things: bid management & rank checking applications that carry out automated searches, varying demographics between different search engines, and different definitions of the word "search."

• Wordtracker: Wordtracker's database of nearly 400 million searches represents the largest public database intended for keyword research. Demographics could be an issue here, because their data comes from two small Metasearch engines, Dogpile and Metacrawler. However, the relative values of related search terms should remain fairly consistent.
   
  Wordtracker's data is skewed slightly by rank checking software, but overall Wordtracker has been the most accurate public source. Wordtracker offers a free trial which provides a limited amount of data, but it's well worth buying a full subscription to get access to tools like "comprehensive search." At this time, we use Wordtracker as our primary source of data on keyword popularity.

 

• Keyword Discovery: Trellian's Keyword Discovery database has been part of our toolkit for a while, but until recently we didn't have a lot of confidence in the counts. Keyword Discovery announced a new keyword database in August 2006, which we are currently testing. This new database is built by analyzing queries from web browsers, and in theory should be immune to skew from rank checking. So far, it appears to be the largest source of reliable data on keyword popularity, with approximately 600 million user queries stored to date.
   
  We have been using Keyword Discovery for research on "seasonal" search queries, because they maintain a full year's worth of data. Once our evaluation of the new database is completed, we will likely switch to Keyword Discovery as our primary data source, and use Wordtracker as a backup.

 

• Overture Suggestion Tool: Overture's counts are, unfortunately, badly skewed. Bid mananagement applications and rank checking both tend to skew counts higher. To make matters even worse, Overture's distribution partnerships can cause even larger inaccuracies.
   
  For example, many searches for popular words like "dating" actually come from pop-up search windows generated by the Gator/GAIN adware application, where no actual "search" has taken place. This problem affects popularity data from smaller pay-per-click search engines as well. The best thing about Overture's tool: it's free.

 

• Google Adwords: Google offers a free search term suggestion tool called the "Keyword Sandbox." Several features of this tool, by design, make it a less desirable tool for keyword research. The main issue is that Google does not provide search counts, instead providing estimates of the average click-throughs for a paid advertisement. Efforts to extrapolate the search counts from this tool have failed. We continue to research the relationship between Google's reported numbers and the actual popularity of search terms.
   
  It's likely that the reported click-through counts does provide some insight into the true value of search terms. The geographic targeting features allow researchers to discover regional variations in search term use, for example between the U.S. and the U.K. No other data source supports this kind of research.
   
  Adwords is also our preferred tool for search term targeting studies. In these studies, paid ads are used to examine the relevance and conversion rate for different search terms. A useful side effect is that we also get real data on the popularity of search terms across Google's network. Running an Adwords campaign costs money, but some of the cost can be offset by increased sales.


• Yahoo Site Match: Paid inclusion programs like Site Match provide valuable data to subscribers. We use reported data from these programs to help estimate the click-through rates for "organic" listings on search engines like Yahoo and MSN. Unfortunately, because Yahoo adds a per-click charge for every referral (without giving the advertiser the ability to use negative matching on irrelevant search terms), it's hard to justify participation in their progam.

Editorial note: yes, we are aware of many new tools including Keyword Intelligence, and a few others. While these are in many cases excellent tools, our conclusions have not changed as to the best sources of data on keyword popularity.

The Impact Of Keyword Relevance

Simply knowing the popularity of a search term isn't enough. You still don't know what they're searching for. The more generic a search term, the less likely it is to be 100% relevant for your web site. We use a simple formula called "weighted popularity" to help our customers identify their most targeted keywords.

With this formula, our clients estimate the relevance of search terms as a percentage. The popularity is multiplied by this percentage to give a weighted value that better reflects the number of targeted searches. Our usage instructions provide guidance on how to estimate the relevance of search terms. Even an imperfect estimate is better than assuming that every search term is equally relevant.

As mentioned above, results from pay-per-click campaigns provide some insight into keyword relevance, through click-through and conversion rates. This is an important area of study at SEO Research Labs, and we continue to seek better ways of measuring or estimating the targeting value of search terms.

How Many Sites Are Competing For Rankings?

One of the most inane expressions we've heard is "you're only competing against the top ten sites." This couldn't be further from the truth. Search engine positioning is a race that never ends. The current leaders may set the pace, but there are always others in the back of the pack trying to reach the front. How many "others" there are, and how fast they are moving, makes a big difference.

The current standard for measuring the level of competition is to check the number of matches on each search engine for the search term in question. Unfortunately, the number of results on a given search engine depends on the size of their database, how they present search results, and how often the individual keywords are used in everyday language. Doesn't it make more sense to look at the intentional competition?

The first thing anyone does to optimize a page for higher rankings is include the keywords in the page title. Well, Google not only has the largest index in the search engine world, they also offer an advanced search operation ("intitle:") that will tell you exactly how many pages have your keywords in the <TITLE> tag.

Anyone who is serious about competing for search engine rankings also takes one more step: they use the keywords in the text of incoming links! Google comes to the rescue again, with another advanced operator ("inanchor:") that shows how many pages have your keywords in their incoming links.

Combining these two measurements of intentional competition gives us the most accurate measurement of the level of competition, and this is what we provide on our keyword analysis reports. By querying Google for all pages that contain your keywords in the <TITLE> and incoming links, we can clearly establish the true number of competing pages. Since the same folks will be competing with you on all search engines, you need look no further than Google to identify the competition.

How Strong Is The Competion?

The immediate challenge is to get ahead of the current top-ranked sites. To get a realistic assessment of how strong the competition is, the best available measurement is the number of incoming links pointing to the top ranked sites. For many search terms, the link popularity scores will vary widely between the top 10, but it's safe to say that if you can't outlink any of them, you probably can't outrank them.

At SEO Research Labs, we use Alexa's paid data services to measure the link popularity of the top 10 sites, and provide this valuable (and expensive!) competition data for the 100 most popular search terms in each keyword report.

Keyword reports usually take 3-5 business days to complete. Our business days are Monday-Friday 8am-6pm, excluding U.S. Federal holidays. If you need more information, feel free to contact us, review the detailed description of our keyword reports, or learn about our process.

If you're ready to order a keyword report now:
Website owners: click here.
SEO/SEM consultants and site designers: click here.