Using N-Gram Analysis to Optimize SEM Account Performance

As paid acquisition marketers, noticing trends and patterns in data are key aspects of optimizing performance; however, sometimes the data sets we work with are large and complex, which can make this very challenging. This is especially true when it comes to analyzing keywords and search terms in paid search advertising, where there might be hundreds of thousands, if not millions, of long tail keywords and search term permutations.

To make things worse, the keyword-level metrics in these large data sets are often too small and insignificant to draw any meaningful insights to act upon. For example, you might have a target CPA of $250 for a client’s account, but have thousands of search terms that have spent less than $2.00 each. What can you do in a situation such as this? In order to get actionable insights from a data set that’s spread over many different keywords each with very little data, it requires aggregating similar keywords to the point where it becomes significant. This allows you to maintain the signal and integrity of the underlying data while arriving at a more significant sample size which you can make decisions off of.

In this article, we will discuss specifically how you can accomplish this data aggregation for large search term data set using n-grams .

What is An N-gram?

Before we dive into our example, first let me explain what an n-gram is. An n-gram is a fixed length neighboring sequence of n items in a text or speech.

For example, given n = 3, the text “live in a van down by the river,” the 3-gram would be “live in a,” “in a van,” “a van down,” “van down by,” “down by the,” and “by the river.” In this example, the n-gram sequencing was done manually; however, this process can be done in multiple more efficient ways such as through Power Query in Excel or from Google Ads scripts.

How does this apply to SEM though? In SEM, n-gram analysis is useful for accomplishing three main activities for large search term data sets with sparse data at the search term level:

  1. Finding new keyword themes or permutations to add to your account.
  2. Determining which keywords should be added as negative keywords.
  3. Applying bid adjustments to a group of keywords based on the collective underlying search term performance data.

Example Case

To better understand the value of n-gram analysis, let’s walk through a hypothetical search term data set for an e-commerce company that sells various types of shoes online. Their goal is to sell 90 pairs of shoes per month at a maximum CPA of $25. Below is a snapshot of some of the search terms and associated performance data for this company over the past few months:

(Table 1: Snapshot search term data)

Immediately, there are search terms that should be either added as negative exact match keywords or receive lower bids due to the CPA relative to goal. For example, “shoes for leisure,” “tennis shoes for exercise,” and “tennis shoes for leisure” all should be taken action on because their CPAs are much higher than our target CPA of $25. If we simply added these search terms as negative exact match keywords, this would result in 92 conversions at a $10.02 CPA if we applied this retroactively. Perfect – the goal has been achieved!

However, we want to do better than that. And we can, through the power of n-grams.

Optimizing with Granular Data

At the simplest level, we have a 1-gram data, as seen below.

(Table 2: 1-gram of snapshot search term data)

Individually, the search terms with the word “lifting” had less than $13 of spend each if you refer back to table 1 above, but in aggregate we can see that the search terms with “lifting” have spent $73 and not led to a single conversion. This is a great opportunity to optimize the account. Depending on your thresholds for taking action, this could warrant bid optimization, adding negative keywords, or testing new ad copy or landing pages for this specific theme to see if there are conversion rate lift opportunities.

Looking at the 1-gram data above, another group is search terms that include the word “for.” Adding in the insight of the 2-gram data (below), we can see this is due to search terms that contain the phrase “shoes for” and not “for sale.” This insight might cause us to break these terms out into separate ad groups that will get their own bids or tailored ads to maximize volume on the “for sale” terms and improve efficiency on the “shoes for” terms.

(Table 3: 2-gram of snapshot search term data)

Running N-Gram Analysis At Scale

While it may be easy to spot these new opportunities and optimizations in a snapshot view for smaller accounts, it would be much harder (if not impossible) to find these patterns and trends if there were thousands or millions of search terms. This is why the n-gram analysis model is useful for accounts that have a large number of search terms where each search term doesn’t have significant data.

Now, this may sound good in theory but actually running this analysis on an account with thousands of unique search terms would certainly be painful and time consuming to do manually. Instead, this process can be done with scripts or Power Query. Here is a simple and clean script   to run an n-gram analysis.

To run this script, the process is as follows:

1. Click into Tools > Scripts

2. Create a new script by pressing the (+) button

3. Name the script and copy-paste the script

4. Create a Google Sheet link with “Anyone with the link can edit” access

5. Change the spreadsheet URL to your Google Sheet URL

6. Save and authorize the script to run

7. Hit Run then Run Without Preview and wait for the output to appear in your Google Sheet

Wrapping Up

In this quick guide, we’ve covered what an n-gram is and how it can be useful for analyzing large search term data sets and drawing actionable insights to improve account performance. We then pointed you to a script that can be used to automate the process of running an n-gram analysis.

Now you will be able to derive unique and powerful insights that you may not have noticed before. Most importantly, you will be able to work with large sets of granular data to better optimize your paid search account and get maximum performance.