We’ve all heard about Google’s Search Quality Raters Guidelines.
But let’s be honest: there’s a lot to digest in this 168-page document.
If you’ve been looking for a basic guide to Google’s Search Quality Raters Guidelines, then keep reading.
In this post, you’ll learn what is covered in Google’s Search Quality Raters Guidelines, how it impacts search, and what you need to know about the concept of E-A-T, which stands for:
Let’s dive in…
What Are the Search Quality Raters Guidelines?
Google’s Search Quality Raters Guidelines are a document given to people that Google employs for the sole purpose of rating websites.
The guidelines outline the conditions and elements that need to be considered, and how the site should be rated, by that person.
In essence, Google has hired many thousands of people to rate websites and record whether the site is good or bad across a variety of areas.
Now, it is important to understand that these people have no impact on the rankings of the sites they rate.
A positive or negative rating by Quality Rater will not directly impact your position on Google.
One can argue (and I do) that their role is so much larger than that.
They don’t influence the rankings of the sites they rate.
They influence the rankings of every site.
How Does That Work?
I need to be clear here, I can’t pick up my Bat Phone and get confirmation on exactly how the algorithms work, directly from Google.
Like everyone else, I pay attention to what they say and make best guesstimates on how certain tasks will be best accomplished.
The most likely structure to make the feedback of ~10,000 quality raters actionable is to do it in the way that Google does best: algorithmically.
In the backend of the system, the raters are using a slider to assign values:
This is then made available to machine learning systems that use this to augment the algorithms based on known signal data.
For example, if a site or group of similar sites are consistently rated High or better, the system would review all the signal data from the site(s) to look for commonality.
The same would be true for site(s) with Low values.
With this, the system would likely:
- Produce a set of results based on what the new algorithm produced across a variety of phrases and niches.
- Send the top ranking sites in that set to the raters.
- And, assuming it is better, put that into the global algorithms we all know and love.
So, while a Quality Rater does not impact an individual site’s ranking, they do impact the entire web.
Also Worth Noting
When you see a new SERP feature, or another change in how results are structured or displayed, it’s been put past the Quality Raters.
If something is deployed that you don’t like, you probably have these folks to blame. 😉
What Are They Looking For?
Before we answer that let me be clear:
This article is meant to be a beginner’s guide.
As such, we are going to cover this area topically in hopes of providing a global understanding of what is going on.
Once you’ve completed this article and have the foundation, I highly recommend:
- Reading Our Search Quality Rater Tests from Google.
- Watching the following video of Marie Haynes being interviewed by Casey Markee for SEMrush.
They’ll round-off a great starting point.
What Is Google Looking For?
When we’re looking at the areas the raters are instructed to look at, we’re essentially:
- Looking at what Google wants the algorithm to produce.
- Getting a glimpse of what their algorithms will focus on.
The guide states:
“As a Search Quality Rater, you will work on many different types of rating projects. The General Guidelines primarily cover Page Quality (PQ) rating and Needs Met (NM) rating; however, the concepts are also important for many other types of rating tasks.”
We won’t be looking at the tasks specifically here and will focus on the more important (from the context of this piece):
- Needs Met
- Page Quality
What do they mean?
Needs Met is a fairly straightforward concept… it means intent.
The question that raters would be asking themselves in assessing it is:
How helpful and satisfying is this result?
During this testing, a rater may visit a single page or visit a search results page and rate every result.
Either will send the signal to Google about the site structure, device, demographic, and location results differences, and I’m sure a number of other factors that apply to the grading of each result (there’s a reason they have more than 10,000 raters around the world).
This data will then be used to drive changes to improve the results to algorithmically determine which signals or signal combinations are common to the higher rankings results.
It’s important to note that as with the real world, the Needs Met rating does require decent Page Quality. In fact, the guidelines state it clearly:
“The Needs Met rating is based on both the query and the result.”
One can have a medium Needs Met with a low Page Quality, but it would be highly unlikely that they could get a high Needs Met rating.
After all, the user intent it’s satisfied if the searcher doesn’t trust the result.
One final element of Needs Met that is worth noting before we move on to Page Quality is the interpretation.
By this, we are to consider queries with multiple possible meanings such as the following, from the examples they give:
When the rater is assigning a Needs Met score, they are to give more weight to pages satisfying higher intents.
- A “Highest” rating being given to an authoritative piece on Apple, Oklahoma.
- Training algorithms to focus on the wrong signals for the majority of users.
Page Quality ratings are based on a number of factors, all of which interconnect (almost like a Google algorithm, right?).
And the weight given to each is based on the type of site and query (again … the similarity is uncanny).
The key ones noted in the guidelines are:
Your Money or Your Life (YMYL)
As Google words it, a YMYL site is one that:
“… potentially impact a person’s future happiness, health, financial stability, or safety.”
With YMYL sites, raters are directed to put more weight on E-A-T.
The guidelines categorize them as:
- News and current events: News about important topics (e.g., international events, business, politics, science, technology). Keep in mind that not all news articles are necessarily considered YMYL (e.g., sports, entertainment, and everyday lifestyle topics are generally not YMYL). Use your judgment and knowledge of your locale.
- Civics, government, and law: Information important to maintaining an informed citizenry, such as information about voting, government agencies, public institutions, social services, and legal issues (e.g., divorce, child custody, adoption, creating a will, etc.).
- Finance: Financial advice or information regarding investments, taxes, retirement planning, loans, banking, or insurance, particularly webpages that allow people to make purchases or transfer money online.
- Shopping: Information about or services related to research or purchase of goods/services, particularly webpages that allow people to make purchases online.
- Health and safety: Advice or information about medical issues, drugs, hospitals, emergency preparedness, how dangerous an activity is, etc.
- Groups of people: Information about or claims related to groups of people, including but not limited to those grouped on the basis of race or ethnic origin, religion, disability, age, nationality, veteran status, sexual orientation, gender, or gender identity.
- Other: There are many other topics related to big decisions or important aspects of people’s lives which thus may be considered YMYL, such as fitness and nutrition, housing information, choosing a college, finding a job, etc.
Most people don’t think of shopping when they think of YMYL … but it’s in there.
According to the guidelines, the sections of a website can be classified into three main categories:
- Main Content (MC): Main Content is any part of the page that directly helps the page achieve its purpose.
- Supplemental Content (SC): Supplementary Content contributes to good user experience on the page but does not directly help the page achieve its purpose. The example they give is navigations links. Critical to the site, but not necessary to satisfy Needs Met.
- Ads: s/Monetization (Ads) is content and/or links that are displayed for the purpose of monetizing (making money from) the page.
Following their direction, understanding a webpage is quite simple.
- What’s outlined in blue in the screenshot below is Master Content.
- What’s outlined in red in the screenshot below is Ads.
- Everything that remains is Supplemental Content.
The ease-of-access and volume of the Main Content play their parts in Page Quality calculations.
It’s what assists the rater value on not just whether the need is met, but also how easily supplemental content is accessed, should it be desired.
E-A-T is certainly the most talked about section of the guidelines.
Before we dive into E-A-T, it’s incredibly important to understand one thing:
E-A-T is not a ranking factor.
Say it with me:
E-A-T is what the raters are looking for.
Raters use E-A-T to assist them in how they are rating websites.
It is in no way a part of any algorithm.
Here’s a tweet from Google’s Danny Sullivan that explains it well:
Our systems aren't looking for EAT. Our raters are using that to see if our systems are working well to show good information. There are many different signals that, if we get it right, align with what a good human EAT assessment would be. See also: https://t.co/1fs2oJ9Gtl pic.twitter.com/GBbnYEjJUV
— Danny Sullivan (@dannysullivan) February 19, 2020
It is not part of the algorithm.
The goal is that if the raters use it to judge websites, and Google uses their ratings to adjust their algorithm, in the end, the algorithm will align with the E-A-T principles.
So, E-A-T can well be used as a guiding principle for design, content creation, and supporting external signals.
You can’t optimize for it specifically.
Without getting into too much detail, they are basically:
The expertise of the content creator. This is related only to the content of the page being judged, and not the site as a whole.
It does not have to be an author and the criteria are not set in stone.
When looking for information on lung cancer, an in-depth study on causes, impact, statistics, etc. from an important medical school or institution or government agency would likely get a high score.
If the searcher was looking for information on what’s it’s like to live with it, a personal account from a bank teller who’s partner lived with it for years would fulfill the Expertise criteria exceptionally.
It may be on an important site, or it may be an exceptional member of a forum answering questions.
It’s all about context and addressing with expertise, the need to be met.
The authority of the content is judged by the authority of the content itself as well as the domain.
In general, this would be based on external signals such as links and link quality, brand mentions, citations, etc. both to the content specifically, and the domain as a whole.
When thinking of authority I can’t help but think of a patent granted in 2015, Ranking Search Results Based On Entity Metrics.
In it, they discuss the following key metrics:
- Relatedness: How related are two entities?
- Example: Empire State Building and Skyscrapers
- Notability: How notable is an entity in its domain?
- Example: SEMrush is more notable in the SEO tools than the software domain
- Contribution: How is an entity viewed by the world?
- Example: Does it have critical reviews, fame rankings, etc.?
- Prizes: Has the entity received prizes?
- Example: The types of awards and prizes an entity has received.
While there are certainly other characteristics, I find these helpful guides.
The trust of the content is judged again by the trust of the content specifically, as well as the trust of the domain.
Trust is similar to authority, but more pointed.
Where authority focuses more on the volume of quality references, trust focuses more on specific signals and sites.
An example they use in the guidelines is the BBB.
While they don’t talk about it being used as a positive signal, they do note that a bad rating based on a significant volume of users could be used as a negative.
If you’re just getting your feet wet, or just looking for a different point of view, I hope you’ve found this helpful.
I’ve tried to avoid getting into strategies around the subject, as that is relative to each niche, starting point, and current E-A-T.
This is an important document. It tells us where Google wants to go, and they’re throwing a lot of resources at it.
Below is some additional reading to help you further.
- Google’s E-A-T: Busting 10 of the Biggest Misconceptions
- E-A-T & YMYL: How to Boost Your On-Page SEO
- How to Use Structured Data to Support E-A-T
- Quality Raters Guidelines and Google Ranking Signals
- How Does Google Use Quality Raters?
All screenshots taken by author, April 2020
In-Post Images: Adobe Stock / Modified by author, April 2020