In today’s web, various and very different spiders can crawl your website. Each different spider can look for different meta tags.
Let’s take a look at how Google, Google’s AMP feature, Bing/Yahoo, Facebook, Twitter, and Pinterest look at your website’s structure and metadata.
Data Types
Some data can be obtained from the context of the page.
For example, Facebook is good at finding images for when Open Graph data is missing.
Another example is that Google can generate a suggestive description determined by the search query, while completely ignoring what you’ve put in the related description meta tag.
To determine what’s important for you, take a look at the table below.
The more pluses, the higher priority it has. So Twitter will first look for Twitter Card and then Open Graph.
AMP | Bing | |||||
---|---|---|---|---|---|---|
Title | +++ | ++ | +++ | + | + | + |
Description | ++ | ++ | +++ | + | – | + |
Open Graph | ※ | – | – | +++ | ++ | +++ |
Twitter Card | – | – | – | – | +++ | – |
JSON-LD | ++ | +++ | ++ | – | – | – |
Microdata | + | ++ | + | – | – | – |
※ Google seems to sometimes use Open Graph images within Google Images.
There are other factors that come into play for ranking. Like the sitemap, your website’s structure (like mobile accessibility), your site’s security, and performance.
Some spiders are very sensitive to errors, like Twitter. Others can automatically correct or even suggest better data when its output is erroneous.
Now, I do not have a lot of experience with Pinterest, so I can’t say how they handle errors.
AMP | Bing | |||||
---|---|---|---|---|---|---|
Robots | +++ | +++ | +++ | + | + | ? |
Sitemap | ++ | – | +++ | – | – | – |
Accessibility | ++ | ++ | – | – | – | – |
Performance | ++ | – | + | – | – | – |
Security | ++ | ++ | + | – | – | – |
Error correction | +++ | +++ | +++ | +++ | + | ? |
Testing data
To make sure all your data is rendered correctly, you should use the following tools.
Testing HTML
A good WordPress theme shouldn’t bring any HTML errors. Good HTML code ensures your website is displayed correctly on many different browsers.
Now I’ve seen sites with lots of errors ranking very well, so do take the test with a grain of salt.
Testing Structured Data
When your web pages include structured data markup, Search Engines can use that data to index your content better, present it more prominently in Search Results, and use it in several different applications.
You can test if the output is working well by using Google’s Structured Data Testing Tool.
A better analogy for this is using the cached version through Google Webmaster Tools. But you’d have to add your website first.
Testing Open Graph Data
Open Graph is used by many social networking sites, like Facebook, Twitter and Pinterest.
Those websites use this data to markup the link when shared with pictures or even a video, a title and excerpt, the author, and any internally linked pages.
Facebook is the leader in this type of data, and they have provided a tool for this.
Please note that the Twitter Card Validator also listens to this data, but more on that in the next section.
Facebook tends to maintain a long-term cache of your pages. With the Open Graph Debugger, you can not only validate your pages, but you can also ask Facebook to refresh their caches. You can do this by hitting “Scrape Again” after validating your URL.
Testing Twitter Cards
Twitter Cards can be displayed in.. well… Twitter. They’re also used in embedded Single Tweets and embedded Timelines that can be displayed on your website.
The WordPress Core team has done awesome work using oEmbed to include various things right in your WordPress posts. So all this data can loop back.
You can test your Twitter markup at the link below.
Testing AMP Data
When you’ve signed up for Google Webmasters’ Search Console, you can test the markup in their AMP Reporting tool.
If you have a direct AMP link to your website (often ends with /amp/
), then you should also test it in the Structured Data Testing Tool, Open Graph Debugger and the Twitter Card Validator.
There is also a live testing tool for when you’re actively testing data.
Testing General Meta Data
As with testing cached AMP Data, you’d have to sign up for Google Webmasters.
Then you can view any errors in their HTML Suggestions Reporting tool.
This data is particularly handy for testing duplicated content and descriptions. Do check in every now and then so you can fix these.
That’s it!
For now, at least. I’ll add more tests in the future.
Feel free to link your go-to tool, be it for performance, security, data, etc. in the comments below.
habs
Thanks :) that was very helpful.
Rutledge Daugette
Thanks for the info, the graphs really shed some great light on how important certain things are to each search engine and social site. Really interesting!
Christopher Baker
I’ve just started using this plugin and now looking around for configuration information and tools. Thank you for your hard work and thank you again for bringing us this useful compilation of testing tools and data. Long live Open Source! :D
Ravi
Hi Sybre Waaijer,
Your plugin has huge potential and I really like it. the one main issue that I found in your plugin is that you haven’t given any option to noindex or nofollow to all or multiple posts, you have these options to others like category or tags but not in post. kindly add this option in your next release also. This may not look wise to noindex or nofollow to all your web post but it does help sometime..
Thanks
Sybre Waaijer
Hi Ravi,
The feature you’re requesting is planned, see https://github.com/sybrew/the-seo-framework/issues/155.
It doesn’t have a milestone yet because we’re limited by the HTML structure of the settings.
So, it won’t be in the next update, but it will be available somewhere in the 3.x release cycle.