5) Internal Link ScoreĪ useful way to evaluate and improve internal linking is to calculate internal PageRank of URLs, to help get a clearer understanding about which pages might be seen as more authoritative by the search engines. You can also now supply the XML Sitemap location into the URL bar at the top, and the SEO Spider will crawl that directly, too (instead of switching to list mode). The new Sitemaps tab and filters allow you to quickly analyse common issues with your XML Sitemap, such as URLs not in the sitemap, orphan pages, non-indexable URLs and more.
You can select to crawl XML Sitemaps under ‘Configuration > Spider’, and the SEO Spider will auto-discover them from robots.txt entry, or the location can be supplied. It’s always been possible to crawl XML Sitemaps directly within the SEO Spider (in list mode), however, you’re now able to crawl and integrate them as part of a site crawl. It also allows the SEO Spider to use a single filter, or two columns to communicate a potential issue, rather than six or seven. It makes it easier at a glance to review whether a URL is indexable when reviewing page titles, rather than scanning columns for canonicals, directives etc. It makes it easier when you export data from the internal tab, to quickly identify which URLs are canonicalised for example, rather than having to run a formula in a spreadsheet. This was introduced to make auditing more efficient. This provides a reason why a URL is ‘non-indexable’, for example, if it’s a ‘Client Error’, ‘Blocked by Robots.txt, ‘noindex’, ‘Canonicalised’ or something else (and perhaps a combination of those). The reason for this is for simplicity, it helps to bucket and organise URLs into two distinct groups of interest.Įach URL will also have an indexability status associated with it for quick reference. This might differ a little from the search engines, which will index URLs which can’t be crawled and content that can’t be seen (such as those blocked by robots.txt) if they have links pointing to them.
#Screaming frog seo spider version 5.0 code
For the SEO Spider, an ‘Indexable’ URL means a page that can be crawled, responds with a ‘200’ status code and is permitted to be indexed. These two phrases are now commonplace within SEO, but they don’t have an exact definition. This is not the third biggest feature in this release, but it’s important to understand the concept of indexability we have introduced into the SEO Spider, as it’s integrated into many old and new features and data.Įvery URL is now classified as either ‘ Indexable‘ or ‘ Non-Indexable‘. We believe this can be an extremely powerful feature, and we’re excited about the new and unique ways users will utilise this ability within their own tech stacks. This also allows running the SEO Spider completely headless, so you won’t even need to look at the user interface if that’s your preference (how rude!).
#Screaming frog seo spider version 5.0 how to
You can read the full list of commands that can be supplied and how to use the command line in our updated user guide. It behaves like a typical console application, and you can use –help to view the full arguments available.
This includes launching, full configuration, saving and exporting of almost any data and reporting.
You’re now able to operate the SEO Spider entirely via command line. 2) Full Command Line Interface & –Headless Mode The keen-eyed among you may have noticed that the SEO Spider will run in headless mode (meaning without an interface) when scheduled to export data – which leads us to our next point. This should be super useful for anyone that runs regular crawls, has clients that only allow crawling at certain less-than-convenient ‘but, I’ll be in bed!’ off-peak times, uses crawl data for their own automatic reporting, or have a developer that needs a broken links report sent to them every Tuesday by 7 am. You can also automatically save the crawl file and export any of the tabs, filters, bulk exports, reports or XML Sitemaps to a chosen location. You’re able to pre-select the mode (spider, or list), saved configuration, as well as APIs ( Google Analytics, Search Console, Majestic, Ahrefs, Moz) to pull in any data for the scheduled crawl. You can now schedule crawls to run automatically within the SEO Spider, as a one off, or at chosen intervals. In our last release, we announced an extremely powerful hybrid storage engine, and in this update, we have lots of very exciting new features driven entirely by user requests and feedback. We are delighted to announce the release of Screaming Frog SEO Spider version 10.0, codenamed internally as ‘ Liger‘.
0 kommentar(er)
