R’ can be very useful for SEO. Here are some examples of how it can be used:

Web Crawling With R’

you can crawl a website and automatically check the technical characteristics of the web pages: title, metadata, but also: counting the number of words, compute website internal page rank or extracting URLs from XML sitemaps.

There are many packages to help you:

  • Check “Rcrawler” and “rvest” package for crawling
  • Check “tidytext” package for words counting
  • Check “igraph” package for iPR
  • Check “xsitemap” package for XML sitemaps (mine!)


here is 2 tutorials that’s have made, that might be usefull:

Use R’ to crawl XML sitemaps

Crawling with R’ using rvest package

[end of edit]

Gather data from SEO tools API

Many SEO tools like semrush, ahref, botify, OnCrawl, adwords, … have APIs. With R’ you can request data in batches, for example, to process thousands of keywords or check the quality of a hundred domain names all at once.Not all APIs are easy to use.
Check Rémi Bacha github you’ll find several interesting packages that help do this kind of things.

Data Crunching

If you have a big website, you can uncover interesting information from Googlebot web server logs. R’ can help you cross-check them with other data sources.
All the “tidyverse” packages are very handy to deal with these big datasets

Other examples and ideas in bulk: keyword clustering, duplicat content detection, webpage language detection (using letter n-grams), automatic detection of the SERP intents …

* *