can be very useful for SEO. Here are some examples of how it can be used:
Web Crawling With R’
you can crawl a website and automatically check the technical characteristics of the web pages: title, metadata, but also: counting the number of words, compute website internal page rank or extracting URLs from XML sitemaps.
There are many packages to help you:
- Check “Rcrawler” and “rvest” package for crawling
- Check “tidytext” package for words counting
- Check “igraph” package for iPR
- Check “xsitemap” package for XML sitemaps (mine!)
here is 2 tutorials that’s have made, that might be usefull:
[end of edit]
Gather data from SEO tools API
Many SEO tools like semrush, ahref, botify, OnCrawl, adwords, … have APIs. With R’ you can request data in batches, for example, to process thousands of keywords or check the quality of a hundred domain names all at once.Not all APIs are easy to use.
Check you’ll find several interesting packages that help do this kind of things.
If you have a big website, you can uncover interesting information from Googlebot web server logs. R’ can help you cross-check them with other data sources.
All the “tidyverse” packages are very handy to deal with these big datasets
Other examples and ideas in bulk: keyword clustering, duplicat content detection, webpage language detection (using letter n-grams), automatic detection of the SERP intents …