useful SEO XPath's & CSS selectors X
⚠️ THIS IS A WORK IN PROGRESS
Last updated
⚠️ THIS IS A WORK IN PROGRESS
Last updated
If you are crawling website for SEO purposes, whether it is with rcrawler, rvest or another way. You will probably need to use some CSS/Xpath selector to extract the useful bit from each page. here is a collection of the most useful ones. Feel free to reach out if you think it misses important ones.
Value
xpath
CSS
Robots (Index/Noindex)
//meta[@name='robots']/@content
head > meta[rel="robots"]
canonical tag
//link[@rel='canonical']/@href
head > link[rel="canonical"]
Page Title
//title
head > title
Meta Description
//meta[@name='description']/@content
head > meta[name="description"]