🤖
R for SEO
  • Using R for SEO, What to expect?
  • Getting started
  • What is R? What is SEO?
  • About this Book
  • Crawl and extract data
    • What's crawling and why is it useful?
    • Download and check XML sitemaps using R'
    • Crawling with rvest
    • Website Crawling and SEO extraction with Rcrawler
    • Perform automatic browser tests with RSelenium
  • Grabbing data from APIs
    • Grab Google Suggest Search Queries using R'
    • Grab Google Analytics Data x
    • Grab keywords search volume from DataForSeo API using R'
    • Grab Google Rankings from VALUE SERP API using R'
    • Classify SEO Keywords using GPT-3 & R'
    • Grab Google Search Console Data x
    • Grab 'ahrefs' API data x
    • Grab Google Custom search API Data x
    • Send requests to the Google Indexing API using googleAuthR
    • other APIs x
  • Export and read Data
    • Send and read SEO data to Excel/CSV
    • Send your data by email using gmail API
    • Send and read SEO data to Google Sheet x
  • data wrangling & analysis
    • Join Crawl data with Google Analytics Data
    • Count words, n-grams, shingles x
    • Hunt down keyword cannibalization
    • Duplicate content analysis x
    • Compute ‘Internal Page Rank’
    • SEO traffic Forecast x
    • URLs categorization
    • Track SEO active pages percentage over time x
  • Data Viz
    • Why Data visualisation is important? x
    • Use Esquisse to create plots quickly
  • Explore data with rPivotTable
  • Resources
    • Launch an R script using github actions
    • Types / Class & packages x
    • SEO & R People x
    • Execute R code online
    • useful SEO XPath's & CSS selectors X
Powered by GitBook
On this page
  • What's Selenium?
  • Why Selenium is an interesting solution?
  • Let's start
  • How to quickly configure a Selenium scenario?

Was this helpful?

  1. Crawl and extract data

Perform automatic browser tests with RSelenium

PreviousWebsite Crawling and SEO extraction with RcrawlerNextGrab Google Suggest Search Queries using R'

Last updated 3 years ago

Was this helpful?

What's Selenium?

Selenium is a classic tool for and it can help perform automatic checks on a website. This is an intro to how to use it

Why Selenium is an interesting solution?

One of the great advantages of using Selenium is that you can alternate automatic and manual actions in the same session. For example, you can log on somewhere and run an automatic script after pretty easily or… fill in a captcha and run your script.

Let's start

The first step is, as always, to install and load the RSelenium package

#install to run once
install.packages("RSelenium")
library(RSelenium)

We’ll launch a selenium server with a Firefox browser in a controlled mode. It will take quite some time the first time but after it will load in a few seconds.

here is the R command:

rd <- rsDriver(browser = "firefox", port = 4444L)

Then we’ll grab the instance to be able to control our browser

remDr <- rd[["client"]]

It’s now possible to send action to our browser. To open a website URL just type

remDr$navigate("http://www.bbc.com")

You will notice the robot head icon which means that it is a remote-controlled browser

Here are some useful commands:

# find a dom element using the class selector and grab inner text
remDr$findElement(using = "class", value ="top-story")$getElementText()
 
 
# find a dom element using a class selector and click on it
remDr$findElement(using = "class", value ="top-story")$clickElement()
 
 
# get h1 textusing a tag selector
remDr$findElement(using ="tag", value = "h1")$getElementText()
 
 
# refresh browser
remDr$refresh() 

When you are done with it, don’t forget to

# close browser
remDr$close()
 
 
# stop the selenium server
rd[["server"]]$stop()
 
# and delete it
rm(rd)

Otherwise, it’s gonna be a mess when you’ll get back on it

How to quickly configure a Selenium scenario?

At the end of the process, it should open a firefox window like this one

You can check the to learn all the functions to find DOM elements and actions but there is a quicker way:

Use a chrome extension like , record and copy/paste the instructions directly

documentation
Katalon recorder
QA