• Skip to primary navigation
  • Skip to main content

data Rebellion

Learning through Adventure

  • Home
  • Blog
  • Beginner Python Course
    • Preface – The Journey Ahead
    • Prologue – The Door
    • Chapter 1 – Arithmetic and Variables
    • Chapter 2 – Strings and Lists
    • Chapter 3 – Conditional Statements
    • Chapter 4 – Functions
    • Chapter 5 – Loops
    • Chapter 6 – Built-in Functions and Methods
    • Chapter 7 – Imports and Nesting
    • Chapter 8 – Opening the Door
    • Epilogue – Only the Beginning
  • About
  • Contact
You are here: Home / Selenium / Using Firefox Extensions with Selenium in Python

Using Firefox Extensions with Selenium in Python

Updated March 25, 2021. Published September 11, 2018. 11 Comments

Firefox is my personal favorite browser, due in part to all of the great extensions available for it. When you try running Firefox with Selenium, however, you’ll probably find that Firefox is missing the extensions you have installed and normally use when browsing. Luckily, there’s a quick and easy way to install all your favorite Firefox extensions when using Selenium.

For example, let’s say we’d like to do a little light web scraping. To keep things simple, let’s just grab what’s trending off of Yahoo’s home page.

Screenshot of Yahoo's Home Page

You can see the top 10 trending subjects off to the right, starting with Beaufort County.

Selenium without Firefox Extensions

Here’s how we’d normally scrape that info:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from selenium import webdriver
geckodriver = 'C:\\Users\\Grayson\\Downloads\\geckodriver.exe'
 
browser = webdriver.Firefox(executable_path=geckodriver)
 
browser.get('http://www.yahoo.com')
 
trending_xpath = '//li[@class="trending-list selected"]/ul/li/a/span'
 
trending = browser.find_elements_by_xpath(trending_xpath)
 
# trending # and subject are separate elements, concatenate like so
for i in range(0, 20, 2):
print(trending[i].text + trending[i+1].text)
 
browser.quit()
1
2
3
4
5
6
7
8
9
10
1.Beaufort County
2.Faith Hill
3.Gretchen Carlson
4.Nicki Minaj
5.Taapsee Pannu
6.Cox Cable
7.Scott Disick
8.Airbnb Vacation Rentals
9.Brett Favre
10.Ally Bank

Great, that seems to work. But let’s say we’d prefer Firefox to be running with a couple of my favorite extensions, namely:

  • HTTPS Everywhere: Automatically enables HTTPS encryption on sites that support it, making for more secure browsing.
  • uBlock Origin: An efficient blocker that can make bloated web pages load much faster.

How do we get these extensions installed on Selenium’s instance of Firefox?

Getting the Necessary Information

First, we’ll need to find where those extensions are stored locally. Note that this means you’ll need to already have them installed on your machine for regular use of Firefox.

To find them, open up Firefox and navigate to the main drop down menu. Go to “Help”, and then “Troubleshooting Information”. Alternatively, you can get to the same place by entering about:support in your Firefox navigation bar.

In the “Application Basics” section click the “Open Directory” button, and in the file browser that pops up open the “extensions” folder. These are the extension installation files we’ll need to reference in our script. There should be a different “.xpi” file for every Firefox extension you have installed, and the file path to this folder should look something like “C:\Users\Grayson\AppData\Roaming\Mozilla\Firefox\Profiles\3rqg4psi.default\extensions”.

It might be difficult to tell which files correspond to which extensions based on the file names, as the file names are sometimes unintelligible. To get around this, go back to your browser and on the same page as before scroll down to the “Extensions” section. Here you’ll find a table that pairs each extension name with its corresponding ID, and the ID should be almost the same as the installation file name, lacking just the “.xpi” suffix.

In our case though, the extension and file names aren’t too hard to match:

  • HTTPS Everywhere: https-everywhere@eff.org.xpi
  • uBlock Origin: uBlock0@raymondhill.net.xpi

Selenium with Firefox Extensions

Now we just need to add a few lines of code to our original script to install these extensions. We’ll perform the installations right after we initialize the browser.

Run this script and see if you get the same results as last time. Look for the extension symbols near the top right of the browser. You should see the blue-and-white “S” symbol for HTTPS Everywhere and the reddish badge symbol for uBlock Origin.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from selenium import webdriver
geckodriver = 'C:\\Users\\Grayson\\Downloads\\geckodriver.exe'
 
browser = webdriver.Firefox(executable_path=geckodriver)
 
extension_dir = 'C:\\Users\\Grayson\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\3rqg4psi.default\\extensions\\'
 
# remember to include .xpi at the end of your file names
extensions = [
'https-everywhere@eff.org.xpi',
'uBlock0@raymondhill.net.xpi',
]
 
for extension in extensions:
browser.install_addon(extension_dir + extension, temporary=True)
 
browser.get('http://www.yahoo.com')
 
trending_xpath = '//li[@class="trending-list selected"]/ul/li/a/span'
 
trending = browser.find_elements_by_xpath(trending_xpath)
 
for i in range(0, 20, 2):
print(trending[i].text + trending[i+1].text)
 
browser.quit()
1
2
3
4
5
6
7
8
9
10
1.Beaufort County
2.Faith Hill
3.Gretchen Carlson
4.Nicki Minaj
5.Taapsee Pannu
6.Cox Cable
7.Scott Disick
8.Airbnb Vacation Rentals
9.Brett Favre
10.Ally Bank

So there you have it. We performed the same operation, but got to take our two favorite Firefox extensions along for the ride.

In addition to the peace of mind knowing that HTTPS security was used whenever possible, you may have noticed that our second script took significantly less time to load the page. This is because uBlock Origin blocked a number of unnecessary, resource-intensive requests, a great feature to have when you’re dealing with the slow, bloated web pages that are all too common nowadays.

Anyways, I hope this gives you a few ideas as to how you can make your life a little more convenient. Let me know if you have any questions, and happy automating.

Python Automation Project Ideas Ebook Front Page

Free Ebook: 88 Python Project Ideas for Automating Your Life

The best way to learn programming is by working on real-world projects, so why not work on projects that also save you time and sanity? In this free, curated collection, you'll find project ideas for automating:

  • Common office tasks
  • Birthday gifts and wishes
  • Grocery and meal planning
  • Relationships (just the tedious parts!)
  • And quite a bit more

Subscribe to Data Rebellion and get this Ebook delivered straight to your inbox, as well as other exclusive content from time to time on efficiently learning to code useful things, vanquishing soul-crushing work, and having fun along the way.

Reader Interactions

Comments

  1. Anonymous says

    January 9, 2020 at 11:40 pm

    Thanks dear this help me solve the issue

    Reply
    • Grayson Stanton says

      January 10, 2020 at 5:58 pm

      Glad I could help!

      Reply
  2. Ayudh says

    March 21, 2020 at 6:33 am

    What’s the Firefox version and selenium version you are using? Because I get AttributeError: ‘WebDriver’ object has no attribute ‘install_addon’ with FireFox 17.0 & selenium 2

    Reply
  3. Anonymous says

    July 26, 2020 at 7:41 pm

    Thank you very much!!!

    Reply
  4. Anonymous says

    October 21, 2020 at 3:01 pm

    thanks man whoever you are. i was stuck with this problem for a month or more and started using manual alternatives. and you gave a 30sec solution. thanks again.

    Reply
    • Grayson says

      October 22, 2020 at 3:05 pm

      Oh awesome, glad to hear it.

      Reply
  5. Akshaya Kumar Viswanathan says

    May 13, 2021 at 9:08 am

    Hey. Is it possible to make the extension work using selenium. I have an extension installed already. I want to run it using selenium in Tor browser.

    Reply
    • Grayson Stanton says

      May 13, 2021 at 7:09 pm

      Hi Akshaya. I’m not familiar with using the Tor browser with selenium, so I can’t help you right now. But that sounds like a good idea for a future blog post.

      Reply
  6. Anonymous says

    May 14, 2021 at 2:05 am

    Dear,

    It’s the same as firefox. if it can be done in Firefox it’s possible in tor browser. I need to make the extension work using selenium. The extension is already installed. To make it work, I have to click on the extension and click one option only then it does what it should do. Is there anyway to do it using selenium?

    Reply
  7. Akshaya Kumar Viswanathan says

    May 14, 2021 at 2:06 am

    It’s the same as firefox. if it can be done in Firefox it’s possible in tor browser. I need to make the extension work using selenium. The extension is already installed. To make it work, I have to click on the extension and click one option only then it does what it should do. Is there anyway to do it using selenium?

    Reply
    • Grayson Stanton says

      May 15, 2021 at 11:08 pm

      Hi Akshaya, I wish I had more time to look into this further, but this article looks like it might have a solution. And if that fails, as a last resort, I suppose you could use something like pyautogui to have your mouse click at the correct points on your screen that would effectively modify the extension options, albeit in a really hacky way.

      Reply

Leave a comment Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Copyright © 2023

Terms and Conditions - Privacy Policy