Using Firefox Extensions with Selenium in Python

Firefox is my personal favorite browser, due in part to all of the great extensions available for it. When you try running Firefox with Selenium, however, you’ll probably find that Firefox is missing the extensions you have installed and normally use when browsing. Luckily, there’s a quick and easy way to install all your favorite Firefox extensions when using Selenium.

For example, let’s say we’d like to do a little light web scraping. To keep things simple, let’s just grab what’s trending off of Yahoo’s home page.

You can see the top 10 trending subjects off to the right, starting with Beaufort County.

Selenium without Firefox Extensions

Here’s how we’d normally scrape that info:

Great, that seems to work. But let’s say we’d prefer Firefox to be running with a couple of my favorite extensions, namely:

• HTTPS Everywhere: Automatically enables HTTPS encryption on sites that support it, making for more secure browsing.
• uBlock Origin: An efficient blocker that can make bloated web pages load much faster.

How do we get these extensions installed on Selenium’s instance of Firefox?

Getting the Necessary Information

First, we’ll need to find where those extensions are stored locally. Note that this means you’ll need to already have them installed on your machine for regular use of Firefox.

To find them, open up Firefox and navigate to the main drop down menu. Go to “Help”, and then “Troubleshooting Information”. Alternatively, you can get to the same place by entering about:support in your Firefox navigation bar.

In the “Application Basics” section click the “Open Directory” button, and in the file browser that pops up open the “extensions” folder. These are the extension installation files we’ll need to reference in our script. There should be a different “.xpi” file for every Firefox extension you have installed, and the file path to this folder should look something like “C:\Users\Grayson\AppData\Roaming\Mozilla\Firefox\Profiles\3rqg4psi.default\extensions”.

It might be difficult to tell which files correspond to which extensions based on the file names, as the file names are sometimes unintelligible. To get around this, go back to your browser and on the same page as before scroll down to the “Extensions” section. Here you’ll find a table that pairs each extension name with its corresponding ID, and the ID should be almost the same as the installation file name, lacking just the “.xpi” suffix.

In our case though, the extension and file names aren’t too hard to match:

• HTTPS Everywhere: https-everywhere@eff.org.xpi
• uBlock Origin: uBlock0@raymondhill.net.xpi

Selenium with Firefox Extensions

Now we just need to add a few lines of code to our original script to install these extensions. We’ll perform the installations right after we initialize the browser.

Run this script and see if you get the same results as last time. Look for the extension symbols near the top right of the browser. You should see the blue-and-white “S” symbol for HTTPS Everywhere and the reddish badge symbol for uBlock Origin.

So there you have it. We performed the same operation, but got to take our two favorite Firefox extensions along for the ride.

In addition to the peace of mind knowing that HTTPS security was used whenever possible, you may have noticed that our second script took significantly less time to load the page. This is because uBlock Origin blocked a number of unnecessary, resource-intensive requests, a great feature to have when you’re dealing with the slow, bloated web pages that are all too common nowadays.

Anyways, I hope this gives you a few ideas as to how you can make your life a little more convenient. Let me know if you have any questions, and happy automating.

Free Ebook: 88 Python Project Ideas for Automating Your Life

The best way to learn programming is by working on real-world projects, so why not work on projects that also save you time and sanity? In this free, curated collection, you'll find project ideas for automating:

• Common office tasks
• Birthday gifts and wishes
• Grocery and meal planning
• Relationships (just the tedious parts!)
• And much, much more

Subscribe to Data Rebellion and get this Ebook delivered straight to your inbox, as well as other exclusive content on using programming to vanquish soul-crushing work, advance your career, and design your lifestyle of choice. I respect your inbox, and it's a breeze to unsubscribe.