Headless Chrome and regular Chrome have the same capabilities, and running them with Selenium is a very similar process. The difference is that Headless Chrome does not generate any sort of user interface. In other words, no browser is visibly launched.
If you happen to be web scraping with Selenium, it’s often helpful to see what exactly the browser is doing in real time for development and debugging purposes. However, using headless mode can be great if your script is working and you don’t want to be bothered with an open browser. And even better, a headless browser should generally run faster than its headed counterpart, given that it doesn’t require the extra resources normally needed to visually render everything happening in the browser.
To run Headless Chrome, you’ll first need to set up Selenium.
Once you’ve got Selenium working, using Headless Chrome is a breeze. For example, let’s see if we can get to DuckDuckGo‘s home page.
Since Headless Chrome has no visible browser, we’ll take a screenshot to confirm what the browser is doing.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
from selenium import webdriver chromedriver = 'C:\\Users\\grayson\\Downloads\\chromedriver.exe' options = webdriver.ChromeOptions() options.add_argument('headless') options.add_argument('window-size=1200x600') # optional browser = webdriver.Chrome(executable_path=chromedriver, chrome_options=options) browser.get('https://www.duckduckgo.com') browser.save_screenshot('C:\\Users\\grayson\\Downloads\\headless_chrome_test.png') browser.quit() |
If the script finishes and your screenshot shows DuckDuckGo’s home page, you’re all set!
If something’s not working, make sure Selenium, Chrome, and chrome driver are all up-to-date and try again. If all else fails, leave a comment and we’ll figure it out.
Aadil says
Hi, how would I add a username and password if authentication is required?
I am trying to access an Intranet page, and a box pops up asking me for a username and password.
So currently I get a 401 error code response.
Thank you