5 Best Ways to Capture a Complete Screenshot of a Webpage in Selenium with Python

πŸ’‘ Problem Formulation: Automating full-page screenshots in testing or web scraping can be challenging, especially when dealing with dynamic content or large pages that do not fit on a single screen. Often, developers want to capture the entire content of a page which may require scrolling. The desired outcome is an image file that represents the full-page content exactly as it would appear if fully scrolled through by a user.

Method 1: Standard Screenshot Method

The standard screenshot method in Selenium is the most straightforward way to capture a screenshot of the visible area of a webpage. The get_screenshot_as_file() function in Selenium’s WebDriver can be used to save the screenshot directly to a file.

Here’s an example:

from selenium import webdriver

driver = webdriver.Chrome()
driver.get('https://www.example.com')
driver.get_screenshot_as_file('example_screenshot.png')
driver.quit()

The output is a PNG file named ‘example_screenshot.png’ that contains the visible portion of ‘https://www.example.com’ at the time of capture.

This code initializes the web driver, navigates to ‘https://www.example.com’, and then saves a screenshot of the visible part of the web page to ‘example_screenshot.png’. The driver is then quit, closing the browser. This is a quick and simple way to capture what’s visible without any additional processing for capturing content beyond the viewable area.

Method 2: Full Page Screenshot Using a Plugin

For capturing the entire webpage including content beyond the viewport, one can use Selenium plugins such as AShot. AShot, a WebDriver screenshot utility, captures the entire webpage by scrolling and stitching together multiple screenshots.

Here’s an example:

from selenium import webdriver
from AShot import AShot

driver = webdriver.Chrome()
driver.get('https://www.example.com')
screenshot = AShot().shooting_strategy(ShootingStrategies.viewport_pasting(100)).take_screenshot(driver)
ImageIO.write(screenshot.get_image(), "PNG", File("entire_screenshot.png"))
driver.quit()

The output is a PNG file named ‘entire_screenshot.png’ providing a full page screenshot of ‘https://www.example.com’.

In this example, the AShot utility is initialized with a strategy to scroll and capture the entire page. The resulting image is written to ‘entire_screenshot.png’. This method is powerful for capturing full-page content but requires additional libraries and may be slower due to the stitching process.

Method 3: Taking Screenshot with JavaScript Execution

Executing JavaScript to scroll through and capture the full page provides more control over the screenshot process. In Selenium, the execute_script() function is used to run custom JavaScript within the context of the browser.

Here’s an example:

from selenium import webdriver

driver = webdriver.Chrome()
driver.get('https://www.example.com')

# Scroll to the bottom to ensure all lazy-loaded images are loaded
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

# Take the screenshot
driver.get_screenshot_as_file('fullpage_screenshot.png')
driver.quit()

The output is a PNG file named ‘fullpage_screenshot.png’ attempting to capture the fully scrolled webpage.

This code snippet scrolls the web page to the bottom using JavaScript, then takes a screenshot, ensuring that lazy-loaded images are captured. While this simple method attempts to ensure that all page content is loaded, it may not always result in a complete full-page screenshot if the page has complex loading mechanisms.

Method 4: Combine Scrolling with Screenshot

To overcome lazy-loading and capture dynamic content, we can combine scrolling through the page with several screenshot captures, and then stitch the images together using an image processing library like PIL (Pillow) in Python.

Here’s an example:

from selenium import webdriver
from PIL import Image

driver = webdriver.Chrome()
driver.get('https://www.example.com')

scrolls = driver.execute_script("return Math.ceil(document.body.scrollHeight / window.innerHeight);")
screenshots = []
for _ in range(scrolls):
    screenshots.append(Image.open(BytesIO(driver.get_screenshot_as_png())))
    driver.execute_script("window.scrollBy(0, window.innerHeight);")

# stitch screenshots
stitched_image = ...
stitched_image.save('full_page_screenshot.png')
driver.quit()

The output is a PNG file named ‘full_page_screenshot.png’ that is a stitched image of the entire webpage.

In this code, we determine the number of scrolls needed to cover the page, capture each segment as an image, and then use the PIL library to stitch together these segments. This results in a complete screenshot of the webpage. This method’s effectiveness is highly dependent on accurate scrolling and stitching logic.

Bonus One-Liner Method 5: Using Third-party Services

Third-party services, like BrowserStack or Sauce Labs, provide APIs that may support full-page screenshots with less hands-on coding required by making use of their infrastructure.

Here’s an example:

import requests

response = requests.get('https://api.third-party-service.com/screenshot?url=https://www.example.com')
with open('full_page_screenshot.png', 'wb') as f:
    f.write(response.content)

The output is a PNG file saved as ‘full_page_screenshot.png’, which contains a full-page screenshot of ‘https://www.example.com’ provided by the third-party service.

This code makes an HTTP request to a third-party service’s API, requesting a screenshot of the specified URL, and then writes the response to a file. This method is highly convenient but may incur costs and is dependent on the service’s availability and capabilities.

Summary/Discussion

  • Method 1: Standard Screenshot Method. Simple and quick. Only captures visible area.
  • Method 2: Full Page Screenshot Using a Plugin. Captures entire page content. Requires additional library and processing time.
  • Method 3: Screenshot with JavaScript Execution. Offers customizability. May miss out on completely capturing dynamic pages.
  • Method 4: Combine Scrolling with Screenshot. Offers a complete capture. Requires complex image processing and may be error-prone with dynamic content.
  • Method 5: Using Third-party Services. Easy to implement. Dependent on external services and may be subject to fees.