Web Scraping

How to Get an HTML Page from a URL in Python?

This tutorial shows you how to perform simple HTTP get requests to get an HTML page from a given URL in Python! Problem Formulation Given a URL as a string. How to extract the HTML from the given URL and store the result in a Python string variable? Example: Say, you want to accomplish the …

How to Get an HTML Page from a URL in Python? Read More »

Scraping The Absolute URL Instead of The Relative Path Using BeautifulSoup In Python

Summary: Use urllib.parse.urljoin() to scrape the base URL and the relative path and join them to extract the complete/absolute URL. You can also concatenate the base URL and the absolute path to derive the absolute path; but make sure to take care of erroneous situations like extra forward-slash in this case. Problem Formulation Problem: How …

Scraping The Absolute URL Instead of The Relative Path Using BeautifulSoup In Python Read More »