How to Disable Security Certificate Checks for Requests in Python

Summary: You can disable security certificate checks for requests in Python in two ways:
(i) Using Session.verify as False
(ii) Setting verify = false inside the request method.


Problem Formulation

Problem Statement: How to disable security certificate checks for requests in Python?

The requests module in Python sends HTTP requests using a specific method to a specified URL. A response object, including data and information like encoding, status, content, etc., gets returned in response to this request. However, whenever we carry out actions such as post, get, delete, etc, we get an error known as SSLCertVerificationError, also called SSL:Certificate_Verify_Failed self-signed certificate

Example:

Import requests
requests.post(url = 'https://data1', data = {'a': 'az'})

Output:

/usr/local/lib/python3.7 /site
packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced
usage.html#ssl
warnings InsecureRequestWarning)

✨ You can eliminate this error by disabling the security certificate checks.

Prerequisite: You must have the requests library installed on your machine. If you don’t have it yet, you can easily install it using pip by copying the following command: pip install requests 

Now let’s look at the methods to disable security certificate checks for requests in Python in detail.

Method 1: By Setting verify = False

The requests module in Python contains different methods like the post, get, delete, request, etc. We can send an HTTP request to these methods as each accepts a URL. You can set the verify parameter as False to disable the security certificate checks for requests in Python.

When the requests library sends a URL, the following set of operations occur:

  • A DNS lookup converts the URL to an IP address (example: 312.245.123.21),
  • The request library sends a request to this IP address,
  • The server attempts to validate this request,
  • The server returns a status code.

Learn more about requests in Python Requests Library – Your First HTTP Request in Python.”

Approach:

  • First, import the requests module.
  • We need to call the request method to make the requests. We need to pass a string with the HTTP method (“GET”) as the first input and an endpoint to which we want to send the request as the second input. 
  • To disable the security certificate checks for requests, we have to pass the verify parameter with the value being False. 
  • The request method returns an object of class Response, which we store in a variable. Finally, we will print the response from the server.

Code:

# Importing the requests module
import requests
# Sending a get http request to the specified url
response = requests.request(
    "GET", "https://finxter.com/", verify = False)
  
# Printing the response
print(response)

Output:

/usr/lib/python3/dist-packages/urllib3/connectionpool.py:999: InsecureRequestWarning: Unverified HTTPS request is being made to host 'finxter.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
<Response [200]>

Discussion: By passing the verify parameter as False to the request method, we have managed to disable the security certificate check and make the code free of errors. However, this method will throw warnings as shown above. To handle the above warnings, we have to use the urlib3.disable_warnings method. The warning gets suppressed by using the urllib3.disable_warnings method when the verify parameter is set as False.

Have a look at the following code that demonstrates how you can get rid of the warning:

# Importing the requests module
import requests
from urllib3.exceptions import InsecureRequestWarning

# Handling the warnings from urllib3
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)
# Sending a get http request to the specified url
response = requests.request(
    "GET", "https://finxter.com/", verify=False)

# Printing the response
print(response)

# Output: <Response [200]>

Note: When we run the code after disabling the security certificate check for requests, the warning is sent to the shell that indicates- “certificate validation is strongly advised “. It is because, in realtime applications, such validations must be performed and thus, we should only follow the method we are looking at for controlled testing scenarios.

Method 2: Use Session.verify and Set it as False

We can disable the security certificate checks for requests in Python by setting the Session.verify to False. Simply declare Session.verify = False instead of passing verify = True. Let’s look at the following code so that we get a better understanding of how to solve the problem:

Solution:

# Importing requests
import requests

# Creating the session object and declaring the verify variable to False
session = requests.Session()
session.verify = False

# Sending a get http request to the specified url
response = requests.get("https://finxter.com/")

# Printing the response data
print(response)

# OUTPUT: <Response [200]>

Tidbit: If you try to print the entire response returned by the server in order to just verify if the security certificate check has been disabled or not then you will literally get a page full of HTML codes. This is unnecessary and it might also seem to be confusing. Hence, it is advisable to print the response status to check whether the security certificate checks have been disabled unless of course, you have a different requirement where you need the entire response text. I hope you got the point!

Conclusion

In this article we looked at the different methods to disable the security certificate checks for requests in Python. I hope you found it helpful. Please stay tuned and subscribe for more such interesting articles.


Web Scraping with BeautifulSoup

One of the most sought-after skills on Fiverr and Upwork is web scraping.

Make no mistake: extracting data programmatically from websites is a critical life skill in today’s world that’s shaped by the web and remote work.

This course teaches you the ins and outs of Python’s BeautifulSoup library for web scraping.