The Hidden Gems: 4 Best Google Search Libraries for Python You Can’t Miss!

by

A few days ago, a coding friend asked me: “What is the best Google search library for Python?”

Well, to be honest, I had no clue, so I did my investigation and did some quick testing. And I thought that it might be useful to share it with some Pythonistas newbies out of there.

So let’s review some great libraries to access the powerful search of a Google query within your Python code.

Module 1: PyWhatKit

First, let’s start simply by using the PyWhatKit Module:

PyWhatKit is a Python Library that allows you to schedule and send WhatsApp messages and perform other functions such as playing a video on YouTube, converting an image to ASCII art, and converting a string to an image with handwritten characters. Other features include sending emails, taking screenshots, and shutting down or canceling a shutdown on a Linux or Mac OS machine.

GitHub :  https://github.com/Ankit404butfound/PyWhatKit

But you can simply use it as well to run your favorite Google Queries.

So let’s start with the basics!

How to install it:

pip install pywhatkit 

First to test your code:

import pywhatkit as pwk

# To perform a Google search and to open your default browser automatically
print("Let's ask Google!")
search1 = "FIFA world Cup"
pwk.search(search1)

Now a better code that asks for the user’s input:

# Importing the search function from the pywhatkit library
from pywhatkit import search

# Prompting for the user input

query = input("Ask Google about? : ")
print("Searching for ...")
# Running the search query
search(query)

You can run multiple queries, but this will open as many tabs in your browser. Maybe you have a use case for that?

# To import the search function from the pywhatkit library
from pywhatkit import search

# Hardcoding your queries
query1 = "Fifa"
query2 = "world cup"
query3 = "Mundial"
query4 = "world soccer"

# Searching at once
search(query1)
search(query2)
search(query3)
search(query4)

Ok, hold on! I can hear you saying but what about all Google search options. Well let’s investigate another Python library: google!

Module 2: ‘Google’

To install it:

pip install google

Now let’s ask the user what is looking for and return the result of all queries with a list of URLs. More useful when doing an investigation.

# Importing the search function from the google library
from googlesearch import search

# Asking the user how many queries he wants to run

num_searches = int(input("How many queries do you to do, (ie: 3): "))
searchQueries = []
while num_searches>0:

# Then asking the user the subject of the search query

    query = input("Ask Google about? : ")

# This will display a max of 5 results per query

    for i in search(query, tld="com", num=5, stop=5, pause=3):
        print(i)
    num_searches=num_searches-1

Let’s have a look at those search function options:

query  # a string which contain what you are looking for; ie: "FIFA World cup"
tld = 'com',  # The top level domain  of google; ie: 'co.uk, fr' , ...
                lang = 'en',  #  Language  of search result, 'ie: fr=French; gr=german',...
                num = 10,     # Number of results per page 
                start = 0,    # First result to retrieve 
                stop = None,  # Last result to retrieve 
                pause = 2.0,  # Lapse between HTTP requests, this is important because if this value is too low Google can block your IP, so I recommend 2 or 3 seconds

You will get some results in the following format (for example):

How many queries do you to do, (ie: 3): 1
Ask Google about? : fifa
https://www.beinsports.com/france/fifa-coupe-du-monde-2022/video/coupe-du-monde-2022-lenorme-occasion-pour-le-/2000861
https://www.tf1.fr/tf1/fifa-coupe-du-monde-de-football/videos/giroud-le-patient-anglais-91919296.html
https://www.sports.fr/football/equipe-de-france/classement-fifa-injustice-vue-bleus-672799.html
https://www.fifa.com/
https://twitter.com/FIFAcom/status/1600418112830115841?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet

Module 3: Google-API-Python-Client

Now, if you are interested in getting your results back in a JSON format, then the solution is to use the official Google API.

Before using the Google Python library, you need to create a Google account and get your API key here. You will need to create as well a service account in the Google Search Console.

I do recommend reading this blog if you are struggling to create your account and keys.

For the installation of the library:

pip install google-api-python-client

Now we are ready to create a script:

from googleapiclient.discovery import build   #Import the Google API library
import json #we will need it for the output of course

Then, you will need to create two variables that contain your token and key to be authenticated by Google.

For example, they may look as follows (do not copy them, they are fake 😊):

my_api_key = "AIzaSyAezbZKKKKKKr56r8kZk"
my_cse_id = "46c457999997"


Now let’s create a function that we can call to do our search:

def search(search_term, api_key, cse_id, **kwargs):
    service = build("customsearch", "v1", developerKey=api_key)
    result = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute()
    return result

In the second line, we are using the build() command, which has many options because this Google API can be used for many things, but the one we are interested here is the: customsearch.

🌍 Resources: More information on all Google services is available here. More information on the build command here.

Let’s try our function to see if everything is working fine:

result = search("FIFA", my_api_key, my_cse_id)
print(json.dumps(result, sort_keys=True, indent= 4))

Hold on, the output is not in a JSON file. No worries, let’s modify the last bit so you can save it directly to a JSON file:

result = search("FIFA", my_api_key, my_cse_id)
json_file = open("searchResult.json", "w")
json.dump(result, json_file)

Et voilà!

Module 4: SerpAPI

There is another option if you do not want to use the official Google API.

The google-search-results library is not free for unlimited searches. But it can be free for 100 searches per day if you create an account at SerpApi.com, and retrieve your API Key can add it to your code as you can guess!

from serpapi import GoogleSearch

params = {
  "device": "desktop",
  "engine": "google",
  "q": "café",
  "location": "France",
  "google_domain": "google.fr",
  "gl": "fr",
  "hl": "fr",
  "api_key": "cfc232bade8efdb3956XXXXXXxx02251b63f7c751b001a800b7c"
}

search = GoogleSearch(params)
results = search.get_dict()

You will get a nice JSON file as a result. This library might be interesting for you if you want to do an internet search on other engines. Serapi.com supports Bing, DuckDuckGo, Yahoo, Yandex, eBay, Youtube, and many more.

I wanted to share this option as well, even though it is commercial. It can be useful for your use case.

Finishing Up

Some fun to finish for all the fans of Google out there? 😊

By using the first library that we play with, you can transform any images into an ASCII art file, special memories for the 80s.

Please download any image by searching: “I love Google” and save it as a PNG.

# To import the search function from the pywhatkit library
import pywhatkit as pwk

pwk.image_to_ascii_art('loveGoogle.png', 'loveGascii.txt')

Thanks for reading! ♥️