How To Read A JSON File With Python

by

Are you confronted with a JSON file and at a loss to know how to extract wanted information, or you don’t know about JSON but would like to? Today we’ll introduce the JSON format and use Python to access a JSON file. We’ll extract the data we need and we’ll use the Python JSON module designed specifically for this purpose. I’ll also mention some other options for achieving the same result.

As an added bonus I’ll also show you how to convert Python data-types into JSON format. So let’s get started.

What Is A JSON file?

? JSON stands for Java Script Object Notation which is a human friendly text-format for exchanging data. Despite the word Java in the name, JSON is language independent and most major programming languages have the ability to generate and parse JSON formatted data.

As a Python user you will be familiar with the list and dictionary data-types, and the foundation of JSON uses two such structures.

  1. A collection of name / value pairs; and
  2. An ordered set of values.

Here is some JSON formatted data called used_vehicles.json, and you can see the similarity to our Python data-types.

{
    "cars" : [
        {
            "make" : "Audi",
            "model" : "A4",
            "year" : 2010
            },
        {
            "make" : "Peugeot",
            "model" : "308",
            "year" : 2019
            },
        {
            "make" : "Toyota",
            "model" : "Landcruiser",
            "year" : 2015
            }
        ],
    "trucks" : [
        {
            "make" : "Citroen",
            "model" : "HY Van",
            "year" : 1979
            },
        {
            "make" : "Bedford",
            "model" : "TJ",
            "year" : 1960
            }
        ]
}

You can see a dictionary style syntax using curly brackets and key : value pairs, and there is a square bracket notation that looks like a list (or an array).

So How Do You Read A JSON File?

Python has a module that encodes and decodes JSON format files. If you wish to read about it in the Python standard library, you can find it here.

In its basic usage the JSON module has two key read methods:

  1. json.load() – takes a JSON file and converts the encoded data to a Python object
  2. json.loads() – takes a JSON formatted string and converts it into a Python object

Let’s write some code to import JSON and convert the file to a Python format.

Using The json.load() Method

import json

with open('used_vehicles.json') as file:
    stock = json.load(file)

With that code we have now opened, read and converted the used_vehicle.json file to a Python dictionary and we’ve passed it to ‘stock’. Remember that if you are using a path name in the open() command, use a raw string by placing the letter r before the opening speech mark – here’s an example – open(r'C://PythonTutorial/used_vehicles.json')

Now that we have our data opened and converted, let’s print it and see what we get.

import json

with open('used_vehicles.json') as file:
    stock = json.load(file) 

print(stock)

# Result

{'cars': [{'make': 'Audi', 'model': 'A4', 'year': 2010}, {'make': 'Peugeot', 'model': '308', 'year': 2019}, {'make': 'Toyota', 'model': 'Landcruiser', 'year': 2015}], 'trucks': [{'make': 'Citroen', 'model': 'HY Van', 'year': 1979}, {'make': 'Bedford', 'model': 'TJ', 'year': 1960}]}

With recognisable raw data we can now apply standard Python techniques to extract our desired information. We’ll break out the car and truck information into a more readable format.

import json

with open('used_vehicles.json') as file:
    stock = json.load(file) 

for cars in stock['cars']:
    print(cars)

print('\n')

for trucks in stock['trucks']:
    print(trucks)

# Result

{'make': 'Audi', 'model': 'A4', 'year': 2010}
{'make': 'Peugeot', 'model': '308', 'year': 2019}
{'make': 'Toyota', 'model': 'Landcruiser', 'year': 2015}


{'make': 'Citroen', 'model': 'HY Van', 'year': 1979}
{'make': 'Bedford', 'model': 'TJ', 'year': 1960}

Then finally into something approaching normal text-format.

import json

with open('used_vehicles.json') as file:
    stock = json.load(file) 

for cars in stock['cars']:
    print(cars['make'], cars['model'], '\tManufactured', cars['year'])

print('\n')

for trucks in stock['trucks']:
    print(trucks['make'], trucks['model'], '\t\tManufactured', trucks['year'])

# Result

Audi A4 		Manufactured 2010
Peugeot 308 		Manufactured 2019
Toyota Landcruiser 	Manufactured 2015


Citroen HY Van 	Manufactured 1979
Bedford TJ 		Manufactured 1960

Note the use of the \t character to insert tabs that will make the printed data look a little more tidy.

Remember that the code we just wrote used json.load() as we were reading from a file. If we had a JSON formatted string we’d be able to use json.loads() – so let’s do that also.

Using The json.loads() Method

With the following example we show a JSON string and then use the json.loads() command to convert the data into a Python object. In this example I’ve also imported pretty print to enable me to show the final output in a format that is somewhat readable, by using the pprint() command rather than the standard print() format.

import json
from pprint import pprint

vehicles_json = """{
    "cars" : [
        {
            "make" : "Audi",
            "model" : "A4",
            "year" : 2010
            },
        {
            "make" : "Peugeot",
            "model" : "308",
            "year" : 2019
            },
        {
            "make" : "Toyota",
            "model" : "Landcruiser",
            "year" : 2015
            }
        ],
    "trucks" : [
        {
            "make" : "Citroen",
            "model" : "HY Van",
            "year" : 1979
            },
        {
            "make" : "Bedford",
            "model" : "TJ",
            "year" : 1960
            }
        ]
}
"""

vehicles_dict = json.loads(vehicles_json)

pprint(vehicles_dict)

# Result

{'cars': [{'make': 'Audi', 'model': 'A4', 'year': 2010},
          {'make': 'Peugeot', 'model': '308', 'year': 2019},
          {'make': 'Toyota', 'model': 'Landcruiser', 'year': 2015}],
 'trucks': [{'make': 'Citroen', 'model': 'HY Van', 'year': 1979},
            {'make': 'Bedford', 'model': 'TJ', 'year': 1960}]}

Process finished with exit code 0

So to drive the lesson home – remember that the singular load() method reads from a file and converts it to a Python object. The plural loads() method reads from a string and similarly returns a Python object.

Bonus: How Do We Write Data To A JSON File?

We saw in the read examples the use of load() vs loads() so unsurprisingly we have a similar syntax in a write scenario, just reversing the process. The two write methods are:

  1. json.dump() – takes a Python object and stores it in a json file
  2. json.dumps() – takes a Python object and writes it to directly to a json string format

Let’s take a look at the code necessary for these tasks.

Using The json.dump() Method

First using the json.dump() method which writes to a file– note that I’ve opened a file using the ‘with’ command which prevents us needing to add an extra close() command at the end of the code block making the code tidier.

We also need to add the ‘w’ command to ‘write’ to the file, and finally we use the json.dump() syntax. Note I’ve included an optional ‘indent’ command to help make the new file more readable when opened, by indenting by four spaces.

import json

employees_dict = {'employees' : [
        {'name' : 'Adam Smith', 'employee_num' : 187456},
        {'name' : 'Sharon McFarlane', 'employee_num' : 847563},
        {'name' : 'Karen Angus', 'employee_num' : 678654}
        ],
    'contractors' : [
        {'name' : 'John Franklin', 'contract_num' : 7658},
        {'name' : 'Kirsty Appleby', 'contract_num' : 3456},
        {'name' : 'Jim Armstrong', 'contract_num' : 5678},
        ]
    }


with open('C:\PythonTutorials\employee1.json', 'w') as file:
    json.dump(employees_dict, file, indent=4)

If we then check the C:\PythonTutorials folder we find the following file, which I’ve opened in Notepad.

Using The json.dumps() Method

The json.dumps() command writes to a JSON string format, so I’ll code that and pass it to a variable which we’ll then print to see how it looks.

import json

employees_dict = {'employees' : [
        {'name' : 'Adam Smith', 'employee_num' : 187456},
        {'name' : 'Sharon McFarlane', 'employee_num' : 847563},
        {'name' : 'Karen Angus', 'employee_num' : 678654}
        ],
    'contractors' : [
        {'name' : 'John Franklin', 'contract_num' : 7658},
        {'name' : 'Kirsty Appleby', 'contract_num' : 3456},
        {'name' : 'Jim Armstrong', 'contract_num' : 5678},
        ]
    }

employees_json = json.dumps(employees_dict, indent=4)

print(employees_json)


# Results

{
    "employees": [
        {
            "name": "Adam Smith",
            "employee_num": 187456
        },
        {
            "name": "Sharon McFarlane",
            "employee_num": 847563
        },
        {
            "name": "Karen Angus",
            "employee_num": 678654
        }
    ],
    "contractors": [
        {
            "name": "John Franklin",
            "contract_num": 7658
        },
        {
            "name": "Kirsty Appleby",
            "contract_num": 3456
        },
        {
            "name": "Jim Armstrong",
            "contract_num": 5678
        }
    ]
}

Process finished with exit code 0

As you can see when comparing the two codes, the json.dumps() requires less lines of code. You’re simply passing your dictionary to the method which outputs the JSON string to our employees_json variable. You can then print that or utilise it elsewhere in your programme.

Another Option To Read JSON Files In Python

Python’s standard method of making HTTP requests is called, not surprisingly, requests. Using the get() method, you are able to download a web page and pass it to a response object. Within the requests module lies the ability to convert that response object, if it is in JSON format, into a Python dictionary datatype in the same way we have previously with the json module.

To install the requests library you will need to run the following command;

$ pip install requests

We then import requests into our code before constructing a get() request. Here’s the code;

import requests
from pprint import pprint

response = requests.get("https://api.github.com/")

convert = response.json()

pprint(convert)

# Results

{'authorizations_url': 'https://api.github.com/authorizations',
 'code_search_url': 'https://api.github.com/search/code?q={query}{&page,per_page,sort,order}',
 'commit_search_url': 'https://api.github.com/search/commits?q={query}{&page,per_page,sort,order}',
 'current_user_authorizations_html_url': 'https://github.com/settings/connections/applications{/client_id}',
 'current_user_repositories_url': 'https://api.github.com/user/repos{?type,page,per_page,sort}',
 'current_user_url': 'https://api.github.com/user',
 . . .
 'repository_url': 'https://api.github.com/repos/{owner}/{repo}',
 'starred_gists_url': 'https://api.github.com/gists/starred',
 'starred_url': 'https://api.github.com/user/starred{/owner}{/repo}',
 'user_organizations_url': 'https://api.github.com/user/orgs',
 'user_repositories_url': 'https://api.github.com/users/{user}/repos{?type,page,per_page,sort}',
 'user_search_url': 'https://api.github.com/search/users?q={query}{&page,per_page,sort,order}',
 'user_url': 'https://api.github.com/users/{user}'}

Process finished with exit code 0

You’ll see I’ve used pretty print again to make the output readable and I’ve also truncated the returned data to save space. In all other respects however, the results are very much the same as we’ve seen previously.

To Summarise

In this article we set out to understand the basics of a JSON file and we learned that it is a file type used for easy data interchange which is able to be read by most major programming languages.

JSON files use text in a very specific format, utilising structures similar to Python dictionary and list data-types.

We learned about a Python module called json which enables us to read JSON files and write them to Python objects using the load() and loads() methods, and similarly enables us to take Python data-types and write them to JSON format using the dump() and dumps() modules.

Finally we introduced the requests module which is a Python package used to handle HTTP requests, and which has a json method to convert the recovered JSON data to a Python object.

I trust this article helped with your understanding in how to read a JSON file. Thank you for reading.