5 Best Ways to Dump a List of Dicts to JSON in Python

💡 Problem Formulation: In Python, developers often need to convert a list of dictionaries into JSON format for easy data interchange or for saving configurations. The challenge is efficiently converting a structured list like [{"key1": "value1"}, {"key2": "value2"}] into a JSON string such as '[{"key1": "value1"}, {"key2": "value2"}]'. In this article, we explore various methods to achieve this conversion.

Method 1: Using json.dumps() Standard Function

Python’s standard library has a built-in json module, which provides the json.dumps() function to convert a Python object into a JSON formatted string. This method is straightforward and recommended for most use cases. The dumps() function ensures that the resulting JSON is correctly formatted and encoded.

Here’s an example:

import json

# A list of dictionaries
data = [{'name': 'John', 'age': 30}, {'name': 'Jane', 'age': 25}]

# Convert the list of dictionaries to a JSON string
json_data = json.dumps(data)

print(json_data)

Output:

[{"name": "John", "age": 30}, {"name": "Jane", "age": 25}]

By calling json.dumps(data), we serialize the list of dictionaries into a JSON string, which we can then print, store in a file, or send over the network. It is fast, reliable, and widely used in various programming scenarios.

Method 2: Using json.dump() to Write Directly to a File

If the aim is to write the JSON data directly into a file, Python’s json.dump() function is the perfect tool. Unlike json.dumps(), which returns a string, json.dump() writes the JSON data to a file-like object. This approach can be more efficient if working with large datasets.

Here’s an example:

import json

# A list of dictionaries
data = [{'apple': 1, 'banana': 2}, {'strawberry': 15, 'blueberry': 22}]

# Write the JSON to a file
with open('data.json', 'w') as file:
    json.dump(data, file)

Using this method, no direct output is printed. Instead, a file named data.json is created containing the JSON data. The use of the with statement ensures the file is properly closed after the writing operation is complete. It’s a clean and efficient way to handle file operations in Python.

Method 3: Pretty Printing with json.dumps()

For better readability, Python’s json.dumps() can be used with additional arguments to format the JSON string in a pretty-printed style. The indent and sort_keys parameters control the formatting.

Here’s an example:

import json

# A list of dictionaries
data = [{'a': 1, 'c': 3, 'b': 2}, {'d': 4, 'e': 5}]

# Convert to a nicely formatted JSON string
pretty_json_data = json.dumps(data, indent=4, sort_keys=True)

print(pretty_json_data)

Output:

[
    {
        "a": 1,
        "b": 2,
        "c": 3
    },
    {
        "d": 4,
        "e": 5
    }
]

Here, the indent parameter adds whitespace to the output, making it easier to read. The sort_keys parameter sorts the dictionary keys alphabetically. This method is not recommended for large datasets or sending over the network because of the added size, but it is excellent for debugging or configuration files.

Method 4: Handling Non-Primitive Types with Custom Encoder

When dealing with types that the default JSON encoder cannot serialize—such as datetime objects—you’ll need to use a custom encoder. By subclassing json.JSONEncoder and overriding the default() method, we can define how to serialize these complex types.

Here’s an example:

import json
from datetime import datetime

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()
        return json.JSONEncoder.default(self, obj)

# A list with a complex data type
data = [{'now': datetime.now()}]

# Dump the list with the custom encoder
json_data = json.dumps(data, cls=CustomEncoder)

print(json_data)

Output:

[{"now": "2023-04-01T12:00:00"}]

In this case, the CustomEncoder checks if an object is an instance of datetime and returns its ISO-format string via obj.isoformat(). The json.dumps() function accepts an extra argument cls to specify the custom encoder which enables it to serialize the complex data. This approach provides flexibility when working with advanced data types.

Bonus One-Liner Method 5: List Comprehension with json.dumps()

For simple transformations or filtering before dumping to JSON, a one-liner using list comprehension and json.dumps() can be very concise. This method is ideal for lightweight data manipulation tasks.

Here’s an example:

import json

# List of dictionaries with some extra data
data = [{'user': 'Alice', 'logged_in': True}, {'user': 'Bob', 'logged_in': False}]

# One-liner to filter out users that are logged in and dump to JSON
logged_in_users = json.dumps([user for user in data if user['logged_in']])

print(logged_in_users)

Output:

[{"user": "Alice", "logged_in": true}]

In this succinct snippet, the list comprehension filters the list of dictionaries to include only those with user['logged_in'] set to True. We then immediately pass the resulting list to json.dumps(). This method shines for its brevity and inline transformation capability.

Summary/Discussion

• Method 1: json.dumps(). Strengths: Simple and fast serialization of Python objects to JSON strings. Weaknesses: Not directly suitable for file writing.
• Method 2: json.dump(). Strengths: Convenient for writing JSON data directly to a file. Weaknesses: Less suitable for quick serialization to a string when file operations are not needed.
• Method 3: Pretty Printing. Strengths: Enhances readability of the JSON string. Weaknesses: Increases the size of the JSON data, less efficient for large data or network operations.
• Method 4: Custom Encoder. Strengths: Offers the flexibility to handle complex data types like datetime. Weaknesses: Requires more code and understanding of custom encoders can be a bit complex for beginners.
• Method 5: One-Liner List Comprehension. Strengths: Very concise, integrates transformations inline. Weaknesses: Limited in complexity, may be less readable for more elaborate data manipulations.