5 Best Ways to Flatten a Nested Dictionary in Python

💡 Problem Formulation: In Python, a common task is to flatten a nested dictionary, which involves converting it into a single-level dictionary with compound keys. This allows for easier access and manipulation of the data. Consider a nested dictionary like {'a': {'b': 1, 'c': 2}, 'd': {'e': 3}}. The goal is to flatten this into {'a_b': 1, 'a_c': 2, 'd_e': 3}.

Method 1: Recursive Approach

The recursive approach involves a function that goes through each key-value pair in the dictionary and flattens it out. If a value is another dictionary, the function calls itself on that value. This method is clear, logical, and works well with deeply nested dictionaries.

Here’s an example:

def flatten_dict(d, parent_key='', sep='_'):
    items = []
    for k, v in d.items():
        new_key = parent_key + sep + k if parent_key else k
        if isinstance(v, dict):
            items.extend(flatten_dict(v, new_key, sep=sep).items())
        else:
            items.append((new_key, v))
    return dict(items)

nested_dict = {'a': {'b': 1, 'c': 2}, 'd': {'e': 3}}
flat_dict = flatten_dict(nested_dict)
print(flat_dict)

The output of this code snippet:

This recursive function begins with an empty string for the parent_key and iterates through the dictionary. If a nested dictionary is found, the function calls itself with the updated parent_key. The results are combined into a new flat dictionary.

Method 2: Using a Queue

Utilizing a queue as an iterative approach can be effective for flattening. It simplifies control flow when dealing with deeply nested dictionaries and maintains order. This approach uses breadth-first traversal to ensure all levels of the dictionary are processed.

Here’s an example:

from collections import deque

def flatten_dict_using_queue(d):
    flat_dict = {}
    queue = deque([((), d)])

    while queue:
        parent_key, curr_d = queue.popleft()
        for k, v in curr_d.items():
            if isinstance(v, dict):
                queue.append((parent_key + (k,), v))
            else:
                flat_dict['_'.join((parent_key + (k,)))] = v

    return flat_dict

nested_dict = {'a': {'b': 1, 'c': 2}, 'd': {'e': 3}}
flat_dict = flatten_dict_using_queue(nested_dict)
print(flat_dict)

The output of this code snippet:

This method utilizes a queue to keep track of dictionaries that need to be flattened. We dequeue an item, process it, and if a value is another dictionary, we enqueue it with a compound key. This continuation until the queue is empty yields a flattened dictionary.

Method 3: Using Itertools to Chain Keys

This method takes advantage of Python’s itertools.chain function to create a new dictionary by chaining keys together. It’s an elegant and pythonic way to flatten a nested dictionary, but might be less intuitive for deeply nested structures.

Here’s an example:

from itertools import chain

def flatten_dict_itertools(d, parent_key=''):
    for k, v in d.items():
        new_key = f'{parent_key}{k}' if parent_key else k
        if isinstance(v, dict):
            yield from flatten_dict_itertools(v, f'{new_key}_')
        else:
            yield (new_key, v)

nested_dict = {'a': {'b': 1, 'c': 2}, 'd': {'e': 3}}
flat_dict = dict(chain.from_iterable(flatten_dict_itertools(nested_dict)))
print(flat_dict)

The output of this code snippet:

This method uses a generator to yield tuple pairs of flattened keys and their corresponding values. The chain.from_iterable function effectively flattens the generator into a sequence, which is then turned into a dictionary.

Method 4: Using a Stack

Similar to the queue method, using a stack allows for a non-recursive, iterative approach to flattening a dictionary. A stack will process the deepest keys first with depth-first traversal, which might be more efficient for certain structures.

Here’s an example:

def flatten_dict_using_stack(d):
    flat_dict = {}
    stack = [('', d)]

    while stack:
        parent_key, curr_d = stack.pop()
        for k, v in curr_d.items():
            new_key = parent_key + k if parent_key == '' else f'{parent_key}_{k}'
            if isinstance(v, dict):
                stack.append((new_key, v))
            else:
                flat_dict[new_key] = v

    return flat_dict

nested_dict = {'a': {'b': 1, 'c': 2}, 'd': {'e': 3}}
flat_dict = flatten_dict_using_stack(nested_dict)
print(flat_dict)

The output of this code snippet:

Using a stack, this method processes each dictionary by popping the last item first. Each nested dictionary encountered is added to the stack with its combined key. The loop continues until the stack is empty, and the result is a flat dictionary.

Bonus One-Liner Method 5: Using Dictionary Comprehension

Dictionary comprehension offers a concise and pythonic way to flatten a dictionary. However, it’s limited to dictionaries with only one level of nesting and might not be suitable for deeper or irregular structures.

Here’s an example:

flattened = {f'{outer_key}_{inner_key}': value for outer_key, inner_dict in nested_dict.items() for inner_key, value in inner_dict.items()}
print(flattened)

The output of this code snippet:

This one-liner uses nested dictionary comprehension to iterate over the outer and inner keys of the nested dictionary simultaneously, constructing a new flat dictionary with compound keys.

Summary/Discussion

• Method 1: Recursive Approach. This method is clear and elegant. It works well with deeply nested dictionaries. However, it may suffer from maximum recursion depth errors with extremely deep nesting.
• Method 2: Using a Queue. The queue provides an orderly way to process nested dictionaries without recursion. It ensures that no nesting is missed but may require more memory to maintain the queue.
• Method 3: Using Itertools to Chain Keys. This method leverages Python’s itertools for a clean solution, and the generator-based approach can be memory efficient. However, it’s less straightforward for highly nested or complex structures.
• Method 4: Using a Stack. Iterative and efficient, the stack approach is great for visualizing a depth-first search of the dictionary. Nonetheless, it may have slightly increased complexity compared to other methods.
• Bonus One-Liner Method 5: Using Dictionary Comprehension. This method is quick and concise for shallow nested dictionaries. However, it lacks the ability to handle more than one level of nesting.