Python – How to Join a List of Dictionaries into a Single One?

by
5/5 - (1 vote)
Python: How to Merge Multiple Dictionaries?

Problem: Say, you’ve got a list of dictionaries:

[{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4, 'a': 4}]

Notice how the first and the last dictionaries carry the same key 'a'.

How do you merge all those dictionaries into a single dictionary to obtain the following one?

{'a': 4, 'b': 1, 'c': 2, 'd': 3, 'e': 4}

Notice how the value of the duplicate key 'a' is the value of the last and not the first dict in the list of dicts.

To merge multiple dictionaries, the most Pythonic way is to use dictionary comprehension {k:v for x in l for k,v in x.items()} to first iterate over all dictionaries in the list l and then iterate over all (key, value) pairs in each dictionary.

Let’s explore all the available options in the remaining article:

Exercise: Run the code—which method generates a different output than all the other methods?

Method 1: Dictionary Comprehension With Nested Context

You can use dictionary comprehension {k:v for x in l for k,v in x.items()} to first iterate over all dictionaries in the list l and then iterate over all (key, value) pairs in each dictionary.

  • Create a new dictionary using the {...} notation.
  • Go over all dictionaries in the list of dictionaries l by using the outer loop for x in l.
  • Go over all (key, value) pairs in the current dictionary x by using the x.items() method that returns an iterable of (key, value) pairs.
  • Fill the new dictionary with (key, value) pairs k:v using the general dictionary comprehension syntax {k:v for ...}.
l = [{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4, 'a': 4}]

d = {k:v for x in l for k,v in x.items()}
print(d)
# {'a': 4, 'b': 1, 'c': 2, 'd': 3, 'e': 4}

This is the most Pythonic way to merge multiple dictionaries into a single one and it works for an arbitrary number of dictionaries.

Method 2: Simple Nested Loop

Use a simple nested loop to add each (key, value) pair separately to a newly created dictionary:

  • Create a new, empty dictionary.
  • Go over each dictionary in the list of dictionaries.
  • Go over each (key, value) pair in the current dictionary.
  • Add the (key, value) pair to the new dictionary—possibly overwriting “older” keys with the current one.
l = [{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4, 'a': 4}]

d = {}
for dictionary in l:
    for k, v in dictionary.items():
        d[k] = v

print(d)
{'a': 4, 'b': 1, 'c': 2, 'd': 3, 'e': 4}

You can visualize the execution flow of this code here:

This is the easiest to read for many beginner coders—but it’s much less concise and less Pythonic.

Method 3: Use the update() Method

The dict.update(x) method updates the dictionary on which it is called with a bunch of new (key, value) pairs given in the dictionary argument x. The method to merge multiple dictionaries is simple:

  • Create a new, empty dictionary.
  • Go over each dictionary in the list of dictionaries.
  • Update the new dictionary with the values in the current dictionary.
l = [{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4, 'a': 4}]

d = {}
for dictionary in l:
    d.update(dictionary)

print(d)
# {'a': 4, 'b': 1, 'c': 2, 'd': 3, 'e': 4}

This is a very readable and efficient way and it’s shorter than method 2.

Method 4: Dictionary Unpacking

When applied to a dictionary d, the double asterisk operator **d unpacks all elements in d into the outer dictionary. You can only use this “dictionary unpacking” method within an environment (in our case, a dictionary) that’s capable of handling the (key, value) pairs.

Side note: Sometimes it’s also used for keyword arguments. 

l = [{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4, 'a': 4}]

d = {**l[0], **l[1], **l[2], **l[3], **l[4]}
print(d)
# {'a': 4, 'b': 1, 'c': 2, 'd': 3, 'e': 4}

This is a concise, efficient, and Pythonic way to merge multiple dictionaries.

However, it’s not optimal because you must manually type each unpacking operation. If the dictionary has 100 elements, this wouldn’t be feasible.

Note: you cannot use dictionary unpacking in dictionary comprehension to alleviate this problem—Python will throw a SyntaxError!

Method 5: Use ChainMap With Unpacking

If you’re not happy with either of those methods, you can also use the ChainMap data structure from the collections library.

l = [{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4, 'a': 4}]

from collections import ChainMap
d = dict(ChainMap(*l))
print(d)
# {'e': 4, 'a': 0, 'd': 3, 'c': 2, 'b': 1}

However, this does not exactly meet our specifications: the fifth dictionary in our collection does not overwrite the (key, value) pairs of the first dictionary.

Other than that, it’s a fast way to merge multiple dictionaries and I wanted to include it here for comprehensibility.

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

🚀 If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!