There comes a time in all our coding lives when we need to compare lists to understand whether items in one list appear in a second list.
In this article we’ll start where we all started, using for-loops, before moving to a more classic Python list comprehension. We’ll then move beyond that to use Python built-in functions any() and all() before looking at a clever use of methods contained within the set() data type.
By the end of this article, you’ll be creating code that not only meets your need but also retains readability while being concise, fast, and elegantly pythonic.
We will also go a bit beyond the remit of checking whether items in one list appear in another, we’ll also find out an easy method to return those duplicates (or as a bonus, return the non-duplicated items of the list) for review or use.
Method 1: List Comprehension
So if we were starting in Python coding we would use a for loop to iterate through the two lists and return a result. In the examples following we first return a True/False on whether a number in List C is also in List A. We then return the actual numbers from List C that are in List A.
# Main list
lst_a = [24, 17, 37, 16, 27, 13, 46, 40, 46, 51,
44, 29, 54, 77, 78, 73, 40, 58, 32, 48,
45, 55, 51, 59, 68, 34, 83, 65, 57, 50,
57, 93, 62, 37, 70, 62
]
# SOME items are in lst_a
lst_c = [93, 108, 15, 42, 27, 83]
# Empty list
lst_result = []
# Check True or False items are in both lists
for i in lst_c:
for j in lst_a:
if i == j:
print(True, end=' ')
else:
continue
print('\n')
# Return values that are in both lists
for i in lst_c:
for j in lst_a:
if i == j:
lst_result.append(i)
else:
continue
print(lst_result)
# Result #
True True True
[93, 27, 83]
So that’s fine as far as it goes; we’ve answered the question. Yet, it took six lines of code for the True/False query and another six lines of code plus the creation of a new list to identify the actual numbers that were common to both lists. Using list comprehension we can improve on that.
Method 2: Lambda
If you read my blog post on the use of lambda expressions and you’ve fallen in love with them as I once did, we could use a lambda for this purpose.
# Main list
lst_a = [24, 17, 37, 16, 27, 13, 46, 40, 46, 51,
44, 29, 54, 77, 78, 73, 40, 58, 32, 48,
45, 55, 51, 59, 68, 34, 83, 65, 57, 50,
57, 93, 62, 37, 70, 62
]
# SOME items are in lst_a
lst_c = [93, 108, 15, 42, 27, 83]
print(list(filter(lambda i: i in lst_a, lst_c)))
# Result #
[93, 27, 83]
Yet, in my post on lambda expressions I did say that they can sometimes make code difficult to read and when looking at the above code, as much as it pains me to admit it, I’m not convinced a lambda expression is necessary in this case. The syntax doesn’t exactly roll off your tongue while reading so let’s look at a couple of easy one-liner list comprehensions that return the same information as the previous methods but they’re more concise and more readable. Here they are.
# Main list
lst_a = [24, 17, 37, 16, 27, 13, 46, 40, 46, 51,
44, 29, 54, 77, 78, 73, 40, 58, 32, 48,
45, 55, 51, 59, 68, 34, 83, 65, 57, 50,
57, 93, 62, 37, 70, 62
]
# SOME items are in lst_a
lst_c = [93, 108, 15, 42, 27, 83]
print([True for i in lst_a if i in lst_c], '\n')
print([i for i in lst_a if i in lst_c])
# Result #
[True, True, True]
[27, 83, 93]
So we have reached a tidy landing place for list comprehensions with short, readable code but now we should inject another variable into our thinking, that being the speed of execution. On small lists such as the ones we’ve used here any speed penalties of different function choices are minor, however, be careful that on a large list this method of list comprehension doesn’t come with a speed penalty. It would pay to check with a timer during a test.
Method 3: Python’s any() and all() Built-in Functions
To avoid writing lengthy code, Python has a range of built-in functions that meet our need to understand whether items in one list are present in another.
The function any() checks if any of the items in a list are True and returns a corresponding True. Here’s a simple example of how it works:
a = [True, False, True, False, True] print(any(a)) # Result # True
That’s straightforward, so let’s apply it to our list of examples. I’ve screenshot all the lists again to save you from scrolling. So if we want a simple True/False response to our question as to whether any items in one list are in another, any() suits our need admirably.
print(any(x in lst_a for x in lst_b)) print(any(x in lst_a for x in lst_c)) print(any(x in lst_a for x in lst_d)) # Result # True True False
Remember that lst_b items are all in lst_a; lst_c has some of its items in lst_a, and lst_d does not have any items in lst_a. Therefore the return of True, True, False makes sense as only the third list, lst_d, does not have any items duplicated in lst_a.
The issue with this method is that it doesn’t tell you if all the items in one list are in another, only that some are. If you need that degree of precision, the built-in function all() can do this for you.
# Main list
lst_a = [24, 17, 37, 16, 27, 13, 46, 40, 46, 51,
44, 29, 54, 77, 78, 73, 40, 58, 32, 48,
45, 55, 51, 59, 68, 34, 83, 65, 57, 50,
57, 93, 62, 37, 70, 62
]
# ALL items are in lst_a
lst_b = [59, 37, 32, 40]
# SOME items are in lst_a
lst_c = [93, 108, 15, 42, 27, 83]
# NO items are in lst_a
lst_d = [23, 101, 63, 35]
print(all(x in lst_a for x in lst_b))
print(all(x in lst_a for x in lst_c))
print(all(x in lst_a for x in lst_d))
# Result #
True
False
False
So in this case the only list that has all of its items contained in lst_a is lst_b, hence the True.
These two functions any() and all() are useful, provide readable code, and are concise, but in the basic list comprehension done previously, we were also able to list out the actual duplicate items. While you could do that using any() and all() the extra code to make it work begs the question of why you’d bother so let’s leave those two to return just True or False and turn our attention to some different approaches.
Method 4: Introducing The set() Data Type And Methods
Now it may seem strange and a bit arbitrary to introduce a new data type when we’re working with lists but the method I’m about to show is an elegant way to answer our question on whether items in one list are in another, and we’ll even return the answer as a list to remain coherent with our code.
For those who don’t do much with sets, they are one of the four Python built-in data types. They are an unordered and unindexed collection of data, and they come with some very clever methods that we can use. There are 17 methods for use on sets and I’ll first introduce you to two of those that I feel suit best this application. The first one gives us much the same as we’ve done using any() and all() , while the second provides an elegant way of returning the items common to two lists.
• issubset() – returns whether another set contains this set or not
• intersection() – returns a set, that is the intersection of two other sets
And here is the code using both methods on each of our three list comparisons.
# Main list
lst_a = [24, 17, 37, 16, 27, 13, 46, 40, 46, 51,
44, 29, 54, 77, 78, 73, 40, 58, 32, 48,
45, 55, 51, 59, 68, 34, 83, 65, 57, 50,
57, 93, 62, 37, 70, 62
]
# ALL items are in lst_a
lst_b = [59, 37, 32, 40]
# SOME items are in lst_a
lst_c = [93, 108, 15, 42, 27, 83]
# NO items are in lst_a
lst_d = [23, 101, 63, 35]
print(set(lst_b).issubset(lst_a))
print(set(lst_c).issubset(lst_a))
print(set(lst_d).issubset(lst_a), '\n')
print(list(set(lst_a).intersection(set(lst_b))))
print(list(set(lst_a).intersection(set(lst_c))))
print(list(set(lst_a).intersection(set(lst_d))))
# Result #
True
False
False
[32, 40, 59, 37]
[27, 83, 93]
[]
Note that in both cases we needed to convert the lists into sets using the set(lst_a), set(lst_b) syntax shown, before allowing the intersection method to do its work. If you want the response returned as a list then you’ll need to convert the response using the list() command as shown. If that’s not important to you, you’ll save a bit of code and return a set.
Methods 5-7: Three Bonus Methods
While moving slightly away from our original question of whether items in one list are in another, there are three other methods in set( ) that may suit your needs in making list comparisons although the answers they return approach the problem from another angle. These are:
• difference() – returns a set containing the difference between two or more sets
• isdisjoint() – returns whether two sets have an intersection or not
• issuperset() – returns whether one set contains another set or not
As you can tell from the descriptions they’re effectively the inverse of what we have done previously with intersection() and issubset().
Using our code examples, difference() will return the numbers in lst_a that are not in lst_b, c or d while isdisjoint() will return False if there is an intersection and a True if there is not (which seems a little counterintuitive until you reflect on the name of the method), and issuperset() will check whether our large lst_a contains the smaller lst_b, c or d in its entirety.
Here’s an example of the three methods in use on our lists.
# Main List
lst_a = [24, 17, 37, 16, 27, 13, 46, 40, 46, 51,
44, 29, 54, 77, 78, 73, 40, 58, 32, 48,
45, 55, 51, 59, 68, 34, 83, 65, 57, 50,
57, 93, 62, 37, 70, 62
]
# ALL items are in lst_a
lst_b = [59, 37, 32, 40]
# SOME items are in lst_a
lst_c = [93, 108, 15, 42, 27, 83]
# NO items are in lst_a
lst_d = [23, 101, 63, 35]
print(set(lst_a).isdisjoint(lst_b))
print(set(lst_a).isdisjoint(lst_c))
print(set(lst_a).isdisjoint(lst_d), '\n')
print(list(set(lst_a).difference(set(lst_b))))
print(list(set(lst_a).difference(set(lst_c))))
print(list(set(lst_a).difference(set(lst_d))), '\n')
print(set(lst_a).issuperset(set(lst_b)))
print(set(lst_a).issuperset(set(lst_c)))
print(set(lst_a).issuperset(set(lst_d)))
# Result #
False
False
True
[65, 68, 70, 73, 13, 77, 78, 16, 17, 83, 24, 27, 29, 93, 34, 44, 45, 46, 48, 50, 51, 54, 55, 57, 58, 62]
[65, 68, 70, 73, 13, 77, 78, 16, 17, 24, 29, 32, 34, 37, 40, 44, 45, 46, 48, 50, 51, 54, 55, 57, 58, 59, 62]
[65, 68, 70, 73, 13, 77, 78, 16, 17, 83, 24, 27, 29, 93, 32, 34, 37, 40, 44, 45, 46, 48, 50, 51, 54, 55, 57, 58, 59, 62]
True
False
False
At the risk of labouring a point, remember that isdisjoint() will return False if any items in one list appear in the other. It will only return True when the two lists are entirely separate without any duplication.
🌍 Recommended Tutorial: How to Find Common Elements of Two Lists
In Summary
To summarise what we’ve covered today, we looked at an oft posed question of how best to check if items in one list are also in another.
- We started with basic list comprehension using
for-loops, before checking to see if alambdaexpression was more suitable. We finally concluded with a one-line list comprehension that returnedTrueorFalseon whether each list item was in another list. We also used a one-line list comprehension to return the actual values of the duplicates as a list. - We then explored Python built-in functions
any()andall()which return true or false depending on whether any or all items in one list repeat in another. - Finally, we introduced some of the methods used within the
set()datatype. Usingissubset()we were able to return true or false on whether list items duplicated in another list and usingintersection()we returned the values of the duplicated items. - As a bonus, we also introduced some other
set()methods which allowed us to further manipulate the lists to return useful data.
I hope the examples used in this article have been useful. They are not the only way to solve the original problem but there are enough examples here to get you started on your journey with lists. I highly recommend the following articles for those who wish to go deeper into some of the methods shown today.
To explore Python’s built-in functions I suggest starting here; https://blog.finxter.com/python-built-in-functions/
For a good start with list comprehension, try this article; https://blog.finxter.com/list-comprehension/
If you’d like to know more about writing one-liner code then you can’t go past this book; https://www.amazon.com/gp/product/B07ZY7XMX8
I hope you enjoyed the article. Thanks for reading!