Python Filter List Based on Another List

💡 Problem Formulation: When working with lists in Python, a common task is to filter elements from one list based on the contents of another list. For example, given a list of numbers list1 = [1, 2, 3, 4, 5] and a list of filters filters = [2, 3], how can we get a result list [1, 4, 5] that excludes any numbers present in filters?

Method 1: Loop and Condition

A simple way to filter a list based on another is to use a loop and a conditional statement. Loop through each item in the main list and only add it to a new list if it is not present in the filter list.

Here’s a minimal example:

def filter_list(main_list, filter_list):
    return [item for item in main_list if item not in filter_list]

In this code, we define a function that takes two lists: main_list and filter_list. It returns a new list that includes only those items from main_list not found in filter_list.

If you don’t yet understand list comprehensions, check out this article and tutorial:

👉 Python List Comprehension

Method 2: Using filter() Function

The filter() function is a built-in Python function that can be used to create an iterator from elements of an iterable for which a function returns True.

Here’s a minimal example:

def does_not_exist_in_filter(item, filter_list):
    return item not in filter_list

filtered_list = list(filter(lambda item: does_not_exist_in_filter(item, filters), list1))

We define a helper function that checks if an item is not in the filter list and then use filter() with a lambda function to apply this helper function to each item in list1.

Method 3: Set Operation

If the order of elements is not important and the lists contain no duplicates, set operations can be efficient.

Here’s a minimal example:

filtered_list = list(set(list1) - set(filters))

By converting both lists to sets, we can use the subtraction operation to remove elements of filters from list1, then convert the result back to a list.

Method 4: List Comprehension with in Operator

We can use list comprehension to create a succinct one-liner that filters the list.

Here’s a minimal example:

filtered_list = [item for item in list1 if item not in filters]

This line iterates over each element in list1 and includes it in filtered_list only if it’s not found in filters.

Method 5: Using filterfalse() from itertools

filterfalse() from the itertools module is the opposite of filter(); it constructs an iterator from elements of iterable for which a function returns False.

Here’s a minimal example:

from itertools import filterfalse
filtered_list = list(filterfalse(lambda item: item in filters, list1))

This uses filterfalse() to keep only those items not in the filters list, making it a compact and readable approach.

Bonus One-Liner Method 6: Using a Function with filter()

The filter() function can also be used with an in-line lambda function for a compact one-liner.

Here’s a minimal example:

filtered_list = list(filter(lambda x: x not in filters, list1))

Here, filter() is applied directly using a lambda function that checks if an item is not in filters.

Summary/Discussion

Python provides several ways to filter a list based on another list, from simple for loops to list comprehensions and set operations.

• The best method depends on your needs, such as preserving the order of elements or ensuring performance with large data sets.
• Set operations are typically the fastest but require elements to be unique and not ordered.
• List comprehension and the filter() function provide a clear and Pythonic way to achieve the same with or without order preserved.
• itertools.filterfalse() is a useful addition when you want to filter based on a condition for exclusion.

Each method shown allows for concise and efficient list filtering, ensuring Python coders have the tools necessary to tackle such common list operations with ease.

👉 How to Filter a List in Python?