5 Best Ways to Get Unique Values from a List in Python

Rate this post

πŸ’‘ Problem Formulation: In Python, when dealing with lists, one might often need to extract the unique elements to eliminate duplicates. This operation is common when processing datasets for machine learning, data analysis, or simply when trying to achieve a collection of distinct items. For instance, given a list input_list = [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], we want the unique set of values [1, 2, 3, 4] as the output.

Method 1: Using Set Data Structure

The set data structure in Python is designed to store unique items by definition. If a list is converted to a set, all duplicate elements are automatically removed. This method is very efficient and the most straightforward way to get unique elements from a list. However, it does not maintain the order of elements.

Here’s an example:

input_list = ['apple', 'banana', 'apple', 'cherry', 'banana', 'cherry']
unique_items = set(input_list)
print(unique_items)

{'banana', 'cherry', 'apple'}

This simple code snippet converts the list into a set that inherently contains only unique elements. The order in which the items appear is not guaranteed.

Method 2: Using a For Loop

A more manual approach is to iterate over the list with a for loop and collect unique elements. This method allows maintaining the original order of appearance within the list.

Here’s an example:

input_list = [1, 2, 2, 3, 3, 3, 4, 4, 4, 4]
unique_list = []
for item in input_list:
    if item not in unique_list:
        unique_list.append(item)
print(unique_list)

[1, 2, 3, 4]

This code iterates through each element and appends it to a new list only if it’s not already included, effectively retaining only unique items in their original order.

Method 3: Using List Comprehension and an Auxiliary Set

Combining list comprehension with an auxiliary set is a neat way to preserve the order of elements and still have an efficient look-up time for unique elements. This method provides a balance between performance and order preservation.

Here’s an example:

input_list = ['a', 'b', 'a', 'c', 'b', 'c', 'a']
unique_list = []
unique_set = set()
[unique_list.append(x) for x in input_list if not (x in unique_set or unique_set.add(x))]
print(unique_list)

['a', 'b', 'c']

Here, the list comprehension goes through each item in the input list, and the auxiliary set tracks elements that have been encountered, efficiently preventing duplicates from being added to the result list.

Method 4: Using Dictionary Keys

Since Python 3.7, dictionaries maintain the insertion order of keys. By creating a dictionary from the list, with list elements as the keys, one can take advantage of this property to maintain the order while removing duplicates.

Here’s an example:

input_list = [10, 20, 10, 30, 20, 40]
unique_dict = dict.fromkeys(input_list)
unique_list = list(unique_dict)
print(unique_list)

[10, 20, 30, 40]

The code snippet uses the fromkeys() method to create a dictionary, where each item of the list becomes a key and thus inherently unique. Converting the keys back to a list results in an ordered list of unique items.

Bonus One-Liner Method 5: Using the “OrderedDict” from “collections”

The “OrderedDict” from the “collections” module is similar to the standard dictionary but has the additional feature of maintaining the order of keys. It can be used to get unique values while preserving the list order in a one-liner approach.

Here’s an example:

from collections import OrderedDict
input_list = [1, 9, 2, 9, 1, 3, 2, 3, 4]
unique_list = list(OrderedDict.fromkeys(input_list))
print(unique_list)

[1, 9, 2, 3, 4]

Using the OrderedDict.fromkeys() function, we generate an OrderedDict with unique keys from the list. Converting its keys to a list gives us the unique ordered items.

Summary/Discussion

  • Method 1: Using Set. Fastest and simplest method. Does not maintain the order of elements.
  • Method 2: Using a For Loop. Maintains order. Less efficient for large lists due to the in operation within a loop.
  • Method 3: Using List Comprehension and an Auxiliary Set. Preserves order with better performance by leveraging a set for look-ups.
  • Method 4: Using Dictionary Keys. Python 3.7+ guarantees order of dictionary keys, reliable for unique ordered list. Slightly less straightforward than using a set.
  • Bonus Method 5: Using “OrderedDict”. Preserves order and is a one-liner, requires importing from collections.