5 Best Ways to Find and Delete the Last Occurrence of Duplicate Elements in Python

Rate this post

πŸ’‘ Problem Formulation: Working with lists in Python often involves scenarios where you need to identify and remove duplicates – but with a twist. Consider a list [3, 5, 2, 3, 8, 5]. The task is to delete the last occurrence of the duplicate elements, resulting in [3, 5, 2, 8], keeping only the first occurrence of each duplicate. This article will explore five effective methods to achieve this in Python.

Method 1: Using a Reverse Traversal Loop

This method relies on reverse iteration over a list. By traversing the list from the end to the beginning, it is simpler to find and remove the last occurrence of each duplicate element without affecting earlier duplicates.

Here’s an example:

lst = [3, 5, 2, 3, 8, 5]
seen = set()
for i in range(len(lst) - 1, -1, -1):
    if lst[i] in seen:
        del lst[i]
    else:
        seen.add(lst[i])

Output: [3, 5, 2, 8]

This code first creates an empty set seen to store unique elements. It then iterates backward through the list and checks if an element is in the seen set. If so, it removes the element using del. Otherwise, it adds the element to the set.

Method 2: Using the OrderedDict

The OrderedDict from the collections module can be used to maintain the order of insertion of the elements and quickly identify duplicates. We can then build a new list with the order preserved but without the last occurrences of duplicates.

Here’s an example:

from collections import OrderedDict

lst = [3, 5, 2, 3, 8, 5]
reversed_lst = lst[::-1]
reversed_unique_lst = list(OrderedDict.fromkeys(reversed_lst))
result = reversed_unique_lst[::-1]

Output: [3, 5, 2, 8]

By reversing the original list and using OrderedDict.fromkeys(), we create a temporary list where the first occurrences (from the end) of each element are kept. Reversing this list provides the desired list with the last duplicates removed.

Method 3: Using a Dictionary

A standard Python dictionary can be utilized to keep track of the index of elements. We can delete the elements that have an index that appears later (indicating they are duplicates) in the list.

Here’s an example:

lst = [3, 5, 2, 3, 8, 5]
indexes = {}
for idx, elem in enumerate(lst):
    indexes[elem] = idx
lst = [elem for idx, elem in enumerate(lst) if indexes[elem] == idx]

Output: [3, 5, 2, 8]

The code uses a dictionary to map each element to its latest index. When constructing the final list, it includes only elements whose current index is the same as the one stored in the dictionary, effectively removing the last duplicate occurrences.

Method 4: Using List Comprehension and Any()

This method leverages Python’s list comprehension and the any() function to iterate over the list. It checks for duplicate occurrences and removes only the last occurrences, while maintaining the list order.

Here’s an example:

lst = [3, 5, 2, 3, 8, 5]
result = [elem for i, elem in enumerate(lst) if elem not in lst[i+1:]]

Output: [3, 5, 2, 8]

The code uses a list comprehension with a conditional statement to check if an element appears again in the list after the current index. The result list contains only elements not followed by duplicates, thereby excluding the last occurrences of duplicates.

Bonus One-Liner Method 5: Using a Function with List Comprehension

A more compact and elegant solution can be achieved by wrapping the list comprehension within a function. This one-liner method quickly finds and excludes the last occurrences of any duplicate elements within the given list.

Here’s an example:

def remove_last_duplicates(lst):
    return [elem for i, elem in enumerate(lst) if elem not in lst[i+1:]]

lst = [3, 5, 2, 3, 8, 5]
result = remove_last_duplicates(lst)

Output: [3, 5, 2, 8]

This cohesive code snippet encapsulates the duplicate removal logic inside a function named remove_last_duplicates. It allows for simple reuse and improved code readability, maintaining a polished codebase.

Summary/Discussion

  • Method 1: Reverse Traversal Loop. This method is straightforward and efficient, especially for larger lists, as it manipulates the list in place. However, its manual nature might make the code less Pythonic than other methods.
  • Method 2: OrderedDict. It is a clean and readable approach but comes with the overhead of importing a module and additional list reversals that might not be necessary.
  • Method 3: Dictionary Indexes. This method provides both clarity and performance, as it uses dict comprehension. However, it may not be as intuitive for beginners due to the dictionary/enum combination.
  • Method 4: List Comprehension and Any(). This method offers a concise solution, leveraging the power of list comprehension and is very Pythonic. The downside is the potential for higher computational complexity due to multiple scans of the list.
  • Bonus Method 5: Functional List Comprehension. This method is elegant and reusable, offering both brevity and clarity. However, hiding the logic in a function can obscure what’s happening for those unfamiliar with the approach.