5 Best Ways to Find All Occurrences of a Substring within a List of Strings in Python

💡 Problem Formulation: We are often faced with the task of finding all instances of a specific substring within a list of strings. This is a common string manipulation challenge in Python. For instance, given a list of sentences and a search term, we want to obtain a list of all occurrences of that term within each string. Let’s tackle this problem using several Python methods.

Method 1: Using a List Comprehension and str.find

Here’s a list comprehension approach that utilizes the str.find method. This will iterate over each string in the list, finding occurrences of the substring and returning their indices.

Here’s an example:

def find_substring_occurrences(string_list, substring):
    return [[i for i in range(len(s)) if s.startswith(substring, i)] for s in string_list]

sample_strings = ["find the substring find", "no occurrence here", "hidden findsub"]
print(find_substring_occurrences(sample_strings, "find"))

The output will be:

[[0, 17], [], [7]]

This code snippet defines a function that uses list comprehensions to iterate through a list of strings, checking for the specified substring. For each string, it creates a list of start indices where the substring is found. The outer list comprehension aggregates these lists into a list of lists, making it clear where each occurrence is found within the input list.

Method 2: Using Regex re.finditer()

Regular expressions are powerful for string searching. Python’s re.finditer() method returns an iterator yielding match objects over all non-overlapping matches for the regex pattern in the string.

Here’s an example:

import re

def find_substring_occurrences_regex(string_list, substring):
    regex = re.compile(substring)
    return [[m.start() for m in regex.finditer(s)] for s in string_list]

sample_strings = ["find the substring find", "again find here yes", "nope!"]
print(find_substring_occurrences_regex(sample_strings, "find"))

The output will be:

[[0, 17], [6], []]

This code snippet defines a function that compiles a regular expression pattern for the given substring and uses finditer() to find all occurrences in each string. The list comprehension captures the start indices of these matches, providing a structured output that represents the positions of all occurrences.

Method 3: Using String Method str.count() with Indices

The str.count() method returns the number of occurrences of a substring in the given string. Used with indices, we can find all positions of the substring.

Here’s an example:

def find_substring_counts(string_list, substring):
    occurrences = []
    for s in string_list:
        count = s.count(substring)
        indices = []
        start = 0
        while count > 0:
            index = s.find(substring, start)
            indices.append(index)
            start = index + len(substring)
            count -= 1
        occurrences.append(indices)
    return occurrences

sample_strings = ["find the substring", "find and find again", ""]
print(find_substring_counts(sample_strings, "find"))

The output will be:

[[0], [0, 9], []]

This function utilizes the str.count() method to determine how many times to loop and find the substring. By tracking the starting index for each subsequent search, we can accumulate all indices in a list, resulting in a comprehensive mapping of all substring occurrences within the strings.

Method 4: Custom Function Using String Indexing and Slicing

Building a custom function that utilizes string indexing and slicing can offer precise control over substring searching functionality.

Here’s an example:

def custom_find_substring(string_list, substring):
    result = []
    for s in string_list:
        temp = []
        index = 0
        while index < len(s):
            index = s.find(substring, index)
            if index == -1:
                break
            temp.append(index)
            index += len(substring)
        result.append(temp)
    return result

sample_strings = ["this is a find test", "still finding", "nothing to find here"]
print(custom_find_substring(sample_strings, "find"))

The output will be:

[[10], [6], [13]]

This code iteratively searches for the substring within each string, using .find() with an updated index each iteration to find subsequent occurrences. The index is increased by the length of the substring to avoid overlapping matches, and the process repeats until no further matches are found.

Bonus One-Liner Method 5: Using re.findall() with List Comprehensions

The re.findall() function can be succinctly combined with list comprehensions to search for all substring occurrences. This method is compact yet powerful.

Here’s an example:

import re

def one_liner_find_substring(string_list, substring):
    return [m.start() for s in string_list for m in re.finditer(substring, s)]

sample_strings = ["find the finds", "not here", "also find here"]
print(one_liner_find_substring(sample_strings, "find"))

The output will be:

[0, 9, 28]

This one-liner function leverages the simplicity of list comprehensions and the practicality of regular expressions. It creates a single list of start indices for each match of the substring across all strings in the list, flattening the results into one aggregated list.

Summary/Discussion

• Method 1: List Comprehension with str.find. Strengths: Simple, readable. Weaknesses: May not handle overlapping substrings elegantly.
• Method 2: Using Regex re.finditer(). Strengths: Powerful for complex patterns, handles overlapping matches. Weaknesses: Can be slower for simple tasks or large data sets.
• Method 3: Using str.count() with Indices. Strengths: Built-in methods, easy to understand. Weaknesses: Inefficient for multiple occurrences.
• Method 4: Custom Function Using Indexing and Slicing. Strengths: Full control over functionality. Weaknesses: More complex code, possibly less efficient.
• Bonus Method 5: One-Liner with re.findall(). Strengths: Concise, one-liner. Weaknesses: Returns a flat list, losing information about which string had the occurrences.