5 Best Ways to Run Two Python Loops Concurrently

💡 Problem Formulation: When developing Python applications, you might face scenarios where you need to execute two or more loops at the same time without waiting for the other to complete. For example, you may want to monitor two data streams simultaneously or perform two independent series of calculations. To achieve concurrency, we’ll explore different methods to run two loops simultaneously, ensuring they execute in parallel resulting in a more efficient process.

Method 1: Using the threading Module

The threading module in Python provides a way to execute functions in separate threads. This method allows simultaneous execution of each loop in their own thread. Functions must be defined for the loops, and then started as threads.

Here’s an example:

import threading

def loop1():
    for i in range(5):
        print("Loop 1:", i)

def loop2():
    for j in range(5):
        print("Loop 2:", j)

t1 = threading.Thread(target=loop1)
t2 = threading.Thread(target=loop2)

t1.start()
t2.start()

t1.join()
t2.join()

Output:

Loop 1: 0
Loop 2: 0
Loop 2: 1
Loop 1: 1
... (output may vary)

This code initializes two threads named t1 and t2 associated with the loop1 and loop2 functions. Upon starting the threads with start(), each loop runs concurrently. The calls to join() ensure that the main program waits for both loops to complete before proceeding.

Method 2: Using the multiprocessing Module

The multiprocessing module allows you to create processes that can run on different CPU cores. By using this module, you can run loops concurrently and utilize multiple cores, which is particularly useful for computationally intensive tasks.

Here’s an example:

from multiprocessing import Process

def loop1():
    for i in range(5):
        print("Loop 1:", i)

def loop2():
    for j in range(5):
        print("Loop 2:", j)

p1 = Process(target=loop1)
p2 = Process(target=loop2)

p1.start()
p2.start()

p1.join()
p2.join()

Output:

Loop 1: 0
Loop 1: 1
...
Loop 2: 0
Loop 2: 1
... (output may vary)

Similar to the threading example, we define functions for our loops and create Process objects for each. Starting them with start() runs them in parallel and the join() methods are used to ensure that the main program waits for both processes to finish.

Method 3: Using asyncio

Python’s asyncio module is ideal for I/O-bound and high-level structured network code. Loops can be defined as asynchronous functions and executed concurrently using event loops.

Here’s an example:

import asyncio

async def loop1():
    for i in range(5):
        print("Loop 1:", i)
        await asyncio.sleep(0.1) # Simulate I/O with a sleep

async def loop2():
    for j in range(5):
        print("Loop 2:", j)
        await asyncio.sleep(0.1) # Simulate I/O with a sleep

async def main():
    await asyncio.gather(loop1(), loop2())

asyncio.run(main())

Output:

Loop 1: 0
Loop 2: 0
Loop 1: 1
Loop 2: 1
... (output may vary)

The asyncio.gather() function takes multiple asynchronous functions, schedules them to run concurrently, and waits for all of them to be completed. The asyncio.run() function is used to run the main event loop.

Method 4: Using Generators with next()

This method uses Python generators where each loop is a generator. The next() function is then used in a loop to alternately pull values from each generator, effectively interleaving the iterations.

Here’s an example:

def loop1():
    for i in range(5):
        yield "Loop 1: " + str(i)

def loop2():
    for j in range(5):
        yield "Loop 2: " + str(j)

gen1 = loop1()
gen2 = loop2()

for _ in range(5):
    print(next(gen1))
    print(next(gen2))

Output:

Loop 1: 0
Loop 2: 0
Loop 1: 1
Loop 2: 1
... (sequential continuation)

This code runs the loops sequentially but alternates between them, giving the appearance of concurrent execution. It is especially useful when the loops must communicate or pass data to one another.

Bonus One-Liner Method 5: Using list comprehensions with zip()

A list comprehension combined with zip() executes loops in parallel by element pairing. This method is limited by the shortest iterable and is not truly concurrent but can be suitable for simple parallel-like looping over data structures.

Here’s an example:

loop1 = (i for i in range(5))
loop2 = (j for j in range(5))

for l1, l2 in zip(loop1, loop2):
    print(f"Loop 1: {l1}, Loop 2: {l2}")

Output:

Loop 1: 0, Loop 2: 0
Loop 1: 1, Loop 2: 1
... (sequential continuation)

While this method appears to run the loops concurrently, it’s important to note that this is not true parallelism—it merely synchronizes iteration over two sequences.

Summary/Discussion

• Method 1: Threading. Easy to implement for lightweight concurrency. Not suitable for CPU-bound tasks due to the Global Interpreter Lock (GIL).
• Method 2: Multiprocessing. Good for CPU-intensive operations, bypasses the GIL. Higher memory overhead and may require IPC mechanisms for sharing data.
• Method 3: Asyncio. Best suited for I/O-bound tasks with high-level concurrency features. Learning curve involved and not suited for CPU-bound tasks.
• Method 4: Generators with next(). Allows manual control of execution order. Provides a level of concurrency within a single thread. Not truly parallel.
• Bonus Method 5: List Comprehensions with zip(). Simple and concise. Offers limited ‘concurrency’ and is restricted to the length of the shortest iterable.