Python Generator Expressions

by

Short Overview

Python Generator Expressions provide an easy-to-read syntax for creating generator objects. They allow you to quickly create an iterable object by using a single line of code. A generator expression is like a list comprehension, but instead of creating a list, it returns an iterator object that produces the values instead of storing them all in memory.

Syntax Simple Generator Expression

A simple generator expression goes over each item of an iterable and applies an expression on each item. The return value is a generator of possibly modified items.

(expression for item in iterable)

Syntax Conditional Generator Expression

A conditional generator expression goes over each item of an iterable that satisfies a specified condition — and applies an expression on each of those. The return value is a generator of possibly modified items.

(expression for item in iterable if condition)

Examples

Here are ten examples of generator expressions:

  1. (num ** 2 for num in range(1,11))
  2. (num + 10 for num in range(1,11))
  3. (num % 2 == 0 for num in range(1,11))
  4. (str.upper() for str in ['cat', 'dog', 'mouse'])
  5. (x + y for x in [1,2,3] for y in [4,5,6])
  6. (x ** 2 for x in range(1, 101) if x % 5 == 0)
  7. (x for x in range(20) if x % 2 == 0)
  8. (x for x in range(0, 101, 3))
  9. (x for x in range(20) if x % 3 == 0 and x % 5 == 0)
  10. (x + y + z for x in [1,2,3] for y in [4,5,6] for z in [7,8,9])

⚽ Exercise: Try to find out what each generator expression does and check you solution against the following results!

  1. Squares of numbers from 1-10
  2. Add 10 to every number from 1-10
  3. Filter even numbers from 1-10
  4. Uppercase every string in list
  5. Cartesian product of [1,2,3] and [4,5,6]
  6. Squares of numbers divisible by 5 from 1-100
  7. Filter even numbers from 0-19
  8. Numbers from 0-100 in steps of 3
  9. Filter numbers divisible by 3 and 5 from 0-19
  10. Cartesian product of [1,2,3], [4,5,6] and [7,8,9]

Here’s the output of these generator expressions after converting them to lists.

  1. [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
  2. [11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
  3. [False, True, False, True, False, True, False, True, False, True]
  4. ['CAT', 'DOG', 'MOUSE']
  5. [5, 6, 7, 6, 7, 8, 7, 8, 9]
  6. [25, 50, 100, 225, 400, 625, 900, 1225, 1600]
  7. [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
  8. [0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54, 57, 60, 63, 66, 69, 72, 75, 78, 81, 84, 87, 90, 93, 96, 99]
  9. [0, 15, 30, 45, 60]
  10. [12, 13, 14, 15, 16, 17, 18, 19, 20, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27]

If I didn’t convert them to a list first, you couldn’t see the results because the real outputs are string representations of generator objects that are not human-readable.

>>> (num ** 2 for num in range(1,11))
<generator object <genexpr> at 0x000001627F4A3EB0>

Deep Dive

Generator expressions are a powerful tool for creating iterators in Python. They provide a concise way to define and create generators, allowing programmers to quickly and easily iterate over data structures with just a few lines of code.

👉 Recommended Tutorial: Understanding Generators In Python

Generator expressions are similar to list comprehensions, but instead of creating a list, they create a generator object. This generator object can be used to iterate over a data structure without having to create a new list.

💡 Advantage: This can save memory and increase performance since the generator object does not need to store the entire data structure in memory.

You can see the memory-saving capabilities of generator expression in this experiment (source):

Using generators instead of lists is significantly more memory-efficient because the objects are not instantiated in memory before needed.

Generator expressions are defined using the same syntax as list comprehensions, but with parentheses instead of brackets.

👉 Recommended: Understanding List Comprehension in Python

Inside the parentheses, an expression is used to define the values of the generator. For example, the following generator expression creates a generator object that contains the numbers from 1 to 10:

(x for x in range(1, 11))

Generator expressions can also be used to filter data. For example, the following generator expression creates a generator object that contains only the even numbers from 1 to 10:

(x for x in range(1, 11) if x % 2 == 0)

Generator expressions can also be used to create a generator object from an existing iterator.

👉 Recommended: Iterators, Iterables and Itertools

For example, the following generator expression creates a generator object from an existing list:

(x for x in my_list)

Summary

Generator expressions are a great tool for quickly and easily creating generator objects. They provide a concise way to define and create generators, allowing programmers to quickly and easily iterate over data structures with just a few lines of code.