Python Namespaces Made Simple

by
5/5 - (4 votes)

Namespace are everywhere in Python whether you realize it or not. If you don’t know about Python namespaces, you’ll eventually introduce nasty bugs into your Python code. Let’s fix this once and for all! 🙂

As you read over the article, you can watch my explainer video:

A Guide to Python Namespaces -- With Harry Potter Examples 🤓

Why Namespaces?

In many classes with 30+ students, two of them share the same name. The reason is a variant of the popular birthday problem: The teacher asks “Alice?” — and two Alices answer with “yes”. The origin of the problem is the existence of ambiguous names. Roughly speaking, real-world namespaces are not properly configured.

The same problem arises for the Python interpreter if you tell it to execute the function “alice()“. The name “Alice” may have been already defined in an imported module or in another part of your code.

? The goal of a namespace is to resolve these naming conflicts.

While the teacher will use more advanced techniques for disambiguation (like pointing to Alice on the left), the Python interpreter cannot decide which Alice is meant.

What is a Namespace?

A namespace is simply a collection of names. A name identifies an object (e.g. a function or a variable). So if you call the function f() in your code, the namespace tells the interpreter what function object to call.

A namespace in Python is implemented as a dictionary that assigns names to objects. The name uniquely identifies an object in your namespace. We will see an example in a moment.

You can print out the current namespace (i.e., the collection of names) with the function dir().

alice = 25

def birthday():
    alice += 1

print(dir())
# ['__annotations__', '__builtins__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'alice', 'birthday']"

Defining the variable "alice" and the function “birthday” has exactly the same effect on the namespace: each time a new name is added to the namespace.

Note that the namespace contains some default dunder (for “double underscore”) names. You’ll learn about them at the end of this article.

How Does Importing Modules Affect the Namespace?

You should always use libraries instead of reinventing the wheel. A library is a collection of modules. A module is a file containing code that you can reuse in your program. You load the code of the file 'module.py' into your program with one of the three following statements.

# 1. Import all the code from the file 'module.py'
import module
module.f()

# 2. Rename module in your own file
import module as m
m.f()

# 3. Import only specific functions of the module
from module import f
f()

What’s the problem? Say you define function f() in your program. But function f() is already defined in an imported module. This is bad. Suppose you tell the Python interpreter to execute f(). Should it execute your function f() or the module’s function f()?

## File module.py
def f():
    print("hi")


## File main.py
from module import f

def f():
    print("hello")


f()
# What's the output??

While the program runs perfectly fine (the result is "hello"), this way of implementation is very confusing. Because you never know which functions overwrite which function from another module.

Python solves this problem with namespaces. In the example above, namespaces resolve the problem of multiple definitions of f().

In the following example, we have the main program 'main.py' and the module 'wordifier.py'.

## File 'wordifier.py'
def negate(word):
    return 'de' + word


def verb(word):
    return word + 'ize'


## File 'main.py'
def add_yes(word):
    return word + ', yes!'


print(dir())
# [..., 'add_yes']

from wordifier import *

print(dir())
# [..., 'add_yes', 'negate', 'verb']

What is going on in this program?

First, we print the namespace BEFORE importing the module ‘wordifier‘. In this case, the namespace contains the name of the function defined in our main file.

Second, we print the namespace AFTER importing all names from the module ‘wordifier‘. In this case, the namespace contains the name of the function defined in both our main file AND the module.

Ok, so now you know that importing modules will change your namespace.

Use ‘from A import B‘ to import ONLY object name ‘B‘ from namespace ‘A‘ into your local namespace.

Use ‘from A import *‘ to import ALL names from namespace ‘A‘ into your local namespace.

Yet, if you use ‘import A‘, you do NOT import any new name from namespace ‘A‘ into your local namespace. Instead, you call function f() in module ‘A‘ like this: A.f().

Now you should be able to solve the following puzzle.

What is the output of this code snippet?

## File 'wordifier.py'
def negate(word):
    return 'de' + word


def verb(word):
    return word + 'ize'


## File 'main.py'
def add_yes(word):
    return word + ', yes!'


print(dir())
# [..., 'add_yes']

import wordifier

print('negate' in dir())
# True or False?

What’s the output of this code snippet?**

In summary, a namespace maps names to objects so that the Python interpreter knows which object you mean when you use a name. You can modify the namespace by importing some or all names from other modules.

**The output of the above code puzzle is “False”.

You have learned about the motivation and reasons for namespaces (disambiguation!), how to explore the current Python namespace (using the dir() function), and how the namespace is modified when you import modules (there are three different ways).

Next, you’ll dive deeper into namespaces by exploring the question:

What are Global, and Local Namespaces?

There are two types of namespaces: global and local namespaces. Global namespaces are visible throughout your Python code. Local namespaces are only visible within a function.

Each time you CALL a function, you implicitly create a new local namespace for this function. As the function terminates, the namespace is destroyed.

Each time you execute your project, you implicitly create a new global namespace. If you import a module or define a function globally, the respective names are added to the global namespace.

Consider the following code puzzle that shows you how to print the global and local namespaces to your shell:

wizards = ['Harry',
           'Hermione',
           'Ron']

## GLOBAL NAMESPACE
print(dir())
"""
['__annotations__', '__builtins__', '__doc__',
'__file__', '__loader__', '__name__',
'__package__', '__spec__', 'wizards']
"""

def encrypt(lst):
    secret = []
    for wiz in lst:
        secret.append(wiz[::-1])

    ## LOCAL NAMESPACE
    print(dir())
    # ['lst', 'secret', 'wiz']
    
    return secret

print(dir())
"""
['__annotations__', '__builtins__', '__doc__',
'__file__', '__loader__', '__name__',
'__package__', '__spec__', 'encrypt', 'wizards']
"""

print(encrypt(wizards))

(By the way: can you solve the code puzzle?)

The first call of the dir() function happens on the global level (no indentation), so the result is the global namespace with the default values and the 'wizards' name.

The second call of the dir() function happens on the local level (indentation), so the result is the local namespace with the variables used within the function. As soon as the function terminates, the namespace is released.

The third call of the dir() function happens on the global level (no indentation). But the result has changed as you have added a new global name to it: 'encrypt'. Note that the names from the local namespace ('lst', 'secret', 'wiz') do not exist within the global namespace anymore.

Say you execute the following statement now:

print(secret)

Python would throw a NameError that the name ‘secret‘ is not defined because the local namespace does not exist anymore.

This brings us to the next question:

What is a Scope?

In the last example, you have seen that you cannot use any name in your code wherever you want. If the namespace defining this name does not exist (or is not visible), you cannot use the name. If you try, Python will throw a NameError.

Thus, every name has a scope that exactly defines the location in your code where you can use it. The name ‘secret‘ can only be used within the function ‘encrypt‘. The name ‘encrypt‘ can be used anywhere in the program.

Interestingly, scopes are hierarchically structured:

  • The outermost (global) scope contains all built-in names (with the dunder notation). You can use them everywhere in any Python program.
  • The module-level scope contains the names defined in your Python program (first-level functions and variables), as well as the imported module names or imported names from these modules. You can use them anywhere in YOUR Python program.
  • The hierarchical local scopes: did you know that you can define functions within functions? So you can have different enclosing scopes of namespaces. We will dive deeper into this particular example in a moment.
  • The innermost scope limited to the current function in which your program executes. If the function terminates, the names in this scope are lost.

So let’s look at such an example with hierarchical scopes:

def outer():
    x = 42
    def inner():
        y = 21
        print(dir())
    inner()
    print(dir())

outer()
print(dir())


'''
Output 1:
['y']

Output 2:
['inner', 'x']

Output 3:
['__annotations__', '__builtins__',
'__doc__', '__file__', '__loader__',
'__name__', '__package__', '__spec__', 'outer']
'''

There are three hierarchical namespace levels in the code example:

  • The global namespace containing the default dunder names and the ‘outer‘ function name,
  • The local namespace of the ‘outer’ function containing the variable name ‘x‘ and the function name ‘inner‘, and
  • The local namespace of the ‘inner’ function containing the variable name ‘y‘.

In summary, there are multiple levels of namespaces defining the scope of the names, i.e., the specific part of the code where your names can be used without Python throwing an error.

[Short notice] This article is based on an email topic in my “Coffee Break Python” email series where I publish daily Python tutorials for continuous improvement in Python during your coffee breaks. Join us, it’s fun!

There are two important questions left:

What’s the Meaning of the ‘global’ Keyword in Python?

Let’s start with a surprising code puzzle:

magic_wiz = 'Harry'

def change_wiz():
    magic_wiz = 'Hermione'

change_wiz()
print(magic_wiz)

What’s the output of this code?

Commit to an answer before reading on.

By now, you’ve already learned about hierarchical namespaces in Python. So you should know that each time we enter a function, a new local namespace is created.

When entering the function “change_wiz()“, the new local namespace contains only the name “magic_wiz“. This name has nothing to do with the global name “magic_wiz” — it temporarily overwrites the global name.

Roughly speaking, when referring to the name “magic_wiz” within the function “change_wiz“, you cannot directly access the global name “magic_wiz“. The correct computer science term for this phenomenon is “name masking” because the name of the inner scope temporarily masks the name of the outer scope [Wiki].

Hence, the result of the above code puzzle is ‘Harry’. Surprising, isn’t it?

The global variable ‘magic_wiz‘ has never been changed by the function — the name ‘magic_wiz‘ in the inner namespace of the function masked the global name ‘magic_wiz‘.

How can we resolve this issue? Simply by using the “global” keyword to indicate that we mean the global name within the function:

magic_wiz = 'Harry'

def change_wiz():
    global magic_wiz
    magic_wiz = 'Hermione'

change_wiz()
print(magic_wiz)

Now, the output is 'Hermione' as the 'magic_wiz' name is explicitly defined as referring to the global namespace.

Before we wrap up with namespaces, I want to quickly answer one last question:

What Are the Default Names in the Namespace with Double Underscore (‘dunder’) Notation?

Have a look at this simple example:

magic_wiz = "Harry"

print(dir())
"""
['__annotations__', '__builtins__',
'__doc__', '__file__', '__loader__',
'__name__', '__package__', '__spec__',
'magic_wiz']
"""

We’ve only defined the last name in the namespace. The remaining names are included in the namespace by default. All of those default names are enclosed in double underscores (“dunders”).

Let’s print them one by one:

print(__annotations__)
# {}

print(__builtins__)
# <module 'builtins' (built-in)>

print(__doc__)
# None

print(__file__)
# C:\Users\xcent\Desktop\Finxter\Blog\PythonNamespaces\global_vs_local.py

print(__loader__)
# <class '_frozen_importlib.BuiltinImporter'>

print(__name__)
# __main__

print(__package__)
# None

print(__spec__)
# None

Without going too much into detail here, you can see that the dunder names provide us with useful meta information. For example, the variable ‘__file__‘ gives us the source file location of this program, while the variable ‘__name__‘ specifies whether this program is executed itself or merely imported as a module.

Conclusion

This article explores an important topic in Python: Namespaces. This is relevant, especially for debugging reasons. You can find some bugs only with a proper understanding of Python namespaces. Consider this done.

The article is based on my writings for the daily “Coffee Break Python” email series. I regularly make new mini-courses about diverse and important topics in computer science.

Check out the Finxter academy to learn about new mini-courses such as this one:

[Mini-Course] “The Coder’s Toolchain: How Everything Works Together in Python”