What Is Python Output Buffering and How to Disable It?

by

Summary: Python output buffering is the process of storing the output of your code in buffer memory. Once the buffer is full, the output gets displayed on the standard output screen. Buffering is enabled by default and you can use one of the following methods to disable it:

  • Execute the Python code with the -u command line switch, for example python -u code.py
  • Use the flush keyword
  • Use sys.stdout.flush()
  • Wrap sys.stdout in TextIOWrapper and set the buffered size to 0
  • Set PYTHONUNBUFFERED to a non-empty string

Problem: Given a Python program; how to disable output buffering?

Overview Of Python Output Buffering

Before we learn buffering in the context of Python, we should ponder upon what buffer means in real life? Let me answer this question with another question. When you are in a stressful situation, whom or what do you look up to as an option to lessen your stress? On a personal level, I look up to my friends and family. So they are my “buffer” against stress.

According to the dictionary definition, a buffer is a person or thing that reduces a shock or that forms a barrier between incompatible or antagonistic people or things. This is what exactly a buffer does when it comes to a programming language like Python (though it is not a person in this case).

In terms of computer science, you may consider buffer as a middle layer which ensures a more efficient way for data to be transferred from one location to another in the system. For example, if you are downloading a 100 MB file and 1 MB gets written at a time to the hard disk would mean that it would require a total of 100 disk writes. However, a buffer could ensure that 10 MB gets written at a time to the disk, and this way only 10 disk writes will be needed.

Advantages and Disadvantages:

From the above scenario, we learned how buffer can act as an intermediate resource and reduce the number of disk writes thereby enhancing write efficiency and memory management. However, when it comes to a programming language, buffering has a downside too. We cannot see the output in real-time. This does not make much of a difference in case of simple or short programs but in complex and long programs that require a lot of computation, this might become a major drawback since we might need a more granular output rather than the entire output being displayed at once for debugging and testing purposes.

Example: Consider the following program for which we shall compare the buffered and unbuffered outputs:

for buffer in range(20):
    print(buffer, end=" ")

We know that the Output is a range of numbers from 0 to 19, that is

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

But we are more interested in understanding the nature of the output. If buffering is enabled you will get the output at once on the screen after waiting for 19 seconds. However, when buffering is disabled the numbers will be displayed one by one every 2 seconds.

You can try this in our interactive shell:

Exercise: Which behavior do you observe?

How To Disable Output Buffering In Python?

By default output buffering is enabled in Python. Let us have a look at the different ways in which we can disable output buffering in python.

Method 1: Running Python With -u Command Line Arguement

If you want to skip buffering for the entire program then the simplest solution to do this is to run your program using the command line interface and use the -u switch to bypass buffering. The syntax to disable buffering through the command line is:

python -u filename.py

Output

Method 2: Using The Flush Keyword

If you want to skip buffering for specific lines of your code, passing the flush keyword as an argument to the print statement is a good option. By default, the flush argument is set as False. To ensure that output buffering is disabled, the flush argument should be set to True.

Let us have a look at the following example to understand the syntax and usage of the flush keyword:

import time
for buffer in range(20):
    print(buffer, end=" ", flush=True)
    time.sleep(2)

Output

This image has an empty alt attribute; its file name is Untitled1.gif

Method 3: Using sys.stdout.flush()

Using the sys.stdout.flush() forces the program to flush the buffer. Therefore everything in the buffer will be displayed on the standard output without delay.

Let us have a look at how sys.stdout.flush() works in the following program:

import time
import sys


for buffer in range(20):
    print(buffer, end=" ")
    sys.stdout.flush()
    time.sleep(2)

Output

This image has an empty alt attribute; its file name is Untitled1.gif

Method 4: Wrapping sys.stdout Using TextIOWrapper And Setting Buffered Size = 0

Another approach to disable the output buffering in python is to open up the stdout in write mode with buffer size set to 0 (unbuffered) and then wrapping it using TextIOWrapper to ensure that you get a TextIO stream and not a binary file object. Also, you must ensure to enable the write_through to eliminate all buffers.

Let us have a look at the following code to get a better grip on the above explanation:

# open the stdout file descriptor in write mode and set 0 as the buffer size: unbuffered
import io
import os
import sys
import time

try:
    # open stdout in binary mode, then wrap it in a TextIOWrapper and enable write_through
    sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True)
    # for flushing on newlines only use :
    # sys.stdout.reconfigure(line_buffering=True)
except TypeError:
    # In case you are on Python 2
    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

print("Enter a Line: ")
line = sys.stdin.readline()
for i in line:
    print(i, end=" ")
    time.sleep(2)

Output

Method 5: Setting Python Environment Variable PYTHONUNBUFFERED

When the PYTHONUNBUFFERED variable is set to a non-empty string, it is equivalent to the -u command line option.

Conclusion

I hope this article helped you to get an overview of Python output buffering and the methods to disable it. Please stay tuned for more interesting articles.

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

🚀 If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!