How to Open Multiple Files in Python

Problem Formulation and Solution Overview

In this article, you’ll learn how to open multiple files in Python.

💬 Question: How would we write Python code to open multiple files?

We can accomplish this task by one of the following options:

• Method 1: Open Multiple Text Files using open()
• Method 2:Open Multiple Text Files using open() and backslash (\)
• Method 3: Open Multiple Text Files using Parenthesized Context Managers and open()
• Method 4: Open Multiple Text Files using the os library and open()

To make it more fun, we have the following running scenario:

You have been contacted by the Finxter Academy to write code that opens multiple files and writes the contents from one file to another.

Preparation & Overview

If your script opens files, it is a good practice to check for errors, such as:

• No Such File Or Directory
• Not Readable
• Not Writable

In this regard, all examples in this article will be wrapped in a try/except statement to catch these errors.

Contents of orig_file.txt

To follow along with this article, create orig_file.txt with the contents below and place it in the current working directory.

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import logging

This allows us to log any error message that may occur when handling the files.

Method 1: Open Multiple Text Files with open()

This method opens multiple text files simultaneously on one line using the open() statement separated by the comma (,) character.

try:
    with open('orig_file.txt', 'r') as fp1, open('new_file.txt', 'w') as fp2:
        fp2.write(fp1.read())
except OSError as error:
    logging.error("This Error Occurred: %s", error)

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Opens two (2) text files on one line of code.
  • The file for reading, which contains a comma (,) character at the end to let Python know there is another file to open: open('orig_file.txt', 'r') as fp1
  • The second is for writing: open('new_file.txt', 'w') as fp2
• Then, the entire contents of orig_file.txt writes to new_file.txt.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

💡 Note: The strip() function removes any trailing space characters. The newline character (\n) is added to place each iteration on its own line.

If successful, the contents of orig_file.txt and new_file.txt are identical.

Output of new_file.txt

Method 2: Open Files Across Multiple Lines with open() and Backslash (\)

Before Python version 3.10, and in use today, opening multiple files on one line could be awkward. To circumvent this, use the backslash (\) character as shown below to place the open() statements on separate lines.

try:
    with open('orig_file.txt', 'r') as fp1, \
         open('new_file3.txt', 'w')  as fp2:
            fp2.write(fp1.read())
except OSError as error:
    logging.error("This Error Occurred: %s", error)

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Opens the first file using open('orig_file.txt', 'r') for reading and contains a backslash (\) character to let Python know there is another file to open.
• Opens the second file using open('new_file.txt', 'w') for writing.
• Then, the entire contents of orig_file.txt writes to new_file.txt.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

💡 Note: The strip() function removes any trailing space characters. The newline character (\n) is added to place each iteration on its own line.

If successful, the contents of orig_file.txt and new_file.txt are identical.

Method 3: Open Multiple Text Files using Parenthesized Context Managers and open()

In Python version 3.10, Parenthesized Context Managers were added. This fixes a bug found in version 3.9, which did not support the use of parentheses across multiple lines of code. How Pythonic!

Here’s how they look in a short example:

try:
    with (
        open('orig_file.txt', 'r')  as fp1,
        open('new_file.txt',  'w') as fp2
    ):
        fp2.write(fp1.read())
except OSError as error:
    logging.error("This Error Occurred: %s", error)

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Declare the opening of with and the opening bracket (with ()).
  • Opens orig_file.txt (open('orig_file.txt', 'r') as fp1,) for reading with a comma (,) to let Python know to expect another file.
  • Open new_file.txt (open('new_file.txt', 'w') as fp2) for writing.
• Closes the with statement by using ):.
• Then, the entire contents of orig_file.txt writes to new_file.txt.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

If successful, the contents of orig_file.txt and new_file.txt are identical.

Method 4: Open Multiple Text Files using the os library and open()

This method calls in the os library (import os) which provides functionality to work with the Operating System. Specifically, for this example, file and folder manipulation.

import os 

os.chdir('files')
filelist = os.listdir(os.getcwd()) 

for i in filelist: 
    try: 
        with open(i, 'r') as fp: 
            for line in fp.readlines(): 
                print(line) 
    except OSError as error: 
        print('error %s', error) 

💡 Note: In this example, two (2) files are read in and output to the terminal.

This code snippet imports the os library to access the required functions.

For this example, we have two (2) text files located in our files directory:
file1.txt and file2.txt. To access and work with these files, we call os.chdir('files') to change to this folder (directory).

Next, we retrieve a list of all files residing in the current working directory
(os.listdir(os.getcwd()) and save the results to filelist.

IF we output the contents of filelist to the terminal, we would have the following: a list of all files in the current working directory in a List format.

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Instantiates a for loop to traverse through each file in the List and does the following:
  • Opens the current file for reading.
  • Reads in this file one line at a time and output to the terminal.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

Output

The contents of the two (2) files are:

Summary

These four (4) methods of how to multiple files should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!

Programmer Humor

❓ Question: How did the programmer die in the shower? ☠️

❗ Answer: They read the shampoo bottle instructions:
Lather. Rinse. Repeat.