5 Best Ways to Delete Specific Lines from a Text File in Python

by

💡 Problem Formulation: Occasionally in programming, you may encounter the need to modify text files by removing specific lines. For instance, say you have a file named “diary.txt” and want to remove the 3rd line from it. The chosen method should efficiently isolate and delete the target line, resulting in a file that no longer contains the unwanted content.

Method 1: Using a Temporary List

This method involves reading all the lines of the file into memory, manipulating the line list to remove the specific line, and then writing the modified content back to the file. It’s simple and direct, but not memory-efficient for very large files.

Here’s an example:

with open('diary.txt', 'r') as file:
    lines = file.readlines()

lines = [line for line in lines if not line.startswith('Dear Diary, today was...')]

with open('diary.txt', 'w') as file:
    file.writelines(lines)

Output: The specified line starting with “Dear Diary, today was…” will be deleted from “diary.txt”.

The code reads the entire file into a list of lines. It then uses a list comprehension to recreate the list without the lines starting with a specific prefix, effectively deleting those lines. Finally, the modified list is written back to the file.

Method 2: Using fileinput Module

The fileinput module allows for in-place file text manipulation whereby the file’s content is processed and the file is updated during iteration. This method avoids loading the entire file into memory, helping with large files.

Here’s an example:

import fileinput
import sys

line_to_delete = "Unremarkable day."
for line in fileinput.input('diary.txt', inplace=True, backup='.bak'):
    if line.strip('\n') != line_to_delete:
        sys.stdout.write(line)

Output: The exact line reading “Unremarkable day.” is removed, and a backup file “diary.txt.bak” is created.

In this snippet, each line of the file is processed one at a time. If a line matches the one to delete, it’s skipped from being written back. Changes are made in-place with a backup file created to prevent data loss.

Method 3: Reading and Writing Simultaneously

By opening the input and output files simultaneously, we can copy lines from the source to the destination, skipping the line to delete. This method is memory-efficient but requires a temporary output file.

Here’s an example:

with open('diary.txt', 'r') as read_file, open('diary_temp.txt', 'w') as write_file:
    for line in read_file:
        if not line.strip() == "I forgot to water the plants.":
            write_file.write(line)

import os
os.replace('diary_temp.txt', 'diary.txt')

Output: A line stating “I forgot to water the plants.” is removed from “diary.txt”.

This approach reads from the original file and writes to a new file, omitting the unwanted line. The original file is then replaced with the new file, effectively deleting the specified line.

Method 4: Using sed Command with subprocess Module

For UNIX-based systems, you can leverage the sed command through Python’s subprocess module to delete lines directly in the terminal. This is highly efficient but system-dependent.

Here’s an example:

import subprocess

line_number = "3"
file_name = "diary.txt"
subprocess.run(['sed', '-i', '{}d'.format(line_number), file_name])

Output: The third line is removed from the file named “diary.txt”.

This method uses the shell utility ‘sed’ to delete a line by its number. The ‘subprocess’ module allows Python to interact with the system’s command-line interface to execute the command.

Bonus One-Liner Method 5: Stream Editing with a Pipe

Leverage Python’s ability to execute shell commands to use commands like sed or awk in a UNIX environment. This one-liner is concise and efficient for users comfortable with command-line operations.

Here’s an example:

import os

os.system("sed -i '3d' diary.txt")

Output: The third line in “diary.txt” will be deleted.

In this command, sed is used with the -i flag to delete the third line of the specified file, all within a single line of Python code.

Summary/Discussion

  • Method 1: Using a Temporary List. Easy to understand. Not suitable for large files due to memory consumption.
  • Method 2: Using fileinput Module. Efficient for large files with in-place editing. Requires careful handling to avoid accidental data loss.
  • Method 3: Reading and Writing Simultaneously. Memory efficient. Less straightforward due to the need for a temporary file and replacing the original file.
  • Method 4: Using sed Command with subprocess Module. Highly efficient and fast. Works only on UNIX-based systems and requires understanding of shell commands.
  • Method 5: Stream Editing with a Pipe. Extremely concise. System-dependent and requires familiarity with Unix command-line tools.