Python Read Binary File

During your career as a Pythonista, you will most likely find yourself required to work with binary data. See the examples outlined below to efficiently read/write to a Binary file.

Preparation

Before any data manipulation can occur, one (1) new library will require installation.

• The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.

To install this library, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.

$ pip install numpy

Hit the <Enter> key on the keyboard to start the installation process.

If the installation was successful, a message displays in the terminal indicating the same.

Feel free to view the PyCharm installation guide for the required library.

• How to install Numpy on PyCharm

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import numpy as np 
import struct
from array import array

💡 Note: The struct and array libraries do not need installation as they come with Python.

Read a Binary File to an Array

For this example, a list of integers is saved to the binary file finxter-01.bin.

fp = open("finxter-01.bin", "wb")
nums = [12, 42, 45, 78, 96]
array = bytearray(nums)
fp.write(array)
fp.close()

• Line [1] shows a new file created in wb (write binary) mode. A file pointer is assigned to reference this file.
• Line [2] contains a list of integers assigned to the variable nums.
• Line [3] creates a bytearray of the variable nums and assigns this value to the variable array. The bytearray() method returns a bytearray object which is an array of bytes.
• Line [4] writes the variable array to the file opened earlier.
• Line [5] closes the previously opened file.

Reading From a Binary File

Reading from a binary file is similar to the steps taken above.

fp = open("finxter-01.bin", "rb")
nums = list(fp.read())
print(nums)
fp.close()

• Line [1] opens the finxter-01.bin file in rb (read binary) mode. A file pointer is assigned to reference the file.
• Line [2] reads in the list of integers from the file created above. The contents are saved to nums.
• Line [3] sends the output to the terminal.
• Line [4] closes the previously opened file.

Output

Read a Binary File into a String

For this example, this code writes a sentence to the finxter-02.bin file.

fp = open("finxter-02.bin", "wb")
fp.write(b"The journey of a thousand miles begins with a single step.")
fp.close()

• Line [1] opens a new binary file in wb (write binary) mode. A file pointer (fp) is assigned to reference this file.
• On line [2], the string is written to the file. Notice the b character at the start of the string. The b character denotes binary.
• Line [3] closes the previously opened file.

The contents of this file are read back in.

fp = open("finxter-02.bin", "rb")
string = fp.read()
print(string)
fp.close()

• Line [1] opens the finxter-02.bin binary file in rb (read binary) mode. A file pointer (fp) is assigned to reference this file.
• Line [2] reads in the file and saves it to a variable named string.
• Line [3] outputs the variable string to the terminal.
• Line [4] closes the previously opened file.

Output

Read in a Binary File and Write to Another Binary File

For this example, this code reads in the contents of finxter-02.bin and writes said contents to finxter-03.bin.

fp_from = open("finxter-02.bin", "rb")
string = fp_from.read()
fp_to = open("finxter-03.bin", "wb")
fp_to.write(string)
fp_from.close()
fp_to.close()

• Line [1] opens an existing binary file in rb (read binary) mode. A file pointer (fp_from) is assigned to reference this file.
• Line [2] reads in the file and saves it to a variable named string. 
• Line [3] opens a new binary file in wb (write binary) mode. Another file pointer (fp_to) is assigned to reference this file.
• Line [4] writes the contents of the variable string to the new file finxter-03.bin. 
• Lines [5-6] close the open files.

Read in a Binary File until EOF

For this example, we will be opening an existing binary file, finxter-03.bin. This file contains a few rows of binary data. This code uses a while loop to move through the file until the EOF marker has been reached.

with open("finxter-03.bin", "rb") as fp:
    while True:
        data = fp.read(10)
        if not data:
            break
        print(data)

• Line [1] opens the file slightly differently from other examples. By using Python’s with open method, a file pointer is still assigned. However, the close() method is not required. The open with code automatically closes the file after the code has stopped.
• Line [2] starts the while loop.
  • Line [3] reads in the data using the file pointer (fp) 10 characters/bytes at a time.
  • Line [4] checks to see if the EOF marker is reached.
    • If the EOF marker is reached, line [5] runs, and the code stops (break).
  • Otherwise, the code moves to line [6] and outputs ten characters/bytes of data to the terminal.

Output

Read a Binary File into a NumPy Array

This example uses Python’s famous NumPy library. The NumPy library makes working with arrays a breeze. NumPy also has functions for working in the domain of linear algebra, Fourier transform, and matrices. The wonderful thing about NumPy is that the code executes up to 50 times faster than Python’s traditional lists.

my_list = [10, 15, 20, 25, 30, 35, 40]
my_array = np.array(my_list).tofile("finxter-04.bin")
print(np.fromfile("finter-04.bin",  dtype=np.int32))

• Line [1] creates a list of integers and assigns it to a variable named my_list.
• Line [2] uses the np.array() method and saves the array to a file.
• Line [3] outputs the contents to the terminal.

Output

💡 Note: You will get different results based on the dtype entered. The choices are int8, int32, or int64. 

Read a Binary File Byte by Byte

From time to time, a binary file will need to be read byte by byte.

fp = open("finxter-04.bin", "rb")
byte = fp.read(1)
while byte:
    print(byte)
    byte = fp.read(1)
fp.close()

• Line [1] opens an existing binary file in rb (read binary) mode. A file pointer (fp) is assigned to reference this file.
• Line [2] reads in one (1) byte from the file is read in and saved to a variable named byte.
• LIne [3] instantiates a while loop that will continue until there are no more bytes to read in.
  • Line [4] outputs the byte to the terminal.
  • Line [5] reads in another byte.
• Line [6] closes the previously opened file.

Output (snippet)

Read a Binary File in Chunks

The best practice for reading in large files is to read in small portions (chunks). 

For this example, an image of a dog (dog.jpg) is placed in the current working directory before running the code.

chunk_size = 20
image_file = "dog.jpg"
with open(image_file, "rb") as infile:
    while True:
        chunk = infile.read(chunk_size)
        if not chunk:
            break
        print(chunk)

• Line [1] assigns the size of the chunk to the variable chunk_size.
• Line [2] assigns the variable image_file to the file to be read in.
• Line [3] opens the image_file.
• Line [4] instantiates the while loop and executes while True.
  • Line [5] reads in a chunk of the open file for each loop.
    • Line [6] checks if the variable chunk contains data; if no data is found, Line [7] executes.
  • Line [8] outputs the chunk to the terminal.

Output (snippet)

Read in a Binary File to ASCII

To convert the contents of a binary file to ASCII, you will need to use the encode() method. This method returns an encoded version of the string. In this case, it would be ascii.

fp = open("finxter-05.bin", "wb")
string = bytearray("Python is cool!".encode("ascii"))
fp.write(string)
fp.close()

• Line [1] creates and opens a file in wb (write binary) mode and assigns a file pointer (fp).
• Line [2] assigns the variable string the encoded bytearray string "Python is cool!".
• Line [3] writes the contents of the string to the file.
• Line [4] closes the previously open file.

fp = open("finxter-05.bin", "rb")
print(fp.read())
fp.close()

• Line [1] shown below opens the file created above in rb (read binary) mode.
• Line [2] displays the output to the console.
• Line [3] closes the previously open file.

Output

Read a Binary File using Little Endian

Four integers are saved to a new binary file in “little-endian” format for this example. The file is then read back in and will display these integers.

with open("finxter-06.bin", "wb") as fp:
    fp.write(struct.pack("<4i", 12, 45, 75, 98))

• Line [1] creates a new binary file named finxter-06.bin in wb (write binary) mode and assigns a file pointer (fp)
  • Line [2] saves four integers to a file using the struct.pack() method. This method requires two parameters:
    • The first parameter (“<4i”) indicates 4 integers saved in “little-endian” format.
    • The second parameter contains the integers to be saved.

The code below reads in the data saved earlier to an array.  

arr = array("i")
with open("finxter-06.bin", "rb") as fp:
    arr.fromfile(fp, 4)
print(arr)
array('i', [12, 45, 75, 98])

• Line [1] assigns the variable arr to an array of integers (as denoted by the: i).
• Line [2] opens the file created above in rb (read binary) mode, and a file pointer (fp) is assigned to reference this file.
  • Line [3] reads in the array from the file (the four integers saved above). 
• Line [4] outputs the data to the terminal.
• Line [5] is the output from this code.

Output

Feel free to join our free email academy with cheat sheets, Python, freelancing, and lots of fun! 🙂