Python Read Binary File

During your career as a Pythonista, you will most likely find yourself required to work with binary data.  To efficiently read/write to a Binary file, see the examples outlined below.

Read a Binary File to an Array

For this example, a list of integers is saved to a new binary file. The file is then read back in and displays the list of integers.

  • The code on line [1] shows a new file created in “wb” (write binary) mode. A file pointer is assigned to reference this file.
  • Line [2] contains a list of integers assigned to the variable “nums“.
  • Line [3] creates a bytearray of the variable “nums” and assigns this value to the variable “array“. The bytearray() method returns a bytearray object which is an array of bytes.
  • Line [4] writes the variable “array” to the file opened earlier. When using the “open” method, the file is not automatically closed when the code ends.
  • Line [5] closes the previously opened file.
fp = open("finxter-01.bin", "wb")
nums = [12, 42, 45, 78, 96]
array = bytearray(nums)
fp.write(array)
fp.close()

Reading from a binary file is quite similar to the steps taken above.

  • On line [1], the file is opened in “rb” (read binary) mode. A file pointer was assigned to reference the file.  
  • Line [2] reads in the list of integers from the file created above. The contents are saved to the variable “nums“.
  • Line [3] displays the output to the console.
  • Line [4] closes the previously opened file.
fp = open("finxter-01.bin", "rb")
nums = list(fp.read())
print(nums)
fp.close()

Read a Binary File into a String

As you will see in the code below,

  • line [1] opens a new binary file in β€œwb” (write binary) mode. A file pointer (fp) is assigned to reference this file.
  • On line [2], the string is written to the file. Notice the “b” character at the start of the string. The “b” character denotes binary.
  • Line [3] closes the previously opened file.
fp = open("finxter-02.bin", "wb")
fp.write(b"The journey of a thousand miles begins with a single step.")
fp.close()

In the code below,

  • line [1] opens the existing binary file in β€œrb” (read binary) mode.  A file pointer (fp) is assigned to reference this file.
  • On line [2], the file is read in and saved to a variable named “string“.
  • On line [3] the variable string is outputted to the console.
  • Line [4] displays the output.
  • Line [5] closes the previously opened file.
fp = open("finxter-02.bin", "rb")
string = fp.read()
print(string)
b'The journey of a thousand miles begins with a single step.'
fp.close()

Read in a Binary File and Write to Another Binary File

As you will see in the code below,

  • line [1] opens an existing binary file in β€œrb” (read binary) mode. A file pointer (fp_from) is assigned to reference this file.
  • On line [2] the file is read in and saved to a variable named “string“. 
  • The code on line [3] opens a new binary file in β€œwb” (write binary) mode. Another file pointer (fp_to) is assigned to reference this file.
  • Line [4] writes the contents of the variable β€œstring” to the new file β€œfinxter-03.bin“. 
  • Two open files need to be closed. Lines [5] and [6] perform this task.
fp_from = open("finxter-02.bin", "rb")
string = fp_from.read()
fp_to = open("finxter-03.bin", "wb")
fp_to.write(string)
fp_from.close()
fp_to.close()

Read in a Binary File until EOF

For this example, we will be opening an existing binary file, β€œfinxter-04.bin”. This file contains a few rows of binary data. This code uses a while loop to move through the file until the EOF marker has been reached.

  • Line [1] opens the file in a slightly different manner than other examples used above. By using Python’s “with open” method, a file pointer is still assigned. However, the close() method is not required. The “open with” code automatically closes the file after the code has stopped.
  • Line [2] starts the while loop.
  • Line [3] reads in the data using the file pointer (fp) 10 characters/bytes at a time.
  • Line [4] checks to see if the EOF marker was reached.
  • The code then moves to line [6] and outputs ten characters/bytes of data to the console.
  • If the EOF marker has been reached, line [5] runs, and the code stops (break).
with open("finxter-04.bin", "rb") as fp:
    while True:
        data = fp.read(10)
        if not data:
            break
        print(data)

Read a Binary File into a NumPy Array

This example uses Python’s famous NumPy library. The NumPy library makes working with arrays a breeze. NumPy also has functions for working in the domain of linear algebra, Fourier transform, and matrices. The wonderful thing about NumPy is that the code executes up to 50 times faster than Python’s traditional lists.

If an error occurs when running this code, you may need to install the NumPy library.

  • Line [1] imports the NumPy library to access the methods required to run this code.
  • On line [2] A list of integers is created and assigned to a variable named β€œmy_list”.
  • Line [3], using the np.array() method and saves the array to a file.
  • Line [4] outputs the contents to the console.

Note: you will get different results based on the dtype entered. The choices are int8, int32 or, int64. 

import numpy as np 
my_list = [10, 15, 20, 25, 30, 35, 40]
my_array = np.array(my_list).tofile("finter-05.bin")
print(np.fromfile("finter-05.bin",  dtype=np.int32))

Read a Binary File Byte by Byte

From time to time, a binary file will need to be read byte by byte.

  • Line [1] opens an existing binary file in β€œrb” (read binary) mode. A file pointer (fp) is assigned to reference this file.
  • On line [2] 1 byte from the file is read in and saved to a variable named “byte“.
  • On line [3], the while loop determines if the variable “byte” contains data. If the variable “byte” contains data, line [4] will output the byte to the console.
  • Line [5] reads in another byte. The while loop continues until no additional bytes are located. At that time, the while loop will stop.
  • Line [6] closes the previously opened file.
fp = open("finxter-06.bin", "rb")
byte = fp.read(1)
while byte:
    print(byte)
    byte = fp.read(1)
fp.close()

Read a Binary File in Chunks

When reading in large files, the best practice is to read in small portions (chunks) at a time. 

  • Line [1] assigns the size of the chunk to the variable β€œchunk_size“.
  • The code on line [2] assigns the variable β€œimage_file” to the file to be read in.
  • Line [3] opens the “image_file“.
  • The while loop starts on line [4]. If the loop evaluates to True, a chunk from the file is read into the variable “chunk” and sent to the console. If there are no chunks left to read, the loop terminates. If this is the case, the break statement will execute. Otherwise, the while loop continues.
chunk_size = 20
image_file = "dog.jpg"
with open(image_file, "rb") as infile:
    while True:
        chunk = infile.read(chunk_size)
        if not chunk:
            break
        print(chunk)

Read in a Binary File to ASCII

To convert the contents of a binary file to ASCII, you will need to use the encode() method.  This method returns an encoded version of the string. In this case, it would be “ascii“.

For this example, a binary file is created, named “finxter-08.bin” in “wb” (write binary) mode.  A file pointer (fp) is assigned to reference this file. On line [2] a variable named β€œstring” is created using the bytearray function to encode "Python is cool!". The contents of the variable “string” are written to the file opened earlier. The previously opened file is closed.

fp = open("finxter-08.bin", "wb")
string = bytearray("Python is cool!".encode("ascii"))
fp.write(string)
fp.close()

Line [1] shown below opens the file created above in β€œrb” (read binary) mode. Line [2] displays the output to the console. Line [3] closes the previously open file.

fp = open("finxter-08.bin", "rb")
print(fp.read())
fp.close()

Read a Binary File using Little Endian

For this example, four integers are saved to a new binary file in “little-endian” format. The file is then read back in and will display these integers.

Line [1] imports the struct library. This library contains the method, struct.pack(). This method converts data (for this example, integers) into binary format. On line [2], a new binary file was created named “finxter-09.bin” in “wb” (write binary) mode. A file pointer (fp) was assigned to reference this file.

Line [3] saves four integers to a file using the struct.pack() method. This method requires two parameters:

  1. The first parameter (β€œ<4i”) indicates 4 integers saved in “little-endian” format.
  2. The second parameter contains the integers to be saved.
import struct
with open("finxter-09.bin", "wb") as fp:
    fp.write(struct.pack("<4i", 12, 45, 75, 98))

The code below reads in the data saved earlier to an array.  

  • Line [1] imports the array library so that it can perform this task.  
  • Line [2] assigns the variable β€œarr” to an array of integers (as denoted by the: β€œi”).
  • Line [3] opens the file created above in β€œrb” (read binary) mode, and a file pointer (fp) is assigned to reference this file.
  • Line [4] then reads in the array from the file (the four integers saved above). 
  • Line [5] outputs the data to the console.
  • Line [6] is the output from this code.
from array import array
arr = array("i")
with open("finxter-09.bin", "rb") as fp:
    arr.fromfile(fp, 4)
print(arr)
array('i', [12, 45, 75, 98])

Feel free to join our free email academy with cheat sheets, Python, freelancing, and lots of fun! πŸ™‚