5 Best Ways to Save a Python NumPy Array to File

by

💡 Problem Formulation: How do you save a NumPy array to a file for future use or data sharing? Whether you’re dealing with large datasets or simply need a convenient way to store array data, saving NumPy arrays efficiently can be a challenge. For example, let’s say you have a NumPy array representing image pixel data and you want to save it in a file for later analysis; choosing the right method to do so will depend on your specific needs for speed, readability, and file size.

Method 1: Using numpy.save()

Saving NumPy arrays using numpy.save() is straightforward and efficient. It saves arrays in a binary format with a .npy extension. This method is quick and the files are easily loaded back into Python using numpy.load().

♥️ Info: Are you AI curious but you still have to create real impactful projects? Join our official AI builder club on Skool (only $5): SHIP! - One Project Per Month

Here’s an example:

import numpy as np

# Creating a NumPy array
array_to_save = np.arange(10)

# Saving the array to a file
np.save('my_array.npy', array_to_save)

Output: A file named ‘my_array.npy’ will be created in the current directory.

This method is a great choice for saving a single array as the saved file is not human-readable but is very compact and perfect for efficient storage and retrieval in Python.

Method 2: Using numpy.savetxt()

If you need a human-readable format or want to save a two-dimensional array as a text file, numpy.savetxt() is ideal. It saves the array as a text file, with options to specify delimiter, header, and footer.

Here’s an example:

import numpy as np

# Generating a 2D NumPy array
array_to_save = np.reshape(np.arange(100), (10, 10))

# Saving as a text file
np.savetxt('my_array.txt', array_to_save, fmt='%d', delimiter=',')

Output: A text file named ‘my_array.txt’ will be created, containing integers separated by commas.

This method is very versatile for saving 2D arrays and can be used when you want to view or edit your data using a text editor or share it with someone who does not use Python.

Method 3: Using numpy.savez() and numpy.savez_compressed()

When it comes to saving multiple arrays at once, numpy.savez() or numpy.savez_compressed() are your go-to methods. The former saves multiple arrays into an uncompressed .npz file, while the latter adds compression to reduce file size.

Here’s an example:

import numpy as np

# Creating multiple arrays
arr1 = np.array([1, 2, 3])
arr2 = np.array([4, 5, 6])

# Saving multiple arrays into a .npz file
np.savez('arrays.npz', arr1=arr1, arr2=arr2)

# For compressed storage
np.savez_compressed('arrays_compressed.npz', arr1=arr1, arr2=arr2)

Output: Two files, ‘arrays.npz’ and ‘arrays_compressed.npz’, will be created in the current directory.

These methods are especially valuable when dealing with multiple related arrays. Using keyword arguments, you can label the arrays individually within the file, making it simple to load them back with recognizable names.

Method 4: Using Pickle

Pickle is Python’s way of serializing and deserializing objects. It can be used to save NumPy arrays, although it’s not specific to NumPy and may not be as optimized as numpy.save().

Here’s an example:

import numpy as np
import pickle

# Create a NumPy array
array_to_save = np.arange(5)

# Pickling the array to a file
with open('array.pkl', 'wb') as f:
    pickle.dump(array_to_save, f)

Output: A pickled file named ‘array.pkl’ will be created in the current directory.

This general-purpose method can save any Python object, including NumPy arrays. However, it is slower than the NumPy-specific methods and may result in larger files.

Bonus One-Liner Method 5: Using numpy.ndarray.tofile()

The numpy.ndarray.tofile() function offers a one-liner alternative to save an array to a binary or text file. This method is simple but does not save shape or dtype information.

Here’s an example:

import numpy as np

# Creating a NumPy array
array_to_save = np.random.rand(3, 3)

# Saving the array to a binary file
array_to_save.tofile('array.bin')

Output: A binary file named ‘array.bin’ will be created in the current directory.

The simplicity of this method is a big plus, but be aware that you lose metadata like the array’s shape and dtype, which can be critical for loading the data correctly later on.

Summary/Discussion

  • Method 1: numpy.save(). Optimized for NumPy arrays, providing fast saving and loading. Produces binary files that are not human-readable.
  • Method 2: numpy.savetxt(). Great for saving 2D arrays in a human-readable format. Offers flexibility with delimiters and formatting but can be less space-efficient.
  • Method 3: numpy.savez() / numpy.savez_compressed(). Best for saving multiple arrays together, with or without compression. Convenient for grouping related data.
  • Method 4: Pickle. Universal method for Python objects but not optimized for NumPy arrays. Use when NumPy-specific methods are not applicable.
  • Bonus One-Liner Method 5: numpy.ndarray.tofile(). Quick and easy, perfect for large datasets where metadata is not necessary or can be stored separately.