Python memoryview() — Tame That Strange Beast!

The Python memoryview(arg) function returns a memoryview object of the given bytes or bytearray argument. This exposes the argument’s internal data (buffers) to help you access the data without intermediate copying.

Syntax: memoryview(object)
ArgumentsobjectBytes or Bytearray object for which the memoryview should be returned
Return ValuememoryviewReturns memoryview of the object.

Python memoryview() — Usage Examples

Learn by example! In the following, we’re going to explore an example of why and how to use the memoryview() built-in function.

A simple use of memoryview() is to pass the byte string b'hello world' as an argument:

>>> view = memoryview(b'hello world')
>>> view
<memory at 0x0000025D2D26B1C8>

The view object is of type memoryview. You can access the Unicodes of each letter using simple indexing:

>>> view[0]
104

If you convert this first Unicode number to a character using the str() built-in function, you see that it relates to the first character of the bytestring, that is 'h'.

>>> chr(view[0])
'h'

Python memoryview() — Video

What is a Memoryview in Python?

The standard Python distribution, cPython, is implemented using the programming language C. If you create a bytes object or a bytearray in Python, this will be mapped to an object in memory—a C object, not a Python object. In essence, Python is just a virtualization on top of C! If you want to do some dirty things like accessing objects in memory, you can use the memoryview object that is really a reference to the real C object in memory—but that feels like any other Python object! More specifically, the memoryview object exposes the buffer interface as a Python object which can then be passed around like any other object.

“While each of these types have their own semantics, they share the common characteristic of being backed by a possibly large memory buffer. It is then desirable, in some situations, to access that buffer directly and without intermediate copying.”Docs

When you use bytes or bytearrays in Python, you often want to directly access the data in memory. The memoryview() function allows you to do so!

Performance Evaluation: How Much More Efficient is Using a Memoryview?

In the following experiment, we want to answer the question how much more efficient is the use of a memoryview object when creating a large number of slices on large bytestrings.

  • If you don’t use memoryviews, you’ll create a large number of intermediate objects—essentially copying the same data again and again.
  • If you use memoryviews, you avoid these redundant copies.

The effect can be very substantial as shown in the following evaluation graphic:

This graphic is the result of running the following code for the performance evaluation on my Win 10, Intel Core i7, 8th Gen machine:

import time

sizes = range(100000, 600000, 100000)


# Without Memoryview
l1 = []
for n in sizes:
    data = b'x' * n
    start = time.time()
    b = data
    while b:
        b = b[1:]
    stop = time.time()
    print(f'bytes {n} {stop-start}')
    l1.append(stop-start)


# With Memoryview
l2 = []
for n in sizes:
    data = b'x' * n
    start = time.time()
    b = memoryview(data)
    while b:
        b = b[1:]
    stop = time.time()
    print(f'memview {n} {stop-start}')
    l2.append(stop-start)


# Plot everything
import matplotlib.pyplot as plt
plt.plot(l1, 'x-', label='Without Memoryview')
plt.plot(l2, 'o--', label='With Memoryview')
plt.xlabel('Size of Bytearray')
plt.ylabel('Time (s)')
plt.legend()
plt.show()

The numerical results are as follows:

bytes 100000 0.1532435417175293
bytes 200000 0.47913265228271484
bytes 300000 1.1720850467681885
bytes 400000 2.15946888923645
bytes 500000 3.440741777420044
memview 100000 0.0
memview 200000 0.015674114227294922
memview 300000 0.03777813911437988
memview 400000 0.04686594009399414
memview 500000 0.05336737632751465

You can see that using a memoryview can result in drastic performance improvements!

When Should a Memoryview Be Used?

The previous example has shown that a memoryview is more efficient in many ways if you use slicing on large bytearrays. In effect, a memoryview is a generalized NumPy array:

“A memoryview is essentially a generalized NumPy array structure in Python itself (without the math). It allows you to share memory between data-structures (things like PIL images, SQLlite data-bases, NumPy arrays, etc.) without first copying. This is very important for large data sets. With it you can do things like memory-map to a very large file, slice a piece of that file and do calculations on that piece (easiest if you are using NumPy).” (source)


But before we move on, I’m excited to present you my new Python book Python One-Liners (Amazon Link).

If you like one-liners, you’ll LOVE the book. It’ll teach you everything there is to know about a single line of Python code. But it’s also an introduction to computer science, data science, machine learning, and algorithms. The universe in a single line of Python!

The book was released in 2020 with the world-class programming book publisher NoStarch Press (San Francisco).

Link: https://nostarch.com/pythononeliners


Summary

The Python memoryview(arg) function returns a memoryview object of the given bytes or bytearray argument.

This exposes the argument’s internal data (buffers) to help you access the data without intermediate copying.

>>> x = memoryview(b'42')
>>> x[0]
52
>>> x[1]
50
>>> x[2]
Traceback (most recent call last):
  File "<pyshell#7>", line 1, in <module>
    x[2]
IndexError: index out of bounds on dimension 1

I hope you enjoyed the article! To improve your Python education, you may want to join the popular free Finxter Email Academy:

Do you want to boost your Python skills in a fun and easy-to-consume way? Consider the following resources and become a master coder!

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!