# NumPy Broadcasting – A Simple Tutorial

Broadcasting describes how NumPy automatically brings two arrays with different shapes to a compatible shape during arithmetic operations. Generally, the smaller array is “repeated” multiple times until both arrays have the same shape. Broadcasting is memory-efficient as it doesn’t actually copy the smaller array multiple times.

Here’s a minimal example:

```import numpy as np

A = np.array([1, 2, 3])
res = A * 3 # scalar is broadcasted to [3 3 3]
print(res)
# [3 6 9]
```

Let’s have a more gentle introduction into the motivation and the concept next.

## A Visual Representation

The following visual representation of two broadcasting operations shows

• how a scalar (0-dimensional) value can be broadcasted to a 1-dimensional array, and
• how a 1-dimensional array can be broadcasted to a 2-dimensional array.

In both cases, you essentially copy the smaller array until you reach the same shape of the larger array.

## Gentle Introduction

Many NumPy operations such as multiplication are performed “element-wise”. For example: say you multiply two NumPy arrays `A` and `B` with the same shape: `A * B`.

In this case, NumPy performs element-wise matrix multiplication by multiplying cell `A[i,j]` with cell `B[i,j]` for every cell in the arrays. As the arrays have the same shape, that’s not a problem.

```import numpy as np

salary = np.array([2000, 4000, 8000])
salary_bump = np.array([1.1, 1.1, 1.1])

print(salary * salary_bump)```

Puzzle: What’s the output of this code snippet?

The puzzle shows a scenario where the company decides that all employees get a salary bump by 10% after a successful year.

Thus, the result of the puzzle is the NumPy array:

`# [2200. 4400. 8800.]`

So far so good. But what happens if you multiply two arrays with different shapes?

The answer is again: broadcasting. Here is the same example but we are a bit lazy now and try to save a few bits.

```salary = np.array([2000, 4000, 8000])
salary_bump = 1.1

print(salary * salary_bump)```

Puzzle: What’s the output of this code snippet?

As all three values of the `salary_bump` are exactly the same numbers, you try to shorten it by multiplying a NumPy array with a number. Thus, it produces exactly the same output.

`# [2200. 4400. 8800.]`

Internally, NumPy creates a second implicit array that is filled with the salary bump value 1.1. This array looks as follows: `np.array([1.1, 1.1, 1.1])`. This procedure is called “broadcasting”.

Not only is the code using broadcasting more concise and readable, but it is also more efficient! In the first example, the `salary_bump` array contains redundancies. But in the second example, NumPy gets rid of these redundancies — it does not really copy the data into a second NumPy array, this is only the conceptual idea but not the implementation.

## How to Apply Broadcasting to Two NumPy Arrays?

Broadcasting is only possible if, along a single dimension, one array has a size of n but the other a size of 1. Otherwise, you cannot perform NumPy operations on two arrays with different shapes.

Missing dimensions are not a problem in this scenario. Here is a nice visualization from the documentation of how NumPy arrays will be broadcasted together:

``` A      (2d array):  5 x 4
B      (1d array):      1
Result (2d array):  5 x 4

A      (2d array):  5 x 4
B      (1d array):      4
Result (2d array):  5 x 4

A      (3d array):  15 x 3 x 5
B      (3d array):  15 x 1 x 5
Result (3d array):  15 x 3 x 5

A      (3d array):  15 x 3 x 5
B      (2d array):       3 x 5
Result (3d array):  15 x 3 x 5

A      (3d array):  15 x 3 x 5
B      (2d array):       3 x 1
Result (3d array):  15 x 3 x 5```

An important observation is the following: For any dimension where the first array has a size of one, NumPy conceptually copies its data until the size of the second array is reached. Moreover, if the dimension is completely missing for array B, it is simply copied as well along the missing dimension.

Exercise: try to understand what is happening here in the first example of the given visualization.

In summary, broadcasting automatically matches two arrays with incompatible shape — a beautiful feature of the NumPy library! 