5 Best Ways to Return the Minimum of an Array Ignoring NaNs in Python

by

💡 Problem Formulation: You have an array in Python which may contain both real numbers and NaN (not a number) values. You need to find the minimum value in this array, ignoring any NaNs and considering the possibility of positive infinity. For example, an input [3, NaN, 7, Inf, -1] should yield an output -1, as it’s the minimum value ignoring NaN and Inf.

Method 1: Using NumPy’s nanmin Function

NumPy’s nanmin function specifically tackles this problem by providing a way to return the minimum of an array while ignoring NaNs. This function treats positive infinity as a valid number, hence allowing to compute the minimum value considering Inf.

Here’s an example:

import numpy as np

array_with_nans = np.array([3, np.nan, 7, np.inf, -1])
min_value = np.nanmin(array_with_nans)

print(min_value)

Output: -1.0

This code snippet creates a numpy array that includes real numbers, NaN, and Inf. By calling np.nanmin(), we obtain the smallest numerical value in the array, disregarding NaN elements. The result is -1.0, which correctly ignores the NaN and treats Inf as a larger number.

Method 2: Using Pandas’ min Function with skipna=True

Pandas Series has a min function with a skipna parameter that can be set to True. This tells Pandas to ignore any NaN values during computation, similarly to how NumPy’s nanmin function works.

Here’s an example:

import pandas as pd

series_with_nans = pd.Series([3, pd.NA, 7, float('inf'), -1])
min_value = series_with_nans.min(skipna=True)

print(min_value)

Output: -1

Here, we use Pandas to create a Series object that includes pd.NA as a NaN representation, alongside regular numbers and float('inf'). Invoking the .min(skipna=True) method yields the minimum value excluding any NaNs. As shown in the result, -1 is correctly identified as the minimum.

Method 3: Filter and Min Function

By using the built-in filter function alongside min, you can remove NaNs from consideration before applying the min function to find the smallest number. Since Python treats positive infinity as a higher value than any real number, it will be naturally ignored in the minimum calculation.

Here’s an example:

import math

array_with_nans = [3, math.nan, 7, float('inf'), -1]
filtered_array = filter(lambda x: not math.isnan(x), array_with_nans)
min_value = min(filtered_array)

print(min_value)

Output: -1

The example leverages a lambda function within filter() to discard any values where math.isnan() returns True. This effectively removes NaN values from the list before finding the minimum with the built-in min() function. The final output is, once again, -1.

Method 4: List Comprehension and Min Function

A similar approach to using filter is to utilize list comprehension to create a new list without NaN values. This way is often more pythonic and readable. The math.isfinite() function helps to check for both NaN and Inf.

Here’s an example:

import math

array_with_nans = [3, math.nan, 7, float('inf'), -1]
cleaned_array = [x for x in array_with_nans if math.isfinite(x)]
min_value = min(cleaned_array)

print(min_value)

Output: -1

This code snippet utilizes list comprehension to exclude non-finite numbers (including both NaN and Inf) from the list before finding the minimum value. The condition math.isfinite(x) ensures that only finite numbers are present, and hence the minimum value calculation is accurately performed.

Bonus One-Liner Method 5: The Walrus := Operator

In Python 3.8 and later, the walrus operator := can be used in a list comprehension to compactly filter out NaN values and calculate the minimum in one line.

Here’s an example:

import math

array_with_nans = [3, math.nan, 7, float('inf'), -1]
min_value = min((x for x in array_with_nans if (is_finite := math.isfinite(x)) & (x != float('inf'))))

print(min_value)

Output: -1

This clever trick nests the conditional statement for finiteness inside the generator expression, assigning the result to the variable is_finite using the walrus operator. It then further refines the condition to also check for positive infinity, providing an immediate, filtered context for min() to operate within. The result is an elegant one-liner that delivers the correct minimum value.

Summary/Discussion

  • Method 1: NumPy’s nanmin. Straightforward with a library designed for numerical computation. Requires NumPy installation.
  • Method 2: Pandas’ min. Convenient for those already using Pandas for data analysis. Requires Pandas installation.
  • Method 3: Filter and Min. Utilizes pure Python without additional libraries. Less compact than other methods but clear and explicit.
  • Method 4: List Comprehension. Pythonic, easily readable, and no external libraries needed. Not as concise as NumPy or Pandas solutions.
  • Method 5: Walrus Operator. Compact and modern Python syntax. Only available in Python 3.8 and above.