Check for NaN Values in Python

by

Overview

Problem: How to check if a given value is NaN?

Here’s a quick look at the solutions to follow:

import math
import numpy as np
import pandas as pd

x = float('nan')
print(math.isnan(x))
print(x != x)
print(np.isnan(x))
print(pd.isna(x))
print(not(float('-inf') < x < float('inf')))

So, what is a NaN value?

NaN is a constant value that indicates that the given value is Not a Number. It’s a floating-point value, hence cannot be converted to any other type other than float. We should know that NaN and Null are two different things in Python. The Null values indicate something which does not exist, i.e. is empty. But that is not the case with NaN.

We have to deal with NaN values frequently in Python especially when we deal with array objects or DataFrames. So, without further delay, let us dive into our mission critical question and have a look at the different methods to solve our problem.

Method 1: Using math.isnan()

The simplest solution to check for NaN values in Python is to use the mathematical function math.isnan().

math.isnan() is a function of the math module in Python that checks for NaN constants in float objects and returns True for every NaN value encountered and returns False otherwise.

Example: 

# Importing the math module
import math


# Function to check for NaN values
def isNaN(a):
    # Using math.isnan()
    if math.isnan(a):
        print("NaN value encountered!")
    else:
        print("Type of Given Value: ", type(a))


# NaN value
x = float('NaN')
isNaN(x)
# Floating value
y = float("5.78")
isNaN(y)

Output:

NaN value encountered!
Type of Given Value:  <class 'float'>

In the above example, since x represents a NaN value, hence, the isNaN method returns True but in case of y , isNan returns False and prints the type of the variable y as an output.

Method 2: Hack NaN Using != Operator

The most unique thing about NaN values is that they are constantly shapeshifting. This means we cannot compare the NaN value even against itself. Hence, we can use the != (not equal to) operator to check for the NaN values. Thus, the idea is to check if the given variable is equal to itself. If we consider any object other than NaN, the expression (x == x) will always return True. If it’s not equal, then it is a NaN value.

Example 1:

print(5 == 5)
# True
print(['a', 'b'] == ['a', 'b'])
# True
print([] == [])
# True
print(float("nan") == float("nan"))
# False
print(float("nan") != float("nan"))
# True

Example 2:

# Function to check for NaN values
def not_a_number(x):
    # Using != operator
    if x != x:
        print("Not a Number!")
    else:
        print(f'Type of {x} is {type(x)}')


# Floating value
x = float("7.8")
not_a_number(x)
# NaN value
y = float("NaN")
not_a_number(y)

Output:

Type of 7.8 is <class 'float'>
Not a Number!

Method 3: Using numpy.isnan()

We can also use the NumPy library to check whether the given value is NaN or not. We just need to ensure that we import the library at the start of the program and then use its np.isnan(x) method.

The np.isnan(number) function checks whether the element in a Numpy array is NaN or not. It then returns the result as a boolean array.

Example: In the following example we have a Numpy Array and then we will check the type of each value. We will also check if it is a NaN value or not.

import numpy as np

arr = np.array([10, 20, np.nan, 40, np.nan])
for x in arr:
    if np.isnan(x):
        print("Not a Number!")
    else:
        print(x, ":", type(x))

Output:

10.0 : <class 'numpy.float64'>
20.0 : <class 'numpy.float64'>
Not a Number!
40.0 : <class 'numpy.float64'>
Not a Number!

💡TRIVIA

Let us try to perform some basic functions on an numpy array that involves NaN values and find out what happens to it.

import numpy as np

arr = np.array([10, 20, np.nan, 40, np.nan])
print(arr.sum())
print(arr.max())

Output:

nan
nan

Now this can be a problem in many cases. So, do we have a way to eliminate the NaN values from our array object and then perform the mathematical operations upon the array elements? Yes! Numpy facilitates us with methods like np.nansum() and np.nanmax() that help us to calculate the sum and maximum values in the array by ignoring the presence of NaN values in the array.

Example:

import numpy as np

arr = np.array([10, 20, np.nan, 40, np.nan])
print(np.nansum(arr))
print(np.nanmax(arr))

Output:

70.0
40.0

Method 4: Using pandas.isna()

Another way to solve our problem is to use the isna() method of the Pandas module. pandas.isna() is a function that detects missing values in an array-like object. It returns True if any NaN value is encountered.

Example 1:

import pandas as pd

x = float("nan")
y = 25.75
print(pd.isna(x))
print(pd.isna(y))

Output:

True
False

Example 2: In the following example we will have a look at a Pandas DataFrame and detect the presence of NaN values in the DataFrame.

import pandas as pd

df = pd.DataFrame([['Mercury', 'Venus', 'Earth'], ['1', float('nan'), '2']])
print(pd.isna(df))

Output:

       0      1      2
0  False  False  False
1  False   True  False

Method 5: By Checking The Range

We can check for the NaN values by using another NaN special property: limited range. The range of all the floating-point values falls within negative infinity to infinity. However, NaN values do not fall within this range.

Hence, the idea is to check whether a given value lies within the range of -inf and inf. If yes , then it is not a NaN value else it is a NaN value.

Example:

li = [25.87, float('nan')]
for i in li:
    if float('-inf') < float(i) < float('inf'):
        print(i)
    else:
        print("Not a Number!")

Output:

25.87
Not a Number!

Recommended read: Python Infinity

Conclusion

In this article, we learned how we can use the various methods and modules (pandas, NumPy, and math) in Python to check for the NaN values. I hope this article was able to answer your queries. Please stay tuned and subscribe for more such articles. 

Authors: SHUBHAM SAYON and RASHI AGARWAL

Do you want to become a NumPy master? Check out our interactive puzzle book Coffee Break NumPy and boost your data science skills! (Amazon link opens in new tab.)

Coffee Break NumPy