**π‘ Problem Formulation:** When working with datasets in Python, it’s common to encounter missing values in your DataFrame columns. This can lead to inaccuracies in your analysis or errors in your code. The goal is to replace missing values with the median of the column as it’s less sensitive to outliers than the mean. For example, given a DataFrame with some NaN values, we want to fill those NaNs with the median value of their respective columns.

## Method 1: Using `DataFrame.fillna()`

with `DataFrame.median()`

This method leverages the pandas `DataFrame.fillna()`

function that allows filling NA/NaN values using specified methods, and `DataFrame.median()`

computes the median of the DataFrame’s numeric columns, which we will use to replace the missing values. This method is particularly useful when you want a quick and direct way to deal with missing values column by column.

Here’s an example:

import pandas as pd # Creating a DataFrame with NaN values df = pd.DataFrame({ 'A': [1, 2, 3, None, 5], 'B': [5, None, None, 4, 5], 'C': [None, 2, 3, 4, 5] }) # Filling the NaN values with median of the columns df.fillna(df.median(), inplace=True) print(df)

Output:

A B C 0 1.0 5.0 3.0 1 2.0 5.0 2.0 2 3.0 5.0 3.0 3 3.0 4.0 4.0 4 5.0 5.0 5.0

This code snippet first creates a DataFrame with some NaN values. Then, it calculates the median for each column and replaces NaN values with these medians. The `inplace=True`

argument updates the original DataFrame without the need to assign it to a new variable.

## Method 2: Using `DataFrame.apply()`

for Selective Column Filling

Using `DataFrame.apply()`

allows you to apply a function along an axis of the DataFrame. When combined with a lambda function to calculate the median, you can selectively fill NaN values in columns with their respective medians. This method gives you more control when you have specific columns to target for filling NaN values.

Here’s an example:

import pandas as pd df = pd.DataFrame({ 'A': [1, None, 3, None, 5], 'B': [5, None, None, 4, 5], 'C': [None, None, 3, 4, 5] }) # Apply a lambda function to fill NaN values with the median df = df.apply(lambda col: col.fillna(col.median())) print(df)

Output:

A B C 0 1.0 5.0 4.0 1 3.0 4.5 4.0 2 3.0 4.5 3.0 3 3.0 4.0 4.0 4 5.0 5.0 5.0

In this example, `apply()`

is used to run a lambda function on each column. The lambda function fills NaN values for a column with the median of that column. Since `apply()`

works on a column-by-column basis, it’s convenient for applying column-specific operations.

## Method 3: Filling with Median using `DataFrame.transform()`

The `DataFrame.transform()`

method is utilized when you wish to perform some operation on a DataFrame or Series and return an object that is the same size. By combining this with the median function, one can fill in missing values with the median while still being able to conduct additional transformations if needed.

Here’s an example:

import pandas as pd df = pd.DataFrame({ 'A': [1, None, 3, 4, 5], 'B': [5, None, None, 4, 5], 'C': [None, 2, 3, 4, 5] }) # Using transform to fill NaN values with the median. df = df.transform(lambda x: x.fillna(x.median())) print(df)

Output:

A B C 0 1.0 5.0 3.5 1 3.0 4.5 2.0 2 3.0 4.5 3.0 3 4.0 4.0 4.0 4 5.0 5.0 5.0

This snippet applies the `transform()`

method with a lambda function that fills NaNs with the median value for that series. It’s a versatile approach that not only fills the missing values but can also be extended to include additional transformations seamlessly.

## Method 4: Using `numpy.where()`

with `DataFrame.isnull()`

This method involves using NumPy’s `where()`

function in tandem with pandas’ `isnull()`

to identify and fill NaN values with the median. This approach is useful when specific condition-based replacements are desired, and allows for customized replacement logic.

Here’s an example:

import pandas as pd import numpy as np df = pd.DataFrame({ 'A': [1, 2, 3, None, 5], 'B': [5, None, 1, 4, 5], 'C': [None, 2, 3, 4, 5] }) # Using numpy.where() to fill NaN values with the median for column in df.columns: df[column] = np.where(df[column].isnull(), df[column].median(), df[column]) print(df)

Output:

A B C 0 1.0 5.0 3.0 1 2.0 5.0 2.0 2 3.0 1.0 3.0 3 3.0 4.0 4.0 4 5.0 5.0 5.0

The code iterates over all columns, using `np.where()`

to check for NaN values with `df[column].isnull()`

. When a NaN is found, it is replaced by the median of that column, otherwise, the original value is retained. This method is powerful but can be more verbose compared to others.

## Bonus One-Liner Method 5: Using `DataFrame.where()`

with `notnull()`

Another concise technique involves using pandas’ `DataFrame.where()`

in combination with `notnull()`

. This one-liner is an efficient way to maintain the original non-NaN values while replacing the NaNs with column medians directly.

Here’s an example:

import pandas as pd df = pd.DataFrame({ 'A': [1, 2, 3, None, 5], 'B': [5, None, 1, 4, 5], 'C': [None, 2, 3, 4, 5] }) # One-liner to fill NaN values with median using DataFrame.where() df = df.where(df.notnull(), df.median(), axis='columns') print(df)

Output:

A B C 0 1.0 5.0 3.0 1 2.0 4.0 2.0 2 3.0 1.0 3.0 3 3.0 4.0 4.0 4 5.0 5.0 5.0

This example showcases how `DataFrame.where()`

can be used to apply the median to NaN elements while keeping the original non-NaN values unchanged. The one-liner is elegant and efficient, but it may not be as readable for those unfamiliar with this style of pandas’ operations.

## Summary/Discussion

**Method 1:**`DataFrame.fillna()`

with`DataFrame.median()`

. Quick and straightforward. Limited customization.**Method 2:**Lambda function with`DataFrame.apply()`

. More control over specific column operations. Slightly more complex.**Method 3:**`DataFrame.transform()`

with median. Versatile for additional transformations. Could be overkill for simple replacements.**Method 4:**`numpy.where()`

combined with`DataFrame.isnull()`

. High level of customization potential. More verbose than other methods.**Method 5:**One-liner using`DataFrame.where()`

with`notnull()`

. Efficient and concise. May lack clarity for some users.