­Pandas DataFrame describe(), diff(), eval(), kurtosis()

The Pandas DataFrame has several methods concerning Computations and Descriptive Stats. When applied to a DataFrame, these methods evaluate the elements and return the results.

Preparation

Before any data manipulation can occur, two (2) new libraries will require installation.

• The Pandas library enables access to/from a DataFrame.
• The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.

To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.

$ pip install pandas

Hit the <Enter> key on the keyboard to start the installation process.

$ pip install numpy

Hit the <Enter> key on the keyboard to start the installation process.

If the installations were successful, a message displays in the terminal indicating the same.

Feel free to view the PyCharm installation guide for the required libraries.

• How to Install Pandas on PyCharm
• How to Install NumPy on PyCharm

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import pandas as pd
import numpy as np 

DataFrame describe()

The describe() method analyzes numeric and object series and DataFrame column sets of various data types.

The syntax for this method is as follows (source):

DataFrame.describe(percentiles=None, include=None, exclude=None, datetime_is_numeric=False)

Also, consider this table from the docs:

For this example, the same Teams DataFrame referred to in Part 2 of this series is used. The DataFrame below displays four (4) Hockey Teams’ stats: wins, losses, and ties.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':   [3, 6, 10],
                         'Leafs':    [2, 7, 11],
                         'Flames':   [1, 8, 12]})

result = df_teams.describe().apply(lambda x:round(x,2))
print(result)

• Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
• Line [2] uses the describe() method to retrieve additional analytical information. Using a lambda, it then formats the output to two (2) decimal places and saves it to the result variable.
• Line [3] outputs the result to the terminal.

Output

Click here to see additional examples.

DataFrame diff()

The diff() method calculates the difference between a DataFrame element compared with another element in the same DataFrame. The default is the element in the previous row.

The syntax for this method is as follows:

DataFrame.diff(periods=1, axis=0)

Code – Example 1

This example reflects the difference in regard to the previous row.

df_teams = pd.DataFrame({'Bruins':  [4, 5, 9],
                         'Oilers':   [3, 6, 10],
                         'Leafs':    [2, 7, 11],
                         'Flames': [1, 8, 12]})

result = df_teams.diff()
print(result)

• Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
• Line [2] uses the diff() method to determine the difference from the previous row and saves it to the result variable.
• Line [3] outputs the result to the terminal.

Output

Code – Example 2

This example reflects the difference in regard to the previous column.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':   [3, 6, 10],
                         'Leafs':    [2, 7, 11],
                         'Flames':   [1, 8, 12]})

result = df_teams.diff(axis=1)
print(result)

• Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
• Line [2] uses the diff() method to determine the difference from the previous column and saves it to the result variable.
• Line [3] outputs the result to the terminal.

Output

Code – Example 3

This example reflects the difference in regard to the previous rows.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':   [3, 6, 10],
                         'Leafs':    [2, 7, 11],
                         'Flames':   [1, 8, 12]})

result = df_teams.diff(periods=1)
print(result)

• Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
• Line [2] uses the diff() method to determine the difference from the previous column and with periods set to 1 and saves to the result variable.
• Line [3] outputs the result to the terminal.

Output

DataFrame eval()

The eval() method evaluates a string describing the operation on DataFrame columns. This is for columns only, not specific rows or elements. This allows the eval to run arbitrary code.

🛑 Note: This can make the code vulnerable to code injection if you pass user input to this method.

The syntax for this method is as follows:

DataFrame.eval(expr, inplace=False, **kwargs)

For this example, the Hockey Teams Bruins and Oilers stats will be added together.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':   [3, 6, 10],
                         'Leafs':    [2, 7, 11],
                         'Flames':   [1, 8, 12]})

result = df_teams.eval('Bruins + Oilers')
print(result)	

• Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
• Line [2] uses the eval() method to evaluate the calculation and saves to the result variable.
• Line [3] outputs the result to the terminal.

Output

DataFrame kurt() and kurtosis()

The DataFrame kurt() and kurtosis() methods are identical and return an unbiased kurtosis over a requested axis. For additional information on Kurtosis, click here.

For this example, the Hockey Teams data is used.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':   [3, 6, 10],
                         'Leafs':    [2, 7, 11],
                         'Flames':   [1, 8, 12]})

result = df_teams.kurtosis()
print(result)

• Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
• Line [2] uses the kurtosis() method to determine the output and saves to the result variable.
• Line [3] outputs the result to the terminal.

Output

Further Learning Resources

This is Part 3 of the DataFrame method series.

• Part 1 focuses on the DataFrame methods abs(), all(), any(), clip(), corr(), and corrwith().
• Part 2 focuses on the DataFrame methods count(), cov(), cummax(), cummin(), cumprod(), cumsum().
• Part 3 focuses on the DataFrame methods describe(), diff(), eval(), kurtosis().
• Part 4 focuses on the DataFrame methods mad(), min(), max(), mean(), median(), and mode().
• Part 5 focuses on the DataFrame methods pct_change(), quantile(), rank(), round(), prod(), and product().
• Part 6 focuses on the DataFrame methods add_prefix(), add_suffix(), and align().
• Part 7 focuses on the DataFrame methods at_time(), between_time(), drop(), drop_duplicates() and duplicated().
• Part 8 focuses on the DataFrame methods equals(), filter(), first(), last(), head(), and tail()
• Part 9 focuses on the DataFrame methods equals(), filter(), first(), last(), head(), and tail()
• Part 10 focuses on the DataFrame methods reset_index(), sample(), set_axis(), set_index(), take(), and truncate()
• Part 11 focuses on the DataFrame methods backfill(), bfill(), fillna(), dropna(), and interpolate()
• Part 12 focuses on the DataFrame methods isna(), isnull(), notna(), notnull(), pad() and replace()
• Part 13 focuses on the DataFrame methods drop_level(), pivot(), pivot_table(), reorder_levels(), sort_values() and sort_index()
• Part 14 focuses on the DataFrame methods nlargest(), nsmallest(), swap_level(), stack(), unstack() and swap_axes()
• Part 15 focuses on the DataFrame methods melt(), explode(), squeeze(), to_xarray(), t() and transpose()
• Part 16 focuses on the DataFrame methods append(), assign(), compare(), join(), merge() and update()
• Part 17 focuses on the DataFrame methods asfreq(), asof(), shift(), slice_shift(), tshift(), first_valid_index(), and last_valid_index()
• Part 18 focuses on the DataFrame methods resample(), to_period(), to_timestamp(), tz_localize(), and tz_convert()
• Part 19 focuses on the visualization aspect of DataFrames and Series via plotting, such as plot(), and plot.area().
• Part 20 focuses on continuing the visualization aspect of DataFrames and Series via plotting such as hexbin, hist, pie, and scatter plots.
• Part 21 focuses on the serialization and conversion methods from_dict(), to_dict(), from_records(), to_records(), to_json(), and to_pickles().
• Part 22 focuses on the serialization and conversion methods to_clipboard(), to_html(), to_sql(), to_csv(), and to_excel().
• Part 23 focuses on the serialization and conversion methods to_markdown(), to_stata(), to_hdf(), to_latex(), to_xml().
• Part 24 focuses on the serialization and conversion methods to_parquet(), to_feather(), to_string(), Styler.
• Part 25 focuses on the serialization and conversion methods to_bgq() and to_coo().

Also, have a look at the Pandas DataFrame methods cheat sheet!