­Pandas Methods count(), cov() & cumX()

Rate this post

The Pandas DataFrame has several methods concerning Computations and Descriptive Stats. When applied to a DataFrame, these methods evaluate the elements and return the results.


Preparation

Before any data manipulation can occur, two (2) new libraries will require installation.

  • The Pandas library enables access to/from a DataFrame.
  • The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.

To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.

$ pip install pandas

Hit the <Enter> key on the keyboard to start the installation process.

$ pip install numpy

Hit the <Enter> key on the keyboard to start the installation process.

If the installations were successful, a message displays in the terminal indicating the same.


Feel free to view the PyCharm installation guide for the required libraries.


Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import pandas as pd 
import numpy as np 

DataFrame count()

The count() method provides the count of all non-NaN values in a DataFrame/Series.

The syntax for this method is as follows:

DataFrame.count()
ParametersDescription
axisIf zero (0) or index is selected, apply the function to each column. Default is None. If one (1) is selected, apply the function to each row.
levelA string specifies the level name.
numeric_onlyThis parameter can be a float, integer, or Boolean value. By default, False.

For this example, the Human Resources Dept. of Rivers Clothing wants to determine the cost of benefit coverage based on the marital status of their staff. The issue here is some data contains the NaN value. 

df_staff = pd.DataFrame({'EID':    [100, 101, 102, 103],
                         'Name':   ['Micah', 'Alycia', 'Philip', 'Josiah'],
                         'Status': ['M', 'S', np.nan, np.nan]})

result = df_staff.count()
print(result)
  • Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_staff.
  • Line [2] uses the count() method to determine how many non-NaN values exist. The output saves to the result variable.
  • Line [3] outputs the result to the terminal.

Output

EID4
Name4
Status2
dtype: int64

DataFrame cov()

The cov() method computes pair-wise co-variances across the series of a DataFrame. This analysis determines the relationship between various measures across time. Any NaN/NULL values do not count.

The syntax for this method is as follows:

DataFrame.cov(min_periods=None, ddof=1)
ParametersDescription
min_periodsThe minimum number of observations required per pair of columns to have a valid result. This parameter is an integer and is optional.
ddofThis parameter is the Delta degrees of freedom. This parameter is the divisor used in calculations (N - ddof), where N represents the number of elements. By default, the value is one (1).

For this example, a random series of numbers generate to see the cov() method in action.

np.random.seed(75)
df = pd.DataFrame(np.random.randn(35, 3),columns=['Level-A', 'Level-B', 'Level-C'])
result = df.cov(min_periods=12)
print(result)
  • Line [1] generates random numbers using the NumPy seed() method.
  • Line [2] creates a DataFrame using the NumPy randn() method and a list. This DataFrame saves to df. Notice the three (3) in randn() corresponds to the number of columns outlined in the DataFrame.
  • Line [3] calls the cov() method and sets the Minimum Period to 12.
  • Line [4] outputs the result to the terminal.

Output

 Level-ALevel-BLevel-C
Level-A1.1338520.1399680.159209
Level-B0.1399680.8984060.540002
Level-C0.1592090.5400021.384775

DataFrame cummax()

The cummax() method returns a cumulative maximum over a DataFrame/Series axis.

The syntax for this method is as follows:

DataFrame.cummax(axis=None, skipna=True, *args, **kwargs)
ParametersDescription
axisIf zero (0) or index is selected, apply the function to each column. Default is None. If one (1) is selected, apply the function to each row.
skipnaThis parameter excludes NaN or NULL values. If a row/column contains these values, the result is NaN. By default, this is True.
*argsAdditional keywords have no effect. However, they might be compatible with NumPy.
**kwargsAdditional keywords have no effect. However, they might be compatible with NumPy.

This example displays the maximum number of wins, losses, and ties for four (4) hockey teams.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':    [3, 6, 10],
                         'Leafs':     [2, 7, 11],
                         'Flames':  [1, 8, 12]})

result = df_teams.cummax(axis=’index’)
print(result)
  • Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
  • Line [2] retrieves the maximum numbers and saves them to the result variable.
  • Line [3] outputs the result to the terminal.

Output

 BruinsOilersLeafsFlames
04444
15678
29101112

💡 Note: By default, Line [6] iterates over all the rows and determines the value for each column. This is equivalent to axis=None or axis=’index’ (used in our example).


DataFrame cummin()

The cummin() method returns a cumulative minimum over a DataFrame/Series axis.

The syntax for this method is as follows:

DataFrame. cummin(axis=None, skipna=True, *args, **kwargs)
ParametersDescription
axisIf zero (0) or index is selected, apply the function to each column. Default is None. If one (1) is selected, apply the function to each row.
skipnaThis parameter excludes NaN or NULL values. If a row/column contains these values, the result is NaN. By default, this is True.
*argsAdditional keywords have no effect. However, they might be compatible with NumPy.
**kwargsAdditional keywords have no effect. However, they might be compatible with NumPy.

This example displays the minimum number of wins, losses, and ties for four (4) hockey teams.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':    [3, 6, 10],
                         'Leafs':     [2, 7, 11],
                         'Flames':  [1, 8, 12]})

result = df_teams.cummin(axis=’index’)
print(result)
  • Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
  • Line [2] retrieves the minimum numbers and saves them to the result variable.
  • Line [3] outputs the result to the terminal.

Output

 BruinsOilersLeafsFlames
04321
14321
24321

💡 Note: By default, Line [6] iterates over all the rows and determines the value for each column. This is equivalent to axis=None or axis=’index’ (used in our example).


DataFrame cumprod()

The cumprod() method returns a DataFrame/Series of the same size containing the cumulative product.

The syntax for this method is as follows:

DataFrame.cumprod(axis=None, skipna=True, *args, **kwargs)
ParametersDescription
axisIf zero (0) or index is selected, apply the function to each column. Default is None. If one (1) is selected, apply the function to each row.
skipnaThis parameter excludes NaN or NULL values. If a row/column contains these values, the result is NaN. By default, this is True.
*argsAdditional keywords have no effect. However, they might be compatible with NumPy.
**kwargsAdditional keywords have no effect. However, they might be compatible with NumPy.

This example displays the cumulative product of the Hockey Team Stats.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':    [3, 6, 10],
                         'Leafs':     [2, 7, 11],
                         'Flames':  [1, 8, 12]})

result = df_teams.cumprod(axis='index')
print(result)
  • Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
  • Line [2] retrieves the cumulative product and saves them to the result variable.
  • Line [3] outputs the result to the terminal.

Output

 BruinsOilersLeafsFlames
04321
12018148
218018015496

💡 Note: By default, Line [6] iterates over all the rows and determines the value for each column. This is equivalent to axis=None or axis=’index’ (used in our example).


DataFrame cumsum()

The cumsum() method returns a DataFrame/Series of the same size containing the cumulative sum.

The syntax for this method is as follows:

DataFrame.cumsum(axis=None, skipna=True, *args, **kwargs)
ParametersDescription
axisIf zero (0) or index is selected, apply the function to each column. Default is None. If one (1) is selected, apply the function to each row.
skipnaThis parameter excludes NaN or NULL values. If a row/column contains these values, the result is NaN. By default, this is True.
*argsAdditional keywords have no effect. However, they might be compatible with NumPy.
**kwargsAdditional keywords have no effect. However, they might be compatible with NumPy.

This example displays the cumulative sum of the Hockey Team Stats.

df_teams = pd.DataFrame({'Bruins':   [4, 5, 9],
                         'Oilers':    [3, 6, 10],
                         'Leafs':     [2, 7, 11],
                         'Flames':  [1, 8, 12]})

result = df_teams.cumsum(axis='index')
print(result)
  • Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
  • Line [2] retrieves the cumulative sum and saves them to the result variable.
  • Line [3] outputs the result to the terminal.

Output

 BruinsOilersLeafsFlames
04321
19999
218192021

💡 Note: By default, Line [6] iterates over all the rows and determines the value for each column. This is equivalent to axis=None or axis=’index’ (used in our example).

Further Learning Resources

This is Part 2 of the DataFrame method series.

  • Part 1 focuses on the DataFrame methods abs(), all(), any(), clip(), corr(), and corrwith().
  • Part 2 focuses on the DataFrame methods count(), cov(), cummax(), cummin(), cumprod(), cumsum().
  • Part 3 focuses on the DataFrame methods describe(), diff(), eval(), kurtosis().
  • Part 4 focuses on the DataFrame methods mad(), min(), max(), mean(), median(), and mode().
  • Part 5 focuses on the DataFrame methods pct_change(), quantile(), rank(), round(), prod(), and product().
  • Part 6 focuses on the DataFrame methods add_prefix(), add_suffix(), and align().
  • Part 7 focuses on the DataFrame methods at_time(), between_time(), drop(), drop_duplicates() and duplicated().
  • Part 8 focuses on the DataFrame methods equals(), filter(), first(), last(), head(), and tail()
  • Part 9 focuses on the DataFrame methods equals(), filter(), first(), last(), head(), and tail()
  • Part 10 focuses on the DataFrame methods reset_index(), sample(), set_axis(), set_index(), take(), and truncate()
  • Part 11 focuses on the DataFrame methods backfill(), bfill(), fillna(), dropna(), and interpolate()
  • Part 12 focuses on the DataFrame methods isna(), isnull(), notna(), notnull(), pad() and replace()
  • Part 13 focuses on the DataFrame methods drop_level(), pivot(), pivot_table(), reorder_levels(), sort_values() and sort_index()
  • Part 14 focuses on the DataFrame methods nlargest(), nsmallest(), swap_level(), stack(), unstack() and swap_axes()
  • Part 15 focuses on the DataFrame methods melt(), explode(), squeeze(), to_xarray(), t() and transpose()
  • Part 16 focuses on the DataFrame methods append(), assign(), compare(), join(), merge() and update()
  • Part 17 focuses on the DataFrame methods asfreq(), asof(), shift(), slice_shift(), tshift(), first_valid_index(), and last_valid_index()
  • Part 18 focuses on the DataFrame methods resample(), to_period(), to_timestamp(), tz_localize(), and tz_convert()
  • Part 19 focuses on the visualization aspect of DataFrames and Series via plotting, such as plot(), and plot.area().
  • Part 20 focuses on continuing the visualization aspect of DataFrames and Series via plotting such as hexbin, hist, pie, and scatter plots.
  • Part 21 focuses on the serialization and conversion methods from_dict(), to_dict(), from_records(), to_records(), to_json(), and to_pickles().
  • Part 22 focuses on the serialization and conversion methods to_clipboard(), to_html(), to_sql(), to_csv(), and to_excel().
  • Part 23 focuses on the serialization and conversion methods to_markdown(), to_stata(), to_hdf(), to_latex(), to_xml().
  • Part 24 focuses on the serialization and conversion methods to_parquet(), to_feather(), to_string(), Styler.
  • Part 25 focuses on the serialization and conversion methods to_bgq() and to_coo().

Also, have a look at the Pandas DataFrame methods cheat sheet!