Pandas DataFrame Methods equals(), filter(), first(), last(), head(), and tail()

The Pandas DataFrame has several Re-indexing/Selection/Label Manipulations methods. When applied to a DataFrame, these methods evaluate, modify the elements and return the results.

Preparation

Before any data manipulation can occur, one (1) new library will require installation:

• The Pandas library enables access to/from a DataFrame.

To install this library, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.

$ pip install pandas

Hit the <Enter> key on the keyboard to start the installation process.

If the installations were successful, a message displays in the terminal indicating the same.

Feel free to view the PyCharm installation guide for the required library.

• How to Install Pandas on PyCharm

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import pandas as pd

💡 Note: To follow along with the examples below, click here to download the finxters.csv file of auto-generated dummy user data. Move this file to the current working directory.

DataFrame equals()

The equals() method compares two (2) DataFrames/Series against each other to determine if they have an identical shape and elements. If identical return True, otherwise return False.

The syntax for this method is as follows:

DataFrame.equals(other)

For this example, we have two (2) DataFrames containing grades for three (3) students.

df_scores1 = pd.DataFrame({'Micah':  [91, 58, 73],
                           'Bob':    [53, 87, 46],
                           'Chloe':  [60, 54, 61]})
  
df_scores2 = pd.DataFrame({'Micah':  [91, 58, 73],
                           'Bob':    [53, 87, 46],
                           'Chloe':  [60, 54, 61]})
  
result = df_scores1.equals(df_scores2)
print(result)
print(df_scores1.shape)
print(df_scores2.shape)

• Line [1-2] creates two (2) DataFrames.
• Line [3] compares df_scores1 against df_scores2 testing the shape and elements. The outcome saves to result (True/False).
• Line [4] outputs the result to the terminal.
• Line [5-6] confirms the shape of the DataFrames is equal by outputting the results to the terminal.

Output

True
(3, 3)
(3, 3)

DataFrame filter()

The filter() method returns a subset of a DataFrame/Series rows/columns based on index label(s). This method does not filter a DataFrame/Series on the contents. The filter applies to the specific label(s) of the selected index. The return value is a subset of the callable.

The syntax for this method is as follows:

DataFrame.filter(items=None, like=None, regex=None, axis=None)

For this method, various scenarios below highlight its capabilities.

This example uses the items parameter to filter the DataFrame and return the result.

Code – Example 1

df_fxs = pd.read_csv('finxters.csv')
result = df_fxs.filter(items=['Username', 'Rank'])
print(result.head())

• Line [1] reads in the comma-separated CSV file and saves it to df_fxs.
• Line [2] filters df_fxs to include only the columns matching the item labels (Username and Rank). These columns (and associated values) save to result.
• Line [3] outputs the first five (5) rows (head()) to the terminal.

Output

This example uses the like parameter to filter the DataFrame and return the result.

Code – Example 2

df_fxs = pd.read_csv('finxters.csv')
result = df_fxs.filter(like='FID', axis=1)
print(result.head())

• Line [1] reads in the comma-separated CSV file and saves it to df_fxs.
• Line [2] filters df_fxs to include only the columns matching the like label ('FID'). This column (and associated values) save to result.
• Line [3] outputs the first five (5) rows (head()) to the terminal.

Output

This example uses the regex parameter to filter the DataFrame and return the result.

Code – Example 3

df_fxs = pd.read_csv('finxters.csv')
result = df_fxs.filter(regex='e$', axis=1)
print(result.head())

• Line [1] reads in the comma-separated CSV file and saves it to df_fxs.
• Line [2] filters df_fxs to include only the columns matching the regex label (First_Name, Last_Name and Username). These columns (and associated values) save to result.
• Line [3] outputs the first five (5) rows (head()) to the terminal.

Output

Note: Each column name in the output ends with the letter e.

DataFrame first()

The first() method retrieves and returns the first set number of rows (periods) based on the value entered. The index must be a date value to return the appropriate results.

The syntax for this method is as follows:

DataFrame.first(offset)

For this example, the blood pressure for three (3) patients over a two (2) month period is retrieved.

r = pd.date_range('2021-01-01', periods=3, freq='1M')
df = pd.DataFrame({'Patient-1': [123, 120, 144], 
                   'Patient-2': [129, 125, 90],
                   'Patient-3': [101, 95,  124]},index=r)

result = df.first('1M')
print(result)

• Line [1] sets up the following:
  • The date range start date ('2021-01-01').
  • The number of periods (3).
  • The frequency ('1M'). This statement equates to 1 Month.
• Line [2] creates a DataFrame containing:
  • Three (3) patient names containing three (3) elements of data for each patient.
• Line [3] saves the first month period to result.
• Line [4] outputs the result to the terminal.

Output

💡 Note: The date range for the selected frequency references the last day of the month.

DataFrame last()

The last() method retrieves and returns the last set number of rows (periods) based on the value entered. The index must be a date value for this method to return the expected results.

The syntax for this method is as follows:

DataFrame.last(offset)

For this example, the blood pressure for three (3) patients over a  two (2) month period is retrieved.

r = pd.date_range('2021-01-01', periods=3, freq='1M')
df = pd.DataFrame({'Patient-1': [123, 120, 144], 
                   'Patient-2': [129, 125, 90],
                   'Patient-3': [101, 95,  124]},index=r)

result = df.last('1M')
print(result)

• Line [1] sets up the following:
  • The date range start date ('2021-01-01').
  • The number of periods (3).
  • The frequency ('1M'). This statement equates to 1 Month.
• Line [2] creates a DataFrame containing:
  • Three (3) patient names containing three (3) elements of data for each patient.
• Line [3] saves the output to result.
• Line [4] outputs the result to the terminal.

Output

💡 Note: The date range for the selected frequency references the last day of the month.

DataFrame head() and tail()

These methods display n numbers of records (top or bottom). These methods are useful when you are accessing large amounts of data.

If no parameter is entered, by default, five (5) rows display.

• The head() method returns the top five (5) rows of the DataFrame.
• The tail() method returns the bottom five (5) rows of the DataFrame

If a parameter is entered in one of the above methods, n number of rows (top/bottom) will display.

As you will note, the head() method was accessed many times during this article. Both methods are must-haves in your knowledge base.

The syntax for these methods is as follows:

DataFrame.head(n=5)
DataFrame.tail(n=5)

For this example, the first five (5) records from the DataFrame and the last five (5) records (tail) from the same DataFrame display.

df_fxs = pd.read_csv('finxters.csv')
print(df_fxs.head())
print(df_fxs.tail())

Output

Further Learning Resources

This is Part 8 of the DataFrame method series.

• Part 1 focuses on the DataFrame methods abs(), all(), any(), clip(), corr(), and corrwith().
• Part 2 focuses on the DataFrame methods count(), cov(), cummax(), cummin(), cumprod(), cumsum().
• Part 3 focuses on the DataFrame methods describe(), diff(), eval(), kurtosis().
• Part 4 focuses on the DataFrame methods mad(), min(), max(), mean(), median(), and mode().
• Part 5 focuses on the DataFrame methods pct_change(), quantile(), rank(), round(), prod(), and product().
• Part 6 focuses on the DataFrame methods add_prefix(), add_suffix(), and align().
• Part 7 focuses on the DataFrame methods at_time(), between_time(), drop(), drop_duplicates() and duplicated().
• Part 8 focuses on the DataFrame methods equals(), filter(), first(), last(), head(), and tail()
• Part 9 focuses on the DataFrame methods equals(), filter(), first(), last(), head(), and tail()
• Part 10 focuses on the DataFrame methods reset_index(), sample(), set_axis(), set_index(), take(), and truncate()
• Part 11 focuses on the DataFrame methods backfill(), bfill(), fillna(), dropna(), and interpolate()
• Part 12 focuses on the DataFrame methods isna(), isnull(), notna(), notnull(), pad() and replace()
• Part 13 focuses on the DataFrame methods drop_level(), pivot(), pivot_table(), reorder_levels(), sort_values() and sort_index()
• Part 14 focuses on the DataFrame methods nlargest(), nsmallest(), swap_level(), stack(), unstack() and swap_axes()
• Part 15 focuses on the DataFrame methods melt(), explode(), squeeze(), to_xarray(), t() and transpose()
• Part 16 focuses on the DataFrame methods append(), assign(), compare(), join(), merge() and update()
• Part 17 focuses on the DataFrame methods asfreq(), asof(), shift(), slice_shift(), tshift(), first_valid_index(), and last_valid_index()
• Part 18 focuses on the DataFrame methods resample(), to_period(), to_timestamp(), tz_localize(), and tz_convert()
• Part 19 focuses on the visualization aspect of DataFrames and Series via plotting, such as plot(), and plot.area().
• Part 20 focuses on continuing the visualization aspect of DataFrames and Series via plotting such as hexbin, hist, pie, and scatter plots.
• Part 21 focuses on the serialization and conversion methods from_dict(), to_dict(), from_records(), to_records(), to_json(), and to_pickles().
• Part 22 focuses on the serialization and conversion methods to_clipboard(), to_html(), to_sql(), to_csv(), and to_excel().
• Part 23 focuses on the serialization and conversion methods to_markdown(), to_stata(), to_hdf(), to_latex(), to_xml().
• Part 24 focuses on the serialization and conversion methods to_parquet(), to_feather(), to_string(), Styler.
• Part 25 focuses on the serialization and conversion methods to_bgq() and to_coo().

Also, have a look at the Pandas DataFrame methods cheat sheet!