The Pandas DataFrame/Series has several methods to re-shape, sort, and transpose the data. When applied to a DataFrame/Series, these methods evaluate and modify the data to accommodate the selections.
Preparation
Before any data manipulation can occur, two (2) new libraries will require installation.
- The Pandas library enables access to/from a DataFrame.
- The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.
To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.
$ pip install pandas
Hit the <Enter> key on the keyboard to start the installation process.
$ pip install numpy
Hit the <Enter> key on the keyboard to start the installation process.
If the installations were successful, a message displays in the terminal indicating the same.
Feel free to view the PyCharm installation guide for the required libraries.
Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.
import pandas as pd import numpy as np
DataFrame nlargest()
The nlargest() method retrieves and returns the first n (number) of rows containing the largest column values in descending order. This method is similar to df.sort_values().
Click here for additional details.
The syntax for this method is as follows:
DataFrame.nlargest(n, columns, keep='first')
| Parameter | Argument |
|---|---|
n | This parameter is an integer that returns the specified (n) rows from a DataFrame/Series. |
columns | This parameter is a list or list of labels to order the DataFrame/Series. |
keep | This parameter deals with how to handle duplicate values. The options are 'first', 'last' and 'all'. – first/last organize the first/last occurrences. – all: includes duplicates. |
For this example, the countries.csv file containing nine (9) rows reads in. The top three (3) countries with the highest areas display in descending order.
💡 Note: Click here to download this file. Move this file to the current working directory.
Code – Example 1
df = pd.read_csv('countries.csv')
result = df.nlargest(3, 'Area')
print(result)
- Line [1] reads in a comma-delimited CSV file and saves to a DataFrame (
df). - Line [2] determines the top three (3) countries with the highest area and sorts in descending order. The output saves to
result. - Line [3] outputs the result to the terminal.
Output
result
| Country | Capital | Population | Area | |
| 5 | Russia | Moscow | 146748590 | 17098246 |
| 6 | USA | Washington | 328239523 | 9833520 |
| 7 | China | Beijing | 1400050000 | 9596961 |
For this example, the countries.csv file containing nine (9) rows reads in. The top three (3) countries with the highest areas display in descending order. In this example, the numeric fields are formatted using a lambda!
Code – Example 2
df = pd.read_csv('countries.csv')
df = df.nlargest(3, 'Population')
df['Area'] = df['Area'].apply(lambda x: '{:,}'.format(x))
df['Population'] = df['Population'].apply(lambda x: '{:,}'.format(x))
print(df)
- Line [1] reads in a comma-delimited CSV file and saves to a DataFrame (
df). - Line [2] determines the top three (3) countries with the highest population and sorts in descending order. The output saves to
df. - Line [3-4] format the Population and Area columns with commas. The
dfupdates accordingly. - Line [5] outputs the DataFrame to the terminal.
Output
df
| Country | Capital | Population | Area | |
| 7 | China | Beijing | 1,400,050,000 | 9,596,961 |
| 8 | India | Dheli | 1,352,642,280 | 3,287,263 |
| 6 | USA | Washington | 328,239,523 | 9,833,520 |
DataFrame nsmallest()
The nsmallest() method retrieves and returns the first n (number) of rows that contain the smallest column values in ascending order.
The syntax for this method is as follows:
DataFrame.nsmallest(n, columns, keep='first')
| Parameter | Argument |
|---|---|
n | This parameter is an integer that returns the specified (n) rows from a DataFrame/Series. |
columns | This parameter is a list or list of labels to order the DataFrame/Series. |
keep | This parameter deals with how to handle duplicate values. The options are 'first', 'last' and 'all'. – first/last organize the first/last occurrences. – all: includes duplicates. |
| N: | This parameter is an integer and returns the specified (n) rows from a DataFrame/Series. |
| Columns: | This parameter is a list or list of labels to order the DataFrame/Series. |
| Keep: | This parameter deals with how to handle duplicate values. The options are ‘first’, ‘last’ and ‘all’. – first/last organize the first/last occurrences. – all: includes duplicates. |
For this example, the finxters.csv file containing fictitious Finxter user information reads in. The users with the smallest number of puzzles solved returns in ascending order. In this example, the numeric fields are formatted using a lambda.
df = pd.read_csv('finxters.csv')
df = df.nsmallest(3, 'Solved')
df['Solved'] = df['Solved'].apply(lambda x: '{:,}'.format(int(x)))
df['Incorrect'] = df['Incorrect'].apply(lambda x: '{:,}'.format(int(x)))
df['Recurring'] = df['Recurring'].apply(lambda x: '${:,}'.format(x))
print(df)
- Line [1] reads in a comma-delimited CSV file and saves to a DataFrame (
df). - Line [2] determines the three (3) users with the smallest number of puzzles solved and sorts in ascending order. The output saves to
df. - Line [3-4] converts to an integer and formats the Solved and Incorrect columns with commas. The
dfupdates accordingly. - Line [5] formats the Recurring column with a dollar sign and a comma. The
dfupdates accordingly. - Line [6] outputs the DataFrame to the terminal.
Output
| FID | Start | First_Name | Last_Name | … | Solved | Incorrect | Recurring | Taxes | |
| 2 | 30022331 | 11/1/2021 | Peter | Dunn | … | 15 | 9 | 9.98 | 15 |
| 18 | 3002285 | 16/6/2021 | Jack | Thompson | … | 91 | 18 | 15.98 | 18 |
| 42 | 30024622 | 6/10/2021 | Jan | Martin | … | 995 | 37 | 9.98 | 10 |
DataFrame swap_level()
The swaplevel() method trades the contents of level i with the contents of level j in a MultiIndex DataFrame/Series.
The syntax for this method is as follows:
DataFrame.swaplevel(i=- 2, j=- 1, axis=0)
| Parameter | Description |
|---|---|
i, j | These parameters can be an integer/string. They are the indexes to be swapped. |
axis | If zero (0) or index is selected, apply to each column. Default is 0 (column). If zero (1) or columns, apply to each row. |
For this example, there are five (5) students. Each student has some associated data with it. Grades generate by using np.random.randint() from the NumPy library.
index = [(1001, 'Micah Smith', 14), (1001, 'Philip Jones', 15),
(1002, 'Ben Grimes', 16), (1002, 'Alicia Heath', 17),
(1002, 'Arch Nelson', 18)]
m_index = pd.MultiIndex.from_tuples(index)
grades_lst = np.random.randint(45,100,size=5)
df = pd.DataFrame({"Grades": grades_lst}, index=m_index)
print(df)
result = df.swaplevel(0,1)
print(result)
- Line [1] creates a list of tuples. Each tuple contains three (3) values. The output saves to the index.
- Line [2] creates a
MultiIndexfrom the List of Tuples created on line [1] and saves it tom_index. - Line [3] generates five (5) random grades between the specified range and saves them to
grades_lst. - Line [4] creates a DataFrame from the variables created on lines [1-3] and saves to
df. - Line [5] outputs the DataFrame to the terminal.
- Line [6] swaps out the levels as specified. The output saves to
result. - Line [7] outputs the result to the terminal.
Output
df
| Grades | |||
| 1001 | Micah Smith | 14 | 55 |
| Philip Jones | 15 | 74 | |
| 1002 | Ben Grimes | 16 | 93 |
| Alicia Heath | 17 | 93 | |
| Arch Nelson | 18 | 63 |
result
| Grades | |||
| Micah Smith | 1001 | 14 | 55 |
| Philip Jones | 1001 | 15 | 74 |
| Ben Grimes | 1002 | 16 | 93 |
| Alicia Heath | 1002 | 17 | 93 |
| Arch Nelson | 1002 | 18 | 63 |
DataFrame stack()
The stack() method returns a re-shaped Multi-Level index DataFrame/Series containing at minimum one (1) or more inner levels. A pivot occurs on the new levels using the columns of the DataFrame/Series.
💡 Note: If a single level, the output returns as a Series. If multi-level, the new level(s) are retrieved from the said levels and return a DataFrame.
The syntax for this method is as follows:
DataFrame.stack(level=- 1, dropna=True)
level | This parameter is the level(s) to stack on the selected axis. Levels can be a string, integer, or list. By default, -1 (last level). |
dropna | This parameter determines if rows containing missing values drop. True, by default. |
We have two (2) students with relevant details that save to a DataFrame. The code below displays the original DataFrame and the DataFrame using the stack() method.
df = pd.DataFrame([[8, 7], [7, 5]],
index=['Micah', 'Philip'],
columns=['Age', 'Grade'])
print(df)
result = df.stack()
print(result)
- Line [1] creates a DataFrame with index labels and columns specified. This output saves to
df. - Line [2] outputs the DataFrame to the terminal.
- Line [3] stacks the DataFrame and saves the output to
result. - Line [4] outputs the result to the terminal (stacked format).
Output
df
| Age | Grade | |
| Micah | 8 | 7 |
| Philip | 7 | 5 |
result
| Micah | Age | 8 |
| Grade | 7 | |
| Philip | Age | 7 |
| Grade | 5 | |
| dtype: int64 |
DataFrame unstack()
The unstack() method returns a re-shaped Multi-Level index DataFrame/Series with a new column level using inner-level labels.
The syntax for this method is as follows:
DataFrame.unstack(level=- 1, fill_value=None)
| Parameters | Description |
|---|---|
level | This parameter is the level(s) to unstack. Levels can be a string, integer, or list. -1 by default (last level). |
dropna | This parameter determines if rows containing missing values drop. True, by default. |
We have two (2) students with relevant details that save to a DataFrame. The code below displays the original DataFrame and the DataFrame using the unstack() method.
df = pd.DataFrame([[8, 7], [7, 5]],
index=['Micah', 'Philip'],
columns=['Age', 'Grade'])
df = df.stack()
print(df)
result = df.unstack(level=0)
print(result)
- Line [1] creates a DataFrame with index labels and columns specified. The output saves to
df. - Line [2] stacks the DataFrame. The
dfupdates accordingly. - Line [3] outputs the DataFrame to the terminal.
- Line [4] unstacks the stacked DataFrame. The output saves to
result. - Line [5] outputs the result to the terminal (unstacked format).
Output
df
| Micah | Age | 8 |
| Grade | 7 | |
| Philip | Age | 7 |
| Grade | 5 | |
| dtype: int64 |
result
| Age | Grade | |
| Micah | 8 | 7 |
| Philip | 7 | 5 |
DataFrame swap_axes()
The swapaxes() method swap axis 1 with axis 2. This parameter returns the called DataFrame/Series.
The syntax for this method is as follows:
DataFrame.swapaxes(axis1, axis2, copy=True)
axis1, axis2 | If zero (0) or index is selected, apply to each column. Default is 0 (column). If zero (1) or columns, apply to each row. |
copy | If True, a copy of the original DataFrame/Series creates. True, by default. If False, the updates occur on the original DataFrame/Series. |
We have six (6) book titles and associated details that save to a DataFrame. The np.random.randint() method retrieves the number of books sold.
index = [('Mystery', 'Sharp Objects', 1982), ('Mystery', 'A Murder', 1973), ('Mystery', 'Wanted', 1990),
('Fiction', 'Thirst', 1992), ('Fiction', 'The Time Keeper', 2014), ('Fiction', 'Eligible', 1997)]
m_index = pd.MultiIndex.from_tuples(index)
num_sold = np.random.randint(5,250,size=6)
df = pd.DataFrame({'Sold': num_sold}, index=m_index)
print(df)
result = df.swapaxes(1, 0)
print(result)
- Line [1] creates a List of tuples. Each tuple contains three (3) values. The output saves to the index.
- Line [2] creates a
MultiIndexfrom the list of tuples created on line [1] and saves it tom_index. - Line [3] generates five (5) random integers between the specified range and saves them to
num_sold. - Line [4] creates a DataFrame from the variables created on lines [1-3] and saves to
df. - Line [5] outputs the DataFrame to the terminal.
- Line [6] swaps out the axes as specified. The output saves to
result. - Line [7] outputs the result to the terminal.
Output
df
| Sold | |||
| Mystery | Sharp Objects | 1982 | 76 |
| A Murder | 1973 | 114 | |
| Wanted | 1990 | 244 | |
| Fiction | Thirst | 1992 | 153 |
| The Time Keeper | 2014 | 207 | |
| Eligible | 1997 | 175 |
result
| Mystery | Fiction | |||||
| Sharp Objects | A Murder | Wanted | Thirst | The Time Keeper | Eligible | |
| 1982 | 1973 | 1990 | 1992 | 2014 | 1997 | |
| Sold | 76 | 114 | 244 | 153 | 207 | 175 |
Further Learning Resources
This is Part 14 of the DataFrame method series.
Also, have a look at the Pandas DataFrame methods cheat sheet!
At university, I found my love of writing and coding. Both of which I was able to use in my career.
During the past 15 years, I have held a number of positions such as:
In-house Corporate Technical Writer for various software programs such as Navision and Microsoft CRM
Corporate Trainer (staff of 30+)
Programming Instructor
Implementation Specialist for Navision and Microsoft CRM
Senior PHP Coder