5 Best Ways to Convert a Pandas DataFrame to an Array

When working with data in Python, you might come across scenarios where you need to convert a Pandas DataFrame into a numpy array for further manipulation or processing. Transforming DataFrames to arrays is a common task, particularly in data preprocessing for machine learning models which require data in numeric array format. This article provides various methods to achieve this conversion efficiently. As an example, consider a DataFrame containing columns [‘x’,’y’,’z’] and the goal is to extract its values into a numpy array.

Method 1: Using to_numpy() Function

The to_numpy() function in Pandas is designed to convert a DataFrame into a numpy array. It simply returns the numpy representation of the data, preserving the dtype of each column if possible. Its ease of use and effectiveness make it a go-to choice for this conversion.

Here’s an example:

import pandas as pd
df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6], 'z': [7, 8, 9]})
array = df.to_numpy()

Output:

[[1 4 7]
 [2 5 8]
 [3 6 9]]

This code creates a DataFrame with three columns and then uses the to_numpy() function to convert it to a numpy array. Each row from the DataFrame becomes a row in the resulting array.

Method 2: Using values Attribute

The values attribute returns the numpy representation of the DataFrame. It is similar to the to_numpy() method and is equally easy to use. However, the values property is somewhat deprecated and it is recommended to use to_numpy() for future proofing your code.

Here’s an example:

import pandas as pd
df = pd.DataFrame({'x': [10, 20, 30], 'y': [40, 50, 60], 'z': [70, 80, 90]})
array = df.values

Output:

[[10 40 70]
 [20 50 80]
 [30 60 90]]

The code snippet demonstrates the use of DataFrame’s values attribute to convert it to a numpy array wherein each DataFrame row corresponds to the array’s row.

Method 3: Using astype() Method

The astype() method can be combined with to_numpy() to convert the DataFrame to a numpy array of a specific data type. This is useful when the array needs to be of a uniform data type, such as float for use in machine learning models.

Here’s an example:

import pandas as pd
df = pd.DataFrame({'x': [1.1, 2.2, 3.3], 'y': [4.4, 5.5, 6.6]})
array = df.astype('float32').to_numpy()

Output:

[[1.1 4.4]
 [2.2 5.5]
 [3.3 6.6]]

Here, the DataFrame is first cast to type ‘float32’ using astype('float32') thereby ensuring the array elements are of that type before it is converted to a numpy array using to_numpy().

Method 4: Using np.array() Function

The numpy library has an array function np.array() that can take a Pandas DataFrame as input and convert it into a numpy array. This method gives you the control to specify the data type at the moment of array creation.

Here’s an example:

import pandas as pd
import numpy as np
df = pd.DataFrame({'x': ['1', '2', '3'], 'y': ['4', '5', '6']})
array = np.array(df, dtype='int')

Output:

[[1 4]
 [2 5]
 [3 6]]

The np.array() function is used to create a numpy array from the DataFrame, specifying the desired data type to ‘int’. This is particularly useful when the DataFrame’s data types are varied or not numerical.

Bonus One-Liner Method 5: Using List Comprehension

This method utilizes Python’s list comprehension to manually iterate through DataFrame rows and store each row’s values in a new list, effectively converting it into a list of lists, which resembles an array structure.

Here’s an example:

import pandas as pd
df = pd.DataFrame({'x': [1, 2], 'y': [3, 4], 'z': [5, 6]})
array = [row.tolist() for index, row in df.iterrows()]

Output:

[[1, 3, 5], [2, 4, 6]]

In this code, the iterrows() method is used in a list comprehension to iterate over the DataFrame rows and the tolist() method is called on each row to convert each row into a list. The resulting structure is a list of lists.

Summary/Discussion

• Method 1: to_numpy(). Direct and simple. Ensures pandas compatibility. Does not allow dtype specification while converting.
• Method 2: values attribute. Quick and easy but slightly deprecated. No control over dtype during the conversion.
• Method 3: astype() with to_numpy(). Allows data type change before conversion. Extra step required to specify dtype.
• Method 4: np.array() function. Good for datatype control upon initializing the array. It’s an external approach, which means it relies on numpy directly.
• Method 5: List Comprehension. More manual and flexible way of converting DataFrames into an array of lists. Useful for custom transformations.