5 Best Ways to Convert a CSV File to a Nested List in Python

by

💡 Problem Formulation: Python developers often need to convert CSV files into a structured nested list, where each sublist represents a row from the CSV, with individual elements corresponding to the cells within that row. For example, the input could be a CSV file with entries separated by commas, and the desired output should be a list of lists, such as [["Header1", "Header2"], ["row1col1", "row1col2"], ["row2col1", "row2col2"]].

Method 1: Using the csv.reader class

The csv.reader class in Python’s csv module is a versatile and straightforward way to parse CSV files into a nested list. It handles different CSV dialects and takes care of complexities such as special character encapsulation and line terminators.

Here’s an example:

import csv

with open('data.csv', 'r') as file:
    csv_reader = csv.reader(file)
    nested_list = list(csv_reader)

print(nested_list)

Output:

[["Header1", "Header2"], ["row1col1", "row1col2"], ["row2col1", "row2col2"]]

The provided code snippet opens a CSV file called data.csv, reads it using csv.reader, and converts it to a nested list with the list() function. Each sublist represents a row from the CSV.

Method 2: Using Pandas

Pandas is a powerful data manipulation library in Python that can simplify CSV parsing. The pandas.read_csv function loads the CSV into a DataFrame, which can then be converted to a nested list with the values.tolist() method.

Here’s an example:

import pandas as pd

df = pd.read_csv('data.csv')
nested_list = df.values.tolist()

print(nested_list)

Output:

[["row1col1", "row1col2"], ["row2col1", "row2col2"]]

This code uses Pandas to read a CSV file into a DataFrame and then converts that DataFrame into a nested list. Note that the default behavior does not include the header row in the resulting nested list.

Method 3: Using List Comprehension with csv.reader

List comprehension provides a concise way to use csv.reader for converting CSV files to nested lists. This method is similar to Method 1 but might be more suitable for Python developers who prefer more Pythonic approaches.

Here’s an example:

import csv

with open('data.csv', 'r') as file:
    nested_list = [row for row in csv.reader(file)]

print(nested_list)

Output:

[["Header1", "Header2"], ["row1col1", "row1col2"], ["row2col1", "row2col2"]]

The list comprehension iterates over the CSV file using csv.reader and constructs the nested list inline. This is a concise alternative to calling list() on the reader object.

Method 4: Using csv.DictReader for Nested Dictionaries

Sometimes, converting CSV data into a list of dictionaries, where each dictionary represents a row, can be more beneficial, especially if you want to address columns by header names. The csv.DictReader object is perfect for this.

Here’s an example:

import csv

with open('data.csv', mode='r') as file:
    dict_reader = csv.DictReader(file)
    nested_list = [dict(row) for row in dict_reader]

print(nested_list)

Output:

[{"Header1": "row1col1", "Header2": "row1col2"}, {"Header1": "row2col1", "Header2": "row2col2"}]

This snippet reads the CSV file into a csv.DictReader object and then uses a list comprehension to create a list of dictionaries, with each dictionary representing one row in the CSV file with header names as keys.

Bonus One-Liner Method 5: Using a Generator Expression

If you seek a minimalist approach and you’re dealing with a large CSV file, a generator expression together with csv.reader can provide an on-demand parsing, one row at a time.

Here’s an example:

import csv

with open('data.csv', 'r') as file:
    nested_list = (row for row in csv.reader(file))

print(next(nested_list))  # Prints the first row
print(next(nested_list))  # Prints the second row

Output:

["Header1", "Header2"]
["row1col1", "row1col2"]

This code turns the CSV parsing into an iterator which yields one row at a time, allowing memory-efficient reading of large files. Note that it only produces values on demand and consequently does not create a complete nested list in memory.

Summary/Discussion

  • Method 1: csv.reader: Simple and robust. Handles different CSV dialects. May not be the fastest for very large files.
  • Method 2: Pandas: Extremely powerful and fast for large datasets. However, requires an external library which might not be ideal for lightweight applications.
  • Method 3: List Comprehension with csv.reader: Pythonic and concise. Works well for Python developers comfortable with list comprehensions. Performance is similar to Method 1.
  • Method 4: csv.DictReader: Best for CSV files with headers, allowing access by column names. May not be suitable for all data structures, especially if a list of lists is necessary.
  • Bonus One-Liner Method 5: Generator Expression is memory efficient. Ideal for very large CSV files, but not practical if you need to access all data at once.