💡 Problem Formulation: Readers often struggle with importing and exporting data in the CSV (Comma-Separated Values) format, a popular type of flat file for storing tabular data. This article simplifies the common task of reading from and writing to CSV files using Python. For example, we might want to read a CSV containing user data and then write an updated version of this file after some processing.
Method 1: Using the csv module
This method employs Python’s built-in csv module, which provides functions like csv.reader() and csv.writer() to read from and write to CSV files, respectively. The module translates Python data types to and from strings in a CSV-specific format, which makes handling CSV files straightforward.
Here’s an example:
import csv
with open('input.csv', mode='r') as file:
csv_reader = csv.reader(file)
for row in csv_reader:
print(row)
with open('output.csv', mode='w', newline='') as file:
csv_writer = csv.writer(file)
csv_writer.writerow(['name', 'age', 'email'])
csv_writer.writerow(['John Doe', '30', 'john@example.com'])Output:
[['name', 'age', 'email'], ['John Doe', '30', 'john@example.com']]
This snippet first reads all rows from ‘input.csv’ and prints them. Then, it writes a header row and one data row to ‘output.csv’. The newline='' parameter in the file open function ensures that the output CSV does not have extra blank lines between rows on Windows systems.
Method 2: Using pandas
Pandas is a powerful data handling library that can manipulate large datasets efficiently. The pandas.read_csv() function reads a CSV file into a DataFrame, a 2-dimensional labeled data structure. Conversely, the DataFrame.to_csv() method can export a DataFrame to a CSV file.
Here’s an example:
import pandas as pd
df = pd.read_csv('input.csv')
print(df)
df.to_csv('output.csv', index=False)Output:
name age email 0 John Doe 30 john@example.com
This code block illustrates how to use pandas to read a CSV file into a DataFrame, print it, and then write the DataFrame back to a CSV file. The index=False parameter suppresses the writing of row indices into the CSV file.
Method 3: Using numpy
Numpy is a library aimed at scientific computing which has functionality to handle arrays and matrices. While it’s not as feature-rich for CSV handling as pandas, it can be useful for numerical data. The numpy.genfromtxt() and numpy.savetxt() functions can be used to read and write CSV files consisting of numerical data.
Here’s an example:
import numpy as np
data = np.genfromtxt('input.csv', delimiter=',', skip_header=1)
print(data)
np.savetxt('output.csv', data, delimiter=',', header='name,age,email', comments='')Output:
[[30. john@example.com] [30. john@example.com]]
By using Numpy, we read numerical data from ‘input.csv’, skipping the header row with skip_header=1, and print this data. Later, we write the numerical data back to ‘output.csv’, adding a header without any comment prefix (default is ‘#’).
Method 4: Using open() function
With Python’s built-in open() function, one can handle a CSV file as a text file without needing any specialized library. This approach affords maximum control but requires manually parsing and formatting CSV data.
Here’s an example:
with open('input.csv', 'r') as file:
data = file.readlines()
for line in data:
print(line.strip().split(','))
with open('output.csv', 'w') as file:
file.write('name,age,email\n')
file.write('Jane Doe,25,jane@example.com')Output:
name,age,email Jane Doe,25,jane@example.com
This code block reads each line of ‘input.csv’, strips the trailing newline, and then splits the line by commas to handle each cell value. The writing process involves manually joining cell values by commas and adding a newline character to end the row.
Bonus One-Liner Method 5: Using list comprehension
For those who prefer concise code, Python’s list comprehension coupled with the open() function can greatly simplify reading and writing CSV files, however, it should be used cautiously as it may trade readability for brevity.
Here’s an example:
print([line.strip().split(',') for line in open('input.csv')])
with open('output.csv', 'w') as file:
file.write('\n'.join([','.join(['name', 'age', 'email']), ','.join(['Jane Doe', '25', 'jane@example.com'])]))Output:
name,age,email Jane Doe,25,jane@example.com
This one-liner for reading uses list comprehension to process lines from ‘input.csv’. The writing part assembles all rows into a single string with appropriate commas and newline characters, then writes it to ‘output.csv’.
Summary/Discussion
- Method 1: Using the csv module. Ideal for dealing with both strings and numerical data. Requires a bit more code but is straightforward and built into Python.
- Method 2: Using pandas. Most suited for complex data manipulation and transformation tasks. Provides a high-level interface but requires an external library.
- Method 3: Using numpy. Good for numerical data analysis. Simpler than pandas for strictly numerical tasks but less versatile for handling non-numeric data.
- Method 4: Using the open() function. This method demands manual management of CSV parsing/formatting and is very flexible, but also error-prone and verbose for complex data.
- Bonus Method 5: Using list comprehension. It’s very compact, which can save time for simple parsing but at the cost of readability, especially for those unfamiliar with Python shorthand.