How to Convert .dat to .csv File in Python?

by

Problem Formulation

Given a file my_file.dat like this one:

A | B | C | D  | E | F
1 | 5 | 9 | 5  | 1 | hello
2 | 6 | 8 | 4  | 0 | finxters
3 | 7 | 7 | 3  | 1 | learn
4 | 8 | 6 | 2  | 2 | python

How to convert this file to a CSV file of the following standard comma-separated values format:

A,B,C,D,E,F
1,5,9,5,1,hello
2,6,8,4,0,finxter
3,7,7,3,1,learn
4,8,6,2,2,python

Method 1: Pandas Read and Write CSV

You can convert a .dat file to a CSV file in Python in four simple steps: (1) Install the Pandas library, (2) import the Pandas library, (3) read the CSV file as DataFrame, and (4) write the DataFrame to the file.

  1. (Optional in shell) pip install pandas
  2. import pandas as pd
  3. df = pd.read_csv('my_file.txt', sep='\s+|\s+')
  4. df.to_csv('my_file.csv', index=None)

Here’s a minimal example:

import pandas as pd
read_file = pd.read_csv('my_file.dat', sep='\s+\|\s+')
read_file.to_csv ('my_file.csv', index=None)

Note that we used the regular expression sep='\s+|\s+' to use any whitespace, followed by the '|' symbol, followed by any whitespace as a separator between two CSV values. If you have a different separator string, you can define it here.

🌍 Related Tutorial: Python Pandas DataFrame to_csv()

Method 2: CSV Module

You can use the csv module to read a .dat file and write it as a .csv file in 5 steps:

  1. Install and import the csv module
  2. Open the .dat file in reading mode and the .csv file in writing mode.
  3. Create a csv.writer() object.
  4. Iterate over the rows in the .dat file and reformat them to a list of values.
  5. Write the list of row values to the CSV and repeat for the next row until completion.

Here’s a concrete example:

import csv

with open('my_file.dat', 'r') as dat_file:
    with open('my_file.csv', 'w', newline='') as csv_file:
        csv_writer = csv.writer(csv_file)
        for row in dat_file:
            row = [value.strip() for value in row.split('|')]
            csv_writer.writerow(row)

The resulting CSV file looks like this:

You can learn more about some of the features used in the code here:

Method 3: Vanilla Python

No library is needed to convert a .csv to a .dat in Python. Simply use the standard file handling functionality open() and file.write(), in addition to basic Python tricks to convert the data in the input file to comma-separated values.

with open('my_file.dat', 'r') as dat_file:
    with open('my_file.csv', 'w', newline='') as csv_file:
        for row in dat_file:
            row = [value.strip() for value in row.split('|')]
            csv_file.write(','.join(row) + '\n')

For example, in the code we used the string.join() method to create a fully compatible CSV row.

🌍 Related Tutorial: Python Convert String to CSV File