Panda's Write CSV - Append vs. Write

I would like to use pd.write_csv to write "filename" (with headers) if "filename" doesn't exist, otherwise to append to "filename" if it exists. If I simply use command:

     df.to_csv('filename.csv',mode = 'a',header ='column_names')

The write or append succeeds, but it seems like the header is written every time an append takes place.

How can I only add the header if the file doesn't exist, and append without header if the file does exist?

Asked By: GPB

Source

Answer #1:

Not sure there is a way in pandas but checking if the file exists would be a simple approach:

import os
# if file does not exist write header 
if not os.path.isfile('filename.csv'):
   df.to_csv('filename.csv', header='column_names')
else: # else it exists so append without writing the header
   df.to_csv('filename.csv', mode='a', header=False)

Answered By: Padraic Cunningham

Answer #2:

with open(filename, 'a') as f:
    df.to_csv(f, mode='a', header=f.tell()==0)

it will add header when writes to the file first time

Answered By: user3657041

Answer #3:

In Pandas dataframe "to_csv" function, use header=False if csv file exists & append to existing file.

    import os

    hdr = False  if os.path.isfile('filename.csv') else True
    df.to_csv('filename.csv', mode='a', header=hdr)

Answered By: VK Singh

Answer #4:

The above solutions are great, but I have a moral obligation to include the pathlib solution here:

from pathlib import Path

file_path = Path(filename)
if file_path.exists():
   df.to_csv(file_path, header=False, mode='a')
else:
   df.to_csv(file_path, header=True, mode='w')

Alternatively (depending on your inlining preferences):

file_exists = file_path.exists()
df.to_csv(file_path, header=not file_exists, mode='a' if file_exists else 'w')

Answered By: DV82XL

The answers/resolutions are collected from stackoverflow, are licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0 .