attempting to dive into classes()
and I thought I'd make a real-world program that would help one of my colleagues at work.
I'm using the watchdog API to watch
a folder, the behavior I'm after is that when a file is moved into this folder I want to rename it according to course_name
column in the csv, simple so far right?
now when I run the above pseudo logic I keep getting a FileNotFoundError
however the code does work - but the API is still searching for the file that was removed/changed?
from what I can see something is executing after my function but I can't for the life of me figure out what?
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import pandas as pd
import os
from shutil import copyfile
my_path = r"<dir_to_watch>"
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print(f'event type: {event.event_type} path : {event.src_path}')
df = pd.read_csv(event.src_path) # read the file
course = df['Course Name'].unique().tolist()[0] # pass course name to a variable
copyfile(event.src_path, f"{course}.csv") # copy file, using os.rename threw up an error.
os.remove(event.src_path) # remove original file.
print("file renamed")
I then execute the above with :
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path=my_path, recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
If any additional information is needed please ask.
the traceback error is quite long my apologies :
Exception in thread Thread-8:
Traceback (most recent call last):
File "\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "\Anaconda3\lib\site-packages\watchdog\observers\api.py", line 199, in run
self.dispatch_events(self.event_queue, self.timeout)
File "\Anaconda3\lib\site-packages\watchdog\observers\api.py", line 368, in dispatch_events
handler.dispatch(event)
File "\Anaconda3\lib\site-packages\watchdog\events.py", line 330, in dispatch
_method_map[event_type](event)
File "<ipython-input-7-30cb2defae10>", line 13, in on_modified
df = pd.read_csv(event.src_path)
File "\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 702, in parser_f
return _read(filepath_or_buffer, kwds)
File "\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 429, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 895, in __init__
self._make_engine(self.engine)
File "\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 1122, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 1853, in __init__
self._reader = parsers.TextReader(src, **kwds)
File "pandas\_libs\parsers.pyx", line 387, in pandas._libs.parsers.TextReader.__cinit__
File "pandas\_libs\parsers.pyx", line 705, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File b'report - Copy.csv' does not exist: b'report - Copy.csv'
I have found that on_modified events can 'fire' twice for the same file. I solved this with a deque of recent events for my application, but a last_event variable would work to prevent duplicates. You may want to find a way of re-setting last_event, if you are expecting input files with the same name.
Also I have found a short wait in on_modified() to allow the file to be fully written to be very useful in preventing unexpected behavior if performing operations on that file.
last_event = ''
def on_modified(self, event):
time.sleep(1) # wait to allow file to be fully written
if not event.src_path == last_event: # files we haven't seen recently
# do something
last_event = event.src_path
An easier option however, and perhaps more pythonic, would be just to handle that exception properly:
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print(f'event type: {event.event_type} path : {event.src_path}')
try:
df = pd.read_csv(event.src_path) # read the file
course = df['Course Name'].unique().tolist()[0] # pass course name to a variable
copyfile(event.src_path, f"{course}.csv") # copy file, using os.rename threw up an error.
os.remove(event.src_path) # remove original file.
print("file renamed")
except FileNotFoundError:
# handle the error
pass