Current behavior
load_directory() in src/Process.py calls load_pdf(filepath) for each file with no error handling. If a single PDF is corrupted, password-protected,
or otherwise unreadable by PyMuPDF, fitz.open() raises an exception, and the exception propagates all the way up through load_directory, killing the run, even though the other files in the directory would have processed fine.
Expected behavior
A single bad file should be skipped (with a warning logged) so the rest of the directory is still processed. The pipeline is meant to run over a
whole corpus; one bad input shouldn't take down the batch.
Current behavior
load_directory()insrc/Process.pycallsload_pdf(filepath)for each file with no error handling. If a single PDF is corrupted, password-protected,or otherwise unreadable by PyMuPDF,
fitz.open()raises an exception, and the exception propagates all the way up throughload_directory, killing the run, even though the other files in the directory would have processed fine.Expected behavior
A single bad file should be skipped (with a warning logged) so the rest of the directory is still processed. The pipeline is meant to run over a
whole corpus; one bad input shouldn't take down the batch.