Skip to content
  • There are no suggestions because the search field is empty.

How Duplicate Detection Works

You're uploading files and you want to know how Valid8 decides whether a file is new or a duplicate, or you've had a file rejected as a duplicate and you're trying to understand why.

The comparison is byte-based, not content-based

Duplicate detection compares files byte-for-byte — it does not open the file to read dates, account numbers, transaction counts, or any other content. A file that differs by even a single byte anywhere in the document is considered distinct and is not flagged as a duplicate.

This means:

  • A re-scanned or re-exported version of the same statement is not a duplicate. If you received a statement PDF from your bank, uploaded it, deleted it, and then requested a fresh export of the same statement from the bank and uploaded that, the second file is treated as new even though it covers the same account and date range. The byte streams differ because the export timestamps, metadata, or rendering details changed.
  • A modified copy is not a duplicate. If you opened a PDF to redact account numbers, add annotations, or combine multiple statements into one file, the edited file is distinct from the original and will not be caught by duplicate detection.
  • File format changes break the duplicate check. If you converted a file from one format to another (e.g., saved an Excel export as CSV with different line endings, or re-rendered a PDF), the new file is not a duplicate even if the visible content is identical.

 

CSV uploads get no duplicate detection at all

Transaction Lists (CSV) and Check Item Lists (CSV) have no duplicate detection of any kind. You can upload the exact same CSV file multiple times to the same project and it will be imported every time, creating duplicate transactions or check records. There is no fingerprint stored, no external service checking for copies, and no rejection on re-upload.

This is expected behavior, not a gap. If you need to re-import a corrected CSV, you must manually delete the original import's data first — there is no automatic detection or merge.

What "duplicate" means for rejection notes

When a file is rejected as a duplicate, the Notes column on the ALL FILES screen (visible once you switch to the REJECTED chip) shows a message explaining the file was already processed. The exact wording depends on which pathway handled it, but both convey the same outcome: the platform recognized the file and blocked it from being processed again.