Re-upload files without creating duplicates

What this does

Explains the promise printed under every upload area: you can re-upload the same file, or overlapping date ranges, without creating duplicates.

Why you would want to

• Your broker caps each export at six months and you want three years.
• You are not sure whether an earlier upload finished.
• You connected an automatic sync and also want the history the sync cannot reach.
• A parser fix has shipped for your broker and you were told to re-import.

All four are normal. Overlap costs you nothing but the upload time.

The three layers

Fill level, before reconstruction. Every fill carries an execution id from your broker. VEXA drops fills whose ids it already holds before it rebuilds anything, so merging a new file with positions it already has open cannot double-count. Brokers that omit an execution id get a deterministic one derived from the fill itself, so the same fill produces the same id every time.

Trade level, after reconstruction. A finished trade is matched two ways: by its id, and by a fingerprint of symbol, account, entry second, exit second and quantity, with an hour of tolerance for daylight-saving shifts. This is the layer that catches the same trade arriving once from a file and once from the automatic sync.

The imported-execution registry. Brokers with rolling windows keep re-delivering old fills. Interactive Brokers, for one, hands back the same 365 days every time. VEXA records every fill it has consumed into a finished trade and filters those out on the next delivery. Exact broker execution ids are dropped at any age. Fingerprint matches are only applied to fills older than thirty days, so two genuine same-second fills you made this morning are never touched.

What a trade id is made of

It depends on how the trade arrived, and the two paths do not use the same key.

A trade built from a file you uploaded is keyed on your user id, the symbol, the entry time, the exit time, the entry price, the exit price, the quantity, the profit, and the broker’s own position or ticket id where the broker provides one. Prices and profit are part of the key on this path, so the same file uploaded twice lands on the same id, but a file whose prices differ in the fourth decimal is a different trade as far as the id is concerned. That is what the two layers above are there for.

A trade built by the Interactive Brokers sync is keyed on your user id, the broker account, the symbol, the entry time, the exit time and the quantity. No price and no profit, so a re-pull that recalculates either of them still lands on the same id.

The scheduled sweep

Twice a day, at 03:00 and 15:00 UTC, VEXA re-checks accounts with recent activity for trades that slipped past the import-time layers. It matches on the same fingerprint plus a two-second window for feed jitter, and it refuses to treat two trades as the same when both carry distinct broker position ids. That last rule matters for hedging accounts, where two positions opened in the same second on the same symbol are genuinely two trades.

What you should see

Upload a file you have already uploaded and the result panel reads: all N trades in this file were already imported from previous uploads. Imported is zero, duplicates is N. Nothing changes in Journal or Metrics.

Upload an overlapping range and you get a mix: the new trades imported, the overlap counted as already in your account.

If it doesn’t work

• You genuinely see the same trade twice. That is a bug, not a setting. Note the broker, whether both an automatic sync and a file cover those dates, and the file you used, then contact support. On Interactive Brokers you also have Remove duplicate trades.
• Trades vanished after a re-upload. Re-uploading never deletes. Check the account filter before anything else, because a new account created by the upload can leave older trades out of view. See The account filter.
• The same trade shows with two different profits. Not a duplicate. That is two accounts holding one symbol, or a multiplier problem. Start with your broker’s article.