Backfill history

TL;DR

Loading a historic period, which source can supply it, and the one database that must never be written to.

What each source can supply

SourceHistoric?
NetSuiteYes, by date. Queue the days you want
verkoopdumpYes, by hand in slices. Pre-2022 and the acquisition gap
GA4Order attribution: no date parameter. Traffic: requested by date
Akeneo, Azure, PriceShapeNo. Current state only
commercetoolsYes, by date. The nightly fan-out covers a trailing window; older days are queued by hand

Asking the GA4 order-attribution worker for an old period returns the recent window again. That is why 2022 to 2025 acquisition data comes from verkoopdump instead.

Before touching verkoopdump

Read-only, without exception. It is the company's only copy of pre-2022 order detail, with no backup story in this repository, and the credential is not read-only. Only the check in the code stands between a statement and a write. Reach it only through the model that performs that check.

Procedure

  1. Decide the window and which source covers it.
  2. Queue it in slices, typically a year, rather than in one run.
  3. Let each slice drain before queueing the next; a large queue blocks everything that depends on the same type.
  4. Switch the historic step off again once the window is complete.
  5. Read real rows back across the window, at both edges and in the middle.

Verify

Row counts for the period, and a spot check that classification and calculation followed the import rather than stopping at it. A finished batch is not evidence.

Evidence

Read-only rule from AGENTS.md, Forbidden paths. Slice-and-switch-off pattern and the historic schedule line from AGENTS.md, Jobs and the batches queue. GA4's rolling window from AGENTS.md, The application. The commercetools trailing window from app/jobs/system/resetdaily.php and ingest.trailingqueuedays in config/settings.json. Basis: code-checked against the current implementation.

About this document
  • Type
    runbook
  • Status
    active
  • Updated
    2026-09-14
  • Created
    2026-09-13