Store Only What Changed

A new Traversal run does not need a new full copy of the data. WDS keeps the recurring collection lean.

Run 41 → Baseline stored
Run 42 → Only price changes added
Run 43 → Only new or updated records added

Unchanged scraped output is not stored again. That limits duplication in the WDS database while preserving a run-by-run history.

With Get Traversal Run Data, your pipeline can still retrieve any run by number in cursor-paginated batches and work with the complete data for that run.

Complete snapshots when you read. Incremental storage underneath.

Track changes over time—without storing the same values over and over.