Question 4
A nightly scraper normally covers 100 pages. Tonight it fails after page 60. The pipeline marks every previously known item not encountered tonight as deleted. What is the integrity error?
Treat unseen items as deleted because each scheduled run represents a complete source snapshot.
Remove previous state after failure so the next successful run reconstructs the catalogue cleanly.
Preserve prior state because incomplete coverage cannot establish that unseen items disappeared.
Trust stored hashes to identify deletions even when later pages were never fetched.