Question 15
When implementing data lineage tracking for a complex pipeline with multiple data sources, transformations, and outputs, which metadata is most critical to capture?
Capture only the final output location of the pipeline for reference
Capture source IDs, transformation versions, timestamps, data quality metrics, and dependencies
Capture only the names of users who executed the pipeline for auditing
Capture only storage costs and query performance metrics for monitoring