Question 1
Which data serialization format is most efficient for storing large numeric arrays with minimal overhead?
XML
Parquet
JSON
Plain text CSV
Tools in Data Science End Term: 21 December 2025 (September 2025 term)
The IIT Madras BS Tools in Data Science (Tools in Data Science (TDS)) End Term paper sat on 21 Dec 2025, in the September 2025 term: 36 questions for 40 marks in 180 minutes. Every question is below with its answer. Take it as a timed mock test to be marked, or read it through first.
- 36
- 40
- 180 min
- 36
Show answer
Correct answer
Parquet
Question 2
In Git, which command shows the differences between your working directory and the last commit?
git status
git diff
git log
git show
Show answer
Correct answer
git diff
Question 3
What does HTTP status code 429 indicate?
Internal server error
Rate limit exceeded
Unauthorized access
Resource not found
Show answer
Correct answer
Rate limit exceeded
Question 4
Which Python library is primarily designed for exploratory data analysis and statistical visualization?
NumPy
Seaborn
Django
Flask
Show answer
Correct answer
Seaborn
Question 5
What is the main purpose of using environment variables in production systems?
To speed up code execution
To store configuration and secrets separately from code
To replace all function parameters
To automatically test applications
Show answer
Correct answer
To store configuration and secrets separately from code
Question 6
When validating incoming data from external APIs, which validation strategy prevents downstream pipeline failures most effectively?
Check only that the data is not empty
Validate data types, required fields, value ranges, and fail fast with clear error messages
Accept all data and let downstream processes handle errors
Log warnings but process all data regardless
Show answer
Correct answer
Validate data types, required fields, value ranges, and fail fast with clear error messages
Question 7
You are building a daily ETL pipeline that processes customer orders. Orders can arrive late or be updated. Which design pattern ensures you don't miss late-arriving data while avoiding duplicate processing?
Process only data from the current day and ignore anything older
Use lookback windows with deduplication based on unique order IDs and update timestamps
Reprocess all historical data every day to ensure completeness
Process data as it arrives without tracking what has been processed
Show answer
Correct answer
Use lookback windows with deduplication based on unique order IDs and update timestamps
Question 8
Your analytics dashboard queries a large dataset that updates hourly but is queried every few seconds by users. Which caching strategy improves performance while maintaining reasonable freshness?
Never cache and always query the live dataset directly
Cache results with a TTL matching update frequency and invalidate on updates
Cache results permanently without refreshing them for future queries
Disable the dashboard whenever the dataset is updated to prevent queries
Show answer
Correct answer
Cache results with a TTL matching update frequency and invalidate on updates
Question 9
When joining datasets from different time zones, you discover that some timestamps lack timezone information. What is the safest approach?
Assume all missing timestamps are in UTC for simplicity
Infer timezone from metadata or location, document assumptions, and flag inferred values
Reject any records that do not have explicit timezone information
The code fails because the data contains timestamps in a non-UTC time zone that pandas cannot parse.
Show answer
Correct answer
Infer timezone from metadata or location, document assumptions, and flag inferred values
Question 10
You maintain a data pipeline that transforms raw logs into analytics tables. The transformation logic changes frequently. How should you version and test these transformations?
Keep all transformation logic in a single script and update it directly
Version transformations with git, write unit tests, test on sample data before production
Manually document changes in a shared document
Avoid making changes to prevent breaking existing processes
Show answer
Correct answer
Version transformations with git, write unit tests, test on sample data before production
Question 11
When aggregating financial transaction data across multiple currencies, why is it important to store both the original currency amount and the exchange rate used for conversion?
It's unnecessary; storing only the converted amount is sufficient
Enables auditing, recalculation with updated rates, and transparency
It wastes storage with no practical benefit
Regulatory requirements mandate it for all data types
Show answer
Correct answer
Enables auditing, recalculation with updated rates, and transparency
Question 12
Your team builds a recommendation system that suggests products to users. After deployment, you notice that recommendations for certain user segments are significantly worse than others. What is the best systematic approach to diagnose and fix this?
Retrain the model with more data and hope it improves
Analyze performance by segment, identify biases in training data, add segment-specific tests, and validate fixes
Disable recommendations for poorly performing segments
Manually override recommendations for affected users
Show answer
Correct answer
Analyze performance by segment, identify biases in training data, add segment-specific tests, and validate fixes
Question 13
Your data pipeline uses a third-party library that has known security vulnerabilities in older versions. The latest version includes breaking API changes. What should you do?
Stay on the older vulnerable version to avoid breaking existing code
Assess vulnerability severity, test the new version in staging, and plan migration or patches
Update to the latest version immediately without any testing
Replace the library entirely with custom code regardless of complexity
Show answer
Correct answer
Assess vulnerability severity, test the new version in staging, and plan migration or patches
Question 14
For ensuring long-term reproducibility of data analysis notebooks that depend on external data sources and multiple libraries, which approach is most comprehensive?
Save only the final notebook with outputs
Use containers, pin dependencies, archive input data, version-control notebooks, and document environment
Share notebooks via email with verbal instructions
Rely on cloud platforms to maintain compatibility
Show answer
Correct answer
Use containers, pin dependencies, archive input data, version-control notebooks, and document environment
Question 15
When implementing data lineage tracking for a complex pipeline with multiple data sources, transformations, and outputs, which metadata is most critical to capture?
Capture only the final output location of the pipeline for reference
Capture source IDs, transformation versions, timestamps, data quality metrics, and dependencies
Capture only the names of users who executed the pipeline for auditing
Capture only storage costs and query performance metrics for monitoring
Show answer
Correct answer
Capture source IDs, transformation versions, timestamps, data quality metrics, and dependencies
Question 16
Your pipeline processes personally identifiable information (PII). A privacy audit requires you to implement data minimization. Which strategy best balances utility and privacy?
Encrypt all data and continue processing it without changes
Keep only necessary fields, anonymize or aggregate data, enforce access controls, and document retention
Delete all PII immediately without considering business requirements
Move PII to a separate database while keeping the same access patterns
Show answer
Correct answer
Keep only necessary fields, anonymize or aggregate data, enforce access controls, and document retention
Question 17
Scenario 1: Restaurant Review Data Analysis
Context
A food delivery platform wants to analyze restaurant reviews to identify trending cuisines and customer satisfaction patterns. They have review data in CSV format with ratings, text comments, and timestamps.
Sample Data (restaurants.csv):
Based on the above data, answer the given subquestions.
The dataset has some missing values in the review_text column (some customers left ratings but no comment). Before analyzing review text for common words, what should you do?
Ignore missing values and let pandas handle them automatically
Remove rows with missing review text using df.dropna(subset=['review_text'])
Replace all missing review text entries with the word "missing"
Delete the entire review_text column from the dataset
Show answer
Correct answer
Remove rows with missing review text using df.dropna(subset=['review_text'])
Question 18
Scenario 1: Restaurant Review Data Analysis
Context
A food delivery platform wants to analyze restaurant reviews to identify trending cuisines and customer satisfaction patterns. They have review data in CSV format with ratings, text comments, and timestamps.
Sample Data (restaurants.csv):
Based on the above data, answer the given subquestions.
To identify the most common words in negative reviews (rating < 3.0), which approach correctly combines filtering and text analysis?
Count all words in all reviews regardless of rating
Filter for low ratings, then extract and count words from the review_text column
Sort reviews by rating and manually read the bottom ones
Use only the rating numbers without looking at text
Show answer
Correct answer
Filter for low ratings, then extract and count words from the review_text column
Question 19
Scenario 1: Restaurant Review Data Analysis
Context
A food delivery platform wants to analyze restaurant reviews to identify trending cuisines and customer satisfaction patterns. They have review data in CSV format with ratings, text comments, and timestamps.
Sample Data (restaurants.csv):
Based on the above data, answer the given subquestions.
After completing the analysis, which file format is most appropriate for sharing summary results (e.g., average rating by cuisine) with non-technical stakeholders?
Python pickle file (.pkl)
CSV file that can be opened in Excel
JSON with nested structures
Binary database file
Show answer
Correct answer
CSV file that can be opened in Excel
Question 20
Scenario 1: Restaurant Review Data Analysis
Context
A food delivery platform wants to analyze restaurant reviews to identify trending cuisines and customer satisfaction patterns. They have review data in CSV format with ratings, text comments, and timestamps.
Sample Data (restaurants.csv):
Based on the above data, answer the given subquestions.
Which Python library is most commonly used for loading and analyzing tabular CSV data like this restaurant reviews dataset?
requests
pandas
BeautifulSoup
Flask
Show answer
Correct answer
pandas
Question 21
Scenario 1: Restaurant Review Data Analysis
Context
A food delivery platform wants to analyze restaurant reviews to identify trending cuisines and customer satisfaction patterns. They have review data in CSV format with ratings, text comments, and timestamps.
Sample Data (restaurants.csv):
Based on the above data, answer the given subquestions.
To find the average rating for each cuisine type (Indian, Italian, Japanese), which pandas operation is most appropriate?
df.sort_values('cuisine')
df.groupby('cuisine')['rating'].mean()
df.filter('cuisine')
df.merge('cuisine', 'rating')
Show answer
Correct answer
df.groupby('cuisine')['rating'].mean()
Question 22
Scenario 2: Automated Git Workflow for Team Project
Context
A data science team uses Git for version control on a shared project. Multiple team members work on different features simultaneously. They follow a workflow where everyone works on separate branches and merges into the main branch through pull requests.
Current Setup:
• main branch: Production-ready code • dev branch: Development/testing code • Feature branches: Individual work (e.g., feature-data-cleaning, feature-visualization)
Based on the above data, answer the given subquestions.
After making changes to your code, what is the correct sequence to save your work and upload it to GitHub?
git push → git add → git commit
git commit → git add → git push
git add → git commit → git push
git merge → git push
Show answer
Correct answer
git add → git commit → git push
Question 23
Scenario 2: Automated Git Workflow for Team Project
Context
A data science team uses Git for version control on a shared project. Multiple team members work on different features simultaneously. They follow a workflow where everyone works on separate branches and merges into the main branch through pull requests.
Current Setup:
• main branch: Production-ready code • dev branch: Development/testing code • Feature branches: Individual work (e.g., feature-data-cleaning, feature-visualization)
Based on the above data, answer the given subquestions.
Two team members both modify the same line in analysis.py on different branches. When the second person tries to merge their branch, what will happen?
Git automatically keeps the newer change
Git creates a merge conflict that must be resolved manually by choosing which change to keep
Git deletes both changes
Git randomly picks one change
Show answer
Correct answer
Git creates a merge conflict that must be resolved manually by choosing which change to keep
Question 24
Scenario 2: Automated Git Workflow for Team Project
Context
A data science team uses Git for version control on a shared project. Multiple team members work on different features simultaneously. They follow a workflow where everyone works on separate branches and merges into the main branch through pull requests.
Current Setup:
• main branch: Production-ready code • dev branch: Development/testing code • Feature branches: Individual work (e.g., feature-data-cleaning, feature-visualization)
Based on the above data, answer the given subquestions.
Before pushing your changes, you want to make sure your branch has the latest updates from the main branch. Which command downloads and integrates those updates?
git commit main
git pull origin main (while on your feature branch)
git delete main
git init main
Show answer
Correct answer
git pull origin main (while on your feature branch)
Question 25
Scenario 2: Automated Git Workflow for Team Project
Context
A data science team uses Git for version control on a shared project. Multiple team members work on different features simultaneously. They follow a workflow where everyone works on separate branches and merges into the main branch through pull requests.
Current Setup:
• main branch: Production-ready code • dev branch: Development/testing code • Feature branches: Individual work (e.g., feature-data-cleaning, feature-visualization)
Based on the above data, answer the given subquestions.
What is the main purpose of creating a pull request (PR) instead of directly merging your branch into main?
Pull requests are faster than merging
Pull requests allow team members to review your code, suggest changes, and approve before merging
Pull requests automatically fix bugs
Pull requests are only for documentation
Show answer
Correct answer
Pull requests allow team members to review your code, suggest changes, and approve before merging
Question 26
Scenario 2: Automated Git Workflow for Team Project
Context
A data science team uses Git for version control on a shared project. Multiple team members work on different features simultaneously. They follow a workflow where everyone works on separate branches and merges into the main branch through pull requests.
Current Setup:
• main branch: Production-ready code • dev branch: Development/testing code • Feature branches: Individual work (e.g., feature-data-cleaning, feature-visualization)
Based on the above data, answer the given subquestions.
When starting work on a new feature, what is the correct Git command sequence to create and switch to a new branch called feature-analysis?
git commit feature-analysis
git branch feature-analysis then git checkout feature-analysis
git push feature-analysis
git merge feature-analysis
Show answer
Correct answer
git branch feature-analysis then git checkout feature-analysis
Question 27
Scenario 3 : Deploying a Data Dashboard with Docker
Context
A team built a Python dashboard using Streamlit that visualizes sales data. They want to deploy it so anyone can access it via a web browser. They decide to use Docker to package the application with all its dependencies.
Based on the above data, answer the given subquestions.
After building the Docker image, which command correctly runs the container and makes the dashboard accessible on port 8501?
docker build -t dashboard .
docker run -p 8501:8501 dashboard
docker push dashboard
None of these
Show answer
Correct answer
docker run -p 8501:8501 dashboard
Question 28
Scenario 3 : Deploying a Data Dashboard with Docker
Context
A team built a Python dashboard using Streamlit that visualizes sales data. They want to deploy it so anyone can access it via a web browser. They decide to use Docker to package the application with all its dependencies.
Based on the above data, answer the given subquestions.
The dashboard application crashes when deployed. Which Docker command shows the application's error messages and logs?
Show answer
Correct answer
Question 29
Scenario 3 : Deploying a Data Dashboard with Docker
Context
A team built a Python dashboard using Streamlit that visualizes sales data. They want to deploy it so anyone can access it via a web browser. They decide to use Docker to package the application with all its dependencies.
Based on the above data, answer the given subquestions.
What is the main advantage of using Docker to deploy the dashboard?
Docker makes the code run faster
Docker packages the application with all dependencies in a container, ensuring it runs consistently across different machines
Docker automatically writes code for you
Docker provides free hosting
Show answer
Correct answer
Docker packages the application with all dependencies in a container, ensuring it runs consistently across different machines
Question 30
Scenario 3 : Deploying a Data Dashboard with Docker
Context
A team built a Python dashboard using Streamlit that visualizes sales data. They want to deploy it so anyone can access it via a web browser. They decide to use Docker to package the application with all its dependencies.
Based on the above data, answer the given subquestions.
In the Dockerfile, which command specifies which Python packages should be installed in the container?
RUN pip install -r requirements.txt
COPY dashboard.py
EXPOSE 8501
FROM python:3.11
Show answer
Correct answer
RUN pip install -r requirements.txt
Question 31
Scenario 3 : Deploying a Data Dashboard with Docker
Context
A team built a Python dashboard using Streamlit that visualizes sales data. They want to deploy it so anyone can access it via a web browser. They decide to use Docker to package the application with all its dependencies.
Based on the above data, answer the given subquestions.
The Dockerfile includes the line EXPOSE 8501. What does this do?
Automatically makes the application accessible to anyone on the internet
Documents the listen port; you must still publish it using -p 8501:8501
Closes port 8501 for security
Changes the application code to use port 8501
Show answer
Correct answer
Documents the listen port; you must still publish it using -p 8501:8501
Question 32
Scenario 4 : OpenRefine for Data Cleaning
Context
A researcher has a dataset of company names from different sources. The same companies are spelled differently across sources, making analysis difficult.
Sample Messy Data (companies.csv):
Based on the above data, answer the given subquestions.
You want to standardize all company names to lowercase, then to title case (First Letter Capitalized). Which OpenRefine transformation approach is correct?
Manually retype each company name
Apply transformations value.toLowercase() then value.toTitlecase()
Delete the column and recreate it
Export to Excel and use find-replace
Show answer
Correct answer
Apply transformations value.toLowercase() then value.toTitlecase()
Question 33
Scenario 4 : OpenRefine for Data Cleaning
Context
A researcher has a dataset of company names from different sources. The same companies are spelled differently across sources, making analysis difficult.
Sample Messy Data (companies.csv):
Based on the above data, answer the given subquestions.
After standardizing names, you have multiple rows for the same company (multiple revenue entries). What should you do to combine them?
Delete all rows that appear to be duplicates
Use faceting to group names and then aggregate or sum the revenue values
Manually total the revenue values using an external calculator
Leave all duplicated company rows exactly as they currently are
Show answer
Correct answer
Use faceting to group names and then aggregate or sum the revenue values
Question 34
Scenario 4 : OpenRefine for Data Cleaning
Context
A researcher has a dataset of company names from different sources. The same companies are spelled differently across sources, making analysis difficult.
Sample Messy Data (companies.csv):
Based on the above data, answer the given subquestions.
OpenRefine saves your data cleaning steps as operations. What is the main benefit of this feature?
It automatically cleans new data
You can export the operations as JSON and replay them on similar datasets.
It makes the software run faster
It deletes your original data
Show answer
Correct answer
You can export the operations as JSON and replay them on similar datasets.
Question 35
Scenario 4 : OpenRefine for Data Cleaning
Context
A researcher has a dataset of company names from different sources. The same companies are spelled differently across sources, making analysis difficult.
Sample Messy Data (companies.csv):
Based on the above data, answer the given subquestions.
What type of data quality issue does this dataset have?
Missing values
Inconsistent naming and capitalization for the same entities
Wrong and unrelated data types
Too much data
Show answer
Correct answer
Inconsistent naming and capitalization for the same entities
Question 36
Scenario 4 : OpenRefine for Data Cleaning
Context
A researcher has a dataset of company names from different sources. The same companies are spelled differently across sources, making analysis difficult.
Sample Messy Data (companies.csv):
Based on the above data, answer the given subquestions.
In OpenRefine, which feature automatically groups similar text values like "Microsoft Corp" and "MICROSOFT CORPORATION" so you can merge them?
Filter
Sort
Text clustering
Delete rows
Show answer
Correct answer
Text clustering