What best describes "big data"?
Big data is used to refer to the set of technologies built around Hadoop
Big data refers to the cloud-native design principle
Big data is about All data, Any Time, Any Method
Big data is about volume, velocity, variety
Which of these are implementations of the divide-and-conquer paradigm?
Spark and Map
Serverless and Message Broker
Hadoop and Spark
MapReduce and Google Cloud Functions
A website sees 1 Billion hits every month. The website owner wants to count average hits per customer in the latest month, where a customer is denoted by the IP address of the device from which the customer is accessing the website. The owner has at his disposal a Hadoop cluster of 5 workers and 2 masters each with 1GB of RAM. Which of the following methods is the most likely to finish fastest?
Write a MapReduce program where the Map does nothing useful, Combine computes the aggregated hits per customer, Shuffle combined data based on IP address across workers, and in the Reduce, build hash table on each machine with hash key = IP address and hash value = counter, followed by another Reduce that finally computes the avg on top of all hash values.
Write a Spark program that forms a Dataframe as grouping by IP address with count as aggregate, followed by a take into a list in the Spark driver which further computes the average of all the individual counts in the list
Write a Spark program that forms a Dataframe as grouping by IP address with count as aggregate, followed by another stage that computes the avg on top of the Dataframe of the first stage
Write a MapReduce program where 2 pairs of Map Reduce are chained together: 1^(st) pair is where the Map does nothing useful, Shuffle data based on IP address across workers, and in the Reduce, build hash table on each machine with hash key = IP address and hash value = counter, while the 2^(nd) pair is another Map that does nothing useful followed by a Reduce that finally computes avg on top of all hash values.
All will finish in approximately the same time.
Sign in with Google — it is free — to see every question with its answer and explanation, practise it in learning mode, or take it as a timed mock test.
The IIT Madras BS Introduction to Big Data (Intro to Big Data) End Term paper sat on 28 Apr 2024, in the January 2024 term: 30 questions for 60 marks in 180 minutes. The first 3 questions are below. Sign in with Google — it is free — to see the whole paper with its answers and explanations, in learning mode or as a timed mock test.
| Feature | Intro to Big Data End Term 28 Apr 2024 at a glance |
|---|---|
| Term | January 2024 term |
| Subject | Introduction to Big Data |
| Course code | BSDA5001 |
| Questions | 30 |
| Marks | 60 |
| Duration | 180 min |
| MCQ | 22 |
| MSQ | 8 |
| Official paper | IIT M DEGREE FN EXAM QDB1 28 Apr 2024 |
| Negative marking | No negative marking. |
| Updated |
