Quiz Space

Introduction to Big Data · End Term · 13 Apr 2025 · January 2025 term · Set 1-6

Question 3: You are asked to build a recommendation engine for an off…

Question 3

+2 marksOne correct option

You are asked to build a recommendation engine for an offline store at the checkout counter, using a combination of cloud and point-of-sale (PoS) terminal resources. Consider the following pipeline choices for effecting the same outcome, where Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications, Google Datastore and HBase are both highly-scalable NoSQL database systems for interactive, real-time applications, and Google Vertex AI is a hosted platform that lets you train and deploy ML models
(i) Recommender model training on Python VM on GCP; Data publisher on VM in PoS → Kafka VM on GCP → Spark Streaming on Hadoop VMs on GCP -> Recommender hosted on Python VM on GCP → Relay recommendation to PoS (ii) Recommender model training on Vertex AI; Data publisher on VM in PoS → Kafka VM on GCP → Dataflow -> Recommender hosted on Vertex AI → Relay recommendation to PoS (iii) Recommender model training on Python VM on GCP; Data publisher on VM in PoS → Pub/Sub → Spark Streaming on Google Dataproc → Recommender hosted on Python VM on GCP -> Relay recommendation to PoS (iv) Recommender model training on Python VM on GCP; Data publisher on VM in PoS → Pub/Sub → Dataflow → Recommender hosted on Python VM on GCP → Relay recommendation to PoS (v) Recommender model training on Python VM on GCP; Data publisher on VM in PoS → Pub/Sub → Spark Streaming on Hadoop VMs on GCP → Recommender hosted on Python VM on GCP → Relay recommendation to PoS
Which option below represents the correct order of pipeline options that has the “most IaaS” entry to the left and the “most PaaS” entry to the right?

  1. A

    (i), (v), (iii), (iv), (ii)

  2. B

    (i), (ii), (iii), (iv), (v)

  3. C

    (ii), (iii), (i), (iv), (v)

  4. D

    (v), (iii), (i), (iv), (ii)

  5. E

    All are equally PaaS / IaaS

Show answer

Correct answer

  • A

    (i), (v), (iii), (iv), (ii)

Question 3 of 19 in the IIT Madras BS Introduction to Big Data (Intro to Big Data) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M IMPROVEMENT FN EXAM QIM2 13 Apr). It carries 2 marks.

This question was also asked in

More questions from this paper

  1. Q1At the onset of every festive season, there is a surge in quick commerce orders (i.e. orders delivered within 15 minute…
  2. Q2You are given a Spark program that runs on a Google Dataproc cluster on a daily schedule starting execution at 7AM runn…
  3. Q4Consider a file “data.bin” which is formatted as follows: every data record has 10 key-value pairs of the format “key,v…
  4. Q5You join the data engineering team at a company that has good big data expertise already. Your first assignment is to c…
  5. Q6What happens when a Spark Structured Streaming pipeline operating with Kafka as source and console output as target is …
  6. Q7A big data streaming application that uses Kafka as source is observed to be really lagging behind currently live data.…
  7. Q8A company with headquarters (HQ) in the Middle East operates on a Sunday-Thursday weekday schedule with Friday \& Satur…
  8. Q9You are appointed as a Data Engineer in a company that has a legacy reporting application written in Java which suffers…
  9. Q10Fielder on the midwicket boundary is wearing a smart watch. Unlike other smart watches, this one is unique in that it h…
  10. Q11Suppose there’s a new transformer model from Facebook that is orders of magnitude smaller than mobilenet_v2 and has a b…
  11. Q12You are given a Spark Streaming pipeline that invokes a pre-trained DL model for every image it receives as input and p…
  12. Q13Consider a Structured Streaming application running on Google Dataproc firing up every 10 seconds, consuming any number…
  13. Q14Let us say we are using structured streaming for continuously reading data from Kafka and storing the results back into…
  14. Q15Observe the below image showing a possible implementation of Kafka or Google Pub/Sub and select the options that are tr…
  15. Q16You are given the task of improving the performance of a Spark SQL program that is doing a simple count after a series …
  16. Q17Which of the following are best practices associated with Streaming applications?
  17. Q18What are some capabilities common to both “streaming processing” \& “batch processing” when using Spark for big data?
  18. Q19Which of the following is true?