Quiz Space

Introduction to Big Data · End Term · 22 Dec 2024 · September 2024 term · Set QDB4

Question 7: You are given a Spark Streaming pipeline that invokes a p…

Question 7

+2 marksOne correct option

You are given a Spark Streaming pipeline that invokes a pre-trained DL model for every image it receives as input and produces the classification result in quick time. The model with the best recall rate from the PyTorch library takes 3.1 seconds on an average to execute on a Quad CPU Spark worker machine. However, the output is expected to be produced consistently within 3 seconds from having received the input. What is the best option to meet the expectations without compromising on false negatives and minimizing the amount of money & effort that is spent further?

  1. A

    Use a different DL model that is faster but has half the recall rate.

  2. B

    Build a custom model that compresses the highest recall rate model just enough to be able to execute within the stipulated time, and measure recall.

  3. C

    Parallelize the DL model code using Divide-and-Conquer so that it runs faster.

  4. D

    Change Spark machine to use GPUs

Show answer

Correct answer

  • B

    Build a custom model that compresses the highest recall rate model just enough to be able to execute within the stipulated time, and measure recall.

See the whole paperTake it as a mock test

Question 7 of 21 in the IIT Madras BS Introduction to Big Data (Intro to Big Data) End Term paper sat on 22 Dec 2024, in the September 2024 term (IIT M DEGREE AN EXAM QDB4 22 Dec 2024). It carries 2 marks.

More questions from this paper

  1. Q1What happens when a Spark Structured Streaming pipeline operating with Kafka as the source is subject to a machine fail…
  2. Q2A company with headquarters (HQ) in the Middle East operates on a Sunday-Thursday weekday schedule with Friday \& Satur…
  3. Q3In the class, we saw the UDF for mobilenet_v2. Specifically, the predict() function contained the below lines of code
  4. Q4Dhoni is on the crease with a bat in hand that has sensors embedded throughout. The sensors talk to the spider cam ever…
  5. Q5In the class, we saw the UDF for mobilenet_v2. By definition, UDFs are scalar. In Spark, there is another class of user…
  6. Q6You are appointed as a Data Engineer in a company that has a legacy reporting application written in Java which suffers…
  7. Q8Which of the following code snippets will give a runtime error? (Note: df is a spark dataframe. It has a column called …
  8. Q9Since it is the onset of summer, there is a surge in railway ticket bookings. The business head at IRCTC is interested …
  9. Q10Consider a Structured Streaming application running on Google Dataproc firing up every 10 seconds, consuming any number…
  10. Q11Consider a Kafka system that has two brokers with the exact same specifications. Let’s consider a topic A with a single…
  11. Q12Let us say we are using structured streaming for continuously reading data from Kafka and storing the results back into…
  12. Q13Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications. Go…
  13. Q14A big data streaming application that reads from using Kafka as source is observed to be really slow. The Kafka cluster…
  14. Q15You are given the task of improving the performance of a Spark SQL program. You suspect that the culprit is the main tr…
  15. Q16What differentiates “streaming processing” from “batch processing” in the context of big data?
  16. Q17What are the core components for a Big Data Streaming application?
  17. Q18Which of the following statements are true?
  18. Q19Observe the below image and select the options that are true.
  19. Q20A Spark Streaming application is configured to execute once per minute. However, each run takes 10+ minutes consistentl…
  20. Q21You are given a Spark program that runs on a Google Dataproc cluster on a daily schedule from 1AM-12PM to produce as ou…