Quiz Space

Introduction to Big Data · End Term · 13 Apr 2025 · January 2025 term · Set 1

Intro to Big Data End Term 13 Apr 2025 — Question 19

Question 19

+4 marksOne or more correct options

Select all that apply.

  1. A

    The given programs are an example of ETL and will be bottlenecked on thestring split happening on the Python VM.

  2. B

    The given programs are together an example of ELT that maximally utilizesthe Spark cluster and no further performance optimizations are required.

  3. C

    By moving the string split into PySpark driver program, I can get rid of thePython program and thus make the program an ELT.

  4. D

    Rewriting the string split to use native PySpark dataframe string functions willbe best suited for performance.

Show answer

Correct answers

  • A

    The given programs are an example of ETL and will be bottlenecked on thestring split happening on the Python VM.

  • D

    Rewriting the string split to use native PySpark dataframe string functions willbe best suited for performance.

See the whole paperTake it as a mock test

Question 19 of 20 in the IIT Madras BS Introduction to Big Data (Intro to Big Data) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M IMPROVEMENT FN EXAM QIM2 13 Apr). It carries 4 marks.

More questions from this paper

  1. Q1Which of the following is a false characterization of "hot potato” principle?
  2. Q2A website that started 3 years ago now sees 1 Billion hits every month. The website owner wants to count average hits p…
  3. Q3An enterprise software designer wants to leverage the best of Google cloud to minimize the number of administrative ove…
  4. Q4Consider an application that can scale from handling 1000 users to handling 100 million users by simply making copies o…
  5. Q5Consider a file “data.bin” which is formatted as follows: every data record has 10 key-value pairs of the format “key,v…
  6. Q6You join the data engineering team at a company that has good big data expertise already. Your first assignment is to c…
  7. Q7What happens when a Spark Structured Streaming pipeline operating with Kafka as source and console output as target is …
  8. Q8A big data streaming application that uses Kafka as source is observed to be really lagging behind currently live data.…
  9. Q9A company with headquarters (HQ) in the Middle East operates on a Sunday-Thursday weekday schedule with Friday \& Satur…
  10. Q10You are appointed as a Data Engineer in a company that has a legacy reporting application written in Java which suffers…
  11. Q11Fielder on the midwicket boundary is wearing a smart watch. Unlike other smart watches, this one is unique in that it h…
  12. Q12Figure question
  13. Q13You are given a Spark Streaming pipeline that invokes a pre-trained DL model for every image it receives as input and p…
  14. Q14Consider a Structured Streaming application running on Google Dataproc firing up every 10 seconds, consuming any number…
  15. Q15Let us say we are using structured streaming for continuously reading data from Kafka and storing the results back into…
  16. Q16Which one of these is not an implementation of the divide-and-conquer data processing paradigm?
  17. Q17What option(s) best describe the differences between MapReduce and Spark?
  18. Q18You are provided with a Spark program that picks out a list of suspicious transactions. Its logic is based on both the …
  19. Q20In a manufacturing facility, the supervisor is interested in improving efficiency of the assembly line using sensor dat…