Quiz Space

Introduction to Big Data · Quiz 2 · 2 Apr 2023 · January 2023 term · Set QPE2

Question 17: You are provided with a Spark program that picks out a l…

Question 17

+3 marksOne or more correct options

You are provided with a Spark program that picks out a list of suspicious transactions based on the amount of the transaction being higher than a threshold and the geographic location of the transaction matching a known set of suspected locations provided in a small file. The Spark program is written to pull all the transactions from the Workers to the Driver and then apply the logic of threshold filtering and geographic location matching. Which of the following changes will you make to the program to improve performance? (Choose any two)

Select all that apply.

  1. A

    Use broadcast variables

  2. B

    Hardcode threshold value as a filter condition in the Driver program

  3. C

    Filter for transactions in the Workers before bringing to the Driver

  4. D

    Hardcode threshold value as a filter condition in the Workers itself

Show answer

Correct answers

  • A

    Use broadcast variables

  • D

    Hardcode threshold value as a filter condition in the Workers itself

See the whole paperTake it as a mock test

Question 17 of 20 in the IIT Madras BS Introduction to Big Data (Intro to Big Data) Quiz 2 paper sat on 2 Apr 2023, in the January 2023 term (IIT M DEGREE AN3 EXAM QPE3 02 Apr 2023). It carries 3 marks.

This question was also asked in

More questions from this paper

  1. Q1What best describes "big data"?
  2. Q2Which of these represent examples of divide-and-conquer?
  3. Q3Which of the following statements about Spark application architecture is correct?
  4. Q4Consider the problem of sorting a 1 petabyte file of numbers stored in a Hadoop cluster of 10 machines where the size o…
  5. Q5A website sees 1 Billion hits every month. The website owner wants to count average hits per customer in the latest mon…
  6. Q6An enterprise software designer wants to leverage the best of cloud to minimize the number of administrative overheads …
  7. Q7Consider an application that can scale from handling 1000 users to handling 100 million users by simply making copies o…
  8. Q8Linux is an example of an operating system. Which of the following is considered as the "operating system of a cluster …
  9. Q9Figure question
  10. Q10Figure question
  11. Q11Figure question
  12. Q12Consider a file “data.bin” which is formatted as follows: every data record is in the form of pairs of values of the fo…
  13. Q13Consider the below mutually-exclusive characterisations about the program: i. Program will not run since RDD transforma…
  14. Q14What is the output of the following code? What is the output of the following code?
  15. Q15What option(s) best describe the differences between MapReduce and Spark?
  16. Q16Would IRCTC's railway ticket booking application be suitable for a serverless implementation?
  17. Q18Which of the following types of data sources can you read successfully without missing data using a program that extrac…
  18. Q19Which of the following is true?
  19. Q20Which of the following statements is/ are true for Google Cloud Functions?