Introduction to Big Data, End Term
What are some capabilities common to both “streaming processing” & “batch processing” in the context of big data?
What are some capabilities common to both “streaming processing” \& “batch processing” in the context of big data? Which of the following are best practices associated with Streaming applications? You are given the task of improving the performance of a Spark SQL program. You suspect that the culprit is the main transformation job in the program. When you run EXPLAIN on that SQL, you see that Spark wrongly estimates that there are only 10 values for the key being aggregated, whereas in reality the underlying data has a million values for that key. What actions would you perform from the below to ensure that the right estimates are used?