Spark Performance Tuning - Part 1

Play Spark Performance Tuning - Part 1
Sign in to queue


This week's Data Exposed show welcomes back Maxim Lukiyanov to kick off a 4-part series on Spark performance tuning with Spark 2.x. Maxim is a Senior PM on the big data HDInsight team and is in the studio today to present Part 1 of his 4-part series.

[03:00] - Spark Use Cases as it pertains to performance.

[06:00] - Performance Demos and why performance matters. Query graphs and query performance depending on query optimization.

[11:00] - Spark 2.x project Tungsten overview, which includes: [13:45] Catalyst query optimizer,  [15:20] Direct Memory management, [18:10] WholeStage Code generation.




The Discussion

Add Your 2 Cents