Big, Fast, and Data-Furious…with Spark

Making teams of data scientists productive is a challenging task. The size of the data in Big Data problems is the first great hindrance to productivity. Apache Spark provides a foundation for the solution to this problem by offering interactive compute engine, but it is not sufficient in itself. In this session we review how a set of open source tools including Jupyter and Livy can be combined with advanced resource management and elasticity of Azure cloud to provide a comprehensive interactive platform for Big Data.