Crunching through big data with MBrace, Azure and F#
For data exploration and rapid prototyping, the productivity of an interactive scripting environment is hard to beat: simply grab data, run code, and iterate based on immediate feedback. However, that story starts to break down when the data you have to process is big, or the computations expensive. Your local machine becomes the bottleneck, and you are left with a slow and unresponsive environment.
In this talk, we will introduce MBrace.net, an open-source and free engine for scalable cloud programming. Using the MBrace programming model, you can keep working in your beloved familiar scripting environment, and easily execute C# or F# code on a cluster of machines on Azure. We will focus primarily on live demos, from provisioning an Azure cluster with Brisk, to analyzing large datasets in a distributed fashion; in particular, we will discuss how this setup is relevant to data science and machine learning.