Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.

Start free today

Treehouse discussion: submitting Spark jobs using Kubernetes scheduler on AKS

Play Treehouse discussion: submitting Spark jobs using Kubernetes scheduler on AKS
Sign in to queue

Description

Follow the discussion between Lena and Bernd and learn about a new Kubernetes scheduler option in the latest release of Spark 2.3.0. Lena gives Bernd a quick overview of Spark, and shows him a hands-on example of submitting a job to run on Kubernetes cluster in Azure Container Service (AKS). They go over the necessary steps to prepare for job submission, as well as details of what happens when the job is being created, running or completed. In addition to learning about technical aspects of the topic, enjoy the magnificent view of the pacific-northwestern nature and explore the Treehouses on Microsoft campus!

More information here:
https://aka.ms/spark-job
https://spark.apache.org/docs/latest/running-on-kubernetes.html

Follow Lena Hall on Twitter: @lenadroid
Follow Bernd Verst on Twitter: @BerndVerst

Create a Free Account (Azure)

Embed

Download

The Discussion

Add Your 2 Cents