Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.

Start free today

Siphon on HDInsight Kafka

Play Siphon on HDInsight Kafka
Sign in to queue


Thomas Alex joins Lara Rubbelke to discuss how Microsoft uses Apache Kafka for HDInsight to power Siphon, a data ingestion service for internal use. Apache Kafka for HDInsight is an enterprise-grade, open-source, streaming ingestion service. Microsoft created Siphon as a highly available and reliable service to ingest massive amounts of data for processing in near real time. Siphon handles ingestion of over a trillion events per day across multiple business-critical scenarios at Microsoft. In this episode, learn how Siphon uses Apache Kafka for HDInsight as its scalable pub/sub message queue.

For more information:





The Discussion

  • User profile image

    what were the deciding factors for Kafka on Hdinsight and not Event Hubs for this real time ingestion requirement

Add Your 2 Cents