Introducing Apache Kafka on Azure HDInsight

The Discussion

• 

  Interesting, with HIPAA compliance, could Hospitals & Primary Care Physicians import data and run analytics on patient data?

• 

  I was waiting for this! What type of subscription is required? Tried a BizSpark and a "Pay as you go" to no avail :(

  See message: Please select a subscription that has access to HDInsight Kafka

   

• 

  Can i get the code how to fetch the data from twitter to Kafka + Storm

• 

  @Rohit:

  To replicate the above demo, you need a setup like the following

  TwitterProducer --> Kafka <-- Storm Consumer

  For a TwitterProducer, you can use a simple spark notebook as detailed below
  (doc for the setup)

  https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-kafka-spark-structured-streaming

  (notebook for the actual producer)
  https://github.com/Azure-Samples/hdinsight-spark-kafka-structured-streaming

  Once you have this working, you can use Storm to consume the messages from Kafka as shown here
  https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-storm-with-kafka

  Let us know if you have more questions.