Stream processing with R in AWS

Play Stream processing with R in AWS
Sign in to queue


useR!2017: Stream processing with R in AWS

Keywords: stream processing, big data, ETL, scale
R is rarely mentioned among the big data tools, although it's fairly well scalable for most data science problems and ETL tasks. This talk presents an open-source R package to interact with Amazon Kinesis via the MultiLangDaemon bundled with the Amazon KCL to start multiple R sessions on a machine or cluster of nodes to process data from theoretically any number of Kinesis shards.
Besides the technical background and a quick introduction on how Kinesis works, this talk will feature some stream processing use-cases at, and will also provide an overview and hands-on demos on the related data infrastructure built on the top of Docker, Amazon ECS, ECR, KMS, Redshift and a bunch of third-party APIs – besides the related open-source R packages, eg AWR, AWR.KMS and AWR.Kinesis, developed at CARD.





Download this episode

The Discussion

Add Your 2 Cents