Stream processing with R in AWS

Play Stream processing with R in AWS
Sign in to queue

Description

useR!2017: Stream processing with R in AWS

Keywords: stream processing, big data, ETL, scale
Webpages: https://CRAN.R-project.org/package=AWR, https://CRAN.R-project.org/package=AWR.KMS, https://CRAN.R-project.org/package=AWR.Kinesis
R is rarely mentioned among the big data tools, although it's fairly well scalable for most data science problems and ETL tasks. This talk presents an open-source R package to interact with Amazon Kinesis via the MultiLangDaemon bundled with the Amazon KCL to start multiple R sessions on a machine or cluster of nodes to process data from theoretically any number of Kinesis shards.
Besides the technical background and a quick introduction on how Kinesis works, this talk will feature some stream processing use-cases at CARD.com, and will also provide an overview and hands-on demos on the related data infrastructure built on the top of Docker, Amazon ECS, ECR, KMS, Redshift and a bunch of third-party APIs – besides the related open-source R packages, eg AWR, AWR.KMS and AWR.Kinesis, developed at CARD.
References

Day:

1

Embed

Download

Download this episode

The Discussion

Add Your 2 Cents