DevCamp 2010 Keynote - Rx: Curing your asynchronous programming blues

Play DevCamp 2010 Keynote - Rx: Curing your asynchronous programming blues
Sign in to queue


Speaker: Bart de Smet

Aufzeichnung vom DEVcamp 2010 in Wien, am 12. Oktober 2010

Organisiert von den Partnern TechTalkCubido und CSS.
Microsoft Österreich unterstützte das Event als Hauptsponsor und hat auch selbst einige Sessions abgehalten.

At Microsofts Professional Developers Conference (PDC) a few years back, we introduced the LINQ project to solve the impedance mismatch between various data models by means of integrated query syntax in mainstream programming languages. Today, we’re seeing a rich ecosystem around LINQ providers that allow developers to reach out to many more data sources. However, there’s a lot of opportunity left to democratize even more data models. Based on the theory of monads, we’ll explore the incredibly powerful nature of query comprehensions to build reactive queries.
Asynchronous, event-driven "reactive" programming is way too hard in today's world of development tools and frameworks. The huge amount of manual and error-prone plumbing leads to incomprehensible and hard to maintain code. As we reach out to services in the cloud, the desire for asynchronous computation is ever increasing, requiring a fresh look on the problems imposed by reactive programming. Centered around the concept of observable data sources, Rx provides a framework that takes care of the hard parts of reactive programming. Instead of focusing on the hard parts, you now can start dreaming about the endless possibilities of composing queries over asynchronous data sources, piggybacking on convenient LINQ syntax.

In this session, we'll cover the design philosophy of Rx, rooted on the deep duality between the interactive IEnumerable interface and the new reactive IObservable interface in .NET 4.

From this core understanding, we'll start looking at various combinators and operators defined over observable collections, as provided by Rx, driving concepts home by a bunch of samples. Finally, if time permits, we'll look at the Reactive Extensions for JavaScript which allows us to take the concepts we already know from Rx and apply them to JavaScript and have deep integration with libraries such as jQuery.  Democratizing asynchronous programming starts today. Don't miss out on it!




Download this episode

The Discussion

  • User profile image
    Eddie Korbs

    One of the best demos I have seen on channel 9 lately. Great explanation and demos...inspiring!!!

  • User profile image

    On the side note... maybe you could buy yourself a small remote page switcher... they are really cheap these days.
    All in all it was really joy to watch... waiting for your next presentation.

  • User profile image

    GREAT presentation ! outstanding Big Smile

  • User profile image

    Great Presentation. I learnt something.

    Though, it would be better not need to use Reflection, and have it 

    So you could write  TextBox. and have Intellisense list some of the possible events.

    In my opinion having encode the event name in a string, could lead to problems.

    Suppose I mistype the name, I don't get a compile-time warning but a runtime exception.

  • User profile image

    If you haven't seen Bart's LINQ to Everything session from PDC10, you should. Another excellent presentation.


  • User profile image

    @AdamSpeight2008: I've actually mentioned during my talk (as far as I remember) that we have an overload that allows you to do more or less what you're describing, by passing in add and remove handler lambda expressions (which will be using the .NET event itself using +- and -= operations, respectively).

    Since I like to do all demos live and always avoid to import code snippets on stage (sure, I could build the entire Windows kernel live on stage by dragging and dropping a code snipet of a few million lines of code, that would be immediately obvious to the audience Smiley), I went for the shorter one. Following this approach of live coding allows the brainwaves of the audience to be in sync with mine while producing the code.

    That said, this overload is there as the closest one to being able to pass in an event. However, .NET events are merely metadata citizens and are not first-class constructs: in other words, you can't grab them nor pass them around (as opposed to methods which you can grab by means of delegate types).

    Hope this helps and thanks for watching,

  • User profile image

    @Waldemar: Thanks for the feedback. In fact I have two of those clickers (<AdWarning>Microsoft Wireless Notebook Presenter Mouse 8000</AdWarning>), which I either forget to bring or forget to switch off such that the battery dies prematurely.

    So yes, this was a little bit of jumping around on stage, which is good for my "calories burned count" anyway. Luckily some conference venues provide this kind of equipment Smiley.

  • User profile image

    @bdesmet: it looks like WPF's RoutedEvent system works better here, it has a static XxxEvent property to reference to the event

  • User profile image

    @felix9: Slightly better for sure, because there's something you can grab. Passing it around though is still cumbersome: you'd need to pass the static event object, as well as the instance to which it applies. All the remainder points made in my .NET events comparison slide hold here too: their "events as data source" characteristic is not apparent, composition lacks, etc. What you got though is a slightly nicer FromEvent bridge.

    In fact, WPF events are not strictly more first class than .NET events, because you still have to pass two pieces of information around: the event object and the instance to which it applies. That's different with delegates, which can capture both the target method and the instance to go with it (remember all the function pointer "fun" in native C/C++ land to achieve a similar effect?).

    Having said all of this, again: we're not replacing existing event sources or sources of asynchrony. Instead, we can build bridges and expose those as observable sequences. Building this particular bridge is incredibly straightforward, too.


    PS: In a former life, I was a developer on WPF, so yes, I'm very well-aware of what that particular platform got too Wink.

  • User profile image

    Thanks Charles for the link to LINQ to Everything. A great presentation. The bit in the end where Bart integrates the optimizer from solver foundation with LINQ is fantastic, and then he creates a time varying solution to the optimization problem by just putting two from's in front that get stock data observables. Wow!

  • User profile image

    All that fictional syntax, Bart. Maybe you guys have a new PL in mind too? Wink

    Funniest quote of the talk: "Nobody thinks of a mouse as a database. They look at you and they lock you up somewhere." Big Smile

  • User profile image

    hi Bart, nice presentation. you have the source code up somewhere?

  • User profile image

    @kevin: You can find the sample as a Hands-On Lab through our forum at See section "Announcements" near the top for the link to the latest HOL documents.

    Hope this helps,

  • User profile image

    Great Presentation! Best Rx material I've seen yet. Now I can make sense of the Rx Design Guidelines.pdf.

    Since async/await is in ctp, I am using a combination Task.FromAsync() and Unwrap() to handle composition of async operations. Now, Rx looks to me like good tool for observing a "stream of Tasks".

    Could you elaborate if and how Rx and Task<T> fit together, especially with the Async Ctp? On the first look, there seem to be similarities between the two.


  • User profile image

    There are indeed similarities between the two, but enough differences to warrant their coexistence. Let's talk about this now.

    First of all, from an interface point of view, IObservable<T> and Task<T> share the same mode of data acquisition, with data being pushed at you through a continuation (cf. IObserver<T> and the Task<T> ContinueWith method). One difference lies in the arity of the data. While IObservable<T> can describe the acquisition of multiple values of type T (cf. OnNext), a Task<T> can only signal a single value. So, IObservable<T> is more general than Task<T> from a one-versus-many data-centric point of view. In other words, IObservable<T> is great to describe push-based streams of data (such as for event processing) while Task<T> covers the single-valued case. No surprise we have conversions from Task<T> into IObservable<T> in the Rx libraries. You could do the other way around as well, for example collecting an IObservable<T>'s data into an IList<T> which is wrapped in a Task<T>. Here you're loosing the timing information of inidividually acquired values though (read on).

    Now on to the Async CTP. What the new async features in C# and VB are all about is sequential composition for asynchronous programming. It's all about writing code that runs in response to an asynchronous task's (task used in the broadest sense here, not just Task<T>) sequentially with respect to the code that triggered the operation. In reality, the compiler turns the sequential-looking code into a continuation on the asynchronous task:

    var res = await op();

    This really turns into something along the following lines (in reality much more complex using a state machine much alike iterators, and with a check for BeginAwait's Boolean return value):

    var aw = op().GetAwaiter();
    aw.BeginAwait(() => {
        var res = aw.EndWait();

    The implementation of GetAwaiter depends on the return type of the "op" call above. In case it's a Task<T>, the awaiter represents an entry-point to establish a ContinueWith operation passed to the BeginAwait method. However, the compiler is by no means tied to any particular implementation of the "await pattern" and will choose whatever is available through regular method resolution (instance methods first, then extension methods, all regular rules apply). In particular, the Rx library implements the await pattern for IObservable<T>:

    var res = await Observable.Range(0, 10);
    Console.WriteLine(string.Join(", ", res));

    The inferred type for res will be IList<int>, collecting all the values from the IObservable<int> sequence (here the range with values 0 through 9) that were signaled before OnCompleted. If an error is signaled through OnError, the EndAwait method call will surface it. Notice the use of an IList<T> for the await on IObservable<T> because await is about single values, while IObservable<T> is about sequences or streams of data.

    To conclude, let's give an other observation (pun intended) here. Rx and async/await are totally complementary. I think we all agree that both technologies deal with asynchronous programming. One difference is in the arity of data, another is in the way you perform composition. We've seen exactly this coexistence situation for synchronous programming as well. Here are the two language features I'm talking about:

    • ; - No, not a typo: semi-colon. Semi-colon is the way to sequence synchronous code operations in imperative style languages like C#. It's also your way to flow data out of an operation, into a variable and propagate it to the next operation.
    • Query expressions, aka LINQ - This allows for composition over sequences of data by chaining (sequencing) together query operators through a fluent method invocation pattern. This is not your typically control flow, but totally complementary to the above.

    Today we're seeing this repeated for asynchronous programming. For the first bullet point, we now have the await keyword, which allows code-centric sequencing of asynchronous operators. For the second bullet point, LINQ can be used to operate on sequences of data (using a pipelining mechanism for sequencing) such as IObservable<T> collections (or IAsyncEnumerable<T>, search C9 for more information on that one's role). Both can nicely coexist.

    Hope this helps!

  • User profile image

    Thanks for the long explanation!

    Glad to see they are complementary, considering the past ways for async operations I was kind of worried.

    I wouldn't have seen ; : linq = Task : Rx if it poked me in the eye - but it makes perfectly sense!

    Thanks again!

  • User profile image

    This video was so clear and inspiring that our team took Rx into real world use. We have used it a few weeks now, and it have already a) made a few complex source code classes (with async events) a lot simpler, and b) solved some really nasty thread safety issues. So, thanks Bart and the Rx team!


  • User profile image

    I have also some problems:

    • I can't really debug Unit tests when using Silverlight Reactive Extensions (Silverlight Rx library with .NET core test cases).
    • One case when calling .Subscribe() I get an exception before onNext evaluation: "Collection was modified; enumeration operation may not execute." But I don't know which collection! How can this be possible, I thought that Observables are like "history of events", and you can't modify history... Because this is before the first onNext, I don't find a way to debug. Sad

    And questions:

    • Ain't there pdb-files to debug the Rx core...?
    • Should I dispose manually? When?
    • How will Rx work with Silverlight two-way data binding?


  • User profile image

    @Thorium: Thanks for your interest. Please use the Rx forums for questions:

  • User profile image
    Mike Pelton

    Truly excellent - clear, rich in information, well paced, and inspiring. Many thanks!

  • User profile image

    Excellent work.. It takes great presentation skill to do presentation on a complicated topic in such an easy to understand way.. Bravo

  • User profile image
    Kevin Burton

    Is there an update for the demos presented in this presentation. It seems that the current version of Rx doesn't support many of the methods in this presentation.

  • User profile image
    Zubin Appoo

    Excellent presentation - we are using RX extensions in our solution and this video really helped me have a clear understanding of the benefits and to explain it to others.

  • User profile image

    Got job with your presentation. It's clear and simple to understand. The only problem is that now that the API has changed some methods you presented are not available anymore.

  • User profile image

    Wow, fantastic presentation although I would recommend a MS presenter mouse (PPT buttons on the bottom)  so you won't have to run back and forth like a mongoose on methamphetamines Smiley but I must say having done maaaannnnyyyy presentations myself this one was a 10. Thank you so much not only for RX but also for your energy and clarity of presentation. I am doing extensive Lync/UCMA work and the entire API is async method pairs for this RX is a godsend.

  • User profile image

    One of the best presentation of technology without wasting any time and to the point.

    Thanks lot

Add Your 2 Cents