Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

E2E: Erik Meijer and Wes Dyer - Reactive Framework (Rx) Under the Hood 2 of 2

Download

Right click “Save as…”

  • High Quality WMV (PC)
  • MP3 (Audio only)
  • MP4 (iPhone, Android)
  • Mid Quality WMV (Lo-band, Mobile)
  • WMV (WMV Video)
Software Developer extraordinaire and language compiler geek Wes Dyer and programming language design guru and LINQ co-creator Erik Meijer dig into the Reactive Framework (Rx). This is part 2 of 2. See part 1 here.

Here, Erik and Wes continue their discussion on the core ideas behind Rx. Rx is deep (as in profound), as you must have gathered by now. Erik, of course, continues to keep the theoretical basis of all this squarely front and center so we understand the relationship between principles and practice.

Enjoy!

Tags:

Follow the Discussion

  • Bent Rasmussenexoteric stuck in a loop, for a while

    Still watching but I must say already that the IO<IO<T>> has something extremely satisfying about it. Very simple (well, superficially so).

  • Let's be honest, wouldn't you just love to be in the room during one of these white board sessions! I know envys not a good thing per say - but you folks who get to work along side Erik et al... Wow.

  • CharlesCharles Welcome Change

    Agreed. But, then again, by releasing (and filming these in the first place) this is exactly what we are doing (bringing into the room). Think "fly on a wall". Some day, we will transport you right into the room as it is unfolding in real time. Some day (not science fiction...).

     

    C

  • William Staceystaceyw Before C# there was darkness...

    Thanks much C, Erik, and Wes.  Another great 2 part.  Naturally, I am thinking away about how you might bring this into ADO.Net Data Services (et al).  Say I wanted to get notified each time someone does an update or delete on specific condition, etc.  I can see how you may do this on the service end inside a method, but am wondering about the comm channel.  Because of firewalls and the http request/reply pattern, you still end up with essentially a client-side pull model instead of a pure push model.  Maybe that is just something we have to live with in a distributed environment.  That said, any ideas on this subject?  You can abstract the server side Push by having some kind of timer on the client side that pulls stuff down from a server's observable queue.   However, things like background timer tasks and what is the correct amount of time between pulls, message ordering starts to get messy.  Any new ideas in the network side of this Beauty? It is really cool to find the ONE.  Because now you can standardize on this for all async Push - Sql Notifications based on Linq queries, Ado.Net Data Services notifications, all .Net services, Windows Services, devices, .Net MicroFramework, .Net POS, etc.    Great work. 

  • William Staceystaceyw Before C# there was darkness...

    Second question.  Do you think there will be a way to remote the observer query to the server, so the server processes any Where clause to reduce unneeded traffic on the wire?  Example, send me X where Y <= 2 || Y >1000

  • Well I think there would be some restrictions (on types that can be used and method calls), that could be passed to the server.  But look at what ..net ria services, what it does is it takes the IQueryable expression tree and serializes it directly and sends it off to the server.  IQueryable is just expression trees that havnt been compiled into a function that returns an IEnumerable yet.  I am sure they could have a simlar thing in IObservable that would have interfaces like  IRequestable Select<source,result>(this IRequestable , Expression<Func<source,result>> ).  I dont know quite what they would call it though.  IQueryable unforchantly derives from IEnumerable, but otherwise it would be identical in its signatures.  IQueryable assumes that something is going to serialize it to pass it to something else for proccessing.  In the case of observables you would be assuming something is going to serialize it to do the proccessing (filtering or other proccessing) remotely.  Maybe something like you can pass it up to the server and have it tell you when some specific row of a database table changes, or off to active directory to let you know when some user gets locked out.  All the server does is provide you with access to everything you have access to as one giant IObservable, and then the client can select (and filter) the type of events it wishes to listen to, and only those are sent down.

     

    Also intresting to think about this in terms of a UI framework.  Where the framework itself has one giant IObserable which sends a new value every time the mouses moves or keyboard is pressed, and it hands off to all its children .Where(...) verions of the giant IObserable that filter for just its selected area of the screen or currently selected child (which then wraps its children in another .Where).  In the end you dont need to even test do any hitbox testing or anything, everything just flows down stream.  I spose most UI frameworks like WPF already do this with thier visual tree's just invoking methods on a child of a known type (like UIElement), but when you look at the 100's of On... overrideable method calls, you got to wonder if filtering only what you need might be a better option.  Where they dont have to keep extending the base class to add additional method calls, and instead add new derived event types that return more specific infromation if you want that extra info.  Or say your a team like Microsoft Surface that has some specific type of "new" interaction events you want to pass to your children based on specific proprietary hardware.  In the current WPF, this is very hard/impossible (I think they derived thier own UIElement type and force all thier children had to derive from that to get the new events).

  • Bent Rasmussenexoteric stuck in a loop, for a while

    Charles, do you think it possible to get perhaps a type proliferation overview, maybe in interview format, talking about where you plan to retrofit these types as well as possibly in what order. I imagine this will be a big undertaking as it basically amounts to every single instance of where you have an event today.

     

    I wonder if it wouldn't make sense to systematically and automatically retrofit all of .Net with these types, that is, using some naming convention, simply inject one IObservable property for every event. If I remember correctly, you're just using an event and holding a subscriber delegate to the event and on dispose you just remove the same delegate. So it should be a mechanical process to retrofit. This way we can also have this retrofitted to good old WF.

     

    Also, it would seem like this would have some interesting deeper implications for concurrency and coordination if we have parallel events going on and we coordinate them with these query combinators, hmm...

  • My guess unfortunately is they aren’t going to retrofit anything, at least initially.  We would be lucky to get the IObservable interface in the BCL (and not just as an extension), although this might happen in .NET 4.0.  If they actually went converted events to be IObservable’s they would break everyones code that depended on those events.  Even if they rewrote the compiler somehow so the event keyword now MEANS IObservable, a lot of code would break (think of the IL changes on a different language).  Ah well, the good thing is that at least all events can easily be converted into IObservable’s.

  • Bent Rasmussenexoteric stuck in a loop, for a while

    No no no, you misread - or I used the word retrofit wrongly. The idea is to *add* an IObservable property to all classes where there is already an event. This would bring the meaning of Reactive Extensions to a whole new level, meaning Reactive Extensions *everywhere*.

     

    Charles: Maybe you should just rename the posts to Reactive Extensions for .Net: ...... to defuse confusion. Or I should just get to work now heh...

  • Allan LindqvistaL_ Kinect ftw

    Rx is really really cool Smiley it will have as big an impact as linq-to-objects (or linq to enumerables really) had..

     

    what is crucial though is interop with exsisting stuff.. tasks for example, or the event based calls in silverlight (not a big fan of those) or begin/end invoke, or events..

     

    there are stories for most of them but some are better than others, converting an event to an iobservable is somewhat clunky and involves magic strings (although there is type safe way of doing it to)

     

    i think what we'll see is a bunch of extension method libraries for getting iobservable versions of various events.. its to bad they didnt have this stuff when wpf was created Smiley

  • CharlesCharles Welcome Change

    I like the sound of that Smiley

     

    To be clear, "Reactive Extensions" is not officially official, but is the direction the team + marketing is going. I will leave the title as is for historical reasons Smiley You will learn more about this technology as it grows. At some point, an official anouncement of some kind will emerge and a new name will be born.

     

    C

  • Bent Rasmussenexoteric stuck in a loop, for a while

    On the other hand, if syntax for creating observables from events is sufficiently easy, this may be overkill, hmm...

  • stevo_stevo_ Human after all

    Maybe I'm confused about your points regarding events, but the reactive framework adds extensions to wrap events, events are reactive anyway, the reason you would want 'iobservable' events is for the composition advantage they give.

     

    edit: the points regarding let at the end were interesting, but perhaps not really discussed properly, ie.. no practical example.

     

    Great video two parter tho, good ending Tongue Out.

  • You guys never got to Until (mentioned in part 1). Any follow up details?

     

    I also wish you'd talk more about Let. How is that implemented? Does it give you lazy computation? E.g. Haskell lets your write:

     

      zero = let x = x in 0

     

    And you won't get an infinite loop. Does Let allow somethign similar?

     

    Good stuff!

  • Bent Rasmussenexoteric stuck in a loop, for a while

    Not sure who you are addressing here but if me, then the point was to make the use of Rx have as low a syntactic overhead as possible. This is an example use of Rx I found,

     

    IObservable<Event<MouseEventArgs>> mouseMoveEventObservable = Observable.FromEvent<MouseEventArgs>(myControl, “MouseMove”); mouseMoveEventObservable.Subscribe(...);

     

    So looking at this it's not the sweetest syntax you could arrive at, but here the lacking use of type-inference is a culpit as well, so we make it a bit sweeter

     

    var mouseMoveEventObservable = Observable.FromEvent<MouseEventArgs>(myControl, “MouseMove”); mouseMoveEventObservable.Subscribe(...);

     

    Then finally the solution we would have wanted from the outset

     

    myControl.MouseMove.Subscribe(...);

     

    Clearly there is some syntactic overhead which will repreat itself in each and every program. Whether one actually cares about something like this is a different matter.

  • This looks very good.

     

    I have a question: you mention that SelectMany "merges" event streams, and you have to use until make sure that you stop listening when a new event comes in (in the dictionary suggest example). I cannot think of an example where you'd NOT want to use until. So why isn't this the default behavior for SelectMany? Can you give an example where you'd want to have SelectMany to merge the event streams instead of switching to the most current one?

  • Let is more like a local variable declaration in C# to force effects to happen once.

     

    If you do not want the side-effect of reading from the console twice:

     

    Console.WriteLine(Console.ReadLine() + Console.ReadLine());

     

    you do it once by assigning the result to a local variable

     


             var s = Console.ReadLine();

     Console.WriteLine(x+x);

            }

     

    In Haskell terms Let  is actually very close to bind :: IO a -> (a -> IO b) -> IO b in the IO monad; but in C# the IO monad is the ambient monad so you do not see it in the type.

  • What you notice is that there is some boring boilerplate needed to make .NET events first class values. However, that is not something that you would expect consumers of types to write, instead, the producer of a type with events writes (or auto-generates) extension methods that do the conversions once.

     

    Of course it would be nice if there would be direct language support for this, but in general when you can do something "good enough" using a library, I am not sure that you want to increase the feature footprint of the language even more.

  • You are correct that in most cases you use Until. However, imagine you are writing an application that collects payments. When you send out a billing statement, the payment comes back asynchronously. However, you do not want to drop the previous payment on the floor when you send out the next billing statement.

     

    In the "marble diagram" below, you want to collect both P1 and P0 that result from B0 and B1, even though they came back out of order:

     

     

    --B0------------B1------------------------->

                 |                       |

                 --------------------------P0.

                                         |                     |

                                         -----P1.        |

                                                    |          |

           ------------------------P1----P0------------->

  • The reason we came up with Rx was exactly to deal with distributed computations.

     

    The reasons why observable streams work so well in this context (and answers your question affermatively) is eloquently explained in this paper http://www.irisa.fr/asap/intranet/the-many-faces-of-publish-subscribe.pdf/

     

     

  • Bent Rasmussenexoteric stuck in a loop, for a while

    I see this example on the "unfold" blog that is probably the approach you recommend - and it looks quite forgivable

     

    public static IObservable<Event<MouseButtonEventArgs>> GetMouseLeftButtonDown(this UIElement that) { return Observable.FromEvent<MouseButtonEventArgs>(that, "MouseLeftButtonDown"); } 

     

    The compositional nature of these things ("IE" & "IO") is very inspiring. Looking forward to the beta 2 drop, hopefully with Rx.

     

    It'll also be interesting to see the extension method proliferation in support of these new types. At least if a legacy API does not support it, one can always add support oneself.

  • William Staceystaceyw Before C# there was darkness...

    Thanks Erik.  That was a good read.  I see Birrell (of MSR and high priest) was referenced in that (RPC) paper.  It would be cool if you could at some point do an expert-to-expert on this subject with him - as he also seems to have some background in distributed systems and concurrency (e.g. Automatic Mutual Exclusion project).  Someone may enjoy following paper before bed - automatic mutual exclusion, multithreading, atomic fragments, transactions, c#, o my... :

    http://research.microsoft.com/en-us/projects/ame/ame-yielding-montanari.pdf

     

  • CharlesCharles Welcome Change

    Great idea!

    C

  • http://community.bartdesmet.net/blogs/bart/archive/2009/09/12/taming-your-sequence-s-side-effects-through-ienumerable-let.aspx

  • Bent Rasmussenexoteric stuck in a loop, for a while

    Thinking about Erik's Fundamentalist Functional Programming talk at OSCON reminds me: 

     

    DateTime.Now should be an IObservable<DateTime>!

  • ivan_ivan_ g

    Charles,

     

    Is there an expert-to-expert episode out there where CCR (Concurrency and Coordination Runtime) is compared to Rx?

    I would like to know of a specific similarities. May be Erik, from his functional and Rx perspective, can lead CCR discussion?

    I've heard CCR is very fast and I would like to know how Rx will fit into this concurency picture.

    Can I apply both CCR and Rx? What would the best use case for CCR and Rx? Where one is better/worse than the other for concurrent apps?

    I watched how MySpace implemented CCR into their code, it was very interesting.

  • CharlesCharles Welcome Change

    There is no CCR vs Rx E2E, but there is an Erik + CCR creator George Chrysanthakopoulos E2E... Great conversation, it is.

     

    C

  • If you are interested in hearing more about Rx make sure to attend my PDC (http://microsoftpdc.com/Sessions/VTL04) or my QCON (http://qconsf.com/sf2009/speaker/Erik+Meijer) talks.

     

     

  • This seems very exciting stuff.

     

    Coming from the videogames industry I got fascinated by functional reactive programming, and started learning Haskell and FRP a couple of years ago.

     

    C# is still my favorite language for writing production software, and it's great to see that my studies of Haskell & FRP will now help me write better reactive C# code. I did not expect this FRP hobby of mine to actually be useful in C#! So many thanks guys!

     

    Now the FRP systems I looked at (Fran, Yampa, Fruit, Reactive) are side effect free; it will be interesting to see how easy it is to master Rx in the presence of side effects.

     

    One thing that seems unclear to me is if Rx uses times at which events happen, and how they are sorted. For example, in Conal Elliott's Reactive library you have a merge function that merges two event streams, ordering them by occurrence time, left-biased if two events happen at exactly the same time.  Not sure how Rx handles that; I guess because it embraces side effects, it doesn't really care, since it is fully push based, what is pushed first is handled first?

     

    Really looking forward to play with it!

  • I am trying to work out what advantages this has over classic tightly coupled event handling or asynchronous WCF service calls. The only thing I can think of is the merging of events via linq however, is this not achievable via a 'normal' messaging model such as msmq where by the queue is doing the merging of events/messages? An uglier example of this is having a session state aware WCF service. Am I missing the point and is this just another way of doing it with the advantage of being easier to implement so developers can concentrate on discovering the needs of a system rather implementing solutions to the problems the needs of the system produces?

     

    Regarding the problem with receiving non linear events and solving this with the until, how does this work when the observers are in distributed system like a cloud environment where the until would need to be able to block/dispose of processes on other physical/virtual machines? Unless the cloud is already doing the thread merging and I just haven't found out yet.

  • WOW ... TKX Charles .... this was simple an amazing video to watch. And YES Erik Rx is both elegant and mathematically correct (just like LINQ) BUT PLEASEEEE do not retire. Wes Dyer dude u rock too bro ... and I am jealous u get to work with Erik Smiley

     

    THANKS again Charles/Erik for putting togather the E2E series and allowing us, "the App Develpers" really learn more about language design and how stuff is actually implemented behind the scenes.

     

    p.s.  I introduced these videos to my hard core Java developers and they are slowly but surely getting warmed up to the beauty of C# and .NET in general Smiley just love it.

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.