Bart De Smet

Bart De Smet bdesmet Bart De Smet ​[MSFT::SQL::​Cloud​Programmabi​lity::Rx]

Comments

  • Bart De Smet: Rx Update - .NET 4.5, Async, WinRT

    @ligAZ: We're actively working with the PLIB folks on making Rx portable. The main thing required is to have IObservable available in all supported platforms. At this point, we can't commit to a time frame, but we're on it Smiley.

  • Bart De Smet: Rx Update - .NET 4.5, Async, WinRT

    @AdamSpeight2008: Remains to be seen. Notice the trend of reducing magic strings in pretty much every recent release of the languages (e.g. LINQ instead of queries in strings, dynamic instead of invocation calls with member names in strings, call info attributes instead of hardcoding the current member's name, etc.).

    Personally, to reduce the event conversion friction I'd invest in a conversion of events to IObservable<T> like F# does, but at this point there are no such plans.

  • Bart De Smet: Rx Update - .NET 4.5, Async, WinRT

    @felix9: The test code you saw on the screen is not flawless indeed. In general, it's possible for the sequence to produce an OnNext message before the Dispose call to the subscription occurs. The fix is quite simple though: use the Finally operator in Rx to run code after the sequence completes or gets disposed.

  • Bart De Smet: Rx Update - .NET 4.5, Async, WinRT

    @Jeff: While Task<T> would be a good fit for single-value asynchronous results coming out of the Rx APIs, there are a few things we don't want to get into.

    First of all, staying in IObservable<T> allows for further composition, which is very rich in the world of Rx. A typical example I always bring up is to apply the Timeout operator to the resulting sequence. With Task<T> you can do similar things using WhenAny or by using cancellation mechanisms, but it's less sexy.

    Secondly, by going to Task<T> there's an implicit change in scheduler affinity. Having the ability to control scheduling through the IScheduler abstraction and having a more free-threaded world is an advantage of Rx we don't want to loose due to implicit changes of primitives.

    Having said all of this, users can still go to a Task<T> object explicitly by using the ToTask operator. However, for the simple task (pun intended) of awaiting the result of such a single-valued observable sequence, support is right there in Rx to use "await" directly.

  • Bart De Smet: Interactive Extensions (Ix)

    @evarlast:Thanks for bringing the broken link to our attention. We'll work on fixing it. In fact, we have a new version of Ix available (v1.1.10823). Hiding the old version from the Download Center caused the link breakage. Sorry for the inconvenience.

  • Bart De Smet: Interactive Extensions (Ix)

    @PerfectPhase: TPL Dataflow has some overlap for sure, but at the same time the two technologies can work together using - as you mention - bridging facilities.

    The main difference is this: with Rx you get a very high degree of compositionality thanks to the composition taking place at the IObservable<T> level. If you look at that layer as the declarative layer, you can think of deeper layers in Rx as operational layers. Concepts that fit in there include subjects and schedulers.

    At that level, it's much harder to compose because of the much more stateful nature of the building blocks. You can already see this being the case in the Rx API where the binding operators meet their subjects, requiring different - imperative-style - interaction through the notion of IConnectableObservable<T>.

    Similarly, composition of things like "backpressure" on an observable channel using queues and producer/consumer interaction is much harder. Say you have multiple sources coming together and you need to throttle the rate at which things are being sent. How do operators propagate the signals required to apply backpressure etc? Typically you deliver those out-of-band with regards to the composed query operators, requiring quite some plumbing.

    All in all, your analysis of "high level down" and "low level up" is pretty accurate. It's much (to use an analogy with a different domain) like the contrast between the querying experience in an RDBMS versus leveraging ISAM style access. The level at which you glue things together is different, and hence you have different levels of high-level expressiveness and/or low-level control.

    The nice thing is you can mix and match the technologies depending on your needs. Also, preferences in programming style contribute to those decisions: functional declarative state-poor combinator-centric query style, versus a more imperative state-rich approach based on message-passing actor/agent style. And thanks to the bridges available on the building blocks, you can sandwich Rx inside TPL Dataflow and vice versa, if needed.

    Also make sure to have a look at the documents available on the TPL Dataflow website. Also see this discussion that summarizes some of the observations (pun intended) made here.

  • Bart De Smet: Interactive Extensions (Ix)

    @AdamSpeight2008: You're lucky I have some flights coming up pretty soon Smiley. In the meantime, here are a few things to think about. How do grouping and windowing operators compare to the idea of a multi-yield? What's the lifetime of the underlying enumerator in case you have multiple sequences that are artifacts of applying an operator over a shared sequence (such as multi-yield)? Think about both the acquisition of the enumerator as well as the disposal thereof.

  • Bart De Smet: Interactive Extensions (Ix)

    @scyonx: We'll work on providing more samples on the subject going forward. Stay tuned.

  • Bart De Smet: Interactive Extensions (Ix)

    @Jules: Thanks for your post. First things first: to be clear, you can write the time-based operations for IE<T> without any change to the type. Where we disagree is in the interpretation of time. Let's go into more details.

    First, a quick reminder about the duality between IE<T> and IO<T>. If you distill the essence of the interfaces using arrows, you get both isomorphisms (ignoring the aspect of resource maintenance, captured by IDisposable, since we only care about the data flow aspect):

    • IE<T> ~ () -> (() -> T | Exception | void)
    • IO<T> ~ ((T | Exception | void) -> ()) -> ()

    The only thing we did is reverse the arrows, and end up with the dual type. Interfaces are a mere manifestation of the lack of discriminated union types, and have a particular imperative feeling to it that some consider to be "more clear".

    With regards to the time notion now. Your interpretation of time is one that treats time as data. The universe you describe is one where Tuple<Time, T> is a coordinate in a n+1 dimensional space with n being the number of spatial coordinates and 1 reflecting the time dimension. The spatial coordinates are defined by the CLR type system and heap here: typeof(T) and the particular instance of the type. This approach is totally valid, and in fact we have it in Rx in the form of Timestamped<T>, which is isomorphic to Tuple<Time, T>.

    However, there's a different interpretation of time that relates to the control flow aspect of computation. Rather than associating time with data (by treating it as data itself), time can be related to the execution itself. In Rx, this is embodied by the IScheduler interface which carries a clock. In this world, the scheduler defines the universe of space (where things happen, e.g. on the thread pool), and time (when things happen, related to the scheduler's clock). Going back to machine architecture analogies, you could see the number of instructions executed (or the number of CPU clock ticks) as the clock.

    Operators like Timeout have been defined in terms of the latter interpretation where time is extrinsic to the data (observe the IScheduler parameter passed in). For example, upon arrival of an OnNext message, the scheduler can be invoked to run a timeout timer to monitor the next OnNext message coming down the pipe. In this worldview, time is associated with the execution of method calls. You can do a very similar thing for IE<T>, now based on the dual MoveNext method. Again, you observe the time behavior of the environment that's sending you the data by running a local timer (ignoring relativistic effects), measuring the time that elapses between method calls: the incoming MoveNext call and the time it takes for the environment to respond to the outgoing MoveNext request. To do so, you need to rely on a clock which is extrinsic to the data, e.g. provided by the IScheduler interface.

    That's not to say you can't define time-based operations using an intrinsic notion of time, e.g. based on Timestamped<T>. In that case, you're performing distance measurements and coordinate transformations on the data itself, which happens to carry time information. I invite the niners to think about all kinds of operators in terms of this (e.g. Select translates an object in space). This approach works very well for analysis of historical data, e.g. log data. However, you could do this too by using virtual time scheduling.

    The discussion where to put time is very similar to the one on where to put other aspects of the control flow such as exceptions. Do you treat those as code or as data? In Rx and Ix, we treat exceptions as control flow aspects (cf. OnError and the dual of MoveNext throwing) at the interface level (note for Ix: the C# language doesn't provide checked exceptions, hence it doesn't show up in the IR<T> interface explicitly). However, one can employ reification to treat those control flow mechanisms as part of the data flow, using operators like Materialize. Similarly, the time aspect can be moved from control flow to data flow using the Timestamp operator. Both treatments are valid, and it depends on your scenario which one is more convenient than the other.

    Finally, the discussion on extrinsic or intrinsic notion of time with regards to the data is one of the essential differences between Rx and StreamInsight. In the latter technology, every event carries its own notion of time (cf. CepStream<T>), including duration. In Rx, we model those things differently using the concept of reactive coincidence. Again, because both notions make sense, there are conversions possible between both worlds; e.g. StreamInsight has an IO<T>-based programming model exposed.

  • Bart De Smet: Interactive Extensions (Ix)

    In case you want to use NuGet to install Ix, look for the Ix_Experimental-Main and Ix_Experimental-Providers packages (following our naming convention used in Rx).

  • Rx Workshop: Introduction

    @exoteric: Ix has shipped again with improved operator parity, some extensions, and some cleanup. More information here.

  • Announcing the Official Release of Rx!

    @wkempf: You can still get System.Threading.dll from old releases (prior to 1.0.10425) and use it, but keep in mind the bits are provided as-is with no support.