In order to use ForkJoin, you'll have to add a reference to the experimental Rx assembly, but it should continue to work as it did before. Alternatively, if this is single-value async, you could also use Task.WaitAll or Observable.CombineLatest to achieve a similar effect, without having to rely on the experimental functionality.

posted by bdesmet

The main thing to decide is what you want to do with platform-specific functionality that ended up in a bottom layer of your library: keep the API surface there, discover it at runtime, and mark it deprecated to stimulate new usage patterns; or have a clean slate release for Portable Library with migration guidance for the types/members that moves. (In fact, if you don't have to slice a type because it had both platform neutral and platform-specific stuff in it, you can simply move the type up the layer map.) As explained, we went with the second approach initially, but the benefits diminished as we went along, simply because Portable Library's intersection between our target platforms got much better by RTM.

If I'd be doing it again now, I'd really strive for a portable core that eliminates platform-specific copies right from the start. Based on what falls out of the intersection, I'd decide whether some kind of enlightenment mechanism is required / warranted or not.

To implement such an enlightenment mechanism, the main trick is Type.GetType(string, bool) to look for a known type in a known assembly (ideally demanding an exact version match, saving you from future headaches) implementing a known interface. Then decide whether you can do a graceful fallback if the module is missing. At the same time, try making all deployment vehicles such that the enlightenment assembly is always installed (e.g. as part of a NuGet package, or in an Extension SDK).

I'll chat with the Portable Library team about documentation and guidance plans.

Hope this helps,
-Bart

posted by bdesmet

Long story short: you don't have to change any of your code.

posted by bdesmet

The analogy extends to a comparison of advantages and disadvantages between both approaches. For example, objects have a cost that may not be warranted for a lot of simple events (e.g. button clicks). Also, events are well-suited for a code-centric usage (e.g. kicking off a simple action), while observable sequences provide for a data-centric approach (e.g. computing statistics over temporal data).

posted by bdesmet

posted by bdesmet

In order to ensure "global" safety, there's a minimal amount of cheap safeguarding based on swapping out observers for mute ones as soon as an operator's input reaches a terminal state. So, multiple terminal messages (invalid from the grammar's point of view) are silenced but the first one, and the place this happens is inside the custom operator noticing this (rather than in the Subscribe method's wrapping of the observer in the past).

As for concurrent notifications (which are invalid too), there has never been true safeguarding against this, so inputs are assumed to be safe. Only when an operator has to deal with multiple sources, it will do proper orchestration internally (e.g. Merge has to ensure we never talk to the outgoing observer in a concurrent manner), using locks or other approaches. However, when an ill-behaved producer is in the mix, behavior is undefined (e.g. a Subject<T> that exhibits concurrent messages). In case such a producer exists, one has to safeguard the pipeline using Synchronize.

When implementing sources or custom operators, we recommend to use ObservableBase<T> and Observable.Create, respectively. Those are the safest way to get things right. In fact, for custom operators, the use of composition of existing operators is a good start. Going forward, we may provide more advanced "power user" facilities to write more efficient custom operators using techniques similar to the ones we're using internally now. However, with power comes responsibility, so users of such primitives and implementation patterns need to have a thorough understanding of expectations with regards to observable sequences, the observer grammar, etc.

posted by bdesmet

The BCL/CLR teams are aware of the need for such tricks in some types of APIs, and we may see more guidance or helper functionality to address this going forward (no promises implied in this statement). For a lot of Portable Libraries though, one can live without such mechanisms. Think about class libraries for a data model that can be reused across Windows Phone, Metro, Silverlight, classic desktop, etc.

posted by bdesmet

As soon as you start composing events, the benefits far outweigh the cost. A lot of the composition of Rx builds on the ability to compose subscriptions. Subscribing to the merge of N sources means returning an IDisposable that holds on to the underlying N source subscriptions. It's as simple as that.

If you'd measure all sorts of performance metrics in real-world scenarios, I doubt the disposable object graph will show up as a relevant influence. (Btw, we have many IDisposable implementations internally in Rx, also to allow for reuse of objects.) It definitely didn't in the Rx v2.0 Beta analysis of performance. Also, typically there's 1 subscription for a whole bunch of events flowing through the pipeline: 1 <<<<< N.

To conclude, my remark is mostly about the debate whether or not observables are a primitive or an abstraction over a primitive. Just like delegates are first-class representations of methods, observables are first-class representations of events (which are method pairs for our purposes). In this analogy, one doesn't "convert" (e.g. through method group conversion in C#, see ECMA 334, section 13.6) a method into a delegate unless you want to do a more functional style composition (e.g. LINQ to Objects heavily relies on delegates). The same holds for observables, with the only difference being a bit more friction to convert an event to its first-class representation (i.e. there's no method group equivalent for event members).

posted by bdesmet

Unfortunately, the namespace shows up in IntelliSense at this point, due to some complexities around the IDE's handling of InternalsVisibleTo. We're aware of this problem and are looking into it.

posted by bdesmet

What you likely want is a smoother way to convert an event into an observable sequence, just like F# has an implicit conversion allowing this (see here and here for starters).

posted by bdesmet

• Data center monitoring scenarios - e.g. computing moving aggregates of CPU utilization, noticing peaks, etc.
• Sensor and signal processing systems - e.g. collecting and processing data in smart grids, network centers, etc.
• Robotics where queries over sensor inputs control servos etc. - e.g. we've seen queries being compiled into embedded code.
• GPS processing systems to tame events of long/lat pairs - e.g. filter out noise, correlate with other data stores, etc.
• Intrusion detection systems by processing historical logs and real-time data - e.g. logs of requests / IP address, try to find suspicious outliers.
• Blending UI events with asynchronous requests to web services etc. - e.g. take input in a text box and launch async web calls for completion etc.

The spectrum of places used has also grown signficantly. We started off by going into Windows Phone 7 as an API to help users process sensor data (accelerometer, GPS, etc.). In full desktop CLR, we've seen adoption ranging from client UI scenarios to server/datacenter workloads. We're seeing an increasing interest in the embedded space as well, including robotics. And we have Rx for JavaScript used in both client and server scenarios (e.g. node.js).

Stay tuned and you'll see us "eat our own dogfood" in various places. Quite a few products today are shipping "powered by Rx" (most lately PowerView in SQL Server 2012, various online services in Bing, but also yet-unannounced projects), and the number is growing fast.

With regards to debuggability, we've done a lot of work in this release to make callstacks readable, approachable, and understandable (see the blog post). This is just the end of the beginning of work we're doing in this field. So again, stay tuned!

posted by bdesmet

posted by bdesmet

posted by bdesmet

Personally, to reduce the event conversion friction I'd invest in a conversion of events to IObservable<T> like F# does, but at this point there are no such plans.

posted by bdesmet

posted by bdesmet

First of all, staying in IObservable<T> allows for further composition, which is very rich in the world of Rx. A typical example I always bring up is to apply the Timeout operator to the resulting sequence. With Task<T> you can do similar things using WhenAny or by using cancellation mechanisms, but it's less sexy.

Secondly, by going to Task<T> there's an implicit change in scheduler affinity. Having the ability to control scheduling through the IScheduler abstraction and having a more free-threaded world is an advantage of Rx we don't want to loose due to implicit changes of primitives.

Having said all of this, users can still go to a Task<T> object explicitly by using the ToTask operator. However, for the simple task (pun intended) of awaiting the result of such a single-valued observable sequence, support is right there in Rx to use "await" directly.

posted by bdesmet

posted by bdesmet

The main difference is this: with Rx you get a very high degree of compositionality thanks to the composition taking place at the IObservable<T> level. If you look at that layer as the declarative layer, you can think of deeper layers in Rx as operational layers. Concepts that fit in there include subjects and schedulers.

At that level, it's much harder to compose because of the much more stateful nature of the building blocks. You can already see this being the case in the Rx API where the binding operators meet their subjects, requiring different - imperative-style - interaction through the notion of IConnectableObservable<T>.

Similarly, composition of things like "backpressure" on an observable channel using queues and producer/consumer interaction is much harder. Say you have multiple sources coming together and you need to throttle the rate at which things are being sent. How do operators propagate the signals required to apply backpressure etc? Typically you deliver those out-of-band with regards to the composed query operators, requiring quite some plumbing.

All in all, your analysis of "high level down" and "low level up" is pretty accurate. It's much (to use an analogy with a different domain) like the contrast between the querying experience in an RDBMS versus leveraging ISAM style access. The level at which you glue things together is different, and hence you have different levels of high-level expressiveness and/or low-level control.

The nice thing is you can mix and match the technologies depending on your needs. Also, preferences in programming style contribute to those decisions: functional declarative state-poor combinator-centric query style, versus a more imperative state-rich approach based on message-passing actor/agent style. And thanks to the bridges available on the building blocks, you can sandwich Rx inside TPL Dataflow and vice versa, if needed.

Also make sure to have a look at the documents available on the TPL Dataflow website. Also see this discussion that summarizes some of the observations (pun intended) made here.

posted by bdesmet

posted by bdesmet

posted by bdesmet

First, a quick reminder about the duality between IE<T> and IO<T>. If you distill the essence of the interfaces using arrows, you get both isomorphisms (ignoring the aspect of resource maintenance, captured by IDisposable, since we only care about the data flow aspect):

• IE<T> ~ () -> (() -> T | Exception | void)
• IO<T> ~ ((T | Exception | void) -> ()) -> ()

The only thing we did is reverse the arrows, and end up with the dual type. Interfaces are a mere manifestation of the lack of discriminated union types, and have a particular imperative feeling to it that some consider to be "more clear".

With regards to the time notion now. Your interpretation of time is one that treats time as data. The universe you describe is one where Tuple<Time, T> is a coordinate in a n+1 dimensional space with n being the number of spatial coordinates and 1 reflecting the time dimension. The spatial coordinates are defined by the CLR type system and heap here: typeof(T) and the particular instance of the type. This approach is totally valid, and in fact we have it in Rx in the form of Timestamped<T>, which is isomorphic to Tuple<Time, T>.

However, there's a different interpretation of time that relates to the control flow aspect of computation. Rather than associating time with data (by treating it as data itself), time can be related to the execution itself. In Rx, this is embodied by the IScheduler interface which carries a clock. In this world, the scheduler defines the universe of space (where things happen, e.g. on the thread pool), and time (when things happen, related to the scheduler's clock). Going back to machine architecture analogies, you could see the number of instructions executed (or the number of CPU clock ticks) as the clock.

Operators like Timeout have been defined in terms of the latter interpretation where time is extrinsic to the data (observe the IScheduler parameter passed in). For example, upon arrival of an OnNext message, the scheduler can be invoked to run a timeout timer to monitor the next OnNext message coming down the pipe. In this worldview, time is associated with the execution of method calls. You can do a very similar thing for IE<T>, now based on the dual MoveNext method. Again, you observe the time behavior of the environment that's sending you the data by running a local timer (ignoring relativistic effects), measuring the time that elapses between method calls: the incoming MoveNext call and the time it takes for the environment to respond to the outgoing MoveNext request. To do so, you need to rely on a clock which is extrinsic to the data, e.g. provided by the IScheduler interface.

That's not to say you can't define time-based operations using an intrinsic notion of time, e.g. based on Timestamped<T>. In that case, you're performing distance measurements and coordinate transformations on the data itself, which happens to carry time information. I invite the niners to think about all kinds of operators in terms of this (e.g. Select translates an object in space). This approach works very well for analysis of historical data, e.g. log data. However, you could do this too by using virtual time scheduling.

The discussion where to put time is very similar to the one on where to put other aspects of the control flow such as exceptions. Do you treat those as code or as data? In Rx and Ix, we treat exceptions as control flow aspects (cf. OnError and the dual of MoveNext throwing) at the interface level (note for Ix: the C# language doesn't provide checked exceptions, hence it doesn't show up in the IR<T> interface explicitly). However, one can employ reification to treat those control flow mechanisms as part of the data flow, using operators like Materialize. Similarly, the time aspect can be moved from control flow to data flow using the Timestamp operator. Both treatments are valid, and it depends on your scenario which one is more convenient than the other.

Finally, the discussion on extrinsic or intrinsic notion of time with regards to the data is one of the essential differences between Rx and StreamInsight. In the latter technology, every event carries its own notion of time (cf. CepStream<T>), including duration. In Rx, we model those things differently using the concept of reactive coincidence. Again, because both notions make sense, there are conversions possible between both worlds; e.g. StreamInsight has an IO<T>-based programming model exposed.

posted by bdesmet

posted by bdesmet

posted by bdesmet

posted by bdesmet

posted by bdesmet