Bart De Smet: Observations on IQbservable - The Dual of IQueryable

Thanks for watching and for the feedback. We're open to all sorts of discussions on the Rx forums, including the naming ones. This said, we tend to try getting the semantics right first, since alpha conversion is a widely understood technique. Not trying to downplay the importance of API design in the large, Wadler's Law seems to capture exactly what we're talking about .

Thanks for the kind words, Jules.. It may be my new diet that's paying off .

A few more pointers for the viewers:

• Types and Programming Languages by Benjamin Pierce (UPenn), containing coverage of the T-shirt
• Advanced Topics in Types and Programming Languages by the same author
• Expert to Expert: Brian Beckman and Erik Meijer - Inside the .NET Reactive Framework (Rx), explaining our interface duality
• Expression Trees on MSDN, as a good starting point to familiarize yourself with the tree APIs

Things not mentioned in the video:

• Some "imperative operators" using interfaces like IConnectableObservable, requiring code blocks (and hence statement trees in a fantasy homo-iconic world), are not available on Qbservable. There are just a handful of overloads that fall in this camp such as the parameterless Publish method (the lambda version is there though).
• Qbservable also provides support for join patterns as expression trees. That is, the And and Then operators from System.Joins are available in an expression tree form as well. It's left to the reader to dream up the endless possibilities this enables for joining multiple data streams "in the cloud".
• Constructors like FromAsync which return a Func<T1, ..., Tn, IObservable<TResult>> are supported too. They return a similar Func returning an IQbservable<TResult> instead. That IQbservable will contain an expression tree with an Invoke node upon invoking the returned function.
• Parameters on Qbservable operators are only protected against null reference inputs. That is, no operator-specific enforcements are made. For example, one is allowed to write Take(-5) against an IQbservable, but not against an IObservable. It's up to the provider to decide what -5 means (take last 5?, don't allow?).
• Observable has a static get-only Provider property that exposes the IQbservableProvider that can be used to create expression trees for "local" queries and can be used to get stuff like Return in an expression form, using Observable.Provider.Return(5), returning an IQbservable<int>. This is complementary to the AsQbservable operator.
• Use of AsQbservable on a "local" observable sequence allows one to write expression trees against such a source. When calling Subscribe on the resulting IQbservable, which also implements IObservable per definition, the tree gets rewritten to target the Observable.* operators, including join patterns and whatnot.
• For signatures Qbservable operators, notice that only a minimum set of parameters has been tweaked to be expression-friendly compared to the corresponding Observable operators. That is, things like Amb(IObservable, IObservable) only get the first parameter as IQbservable. This opens up for quite some flexibility; providers can type-check on the second argument.

Play and amaze us with great observable providers .

Changing names is easy, especially in our world of incubating ideas. Changing semantics on the other hand is hard. So let's focus on the latter first and then get to the aesthetics of it all. De gustibus non est disputandum and The colour of the bikshed apply here.

Oh, and the "visually close" argument raised earlier is rather weak, especially since homo-iconicity is all about that . Is x => x too much like ... x => x, while they may be radically different behind the scenes (delegate versus expression tree)? Also keep in mind the interface is not what users of providers see, at all.

This said, feel free to discuss naming on a dedicated thread on the Rx forums.

Bridging Rx with MSMQ is totally doable. At first sight, it doesn't look like there's a domain-specific queue query language you can apply to contents of a query, so the queryable approach seems redundant. Nonetheless, some of the System.Messaging APIs may benefit from mapping onto Rx equivalent operators. For example, the TimeSpan parameter used to specify a timeout on various Send/Receive operations is an ideal candidate for exposure through the Timeout operator of Rx.

Concerning this bridging, simply creating operators that expose a message queue as an IObservable and vice versa would likely suffice. In fact, I've done such a thing in the past. Transactions could be totally part of this too. Basically, you'd check whether a MessageQueue object is marked as Transactional, and if so, you'd likely do something along those lines:

• When importing an IObservable to MSMQ, you'd likely want a transactional send of many OnNext messages, rolling back the transaction on OnError and committing it on OnCompleted. This won't work well for infinite sequences of course (and likely you don't want to do a transactional "send just one message" kind of thing).
• When importing an MSMQ queue as an IObservable, you'd likely want transactional receive to work on a per-message basis, only committing the transaction if the OnNext message you send out to the consumers gracefully returns. That is, you guarantee the user has seen the message successfully and could handle it.

If this automatic transaction management doesn't work for you, a better option may be to have the user control the transaction himself, simply by use of MessageQueueTransaction objects. Not as sexy or smart as the above, but maybe necessary to accommodate for various more complicated scenarios.

I'll leave it as an exercise to the reader to implement such bridging operators.

(Edit: wanted to paste code, but it went terribly wrong; will try again later)

Hope this helps. Happy hacking!

The sample shown in the video will be posted at a later point at either the Rx team blog or my blog. I'll keep you current on that one.

One versus many is a popular debate amongst language people. In fact, certain functional languages that will remain unnamed encode a lot of stuff using lists (e.g. "nullability" in terms of zero-length lists to represent "not present"), while others introduce whole separate language features to deal with 0 versus 1 cardinality (e.g. "option"). I'm much more in the former camp as many can encode one, but not vice-versa.

Whether or not the IE/IQ design for aggregates it the right one is something that can be debated for a long time in a similar manner. If we're doing LINQ to SQL using it anyway, isn't a single result a single-column, single-row table? People have always used different methods on low-level APIs a la SqlCommand depending on the desired outcome. While you could use ExecuteReader to get a scalar value back, ExecuteScalar makes things simpler. But if LINQ generalizes operators over sequences, something can be said about it being closer to the ExecuteReader approach. With the sole argument of one more method call adding friction, xs.Sum().Single() isn't too bad. This said, I do buy the arguments made.

The thing that's certain is that in an asynchronous world you don't want to leave the monad. There are multiple reasons for that. One is cancellation, but the more important one is to preserve composition. The former is definitely an IO thing, but the latter applies equally well in the world of IE. We may introduce similar aggregates in our Interactive Extensions (Ix) assembly for LINQ to Objects as well. Simply because the original LINQ design didn't account for further composition, doesn't mean it's not better. In particular, with operators like Amb and Timeout (amongst many others) introduced in Rx and Ix, composition for aggregates is incredibly useful. Also, a different return type outside IE requires people to write different code to do data binding to the result of an aggregation versus a grid-shaped result.

Finally, in the IQ case, the fact you loose the expression tree breaks composition on that level in certain cases. Not if you're "lucky" and things like Sum() appear in a selector lambda expression. But if you want to translate computations involving multiple aggregates (agreed, not often done in today's world), you're left out in the dark as there's no way to get the tree representing Sum without triggering its execution (unless you manually reconstruct it using infoof(Queryable.Sum) in a MethodCallExpression). Btw, if aggregates stayed in IE, computation over multiple aggregates could be done easily using operators like Zip, which is just a "lifted binary operator" for single-element sequences.

There are a few samples where the form quits before event subscriptions are disposed; it may be that's the case you're hitting, which looks similar in terms of exceptions. I'll have a look. Thanks for letting me know.

Hi folks,

The sample of the LINQ to WQL provider can (finally) be downloaded. More information can be found on this post on the Rx forum. Please use the forum for further questions and/or discussions on the subject.

Have fun,

-Bart