Maestro: A Managed Domain Specific Language For Concurrent Programming

Posted By: Charles | Dec 22nd, 2008 @ 11:46 AM | 94,121 Views | 12 Comments
Josh Phillips(PM), Niklas Gustafsson(Architect), and Artur Laksberg(Developer) of the Parallel Computing Platform Team spend some time with me to discuss a managed (.NET-based) DSL (Domain Specific Language) for concurrent programming, Maestro. Maestro incorporates well-entrenched language patterns (imperative, OO, C style syntax, etc) and language constructs (channels, agents, domains) in a compelling way to make concurrent composition more accessible and familiar to the legions of sequential code composers.

Here we dig into the architecture and design of the Maestro language and compiler as well as discuss the philosophy behind this incubation project (at this point in time there are no plans to release Maestro as a product - it's a research project, an incubation...). Why create another language to help solve the Concurrency Problem? What's the advantage over implementing a library (this is .NET after all -> CLR + BCL = most of the power of the platform)? There's obviously good reasons for implementig Maestro as a language, but you'll need to watch and listen to find out.

Enjoy.

Note: After this interview was conducted and posted to Channel 9, the Maestro team has renamed their technology to Axum. So, they are now the Axum team and the managed DSL for concurrent programming they're incubating is called Axum. Smiley
Rating:
4
0
#Dec 22nd, 2008 @ 1:10 PM
ilmar
ilmar
Super interesting stuff, keep this coming!
#Dec 22nd, 2008 @ 6:06 PM
ZippyV
ZippyV
Fired Up
I think I get the concept of it but what are some of the applications that could benefit from this? A business application, a game, ... ?
#Dec 22nd, 2008 @ 10:34 PM
Charles
Charles
Welcome Change
Any application with concurrent tendencies Smiley It's a DSL, but the specific domain is concurrency/parallelism and what it offers (at this point theoretically since you can't play with it yet) is a readily comprehensible way (providing sequential semantics that you're most likely very used to) to program multiple asychnronous domains that communicate via channels and the agents they possess. At least, this is my understanding after listening to the big brains explain their thinking.

C
#Dec 23rd, 2008 @ 1:37 PM
mkoops
mkoops

Charles,
Brilliant stuff, one question though -  Why is the MP4 version 500 MB v 160 odd for WMV?

Mark

#Dec 23rd, 2008 @ 1:40 PM
Charles
Charles
Welcome Change
In the new year, this will be fixed (most likely with the addition of a low res MP4). Right now, MP4's are being encoded at a high bit rate (maybe too high). Sampy (Mike Sampson), the inventor of Magic Folder (the application that does all of the processing magic for our media files) is aware of this and has plans to address this.

Glad you liked the interview!

C
#Dec 23rd, 2008 @ 4:20 PM
esoteric
esoteric
λ
I have thoroughly enjoyed all the M and Parallelism talks. For parallelism there is enough to keep you busy for quite some time.

So now we know that Maestro is built on CCR.

CLR
  • TPL
  • CCR
  • Maestro
  • ?
C/C++
  • CRT
  • PPL
  • ?
I would be interested in
  • Native vs managed integration? - CRT/PPL vs TPL/threadpool
  • TPL/CCR integration?
  • What scale do you expect what libraries to be able to manage?
  • What level of concurrency do what libraries efficiently enable?
  • How do what concurrency paradigms affect program complexity and predictability and how future proof will they be
  • In the past sequential programs became faster with faster processors; we cannot just expect parallel programs to continue to scale - exposing concurrency at some level may seem fine, but it will start to hurt at some level -- program flow prevents absolute parallelism; maximization how? - Niklas talks about speculative parallelism...
  • Dogfooding -
    • Where is Microsoft be dogfooding these libraries - applications and services; what is being done in Windows itself, going forward?
    • What will it dogfood where and why?
    • How deep into the OS will it dogfood and what will be used where?
    • What does concurrency mean for Windows?
  • Implicit concurrency
    • Is it feasible?
    • What languages?
    • What timescale?
    • On what scale?
    • Dynamic program optimization for needed parallelism (no. of cores) or static partitioning with "fixed-grained" tasks?
  • Parallelism concepts, how many are there both research-stage and applied
  • Will the asking ever stop?

#Dec 27th, 2008 @ 10:11 PM
nagunn
nagunn
It's interesting that the CCR is the machinery behind all this. It's an excellent piece of work and a shame (to my mind) that its adoption does not seem to have been wider. I think the actor model is certainly one of the better models for taming concurrency, and the CCR provides a strong foundation for it, amongst other things.

I think anything that Microsoft do to promote this model is going to be a good thing in the long run, and I look forward to a possible future release of Maestro in some form.

The deep isolation present in Erlang, which supports the actor model as a first class concept, is also available in Decentralized Software Services, which builds atop of the CCR and is available to C# developers now.

For those interested in the functional perspective, there is a non-blocking asynchronous message passing implementation available within the F# CTP and I've been toying with using the CCR from within F# computation expressions to simplify the syntax somewhat.
#Dec 29th, 2008 @ 2:05 PM
Charles
Charles
Welcome Change
What makes you think that CCR adoption is not wide? Consider that many people are using CCR and in some cases very novel (and unexpected ways, like MySpace...).

C
#Dec 30th, 2008 @ 12:12 AM
nagunn
nagunn
Charles hi,

I don't know how wide the adoption has been of course. I did say that it *seemed* not to have been wide and of course I could be wrong - wouldn't be the first time - but using the only benchmarks I have, namely (1) the number of articles in the blogosphere on CCR and (2) the amount of activity on the CCR/DSS forums, it's not an entirely invalid assumption.

Of course it could be that not many think the CCR is worth blogging about, or that it is such an obviously intuitive model that not many need the forums, but I think neither of these is true and your own comments in the video suggest that some developers may consider the programming model something of a mental leap from where they currently are.

I personally think the CCR is a really powerful model and *well worth* the investment. We have commercial licences for it at my place of work and maybe one day I'll be able to talk about that.

On the other hand, it could be that by helping solve issues both of concurrent systems design and of scalability, everyone is using it, but keeping quiet, hoping that no-one else will. But I doubt it... Wink

Happy New Year!
Microsoft Communities