Concurrency and Coordination Runtime

Hi, we developed the CCR independently from the futures work. We wanted to investigate a model where the asynchronous interaction is visible, not implicit. Futures have been around for awhile and we believe that we can express the patterns futures enable easily using ports and our basic arbiters. We are not certain having ports exposed directly is the way to go, but its a concrete, useful alternative to locks and custom one-off queuing implementations we all end up writing when we want to hook up things together.

We have discussed with Herb to see if the CCR can be the execution target for the futures/active work, assuming a native version is written. The CCR right now is only available in managed environment.

Wow! Really Really Smart guys. They were/are light years ahead of me. It's like their brains are working at mach 5 speed and their mouths are only going 60 miles an hour. Most of this content went over my head but its super sexy stuff to hear about.

And I loved how robert stayed quiet and did an awesome job while filming.....but the best part was the number of time I heard Charles say "Fantanstic". I stopped counting at 10. Looks like I've stumbled onto the new buzz word around concurrency...."It's Fantastic" should be the tag line!

Keep up the great work and showing us 9ers these really smart people and their sexy forward looking technologies

scobleizer wrote:

See, I'm trying to not do my stupid laugh so much.

*snif* I always liked it.

jskeet, you are right CSPs, Pi calculus and alot of other areas are all related. The CCR definately is influenced by previous work. Alot of the influence came from my experience in the Xbox team writing asynchronous APIs to keep things responsive and also from my time in the NT Kernel (Base) team working on device drivers. Soon it became apparent that a nice way to write message based, asynchronous code, but without the complication of continuations, callbacks etc and with a nice way to coordinate among many things at once (like scatter/gather) would be very useful. I also wanted to be available in C# or VB or any CLR language! Yes the CCR is accesible from VB.Net.

Check out the Wiki linked above for examples on how we used Iterators to make code look like "blocking" calls even if its full asynchronous.

Iterators in CCR are used in a very interesting way that allows you to even do loops!

foreach(Message m in MyOutboundMessages)
{
    Success successResult = null;
    UIPort.post(m);
    yield return
    m.Response.with(delegate(Success s) {successResult = s;}))
    |
    m.Response.with(delegate(Failure F) {}));

    // this line will execute after *either branch above, but with no
    // thread blocking. We just wake up here when a response
    // happens!

    if (successResult != null)
        Console.WriteLine(successResult.ToString());
    else
        Console.WriteLine("Failed UI message result");
   
}

Hi, we just think that a message passing coordination library is not  the only solution out there. I think its plenty good and usable now but people might find it too "low level" or explicit. I think the CCR is complementary to other solutions like Software Transactional Memory (STMs) if you use STMs to get isolation and protect shared memory.
One good thing about the CCR is that lines up well with distributed solutions that use message passing.

While CCR has things like interleave which can easily protect access to shared data structures, a version based scheme, in some cases, will give you better results.
Example:

// we have two methods that access some shared state. Unlike locks
// i dont need to know what that state is in the Ccr. This way
// you dont deadlock by grabing locks in the wrong order, etc

activate(
    exclusive(!MainPort.with(UpdateHandler))) 
    ^
    concurrent(!MainPort.with(ReaderHandler)));

in the snippet above CCR will run as many instances of ReaderHandler as messages posted in the MainPort,as long as no instance of MyUpdate Handler is running. It will stop scheduling concurrent handlers if a exclusive handler gets queued. This is like a writer biased reader/writer "lock". In general we use things like interleave to get incredible concurrency (millions of work items).

A version of the CCR with STM support could do this:

activate(
 !MainPort.withatomic(UpdateHandler)
 |
 !MainPort.with(ReaderHandler));

Other solutions to concurrency are data level parallelism libraries

Hi Multicore, the paper linked in the header of this forum mentions C-Omega and its roots in the Join Calculus. DavidRichter also mentioned it earlier. I became aware of Join Calculus after CCR was already fully developed (I was aware of the Pi calculus and CCS/CSP). For the CCR, the join primitive is just one of the possible coordination primitives. Our dispatching architecture and arbiters allow you to do things Join describes but our primary motivation was patterns we run into coding various applications. We then codified those patterns. One of them happened to be join plus the variable and dynamic versions of join not described in earlier work (althoutgh they are expressible in the framework of the join calculus)

One thing I need to mention is that Join, choice etc are not rooted in the Join or Pi calculus. These are primitives OS developers and application developers have used for a very long time. In win32 they have names such WaitForMultipleObjects and the semantics are very similar, but on event sychronization objects not queues, like we do.

All these deep CS stuff is not very approachable to most people and from the start my motivation was to improve, using a common, small set of primitives, alot of the existing OS synchronization APIs, not encode something found in theory.

Hope this makes sense.

I am glad you asked about debugging. We have worked on a debugging story that is message and asynchrony aware.

Unlike stack based debugging which is little to no help here (since the thread you wake up is arbitrary and the stack frames above you are just the scheduler) you need something thet logically relates your messages and the handlers which wake up to service them. This creates an execution tree you can follow or set breakpoints on messages. Yes breakpoints on messages so no matter who picks them, you break there. In the CCR, when you post to a port, you dont know who is listening (part of the cool loose coupling we give you) but that is a nightmare for regular debugging, since you cant "step-into". Well, with our message based debugging, you can "step-into" the receiver of the message and then any others receivers that fire because of messages logically related.

Its sounds a bit fuzzy right now, but when the CCR bits become available for download as a preview (first half of next year or sooner) a debugging story will go with it

Absolutely. CCR APIs will pass through FxCop with zero issues. So far we have been experimenting with syntax that is concise and readable but before we release, even as a CTP, it will conform to the standard MS rules.

Btw, if people read the wiki, although by far not an exhaustive reference it allows you to give feedback on the syntax we have chosen. We welcome all feedback. Underneath the CCR has a concistent design where all arbiters, receivers are schedulable tasks.

If you dont like the operators, you can always create directly the arbiter and schedule it. Example (with operators first):

// create a port with two queues, one for each message type
Port<SuccessMessage,FailureMessage> Response = new Port<SuccessMessage,FailureMessage>();

pFile.post(new Read(Response));

// attach a choice with success or failure handlers
activate(Response.with(Success) | Response.with(Failure));

Now instead, lets create the choice directly:

// the last line above is the same as this:
_pDispatcher.post(
    new Choice(
       Response.with(Success),
       Response.with(Failure)
   ));

The activate() api is of a base class that has a default dispatcher set. Programs can choose to create different dispatcher to interop well with STA threaded legacy apis or for more explicit load balancing. But they key is that the simple, consistent treatment of everything as a scheludable task, allows you construct things the way you want, in an imperative manner, and then issue them as commands.

Short answer: Yap we take care of it. And yes, if you want to, you can specify in more detail how the scheduling will occur.

Long answer:
The CCR takes care of automatically distributing work across processors. All you do is associate receivers with ports, constrainted under some arbiter (choice, one-time/re-issue receive, interleave etc) and we will generate work items when messages arrive, and schedule them across task queues with some number of OS threads in the back end picking them apart. Usually we use 2 threads per processor but you can set that number.

We can achieve near perfect load balancing across heterogenous tasks and keep all your CPUs hapilly busy. A CCR program just needs one instance of a dispatcher process, and if you set the number of threads to use to 0, if you want us to choose how many threads per processor to utilize in the back end.

Dispatcher creation:

DispatcherPort MyDispatcher =
Dispatcher.Create(0,"FriendlyName");

// now we can use it to associate a piece of code
// that will execute in one of the CPUs, when message arrives

MyDispatcher.post(!MyMainPort.with(delegate(DoWork W)
{
    // do work here. If you get N DoWork messages, we will
    // run as many versions of this code concurrently as your CPUs!
}));

// you can also set the dispatcher as  field in a base class

class MyFirstCcrProgram : CcrServiceBase
{
     public MyFirstCcrProgram(DispatcherPort Dispatcher):
         base (Dispatcher) {}

     void Init()
     {
          // because we associated a dispatcher
          // when we were constructed, we can just
          // use the base member activate()
         
    activate(!MyMainPort.with(delegate(DoWork W)
    {
        // do work here. If you get N DoWork messages,
        // we will run as many versions of this code
        // concurrently as your CPUs!
       
    }));
    
    }
}


If you want greater control you can create multiple dispatchers, and then each item you want to activate a receiver, use a particular receivers. This is very powerfull and unlike global threadpool schemes: By being able to  associate a dispatcher per receiver, you can interoperate very well with Single-Threaded-Apartment components, like COM, but your CCR code does not look any different!

We will give you at least one simple "adaptor", used for UI, where you post messages on ports, but using its own dispatcher, with just one thread, it does Invoke in the back, on your behalf, toa winform.

All this helps you go the next level, if you wish, in controlling execution, scheduling etc and interoperating with the OS.

This indeed offers a middle ground. Using iterators and anonymous delegates you dont have to explicitly pass state around or do you have to explicitly code continuations!. You can scale to hundreds of thousands of work items and the same time keep your code in one place. Using the yield return keyword, you get "blocking" behavior but without blocking. This way you keep all the logic in one place, without jumping around continuations.

A server example is actually given in the wiki and in the examples above but i will show one here. Its so concise, its easy to miss:

class MyHttpServer:CcrServiceBase
{
      Port<HttpRequest> MainPort = new
          Port<HttpRequest>();

      void Init()
      {
          // assume you have some api wrapper around http listener
          // binding a CCR port to listener requests

          Bind(MainPort,"MyUri/");

          // now activate an iterator handler that gets invoked
          // whenever a http request comes in.
          // notice "!" infront which makes this handler stick around

          activate(!MainPort.iteratewith(RequestHandler);
      }

       IEnumerator<ITask> RequestHandler(HttpRequest httpReq)
      {
           byte [] fileData = null;

           // lets say you need to two things for every request that 
           // comes in: Read from a file and send a log update.
           // again we have a simple stream wrapper around a file
           FileReadRequest fr = new FileReadRequest();
           LogUpdate log = new LogUpdate();

           FileService.post(fr);
           LogService.post(log);

           yield return join<byte [],bool>(fr.Response,log.Response).
               with(delegate (byte [] data, bool logDone) 
               {
                   // export a variable to outer scope
                   fileData = Data;
               })
               |
               fr.Response.with(Exception Ex) { // error })
           );
           if (fileData == null)
               // fail request, deal with error. Notice how the delegate
               // inside the yield above can touch stack variables
               // in the scope of this routine!

           // this line executes after the yield above but without ever
           // blocking a thread. We get logically sequential
           // behavior but without wasting a thread.

           // use the data here and return result. 
           httpReq.post(fileData);
      }

}

Please note the section in John Vert's article where he mentions all the benefits of using IoCompletion ports to simplify handling I/O requests and getting just a handfull of threads to do all the work. The CCR, using one consistent interaction mechanism, the CCR Port, provides a very similar model. You use ports to indicate completion of tasks, and you attach Reissue receivers (using activate) to deal with them. You dont care how many CPUs your system has or how many threads you need. We will do the right thing. The key, in both schemes (NT and CCR) is asynchrony.

g

Staceyw,  you will actually be able to service alot more than the number of OS threads in 32 bit systems, if you use asynchronous socket APIs, even from C#. The underlying OS will use overlapped I/O and will not block a thread per connection. Then, the total number of simultaneous requests you can service will not be bound by OS threads. The key ofcourse is to use asynchronous APIs all the way through, so you dont consume a CCR or C# threadpool worker. YOur actuall throughput might not increase btw, but your system will be able to service hundreds of thousands of packets, independent of each other so appear more responsive.

See one of the examples of iterators in this thread or the wiki.We actually have implemented a CCR based transport that does this (but cant post the code, yet , and it uses the C# asynchronous socket calls underneath. We just had to wrap them with CCR ports and simple processes so we did not have to deal with Begin/End.

The CCR, using iterators allows you to get "blocking" behavior so your code does not look like the spaghetti of continuations we find ourselves writing again and again. You can actually encode a while loop , processing packets but with no blocking! One of the examples in the wiki shows a for loop using iterators to encode asyncronous but logically sequentially req/rsp operations. It keeps it all in one routine.

So really what we are doing for you, is allow you to use the asynchronous OS APIs, through a much more readable front end, to get the logical behavior of threads, but without the overhead and limitations.

When threads become super lightweight one day( if ever), then the CCR can use alot more of them and get rid of iterators, but its syntax will not really change. The CCR can have blocking arbiters (join, choice etc)

hope this helps.

I updated the wiki with some examples of how you can do value based filtering with the CCR, allowing you to concurrently parse message streams and coordinate among various logic based on just values.

we plan to have Beta quality bits available for download around the may/june timeframe.

yes UI responsiveness it can definatelly help you with, especially coordination among multiple things , like IO, while keeping the UI going.

The Win32 and Winforms UI model is a bit hard to get right however because of its built in SingleThreadApartment design and various re-entrancy issues. Right now i would recommend you look at the latest PDC talks on concurrency, especially the talk from Jan Gray and Joe Duffy on CLR concurrency best practices. Also look into the background worker CLR 2.0 feature that helps with UI concurrency.

Someone at work made me aware of that lock as well, 3 or 4 months ago. The reader/writer lock jeff wrote is a subcase of the interleave primitive we have. We have talked to jeff about it and Jeff was thinking down the same path that led to the CCR. The CCR however normalizes all operations to be coordination of messages over ports, so reader/writer is just one of the coordination primitives you can implement cleanly. Many other things, like its dispatching model, port behavior etc are very different. Its more of a complete runtime than a single primitive. That can be both good and bad.

Anyway, stay tuned, Jeff is working with us, giving good feedback, and in the future help with articles on the CCR.

We actualy have built distributed system infrastructure on top of the CCR. The CCR itself is not ofcourse about distribution (you need other machinery for that, which sounds you have built already) but it helps you with coordinating I/O, so it was a good fit for our distributed work. With some simple adaptors over .NET system apis (like web requests, sockets, streams) we were able to have one consistent way to deal with i/o and load balance well.

So i see no reason why you cant use this in your system.

I really want to get this out there, and have people play with it. As i said, in a couple of months you can play with the CCR, plus a bunch more that has been built on top of it Its currently in good shape and as i mentioned earlier Jeff Richter is working with us to simplify the object model and then write up an article on it.

g

current release is planned for this june. Just 2 months away

what you are doing is perfectly valid. In the recent refactoring we did, we removed operator overloads. All runtime features are still there however, including dynamically creating choice, join, etc.

The wiki has the latest API.

Yes, ports, dispatchers etc are regular C# objects so they can be used anywhere you use and store object references.

Yes ports can be extended. You just implement the appropriate interfaces or easier, derive from Port<T>. We used to have LIFO, random insert implementations but again and again FIFO became the norm due to to starving issues related to LIFO and the distruptive attributes non-FIFO methods have on streaming or preserving sequence of events.

messages stored on ports, are by reference. We have extended some of our Arbiters (for example we have a variant of Receiver<T>) that deep clone objects before waking up the user task+ delegate associated with the port. The CCR allows you to do this type of extensies trivially and in a manner that the sender (the process that Posts) never has to know about. As i mentioned earlier we have a distributed system built on top of it

yes ports are definately thread safe!

Hi, the CCR is ready for its first public release. It will be available tuesday as part of a new product CTP we will announce then. It is an independent part, so people just interested in the CCR can use it by itself. I will post the link tuesday. Also expect another channel9 video showing the ccr in action

ok the bits are live. You can go to msdn.microsoft.com/robotics, and download the setup. its kind of big since it includes a few external components but once it installs, the CCR dll will be under \bin. Take a look at the service tutorials that come as part of the robotics SDK. The CCR is used as part of a service model that will give you cross machine distribution, in addition to concurrency and coordination within a machine.

July 9th you will also get a MSDN article ont he CCR with sample code using it in various scenarios.

sumothecat wrote:

Very enjoyable video! I have a simplistic question -- I thought a "runtime" was something that loaded code and managed its execution (e.g. CLR, Java runtime). This is a type library -- what makes it a "runtime"? Thanks!

The CCR, even if its a library, actually manages all execution, replacing the CLR thread pool or any manual threading code. Underneath ofcourse it uses the OS+CLR.

The CCR manages messages on ports, task load balancing and execution, iterator task stepping, dispatcher lifetime etc. So its somewhere between a library and an execution environment. If you check out our robotics video, it might make it a little bit more clear why the CCR is a fundemental piece of our execution environment

http://channel9.msdn.com/Showpost.aspx?postid=206574