Concurrency and Coordination Runtime
- Posted: Dec 07, 2005 at 11:33 AM
- 165,906 Views
- 77 Comments
The Concurrency and Coordination Runtime (CCR) is a lightweight port-based concurrency library for C# 2.0 developed by George Chrysanthakopoulos in the
Advanced Strategies group at Microsoft. Here, we have a deep discussion about CCR with George, a Software Architect, and Satnam Singh, Architect. You can get more info about CCR on the
CCR Wiki. This is super cool stuff and represents a really innovative approach to making managed threaded programming more readily understandable and predictable.
Please check out the OOPSLA/SCOOL paper on the CCR.
Click here to see how the CCR is being used by the Microsoft Robotics Group.
Please check out the OOPSLA/SCOOL paper on the CCR.
Click here to see how the CCR is being used by the Microsoft Robotics Group.
Comments Closed
Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums,
or
Contact Us and let us know.
More episodes in this show
-
Surendra Verma: Vista Transactional File System
-
Concurrency and Coordination Runtime
-
Vista Audio Stack and API
Follow the Discussion
Oops, something didn't work.
What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in. You need to be signed in to Channel 9 to use this feature.What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in and view them all on your notifications page.sign up for email notifications?
Also, I think you should change the title to "Shat, I'm deadlocked"
I like this setup of an interviewer & seperate camera guy; keep up the good work.
Thanks for putting out the Going Deep series Charles, I'm looking forward to it.
Sound: good.
Picture: good.
Questions: good.
Subject: interesting and important.
George Chrysanthakopoulos: very convincing.
Conclusion: Heck, I thought I was in the room. I love it.
extremely interesting stuff! and the video was really fun to watch, too! reading through the paper i'm wondering how the port based ccr solution compares to the active/future objects idiom Herb Sutter is thinking about for c++0x/c++cli. did the ccr team and the guys around active<> objectives share some thoughts on the topic or are these two completely independent efforts?
again a worthy installment of going deep! nice work, Charles and Scoble!
best regards,
martin
We have discussed with Herb to see if the CCR can be the execution target for the futures/active work, assuming a native version is written. The CCR right now is only available in managed environment.
Hope Microsoft is still around, and the leader in the industry, when this paradigm shift happens.
EDIT: Just finished watching this video, this thing, combined with the Singulairty OS makes me mouth water at what OS will be out in the next decade.
Where are the bits to download? or is it not released to the public yet?
I'm excited about this.. release it
Great video!
We need the effective concurrent environments!
The free lunch is almost over, even though indium antimonide may realize 6GHz CPU.
Intel aims for faster, cooler transistors
http://news.com.com/2100-1008_3-5984635.html
Looks like we may need to wait 10 years though for that technology. Looks like concurrency is the only option for now.
Stephen.
I think this library could be very helpfull, however I can't see the SDK download is it out yet? Also could you elaborate into more details (perhaps with some code or UML) on the futures/ActiveObject pattern?
Thanks and keep up the great work!
Wow! Really Really Smart guys. They were/are light years ahead of me. It's like their brains are working at mach 5 speed and their mouths are only going 60 miles an hour. Most of this content went over my head but its super sexy stuff to hear about.
And I loved how robert stayed quiet and did an awesome job while filming.....but the best part was the number of time I heard Charles say "Fantanstic". I stopped counting at 10. Looks like I've stumbled onto the new buzz word around concurrency...."It's Fantastic" should be the tag line!
Keep up the great work and showing us 9ers these really smart people and their sexy forward looking technologies
See, I'm trying to not do my stupid laugh so much.
Not to mention I was way in over my head too. Those guys are freaky smart.
*snif* I always liked it.
I only started learning about CSPs yesterday, and now I see this - I guess I won't need to rewrite JCSP in C# after all
Of course, I could be way off... any comments?
(For more info about CSPs, see the links I've included in my blog entry on these topics.)
Jon
As soon as there is an SDK available, we will let you know. In the meantime, read the paper by George and Satnam, watch the video a few times, get yourself ready. There's a fair amount to get under your belt in terms of understanding what this is and how/when to use it.
C
PS: Scoble generally keeps quiet and does great camera work for Going Deeps. But, he'll always get a question or two in, which is good! I do think his antics are best served when he's interviewing product team PMs, etc
Given the amount of research being done on the CLR, one can probably assume that somewhere around 2010, Windows will become a managed OS built using the work Microsoft Research is doing today - CCR, Comega, Singularity etc.
Check out the Wiki linked above for examples on how we used Iterators to make code look like "blocking" calls even if its full asynchronous.
Iterators in CCR are used in a very interesting way that allows you to even do loops!
foreach(Message m in MyOutboundMessages)
{
Success successResult = null;
UIPort.post(m);
yield return
m.Response.with(delegate(Success s) {successResult = s;}))
|
m.Response.with(delegate(Failure F) {}));
// this line will execute after *either branch above, but with no
// thread blocking. We just wake up here when a response
// happens!
if (successResult != null)
Console.WriteLine(successResult.ToString());
else
Console.WriteLine("Failed UI message result");
}
CSP has a single-entry, single-exit paradigm, requiring proper nesting of concurrent code in fork/join pairs.
The pi-calculus and join-calculus work which inspired the CCR has no such requirement.
One good thing about the CCR is that lines up well with distributed solutions that use message passing.
While CCR has things like interleave which can easily protect access to shared data structures, a version based scheme, in some cases, will give you better results.
Example:
// we have two methods that access some shared state. Unlike locks
// i dont need to know what that state is in the Ccr. This way
// you dont deadlock by grabing locks in the wrong order, etc
activate(
exclusive(!MainPort.with(UpdateHandler)))
^
concurrent(!MainPort.with(ReaderHandler)));
in the snippet above CCR will run as many instances of ReaderHandler as messages posted in the MainPort,as long as no instance of MyUpdate Handler is running. It will stop scheduling concurrent handlers if a exclusive handler gets queued. This is like a writer biased reader/writer "lock". In general we use things like interleave to get incredible concurrency (millions of work items).
A version of the CCR with STM support could do this:
activate(
!MainPort.withatomic(UpdateHandler)
|
!MainPort.with(ReaderHandler));
Other solutions to concurrency are data level parallelism libraries
The joins in Comega are declarations which makes them rather static. The joins in CCR are statements and they can be dynamically constructed which makes them more flexible.
One thing I need to mention is that Join, choice etc are not rooted in the Join or Pi calculus. These are primitives OS developers and application developers have used for a very long time. In win32 they have names such WaitForMultipleObjects and the semantics are very similar, but on event sychronization objects not queues, like we do.
All these deep CS stuff is not very approachable to most people and from the start my motivation was to improve, using a common, small set of primitives, alot of the existing OS synchronization APIs, not encode something found in theory.
Hope this makes sense.
Where can I download CCR from?
I'm wondering what the debugging story is like?
Unlike stack based debugging which is little to no help here (since the thread you wake up is arbitrary and the stack frames above you are just the scheduler) you need something thet logically relates your messages and the handlers which wake up to service them. This creates an execution tree you can follow or set breakpoints on messages. Yes breakpoints on messages so no matter who picks them, you break there. In the CCR, when you post to a port, you dont know who is listening (part of the cool loose coupling we give you) but that is a nightmare for regular debugging, since you cant "step-into". Well, with our message based debugging, you can "step-into" the receiver of the message and then any others receivers that fire because of messages logically related.
Its sounds a bit fuzzy right now, but when the CCR bits become available for download as a preview (first half of next year or sooner) a debugging story will go with it
Thx, and let's hope it'll come out during the month of December
I am a little surprised, though, that method names (e.g., Port.post) do not follow .NET Class Library Guidelines; do Krzysztof Cwalina and Brad Abrams know?
Cheers,
Robert
And if the CCR does take care of it, are there ways to specify how you want that work distributed? Meaning do you have the ability to say I want thread A and B to run on processor 1 and thread C, D, E to run on processor 2?
Btw, if people read the wiki, although by far not an exhaustive reference it allows you to give feedback on the syntax we have chosen. We welcome all feedback. Underneath the CCR has a concistent design where all arbiters, receivers are schedulable tasks.
If you dont like the operators, you can always create directly the arbiter and schedule it. Example (with operators first):
// create a port with two queues, one for each message type
Port<SuccessMessage,FailureMessage> Response = new Port<SuccessMessage,FailureMessage>();
pFile.post(new Read(Response));
// attach a choice with success or failure handlers
activate(Response.with(Success) | Response.with(Failure));
Now instead, lets create the choice directly:
// the last line above is the same as this:
_pDispatcher.post(
new Choice(
Response.with(Success),
Response.with(Failure)
));
The activate() api is of a base class that has a default dispatcher set. Programs can choose to create different dispatcher to interop well with STA threaded legacy apis or for more explicit load balancing. But they key is that the simple, consistent treatment of everything as a scheludable task, allows you construct things the way you want, in an imperative manner, and then issue them as commands.
Short answer: Yap we take care of it. And yes, if you want to, you can specify in more detail how the scheduling will occur.
Long answer:
The CCR takes care of automatically distributing work across processors. All you do is associate receivers with ports, constrainted under some arbiter (choice, one-time/re-issue receive, interleave etc) and we will generate work items when messages arrive, and schedule them across task queues with some number of OS threads in the back end picking them apart. Usually we use 2 threads per processor but you can set that number.
We can achieve near perfect load balancing across heterogenous tasks and keep all your CPUs hapilly busy. A CCR program just needs one instance of a dispatcher process, and if you set the number of threads to use to 0, if you want us to choose how many threads per processor to utilize in the back end.
Dispatcher creation:
DispatcherPort MyDispatcher =
Dispatcher.Create(0,"FriendlyName");
// now we can use it to associate a piece of code
// that will execute in one of the CPUs, when message arrives
MyDispatcher.post(!MyMainPort.with(delegate(DoWork W)
{
// do work here. If you get N DoWork messages, we will
// run as many versions of this code concurrently as your CPUs!
}));
// you can also set the dispatcher as field in a base class
class MyFirstCcrProgram : CcrServiceBase
{
public MyFirstCcrProgram(DispatcherPort Dispatcher):
base (Dispatcher) {}
void Init()
{
// because we associated a dispatcher
// when we were constructed, we can just
// use the base member activate()
activate(!MyMainPort.with(delegate(DoWork W)
{
// do work here. If you get N DoWork messages,
// we will run as many versions of this code
// concurrently as your CPUs!
}));
}
}
If you want greater control you can create multiple dispatchers, and then each item you want to activate a receiver, use a particular receivers. This is very powerfull and unlike global threadpool schemes: By being able to associate a dispatcher per receiver, you can interoperate very well with Single-Threaded-Apartment components, like COM, but your CCR code does not look any different!
We will give you at least one simple "adaptor", used for UI, where you post messages on ports, but using its own dispatcher, with just one thread, it does Invoke in the back, on your behalf, toa winform.
All this helps you go the next level, if you wish, in controlling execution, scheduling etc and interoperating with the OS.
Developing dynamic analyses for message passing programs is of course important (e.g. run-time debugging). However, I feel that for message passing programs that is not going to be enough. We will also need the help of more powerful static analyses to help catch problems like deadlocks. There is some interesting work involving contracts and behavioral types which looks like a promising approach. For exampe: http://www.research.microsoft.com/behave/index.htm
I too wish to some day find something that i can be that passionate about.
Cheers,
LB
Like everyone else, I am itching to get my hands on the library and try out some stuff. It looks like it will solve some very specific problems for me.
On a related topic, this prompted me to check out the latest C# threading books to see if there's anything good out there. Scoping Amazon and B&N only provided the "C# Threading Handbook" by Tobin Titus, et al. Might have the content, but doesn't get a good rating (plus it is about $80 on Amazon for a 300 page book). Anyone got a better suggestion for sources on doing it "old style"?
Glad you liked the video!
Can you elaborate more on what you're looking for? Threading is a pretty broad topic. What problem are you trying to solve right now? For example, are you developing a massively parellelized, loosely-coupled, distributed system?
C
I mean today you have the thread-per-socket method, which allows easier programming and correctness verify, but does not scale much beyond 1700+- threads. The async method scales better, but is a nightmare to program and reason about correctness. Even using a thread pool, you have same kind of issue with managing state as the pure async play. Would this offer some kind of new pattern or middle ground where it would feel like a thread per user model, but actually be async in terms of threads? Any simple server example to show idea? Thanks guys, this is great stuff!
--
William Stacey [MVP]
A server example is actually given in the wiki and in the examples above but i will show one here. Its so concise, its easy to miss:
class MyHttpServer:CcrServiceBase
{
Port<HttpRequest> MainPort = new
Port<HttpRequest>();
void Init()
{
// assume you have some api wrapper around http listener
// binding a CCR port to listener requests
Bind(MainPort,"MyUri/");
// now activate an iterator handler that gets invoked
// whenever a http request comes in.
// notice "!" infront which makes this handler stick around
activate(!MainPort.iteratewith(RequestHandler);
}
IEnumerator<ITask> RequestHandler(HttpRequest httpReq)
{
byte [] fileData = null;
// lets say you need to two things for every request that
// comes in: Read from a file and send a log update.
// again we have a simple stream wrapper around a file
FileReadRequest fr = new FileReadRequest();
LogUpdate log = new LogUpdate();
FileService.post(fr);
LogService.post(log);
yield return join<byte [],bool>(fr.Response,log.Response).
with(delegate (byte [] data, bool logDone)
{
// export a variable to outer scope
fileData = Data;
})
|
fr.Response.with(Exception Ex) { // error })
);
if (fileData == null)
// fail request, deal with error. Notice how the delegate
// inside the yield above can touch stack variables
// in the scope of this routine!
// this line executes after the yield above but without ever
// blocking a thread. We get logically sequential
// behavior but without wasting a thread.
// use the data here and return result.
httpReq.post(fileData);
}
}
I have an immediate need for more threading info and a future need for this concurrancy and coordination runtime.
In the near term, I'd like to be better at threading with common related ASP.NET patterns or functions. Such as: file access and update coordination, access and update of memory-cached single database objects, scheduled process coordination, and so on. I'm expecting to discover more uses with better skills in this area, especially for the AJAX type processing or other things I haven't encountered yet as a deliverable.
In the future, I'd like to revisit some past ideas on physical system simulation that permit calculations without the level of explicit synchronization that has been required in the past. The CCR seems to be a great fit for that. If this can lead to easier deployment to calculation on arbitrary processor/network configuration, that would be great. In fact, there are some natural network problems that would really benefit (think simulation of electrical distribution networks. Or flooding analysis on major rivers.). So simulate a network on a network (a statement only a geek would appreciate!).
Any pointers on either topic would be appreciated. You can email me direct at Walt AT NOSPAM Lounsbery.com. Thanks!
Larry Osterman had an excellent series of blog posts about concurrency. Here's his wrap-up of the series, with links and descriptions of all entries:
http://blogs.msdn.com/larryosterman/archive/2005/03/10/392258.aspx
Thanks! Great link to Osterman's stuff!
Walt
g
hmm. Nice. Thank you sir. If you ever get time, maybe you could point/post a simple message stream loop server to handle multiple users (i.e. sockets) with open connections and sending/receiving messages in a receive loop for each client. I am missing it, but would seem to still be bound by 1700+- user connections as each active connection would take at least one main thread. Or am I wrong? Thanks much!!!
--
wjs
Staceyw, you will actually be able to service alot more than the number of OS threads in 32 bit systems, if you use asynchronous socket APIs, even from C#. The underlying OS will use overlapped I/O and will not block a thread per connection. Then, the total number of simultaneous requests you can service will not be bound by OS threads. The key ofcourse is to use asynchronous APIs all the way through, so you dont consume a CCR or C# threadpool worker. YOur actuall throughput might not increase btw, but your system will be able to service hundreds of thousands of packets, independent of each other so appear more responsive.
, and it uses the C# asynchronous socket calls underneath. We just had to wrap them with CCR ports
and simple processes so we did not have to deal with Begin/End.
See one of the examples of iterators in this thread or the wiki.We actually have implemented a CCR based transport that does this (but cant post the code, yet
The CCR, using iterators allows you to get "blocking" behavior so your code does not look like the spaghetti of continuations we find ourselves writing again and again. You can actually encode a while loop , processing packets but with no blocking! One of the examples in the wiki shows a for loop using iterators to encode asyncronous but logically sequentially req/rsp operations. It keeps it all in one routine.
So really what we are doing for you, is allow you to use the asynchronous OS APIs, through a much more readable front end, to get the logical behavior of threads, but without the overhead and limitations.
When threads become super lightweight one day( if ever), then the CCR can use alot more of them and get rid of iterators, but its syntax will not really change. The CCR can have blocking arbiters (join, choice etc)
hope this helps.
Any idea when we can play with some bits? TIA
Is there anything similar out there that we might also want to take a look at, that is downloadable now?
The scenario I have right now is, I am building a product that uses a lot of multithreading. It is a smart client application that is using the Smart Client Offline Application Block along with a number of other threads handling various tasks like incremental synchronization of data from the offline block, user interface updating, etc. I also have a number of singleton classes that act as gateways into different core functionality. I'm using the lock statement in C# in my singleton classes to control access to public property setters from multiple threads. I'm looking for ways to improve performance in some areas.
One issue I'm working on right now is, there are various bits of user interface information that get updated from multiple threads. I'm using Control.BeginInvoke etc. to manage the user interface updates when some of this data changes (for instance, wireless signal strength, online/offline status, data synchronization status information, etc.). I'm working on solving some cases when it seems there is contention to update the UI on the UI thread and it causes "pauses" that are noticeable when mousing over toolbar icons that hot track for instance (I have a XAML toolbar where the icons "zoom" 50% when moused over).
Are these types of scenarios things that the CCR can help me with?
The Win32 and Winforms UI model is a bit hard to get right however because of its built in SingleThreadApartment design and various re-entrancy issues. Right now i would recommend you look at the latest PDC talks on concurrency, especially the talk from Jan Gray and Joe Duffy on CLR concurrency best practices. Also look into the background worker CLR 2.0 feature that helps with UI concurrency.
I too am looking forward. I might approach it this way:
Instead of updating UI all over the place, I would have a focus point.
I clean pattern I have been using is a Blocking queue. So each thread that needs to update the UI will jusend a message to the queue. Your UI thread can't be the consumer as you UI will block, so have one UI helper thread that is the single consumer of the queue. It just pops the queue and calls BeginInvoke to update UI controls depending on the Message type. So you can have any number of producer threads all pushing the single queue - all they need is a ref to the queue. It is also a single location to signal all producers to shutdown (i.e. stop the queue). You also have a single place to "throttle" the speed of UI updates with Thread.Sleep(). You can check my BlockingQueue<T> and samples in the Sandbox. Cheers!
--William
There are hacker forces trying to break into your machine right at this very moment in order to temporarily lease your CPU and other resources on the black market. It sounds like science fiction but it isn't -- the black market is merely ahead of legitimate business models, something akin to big pharma selling pot, which they already do.
Eventually this will be the way we "buy" computers. We'll rent computer time on machines that we never see. I think that was the idea behind the Chi Corporation in the late 60s...
Anyway, stay tuned, Jeff is working with us, giving good feedback, and in the future help with articles on the CCR.
I have to admit this looks amazing and is exactly what we need on my current project. I currently have a distributed server system that accesses data locally and on other machines, but I'm having difficulty adding parallelism to the system. I believe this would allow me to easily add this if I can massage the dispatcher to not only load balance across processors but across machines.
Georgio do you see this as being already built in or at least a workable implementation on my part? I already have a running system (in .net 2.0) capable of handling thousands of requests a second, and I believe I can easily shoehorn this into it. We are still months away from going live so hopefully the timing will be workable on my part to use this.
So i see no reason why you cant use this in your system.
I really want to get this out there, and have people play with it. As i said, in a couple of months you can play with the CCR, plus a bunch more that has been built on top of it
g
Hi! I have been playing with the CCR and I think it takes away some of the pain of working with concurrency, which is very good!
However I'm a little confused of how should one program with the CCR
For instance, is it correct to send ports to ITasks and post from within them (like in the following example)?
static void HandlerPorts(Port<int> pInts)
{
pInts.Post(42);
}
static void Main(string[] args)
{
using (Dispatcher dispatcher = new Dispatcher(0, "CcrDispatcher"))
{
DispatcherQueue queue = dispatcher.AddQueue("MainQueue");
Port<int> pInts = new Port<int>();
Port<Port<int>> pPorts = new Port<Port<int>>();
Arbiter.Activate(queue, Arbiter.Receive<int>(true, pInts,
delegate(int value){
Console.Out.WriteLine(value);
}
));
for(int i=0; i<2; i++){
Arbiter.Activate(queue, Arbiter.Receive<Port<int>>(false, pPorts, HandlerPorts));
pPorts.Post(pInts);
}
System.Threading.Thread.Sleep(1000);
}
}
This seems to work as I want, printing twice the value 42. But am I messing with the CLR in any way I shouldn't?
Also, is the API defined anywhere? I've heard that there are some quite neat ways of using the CLR (operator overloading, like the | for choice).
Thanks and keep up the good work!
Nuno
The wiki has the latest API.
Hi again!
Can the Ports, Queues and Dispatchers be used as "globals" (class variables, for instance) from within the tasks?
Is it possible to extend a Port with a different strategy (like a LIFO or a priority queue instead of a FIFO)?
How are the objects passed by the ports to the tasks? By reference or copied?
Thanks!
Yes ports can be extended. You just implement the appropriate interfaces or easier, derive from Port<T>. We used to have LIFO, random insert implementations but again and again FIFO became the norm due to to starving issues related to LIFO and the distruptive attributes non-FIFO methods have on streaming or preserving sequence of events.
messages stored on ports, are by reference. We have extended some of our Arbiters (for example we have a variant of Receiver<T>) that deep clone objects before waking up the user task+ delegate associated with the port. The CCR allows you to do this type of extensies trivially and in a manner that the sender (the process that Posts) never has to know about. As i mentioned earlier we have a distributed system built on top of it
Great! Those features really do simplify the implementation of our app.
What I meant in the the first question was if the Ports are threadsafe: for instance, can I post to them from different CCR Tasks, eventually running simultaneously, without having to worry about race conditions in the Port's object, right?
July 9th you will also get a MSDN article ont he CCR with sample code using it in various scenarios.
The CCR, even if its a library, actually manages all execution, replacing the CLR thread pool or any manual threading code. Underneath ofcourse it uses the OS+CLR.
The CCR manages messages on ports, task load balancing and execution, iterator task stepping, dispatcher lifetime etc. So its somewhere between a library and an execution environment. If you check out our robotics video, it might make it a little bit more clear why the CCR is a fundemental piece of our execution environment
http://channel9.msdn.com/Showpost.aspx?postid=206574
Thanks in advance.
Greets,
Bas Bossink
Hi everyone! You are welcome to try my commit to the community and play with my framework samples at http://plugins.codeplex.com/">http://plugins.codeplex.com
Sincerely yours,
hack2root
Remove this comment
Remove this thread
close