Comment Feed for CCR Programming - Jeffrey Richter and George Chrysanthakopoulos (Going Deep on Channel 9)

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

sukumar — Wed, 04 Jul 2007 14:21:48 GMT

Hai ,

This is sukumar . i am very new is this site. i want to implement CCR concept in my project . i gone through this video file(http://channel9.msdn.com/ShowPost.aspx?PostID=143582) but i didnt get properly .if u have any document abt CCR plz send to my inbox. its help for my team .

my mail id : honeysukumar@yahoo.com

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

DonSch — Fri, 16 Mar 2007 14:26:57 GMT

Thanks George - I appreciate the quick reply.. I'll post any further questions on the newsgroup. It would in fact be quite helpful to have a subsection specific to CCR. The application I'm doing a 'proof-of-concept' for is within the context of asycnhronous web services (asp.net/wse3.0), as well as inwith a number of windows services and it looks very promising! :)

Don

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Fri, 16 Mar 2007 01:41:52 GMT

DonSch wrote:

George,

I'm wondering if you could comment on the recommended use of ThreadCausalities in CCR (in my case non-DSS environment). I can see that Causalities will be transferred from the current thread to the post operation, and that they are conversely transferred from the ITask back to the executing thread, making the context available to delegates as called from Receivers, etc.

The obvious use is to provide an ExceptionPort that can take care of any unhandled exceptions that might have been thrown during events, etc. I have not really seen too many examples, nor is is clear what the appropriate patterns of use really are..

Possible example: I'm inside an iterator doing a number of yield return operations, etc, to Choices, etc. and may want that iterator to have a way to handle any unhandled exceptions during those operations, so I add a CausalityContext within that iterator.

Then perhaps one of the services that I do a yield return call to (it returning a Choice) decides that it needs a 'nested' causality, so it creates one for it's exception handling (of unhandled exceptions in the dispatcher chain).. Once that service posts back thru the Choice, it appears that the original Iterator has it's restored CausalityContext in place again.

Any insight on the appropriate usage patterns would really be appreciated. Also the intent of the CoordinationPort?

The CCR appears to be an extremely useful library for a vast number off cases and I really appreciate your time in filling in the gaps!

tnx again!
Don

Hi Don, we actually use causalities, for exactly the same reasons you guessed :) Its a generalization, across threads and processors, of the nested exception handling mechanism. Much more powerfull however since it can deal with joins (two different causalities coming together in a common handler). And no, they are not transactions ;)

Causalities are used for every single operation in the DSS runtime, our services infrastructure. They make sure that when an exception is thrown, within any logically related descendant of a Post operation, that exception can be cleanly caught and dealt with in one place.

The CoordinationPort allows anyone within a causality to post a message to the "owner" of the causlity, allowing you to implement all kinds of protocols (like N phase commit, etc)

Please post any further questions in the robotics newsgroup since thats the one we acively monitor. Soon we will have a forum in MSDN where we can talk about CCR stuff exclusively (it will be under the Robotics family of forums).

For examples, i have posted something in the past in our robotics newsgroup on this (search for Causalities, Causality)

btw we always want to know how the CCR is used and applied, especially in large scale commercial applications.

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

DonSch — Fri, 16 Mar 2007 00:10:11 GMT

George,

I'm wondering if you could comment on the recommended use of ThreadCausalities in CCR (in my case non-DSS environment). I can see that Causalities will be transferred from the current thread to the post operation, and that they are conversely transferred from the ITask back to the executing thread, making the context available to delegates as called from Receivers, etc.

The obvious use is to provide an ExceptionPort that can take care of any unhandled exceptions that might have been thrown during events, etc. I have not really seen too many examples, nor is is clear what the appropriate patterns of use really are..

Possible example: I'm inside an iterator doing a number of yield return operations, etc, to Choices, etc. and may want that iterator to have a way to handle any unhandled exceptions during those operations, so I add a CausalityContext within that iterator.

Then perhaps one of the services that I do a yield return call to (it returning a Choice) decides that it needs a 'nested' causality, so it creates one for it's exception handling (of unhandled exceptions in the dispatcher chain).. Once that service posts back thru the Choice, it appears that the original Iterator has it's restored CausalityContext in place again.

Any insight on the appropriate usage patterns would really be appreciated. Also the intent of the CoordinationPort?

The CCR appears to be an extremely useful library for a vast number off cases and I really appreciate your time in filling in the gaps!

tnx again!
Don

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

raghur — Tue, 26 Dec 2006 20:11:55 GMT

Hi,

How does CCR compare to DataRush from Pervasive in Java land (http://www.pervasivedatarush.com/technology).

Are these two in similar concept?

Thanks.
Raghu/..

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Wed, 13 Dec 2006 15:00:06 GMT

Microsoft Robotics Studio version 1.0 and with it the CCR library have released!

Please visit www.microsoft.com/robotics for the latest bits.

Note on the licensing: CCR, which is part of the MSRS runtime is covered by the same license as the robotics runtime. For non-commercial use (please read the license for more details) its use is free. For commercial use however there is a fee (good news is that with one MSRS license you get permission for multiple runtime instances)

For any questions please refer to the license and also feel free to post to the newsgroups.

thanks to all for all the great feedback

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Mon, 04 Dec 2006 16:05:47 GMT

dtspence wrote:

George,

Thanks for the reply. Your suggestion worked great. I have another related question.

Could you explain what the runtime does with the return type of IEnumerator<ITask> a little? The runtime seemed to execute the first task, but not the others.

Thanks.

.....

The C# compliler re-writes all functions that return IENumerator to look like a heap allocated class that implements a state machine. Every time you yield, the state machine is updated, and the CCR uses that to do MoveNext when the constraint (for a example a receive occuring) is satisfied. Basically the CCR figured out how to use iterator for async scheduling.
Now, for all this to happen, you must implement the proper Task which can support linked iterators. This is what choice, Receive, Join etc have to do. Its not enough to just derive from task, since you cant yield to a simple task: Its not clear when a task is "done". To do this, you will need to support the LinkedIterator property plus some other fields.

Its a bit of an advanced subject that i think should be covered by a new article or even book on the ccr :)

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

dtspence — Mon, 04 Dec 2006 13:54:06 GMT

George,

Thanks for the reply. Your suggestion worked great. I have another related question.

Could you explain what the runtime does with the return type of IEnumerator<ITask> a little? The runtime seemed to execute the first task, but not the others.

Thanks.

georgioc wrote:

dtspence wrote:
George,

I am trying to create an alternate receiver that would send a message to all predicates and applicable handlers, versus stopping on a single Predicate match.

There is a problem with my implementation not getting a second pass through the EvaluatorAggregateTask's Execute method after the first yield return. I also tried to return an alternate IEnumerator<> from a List<> and received the same behavior.

I was able to get all the EvaluatorTask<> enqueued by calling the DispatchQueue.Enqueue, not sure if this is correct. Do you know if the framework should be executing all of the tasks returned back in the enumerator? Also please mention anything I look at doing differently.

Thanks.

Code snippets as follows:

public class TestReceiver<T>
    : Receiver
{
     public TestReceiver(bool persist, IPortReceive port,
          params Tuple<Predicate<T>, Handler<T>>[] tuples)
       : base(true, port, new EvaluatorAggregateTask<T>(tuples))
     {
          // no code
     }
}

internal class EvaluatorAggregateTask<V>
     : ITask
{
      public IEnumerator<ITask> Execute()
      {
          foreach (Tuple<Predicate<V>, Handler<V>> tuple in this.tuples)
          {
              Predicate<V> predicate = (Predicate<V>)tuple.Item0;
              Handler<V> handler = (Handler<V>)tuple.Item1;
              yield return new EvaluatorTask<V>((PortElement<V>)this[0],
                     handler, predicate);
          }
      }

   [Rest of ITask implementation here ... ]
}

internal class EvaluatorTask<V>
     : ITask
{
      public IEnumerator<ITask> Execute()
      {
          if (this.predicate(this.param.TypedItem) == true)
          {
              this.handler(this.param.TypedItem);
          }
          return null;
      }

     [Rest of ITask implementation here ... ]
}

Hi Tasks never get their execute method called more than once. Persistent receivers even, will clone the task that runs the handler, and call it once for each message.

To achieve what you want, there is a different approach. Create the alternate receiver, just like you did. In the receiver's Evaluate method (override it) take the IPortElement.Value contents, and essentially broad cast to any other port or handlers you want. Make sure you return true from the Evaluate, so the port does not keep the message in its queue

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

kroki — Thu, 30 Nov 2006 23:45:34 GMT

Thank you,

Kroki

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Thu, 30 Nov 2006 22:44:21 GMT

kroki wrote:

George,

Can you refer me to a library with similar functionality as the CCR, though for unmanaged C++?

Thanks in advance,

Kroki

Hi i am not aware of any such library. A CCR C++ version would be nice but its not in our plans at this point.

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Thu, 30 Nov 2006 22:43:41 GMT

dtspence wrote:

George,

I am trying to create an alternate receiver that would send a message to all predicates and applicable handlers, versus stopping on a single Predicate match.

There is a problem with my implementation not getting a second pass through the EvaluatorAggregateTask's Execute method after the first yield return. I also tried to return an alternate IEnumerator<> from a List<> and received the same behavior.

I was able to get all the EvaluatorTask<> enqueued by calling the DispatchQueue.Enqueue, not sure if this is correct. Do you know if the framework should be executing all of the tasks returned back in the enumerator? Also please mention anything I look at doing differently.

Thanks.

Code snippets as follows:

public class TestReceiver<T>
    : Receiver
{
     public TestReceiver(bool persist, IPortReceive port,
          params Tuple<Predicate<T>, Handler<T>>[] tuples)
       : base(true, port, new EvaluatorAggregateTask<T>(tuples))
     {
          // no code
     }
}

internal class EvaluatorAggregateTask<V>
     : ITask
{
      public IEnumerator<ITask> Execute()
      {
          foreach (Tuple<Predicate<V>, Handler<V>> tuple in this.tuples)
          {
              Predicate<V> predicate = (Predicate<V>)tuple.Item0;
              Handler<V> handler = (Handler<V>)tuple.Item1;
              yield return new EvaluatorTask<V>((PortElement<V>)this[0],
                     handler, predicate);
          }
      }

   [Rest of ITask implementation here ... ]
}

internal class EvaluatorTask<V>
     : ITask
{
      public IEnumerator<ITask> Execute()
      {
          if (this.predicate(this.param.TypedItem) == true)
          {
              this.handler(this.param.TypedItem);
          }
          return null;
      }

     [Rest of ITask implementation here ... ]
}

Hi Tasks never get their execute method called more than once. Persistent receivers even, will clone the task that runs the handler, and call it once for each message.

To achieve what you want, there is a different approach. Create the alternate receiver, just like you did. In the receiver's Evaluate method (override it) take the IPortElement.Value contents, and essentially broad cast to any other port or handlers you want. Make sure you return true from the Evaluate, so the port does not keep the message in its queue

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

kroki — Thu, 30 Nov 2006 15:34:09 GMT

George,

Can you refer me to a library with similar functionality as the CCR, though for unmanaged C++?

Thanks in advance,

Kroki

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

dtspence — Thu, 30 Nov 2006 13:41:59 GMT

George,

I am trying to create an alternate receiver that would send a message to all predicates and applicable handlers, versus stopping on a single Predicate match.

There is a problem with my implementation not getting a second pass through the EvaluatorAggregateTask's Execute method after the first yield return. I also tried to return an alternate IEnumerator<> from a List<> and received the same behavior.

I was able to get all the EvaluatorTask<> enqueued by calling the DispatchQueue.Enqueue, not sure if this is correct. Do you know if the framework should be executing all of the tasks returned back in the enumerator? Also please mention anything I look at doing differently.

Thanks.

Code snippets as follows:

public class TestReceiver<T>
    : Receiver
{
     public TestReceiver(bool persist, IPortReceive port,
          params Tuple<Predicate<T>, Handler<T>>[] tuples)
       : base(true, port, new EvaluatorAggregateTask<T>(tuples))
     {
          // no code
     }
}

internal class EvaluatorAggregateTask<V>
     : ITask
{
      public IEnumerator<ITask> Execute()
      {
          foreach (Tuple<Predicate<V>, Handler<V>> tuple in this.tuples)
          {
              Predicate<V> predicate = (Predicate<V>)tuple.Item0;
              Handler<V> handler = (Handler<V>)tuple.Item1;
              yield return new EvaluatorTask<V>((PortElement<V>)this[0],
                     handler, predicate);
          }
      }

   [Rest of ITask implementation here ... ]
}

internal class EvaluatorTask<V>
     : ITask
{
      public IEnumerator<ITask> Execute()
      {
          if (this.predicate(this.param.TypedItem) == true)
          {
              this.handler(this.param.TypedItem);
          }
          return null;
      }

     [Rest of ITask implementation here ... ]
}

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Tue, 14 Nov 2006 03:13:04 GMT

An update on the FIFO issue and Interleave that was raised by staceyw in this forum. I have found and fixed the issue causing this and it turns out interleave should have worked FIFO for exclusive receivers all along.
See more details here:
http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.msroboticsstudio&mid=618d9b8a-7194-4185-8d89-e8fff887395b&sloc=en-us

Re: Usage of UseBackgroundThreads

georgioc — Mon, 13 Nov 2006 05:28:24 GMT

enelson wrote:

Hi George,

Dan is a coworker of mine and posted this after I commented about that.

We don't necessarily have a situation where we NEED the ability to have different settings, but we do have a situation where changing the value could affect code in different libraries, and I felt that was a Bad Thing (tm).

Basically, CCR has caught on like wildfire around here, so we have multiple libraries in the same AppDomain using it simultaneously. I personally have a component based service where the service itself has a Dispatcher for message distribution and then one the components has its own for part of its message processing tasks.

Since I happen to have written both the service and this component, it's not much of a concern. But other developers will be writing components for the service, and it feels programmatically rude to change that static value and have it affect their instantiations. It's not a huge deal to save the old value and change it back, but it is kind of ugly

Thanks for the wonderful library! This is a situation that could only arise from us loving it so much.

First of all i am glad you are loving it so much :)

Yes, indeed it does feel wierd to have a static for this. However remember that the background thread setting only makes sense if its global: If its not, its effect on application shutdown is eliminated. Even one dispatcher with foreground threads would keep the app from exiting.

May I ask why is the value being touched? Is it to control application exit behavior or to simply request background threads for the benefits they provide (better cpu sharing with other apps etc)

Re: Usage of UseBackgroundThreads

enelson — Fri, 10 Nov 2006 08:52:10 GMT

Hi George,

Dan is a coworker of mine and posted this after I commented about that.

We don't necessarily have a situation where we NEED the ability to have different settings, but we do have a situation where changing the value could affect code in different libraries, and I felt that was a Bad Thing (tm).

Basically, CCR has caught on like wildfire around here, so we have multiple libraries in the same AppDomain using it simultaneously. I personally have a component based service where the service itself has a Dispatcher for message distribution and then one the components has its own for part of its message processing tasks.

Since I happen to have written both the service and this component, it's not much of a concern. But other developers will be writing components for the service, and it feels programmatically rude to change that static value and have it affect their instantiations. It's not a huge deal to save the old value and change it back, but it is kind of ugly :s

Thanks for the wonderful library! This is a situation that could only arise from us loving it so much.

Re: Usage of UseBackgroundThreads

georgioc — Thu, 02 Nov 2006 16:47:54 GMT

dfarino wrote:

Hi George,

I have a comment about the UseBackgroundThreads static property on Dispatcher.

It looks like you can set this property and its value is used during the creation of a Dispatcher instance. It feels to me like it should be a contructor parameter instead of a static property.

The reason I ask is that we have CCR code (running in the same AppDomain) that was written by several developers. It seems like having to do something like this introduces a race condition:

bool old = Dispatcher.UseBackgroundThreads;
Dispatcher.UseBackgroundThreads = true;
Dispatcher d = new Dispatcher();
Dispatcher.UseBackgroundThreads = old;

Am I missing the reason for this design decision?

Thanks,
Dan

I decided to make it a static since it needs to be applied to all dispatchers to have the desried effect on the app domain. If even one dispatcher is created with a different setting, then app domain will not exit/exit as expected. Its not meant to be change dmultiple time.

Now what this means is that it should probably be static single assignment, aka write once, read many. The other option is to let people manage tis, like you suggest and make it a constructor parameter.

Can you describe your scenario where different dispatcher need different settings, within one appdomain?

Usage of UseBackgroundThreads

dfarino — Wed, 01 Nov 2006 20:38:08 GMT

Hi George,

I have a comment about the UseBackgroundThreads static property on Dispatcher.

It looks like you can set this property and its value is used during the creation of a Dispatcher instance. It feels to me like it should be a contructor parameter instead of a static property.

The reason I ask is that we have CCR code (running in the same AppDomain) that was written by several developers. It seems like having to do something like this introduces a race condition:

bool old = Dispatcher.UseBackgroundThreads;
Dispatcher.UseBackgroundThreads = true;
Dispatcher d = new Dispatcher();
Dispatcher.UseBackgroundThreads = old;

Am I missing the reason for this design decision?

Thanks,
Dan

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Wed, 25 Oct 2006 15:39:04 GMT

tartufella wrote:

Hi George,

My implementation of the CCR is going really well and it's great to see new versions coming out every month.

I have an issue that is niggling at me though and it's to do with the ProcessedTaskCount of a Dispatcher. I have a long running Windows Service that has 1 Dispatcher and multiple Dispatcher Queues. The service is quite busy, so the ProcessedTaskCount goes up quite quickly. Given that ProcessTaskCount is an Int32, it's not unreasonable that over some time I will hit the Int32.MaxValue and the ccr will then bring my AppDomain down. Does this value recycle before it reaches Int32.MaxValue?

Many thanks

Marshall

Hi, we do handle the rolling over case, so the CCR will not affect the appdomain when this happens.

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Fri, 20 Oct 2006 02:54:40 GMT

shmarya wrote:

Hi George,

I have a question:

I am trying to build a method which may be either asynchronous or synchronous, using CCR internally (to the method) to accomplish the asynchronousicity...

So basically I have a method which has two local ports, one for the input message and the other for the returned results.

Everything is fine until I want to block on the receiver for the result port in order to acheive 'synchronous' operation.

How can I make the Activate call block?

Is there a better method for doing this?

Thanks,
Shmarya

Make the parent method an Iterator, spawn it (either enqueue an IterativeTask with some arguments, or post a message to a port with an iterator receiver attached.

In the iterator method, you can do yield return, which is the synchronous version of Activate().

There lots of examples of CCR with iterators in the robotics tutorials and samples. The MSDN article also covers them.

thanx
g

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

tartufella — Tue, 17 Oct 2006 10:47:53 GMT

Hi George,

My implementation of the CCR is going really well and it's great to see new versions coming out every month.

I have an issue that is niggling at me though and it's to do with the ProcessedTaskCount of a Dispatcher. I have a long running Windows Service that has 1 Dispatcher and multiple Dispatcher Queues. The service is quite busy, so the ProcessedTaskCount goes up quite quickly. Given that ProcessTaskCount is an Int32, it's not unreasonable that over some time I will hit the Int32.MaxValue and the ccr will then bring my AppDomain down. Does this value recycle before it reaches Int32.MaxValue?

Many thanks

Marshall

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

shmarya — Tue, 17 Oct 2006 09:23:40 GMT

Hi George,

I have a question:

I am trying to build a method which may be either asynchronous or synchronous, using CCR internally (to the method) to accomplish the asynchronousicity...

So basically I have a method which has two local ports, one for the input message and the other for the returned results.

Everything is fine until I want to block on the receiver for the result port in order to acheive 'synchronous' operation.

How can I make the Activate call block?

Is there a better method for doing this?

Thanks,
Shmarya

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Wed, 11 Oct 2006 16:33:44 GMT

Hi, CCR has some new features relating to throttling and rate based scheduling. They are available in the latest robotics CTP and explained briefly here (towards the bottom, where it talks about ccr/dss):
http://msdn.microsoft.com/robotics/getstarted/ctp4/default.aspx

Basically the ccr dispatcher queue allows you to specify task execution based on constraints. This eliminates alot of manual throttling and increases determinism:

/// <summary>

/// Specifies dispatcher queue task scheduling behavior

/// </summary>

public enum TaskExecutionPolicy

{

/// <summary>

/// Default behavior, all tasks are queued with no constraints

/// </summary>

Unconstrained = 0,

/// <summary>

/// Queue enforces maximum depth (specified at queue creation)

/// and discards tasks enqueued after the limit is reached

/// </summary>

ConstrainQueueDepthDiscardTasks,

/// <summary>

/// Queue enforces maximum depth (specified at queue creation)

/// but does not discard anny tasks. It forces the thread posting any tasks after the limit is reached, to

/// sleep until the queue depth falls below the limit

/// </summary>

ConstrainQueueDepthThrottleExecution,

/// <summary>

/// Queue enforces the rate of task scheduling specified at queue creation

/// and discards tasks enqueued after the current scheduling rate is above the specified rate

/// </summary>

ConstrainSchedulingRateDiscardTasks,

/// <summary>

/// Queue enforces the rate of task scheduling specified at queue creation

/// and forces the thread posting tasks to sleep until the current rate of task scheduling falls below

/// the specified average rate

/// </summary>

ConstrainSchedulingRateThrottleExecution

}

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

georgioc — Sat, 07 Oct 2006 02:47:17 GMT

oddurmagnusson wrote:

This stuff is really intresting. Would it be possible to use this to implement something similiar to stackless python( ironpython + ccr = stackless python in clr, now that would just make my day )

we have CCR + python running, so yes :)
See the python examples in the robotics tutorials (ccr ships as an independent, reusable part of the robotics CTP)

Re: CCR Programming - Jeffrey Richter and George Chrysanthakopoulos

oddurmagnusson — Fri, 06 Oct 2006 18:55:57 GMT

This stuff is really intresting. Would it be possible to use this to implement something similiar to stackless python( ironpython + ccr = stackless python in clr, now that would just make my day )