Jeffrey Richter and his AsyncEnumerator

The Discussion

• 

  This library seems interesting. Can someone comment on similarities/differences to the Parallel.For library?
  
  IIRC, there's no cancelling nor timeout mechanism in Parallel.For, but is that all?
  

• 

  Differences with Parallel.For? One that springs to mind is that an application on current or myabe not-so current .NET technology can leverage this technique.
  
  Top marks, Jeff.

• 

  They are very different. Parallel.For in particular is about performing a bunch of compute-bound operations in parallel scaling out across all the CPUs in teh machine. My AsyncEnumerator is mostly about issuing one or more concurrent I/O-bound operations without have any threads block for them to complete. 

• 

  You guys should look at asynchronous workflows in F#, here is a Channel9 video on it:
  
  https://channel9.msdn.com/posts/Charles/Don-Syme-Whats-new-in-F-Asynchronous-Workflows-and-welcome-to-the-NET-family/
  
  Asynchronous workflows allow you to express asynchronous computations without having to wrap their continuations.

• 

  Couldn't the function use yield return to hand back a lambda, and the lambda would contain the asynchronous call(s) to fire off?
  
  The calls could be made on an interface that wraps the socket and hides the details of managing the asynchronous nature of the call altogether.
  
  Hmm... this sounds like a good idea... I'll whip up something to demonstrate what I mean more clearly.

• 

  Here's what I mean:
  
  http://incrediblejourneysintotheknown.blogspot.com/2008/12/asynchronous-sockets-with-yield-return.html
  
  Would be very interested to hear what Jeffrey thinks about it!

• 

   

   

  Regarding the difference between this and PFX, they are solving different problems. PFX is about providing simple means of parallel execution; Jeff's library is about maximizing thread efficiency by avoiding thread blocking altogether by following the APM model. APM is a requirement when writing programs that:
  (1) have potentially hundreds or thousands of parallel execution paths
  (2) spend most of their time blocking
  Without (1), you won't gain enough through the APM to justify the extra programming effort. Without (2), you won't save anything with the APM. The quintessential application is a sockets server that accepts requests over the net. Because the Internet is slow, requests may take many seconds from end to end, and yet spend most of their time blocked waiting on a slow network. A well-written asynchronous TCP server can handle a thousand concurrent requests while consuming just a few threads.
  
  APM relies on methods returning with a callback rather than blocking the underlying thread. There are actually fairly few truely asycnhronous methods in the .NET Framework - most are related to networking, where the most gains are to be had.
  

   

  Jeff's idea is very clever - but it's dubious using it for retrieving web pages in the background in a Win Forms application. The only thing he's achieving is saving the overhead of tying up one or two threads for a few seconds. There was a time when tying up 1MB of memory for a few seconds was a big deal, but those days are long gone. Using PFX (or asynchronous delegates, which interestingly don't follow the APM) would be simpler and just as effective.
  

   

  By far, the most common use for the async-callback-method model is writing a TCP or HTTP server capable of handling hundreds of concurrent requests. It's NOT required within ASP.NET apps because the ASP.NET infrastructure already does this for you. It's also not needed around making database calls within a web app because the DB server will nearly always be more of a bottleneck than the accompanying thread overhead. And who in real life would give up the productivity of the high-level synchronous DB libraries such as LINQ to SQL and revert to using low-level async-compliant SqlCommand objects and the like?

   

  So if you were writing a large TCP or HTTP server from scratch yourself, Jeff's library may be very helpful. The only problem I can see (unless I'm mistaken) is that you can't refactor iterator methods into smaller chunks - as its complexity grows, the method will get bigger and bigger and there's nothing you can do about it. The problem is that any methods that you call from an iterator can't themselves yield on behalf of the caller. So if you wanted to do this:
  
  foreach (var foo in far)
     AnotherMethod (foo);
  
  then AnotherMethod couldn't do anything asynchronous within its implementation.
  
  Joe
  

• 

  I certainly agree with your comments about server-side code. However, I disagree with your thoughts about client-side code. On my machine Outlook (just to pick some app as an example) has over 50 threads in it because the Outlook developers have the mindset that it isn't bad to just spawn off a few more threads and allocate a few more megabytes of memory. However, this wastes a lot of resources and if Outlook is running on a terminal server machine hosting 100 client sessions, then that means 5,000 threads on the system! In fact, because this problem is so bad, Windows 7 is doing a lot to reduce the number of threads used throughout system services and applications that ship with the OS so that Windows 7 will run well on small footprint machines with only 1GB of RAM in them. Clearly the Windows team feels that all the threads they have been creating has been hurting them and they are finally doing something about it.
  
  And, since I'd like to make things better for Windows users,  I highly recommend asynchrounous programming for client-side applications. Also, delegates DO support the APM via their BeginInvoke/EndInvoke methods and because of this, asynchronous delegate invocations integrate quite nicely with my AsyncEnumerator. In addition, my AsyncEnumerator automatically marshals the result back to the GUI thread so there is no need to call Control's BeginInvoke method (for Windows Forms) or DispatcherObject's BeginIvoke method (for WPF). Furthermore, my AE offers cancelation/timeout support as well as discard support which are all useful features for client-side applications that you do not get with the APM alone or by just spawning up another thread.
  
  ASP.NET does support the ability for developers to implement their web form or web service app asynchronously but few developers take advantage of this. However, ASP.NET offers it as an option because it does not simply give it to you by default. Also, it is very useful to perform async programming when accessing a DB because the DB IS a bottleneck (as you point out). If you make synchronous requests to a DB then your server will create a ton of threads which are all blocked; your server will handle just a few concurrent requests and memory consumption will be very high with thread stacks which your code is not using at all.
  
  In the real world, there are many developers who will avoid high-level synchronous libraries in order to gain scalability. I know that MS does avoid synchronous DB access for Hotmail and many of its other highly-scalable services. However, it is true, that if your service has few concurrent users, then you do get a lot of benefit from various synchronous abstractions (like Linq to SQL). In an ideal world, these abstractions that offer such productivity will offer asynchronous ways of using them. Linq to SQL doesn't offer this today but it could in the future (or Linq to Entities could).
  
  And, as for refactoring...Iterators do offer some challenges here, I agree. However, from an iterator, you can call compute-bound synchronous methods or methods that initiate an asynchronous compute-bound or I/O-bound operation that itself returns an IAsyncResult. And, also my AsyncEnumerator does support composition where an iterator can invoke another iterator. For example, you could create an iterator that asynchronously queries a web server to get some data and process it. And then, you can create another iterator that consumes the first in a loop to do asynchronous processing for several web sites. In fact, all of this processing can happen concurrently allowing for phenominal scaling. If your iterator methods get overwhelming, then you can resort to call back methods which is what all native applications have to do and what developers writing scalable managed application have to do today. Iterators give you a new ability; they don't take away any of the old abilities. This ability has very few drawbacks but if you come across one, then don't use it; go back to an older tried and true way.

• 

  Very very nice. Hats off to Jeff for this.
  

• 

  Hi Jeff
  
  Nice to hear from you!  Regarding Outlook, I agree that spawning 50 threads is way over the top. There's a difference, though, between consuming 50 threads for the life of the application and starting two threads that will run for a second.
  
  I wonder what Outlook is doing to consume that many threads. I never would have imagined that checking one's e-mail was that complicated!
  
  Cheers
  
  Joe
  

• 

  I find this quite useful and it probably does make developing code faster in some circumstances.
  Thanks Jeff
  

• 

  Very clever idea Jeff. I got the feeling from the latest .NET framework that the APM model was being phased out by MS and replaced with the xxxAsync xxxCompleted event pattern. Just look at the code generated for WCF services for an example of this (even though I'm pretty sure it uses the APM pattern under the covers).  My only criticism of your AsyncEnumerator would be how large the iterator methods start to get when you have a fair bit of logic in each stage. As you said you can always switch back to separate event methods, but I'm wondering if there's not a happy medium somewhere in between, perhaps with sub-iterators, or some kind of iterator stack? Just thinking out loud...
  

• 

  Just to say that I am a big fan of Jeff's work, and have been using his AsyncResult and AsyncEnumerator for some time, on a number of different projects.  I use the AE when loading WPF controls to make multiple asycn calls to a DB when fetching various bits of data that the control needs.  Subsequently the control loads up much more quickly, and when all of the data returns, I can update the control, all without having to use the Dispatcher.
  
  I do have one question though Jeff, you used to make the source code for your async library available on Wintellect, but now you only release a compiled library.  Why is this?  I have found that the latest release has caused a breaking change to my code, and so I am still using the version from May.  Can you at least outline what has changed between that version and Octobers release?
  
  Steve

• 

  Have you guys looked at my lirary, which I created in Mid 2006
  
  www.codeproject.com/KB/cs/AsynchronousCodeBlocks.aspx
  
  This library with my accompanying Managed IOCP library were very similar to Microsoft's Parallel FX and Task Library. I wrote these libraries in Mid 2005 through 2006.
  
  Regards,
  Aditya.P
  (Be Original)

• 

  CCR has been using almost idenctical syntax to do what Jeff is showing here for about 4 years (publicly). Jeff was even in some of our videos 3 years ago.
  
  Various channel9 videos and PDC talk have examples... 
  https://channel9.msdn.com/pdc2008/TL55/
  
  https://channel9.msdn.com/tags/CCR/
  
  CCR example for file read (from PDC session):
  
  

  IEnumerator<ITask> CcrReadFileAsync(string file)
  {
      var result = new Port<IAsyncResult>(); 
          using (var fs = new FileStream(file,…,FileOptions.Asynchronous))
          {

              var buf = new byte[fs.Length];
              fs.BeginRead(buf, 0, buf.Length, result.Post, null); 
              yield return result.Receive();
              var ar = (IAsyncResult)resultPort.Test(); 
              try  { 
                  fs.EndRead(ar); 
                  ProcessData(buf); 
              } catch { // handle exception }
        }
  }

• 

  The use of C# iterators for aynchronous programming was first done by the CCR team. This is true. In fact, I remember thinking how outrageously cool it was to use C# iterators in this way. Remember this interview?
  
  In this case, Jeff has extended on the basic functionality present in CCR's implementation of async using C# iterators to include some new and powerful capabilities. Great work, Jeff!
  
  C

• 

  Thank you my dear friend. I like you and your books.............

• 

  With the CCR going commercial, how does this fit. Seems like dirct competition.
  When would you use the CCR versus this?

• 

  Hi Jeffrey,

   

  I wanted to show your video about the AsyncEnumerator to a colleague, but it seems to have disappeared from Channel9...

  Was that a result of a conscious action, or did something go wrong? Is there any other place I can find it? - Peter

• 

  Fixed. Thanks for letting us know!

  C

• 

  This seems to be an implementation of trampoline-style threading in .Net.

  You write your code as a method which periodically gives up control of its calling thread by use of the "yield" statement. And you expect that the scheduler/engine will revive you so that you can continue from where you left off.

   

  This same technique can be used to write multi-threaded programs in single-thread languages such as javascript:  http://blog.monstuff.com/archives/000315.html

   

  This technique would benefit from C# supporting the "yield foreach" construction. This would allow for an iterator method calling into another iterator method passing the result directly through. This is an especially useful syntax and optimization for recusive calls (tree navigation).

  Read more on "yield foreach" at https://blogs.msdn.com/wesdyer/archive/2007/03/23/all-about-iterators.aspx