Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

AppFabric.tv - Threading with Jeff Richter

Download

Right click “Save as…”

Years ago I did one of my first podcasts with Jeff Richter, and later I took his advanced threading class, which was fantastic. Jeff really helped me to understand threads, thread pools, and why async is so important when building server side code. In this episode, Jeff joins me to bring you up to speed on threads, threading, and async so you can build really fast and efficient server apps.

Ron Jacobs
http://blogs.msdn.com/rjacobs
Twitter: @ronljacobs http://twitter.com/ronljacobs

Tags:

Follow the Discussion

  • felix9felix9 the cat that walked by itself

    Oh Jeffrey Richter is the man ! Cool

  • GordonGordon

    Using async Begin* / End* model:
    How many calls to Begin* can you do before you crash ? That queue must have a limit too, right ?

    If your request (Begin*) rate is faster then the respond (End*) rate you will get into trouble anyway.

    So you've switch from one out-of-memory exception to another.

  • Jeffrey RichterJeffRichter Jeffrey Richter

    All computer resources are finite. You are always trading off one thing for another thing. When calling a Begin* method, you give up 1MB+ of memory (the thread's stack) and replace it with an I/O request packet (maybe 100 bytes). This is a huge diference and allows your application to scale much, much better. If your service is busy enough, then 1 machine can't handle the load, and you need multiple machines to handle the load. Trading threads for I/O requets packets allows you to defer or reduce the number of machines. Adding machines adds significant additional costs: hardware, electricity, software licenses, IT costs (backup, patching, maintenance) etc.

  • Great video, and excellent presentation on Azure in Waltham.

     

    How do you work asynchronously with memory mapped files though? After all you are just working with pointers in memory but the system must issue I/O requests behind the scenes?

     

    Thanks!

     

     

  • Jeffrey RichterJeffRichter Jeffrey Richter

    Memory-mapped files are all about making file I/O look like RAM and so you can't work asynchronously with memory-mapped files. You are tradiing the simpler progrmaming model for reduced scalability (when acessing MMF data actaully results in I/O as opposed to accessing cached data in RAM).

  • OleksiiOleksii

    Fantastic video!

    Jeffrey, where can I read more about the objects that are created for each new thread in .NET, why they needed and how much memory they take on different platforms (x86, x64)?

    Is managed thread is a wrapper for the native OS thread?

    Thanks!

  • Jeffrey RichterJeffRichter Jeffrey Richter

    I cover a lot of this in my CLR via C#, 3rd Edition book. My Windows via C/C++ also talks alot of threads and their overhead. The Windows Internals book by Mark Russinovich & David Solomon also has a lot of info in it. There is no such thing as a "managed thread". In managed code, you can ask Windows to create a thread. Again, my CLR via C#, 3rd Edition book goes into all of this in great detail.

  • SarangSarang

    So does that mean have to make a careful conscious choice of when to use a begin-end model vs a queueuserworkitem depending on if the operation is IO vs compute bound respectively?

  • Jeffrey RichterJeffRichter Jeffrey Richter

    With each operation, you should always consider whether it is an I/O operation or a compute-bound operation. I/O operations do not use the CPU on the motherboard at all and so you scale them out using asynchronous I/O operations (Begin/End methods); do NOT use threads to perform I/O operations in parallel as this just wastes threads. Compute operations DO use the CPU on the motherboard and so you improve performance by having multiple threads (up to 1 thread per CPU in the machine) execute different pieces of the work concurrently. For compute operations, you can use QueueUserWorkItem (but this is fire & forget), or a delegate's BeginInvoke/EndInvoke methods, or create & start .NET 4's Task object. Delegate's BeginInvoke/EndInvoke allow you to have the same programming model for compute operations as you get for I/O operations.

  • Dan Ballaserializable Dan Balla

    Hi Jeff
    Thank you for an awesome session. I apologize if I am missing the obvious, but would you mind giving me a little more detail on how would you implement the described async scenario? As far as I can see, at some point something has to lock and wait for the time-consuming operation. How can you do that in a manner that would release the thread back to the threadpool? Take for example your slow database scenario: if this is called from a WCF operation that has to return the result to the client, it will need to employ something like a waithandle to await the result. As far as I know the waithandle will not release the thread.
    What I have done in the past was to push the "locking" all the way to the client so that the initial command going through IIS does nothing but delegate to another process, and then the client polls another service operation for the result. I was even thinking of improving this using websockets. However if this could be achieved all in process it would be a much more elegant solution.
    Thanks,
    Dan

  • Jeffrey RichterJeffRichter Jeffrey Richter

    All of Microsoft's hosting infrastructures -- ASP.NET, WCF, etc -- support asynchronous programming. The server gets a client request, the server makes a request to another server (like SQL)  asynchronously, and the thread returns back to the thread pool. The hosting infrastructure knows NOT to send the response back to the client. When the server (SQL) responds, its response it put in the thread pool, another threadpool thread wakes up and your code processes the response and returns. When the thread returns to the pool this time, the hosting infrastructure DOES send the response to the client. Lookup how to implement your service asynchronously in whatever infrastrucutre is hosting you. For example, do a web search for "implementing ASP.NET Web form service asynchronously".

  • Dan Ballaserializable Dan Balla

    @JeffRichter:Thanks Jeff. That makes sense.

  • AnonymiousAnonymious

    Can you recommend to use APM for slow database request?

    I am asking because here Rick discouraging to use them?

    http://blogs.msdn.com/b/rickandy/archive/2009/11/14/should-my-database-calls-be-asynchronous.aspx


    I am just wanting to know your suggestion?

  • Jeffrey RichterJeffRichter Jeffrey Richter

    Rick may have some valid ideas here but he is also assuming that every web request results in a DB request which, in many web servers, is not true. I have written web sites where many requests are handled from memory or from cache or possibly from a store other than a DB. In this case, there are NOT lots of threads blocking on the DB and the threads can do other work. In addition, his only argument for not making things async is really just to simplify the programming model. He's not suggesting making things sync because your service is faster or uses less resources (both of which hurt scalability). He's suggestion is purely about simplifying the code. But with technologies like my Power Threading library's AsyncEnumerator, or with the new async/await feature being put into the next version of the .NET Fx, the programming model is much simpler than it has been in the past and so a simplified programming model is much less compelling of an argument.

  • SuneetSuneet

    Brilliant video on thread, specially when I am done with your 2nd edition CLR Via C# and reading on the 3rd edition.

  • Dan Ballaserializable Dan Balla

    Hi Jeff

    Could you please explain one more thing: I have created a simple test service invoking a slow sql command in order to observe the thread consumption (listed here https://gist.github.com/1032983). When my test client hits the service with n concurrent requests, if the service is running the query synchronously I notice about n worker threads and n IO threads being allocated for the duration of the calls, as expected. When the service is running the query asynchronously, I notice about n worker threads and 0 IO threads being used. Why does WCF hang on to those worker threads? Are they locked just because the WCF connection is open? Or is it that I am doing something stupid? Is there any way to work around this?

    Thanks

    Dan

  • Dan Ballaserializable Dan Balla

    PS. I must confess, your last book that I've read is an old edition of "applied .net framework programming". I will pick up your latest CLR via C# today Smiley

  • Jeffrey RichterJeffRichter Jeffrey Richter

    @serializable:I am not a WCF expert and I don't have time to examine your code. So I'm not sure if the WCF infrastructure is implemented poorly or if your code is not using the infrastructure correctly. I do know that the WCF team cared a lot about async operations and so the most likely problem is that your code is not using it correctly.

  • Dan Ballaserializable Dan Balla

    @JeffRichter:Thank you.

  • MaranMaran

    @JeffRichter: I am into Asynchronous programming in CLR via C# third ed. I am exploring it for the demands of my own. Jeff, you are a genius.
    Do you forsee a timeline as when we would be able to take advantage of the Processor groups so we could have our threads running on over 64 CPU cores. That could be a hypothetical question but I am just curious for information.:)

    I hate to say this, but if you don't write books for the upcoming versions of the .Net framework, then most of us here would be a lot more than upset. :(

  • Jeffrey RichterJeffRichter Jeffrey Richter

    You should be able to use processor groups today in .NET if you P/Invoke out to the native Windows APIs.The CLR team could wrap these for you but it is trivial for you to do it yourself. There are few machines with this many cores and Azure machines have no more than 8 cores so this would be very low priority for the CLR team. 

     

    I plan to revise my CLR vua C# book for the next version of .NET but I haven't started working on it yet. I'll probably start when it enters beta.

  • Greg Braygbrayut Check out PhraseMeme Scanner for Windows Phone at ​PhraseMeme.​com

    Great video! It helps show why some newer frameworks like node.js focus on event driven or asynchronous programming to limit the amount of time a thread spends being blocked. It really is amazing how many requests you can server if you avoid blocking in your threads.

    I am curious as to if/when you think something like lightweight threads (aka fibers, wikipedia) with co-operative multitasking should be used in a server architecture. SQL Server is the only Microsoft server product that I am aware of that can be configured to use fibers instead of full threads, and even they don't recommend it except for in very special situations. In what situation should a highly specialized tool like fibers be used?

  • @JeffRichter:

    Hi

    Great video... I enjoyed it.

    Yet I think you glossed over the real reason people use threads. Unless an application has been designed from its inception to be asynchronous, threading does one thing that begin/end asynch does not: it preserves application state through the stack.

    In particular, imagine a legacy application that is deep down in its stack of A calling B calling C calling ... and somewhere deep in the stack it needs to do an I/O.

    If it can't proceed until the I/O is done there is little to do but block (which is why you need to be on a thread, otherwise the whole application freezes).

    If you spawned an asynch request at this point, what is the thread supposed to do until the I/O is done? It can't just return !!

    I wish you had addressed this point - in my opinion it is the key issue regarding threading.

    The answer, I believe, is to have the stack be independent of threading of execution (which is after all just a way of sharing the CPU across multiple logical work requests). A call stack represents the true application state (along with the heap, ...).

    In this model of an application thread, it is the stack that is important. When a 'stack' is blocked the underlying execution (managed by the system) can just jump to another 'stack'.

    I think this is what tasks are all about, no?

    -David

     

  • Jeffrey RichterJeffRichter Jeffrey Richter

    @gbrayut:Fibers are a pretty old technology. They were originally added to NT4 to ease porting of apps from other OSes to Windows NT. While SQL server has a fiber mode, the mode is usually used for benchmarking and not for real day to day running of the DB. There was an attempt in .NET 2 to add fibers to the CLR but the attempt failed due to separating state to give the impression of different fibers being different threads. Since fibers are not threads, they do not work like threads and using them can become quite difficult. This is why the CLR dropped the feature. Maybe on a platform where fibers are a 1st class citizens from the very beginning, they could work well.

  • Jeffrey RichterJeffRichter Jeffrey Richter

    @djmarcus:You are correct that his is a problem but a lot fo work has been going on in this space. In my Power Threading library, my AsyncEnumerator class (along with C#'s iterator feature) allows you to have a synchronous programming model for doing asynchronous operations. This has been available for about 5 years now. It has received such support and success, that the .NET team at MS is adding very similar features to the next version of .NET and, the new features will support a deep call stack where you can initiate an async I/O at the bottom of the stack, return all the way up and have execution continue at the bottom of the logical stack when the I/O compeletes.

  • @JeffRichter:

    I thought I was familiar with the AsyncEnumerator class.. but apparently I am not.I will take a closer look.

    Being able to 'resume execution' at the point of I/O with the stack intact and the same as it was prior to the I/O is exactly what I was referring to.

    Thanks for your quick reply.

    -David

  • I recommend Jeff's CLR Book to anyone who wants to understand .NET Properly. It is a masterpiece. Thank you Jeff

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.