Comment Feed for Channel 9 - AppFabric.tv - Threading with Jeff Richter http://ak.channel9.msdn.com/ch9/21fe/2b294ef5-5ff9-47fb-a368-9eed013521fe/endpointAFThreading_100_ch9.jpg Channel 9 - AppFabric.tv - Threading with Jeff Richter Years ago I did one of my first podcasts with Jeff Richter, and later I took his advanced threading class, which was fantastic. Jeff really helped me to understand threads, thread pools, and why async is so important when building server side code. In this episode, Jeff joins me to bring you up to speed on threads, threading, and async so you can build really fast and efficient server apps. Ron Jacobs http://blogs.msdn.com/rjacobs Twitter: @ronljacobs http://twitter.com/ronljacobs en Sat, 18 May 2013 22:36:05 GMT Sat, 18 May 2013 22:36:05 GMT Rev9 Re: AppFabric.tv - Threading with Jeff Richter Oh Jeffrey Richter is the man ! Cool

posted by felix9

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634431446520000000 Wed, 08 Jun 2011 15:44:12 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634431446520000000 felix9 Re: AppFabric.tv - Threading with Jeff Richter How many calls to Begin* can you do before you crash ? That queue must have a limit too, right ?

If your request (Begin*) rate is faster then the respond (End*) rate you will get into trouble anyway.

So you've switch from one out-of-memory exception to another.

posted by Gordon

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634431596410000000 Wed, 08 Jun 2011 19:54:01 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634431596410000000 Gordon Re: AppFabric.tv - Threading with Jeff Richter All computer resources are finite. You are always trading off one thing for another thing. When calling a Begin* method, you give up 1MB+ of memory (the thread's stack) and replace it with an I/O request packet (maybe 100 bytes). This is a huge diference and allows your application to scale much, much better. If your service is busy enough, then 1 machine can't handle the load, and you need multiple machines to handle the load. Trading threads for I/O requets packets allows you to defer or reduce the number of machines. Adding machines adds significant additional costs: hardware, electricity, software licenses, IT costs (backup, patching, maintenance) etc.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634431608770000000 Wed, 08 Jun 2011 20:14:37 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634431608770000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter Great video, and excellent presentation on Azure in Waltham.

 

How do you work asynchronously with memory mapped files though? After all you are just working with pointers in memory but the system must issue I/O requests behind the scenes?

 

Thanks!

 

 

posted by fantassin

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634432252540000000 Thu, 09 Jun 2011 14:07:34 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634432252540000000 fantassin Re: AppFabric.tv - Threading with Jeff Richter Memory-mapped files are all about making file I/O look like RAM and so you can't work asynchronously with memory-mapped files. You are tradiing the simpler progrmaming model for reduced scalability (when acessing MMF data actaully results in I/O as opposed to accessing cached data in RAM).

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634432313110000000 Thu, 09 Jun 2011 15:48:31 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634432313110000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter
Jeffrey, where can I read more about the objects that are created for each new thread in .NET, why they needed and how much memory they take on different platforms (x86, x64)?

Is managed thread is a wrapper for the native OS thread?

Thanks!

posted by Oleksii

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634433053100000000 Fri, 10 Jun 2011 12:21:50 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634433053100000000 Oleksii Re: AppFabric.tv - Threading with Jeff Richter I cover a lot of this in my CLR via C#, 3rd Edition book. My Windows via C/C++ also talks alot of threads and their overhead. The Windows Internals book by Mark Russinovich & David Solomon also has a lot of info in it. There is no such thing as a "managed thread". In managed code, you can ask Windows to create a thread. Again, my CLR via C#, 3rd Edition book goes into all of this in great detail.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634433209630000000 Fri, 10 Jun 2011 16:42:43 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634433209630000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter posted by Sarang

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634433781540000000 Sat, 11 Jun 2011 08:35:54 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634433781540000000 Sarang Re: AppFabric.tv - Threading with Jeff Richter With each operation, you should always consider whether it is an I/O operation or a compute-bound operation. I/O operations do not use the CPU on the motherboard at all and so you scale them out using asynchronous I/O operations (Begin/End methods); do NOT use threads to perform I/O operations in parallel as this just wastes threads. Compute operations DO use the CPU on the motherboard and so you improve performance by having multiple threads (up to 1 thread per CPU in the machine) execute different pieces of the work concurrently. For compute operations, you can use QueueUserWorkItem (but this is fire & forget), or a delegate's BeginInvoke/EndInvoke methods, or create & start .NET 4's Task object. Delegate's BeginInvoke/EndInvoke allow you to have the same programming model for compute operations as you get for I/O operations.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634434037780000000 Sat, 11 Jun 2011 15:42:58 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634434037780000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter Hi Jeff
Thank you for an awesome session. I apologize if I am missing the obvious, but would you mind giving me a little more detail on how would you implement the described async scenario? As far as I can see, at some point something has to lock and wait for the time-consuming operation. How can you do that in a manner that would release the thread back to the threadpool? Take for example your slow database scenario: if this is called from a WCF operation that has to return the result to the client, it will need to employ something like a waithandle to await the result. As far as I know the waithandle will not release the thread.
What I have done in the past was to push the "locking" all the way to the client so that the initial command going through IIS does nothing but delegate to another process, and then the client polls another service operation for the result. I was even thinking of improving this using websockets. However if this could be achieved all in process it would be a much more elegant solution.
Thanks,
Dan

posted by serializable

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634436750870000000 Tue, 14 Jun 2011 19:04:47 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634436750870000000 serializable Re: AppFabric.tv - Threading with Jeff Richter All of Microsoft's hosting infrastructures -- ASP.NET, WCF, etc -- support asynchronous programming. The server gets a client request, the server makes a request to another server (like SQL)  asynchronously, and the thread returns back to the thread pool. The hosting infrastructure knows NOT to send the response back to the client. When the server (SQL) responds, its response it put in the thread pool, another threadpool thread wakes up and your code processes the response and returns. When the thread returns to the pool this time, the hosting infrastructure DOES send the response to the client. Lookup how to implement your service asynchronously in whatever infrastrucutre is hosting you. For example, do a web search for "implementing ASP.NET Web form service asynchronously".

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437056270000000 Wed, 15 Jun 2011 03:33:47 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437056270000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter @JeffRichter:Thanks Jeff. That makes sense.

posted by serializable

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437237510000000 Wed, 15 Jun 2011 08:35:51 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437237510000000 serializable Re: AppFabric.tv - Threading with Jeff Richter
I am asking because here Rick discouraging to use them?

http://blogs.msdn.com/b/rickandy/archive/2009/11/14/should-my-database-calls-be-asynchronous.aspx


I am just wanting to know your suggestion?

posted by Anonymious

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437333150000000 Wed, 15 Jun 2011 11:15:15 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437333150000000 Anonymious Re: AppFabric.tv - Threading with Jeff Richter Rick may have some valid ideas here but he is also assuming that every web request results in a DB request which, in many web servers, is not true. I have written web sites where many requests are handled from memory or from cache or possibly from a store other than a DB. In this case, there are NOT lots of threads blocking on the DB and the threads can do other work. In addition, his only argument for not making things async is really just to simplify the programming model. He's not suggesting making things sync because your service is faster or uses less resources (both of which hurt scalability). He's suggestion is purely about simplifying the code. But with technologies like my Power Threading library's AsyncEnumerator, or with the new async/await feature being put into the next version of the .NET Fx, the programming model is much simpler than it has been in the past and so a simplified programming model is much less compelling of an argument.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437482260000000 Wed, 15 Jun 2011 15:23:46 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634437482260000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter posted by Suneet

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634438487800000000 Thu, 16 Jun 2011 19:19:40 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634438487800000000 Suneet Re: AppFabric.tv - Threading with Jeff Richter Hi Jeff

Could you please explain one more thing: I have created a simple test service invoking a slow sql command in order to observe the thread consumption (listed here https://gist.github.com/1032983). When my test client hits the service with n concurrent requests, if the service is running the query synchronously I notice about n worker threads and n IO threads being allocated for the duration of the calls, as expected. When the service is running the query asynchronously, I notice about n worker threads and 0 IO threads being used. Why does WCF hang on to those worker threads? Are they locked just because the WCF connection is open? Or is it that I am doing something stupid? Is there any way to work around this?

Thanks

Dan

posted by serializable

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634439913540000000 Sat, 18 Jun 2011 10:55:54 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634439913540000000 serializable Re: AppFabric.tv - Threading with Jeff Richter PS. I must confess, your last book that I've read is an old edition of "applied .net framework programming". I will pick up your latest CLR via C# today Smiley

posted by serializable

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634439914740000000 Sat, 18 Jun 2011 10:57:54 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634439914740000000 serializable Re: AppFabric.tv - Threading with Jeff Richter @serializable:I am not a WCF expert and I don't have time to examine your code. So I'm not sure if the WCF infrastructure is implemented poorly or if your code is not using the infrastructure correctly. I do know that the WCF team cared a lot about async operations and so the most likely problem is that your code is not using it correctly.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634440362060000000 Sat, 18 Jun 2011 23:23:26 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634440362060000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter @JeffRichter:Thank you.

posted by serializable

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634440647750000000 Sun, 19 Jun 2011 07:19:35 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634440647750000000 serializable Re: AppFabric.tv - Threading with Jeff Richter Do you forsee a timeline as when we would be able to take advantage of the Processor groups so we could have our threads running on over 64 CPU cores. That could be a hypothetical question but I am just curious for information.:)

I hate to say this, but if you don't write books for the upcoming versions of the .Net framework, then most of us here would be a lot more than upset. :(

posted by Maran

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634441496710000000 Mon, 20 Jun 2011 06:54:31 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634441496710000000 Maran Re: AppFabric.tv - Threading with Jeff Richter You should be able to use processor groups today in .NET if you P/Invoke out to the native Windows APIs.The CLR team could wrap these for you but it is trivial for you to do it yourself. There are few machines with this many cores and Azure machines have no more than 8 cores so this would be very low priority for the CLR team. 

 

I plan to revise my CLR vua C# book for the next version of .NET but I haven't started working on it yet. I'll probably start when it enters beta.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634441629340000000 Mon, 20 Jun 2011 10:35:34 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634441629340000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter Great video! It helps show why some newer frameworks like node.js focus on event driven or asynchronous programming to limit the amount of time a thread spends being blocked. It really is amazing how many requests you can server if you avoid blocking in your threads.

I am curious as to if/when you think something like lightweight threads (aka fibers, wikipedia) with co-operative multitasking should be used in a server architecture. SQL Server is the only Microsoft server product that I am aware of that can be configured to use fibers instead of full threads, and even they don't recommend it except for in very special situations. In what situation should a highly specialized tool like fibers be used?

posted by gbrayut

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442086530000000 Mon, 20 Jun 2011 23:17:33 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442086530000000 gbrayut Re: AppFabric.tv - Threading with Jeff Richter @JeffRichter:

Hi

Great video... I enjoyed it.

Yet I think you glossed over the real reason people use threads. Unless an application has been designed from its inception to be asynchronous, threading does one thing that begin/end asynch does not: it preserves application state through the stack.

In particular, imagine a legacy application that is deep down in its stack of A calling B calling C calling ... and somewhere deep in the stack it needs to do an I/O.

If it can't proceed until the I/O is done there is little to do but block (which is why you need to be on a thread, otherwise the whole application freezes).

If you spawned an asynch request at this point, what is the thread supposed to do until the I/O is done? It can't just return !!

I wish you had addressed this point - in my opinion it is the key issue regarding threading.

The answer, I believe, is to have the stack be independent of threading of execution (which is after all just a way of sharing the CPU across multiple logical work requests). A call stack represents the true application state (along with the heap, ...).

In this model of an application thread, it is the stack that is important. When a 'stack' is blocked the underlying execution (managed by the system) can just jump to another 'stack'.

I think this is what tasks are all about, no?

-David

 

posted by djmarcus

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442732370000000 Tue, 21 Jun 2011 17:13:57 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442732370000000 djmarcus Re: AppFabric.tv - Threading with Jeff Richter @gbrayut:Fibers are a pretty old technology. They were originally added to NT4 to ease porting of apps from other OSes to Windows NT. While SQL server has a fiber mode, the mode is usually used for benchmarking and not for real day to day running of the DB. There was an attempt in .NET 2 to add fibers to the CLR but the attempt failed due to separating state to give the impression of different fibers being different threads. Since fibers are not threads, they do not work like threads and using them can become quite difficult. This is why the CLR dropped the feature. Maybe on a platform where fibers are a 1st class citizens from the very beginning, they could work well.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442971400000000 Tue, 21 Jun 2011 23:52:20 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442971400000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter @djmarcus:You are correct that his is a problem but a lot fo work has been going on in this space. In my Power Threading library, my AsyncEnumerator class (along with C#'s iterator feature) allows you to have a synchronous programming model for doing asynchronous operations. This has been available for about 5 years now. It has received such support and success, that the .NET team at MS is adding very similar features to the next version of .NET and, the new features will support a deep call stack where you can initiate an async I/O at the bottom of the stack, return all the way up and have execution continue at the bottom of the logical stack when the I/O compeletes.

posted by JeffRichter

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442974150000000 Tue, 21 Jun 2011 23:56:55 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442974150000000 JeffRichter Re: AppFabric.tv - Threading with Jeff Richter @JeffRichter:

I thought I was familiar with the AsyncEnumerator class.. but apparently I am not.I will take a closer look.

Being able to 'resume execution' at the point of I/O with the stack intact and the same as it was prior to the I/O is exactly what I was referring to.

Thanks for your quick reply.

-David

posted by djmarcus

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442985080000000 Wed, 22 Jun 2011 00:15:08 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634442985080000000 djmarcus Re: AppFabric.tv - Threading with Jeff Richter I recommend Jeff's CLR Book to anyone who wants to understand .NET Properly. It is a masterpiece. Thank you Jeff

posted by SOA_Architect

]]> http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634461019680000000 Tue, 12 Jul 2011 21:12:48 GMT http://channel9.msdn.com/Shows/AppFabric-tv/AppFabrictv-Threading-with-Jeff-Richter#c634461019680000000 SOA_Architect