Stephen Toub: Inside TPL Dataflow
- Posted: Feb 02, 2011 at 10:46 AM
- 55,468 Views
- 16 Comments
Download
How do I download the videos?
- To download, right click the file type you would like and pick “Save target as…” or “Save link as…”
Why should I download videos from Channel9?
- It's an easy way to save the videos you like locally.
- You can save the videos in order to watch them offline.
- If all you want is to hear the audio, you can download the MP3!
Which version should I choose?
- If you want to view the video on your PC, Xbox or Media Center, download the High Quality WMV file (this is the highest quality version we have available).
- If you'd like a lower bitrate version, to reduce the download time or cost, then choose the Medium Quality WMV file.
- If you have a Zune, WP7, iPhone, iPad, or iPod device, choose the low or medium MP4 file.
- If you just want to hear the audio of the video, choose the MP3 file.
Right click “Save as…”
- High Quality WMV (PC, Xbox, MCE)
- MP3 (Audio only)
- Mid Quality WMV (Lo-band, Mobile)
- High Quality MP4 (iPad, PC)
- MP4 (iPod, Zune HD)
TPL Dataflow (TDF), System.Threading.Tasks.Dataflow, builds upon the foundational layer for asynchronous and concurrent programming using Tasks provided in TPL in .NET 4. TDF is a complementary set of primitives to those delivered in TPL in .NET 4, addressing additional scenarios beyond those directly and easily supported by the original APIs. TPL Dataflow utilizes tasks, concurrent collections, tuples, and other features introduced in .NET 4 to bring support for parallel dataflow-based programming into the .NET Framework. It also directly integrates with new language support for tasks and asynchrony provided by both C# and Visual Basic, and with existing language support in .NET 4 for tasks provided by F#.
Several of you probably wonder how TDF differs from Rx in terms of core capabilities and specific usage scenarios. Well, Stephen talks about this right here.
As usual, Stephen goes deep and clearly articulates many technical details of TDF. Another great Toub session!
Please watch the great screencast Stephen posted to C9 a while back covering some simple examples of programming with TDF.
Make sure to ask techincal questions over in the TPL Dataflow forums, where the TDF engineers spend time and look for questions to answer: http://social.msdn.microsoft.com/Forums/en-US/tpldataflow/threads
Tune in. Enjoy. Learn.
Comments Closed
Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums,
or
Contact Us and let us know.
Follow the Discussion
Hello - this is probably a big ask, but since starting a new contract the internet connection is a shocker and I'm missing out on all this fabulous information. I'm fast becoming obsolete.
Can you run off a super lightweight mobile encoding so I can synch with my WP7 and catch up on the train home?
Oh go on. I'll be your best friend.
Mmmm.... I just saw a proposed session in the TechEd session preference survey with an interesting title:
Parallel Extensions in .NET 4.5
We will explore what's new in the Framework for parallel programming in 4.5. Specifically, we'll dive into the new TPL Dataflow library and the various improvements around partitioning, cancellation, etc.
Much coolness. Could you standardize on Add/Take instead of mixing Post/Receive/Add/Take in all blocks? Everything else is ready to go! JK
Will have to spend some more time with dataflow now.
@LukePuplett: We do provide a low quality MP4 -> http://files.ch9.ms/ch9/3276/425571a2-d072-4102-ab2e-9e7e00033276/StephenToubInsideTPLDataflow_low_ch9.mp4
Is that still too big?
C
Shouldn't DeclinePermanently() instead be something like PostLast(params T[] lastItems) ? At some point, I am going to be the last poster. But doing this is not atomic:
Another producer could (and eventually will) sneak in between the calls. The only way to avoid that is code in another sync structure (i.e. lock), which your trying to avoid. Thoughts?
Thanks for the question, staceyw. I've responded to it over on the TPL Dataflow forum at http://social.msdn.microsoft.com/Forums/en-US/tpldataflow/thread/2c5840ad-75d8-4c80-8a63-3782f18ccbf5.
Great video! I was curious how you'd project out of a block a type different then the input type and was pleased to see it in the last ten minutes. I'm very interested to see the relationship be Dataflow and Rx develop.
thanks for the clarification about the relationship with rx
i havent watched the whole video yet but it just struck me when you where talking about the coordination stuff in the beginning, TDF also feels a bit like the CCR.. its been a while since we saw George chrysanthakopoulos on c9 
more TDF screencasts would also be awsome
Perhaps custom TaskSchedulers based on TDF? seems like a very good fit
@felix9:
i knew it.... ron jacobs aaalmost slipped the name 4.5 in his wf talk at pdc
hopefully that will include that xaml editor goodness they showed for silverlight [assuming its linked to a vs service pack]
hopefully they'll start talking about 4.5 soon
Hi aL_
Good eye. TPL Dataflow is in part based on and inspired by concepts from CCR, along with concepts from Axum and Visual C++ 10's Asynchronous Agents library, so you'll see a lot of similarities in terms of the kinds of problems you can solve. The APIs were redesigned to fit in well with the rest of the .NET Framework and to take advantage of what the Task Parallel Library and other .NET goodies have to offer, as well as redesigned to incorporate some more scenarios and patterns we felt were important.
Regarding TaskSchedulers, sure, we'll see what we can pull together. Note that these dataflow blocks can be targetted to run on any TaskScheduler instance, so you could configure a block to run on the thread pool, or on the UI, or in a concurrent/exclusive fashion, or whatever underlying semantics you want to achieve by plugging in a custom scheduler. It sounds like you're interested in and asking about the other direction, implementing a TaskScheduler with a dataflow block (like ActionBlock)... you could certainly do that, too.
I'd love to see another video on this stuff. One that focusus on how you solve specific problems using TDF would be best. Like Rx, it seems like good stuff, but knowning when to use it isn't always obvious.
@Toub:
its great to see elements of the CCR getting pulled closer to the framework, i always felt the ccr got underrated or otherwise didnt enter the developer mainstream.
you're right, i was referring to implementing TaskSchedulers using TDF, maybe like a throttling ui scheduler that ensures the ui thread doesnt get flooded with updates. Just a simple scheduler that process one task at the time [just an actionblock basically] is a good practical sample too i think. Really, all the custom schedulers in the parallel programming samples are good candidates
I am curious as to why the blocks in the dataflow library are mutable. In a recent project, we designed a dataflow library in .net for processing energy usage data. Early on we decided to make the dataflow graph objects (blocks) immutable. In this library, the enumerators used to get data out of the dataflow graph are mutable. Enumerators are created on demand by block nodes when you begin to iterate over the dataflow graph (the enumerators form a parallel type of graph).
Immutable block objects had advantages. We could share them between threads, and we could share blocks as input components in multiple larger graphs. By computing an immutable hash code on a block on construction, using the hash code of input blocks in the computation, we could compare block graphs for equality, and use them as keys in dictionaries that memoized graph outputs (as long as inputs were immutable, which was the case for historical data).
In the TPL Dataflow library, blocks are both graph nodes and enumerators.
@Frank Hileman: Frank, create an account on 9 so you get the benefits of text formatting...
C
Hi Charles,
I did not notice I was not logged in. Maybe previously I had to log in to comment at all. Thanks.
@Frank Hileman: Cool
C
Hi guys, great video, keep 'em coming!
@Al_: you could have shared some credit with me on the Rx comments :P http://channel9.msdn.com/posts/TPL-Dataflow-Tour#c634317373800000000
Cheers!
Remove this comment
Remove this thread
close