Loading user information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading user information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Stephen Toub: Inside TPL Dataflow

1 hour, 3 minutes, 46 seconds


Right click “Save as…”

TPL Dataflow (TDF), System.Threading.Tasks.Dataflow, builds upon the foundational layer for asynchronous and concurrent programming using Tasks provided in TPL in .NET 4. TDF is a complementary set of primitives to those delivered in TPL in .NET 4, addressing additional scenarios beyond those directly and easily supported by the original APIs. TPL Dataflow utilizes tasks, concurrent collections, tuples, and other features introduced in .NET 4 to bring support for parallel dataflow-based programming into the .NET Framework. It also directly integrates with new language support for tasks and asynchrony provided by both C# and Visual Basic, and with existing language support in .NET 4 for tasks provided by F#.

Several of you probably wonder how TDF differs from Rx in terms of core capabilities and specific usage scenarios. Well, Stephen talks about this right here.

As usual, Stephen goes deep and clearly articulates many technical details of TDF. Another great Toub session!

Please watch the great screencast Stephen posted to C9 a while back covering some simple examples of programming with TDF.

Make sure to ask techincal questions over in the TPL Dataflow forums, where the TDF engineers spend time and look for questions to answer: http://social.msdn.microsoft.com/Forums/en-US/tpldataflow/threads

Tune in. Enjoy. Learn.



Follow the discussion

  • Oops, something didn't work.

    Getting subscription
    Subscribe to this conversation
  • Hello - this is probably a big ask, but since starting a new contract the internet connection is a shocker and I'm missing out on all this fabulous information. I'm fast becoming obsolete.

    Can you run off a super lightweight mobile encoding so I can synch with my WP7 and catch up on the train home?

    Oh go on. I'll be your best friend.

  • felix9felix9 the cat that walked by itself

    Mmmm.... I just saw a proposed session in the TechEd session preference survey with an interesting title:

    Parallel Extensions in .NET 4.5

    We will explore what's new in the Framework for parallel programming in 4.5.  Specifically, we'll dive into the new TPL Dataflow library and the various improvements around partitioning, cancellation, etc.

  • William Staceystaceyw Before C# there was darkness...

    Much coolness.  Could you standardize on Add/Take instead of mixing Post/Receive/Add/Take in all blocks? Everything else is ready to go! JK Smiley  Will have to spend some more time with dataflow now.

  • CharlesCharles Welcome Change

    @LukePuplett: We do provide a low quality MP4 -> http://files.ch9.ms/ch9/3276/425571a2-d072-4102-ab2e-9e7e00033276/StephenToubInsideTPLDataflow_low_ch9.mp4 

    Is that still too big?


  • William Staceystaceyw Before C# there was darkness...

    Shouldn't DeclinePermanently() instead be something like PostLast(params T[] lastItems) ? At some point, I am going to be the last poster. But doing this is not atomic:

    // Not atomic.

    Another producer could (and eventually will) sneak in between the calls. The only way to avoid that is code in another sync structure (i.e. lock), which your trying to avoid. Thoughts?

  • Thanks for the question, staceyw.  I've responded to it over on the TPL Dataflow forum at http://social.msdn.microsoft.com/Forums/en-US/tpldataflow/thread/2c5840ad-75d8-4c80-8a63-3782f18ccbf5.

  • James AlexanderJames Alexander

    Great video! I was curious how you'd project out of a block a type different then the input type and was pleased to see it in the last ten minutes. I'm very interested to see the relationship be Dataflow and Rx develop.

  • Allan LindqvistaL_ Kinect ftw

    thanks for the clarification about the relationship with rx Smiley i havent watched the whole video yet but it just struck me when you where talking about the coordination stuff in the beginning, TDF also feels a bit like the CCR.. its been a while since we saw George chrysanthakopoulos on c9 Smiley

    more TDF screencasts would also be awsome Smiley Perhaps custom TaskSchedulers based on TDF? seems like a very good fit


    i knew it.... ron jacobs aaalmost slipped the name 4.5 in his wf talk at pdc Smiley hopefully that will include that xaml editor goodness they showed for silverlight [assuming its linked to a vs service pack]

    hopefully they'll start talking about 4.5 soon

  • Hi aL_

    Good eye. TPL Dataflow is in part based on and inspired by concepts from CCR, along with concepts from Axum and Visual C++ 10's Asynchronous Agents library, so you'll see a lot of similarities in terms of the kinds of problems you can solve.  The APIs were redesigned to fit in well with the rest of the .NET Framework and to take advantage of what the Task Parallel Library and other .NET goodies have to offer, as well as redesigned to incorporate some more scenarios and patterns we felt were important.

    Regarding TaskSchedulers, sure, we'll see what we can pull together.  Note that these dataflow blocks can be targetted to run on any TaskScheduler instance, so you could configure a block to run on the thread pool, or on the UI, or in a concurrent/exclusive fashion, or whatever underlying semantics you want to achieve by plugging in a custom scheduler.  It sounds like you're interested in and asking about the other direction, implementing a TaskScheduler with a dataflow block (like ActionBlock)... you could certainly do that, too.

  • BrandonBrandon

    I'd love to see another video on this stuff.  One that focusus on how you solve specific problems using TDF would be best.  Like Rx, it seems like good stuff, but knowning when to use it isn't always obvious.

  • Allan LindqvistaL_ Kinect ftw


    its great to see elements of the CCR getting pulled closer to the framework, i always felt the ccr got underrated or otherwise didnt enter the developer mainstream.

    you're right, i was referring to implementing TaskSchedulers using TDF, maybe like a throttling ui scheduler that ensures the ui thread doesnt get flooded with updates. Just a simple scheduler that process one task at the time [just an actionblock basically] is a good practical sample too i think. Really, all the custom schedulers in the parallel programming samples are good candidates Smiley


  • Frank HilemanFrank Hileman

    I am curious as to why the blocks in the dataflow library are mutable. In a recent project, we designed a dataflow library in .net for processing energy usage data. Early on we decided to make the dataflow graph objects (blocks) immutable. In this library, the enumerators used to get data out of the dataflow graph are mutable. Enumerators are created on demand by block nodes when you begin to iterate over the dataflow graph (the enumerators form a parallel type of graph).
    Immutable block objects had advantages. We could share them between threads, and we could share blocks as input components in multiple larger graphs. By computing an immutable hash code on a block on construction, using the hash code of input blocks in the computation, we could compare block graphs for equality, and use them as keys in dictionaries that memoized graph outputs (as long as inputs were immutable, which was the case for historical data).
    In the TPL Dataflow library, blocks are both graph nodes and enumerators.

  • CharlesCharles Welcome Change

    @Frank Hileman: Frank, create an account on 9 so you get the benefits of text formatting... Smiley


  • Hi Charles,

    I did not notice I was not logged in. Maybe previously I had to log in to comment at all. Thanks.

  • CharlesCharles Welcome Change

    @Frank Hileman: Cool Smiley

  • GustavoGustavo

    Hi guys, great video, keep 'em coming!
    @Al_: you could have shared some credit with me on the Rx comments :P http://channel9.msdn.com/posts/TPL-Dataflow-Tour#c634317373800000000

Remove this comment

Remove this thread


Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.