Blog Post

Frank McSherry: Introduction to Naiad and Differential Dataflow

Sign in to queue

The Discussion

  • User profile image

    looks like a really interesting project, thanks

  • User profile image

    That's some heavy stuff once you start thinking about it Smiley naiad + azure = skynet?


    Hope too see more naiad on c9!

  • User profile image

    Can you explain more about the .FixedPoint() linq method?  I found the naiad.pptx presentation with a lot of animations, but I'm unable to follow it without the narrative that I presume went with it.  I'll check out the paper, but if it's anything like this video, perhaps there are some prerequisite materials you can recommend?

    From a practical standpoint, I'm interested in how to integrate this with a datasource like SQL Server for persistence.  To maintain a twitter-like service optimized for individual user/viewer queries, I assume that we'd have the data in an intuitive, application-specific schema of Users, Tweets, and Mentions.  As changes are committed to the persisted data store (SQL Server), we must independently notify the Naiad cluster of the change - and then the Naiad cluster members (or is it just the Controllers?) can service user queries quickly.  Is it advisable to allow this notification to Naiad to come from the database layer rather than from the application layer, using something like SQL Query Notifications?  Or does the high volume of changes that might be expected rule out Query Notifications?  The benefit of being able to listen to SQL Server directly for relevant changes would be relative transparency to existing applications, less chance of data getting out of sync due to a forgotten call to Naiad.

    Alternatively, perhaps Naiad becomes the primary datastore and is augmented to persist its Dataflow dataset in a Naiad-friendly relational schema?

Add Your 2 Cents