Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

SQL Database Sharding Patterns

Download

Right click “Save as…”

  • MP3 (Audio only)
  • MP4 (iPhone, Android)
  • High Quality MP4 (iPad, PC, Xbox)
  • Mid Quality MP4 (Windows Phone, HTML5, iPhone)

Join Scott Klein as he welcomes Shaun Tinline-Jones, Senior Program Manager for AzureCat, as they discuss patterns and practices for sharding Azure SQL Database. Scott has been working with Shaun and the AzureCAT team for some time now on a sample library that provides elastic scalability in SQL Database. Scott and Shaun talk about some of the complexities of database sharding but introduce and demo a library that help overcome those complexities and provide the elastic scale needed in SQL Database. Shaun highlights the key components and concepts when working with the sharding and then dives right into demoing the library to illustrate how the library can be used to implement sharding patterns into SQL Database. The sample library is available for customers to modify/extend/use and is compiled from engaging with customers who have taken on the task of creating customer sharding solutions.

Tags:

Follow the Discussion

  • Hi, thank you, it was great conversation, custom sharding is very important.

    When can we start using this library? is it ready now, or if not where can we find it in the future?

  • @gkaya: Thank you for viewing the session.  The library and supporting guidance documents are undergoing peer and legal review. Our timelines are targeting July/August 2014.  Please do contact me if you would like to discuss earlier access.

  • pinipini

    Hi when we will have it?
    pini

  • @pini: Thank you for viewing the session.  The library and supporting guidance documents are undergoing peer and legal review. Our timelines are targeting July/August 2014.  Please do contact me if you would like to discuss earlier access.  I wish I had a different answer for you but this is the required steps, and we are working at meeting this timeline.

  • I am user of Sql Federation and looking for alternatives now as federation is being dropped, and that's how I stumbled upon the session.

    So the session talks about creating/spiting/merging a shard which is sort of akin to Create/Split Federation etc. in Sql Azure. But how about connection filtering equivalent in Sql Azure federation, how that is going to work? Currently we have lots of stored procs where we don't have to explicitly do "WHERE CustomerId = @CustomerId" as Connection Filtering (i.e. USE FEDERATION.....) takes care of returning only customer specify data, is there some equivalent or guidance on how to migrate that will be available?

  • @pateketu: IIRC Federations was achieving this functionality via the gateway.  This is not available to database calls, especially to non-Azure DB environments, and therefore really tricky (if even possible) to implement.  Our own personal research into this topic bubbled a number of challenges that are simply unwieldy to deal with in a "generic" way.  I am aware of one customer that achieves this by constraining all database interactions to EF calls.  This means that they can add the predicate as each query is created.  If you were willing to make strict rules, such as all SPs must have sharding key parameter, all T-SQL commands were first passed through the transact script dom to get the sharding key predicate, then this could be added to the sharding code.

    In summary, this functionality (connection filtering) is not available in the current work discussed in this recording, nor is it something we have found a reasonable resolution for.  It is certainly something that would be great to have, and I would be very interested in hearing ideas on how this can be achieved.

  • @Shaunbg, thanks for the response, I am in touch with someone in Azure CAT team (I am guessing it's probably same as your team) and hoping to get access to the library talked about in the recording soon'ish. I just looked up "transact script dom" and found TransactSql.ScriptDOM. Never knew of it's existence, it does sound like a promising approach to inject WHERE Predicts!

  • JorgeJorge

    Any news on when the library will be avail? I need to do custom sharding but it looks likes federation will no more be supported. Thanks!

  • I just returned from vacation and received the output from the editors who have been working on the refining final content.  The most recent dates that were shared with me are beginning September.  You can contact me offline if that will help you.

  • Is there any news on when the library will be released?

Remove this comment

Remove this thread

close

Comment on the Post

Already have a Channel 9 account? Please sign in