Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Cloud Cover Episode 16 - Big Compute with Full Monte

Download

Right click “Save as…”

Join Ryan and Steve each week as they cover the Microsoft cloud. You can follow and interact with the show at @cloudcovershow

In this episode:  
  • Discuss the lessons learned in building the HPC-style sample called Full Monte.
  • For 'big compute' apps, learn the gotchas around partitioning work, message overhead, aggregation, and sending results.
  • Listen in on how to effectively use queues in Windows Azure.
  • Discover a tip for keeping your hosted services stable during upgrades.

Show Links:

Using Affinity Groups in Windows Azure
Enzo SQL Shard (SQL Azure Sharding)
CloudCache (Memcached in Windows Azure)
Windows Azure Training Kit - June Release
Full Monte Sample

Tags:

Follow the Discussion

  • I'm not sure why, but I chuckle every time I hear "lease the blob" .. Smiley

  • That was an awesome presentation. I really enjoyed the theory-oriented way this episode took... I guess I like episodes like this --with less code-- beacause I can get into the details after, if I really need to --thing is, this time I actually had to dwelve into the details, but I liked being able to choose... long live freedom!

     

    Anyway, great show.  Now, I want to ask you where can I find the work-partitioning theory you talked about today... I know it's a preety common thing in computer science (at least nowadays, where paralelization is more rule than choice), but... what book/paper/source whould you recommend about this topic?

  • Where can I find the code for the Full Monte HPC example?

  • I updated the post above to link to it.  I really should have done that to begin with.  Sorry!

  • Hi Pablo - thanks for watching!  Glad you liked the show.  I haven't seen much in the way of books or papers on partitioning.  What Steve and I were talking about came from experience trying to build this ourselves.  I suspect that how your partition your work will also be highly dependant on not only the application, but some characteristics of the cloud you are working on.  In general, you have to balance your work to overhead ratio.  You have conflicting goals in some cases - maximizing available work (and hence parallelization) while minimizing performance loss by overhead of more work.

     

    For Full Monte, we literally tested a number of ratios of work to message size to find the right balance.  We did not want to starve roles or lose huge chunks of work in failure, but we also did not want to spend more time on overhead than computation.  Turns out it was a tricky balancing act.

     

    Take a look at the code (linked above now) and see how we did it.  That might be a good start.  Thanks again!

     

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.