Cloud Cover Episode 16 - Big Compute with Full Monte
- Posted: Jun 25, 2010 at 3:08 AM
- 34,439 Views
- 5 Comments
Join Ryan and Steve each week as they cover the Microsoft cloud. You can follow and interact with the show at
@cloudcovershow
In this episode:
In this episode:
- Discuss the lessons learned in building the HPC-style sample called Full Monte.
- For 'big compute' apps, learn the gotchas around partitioning work, message overhead, aggregation, and sending results.
- Listen in on how to effectively use queues in Windows Azure.
- Discover a tip for keeping your hosted services stable during upgrades.
Show Links:
Using Affinity Groups in Windows Azure
Enzo SQL Shard (SQL Azure Sharding)
CloudCache (Memcached in Windows Azure)
Windows Azure Training Kit - June Release
Full Monte Sample
Comments Closed
Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums,
or
Contact Us and let us know.
More episodes in this show
-
Cloud Cover Episode 15 - Certificates and SSL
-
Cloud Cover Episode 16 - Big Compute with Full…
-
Cloud Cover Episode 17 - Using Queues
Follow the Discussion
Oops, something didn't work.
What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in. You need to be signed in to Channel 9 to use this feature.What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in and view them all on your notifications page.sign up for email notifications?
I'm not sure why, but I chuckle every time I hear "lease the blob" ..
That was an awesome presentation. I really enjoyed the theory-oriented way this episode took... I guess I like episodes like this --with less code-- beacause I can get into the details after, if I really need to --thing is, this time I actually had to dwelve into the details, but I liked being able to choose... long live freedom!
Anyway, great show. Now, I want to ask you where can I find the work-partitioning theory you talked about today... I know it's a preety common thing in computer science (at least nowadays, where paralelization is more rule than choice), but... what book/paper/source whould you recommend about this topic?
Where can I find the code for the Full Monte HPC example?
I updated the post above to link to it. I really should have done that to begin with. Sorry!
Hi Pablo - thanks for watching! Glad you liked the show. I haven't seen much in the way of books or papers on partitioning. What Steve and I were talking about came from experience trying to build this ourselves. I suspect that how your partition your work will also be highly dependant on not only the application, but some characteristics of the cloud you are working on. In general, you have to balance your work to overhead ratio. You have conflicting goals in some cases - maximizing available work (and hence parallelization) while minimizing performance loss by overhead of more work.
For Full Monte, we literally tested a number of ratios of work to message size to find the right balance. We did not want to starve roles or lose huge chunks of work in failure, but we also did not want to spend more time on overhead than computation. Turns out it was a tricky balancing act.
Take a look at the code (linked above now) and see how we did it. That might be a good start. Thanks again!
Remove this comment
Remove this thread
close