Burton Smith: On General Purpose Super Computing and the History and Future of Parallelism

Posted By: Charles | Feb 13th, 2008 @ 8:34 AM | 23,666 Views | 59 Comments

Formats:

Burton Smith is a Technical Fellow at Microsoft who thinks about ways in which our platform needs to be structured to support general purpose computers that will soon have clustered super computer processing power as we move closer to manycore everywhere (not too far off into the future...). Burton is a parallel computing expert, an industry thought leader in high performance, massively parallel distributed (aka super) computing. Winner of the Seymour Cray Computer Engineering Award, Burton knows a thing or two about how to architect and implement software systems that can succeed in the Age of Manycore.

This is a long and great conversation, unedited of course. You'll want to make some time for this and listen carefully to what Burton says. This is a very important general introduction to parallelism and high performance computing. As always, we can't talk about super computing without addressing program language evolution in the context of manycore (you've seen this quite a bit on C9 over the years). We cover a lot of ground here including Burton's insights into functional programming, transactions, compatability, shared mutable state, operating systems, technical redunancy and the role of Technical Fellows in the post-Bill era.

Enjoy this great introduction to parallelism and the future of our platform technologies and tools as we head into the age of manycore. This is the first in a series of several interviews covering parallel computing and Microsoft's Parallel Computing Platform technologies, specifically.

Low res file for the bandwidth-challenged.

Tags: Computing, High Performance Computing, Parallel Computing, Parallel Extensions, Programming, Software Composability

Rating:

evildictaitor
if( !succeed( try() ) ) { while(true) try(); }

I'm such a pedant, but "Low res file for the bandwidth challenged" means that the file was challenged, not that the people are bandwidth-challenged. You need a hyphen or the entire sentence means something else. (I sat here for a minute wondering why the hell someone would challenge the file)

Charles
Welcome Change

evildictaitor wrote:

I'm such a pedant, but "Low res file for the bandwidth challenged" means that the file was challenged, not that the people are bandwidth-challenged. You need a hyphen or the entire sentence means something else. (I sat here for a minute wondering why the hell someone would challenge the file)

Fixed. Now, watch the interview. Smiley

dot_tom

You might want to run a spell checker over the accompanying text, hint "clustered" Wink

Great video though!

esoteric
λ

I think most of us managed to get the meaning. Anyhow - excellent video! Smiley

deltalmg911

Good to hear from a fellow HPC geek;)

I'm not sure if silos have to be eliminated in quite the way that Burton mentions. Sometimes (in my experience most the time) one group needs a component before another group will be ready to deliver it, even if they have the "winning solution". Say for example the winner is in the middle of a reliablity scrum so they aren't going to be coding anything for the next 1-2 weeks. That will push any component of any meaningful size off for at least a month.

In my view it isn't so much picking the winner up front, but getting a rough API design and letting whoever needs it first code it. Then everyone needs to know that feature has been made so when they run into a need for it they can "cut and paste" it into their app. Otherwise what happens is you pick a winner and the other projects get a stop sign placed on their Gantt charts wherever the point at which the project leaders can deliver that component.

I would like to see a little bit of what I'd like to term "architect in a cloud" where an architect isn't tasked to a particular product group but is free to roam (Bill Gates probably has demonstrated that role best in the past). That way if they have the winning idea but another group than the one they currently are in has spare dev resources or an earlier dependance on the component the architect can float on over to that group and lead the project rather than wait for the resources to be free in his department or steal the resources from another department.

Two arguements for it:

1) It is easier to move one person then a dev team.
2) Architects spending more time circulating about will have a deeper grasp of the company and a better idea of who the "thought leaders" are in a bunch of different niches.
3) Components that are depended on by multiple projects get started earlier.

evildictaitor
if( !succeed( try() ) ) { while(true) try(); }

Just got round to watching it, and I'm impressed. Burton mentions a lot of very important things (like that functional programs can be easilly serialized) and goes into a lot of depth here about why functional-geeks care about functional languages.

Can I just help to clarify for some people - when Burton goes on about everything in functional languages being constant and making things out of these constants, a much better way of putting this is that are composed of functions which are constant with respect to their arguments
, and this allows for more benefits than just compile (and just-in-time) compilation for parallelisation - it also has serious impacts as to how you compute them, so it's even more important than he was suggesting.

Anyway, this was an excellent video, and I hope to see more where they came from in the near future! Good work Burton! (and Charles, yeah. shouldn't forget him Tongue Out

)

sylvan

Great video! Reassuring! He seems to really "get it" (IMO) so it's nice to know that someone like that is walking the halls and nudging things in the right direction.

Dark_Halmut

SQL is a fine example of a functional language that most of us are used to already... but even can become complex when considering a database getting hit by many users simulaneously with row/table locks and buffering modes. It doesn't really solve the race condition that crops up in parallel computing.

evildictaitor
if( !succeed( try() ) ) { while(true) try(); }

Dark_Halmut wrote:

SQL is a fine example of a functional language that most of us are used to already... but even can become complex when considering a database getting hit by many users simulaneously with row/table locks and buffering modes. It doesn't really solve the race condition that crops up in parallel computing.

The bit where the locks are is where the functional bit (SQL) is put down in state - i.e. where SQL stops being functional (storing things is a non-functional thing to do). Of course, if we didn't do that bit, it would never bother to store the data (which would suck as databases go). If your database was immutable (i.e. you never wrote to it), SQL would be purely functional, and achieves massive parrallelism.