Expert to Expert - Joe Duffy: Perspectives on Concurrent Programming and Parallelism

Posted By: Charles | Feb 17th @ 9:39 AM | 55,612 Views | 55 Comments
Joe Duffy spends a lot of time thinking about the future of concurrent programming and parallelism. In his role as Lead Developer in the Parallel Computing Platform team, Joe is the creator of PLINQ and a key contributor to many of the managed (.NET) concurrency incubations happening in and around his broader team. He's also an author (check out his latest book, Concurrent Programming on Windows)

You've met Joe many times before on C9 and the concurrency topic should be quite familiar to you by now (There's a lot of very innovative thinking going on in the parallel computing platform team (and it's not just about the managed world, as you know)).

We've spent a lot time discussing library-based approaches to enabling parallelism in a readily understanable, predictable, safe and scalable way for .NET programmers. We've also spent time on language level approaches to the problem (new constructs in C# that make it easier to compose in a semi-functional way (lamdas, LINQ, etc) or purely in a hybrid-functional way in F# or with experimental DSLs like Maestro).

Erik Meijer, Expert to Expert host, programming language designer and one of the high priests of the lamda calculus  spends a great deal of time thinking about the problem of software's capability to scale effectively (as efficiently, safely, and as composable as possible) in the Many-Core age. So, we add Joe + Erik and we get many excellent, insightful questions and answers. Of course the notion of side-effects plays a big role here and we even debate the merits of Haskell in the real world. This is a great conversation.  It goes deep, but not so far into the rabbit hole that you won't be able to find your way back. Smiley

Enjoy!
Media Downloads:
Rating:
8
0
#Feb 17th @ 11:30 AM
littleguru
littleguru
<3 Seattle

Niiiiiiiiiiiiiiiiiiiice! Thanks guys.

Joe has a point: Haskell only looks complicated. Probably if the language would look more like C/C++, C# or JAVA people would probably use it more and not get scared simply by checking it out. The stuff often is only named in a way that people get scared.

I have already had a similar experience with trying to explain Lambda Expressions in C#. They are not that complicated to get but people are scared when you tell them the term "Lambda Expression(s)". It's as if the name activates a blocking system in the brain.

#Feb 17th @ 11:36 AM
Cyonix
Cyonix
Me
That discussion was great, this is what C9 is about. It's great having Joe on again, his problem domain is an extremely interesting one. If you get the chance I want to know about Joe's new unnamed language that he said he'd been working on.

This video followed the C9 formula for success (in my opinion):

1. Passionate discussion
2. Interesting topic
3. Reference and/or sneak-peak to future direction
4. A conversation not a press release

Note: Although this video followed the above pattern for success, anything with Erik Meijer/Brian Beckman == win
#Feb 17th @ 12:59 PM
tuxbear
tuxbear
I second Cyonix. I just ordered Joe's book Smiley
#Feb 17th @ 1:42 PM
aL_
aL_
Rx ftw
oooo awsome :O i've been dying to get an update from those guys Smiley

--midwatch edit (17:24)--
now this is why i love channel 9.. right here. cutting edge stuff. stuff you're not even sure you can talk about yet. thats what i love about channel9 Smiley just wanted to call that out.. that should be the motto of channel9 Wink

--midwatch edit2 (40:40)--
erics talks about how the dlr is alot about talking to legacy. i dont think thats entirely true.. diffrent problems requre diffrent things of the language and one thing the dlr really enables imo is to have a wider choice of approaches to those problems Smiley i dont think its only for old code, i may want to have things more static in one part of the system and more dynamic in another Smiley
#Feb 17th @ 1:48 PM
sylvan
sylvan
One of the best C9 episodes yet, IMO. Get two people in a room and let them discuss stuff without fear of getting too technical.

I'm with Eric on this one personally. My main beef with Duffy's side of the debate (which is essentially just saying "Yes, we agree in principle, but we don't want to force everyone to learn something completely new so we're going to try to tack it on to what we already have") is the following:

1. I think the cost of not having a "robust" solution to this issue is underestimated. IMO the cost of forcing every man, woman and child with a compiler to learn a Haskell-like purely functional language is peanuts compared to the cost of letting them loose in a manycore world without the adequate tools. The perceived cost may be different, but that's what marketing is for.

2. The complexity of adding pure "bubbles" to an impure language quickly mounts to the point where the overall system is FAR more complicated than Haskell is. Think of all the different "kinds" of effects annotations you would need for a pure subset (transitive const, immutable, locally mutable but not externally visible, etc.). There are many ways of being impure, but only one way of being pure. Pure default makes life easier for both compiler writers and users.

3. It's tempting to be "nice" to existing .Net developers by not forcing them to learn something new, but I wonder if you're really helping them in the long run.  Think about Hoare's nullable pointers - it was added on a whim because it was easy, and now he refers to it as his Billion Dollar Mistake. Sometimes doing the hard thing is what will end up making life easier for people. Related to 1. above, but the main point is "don't be nice for the sake of being nice".

4. Very few entities could pull off kick starting a paradigm shift. If not microsoft, then who? Nobody is my guess.


That said, the disagreement is far smaller than it may appear. Pretty much everyone agrees that in a perfect world we'd all just switch to a Haskell-like language, the question is if the real-world cost of doing that outweighs the long term cost of not doing it. Some say no (me included), some say yes.

As a suggestion for future interviews, I'd recommend heading back to the UK and checking up with SPJ et al, they're working on some really cool parallelism stuff in Haskell (specifically nested data parallelism).

Oh, and about SPJs graph that you had on C9 a while back, I don't actually think that he said Haskell was useless, he said Haskell started out being useless (circa 1990) and then progressed to become more and more useful (while remaining safe), whereas other language started out being useful but unsafe, and are now adding features for becoming more safe. So the idea is that both prongs of attacks are nearing the same Nirvana, but with different strategies.
#Feb 17th @ 2:08 PM
Charles
Charles
Welcome Change
The great Simon Peyton-Jones will appear on C9 the next time he's in town (Redmond, WA, USA). And, yes, Erik Meijer will be there too! Smiley That should be fun.

I really enjoyed this particular conversation because it really surfaced the vision behind Expert to Expert!

Thank you Erik and Joe! We will meet again.
C
#Feb 17th @ 2:13 PM
aL_
aL_
Rx ftw
hehe i gotta go with duffy on this Wink
imo even the most perfect system is useless if no one uses it [because its too hard or whatever] Smiley

purity is great but what i like about duffy and that whole team is that they are still pragmatic, they want people to have use for their stuff, not just create something thats perfectly pure beautiful but not useable in a practical sense Smiley

maybe we wont be able to create completly linear scaling programs with out errors while using side affects, but i think we'll always have errors in our programs, no matter how pure they are Smiley

however, there is no right awnser to this cunandrum imo, just diffrent oppinions Smiley if nothing else, you could make a whole lot of money with your haskell programs when they kick my .net programs butt on the million core machines of tomorrow Smiley
#Feb 17th @ 2:35 PM
sylvan
sylvan
But by "practical" you're just talking about marketing, not any technical issues. The issue isn't about using the actual language, that's clearly doable an perfectly practical, the issue is convincing people that they should make the investment to learn something new. That might be difficult, and costly. I guess my point is that making something that's only "90% pure" (which is really a nonsensical concept, like "90% virgin", Smiley) may be less costly up front (because you can extend C# into something that's more messy and complicated in an absolute sense, but offers a smaller relative learning curve because people already know what we have now), but those "10% problems" will add up over the coming decades to dwarf the one-time cost of a paradigm shift.

So I think it's a false dichotomy that we're choosing between "perfect but unusable" and "imperfect but useful".

A small company wouldn't have an option, they would have to go for an evolutionary approach, but a company like Microsoft does have the option of whole-sale paradigm shift. They'd have seminars, and books, and visual studio integration, and maybe even get hardware partners to produce Reduceron-style co-processors for graph-reduction. MS can throw their weight and marketing behind it to make it happen, but very few other companies could.
#Feb 17th @ 2:59 PM
aL_
aL_
Rx ftw
no im not talking about marketing.. im talking about average joe programer who has a problem to solve within a set timeframe. he wants a solution to his problem first and formost, not something that is completly pure and "beautiful" 

purity adds a bunch of constrains on programs (thats the whole point) to make absolutly double sure that they are "correct" but people are able to create things that work even without these constraints Smiley people are writing all kinds of useful stuff in non pure languages. so purity is not a requirement, its an aid imo.

you mention small companies. well its in those companies that .net really gives an advantage because its such a boost in productivity. if microsfot was to make  the entire runtime and bcl pure, those companies would have to look elsewhare for that productivity boost

i dont think c# should become haskel because we already have haskell, if we want to use haskell, use haskell Smiley 

you talk about the "coming decades". yes that is probobly true that in 50 years time we'll open a vs2008 project and sigh loudly but im abolutly sure that we would do that even if we switch to haskell right now Smiley i just dont think haskell or functunal purity is the end-all-perfect solution for everything. new paradigms will always come.. Smiley
Microsoft Communities