Expert to Expert - Joe Duffy: Perspectives on Concurrent Programming and Parallelism

I second Cyonix. I just ordered Joe's book

oooo awsome :O i've been dying to get an update from those guys

--midwatch edit (17:24)--
now this is why i love channel 9.. right here. cutting edge stuff. stuff you're not even sure you can talk about yet. thats what i love about channel9 just wanted to call that out.. that should be the motto of channel9

--midwatch edit2 (40:40)--
erics talks about how the dlr is alot about talking to legacy. i dont think thats entirely true.. diffrent problems requre diffrent things of the language and one thing the dlr really enables imo is to have a wider choice of approaches to those problems i dont think its only for old code, i may want to have things more static in one part of the system and more dynamic in another

One of the best C9 episodes yet, IMO. Get two people in a room and let them discuss stuff without fear of getting too technical.

hehe i gotta go with duffy on this
imo even the most perfect system is useless if no one uses it [because its too hard or whatever]

purity is great but what i like about duffy and that whole team is that they are still pragmatic, they want people to have use for their stuff, not just create something thats perfectly pure beautiful but not useable in a practical sense

maybe we wont be able to create completly linear scaling programs with out errors while using side affects, but i think we'll always have errors in our programs, no matter how pure they are

however, there is no right awnser to this cunandrum imo, just diffrent oppinions if nothing else, you could make a whole lot of money with your haskell programs when they kick my .net programs butt on the million core machines of tomorrow

But by "practical" you're just talking about marketing, not any technical issues. The issue isn't about using the actual language, that's clearly doable an perfectly practical, the issue is convincing people that they should make the investment to learn something new. That might be difficult, and costly. I guess my point is that making something that's only "90% pure" (which is really a nonsensical concept, like "90% virgin", ) may be less costly up front (because you can extend C# into something that's more messy and complicated in an absolute sense, but offers a smaller relative learning curve because people already know what we have now), but those "10% problems" will add up over the coming decades to dwarf the one-time cost of a paradigm shift.

no im not talking about marketing.. im talking about average joe programer who has a problem to solve within a set timeframe. he wants a solution to his problem first and formost, not something that is completly pure and "beautiful" 

purity adds a bunch of constrains on programs (thats the whole point) to make absolutly double sure that they are "correct" but people are able to create things that work even without these constraints people are writing all kinds of useful stuff in non pure languages. so purity is not a requirement, its an aid imo.

you mention small companies. well its in those companies that .net really gives an advantage because its such a boost in productivity. if microsfot was to make  the entire runtime and bcl pure, those companies would have to look elsewhare for that productivity boost

i dont think c# should become haskel because we already have haskell, if we want to use haskell, use haskell  

you talk about the "coming decades". yes that is probobly true that in 50 years time we'll open a vs2008 project and sigh loudly but im abolutly sure that we would do that even if we switch to haskell right now  i just dont think haskell or functunal purity is the end-all-perfect solution for everything. new paradigms will always come..

I prefer videos that talk about, not now, not now.next, but now.next.next. It's really the strategic research that's interesting. Short-term stuff can be learned from blogs and books, but knowing what directions the research and development is going is very interesting. This Expert-2-Expert interview fits the bill. We get an insight into the status quo, what's comming and further out what's being investigated as potential solutions. And even (although only shortly) the idea of "what would you do if you could start all over (with the CLR)". Another excellent interview.

great, great interview as always.. i dont want to take a break at all   

it touches on alot of things ive been burning cycles in my puny brain on..
the pull between more static and more dynamic, both are beeing called "the solution" but they pull in so diffrent directions and c# is in the middle..

i think what anders said at pdc and other times is so very true. there is no one correct model. sometimes we need to be dynamic, sometimes we want o be really static.. there just isnt a single solution, a single model. so should we accept this and try making using diffrent models easier or should we make it more difficult

i know you dont agree sylvan but i think its better to get 80%-90% there and get high adoption that to get 100% and make 99% of programmers relearn 99% of what they know we cant just ignore the people.. that is unfortunate

banans must be one of the more paralell fruits btw..

I have just ordered Joes book. He obviously is an industry leader in his field. Your right about the .net being used in small companies like the one I work for. We took the business decision to design our financial package in .net due to the customisation offered by a .net environment.

Are the MP3 and MP4 versions going to be made available soon?

Thanks Charles. Any chance the mp3 could be added to Building-Channel-9-Inside-EvNet-Part-1 as well?

Awesome! Joe's Concurrency book was one of the very few comp.sci books I instantly re-read a second time cover to cover last year. Brilliant stuff and I've been waving it under the noses of anyone who codes at work.

I still think you're presenting a false dichotomy. It's not about "beautiful" vs "getting the job done". It's about "the best way to get the job done in 5-10 years when we'll have hundreds of cores". The only thing stopping the "average joe programmer" from spending the effort to learn something radically different is marketing.

its time, not marketing that stopping joe programmer from learning from learning something radically diffrent.. marketing may help convince people to spend that time, but in a lot of cases its just not possible.. if the boss man tells you that he wants the serverapp back up you cannot tell him/her that you gotta go relearn a large portion of your programming skills first yes i know thats an extreme example but time is always an issue in industrial programming

Adding things to c# will make it more complex, this is true. but when its "too" complex is a subjective thing. when c# does become "too" complex for someone, that person will move to another language (hopefully still .net)  if c# becomes more complicated than haskell for newbies, arent the newbies free to use haskell?

My point is that you already do have a language that is pure from the start, haskell, so why not use that if thats what you prefer.. perhaps the best thing would be to have a haskell compiler for the clr (with its own pure bcl). also, theres f#, its a lot more pure than c# and its beeing added as first class language, isnt that sort of what your calling out for?

the lines of programming is pretty blurry, sometimes you need some purity and sometimes you really need to be dynamic, c#4, as a true general purpose language (i dont agree with eric that haskell is GP, not in practice anyway), tries to find a middle road. its not optimal for dynamic programming and its not optimal for pure functional programming, if you want more dynamic/more pure, use a diffrent language
i dont think there can be one language that pleases everyone and trying to force people to do this or that just doesnt work. they have to come willingliy.. i think exposing people to functual constructs through c#4 will entice them to try purer languages like f# or haskell where its applicable and that cant be bad right?

sylvan: I agree with your original post, that the cost of learning something more suitable for many-core development outweighs the cost of the messes that people will produce with C#.... The problem is not that Microsoft are promoting parallel extensions over miracle language 'x', the problem is that 'x' does not exist *and* no one really knows what 'x' is. People have some ideas what 'x' may look like but they are just that: ideas. It doesn't exist because no one in the field really knows what it looks like and how to solve the (many) problems. There are some proposed solutions, most of which while having some positive points look just like what they are: guesses. I defy anyone to say here even what colour the solution will be (yes that's a joke).

can we have the video download for iPhone as well?

Very nice work gents.

I'm still bullish on message passing in the long term for a number of reasons. I find the Erlang world view (if not the details of the language) very appealing. And I systems like Singularity make me even more optimistic about it. I look at IOCP and see the same sort of thing going on.

It might be interesting for a future video to "go deep" on how synchronization primitives work (right down to the opcodes). I'd like to know if we'll ever get to the point where hardware designers can no longer maintain cache coherency system wide.

It might also be interesting to look into how Second Life's new Mono based script engine handles massive parallelism. You could also touch on Erlang style processes and the way the CCR can leverage iterators.

I need a coffee.

> I'd like to know if we'll ever get to the point where hardware designers can no longer maintain cache coherency system wide.

Don't you think we have a big enough problem without that rearing its head   I suppose NUMA may mitigate that ... assuming the overlying system can manage it of course.

That's still just a marketing issue. If your boss doesn't want you to learn "H#", then clearly more marketing is needed until he sends you off on a Microsoft H# seminar.

If you're going to wait until a miracle language appears you'll have a long wait ahead of you. Meanwhile we have languages that are sufficiently more sophisticated (than C# et al) in this area that they almost seem miraculous if you squint your eyes. Why waste that massive benefit because we haven't yet found a magical cure for all problems?

I'm glad you enjoyed the discussion.

I'm reading over all of your comments, and I see many great points being made.

One that I'd like to deposit for your consideration.  Haskell is an ideal language _in certain contexts_.  For some people, those contexts are important enough to learn an entirely new language.  Highly parallel programs may be one such context, where the safety moving wholesale to Haskell brings is worth the cost of switching.  Even if that were true, _most_ .NET developers are not currently salivating for parallelism.  In 5 years?  Maybe.  But not today.  So as a broad blanket statement, it is safe to say that the perceived cost of switching to Haskell is far higher than the perceived benefit for the bulk of the development community.  This is why an incemental, move-select-parts-of-your-program-over-to-the-safe-world-piecemeal, strategy is so attractive.

In addition to that, I mentioned in the video that Haskell is not a panacea.  It has many interesting ideas, but some that I consider to be debatable for the .NET community at large.  Algebraic data types mixed with structural pattern matching -- with type classes for polymorphism -- are useful for a certain class of programming, but telling a whole community of object-oriented developers to switch overnight will not only result in religious clashes, but is probably just plain wrong anyway.  There is a plethora of shared knowledge (e.g., in patterns -- see GoF), collateral (books, articles, training), and frameworks that Windows developers rely on each day, which are strongly tied to the C++-family of languages.  Moreover, I don't believe vanilla Haskell (98) has solved _all_ of the problems associated with composition of separate agents that are performing I/O.  The "one top-level I/O loop" style of programming doesn't scale beyond one coarse-grained agent.  For that, something more like Occam or Erlang is neeeded, and this is crucial to address in order to enable composition of fine-grain with coarse-grain concurrency.

Food for thought, I hope.

Best Regards,
---joe

I agree that Haskell isn't a panacea. I never said it was. The main thing I would want from it is purity with some way of abstracting over effects (and ideally being able to write your own for EDSLs). This is precisely why I think it would be useful for someone with deep pockets to take a stab at creating a new purely functional programming language knowing what we now know. Haskell is about 20 years old, so it certainly has some warts accumulated like any language of that age - I think a benevolent dictator is needed to take the main lessons from Haskell and lift them over onto a clean slate - possibly with provisions to make it less scary for newbies (e.g. C style syntax).

I agree that there are many useful extensions to Haskell that solve particualr problems.  (This is why I was careful to say "vanilla Haskell (98)".)  Concurrent Haskell, Data Parallel Haskell, and STM TVars are all great examples.  It is still unclear to me whether a new langauge ought to dictate a more constrained model for constructing programs, or whether providing a great collection of independent (but composable) packages is better.  The former is typically needed to reach a broad developer audience, but is at odds with some of the most fundamental language design principles that I strongly believe in (e.g., the C++ model of helping developers to do the "right" thing but not preventing them from doing the "wrong" thing).

In any case, we probably agree on one thing: in the long-term, a new language is needed.  We're just debating execution strategies to get there.  I firmly believe we need a stepping stone from here to there, but that "there" is still a very important place to end up.

---joe

haskell might be great for paralellism but like joe said, its not great for everything.  c# is really good for a very wide amout of stuff and i think that means more to most general purpose programmers that excelence in a perticular area

As far as new language goes.. well.. yeah a better language is always great, but i think its easy to over state the importance of languages and understate the importance of philosophies and understanding of what one is doing.. alot of these problems really transcends computing in my view the world is massibly paralell after all.

what i mean is that its prefectly possible to create perfectly pure-no-side-effects applications in any language, its the philosophy thats the core. no language will allways produce perfectly scalabe programs. we must not forget that we, the humans, are the ones who wants to express something and the language is just a means (all be it, an important one) to do that

Can you give an example of something it's not great at?

Why not have something new, not .NET, which is based on what was learnt from .NET, so an updated static type system (perhaps one that can better enable a ~dynamic lang), an intermediate language, native interop, GC.. but base it from functional immutability and native / .net interop.

Wasn't .NET a revolution? it seems to have done well, I don't think you need expect developers to give up their investment in .net code, just as .NET was harnessed and interoped with their native 'investments' (plus I bet companys would actually WANT to 'port' their code up for the advances it gives).


oh and get nullability right.. etc

But really, why not? it won't sell?

Hi sylvan

I wasn't suggesting nothing is done or people do nothing. After all fully parallel applications have been possible for a long time (I have even written some). The problem is not 'can it be done' but 'how do we make it easier'.  Your original point was that it would be better to teach people something new rather than (say) relying on C# to provide the necessary concurrency. Sure it would be good to teach programmers a new paradigm that really helps solve the concurrency issues -- I am just sure we have much idea as to what it is.

sure, though i gotta wonder why you dont choose to ask joe or eric that question..
its not great for anything where the static type of something you need to talk to isnt known.
beeing completly pure also restricts what kind of optimizations you can make. all those guarantees about isolation just does not come for free.
basicaly every arguments that is used against the static:ness of c# is even more applicable to haskell

you can use c#/.net to write
web apps,
ria apps,
desktop apps,
xbox games,
mobile apps and 
embedded apps
all without having to write your own runtime and without using obscure 3rd party libraries. is that true for haskell?

its interesting that you ask me to "compare the amount of work required to get it running" because that doesnt seem to be a requirement for you.. just how much work would it be to create a graphically rich, multi touch application in haskell? id image you'd have to re-invent a whole bunch of things already present in c#/.net

"C# can't easily be compiled" not sure what you're talking about.. the c# compiler is included in the framework but surely you must know that
"Can you generate x86 assembly on the fly into a buffer and then jump into it and start executing in C#?" well, c# is jitted so no. but you can generate IL with AssemblyBuilder/Reflection.Emit. however i dont see that as a very general purpose or command thing to do
"Can you use C# to write hardware?" yes? pinvoke/CreateFile?
"How about reactive animation?" from what i understand, that pretty much what depedency properties do

im not saying it doesnt matter what language we use. im saying that language is not the most important thing, the understanding of the concept is. you seem to think that if we all used haskell, everything would just work, and thats not true.
no language will let you escape the need to understand the problem and if you do understand the problem you truly can implement it in almost any language (even assembler). its just a matter of preference what language you actually use.

the debate between static/dynamic has been going on for what? 40-50 years? knowing that i dont understand how anyone can think there is a simple solution that will just work in any situation.. thats just religion..

the cold hard fact is that haskell is a minority langunage. if it was sooo great, sooo easy to learn, soo general purpose, soo free from problems, it would be more widely used. its not, so it isnt. blaming marketing is pretty lame because haskell has been around for ~20 years, and still hasnt become mainstream. thats not just a marketing problem..

im not married to c#. it has problems and can be made better. its not great for everything.
the fact that you cant admit that about haskell just makes you look silly, and worse than that, you're hindering the progress of haskell by not admitting its flaws..

I didn't ask Eric or Joe because they didn't make the claims you did.

Pretty sure you said explicitly that it wasn't general purpose in the video, at which point Eric protested wildly.

Okay, I guess I my recollection confused that with the statement later where you said that it "doesn't scale" (49:40). Mea culpa. Anyway, that's unsupported too. At the end of the day you could always choose to write 100% of your code in the IO monad at which point you're no worse  (or better) off than an "effects-everywhere" language like C#.

I think we should give people more credit. Most people probably didn't understand basic algebra before they were taught it either. I really don't think Haskell is that big of a deal compared to something like OOP with all the nuances it has. It's just that it doesn't share too much with OOP so it's not as easy to learn as "yet another OOP language", but if you think in terms of the overall effort to go from zero to OOP or zero to Haskell, I'd probably expect Haskell to win.

I agree with that Charles.  Take this Haskell/GUI sample from wikipedia:
gui :: IO ()
gui = do
  f <- frame [ text := "Event Handling" ]
  st <- staticText f [ text := "You haven\'t clicked the button yet." ]
  b <- button f [ text := "Click me!"
                , on command := set st [ text := "You have clicked the button!" ]
                ]
  set f [ layout := column 25 [ widget st, widget b ] ]

Ooo my...I don't see myself ever being comfortable mentally with that and being able to write (or read) my windows apps that way.  Not saying it will never happen, but I just don't feel it (and I am pretty open minded).  Moreover, would Intellisense be able to work with this?

I am interested to hear more thoughts on linear types and more thoughts on types that could have attributes to dictate concurrency behaviors and useage rights (i.e. read only, read/write, reads for these callers, etc.) with compiler enforcement.

I have to watch this show again.  Good show guys.

A random related side:  My 4-way 2.66GHz HP is noticiably slower then my 2-way 3GHz HP in the Vista look and feel category.  I know there is many factors.  But more cores is not really helping the user UI experience more then faster cores in my experience (from the clock on the wall).  This is because the "user" things are bound by the UI loop (including the shell).  So we can not forget about the "fast" path while reasearching many-core solutions.  Hardware and software vendors still need to push each core and not think adding cores is the ultimate solution to the problem.   

Uh, I don't see what's so bad about that? This is pretty much what you'd write in any language. Is it just that the syntax is unfamiliar? That's a fairly shallow reason to dismiss something forever.

I was careful not to say "ever".  I just don't find myself "thinking" in it any time soon.   I am not defending c#, nor am I dissing haskell.  However, I have seen many times people say they can "think" in c# and it just rolls off the brain.  I am in that crowd.  I could think in c# after a couple days.  This can not be down played as this ability (IMO) makes a language useable and popular more then anything else.  If it is not approachable right away, it tends to favor a limited crowd - for right or wrong.  I don't believe in absolutes or language realigions as I have used too many of them and seen them come and go (I admit I loved Cobol and RPG back in the day).  They have to be very natural and readable.   I would rather see more lines for clear intent, then a terse alternative any day.  The "dot" in .Net is also lost here from what I can see, and that would be a big deal.

There is also an issue which you pointed out.  All these 3rd party libriaries with limited or no support that come and go.  That was same reason I finally left my beloved Perl.  It turns into a treasure hunt all the time looking for and downloading libraries and extentions and patches and version mismatches, etc.  That part could be fixed if MS made it a .net language however.  I say, use what you like to write in.  The syntax question however is OT to the larger issue of the right Model for concurrency.  I think the truth lies somewhere between Erlang, CCR, Linq, and Agents (not Agent Smith) as I think Joe was leading to.

BTW.  Anyone *here actually write a non-trivial Windows application written in Haskell and have a URL to sample?  Would be interested in looking at it.  How about Eric?

Could explain specifically what about that snippet makes it hard to "think" about it? You're just saying kinda vaguely that you can't find yourself "thinking" in it, but I don't understand what specifically is posing a barrier to  you.

The one question:

    " If all .net languages eventually compile down to CIL, then why not just focus on analysing the CIL code for parallel optimisations and then literally restructure the code on that common level? (rather than finding a common model for all high level paradigms) "

The simple answer is that when you get down to the CIL a lot of the context has been lost. It is true, of course, that the CIL *is* a representation of whatever you wrote but the translation is one way and the higher level information is lost. For example, this post could be read one letter at a time but without assembling the letters into words the content will be lost.  It may be possible to get some speedup by examining the CIL, CPUs do that right now on the 'real' machine code by doing out-of-order and speculative execution.

thats exactly the point im trying to make (all though i might not be so good at it)
diffrent tools are good for diffrent things, c# id good at alot of thing while say, haskell is really great for parallelization

another point you make that ive also tried to put forth is that the functional thinking is the real gold and that it can be applied across alsmost all languages, purely functional or not

all im saying is that c# and .net is more general purpose given whats available today. and really, thats all that counts. if may be possible to write xbox games in haskell but as you say, it (along with alot of other things) its not supported. and if its not supported at least to some extent, it might as well be impossible im most buisness scenarios.

we differ in what we mean by general purpose. the examples you give, compileing to native and writing hardware are general purpose in a very formal way, but they are very specific applications and not sovething that you do very often as a regular programmer. what i mean is, given all programers, very few do what you describe.. you see, most programmers dont care if its the "language" or if its "api". if its possible with an api, its possible with the language. i know thats not formally correct, but thats the way people reason..
in that regard, haskell is less general purpose as it has a smaller api. it does have the potential to be more general purpose, but in practice it isnt.

you are right though. i dont know much about haskell. im sorry if you feel ive been putting word in your mouth, that was not my intent.

one thing you dont seem to realize however is that just because someone cant (or has the time) to prove to you that something is difficult, that doesnt mean it isnt difficult. no i dont care enough about winning a discussion at a forum to create a whole app proving some point about c# or haskell. but that doesnt mean that i or stacy or charels are lieing when we say we feel something is unintuative.
thats how we feel, theres nothing you can do about that

also, where are these awsome 3dshooters and webapps written in haskell? love to see some links

Specifically, I don't like the " <- ". I don't like loosing the "dot". I don't like lack of explicit statement terminator.

But this is all subjective. Some people love the Mona Lisa, other people see a dude with a wig on.

sounds great charels
as ive said before (though in a far less eloquent manner than you) i truly belive that the concepts, the thinking, is where its at.

learning language x is in a sense a byproduct of learning the concepts behind x imo.. in a way the language is a means to express the concepts.
 you can in some cases learn the language without grasping the concepts, but that makes you a lousy programmer

looking forward to that anders video and more conceptual goodness

Have you really tried "tasting" the syntax (ie worked with it for a couple of weeks)? If not, how can you know you do not like it?

Even though there are religious wars about syntax, I guess most people would adapt pretty easily to another syntax if they really were convinced there was a substantial benefit in using another language.
If our brains didn't adapt easily to weird syntaxes, we wouldn't have seen all those C-inspired syntaxes and Perl would have remained a sick fantasy inside only one man's brain.

RE: “CIL *is* a representation of whatever you wrote but the translation is one way and the higher level information is lost”

I’m not entirely convinced that the metadata relating to the high level thinking/interpretation is even required to solve this kind of problem? (i.e. mapping serial/sequential code to an equivalent and more parallelized version). In some packages such as Simulink (Mathworks) and BlockBuilder for Simulink (Maplesoft) the model of the algorithm/application is analysed and algebraically simplified into a form that is far more efficient, but equivalent. As, at the low level, the entire application can ultimately be described as a system of equations, why would this abstract principle even require metadata to reform the solution? Only the basic procedural/functional subsets/partitions need be maintained.

In your example of a string of characters, even this can be considered a single value as is commonly used in databases and cryptography (see gmp mp bignum library applications for references). So on the base, all of this can simply be considered one large algebraic problem? Or do you disagree?

This simple yet elegant maxi dress from Red Herring is perfect for wearing in the summer   months allowing you too feel cool and comfortable in the heat. With the beautiful       jewelled detail across the bust and subtle lace and stripes running through the dress,     this outfit can also be worn for a night out!