Anders Hejlsberg, Herb Sutter, Erik Meijer, Brian Beckman: Software Composability and the Future of

Charles, why didn't you take advantage of Roundtable for this interview? I just started watching...and this lineup looks like it's potentially one of the coolest interviews in a long time

Sounds good man! Your interviews definitely aren't lacking! Keep up the great work.

I think the reason why you had compiler developers saying that "F# was their favourite dynamic language" is more due to an issue of your refined (limited) definition of "dynamic", in this context. ie A question that (mischievously ?) sets up the answerer to fail your implicit, scope-limited defintion... [A]

This kinda reminds me of some experients that were done on children of varying ages when asked "which of these two straws is taller". The children were individually sat down at a table with two identical straws (length, colour, etc). However, one of the straws was moved further away. The three year olds (I'm not actually sure of this age, but it'll do) always selected the straw furthest away. In other words, the problem was actually a mismatch between the experience of the questioner and answerer. The questioner's definition of "taller" was more refined than the child's. However, once those children had been shown what was actually meant by the question, they moved the straws next to each other, and (effectively) answered "they are both the same".


PS: Hey, I couldn't resist given Anders' little dig about Academics and functional programming...

REALLY good interview !

Actually "weak delegate" is the thing which I hope can be built into CLR3.0

Sheva.

footballism wrote:

Actually "weak delegate" is the thing which I hope can be built into CLR3.0

Sheva.

Agreed too.

THis video is very cool. The topics answer some good questions, although other stuff could have been answered there.




On the whole its a very good interview. (Now if we could have had the VB team there too (Amanda Silver and the others) it would have been even better.

I hope this gets repeated again, because its like a dream come true for alot of people including me, for all the lead devs in the languages discussing the future and the existing problems that devs face today and how to solve these.

I agree that Locks in threads are very limiting and this whole model must be gone and replaced with a new multi-threading sub-model, where things are faster without the problems surrounding locks.

Linq is I think the best thing that happened to .NET in quite a while, and its adoption in the .NET would make .NEt even more attractive. I am a big fan of Linq because it makes programming fun and more productive on the whole.

If we can only find a way to emplement a zero-tolerance policy on tampering with MSIL as it goes form dev machine to users machine to CLR, I would be a very happy .NET developer and I think alot of people who depend on .NET to develop would be happier too.

All I can say, keep the good stuff coming!

Onward!!!

How much did Anders get for plugging the TalkingRain Beverage Company?

Dummy wrote:

How much did Anders get for plugging the TalkingRain Beverage Company?

Talking rain have been featured at C9 several times.
And sparkling water is the win anyways!

Excellent premise! A historical video and I'm one of the few that is actually interested in history and genius personality.

An exceptional interview.

One of your very best Charles.
 
Congratulations and thank you.

The trend is towards the CLIFF:

Composable Languages Imperative Functional Fussion (CLIFF)

It is enlightening to learn functional programming just for the different perspective on coding.  However, I doubt that it will be very useful in real world application development.

I would have been more interested in a discussion about how Microsoft language developers intend to improve support for concurrency and transactional software in the next versions of their respective languages.  Workflow Foundation looks like it has potential, but it would be great if there were better support for it in the languages and good examples of how it would actually be useful in real world application development scenarios.

Good discussion to watch nonetheless.  One of the first in a long time I actually watched from beginning to end.

i have been watching channel9 videos for a while now and i just had to sign up to leave a comment about this video ..so far this has been the most educational video i have seen.

i just graduated from college and i remember studying lisp as a freshman and haskell as a junior ..together with the traditional ones(C/C++/java/assembly) .. didnt like functional programming that much, they seem unnatural to me and i didnt see why i had to learn them(especially lisp) but i enjoy seeing(and using) traces of them in python.

this video just brough back memories(painful ones about functional programming) ...

this was very educational

Great video guys!

I say yes we need better concurrency abstractions for sure.  But until then, we will still need tooling, static analysis, and runtime analysis to help me answer the following:

1) Do I really need a lock here? Might an interlocked or something else be better. Maybe no lock is even needed in some cases - why or why not?

2) How many times am I taking this lock?  How hot is it?  Am I needlessly taking it multiple times?  Should I refactor with finer grain locking or would that just add more overhead?  Tool should help me discover this easily and help me refactor.

3) Where am I at risk of dead-lock or live-lock?

4) We need a way to gaureentee a method (or property) is always called in a locked context. Likewise I may want a gaureentee a lock is not held when a certain method is called.  A set of invariants could also have such compile and runtime debug checks.  Attributes would seem reasonable here.  This would seem easy for the computer to do for me, but hard to do manually as programs get complex.

5) Runtime collection performance analysis.  Should I be using a linked list here or a List<T>.  What would be the projected pros or cons using this or that?

6) Should I be using a closure here or a plain delegate and method. I should be able to know the cost easily.

7) Need easy dynamic code for user level changes to behavior:

RunThis(true,
    delegate(string s)
    {
          x = x + 1;  // may what dynamic change
    }
);

Why is it so hard just to replace 1 line with something else at runtime?   All I really want to do is allow user to plug-in "x = x +5".  But today, they need to write and compile a dll, I need to figure out how to load the assembly and find right namespace and call a delegate and somehow get versioning correct, etc, etc.  I may also want this in a sandbox, with certain things the user code and do and not do.

This really fills me with hope for the future. I've been quite concerned that MS was completely missing the boat with what we need for the future, and I still think they're a bit late on the ball here, but at least it seems like all the head honchos understand that functional programming is the only direction we can move in if we are to have any hope of effectively programming for tomorrow's machines.

I think we're kind of late to the party on the sofware side though. The hardware is already here, and we don't know how to use it yet. We should've been better at predicting this "crisis" and moved to FP much sooner.

Very interesting video Charles! It really fills me with hope for the future.

That rocked! 

staceyw wrote:

Great video guys!

I say yes we need better concurrency abstractions for sure.  But until then, we will still need tooling, static analysis, and runtime analysis to help me answer the following:

1) Do I really need a lock here? Might an interlocked or something else be better. Maybe no lock is even needed in some cases - why or why not?

2) How many times am I taking this lock?  How hot is it?  Am I needlessly taking it multiple times?  Should I refactor with finer grain locking or would that just add more overhead?  Tool should help me discover this easily and help me refactor.

3) Where am I at risk of dead-lock or live-lock?

4) We need a way to gaureentee a method (or property) is always called in a locked context. Likewise I may want a gaureentee a lock is not held when a certain method is called.  A set of invariants could also have such compile and runtime debug checks.  Attributes would seem reasonable here.  This would seem easy for the computer to do for me, but hard to do manually as programs get complex.

5) Runtime collection performance analysis.  Should I be using a linked list here or a List<T>.  What would be the projected pros or cons using this or that?

6) Should I be using a closure here or a plain delegate and method. I should be able to know the cost easily.

7) Need easy dynamic code for user level changes to behavior:

RunThis(true,
    delegate(string s)
    {
          x = x + 1;  // may what dynamic change
    }
);

Why is it so hard just to replace 1 line with something else at runtime?   All I really want to do is allow user to plug-in "x = x +5".  But today, they need to write and compile a dll, I need to figure out how to load the assembly and find right namespace and call a delegate and somehow get versioning correct, etc, etc.  I may also want this in a sandbox, with certain things the user code and do and not do.

Just to add another.  Ability to step thru anon delegates like normal code.  It seems to jump around a bit spastic and highlight the whole block which feels wierd.  Maybe just me and not know how to debug anon block right yet. 

Great interview.  I would love to work with any of these guys.  Here's a crazy thought ... probabilistic concurrency.  Instead of locking, duplicate and execute concurrently and keep track of side effects (split it across cores, whatever you like).  Execution paths that result in error once you smack the results back together are probably out of sequence and need to be reordered.  The more concurrent operations, the lower the probability that the compiler or runtime could determine the correct order of execution.  However for 2 operations, say, it would be very likely that the correct order is the path that does not result in an exception.  The depth of the composed objects would also reduce the probability of correctness.  However, if one approached the problem from a probabilistic point of view, tools, and advances in statistical analysis in the compiler and runtime could perhaps improve this over time.

Just a thought.  I am sure this is a vast oversimplification, I see many problems already.

Charles wrote:

How many of you have experiene with functional programming? What do you think about the trend towards a comingling of imperative and functional?

In the video they point out several times that we really need to be able to statically verify what code has side effects, and what kind of effects, and what doesn't. That to me implies purely functional programming. So where's the mix?

I'd love a language like Haskell, but where some of the cool concepts are made a bit more accessible. Eric Meijer points out that while you can do everything they're talking about right now in Haskell, including side effects with static localization guarantees, it is a bit painful to use. I'm not sure that this is necessarily a bad thing (we do want people to use side effects sparingly right? So perhaps making it extremely explicit and verbose is a good way of discouraging its use)?
Anyway, my point is that this is not inherently something wrong with the Haskell way of doing things on a semantic level (purely functional by default, with ways of encapsulating effectful programming).
Take the ST monad for example, this gives you a way of using mutable states in pure functions (ensuring statically that no effects "leak out"). This is possible to do very much in the flavour of a library in Haskell, which is also the reason it's a bit clunky to use. Take the following Haskell code for example, which uses mutable variables (for no good reason) to add two numbers (and a couple of pure versions for comparisons below):

-- this defines an imperative function using mutable state

foo xref yref = do
  x <- readSTRef xref -- extract the value from reference
  y <- readSTRef yref
  return (x+y)

-- here we "run" the imperative function above with "runST"
-- ensuring statically that no side effects "leak"
fooPurified x y = runST ( do
  xref <- newSTRef 5
  yref <- newSTRef 8 
  foo xref yref 
)

-- two versins without imperative features
fooPure1 x y = x+y
fooPure2 = (+)

(btw, note the "but it looks like a dynamically typed language"-effect above, though it is all 100% strongly typed)

Now, clearly the first version is not as elegant as the following two versions. But there is no reason for why the language couldn't have a more convenient built in syntax for doing all this with a C-ish look and feel. The "assembly" like programming style that you must use to do mutable state in purely functional languages is syntactical. There is no reason for why a C-like version of the function above couldn't insert the "readSTRef" style commands automatically with some translation rules (I'm handwaving the details here).

I think a lot of really cool concepts in Haskell could be put in a language just the way they are now (i.e. without sacrificing purity) by just adding a bit of syntacic sugar. For example local mutable state (ST), transactional memory (STM) and IO (IO). The imperative bits of Haskell would thus get a little extra sugar and special translation rules to make it look more like C# or something, without losing the ability to guarantee that a function doesn't have side effects (and thus can be exectued safely on a separate thread if the runtime system decides that it would be apropriate).

What I'm saying is different from what's happening in C# 3.0 because they're taking some of the more superficial aspects of FP and plugging it into an impure imperative envirnoment, which while cool completely misses out on the main benefits of being pure (from a reasoning stand point).
What I'm proposing is the other way around. Start out with a functional language, and add features to make the encapsulation of imperative features within this language be just as convenient as in languages which are imperative at the core. This would seem to me to be the best strategy going forward because it errs on the side of "not losing your ability to reason about the program" which, as we heard in the video, is critical for the compiler and runtime system's ability to do smart stuff with our code (like running it on multiple CPUs). And as they learn more stuff they might make it easier to do imperative programming in this functional environment, but the critical philosophy is that you should start in the pure side of things, because there is no way to add purity later.

Thank you! A really nice show!

... one more thing.. I just signed up to say the above thingie.. thank you charles!

oh.. its possible to edit posts.. sheet...

Declarative languages learn and use 18 - 28 year old.

Funcutional Languages learn and use 35 - 55 years old.

SEP2007 wrote:

Declarative languages learn and use 18 - 28 year old.

Funcutional Languages learn and use 35 - 55 years old.

Natural Languages ... learned and used for at least thousands or 10s of thousands of years.

I periodically check TIOBE to see the relative popularity of programming languages I currently use, C#, Visual Basic, JavaScript, and Transact-SQL.  I used to code in C++ and ActionScript but have given them up for C# and Visual Basic.

The downward trend in Java is interesting.


TIOBE Programming Community Index for January 2007 January Headline: Ruby declared TIOBE's Programming Language of 2006!

The TIOBE Programming Community index gives an indication of the popularity of programming languages. The index is updated once a month. The ratings are based on the world-wide availability of skilled engineers, courses and third party vendors. The popular search engines Google, MSN, and Yahoo! are used to calculate the ratings. Observe that the TIOBE index is not about the best programming language or the language in which most lines of code have been written.

The index can be used to check whether your programming skills are still up to date or to make a strategic decision about what programming language should be adopted when starting to build a new software system. The definition of the TIOBE index can be found here.

Position Jan 2007	Position Jan 2006	Delta in Position	Programming Language	Ratings Jan 2007	Delta Jan 2006	Status
1	1		Java	19.160%	-3.10%	A
2	2		C	15.807%	-3.20%	A
3	3		C++	10.425%	-1.04%	A
4	5		(Visual) Basic	9.123%	+0.03%	A
5	4		PHP	7.943%	-1.46%	A
6	6		Perl	6.237%	-0.81%	A
7	7		C#	3.521%	-0.03%	A
8	8		Python	3.502%	+0.90%	A
9	10		JavaScript	2.845%	+1.31%	A
10	21	11 *	Ruby	2.519%	+2.15%	A
11	11		SAS	2.343%	+1.18%	A
12	9		Delphi	2.336%	+0.75%	A
13	12		PL/SQL	1.570%	+0.54%	A
14	22	8 *	D	1.335%	+0.97%	A-
15	20		ABAP	1.229%	+0.82%	A-
16	14		Lisp/Scheme	0.674%	+0.07%	B
17	18		Ada	0.638%	+0.17%	B
18	13		COBOL	0.637%	-0.13%	B
19	15		Pascal	0.570%	+0.04%	B
20	34	14 *	Transact-SQL	0.510%	+0.34%	B

Long term trends

The long term trends for the first 10 programming languages can be found in the line diagram below.

http://www.tiobe.com/tpci.htm

Long term trends

The long term trends for the first 10 programming languages can be found in the line diagram below.

[/quote]

What was it that happened in 2004?

For those of you who havent seen this link here it is.

A research project of compiling c# to javascript.

Leave any thoughts at the google groups

http://jsc.sourceforge.net/

Naturally, there is some low hanging fruit in concurrent space.  Domain specific apis can have collection abstractions.  They understand their own domain so can very precisely reason about what can go in parallel.  As a simple example GetWebPage(string) can have another overload for GetWebPage(string[]) to take an array.  Then the library can scatter multiple url requests and block for all replies or exception.  But then we need to settle on a common pattern for results and handle in a common way across all apis.  How are exceptions handled?  Should all processing just stop and exception thrown or  should I get parcial results?  Should this be a blocking model or IAsyncResult or something else like an async Port result model (i.e. ccr or my www.codeplex.com/pcr library)  The Iasync model is well know, but levels of nesting makes it hard to deal with and reason about.

We also get into all kinds of side issue like fifo and order of results.  Sometimes you care, sometimes you don't.  The service broker or port/queue model seem more likely to solve this problem cleanly, but is not API based. Plus tooling and diag are ~easy with that model as you have queues and can track work and see it flow.  We know things like sql and service broker have this kind of thing near perfect or very close so how to factor that kind of thing into a language. 

Charles wrote:



How many of you have experiene with functional programming? What do you think about the trend towards a comingling of imperative and functional?

C

I've been playing with functional langauges, (mainly Lisp and Haskell),  for about a year or two and have really been enjoing the new angle of looking at things.  Some of these functional features that are now showing up in C# are some of my favorite features from other langauges that I always missed when I've used C++/C# in the past.  I'm extremely happy to see them being added to C# and I'm really excited to see where the language will go in the coming years.

This discussion is great. Forgive late comment, but better late than ever;)

I am very much impressed with capabilities of functional programming as noted here. It is so good to see that aspects of functional programming migrate more mainstream making it evolve in such a nice direction.

Erik Meijer is a Dutchman. I could hear the moment he started talking.

Doei Erik