JAOO 2007: Erik Meijer and Dave Thomas - Objects, Functions, Virtual Machines, IDEs and Other Fun St

"I mean I think the .Net library is great.  It is wide, but does not really feel fat to me.   I mean how does a functional language help you call something like Dns.GetHostEntry() any better?"

You’re right. If you need a library function like Dns.GetHostEntry() who gives a damn if it’s being called from IronPython or C#. It’s there because people need it, not because of static or dynamic typing.

I thought his objection to fat libraries was that developers should think about the problem more, and that given appropriate languages they can build precisely what they need for their app. themselves without necessarily modelling every aspect in OO and/or automatically falling back on a large stack of OO frameworks.

Having said that, I've never really worked with frameworks like Spring or Hibernate so I don't know how bad the problem is. What are other people's experiences with these kind of frameworks? Good for technical-publishers or good for developers?

In .Net’s case I’d agree that the library weight is muscle, not fat; it’s there to do work for people – though with time it's naturally going to become more convoluted.

very interessting interview (i really like erik). but i have to say some words to charles.

I think charles is in a mindset that functional programming is the way to go and all problems just fade away as soon as you write your code in ruby or F# and so on. I don't know what kind of programms charles is writing, but my experience as a professional developer is that "the old style" of programming is very appropriate for real world problems.

even in the world of multicore CPUs it's not that hard to use all those core for your programm. especially for web and database related software it's very easy. libraries like ParallelFX will help to write nice code. so i don't see that functional programming is the way how programms will be written in the future. (Of course there are some perfect exceptions but please don't nail me on that).

Thank you charles for your reply. I agree that all those nice features like lambda expressions in c# can help to write better code (and of course everyone should use it). But unfortunately a lambda expression alone does not solve the concurrency problem. Take a look at your interview with Anders Hejlsberg and Joe Duffy, even with ParallelFX a lambda expression is just a nice way to express what the programm should do. In fact it's just a delegate and the code that is executed is still the same as before (with all the concurreny problems).

Don't get me worng, i don't blame lambda expressions that they can't solve the concurreny problem - because they are not designed for it. Anders Hejlsberg said in his interview: "You still have to think" and that's the truth.

This was a really interesting discussion that came far around. Very good job.

 

Im very much looking forward to the second part!

I could listen to Dave Thomas for several hours.

He really nailed it about what is going on at colleges in the USA with Java.

More interviews like this Charles!

berni wrote:

But unfortunately a lambda expression alone does not solve the concurrency problem.

Actually, to be more correct a lambda does NOTHING to solve the concurrency problem.  Functional languages solve concurrency by disallowing mutable state.  Lambdas are just one of many techniques that you need to get real work done in the presence of that draconian restriction.

I see a looming disaster if programmers misunderstand the implications of "functional elements" in imperative programming languages.  Functional programming is as much about what you don't do as what you do.  If you use typical functional constructs in a imperative language, you might accidentally take an implicit dependency on the implementation of everything you call -- nothing you call can mutate any state.  This introduces versioning and subtle edge case bugs, as the author is probably unaware and cannot discover that you have taken that dependency.

This brings us to the second looming disaster.  It simply doesn't work to have two classes of code, one of which cannot call the other.  Several of the C# team members have discussed on channel 9 that this is why the C++ const modifier was not carried over into C#.  It has proven frustratingly difficult to predict how other programmers will want to compose code.  When one class of code cannot call another (ie "functional code" with no mutation and "imperative" code that can) one eventually sees parallel frameworks developing as common operations are eventually needed in both classes.

Don't get me wrong, local variable type inference and lambdas in C# are hot features.  Linq has been my biggest boost in productivity since, well C#.  But these features do not make the language functional, and you're setting yourself up for pain if you think they do.

Can you give an example of what you're talking about, John?

John Melville, MD wrote:


Actually, to be more correct a lambda does NOTHING to solve the concurrency problem.  Functional languages solve concurrency by disallowing mutable state.  Lambdas are just one of many techniques that you need to get real work done in the presence of that draconian restriction.

I see a looming disaster if programmers misunderstand the implications of "functional elements" in imperative programming languages. 

That's exactly what i tried to say to charles (my english is not so good), thank you john. He missed this discussion in his last interviews when he was talking to Erik, Anders or Joe Duffy. Actually Joe Duffy was pointing at this problem when he wrote his "a(i)" function to the whiteboard - but it was only one short sentence.

JChung2006 wrote:

Can you give an example of what you're talking about, John?

Say I want to convert a list of strings to a numbered list of strings. This works:

IEnumerable<String> NumberStrings(IEnumerable<String> input) {
  int i;
  return input.Select(s=>String.Format("{0}. {1}", ++i, s));
}

but this does not work:

IEnumerable<String> NumberStrings(IEnumerable<String> input) {
  int i;
  return input.AsParallel().Select(s=>String.Format("{0}. {1}", ++i, s));
}

Evemtually numbers will be repeated or skipped because of race conditions involving the ++i expression.

[For the nitpickers out there: I know that numbering n items is inherently sequential and that trying to do it in parallel is foolish.  I am trying to write a simple example of code that looks like it should work because its "functional," but doesn't.]

Even though I am using ParallelFX, lambdas, LINQ, and all the other functional goo, its not a magic wand that lets me ignore the problem of shared mutable state. If I were using lisp, haskel, or another functional language it wouldn't let me do the ++i, because they don't have mutable state at all.  The success of functional languages (in which no beginning student really believes) is that you can actually write meaningful programs without it.

Then it gets worse because if the ++i were actually a function call.  In functional languages, the language guarentees no mutable state.  In C# I may not know if it mutates state, or it may not do so now and change in the next version.  It might mutate state only in an edge case I didn't test for.  All of the sudden, my correctness depends on a sometimes very subtle implementation detail of (potentially) someone else's code.

Great vid guys.  I did not follow the Sql-to-xml-to-objects slam by Dave.  Sure, you can trans data over the wire more efficient by using a byte[] as we have today with SqlReader.  But I don't want to use SqlReader as using Linq is much easier.  I want an object on the client side and access properties using a "dot" and Intellisense.  Using an object array or late-binding, does not help me here as I have to fall-back to somehow "knowing" field locations by index and converting types from object.  How would a functional approach help this matter of working with sql data?  Any code examples?

John Melville, MD wrote:



berni wrote: But unfortunately a lambda expression alone does not solve the concurrency problem.

Actually, to be more correct a lambda does NOTHING to solve the concurrency problem.  Functional languages solve concurrency by disallowing mutable state.  Lambdas are just one of many techniques that you need to get real work done in the presence of that draconian restriction.

I see a looming disaster if programmers misunderstand the implications of "functional elements" in imperative programming languages.  Functional programming is as much about what you don't do as what you do.  If you use typical functional constructs in a imperative language, you might accidentally take an implicit dependency on the implementation of everything you call -- nothing you call can mutate any state.  This introduces versioning and subtle edge case bugs, as the author is probably unaware and cannot discover that you have taken that dependency.

This brings us to the second looming disaster.  It simply doesn't work to have two classes of code, one of which cannot call the other.  Several of the C# team members have discussed on channel 9 that this is why the C++ const modifier was not carried over into C#.  It has proven frustratingly difficult to predict how other programmers will want to compose code.  When one class of code cannot call another (ie "functional code" with no mutation and "imperative" code that can) one eventually sees parallel frameworks developing as common operations are eventually needed in both classes.

Don't get me wrong, local variable type inference and lambdas in C# are hot features.  Linq has been my biggest boost in productivity since, well C#.  But these features do not make the language functional, and you're setting yourself up for pain if you think they do.

Agreed.

John Melville, MD wrote:



berni wrote: But unfortunately a lambda expression alone does not solve the concurrency problem.

Actually, to be more correct a lambda does NOTHING to solve the concurrency problem.  Functional languages solve concurrency by disallowing mutable state.  Lambdas are just one of many techniques that you need to get real work done in the presence of that draconian restriction.

I see a looming disaster if programmers misunderstand the implications of "functional elements" in imperative programming languages.  Functional programming is as much about what you don't do as what you do.  If you use typical functional constructs in a imperative language, you might accidentally take an implicit dependency on the implementation of everything you call -- nothing you call can mutate any state.  This introduces versioning and subtle edge case bugs, as the author is probably unaware and cannot discover that you have taken that dependency.

This brings us to the second looming disaster.  It simply doesn't work to have two classes of code, one of which cannot call the other.  Several of the C# team members have discussed on channel 9 that this is why the C++ const modifier was not carried over into C#.  It has proven frustratingly difficult to predict how other programmers will want to compose code.  When one class of code cannot call another (ie "functional code" with no mutation and "imperative" code that can) one eventually sees parallel frameworks developing as common operations are eventually needed in both classes.

Don't get me wrong, local variable type inference and lambdas in C# are hot features.  Linq has been my biggest boost in productivity since, well C#.  But these features do not make the language functional, and you're setting yourself up for pain if you think they do.

Agreed.

I do like the idea of lambdas in sql.  In my mind being able to pass a lambda to be executed remotely on sql would be very interesting and reduce a lot of manual slog of writing stored procs and bouncing back and forth between DSLs.  Example:

List<person> list = ...
db.Exec(a =>
    foreach(Person p in a)   
      SqlDB.Person.Add(p));  //Run code on server passing array.

Constrainsts and triggers could be lambas too, or Services so the middle tier could use same constraint code as db constraints and visa-versa.

Just an awesome series of videos Charles - but when are we going to Brian Beckman back on? That dude is in no small part responsible for my falling in love with math again recently.

I did not understand the issue Dave seemed to have with libraries either.  I mean I think the .Net library is great.  It is wide, but does not really feel fat to me.   I mean how does a functional language help you call something like Dns.GetHostEntry() any better?  Maybe I missed the point.

John Melville, MD wrote:



JChung2006 wrote: Can you give an example of what you're talking about, John?

Say I want to convert a list of strings to a numbered list of strings. This works:

IEnumerable<String> NumberStrings(IEnumerable<String> input) {
  int i;
  return input.Select(s=>String.Format("{0}. {1}", ++i, s));
}

but this does not work:

IEnumerable<String> NumberStrings(IEnumerable<String> input) {
  int i;
  return input.AsParallel().Select(s=>String.Format("{0}. {1}", ++i, s));
}

Evemtually numbers will be repeated or skipped because of race conditions involving the ++i expression.

[For the nitpickers out there: I know that numbering n items is inherently sequential and that trying to do it in parallel is foolish.  I am trying to write a simple example of code that looks like it should work because its "functional," but doesn't.]

Even though I am using ParallelFX, lambdas, LINQ, and all the other functional goo, its not a magic wand that lets me ignore the problem of shared mutable state. If I were using lisp, haskel, or another functional language it wouldn't let me do the ++i, because they don't have mutable state at all.  The success of functional languages (in which no beginning student really believes) is that you can actually write meaningful programs without it.

Then it gets worse because if the ++i were actually a function call.  In functional languages, the language guarentees no mutable state.  In C# I may not know if it mutates state, or it may not do so now and change in the next version.  It might mutate state only in an edge case I didn't test for.  All of the sudden, my correctness depends on a sometimes very subtle implementation detail of (potentially) someone else's code.

I pointed out the flaw in the logic of somebody who innocently asked for a parallel version of a for loop in another video thread (the Parallel Fx video thread, I believe).

There are however many problems that can be solved efficiently by leveraging data parallelism, e.g., map-reduce, for which we have no really clean or concise semantics built into our current set of CLR programming languages.

It isn't so much the "functional programming" aspects of functional programming languages that solve this so much as the semantics and syntax for expressing those semantics in such a way that lends itself well to parallelism.

We'll need documented guidance to use these things wisely.  You can write grossly inefficient and-or bug-ridden code in any language like SQL or C#.  That doesn't make these any of these languages less than useful.

Hopefully universities are doing a better job these days of teaching the concepts of functional programming.  When I was in school, I had to go to graduate school for that sort of thing.  I hope computer science curriciula has improved since those days.

Good stuff.

I will have to disagree with (or at least complement) one thing Eric
said: that you have to have lazy evaluation to really avoid side-effects.
I would claim that not even with lazy evaluation, can you avoid
side-effects completely.

The problem is when you include aspects as memory footprint and
timing aspects. These things are very difficult to predict with lazy evaluation, and in many applications, controlling timing and memory
is vital. Partly for this reason, Haskell has strictness annotations to
turn off lazy evaluation.

Not that this is a big problem. You can do lazy evaluation in Erlang
(observe Erlang QuickCheck), but you have to be explicit about it.
Conversely, you must be explicit about strict evaluation in Haskell.
The notion of side-effect free programming is more fundamental,
and states that you must strive to take control of side-effects and
use them only where necessary.

Some side-effects are unavoidable (e.g. the processor gets warm,
as Simon Peyton-Jones pointed out), but may not matter to the
application; other side-effects are unwanted and perhaps
disastrous (some thread frees data which is still needed by
another). In complex systems, control of side-effects is
essential.

/Ulf Wiger