Hypothesis: The time it takes for the current set of software development tools (compilers, languages, libraries and runtimes) to evolve to take full advantage of radically new processor architectures (both on the horizon and here now) may be on par with the time it takes to reach a broad level of adoption of something new.

C

What is that saying... Stripe, stream or struggle?

magicalclick said:

Yeah, but, the problem is syncing the data. That is always the hard part. One way I look at it is, something like between SQL and OO programming. Because SQL takes care of the data locked automatically, and SQL itself is parralle. Not sure, if there is way to bring that to OO style.

 

I'm not completely convinced that object orientation is the only way forward.  Speaking SQL for a moment, if you were to look at tables as types and were able to pass them around like variables, you could enable a class of staticly typed solutions for problems that would otherwise require all kinds of dynamic muck or schema diarrhea. 

For linq, I am still trying to wrap my mind around this idea that grouping and aggregation do not need to be coupled.  Erik was talking about this in one of his posts.

I vote just change the plumbing for existing languages and run-times... let me continue to say what to do, using my existing knowledge and code base, and let the platform decide how best to do it.

elmer said:

I don't understand why there should be the need to learn a new language, in order to continue doing the same thing.

 

I vote just change the plumbing for existing languages and run-times... let me continue to say what to do, using my existing knowledge and code base, and let the platform decide how best to do it.

The same thing is in fact part of problem. What I mean is that in order for plumbing to be effective, the higher level expressive abstraction (like a language) needs to describe what eventually gets plumbed. This is precisely why things like metaprogramming (and declarative expression, generally) is the trend. The only reason functional programming has moved into the mainstream mindset, for example, is due to its declarative nature, explicitly controlable side effects, composability, etc, all paramount to building reliable concurrent systems that are compositional. Part of me agrees with your vote, but another part can't help but wonder if we don't, in parallel, start from first principles in the  (con)current context.

C

exoteric said:

Where did that come from? Are you talking about auto-parallelization - or else how could you continue with imperative programming? I must say I like LINQ for the compositionality and the way I can uniformly attack every problem I can think of and build up small building blocks but the same compositionality is not as easy with "purely imperative" programming - or is it?

Are you talking about auto-parallelization

Yes. I really don't want to know or have to care about what I can/can't do on a particular platform.

The complier should be able to work out what is possible, and the runtime/opsys should provide the required services to make the best possible use of the hardware. Certainly, it should be able to do it a lot more accurately and reliably than I could.

e.g. If I want to run a FOR loop, I just want it to go as fast as the platform will allow, and it should be the job of the compiler and underlying runtime/os to decide how best to implement/run it.

Of course, all of that is much easier said than done, and it might not be possible to get ultimate performace, but asking the application programmer to make decisions on when to implement parallel code, is just not going to work for 99% of programmers.

elmer said:

exoteric said:

*snip*

Are you talking about auto-parallelization

 

Yes. I really don't want to know or have to care about what I can/can't do on a particular platform.

 

The complier should be able to work out what is possible, and the runtime/opsys should provide the required services to make the best possible use of the hardware. Certainly, it should be able to do it a lot more accurately and reliably than I could.

 

e.g. If I want to run a FOR loop, I just want it to go as fast as the platform will allow, and it should be the job of the compiler and underlying runtime/os to decide how best to implement/run it.

 

Of course, all of that is much easier said than done, and it might not be possible to get ultimate performace, but asking the application programmer to make decisions on when to implement parallel code, is just not going to work for 99% of programmers.

Much easier said than done, yes. A single for loop does not an application make.

I was asking Charles though.

Charles said:

elmer said:

*snip*

The same thing is in fact part of problem. What I mean is that in order for plumbing to be effective, the higher level expressive abstraction (like a language) needs to describe what eventually gets plumbed. This is precisely why things like metaprogramming (and declarative expression, generally) is the trend. The only reason functional programming has moved into the mainstream mindset, for example, is due to its declarative nature, explicitly controlable side effects, composability, etc, all paramount to building reliable concurrent systems that are compositional. Part of me agrees with your vote, but another part can't help but wonder if we don't, in parallel, start from first principles in the  (con)current context.

C

More and more these day I like the ideas of Languages such as Erlang and Axum, and find my self trying to use some of their ideas in my everyday work.

exoteric said:

elmer said:

*snip*

Much easier said than done, yes. A single for loop does not an application make.

 

I was asking Charles though.

I'm just trying to have a conversation around developing new (revolutionary versus evolutionary) methodologies versus modifying old ones to exploit the advancements in hardware in the most effective manner as possible. Auto-parallelization at the machine level is pretty much science fiction without explicit support at the expressive level way up the abstraction stack.... Or is it?

Of course, throwing everything out that's been invested in for so long is unrealistic, but this is why theory is fun

C

Charles said:

exoteric said:

*snip*

I'm just trying to have a conversation around developing new (revolutionary versus evolutionary) methodologies versus modifying old ones to exploit the advancements in hardware in the most effective manner as possible. Auto-parallelization at the machine level is pretty much science fiction without explicit support at the expressive level way up the abstraction stack.... Or is it?

 

Of course, throwing everything out that's been invested in for so long is unrealistic, but this is why theory is fun

C

Auto-parallelization at the machine level is pretty much science fiction without explicit support at the expressive level way up the abstraction stack.... Or is it?

Yep, it’s not an easy problem to solve, and people have been working on it for a long time... a google search of ‘automatic code parallelization’ will turn up a long list of stuff on it... including some interesting research papers.

In essence, the problem seems to be that it requires some level of “comprehension” by the compiler, understanding what you are trying to do, rather than simple structure analysis.

All way over my head... I just *KNOW* that I could not hand-write parallel code, even if my life depended on it.

elmer said:

Charles said:

*snip*

Auto-parallelization at the machine level is pretty much science fiction without explicit support at the expressive level way up the abstraction stack.... Or is it?

 

Yep, it’s not an easy problem to solve, and people have been working on it for a long time... a google search of ‘automatic code parallelization’ will turn up a long list of stuff on it... including some interesting research papers.

 

In essence, the problem seems to be that it requires some level of “comprehension” by the compiler, understanding what you are trying to do, rather than simple structure analysis.

 

All way over my head... I just *KNOW* that I could not hand-write parallel code, even if my life depended on it.

Not over your head at all. I've already stated the problem you mention: in order for plumbing to be effective, the higher level expressive abstraction (like a language) needs to describe what eventually gets plumbed. New programming abstractions designed to express solutions to general purpose problems in the many core domain will go a long way in speeding up innovation in the software that instructs the eventual machine instructions.

Where's Hal when you need him?

C

Charles said:

elmer said:

*snip*

Not over your head at all. I've already stated the problem you mention: in order for plumbing to be effective, the higher level expressive abstraction (like a language) needs to describe what eventually gets plumbed. New programming abstractions designed to express solutions to general purpose problems in the many core domain will go a long way in speeding up innovation in the software that instructs the eventual machine instructions.

 

Where's Hal when you need him?

 

C

There are many parrallel based language out there. Usually they have some kind of internal messaging system for parralel sync. I still think it is best to go for the route of OO methodology. The main reason OO is so much more "useful" is because it is easier to debug, easier to predict the result, and very "expressive".

Most of the language that failed are because they are hard to use for the most part. SQL survived because it is actually easy comepare to the rest of sementic DB languages. But, even that, people is moving DB to OO because OO is just that much easier to deal with (especially on high performance tasks).

Charles said:

exoteric said:

*snip*

I'm just trying to have a conversation around developing new (revolutionary versus evolutionary) methodologies versus modifying old ones to exploit the advancements in hardware in the most effective manner as possible. Auto-parallelization at the machine level is pretty much science fiction without explicit support at the expressive level way up the abstraction stack.... Or is it?

 

Of course, throwing everything out that's been invested in for so long is unrealistic, but this is why theory is fun

C

There is simply some practical problems that can not be parallelied effectively, if algorithms depend on intermediate data you are SOL until that intermediate data is computed. It's not science fiction it's logicial impossibility. This whole parallel affliction is one of the worst things to ever happen to the software industry. I don't think it's something that most software developers should have to worry about.

Bass said:

Charles said:

*snip*

There is simply some practical problems that can not be parallelied effectively, if algorithms depend on intermediate data you are SOL until that intermediate data is computed. It's not science fiction it's logicial impossibility. This whole parallel affliction is one of the worst things to ever happen to the software industry. I don't think it's something that most software developers should have to worry about.

So use different algorithms.  

Bass said:

Charles said:

*snip*

There is simply some practical problems that can not be parallelied effectively, if algorithms depend on intermediate data you are SOL until that intermediate data is computed. It's not science fiction it's logicial impossibility. This whole parallel affliction is one of the worst things to ever happen to the software industry. I don't think it's something that most software developers should have to worry about.

Unfortunately this "affliction" is caused by the laws of physics that prevent us from linearly scaling up the speed of CPUs. So unless someone can come up with a completely different way to make CPUs that doesn't have this issue, we're stuck with it.

Bass said:

Charles said:

*snip*

There is simply some practical problems that can not be parallelied effectively, if algorithms depend on intermediate data you are SOL until that intermediate data is computed. It's not science fiction it's logicial impossibility. This whole parallel affliction is one of the worst things to ever happen to the software industry. I don't think it's something that most software developers should have to worry about.

Sometimes that is true, others it is only true because we are attempting to do the absolute minimum amount of work possible to solve a problem. I suspect that as CPU cores become more numerous, you'll start to see much wider use of algorithms and languages that rely on attempting multiple possible solutions simultaneously and discarding the results of those that turn out to be unnecessary.

Charles said:

exoteric said:

*snip*

I'm just trying to have a conversation around developing new (revolutionary versus evolutionary) methodologies versus modifying old ones to exploit the advancements in hardware in the most effective manner as possible. Auto-parallelization at the machine level is pretty much science fiction without explicit support at the expressive level way up the abstraction stack.... Or is it?

 

Of course, throwing everything out that's been invested in for so long is unrealistic, but this is why theory is fun

C

It is indeed an interesting topic.

Let me express what annoys me more than anything, which is this

There is not a single platform for software development; .Net did not kill off native code dispite its all-inclusive mantra, Microsoft itself still creates a lot of native software for maximum performance; maybe this picture will change a little as the per-machine processor count increases

This in term means that

Everything is created  twice - native and .Net. The latest example is Rx (which is even triply created; Javascript as well; - cool as that is).

C# and F# are awesome but

Is the "object-functional integration" good enough and does IL have the right abstractions?

Also, given a superbly expressive language like Scala

Wouldn't it be nice to be able to express a whole operating system using extremely modular, fine-grained and clean semantics (assuming better than C/C++) And is the Singularity/Sing# experiment the future for Microsoft?

This also ties into the parallelism and concurrency problem in that

It is preferable to be able to express as much code using a high level of abstraction, thereby pushing abstractions closer to the metal, to have as much code as possible yield the benefits of those abstractions and to maximize integration (IL + P/Invoke vs C# + F#?)

Some quick thoughts from a random developer...

2) The other thing I've not understood is WHY today we are limited to a single main function!!!  To me, if code entry execution point and exit points could be refrenced or indexed like a fucntion table at the beginning of the file, the binary could/should be able to run different functions at the same time, instead of the one large main which must start at one point in code/binary and end at specified exit/exception points. This is an alternate way in my head I've been wondering could be a different approach to the "thread".