rhm said:

I can see computers eventually getting good at the turing test (in fact I'm so surprised that they aren't already, I'm kindof tempted to have a crack at it myself), but there's a big difference between that doing useful work or anything that you might call 'intellectual'.

 

There are big limitations on what a computer can do in any case - it's not just a case of 'we'll write better software and make faster computers and it'll all be ok'. It's provable that a program cannot produce a program that is greater in complexity than itself. Thus you cannot have AI automatically getting more sophisticated by itself (the Skynet situation). That's not to say AI techniques cannot produce more interesting and useful systems and services, but I find it frustrating that personalities are still willing to make 30 year predictions that assume things will just get solved because people are working on them.

 

A program can produce a program more complex than itself. It is trivial to write a program that brute force outputs every combination of assembly instruction, which will given infinite time, result in every single program that you can possibly write/run on that computer.

 

Now validating which of those programs is correct and does what you want, results in the halting problem.

 

And how do you quantify the complexity of a program? If the brute force assembly program can produce every other program possible, it must be the most complex program?