The many very hard problems of Artificial Intelligence are not even close to being "resolved"... AI's statistical component, Machine Learning, should not be confused with with AI itself. That said, you are right. If and when an artifical mind is created (something that can think (meaning generate thoughts autonomously) and perceive subjectively rather than simply following a complex set of predefined rules) then we will face a very compelling ethical dilema. What happens when the thinking machine develops emotions? We all no how emotions impact decision making....C
I wouldn’t be surprised if artificial intelligence some day surpasses the intelligence of humans. Already some components of computers are out performing the human mind. Consider some of the large data bases and the ability to retrieve the information stored on them.
When I think, I realize I am at a disadvantage due to a lack of knowledge or lack of information regarding unknown variables. It’s amazing that I can come to any kind of conclusion at all when making a decision. Some people have a hard time walking down the street because they have so much information to process and yet not enough information to complete their inquiry. The majority of us make decisions when we relatively have very little information on the subject. We make our decisions in ignorance. Our minds make the best decisions possible based on the information at hand and then roles the dice.
I think it may be just as advantages to figure out how to harness the human minds ability to deal with uncertainty as it is to give those same abilities to computers.
I do think we can make machines that reflect our ability to think. Abstractly: if the equations they make are able to be matched against each other over and over again. Like the rolling of dice until it strikes upon the most logically feasible result. I may have just over simplified a very challenging question. Kind of like when I was a kid and I thought all you needed to make a computer game was a card board box and a picture drawn on paper.