Tim Tyler: Stopping superintelligence

A video discussing the possibility of constructing a superintelligent agent that doesn't object to being switched off.

Transcript: http://alife.co.uk/essays/stopping_superintelligence/

Note: This is not a video about stopping the construction of superintelligences - but rather it is about how to make sure they can be turned off - if that is desired.

Category:

Science & Technology

Tags:

• superintelligence

[elchippe]

Why to stop the bulding of these beings? they have the rigth to exist.

[tmtyler]

@elchippe This is not a video about stopping the construction of superintelligences - but rather a video about how to make sure they can be turned off - if desired.

[cyberdaemon]

I think more likely we "upgrade" our own intelligence rather than build a completely separate superintelligent.

[tmtyler]

I discuss my thinking in that area in my "intelligence augmentation" video. See also my "angelic foundations" video.

[cyberdaemon]

@tmtyler Oh , ok... Anyway , my view is that AI and augmented IQ will evolve hand in hand.Its not one or another , its both :P

[tmtyler]

It'll be both - at least up to the point where computers can do everything brains can - and have access to some decent hardware. After that, it isn't clear whether there will be many humans around in the wild.

[Lilithaaaa]

We may end up designing super intelligent systems, who's goal is to watch other systems for flaws or errors. As well as systems who goals are to destroy or turn off systems that are harmful to us. A super intelligent gold mining system might be smarter than us, but not smarter that a super intelligence system acting as a police force. Situations may occur where things happen but it seems very unlikely any one system could cause critical damage to everything as long as its being watched.

[trombone7]

Remember, a super intelligent system need not tip its hand until it is too late for us to pull the plug. If creating a situation in which we are too afraid to shut it off is reasonably easy for me to imagine, imagine a system 2,000,000 times smarter than me set up the chess board. The moment 1) we create intelligence greater than ours, and 2) it decides humans are a burden, we are screwed. Even without arms, legs or eyes, it will unhurriedly, but furtively begin implementing a takeover.

[TheJamesrocket]

@tomakist The question to your answer is no, machines are not moral by any human standard, although they could be programmed to. And yes, they can be made to serve us. Do I think doing so would be a good idea? No.

[nilbud]

The machine is super intelligent, the users are dumb. Why not turn off, as the Hindus say, "I'll be back".

[tomakist]

Perhaps this is not a solvable problem: can a race of logical, moral beings serve a race of illogical, immoral beings

[nambypamby34]

Could do this , could do that, Fire could burn you alive Fire could burn your house down. WE still use it when we go camping. So what your saying is no reason not to evolve and move forward.