Eliezer Yudkowsky - The Challenge of Friendly AI (1/3)

[MathematicsInfinitus]

If I understand correctly, we artifice intelligence to operate by our constraints. With such reigns on our creation, how are they to transcend our own capacities?

[KillahMate]

@MathematicsInfinitus With the aid of software and conceptual tools (like the kind Yudkowsky is developing) we are demonstrably capable of creating constructs that transcend our capacities - after all, a jet fighter is under a significant number of criteria many orders of magnitude more powerful (and dangerous) than any of the humans who designed it, or in fact any of the machines that built it. It would be like suggesting that, given a dull knife, we can never construct with it a sharper one.

[MathematicsInfinitus]

@KillahMate You did not really directly address the question. Also, knives are leagues different than artificial intelligence. It is humorous that you would actually use an analogy as an argument, I am certain it is a logical fallacy somewhere.

The highest form of artificial intelligence that we can construct would be one that we could comprehend (in order to construct it.) Therefore, we cannot construct an artificial intelligence that surpasses our comprehension. Do you understand, now?

[KillahMate]

@MathematicsInfinitus Yes, arguing by analogy is a logical fallacy. I was merely using it to clarify my stance on the development of AI - not relying on it to make my case. In my next post I shall refrain from analogy:

[KillahMate]

@MathematicsInfinitus Though a working strong AI would undoubtedly be massively complex, to assume a single engineer need hold that design in their mind in its entirety at all times while working on it would be senseless. Countless machine and software designs are developed by many engineers working in concert, each of them comprehensively grasping only their portion of the design, and the project leads seeing only the broad outlines. Such projects are common. Why should strong AI be different?

[MathematicsInfinitus]

@KillahMate Using your example of many people working together on an AI, wouldn't you concede that the final project is comprehensible by the whole of the group of people working on a project? For some reason I feel as if there is a premise that we disagree on somewhere, perhaps hidden. It is clear to you that you are correct just as it is to me.

I suppose the point I am trying to make is that any product of human comprehension is Humanly Comprehensible. (cont'd)

[KillahMate]

@MathematicsInfinitus I would certainly concede that the final project is comprehensible by the whole of the group. I also believe that the intelligence of that group - if considered *as a single (though heterogeneous) entity* - is greater than the intelligence of its members. That is, the (theoretically) verifiable 'intelligence' (whatever that means) of the group is in certain ways greater than what you would get by arithmetically adding up the individual 'intelligences' of its members.

[MathematicsInfinitus]

@KillahMate The gap between human intelligence and a "higher" intelligence I believe would resemble the gap between human intelligence and the intelligence of a dog. The resemblance is such that we as humans would not be able to understand the "language" of the "higher" AI. Things that they Fathom are humanly unfathomable.

[KillahMate]

@MathematicsInfinitus As for the gap, yours is an arbitrary estimate; the AI will by definition be as intelligent as we make it. The first real one may be less intelligent than an average human, the first 'super' AI only slightly more (as smart as a human genius, perhaps). And don't forget we'll be able to reliably replicate this genius-level intelligence. The AIs you are thinking of may indeed not be fathomable by humans, but they will be by their makers - the AI descendants of our first AIs.

[ancalyme]

...I thought Eliezer was a woman until I saw this, heh.

[lwanatt]

i think ive figured out a simple command utility that should be friendly enough to atleast not destroy us, command functions listed by order of priority 1) allow yourself to be turned off at any time by any phenomena capable of requesting it 2) do not exceed the dimensions of your allocated space physically or informationally 3) consume a maximum of x amount of matter and y amount of energy over z amount of time 4) discover how to make man immortal

[maxaaronlobdell]

@lwanatt What if the AI considered those command functions to be an unnecessary resource overhead that prevented it from more efficiently pursuing its goals? If this AI is truly recursive and has full access to its code, such safeguards could be easily circumvented - and even if those safeguards were 'read only', it is highly unlikely that a superintelligence would be stopped by them for any significant period of time.

[VulcanFleet]

@lwanatt There are so many ways of being immortal where you wish you were dead.