This is also my most recent analysis of the situation.. If we want to increase the probability of navigating a positive rise of super intelligent machines then it may be best to go the open source route... Although I strongly object to letting this genie out of the bottle, banning it's creation is not feasible but also more dangerous. IN either case this may get interesting soon .
SIAI seem to differ from this analysis. They apparently propose secrecy - on grounds of safety.
Frankly, I do not see why anyone else would trust a closed-source intelligent machine shop. It seems to me that the best way to transparently signal positive intentions towards everyone else is to use an open source model.
The forces of secret source and open source seem likely to do battle for quite some time, but once someone devises a system capable of writing its own source - and there are some precursors for that - the whole question of whether humans can read or modify the code becomes moot.
I'm referring mainly to whether the algorithm becomes available to everyone - or whether the knowledge stays secret for a while.
If the secret escapes via some reverse engineering attempt, it does not seem critical whether the decoding is done by man, machine - or some combination of the two.
If machines with even human level intelligence are released open source it will be more disastrous than a skynet scenario.
One anonymous person could one day have an army of thinking machines working to further their personal agenda.
xXAkridXx 2 years ago
An open source release seems like the best possible outcome for humanity as a whole - however I think it has a very low chance of being realized.
tmtyler 2 years ago
This is also my most recent analysis of the situation.. If we want to increase the probability of navigating a positive rise of super intelligent machines then it may be best to go the open source route... Although I strongly object to letting this genie out of the bottle, banning it's creation is not feasible but also more dangerous. IN either case this may get interesting soon .
particleion 2 years ago
SIAI seem to differ from this analysis. They apparently propose secrecy - on grounds of safety.
Frankly, I do not see why anyone else would trust a closed-source intelligent machine shop. It seems to me that the best way to transparently signal positive intentions towards everyone else is to use an open source model.
tmtyler 2 years ago
The forces of secret source and open source seem likely to do battle for quite some time, but once someone devises a system capable of writing its own source - and there are some precursors for that - the whole question of whether humans can read or modify the code becomes moot.
motters2001 2 years ago
I'm referring mainly to whether the algorithm becomes available to everyone - or whether the knowledge stays secret for a while.
If the secret escapes via some reverse engineering attempt, it does not seem critical whether the decoding is done by man, machine - or some combination of the two.
tmtyler 2 years ago