Part of the problem is that it shouldn’t fall to one person, one group, or even one country to decide what a superintelligence’s ultimate goal should be, especially when that utility function will likely be the guiding light of humanity’s evolution or eradication over the next millennium. Yet Brian is putting me in that very position.
