into four principles: • Utilitarianism: Positive conscious experiences should be maximized and suffering should be minimized. • Diversity: A diverse set of positive experiences is better than many repetitions of the same experience, even if the latter has been identified as the most positive experience possible. • Autonomy: Conscious entities/societies should have the freedom to pursue their own goals unless this conflicts with an overriding principle. • Legacy: Compatibility with scenarios that most humans today would view as happy, incompatibility with scenarios that essentially all humans
...more
This one is interesting. How could we possibly be talking about not allowing Super AIs to have autonomy if it is one of our four ethical principles. I would rather trust them to make wiser decisions than us and live/or die with that, the purposfully impose our will on yet another swath of history.
