AI Quotes

“For example, an AI utilized in the mortgage industry may look at an application to decide credit worthiness of a person in order to approve them for a loan. For simplicity, let’s say the system looks at only a hundred descriptors of the applicant and utilizes a neural network to arrive at a binary approval decision. An explanation which included all hundred features and weights of the neural network would not be very useful, so the system may instead select one of the two most important features and explain its decision with respect to just those top properties, ignoring the rest. This highly simplified explanation would not be accurate as the other 98 features all contributed to the decision and if only one or two top features were considered the decision could have been different. This is similar to how principal component analysis works for dimensionality reduction [75].”
― Roman V. Yampolskiy, AI: Unexplainable, Unpredictable, Uncontrollable

“Comprehensibility Theorem is the first mathematical theorem implying the impossibility of any AI agent or natural agent—including a not-necessarily infallible human agent—satisfying a rigorous and deductive interpretation of the self-comprehensibility challenge. … Self-comprehensibility in some form might be essential for a kind of self-reflection useful for self-improvement that might enable some agents to increase their success”. It is reasonable to conclude that a system which doesn’t comprehend itself would not be able to explain itself.”
― Roman V. Yampolskiy, AI: Unexplainable, Unpredictable, Uncontrollable