Jeremie Averous's Blog, page 29

How the Paperclip-Maximiser Syndrome Has Become a Meme of AI

Have you heard about the paperclip-maximiser syndrome? It is a viral game and is used as a meme for negative consequences of a too powerful Artificial Intelligence. If this AI’s only objective is to improve paperclip production it may finally exploit all of Earth’s resources and beyond doing just that – destroying everything else in its path. This Wired Column explains the idea: ‘The Way the World Ends: Not with a Bang But a Paperclip‘. (an alternative AI meme seems to be the strawberry-picking AI transforming the Earth in a single strawberry plantation)

In this interesting speech ‘Dude, you broke the future!‘, Charlie Stross a known Science Fiction author refers to the Elon Musk feared singularity exactly as the “paper syndrome”… and then points wisely that “Musk isn’t paying enough attention. Consider his own companies. Tesla is a battery maximizer—an electric car is a battery with wheels and seats. SpaceX is an orbital payload maximizer, driving down the cost of space launches in order to encourage more sales for the service it provides. Solar City is a photovoltaic panel maximizer. And so on. All three of Musk’s very own slow AIs are based on an architecture that is designed to maximize return on shareholder investment, even if by doing so they cook the planet the shareholders have to live on. (But if you’re Elon Musk, that’s okay: you plan to retire on Mars.)” So it seems that Elon Musk is exactly doing what he fears AI would do.

This all serves to remind us that any “intelligence” should not pursue a single goal but a balanced set of goals, because maximizing a single indicator is always at the detriment of the overall balance. This is true in management, and could possibly take unexpected proportions when AI gets involved.

How Mindfulness Requires Compassion

Following up from the previous post ‘How We Need to Remember that Mindfulness is Difficult and Messy‘ and the excellent post by Leo Babauta ‘The Honest Guide to Mindfulness‘, the author underlines an essential element: the need for compassion – for oneself and for others.

I like the conclusion of the post: “Mindfulness is only part of the work. The work also requires compassion — for yourself and others. It requires vulnerability and the ability to open your heart. It requires honesty and the willingness to face things. It requires being willing to love things as they are, without needing to control things. It requires letting go of what you think things should be like, letting go of what you think you should have or shouldn’t have. The work requires you to be willing to be curious, to be open, to remain in not knowing.”

Mindfulness is a journey and to be successful, a measure of acceptance of oneself and others is needed. It is actually quite a necessary condition to progress beyond a certain point. Be more compassionate to yourselves and to others!

How We Need to Remember that Mindfulness is Difficult and Messy

This excellent post by Leo Babauta ‘The Honest Guide to Mindfulness‘ reminds us that mindfulness – a highly trendy concept – is difficult and messy.

“If you’re new to mindfulness, it’s easy to get the wrong idea from all the marketing you’ll find online. Images of people at complete peace with the world and themselves, full of bliss, simply by sitting still and meditating for a few minutes … they are beautiful images, but they don’t tell the whole truth.”

It is hard to be mindful, and it will take substantial practice and exercise to reach a satisfactory mindfulness stage. Leo Babauta also underlines that it is very uncomfortable as it will real things you’d probably prefer to remain hidden.

This also means that probably much less people have reached a satisfactory level of mindfulness than what they advertise. Difficult is always filters out those that are really motivated and ready to put the effort. Much less people are probably truly mindful than what they say.

If you want to embrace mindfulness, be ready for quite a tough journey of introspection and seeing the world differently, which can be quite unsettling. Still it is quite a beneficial experience, just fasten your seat-belt!

How to Address potential global ‘Common-Cause Failures’ like Covid-19

Following on our post ‘How the Covid-19 is a ‘Common Cause Failure’ Crisis‘, let’s examine how this type of issues or their consequences could be better prevented.

There are quite a few potential common cause failures that could affect humankind with a sizable probability. Climate change is one obvious. Another example is some kind of solar magnetic eruption that could destroy electronic systems.

How can societies be made more resilient to this type of situation? The only approach is to diversify the means to minimise the possibility that they are all affected at the same time; and in addition build-up reserves to be able to sustain a painful transition to some other operating mode.

The important aspect here is diversity. And we need to recognize that the Collaborative Age needs to foster diversity, contrary to the standardisation that was so characteristic of the Industrial Age. Diversity needs to be express itself in society setup, and also geographically. Collaborative Age does enhance diversity; our supply chains also need to follow suit.

How the Covid-19 is a ‘Common Cause Failure’ Crisis

An aspect which strikes me in the Covid-19 crisis is that it is a typical case of a situation made significantly worse by ‘common-cause’ failures. This is a typical situation in snowballing industrial accidents, which we now see unfolding at global scale.

‘Common-cause failures’ is a situation where the same root cause affects several aspects of the system, and specifically those aspects which were supposed to back-up each other. When they happen, they worsen significantly the outcome of incidents because they remove redundancy. Recent examples in large accidents include: tsunami and subsequent flooding in Fukushima, which damaged all redundant nuclear reactor cooling systems; a flock of geese that stuck both engines at the same time on the aircraft that finally landed on the Hudson river, etc.

For industrial risk engineers, preventing common cause failures is the number one action to prevent major accidents because major accidents by definition are accidents that will bypass all redundancies built in the system.

And this is exactly what happens with the Covid-19. A lot of complaints on the availability of hospital beds and medical supplies are based on the fact that no planning considered the simultaneous problem to happen nationwide and globally. Spare capacity elsewhere was not available any more to compensate for a local overwhelming need. Europe plans was relying on China providing supplies; US emergency response was relying on relocating local casualties to other states…

Thus Covid-19 is a common cause failure and this explains the extent of the snowballing crisis we observe, as many redundancies built in our institutions and supply chains have been affected.

How to Address the Challenge of Weak Signal Detection

As mentioned in our previous post ‘How to Explain Covid-19 Blindness‘, the Covid-19 situation illustrates the more general challenge in complex systems to identify weak signals early and specifically, those that can, with some probability, develop into a crisis of significant consequences.

It is a challenge many organisations are regularly facing. For example in my professional field, project management in complex projects, the challenge to detect weak signals early and act on them is addressed by advanced project control approaches.

Nevertheless, it remains a difficult issue. This monitoring is prone to generate many false alarms; and some actions taken early will also avoid some of those weak signals develop into a situation or a crisis. Therefore, there is a risk that responsible bodies become fed up by too many weak signals and lose their vigilance. Still, maintaining this detection capability remains obviously essential.

In the Covid-19 situation as in some other challenges of humanity, the weak signal was identified and clearly delineated at least in some pockets of medical specialists, and even in some strategic analysis by the military. What was not anticipated was the consequential impact on the economy. This was probably because of a lack of pluri-discipline linkage and scenario planning. In addition there has been a lack of anticipation as soon as the first signs of a possible looming scenario appeared.

As a learning point, it is probably worth as in all complex system issues to setup a multi-disciplinary weak signal challenge team to review on a regular basis those signals and recommend actions.

How to Explain Covid-19 Blindness

In this post “COVID-19’s General Blindness is Also a Journalistic Failure“, Frederic Filloux explains the reasons for a lack of anticipation of journalism on the epidemics.

The observation is that while the possibilities of a pandemics were exposed in many scientific publications and widely available, journalists have not raised the alarm early. Of course this blindness is not limited to journalists, but they could have played a significant role.

According to Frederic Filloux this can be explained by loss of in-house expertise due to newsroom shrinking in the current economic situation of the press. When there is a situation, external experts are asked to help, but the very possibility of detecting a situation is lost.

According to him “Newsrooms harboring experts — in house, or more realistically, on retainers — would have been more likely to read low-noise signals or even connect the dots of apparently unrelated facts, to put together a true picture of what is unfolding.“

It is well known that it is always difficult to detect low-noise signals and raise the awareness of a wider group. But in that case, the low noise signal was apparently not even identified, which is a concern.

What could be the solution? Frederic Filloux is currently supporting the development of an AI-based content editor, and is quite confident that such solutions could help. In my mind, in an ever-accelerating world, keeping more emphasis on memory and long term approaches is also important: countries that had been exposed to SARS 15 years ago did remember what had to be done.

Lack of memory and general loss of expertise in groups that could relay the issues we are facing are certainly important culprits.

How the AIDS Epidemics Gives Us Pointers as to Behavior Changes Post Covid-19

This interesting article in the Atlantic “How the Pandemic Will End” points an interesting parallel with the AIDS epidemics as to how behaviors changed as a result.

“The rise of HIV and AIDS completely changed sexual behavior among young people who were coming into sexual maturity at the height of the epidemic. The use of condoms became normalized. Testing for STDs became mainstream. Similarly, washing your hands for 20 seconds, a habit that has historically been hard to enshrine even in hospitals, may be one of those behaviors that we become so accustomed to in the course of this outbreak that we don’t think about them.”

We can certainly aspect that certain habits and behaviors related to health and personal hygiene will change. We can for example expect that in western countries, wearing masks in public will probably become mainstream and polite as it is in Asia (as a way to protect others from your microbes). It will become more normal to stay home when we have the flu.

Just as AIDS had a deep and profound impact on the young generation at that time, so we can expect this crisis to have a deep impact on the current younger generation as it will influence their world view for the years to come.

How Our Worldview Can Change In Days after Years of Stagnation

I like this amusing little post by Tyler Cowen “World 2.0 — “There are decades where nothing happens, and weeks where decades happen”

Just a reminder how our worldview can change in a few days after having remained stuck for years.

Beyond the fun, this serves to remind us that in nature, the main shifts happen in a catastrophic and sudden manner. Hours of flooding will change more the riverbed than years of flowing, earthquakes or volcano eruptions will model the ground more than years of slow shifts… And this is a common characteristics of all complex systems.

A stable world and civilisation is an illusion, and change always happens suddenly. Let’s get up to it.

How Fast Our Worldview Can Change

One thing I found interesting with the Covid-19 crisis is how quick our worldview can change. I got caught too: one day it seemed quite ludicrous to envisage that we would be submitted to confinement (and the economic consequences of such a decision), and less than one week later it was the most sensible thing to do.

This crisis is not anything that we as humankind should not know how to manage: there have been many before, and there will be many in the future too. Lessons learnt and practices are available. Models of contamination exist. Quarantine as a concept is quite old. For example, in 1918 there were already instructions about how to make individual masks, like today.

And we got caught because we don’t have the individual memory of such an event. The collective memory was not sufficient.

Still we collectively managed to change our worldview, our way of living and working, often in less than a week. And after some adjustment, most of us live through this crisis in a reasonable manner.

We thus seem to remain quite adaptable, individually and collectively, when circumstances so require. And quite fast too. That’s rather good news in the face of the increasing volatile world we will be facing in the next decades.