Calum Chace's Blog, page 3

Longevity, a $56 trillion opportunity. With Andrew Scott

In unguarded moments, politicians occasionally wish that retired people would “hurry up and die”, on account of the ballooning costs of pensions and healthcare. Andrew J Scott confronts this attitude in his book, “The 100-Year Life”, which has been sold a million copies in 15 languages, and was runner up in both the FT/McKinsey and Japanese Business Book of the Year Awards. Scott joined the London Futurists Podcast to discuss his arguments.

Scott is a professor of economics at the London Business School, a Research Fellow at the Centre for Economic Policy Research, and a consulting scholar at Stanford University’s Centre on Longevity. He is an adviser to the UK government and a variety of international organisations.

The aging of society is good news, not bad

People are living longer. Fewer children are dying before the age of five, fewer parents are dying while their children are young, and more people are surviving long enough to meet their grandchildren. This is surely a good news story, but it is too often seen as a bad thing because society is aging, and there are fewer working people to fund the pensions of those who have retired.

The aging of society means we must change public policy towards aging, and also our own attitudes. Whatever age you are today, you have more life ahead of you than previous generations did, which means that more needs to be invested in your future – by your government, but more importantly by you. You need to make investments in your future health, skills, and relationships.

We need more life stages

We need a new way of looking at the three stages of life: education, work, and retirement. It is easy to forget that this perspective is quite new: both teenagers and retirees are recently invented concepts. Governments are already trying to raise the retirement age, and reduce the level of pension payments, and this is understandably unpopular with many voters. It is unacceptable to simply extend the kind of working life familiar to baby boomers to make it last 60 years. We need to move to a multi-stage working life, with career breaks, periods of part-time work, and transitions to new industries becoming the norm rather than the exception.

In the twentieth century, people took much of their lifetime’s leisure after they retired. In this century, people will take probably more of their leisure before they retire.

“The 100-year life” is a non-academic book, and it has clearly struck a chord with a large audience, but Scott wants to help mould public policy, so he has also been writing papers on longevity economics in order to influence central bankers and government officials, the people who make the rules. One of his priorities has been to put a dollar value on extended lifespans. Governments have long placed a value on human life in the context of healthcare, with concepts like quality-adjusted life years, or QALYs, used to inform decisions about whether to fund particular medicines, for instance. They have not typically applied the same logic to the value of extending lifespan.

Four archetypes

Scott and some colleagues employed four archetypal characters to portray the possible outcomes of aging. The first are the Struldbruggs, from Jonathan Swift’s 1726 satirical novel Gulliver’s Travels. Members of this unfortunate race are immortal, but they do age. Struldbruggs above 80 are regarded as useless, and their health keeps deteriorating. Scott thinks this is how many of us currently view the prospects for senior citizens, and this is a serious challenge for policy makers.

The second archetype is Dorian Gray, the Oscar Wilde character who stopped aging until the moment he died, because he magically arranged for a portrait to age and decay instead. This archetype represents the outcome where people get older but do not age biologically, and then die suddenly.

The third archetype is Peter Pan, the character invented by J.M. Barrie. Peter ages, but only very slowly, and he lives a very long time. The fourth and final archetype is Wolverine, the Marvel comics character, who can reverse physical damage, including the damage caused by aging.

$56 trillion

Applying economic values to the extra healthspan and lifespan implied in the Dorian Gray and Peter Pan archetype yields very large numbers. For example, extending every life in the USA for one year with no loss of health would generate $56 trillion. For most countries the result is 4% of GDP.

Scott’s next book, called “Evergreen”, due out in March 2024, poses a simple question. You are very likely to get old. You fear getting old. What are you going to do to ensure that you age well?

Beyond 100

Adding an extra year of healthy life is all well and good, but there is an increasing number of smart people (including people that Scott works with) who think we could live much longer than that. Indeed, many people think that longevity escape velocity is not far away – the time when medical science gives you an extra year of life every year that passes, so you never actually get older. Scott is nervous about this because he thinks our institutions and our attitudes are not yet ready for it.

However, the social, economic and legal mechanisms required to handle everyone surviving to 100 in good health are pretty much the same as those required for us all to live to 120, or 500, or beyond.

Revolutions in longevity

We’ve had the first longevity revolution, which was to get survival rates for the first five years much higher. Arguably there was a second longevity revolution in which we tackled smoking, and many of the communicable diseases which stopped people progressing through middle age into old age. Globally, life expectancy at birth is now 73, which is an extraordinary achievement. We’re about to have the next longevity revolution (either the second or the third), which is to change how we age.

Many people find the idea of greatly increased longevity fantastical. They don’t believe it, and they often don’t like it. But if you talk instead about the malleability of aging, it makes more sense. We all know that rich people live longer, healthier lives than poor people. We all know some old people who are in great shape and others who are frail, and we know that these outcomes can be produced by healthier lifestyles as well as by the genetic lottery. It’s short step from improving healthspan to improving lifespan.

The three-dimensional longevity dividend

Governments around the world are finally facing up to the demographic challenge, which means we need to work for longer. What they are signally failing to do, says Scott, is to help us to age better, and help us to re-train successfully, so that we can all enjoy the extra years of work. As our longevity increases, we need our healthspan to keep up, and we also need our productive capacity to do the same. Not just our ability to work for more years, but our ability to remain fully engaged with life – to enjoy it. Scott calls this a three-dimensional longevity dividend, and he thinks it is almost trivially obvious that this would be good for the economy too.

As usual, Singapore is ahead of the curve on these matters. The UAE is also focused on geroscience and associated social planning. More broadly, governments everywhere are addressing the problem that people start withdrawing from the labour market from the age of fifty. It’s not usually because they decide to retire early and enjoy a life of leisure. It’s usually because of poor health, or the need to care for someone in poor health, or because their skills become outdated, or because they face ageism in the workplace. Figuring out how to keep these people working – and enjoying their work – should be a top priority for policy makers.

Another development that Scott finds interesting is the concern raised by the falling life expectancy figures in the US and the UK. These governments do finally seem to be considering life expectancy to be an important indicator of policy success, along with GDP. He would like to see governments set targets for healthy life expectancy.

Make friends with your future self

You have more life ahead of you, so you need to make a friend of your future self, and give your future self more options. Those options require good health, good relationships, and good finances. It’s not rocket science, but it’s important.

The post Longevity, a $56 trillion opportunity. With Andrew Scott first appeared on .

How to use GPT-4 yourself. With Ted Lappas

The last few episodes of the London Futurists Podcast have explored what GPT (generative pre-trained transformer) technology is and how it works, and also the call for a pause in the development of advanced AI. In the latest episode, Ted Lappas, a data scientist and academic, helps us to understand what GPT technology can do for each of us individually.

Lappas is Assistant Professor at Athens University of Economics and Business, and he also works at Satalia, which was London’s largest independent AI consultancy before it was acquired last year by the media giant WPP.

Head start

Lappas uses GPTs for pretty much every task that involves generating or manipulating text. This includes drafting emails, writing introductions to book chapters, summarising other people’s written work, and re-writing text. He also uses it to write computer code, something that GPT-4 is much better at than its predecessors. The main value in all of these use cases is to provide a head start. It gets past any terror of the blank page with a rough first draft – and in many cases the first draft is not all that rough, it’s maybe 70% of the way to final.

What is slowing GPTs down?

Given this formidable level of usefulness, why has GPT-4 not turned the world of work upside-down, at least for knowledge workers? Lappas thinks there are two reasons. The first is the fear that any data people are working with may find its way into the hands of competitors. The second reason is that before a mature ecosystem of plug-ins develops, GPT is just a brain: it cannot interact with the world, or even with the internet, which reduces its usefulness.

Plug-ins from the likes of Expedia and Instacart are changing this, as will systems like AutoGPT and BabyAGI, which have the ability to connect with other systems and apps. (AutoGPT and its like are hard to use and frustrating at the moment, but that will improve.) This unfolding ecosystem of extensions has been compared with the development of the iPhone app store, which made Apple’s smartphone an invaluable tool. It is much easier to create a plug-in for GPT-4 than it was to create an app for the iPhone: all it takes is 30 minutes and a basic grasp of the Python programming language.

Specific use cases

Lappas gives a specific example of how he uses GPT-4 in his work. He is a reviewer for an academic journal, and for each issue he is allocated around 15 papers. This means he has to read and summarise the papers themselves, plus around five reviews of each paper, and the “rebuttal” (responses) by the paper’s authors. This adds up to around 60 documents. There is no way to avoid reading them all, but GPT-4 makes it dramatically easier and faster for him to produce the summaries. He says it makes him more like a conductor of an orchestra than a writer, and it saves him five or six hours on each issue of the journal.

GPT-4 and marketing copy

Another field where Lappas is aware of GPT-4 making waves is the writing of marketing copy. He lives and works in Athens, and several of his friends there write posts for Amazon and other websites. If they are productive, they can churn out fifteen a day. Some are now using GPT-4 to produce the initial drafts, and some are not. Those who are not are starting to fall behind. He thinks that if he is seeing this happening in Greece, which is not a particularly technophile country, then it must be happening elsewhere too.

Websites like Fiverr and Upwork are forums for companies needing copy to advertise projects for freelance copywriters to bid for. These sites are seeing an increase in the number of pitches for each piece of work, and the suggestion is that freelancers are using GPT-4 to increase the amount of projects they can bid for and fulfil. Unfortunately the quality of the resulting work is not always high – it hasn’t been edited thoroughly – and clients often warn that copy produced by machines will not be accepted. After all, if a machine can produce the copy then the client company could instruct GPT-4 themselves, rather than paying a freelancer to do it.

Higher up the value chain are copy writers who generate their content through bespoke interviews with the personnel of the client, or people suggested by the client, and then transform those discussions into sparkling English. Reports suggest that GPT-4 is making fewer inroads into this level of the market, although that will probably change in time. Lappas reports that his colleagues at WPP are acutely interested in how quickly this march up the value chain will happen.

GPT and travel writing

One of the ways this will happen is that the AI models can be fine-tuned by ingesting samples of work by an experienced copy writer. I do this to help with my “Exploring” series of illustrated travel books. On the GPT-4 platform Chatbase, I trained a new bot by feeding it the contents of six of my previous books. In the settings, I specify the parameters for the book in general, for instance telling the bot to avoid flowery language with lots of adjectives, and to avoid impressionistic introductions and conclusions.

I then write a tailored prompt for each new chapter of the book I am currently working on. Churning out copy at several hundred words a minute, the bot does a reasonable job of imitating my writing style, although I still have to do a substantial amount of editing and fact checking, both to improve the readability, and also to weed out the factual errors and the so-called “hallucinations” – the apparently random inventions it includes.

Advice for the curious and the nervous

Lappas strongly advises everyone who wants to understand what GPTs are capable of now and what they will be capable of in future to play with the models. He also urges us all to invest the time and effort to learn a bit of Python, which will make a wide range of tools available. He is seeing some evidence that people are taking this advice in the rapidly growing number of TikTok videos sharing GPT-based success stories. He notes that GPT-4 itself can actually help you learn how to use the technology, which means it is now easier to learn Python than it was last year, and it is also more worthwhile.

Even if the idea of doing any coding at all is abhorrent, it is still worth playing with GPT-4, and it is worth using it on a project that has legs. It costs £20 a month to get access to GPT-4, so you might as well get your money’s worth. Diving into the model with no clear goal in mind may leave you relatively unimpressed. It is when the model helps you achieve a task which would otherwise have taken a couple of hours or more that you realise its power.

Generating images

GPT-4 was preceded by Dall-E, Midjourney, and many other image generating AIs, but they are still harder to use effectively than their text-oriented relatives. GPTs are now pretty good at analysing and explaining imagery, identifying celebrities in photos, for instance, or explaining that a map of the world made up of chicken nuggets is a joke because maps are not usually created that way.

Midjourney is often said to be the best system for generating images from scratch, although it isn’t the easiest one to use. Like other, similar systems, it still struggles with certain kinds of images, notably fingers, which often appear six to each hand instead of five.

Another useful process is known as in-painting, where the system ingests an image and edits it. For instance it could replace a cocker spaniel with a Labrador, and adapt the background to make the new image seamless. This process is not yet good enough for use by WPP agencies, Lappas reports, but it is close.

Open source

On platforms like Hugging Face you can find open source models, which you can tailor to your own requirements. The open source models are not yet as powerful as the models from the tech giants and the AGI labs (OpenAI and DeepMind) but they are improving quickly, and according to a recently tweeted memo that was presented as a leak from Google, they will soon overtake the commercial models. Their interfaces are less user-friendly at the moment, but again, that will probably change quickly.

2024 is the year of video

Lappas thinks that image generation will be largely solved by the end of 2023, and that 2024 will be the year of video. He concludes by saying that now is the time to jump in and explore GPTs – for profit and for fun.

The post How to use GPT-4 yourself. With Ted Lappas first appeared on .

GPT: to ban or not to ban? That is the question

OpenAI launched GPT-4 on 14th March, and its capabilities were shocking to people within the AI community and beyond. A week later, the Future of Life Institute (FLI) published an open letter calling on the world’s leading AI labs to pause the development of even larger GPT (generative pre-trained transformer) models until their safety can be ensured. Geoff Hinton went so far as to resign from Google in order to be free to talk about the risks.

Recent episodes of the London Futurists Podcast have presented the arguments for and against this call for a moratorium. Jaan Tallinn, one of the co-founders of FLI, made the case in favour. Pedro Domingos, an eminent AI researcher, and Kenn Cukier, a senior editor at The Economist, made variants of the case against. In the most recent episode, David Wood and I, the podcast co-hosts, summarise the key points and give our own opinions. The following nine propositions and questions are a framework for that summary.

1. AGI is possible, and soon

The arrival of GPT-4 does not prove anything about how near we are to developing artificial general intelligence (AGI), an AI with all the cognitive abilities of an adult human. But it does suggest to many experts and observers that the challenge may be less difficult than previously thought. GPT-4 was trained on a vast corpus of data – most of the internet, apparently – and then fine tuned with guidance from humans checking its answers to questions. The training took the form of an extended game of “peekaboo” in which the system hid words from itself, and tried to guess them from their context.

The result is an enormously capable prediction machine, which selects the next best word in a sentence. Many people have commented that to some degree, this appears to be what we do when speaking.

Opinion among AI researchers is divided about what is required to get us from here to AGI. Some of them think that continuing to scale up deep learning systems (including transformers) will do the trick, while others think that whole new paradigms will be needed. But the improvement from GPT-2 to 3, and then to 4, suggests to many that we are nearer than we previously thought, and it is high time to start thinking about what happens if and when we get there. The latest median forecast on the Metaculus prediction market for the arrival of full AGI is 2032.

2. AGI is an X-risk

It is extremely unlikely that humans possess the greatest possible level of intelligence, so if and when we reach AGI, the machines will push past our level and become superintelligences. This could happen quickly, and we would soon become the second-smartest species on the planet by a significant margin. The current occupants of that position are chimpanzees, and their fate is entirely in our hands.

We don’t know whether consciousness is a by-product of sufficiently complex information processing, so we don’t know whether a superintelligence will be sentient or conscious. We also don’t know what would give rise to agency, or self-motivation. But an AI doesn’t need to be conscious or have agency in order to be an existential risk (an X-risk) for us. It just needs to be significantly smarter than us, and have goals which are problematic for us. This could happen deliberately or by accident.

People like Eliezer Yudkowsky, the founder of the original X-risk organisation, now called the Machine Intelligence Research Institute (MIRI), are convinced that sharing the planet with a superintelligence will turn out badly for us. I acknowledge that bad outcomes are entirely possible, but I’m not convinced they are inevitable. If we are neither a threat to a superintelligence, nor a competitor for any important resource, it might well decide that we are interesting, and worth keeping around and helping.

3. Four Cs

The following four scenarios capture the possible outcomes.

• Cease: we stop developing advanced AIs, so the threat from superintelligence never materialises. We also miss out on the enormous potential upsides.
• Control: we figure out a way to set up advanced AIs so that their goals are aligned with ours, and they never decide to alter them. Or we figure out how to control entities much smarter than ourselves. Forever.
• Consent: the superintelligence likes us, and understands us better than we understand ourselves. It allows us to continue living our lives, and even helps us to flourish more than ever.
• Catastrophe: either deliberately or inadvertently, the superintelligence wipes us out. I won’t get into torture porn, but extinction isn’t the worst possible outcome.

4. Pause is possible

I used to think that relinquishment – pausing or stopping the development of advanced AIs – was impossible, because possessing a more powerful AI will increasingly confer success in any competition, and no company or army will be content with continuous failure. But I get the sense that most people outside the AI bubble would impose a moratorium if it was their choice. It isn’t clear that FLI has got quite enough momentum this time round, but maybe the next big product launch will spark a surge of pressure. Given enough media attention, public opinion in the US and Europe could drive politicians to enforce a moratorium, and most of the action in advanced AI is taking place in the US.

5. China catching up is not a risk

One of the most common arguments against the FLI’s call for a moratorium is that it would simply enable China to close the gap between its AIs and those of the USA. In fact, the Chinese Communist Party has a horror of powerful minds appearing in its territory that are outside its control. It also dislikes its citizens having tools which could rapidly spread what it sees as unhelpful ideas. So it has already instructed its tech giants to slow down the development of large language models, especially consumer-oriented ones.

6. Pause or stop?

The FLI letter calls for a pause of at least six months, and when pressed, some advocates admit that six months will not be long enough to achieve provable permanent AI alignment, or control. Worthwhile things could be achieved, such as a large increase in the resources dedicated to AI alignment, and perhaps a consensus about how to regulate the development of advanced AI. But the most likely outcome of a six-month pause is an indefinite pause. A pause long enough to make real progress towards permanent provable alignment. It could take years, or decades, to determine whether this is even possible.

7. Is AI Safety achievable?

I’m reluctant to admit it, but I am sceptical about the feasibility of the AI alignment project. There is a fundamental difficulty with the attempt by one entity to control the behaviour of another entity which is much smarter. Even if a superintelligence is not conscious and has no agency, it will have goals, and it will require resources to fulfil those goals. This could bring it into conflict with us, and if it is, say, a thousand times smarter than us, then the chances of us prevailing are slim.

There are probably a few hundred people working on the problem now, and the call for a pause may help increase this number substantially. That is to be welcomed: human ingenuity can achieve surprising results.

8. Bad actors

In a world where the US and Chinese governments were obliging their companies and academics to adhere to a moratorium, it would still be possible for other actors to flout it. It is hard to imagine President Putin observing it, for instance, or Kim Jong Un. There are organised crime networks with enormous resources, and there are also billionaires. Probably, none of these people or organisations could close the gap between today’s AI and AGI at the moment, but as Moore’s Law (or something like it) continues, their job would become easier. AI safety researchers talk about the “overhang” problem, referring to a future time when the amount of compute power available in the world is sufficient to create AGI, and the techniques are available, but nobody realises it for a while. The idea of superintelligence making its appearance in the world controlled by bad actors is terrifying.

9. Tragic loss of upsides

DeepMind, one of the leading AI labs, has a two-step mission statement: step one is to solve intelligence – i.e., create a superintelligence. Step two is to use that to solve every other problem we have, including war, poverty, and even death. Intelligence is humanity’s superpower, even if the way we deploy it is often perverse. If we could greatly multiply the intelligence available to us, there is perhaps no limit to what we can achieve. To forgo this in order to mitigate a risk – however real and grave that risk – would be tragic if the mitigation turned out to be impossible anyway.

Optimism and pessimism

Nick Bostrom, another leader of the X-risk community, points out that both optimism and pessimism are forms of bias, and therefore, strictly speaking, to be avoided. But optimism is both more fun and more productive than pessimism, and both David and I are optimists. David thinks that AI safety may be achievable, at least to some degree. I fear that it is not, but I am hopeful that Consent is the most likely outcome.

The post GPT: to ban or not to ban? That is the question first appeared on .

Evomics deploys AI and nuclear medicine in the fight against cancer

AI and nuclear medicine

Some of the most innovative uses of artificial intelligence in healthcare today are in the field of nuclear medicine, and thanks to AI, nuclear medicine is demonstrating great potential for cancer treatment. There are around 20 million new cancer cases a year, and around 10 million deaths, which is around one in six of all deaths. The problem, and therefore the opportunity, is vast.

One of the leading companies in the field is Evomics, based in Shanghai and Vienna. It is using the same technology to develop both diagnostics and therapies, and it has ambitious plans for its EV101 compound, which it thinks could become a $10bn blockbuster.

Nuclear medicine compared with radiation therapy

It is easy to confuse nuclear medicine with radiation therapy, or radiotherapy. Radiotherapists bombard tissue with radiation from an external source in order to remove or reduce cancerous cells. Nuclear medicine, by contrast, injects radioactive molecules into the bloodstream, where they act as a drug. The radioactive molecule is known as a “radioligand” (from the Latin “ligare”, to bind), and it can perform a diagnostic function or a therapeutic one.

Once inside the body, the molecule “recognises” proteins expressed by tumorous cells and binds to them. This causes the radioligand to decay, and emit positrons, the antimatter counterpart to electrons. When the positrons encounter electrons they annihilate each other, producing a pair of high-energy photons, which are detected by a device called a Positron Electron Tomography (PET) scanner. This is the diagnostic mode of nuclear medicine.

In therapeutic mode, the radioligand delivers a dose of radiation to the tumorous cell that it binds to, and causes it to die. The radioactivity is limited, so several rounds of treatment are usually needed to tackle the cancer.

Theranostics

Evomics uses the same compound for both diagnostic and therapeutic applications, but in different dosages. EV101 is the therapeutic application and EV201 is the diagnostic version. In combination they are known as a “theranostic” (therapy + diagnostic).

The proteins targeted by the Evomics radioligand are known as fibroblast activation proteins (FAPs). Fibroblasts are long, thin cells which normally help create tissue structure, including collagen, but when they malfunction they replicate uncontrollably, which is what causes cancer. The FAP targeted by EV101 and EV201 is associated with almost all forms of cancer.

“I don’t fear failure”

Evomics’ CEO, Dr Shiwei Wang, co-founded the company with his twin brother Shifeng, and two Vienna-based professors of nuclear medicine, Dr Li Xiang and Dr Marcus Hacker. Before starting Evomics, Dr Wang spent a decade investing in biotech startups, while working in corporate development in the pharmaceutical industry. His employer was not a financial investor, but was looking to secure access to and experience of important emerging technologies. This gave Dr Wang privileged insights into the most promising new technologies. He decided that the combination of nuclear medicine and AI would generate enormous benefits for patients, and would fuel the rise of major new businesses.

Dr Wang started his new company with backing from his former employer, but Evomics remains low-profile at present, so its sources of funding are undisclosed at present.

During his stint in corporate development, Dr Wang collaborated with the venture capital community centred on Sand Hill Road in Silicon Valley, and one of the things he learned from them was that a great technology or a great product are not enough to guarantee success. In addition to that, a business founder needs exceptional motivation. Starting and growing a business is not for the faint-hearted: resilience and self-belief is essential for anyone who is trying to do things a new way. Dr Wang told me that he thinks his single greatest asset as CEO of Evomics is that “I don’t fear failure”. Interestingly, he thinks his self-confidence might stem in part from being a twin.

Applying AI

Deep learning AI systems are at the heart of Evomics’ philosophy, and essential to its work. They optimise the planning and preparation of medical interventions, analyse the images produced, and automate and accelerate the production of reports that clinicians can use. By making all stages of the process more efficient, the AI can reduce the number of scans required, and also the dosage of radiation the patient is subjected to. As we all know, speed is essential in cancer treatment, so by speeding up the analysis process and getting valuable insights to clinicians sooner, diagnoses can be made sooner and more accurately, and lives saved.

AI algorithms can also provide a valuable second opinion when clinicians disagree about the interpretation of images or data.

In future, Evomics is also working on the deployment of large language models, the transformer AI models which have captivated the world in the form of ChatGPT and GPT-4, to analyse and write up the findings of its other techniques.

Heart disease

The gamma rays given off by radioligands can be useful beyond cancer: Evomics is also developing software for SPECT imaging. SPECT stands for single-photon emission computed tomography, which can detect the presence of clogged arteries which could cause heart disease.

Hurdles and opportunities

Significant challenges remain to the successful application of AI in nuclear medicine. The models become more effective when trained on large data sets, and the volume of data available today is limited, and it is rarely organised and labelled consistently. Most deep learning models are trained on two-dimensional images, but the scanned images from nuclear medicine are three-dimensional. And most important, patients, clinicians, and regulators must trust AI before it can be universally deployed. Understandably, many still regard AI systems as unexplainable black boxes.

These hurdles were reduced to some degree by the Covid-19 pandemic, which provided a significant boost to the use of AI techniques in medical imaging, including in nuclear medicine.

In 2016, AI researcher Geoff Hinton, known as the father of deep learning AI, famously said that “if you work as a radiologist you are like Wile E. Coyote in the cartoon. You’re already over the edge of the cliff, but you haven’t yet looked down. There’s no ground underneath.” He was wrong, in that today, seven years later, human radiologists are still in demand, and indeed many countries have a shortage of them. But as the cliché goes, even if AI won’t take your job any time soon, a human who knows how to work with AI probably will.

The bigger picture is that AI is making more and better technologies available, and we all benefit from that.

The post Evomics deploys AI and nuclear medicine in the fight against cancer first appeared on .

The AI suicide race. With Jaan Tallinn

From Skype to Safe AI

In the 1990s and early noughties, Jaan Tallinn led much of the software engineering for the file-sharing application Kazaa and the online communications tool Skype. He was also one of the earliest investors in DeepMind, before they were acquired by Google. Since then, he has been a prominent advocate for study of existential risks, including the risks from artificial superintelligence. He joined the London Futurists Podcast to discuss the recent calls for a pause in the development of advanced AI systems.

Two Cambridge XRisk organisations

In the previous decade, Tallinn co-founded not one but two Cambridge organisations studying the existential risk from AI and other developments. He describes the Centre for the Study of Existential Risk (CSER, pronounced as Caesar), founded in Cambridge, England in 2012, as a “broad-spectrum antibiotic” because it looks at a wide range of risks, including climate change as well as AI. It is more research-based and more academic, and hence slower moving than the Future of Life Institute (FLI). Established in Cambridge, Massachusetts, in 2014, FLI focuses on the two most urgent risks – nuclear weapons and AI.

From Jaan to Yann

FLI’s leadership claims it is now apparent that AI risk is orders of magnitude more serious than all other risks combined, but this is a controversial claim, rejected by highly credentialed members of the AI community such as Yann Lecun, Chief AI scientist at Meta. In fact the debate over AI risk has been tagged as a struggle between Jaan and Yann.

There is absolutely no doubt in Tallinn’s mind that the risk from advanced AI is important and urgent. He invites people to look at the progress from GPT-2 to GPT-4, and ask themselves what probability they would assign to control over AI development being yanked from human hands. How likely is it that GPT-6 will be developed by GPT-5 rather than by humans?

Since the launches of ChatGPT and GPT-4, there seem to be fewer people arguing that artificial general intelligence (AGI), an AI with all the cognitive abilities of an adult human, is impossible, or centuries away. There are some, including members of the editorial team at The Economist, as demonstrated in the previous episode of the London Futurists Podcast. But most XRisk sceptics argue that AGI is decades away rather than centuries away, or impossible. All over the world, Tallinn, says, people are waking up to the progress of AI, and asking how a small number of people in Silicon Valley can be allowed to take such extraordinary risks with the future of our species.

Moratorium

As a result, FLI has changed its strategy. It used to call for debate, but now it is calling for an immediate pause in the development of advanced AI. On 22 March – a week after the launch of GPT-4 – it published an open letter, calling for a six-month pause in the development of large language models like GPT-4.

In a sense, this means that OpenAI’s strategy is working. Its CEO Sam Altman has long argued that making large language models available to the public was important to spark a general debate about the future of AI, and that is now happening. What is not happening, however, is collaboration between the leading labs on AI alignment. Instead they seem to be locked, perhaps reluctantly, in a race to develop and deploy the most advanced systems. Tallinn labels this a suicide race.

People who are uncertain about the likely timing of the arrival of superintelligence should be open to the suggestion of a delay. The harder job is to persuade people – like Yann LeCun – who are confident that superintelligence is nowhere near. Tallinn says he is unaware of any valid arguments for LeCun’s position.

Pause or stop?

FLI had a lot of internal discussion about whether to call for a six-month pause, or an indefinite one. One argument for the six-month version was that if the leading labs were unable to agree it, that would demonstrate that they are caught up in an unhealthy and uncontrollable competition. Another argument for it was to forestall the objection that a moratorium would enable China to overtake the US in AI. It is implausible that China can erase the US’ lead in advanced AI systems in just six months. As it turns out, China has already instructed its tech giants to slow down the development of large language models: the Chinese Communist Party does not welcome consumer-facing products and services which could dilute its control.

Tallinn acknowledges that he is not really seeking a six-month pause, but a longer one, perhaps indefinite. He accepts that six months is not likely to be sufficient to ensure that large language models are provably safe in perpetuity. He argues that they are “summoned” into existence by being trained on vast amounts of data. They are then “tamed” by Reinforcement Learning from Human Feedback (RLHF). He fears that before long, a trained model may emerge which is impossible to tame.

So what can be achieved during a pause, if not AI alignment? FLI has published a document called “Policymaking in the Pause”, which calls for more funding for AI alignment research, and for the strict monitoring and regulation of leading AI labs.

Losing the upside potential of AI

A major problem with the pause, and one of the main reasons why people oppose it, is that an indefinite pause would deprive us of the upside potentials of advanced AI – which many people believe to be enormous. DeepMind has a two-step mission statement: solve intelligence, and use that to solve everything else. And they really mean everything: problems like war, poverty, even death may well be preventable if we can throw enough intelligence at them.

An end to the development of large language models does not necessarily mean an end to the development of all AI. But there is no doubt that these models are by far the most promising (and hence risky) types of AI that we have developed so far.

Bad actors

Even if the labs and the governments in the US and China agreed to a moratorium, it is hard to imagine North Korea or Putin’s Russia agreeing. There are also international crime syndicates and a large number of billionaires to consider. It may be possible to detect the creation of large language model labs today, as they require such large numbers of expensive GPUs, and consume so much energy. But these constraints will fall rapidly in the coming years as Moore’s law or something like it continues.

If push comes to shove, Tallinn accepts that it might one day be necessary to take military action against an organisation or a state which persisted in creating the wrong kind of AI lab.

But whatever bad actors might do, Tallinn argues that a race to create ever-more powerful AIs is a suicide race. If you know that a mass murderer is rampaging, and that your family’s death is inevitable, you wouldn’t kill them yourself. He also argues that a pause will buy us time to either make AI safe, or work out how to avoid it being developed when the necessary hardware and energy costs are much lower.

AI leaders and public opinion

Tallinn reports that the leaders of the main AI labs – with the exception of Yann LeCun at Meta – are somewhat sympathetic to FLI’s arguments. They don’t say so in public, but they are aware that advanced AI presents serious risks, and many of them have said that there could come a point when the development of advanced AI should slow down. Tallinn thinks that like everyone else, they have been taken by surprise by the dramatic improvement in large language model performance.

Anecdotal evidence suggests that when lay people think seriously about the possible risks from advanced AI, they support the idea of a pause. The question now is whether Tallinn and his colleagues can raise it far enough up the media agenda. He is getting numerous calls every day from journalists, and he thinks the public will become more and more agitated, and the question will become, how to “cash out” that concern to disrupt the suicide race.

Over the next few months, Tallinn hopes that the US government will require all AI labs to be registered, and then that no AI models requiring more than 1025 FLOPs (floating point operations) will be allowed. Long enough for us to work out how to stop the suicide race.

 

 

The post The AI suicide race. With Jaan Tallinn first appeared on .

Is AGI possible? With Kenn Cukier

Most media coverage of AI is weak

The launch of the large language model known as GPT-4 has re-ignited the debate about where AI is going, and how fast. A paper by some researchers at Microsoft (which is the major investor in OpenAI, the creator of GPT-4) claimed to detect in GPT-4 some sparks of AGI – artificial general intelligence, a system with all the cognitive abilities of an adult human. The Future of Humanity Institute, an organisation based at MIT that studies existential risks, published an open letter calling for a six-month pause in the development of advanced AI.

But with a few honourable exceptions, and despite the best efforts of many individual journalists, the coverage of AI in most media outlets remains pretty poor. The usual narrative is “Look at this shiny, scary new thing. But don’t worry, it will all turn out to be hype in the end.”

The Economist, an honourable exception

One of those honourable exceptions is The Economist, and Kenneth Cukier, its Deputy Executive Editor, joined the London Futurists Podcast to discuss the prospects for advanced AI. Until recently, Cukier was the host of the paper’s weekly tech podcast Babbage, and he is a co-author of the 2013 book “Big Data”, a New York Times best-seller that has been translated into over 20 languages.

It has been said that The Economist is great at predicting the past but bad at predicting the future. Recently, it has improved its coverage of the future a great deal in one respect – namely its coverage of AI. For the first few years after the 2012 Big Bang in AI, The Economist used to delight in sneering about AI hype, but now it is hard to think of any media outlet that understands today’s AI better or explains it better. Not sparing his blushes, Cukier played a significant role in that change.

One thing that The Economist still doesn’t do with regard to AI is to cast its eye more than five years or so into the future. It avoids discussing what AGI and superintelligence will mean, or a genuine exploration of the Economic Singularity, when machines do pretty much everything that humans can do for money. Cukier suggests that these developments are probably fifty years away, and that although this is within the probable lifespan of some of the paper’s younger staff members, newspapers have not generally been in the business of looking that far ahead.

Increasingly sceptical

He acknowledges that informed speculation could be useful to readers, as perceptions of the future have important secondary effects, such as determining choices about what to study, or what careers to aim for. But he is increasingly sceptical that machines will ever fully replace humans in the workplace, or that AGI is possible. In this respect, he seems to be heading in the opposite direction to most observers, and he seems to be at odds with the people who are working towards the goal of AGI, including Sam Altman and Demis Hassabis, who run the world’s two leading AI labs, OpenAI and DeepMind respectively. Both are confident that AGI is possible, and Altman thinks it may be created within a decade. The central estimate on the prediction market Metaculus for the arrival of a basic form of AGI is currently 2026, just three years away. (It was 2028 when we recorded the episode, which was before the release of GPT-4.)

Cukier thinks that the debate over advanced AI is challenging because people have different definitions of things like AGI, and some of the underlying concepts have turned out to be unhelpful. For instance he suggests that machines first passed the Turing Test some years ago, but the test turned out to be about deception and human frailty rather than about machine capability. This is only true if you regard the test as being passed in a few minutes, whereas its more interesting version would take at least 24 hours, and involve a number of people well-versed in AI. The futurist Ray Kurzweil has bet the entrepreneur Mitch Kapoor $20,000 that a machine will pass this version of the test in 2029. (Personally, I think the Turing Test identifies consciousness, not intelligence.)

AGI is a tricky concept – even a crazy one

The concept of “general” intelligence is tricky. Pretty much all humans have it, but the level varies enormously between us. So what level does an AI have to reach to be considered an AGI?

Cukier goes further. He thinks the idea of an AI which has all the cognitive abilities of an adult human is “crazy” and unattainable. He also thinks it would be undesirable even if it was possible, because humans do so many unwise things – falling in love, smoking cigarettes, getting confused about maths.

He also thinks there is a magical, spiritual dimension to human intelligence, which can never be replicated within a machine. This leads him to conclude that machines can never become conscious, whatever AI engineers may claim about consciousness (as well as intelligence) being substrate independent.

The ship of Theseus

A useful thought experiment to test this claim is to consider a future person who has been diagnosed with a fatal brain disease. They can be rescued by replacing their fleshy neurons, one at a time, with silicon equivalents. Obviously we don’t have the technology to do this today, but there seems to be no reason why it couldn’t happen in the future. At what point in the changeover process would the person’s consciousness disappear? This is known as the Ship of Theseus question, after a famous ship in ancient Greece which underwent so many repairs down the years that not one original component remained. There was vigorous debate among Greek philosophers about whether it remained the same ship or not.

In the case of the ship, the question is academic: it doesn’t really matter whether the ship’s identity is preserved. In the case of the patient, it matters a great deal whether consciousness is preserved.

J.S. Bach and Carl Sagan

Until recently, it was thought that only humans could write music that would inspire profound emotions. Today, machines can write music in the style of Bach which affects listeners even more profoundly than the original. The choice of Bach is not random: Carl Sagan was asked why no Bach was included in the music that is accompanying the tiny spaceship Voyager on its journey into deep space. His reply was that Bach’s music is so sublime that it would have been boasting.

Cukier responds that machines can only imitate existing creators; they cannot create a new type of art in the way that human innovators have done repeatedly throughout history.

Understanding flight

One argument advanced by Cukier and others to support the claim that machines can never be created that will equal the human mind is that we do not understand how the mind works. The problem with this argument is that we know it is not necessary to understand something to be able to create or re-create it. The Wright brothers did not understand how birds fly, and they did not understand how their own flying machines worked. But they did work. Likewise, steam engines were developed before the laws of thermodynamics.

Cukier offers a final, dramatic thought on the subject of fully human-level AI: it would be the pinnacle of human hubris, and even idolatrous of us to seek to become godlike by creating AGI.

The post Is AGI possible? With Kenn Cukier first appeared on .

Against pausing AI research. With Pedro Domingos

Should AI research be paused?

Is advanced artificial intelligence reaching the point where it could result in catastrophic damage? Is a slow-down desirable, given that AI can also lead to very positive outcomes, including tools to guard against the worst excesses of other applications of AI? And even if a slow-down is desirable, is it practical?

Professor Pedro Domingos of the University of Washington is best known for his book “The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World”. It describes five different “tribes” of AI researchers, each with their own paradigms, and it argues that progress towards human-level general intelligence requires a unification of these different approaches – not just a scaling up of deep learning models, or combining them with symbolic AI. (The other three tribes use evolutionary, Bayesian, and analogical algorithms.) Domingos joined the London Futurists Podcast to discuss these questions.

GPTs

Generative Pre-Trained Transformers, or GPTs, are currently demonstrating both the strengths and the weaknesses of deep learning AI systems. They are very good at learning, but they also just make things up. Symbolic learning AIs don’t do this because they are reasoning systems. Some researchers still think that the remarkable abilities of GPTs indicate that there is a “straight shot” from today’s best deep learning systems to artificial general intelligence, or AGI – a system with all the cognitive abilities of an adult human. Domingos doubts this, although he can imagine a deep learning model being augmented with some other types of AI to produce a hybrid system which was widely perceived as an AGI.

In fact, Domingos thinks that even a hybrid system which employed techniques championed by all five of the tribes he describes would still fall short of AGI. Humans can recognise many breeds of dogs as dogs, after seeing a couple of pictures of just one breed. None of the AI tribes has a clear path to achieving that. He thinks that AI is doing what all new scientific disciplines do: it is borrowing techniques from other fields (neuroscience, statistics, evolution, etc.) while it figures out its own, unique techniques. He suspects that AI cannot be a mature field until it has developed its own unique techniques.

Timeline to AGI

Domingos has developed a neat answer to the impossible but unavoidable question of when AGI might arrive: “a hundred years – give or take an order of magnitude”. In other words, anywhere between ten years and a thousand. Progress in science is not linear: we are in a period of rapid progress right now, but such periods are usually separated by periods where relatively little happens. The length of these relatively fallow periods are determined by our own creativity, so Domingos likes the American computer scientist Alan Kay’s dictum that the best way to predict the future is to invent it.

The economic value of AGI would be enormous, and there are many people working on the problem. The chances of success are reduced, however, because almost all of those people are pursuing the same approach, working on large language models. Domingos sees one of his main roles as trying to widen the community’s focus.

Criticising the call for a moratorium

Domingos is vehemently opposed to the call by the Future of Life (FLI) for a six-month moratorium on the development of advanced AI. He has tweeted that “The AI moratorium letter was an April Fools’ joke that came out a few days early due to a glitch.”

He thinks the letter’s writers made a series of mistakes. First, he believes the level of urgency and alarm about existential risk expressed in the letter is completely disproportionate to the capability of current AI systems, which he is adamant are nowhere near to AGI. He can understand lay people making this mistake, but he is shocked and disappointed that genuine AI experts – and the letter has been signed by many of those – would do so.

Secondly, he ridicules the letter’s claims that GPTs will cause civilisation to spin out of control by flooding the internet with misinformation, or by destroying all human jobs in the near term.

Third, he thinks it is a risible idea that a group of AI experts could work with regulators over a six-month period to mitigate threats like these, and ensure that AI is henceforth safe beyond reasonable doubt. We have had the internet for more than half a century, and the web for more than thirty years, and we are far from agreeing how to regulate them. Many people think they cause significant harms as well as great benefits, yet few would argue that they should be shut down, or development work on them paused.

Three camps in the AI pause debate

There are three schools of thought regarding a possible pause on AI development. Domingos is joined by Yann LeCun, Andrew Ng and others in thinking we should not pause, because the threat is not yet great, and the upsides of advanced AI outweigh the threat. The second school is represented by Stuart Russell, Elon Musk and others who are calling for a pause. The third school’s most prominent advocate is Eliezer Yudkowsky, who thinks that AGI may well be near, and that the risk from it is severe. He thinks all further research should be subject to a relentlessly enforced ban until safety can be assured – which he thinks could take a long time.

These camps consist largely of people who are smart and well-intentioned, but unfortunately the debate about FLI’s open letter has become ill-tempered, which probably makes it harder for the participants to understand each other’s point of view. Domingos acknowledges this, but argues that the signatories to the letter have raised the temperature of the debate by making outlandish claims.

In fact he notes that the debate about the open letter is not new. Rather, it is surfacing a long-standing debate between people in and around the AI community, which was already acrimonious.

Stupid AI and bad actors

Domingos thinks another of the mistakes in the letter is that it addresses the wrong problems. Even though he thinks AGI could conceivably arrive within ten years, he thinks it is about as likely that he will get struck by lightning, something he does not worry about at all. He does think it would be worthwhile for some people to be thinking about the existential risk from AGI, but not a majority. He thinks that by the time AGI does arrive, it is likely to be so different from the kinds of AI we have today that such preparatory thinking might turn out to be useless.

Domingos has spent decades trying to inform policy makers and the general public about the real pros and cons of AI, and one of the reasons the FLI letter irritates him is that he fears it is undoing any progress he and others have made.

GPT-4 has read the entire web, so we humans make the mistake of thinking that it is smart, like any human who had read the entire web would be. But in fact it is stupid. And the solution to that stupidity is to make it smarter, not to keep it as stupid as it is today. That way it could make good judgements rather than bad ones about who gets a loan, who goes to jail, and so on.

In addition to its stupidity, the other main short-term risk that Domingos sees from AI is bad actors. Cyber criminals will deploy and develop better AIs regardless what the good actors do, and so will governments which act in bad faith. Arresting the development of AI by the better actors would be like saying that police cars can never improve, even if criminals are driving faster and faster ones.

Control

Domingos thinks that humans will always be able to control the objective function (goal) of an advanced AI, because we write it. It is true that the AI may develop sub-objectives which we don’t control, but we can continuously check the AI’s outputs, and look for constraint violations. He says, “solving AI problems is exponentially hard, but checking the solutions is easy. Therefore powerful AI does not imply loss of control by us humans.” The challenge will be to ensure that control is exercised wisely, and for good purposes.

He speculates that maybe at some point in the future, the full-time job of most humans will be checking that AI systems are continuing to follow their prescribed objective functions.

The post Against pausing AI research. With Pedro Domingos first appeared on .

What do professional futurists do? With Nikolas Badminton

GPT-4 has got people interested in the future again

We’re only a few weeks into 2023, but there has been a sea-change in the thinking of many business people regarding the future. GPT-4 and similar systems look likely to usher in major changes to the way many of us work and play, and they will probably have significant impacts on markets, economies, politics, and international relations. How can businesses become more effective in anticipating and managing these changes in their business landscapes?

A new book which addresses this question of managing rapid change is Nikolas Badminton’s “Facing our Futures: How foresight, futures design and strategy creates prosperity and growth”. Over the last few years, Badminton has worked with over 300 organizations, including Google, Microsoft, NASA, the United Nations, American Express, and Rolls Royce. He also advised Robert Downey Jr.’s team for the “Age of AI” documentary series. Badminton joined the London Futurists Podcast to discuss what it means to be a professional futurist, and how to become one.

Becoming a futurist

At the age of eight, Badminton was captivated by “The Usborne Book of the Future”. Published in 1980, this book described what life would be like in 2000, at the start of the next century. It got some things right, including wearable computing, but it predicted other things which have still not come to pass, including colonies on the moon and under the ocean. Around the same time, he started playing with computers, and he carried on working with computers at university and in his early career, which focused on the use of data in advertising. Before becoming a futurist, Badminton spent most of his career as a consultant, advising clients on technology and business strategy.

In 2008, Badminton moved to Canada. In North America there was much more interest in how technology will shape our futures, and for the first time he encountered people calling themselves futurists. But coincidentally, it was after giving a presentation at a London Futurist event in 2015 that Badminton decided to become a full-time futurist.

The nature of futurism

He describes a typical engagement. Working for a large technology company with 180,000 employees, he said “here are the signals, here are the trends, here are some scenarios we can expect to see, and this is how your objectives and your strategies will be impacted.” Together with an executive – that would become a futurist-in-residence – he reviewed a wide range of outcomes, from demographic trends such as population shrinkage, to government policy on gun ownership and gun crime.

Another example is an engagement for a frozen food company which was poised to make an investment in Valencia, in southern Spain. Badminton drew their attention to a report which described a difficult future of excess heat in that part of the world, and a corresponding “apocalypse windfall” in northern Europe. The company reviewed and revised its investment plans accordingly.

The work generally consists of workshops, and collaborative projects to write fiction that captures what emerge as the most likely scenarios and bring them to life. Clients can be large corporates or startups, and they are all over the world. Badminton works on these engagements with colleagues in an organisation called the Futurist Think Tank. In the last three years, this has become 60% of his work, whereas previously most of his revenue came from keynote talks. So far, he is finding it fascinating and rewarding.

How futurists are trained

If you are an engineer, or a government administrator, and you want to become a strategy consultant, you typically do an MBA and join McKinsey or one of the other strategy firms. There is as yet no equivalent pathway for futurists. There are universities which now offer degrees in foresight, but Badminton’s advice is to travel and experience different types of life and work. You need a broad horizon to be a futurist, and most people will not glean that from the pages of books.

The key is to be open-minded and curious, and shift your mindset from “what-is” to “what-if”. You need to be prepared to ask difficult questions, and see where the logic goes. If the people who championed Brexit had sincerely considered the potential downsides, they might not have pushed as hard for the extreme form of Brexit that was eventually imposed. Who knows, maybe David Cameron, the UK prime minister who launched the referendum which led to the decision, might not have done so.

Dark Futures

Lots of people who call themselves futurists peddle what could be called “future porn”. They gush about what a wonderful world we will live in when we have self-driving cars and robot valets, and they fail to think rigorously about the timelines for the individual technologies, the potential for resistance to their adoption, and the possible downsides. When he hears these people talk, Badminton often wants to put his head in his hands and weep: these glib accounts don’t help people think seriously about the challenges ahead.

We are, he says, born of struggle, perennially subject to war and taxes. He is sceptical that we can reach a society of abundance any time soon. He describes the world envisaged by Peter Diamandis and others as a “pay-to-play situation”, which will not be available to the large swathes of humanity who struggle under a calorific deficit, and are beset by wars and corrupt government.

There are plenty of ways that technology could enable or create negative economic and social outcomes. Badminton has run around 45 “Dark Futures” meetings, in which guests give short presentations about bad or dangerous scenarios such as inappropriate use of surveillance technology, or dirty money being laundered through the property markets of desirable cities like Vancouver and London. These meetings have been held in Vancouver, Toronto, and San Francisco. Expansion to New York and London was stymied by the pandemic.

However, Badminton is an optimist. He thinks our societies are in a constant state of collapse, but we are very resilient, and we keep bouncing back in better shape than before. Alarmism about climate change and capitalism is overblown, he thinks. We do face a number of existential risks like nuclear weapons and asteroids, but there are few of these, and he likes to think of futurists as “hope engineers”.

De-growth

Jason Hickel and others argue that rish humans are placing too much strain on the Earth’s ecosystems. They think we should shrink our economies and become more sustainable. Badminton is attracted to parts of this ideology, but he regards it as “entirely imaginary”, and therefore a useful tool to help develop new ways of thinking, new planning scenarios and new stories.

He does think we should wean ourselves off fossil fuels more quickly, and build circular economies where nothing is wasted and everything is recycled. He would like to see cities built for pedestrians rather than cars (although he is cynical about the Saudi project to build a car-free city called The Line), and greater equality of income.

But unless abundance for all is just around the corner, then slowing economic growth is an affordable indulgence for wealthy people in the West, but a disaster for hungry people in the global south, and for people just about clawing their way into a middle class existence.

Futurism improves company performance

The growth of interest in futurism as a career is a good thing, he thinks, but it is still the case that too few large companies employ full-time futurists. A study carried out in Belgium in 2018 by Rene Rohrbeck and Menes Etingue Kum found that over an eight-year period, future-prepared firms outperformed the average by a 33% higher profitability and with a 200% higher growth. If there is a causal link, then whichever direction it goes, the finding is positive for futurists.

The post What do professional futurists do? With Nikolas Badminton first appeared on .

GPT-4. Commotion and controversy

Call for a moratorium on advanced AI development

On the day that a London Futurists Podcast episode dedicated wholly to OpenAI’s GPT-4 system dropped, the Future of Life Institute published an open letter about the underlying technology. Signed by Stuart Russell, Max Tegmark, Elon Musk, Jaan Tallinn, and hundreds of other prominent AI researchers and commentators, the letter called for a pause in the development of the large language models like OpenAI’s GPT-4, and Google’s Bard.

It was surprising to see the name of Sam Altman, OpenAI’s CEO, on the list, and indeed it soon disappeared again. At the time of writing, there were no senior signatories from either of the two AGI labs, OpenAI and DeepMind, or from any of the AI-driven tech giants, Google, Meta, or Microsoft, Amazon or Apple. There was also no representation from the Chinese tech giants, Baidu, Alibaba, or Tencent.

Whatever you think of the letter’s prospects for success, and even the desirability of its objective, it was a powerful demonstration of the excitement and concern being generated in AI circles about GPT-4 and the other large language models. The excitement about GPT-4 is not overdone. The model is a significant advance on any previous natural language processing system.

Two big bangs

The last time there was this much excitement about AI was in 2016, when DeepMind’s AlphaGo system beat Lee Sedol, the world’s best player of the board game Go. That achievement was the result of the Big Bang in AI which occurred four years before, in 2012. When a neural network developed by Geoff Hinton and colleagues won the ImageNet competition, it was the start of the deep learning revolution, and for the first time ever, AI began to make serious money. Interestingly, Hinton’s colleagues included Ilya Sutskever, who went on to help found OpenAI, and become its Chief Scientist.

GPT is the result of what may come to be called the second big bang in AI, which happened in 2017 when some Google researchers published a paper called “Attention is all you need”. This described a new type of deep learning called Transformers, which enable systems like Dall-E and Midjourney, that generate photorealistic images when prompted by short instructions in natural language. They also enable natural language systems like GPT-4.

More attention needed

This renewed public focus on AI is a good thing. The impact of AI on every aspect of life over the coming years and decades will be so profound that the more people are thinking about it in advance, the better. Even with GPT-4 hitting headlines, and obsessing many technophiles, AI still isn’t getting the wider public attention it really deserves. It still gets beaten by populist attacks on transgender people and other somewhat spurious stories, but we must be grateful for whatever progress we can get.

Peekaboo

The operation of Transformers is often summarised as token prediction. They are trained on vast corpuses of text – all of Wikipedia, and millions of copyright-free books, for instance. They ingest this text and select tokens (words, or parts of words) to be “masked”, or hidden. Based on their model of how language works, they guess what the masked token is, and according to whether the guess was right or wrong, they adjust and update the model. By doing this billions of times, Transformers get really good at predicting the next word in a sentence. In order to avoid generating repetitive text, they make some arbitrary tweaks to the probabilities. When a system is tuned to make more tweaks, it is said to have a higher “temperature”.

Critically, this masking process does not require the training data to be labelled. The systems are engaged in self-supervised training. This is unlike the deep learning systems trained on massive datasets like ImageNet, where each image has been labelled by humans.

There is a human component in the training of transformers, though, which is Reinforcement Learning from Human Feedback, or RLHF. After the masking training is complete, the system’s responses to prompts are evaluated by humans for a period, and the evaluations are fed back into the system in order to minimise error and bias.

GPT-3, 3.5, 4

GPT stands for generative pre-trained transformer. GPT-3 was launched in November 2020 and boasted what was then an unheard-of number of 175 billion parameters (analogous to the synapses in a human brain). GPT-4 was released on 14 March 2023, and the number of parameters has not been disclosed. OpenAI has been criticised for reversing its policy of publishing as much as possible about its systems. It replies, not unreasonably, that if these models can cause harm in the wrong hands, it would be silly to make it easier for the bad guys to replicate them.

What is known is that the number of tokens that GPT-4 can handle – 32,000 – is much larger than the 4,100 that GPT-3 could manage. Among other things, this enables it to work with longer texts.

ChatGPT was a chatbot based on a halfway system, GPT-3.5. It was released in November 2022, and within a month it had 100m users, which was the fastest adoption rate of any app or platform.

OpenAI’s short but turbulent history

The story of OpenAI is as dramatic as its products are impressive. The company was formed in San Francisco in 2015 by Elon Musk, Sam Altman, and friends. They invested $1 billion of their own money to get the non-profit started. Musk stood down in 2018, and the reason given at the time was potential conflict of interest with his car company Tesla, which is also a significant developer of AI technology.

More recently there are reports that he left because he feared that OpenAI was failing to compete with the other leading AGI lab, DeepMind. (I call these two labs AGI labs because they are both explicitly targeting the development of artificial general intelligence, an AI with all the cognitive abilities of an adult human.) He offered to lead the company himself, and invest a further $1bn of his own money. When his leadership bid was declined, he left, taking the $1bn with him. OpenAI was unable to pay for the AI talent it needed, and its management decided it had to become, in part, a for-profit organisation.

Microsoft is very interested in GPT technology. It contributed $2bn before the launch of ChatGPT, and has agreed to invest a further $10bn since. OpenAI’s parent company is still a non-profit, and the returns to investors in the revenue-generating subsidiary are capped at 100x. Sam Altman, it turns out, has zero financial interest in the company. He doesn’t mind: he is already a rich man.

Musk has become a trenchant critic of OpenAI, especially on Twitter. Altman has expressed continued deep respect for his former business partner, but has also observed that he sometimes behaves like a jerk.

GPT-4’s advances

OpenAI’s latest system makes fewer mistakes than its predecessors – in the jargon, it hallucinates less. It is better at passing exams too. It passed the US bar exam with a score in the top 10% of candidates, whereas GPT-3 only managed the bottom 10%. This doesn’t tell us whether the system could actually be an effective lawyer, but it is impressive.

Unlike earlier systems, GPT-4 also seems to have learned simple maths. And it often appears to be doing something indistinguishable from reasoning. This was not expected from what are essentially pattern recognition systems. It has even led a group of Microsoft employees to publish an article claiming that GPT-4 shows the first sparks of AGI, although that has been characterised as hype.

Revised timelines and threat estimates

GPT-4 is impressive enough to be causing well-informed people to revise their timelines for the arrival of AGI, and of lasting widespread unemployment. Geoff Hinton, often called the godfather of deep learning, remarked in a recent interview that he used to think AGI was at least 20 years away, and very possibly 50. Now he thinks it might be less than 20 years. He also (for the first time, as far as I know) said that it is “not inconceivable” that advanced AI could cause human extinction.

The post GPT-4. Commotion and controversy first appeared on .

Benign superintelligence, and how to get there. With Ben Goertzel

During a keynote talk that Ben Goertzel gave recently, the robot that accompanied him on stage went mute. The fault lay not with the robot, but with a human who accidentally kicked a cable out of a socket backstage. Goertzel quips that in the future, the old warning against working with children and animals may be extended to a caution against working with any humans at all.

Source