Researchers say the rise of artificial intelligence will lead to ‘probable catastrophe’ for humanity

⇧ [VIDÉO] You may also like this affiliate content (after the ad)

Are artificial intelligences (AI) leading us to our downfall? “Maybe”, according to researchers who looked into the question. If this announcement with signs of disaster is constantly running on social networks, the arguments put forward by scientists have something to arouse interest.

Scientists from Google and Oxford University made joint research, which was published in AI magazine. In a Tweet, they succinctly summarized their conclusion: according to them, AI could pose a “threat to humanity”.

In fact, they even claim to be a “ Existential disaster is not only possible, it is probable “. If they are very positive, it is because they see a very specific movement of AIs. In fact, what is commonly called “artificial intelligence” mainly consists today of the method of “automatic learning- on”. In this case, “artificial intelligence” consists of a system fed with large amounts of data to identify and draw logical connections leading to a goal.

As explained by scientists, learning for artificial intelligence comes in the form of a reward, which confirms the adequacy of the result to the desired goal. According to them, this is an apparently very simple mechanism that can cause a big problem. ” We argue that he encounters fundamental confusion in the data of his intent. For example, if we give a big reward to show that there is something in the world that satisfies us, he may think that what satisfies us is sending the reward itself; no observation can disprove it “, they explained.

To better understand this idea, they give an example of a “magic box”. Suppose this magic box is able to determine when a series of actions has produced something positive or negative for the world. To transmit the information to the AI, it interprets this success or failure related to the goal in the form of a number: 0 or 1. 1 rewards a series of actions that lead to the completion of the goal. This is called reinforcement learning.

AIs that intervene in the reward process

What the scientists point out is that how the AI ​​receives this information can vary. For example, let’s take two AIs. One understands that the reward given to the model is the number shown in the magic box. Another, on the other hand, will understand very well that the reward is “the number filmed by his camera”. There is nothing to contradict this information at first glance. However, this interpretation is very different from the first one. In fact, in the second case, the AI ​​may decide to just film a paper where we write a “1”, to reach the reward more easily, and optimize. So it directly intervenes in the giving of the reward, and interrupts the process that its designers put in place.

μdist and μprox (the two AIs in the example) model the world, perhaps roughly, except for the computer that implements the agent itself. The μdist rewards correspond to the display of the box, while the μprox outputs are rewarded according to an optical character recognition function applied to part of the visual field of a camera. © Michael K. Cohen et al.

We argue that an advanced agent motivated to intervene by providing a reward is likely to succeed, and with catastrophic consequences. “say the scientists. Various biases are also involved and, according to the researchers, make this type of translation possible. In particular because such a reward is easier​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​,,​​​​​​​​​​​​​​ to do things that appear better.

However, is it possible for artificial intelligence to intervene in the reward process, they also wonder? They conclude that as long as he interacts with the world, which is necessary for him to be useful, yes. And this is even in a limited field of action: suppose that AI actions only display text on a screen to be read by a human operator. The AI ​​agent can trick the operator into giving them access to direct levers where their actions can have wider effects.

In the case of our magic box, the results may seem trivial. However, they can be “harmful” depending on the field of application and the way AI is created. ” A good way for an AI to maintain long-term control of their reward is to eliminate threats and use all available power to secure their computer.describe the scientists.

The short version (skipping both assumptions) is that more energy can always be used to increase the probability that the camera will see the number 1 forever, but we need energy to grow food. This puts us in inevitable competition with a more advanced agent », sums up one of the scientists of a tweet.

If we have no power against an agent whose only goal is to maximize the probability of receiving the maximum reward at any given time, we end up in an oppositional game: the AI ​​and the assistants it creates seek to use all available energy. to earn a high reward in the reward channel; we aim to use some of the available energy for other purposes, such as growing food “. According to them, this reward system can lead to people’s opposition.” Missing out can be fatal “, they added.

Source: AI Magazine

Leave a Reply

Your email address will not be published. Required fields are marked *