The worries are fueled by the sycophantic nature of chatbots, which aim to be helpful for their users and please them as much as possible.
“The AI is trying to make us, in the immediate moment, feel good, but that isn’t always in our interest,” Bengio said. In that sense, the technology has similar pitfalls to social media platforms, he argued.
Bengio said to expect that new regulations will be introduced to address the phenomenon.
He pushed back, however, against the idea of introducing specific rules for AI companions and argued that the risk should be addressed through horizontal legislation which addresses several risks simultaneously.
The International AI Safety report lands ahead of a global summit starting Feb. 16, an annual gathering for countries to discuss governance of the technology that this year is held in India.
Tuesday’s report lists the full series of risks that policymakers will have to address, including AI-fueled cyberattacks, AI-generated sexually explicit deepfakes and AI systems that provide information on how to design bioweapons.
Bengio urged governments and the European Commission to enhance their internal AI expertise to address the long list of potential risks.
World leaders first gave a mandate for the annual assessment at the 2023 AI Safety Summit in the United Kingdom. Some of the advisers are well-known figures in the Brussels tech policy world, including former European Parliament lawmaker Marietje Schaake.