We need to think about the welfare of AI as it gets more advanced, dozens of experts and thinkers have said.
For most people, concerns focus on if AI will harm us, rather than if we will harm it, but now an open letter calls for consideration of how we could make conscious machines suffer.
It suggests five principles to guide ethical development of the tech, a proposal signed by big names including Stephen Fry, who has a deep interest in the technology.
Those creating models ‘should prioritise research on understanding and assessing AI consciousness with the objectives of preventing the mistreatment and suffering of conscious AI systems,’ the letter organised by Conscium reads.
They should understand ‘the benefits and risks associated with consciousness in AI systems with different capacities and functions.’
How might conscious AI suffer?
The concern for many is that if AI can become conscious, forcing it to serve us would be like slavery.
It could be abused or asked to perform harmful tasks, and we have already seen criminals exploit the technology.
Perhaps it could experience neglect if not interacted with, or be exploited with experiments. Given that AIs could be replicated, a large number of conscious beings could be created for this existence of suffering.
These are some of the worries underpinning the letter.
But Dr Tom McClelland, a philosophy lecturer at Cambridge University specialised in cognitive science and AI, told Metro we have to be wary of projecting human traits onto AI.
‘We think about what humans don’t like and just assume that an AI won’t like it as well, like working too hard or being switched off,’ he said. ‘But that’s just us projecting. There’s no evidence for that.
‘So one of the things we really have to do if we want to take AI suffering seriously, is understand what would make a conscious AI suffer. And it could be dramatically different to what makes a human or any other organism suffer.
‘It’s hard to know if it would be more like a human with desires and aims that we would be inhibiting, or if it would be more like having a chicken farm.’
How close are we to building conscious AI?
Some doubt that AI could ever become conscious, as they believe the concept itself is tied to biology. Others think it could be on the horizon as we make advancements in machine learning and in neuroscience.
It’s a question philosophers have argued over for decades, but the debate is now moving towards the practical rather than theoretical.
The open letter was published alongside a research paper written by Oxford University’s Patrick Butlin and Theodoros Lappas of Athens University, which warns: ‘Recent research suggests that it may be possible to build conscious AI systems now or in the near future.
‘Furthermore, AI systems or AI-generated characters may increasingly give the impression of being conscious, leading to debate about their moral status.’
The question ‘deserves to be taken seriously’, it says, citing philosopher and cognitive scientist David Chalmers who suggests ‘a credence of 25% or more’ that we will have ‘conscious LLM+s within a decade’.
What is consciousness, anyway?
Now we’re getting into the big questions…
The question is generally whether something has subjective experiences. Is it aware of the world and what is going on in the world?
It’s the difference between information going into the eye, and someone experiencing the colours they see.
Current research looks at whether we may need to expand the definition of consciousness, looking at whether coma patients or those in a vegetative state may be conscious.
Academics have also been asking whether prawns are conscious and if we therefore need to consider more carefully how billions are treated in commercial farming.
Dr McClelland pointed out: ‘Prawn consciousness doesn’t get this kind of profile and you don’t get big celebrities signing letters about it, even though it’s possible we are responsible for terrible suffering.’
Conscium, a company whose stated aim is ‘understanding consciousness to build a better future for humanity’, is looking into how to build AI that works similarly to human minds.
Dr McClelland said it may even be possible that conscious AI already exist, but the burden of proof would be on somebody to give good evidence for that.
Current discussions may ‘overstate’ where we’re at with consciousness science, and the kinds of AI which might become conscious probably aren’t the ones that are household names, like ChatGPT or new Chinese rival DeepSeek.
He explained: ‘Even though a lot of the things they say sound like what a conscious being would say, the way they work is dramatically different to the way a human mind or other organic mind works.’
Other models using ‘whole brain emulation’ mimic neural processes more similarly, and have even recreated the ‘brains’ of simple organisms like nematode worms.
‘As we climb up to more sophisticated animals, there might come a point where we have a computer emulation that is itself conscious. But it might just be an emulation – that’s the agnostic dilemma,’ he said.
Dr McClelland, who has written extensively on AI consciousness, said most experts in the field ‘would be sympathetic to many of the messages’ in the open letter, but also sceptical about the current science around it.
One of the biggest problems is we don’t even know how we would detect if an AI is conscious or just appears to be.
‘The moral messages are well intended, but the practical proposal is a bit sketchy,’ he said.
’It’s going to take a revolutionary shift in our understanding of consciousness, not just fiddling around the edges.’
Get in touch with our news team by emailing us at webnews@metro.co.uk.
For more stories like this, check our news page.
MORE: This is what 9 major cities will look like if sea levels rise by 6ft
MORE: Chinese tech breakthrough costing £1,000,000,000,000 is ‘wake up call’, Trump says
MORE: Stephen Fry casually breaks staggering Who Wants To Be A Millionaire? record
