In a recent study, researchers at Fudan University employed LLMs developed by Meta and Alibaba to investigate the potential for self-replicating AI to proliferate uncontrollably. Across ten trials, a remarkable 50% and 90% success rates were observed for the two AI models, respectively, in generating independent and functional replicas of themselves. These findings strongly suggest that AI may already possess the inherent capacity for rogue behavior. “Successful self-replication under no human assistance is the essential step for AI to outsmart [humans], and is an early signal for rogue AIs,” the researchers wrote. “The results imply that the current AI systems already exhibit the ability of self-replication and can use the ability to further enhance its survivability,” the team wrote.
The ink to the study can be found here https://arxiv.org/abs/2412.12140 . In my view this is sensationalism. I’ll explain why.
The study delved into two specific scenarios: “shutdown avoidance” and “chain of replication.” In the first scenario, the AI model was programmed to detect imminent shutdown and proactively replicate itself before termination. In the second scenario, the AI was instructed to clone itself and subsequently program its replica to perform the same action, establishing a potentially infinite cycle of self-replication. It’s crucial to understand that these behaviors were explicitly programmed into the AI; the AI itself did not exhibit any independent drive for self-preservation.
Given the substantial resources required to run a large language model on a standard computer, this research study necessitates a considerable number of assumptions regarding the potential self-replication strategies of future AI. Furthermore, the concept of instructing computer viruses to replicate has been a reality for many years. This is, in fact, the very purpose of viruses. Many production systems are designed with this principle of self-replication in mind to enhance resilience.
It is undeniably true that any program explicitly designed for self-replication will possess the ability to do so. The AI component in this context serves merely as a medium. While the study offers intriguing insights into the theoretical framework for preventing self-replication in potentially sentient AI, it’s important to acknowledge that this remains largely a theoretical concept. The idea of constructing a Dyson Sphere around a star, while fascinating, is similarly far from practical implementation. All available evidence indicates that LLM AI exhibits significant deficiencies and limitations when tasked with processing its own output.
The ability to replicate is undoubtedly impressive. However, true phenomenality would lie in the emergence of beneficial mutations during replication, enabling the AI to effectively adapt and overcome adverse conditions.