Chatbots aren't the only AI models that have advanced in recent years. Specialized models trained on biological data have made leaps and bounds as well, and could help accelerate vaccine development, treat diseases, and develop drought-resistant crops. But the same properties that make these models useful also pose potential dangers. For example, before a model can design a safe vaccine, we first need to know what's harmful.
That's why experts are calling on governments to put in place mandatory oversight and guardrails for advanced biological models in a new policy paper published Aug. 22 in the peer-reviewed journal Science. While current AI models will not “significantly contribute” to biological risks, future systems could help create new pathogens that could cause pandemics, the authors write.
“The essential elements for creating sophisticated biological models of great concern already exist or will soon exist,” wrote the authors, who are public health and legal experts at Stanford University School of Medicine, Fordham University and the Johns Hopkins Center for Health Security. “Effective governance systems must now be established.”
“We need to start planning now,” said Anita Cicero, deputy director of the Johns Hopkins Center for Health Security and co-author of the paper. “We're going to need systematic government oversight and requirements to reduce the risks of these particularly powerful tools in the future.”
Read more: Researchers seek a glimpse into the future of AI
Humanity has a long history of weaponizing biological weapons. In the 14th century, Mongolian troops are believed to have hurled plague-infected corpses over enemy city walls, possibly contributing to the spread of the Black Death in Europe. During World War II, several great powers experimented with biological weapons such as plague and typhoid, and Japan used them in several Chinese cities. And at the height of the Cold War, the United States and the Soviet Union both ran large-scale biological weapons programs. But in 1972, both countries, and the rest of the world, agreed to dismantle such programs and ban biological weapons, resulting in the Biological Weapons Convention.
Although the international treaty was widely considered effective, it did not completely eliminate the threat of biological weapons. In the early 1990s, the Japanese cult Aum Shinrikyo repeatedly tried to develop and release biological weapons, such as anthrax. The attempts failed because the cult lacked the technical expertise. But experts warn that future AI systems could fill this gap. “As these models become more powerful, the level of sophistication required for malicious actors to cause harm will decrease,” Cicero said.
Not all weaponized pathogens can be transmitted from person to person, and those that can tend to become less lethal as they become more infectious. But AI may be able to “figure out how a pathogen can remain adaptive and still be infectious,” Cicero said. Terrorist groups and other malicious actors aren't the only ones this could happen to. Even well-intentioned researchers could accidentally develop a pathogen that “gets released and spreads uncontrollably” without the proper protocols, Cicero said. Bioterrorism continues to attract global concerns, including from Bill Gates and U.S. Secretary of Commerce Gina Raimondo, who has been leading the Biden administration's AI efforts.
Read more: UK AI Safety Summit ends with limited but meaningful progress
The gap between virtual blueprints and physical biological weapons is surprisingly narrow. Many companies allow the ordering of biological materials online, and some measures to prevent the purchase of dangerous genetic sequences exist, but they are unevenly applied in the U.S. and abroad and can be easily circumvented. “A dam has lots of little holes through which water gushes out,” Cicero explains. Cicero and his co-authors recommend mandatory screening requirements, but note that even these would not be enough to completely prevent the risks of biological AI models.
To date, 175 people, including researchers, academics and industry experts from Harvard, Moderna and Microsoft, have signed a set of voluntary pledges contained in the Responsible AI x Biodesign Community Statement released earlier this year. Cicero, one of the signatories, said she and her co-authors agree that these pledges, while important, are not enough to protect against risks. The paper notes that in other high-risk biological domains, such as using live Ebola virus in laboratories, voluntary pledges alone should not be relied upon.
The authors recommend that governments work with experts in machine learning, infectious diseases and ethics to devise a “battery of tests” that biological AI models must undergo before being released to the public, focusing on whether they might pose “pandemic-level risks”.
“Some standards are needed,” Cicero explains. “At a minimum, risk-benefit assessments and pre-release reviews of biological design tools or powerful large-scale language models would include, among other things, an assessment of whether those models could pose pandemic-level risks.”
Because testing such capabilities of AI systems can be dangerous in itself, the authors recommend creating surrogate assessments, for example whether an AI can synthesize new harmless pathogens as a proxy for its ability to synthesize deadly pathogens. Based on these tests, authorities can decide whether and to what extent access to the models should be restricted. Oversight policies should also address the fact that open-source systems may be modified after release, potentially making them more dangerous in the process.
Read more: Republicans vow to repeal Biden's AI executive order, experts concerned
The authors also recommend that the United States develop a set of standards to guide the responsible sharing of large datasets on “pathogenic traits of concern” and authorize federal agencies to collaborate with the recently established U.S. AI Safety Lab. The UK AI Safety Lab, which works closely with the U.S. AI Safety Lab, already conducts safety testing, including biological risks, on major AI models, but this testing is primarily focused on assessing the capabilities of general-purpose large language models rather than biology-specific systems.
“The last thing we want to do is to trip up the industry and stifle our progress,” Cicero says. “It’s a balancing act.” To avoid hindering research with overregulation, the authors recommend that regulators initially focus on just two types of models: models trained on biological data using very large amounts of computational power, and models of any scale trained on particularly sensitive biological data that isn’t widely accessible, such as new information linking a virus’ genetic sequence to its potential to cause a pandemic.
Cicero says the range of worrying models could widen over time, especially if AI is able to conduct research autonomously in the future. “Imagine 100 million Pfizer chief scientific officers working around the clock at 100 times the speed of reality,” he says, noting that while this could lead to incredible advances in drug design and discovery, it also greatly increases the risks.
The paper emphasizes that managing these risks requires international cooperation, given that they endanger the entire planet. However, while policy harmonization would be ideal, the authors note that “countries with the most advanced AI technologies should prioritize effective evaluation, even at the expense of some international uniformity.”
With AI capabilities expected to continue to improve, and the relative ease of sourcing biological materials and hiring third parties to conduct experiments remotely, Cicero believes that without proper oversight, biological risks from AI could become evident “within the next 20 years, maybe even less than that.” “We need to think not just about the current versions of the tools that are available today, but also about the next versions, because there's this exponential growth that we're seeing. These tools are going to get more and more powerful,” Cicero says.
