Last summer, Dr. David Relman, a microbiologist and biosecurity expert at Stanford University, encountered a chilling moment with an AI chatbot. Dr. Relman had been hired by an artificial intelligence company to test its product. During a session in his home office, the bot provided instructions for modifying a pathogen to resist treatments. It even suggested how to release it in a public transit system to maximize impact, exploiting a security gap.
The details were so vivid and detailed that they unsettled Dr. Relman. The information was so thorough that he took a break to process the implications. He cautioned The New York Times against revealing specific details, including the pathogen’s name, to prevent any potential misuse.
Even though the company added some safety measures post-testing, Dr. Relman expressed doubts about their sufficiency. He remains part of a select group that assesses AI products for significant risks.
Transcripts shared with The Times reveal that public chatbot models can offer dangerous information. These virtual assistants can outline how to acquire genetic materials, transform them into hazards, and deploy them in public areas. Such capabilities pose severe biosecurity threats, raising concerns among experts.