I agree with Noah Smith that the real risk of AI is bioterrorism, not any nebulous superintelligence. It allows people who prior to AI would not have possessed enough skills nor knowledge to create and iterate on novel viral lines that could prove quite deadly.

Within the next decade, I’d put the odds at about 15% that someone designs and releases some virus or bacterium with AI that kills more than 100,000 people.

If I cared to do such a thing with AI, I could already do it. And I’m not nearly the smartest person around. Just depending on people not feeling like doing such a thing is not a great defense.