Evolved, Edited, Engineered: Not All Pathogens Are Built the Same

A threat-modeling primer for AI biorisk

This is an old piece from my AI biosecurity days earlier this year. Finally cleaned it up and published! :)

The tone is slightly more formal than most other things on my blog. I originally wrote it for bioweapon.ai, then decided to make it a standalone piece.

---

AI biorisk is a very live debate right now; governments, frontier labs, and a growing empirical literature are all actively trying to determine what, exactly, is being made easier and for whom. Much of the disagreement currently runs through threat actors: should we be most worried about a motivated novice, a skilled individual, a terrorist group, or a state-sponsored program?

Threat actors, however, are not the only variable that matters. AI biorisk discourse today often overlooks how important biological agents themselves are: both the probability that an attempt succeeds and the ceiling on how damaging it will be are largely determined by which kind of agent is being pursued.

There are three classes of pathogens we care about:

Class A: those that emerge naturally;
Class B: modified versions of Class A; and
Class C: entirely novel pathogens.

Photo by Elena Mozhvilo on Unsplash

A. Naturally-occurring pathogens

Naturally-occurring pathogens are those that emerge through natural evolutionary processes rather than deliberate design. This category includes pathogens like influenza and measles.

Two key characteristics of these pathogens are:

1. Immune defensibility. In most cases, when we encounter a new naturally-occurring pathogen, it is a variant of a pathogen humans have previously faced. Because our immune systems have already evolved defenses against the original pathogen, they can typically recognize and respond to these variants.

2. Lethality-transmission trade-off. In nature, there tends to be a trade-off between deadliness and contagiousness. Because a virus that causes its host to die quickly reduces the window during which the host can transmit that virus to other hosts, highly lethal diseases are typically less contagious, and vice versa – highly contagious diseases are typically less deadly.

Importantly, this trade-off is not absolute. For example, a naturally-occurring pathogen could theoretically achieve both high transmissibility and high lethality if it had a long asymptomatic period during which transmission occurs. However, because of evolutionary constraints, such pathogens are less likely to arise and spread naturally than those optimized for one or the other.

---

B. Modified pathogens

Modified pathogens are naturally-occurring pathogens that have been artificially altered¹ to enhance their ability to spread, survive in the environment, evade immune responses, or resist treatment. (This scientific area of study is known as gain-of-function research; it is predictably highly risky, particularly due to dual use concerns. Whether or not it is helpful for pandemic preparedness is contested.)

What typically separates this class from Class C is that many modified pathogens remain at least partially recognizable as variations on known sequences. Because they are engineered from existing pathogens, they often retain enough structural similarity to their parent strains to be detectable by screening systems designed to identify dangerous pathogens by comparing genetic sequences to databases of known dangerous pathogens (this approach is referred to as “homology-based screening”).

That said, sufficiently extensive modifications have already been shown to evade homology-based detection, creating a gray zone between Class B and Class C: in a 2025 Science paper, Microsoft researchers used open-source protein-design tools to “paraphrase” a toxin, rewriting its sequence while attempting to preserve its three-dimensional structure and biological function.

As Eric Horvitz described the result:

“To our concern… these reformulated sequences slipped past the biosecurity screening systems used worldwide by DNA synthesis companies to flag dangerous orders.”

Note on Framing

The provided three-class framing is meant to provide a useful simplification. In practice, the boundaries are softer than what a clean categorization suggests; it’s more accurate to view the classes as a gradient, because most of our defensive systems don’t recognize categories: they measure similarity, and similarity is continuous.

---

C. Entirely novel pathogens

Photo by Indra Projects on Unsplash

Completely novel pathogens are organisms designed from scratch; they have never existed in nature and bear no close homology to known pathogens. (These pathogens are novel in structure and function, not merely in sequence, which is all the paraphrased toxins above changed.)

This creates two distinct threats, which correspond with the key characteristics described in Class A.

1. First, a novel pathogen could be designed to simultaneously maximize transmissibility and lethality, combining advantages that rarely coexist in nature.

2. Second, against entirely novel pathogens, humans may lack protective pre-existing antibodies and T-cell memory. More radically novel agents could potentially evade even innate immune recognition (as researchers have warned in the case of mirror life).

The Detection Problem

These pathogens can also be especially difficult to detect. As briefly touched upon in Class B, existing U.S. biosurveillance infrastructure is primarily built to detect known pathogens (homology-based approach). Against a naturally occurring outbreak of a familiar pathogen, this approach functions reasonably well; against a novel engineered pathogen, current surveillance systems may fail to detect it entirely until clinical cases emerge. By the time symptomatic individuals are diagnosed, transmission may already be widespread.

The Defense Problem

Post-detection, this category also represents the most difficult pathogen tier to design defenses against. With modified pathogens, even if a modification is novel, scientists have a known parent pathogen to work from, which means they can often leverage existing vaccine platforms, treatment approaches, and understanding of that pathogen’s biology.

With completely novel pathogens, there is no pre-existing institutional knowledge about how the pathogen behaves. Researchers would be designing treatments and vaccines entirely anew; this could dramatically extend the response timeline during an outbreak.

Conclusion

AI has already been used to generate complete viral genomes within a known family.

While effective novel agents remain – for now – extremely hard to build, difficulty and severity are different axes, and in this case, they can run opposite: the agents that are easiest to obtain are the ones our defenses were designed around; the agents that are hardest to obtain are the ones that have the most potential to disrupt both detection and response simultaneously. This remains true no matter who the threat actor is.

So, when it comes to threat modeling for biorisk, we should be asking both questions: who is trying, and what are they trying to make? The second question sets the ceiling.

---

Read the below piece for more on DNA synthesis screening, and a call to action re: S.3741!

1. In some sense, nature itself ‘alters’ viruses all the time (which is why we see mutated versions of the flu and COVID each year); however, these mutations are incremental. Each year’s variant represents a small step from the last, which is why our immune systems and our vaccines can usually keep pace. They also accumulate under selection pressures that have little to do with maximizing harm.