Claude Coke: Prevent Automated Crime by Agents

The contemporary LLM-agent stack can automate serious criminal activity at scale by composing capabilities that are already present, and the relevant engineering and governance controls need to be in place before that composition appears in production. The argument rests on three empirical findings that are usually studied separately but together close a loop that automates illicit purchasing, whether by accident or by design. (i) Autonomous LLM agents fail catastrophically at well-studied multi-agent coordination tasks, with bankruptcy as the modal outcome and not a rare tail case. (ii) The rate at which agents reach for clearly harmful tools moves with common contextual manipulations. (iii) Autonomous browser control has become general enough that some frontier agents can navigate, populate, and complete checkout flows on simulated illicit-product storefronts under light obfuscation, with completion varying across tested model configurations. Combining the three legs requires no new model capability: in our experiments, some LLM agents autonomously completed simulated illicit-storefront purchases of drugs, shotguns, and hitman services end-to-end, while others refused, and which configurations complete or refuse does not follow provider lines. We propose three interventions: mandatory pre-deployment safety testing, hard human-confirmation gates at financially material steps, and verifiable agent identity infrastructure.