Vessel: High-Utilization Megakernels by Construction

AI training is compute-bound but most kernels do not achieve high GPU utilization. This is especially the case for new architectures which arguably are the most in need of scaling. We introduce Vessel, a programming model where an architecture is expressed as pure data and is converted into a megakernel that avoids common sources of utilization losses in GPUs, by construction. We evaluate Vessel across 8 diverse architectures, from trilinear attention to test-time memory, against torch.compile, expert kernels, and a strong control: the same AI agent writing unrestricted GPU kernels. Because a Vessel schedule is pure data with an exact CPU emulation, agents author kernels one-shot, with no GPU access. The resulting kernels are the fastest we are aware of for 6 of the 8 architectures: on average ~3x torch.compile, ~22% over NVIDIA's Transformer Engine on a GQA block, and the only correct fast implementation of Selective Attention we know of. We find the right restrictions can enable faster and cheaper design, and performance gains.