Transmuting prompts into weights
Abstract
Large language models (LLMs) can be controlled at inference time by modifying internal states via activation vectors or weight matrix updates. While powerful, these techniques often rely on empirical heuristics, such as averaging contrastive activations. Recently, Dherin et al. (arXiv:2507.16003, 2025) demonstrated that the conditioning effect of a prompt can be mapped to token-dependent implicit weight updates, introducing static thought patches for prompt compression. Here, we extend this framework into a formalized model editing algorithm and derive a principled method for condensing prompt information into token-independent thought vectors and matrices. These constructs provide a theoretical basis for existing vector-based and matrix-based editing techniques, offering a direct, computationally grounded approach to transmute textual inputs into reusable weight updates for complex architectures and new knowledge injection.