Frontier models require substantial compute for both training and inference, which typically keeps them on servers. Yet on-device AI is increasingly compelling. Edge hardware keeps getting more powerful, on-device deployment scales naturally to massive user bases, and, most importantly, it is inherently private, enabling personalized experiences built on personal data. In this talk, we present strategies for advancing on-device AI by co-designing model architectures and training methods for both learning and runtime efficiency. We cover MobileCLIP and FastVLM, families of efficient vision-language models that achieve state-of-the-art accuracy–latency tradeoffs on mobile devices. Finally, we rethink how language models store knowledge. With pretraining with hierarchical memories, a compact model holds common knowledge and reasoning, while long-tail knowledge is fetched on demand from a large memory bank, allowing small models to match models more than twice their size.