Pre-training is fundamental to foundation models, enabling them to acquire broad knowledge that gives rise to emerging capabilities at later training stages, and scaling is the key for pre-training. In this talk, we present a recipe for building and curating pre-training, multimodal image-text paired data from scratch on a global scale, enabling mutual benefits between English and non-English data. We would like to share our key observations and insights with the community on: (1) why scaling matters, including the foundational role of data and key principles to hold for scaling; (2) how to design simple yet scalable data algorithms that enable industry-scale data collection and training without data filters, serving both research and production needs; (3) how the scaling improves Meta’s products at conventional and frontier machine learning areas. Submission is facilitated by Cogs & Marvel but is entirely organized, executed, and implemented by Meta.