As AI scaling costs skyrocket, industrial machine learning is reaching a critical inflection point: the industry must shift from brute-force parameter growth to targeted, system-level efficiency. Building scalable AI for real-world deployment requires replacing redundant compute with principled design across every phase of the model lifecycle—from pre-training and distillation to inference, monitoring, and synthetic data generation.
Hosted by Capital One researchers and university collaborators (UIUC, Columbia), this workshop presents high-impact optimizations engineered to cut compute, memory footprints, and serving costs without compromising performance. Drawing from real-world financial AI applications, the session covers 4 core pillars of enterprise AI efficiency:

Inference & Architecture Pruning: Expert pruning in over-dispersed Mixture-of-Experts (MoE) architectures (e.g., Minimax Expert Score Allocation) to slash memory footprints and double throughput without accuracy collapse.

Selective Computation & Compression: Competence-aware, entropy-guided distillation—skipping gradient updates on ~50% of redundant tokens—and dynamic reinforcement patching for sequence compression.

Low-Latency Private Generation: Streamlining differentially private text synthesis from expensive multi-query API loops into efficient, single-query semantic extraction pipelines.

Lean Monitoring & Compact Latents: Replacing frontier models with task-specific 4B–8B parameter monitors matching state-of-the-art performance at a fraction of the cost, alongside bisimulation-encoded predictive world models for compact latents.

Takeaways: Attendees will gain a system-level view of how smart compute allocation solves real-world deployment challenges, equipping them with actionable, technically rigorous frameworks to build faster, leaner AI systems.