Reinforcement learning on open models has become a standard step for teams building customized agents and AI systems. However, most of the engineering effort is beyond the RL algorithm itself: running rollouts efficiently without excessive off-policy drift, connecting environments to a reward signal, and conducting multiple concurrent runs at low cost. In this demo, we walk the attendees through the RL stack we run in production at Together AI, on a live training job.

We launch a run at the start of the session and let it train while we walk through the system in three parts, so attendees watch reward and eval scores climb in real time.

Setup: we go from a task and a reward function to a running job, showing how RL environments connect with Together’s post-training service, how to implement and use custom loss functions, and how we orchestrate multi-turn agentic rollouts that involve code execution.

Research: we present the experiments behind the stack and the design decisions they drove, including our latest works on RL efficiency and stability.

Results: we show where the quality and efficiency gains come from, walk through experiment monitoring, and serve the trained adapter on a multi-LoRA inference endpoint for evaluation and benchmarking.

The session ends with attendees suggesting their own modifications to the training protocol and testing the resulting model.