Agents that look strong on public tool-calling benchmarks often struggle in real enterprise settings. There, they face long conversations, ambiguous customer requests, sprawling internal APIs, and company policy, all at once. So what does it actually take to teach a small open model to do this work reliably?

This talk shares a practical, end-to-end approach to specializing small models for enterprise tool use. We cover how to design realistic environments and tasks that reflect how agentic work succeeds or fails in practice, how to make rewards and evaluation trustworthy, and how supervised fine-tuning and reinforcement learning each contribute to the final agent. We also look at the question that matters most for deployment: do these skills generalize beyond the training data, to new business settings and to public benchmarks? We close with lessons learned and open questions for teams building domain-specific agents. All environments are simulated, and the focus is entirely on methodology and evaluation.

Additional details

This talk is based on ongoing applied research at Toloka on training small open-weights models for enterprise agentic tool use. The session will cover:
- Environment design
- Training recipe
- Reward-hacking safeguards
- Transfer analysis
- Efficiency