SocialRL Enhances Negotiation Skills in Small Language Models.

AI agents are increasingly tasked with acting on behalf of users, handling complex interactions like scheduling, comparing offers, and price negotiation. These principal-driven tasks often involve confronting counterparts whose objectives may conflict with the user's. Traditional frontier models, while helpful, can be too accommodating, potentially revealing private information or conceding too readily. To address this, researchers developed SocialRL, a general training methodology designed to instill social reasoning directly into language models. Applied to a 4-billion parameter model across six diverse domains, including negotiation games and marketplace scenarios, SocialRL demonstrated remarkable efficacy. The in-domain trained 4B model achieved performance comparable to or surpassing GPT-5 family models on held-out scenarios, significantly closing the performance gap in negotiation tasks. The study also revealed insights into cross-domain transfer, showing that structurally similar games benefit each other, and a broad multi-issue donor model can lift performance across nearly all domains. By consolidating per-domain specialists using strategies like cascade RL and multi-teacher on-policy distillation, a unified 4B model was created that matched or exceeded the average utility of GPT-4.1 and GPT-5. Theory-of-mind scaffolding was found to be beneficial primarily through training, with distilling the ToM trace proving more effective than actions alone for improved utility and generalization.