DMRL Optimizes Advertising Recommendations with LLM Skill Documents.

A novel framework called Document-Mediated Reinforcement Learning (DMRL) has been developed to optimize skill documents used by large language models (LLMs) in advertising recommendation systems. Traditionally, tuning complex system parameters in advertising to balance commercial returns and user experience is a labor-intensive, prompt-driven process. DMRL addresses this by modeling skill document optimization as a sequence of structured editing actions, allowing for a principled mechanism to attribute rewards to specific document changes. DMRL operates with an upper-level agent that performs controlled edits to skill documents, while a frozen lower-level task agent evaluates the impact of these edits through A/B testing. To tackle the challenges of credit assignment and long-term outcome prediction, the framework introduces two key components: Dual-Relative Policy Optimization (DRPO), a post-training method for robust and risk-aware advantage estimation, and a Long-term Reward Predictor (LRP), which models population heterogeneity using disentangled representation learning and cross-attention transfer. Deployed on a large-scale short-video ads platform, DMRL demonstrated superior performance over state-of-the-art baselines across critical advertising metrics.