VICT Improves Credit Assignment for Long-Horizon LLM Agents.

Researchers have developed VICT (Verifier-Instrumented Credit Tracing), a new method to address the challenge of fine-grained credit assignment in long-horizon reinforcement learning for LLM agents. Traditional approaches often assign credit broadly based on a sparse terminal reward, making it difficult to pinpoint which specific actions contributed to success or failure over extended sequences. VICT, however, leverages the internal structure of task verifiers, which often contain executable or evidence-backed "atoms" that define sub-goals or checks. VICT traces these verifiable atoms back to the agent's actions using dependency-valid proof edges, redistributing group-relative advantage only along these relevant paths. This shifts the burden of credit assignment from complex rollout-side inference to a more structured verifier-side tracing mechanism. The method preserves the original terminal reward, handles incomplete evidence gracefully, and only modifies the training-time advantage tensor, avoiding the need for learned critics or additional inference-time components. Experiments on ALFWorld and WebShop demonstrated that VICT substantially outperforms training based solely on terminal outcomes and achieves strong results compared to other fine-grained credit methods.