PROPEL Accelerates AI Task Generator Training by Predicting Solver Success

Training AI agents through reinforcement learning often hits a bottleneck: the scarcity of suitable tasks that are challenging yet solvable for the current model. Traditional synthetic task generation frequently produces tasks that are too easy, impossible, or poorly defined, hindering effective model improvement. Directly optimizing task generators with reinforcement learning is computationally intensive, as it requires numerous solver rollouts for each candidate task, which can take tens of minutes for complex tasks like software engineering. This makes solver-in-the-loop training impractical. PROPEL tackles this by training a lightweight "activation probe" on a pre-labeled dataset of generated tasks and their solver outcomes. This probe then acts as a proxy for the actual solver pass rate during generator optimization, significantly reducing evaluation time to a single forward pass. Experiments show PROPEL effectively shifts task generation towards the targeted solve rate across various domains, including math, code, and software engineering, substantially increasing the proportion of learnable tasks.