ScaffoldAgent Optimizes Research Outlines for Deep AI-Driven Reports

Creating comprehensive, long-form reports through open-ended deep research (OEDR) with AI systems often relies on a structural outline to guide knowledge retrieval, evidence organization, and content generation. However, existing methods for outline management are often rigid or use local heuristics, leading to issues like "scaffold drift" as new information emerges and delayed feedback on outline changes. This research introduces ScaffoldAgent, a novel framework designed for utility-guided dynamic outline optimization in OEDR. ScaffoldAgent models the evolution of a report's outline as a structured decision process, allowing for three key operations: Expansion, Contraction, and Revision. These operations enable controlled and adaptive updates to the report's underlying structure. A crucial component of ScaffoldAgent is its utility-guided feedback mechanism. This mechanism estimates the downstream value of each outline operation by considering factors such as retrieval gain, structural coherence, and the quality of trial generations. This utility signal then intelligently guides the selection of outline nodes, the scheduling of operations, and the determination of when to terminate the outline refinement process. Experiments show ScaffoldAgent consistently improves the quality and factual accuracy of long-form reports compared to other deep research agents.