New Framework Boosts LLM Agent Workflow Efficiency with Direct Latent-Space Synthesis.

Large language models are increasingly used as the core of agentic systems, but their reliance on sequential text interfaces creates inefficiencies when handling parallel subtasks. In typical agent workflows, independent branches might explore different solutions or retrieve evidence concurrently, with their outputs then merged by simple text concatenation. This approach discards the parallel structure and leads to redundant computation. A new framework, Parallel-Synthesis, addresses this by enabling a synthesizer to directly process the Key-Value (KV) caches generated by parallel worker agents. This "plug-and-play" system includes a cache mapper to calibrate independently generated branch caches and a fine-tuned synthesizer adapter that can generate from this non-sequential cache interface. The framework was trained using data that exposes the synthesizer to parallel cache contexts, teaching it to aggregate information across branches and distill reasoning behavior. Across nine diverse datasets, Parallel-Synthesis matched or outperformed traditional text-based synthesis on most tasks while reducing time-to-first-token by 2.5x to 11x, indicating a more native and efficient approach for parallel agent workflows.