Agentic Models Show Controllable Improvements in Information Extraction

This study explores the effectiveness of agentic large language models (LLMs) in complex information extraction tasks, specifically focusing on extracting structured data from scholarly PDFs. Researchers compared a standard, fixed LLM workflow against several reflective agent variants, including an optimized version that incorporated richer PDF tools and dynamic tool selection. The evaluation prioritized understanding the process-level behavior of these agents, such as tool execution, retries, reflection, and memory usage, alongside traditional outcome measures like extraction coverage. The findings indicate that agentic mechanisms can significantly alter system behavior, and these changes often lead to improved task completion. The paper also meticulously characterizes common failure modes, which in turn inform the design of more robust and optimized agent architectures. This work provides insights into when and how agentic features contribute to more effective and controllable information extraction.