New Framework Boosts Reliable Web Data Collection with LLMs.

While large language models (LLMs) and agents can generate web scrapers from natural language, their direct output often suffers from unreliability due to various errors like broken selectors or schema mismatches. To address this, researchers have developed a new framework designed to make open-web data collection safer and more reliable. This framework shifts the LLM's role from generating free-form code to producing structured, typed JSON collector configurations. It incorporates a six-type collector taxonomy, template and utility-function constraints, static Airflow DAG execution, rule-based quality checking, and structured feedback for correction. Experiments show that this approach, while requiring more initial constraint completion, leads to a reusable, deterministic, and verifiable execution path for repeated data collection, outperforming direct LLM code generation in terms of reliability and execution time.