Agentic AI Poses New Threat to Online Survey Data Quality

Online surveys are a fundamental data collection tool, relying on attention checks to ensure response quality. However, the rise of agentic AI—goal-directed systems powered by LLMs and multimodal capabilities—introduces new challenges to the robustness of these safeguards. This study evaluates a single-agent architecture with multimodal input processing and web interaction capabilities on a controlled survey sandbox. From an "attack" perspective, the research shows how structural vulnerabilities, such as exposed DOM metadata and predictable option encoding, allow these agents to resolve attention checks simply by structured parsing of the web page. From a "defense" perspective, the study implements a mitigation strategy involving DOM metadata obfuscation to remove semantic cues from text-based questions. Evaluations of various open-source language and multimodal models highlight the agents' capabilities and the effectiveness of the obfuscation. The findings offer insights for both empirical researchers and those developing agentic AI.