Conversational XAI Improves UAV Intrusion Detection Trust, Risks Over-Reliance

This research investigates the effectiveness of different Explainable AI (XAI) interfaces for Unmanned Aerial Vehicle (UAV) intrusion detection systems. Specifically, it compares a conversational XAI interface, which leverages large language models (LLMs) for on-demand investigation, against a conventional static visualization dashboard. The goal was to understand their impact on operator comprehension, trust, and reliance during post-incident auditing. The empirical study revealed that the conversational interface was perceived as more useful, likely due to its ability to facilitate easier access and synthesis of relevant information. However, this benefit came with a notable drawback: a lower level of appropriate self-reliance among participants. This suggests a potential risk of operators over-relying on the AI's advice, possibly because natural language responses make the information easier to accept without critical verification. The findings underscore a critical trade-off in human-AI collaboration, particularly in high-stakes domains like UAV security. While intuitive interaction mechanisms can boost perceived usability, they might inadvertently increase the risk of inappropriate reliance. Future XAI system designs should aim to balance seamless interaction with features that encourage critical thinking and verification to foster appropriate reliance.