Janus Playground Explores User-Involved AI Agent Permissions

As AI agents gain autonomy in executing tool calls on behalf of users, effective permission management becomes a critical concern. This research introduces Janus, a novel playground system specifically designed to explore and evaluate different approaches to user involvement in agentic permission management. Janus comprises Janus-Core, a modular agentic system supporting diverse permission designs, and Janus-Harness, an automated evaluation framework. Grounded in a conceptual model that identifies key design axes for user involvement, the researchers implemented six distinct permission assistants. These assistants were evaluated across three scenarios and with three synthetic user responders. The findings highlight that direct user input is essential for strengthening privacy and security in agentic systems. Furthermore, the study shows that AI augmentation can help reduce the cognitive load on users when making permission decisions. However, it also emphasizes the importance of accounting for realistic user behavior, such as "permission fatigue," in system design. The conclusion is that no single permission design is universally optimal, necessitating a more principled and context-sensitive approach to deploying permission assistants. Janus is publicly available to foster further research in this vital area.