NEXUS Enhances Runtime Safety for Tool-Using LLM Agents

Elias Hossain, Md Mehedi Hasan Nipu, Tasfia Nuzhat Ornee, Rajib Rana, Niloofar Yousefi· July 23, 2026 View original

Summary

NEXUS is a structured-plan safety monitor for tool-using LLM agents that applies a formal intervention policy, combining deterministic rules, argument-level inspection, and a calibrated risk score. It achieves high F1 scores and intervention accuracy, outperforming rule-only methods with minimal latency.

As large language model (LLM) agents increasingly perform high-impact actions through tool use, ensuring their runtime safety becomes paramount. This paper introduces NEXUS (Neural EXecution Utility and Safety), a structured-plan safety monitor designed to provide robust runtime oversight for these agents. NEXUS operates by applying a formal intervention policy, which allows it to select among four actions: allow, block, request confirmation, or request revision. This decision-making process integrates deterministic safety rules, granular argument-level inspection, and a calibrated logistic-regression risk score, enabling a graded escalation of intervention based on perceived risk. Evaluated on a synthetic benchmark, NEXUS achieved an impressive F1 score of 0.949 and a 4-class intervention accuracy of 0.6406, significantly outperforming rule-only intervention selection by 27.3 percentage points. It also demonstrated strong performance on other benchmarks with minimal latency (0.205 ms median), adding less than 0.1% overhead to typical agent loops, making it practical for real-world deployment.

Why it matters

Professionals developing or deploying AI agents that interact with real-world systems or sensitive data can use NEXUS to implement robust safety mechanisms, preventing unintended actions, mitigating risks, and building more trustworthy autonomous systems.

How to implement this in your domain

  1. 1Assess the potential risks and failure modes of your tool-using LLM agents in production environments.
  2. 2Investigate integrating structured safety monitors like NEXUS into your agent architectures to control high-impact actions.
  3. 3Develop formal intervention policies that define how your agents should respond to detected risks (e.g., block, confirm, revise).
  4. 4Utilize argument-level inspection and calibrated risk scoring to enable fine-grained and adaptive safety interventions.

Who benefits

AI InfrastructureRoboticsAutonomous SystemsCybersecuritySoftware Development

Key takeaways

  • NEXUS provides structured runtime safety for tool-using LLM agents.
  • It combines deterministic rules, argument inspection, and a calibrated risk score.
  • The system offers four intervention actions: allow, block, confirm, revise.
  • NEXUS significantly outperforms rule-only methods with minimal latency overhead.

Original post by Elias Hossain, Md Mehedi Hasan Nipu, Tasfia Nuzhat Ornee, Rajib Rana, Niloofar Yousefi

"arXiv:2607.19356v1 Announce Type: new Abstract: Tool-using LLM agents increasingly execute high-impact actions, making runtime safety monitoring essential. We present NEXUS (Neural EXecution Utility and Safety), a structured-plan safety monitor that applies a formal intervention…"

View on X

Originally posted by Elias Hossain, Md Mehedi Hasan Nipu, Tasfia Nuzhat Ornee, Rajib Rana, Niloofar Yousefi on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses