Trustworthy Agentic AI: A Framework for Critical Systems.

Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat, Michael Mandulak, Jaewon Kim, Eman Hammad· July 22, 2026 View original

Summary

This survey proposes a trustworthiness model for agentic AI in critical engineering domains, focusing on safety, robustness, transparency, accountability, and privacy. It outlines an assurance workflow and surveys architectures, threats, and mechanisms for developing and evaluating trustworthy agentic systems.

As agentic AI systems, capable of autonomous perception, planning, and multi-step action, are increasingly considered for critical engineering applications, their trustworthiness becomes paramount. This research addresses a gap by defining trustworthiness as a core engineering property, rather than solely focusing on task capability. It introduces a comprehensive trustworthiness model built around five key dimensions: safety, robustness, transparency, accountability, and privacy. The study maps these dimensions onto an agentic assurance workflow, covering everything from perception to audit. It then surveys existing agentic system architectures, potential threats, concrete trust mechanisms, and quantitative metrics relevant for development and evaluation. These principles are examined across diverse, constraint-bound engineering domains such as power systems, autonomous vehicles, high-performance computing, and communication networks. By synthesizing findings across these domains, the research concludes that agentic AI trustworthiness is a unified problem. It outlines a path towards a reusable, cross-domain assurance framework, drawing parallels to the graded certification regimes found in mature safety-critical engineering fields. This aims to provide a structured approach for ensuring that AI agents can be verified, audited, and trusted in high-stakes environments.

Why it matters

Professionals deploying AI in critical infrastructure or products need a robust framework to ensure these systems are safe, reliable, and auditable, mitigating significant operational and reputational risks.

How to implement this in your domain

  1. 1Adopt the proposed trustworthiness model as a foundational principle for designing agentic AI systems.
  2. 2Integrate the outlined assurance workflow into the development lifecycle of critical AI applications.
  3. 3Evaluate existing agentic AI architectures against the identified threats and trust mechanisms.
  4. 4Develop quantitative metrics to continuously monitor and assess the trustworthiness of deployed AI agents.
  5. 5Collaborate with industry bodies to establish cross-domain certification standards for agentic AI.

Who benefits

AutomotiveEnergyAerospaceTelecommunicationsHealthcare

Key takeaways

  • Trustworthiness is a critical, first-class engineering property for agentic AI.
  • A comprehensive model includes safety, robustness, transparency, accountability, and privacy.
  • An assurance workflow spans perception to audit for agentic systems.
  • Cross-domain patterns suggest a unified approach to agentic AI trustworthiness.

Original post by Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat, Michael Mandulak, Jaewon Kim, Eman Hammad

"arXiv:2607.18548v1 Announce Type: new Abstract: Agentic artificial intelligence systems, capable of autonomous perception, planning, tool use, and multi-step action, are increasingly proposed for critical engineering domains where decisions carry physical, operational, or economi…"

View on X

Originally posted by Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat, Michael Mandulak, Jaewon Kim, Eman Hammad on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses