IBM LinuxONE Accelerates Secure Enterprise RAG with Spyre

IBM has introduced a new architecture for Retrieval-Augmented Generation (RAG) that operates entirely within its LinuxONE and IBM Z mainframe environments. This solution addresses the common enterprise challenge of securely processing sensitive data with AI, where data movement between different systems can introduce latency, security risks, and regulatory compliance issues. The core of this architecture is the Spyre accelerator PCIe inference card, specifically designed for LinuxONE, which handles generative inference. The system also leverages the Telum II on-chip accelerator for lighter classification tasks and Red Hat OpenShift for container orchestration. Every stage of the RAG pipeline, from query intake to final response delivery, remains within the secure LinuxONE system, ensuring sensitive data never leaves the hardware perimeter. Benchmarking indicates that this integrated approach achieves end-to-end RAG latencies under two seconds, representing up to a 20x reduction compared to off-platform inference. Crucially, it maintains the robust encryption and auditability required by highly regulated industries, extending confidential computing guarantees to AI workloads through LinuxONE's Secure Execution technology.