AgenticCANN Automates Ascend C Operator Generation for NPU Optimization

Optimizing Ascend C operators is crucial for maximizing Neural Processing Unit (NPU) inference performance, but it demands extensive hardware expertise. While large language models (LLMs) have shown promise in generating CUDA kernels, the distinct programming model of Ascend C presents unique, unexplored challenges. To tackle this, researchers propose AgenticCANN, a framework that uses knowledge-augmented agentic evolution specifically for automated Ascend C operator synthesis in environments with limited NPU-specific data. AgenticCANN overcomes the scarcity of platform knowledge by integrating a knowledge-orchestrated generation system. This system delivers structured, multi-level domain insights throughout the development lifecycle, resolving the initial feasibility bottleneck. Building on this, it employs a stage-adaptive agentic evolution strategy that dynamically adjusts LLM interaction modes to suit different generation and evolution phases. This balances the need for high-exploration candidate discovery with high-convergence performance tuning. Extensive experiments on Huawei Ascend 910B across various operators demonstrate high feasibility (90-100% for elementwise and normalization, 56% for fusion) and significant speedups, including up to 6.65x for 1B Pangu model inference kernels. Analysis confirms that knowledge injection consistently improves feasibility, highlighting its general benefit.