CANN Bench: New Benchmark for AI-Generated Kernels on Huawei NPUs.

This new research introduces CANN Bench, an open-source benchmark designed to evaluate the performance of AI-generated operator kernels specifically on Huawei's Ascend Neural Processing Units (NPUs). Unlike existing benchmarks that primarily focus on CUDA and Triton, CANN Bench addresses the need for a standardized evaluation framework in less-exposed hardware ecosystems. The benchmark includes a comprehensive set of 53 operators and 1060 test cases, ranging from basic element-wise operations to complex FlashAttention kernels, supporting multiple precision formats like FP16, BF16, FP32, and INT8. The evaluation methodology employs a sophisticated three-dimensional weighted composite score, which independently assesses compilation success, functional accuracy, and execution performance. Performance is measured against both a PyTorch-on-Ascend baseline and an analytical Hardware-Anchored Performance (HAP) limit, ensuring that reported scores reflect genuine optimization potential rather than measurement artifacts. The framework is built to prevent reward hacking and is intended for long-term community collaboration within the official CANN repository, providing a robust tool for advancing AI operator-authoring capabilities on Ascend NPUs.