Kernel Forge Optimizes CUDA Kernels with LLM Agents for Faster AI Models

Kernel Forge is a new open-source, end-to-end agentic harness designed to automate the generation and optimization of CUDA kernels using large language models (LLMs). This system addresses the traditional challenge of requiring expert engineers to hand-write low-level GPU code for performance-critical machine learning operations. Kernel Forge can accept any unmodified PyTorch model, supporting a wide range of workloads including vision, diffusion, and LLMs. A key innovation in Kernel Forge is its use of Monte Carlo Tree Search (MCTS) to explore multiple optimization paths, moving beyond simple linear refinement chains. It also provides a graphical user interface for monitoring progress, inspecting candidate kernels, and debugging. Evaluations on an NVIDIA DGX Spark with a GB10 GPU demonstrated significant performance gains, with Kernel Forge optimizing 14 kernels to outperform PyTorch eager mode. It achieved speedups of up to 2.83x on softmax in Gemma 4 E2B and 1.70x on group_norm in Stable Diffusion 3.5 Medium, showcasing its potential to reduce latency and cost in ML deployments.