Axon DSL Enables Portable, High-Performance LLM Architectures.

The current ecosystem for open-source large language models (LLMs) is heavily reliant on a few dominant platforms, creating potential single points of failure and limiting flexibility. Implementing and maintaining efficient model definitions, along with translating them across various training and inference regimes, is a resource-intensive task that hinders both scaling and deployment efforts. To address these challenges, researchers present Axon, a strongly typed domain-specific language (DSL) with a Haskell-like syntax. Axon enables a "write-once, run-everywhere" paradigm for LLM architectures, fostering open collaboration by basing development on a language specification rather than a specific framework's vision. This approach empowers researchers to create highly specialized architectures without sacrificing optimization infrastructure or facing deployment lock-in. Axon allows for concise and auditable specifications that can be automatically compiled into standalone implementations for leading frameworks, including PyTorch, PyTorch with Triton, JAX, MLX, and vLLM. In extensive inference benchmarking experiments involving models from 135M to 32B parameters, Axon demonstrated impressive median speedups: 7% on PyTorch, 12% on PyTorch with Triton, 91% on JAX, and 107% on MLX, compared to reference implementations from the Transformers library. When deployed as native vLLM architectures with PagedAttention and KV-cache, Axon models achieved a 58% median speedup over Transformers implementations.