Learnijoy
ResearchAI ResearchAI Engineering & DevTools

HyenaND Offers Subquadratic Multi-Dimensional Operators for AI Models

David R. Wessels, Farhad Ramezanghorbani, David W. Romero, Alireza Moradzadeh, Olivia Viessmann, Maksim Zhdanov, John St. John, Ken Janik, David M Knigge, Yucheng Tang, Erik J Bekkers, Saee Gopal Paliwal· July 23, 2026 View original

Summary

HyenaND introduces subquadratic, global, input-dependent operators that process multi-dimensional data directly, avoiding the compromises of standard convolutions or 1D rasterization. Its CUDA implementation, nSubQ, achieves wall-clock speedups and matches or outperforms attention baselines across various long-context tasks.

Researchers have developed HyenaND, a novel class of subquadratic operators designed to efficiently process multi-dimensional data in AI models. Traditional subquadratic alternatives to attention mechanisms often struggle with multi-dimensional inputs, either lacking global receptive fields (like standard convolutions) or requiring data to be flattened into a 1D sequence, which distorts spatial relationships. HyenaND overcomes these limitations by directly operating on the native geometry of data, such as images, volumes, and PDE solutions. The core of HyenaND involves using convolutions with implicitly parameterized global, input-dependent multi-dimensional convolutional kernels. A specialized CUDA implementation, named nSubQ, further enhances its performance by fusing the FFT-convolution path, leading to significant wall-clock speedups. Across diverse applications including long-context genomics, computer vision, medical imaging, and PDE modeling, pure HyenaND stacks have demonstrated accuracy comparable to strong attention-based models. Hybrid configurations, which combine HyenaND with attention layers, have even surpassed the performance of both pure attention and recurrence-based hybrid models.

Why it matters

This innovation offers a more efficient and accurate way to handle multi-dimensional data in AI models, potentially accelerating training and inference for complex tasks in various domains.

How to implement this in your domain

  1. 1Evaluate HyenaND as an alternative to attention mechanisms for processing multi-dimensional data in your AI models.
  2. 2Experiment with integrating HyenaND layers into existing computer vision or scientific computing architectures.
  3. 3Benchmark the performance and accuracy of HyenaND against current state-of-the-art models on relevant datasets.
  4. 4Consider contributing to or leveraging the nSubQ CUDA implementation for optimized performance.

Who benefits

Computer VisionMedical ImagingScientific ComputingGenomicsAutonomous Systems

Key takeaways

  • HyenaND provides subquadratic, global, input-dependent operators for multi-dimensional data.
  • It avoids the compromises of 1D rasterization or limited receptive fields in other methods.
  • The nSubQ CUDA implementation delivers significant wall-clock speedups.
  • HyenaND matches or outperforms attention baselines in various long-context applications.

Original post by David R. Wessels, Farhad Ramezanghorbani, David W. Romero, Alireza Moradzadeh, Olivia Viessmann, Maksim Zhdanov, John St. John, Ken Janik, David M Knigge, Yucheng Tang, Erik J Bekkers, Saee Gopal Paliwal

"arXiv:2607.19378v1 Announce Type: new Abstract: Subquadratic alternatives to attention require compromises when applied to multi-dimensional data: standard convolutions lack global receptive fields and input dependency, while recurrent models require rasterizing data such as imag…"

View on X

Originally posted by David R. Wessels, Farhad Ramezanghorbani, David W. Romero, Alireza Moradzadeh, Olivia Viessmann, Maksim Zhdanov, John St. John, Ken Janik, David M Knigge, Yucheng Tang, Erik J Bekkers, Saee Gopal Paliwal on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses

More in AI Research

AI ResearchAI Engineering & DevTools

New Adaptive Filter Improves Time-Series Prediction with Input Noise

Researchers developed the RFFBCGA algorithm, a new nonlinear adaptive filter that effectively mitigates both input and output noise in time-series prediction. This method maintains a fixed network structure while enhancing robustness across various noise scenarios.

Yi Peng, Haiquan Zhao, Jinhui HuJul 23, 2026
AI ResearchAI Engineering & DevTools

New Algorithm Learns Local Causal Structures with Latent Variables

Researchers propose LoCaLS, a new algorithm for learning local causal structures around a target variable from observational data, even when latent variables and selection bias are present. LoCaLS achieves high accuracy with significantly less computational effort than global causal discovery methods.

Zheng Li, Hao Zhang, Ruxin Wang, Ruichu Cai, Kun Zhang, Feng XieJul 23, 2026
AI Engineering & DevToolsAI Research

New Framework Evaluates AI Robustness with Minimum-Norm Attacks

Researchers introduce a unified framework for evaluating adversarial robustness using a comprehensive pool of minimum-norm attacks and robustness-perturbation curves across multiple norms. This approach addresses limitations of fixed-epsilon evaluations, providing a more stable and controllable assessment of AI model defenses.

Luca Scionis, Luca Melis, Maura Pintor, Fabio Brau, Ambra Demontis, Giorgio Fumera, Fabio Roli, Battista BiggioJul 23, 2026