Why Do Sparse Neural Activations Make Language Models Run Faster on Neuromorphic Chips?

The problem with dense projections

Transformer models keep every token in a key-value cache. That cache grows with sequence length. Attention scores scale quadratically. State-space models replace the cache with a fixed recurrent state and linear attention. The memory problem shrinks. The compute problem stays. Each layer still runs large dense matrix multiplications. Quantization helps but does not remove the operations. The paper calls them computationally expensive even after quantization.

How the threshold works

The authors add a trainable threshold Δ to every linear projection. Activations smaller than ±Δ become zero. Larger values pass through unchanged. Outliers survive. The threshold learns per projection during training. No hand-tuning required. The model stays quantized. Sparsity emerges without extra parameters. Performance stays comparable to dense models. Effective arithmetic operations drop up to four times.

Sparsity is unstructured.

From sparsity to throughput

Unstructured sparsity usually hurts GPU performance. Threads diverge. Memory accesses scatter. Neuromorphic chips work differently. They process events. A zero activation produces no event. No event means no compute and no communication. The paper projects up to four times fewer effective arithmetic operations versus the dense baseline. On a multi-core multi-chip platform that translates to thirty-seven times higher throughput and sixteen times lower power. The non-sparsified baseline on the same hardware improves five point four times.

Where the numbers come from

The thirty-seven times and sixteen times figures are projections. They compare against a comparable transformer model on an edge GPU. The five point four times gain compares against the same linear-attention model without sparsity on the neuromorphic platform. Real silicon measurements are not reported in the document. The paper was accepted at IEEE MCSOC2026 and spans eight pages with four figures.