How Does a 37k Parameter Model Run Real-Time Speech Enhancement on a Microcontroller?

The Hardware Wall

Microcontroller NPUs accelerate neural inference under strict latency, memory, and energy constraints. They only support a restricted set of operators in static, integer-quantized graphs.

Recent speech-enhancement networks have reduced parameter counts and MACs to levels nominally suitable for microcontrollers. Their operators and execution patterns often remain incompatible with restricted NPUs. The gap is not theoretical. It appears when a model that looks small on paper cannot run on the actual silicon. This is the problem the authors set out to solve.

Redesigning LiSenNet for the Neural-ART Accelerator

The authors target the STM32N6570-DK Neural-ART accelerator. They start from LiSenNet, a 37k parameter sub-band dual-path model. The recurrent bottleneck is replaced with convolutional frequency and temporal mixers. Unsupported operations are reformulated as static int8-compatible primitives. Bounded decoder activations preserve quality after quantization. Each change serves the hardware constraint directly. No change is made for algorithmic novelty alone. The redesign touches architecture, quantization range, and streaming state together.

Quantization Results on VoiceBank-DEMAND

On VoiceBank-DEMAND the final NPU-compatible model matches or exceeds the recurrent LiSenNet baseline. FP32 PESQ reaches 3.08 versus 3.01 for the baseline. Int8 PESQ reaches 3.01 versus 2.93. Quality does not collapse after quantization. The bounded activations hold the dynamic range. The paper reports no subjective listening tests. It does not compare against larger server-side models.

On-Device Timing and Co-Design

Deployed on the microcontroller the model processes each 16 ms input hop in 4.83 ms. That corresponds to a real-time factor of 0.30. Stateless receptive-field recomputation is an order of magnitude slower at the same frame rate despite higher accelerator utilization. Persistent streaming state makes the difference. The document does not give power measurements. It does not specify memory footprint in kilobytes. Code and checkpoints are not released. Other microcontroller NPUs are not tested. Parameter count and operator compatibility, quantization range, and persistent streaming state must be co-designed. Optimizing one dimension in isolation fails on restricted NPUs. The paper demonstrates this with a working implementation. The principle transfers. The exact numbers belong to this hardware and this model.