How Does a Convolutional Network Learn to Listen Like a Recurrent One on a Smartwatch?
Keyword spotting lets a device wake up when it hears a specific phrase. On a phone the processor has headroom; on a watch every milliwatt counts. The Interspeech 2026 paper shows how to keep CNN training simplicity while adding LSTM‑like streaming.
Why a watch needs a different kind of listening
A conventional CNN consumes a fixed‑length audio window at once. To run continuously it must re‑evaluate the whole window every 10 ms or 20 ms, repeating most of the math for overlapping frames. On heavily resource‑constrained edge devices such as wearables the repeated multiply‑accumulate operations dominate the energy budget. Real‑time latency adds a third constraint alongside compute and memory limits.
How the sub‑model cache works
Short‑Term Memory Convolutions cache feature maps for reuse when the input shifts. The new work splits the network into independent sub‑models, each with its own buffer. Only the affected sub‑model updates.
What the numbers show
The team reports up to 82 % fewer MCPS than an equivalently frequent standard CNN and 46 % fewer than vanilla STMC. The best configuration reaches 93.8 % accuracy on the 11‑class Google Speech Commands task and 97.1 % with zero‑padded data.
Where the five pages stop
Latency on a specific microcontroller is not reported. Tests cover only the 11‑class subset.
The sources this article was written from
- "arXiv:2609.35005v1
https://arxiv.org/abs/2609.35005v1"