Why do pointwise convolutions choke large-kernel CNNs on phones?
Large-kernel convolutional networks expand receptive fields and boost vision accuracy, but their parameter count grows quadratically. The paper shows that pointwise convolutions alone account for more than 87 percent of parameters in models such as RepLKNet-31B, making storage and loading the real barrier on devices with 4–12 GB of RAM.
The blind spot in current compression
Depthwise separable backbones have been the standard trick for keeping large-kernel CNNs lean. Researchers compress the depthwise part with low-rank approximation and weight sharing. The paper points out a blind spot: the pointwise convolutions that sit between depthwise stages. In RepLKNet-31B they contribute over 87 percent of all parameters. The same pattern holds for ConvNeXt and SLaK. On a phone with 4–12 GB RAM the model simply does not fit, or loading it stalls the memory bus. Storage fills fast.
How Channel Group-Shared low-rank approximation rewrites the math
The authors propose a structured low-rank paradigm isomorphic to singular value decomposition. Shared down-projection and up-projection matrices span channel groups inside a layer. Each group keeps its own lightweight diagonal matrix. The high-cost matrices are reused; the per-group matrices stay tiny. Parameter count drops sharply while the decomposition remains mathematically equivalent to SVD.
What changes on the device
Smaller checkpoints mean the model fits in storage. Reduced memory bandwidth pressure during loading cuts latency. Pre-trained RepLKNet and ConvNeXt models enhanced with CGS become feasible to deploy on edge hardware. SLaK sees the same benefit. The method was accepted at MobiCom 2026. Phones choke.
The sources this article was written from
- "arXiv:2608.26069v1
https://arxiv.org/abs/2608.26069v1"