What Is RLX and Why Does a Single Rust Codebase Replace Half a Dozen ML Stacks?

One primitive IR, three levels, fourteen devices

RLX builds its entire stack around a single primitive-level intermediate representation that exists in three levels.

The same IR drives code generation for cpu, metal, mlx, ane, cuda, rocm, oneapi, tpu, hexagon, gpu, vulkan, opengl, directx, and webgpu. Two additional codegen paths target Cortex-M INT8 microcontrollers and FPGA fabric. The dispatch contract is transparent: every operator resolves to a native lowering, a common-IR lowering, or a rewritten lowering. If none of those paths is legal, compilation stops with an error instead of falling back silently. No silent fallbacks. How many of the fourteen backends have reached production maturity is not disclosed.

Formats, numerics, and collectives

The front end ingests safetensors, GGUF, ONNX, and rten model formats without external converters. Numeric support spans F16, BF16, F64, C64, and quantized INT4 and INT8 flows with automatic mixed precision, post-training quantization, and quantization-aware training. For distributed runs RLX uses tensor-parallel and pipeline-parallel collectives over both TCP and RDMA transports. The runtime does not require a separate orchestration layer; the same binary that compiles the graph also launches the collectives. Memory footprint and compile-time latency for large models are not reported.

Beyond neural nets

Sparse and dense linear algebra extensions bring CSR LU, conjugate-gradient, matvec, and LAPACK-backed factorizations into the same IR. 3D Gaussian splatting operators are also implemented, letting the compiler target scientific and physics workloads without switching toolchains. The paper notes that these extensions share the same dispatch contract and legalization checks as the neural operators.

Measured against thirteen rivals

Evaluation runs on one host used identical input generation and p50 latency methodology. On all-MiniLM-L6-v2 RLX-Metal was fastest at every batch size; at batch 32 it recorded 16.6 ms versus 26.7 ms for PyTorch-MPS. In MNIST training the graph-fused MLP reached 946,487 images per second, ahead of NumPy+BLAS at 787,349 images per second. Reference checks on Qwen3 showed 100 percent top-1 parity. The comparison set included PyTorch, TensorFlow, JAX, candle, burn, tch, rten, MLX, CoreML, IREE, Glow, TensorRT, and tinygrad. Power draw was not measured.