What Does Google's HEIR Compiler Mean for Running AI on Encrypted Data?

What HEIR Actually Does

Homomorphic encryption lets a server compute on ciphertext and return an encrypted result. The server never sees the plaintext. Google's blog post frames this as a shift from a capability-versus-privacy trade-off to a question of cost. HEIR (Homomorphic Encryption Intermediate Representation) is the compiler that tries to make that cost manageable for ordinary engineers.

It takes a pre-trained model written for unencrypted inference and rewrites it for encrypted inference. No cryptographer required. That is the stated vision: a one-click path from a standard model to a production service that never touches user data in the clear. The project was announced in 2023. Today it is open source and hosted on GitHub.

Latency figures in the post are measured on a single-threaded CPU. Hardware accelerators from Belfort, Niobium, Cornami, and Optalysys are being integrated. Google says it will demonstrate the latency gains from those accelerators soon.

The Four Demonstrations Google Showed

A deep learning recommendation model compiled with HEIR serves private content recommendations. The work was done with Belfort Labs, LG, and New York University. A credit card fraud detector was compiled together with Niobium and hardshell.ai. The Kitsune system for anomaly detection on encrypted network traffic was compiled with Niobium. A hotword detection model was compiled with Belfort Labs.

Each demo proves a different use case. Recommendations without seeing user features. Fraud detection without seeing transaction details. Network intrusion detection without seeing packet contents. Voice activation without hearing the audio. The source code for all four is in the repository.

Who Is Building on Top of It

Academic groups have adopted HEIR as a research platform. Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Institute of Technology, Purdue, the University of Edinburgh, and Tsinghua University are listed collaborators. Four peer-reviewed publications have been built on HEIR. More are in preparation. The compiler has accumulated numerous citations.

Hardware partners are building accelerators that target HEIR's intermediate representation. Belfort, Niobium, Cornami, and Optalysys are named. The post does not give ship dates for those chips. It does not say which cloud regions will offer them first.

Where the Friction Still Lives

The post acknowledges a nontrivial cost overhead. It does not quantify that overhead in dollars or watts. It does not compare HEIR-compiled latency to plaintext inference on the same hardware. It does not say what model sizes fit in memory today. The single-threaded CPU numbers are a baseline, not a promise.

Shipping proprietary models to devices risks leaking the model. Homomorphic encryption keeps the model on the server. The server still needs the model in the clear to compile it. The post does not describe a flow where the model owner encrypts the model itself before handing it to the compiler.

Open source compiler. Open source demos. Hardware integrations not yet public. The one-click vision remains a vision.