How Did OpenAI Design Its First Chip in Under Two Years?

OpenAI unveiled Jalapeño on 25 August. The company claims up to 13.4 petaflops of 4-bit compute, 232 gigabytes of HBM4 memory, and 15.4 terabytes per second of memory bandwidth. Benchmarks cited by OpenAI show up to 3.6 times lower end-to-end latency than Nvidia's GB300 while drawing less power. Whether those figures hold in fleet-wide inference service remains to be seen. The other half of the story is how the chip was designed.

A timeline veterans call unprecedented

Jalapeño moved from first architecture concept to first silicon in under 20 months. Only nine months separated the first register-transfer level code from tape-out.

Andrew Kahng, distinguished professor at the University of California San Diego, called that speed "likely best in class today." He co-organized a 2016 IEEE Design Automation Futures workshop where Richard Ho, then at Google, framed design time as a function of how many iterations a team could complete in a day. Ho now leads hardware at OpenAI.

The design team averaged fewer than 100 people across the project. That number covers system design, software, and supply chain roles but not Broadcom engineers. Broadcom handled physical design from the gates onward. OpenAI owned end-to-end system design: the inference accelerator, the memory hierarchy, and networking.

David Chin of Verkor.io said Broadcom's help was essential. Ravi Krishna, also at Verkor, called the speed "a relatively impressive result" but expects even quicker timelines if the project started today.

The front-end workflow that LLMs could actually accelerate

Automation in chip design has existed for decades. Ankur Srivastava of the University of Maryland said LLMs differ because they understand language and code. That makes them suited for tasks still in the linguistic domain.

The front-end workflow was built around Accelerated Hardware Synthesis (XLS), an open-source high-level synthesis chain originally developed at Google. XLS lets engineers write in DSLX — a domain-specific language inspired by Rust — and C++. XLS then converts those to Verilog.

Chris Leary, a member of technical staff at OpenAI, said the team thought about how to leverage AI to move faster. The AI was much better at software-looking things. XLS in some ways looks like software so it got that benefit. Leary started XLS during his time at Google. Kahng agreed that using AI to accelerate high-level synthesis makes sense because it is more natural for the LLM and enables fast iteration.

The same logic pushed the team toward software optimization. When first silicon returned from the foundry in May, internal AI models were pointed at writing kernels for benchmarks such as SemiAnalysis's InferenceX. On DeepSeek's multi-head latent attention kernel, utilization climbed from 0.31 percent of the theoretical ceiling to 88.94 percent in roughly 40 hours. Ho said the result is repeatable. The gap between foundry delivery and production ramp can shrink. All schedule assumptions now factor in that capability.

Models evolved during the project

The project began with assistance from models like o3, released publicly in April 2025 but available to the Jalapeño team earlier. By tape-out the team had access to precursors of GPT-6 Astra, which was not publicly released until 3 September 2026. The newer model can work directly in Verilog without XLS translation. It is close to operating proprietary design tools on its own. Ho confirmed the team also used internal LLMs fine-tuned for chip design that are not available to the public. He declined to detail them. The goal is to fold lessons from Jalapeño into commercial models. Ho said it is safe to say that Astra and following models will be very good at chip design.

Backend optimization stayed human for now

The bulk of OpenAI's work focused on the front end: concept through RTL through verification. Backend tasks — routing interconnects, clock and power sign-off, foundry data prep — went to Broadcom. OpenAI physical design engineers still collaborated on floor plan and routing guidance. At Hot Chips 2026 Ho and Leary put numbers on AI-guided physical design optimization: a 10 percent area reduction for matrix multiplication units measured against an optimized human baseline. Broadcom used its own workflow. Its team did not have OpenAI's internal models but did have access to public commercial models.

Verkor's Ravi Krishna said OpenAI's backend approach already feels conservative. He attributed that to when the project started — October 2024. Models from the last four to five months improved. From April 2026 onward they really started to handle those tasks better. Suresh Krishna, also at Verkor, said there is no reason an agentic loop could not largely accelerate the backend as well.

The next generation resets the workflow

Leary said Jalapeño forced a trade-off: innovate on workflow or use what works historically. The second generation gets a reset. Ho said the new workflow has "a lot of places that we are introducing AI." He mentioned verification and physical design. Leary added that the team now has tools for automatic waveform manipulation and viewing. That automates analysis to identify clock signals associated with failures and could improve debugging while the chip is still being designed.

Ho and Leary were clear that chip design cannot be fully automated. Ho said they are not saying anyone can build frontier AI accelerators using just Codex. They are saying specific things about how to be better at Codex and how a small team on a fast timeline can reach quality results. The boundary of what LLMs can replace in the backend remains open. The document does not say when or if that boundary will disappear.