OpenAI unveils its first custom inference chip, built with Broadcom

OpenAI has introduced its first custom-designed processor, a move that signals the company’s deepening push into hardware to reduce reliance on Nvidia’s dominant GPUs. The chip, named Jalapeño, was developed in partnership with Broadcom and is purpose-built for inference workloads—the process of running pre-trained AI models to respond to user commands.

A strategic shift toward vertical integration

Announced Wednesday, the Jalapeño chip represents a significant step in OpenAI’s strategy to control more of its technology stack. The company has long been rumored to be exploring custom silicon as a way to cut costs and improve performance. OpenAI said its own AI models assisted in the chip’s design, and early testing shows substantially better performance-per-watt compared to current state-of-the-art alternatives.

The partnership with Broadcom was first disclosed in October, but Wednesday’s announcement provides the first concrete details about the chip’s capabilities and design philosophy. OpenAI joins Google and Amazon, both of which have developed custom AI accelerators to optimize their cloud and AI operations.

Why inference matters for AI economics

Inference is a critical and increasingly costly part of AI deployment. While training large models remains a resource-intensive process that will likely continue to rely on Nvidia hardware, inference costs add up quickly as usage scales. Even small efficiency gains in inference can significantly improve a company’s bottom line.

OpenAI emphasized the chip’s low operating cost when running real-time coding models, such as those powering its Codex product. The company noted that Jalapeño is designed specifically for these kinds of workloads, which are often underserved by general-purpose GPUs.

Full-stack optimization

In its announcement, OpenAI framed the chip as part of a broader effort to optimize every layer of its infrastructure. “OpenAI is not only developing frontier models or building products on top of them; it is designing the infrastructure underneath them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience,” the company wrote.

This full-stack approach allows OpenAI to tune hardware, software, and model design around a unified goal: making models faster, more reliable, and more affordable for users. President Greg Brockman previously explained the company’s rationale on its in-house podcast, saying the team identified specific workloads that were underserved and asked how to build something that could accelerate what’s possible.

What this means for the AI hardware landscape

OpenAI’s entry into custom chip design adds a new dynamic to the already competitive AI hardware market. Nvidia remains the dominant supplier of AI training and inference hardware, but the rise of custom accelerators from hyperscalers and AI labs is gradually shifting the balance. Broadcom, which has extensive experience designing custom chips for major tech companies, stands to benefit from this trend.

While Jalapeño is still in testing, its successful deployment could pave the way for OpenAI to further reduce its dependence on external suppliers. The company is also building its own data centers and agentic products, making chip design a natural extension of its existing capabilities.

Conclusion

OpenAI’s unveiling of the Jalapeño inference chip marks a pivotal moment in the company’s evolution from AI model developer to full-stack infrastructure builder. By designing custom silicon with Broadcom, OpenAI is positioning itself to better control costs, performance, and innovation in an increasingly competitive market. The chip’s focus on inference efficiency underscores the growing importance of optimizing the entire AI stack, from model architecture to hardware design.

FAQs

What is the Jalapeño chip?
Jalapeño is OpenAI’s first custom-designed processor, built in collaboration with Broadcom specifically for AI inference workloads.

Why did OpenAI build its own chip?
The chip aims to reduce OpenAI’s dependence on Nvidia GPUs, lower inference costs, and improve performance-per-watt for running AI models.

Will Jalapeño replace Nvidia hardware entirely?
No. OpenAI says more performance-intensive tasks like pre-training will likely still rely on Nvidia hardware. Jalapeño is designed specifically for inference, where efficiency gains can have a significant impact on operating costs.