OpenAI’s first published Jalapeño results are strong enough to change an infrastructure shortlist. They are not an independent lab verdict. SemiAnalysis engineers ran the public benchmark with OpenAI engineers inside OpenAI’s lab.
OpenAI Jalapeño is a custom inference chip designed as part of the company’s full serving stack. OpenAI reports 1.5 to 1.9 times more work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency in selected comparisons with NVIDIA GB200 and GB300 systems.
The tests covered GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. OpenAI plans a limited internal deployment by the end of 2026. Outside buyers cannot order the chip.
The result is public, but the run was collaborative
The benchmark framework, InferenceX, is public. That helps because workloads and scoring can be inspected. The execution context still matters. SemiAnalysis says its engineers ran the tests with OpenAI engineers on machines in OpenAI’s lab.
Call that a collaboratively run public benchmark. Do not call it a blind independent test. The ownership label does not erase the result; it tells readers what has and has not been reproduced.
What the reported numbers cover
| Measure | Reported result | Boundary |
|---|---|---|
| Peak-throughput efficiency | 1.5x to 1.9x more work per watt | Selected models and tested configurations |
| End-to-end latency | 1.7x to 3.6x lower | Workload- and operating-point dependent |
| Rated board power | 700 W | Nameplate rating |
| Sustained test power | At or below 550 W | OpenAI’s tested workloads |
| Availability | Limited internal deployment planned | Not an external product |
The competitor systems are GB200 and GB300. They are not NVIDIA’s later Vera Rubin platform. Our Vera Rubin efficiency analysis covers that different generation and should not be used as the baseline for this comparison.
Work per watt is useful only at a named operating point
Inference hardware can trade latency for throughput through batching, speculative techniques, quantization and scheduling. A system may look excellent at maximum batch throughput and less impressive when a chat product needs a fast first token for one user.
- Match model revision and numerical precision.
- Record batch size, prompt length and output length.
- Separate time to first token from generation speed.
- Measure wall power for the full serving system.
- Include host, networking and idle capacity when pricing a cluster.
A 1.9x peak result does not mean every production request costs 47% less. It means the tested system completed 1.9 times as much benchmark work per unit of power at a specified point. Workload mix and utilization decide how much of that advantage survives in production.
The full-stack claim is the larger bet
OpenAI is not presenting Jalapeño as a standalone accelerator. Its full-stack account ties chip design to kernels, compilers, model architecture, networking, datacenters and inference software.
That integration can remove waste that a general platform must tolerate. It also makes the result harder for another operator to copy. If the improvement depends on OpenAI’s compiler, kernels and scheduler, buying a chip with similar arithmetic capacity would not reproduce the serving result.
AI-generated kernels need a narrow label too
OpenAI describes using AI to generate or optimize some kernels. Kernel-level gains can be real and substantial. They should not be generalized into a whole-model speedup unless the end-to-end test shows it.
The clean reporting chain is:
- Name the operator or kernel that changed.
- Report its local benchmark and precision.
- Measure the full model at a fixed quality target.
- Measure the serving system at realistic load.
- Price the accepted output, not the isolated kernel.
This is the same discipline we apply to model leaderboards. A component improvement matters after it survives the system around it.
What buyers can do now
Jalapeño is not for sale, so there is no buyer decision yet. Infrastructure teams can still use the disclosure to improve their own evaluations.
- Require benchmark-ownership and operator labels.
- Compare at both latency and throughput targets.
- Use wall power and rack constraints, not accelerator TDP alone.
- Keep model quality constant when comparing quantization.
- Ask which gains depend on proprietary software or model changes.
The power framing also connects to our local AI agent economics guide: hardware efficiency only creates savings when the system stays utilized and completes accepted work.
Translate the benchmark into a rack decision
An operator should model three demand points: quiet traffic, the normal production band and a peak that still meets the latency target. For each point, record useful tokens or completed requests, wall power, rejected output and spare capacity. Peak throughput alone can hide an expensive idle fleet or a latency cliff.
The comparison also needs capital, networking, cooling, software labor and service life. A chip that completes more work per watt can still have a higher cost per accepted request if utilization is low or the surrounding stack is expensive to operate. Jalapeño’s internal status means those purchase and support terms are not public yet.
- Hold model quality and numerical format constant.
- Compare at the same latency service level.
- Measure the entire rack rather than board power alone.
- Publish both operator-owned and independently repeated results.
My verdict: promising result, incomplete market test
Jalapeño appears technically credible and strategically important. The public benchmark and SemiAnalysis participation provide more evidence than a vendor slide alone.
The next proof is operational: broader workloads, reproducible configurations, sustained deployment data and a cost model. Until then, treat 1.9x as a strong tested result with a clear ownership label, not a universal chip ranking.
Read the source record
- Read OpenAI’s Jalapeño first-results post.
- Review OpenAI’s full-stack infrastructure explanation.
- Compare SemiAnalysis’s benchmark account and operator disclosure.
Checked August 29, 2026. Performance, power and deployment claims are OpenAI-reported. SemiAnalysis supplies the benchmark-operator context. Musthave.ai has not independently reproduced the hardware tests.
