OpenAI has published a detailed account of Habitat, the online storage platform that sits beneath products including ChatGPT, the API, Codex, and internal services. The company says Habitat now handles more than 70 million requests every second, supports products used by over 1 billion people each week across almost 40 geographic regions, and serves more than 500 petabytes of data. The post is the first half of a two-part engineering series about how that system grew under unusually rapid demand.
From a library to a shared platform
Habitat started in mid-2024 as a small Python library that hid database-management details from product teams. The initial implementation connected to Azure Cosmos DB and handled concerns such as schema lookup, routing, authorization, encryption, serialization, and connection pooling. As more teams adopted the library, those shared behaviors became a way to give different OpenAI products a common storage interface without asking every product engineer to become a database specialist.
That client-side design became harder to operate as the product surface expanded. OpenAI describes a regional-failover change that required new routing logic, feature-flag coordination, shadowing, and rollouts across dozens of services. A rollback by one team could reintroduce an older client at exactly the point where the wider system was trying to avoid an outage. Moving Habitat into its own service reduced that operational fan-out: the company could deploy improvements once, observe them centrally, and apply access-control and audit policies at a single boundary.
Reliability work in Python
OpenAI kept the first production service in Python even though a high-throughput network service adds more CPU and memory overhead than a local library. The short-term goal was to unblock product developers and establish a stable platform, while the team learned where the latency budget was going. Because one user action can trigger hundreds of data calls, the slowest call in the chain can dominate the experience. The engineers therefore tracked asyncio scheduling delay alongside ordinary CPU, memory, network, and disk metrics.
The post describes several small operational changes with outsized effects. Large feature-flag configurations were being parsed at the same time on multiple workers, so the team reduced the configuration size, lengthened refresh intervals, and added jitter. Connection reuse created another trap: an aiohttp pool using last-in-first-out behavior could send new work back toward already-slow processes, creating a metastable failure loop. Switching to first-in-first-out reuse improved balance; OpenAI says the service now relies mostly on Istio and Envoy for pooling and load-aware routing.
The same concern shaped how Habitat talks to downstream systems. Envoy provides connection fan-in, HTTP/2 multiplexing, rate limits, and circuit breakers, reducing the chance that a large fleet of workers overwhelms a dependency during deploys, connection churn, or a leak. These are not model capabilities, but they are part of the infrastructure needed to make AI products feel dependable at global scale.
A deliberately narrow data plane
Habitat avoids letting clients construct arbitrary SQL queries that might trigger unbounded scans or expensive joins. Instead, it exposes a constrained NoSQL interface built around predictable, constant-work requests. That choice makes complex data access less convenient for individual teams, but it also makes cost and failure modes easier to reason about. OpenAI uses change-data capture to stream updates into isolated Rockset instances when teams need richer offline views, keeping analytical or search-heavy workloads away from the online path.
Codex helps finish the rewrite
Once Habitat had matured, OpenAI revisited the language trade-off. The company says that in Q2 2026, two engineers working with Codex and GPT-5.5 rewrote the service in Rust. The Rust version now handles 95% of production requests, and OpenAI reports 6x better CPU efficiency and 15x better memory efficiency than the Python implementation. The company plans to deprecate Python in the coming weeks.
Newsroom analysis
The important signal is not simply that OpenAI uses Rust or a particular database. It is that the company treats the storage boundary as a product-wide control plane: narrow interfaces limit blast radius, central services make policy enforceable, and telemetry turns tail latency into an engineering target rather than a vague user complaint. The Codex-assisted rewrite also shows where coding agents can have immediate leverage—compressing a large migration for a small team—without removing the need for humans to choose the architecture, define the safety boundary, and validate production behavior.
The numbers in this article are OpenAI’s own operational measurements, so they should be read as a first-party engineering account rather than an independently audited benchmark. The promised second installment is likely to fill in the missing storage-layer details, including how Habitat uses Azure Cosmos DB to serve the reported data volume and request rate.