OpenAI says its Habitat storage platform now handles more than 70 million requests per second for products used by over one billion people weekly, revealing how ordinary infrastructure choices—not just bigger models—determine whether AI services remain fast and reliable at global scale. The company published the first part of its Habitat engineering account on September 11, 2026, describing a system that grew from a Python library connected to one database into a shared service spanning almost 40 geographic regions and more than 500 petabytes of data.
Evidence note: researched September 12, 2026, from OpenAI’s engineering disclosure, Microsoft’s Azure Cosmos DB documentation, a USENIX systems paper, and independent adoption reporting from TechCrunch and The Next Web. Toolsfine did not test Habitat or independently verify OpenAI’s internal traffic, storage, staffing, or efficiency measurements; those figures remain vendor-reported.
At a glance
| Question | OpenAI’s reported answer |
|---|---|
| What is Habitat? | A shared online-storage layer used by ChatGPT, Codex, the API, and internal services. |
| How large is it? | More than 70 million requests per second and over 500 petabytes across almost 40 regions. |
| What changed? | It moved from a client library to a centralized service, initially in Python and now mostly in Rust. |
| What backs it? | Azure Cosmos DB, internal Nanobase storage, Valkey caches, blob storage, and change-data-capture systems. |
| What is independently verified? | ChatGPT’s enormous adoption is independently reported; Habitat’s internal metrics are not publicly audited. |
What OpenAI disclosed
In its September 11 engineering post, OpenAI describes Habitat as the layer between product services and multiple storage systems. It handles schema lookup, routing, authorization, encryption, isolation, rate limiting, caching, connection management, and data residency. That shared layer lets product teams request objects and direct edges without managing the underlying database topology.
The scale claims are striking: more than 70 million requests each second, over 500 petabytes, and usage by products reaching more than one billion people weekly. Independent reports establish the direction and order of magnitude, though not Habitat’s exact measurements. TechCrunch reported 900 million weekly ChatGPT users in February from an OpenAI disclosure, while The Next Web cited Sensor Tower in June for an estimated one billion monthly app users. Weekly, monthly, app, and cross-product figures are different measures and should not be treated as interchangeable.
Why OpenAI centralized the storage layer
Habitat began in mid-2024 as a Python library talking to Azure Cosmos DB. That was fast to build, but every routing change or bug fix eventually required coordinated client deployments across many services. OpenAI says a rollback by one team could restore old behavior and undermine a carefully staged infrastructure change.
Turning Habitat into a service created one control point for deployments, observability, access policies, audit logging, and storage protection. The tradeoff was overhead: network hops, more compute, and new failure modes. OpenAI consciously kept Python while stabilizing the interface, accepting temporary inefficiency instead of attempting an early full rewrite.
The practical engineering lessons
The most transferable lesson is to optimize tail behavior, not only averages. OpenAI found that CPU-heavy work could delay Python asyncio coroutines even after a database response was ready. It monitored event-loop scheduling delay directly and spread work across many low-concurrency processes.
A second lesson came from connection reuse. OpenAI says aiohttp’s last-in, first-out pool could send fresh work back to recently slow processes, reinforcing overload after a burst. Switching to first-in, first-out reuse broke that feedback loop; Envoy and HTTP/2 multiplexing later reduced connection pressure on downstream systems.
Third, Habitat deliberately exposes a constrained NoSQL-style API rather than arbitrary SQL. Predictable point operations are easier to partition, rate-limit, and cost than unbounded joins. Microsoft’s Cosmos DB documentation similarly explains that point reads consume fewer request units than queries and that complexity, indexing, item size, consistency, and region count affect capacity. The approach also echoes Facebook’s TAO design, documented in a USENIX paper, which uses a limited object-and-association interface to serve a globally distributed social graph.
Why the Python-to-Rust rewrite matters
OpenAI says Python served more than 20 million requests per second at Habitat’s peak. In the second quarter of 2026, two engineers, assisted by Codex and GPT-5.5, rewrote the service in Rust. The Rust version now handles 95% of production requests and is reportedly six times more CPU-efficient and 15 times more memory-efficient, with lower average and tail latency.
Those are internal measurements without a public methodology, hardware profile, workload distribution, or independent benchmark. They demonstrate OpenAI’s reported outcome, not a universal rule that Rust will deliver the same gains. The sequencing is more useful than the headline multiplier: validate the service boundary first, measure bottlenecks under real traffic, and only then replace the implementation behind a stable interface.
Limits and unanswered questions
This is part one of a planned two-part account, so important details remain absent. OpenAI has not yet explained its full multi-tenant isolation strategy, read-performance layers, regional data-placement rules, incident history, or the cost structure behind 500 petabytes. Microsoft documents that Cosmos DB replicates data across configured regions, but that does not reveal OpenAI’s effective configurations or consistency choices.
Centralization also concentrates responsibility. A shared service can enforce stronger policy and observability, but a defect can affect many products at once. Teams copying this architecture should pair the convenience of one gateway with strict isolation, gradual rollouts, circuit breakers, per-tenant quotas, rollback testing, and independent recovery paths.
Practical takeaways
- Measure scheduling delay: CPU utilization alone can miss overloaded event loops and slow tail requests.
- Inspect feedback loops: connection pools and retry policies can repeatedly select unhealthy capacity.
- Constrain hot-path APIs: make expensive fan-out and analytical queries explicit rather than easy defaults.
- Centralize carefully: shared authorization and routing improve consistency but increase blast-radius risk.
- Rewrite after learning: stabilize semantics and collect production evidence before changing languages.
- Label vendor metrics: internal benchmarks are useful evidence, but not independently reproducible facts.
Bottom line
Habitat shows that serving AI at consumer scale is as much a distributed-systems problem as a model problem. OpenAI’s account offers unusually concrete lessons about event-loop delay, connection-pool behavior, constrained APIs, staged centralization, and language migration. The numbers should remain attributed to OpenAI, but the broader lesson is credible: reliable AI products depend on disciplined storage interfaces, observability, isolation, and carefully sequenced engineering—not a single dramatic rewrite.
Sources
- OpenAI: Rapidly scaling online storage to serve over 1 billion ChatGPT users — September 11, 2026.
- TechCrunch: ChatGPT reaches 900M weekly active users — February 27, 2026.
- The Next Web: ChatGPT hits 1 billion monthly users — June 3, 2026.
- Microsoft Learn: Request Units in Azure Cosmos DB — updated April 27, 2026.
- Microsoft Learn: Distribute data globally with Azure Cosmos DB — updated April 27, 2026.
- USENIX: TAO—Facebook’s Distributed Data Store for the Social Graph — June 2013.