For Sam Altman, head of OpenAI, computers mobilize a resource that the artificial intelligence industry severely lacks: RAM. During the podcast Shameless pintThe executive said it would be more efficient globally to concentrate RAM in data centers where it could be used by more people and services. A position that comes at a time when demand from AI players is accentuating tensions in the memory components market.
“It’s pretty terrible” have unused RAM on your PC, says Sam Altman
Sam Altman was responding to discussions about local inference, which is running AI models directly on your computer. Some see it as a way to reduce dependence on data centers, particularly thanks to the availability of open source models. The director of OpenAI questions this vision:
Many people want to make inferences locally. They think that the problem is already solved because we will have open source models that we can run on our computers and therefore we will no longer need all this infrastructure in the data centers. From a global allocation perspective, it’s actually pretty terrible that everyone keeps some of the world’s limited supply of DRAM in their computers, even though it’s only used 5% of the time. If supply is also limited, cloud memory usage should be maximized.
Sam Altman’s argument is set in a context of high memory demand for artificial intelligence infrastructures. Large cloud operators and data center projects seek to secure their supply for several years. Samsung and SK Hynix, two of the main memory manufacturers, sign long-term agreements with players in the sector. This pressure contributes to tightening the market and making certain components for consumer devices more expensive.
However, DRAM is not just for running AI models. It is essential for PC, smartphones and many other devices. Therefore, the demand of data centers comes into direct competition with that of the consumer market, although the needs and types of memory are not always identical.
A vision that favors the cloud model
This position is not neutral for OpenAI. The company operates models, including ChatGPT, which has a cloud-based execution and therefore servers in data centers. A generalization of local inference could reduce the use of cloud services and subscriptions for certain uses. Sam Altman therefore advocates a logic of resource optimization that also corresponds to the needs of your company. This is not enough to invalidate his technical argument, but keep in mind that he is speaking as a director of a major player in cloud AI.
The argument also deserves to be qualified. A computer does not dedicate its RAM only to AI: it is constantly used for the system, applications, games, and other tasks. And some users can run local models extensively. The 5% figure Altman presents is an illustration of his reasoning, not a proven measure of average RAM usage across all PCs.
Therefore, the debate goes beyond the single question of the amount of RAM available. Local inference provides more control over data, can work offline, and avoids complete dependence on a remote provider. The cloud allows you to pool expensive resources, but it means entrusting processing to a service provider and depending on their prices, availability and capabilities. Sam Altman’s words clearly describe OpenAI’s priority: reserving an increasing share of hardware resources for centralized infrastructures. However, they do not prove that local execution is useless or less relevant for all uses.
