The Growing Demand for Data Centers Amid the AI Boom: An Overview of the Challenges and Solutions

As artificial intelligence (AI) continues transforming industries, the demand for robust computing infrastructure has surged. AI's popularity has amplified demand for data centers, a vital hub for housing, processing, and distributing large amounts of data and applications alongside the equipment that powers them.  

New AI models, such as generative AI and large language models (LLMs), such as OpenAI’s ChatGPT, require massive data and power to generate content or responses quickly and efficiently. To power these models, the data center industry is racing to find more power to fuel these hubs as well as expanding the existing number of centers in existence.  

However, building more data centers is not a simple task. With AI's increased use and heightened demands, growing concerns around the data center industry regarding electricity consumption, environmental impacts, and scalability are dampening expectations of an AI-powered future.  

The AI Boom and Its Impact on Data Centers

The ongoing AI boom has placed immense pressure on existing data center infrastructure. AI models require vast computational power to train and deploy. OpenAI, the company behind ChatGPT, which initially sparked the AI craze, uses hundreds of GPUs to power one of its AI models.

In 2020, Microsoft’s massive supercomputer for OpenAI, hosted in the Azure Cloud, used around 10,000 GPUs and over 285,000 CPU cores. AI models double in size every few months, which means this monumental amount has been surpassed four times over. That puts a lot of demand on the data center responsible for hosting the information these supercomputers will process and recall.

Microsoft and OpenAI aren’t the only companies creating new AI models and testing their limits. Enterprise demand rose over 2024 as organizations sought to implement automation and machine learning into their businesses to optimize workflows. This has placed significant pressure on existing data centers and current infrastructure.  

In the coming years, data center capacity and power demand will rapidly increase to support the expansion of artificial intelligence applications. Research conducted by Goldman Sachs estimates that “data center power demand will grow 160% by 2030.”

Similarly, McKinsey & Company reports, “Current trends suggest that global demand for data center capacity could rise at an annual rate of 19 and 22 percent from 2023 to 2030 to reach an annual demand of 171 to 219 gigawatts (GW).”

The article goes on to state, “To keep pace with the current rate of adoption, the power needs of data centers are expected to grow to about three times higher than current capacity by the end of the decade, going from between 3 and 4% of total US power demand today to between 11 and 12% in 2030.

Data centers are expected to support AI workloads with minimal latency, or responses with minimal delay. This area is booming, especially with the growing interest in AI-as-a-service (AIaaS). This cloud-based platform provides access to artificial intelligence tools and capabilities on a subscription or pay-per-use basis. Demand for AIaaS started to rise when it became difficult for companies to obtain their supply of expensive and scarce GPUs.

As a result, data centers need more energy and capacity to support these growing endeavors. However, these emerging demands are straining the existing power infrastructure, environmental sustainability, and space availability.

Key Challenges Facing Data Centers

In a recent article by Quartz, Microsoft described its difficulty building out data centers fast enough to support its many cloud and AI services. Microsoft Chief Financial Officer Amy Hood said the recent 5% stock dip was due to higher demand than available capacity.  

Richard Windsor, Founder of Radio Free Mobile, said, “Microsoft's inability to build data centers fast enough has constrained its guidance for the coming quarter.”

This was further illustrated during a call with Microsoft CEO Satya Nadella. Nadella says that Microsoft is struggling with external constraints due to the high demand for AI training and inference. “[Data centers] don’t get built overnight. Even in Q2, for example, some of the demand issues we have, or rather our ability to fulfill demand, are caused by, in fact, external third-party stuff that we lease moving up. That’s the constraints we have.”

Microsoft isn’t the only one struggling to build new data centers to power the growing use of artificial intelligence applications. According to a report by the Wall Street Journal, the heightened demand for AI has led to diminished capacity for the parts, property, and power required for data centers.  

Data center executives told the WSJ, “The lead time to get custom cooling systems is five times longer than a few years ago. Delivery times for backup generators have gone from as little as a month to as long as two years.”

Likewise, property availability to construct a data center with access to efficient power and data connectivity has severely declined. One company, Hydra Host, tried to locate a property that could provide 15 megawatts of energy necessary to power 10,000 AI chips in early 2024 and is still looking for a location. The greatest challenge to Hydra Host’s search was the inability to find a property with suitable cooling systems or the required transformers. Ordering custom systems to support the prospective data center would take up to a year.

“With what we’re seeing, the fervor to build is probably the greatest since the first dot-com wave,” said Hydra Host Chief Executive Aaron Ginn. He told WSJ that searching for the right parts and space has taken months longer than expected.

In the same article, WSJ states that the 2022 version of ChatGPT required more than 10,000 Nivida GPUs. More recent AI models require more, which puts more significant strain on existing data centers. Real estate firm CBRE said that data center space within the United States has grown by 26%, also contributing to the rising costs for available space.

Jon Lin, the general manager of data center services at Equinix, told the WSJ that it takes work for the industry to scale up fast. “These are not easy problems where you can pivot on a dime and triple capacity very quickly.”

Similarly, the investments required for new data centers are massive. Tech giants have been spending millions, if not billions, to grow this sector immensely after the AI boom. Google only saw its capital expenditure jump 45% compared to a year ago in this regard.  

Components used in data centers, which power AI, have been in extremely short supply since ChatGPT’s debut. Nvidia, the leading supplier of these components, has seen lead times for its products explode in recent months. This has contributed to lead times for items such as backup power generators, which take less than a year to nearly two years.  

This isn’t even touching the energy challenges impacting the data center industry.  

“The data-center market needs power yesterday,” said Clayton Scott, the Chief Commercial Officer at NuScale Power.  

In California’s Silicon Valley, the race to build data centers could result in a boom in gas demand that will be equivalent to the demand for the entire state of California or New York within the next decade. With tech giants and industries pledging to reduce their carbon footprints, data centers must find ways to become more energy-efficient and sustainable.

The growing utilization of power is also starting to impact local communities. The Washington post reports, “extraordinary demand for electricity to power and cool computers inside can drive up the price local utilities pay for energy and require significant improvements to electric grid transmission systems.”

This has caused concerns that tech companies aren’t paying their fair share, and the additional costs are being unfairly pushed onto the consumers. Several groups, including the Sierra Club, an environmental organization that has been highly critical of the semiconductor industry’s toxic past, are filing letters to the government to ensure that tech companies pay what is considered their fair share.

Likewise, many are concerned with the environmental impact from these new data centers will affect sustainability goals and citizens.

Goldman Sachs analysts wrote that “the expected rise of data center carbon dioxide emissions will represent a social cost of $125-140 billion (at present value).”

The Goldman Sachs analysts went on to say, “Conversations with technology companies indicate continued confidence in driving down energy intensity but less confidence in meeting absolute emissions forecasts on account of rising demand.”

According to McKinsey & Company, “Skyrocketing compute and data demands are being further accelerated by gains in computing capabilities alongside reductions in chip efficiency relative to power consumption.”

The article went on to add that “providing the more than 50 gigawatts (GW) of additional data center capacity,” to keep up with growing computing demands would require a $500 billion investment for data center infrastructure alone by the end of this decade. That’s without even touching the mess that is the U.S. power grid industry.  

Aneesh Prabhu and his colleagues at S&P Global, a financial and analytics firm, state, “If the world wants all [AI] workloads to be powered only by sustainable power in 2030 from currently available technologies, it will have to temper its AI initiative.”

Based on earnings reports from TSMC, the biggest benefactor from the AI boom alongside Nvidia, this demand will likely grow in the next several years. Goldman Sachs reports did reveal that the data center industry has worked to increase efficiency in the past, but since 2020, these efficiency gains have dwindled as power consumption has grown.

To keep up with the demands, massive investments must be made in modernizing power grid infrastructure, expanding renewable energy sources, and improving power efficiency within data centers.

Solutions for a Smoother Future

Firstly, regarding data center power needs, countries and companies are turning to renewable solutions to support growing energy demands. In Europe, Goldman Sachs estimates that the EU would require a $1 trillion-plus investment to prepare its grid for AI. This is an immense cost, but Europe has one of the oldest power grid systems.  

The U.S. is experiencing similar issues with its power grid infrastructure. In fact, “Over 70% of U.S. transmission lines are over 25 years old and approaching the end of their 50-80-year lifecycle.”

These challenges contribute to “power grid disruptions, or power outages, can contribute to operational losses, safety risks, increased costs, and data corruption.”

The growing adoption of electric vehicles, rising electrification efforts, and the boom of AI will require heavy investments to help contribute to building out power grids to support the expanding data center industry. Renewable energy and other sustainability efforts are being placed at the forefront of new data center expansion efforts to provide energy and meet carbon emissions goals set forth by the Paris Agreement.  

This includes wind, solar, hydroelectric, and nuclear power. In an ironic twist of fate, Meta had been planning to build a nuclear-powered data center but a rare species of bee discovered at the site upended construction. The Financial Times reported that alongside rare bees, regulatory challenges also contributed to the plant’s cancellation.

However, Meta’s stumbling block doesn’t mean that the future of nuclear-powered data centers has been permanently shelved. Google recently announced its partnership to buy nuclear energy from “small modular reactors to be built by Kairos Power, making it the first tech giant to broker a deal for new nuclear power plants.”

Similarly, Microsoft has signed a 20-year power purchase agreement with Constellation, which would bring part of Three Mile Island back online. There are concerns about bringing Three Mile Island back online as many consider the site to be America’s worst nuclear energy accident. However, the “disaster” was more of a disasterous breakdown of communication than potentially catastrophic atomic destruction.

At the same time, companies are exploring innovations in liquid cooling solutions that can lower the environmental impact and the parts required by these systems. Liquid and free-air cooling can significantly reduce the energy needed to keep data centers at optimal temperatures. This also widens the range of potentially available properties to build new data centers.  

Supermicro President and CEO Charles Liang stated, “We’re driving the future of sustainable AI computing.”  

Giampiero Frisio, President of electrification at Swiss multinational ABB, laid out a plan to prepare for the smooth adoption of expanding data centers without taxing the energy grid and implementing more sustainability.

“I think the best way to act now is to increase the energy efficiency. That’s the best way because there is technology, such as the medium voltage HiPerGuard UPS. You can do it, and you can do it tomorrow morning,” said Frisio.

“The second one is to move on to liquid cooling, there is no doubt. Again, this is in the optic of better energy efficiency. Why? Because of a single rack— the black boxes that look like a wardrobe with all the servers inside—the power density will be four to six times higher than before.”

Frisio concluded, “After that, we are talking about five to 10 years from now, it is the nuclear modular system.”

One last step that can help support the rapid adoption of AI with more scalable data center growth is smaller, localized data centers close to the network or edge. These so-called edge data centers can reduce latency and help alleviate bandwidth constraints. This decentralized approach is particularly valuable for applications that require real-time processing, such as autonomous vehicles or smart cities.

Modular data centers are another scalable solution. These prefabricated, containerized facilities can be rapidly deployed and easily scaled up or down based on demand. They offer flexibility and can be in remote areas where traditional data centers might be impractical to build. Modular designs also allow for more efficient use of space and power, which is pertinent when space is unavailable.

Moving Toward a Sustainable and Electric Future

As AI continues to reshape industries, the role of data centers will only grow in importance. However, this growing demand brings various challenges, including energy consumption and scalability issues. Adopting renewable energy, modular and edge data centers and power grid infrastructure support can achieve a smoother transition.

In the wake of the AI boom, the future of data centers will be defined by their ability to balance performance, sustainability, and security—ultimately serving as the backbone of our increasingly data-driven world. This will require sufficient support and access to the parts supply necessary to establish these critical facilities.  

Sourceability combines a traditional, global sales force of sourcing experts with a global e-commerce platform, Sourcengine, to simplify electronic component procurement. Our experts partner with hundreds of franchised, authorized, and qualified third-party distributors. Through our franchise partnerships, organizations of any size can obtain the parts they need to build sustainable solutions that the data center industry can utilize.  

The world will only see a more significant proliferation of AI as automation's benefits continue to aid consumer and business lives and workflows. To support an AI-integrated future, steps such as establishing a secure and transparent supply chain must be taken now. Sourceability can help you get there.