An AI answer appears on your screen in seconds. Behind it, a model may be running across expensive chips inside a building with its own cooling plant, backup power and high-speed connections. The answer looks like software. Delivering it is also a physical job.
An AI data center houses the computing equipment used to train models and run them for users. Some buildings mainly serve AI workloads; others combine them with ordinary cloud services. Understanding what happens inside helps explain why chip announcements, electricity contracts and construction delays keep appearing in the same business stories.
The basics apply around the world. The cost of land, power, financing, labor and regulatory approval changes by location. There is no universal price tag for an AI data center, and a proposed campus is not necessarily an operating facility.
What makes an AI data center different?
Ordinary data centers already run websites, store records and support business software. AI can demand much denser computing, more memory and faster communication between machines. Large training jobs can require many processors to work together instead of handling unrelated customers independently.
The equipment sits in racks. A server contains processors, memory and connections to other equipment. Central processing units, or CPUs, handle general computing tasks. Graphics processing units, or GPUs, can perform many suitable calculations in parallel, making them useful for training and running many AI models.
GPUs are not the only option. Companies also use specialized accelerators, including chips designed for particular AI workloads. The useful question is whether the whole system can run the intended model efficiently. A powerful processor can still spend time waiting for memory, data or another machine.
Model weights, the numerical settings learned during training, must be available to the processors doing the work. Memory capacity affects which models fit and how many requests a system can handle together. Memory bandwidth affects how quickly data moves between memory and computing units.
This makes the chip only one part of the purchase. Servers need suitable memory, cables, switches, software and support equipment. Our guide to semiconductors explains the chip-making process; the data center is where those manufactured parts become a working service.
Training and inference spend computing power differently
Training adjusts a model's weights using data and an optimization process. A large job can run for an extended period, with processors repeatedly exchanging information. If communication becomes a bottleneck, adding more chips does not necessarily deliver a matching increase in useful work.
Inference puts the trained model to use. In a text assistant, the system processes the prompt and generates output. It usually does not retrain the entire model with every conversation. Retrieval tools, search and other connected services can add separate operations around that model call.
The two stages can use similar equipment, but their operating priorities differ. Training focuses on completing learning work efficiently and reliably. An interactive service must also manage response time, queues and changing demand as people arrive. Video generation and other demanding tasks can require very different resources from a short text answer.
Nvidia's technical explanation distinguishes the learning stage from applying what the model has learned. The distinction matters commercially: finishing a training run doesn't remove the ongoing cost of serving customers. Each additional user can create more computing work.
Follow a request through the building
Consider a simplified cloud-hosted text assistant. Your device sends a request over the internet to the service. Software checks access, selects an available system and places work in a queue when capacity is busy. The exact arrangement depends on the provider and product.
The model's computation uses processors and memory. A larger model may need multiple accelerators connected within a server or across servers. Networking carries information between them; storage holds model files and other data that the service needs. The answer returns through the service to your device.
Not every component touches every request in the same way. A model already loaded in memory doesn't need its entire file read from storage for every word. Some requests can reuse earlier computation, while others require more work because of a longer prompt, a longer answer or additional tool calls.
That is why buying the fastest chip does not settle the performance question. A service also needs scheduling, suitable memory and a network that can move information without leaving expensive processors idle. Capacity has value when the complete system can do useful work for customers.
Why the hardware bill gets so large
The initial spending includes the site, building, electrical equipment, cooling systems, racks, computing hardware and networking. Depending on the project, owners may also pay for major utility connections. A cost quoted for a finished building may exclude the servers, so check what an announced budget includes.
Advanced accelerators bring together complex chip design, manufacturing, packaging and memory. Operators then buy them as part of larger systems. Spare equipment, maintenance and software support add costs beyond the advertised purchase price. Individual components and complete rack systems aren't interchangeable price comparisons.
Equipment also wears out or becomes less attractive compared with newer alternatives. Accounting depreciation spreads its cost over an estimated useful life; the cash purchase and depreciation expense occur on different schedules. An operator can therefore report an accounting profit while spending heavily on new equipment.
Owning everything isn't the only business model. A company can rent computing from a cloud provider, lease hardware or use a colocation facility that supplies space, power and cooling. Each arrangement moves costs and responsibilities around. Long contracts can secure capacity while creating payment obligations that remain if demand disappoints.
Power is a running cost and a construction problem
Electricity powers computing and the equipment supporting it. Getting enough reliable electricity to the right place can be harder than arranging a chip delivery. A grid connection may require new substations, transformers or transmission work, with approvals and construction taking time.
The International Energy Agency's April 2026 report estimates global data-center electricity consumption at 485 terawatt-hours in 2025. Its updated projection reaches about 950 TWh in 2030. These totals cover data centers overall, not just AI, and the later figure is a forecast.
Energy and power measure different things. A megawatt describes a rate of electricity use or capacity. A megawatt-hour describes energy used over time. A 100-megawatt site running at that level for an hour consumes 100 megawatt-hours. An announced capacity figure does not establish how much energy the facility actually uses across a year.
Even the capacity label needs checking. A developer might quote computing-equipment load, total facility load or the potential capacity of a future campus. Those figures can describe different projects. Ask how much equipment is installed, how much has power available and how much is actually operating.
The agency also identifies constraints in transformers, generation equipment and grid connections. A signed electricity contract doesn't necessarily remove those physical limits. Equally, a new facility does not automatically cause blackouts: the outcome depends on available supply, network capacity and how the project connects.
Cooling moves the heat somewhere else
Processors turn electrical energy into heat. Cooling systems keep equipment within suitable operating temperatures. Air cooling moves heat through airflow and heat exchangers. Liquid cooling can collect heat close to a processor, including through cold plates, then carry it to another part of the cooling system.
The heat still needs to leave the facility. Designs may use chillers, dry coolers, cooling towers or combinations of equipment. Local weather and water availability affect those choices. A liquid-cooled server does not automatically mean the whole site uses little water or electricity.
Water consumption also depends on where you draw the boundary. Water evaporated by onsite cooling is different from water withdrawn and returned. Electricity generation can have its own water requirements elsewhere. Comparing two headline water figures without checking those definitions can produce a misleading result.
A worked example of the electricity bill
Power usage effectiveness, or PUE, compares total facility energy with the energy used by IT equipment. It captures overhead such as cooling and power distribution. Lower values mean less overhead for the same IT energy, but the measure doesn't tell you whether the computing itself is useful.
Assume a hypothetical facility averages 100 MW of IT load continuously, with a PUE of 1.20. Total facility load is 120 MW. The remaining 20 MW supports the facility rather than the IT equipment. These are teaching assumptions, not an industry average or a particular operator's figures.
At constant load over a 365-day year, that facility uses 1,051,200 megawatt-hours. At an assumed electricity price of $0.08 per kilowatt-hour, the energy charge is $84.096 million. The illustration excludes demand charges, taxes, connection costs and other fees that an actual bill may include.
A small change in efficiency can matter across that much equipment, but a good PUE doesn't make electricity free. It also says nothing by itself about local water stress, carbon emissions, hardware utilization or the quality of the answers being generated. Those need different measurements.
Google's public efficiency page reports how it measures its fleet, including the boundaries used. When comparing operators, look for those boundaries and the period covered. A design target, a particularly cool month and a full-year operating result should not be treated as the same evidence.
Reliability costs money even when nothing goes wrong
A service people depend on needs protection against failures. Batteries and uninterruptible power supplies can bridge short interruptions while backup systems respond. Generators can support longer outages, subject to fuel, operating limits and the design of the facility. These systems need maintenance and testing.
Redundancy can mean installing additional power or cooling equipment beyond what normal operation requires. That increases spending, but reduces dependence on an individual component. The appropriate design depends on the service and its tolerance for interruption. No arrangement guarantees that every possible failure will be avoided.
Staffing, physical security, fire protection and software operations also belong in the cost. Cybersecurity involves protecting accounts, networks and customer environments, alongside managing the equipment itself. A well-cooled server is still vulnerable if someone gains inappropriate access to the service running on it.
The business question is how much useful work gets sold
Owning a full rack is different from keeping it busy with paid work. Utilization matters because buildings, financing and some other costs continue while demand is weak. Idle equipment may draw less power than busy equipment, but that does not erase the investment already made.
Software can improve economics by batching compatible requests, choosing suitable models, reducing unnecessary computation and using available memory efficiently. Those improvements can reduce the cost of a task. They can also encourage more use, so better efficiency doesn't guarantee lower total electricity demand.
An agent that repeatedly calls models and tools can consume more resources than a single short answer. Model size, output length and response-time requirements all affect the service's cost. A subscription price therefore doesn't reveal the provider's cost for every individual customer or request.
For a listed operator, examine revenue, operating cash flow, capital spending and lease or debt commitments together. Check whether booked capacity is supported by binding customer contracts, and whether customers are concentrated among a few buyers. Announced demand and collected cash answer different questions.
Compare what the customer is actually buying
A rented GPU-hour measures access to a processor for a period. A model service might instead charge for input and output tokens, which are pieces of text processed or generated by the model. These are different products. One price buys computing access; the other buys a service that includes the provider's software and operating work.
A cheaper hourly chip rental can still produce a higher cost per completed task if the system runs the model poorly. Compare the same workload, required accuracy and response time, then include memory, storage and network charges where applicable. Otherwise, the cheapest number on the price list may describe the wrong job.
Read the project announcement before trusting the headline
Separate the stages: securing land, obtaining approvals, arranging power, constructing the building, installing equipment and serving customers. Progress at one stage doesn't prove the others are finished. A multiyear investment plan may cover several sites and include both construction and equipment purchases.
Ask who owns the assets, who owes the payments and what happens if the planned launch slips. For communities, ask who funds grid upgrades and how water, emissions and operating conditions are regulated locally. For investors, ask what level of paid usage the expected return requires.
If a developer promises a launch date, look for the utility connection schedule and equipment delivery milestones that make that date possible. A rendering of a finished campus can't supply those details. Data and technical references in this guide were checked on October 27, 2026, UTC; the energy projection remains an estimate of a future outcome.