AI servers: what they look like, how they are cooled and what determines their price
A modern AI server is above all a powerful GPU accelerator with large memory, power in the kilowatt range and, increasingly, liquid cooling – air is no longer enough for the densest builds. The price is determined mainly by the class of graphics cards and the amount of their memory (VRAM): from a workstation with one or two cards for smaller models up to entire NVIDIA Blackwell racks for data centres. Below is an overview of how AI servers are put together, how they are cooled and why companies are moving from the cloud to their own hardware.
Summary
- Companies are moving to local AI servers because of growing cloud API bills and to protect their data.
- Smaller open models (Gemma 3, Phi-4, Mistral Small 3) can handle a large share of everyday business tasks on in-house hardware.
- Racks with the latest chips have a heat load of over 100 kW – they cannot be operated without liquid cooling.
- The price of a server is most affected by the type and number of GPUs and the size of their memory.
Why companies are moving to local AI servers
Until recently, companies counted mainly on cloud models. But constant queries from hundreds of employees and AI agents running in the background led to a phenomenon the industry calls “token bill shock”.
In response, there is a growing trend towards sovereign AI and local servers. Instead of sending sensitive data to foreign cloud servers, companies deploy infrastructure on their own network. Optimised smaller language models such as Llama 4 Scout, Mistral Small 3 or Phi-4 have shown that a large share of business tasks can be handled locally without an expensive subscription.
Why air cooling is no longer enough
The main technical problem with today's AI servers is heat. Traditional air cooling – powerful fans and air conditioning – can't cope with the density of these computations.
The latest generation of chips, such as NVIDIA Blackwell (GB200) GPUs or AMD MI-series accelerators, have enormous power draw. The heat load of a single server rack today exceeds 100–130 kW. For perspective: that much heat could warm an entire block of flats in winter.
How AI infrastructure is cooled
- Direct-to-chip liquid cooling – the most sought-after standard today. Copper cold plates are mounted directly on the GPU and processor, and the liquid carries heat away right at the source.
- Immersion cooling – servers are fully submerged in a tank of dielectric fluid. It allows maximum hardware density, but maintenance is more complex.
Thanks to liquid loops, data centres can reduce large external chillers and lower their PUE to values around 1.15–1.2. Cryptocurrency miners deal with the same physics – you will find a comparison in our article Where air cooling ends.
What an AI server consists of and what determines its price
For large IT ecosystems, ready-made systems from Dell or HPE or racks based on the NVIDIA Blackwell NVL72 architecture are becoming the standard. Mid-sized companies and local deployments more often choose custom-built configurations.
| System class | Target models | Typical hardware | Cooling |
|---|---|---|---|
| Local stack (small business) | Smaller models (Gemma 3, Phi-4, Mistral 24B) | Workstation with 1–2 RTX cards (24–32 GB VRAM) or a Mac Studio with 64 GB | Powerful air or a closed-loop water cooler (AIO) |
| Business AI server | Large open models (Llama 4 Scout, 109B) | Multi-GPU server – gaming cards with large VRAM or NVIDIA H100/H200, AMD MI300 | Air for smaller builds, direct-to-chip liquid for data centre GPUs |
| Data centre | Model training, AI agent clusters | Racks based on the NVIDIA Blackwell GB200 architecture | Fully liquid (CDU units, cooling towers) |
The price of a build is determined mainly by:
- GPUs and their memory (VRAM) – the size of model the server can hold in memory is given by the total VRAM of all the cards. This is the biggest item in the budget.
- Processor and system memory – a server CPU with enough PCIe lanes and hundreds of GB of RAM for working with data.
- Power and cooling – a server with four powerful GPUs draws around 2.5–3 kW, a data centre rack tens to over a hundred kW.
- Networking – connecting several servers for training requires a fast network (InfiniBand or 400GbE).
A mid-range example from our range: the AI server with 4× RTX 5090 has 128 GB of VRAM in total, an AMD EPYC processor, 512 GB of RAM and a power draw of around 2,650 W. You will find the current price and configurations on the product page.
What software is used on AI servers
To manage local compute, tools such as vLLM (for busy business systems, with the PagedAttention technique) or Ollama and LM Studio for quick deployment of pilot projects are used.
Where AI infrastructure is heading
The AI server industry is quickly moving away from the “just rent the cloud” model. Performance per watt, data protection and the ability to connect systems via open protocols are becoming decisive.
The companies that succeed will be those that can match the performance of new chips with the real possibilities of power and cooling. Local AI is no longer an expensive toy for enthusiasts and is becoming the basis of companies' independence and efficiency.
What next
- PC Praha AI servers – custom configurations based on the model you want to run.
- Can a Bitcoin ASIC be converted into an AI server? – why it can't, and what from a mining farm can be reused.
- Mining 2.0 – how miners are reorienting towards AI infrastructure.
Frequently asked questions
How much VRAM do I need for a local language model?
Why do AI servers need liquid cooling?
Is a local AI server worth it instead of the cloud?
What software is used for local AI?
How much electricity does an AI server with four GPUs use?
This text was prepared with the help of AI. Facts, figures and procedures were checked by Yegor.
