Evaluating AI Infrastructure: Power & Capacity

This guide walks you through evaluating AI infrastructure investments by focusing on the two most critical constraints: data center power and physical capacity. You will learn how to translate a planned GPU deployment into real-world power, cooling, and space requirements, and how to compare that against what a facility can actually deliver. Before starting, have your target hardware specifications (GPU wattage, server count), facility power agreements, and existing floor plan available.

What You’ll Need

• Target AI hardware specs: GPU model, per-GPU power draw (TDP), and server chassis configuration.
• Number of GPUs or servers planned for deployment.
• Current or prospective data center lease or power purchase agreement (PPA) with total available megawatts (MW).
• Facility floor plan with rack layout, aisle dimensions, and ceiling height.
• Cooling system details: type (air, liquid, direct-to-chip), and rated cooling capacity in kW or tons.
• Utility feed details: incoming voltage, transformer capacity, and redundancy level (N, N+1, 2N).

Step-by-Step

1. List your hardware power requirements. For each GPU model, note the maximum power draw (TDP) in watts. For example, a high-end accelerator might draw 700W. Multiply by the number of GPUs per server (e.g., 8 GPUs per node) to get per-server GPU power. Add server overhead (CPU, memory, fans) — typically 15–25% of GPU power — to get total server power draw.
2. Calculate total IT load. Multiply per-server power by the total number of servers planned. This gives you the IT load in kilowatts (kW). For example, 10 servers each drawing 10 kW = 100 kW. Convert to megawatts (MW) by dividing by 1,000.
3. Add facility overhead. Data centers consume extra power for cooling, lighting, and power distribution losses. The Power Usage Effectiveness (PUE) measures this. A typical efficient facility has a PUE of 1.3–1.5. Multiply your IT load by the PUE to get total facility power demand. For 100 kW IT load with PUE 1.4, total demand = 140 kW.
4. Compare against available power capacity. Check your facility’s utility contract or electrical design. Look for the total connected power in MW and the actual usable power after redundancy. If the facility is N+1 redundant, you may only have 80–90% of the nameplate capacity available for IT. Subtract existing tenant loads if it’s a colocation environment.
5. Assess cooling capacity. Determine the cooling system’s rated capacity in kW. This is usually the heat rejection capability of the CRAC/CRAH units or liquid cooling loops. Your total IT load (before PUE) must be less than the cooling capacity, ideally with a 10–20% safety margin. For liquid-cooled systems, check the coolant flow rate and inlet temperature limits.
6. Evaluate physical space. Calculate the number of racks needed: divide total servers by servers per rack (typically 1–4 for AI due to power density). Then check floor space: each rack footprint is about 2 ft x 3 ft (0.6 m x 0.9 m) plus aisle space. A typical AI rack with high power density may require 30–40 sq ft (2.8–3.7 m²) including service clearance. Multiply by rack count and compare to available white space.
7. Check power density per rack. AI servers often draw 30–50 kW per rack, far above traditional 5–10 kW. Ensure your rack power distribution (busway, PDUs, breakers) can handle the amperage at your voltage (e.g., 208V or 415V). Use the formula: Amps = kW * 1000 / (Voltage * Power Factor). For 40 kW at 415V with PF 0.95, amps = 40,000 / (415 * 0.95) ≈ 101 A per phase.
8. Plan for redundancy and growth. Decide if you need N+1 for cooling and power. This means one extra unit for every critical component. For a 1 MW IT load, N+1 power might require 1.2 MW of UPS capacity. Always reserve at least 15–20% headroom for future expansion or power spikes during training runs.
9. Create a capacity summary sheet. List all inputs and outputs: IT load, facility demand, cooling capacity, rack count, floor space, and power density. This becomes your decision matrix for comparing multiple data center options.
10. Run a sensitivity analysis. Change the GPU power draw by ±10% (due to undervolting or overclocking) and see how it affects total capacity. Also test different PUE values (1.2 to 1.6) to see how much headroom you have.

Common Mistakes

• Ignoring PUE: Only counting IT load, not facility overhead, leads to underestimating power needs by 30–50%.
• Using nameplate vs. actual power: GPUs rarely run at 100% TDP during all workloads, but they can during training. Always plan for peak, not average.
• Overlooking cooling type: Air cooling may not handle racks above 20 kW. Liquid cooling requires different infrastructure (piping, manifolds) that may not exist.
• Forgetting redundancy: If you need N+1, you must reserve capacity for the extra unit, effectively reducing usable power by one unit’s worth.
• Ignoring floor weight: High-density racks can exceed floor load ratings (typically 100–150 lb/sq ft). Check the slab thickness and reinforcement.
• Not accounting for network and storage: AI clusters need high-speed switches and storage arrays that also draw power and take space.

Tips

• Use a simple spreadsheet or calculator to model different scenarios. Change the GPU count and see how quickly you hit limits.
• Talk to the facility’s chief engineer, not just the sales rep. Ask for the actual as-built single-line diagram and cooling schematics.
• Consider prefabricated modular data centers if you need rapid deployment—they come with integrated power and cooling.
• If possible, negotiate for a higher power allocation than you currently need. Adding power later is expensive and slow.
• For liquid cooling, check the water quality and flow rate. Direct-to-chip cooling can handle 100+ kW per rack but requires precise engineering.
• Always include a 10% buffer for unexpected power spikes during job startup or checkpointing.

Troubleshooting

Your calculated power exceeds facility capacity. Options: reduce GPU count, choose a lower-power GPU model, or implement time-slicing to run jobs sequentially. You can also negotiate for a higher utility feed, but that takes months.

Cooling is insufficient. If air cooling can’t handle the heat, consider retrofitting liquid cooling to the racks, or reduce density by spreading servers across more racks. Sometimes raising the room temperature setpoint (within ASHRAE guidelines) helps, but only marginally.

Rack power density exceeds circuit limits. You may need to split the load across multiple circuits or install a higher-amperage busway. Check the breaker ratings and wire gauges. This is a common retrofit challenge.

Floor space is tight. Use high-density racks (more servers per rack) but verify power and cooling. Alternatively, use a hot/cold aisle containment system to improve cooling efficiency, allowing more racks in the same footprint.

Utility provider says no additional power available. This is a hard constraint. You may need to look at a different site or consider on-site generation (solar, fuel cells) but that adds complexity and cost.

PUE is higher than expected in practice. Recalculate with a higher PUE (1.5–1.8) for older facilities. If the gap is large, you may need to upgrade cooling systems or accept lower IT capacity.

Frequently Asked Questions

Q: How do I estimate GPU power draw accurately?
A: Use the TDP from the spec sheet, but also check real-world benchmarks for your specific workload. Training typically runs at 80–95% of TDP, while inference may be lower. Always plan for max TDP.

Q: What is the difference between IT load and facility load?
A: IT load is the power consumed by servers, storage, and network gear. Facility load includes that plus cooling, lighting, and power distribution losses. PUE = facility load / IT load.

Q: How much power does a typical AI rack need?
A: It varies widely. A rack with 8x 700W GPUs plus servers can draw 15–20 kW. With 4x 1000W GPUs, it could be 10–15 kW. High-density liquid-cooled racks can exceed 100 kW.

Q: Can I use my existing data center for AI workloads?
A: Possibly, but you must check power density, cooling capacity, and floor weight. Many legacy facilities are limited to 5–10 kW per rack, which is insufficient for modern AI servers.

Q: What is the typical PUE for a modern data center?
A: New hyperscale facilities can achieve 1.1–1.2, while typical enterprise data centers run 1.3–1.5. Older facilities may be 1.6–2.0. Use 1.4 as a conservative default.

Q: How much floor space do I need for 100 GPUs?
A: Assuming 8 GPUs per server, that’s about 13 servers. At 1 server per rack, you need 13 racks, each taking ~30 sq ft including aisle space, so about 390 sq ft (36 m²). Add space for network and storage.

Related

• Intel HD Audio HDMI Not Working on Windows 10? Fix
• Foldable iPhone Case: Hinge-Safe Picking Guide
• Fix iPhone Making Random Calls Issue