Sealed Server Liquid Cooling™

 

Liquid Cooling for the Entire Server.



Sealed Server Loop

Sealed servers are fully liquid cooled delivering the maximum performance, density and energy savings benefits. Direct liquid cooling is used for the high heat flux components, including CPUs and GPUs. Liquid also cools the air inside the server via an air to liquid heat exchanger. This cooled air circulates within the server keeping the other components cool. Liquid then transports the heat out of servers via a Rack CDU to facilities water.

 

The superior efficiency of directly liquid cooling high heat flux parts allows the use of higher performance components in denser configurations.  As the air circulating within a server does not need to cool the hottest parts of the server, warmer air can maintain the other components at safe operating temperatures. This in turn makes it possible to use higher temperature facilities water, increasing free cooling and saving power.

 

Key Attributes:

  • Liquid removes all server heat from data center, eliminating need for Computer Room Air Conditioners (CRAC).
  • No air is exchanged between server and data center.

Benefits:

  • Increases energy efficiency by eliminating CRAC and increasing utilization of free cooling, resulting in power savings up to 50%.
  • Enables operation in harsh environments by eliminating need for air temperature and quality control within the data center.
  • Increases density per rack and for total data center.
  • Enables maximum performance processor (max TDP) use in high density servers.

While every data center and cluster is different and power costs vary around the globe, Rack CDU is generally priced to achieve a one-year payback period.

 

Gallery:


______________________________________________________________________________________________________________________

 

How Sealed Server Liquid Cooling Works:


Sealed servers are self-contained cooling units that do not exchange air with their surroundings. All of the heat generated within the server is removed via cooling liquid to the exterior environment. The result is a server that is thermally room temperature neutral and can be deployed in warehouse style data centers with no air conditioning as well as in rugged, dusty environments including inside military vehicles and under trackside awnings at Formula 1 type events.

 

Sealed Server utilizes Asetek’s standard direct liquid cooling components to cool high heat flux components, including CPUs and GPUs, and a re-circulating “cold” internal air stream that cools the other components within the server. The re-circulating air stream is cooled via an air-to-liquid heat exchanger that uses “cold” liquid to cool the air before this liquid moves on to cool the high heat flux components. The lower cooling loop in the illustration to the left shows the combination of the air-to-liquid heat exchanger cooling the air circulating within the server chassis and the use of direct liquid cooling components removing heat from the processors. 100% of the heat generated within the server is removed via liquid. Two CPUs are liquid cooled in this illustration; however any combination of CPUs and GPUs can be cooled. Direct liquid cooling of memory is also available for servers with large amounts of memory.

 

The Asetek approach contrasts sharply with competitive products that flood the server with cooling oil or bring all the heat to a single surface of the server with a series of custom made and expensive cold plates that must touch nearly all the components on a motherboard. Sealed servers usually operate with less than an air tight seal, maintaining the same serviceability as non-sealed servers. The use of standard cold plates for high heat flux components enables sealed server to easily accommodate motherboard and component changes.

 

The liquid path used by Sealed Server systems is made up of two separate liquid loops, a low pressure server loop (lower loop in illustration) and a facilities loop (upper loop in illustration). No liquid passes between these loops. Facilities water never enters the servers, instead heat moves between the loops via a liquid-to-liquid heat exchanger in the Rack CDU. Separating the loops at the rack level enables the Asetek server loop to operate at such low pressures that a pressure spike leading to a failure within a server simply cannot occur. Additionally, it tightly limits the amount of liquid that is available for spillage within the rack should the loop be damaged.

 

The server loop runs through each server and connects in parallel to a hot/cold liquid distribution manifold within each rack. The hot/cold distribution manifolds are connected via a liquid-to-liquid heat exchanger and a reservoir. Low power, low pressure pumps are integrated with each CPU cold plate within the server. These are the same pumps that are in service and cooling nearly a million CPUs and GPUs in tower PCs and workstations. The pumps draw cold liquid from the cold manifold, through the air-to-liquid heat exchanger, and then move the liquid through the direct liquid cooling cold plates and into the hot manifold. Each pump is capable of providing sufficient flow for the server with the second pump stopped, providing N-1 redundancy. In addition, the pump motors are hot swappable.

 

Liquid from the hot manifold is directed into the liquid-to-liquid heat exchanger by a circulation pump. The liquid-to-liquid heat exchanger transfers processor heat from the server cooling liquid into facilities water, and server cooling liquid returns to the cold manifold. The reservoir also serves as a bypass flow valve, preventing the circulation pump from building up excess pressure. Each rack is equipped with a monitoring system that provides leak detection and reports liquid temperature, flow, pressure and level. All components of the server liquid loop are 100% helium integrity test by Asetek and factory filled with cooling liquid. Data center operators (and server OEMs) never need to touch the liquid in or refill the server loop.

 

The facilities water loop delivers “cool” water to the liquid-to-liquid heat exchanger in each Rack CDU unit. The requirement for Sealed Server is to keep all of the components inside the server operating within specification. The temperature requirements of the air cooled components within the server determine what temperature liquid must be delivered by facilities water. Power savings are maximized by using air cooled components that are designed to operate within the “allowable” limits of Class A4 of the 2011 ASHRAE Thermal Guidelines, which allows for operation with 45°C “cold” air. Power savings result from Sealed Server to operating without using chilled water in many locations and by the elimination of the power required to operate CRAC units.

 

______________________________________________________________________________________________________________________

 

Calculating the Benefits:


Data Center Density:

 

Simple Example:

  • If 1U server draws 1kW total.

  • Data center is provisioned with CRAC supporting 8kW / rack.

  • Density is limited to 8 servers per rack.

With Sealed Server:
  • All 1000W is removed by liquid.

  • CRAC units removed and CRAC infrastructure is converted to support sealed server.

  • Density can expand to 42 servers per rack, a 425% increase.

Data Center Power Savings:

 

Simple Example:

  • If 1U server draws 500W and its CPUs use 60% of the wattage under load. A fully populated rack draws 21kW

  • CRAC power draw to support 21kW is 7kW per rack. (Cooling PUE = 1.34)
With Sealed Server:

  • 100% of the thermal load is removed by liquid cooling

  • CRAC infrastructure is converted to support sealed server.

  • Liquid cooling uses 3kW per rack

  • $3,460 in net power savings per rack per year (@10 Cents/kWh)
Capital Cost Dynamics:
  • CRAC units and CRAC infrastructure is converted to support sealed server

  • No server heat sinks