Chilled water meets high-density cooling
Chilled water takes on the cooling challenge of high-density blade servers — and Mark Luxford of Eaton-Williams is already working on using chilled water to cool micro-processor chips directly.
Just as the industry might have believed that high-density computing equipment was moving out of the ability of chilled water to cool, Eaton-Williams has tackled the problem — and is already moving on to the next stage.It is common knowledge that conventional methods of cooling computer rooms by drawing hot air from the room, cooling it and then returning it to the space via a raised-floor plenum has a practical limit of around 3 kW per rack of computer equipment. Unfortunately, today’s generations of cabinets house blade servers with such high power densities that a rack of such equipment could easily generate 25 kW. Just imagine eight 3-bar electric fires inside such a cabinet! Various solutions have been devised to remove heat from air that has passed through a cabinet before it is returned to the space — so as not to impose any extra cooling load on equipment cooling the room. New racks can have cooling equipment built into them, and other systems on the market can be retrofitted to existing cabinets so that they can accommodate high-power blade servers. Over the last year or two, systems based on the partial evaporation of liquid carbon dioxide or refrigerants have been developed. Experience
Chilled water, however, has not been regarded as capable of removing such large quantities of heat —until Eaton-Williams started applying its experience in cooling magnetic resonance imagers for medical applications to develop a range of equipment for cooling blade servers that can be incorporated into new cabinets and retrofitted using an existing chilled-water circuit via a CDU (cooling distribution unit). This approach separates the rack cooling from the chilled water loop and maintains the water temperature above dew-point to avoid condensation. Mark Luxford, who heads new technologies at Eaton-Williams tells us that the development of this approach to cooling blade-server cabinets is based on over 10 years’ experience in MRI cooling. He explains that, like blade servers, MRI systems are compact, critical pieces of kit with high power densities requiring the removal of anything from 20 to 50 kW of heat and that Eaton-Williams has produced and supplied cooling equipment to the three main manufacturers of MRIs. In response to concerns about the reliability of water cooling, Mark Luxford explains that MRI equipment is now used during surgery in operating theatres. The first application of a chilled-water solution for cooling blade servers came when IBM approached Eaton-Williams, having failed to get a satisfactory solution from two major specialists in cooling computer-room equipment. The solution devised has now been branded Cooling IT Servercool. The prototype installation for IBM was at Poughkeepsie in New York State. An installation was recently completed in Peterborough involving 200 kW of cooling, of which 50 kW is provided by rear-door heat exchangers on the racks themselves. The range of equipment devised by Eaton-Williams provides solutions for retrofitting and for new cabinets and installations. The key to the approach is a rear-door heat exchanger for fitting to the back of a server cabinet and with the capability of removing up to 35 kW of heat. Six fans draw air through the rack and out through the coil. The fans have variable-speed control for soft start and capacity control. Different thermodynamics
The reason that chilled water can remove such large quantities of heat from server cabinets is, as Mark Luxford explains, that the thermodynamics is quite different from comfort cooling. Room air onto a chilled-water coil might be at around 25°C, with an average water temperature in the coil of around 9°C — a difference of 16 K. In contrast, the temperature of air that has passed through rack full of blade servers could readily have risen by 25 K and come on to the cooling coil at 45 to 50°C. However, to avoid condensation the temperature of the air passing through the coil should not be reduced below its dew point — which implies a water flow temperature to the coil of 13 to 16°C and an average of 16 to 19°C, depending on the relative humidity of the air. This is established by a novel dewpoint control system to maximise cooling whilst eliminating condensation risk. For removing heat from the outgoing air from a blade-server cabinet, there is a temperature difference of around 30 K — compared with just 16 K for comfort cooling. The aim is to remove from the air the heat that has been added by the blade servers and return the air to the room at the same temperature that it entered the front of the cabinet. The Cooling IT Servercool approach can remove 25 kW of heat from a 600 mm wide cabinet and 35 kW from a cabinet 800 mm wide. There are two ways of interfacing to the main chilled-water circuit. One way is via a CDU (cooling distribution unit) module that uses a plate heat exchanger to provide an interface with the low temperatures in the main chilled-water circuit and deliver water to the cabinet heat exchanger above dew point. These modules can be housed in the bottom of standard 19 in rack and deliver a total of 20 kW of cooling to one or two rear-door heat exchangers. Upgrade path
CDU modules provide an easy route to upgrading a small number of racks or installing a few new high-density racks. Up to six CDU modules can be housed in a dedicated rack, complete with water and power distribution, providing the opportunity for progressive upgrading as heat loads increase and the data centre grows whilst installing all the necessary facilities day one. For larger-scale applications, Eaton-Williams has also developed a CDU unit that can deliver up to 150 kW of cooling to up to six rear-door heat exchangers in a standard rack giving a compact footprint. Benefits
Mark Luxford believes that a water cooling solution offers both installation and capital-cost benefits compared to carbon dioxide. Assuming that the existing chilled-water system has spare capacity and that spare tappings can be found, Cooling IT components can be installed using 3/4 in flexible pipes under the raised floor with ISOB couplings. Such pipes can each be up to 30 m long. In comparison, says, Mark Luxford, a carbon-dioxide system needs pipework rated at over 25 bar. Nor is he concerned about water leaks. Not only is the temperature of the chilled water kept above dewpoint, but the air speed of 3 m/s would certainly keep any water away from servers in the cabinets. Mark Luxford is unfazed by having to cope with even higher heat densities. The answer is already known, and that is to use chilled water to cool the chips themselves. Eaton-Williams is already working on this approach and has produced 36 prototypes for evaluation by a global IT company.