This is a continuation of our look at future trends in high performance computing. In part 1 we covered the first five of the top ten trends. In this installment we’ll wrap up with the remaining five.
Future Trends in HPC, part 2
Reply
Category Archives: Cooling System
Future trends in HPC, part 1
As we near the end of 2011, we take a moment to reflect on the past year. It’s been a busy year for IT across virtually all verticals, from mobile and search to enterprise servers and cloud computing. When we attended HPC360 a few weeks ago, we had the pleasure to attend a keynote presentation by Addison Snell, CEO of Intersect Research in which he discussed the most important trends in high performance computing (HPC).
HPC is an exciting and growing industry that ICC has been moving into the past couple years. The traditional HPC space revolved around high-end research facilities particularly in science and engineering. However, with each year technological innovations and tailored systems such as our Supermicro GPU Simcluster have brought the realm of HPC closer to reality for many small/medium-sized business and organizations.
In this 2-part series we will look at the top 10 future trends in HPC from Intersect360′s research, coupled with our own analysis and thoughts. No better way for us computer nerds to close the year right? Let’s get started.
Top 10 HPC Trends for 2012 and Beyond
Facebook’s “Green” European Data Center
As power consumption eats away at the environment and company budgets, many organizations constantly look for ways that they can harness renewable energy to cut down costs and improve their image with regard to the environment.
Facebook’s first data center in Europe, based in Lulea, Sweden, is a good example of the possibilities of this: Continue reading
Cooling Servers Under Oil
Micheal Feldman of HPCwire has written an interesting article about a startup company called Green Revolution Cooling (GRC) that has a competitive and innovative product in the server cooling market.
GRC has developed a server rack that lies horizontally on the floor and is filled with an oil-based cooling fluid. Any server that is built according to the standard form factor can slide into the rack and be cooled by the oil bath.
Servers stored in a liquid solution? For this stroke of innovation (actually, the idea is not entirely new, as Feldman notes, but this is its most recent reiteration) GRC has been selected as one of the “Disruptive Technologies of the Year” for SC ’09 and SC ’10.
So how can servers live underwater (actually, under oil, since oil does not conduct electricity) safely? GRC can take any standard server (including blades and GPUs), remove its fans, and seal it with a special coating to make it safe for oil immersion. According to GRC, 250,000+ server hours of testing their racks has not revealed any malfunctions due to the cooling system.
So what are the advantages of this new kind of cooling system? As Feldman states: “The solution is advertised to reduce the cooling energy by 90 percent and cut overall power consumption in the datacenter by up to 45 percent. The pitch is that a single 10kW server rack at 8 cents per kWh will save over $5,000 per year on energy costs alone.”
Although at least one large supercomputing location (the Texas Advanced Computing Center) has begun using these server racks with happy results, GRC is having a difficult time partnering with server manufacturers to cover their servers under warranty if they are used in a GRC rack.
There are various liquid-cooling solutions on the market, but GRC’s is one of the most creative and cost-effective. It is indeed disruptive technology with lots of cost-savings potential. If GRC can overcome the stigma in the market against dunking servers into liquid, its technology can perhaps become a key player in the cooling industry.
Floating data center: a ship full of servers
As Data Center Knowledge reports, a proof-of-concept is currently being developed for a cargo vessel to be used as a dedicated data center. The effort is being led by International Data Security (IDS), a relatively new company, which began the project in 2008 but experienced financial setbacks until this month, when it announced that they are continuing the project. According to the article, Google also began designing a floating data center in 2008, but has not revealed any more information about the current status of this project.
The IDS is commandeering an old training ship from the California Maritime Academy to create a working prototype of its idea. Although the Navy has been deploying data centers on its combat vessels for many years, IDS would be creating the first ship that is specifically dedicated to housing a data center.
The first prototype vessel is not being built as a modular data center, although once the concept is proven, that may be the method of deploying servers throughout future ships. The benefits of having a dedicated data center at sea are that the ocean provides a natural coolant for heated servers, ships are supposedly less prone to natural land disasters like earthquakes and fires, and, as one commentator on the above article noted, a ship can travel and dock between states to take advantage of more favorable tax rates.
Although various practical hurdles still have to be overcome (another commentator noted that materials stored near ports can seep into a docked data center and potentially disrupt the electronics), floating server rooms are just one of many innovative ideas to cool and deploy data centers.
Server rooms get hotter in Zurich
IBM’s new data center offers bold new ideas about how to cool servers. It has long been taken for granted that the best way to control the temperature in a server room was to keep it as cool as possible by blowing out the heat generated by processors inside the servers with chassis fans. IBM’s new data center in Zurich, Switzerland uses liquid cooling to do the same job. In a previous post, I talked about liquid cooling as an alternative to air cooling, but the new data center in Zurich is unique: it’s cooled by hot, not cold, water.
As described in an article on ServerWatch.com, a new idea is emerging in the server industry about how to moderate the heat produced by running servers. Instead of controlling the temperature of the server room itself, it’s much more important to focus on cooling the server components, regardless of how high the temperatures rise in the server room. Recent innovations in air cooling, such as allowing air from outside the building to be cycled through the server room even in locations as hot as New Mexico, show that a hot server room by itself does not dampen the performance of the servers.
But CPUs overheating does. The new Zurich data center, unlike previous liquid-cooled clusters, uses water at 140 degrees F to cool CPUs that run at 185 degrees F. It’s not necessary to chill the water, IBM reasoned, because even warm water is much colder than the temperature of the processors that it heats. So, to save money, the Zurich data center does not chill the water, and moreover pumps it (once it’s been boiled by the servers) to help heat nearby homes.
According to EnterpriseITPlanet.com, “The combined carbon reduction from using less electricity to cool the servers to the recaptured heat for heating purposes is a whopping 85 percent.” As equally important, IBM saved about 40% on their energy spending by switching to hot-water cooling of their servers.
The bottom line, it seems to me, is summed up by Jed Scaramella, senior research analyst for IDC’s Enterprise Platforms and Datacenter Trends as quoted in the ServerWatch article, when he said, “if you walk into the room and the room is cold, that doesn’t tell you much [about how well the servers are cooled]“.
Performance improvements and energy efficiency for HPC data centers
Scientific Computing magazine’s latest Q&A article addresses a topic very important to most professionals who own or operate a data center: how to improve performance and save money on energy costs.
The magazine asked seven industry leaders in the HPC industry for their opinions about this subject. Their answers are very telling and reveal the state of data center technology today and the directions it will likely take in the near future.
Reducing the cost of power consumption
Perhaps the greatest expenditure facing data center operators today is in the electric bill they pay to keep their servers running. While HPC (high-performance computing) servers have become relatively cheaper in recent years with such innovations as GPUs (graphics processing unit), the price of powering ever-expanding data centers has only grown.
There are several bottlenecks that are contributing to these higher costs. The most common problem mentioned by the industry experts in the Q&A article is how to reduce the cost of the cooling systems in server rooms. As Bob Masson of Convey Computer noted, “Every watt required to power a server nominally requires a watt to cool it.”
For this reason, VP of HPC Consulting at NAG Andrew Jones recommends that data center operators make a long-run analysis before they buy new equipment. The price of a server isn’t just what you pay on the day you buy it. The cost also includes what it will take to power that server and cool it over time (and, if you’re buying dozens or hundreds of servers, this begins to add up significantly). That is the true cost of a server, and from this perspective, power and cooling specs become very important.
This kind of an analysis is not only useful before buying a server but also for evaluating an existing data center. Old equipment, as Blake Gonzalez of Dell HPC describes, can be so expensive to run due to power inefficiencies that some companies can save money by buying entirely new servers altogether. But before any such decisions can be made, Gonzalez notes, one has to be able to measure current power consumption accurately and know how to estimate the long-run cost of replacement servers before buying them.
Servers and liquid cooling
If you’ve ever been in a hot server room where noisy fans were whistling from all corners of the server racks, then perhaps you may have wondered to yourself, Isn’t there a better way to cool this place?
As it turns out, there are different ways to cool servers. ServerWatch.com published an article about these alternative methods of temperature control in a server room.
The first method is liquid cooling. This is the “coolest” method, in my opinion, because submarines use the same technique to control the heat from their nuclear reactors. This process works by piping cold water or refrigerant through the server racks or even directly inside the server itself. One system, SprayCool M-series by ISR, squirts coolant right onto the CPU.
An advantage of liquid cooling is that it allows for greater server density in a single room. Water, according to the above article, has “3,500 times the heat transfer capability per unit of volume compared to air”, and if that heat is trapped in liquid form, it’s not going to warm up the air in the server room. Thus, more servers can be packed into a smaller space without melting the ceiling.
Liquid cooling can save on energy costs, as well. IBM’s Data Center Cooling Solution controls the temperature in large-scale clusters by cooling the heated water during the night, when electricity is cheaper. Some companies even offer liquid cooled hard drives that cut down on the sound created by their spinning parts (you can also replace HDDs with SSDs to achieve the same effect).
The other alternative method of cooling servers that the article described was not as “cool”, in my humble opinion. Some data centers are cooled by letting in outside air into the server room. This process is monitored automatically and optimizes the intermingling of inside (hot) and outside (cold) air to keep the servers at the right temperature.
Although we at ICC do not sell any liquid-cooled servers ourselves, I couldn’t resist writing about them. They are a niche technology and probably wouldn’t be the right choice for many server rooms (fans, after all, are much more economical). But they’re so darn cool!