HPC clusters starting to run Windows, not Linux

Microsoft Windows is emerging as an acceptable operating system for high-performance computing (HPC) clusters in place of Linux, lowering the bar for entry into that space, according to some analysts and major vendors.

In 2006, Microsoft announced the release of Windows Compute Cluster Server (CCS) 2003, marking the company's first attempt at an operating system for HPC applications. Industry observers say that Windows CCS has been well accepted, particularly for small computing clusters.

Vendors have jumped on the Windows CCS bandwagon, including HP, which recently extended a multimillion-dollar investment agreement with Microsoft to drive high-performance computing into the mass market and is selling CCS 2003 as part of its HP Unified Cluster Portfolio, and Fujitsu Computer Products of America Inc., which recently published a best-practices paper for HPC cluster deployment using Microsoft Windows instead of Linux.

In February, IBM also partnered with Microsoft to offer Windows on its HPC clusters and advance itself in the expanding HPC space. Traditionally IBM has offered Linux technology for its HPC cluster offerings, but it wanted to appeal to Windows users as well, said Stuart Alexander, IBM marketing manager for IBM clusters.

"This is something I've been wanting for years, and when Microsoft announced it in June [2006], we already had a strong interest," said Alexander. "Linux has been our bread and butter, but the single largest chunk of growth in the cluster market is the departmental cluster space [clusters priced at $250,000 and under], and Microsoft is targeting that market."

Users with Windows on their workstations can easily integrate an HPC cluster into their Microsoft environment, according to Alexander.

"The workstation world uses Windows, so having it available on clusters gives these users something familiar to work with," Alexander said. "Many users are Microsoft-centric. The Windows offering is perfect for those users who are familiar with Windows and understand it and who don't want to use a different OS."

The research conducts data modeling and molecular analysis of human diseases like cancer and heart disease. To oversimplify, researchers perform molecular classifications of diseases as well as patients' responses to different medications.

Currently the cluster has 16 nodes. The head node is an HP ProLiant DL385 G1 server with 4 GB of RAM and two dual-core 2.4 GHz AMD64 processors. Prior to building the HPC cluster, Rahman's group ran data on separate machines and put the results together manually.

Rahman chose Windows rather than traditional Linux because students are familiar with Microsoft software.

"Students have experience in running similar experiments under a Windows environment," Rahman said. "So using a Windows-based CCS platform was very beneficial. We didn't want to spend time building, configuring and running a Linux-based Beowulf cluster."

Don Becker -- who is the founder and chief technical officer of Scyld Software and the co-founder of the original Beowulf project, which is the cornerstone of commodity-based, high-performance cluster computing -- was surprised to hear such positive feedback about Windows on clusters.

Becker said that, in the HPC space, Windows is not considered a formidable threat to Linux, though Microsoft "certainly ha[s] the money to invest, and could one day be."

Microsoft also has the advantage of application tie-ins. Microsoft controls its widely used applications such as Excel, and Windows CCS ties into that environment, Becker said.

Despite this advantage, the idea of using Windows in the HPC space is outlandish to Becker, who believes that Linux is the ideal operating system for clusters; the OS, he said, "grew up in that space and has good high-performance communications."

Becker believes that with its Unix background, Linux is also designed for unattended remote administration with multiple users and is best for the type of software run on clusters. Microsoft Windows, on the other hand, was designed for a single user to log in to a console.

Another advantage of Linux is the ability to boot off networks easily and run on diskless machines without compromising performance, Becker said. "Microsoft is still struggling to develop that type of technology," he said.

Not surprisingly, where Windows shines is in its ease of use, said IBM's Alexander. "I'm not picking on Linux, but the technical bar for entry into HPC has been lowered by Microsoft. Users can get a cluster up and running much easier with Windows," he said.

As for disadvantages when comparing Linux with Windows on HPC clusters, Alexander said he doesn't see any.

"Performance is comparable; you aren't compromising anything there, and the cost is comparable. The only thing I can think of is that Windows is a new version, so there is that newness of it, and there will be future releases," he said.

Blue Bell, Pa.-based Unisys Corp., a technology services company that provides enterprise support for Linux and other open source technologies, assists users in applying both Microsoft Windows and Linux on HPC clusters.

"Each has its own virtue. Microsoft is new in the [HPC cluster] space. So far it seems to be best for smaller clusters. We haven't seen much in the large-cluster environments yet, but I think [Windows] will do well there. We've worked with Microsoft on this, and Windows scales very well now compared to years ago," said Anthony Gold, vice president and general manager of open source business solutions for Unisys.

While some users consider adopting Microsoft for high-performance computing, there are many Linux devotees, Gold said.

"Windows environments are widely used by clients, so from our perspective, moving users off Windows on to Linux isn't always the best choice," Gold said. "Conversely, there are many people in the Unix space who will never move off Linux; it's like a religion for them, and they won't change."

Adoption of Windows on IBM's HPC clusters was slow at first, because people weren't in the mind-set to use anything other than Linux on clusters, but acceptance is growing, said IBM's Alexander.

"Microsoft had to do some things right to get credibility in the HPC market, and they've done that fairly quickly," Alexander said. "We are still selling lots of Linux clusters; but there is explosive growth of small clusters with multicore processors, and those people want Windows."

As far as cost goes, Windows CCS 2003 is available via Microsoft's volume license channel for about $469 per node. Customers can qualify for discounts depending on volume purchases and licenses. "This is a onetime charge, whereas many Linux-based HPC clusters are priced on a subscription basis and require acquisition and integration of separate job scheduler, message passing interface (MPI) and other software utilities," a Microsoft spokesperson said. Windows CCS, however, "comes with a job scheduler, MPI and utilities."

There are several different Linux cluster distributions, with different architectures and levels of support. The support price varies with the vendor, but it's usually priced per node, Becker said.

Principal analyst at Hayward, Calif.-based research firm Pund-IT Inc. Charles King said vendors are supporting Windows on their clusters now because there is downward percolation of HPC into small and midsized businesses.

"Half a decade ago, when HPC/supercomputing solutions were monolithic, largely custom-built systems, that scenario would have been unthinkable," King said. "But the emergence of cost-effective, high-performance x86-based clustered solutions has fundamentally changed the cost and market dynamics of supercomputing. That shift should work to Microsoft's benefit over time."

Microsoft enters the high-performance computing space as customers are presented with quad-core processors, standards-based, high-speed interconnects and x64 computers (i.e., those with 64-bit x86 architecture) that deliver high performance. The HPC market is experiencing a growth spurt, averaging growth of more than 20% a year over the past four years, with standards-based HPC clusters expanding at even higher rates, according to Earl Joseph, program vice president at IDC, an industry analyst firm.

"The profile of HPC is becoming more commercial, where it was once mostly used in the scientific community," said Unisys' Gold. "Moving into the larger HPC clusters will require Microsoft being accepted as a viable option. [Microsoft] is putting talented people behind their product, and in the small-cluster space we are seeing people eating it up quickly."

Jeff Wierer, senior product manager for Microsoft Windows HPC said, "It seems the HPC definition is evolving and is being associated with a high level of productivity. We've seen sales across the board and been surprised. There has been a lot of progress in the area of personal supercomputing."

For Windows CCS, about 40% of Microsoft's customers are in financial services, some running thousands of Windows clusters. About 45% of those users said they need supercomputers to meet customer demands, according to Wierer.

A title designed for a flame war but when you read the article it becomes pretty clear:

"If you are a completely Microsoft shop with a small cluster, fine," Becker said, "But as your clusters grow, the administrative burden will become overwhelming. You have to learn to use an OS like Linux, because if one or two windows pop up, that's fine. But when 100 pop up on your screen, it is a nightmare."

Another advantage of Linux is the ability to boot off networks easily and run on diskless machines without compromising performance, Becker said. "Microsoft is still struggling to develop that type of technology," he said.

He summed it up, small clusters in an all Windows environment ok, large clusters in a hybrid environment? why? Microsoft is defiantly improving on the high end but there is alot of inertia to overcome..

“Another advantage of Linux is the ability to boot off networks easily and run on diskless machines without compromising performance, Becker said. “Microsoft is still struggling to develop that type of technology,” he said.”

This is important.

I built several Linux HPC clusters back in the 98/99 timeframe - we were a vendor that was involved in the commercialization effort real early.

Because of UNIX diskless workstation capabilities, I was able to build 16 node clusters with only one set of disks - on the master. The 16 nodes had no local disk, dramatically dropping their cost. They all remote booted from the master.

The Microsoft comments about MPI (or PVM) and job control are silly - for Linux they are free. As in free beer and free speech. You get the source code. Good stuff.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.