Contributed by paul on from the openbsd-hpc-wtf? dept.
Chris Kuethe (ckuethe@) wrote in about how he and his team (University of Alberta) used OpenBSD in their win in the Cluster Challenge at the Supercomputing '07 Conference in Reno, Nevada. The competition included six teams from different universities and their partners:
Indiana University & Apple,
National Tsing Hua University (Taiwan) & ASUSTek,
Purdue University & HP,
Stony Brook University & Dell,
University of Alberta (Canada) & SGI and
University of Colorado & Aspen Systems.
Below is Chris's account of the competition (and OpenBSD's role in it).
In November 2007 I participated in the Supercomputing '07 Cluster Challenge, along with Gordon Klok (gwk@), Paul Greidanus, Stephan Portillo, Antoine Fillion and Andrew Nisbet. Coaching the team was a professor at the University of Alberta, Dr. Paul Lu, as well as Bob Beck (beck@). The idea of the competition was to build a high-performance cluster in a small (some would say unrealistically low) power budget: 26A at 120V. Once the cluster was built, commonly used scientific applications were run on the clusters. Points were awarded for the successful completion of data sets and penalties were assessed for power overages. I also worked on the DHCP server and handled the security concerns; showing up at a conference with a cluster of linux machines optimized for performance rather than security is not a recipe for high uptime.
Because conference networks are not to be trusted, and to simplify management and because the competition rules forbade incoming connections to the cluster, we hid our SGI Altix XE310 cluster behind a Nexcom EBS1563 running OpenBSD. This machine was also our DNS and DHCP server. Because the cluster itself was built with SystemImager (which uses special DHCP options for configuration), we used the isc-dhcpd port which supports a few more dhcp options than the system dhcpd. Installing the dhcpd port meant that we could have one single dhcp server for our laptops, the compute nodes, and their onboard baseboard management controllers (BMC). We installed the ipmitool port to interact with the BMC cards. IPMI is quite powerful it can query sensors, peek at the operating system and control the hardware but, we just used it to power cycle the nodes.
A part of the competition was visualization. We were given workloads for POP (Parallel Ocean Program), GAMESS (General Atomic and Molecular Electronic Structure System) and POV-Ray(port); it was strongly suggested that we be prepared to display the results of our computation in an aesthetically appealing visual form. The POV-Ray jobs were obvious, each job was a movie. Molden could be used to quickly render molecular geometry as calculated by GAMESS. We also used openbabel to convert GAMESS output to PDB format. Povchem can take PDB files and convert them into POV-ray files, which we then used to produce shiny anti-aliased molecule(s).
It was decided that we'd use an AppleTV to simplify the display of the generated imagery. Single frames were a non-issue, but the AppleTV could be quite picky about what video formats it would accept. For whatever reason, the various linux laptops we'd brought along didn't generate videos that would play correctly, at least not easily. Since we were operating under a deadline, we didn't have time to figure out why 2 or 3 distributions couldn't get a working video transcoding pipeline together. Ffmpeg from the ports tree to save the day! Just one command and a few minutes of transcoding later, we had movies that would play. I quickly realized that I could test whether a movie would play by loading it on to my ipod with gtkpod, which saved me the effort of interrupting a presentation on the AppleTV to see if a movie would play. Unfortunately the AppleTV has a glaring defect - the inability to loop videos. We initially worked around this by using mencoder to generate a long video by copying the shorter videos end to end a few times.
A big thank you goes to everyone involved in keeping ports working well - the ports tree allowed us to install third party software with reasonable assurance that it "does what it says on the tin."
Thank you to Chris Kuethe for his account of the event.
(Comments are closed)