Supercomputer

Purpose

I have a bunch of older machines. There's nothing wrong with them, and they're not really all that old. I have a need for a multi-CPU system for some development work and for running Virtual Machines. Cluster computing and parallel processing have always interested me, so why not build a supercomputer in my basement?

Choosing A Cluster Platform

There are quite a few open source cluster platforms. My requirements include:

• Must be free (as in no monetary cost), preferably Open Source
• Must not require any special compilers, procedures, or programming languages - I want to run programs and have them execute somewhere in the cloud automatically, and transparently.
• Must support a graphical environment - I want to have a multi-monitor setup (starting with 2, maybe up to 6 at a later date) to fit all the programs I'm running in parallel and to impress the less technically savvy.
• Preferably Linux based - since I know it, and it is no monetary cost
• Preferably supports running virtual machines that can use cores from multiple cluster nodes simultaneously, or at least allows some way to specify that the virtual machine's process should consume all available resources on one cluster node

Kerrighed

Kerrighed

• Latest Release:
• Activity:
• Technology:
• Parallelism Strategy:

OpenSSI

OpenSSI

• Latest Release: August 2006
• Activity:
• Technology:
• Parallelism Strategy:

openMosix

openMosix

• Project terminated on March 1, 2008

LinuxPMI

LinuxPMI

• Continuation of openMosix
• Activity: Work on updating for newer kernels is ongoing
• Technology:
• Parallelism Strategy:

PelicanHPC

PellicanHPC

• A Knoppix-based LiveCD for creating clusters quickly
• Latest Release: September 10, 2009
• Activity: Recent news postings, uses recent kernel (2.6.30)
• Technology:
• Parallelism Strategy:

Chromium

Chromium

• Specific to graphics and rendering

Sun Grid Engine

Grid Engine

OpenNebula

OpenNebula

• A virtual machine cluster management system

Implementation Plan

Stage 1 - Proof of Concept

This stage will use minimal hardware (1 storage server, 2 cluster nodes, 10/100 networking) to demonstrate that the solution can meet the requirements set forward. An installation procedure and power up/down procedures will also be produced during this stage.

Stage 2 - Basic Implementation

This stage will use expanded hardware (maximum number of cluster nodes available - probably about 6, Gigabit networking) to demonstrate the full potential of the cluster. This stage will start with a fresh install, using the procedure created in the Proof-of-concept. The setup created here will be expanded into the final implementation. Procedures for managing processes throughout the cluster will be investigated and developed. A benchmark procedure will be investigated, developed, and demonstrated.

Stage 3 - Expanded Implementation

This stage will build on the implementation from Stage 2 - no new computing hardware will be added. The graphical environment will be installed and tested. Multi-monitor support will be added. Scripts will be created to automatically power up all nodes, to initialize the cluster, to shutdown all nodes, and for other tasks.

Stage 4 - Performance Evaluation

Benchmark tests will be run to test the system's capability. If problems are found, they will be addressed and the tests will be run again. The affected procedures will be updated accordingly.

Journal

November 11, 2009

Started this page, outlined the plans

References

• How to deploy kerrighed nodes massively using DRBL
• Ubuntu Kerrighed Cluster Guide (on edubuntu wiki)