StarCluster

StarCluster is an open source cluster-computing toolkit for Amazon’s Elastic Compute Cloud (EC2) released under the LGPL license.

StarCluster has been designed to automate and simplify the process of building, configuring, and managing clusters of virtual machines on Amazon’s EC2 cloud. StarCluster allows anyone to easily create a cluster computing environment in the cloud suited for distributed and parallel computing applications and systems.

Quick-Start Screencast

Getting started with StarCluster:

StarCluster AMIs

StarCluster comes with publically available AMIs on EC2 that include a minimal software stack for distributed and parallel computing. Currently, the AMIs are based on Ubuntu 12.04 and come in both 32bit and 64bit flavors.

Latest Stable AMI IDs (us-east-1):

• ami-7c5c3915 (i386)
• ami-765b3e1f (x86_64)
• ami-52a0c53b (HVM+GPU)

Use the following command to see the available AMIs in regions other than us-east-1:

$ starcluster --region sa-east-1 listpublic

The AMIs include the following software:

• OpenMPI - Library used for writing/running parallel applications
• Hadoop - Framework for writing massively distributed map/reduce tasks
• Open Grid Scheduler - Queueing system for scheduling jobs on the cluster and handling load balancing.
• NFS - Network File System for sharing folders across the cluster.
• ATLAS - Automatically Tuned Linear Algebra Subroutines (custom built for larger EC2 instance types)
• Scipy - Scientific algorithms library for Python (compiled against custom Atlas)
• Numpy - Fast array and numerical library for Python (compiled against custom Atlas)
• IPython - An advanced interactive shell for Python.
• and more...