New StarCluster Ubuntu 11.10 EBS AMIs available including a new HVM AMI for use with CC/CC2/CG instance types! Includes OpenGridScheduler (to replace SGE), Condor, Hadoop/Dumbo, MPICH2, OpenMPI, IPython 0.12 (parallel+notebook), and more. Run dpkg -l on the AMIs for the full list of packages.
The new StarCluster Ubuntu 11.10 EBS HVM AMI additionally comes with NVIDIA driver, CUDA Toolkit/SDK, PyCuda, PyOpenCL, and Magma for GPU computing.
These new AMIs are available in all AWS regions. The new us-east-1 11.10 EBS AMIs are now the defaults in the StarCluster config template. Use the listpublic command to obtain a list of available StarCluster AMIs in other regions:
$ starcluster -r sa-east-1 listpublic
Note
The HVM AMI is currently only available in the us-east-1 region until other regions support HVM.
Added support for new CC2 instance types
Added support for specifying a proxy host to use when connecting to AWS:
[aws info]
aws_proxy = your.proxy.com
aws_proxy_port = 8080
aws_proxy_user = yourproxyuser
aws_proxy_pass = yourproxypass
See Using a Proxy Host for more details
Updated IPCluster plugin to support IPython 0.12 with parallel and notebook support (SSL+password auth). See IPython Cluster Plugin for more details
Keypairs are now validated against EC2 fingerprint which notifies users ahead of time if the file they specified in KEY_LOCATION doesn’t match the EC2 keypair specified
Support for Windows (tested on 32bit Windows 7). See Installing on Windows for more details
| release-date: | 11-05-2011 |
|---|
Support for splitting the config into an arbitrary set of files:
[global]
include=~/.starcluster/awscreds, ~/.starcluster/myconf
See Splitting the Config into Multiple Files for more details
createvolume: support naming/tagging newly created volumes:
$ starcluster createvolume --name mynewvol 30 us-east-1d
See Create and Format a new EBS Volume for more details
listvolumes: add support for filtering by tags:
$ starcluster listvolumes --name mynewvol
$ starcluster listvolumes --tag mykey=myvalue
See Managing EBS Volumes with StarCluster for more details
sshmaster, sshnode, sshinstance: support for running remote commands from command line:
$ starcluster sshmaster mycluster 'cat /etc/fstab'
$ starcluster sshnode mycluster node001 'cat /etc/fstab'
$ starcluster sshinstance i-99999999 'cat /etc/hosts'
See Running Remote Commands on a Cluster from Command Line for more details
The following bugs were fixed in this release:
spothistory command
start command
terminate command
listkeypairs command
| release-date: | 10-17-2011 |
|---|
Note
Before upgrading please be aware that rc1 does not support clusters created with previous versions and will print a warning along with instructions on how to proceed if it finds old clusters. With that said, it’s best not to upgrade until you’ve terminated any currently running clusters created with an old version. Also, the @sc-masters group is no longer needed and can be removed after upgrading by running “starcluster terminate masters”
You can find the new docs for 0.92rc1 here:
http://web.mit.edu/stardev/cluster/docs/0.92rc1/
NOTE: these docs are still very much a work in progress. If anyone is interested in helping to improve the docs, please fork the project on github and submit pull requests. Here’s a guide to get you started:
http://web.mit.edu/stardev/cluster/docs/0.92rc1/contribute.html
Please upgrade at your leisure, test the new release candidate, and submit any bug reports preferably on StarCluster’s github issue tracker or this mailing list.
Cluster Compute/GPU Instances- Added support for the new Cluster Compute/GPU instance types. Thanks to Fred Rotbart for his contributions
EBS-Backed Clusters- Added support for starting/stopping EBS-backed clusters on EC2. The stop command now stops (instead of terminate) an EBS-backed cluster and terminates an S3-backed cluster. Added a terminate command that terminates any cluster
Improved performance - using workerpool library (http://pypi.python.org/pypi/StarCluster) to configure nodes concurrently
New ebsimage and s3image commands for easily creating new EBS-backed AMIs. Each command supports creating a new AMI from S3 and EBS-backed image hosts respectively. (NOTE: createimage has been renamed to s3image. you can still call createimage but this will go away in future releases)
Add/Remove Nodes - added new addnode and removenode commands for adding/removing nodes to a cluster and removing existing nodes from a cluster
Restart command - Added new restart command that reboots the cluster and reconfigures the cluster
Create Keypairs - Added ability to add/list/remove keypairs
Elastic Load Balancing - Support for shrinking/expanding clusters based on Sun Grid Engine queue statistics. This allow the user to start a single-node cluster (or larger) and scale the number of instances needed to meet the current SGE queue load. For example, a single-node cluster can be launched and as the queue load increases new EC2 instances are launched, added to the cluster, used for computation, and then removed when they’re idle. This minimizes the cost of using EC2 for an unknown and on-demand workload. Thanks to Rajat Banerjee (rqbanerjee)
Security Group Permissions - Added ability to specify permission settings to be applied automatically to a cluster’s security group after it’s been started
Multiple Instance Types - Added support for specifying instance types on a per-node basis. Thanks to Dan Yamins for his contributions
Unpartitioned Volumes- StarCluster now supports both partitioned and unpartitioned EBS volumes
New Plugin Hooks - Plugins can now play a part when adding or removing a node as well as when restarting or shutting down the entire cluster by implementing the on_remove_node, on_add_node, on_shutdown, and on_reboot methods respectively
Added a runplugin command and fixed the run_plugin() method in the development shell. You can now run a plugin in the dev shell like so:
~[1]> cm.run_plugin('myplugin', 'mycluster')
Added support for easily switching regions (without config changes) using a global -z option. For example:
$ starcluster -r eu-west-1 listclusters
StarCluster - (http://web.mit.edu/starcluster) (v. 0.92rc1)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu
>>> No clusters found...
Added new resizevolume command for easily resizing existing EBS volumes
Added listregions/listzones commands