StarCluster User Manual

Contents:

• Configuration File
  • Creating the configuration file
    • Storing the config in an alternate location
    • Loading the config from the web
  • Amazon Web Services Credentials
  • Amazon EC2 Regions
    • Switching Regions via Command Line
  • Amazon S3 Region-Specific Endpoints
  • Using a Proxy Host
  • Amazon EC2 Keypairs
  • Defining Cluster Templates
  • Defining Multiple Cluster Templates
  • Setting the Default Cluster Template
  • Amazon EBS Volumes
  • Amazon Security Group Permissions
  • Defining Plugins
  • Splitting the Config into Multiple Files
    • Loading Configs from the Web
  • Tips for Switching Regions
• Using EBS Volumes for Persistent Storage
  • Create and Format a new EBS Volume
  • Managing EBS Volumes
    • Getting Volume Status
    • Removing Volumes
    • Resizing Volumes
    • Moving Volumes Across Availability Zones
• Launching a Cluster on Amazon EC2
  • Managing Multiple Clusters
  • Logging into the master node
  • Logging into the worker nodes
  • Running X11 (Graphical) Applications on the Cluster
  • Rebooting a Cluster
  • Terminating a Cluster
  • Stopping an EBS-backed Cluster
• Running Remote Commands from Command Line
  • Running X11 (Graphical) Applications
• Copying Data to and from a Cluster
  • Copying Data to a Cluster (put)
  • Copying Data from a Cluster (get)
• Using the Cluster
  • Logging into the master node
  • Logging into a worker node
  • Verify /etc/hosts
  • Verify Passwordless SSH
  • Verify /home is NFS Shared
  • Ensure EBS Volumes are Mounted and NFS shared (OPTIONAL)
  • Verify scratch space
  • Compile and run a “Hello World” OpenMPI program
• Customizing the StarCluster AMI
  • Launching and Customizing an Image Host
    • Launching a New Image Host
    • Using an Existing Instance as an Image Host
  • Creating a New AMI from an Image Host
    • Creating an EBS-Backed AMI
    • Creating an S3-backed (instance-store) AMI
• Listing the Official StarCluster AMIs
• Plugin System
  • Creating a New Plugin
  • Using the Logging System
  • Adding Your Plugin to the Config
  • Using the Development Shell
  • Plugin Development Workflow
• Adding and Removing Nodes
  • Adding Nodes
    • Re-adding a Node
  • Removing Nodes
    • Remove Without Terminating
• Elastic Load Balancer
  • Goals
  • Usage
    • Cluster Size Limits
    • Increasing Cluster Growth Rate
    • Load Balancer Statistics
    • Advanced Configuration
    • Experimental Features
  • How it Works
    • Criteria for Adding a Node
    • Criteria for Removing a Node
    • The 45 Minutes Past the Hour Rule
    • The Process of Adding a Node
    • The Process of Removing a Node
    • Learning More
• StarCluster Shell Completion (Bash/Zsh)
  • Enabling Tab Completion in BASH
  • Enabling Tab Completion in ZSH
  • Using StarCluster Tab Completion