StarCluster User Manual

Contents:

• StarCluster Configuration File
  • Creating the configuration file
  • Amazon Web Services Credentials
  • Amazon EC2 Regions
  • Amazon S3 Region-Specific Endpoints
  • Amazon EC2 Keypairs
  • Defining Cluster Templates
  • Defining Multiple Cluster Templates
  • Setting the Default Cluster Template
  • Amazon EBS Volumes
  • Amazon Security Group Permissions
  • Defining Plugins
• Using EBS Volumes for Persistent Storage
  • Create and Format a new EBS Volume
  • Managing EBS Volumes with StarCluster
    • Getting Volume Status
    • Removing Volumes
    • Resizing Volumes
    • Moving Volumes Across Availability Zones
• Launching a StarCluster on Amazon EC2
  • Managing Multiple Clusters
  • Logging into the master node
  • Logging into the worker nodes
  • Rebooting a Cluster
  • Shutting Down a Cluster
• Copying Data to and from a Cluster
  • Enable Experimental Features
  • Copying Data to a Cluster (put)
    • Copy a file or directory to the master as a normal user
    • Copy a file or directory to a cluster node
  • Copying Data from a Cluster (get)
    • Copy a file or directory from the master as a normal user
    • Copy a file or directory from a cluster node
• Using the Cluster
  • Logging into the master node
  • Logging into a worker node
  • Verify /etc/hosts
  • Verify Passwordless SSH
  • Verify /home is NFS Shared
  • Ensure EBS Volumes are Mounted and NFS shared (OPTIONAL)
  • Verify scratch space
  • Compile and run a “Hello World” OpenMPI program
• Customizing the StarCluster AMI
  • Launching and Customizing an Image Host
    • Launching a New Image Host
    • Using an Existing Instance as an Image Host
  • Creating a New AMI from an Image Host
    • Creating an EBS-Backed AMI
    • Creating an S3-backed (instance-store) AMI
• Listing All Public StarCluster AMIs
• StarCluster Plugin System
  • Creating a New Plugin
  • Using the Logging System
  • Adding Your Plugin to the Config
  • Using the Development Shell
  • Plugin Development Workflow
• Adding and Removing Nodes from StarCluster
  • Adding Nodes
  • Removing Nodes
• StarCluster Elastic Load Balancer
  • Goals
  • Usage
    • Cluster Size Limits
    • Increasing Cluster Growth Rate
    • Load Balancer Statistics
    • Advanced Configuration
    • Experimental Features
  • How it Works
    • Criteria for Adding a Node
    • Criteria for Removing a Node
    • The 45 Minutes Past the Hour Rule
    • The Process of Adding a Node
    • The Process of Removing a Node
    • Learning More
• Enabling Tab Completion for Bash/Zsh
  • Enabling StarCluster Tab Completion in BASH
  • Enabling StarCluster Tab Completion in ZSH
  • Using StarCluster Tab Completion