Introduction
This tutorial will guide you through
the (very few!) steps needed to:
- Setup your configuration tools (once and for all)
- Decide on a cluster size and build it on Amazon
- Log into the cluster and set up MPI
- Finish your work and shut down the cluster
Conventions
If I start a line with the following characters,
$>, it means that I will be writing some code that
you should enter “as is” in a console or terminal on
your computer. (of course this syntax will be valid
only for Unix-like systems…)
Pre-requisite Requirements
Throughout this tutorial, I assume that the reader
has already created an Amazon Web Services account
and by doing so has obtained all the necessary
credentials (AWS access key ID, AWS secret access
key, keyname, RSA private key, etc…) To create an
account and get all the necessary credentials, visit
:
To get started with Amazon Elastic Cloud Compute, and
in particular getting the Amazon toolkit, follow all
the instructions given here :
Instructions
Setting up your configuration tools
- Download the archive: ec2-cluster-scripts-x.y.zip
from the following web page:
StarHPC Downloads
NOTE: x
and y are two numbers signaling the version of
the scripts.
- Unzip this archive on your computer (for example
in an .ec2/ directory)
$> unzip
ec2-cluster-scripts-x.y.zip
- Move into the created directory
$> cd
ec2-cluster-scripts-x.y/
- Edit the EC2config.py file with your favorite
editor (vi, emacs, gedit, etc…):
$> vi
EC2config.py
- Fill in this file with your personal Amazon Web
Services account information (like the AWS access key
ID, AWS secret access key, AWS user ID which
corresponds to your account number, the keyname,
etc…).
- Save your changes and quit.
Choosing the size of your cluster and building it
- In order to choose the size for your cluster
(number of CPU's, memory size, storage capacity), you
will have to enter the type of instances that you
wish to boot on Amazon EC2 (m1.large, c1.large,
c1.xlarge) and the number of instances. This is
accomplished by selecting the INSTANCE_TYPE you want
and choosing the number of instances to boot by
assigning a number to DEFAULT_CLUSTER_SIZE in
EC2config.py.
- The next step is really trivial, since all you
have to do is to execute the following script (from
the ec2-cluster-scripts-x.y/ directory):
$>
./ec2-start-cluster.py
Logging in and setting up MPI
- Once the cluster is up and running (you can check
that by refreshing ElasticFox if you installed that
extension into Firefox, or using $>
ec2-describe-instances. This command line option
requires the Amazon toolkit.), execute the following
script:
$>
./ec2-mpi-config.py
-
- Wait until the script is finished, and then
execute this script:
$>
./ec2-login-master.py
- Now you should be root on the Master node of your
new EC2 cluster! What you want is to become a simple
user, lamuser to be precise. To do this just
type:
$> su -
lamuser
- The final step is to launch lamMPI by
typing:
$> lamboot
mpd.hosts
How to run Quantum Espresso on your EC2 Cluster
- After MPI has been setup, you can run Quantum
Espresso jobs on your EC2 cluster. First make sure
you have prepared your input file correctly. Let us
imagine you named it test.in . The next thing to do
is to issue this command:
$> mpirun
-np <number_of_processors>
<path_to_executable> < test.in >
test.out
-
- The above command will execute
<path_to_executable> (it could be pw.x for
example) running the code in parallel using
<number_of_processors> CPU's on the system
described in your input file test.in and will spit
the output to the file test.out.
Shutting down your cluster
- When you are finished with your work on your EC2
cluster, log out from the Master node by
typing:
$>
logout
-
- Then on your Desktop computer (from which you
logged onto the EC2 cluster), type:
$>
./ec2-stop-cluster.py
And you're done! Now you can use the cluster as you
please (like running the Quantum Espresso package in
parallel!)
