Re: [Starcluster] create_image.py vs. createimage

From: Justin Riley <no email>
Date: Thu, 15 Apr 2010 08:38:04 -0400

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Thomas,

Sorry to hear you're having issues with the createimage command.

> I already solved a few problems but now I am at a point where I don't
> know what to do anymore.

Could you describe the few problems you've already solved? I'd like to
understand what caused the very first error you had leading up to this
point.

Thanks,

~Justin

On 04/15/2010 05:54 AM, Thomas Deselaers wrote:
> Hi Justin,
>
> Thanks a lot for your prompt reply.
>
> I just played a bit with the new version to create a new image.

>
> Could you give me a pointer what I should change?
>
> I manually started an instance (64bit image) and modified it a bit.
> Then I tried to run the commands below.
>
> You also find the output of starcluster.
>
>
> Thanks in advance
>
> export EC2_PRIVATE_KEY="/home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa"
> export EC2_CERT=/home/desthoma/work/text/aws/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXxx.pem
>
> starcluster createimage i-73eXXXXX myStarcluster starclusterAMIBucket
>
> StarCluster - (http://web.mit.edu/starcluster)
> Software Tools for Academics and Researchers (STAR)
> Please submit bug reports to starcluster_at_mit.edu
>
>>>> Using private key /home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa (rsa)
> ---------------------------------------------------------------------------
> AuthenticationException Traceback (most recent call last)
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
> in <module>()
> 3 __requires__ = 'StarCluster==0.9999'
> 4 import pkg_resources
> ----> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
> 6
> 7
>
> /home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
> in run_script(self, requires, script_name)
> 487 ns.clear()
> 488 ns['__name__'] = name
> --> 489 self.require(requires)[0].run_script(script_name, ns)
> 490
> 491
>
> /home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
> in run_script(self, script_name, namespace)
> 1212 )
> 1213 script_code = compile(script_text,script_filename,'exec')
> -> 1214 exec script_code in namespace, namespace
> 1215
> 1216 def _has(self, path):
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
> in <module>()
> 4 import pkg_resources
> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
> ----> 6
> 7
> 8
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
> in main()
> 750 sys.exit(0)
> 751 try:
> --> 752 sc.execute(args)
> 753 except exception.BaseException,e:
> 754 log.error(e.msg)
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
> in execute(self, args)
> 359 'bucket': bucket,
> 360 })
> --> 361 icreator = image.EC2ImageCreator(**kwargs)
> 362 icreator.create_image()
> 363
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
> in __init__(self, instance, aws_access_key_id, aws_secret_access_key,
> aws_user_id, bucket, ec2_cert, ec2_private_key, prefix,
> remove_image_files, **kwargs)
> 51 }
> 52 if self.host:
> ---> 53 self.config_dict['arch'] = self._get_arch()
> 54
> 55 _at_print_timing
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
> in _get_arch(self)
> 72
> 73 def _get_arch(self):
> ---> 74 conn = self.host.ssh
> 75 arch = conn.execute(
> 76 'python -c "import platform; print
> platform.architecture()[0]"'
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/node.pyc
> in ssh(self)
> 147 self._ssh = ssh.Connection(self.instance.dns_name,
> 148 username=self.user,
> --> 149 private_key=self.key_location)
> 150 return self._ssh
> 151
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/ssh.pyc
> in __init__(self, host, username, password, private_key,
> private_key_pass, port)
> 77 raise TypeError, "You have not specified a
> password or key."
> 78
> ---> 79 self._transport.connect(username = username, pkey = pkey)
> 80
> 81 def _load_rsa_key(self, private_key, private_key_pass=None):
>
> /var/lib/python-support/python2.5/paramiko/transport.pyc in
> connect(self, hostkey, username, password, pkey)
> 987 else:
> 988 self._log(DEBUG, 'Attempting public-key auth...')
> --> 989 self.auth_publickey(username, pkey)
> 990
> 991 return
>
> /var/lib/python-support/python2.5/paramiko/transport.pyc in
> auth_publickey(self, username, key, event)
> 1214 # caller wants to wait for event themselves
> 1215 return []
> -> 1216 return self.auth_handler.wait_for_response(my_event)
> 1217
> 1218 def auth_interactive(self, username, handler, submethods=''):
>
> /var/lib/python-support/python2.5/paramiko/auth_handler.pyc in
> wait_for_response(self, event)
> 172 if issubclass(e.__class__, PartialAuthentication):
> 173 return e.allowed_types
> --> 174 raise e
> 175 return []
> 176
>
>
>
>
>
> On Wed, Apr 14, 2010 at 21:20, Justin Riley <jtriley_at_mit.edu> wrote:
> Hi Thomas,
>
> The github code's createimage command is the recommended way, however, I
> haven't made a new release out of this code yet. You're welcome to try
> out the latest code; it's very close to ready. The config has been
> enhanced quite a bit past the previous 0.90.1 version so you'll
> certainly want to look at the docs in github:
>
> http://github.com/jtriley/StarCluster/tree/master/docs/sphinx
>
> The *.rst files are the raw text documents. These docs are also
> available on the web here:
>
> http://web.mit.edu/stardev/cluster/docs/
>
> I'm in the process of cleaning up the code in github and finishing up
> some documentation for a new release that supports multiple clusters,
> multiple ebs volumes, cluster 'plugins' that allow users to write their
> own custom setup routines on top of starcluster's defaults, automated
> EBS volume creation (including partition/format), and more.
>
> If you could also report any issues (either bugs or usability issues)
> with the github code to this list I'd appreciate it.
>
> Thanks!
>
> ~Justin
>
>
>
> On 04/14/2010 11:39 AM, Thomas Deselaers wrote:
>>>> Dear all,
>>>>
>>>> I am a new (since today) StarCluster user and I already managed to get
>>>> a few things done (start my "own" cluster, run jobs, etc). The
>>>> documentation is really fine (and I know SGE from the past).
>>>>
>>>> Now I would like to modify the starcluster AMIs to my needs.
>>>>
>>>> In the mailinglist I found various references to a create_image.py
>>>> script, which I cannot find: neither in the mist recent stable version
>>>> (0.90.1) nor in the git-version.
>>>>
>>>>
>>>> In the git version, I found the command `createimage' for starcluster
>>>> (starcluster -h), and assume that this is the replacement. Now I am
>>>> wondering if this is the recommended way at the moment or if there are
>>>> better choices.
>>>>
>>>> Best, and thanks for making star cluster available.
>>>> thomas
>
>>
>>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvHCKwACgkQ4llAkMfDcrm+bACfduEW9Ffh25493EpIW2suda8p
9JIAnRwGlEKm2nqmVWcsoqT+IXFuGhZe
=iv9L
-----END PGP SIGNATURE-----
Received on Thu Apr 15 2010 - 08:38:06 EDT