Hi,
I just found one more mistake in my usage:
I confused X509 certificates and other keypairs.
Now I do
export EC2_CERT=/home/desthoma/work/text/aws/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx.pem
export EC2_PRIVATE_KEY=/home/desthoma/work/text/aws/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx.pem
starcluster createimage i-XXXXXX myStarcluster starclusterAMIBucket
I also put these files into the ec2_cert and ec2_private_key fields of
my ~/.starcluster/config file.
The output is below.
Thanks in advance,
cheers,
thomas
----------------
StarCluster - (
http://web.mit.edu/starcluster)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster_at_mit.edu
>>> Using private key /home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa (rsa)
---------------------------------------------------------------------------
AuthenticationException Traceback (most recent call last)
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
in <module>()
3 __requires__ = 'StarCluster==0.9999'
4 import pkg_resources
----> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
6
7
/home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
in run_script(self, requires, script_name)
487 ns.clear()
488 ns['__name__'] = name
--> 489 self.require(requires)[0].run_script(script_name, ns)
490
491
/home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
in run_script(self, script_name, namespace)
1212 )
1213 script_code = compile(script_text,script_filename,'exec')
-> 1214 exec script_code in namespace, namespace
1215
1216 def _has(self, path):
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
in <module>()
4 import pkg_resources
5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
----> 6
7
8
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
in main()
750 sys.exit(0)
751 try:
--> 752 sc.execute(args)
753 except exception.BaseException,e:
754 log.error(e.msg)
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
in execute(self, args)
359 'bucket': bucket,
360 })
--> 361 icreator = image.EC2ImageCreator(**kwargs)
362 icreator.create_image()
363
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
in __init__(self, instance, aws_access_key_id, aws_secret_access_key,
aws
_image_files, **kwargs)
51 }
52 if self.host:
---> 53 self.config_dict['arch'] = self._get_arch()
54
55 _at_print_timing
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
in _get_arch(self)
72
73 def _get_arch(self):
---> 74 conn = self.host.ssh
75 arch = conn.execute(
76 'python -c "import platform; print
platform.architecture()[0]"'
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/node.pyc
in ssh(self)
147 self._ssh = ssh.Connection(self.instance.dns_name,
148 username=self.user,
--> 149 private_key=self.key_location)
150 return self._ssh
151
/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/ssh.pyc
in __init__(self, host, username, password, private_key,
private_key_pass,
77 raise TypeError, "You have not specified a
password or key."
78
---> 79 self._transport.connect(username = username, pkey = pkey)
80
81 def _load_rsa_key(self, private_key, private_key_pass=None):
/var/lib/python-support/python2.5/paramiko/transport.pyc in
connect(self, hostkey, username, password, pkey)
987 else:
988 self._log(DEBUG, 'Attempting public-key auth...')
--> 989 self.auth_publickey(username, pkey)
990
991 return
/var/lib/python-support/python2.5/paramiko/transport.pyc in
auth_publickey(self, username, key, event)
1214 # caller wants to wait for event themselves
1215 return []
-> 1216 return self.auth_handler.wait_for_response(my_event)
1217
1218 def auth_interactive(self, username, handler, submethods=''):
/var/lib/python-support/python2.5/paramiko/auth_handler.pyc in
wait_for_response(self, event)
172 if issubclass(e.__class__, PartialAuthentication):
173 return e.allowed_types
--> 174 raise e
175 return []
176
On Thu, Apr 15, 2010 at 11:54, Thomas Deselaers
<deselaers_at_vision.ee.ethz.ch> wrote:
> Hi Justin,
>
> Thanks a lot for your prompt reply.
>
> I just played a bit with the new version to create a new image.
>
> I already solved a few problems but now I am at a point where I don't
> know what to do anymore.
>
> Could you give me a pointer what I should change?
>
> I manually started an instance (64bit image) and modified it a bit.
> Then I tried to run the commands below.
>
> You also find the output of starcluster.
>
>
> Thanks in advance
>
> export EC2_PRIVATE_KEY="/home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa"
> export EC2_CERT=/home/desthoma/work/text/aws/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXxx.pem
>
> starcluster createimage i-73eXXXXX myStarcluster starclusterAMIBucket
>
> StarCluster - (http://web.mit.edu/starcluster)
> Software Tools for Academics and Researchers (STAR)
> Please submit bug reports to starcluster_at_mit.edu
>
>>>> Using private key /home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa (rsa)
> ---------------------------------------------------------------------------
> AuthenticationException Traceback (most recent call last)
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
> in <module>()
> 3 __requires__ = 'StarCluster==0.9999'
> 4 import pkg_resources
> ----> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
> 6
> 7
>
> /home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
> in run_script(self, requires, script_name)
> 487 ns.clear()
> 488 ns['__name__'] = name
> --> 489 self.require(requires)[0].run_script(script_name, ns)
> 490
> 491
>
> /home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
> in run_script(self, script_name, namespace)
> 1212 )
> 1213 script_code = compile(script_text,script_filename,'exec')
> -> 1214 exec script_code in namespace, namespace
> 1215
> 1216 def _has(self, path):
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
> in <module>()
> 4 import pkg_resources
> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
> ----> 6
> 7
> 8
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
> in main()
> 750 sys.exit(0)
> 751 try:
> --> 752 sc.execute(args)
> 753 except exception.BaseException,e:
> 754 log.error(e.msg)
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
> in execute(self, args)
> 359 'bucket': bucket,
> 360 })
> --> 361 icreator = image.EC2ImageCreator(**kwargs)
> 362 icreator.create_image()
> 363
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
> in __init__(self, instance, aws_access_key_id, aws_secret_access_key,
> aws_user_id, bucket, ec2_cert, ec2_private_key, prefix,
> remove_image_files, **kwargs)
> 51 }
> 52 if self.host:
> ---> 53 self.config_dict['arch'] = self._get_arch()
> 54
> 55 _at_print_timing
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
> in _get_arch(self)
> 72
> 73 def _get_arch(self):
> ---> 74 conn = self.host.ssh
> 75 arch = conn.execute(
> 76 'python -c "import platform; print
> platform.architecture()[0]"'
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/node.pyc
> in ssh(self)
> 147 self._ssh = ssh.Connection(self.instance.dns_name,
> 148 username=self.user,
> --> 149 private_key=self.key_location)
> 150 return self._ssh
> 151
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/ssh.pyc
> in __init__(self, host, username, password, private_key,
> private_key_pass, port)
> 77 raise TypeError, "You have not specified a
> password or key."
> 78
> ---> 79 self._transport.connect(username = username, pkey = pkey)
> 80
> 81 def _load_rsa_key(self, private_key, private_key_pass=None):
>
> /var/lib/python-support/python2.5/paramiko/transport.pyc in
> connect(self, hostkey, username, password, pkey)
> 987 else:
> 988 self._log(DEBUG, 'Attempting public-key auth...')
> --> 989 self.auth_publickey(username, pkey)
> 990
> 991 return
>
> /var/lib/python-support/python2.5/paramiko/transport.pyc in
> auth_publickey(self, username, key, event)
> 1214 # caller wants to wait for event themselves
> 1215 return []
> -> 1216 return self.auth_handler.wait_for_response(my_event)
> 1217
> 1218 def auth_interactive(self, username, handler, submethods=''):
>
> /var/lib/python-support/python2.5/paramiko/auth_handler.pyc in
> wait_for_response(self, event)
> 172 if issubclass(e.__class__, PartialAuthentication):
> 173 return e.allowed_types
> --> 174 raise e
> 175 return []
> 176
>
>
>
>
>
> On Wed, Apr 14, 2010 at 21:20, Justin Riley <jtriley_at_mit.edu> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi Thomas,
>>
>> The github code's createimage command is the recommended way, however, I
>> haven't made a new release out of this code yet. You're welcome to try
>> out the latest code; it's very close to ready. The config has been
>> enhanced quite a bit past the previous 0.90.1 version so you'll
>> certainly want to look at the docs in github:
>>
>> http://github.com/jtriley/StarCluster/tree/master/docs/sphinx
>>
>> The *.rst files are the raw text documents. These docs are also
>> available on the web here:
>>
>> http://web.mit.edu/stardev/cluster/docs/
>>
>> I'm in the process of cleaning up the code in github and finishing up
>> some documentation for a new release that supports multiple clusters,
>> multiple ebs volumes, cluster 'plugins' that allow users to write their
>> own custom setup routines on top of starcluster's defaults, automated
>> EBS volume creation (including partition/format), and more.
>>
>> If you could also report any issues (either bugs or usability issues)
>> with the github code to this list I'd appreciate it.
>>
>> Thanks!
>>
>> ~Justin
>>
>>
>>
>> On 04/14/2010 11:39 AM, Thomas Deselaers wrote:
>>> Dear all,
>>>
>>> I am a new (since today) StarCluster user and I already managed to get
>>> a few things done (start my "own" cluster, run jobs, etc). The
>>> documentation is really fine (and I know SGE from the past).
>>>
>>> Now I would like to modify the starcluster AMIs to my needs.
>>>
>>> In the mailinglist I found various references to a create_image.py
>>> script, which I cannot find: neither in the mist recent stable version
>>> (0.90.1) nor in the git-version.
>>>
>>>
>>> In the git version, I found the command `createimage' for starcluster
>>> (starcluster -h), and assume that this is the replacement. Now I am
>>> wondering if this is the recommended way at the moment or if there are
>>> better choices.
>>>
>>> Best, and thanks for making star cluster available.
>>> thomas
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2.0.14 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAkvGFZYACgkQ4llAkMfDcrmc1QCdEWkhSUtP9K2s6xw8oOmtzJ+5
>> pDMAnjRV4yBGjRvgRgrmUJLoNSeBLLuz
>> =diB3
>> -----END PGP SIGNATURE-----
>>
>>
>
>
>
> --
> http://thomas.deselaers.de
>
--
http://thomas.deselaers.de
Received on Thu Apr 15 2010 - 06:16:18 EDT