Re: [Starcluster] create_image.py vs. createimage

From: Justin Riley <no email>
Date: Thu, 15 Apr 2010 08:44:48 -0400

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Thomas,

This is because of the files left over from a previous run.

So, if you encounter such errors as below you'll need to:

1. restore authorized keys file (simply reboot and these will be restored)

2. unmount /mnt/img-mnt, remove /mnt/img-mnt directory, and remove all
myStarcluster* image parts and manifests generated by the ec2-bundle script.

After this, retrying the createimage command *should* work. I will
likely add a check that reboots the instance if ssh fails. I will also
add a check to automatically do #2 above before attempting to create the
image.

Thanks,

~Justin

On 04/15/2010 08:01 AM, Thomas Deselaers wrote:
> Hi,
>
> once more me. I solved this problem and now have the next.
>
> The reason for this problem was that the first tries to create the
> image deleted the /root/.ssh/authorized_keys file and thus afterwards
> authentication did not work anymore.
>
>
> I restored that file, and retried.
>
> Now I get the error message related to the manifest file below.
>
> I looked into the machine and found that the file
> '/mnt/myStarcluster.manifest.xml' does not exist.
>
> Again, any help is greatly appreciated and error messages are just below.
>
> Thanks,
> thomas
>
>
> [desthoma_at_galaxoid:~/work/text/aws$] starcluster createimage
> i-73e98818 myStarcluster starclusterAMIBucket
> StarCluster - (http://web.mit.edu/starcluster)
> Software Tools for Academics and Researchers (STAR)
> Please submit bug reports to starcluster_at_mit.edu
>
>>>> Using private key /home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa (rsa)
>>>> Removing private data...
>>>> Creating the bundled image:
> Copying / into the image file /mnt/myStarcluster...
> Excluding:
> /sys
> /myopt/installdisk
> /proc/fs/nfsd
> /proc
> /dev/pts
> /dev
> /media
> /mnt
> /proc
> /sys
> /etc/udev/rules.d/70-persistent-net.rules
> /etc/udev/rules.d/z25_persistent-net.rules
> /mnt/myStarcluster
> /mnt/img-mnt
> 1+0 records in
> 1+0 records out
> 1048576 bytes (1.0 MB) copied, 0.00163 s, 643 MB/s
> mke2fs 1.41.4 (27-Jan-2009)
> ERROR: execution failed: "rsync -rlpgoD -t -r -S -l --exclude /sys
> --exclude /myopt/installdisk --exclude /proc/fs/nfsd --exclude /proc
> --exclude /dev/pts -
> -exclude /dev --exclude /media --exclude /mnt --exclude /proc
> --exclude /sys --exclude /etc/udev/rules.d/70-persistent-net.rules
> --exclude /etc/udev/rules.d
> /z25_persistent-net.rules --exclude /mnt/myStarcluster --exclude
> /mnt/img-mnt -X /* /mnt/img-mnt 2>&1 > /dev/null"
>
>>>> Cleaning up...
>>>> _bundle_image took 8.388 mins
>>>> Uploading bundled image:
> --manifest has invalid value '/mnt/myStarcluster.manifest.xml': File
> does not exist or is not a file.
>
> Try 'ec2-upload-bundle --help'
>>>> _upload_image took 0.012 mins
> ---------------------------------------------------------------------------
> EC2ResponseError Traceback (most recent call last)
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
> in <module>()
> 3 __requires__ = 'StarCluster==0.9999'
> 4 import pkg_resources
> ----> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
> 6
> 7
>
> /home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
> in run_script(self, requires, script_name)
> 487 ns.clear()
> 488 ns['__name__'] = name
> --> 489 self.require(requires)[0].run_script(script_name, ns)
> 490
> 491
>
> /home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
> in run_script(self, script_name, namespace)
> 1212 )
> 1213 script_code = compile(script_text,script_filename,'exec')
> -> 1214 exec script_code in namespace, namespace
> 1215
> 1216 def _has(self, path):
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
> in <module>()
> 4 import pkg_resources
> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
> ----> 6
> 7
> 8
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
> in main()
> 750 sys.exit(0)
> 751 try:
> --> 752 sc.execute(args)
> 753 except exception.BaseException,e:
> 754 log.error(e.msg)
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
> in execute(self, args)
> 360 })
> 361 icreator = image.EC2ImageCreator(**kwargs)
> --> 362 icreator.create_image()
> 363
> 364 class CmdCreateVolume(CmdBase):
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/utils.pyc
> in wrapper(*arg, **kargs)
> 22 """Raw timing function """
> 23 time1 = time.time()
> ---> 24 res = func(*arg, **kargs)
> 25 time2 = time.time()
> 26 log.info('%s took %0.3f mins' % (func.func_name,
> (time2-time1)/60.0))
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
> in create_image(self)
> 57 self._bundle_image()
> 58 self._upload_image()
> ---> 59 self._register_image()
> 60 if self.remove_image_files:
> 61 self._remove_image_files()
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
> in _register_image(self)
> 124 self.prefix,
> 125 image_location=
> "%(bucket)s/%(prefix)s.manifest.xml" % config_dict,
> --> 126 architecture=config_dict.get('arch'),
> 127 )
> 128
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/awsutils.pyc
> in register_image(self, name, description, image_location,
> architecture
> , kernel_id, ramdisk_id, root_device_name, block_device_map)
> 137 return _register_image(self.conn, name, description,
> image_location,
> 138 architecture, kernel_id, ramdisk_id,
> --> 139 root_device_name, block_device_map)
> 140
> 141 def get_keypair(self, keypair):
>
> /home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/hacks.pyc
> in register_image(conn, name, description, image_location,
> architecture, k
> ernel_id, ramdisk_id, root_device_name, block_device_map)
> 55 if block_device_map:
> 56 block_device_map.build_list_params(params)
> ---> 57 rs = conn.get_object('RegisterImage', params, ResultSet)
> 58 image_id = getattr(rs, 'imageId', None)
> 59 return image_id
>
> /home/desthoma/work/lib/python/boto-1.9b-py2.5.egg/boto/connection.pyc
> in get_object(self, action, params, cls, path, parent, verb)
> 629 boto.log.error('%s %s' % (response.status, response.reason))
> 630 boto.log.error('%s' % body)
> --> 631 raise self.ResponseError(response.status,
> response.reason, body)
> 632
> 633 def get_status(self, action, params, path='/',
> parent=None, verb='GET'):
>
> EC2ResponseError: EC2ResponseError: 400 Bad Request
> <?xml version="1.0"?>
> <Response><Errors><Error><Code>InvalidManifest</Code><Message>HTTP 404
> (Not Found) response for URL
> http://s3.amazonaws.com:80/starclusterAMIBucket/myStarcl
> uster.manifest.xml: check your manifest path is correct and in the
> correct region.</Message></Error></Errors><RequestID>99c7de06-b84c-48a4-8145-596d9c4fa34c
> </RequestID></Response>
>
>
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvHCkAACgkQ4llAkMfDcrl19QCfe9DSI9JqLguOQn0r1m5ZSoLu
MSgAnjjJgfbVh9JvnPs5afYl2GYdZ8vu
=fO8w
-----END PGP SIGNATURE-----
Received on Thu Apr 15 2010 - 08:44:49 EDT