Re: [Starcluster] Bug VolumeCreator _request_instance

From: Joachim Van den Bogaert <no email>
Date: Fri, 9 Jul 2010 00:52:54 +0200

Hi Justin,

I thought you would need a StarCluster AMI to create a new volume.
I didn't think of using *any* 32-bit image, which is - I agree - a lot more
sensible
and indeed cheaper.

Making the StarCluster AMI's available for all zones is a good thing
(ec2-migrate-instance may take quite some time ...).
And while you're at it: do you think it would be possible to create EBS'es
instead of instance store images?

Joachim

-----Oorspronkelijk bericht-----
Van: Justin Riley [mailto:jtriley_at_mit.edu]
Verzonden: vrijdag 9 juli 2010 0:16
Aan: Joachim Van den Bogaert
CC: starcluster_at_mit.edu
Onderwerp: Re: [Starcluster] Bug VolumeCreator _request_instance

Hi Joachim,

Thanks for reporting this. Adding an --instance-type option to the
createvolume
command is on my todo list now and this should fix your issue.

For the record though, there's no need to use a 64bit image to create a new
volume. The reason I chose m1.small as a default is because this instance is
*only* used to mount the volume and automate the partitioning/formatting.
For
this there really isn't any advantage to using a 64bit image and using
m1.small
is your cheapest option.

I would recommend using a 32bit alestic image (alestic.com), or *any* 32bit
image, in the eu-west-1 zone with the createvolume command. This avoids
needing
a different instance type. Regardless, I'll be adding checks for invalid
instance_type/image_id combos though in the VolumeCreator class in addition
to
allowing instance_type to be passed to the createvolume action at the
command
line.

Also, I've decided to replicate the StarCluster AMIs in all availability
zones
so that folks outside the us-east-1 zone do not have to migrate the AMIs
themselves. The whole point of StarCluster is to abstract most of the
details
of using EC2 to create HPC clusters in the cloud. Having users migrate the
AMI
themselves to other zones is suboptimal. I'll send a notice to the list in
the
next week or so when the us-west/eu-west/ap-souteast AMIs are available

Hope that helps and thanks again for reporting.

~Justin

Quoting Joachim Van den Bogaert <joachim_at_ccl.kuleuven.be>:

> Hi Justin,
>
>
>
> I tried to create a volume with the following command:
>
>
>
> joachim_at_tm1:~$ starcluster createvolume 100 eu-west-1b --image-id
> ami-6be5cf1f
>
>
>
>>>> Using keypair sge
>
>>>> Requesting host instance in zone eu-west-1b to attach volume to...
>
>>>> Creating security group _at_sc-volumecreator...
>
>>>> No instance in group _at_sc-volumecreator for zone eu-west-1b, launching
> one now.
>
>
>
> volume.py:186 - ERROR - exception thrown: EC2ResponseError: 400 Bad
Request
>
> <?xml version="1.0"?>
>
> <Response>
>
> <Errors>
>
> <Error>
>
> <Code>InvalidParameterValue</Code>
>
> <Message>The requested instance type's architecture (i386) does not
> match the architecture in the manifest for ami-6be5cf1f (x86_64)</Message>
>
> </Error>
>
> </Errors>
>
> <RequestID>4ff61a79-6a82-480e-b1dd-e26d8965eb70</RequestID>
>
> </Response>
>
>
>
> I was looking for an architecture parameter, but I couldn't find one
>
>
>
> (volume.py line 32 - 56)
>
>
>
> def _request_instance(self, zone):
>
> for i in self.security_group.instances():
>
> if i.state in ['pending','running'] and i.placement == zone:
>
> log.info("Using existing instance %s in group %s" % \
>
> (i.id,self.security_group.name))
>
> self._instance = Node(i, self._key_location, 'vol_host')
>
> break
>
> if not self._instance:
>
> log.info("No instance in group %s for zone %s, launching one
> now." % \
>
> (self.security_group.name, zone))
>
> self._resv = self._ec2.run_instances(image_id=self._image_id,
>
> instance_type='m1.small',
>
> min_count=1, max_count=1,
>
> security_groups=[self.security_group.name],
>
> key_name=self._keypair,
>
> placement=zone)
>
> instance = self._resv.instances[0]
>
> self._instance = Node(instance, self._key_location, 'vol_host')
>
> s = Spinner()
>
> log.info("Waiting for instance %s to come up..." %
> self._instance.id)
>
> s.start()
>
> while not self._instance.is_up():
>
> time.sleep(15)
>
> s.stop()
>
> return self._instance
>
>
>
> Apparently the code in line 43
>
>
>
> instance_type='m1.small'
>
>
>
> throws an error if you try to create a volume with a 64-bit StarCluster
AMI,
> since you can't launch an m1.small with 64 bit images.
>
>
>
> I am in the eu zone, so I can't use the AMI defined in
> image_id=static.BASE_AMI_32.
>
> With the 64-bit image I use instead, an extra parameter could prevent this
> error
>
>
>
> After I changed the line into
>
>
>
> instance_type='m1.large'
>
>
>
> the volume was created successfully.
>
>
>
>
>
>
>
> Kind regards,
>
> Joachim
>
>
>
>
>
>
>
>
Received on Thu Jul 08 2010 - 18:52:56 EDT