Re: [Starcluster] Bug VolumeCreator _request_instance

From: Austin Godber <no email>
Date: Thu, 08 Jul 2010 18:19:56 -0400

Oh man, I just implemented my own createcluster, I didn't realize
starcluster did it already.

Austin

On 07/08/2010 06:16 PM, Justin Riley wrote:
> Hi Joachim,
>
> Thanks for reporting this. Adding an --instance-type option to the
> createvolume
> command is on my todo list now and this should fix your issue.
>
> For the record though, there's no need to use a 64bit image to create a new
> volume. The reason I chose m1.small as a default is because this instance is
> *only* used to mount the volume and automate the partitioning/formatting. For
> this there really isn't any advantage to using a 64bit image and using
> m1.small
> is your cheapest option.
>
> I would recommend using a 32bit alestic image (alestic.com), or *any* 32bit
> image, in the eu-west-1 zone with the createvolume command. This avoids
> needing
> a different instance type. Regardless, I'll be adding checks for invalid
> instance_type/image_id combos though in the VolumeCreator class in addition to
> allowing instance_type to be passed to the createvolume action at the command
> line.
>
> Also, I've decided to replicate the StarCluster AMIs in all availability zones
> so that folks outside the us-east-1 zone do not have to migrate the AMIs
> themselves. The whole point of StarCluster is to abstract most of the details
> of using EC2 to create HPC clusters in the cloud. Having users migrate the AMI
> themselves to other zones is suboptimal. I'll send a notice to the list in the
> next week or so when the us-west/eu-west/ap-souteast AMIs are available
>
> Hope that helps and thanks again for reporting.
>
> ~Justin
>
> Quoting Joachim Van den Bogaert<joachim_at_ccl.kuleuven.be>:
>
>
>> Hi Justin,
>>
>>
>>
>> I tried to create a volume with the following command:
>>
>>
>>
>> joachim_at_tm1:~$ starcluster createvolume 100 eu-west-1b --image-id
>> ami-6be5cf1f
>>
>>
>>
>>
>>>>> Using keypair sge
>>>>>
>>
>>>>> Requesting host instance in zone eu-west-1b to attach volume to...
>>>>>
>>
>>>>> Creating security group _at_sc-volumecreator...
>>>>>
>>
>>>>> No instance in group _at_sc-volumecreator for zone eu-west-1b, launching
>>>>>
>> one now.
>>
>>
>>
>> volume.py:186 - ERROR - exception thrown: EC2ResponseError: 400 Bad Request
>>
>> <?xml version="1.0"?>
>>
>> <Response>
>>
>> <Errors>
>>
>> <Error>
>>
>> <Code>InvalidParameterValue</Code>
>>
>> <Message>The requested instance type's architecture (i386) does not
>> match the architecture in the manifest for ami-6be5cf1f (x86_64)</Message>
>>
>> </Error>
>>
>> </Errors>
>>
>> <RequestID>4ff61a79-6a82-480e-b1dd-e26d8965eb70</RequestID>
>>
>> </Response>
>>
>>
>>
>> I was looking for an architecture parameter, but I couldn't find one
>>
>>
>>
>> (volume.py line 32 - 56)
>>
>>
>>
>> def _request_instance(self, zone):
>>
>> for i in self.security_group.instances():
>>
>> if i.state in ['pending','running'] and i.placement == zone:
>>
>> log.info("Using existing instance %s in group %s" % \
>>
>> (i.id,self.security_group.name))
>>
>> self._instance = Node(i, self._key_location, 'vol_host')
>>
>> break
>>
>> if not self._instance:
>>
>> log.info("No instance in group %s for zone %s, launching one
>> now." % \
>>
>> (self.security_group.name, zone))
>>
>> self._resv = self._ec2.run_instances(image_id=self._image_id,
>>
>> instance_type='m1.small',
>>
>> min_count=1, max_count=1,
>>
>> security_groups=[self.security_group.name],
>>
>> key_name=self._keypair,
>>
>> placement=zone)
>>
>> instance = self._resv.instances[0]
>>
>> self._instance = Node(instance, self._key_location, 'vol_host')
>>
>> s = Spinner()
>>
>> log.info("Waiting for instance %s to come up..." %
>> self._instance.id)
>>
>> s.start()
>>
>> while not self._instance.is_up():
>>
>> time.sleep(15)
>>
>> s.stop()
>>
>> return self._instance
>>
>>
>>
>> Apparently the code in line 43
>>
>>
>>
>> instance_type='m1.small'
>>
>>
>>
>> throws an error if you try to create a volume with a 64-bit StarCluster AMI,
>> since you can't launch an m1.small with 64 bit images.
>>
>>
>>
>> I am in the eu zone, so I can't use the AMI defined in
>> image_id=static.BASE_AMI_32.
>>
>> With the 64-bit image I use instead, an extra parameter could prevent this
>> error
>>
>>
>>
>> After I changed the line into
>>
>>
>>
>> instance_type='m1.large'
>>
>>
>>
>> the volume was created successfully.
>>
>>
>>
>>
>>
>>
>>
>> Kind regards,
>>
>> Joachim
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
Received on Thu Jul 08 2010 - 18:19:59 EDT