StarCluster - Mailing List Archive

Re: [Starcluster] Eucalyptus support

From: Justin Riley <no email>
Date: Wed, 24 Mar 2010 14:18:52 -0400

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Nicholas,

I've added support for eucalyptus to starcluster. Could you pull the
latest development code and give it a shot? In the [aws info] section
you will need to add the following settings:

AWS_PORT=8773
AWS_IS_SECURE=False
AWS_REGION_NAME=eucalyptus
AWS_REGION_HOST=#eucalyptus host ip here
AWS_EC2_PATH=/services/Eucalyptus
AWS_S3_PATH=/services/Walrus

I was able to use listimages, listvolumes, showimage, etc actions from
starcluster successfully. I have not tried to launch a cluster or mount
EBS volumes.

If I were you I'd leave out the EBS stuff until a bare-bones cluster is
able to start without issues on your Eucalyptus cluster.

BTW, did you have any luck ssh'ing into your StarCluster EMI on your
local eucalyptus cluster?

~Justin

On 03/24/2010 04:43 AM, Nicholas Ampazis wrote:
> Dear Justin,
>
> Excellent news! I'm not sure though why you could not connect to our server...
>
> Yesterday I also downloaded the source code of euca2ools from here:
>
> http://open.eucalyptus.com/downloads
>
> and I compiled it within my own working environment where I've also
> compiled starcluster. All euca-* commands seem to be working allright
> and I can connect to my server. Unfortunately however I still have
> trouble connecting using the starcluster code, even though I've
> included all the necessary modifications which I've copied from the
> "__init__.py" file of the euca2ools source code (attached).
>
> In any case you seem to be making good progress with this and I really
> can't wait to hear for the github announcement.
>
>
> Many thanks for the support,
>
>
> Nicholas
>
>
>
> On Wed, Mar 24, 2010 at 6:45 AM, Justin Riley <justin.t.riley_at_gmail.com> wrote:
>> Hi Nicolas,
>>
>> I've cc'd the list on this one, hope you don't mind.
>>
>> Before modifying starcluster to connect to Eucalyptus I decided to try it out
>> manually using ipython shell and boto. I wasn't able to get it to work using
>> your config settings for some reason, however, I set up my own single-node
>> eucalyptus cluster today locally and I'm happy to report that boto seems to
>> work well with my local eucalyptus cluster.
>>
>> Keep in mind that I haven't done any rigorous testing outside of just playing
>> around with a few of the boto API calls and there very well may be other
>> issues lurking. Initial tests seem to indicate that it's possible....we'll
>> see. I'm adding in configuration support now and will make the necessary
>> modifications to awsutils.py in the github code so that we can test this out
>> further.
>>
>> Stay tuned,
>>
>> ~Justin
>>
>> On Sunday 21 March 2010 6:01:22 pm you wrote:
>>> Dear Justin,
>>>
>>> Many thanks for the Starcluster ISOs that you provided recently.
>>>
>>> I've successfully managed to bundle and upload the x86_64 iso to a UEC
>>> (eucalyptus) ami. I've also been successful in getting the instance
>>> running but I'm facing two problems:
>>>
>>> 1) I cannot ssh into the instance using e.g.
>>>
>>> ssh -i ~/.ssh/blah.priv sgeadmin_at_XXX.XXX.XXX.XXX
>>>
>>> I've also tried "ubuntu" and "root" as usernames but still no luck.
>>> Please note that I can login to other ubuntu-based amis started with
>>> this keypair with no problem, so there's nothing wrong with my
>>> keypairs or with group permissions and such (e.g. ec2-authorize
>>> default -P tcp -p 22 -s 0.0.0.0/0, etc)
>>>
>>>
>>> 2) I installed the latest starcluster code using git and then I tried
>>> to modify the python code in order to make it work with Eucalyptus.
>>> I've discovered some extra parameters that I can pass to
>>> boto.connect_ec2 and boto.connect_s3 in order to make them connect to
>>> my UEC front-end server rather than Amazon, and the code compiles
>>> allright. I've also set up a config file with my UEC credentials.
>>>
>>> However, when I run e.g. "starcluster start smallcluster test" I'm
>>> getting the following error:
>>>
>>> Traceback (most recent call last):
>>> File "/usr/local/bin/starcluster", line 5, in <module>
>>> pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/pytho
>>> n/pkg_resources.py", line 442, in run_script
>>> self.require(requires)[0].run_script(script_name, ns)
>>> File
>>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/pytho
>>> n/pkg_resources.py", line 1167, in run_script
>>> exec script_code in namespace, namespace
>>> File
>>> "/Library/Python/2.6/site-packages/StarCluster-0.9999-py2.6.egg/EGG-INFO/s
>>> cripts/starcluster", line 6, in <module>
>>>
>>> File "build/bdist.macosx-10.6-universal/egg/starcluster/cli.py",
>>> line 546, in main
>>> File "build/bdist.macosx-10.6-universal/egg/starcluster/cli.py",
>>> line 157, in execute
>>> File "build/bdist.macosx-10.6-universal/egg/starcluster/config.py",
>>> line 291, in get_cluster
>>> File "build/bdist.macosx-10.6-universal/egg/starcluster/cluster.py",
>>> line 148, in __init__
>>> File "build/bdist.macosx-10.6-universal/egg/starcluster/awsutils.py",
>>> line 51, in __init__
>>> TypeError: __init__() takes exactly 4 arguments (8 given)
>>>
>>> I understand that this error that I'm getting is due to the extra
>>> arguments that I've added but, unfortunately, my python skills are not
>>> that good and the _init__() is defined in multiple places. So, for
>>> example, I'm not sure whether I have to make a global change in "class
>>> EasyAWS(object)" or to each individual class (e.g. class
>>> EasyEC2(EasyAWS)). In addition, I'm not certain that more UEC related
>>> parameters aren't needed in other parts of the code.
>>>
>>> To this end, I've decided to ask for your help in case that you can
>>> enlighten me further. I'm attaching the file awsutils.py where I've
>>> made the changes and the config file that I'm using, so perhaps you
>>> can try it out for yourelf in our UEC server.
>>>
>>>
>>> Many thanks,
>>>
>>>
>>> Nicholas
>>
>>
>>
>>
>> _______________________________________________
>> Starcluster mailing list
>> Starcluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkuqV4wACgkQ4llAkMfDcrljrwCfUHJcW2PHcBdeKTxcB20eiOxx
hwMAn2NY0lW6F9IeUpWYzvWAAVrIESho
=GmyF
-----END PGP SIGNATURE-----
Received on Wed Mar 24 2010 - 14:18:55 EDT
This archive was generated by hypermail 2.3.0.

Search:

Sort all by:

Date

Month

Thread

Author

Subject