Re: [Starcluster] failed installing Sun Grid Engine...

From: Dan Yamins <no email>
Date: Wed, 17 Mar 2010 14:38:56 -0400

Which EC2 processors can you not use with either the 64-bit or 32 -bit
starcluster AMI?




On Wed, Mar 17, 2010 at 2:04 PM, chuan gao <aggie.gao_at_gmail.com> wrote:

> Hi Justin,
> Sorry for the confusion, I used the wrong AMI IDs. stupid mistake that
> everybody can avoid by following the instructions. I saw it on the webpage
> but still went ahead and changed that because I want to use the fastest
> processor on amazon. :(
> thank you very much for the software, truly nice work.
> Chuan
>
>
>
> On Wed, Mar 17, 2010 at 1:51 PM, Justin Riley <jtriley_at_mit.edu> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Chuan,
>>
>> Responding to your other question concerning GSL and gotoBLAS:
>>
>> Installing these locally into a EBS volume is a perfectly reasonable
>> solution. In fact, this is how I envisioned most people storing their
>> software/data on StarCluster.
>>
>> The other way to do this is to launch an instance, install the software
>> globally (using apt-get and/or source packages), and then rebundle the
>> AMI. You could then tell starcluster to use this new AMI in your config
>> file.
>>
>> I will add GSL to the next version of StarCluster's AMI. goto blas must
>> be custom compiled. I'll have a look at this and if it's not too much
>> work I'll consider adding it in.
>>
>> Hope that helps,
>>
>> ~Justin
>>
>> On 03/17/2010 01:35 PM, chuan gao wrote:
>> > Hi Justin and Dan,
>> > Sorry for all the trouble, it's all my fault, it turned out that I used
>> > the wrong AMI type. it worked like a charm after I changed that. I do
>> > have another question though, I need to use GSL and gotoBLAS for my
>> > computation, which I didn't find it installed on the cluster, I am
>> > thinking about compiling these locally on my EBS volume and include the
>> > library in my code. is there any other better way?
>> > Thank you all guys for the help!
>> > Chuan
>> >
>> >
>> >
>> > On Wed, Mar 17, 2010 at 10:51 AM, Justin Riley <jtriley_at_mit.edu
>> > <mailto:jtriley_at_mit.edu>> wrote:
>> >
>> > Hi Chuan,
>> >
>> > Have you checked whether or not the EBS volume ever gets to an
>> > 'attached'
>> > state using either ElasticFox or the AWS web console?
>> >
>> > Also, you can run starcluster in debug mode by passing -d option:
>> >
>> > $ starcluster -d -s
>> >
>> > This will do the same thing as -s only with debug output enabled.
>> >
>> > Would you mind sending me that output? Please be careful about any
>> > sensitive
>> > data in the output if you cc this list.
>> >
>> > Thanks,
>> >
>> > ~Justin
>> >
>> >
>> > On Wednesday 17 March 2010 1:18:25 am chuan gao wrote:
>> > > Hi Justin and Dan,
>> > >
>> > > Sorry for the late reply cuz I was tied up on a few things.
>> > >
>> > > To Justin: I tried it without mount EBS and I still have that
>> > problem. I
>> > > believe it's related to this though cuz it happened at the last
>> > step of
>> > > configuring the cluster(all machines have been started by that
>> time).
>> > >
>> > > To Dan: I did tried reinstall paramiko (not in debug mode though
>> > since I
>> > > didn't find out how to do so,) and the system said installed
>> > sucessfully. I
>> > > also tried starting the cluster outside the directory without
>> success.
>> > >
>> > > I also tried the same thing on another machine which has ubuntu on
>> > it and I
>> > > got the same error. I am not the administrator to that machine but
>> > I guess
>> > > it should be ubuntu 9.04.
>> > >
>> > > any ideas? thank you guys very much for handling this.
>> > >
>> > > Chuan
>> > >
>> > > On Tue, Mar 16, 2010 at 7:13 PM, Justin Riley <jtriley_at_mit.edu
>> > <mailto:jtriley_at_mit.edu>> wrote:
>> > > > Hi Chuan,
>> > > >
>> > > > Sorry to hear you're having issues with StarCluster. I just got
>> > back in
>> > > > town,
>> > > > sorry for the delayed response.
>> > > >
>> > > > Dan, as always, thanks for responding :D
>> > > >
>> > > > Could you try launching a cluster without using the EBS volume
>> > and let me
>> > > > know
>> > > > if it succeeds? Nicolas Pinto had a similar issue with
>> > StarCluster and I
>> > > > believe it only happened for him when using EBS.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > ~Justin
>> > > >
>> > > > On Tuesday 16 March 2010 2:15:12 pm chuan gao wrote:
>> > > > > I am using the standard AMI
>> > > > > the OS is ubuntu 9.10
>> > > > > yes. I can ssh into AMI with no problem. actually, the error
>> > occured at
>> > > >
>> > > > the
>> > > >
>> > > > > last step when it is configuring NFS. I checked the AMI on
>> > amazon and
>> > > >
>> > > > they
>> > > >
>> > > > > have been started. and my permanent volume has been mounted.
>> > > > > concerning the error,
>> > > > > build/bdist.linux-i686/egg/paramiko/sftp_client.py
>> > > > > I looked into directory
>> > StarCluster-0.90.1/build/bdist.linux-i686 and
>> > > > > nothing is there. should there be egg/paramiko/sftp_client.py
>> ?
>> > > > > Thanks
>> > > > >
>> > > > > On Tue, Mar 16, 2010 at 1:46 PM, Dan Yamins <
>> dyamins_at_gmail.com
>> > <mailto:dyamins_at_gmail.com>> wrote:
>> > > > > > Justin is on this list, so I'm sure he'll respond soon.
>> > > > > >
>> > > > > > Are you using the standard AMI, or one you built yourself?
>> > > > > >
>> > > > > > Also, what is your operating system and version?
>> > > > > >
>> > > > > > Also, can you ssh into instances of the AMI, independently
>> of
>> > > > > > starcluster?
>> > > > > >
>> > > > > > On Tue, Mar 16, 2010 at 1:17 PM, chuan gao
>> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>>
>> > > >
>> > > > wrote:
>> > > > > >> Thanks Dan, I am pretty sure that I got paramiko installed
>> > > > > >> correctly. I'll work on it a bit more.
>> > > > > >> will Justin have a chance to look at this email list and
>> try to
>> > > > > >> point out what could be the problem?
>> > > > > >>
>> > > > > >> On Tue, Mar 16, 2010 at 11:21 AM, Dan Yamins
>> > <dyamins_at_gmail.com <mailto:dyamins_at_gmail.com>>
>> > > >
>> > > > wrote:
>> > > > > >>> Hm. I'm not sure what the problem is (your version is
>> fine).
>> > > > > >>> Something is clearly wrong with either your paramiko
>> > installation
>> > > > > >>> or the way that starcluster is using it.
>> > > > > >>>
>> > > > > >>> Have you tried testing paramiko outside the context of
>> > starcluster?
>> > > > > >>> Try testing it for normal ssh usage. If that fails, then
>> > you'll
>> > > > > >>> probably have identified the problem. (I don't know if
>> > you just
>> > > > > >>> installed it, perhaps reinstallation would then help.)
>> > > > > >>>
>> > > > > >>> If paramiko works normally, then perhaps there's an
>> > argument that
>> > > > > >>> is being passed on line 109 of starcluster/ssh.py that is
>> > supposed
>> > > > > >>> to represent an existing file object, that somehow isn't
>> being
>> > > > > >>> properly created (probably earlier in the ssh cycle). In
>> this
>> > > > > >>> case, Justin
>> > > >
>> > > > (the
>> > > >
>> > > > > >>> creator of starcluster) should probably be the one to
>> > address your
>> > > > > >>> problem.
>> > > > > >>>
>> > > > > >>> Dan
>> > > > > >>>
>> > > > > >>> On Tue, Mar 16, 2010 at 10:57 AM, chuan gao
>> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>>
>> > > >
>> > > > wrote:
>> > > > > >>>> paramiko-1.7.6-py2.6.egg
>> > > > > >>>> Thanks for replying.
>> > > > > >>>>
>> > > > > >>>> On Tue, Mar 16, 2010 at 9:02 AM, Dan Yamins
>> > <dyamins_at_gmail.com <mailto:dyamins_at_gmail.com>>
>> > > >
>> > > > wrote:
>> > > > > >>>>> On Tue, Mar 16, 2010 at 1:05 AM, chuan gao
>> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>
>> > > > >
>> > > > >wrote:
>> > > > > >>>>>> Here is the error message:
>> > > > > >>>>>> >>> Installing Sun Grid Engine...
>> > > > > >>>>>>
>> > > > > >>>>>> [SNIP]
>> > > > > >>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>> File
>> > "build/bdist.linux-i686/egg/paramiko/sftp_client.py",
>> > > > > >>>>>> line 675, in _read_response
>> > > > > >>>>>> File
>> > "build/bdist.linux-i686/egg/paramiko/sftp_client.py",
>> > > > > >>>>>> line 701, in _convert_status
>> > > > > >>>>>> IOError: [Errno 2] No such file
>> > > > > >>>>>
>> > > > > >>>>> What version of paramiko do you have installed?
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>> _______________________________________________
>> > > > > >>>>> Starcluster mailing list
>> > > > > >>>>> Starcluster_at_mit.edu <mailto:Starcluster_at_mit.edu>
>> > > > > >>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>> > >
>> >
>> >
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2.0.14 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAkuhFqwACgkQ4llAkMfDcrkcEwCgi0EdVkC1/5Ne578BNYkQYMpO
>> +fYAn13tNTBQ2M2P29hgJoltz4nwj2DU
>> =Vuxo
>> -----END PGP SIGNATURE-----
>>
>
>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
Received on Wed Mar 17 2010 - 14:38:57 EDT