Re: [Starcluster] failed installing Sun Grid Engine...

From: chuan gao <no email>
Date: Wed, 17 Mar 2010 14:04:35 -0400

Hi Justin,
Sorry for the confusion, I used the wrong AMI IDs. stupid mistake that
everybody can avoid by following the instructions. I saw it on the webpage
but still went ahead and changed that because I want to use the fastest
processor on amazon. :(
thank you very much for the software, truly nice work.
Chuan


On Wed, Mar 17, 2010 at 1:51 PM, Justin Riley <jtriley_at_mit.edu> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Chuan,
>
> Responding to your other question concerning GSL and gotoBLAS:
>
> Installing these locally into a EBS volume is a perfectly reasonable
> solution. In fact, this is how I envisioned most people storing their
> software/data on StarCluster.
>
> The other way to do this is to launch an instance, install the software
> globally (using apt-get and/or source packages), and then rebundle the
> AMI. You could then tell starcluster to use this new AMI in your config
> file.
>
> I will add GSL to the next version of StarCluster's AMI. goto blas must
> be custom compiled. I'll have a look at this and if it's not too much
> work I'll consider adding it in.
>
> Hope that helps,
>
> ~Justin
>
> On 03/17/2010 01:35 PM, chuan gao wrote:
> > Hi Justin and Dan,
> > Sorry for all the trouble, it's all my fault, it turned out that I used
> > the wrong AMI type. it worked like a charm after I changed that. I do
> > have another question though, I need to use GSL and gotoBLAS for my
> > computation, which I didn't find it installed on the cluster, I am
> > thinking about compiling these locally on my EBS volume and include the
> > library in my code. is there any other better way?
> > Thank you all guys for the help!
> > Chuan
> >
> >
> >
> > On Wed, Mar 17, 2010 at 10:51 AM, Justin Riley <jtriley_at_mit.edu
> > <mailto:jtriley_at_mit.edu>> wrote:
> >
> > Hi Chuan,
> >
> > Have you checked whether or not the EBS volume ever gets to an
> > 'attached'
> > state using either ElasticFox or the AWS web console?
> >
> > Also, you can run starcluster in debug mode by passing -d option:
> >
> > $ starcluster -d -s
> >
> > This will do the same thing as -s only with debug output enabled.
> >
> > Would you mind sending me that output? Please be careful about any
> > sensitive
> > data in the output if you cc this list.
> >
> > Thanks,
> >
> > ~Justin
> >
> >
> > On Wednesday 17 March 2010 1:18:25 am chuan gao wrote:
> > > Hi Justin and Dan,
> > >
> > > Sorry for the late reply cuz I was tied up on a few things.
> > >
> > > To Justin: I tried it without mount EBS and I still have that
> > problem. I
> > > believe it's related to this though cuz it happened at the last
> > step of
> > > configuring the cluster(all machines have been started by that
> time).
> > >
> > > To Dan: I did tried reinstall paramiko (not in debug mode though
> > since I
> > > didn't find out how to do so,) and the system said installed
> > sucessfully. I
> > > also tried starting the cluster outside the directory without
> success.
> > >
> > > I also tried the same thing on another machine which has ubuntu on
> > it and I
> > > got the same error. I am not the administrator to that machine but
> > I guess
> > > it should be ubuntu 9.04.
> > >
> > > any ideas? thank you guys very much for handling this.
> > >
> > > Chuan
> > >
> > > On Tue, Mar 16, 2010 at 7:13 PM, Justin Riley <jtriley_at_mit.edu
> > <mailto:jtriley_at_mit.edu>> wrote:
> > > > Hi Chuan,
> > > >
> > > > Sorry to hear you're having issues with StarCluster. I just got
> > back in
> > > > town,
> > > > sorry for the delayed response.
> > > >
> > > > Dan, as always, thanks for responding :D
> > > >
> > > > Could you try launching a cluster without using the EBS volume
> > and let me
> > > > know
> > > > if it succeeds? Nicolas Pinto had a similar issue with
> > StarCluster and I
> > > > believe it only happened for him when using EBS.
> > > >
> > > > Thanks,
> > > >
> > > > ~Justin
> > > >
> > > > On Tuesday 16 March 2010 2:15:12 pm chuan gao wrote:
> > > > > I am using the standard AMI
> > > > > the OS is ubuntu 9.10
> > > > > yes. I can ssh into AMI with no problem. actually, the error
> > occured at
> > > >
> > > > the
> > > >
> > > > > last step when it is configuring NFS. I checked the AMI on
> > amazon and
> > > >
> > > > they
> > > >
> > > > > have been started. and my permanent volume has been mounted.
> > > > > concerning the error,
> > > > > build/bdist.linux-i686/egg/paramiko/sftp_client.py
> > > > > I looked into directory
> > StarCluster-0.90.1/build/bdist.linux-i686 and
> > > > > nothing is there. should there be egg/paramiko/sftp_client.py ?
> > > > > Thanks
> > > > >
> > > > > On Tue, Mar 16, 2010 at 1:46 PM, Dan Yamins <dyamins_at_gmail.com
> > <mailto:dyamins_at_gmail.com>> wrote:
> > > > > > Justin is on this list, so I'm sure he'll respond soon.
> > > > > >
> > > > > > Are you using the standard AMI, or one you built yourself?
> > > > > >
> > > > > > Also, what is your operating system and version?
> > > > > >
> > > > > > Also, can you ssh into instances of the AMI, independently of
> > > > > > starcluster?
> > > > > >
> > > > > > On Tue, Mar 16, 2010 at 1:17 PM, chuan gao
> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>>
> > > >
> > > > wrote:
> > > > > >> Thanks Dan, I am pretty sure that I got paramiko installed
> > > > > >> correctly. I'll work on it a bit more.
> > > > > >> will Justin have a chance to look at this email list and try
> to
> > > > > >> point out what could be the problem?
> > > > > >>
> > > > > >> On Tue, Mar 16, 2010 at 11:21 AM, Dan Yamins
> > <dyamins_at_gmail.com <mailto:dyamins_at_gmail.com>>
> > > >
> > > > wrote:
> > > > > >>> Hm. I'm not sure what the problem is (your version is
> fine).
> > > > > >>> Something is clearly wrong with either your paramiko
> > installation
> > > > > >>> or the way that starcluster is using it.
> > > > > >>>
> > > > > >>> Have you tried testing paramiko outside the context of
> > starcluster?
> > > > > >>> Try testing it for normal ssh usage. If that fails, then
> > you'll
> > > > > >>> probably have identified the problem. (I don't know if
> > you just
> > > > > >>> installed it, perhaps reinstallation would then help.)
> > > > > >>>
> > > > > >>> If paramiko works normally, then perhaps there's an
> > argument that
> > > > > >>> is being passed on line 109 of starcluster/ssh.py that is
> > supposed
> > > > > >>> to represent an existing file object, that somehow isn't
> being
> > > > > >>> properly created (probably earlier in the ssh cycle). In
> this
> > > > > >>> case, Justin
> > > >
> > > > (the
> > > >
> > > > > >>> creator of starcluster) should probably be the one to
> > address your
> > > > > >>> problem.
> > > > > >>>
> > > > > >>> Dan
> > > > > >>>
> > > > > >>> On Tue, Mar 16, 2010 at 10:57 AM, chuan gao
> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>>
> > > >
> > > > wrote:
> > > > > >>>> paramiko-1.7.6-py2.6.egg
> > > > > >>>> Thanks for replying.
> > > > > >>>>
> > > > > >>>> On Tue, Mar 16, 2010 at 9:02 AM, Dan Yamins
> > <dyamins_at_gmail.com <mailto:dyamins_at_gmail.com>>
> > > >
> > > > wrote:
> > > > > >>>>> On Tue, Mar 16, 2010 at 1:05 AM, chuan gao
> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>
> > > > >
> > > > >wrote:
> > > > > >>>>>> Here is the error message:
> > > > > >>>>>> >>> Installing Sun Grid Engine...
> > > > > >>>>>>
> > > > > >>>>>> [SNIP]
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> File
> > "build/bdist.linux-i686/egg/paramiko/sftp_client.py",
> > > > > >>>>>> line 675, in _read_response
> > > > > >>>>>> File
> > "build/bdist.linux-i686/egg/paramiko/sftp_client.py",
> > > > > >>>>>> line 701, in _convert_status
> > > > > >>>>>> IOError: [Errno 2] No such file
> > > > > >>>>>
> > > > > >>>>> What version of paramiko do you have installed?
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> _______________________________________________
> > > > > >>>>> Starcluster mailing list
> > > > > >>>>> Starcluster_at_mit.edu <mailto:Starcluster_at_mit.edu>
> > > > > >>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
> > >
> >
> >
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.14 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkuhFqwACgkQ4llAkMfDcrkcEwCgi0EdVkC1/5Ne578BNYkQYMpO
> +fYAn13tNTBQ2M2P29hgJoltz4nwj2DU
> =Vuxo
> -----END PGP SIGNATURE-----
>
Received on Wed Mar 17 2010 - 14:04:39 EDT