Re: [Starcluster] failed installing Sun Grid Engine...
yeah that's true, I guess what I really mean is the 64 bit extra large
instance type for certain AMI. anyway, glad that it's working. thank you
very much for helping on this.
Chuan
On Wed, Mar 17, 2010 at 4:08 PM, Dan Yamins <dyamins_at_gmail.com> wrote:
> Sorry to be dumb ... but I still don't understand why that's the "fastest
> processor on Amazon" ... I thought machine resources were a function of
> instance type, not AMI -- once you determined the platform. Won't the
> processor used for that fedora AMI be whatever is associated with the
> instance type you choose? Can't you choose any of the 64-bit instance
> types listed here: http://aws.amazon.com/ec2/instance-types <http://aws.amazon.com/ec2/instance-types/>? I would think that the "fastest processor on Amazon" would be
> *High-CPU Extra Large Instance* c1.xlarge, which has 20 times the compute
> power of the m1.small instance.
>
> Won' t Starcluster's 64-bit AMI work just fine on c1.xlarge or any of the
> other 64-bit-required instances? If so, does that not deliver the
> performance you were seeking?
>
>
>
>
>
>
>
> On Wed, Mar 17, 2010 at 3:14 PM, chuan gao <aggie.gao_at_gmail.com> wrote:
>
>> That is the amazon one with fedora installed. it is 64 bit. ami-86db39ef .
>> not surprising that it didn't work. :D
>> Sorry guys.
>>
>>
>> On Wed, Mar 17, 2010 at 2:38 PM, Dan Yamins <dyamins_at_gmail.com> wrote:
>>
>>> Which EC2 processors can you not use with either the 64-bit or 32 -bit
>>> starcluster AMI?
>>>
>>>
>>>
>>>
>>> On Wed, Mar 17, 2010 at 2:04 PM, chuan gao <aggie.gao_at_gmail.com> wrote:
>>>
>>>> Hi Justin,
>>>> Sorry for the confusion, I used the wrong AMI IDs. stupid mistake that
>>>> everybody can avoid by following the instructions. I saw it on the webpage
>>>> but still went ahead and changed that because I want to use the fastest
>>>> processor on amazon. :(
>>>> thank you very much for the software, truly nice work.
>>>> Chuan
>>>>
>>>>
>>>>
>>>> On Wed, Mar 17, 2010 at 1:51 PM, Justin Riley <jtriley_at_mit.edu> wrote:
>>>>
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA1
>>>>>
>>>>> Chuan,
>>>>>
>>>>> Responding to your other question concerning GSL and gotoBLAS:
>>>>>
>>>>> Installing these locally into a EBS volume is a perfectly reasonable
>>>>> solution. In fact, this is how I envisioned most people storing their
>>>>> software/data on StarCluster.
>>>>>
>>>>> The other way to do this is to launch an instance, install the software
>>>>> globally (using apt-get and/or source packages), and then rebundle the
>>>>> AMI. You could then tell starcluster to use this new AMI in your config
>>>>> file.
>>>>>
>>>>> I will add GSL to the next version of StarCluster's AMI. goto blas must
>>>>> be custom compiled. I'll have a look at this and if it's not too much
>>>>> work I'll consider adding it in.
>>>>>
>>>>> Hope that helps,
>>>>>
>>>>> ~Justin
>>>>>
>>>>> On 03/17/2010 01:35 PM, chuan gao wrote:
>>>>> > Hi Justin and Dan,
>>>>> > Sorry for all the trouble, it's all my fault, it turned out that I
>>>>> used
>>>>> > the wrong AMI type. it worked like a charm after I changed that. I do
>>>>> > have another question though, I need to use GSL and gotoBLAS for my
>>>>> > computation, which I didn't find it installed on the cluster, I am
>>>>> > thinking about compiling these locally on my EBS volume and include
>>>>> the
>>>>> > library in my code. is there any other better way?
>>>>> > Thank you all guys for the help!
>>>>> > Chuan
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Wed, Mar 17, 2010 at 10:51 AM, Justin Riley <jtriley_at_mit.edu
>>>>> > <mailto:jtriley_at_mit.edu>> wrote:
>>>>> >
>>>>> > Hi Chuan,
>>>>> >
>>>>> > Have you checked whether or not the EBS volume ever gets to an
>>>>> > 'attached'
>>>>> > state using either ElasticFox or the AWS web console?
>>>>> >
>>>>> > Also, you can run starcluster in debug mode by passing -d option:
>>>>> >
>>>>> > $ starcluster -d -s
>>>>> >
>>>>> > This will do the same thing as -s only with debug output enabled.
>>>>> >
>>>>> > Would you mind sending me that output? Please be careful about
>>>>> any
>>>>> > sensitive
>>>>> > data in the output if you cc this list.
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > ~Justin
>>>>> >
>>>>> >
>>>>> > On Wednesday 17 March 2010 1:18:25 am chuan gao wrote:
>>>>> > > Hi Justin and Dan,
>>>>> > >
>>>>> > > Sorry for the late reply cuz I was tied up on a few things.
>>>>> > >
>>>>> > > To Justin: I tried it without mount EBS and I still have that
>>>>> > problem. I
>>>>> > > believe it's related to this though cuz it happened at the last
>>>>> > step of
>>>>> > > configuring the cluster(all machines have been started by that
>>>>> time).
>>>>> > >
>>>>> > > To Dan: I did tried reinstall paramiko (not in debug mode
>>>>> though
>>>>> > since I
>>>>> > > didn't find out how to do so,) and the system said installed
>>>>> > sucessfully. I
>>>>> > > also tried starting the cluster outside the directory without
>>>>> success.
>>>>> > >
>>>>> > > I also tried the same thing on another machine which has ubuntu
>>>>> on
>>>>> > it and I
>>>>> > > got the same error. I am not the administrator to that machine
>>>>> but
>>>>> > I guess
>>>>> > > it should be ubuntu 9.04.
>>>>> > >
>>>>> > > any ideas? thank you guys very much for handling this.
>>>>> > >
>>>>> > > Chuan
>>>>> > >
>>>>> > > On Tue, Mar 16, 2010 at 7:13 PM, Justin Riley <jtriley_at_mit.edu
>>>>> > <mailto:jtriley_at_mit.edu>> wrote:
>>>>> > > > Hi Chuan,
>>>>> > > >
>>>>> > > > Sorry to hear you're having issues with StarCluster. I just
>>>>> got
>>>>> > back in
>>>>> > > > town,
>>>>> > > > sorry for the delayed response.
>>>>> > > >
>>>>> > > > Dan, as always, thanks for responding :D
>>>>> > > >
>>>>> > > > Could you try launching a cluster without using the EBS
>>>>> volume
>>>>> > and let me
>>>>> > > > know
>>>>> > > > if it succeeds? Nicolas Pinto had a similar issue with
>>>>> > StarCluster and I
>>>>> > > > believe it only happened for him when using EBS.
>>>>> > > >
>>>>> > > > Thanks,
>>>>> > > >
>>>>> > > > ~Justin
>>>>> > > >
>>>>> > > > On Tuesday 16 March 2010 2:15:12 pm chuan gao wrote:
>>>>> > > > > I am using the standard AMI
>>>>> > > > > the OS is ubuntu 9.10
>>>>> > > > > yes. I can ssh into AMI with no problem. actually, the
>>>>> error
>>>>> > occured at
>>>>> > > >
>>>>> > > > the
>>>>> > > >
>>>>> > > > > last step when it is configuring NFS. I checked the AMI on
>>>>> > amazon and
>>>>> > > >
>>>>> > > > they
>>>>> > > >
>>>>> > > > > have been started. and my permanent volume has been
>>>>> mounted.
>>>>> > > > > concerning the error,
>>>>> > > > > build/bdist.linux-i686/egg/paramiko/sftp_client.py
>>>>> > > > > I looked into directory
>>>>> > StarCluster-0.90.1/build/bdist.linux-i686 and
>>>>> > > > > nothing is there. should there be
>>>>> egg/paramiko/sftp_client.py ?
>>>>> > > > > Thanks
>>>>> > > > >
>>>>> > > > > On Tue, Mar 16, 2010 at 1:46 PM, Dan Yamins <
>>>>> dyamins_at_gmail.com
>>>>> > <mailto:dyamins_at_gmail.com>> wrote:
>>>>> > > > > > Justin is on this list, so I'm sure he'll respond soon.
>>>>> > > > > >
>>>>> > > > > > Are you using the standard AMI, or one you built
>>>>> yourself?
>>>>> > > > > >
>>>>> > > > > > Also, what is your operating system and version?
>>>>> > > > > >
>>>>> > > > > > Also, can you ssh into instances of the AMI,
>>>>> independently of
>>>>> > > > > > starcluster?
>>>>> > > > > >
>>>>> > > > > > On Tue, Mar 16, 2010 at 1:17 PM, chuan gao
>>>>> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>>
>>>>> > > >
>>>>> > > > wrote:
>>>>> > > > > >> Thanks Dan, I am pretty sure that I got paramiko
>>>>> installed
>>>>> > > > > >> correctly. I'll work on it a bit more.
>>>>> > > > > >> will Justin have a chance to look at this email list and
>>>>> try to
>>>>> > > > > >> point out what could be the problem?
>>>>> > > > > >>
>>>>> > > > > >> On Tue, Mar 16, 2010 at 11:21 AM, Dan Yamins
>>>>> > <dyamins_at_gmail.com <mailto:dyamins_at_gmail.com>>
>>>>> > > >
>>>>> > > > wrote:
>>>>> > > > > >>> Hm. I'm not sure what the problem is (your version is
>>>>> fine).
>>>>> > > > > >>> Something is clearly wrong with either your paramiko
>>>>> > installation
>>>>> > > > > >>> or the way that starcluster is using it.
>>>>> > > > > >>>
>>>>> > > > > >>> Have you tried testing paramiko outside the context of
>>>>> > starcluster?
>>>>> > > > > >>> Try testing it for normal ssh usage. If that fails,
>>>>> then
>>>>> > you'll
>>>>> > > > > >>> probably have identified the problem. (I don't know
>>>>> if
>>>>> > you just
>>>>> > > > > >>> installed it, perhaps reinstallation would then help.)
>>>>> > > > > >>>
>>>>> > > > > >>> If paramiko works normally, then perhaps there's an
>>>>> > argument that
>>>>> > > > > >>> is being passed on line 109 of starcluster/ssh.py that
>>>>> is
>>>>> > supposed
>>>>> > > > > >>> to represent an existing file object, that somehow
>>>>> isn't being
>>>>> > > > > >>> properly created (probably earlier in the ssh cycle).
>>>>> In this
>>>>> > > > > >>> case, Justin
>>>>> > > >
>>>>> > > > (the
>>>>> > > >
>>>>> > > > > >>> creator of starcluster) should probably be the one to
>>>>> > address your
>>>>> > > > > >>> problem.
>>>>> > > > > >>>
>>>>> > > > > >>> Dan
>>>>> > > > > >>>
>>>>> > > > > >>> On Tue, Mar 16, 2010 at 10:57 AM, chuan gao
>>>>> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>>
>>>>> > > >
>>>>> > > > wrote:
>>>>> > > > > >>>> paramiko-1.7.6-py2.6.egg
>>>>> > > > > >>>> Thanks for replying.
>>>>> > > > > >>>>
>>>>> > > > > >>>> On Tue, Mar 16, 2010 at 9:02 AM, Dan Yamins
>>>>> > <dyamins_at_gmail.com <mailto:dyamins_at_gmail.com>>
>>>>> > > >
>>>>> > > > wrote:
>>>>> > > > > >>>>> On Tue, Mar 16, 2010 at 1:05 AM, chuan gao
>>>>> > <aggie.gao_at_gmail.com <mailto:aggie.gao_at_gmail.com>
>>>>> > > > >
>>>>> > > > >wrote:
>>>>> > > > > >>>>>> Here is the error message:
>>>>> > > > > >>>>>> >>> Installing Sun Grid Engine...
>>>>> > > > > >>>>>>
>>>>> > > > > >>>>>> [SNIP]
>>>>> > > > > >>>>>>
>>>>> > > > > >>>>>>
>>>>> > > > > >>>>>> File
>>>>> > "build/bdist.linux-i686/egg/paramiko/sftp_client.py",
>>>>> > > > > >>>>>> line 675, in _read_response
>>>>> > > > > >>>>>> File
>>>>> > "build/bdist.linux-i686/egg/paramiko/sftp_client.py",
>>>>> > > > > >>>>>> line 701, in _convert_status
>>>>> > > > > >>>>>> IOError: [Errno 2] No such file
>>>>> > > > > >>>>>
>>>>> > > > > >>>>> What version of paramiko do you have installed?
>>>>> > > > > >>>>>
>>>>> > > > > >>>>>
>>>>> > > > > >>>>> _______________________________________________
>>>>> > > > > >>>>> Starcluster mailing list
>>>>> > > > > >>>>> Starcluster_at_mit.edu <mailto:Starcluster_at_mit.edu>
>>>>> > > > > >>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>>>> > >
>>>>> >
>>>>> >
>>>>>
>>>>> -----BEGIN PGP SIGNATURE-----
>>>>> Version: GnuPG v2.0.14 (GNU/Linux)
>>>>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>>>>
>>>>> iEYEARECAAYFAkuhFqwACgkQ4llAkMfDcrkcEwCgi0EdVkC1/5Ne578BNYkQYMpO
>>>>> +fYAn13tNTBQ2M2P29hgJoltz4nwj2DU
>>>>> =Vuxo
>>>>> -----END PGP SIGNATURE-----
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Starcluster mailing list
>>>> Starcluster_at_mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>>>
>>>>
>>>
>>
>
Received on Wed Mar 17 2010 - 16:43:42 EDT
This archive was generated by
hypermail 2.3.0.