Re: Issues with using an EBS volume

From: Rayson Ho <no email>
Date: Mon, 29 Jul 2013 00:39:39 -0400

Hi,

Some background info: The EBS mounting bug only affects launching from
Windows, which is what you are running...

To fix the bug without upgrading to 0.94, you can merge the fix by
hand in 0.93.3:

https://github.com/jtriley/StarCluster/commit/64fa4b8fac7318c7a77d9d769a5826fa76e62341

It is a 2-line fix, basically changing "os.path.join(path, f)" to
"posixpath.join(path, f)". The reason this is needed because
os.path.join() uses the local OS's path separator to join the path,
and on Windows it is "\", so the result becomes "/dev\sdz" instead of
"/dev/sdz". Launching on Unix or Linux it is only an issue because the
path separator is "/".

It is a safe and simple 2-line fix, and should fix the issue you are
encountering...

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/


On Mon, Jul 29, 2013 at 12:25 AM, Jacob Barhak <jacob.barhak_at_gmail.com> wrote:
> Thanks Rayson,
>
> If this is the case, then there is no quick solution to the disk space issue
> that you helped identify this last week.
>
> I cannot install version 0.94 since easy install does not work well in its
> current form. See the following link for a full report:
> http://star.mit.edu/cluster/mlarchives/1798.html
>
> Alternatively, the solution you suggested of exporting the volumes from the
> master to share these with the nodes would help, yet it involves many
> instructions made manually after the cluster is up and running. So as it is,
> it is too cumbersome to implement and for sure hard to document so someone
> inexperienced can follow easily.
>
> The quickest least painful solution that may be open for me now would be to
> try the plugin pointed to by scrappythekangaroo about 3 months ago in:
> https://github.com/jtriley/StarCluster/issues/44
>
> So unless you can fix the 0.94 installation quick, I will have to try the
> plugin solution in hope it fully resolves the issue.
>
> I hope I can reach an easy and quick solution for increasing the available
> disk space.
>
> Jacob
>
>
> Sent from my iPhone
>
> On Jul 28, 2013, at 10:04 PM, Rayson Ho <raysonlogin_at_gmail.com> wrote:
>
> I believe you are hitting this bug:
>
> https://github.com/jtriley/StarCluster/pull/147
>
> And I verified that 0.93.3 does not have the fix, and 0.94 or the dev
> version should be fine.
>
> Rayson
>
> ==================================================
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
>
>
> On Sun, Jul 28, 2013 at 3:58 AM, Jacob Barhak <jacob.barhak_at_gmail.com>
> wrote:
>
> Hello,
>
>
> Perhaps someone in the group can help out with using an EBS volume.
>
>
> I created an EBS volume and want to launch a cluster that uses it. I am
>
> doing this in an attempt to solve the disk limitation problem I encountered
>
> and is reported in this list at:
>
> http://star.mit.edu/cluster/mlarchives/1795.html
>
>
> However, I encounter the following error during starting the cluster.
>
>
> Setting up the cluster...
>
> Attaching volume vol-f0ae61cb to master node on /dev/sdz ...
>
> Configuring hostnames...
>
> 2/2 ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
>
> 100%
>
> !!! ERROR - volume has more than one partition, please specify which
>
> partition t
>
> o use (e.g. partition=0, partition=1, etc.) in the volume's config
>
>
> The full transcript is attached.
>
>
> I have the following lines in my configuration file:
>
>
> VOLUMES = mydata
>
> ...
>
> [volume mydata]
>
> VOLUME_ID = vol-f0ae61cb
>
> MOUNT_PATH = /mydata
>
>
> I tried adding PARTITION = 0, and PARTITION = 1 to the volume definition in
>
> the configuration file, yet nothing seems to fix this.
>
>
> I also tried using "starcluster createvolume" to create the volume yet I
>
> encountered the same issue as above in whatever method I created the 20gb
>
> volume.
>
>
> I am using ami-a4d64194 for my node images. My configuration and plugin are
>
> derived from the files in https://github.com/ContinuumIO/anaconda-ec2
>
>
> I am operating in us-west-2. I am using starcluster 0.93.3 with windows 7
>
>
> If anyone has a quick solution or diagnosis test, I will appreciate the
>
> feedback.
>
>
> Jacob
>
>
>
>
>
> _______________________________________________
>
> StarCluster mailing list
>
> StarCluster_at_mit.edu
>
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
Received on Mon Jul 29 2013 - 00:39:42 EDT