Re: working with EBS volumes

From: Justin Riley <no email>
Date: Mon, 18 Oct 2010 10:47:44 -0400

Hi Austin,

It's clear now that the user generation code doesn't get a chance to see
that there's a /home/ubuntu folder because the volume is mounted on top
of the existing /home (containing the actual ubuntu folder). I'm going
to update the user-generation code to inspect /etc/passwd and
/etc/shadow rather than /home folders in the case that
/home/CLUSTER_USER folder does not exist already. That should fix the issue.

~Justin

On 10/15/2010 09:43 AM, Austin Godber wrote:
> Hi Justin,
> I have verified this issue. Its not quite what it seems. The real
> issue at hand is that the CLUSTER_USER gets created with the same UID as
> the existing ubuntu user. So anything that uses a UID is going to
> report back the 'ubuntu' user rather than the CLUSTER_USER. Starcluster
> needs to get rid of or replace the ubuntu user or use a new starting
> UID/GID. But we would have to be careful to make sure that sudo still
> works, since that is indexed by name rather than uid. In this case, its
> just luck that sudo works for the CLUSTER_USER since the UID matches the
> ubuntu user. The GIDs collide as well.
>
> To reproduce, do the following:
> pip install starcluster
> pip install boto=1.9b # to get the working version of boto
> starcluster createvolume 1 use-east-1a
> # edit your .starcluster/config to use the new volume and mount it at /home
> starcluster start smallcluster
>
> Here is a variety of the consequences:
>
> $ grep 1000 /etc/passwd
> ubuntu:x:1000:1000:Ubuntu,,,:/home/ubuntu:/bin/bash
> sgeadmin:x:1000:1000::/home/sgeadmin:/bin/bash
>
> $ grep 1000 /etc/group
> ubuntu:x:1000:
> sgeadmin:x:1000:
>
> ubuntu_at_domU-12-31-38-04-C8-B5:~$ w
> 13:30:30 up 19 min, 1 user, load average: 0.00, 0.00, 0.00
> USER TTY FROM LOGIN_at_ IDLE JCPU PCPU WHAT
> sgeadmin pts/1 oh-76-1-37-133.d 13:29 0.00s 0.25s 0.00s w
>
> ubuntu_at_domU-12-31-38-04-C8-B5:~$ id
> uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu)
>
> ubuntu_at_domU-12-31-38-04-C8-B5:~$ ls -l /home/
> total 20
> drwx------ 2 root root 16384 2010-10-15 13:06 lost+found
> drwxr-xr-x 4 ubuntu ubuntu 4096 2010-10-15 13:20 sgeadmin
>
> Austin
>
> On 10/14/2010 04:58 PM, Justin Riley wrote:
>>
> Hi Adam,
>
> Sorry for the late response.
>
> There is some magic that occurs when creating users in order to avoid
> having to chmod /home folders which might contain hundreds of
> gigabytes of data. Basically, StarCluster inspects the top level
> folders under the /home folder and if the CLUSTER_USER's home folder
> already exists, then the CLUSTER_USER is created with the same uid/gid
> as the existing home folder to avoid a recursive chmod. Otherwise,
> StarCluster looks at the uid/gid of the other directories in /home and
> chooses the highest gid/uid combo plus 1 to be the uid/gid for the
> CLUSTER_USER. If that calculation ends up with a uid/gid less than
> 1000 then it defaults to 1000 for the gid/uid of CLUSTER_USER.
>
> A couple questions that might help me to understand what happened:
>
> 1. I'm assuming you must have had MOUNT_PATH=/home for the volume in
> your cluster template's VOLUME list, correct?
>
> 2. Did your volume already contain a 'sgeadmin' folder at the root of
> the volume?
>
> 3. What does "ls -l" look like on the root of the volume that exhibits
> this behavior?
>
> Also, you will find useful information about the uid/gid chosen by
> StarCluster for the CLUSTER_USER in your debug file:
>
> /tmp/starcluster-debug-<your_username>.log
>
> (if you're on mac, this file will be in the directory returned by
> "python -c 'import tempfile; print tempfile.gettempdir()'")
>
> Just grepping for gid or uid in your log file(s) should print out the
> relevant messages: "grep -ri gid /tmp/starcluster-*"
>
> ~Justin
>
>
> On 10/3/10 5:19 PM, Adam Marsh wrote:
>
>
>> I've had some challenges to getting SC running correctly
> when EBS
>
>> volumes are mounted to the head node during configuration.
> I
>
>> initially setup the EBS volumes with database files by
> first
>
>> configuring and mounting the volume on any available EC2 VM
> I had
>
>> running at the time. By default, most of the time I was
> working as
>
>> user 'ubuntu'. However, whenever an EBS volume with files
> or folders
>
>> having 'ubuntu' as the owner and group were included in the
> VOLUMES
>
>> list of the SC config file and were mounted during setup to
> the head
>
>> node, two odd things occurred:
>
>> 1. when the cluster_user account was setup by SC (like
> 'sgeadmin'),
>
>> the owner and user of the /sgeadmin folder under /home was
> 'ubuntu';
>
>> 2. connecting via ssh to the sgeadmin account always
> defaulted to
>
>> logging in to the 'ubuntu' user account.
>
>
>
>> I worked around the problem by changing own/grp settings on
> all EBS
>
>> folders/files to the cluster_user name used in the config
> file.
>
>> All works fine now.
>
>
>
>> Is this just a rare instance of SC system behavior? If not,
> is there
>
>> a better way to prepare EBS volumes for use with SC to
> avoid own/grp
>
>> conflicts?
>
>
>
>> Thanks,
>
>
>
>> <Adam
>
>
>
>
>
>>
>>
_______________________________________________
StarCluster mailing list
StarCluster_at_mit.edu
http://mailman.mit.edu/mailman/listinfo/starcluster

> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Mon Oct 18 2010 - 10:47:48 EDT