Re: dealing with security groups

From: Justin Riley <no email>
Date: Sun, 19 Dec 2010 02:03:49 -0500

Hi Igor,
> I was really impressed how easy it was to use StarCluster - thanks!
> :) but I can't figure out how to work around EC2 security groups to
> make it work for us.
Thanks, glad you like it :D
> We need to be able to mount an NFS share (which is an EBS volume). I
> understand that StarCluster can attach and share an EBS volume but we
> already have our EBS volumes attached and used by other nodes. So
> instead of attaching an EBS volume we need to mount an existing NFS
> share. Does this make sense? I wonder how many people have a similar
> set up? (We also need access to our NIS server and possibly other
> services)

This makes sense and you'll certainly have to manage mounting the
external NFS share in a plugin.

> By default, new EC2 security groups have all of the ports closed
> (except ssh) so one thing I could do is to open the relevant ports at
> cluster startup.

Yes, you'll also have to add the relevant lines to /etc/fstab and run
the 'mount' command for each share on each node off the top of my head.
Modifying fstab and running 'mount' will need to be done in a plugin.

> Alternatively, I'd rather not deal with security groups at all.
> 'default' would work fine for us. Unfortunately, it looks like the
> code assumes that a cluster has its own security group with a certain
> name. So I guess that's not an option, right?

Currently each cluster has it's own security group. This group is mostly
used for accounting but is also useful if you want to apply a firewall
to each individual cluster launched. The latest github code has support
for applying security group permissions after the group is created which
might meet your needs, however, currently the permission settings do not
support specifying group-group permissions. I can add this fairly easily
though...

Also, this use case suggests it would be handy to be able to specify
additional security groups (such as 'default') to add each of the
cluster nodes to in addition to their own (_at_sc-*). This would remove the
need to add permissions since they'd all be in a common group (e.g.
default).

> If I follow the first approach, I would need to do something like this:
> ec2-authorize default -p <nfs-port> -o _at_sc-mycluster
> right? or do I also need to allow access from sc-cluster to default?
>
> Do I need to revoke these permissions when the cluster shuts down or
> will EC2 take care of that as long as StarCluster deletes the security
> group (does it?)

When the cluster shuts down, StarCluster deletes the cluster's security
group so I'd imagine that would take care of the group-group permission
but we'll need to test this to make sure that's the case. If the group
is deleted the rule can't apply but it may still be defined. If this is
the case the rule would have to be cleaned up.

~Justin
Received on Sun Dec 19 2010 - 02:03:31 EST