Re: adding parameters to create_volume function - encryption

From: ronen artzi <no email>
Date: Thu, 3 Jul 2014 20:48:22 -0400

I had a niche case where ebs were attached to master and used as GlusterFS
bricks and than we shared that GlusterFS volume across starcluster nodes.

So the ebs is encrypted now. But the connection between clients to
glusterfs nodes is not secured yet.

But it achieved my immediate goal for this test.

Thanks for all your help.

Ronen
On Jul 3, 2014 8:25 PM, "Rayson Ho" <raysonlogin_at_gmail.com> wrote:

> On Thu, Jul 3, 2014 at 1:55 AM, ronen artzi <ronen.artzi_at_gmail.com> wrote:
>
>>
>> This one does not solve (?) the protocol encryption ( hence the
>> encryption on the fly challenge but provide encryption at rest ).
>>
>
>
> EBS provides encryption at rest, but according to Amazon data is already
> encrypted when it leaves the instance.
>
> "... your data is encrypted before it leaves the EC2 instance. In order
> to be able to do this efficiently and with low latency, the EBS encryption
> feature is only available on EC2's M3, C3, R3, CR1, G2, and I2 instances":
>
> http://aws.amazon.com/blogs/aws/protect-your-data-with-new-ebs-encryption/
>
> However, I do get what you mean -- you want to encrypt the data before it
> reaches EBS or AWS, but then you most likely don't need to change the
> StarCluster code to pass encrypted=True to boto's create_volume(), as EBS
> encryption is not that useful in your use case.
>
> Rayson
>
> ==================================================
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>
>
>
>
>
>
>>
>>
>>
>>
>> On Wed, Jul 2, 2014 at 11:58 PM, Rayson Ho <raysonlogin_at_gmail.com> wrote:
>>
>>> Hmm, I was not very clear in my previous email...
>>>
>>> create_volume() called in volume.py is defined in the StarCluster's
>>> EasyAWS class (awsutils.py). So if you want to pass encypted=True to boto's
>>> create_volume(), you will need to add the extra parameter in awsutils.py:
>>>
>>> def create_volume(self, size, zone, snapshot_id=None):
>>> ...
>>> return self.conn.create_volume(size, zone, snapshot_id,
>>> encypted=True)
>>>
>>> So you just need to work your way up the call chain so that you can
>>> determine the value passed in by the user from the command line.
>>>
>>> (When your code is general purpose enough to be included in the
>>> StarCluster repo, create a pull request in github.)
>>>
>>> Rayson
>>>
>>> ==================================================
>>> Open Grid Scheduler - The Official Open Source Grid Engine
>>> http://gridscheduler.sourceforge.net/
>>> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>>>
>>>
>>> On Wed, Jul 2, 2014 at 11:07 PM, ronen artzi <ronen.artzi_at_gmail.com>
>>> wrote:
>>>
>>>> Thanks Rayson for your prompt response.
>>>>
>>>> I am running on the "Cluster Launcher/controller" node with Boto
>>>> 2.30.0 installed.
>>>> in my code I am using (encrypt is the option I am getting from the user
>>>> : True/False):
>>>>
>>>> log.info(">>>>>>>>>>>> Creating new Volume with
>>>> encrypt option : %s" % encrypt)
>>>> vol = self.ec2.create_volume(size, zone,
>>>> None,None,None,encrypt,None)
>>>> log.info("New volume id: %s" % vol.id)
>>>>
>>>> The error I am getting :
>>>>
>>>> File "/home/ubuntu/.starcluster/plugins/glusterfsencrypt.py", line
>>>> 110, in run
>>>> volume = self._create_ec2_volume(self.name, self.ebs_size, zone ,
>>>> self.encrypt)
>>>>
>>>> File "/home/ubuntu/.starcluster/plugins/glusterfsencrypt.py", line 363,
>>>> in _create_ec2_volume
>>>> vol = self.ec2.create_volume(size, zone, None,None,None,encrypt)
>>>>
>>>> TypeError: create_volume() takes at most 4 arguments (8 given)
>>>>
>>>>
>>>>
>>>> On another note : when I am calling the create_function with
>>>> encypted=encrypt
>>>>
>>>> i am getting :
>>>>
>>>> vol = self.ec2.create_volume(size, zone, None,encrypted=encrypt)
>>>> TypeError: create_volume() got an unexpected keyword argument
>>>> 'encrypted'
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jul 2, 2014 at 6:35 PM, Rayson Ho <raysonlogin_at_gmail.com>
>>>> wrote:
>>>>
>>>>> StarCluster only needs the first 3 function parameters, thus it relies
>>>>> on the default parameters for boto's create_volume() function. If all you
>>>>> want is to pass in the encrypted boolean, then you can add encrypted=True
>>>>> to the list of parameters for create_volume().
>>>>>
>>>>> For you code to be general purpose enough to be included, you will
>>>>> need to add logic so that is does not always pass the encrypted=True bool
>>>>> into the function all the time, and instead read the user's choice from the
>>>>> command line (see createvolume.py).
>>>>>
>>>>> Rayson
>>>>>
>>>>> ==================================================
>>>>> Open Grid Scheduler - The Official Open Source Grid Engine
>>>>> http://gridscheduler.sourceforge.net/
>>>>> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>>>>>
>>>>>
>>>>> On Wed, Jul 2, 2014 at 5:42 PM, ronen artzi <ronen.artzi_at_gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I am trying to extend a glusterfs plugin we have , and would like
>>>>>> to introduce the options of encrypting the ESB bricks.
>>>>>>
>>>>>> I'd like to be able to use the full parameter list coming with ec2
>>>>>> fro volume creation :
>>>>>>
>>>>>> create_volume(*size*, *zone*, *snapshot=None*, *volume_type=None*,
>>>>>> *iops=None*, *encrypted=False*, *dry_run=False*)
>>>>>>
>>>>>> Being very new to the starcluster code base I am not sure what areas
>>>>>> are dealign with such volume creation.
>>>>>>
>>>>>> it seems that under star cluster python packages area , volume.py
>>>>>> support only the first 4 parameters.
>>>>>>
>>>>>> Any insights here are appreciated.
>>>>>> --
>>>>>> Ronen
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> StarCluster mailing list
>>>>>> StarCluster_at_mit.edu
>>>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> - Ronen
>>>> Cell: +1-508-308-2417
>>>> My profiles: [image: LinkedIn]
>>>> <http://www.linkedin.com/pub/ronen-artzi/0/30/4b9> [image: Tungle.me]
>>>> <http://tungle.me/ronenartzi>
>>>>
>>>>
>>>
>>
>>
>> --
>> - Ronen
>> Cell: +1-508-308-2417
>> My profiles: [image: LinkedIn]
>> <http://www.linkedin.com/pub/ronen-artzi/0/30/4b9> [image: Tungle.me]
>> <http://tungle.me/ronenartzi>
>>
>>
>
Received on Thu Jul 03 2014 - 20:48:24 EDT