Re: Ganglia Plugin ?

From: Ryan G <no email>
Date: Mon, 25 May 2015 13:04:47 -0400

Very nice. I'll have to try this out this week when I'm back at work. Thanks for putting the effort in!

Sent from my iPhone

> On May 25, 2015, at 10:42 AM, Sergio Mafra <sergiohmafra_at_gmail.com> wrote:
>
> Just forgot to add that you need to open port 80 in StarCluster security groups in order to allow to access Ganglia via web.
> So, in config file you need to add:
>
> [permission www]
> # open port 80 to the world
> from_port = 80
> to_port = 80
>
> and in the [Cluster ....]
> permissions = www
>
>
> Good luck to all,
>
> Sergio
>
>
>> On Mon, May 25, 2015 at 11:14 AM, Sergio Mafra <sergiohmafra_at_gmail.com> wrote:
>> Hi SC_Folks,
>>
>> I´ve made a little research and found interesting things that helped to build a plugin of configuring Ganglia to run with StarCluster.
>> Here comes what I did in case if you´re interested:
>>
>> 1. First, I used the Shared StarCluster AMI Ubuntu 14.04. It can be found on Public AMIs. (us-east-1 -> ami-38b99850).
>> 2. This tutorial was great: https://www.digitalocean.com/community/tutorials/introduction-to-ganglia-on-ubuntu-14-04
>> 3. I´ve installed these apps in base AMI using: sudo apt-get install -y ganglia-monitor rrdtool gmetad ganglia-webfrontend
>> 4. Ganglia needs to be configured using two config files in /etc/config: gmetad.conf and gmond.conf
>> 5. The gmetad.conf needs to have the name of the cluster you intend to monitor. I used a fixed name (StarCluster) but you can chance in the future with the name tag of the cluster you´re provisioning, So this is what I´ve chnaged:
>> data_souce "StarCluster" 60 localhost
>> 6. The gmond.conf needs to changed in 3 sections:
>> a) Cluster section: Note that I´ve change the unspecified to StarCluster (same name in the gmetad.conf)
>> cluster {
>> name = "StarCluster"
>> owner = "unspecified"
>> latlong = "unspecified"
>> url = "unspecified"
>> }
>>
>> b) udp_send_channel section: You have to comment the mcast_join line and add the ip of the cluster master. The plugin will handle this. So I add the line "host = master_ip". This will be the same if the node is master or slave.
>> udp_send_channel {
>> # mcast_join = 239.2.11.71
>> host = master_ip
>> port = 8649
>> ttl = 1
>> }
>>
>> c) udp_recv_channel: You will have to have two kinds of configuration regarding the master and the slaves. In master, the section will be the following: The mcast_join line commented and bind line too.
>>
>> > Master node:
>> udp_recv_channel {
>> # mcast_join = 239.2.11.71
>> port = 8649
>> # bind = 239.2.11.71
>> }
>>
>> > Slave node: All section must be commented:
>> #udp_recv_channel {
>> # mcast_join = 239.2.11.71
>> # port = 8649
>> # bind = 239.2.11.71
>> #}
>>
>> So I decided to create two gmond.conf in base AMI. The first one will be used to all Slaves and the second will be used to Master.
>> This will be changed in Plugin time.
>>
>> I´ve prepared the gmond.conf with a) , b) and c) Slave node and gmond_master.conf with a), b) and c) Master node.
>>
>> With the ganglia plugin:
>>
>> 1. Gets the private ip of the Master node (master_ip)
>> 2. Checks if the node is master.
>> 3. If true, replaces the gmond.conf file from gmond_server.conf, replaces the master_ip in this file and restart the services needed.
>> 4. If false, replaces the master_ip in gmond.conf and restart the ganglia-monitor service.
>>
>>
>> from starcluster.clustersetup import ClusterSetup
>> from starcluster.logger import log
>>
>> class ganglia(ClusterSetup):
>> def __init__(self):
>> log.debug('Installing Ganglia...')
>> def run(self, nodes, master, user, user_shell, volumes):
>> log.info("Installing Ganglia in all nodes")
>> master_ip = None
>> for node in nodes:
>> if node.is_master():
>> master_ip = node.private_ip_address
>> master.ssh.execute('mv /etc/ganglia/gmond_server.conf /etc/ganglia/gmond.conf')
>> master.ssh.execute("sed s/master_ip/%s/ /etc/ganglia/gmond.conf > ar&&mv ar /etc/ganglia/gmond.conf" % master_ip)
>> master.ssh.execute('service ganglia-monitor restart && sudo service gmetad restart && sudo service apache2 restart')
>> else:
>> node.ssh.execute("sed s/master_ip/%s/ /etc/ganglia/gmond.conf > ar&&mv ar /etc/ganglia/gmond.conf" % master_ip)
>> node.ssh.execute('service ganglia-monitor restart')
>>
>>
>> That´s it.. You now can check on http://<your_master_ip_address>/ganglia and monitor all cluster´s stuff.
>>
>> Hope this helps you,
>>
>> Sérgio Mafra
>>
>>> On Fri, May 22, 2015 at 8:47 AM, Sergio Mafra <sergiohmafra_at_gmail.com> wrote:
>>> Hi SC_Folks,
>>>
>>> Is there any plugin around developed to install Ganglia on StarCluster?
>>>
>>> All best,
>>>
>>> Sergio
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Mon May 25 2015 - 13:04:53 EDT