On a related topic, does anyone know how the load balancing feature tracks the cluster and its compute nodes? I have gotten into situations where listclusters correctly reports that a cluster and its nodes are running (I can ssh into master and the nodes, etc.); however, loadbalance reports that the cluster is not running and refuses to balance the cluster.
From: starcluster-bounces_at_mit.edu [mailto:starcluster-bounces_at_mit.edu] On Behalf Of Eduardo Gurgel Valente
Sent: Tuesday, March 10, 2015 2:08 PM
To: Nicholas Chammas
Subject: Re: [StarCluster] How does StarCluster track the clusters it's managing?
Look at the security group it creates. It follows a naming convention. In addition there are tags with encrypted information at play.
On Mon, Mar 9, 2015 at 11:16 PM, Nicholas Chammas <nicholas.chammas_at_gmail.com<mailto:nicholas.chammas_at_gmail.com>> wrote:
At this point in the StarCluster demo video<http://youtu.be/vC3lJcPq1FY?t=7m20s
>, the presenter runs the following command to list all the clusters being managed by StarCluster:
How does StarCluster track all the clusters it’s managing? Is it through the use of EC2 instance tags? A pointer to the relevant code would also be helpful.
I’m looking to implement a feature similar to listclusters but for spark-ec2<http://spark.apache.org/docs/1.2.1/ec2-scripts.html
>. Tagging seems like the way to go to do that, but we had some issues with it<https://issues.apache.org/jira/browse/SPARK-3332
> when we used it with spark-ec2.
So I’m curious to know how StarCluster did things.
StarCluster mailing list
Received on Tue Mar 10 2015 - 16:04:22 EDT