Hi Vasisht,
Thanks for the Ubuntu 16.04 clarification. Also, I didn’t know about cfncluster. A quick search indicates that it supports elastic clusters to scale the cluster and has sge, torque, and slurm support out of the box.
·
https://aws.amazon.com/getting-started/projects/deploy-elastic-hpc-cluster/
·
http://cfncluster.readthedocs.io/en/latest/configuration.html#scheduler
Looks like the best of both worlds!
-Steve
From: Vasisht Tadigotla [mailto:vasishtreddy_at_gmail.com]
Sent: Monday, January 8, 2018 1:21 PM
To: starcluster_at_mit.edu; Steve Darnell; Martin Fengler
Subject: Re: [StarCluster] Updates
Hi Martin, Steve,
There’s support for Ubuntu 16.04 using Mich’s fork. There aren’t any official AMIs, but I’m happy to share one that I’ve been using if needed. I would second Steve’s suggestion of looking into AWS Batch or cfncluster (
https://github.com/awslabs/cfncluster) although neither elasticluster nor cfncluster have a good option for load balancing.
Thanks,
Vasisht
On January 8, 2018 at 2:01:36 PM, Steve Darnell (darnells_at_dnastar.com<mailto:darnells_at_dnastar.com>) wrote:
Hi Martin,
*** Disclaimer: I am only a StarCluster user. My thoughts are not intended to represent the plans or intentions of Justin Riley or the StarCluster project. ***
I would not expect much in the way of further development. The last commits to the StarCluster develop branch were in December 2016 (
https://github.com/jtriley/StarCluster/commits/develop) while the mailing list and the project issue tracker has each discussed five topics in 2017. Mich at Datacratic has his own branch with improvements (
https://github.com/datacratic/StarCluster). You might want to check that out. His last commits were in May 2017.
If you search the mailing list, you will see references on how to create an Ubuntu 14.04 StarCluster image. If you don’t want to build it yourself, BioConductor distributes a Ubuntu 14.04 StarCluster image with bioc preinstalled (
https://www.bioconductor.org/help/bioconductor-cloud-ami/).
I would not expect to see StarCluster support for more modern Ubuntu versions since they all use systemd for initialization (15.04 and later) rather than Upstart. I imagine it will take a significant effort for StarCluster and all of its preconfigured third party packages and services to support systemd and coexist happily together.
I remain grateful for the work of Justin and the rest of the StarCluster team. I believe it was the right tool for the right job at the right time. StarCluster filled an important need in the early days of AWS when Amazon preferred Hadoop and you had to roll your own cluster-based solutions. Over the years, integrated AWS solutions like Elastic Beanstalk, ECS, and Batch have made configuring and managing a EC2 computing grid much easier. My group has started the transition over to them. If you don’t want to use these tools directly, perhaps an alternative like ElastiCluster (
https://github.com/gc3-uzh-ch/elasticluster) is worth consideration.
Good luck and best regards,
Steve
From: starcluster-bounces_at_mit.edu<mailto:starcluster-bounces_at_mit.edu> [mailto:starcluster-bounces_at_mit.edu<mailto:starcluster-bounces_at_mit.edu>] On Behalf Of Martin Fengler
Sent: Friday, January 5, 2018 3:29 AM
To: starcluster_at_mit.edu<mailto:starcluster_at_mit.edu>
Subject: [StarCluster] Updates
Dear all,
Are there any plans to provide also access to more recent Ubuntu versions? Such as 16.04? And to provide also images for eu-central-1 ?
Best regards
Martin
d/dx meteomatics
Your Experts in Weather Data Processing.
Meteomatics AG
Dr. Martin J. Fengler
Lerchenfeldstrasse 3
CH-9014 St. Gallen
Tel.: +41 (0) 71 272 66 50
Schiffbauerdamm 40, Büro 4406
D-10117 Berlin
Tel.: +49 (0) 30 200 74 280
www.meteomatics.com<
http://www.meteomatics.com/>
_______________________________________________
StarCluster mailing list
StarCluster_at_mit.edu<mailto:StarCluster_at_mit.edu>
http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Mon Jan 08 2018 - 14:35:42 EST