Re: plugin shell help

From: Marc Resnick <no email>
Date: Sat, 30 Oct 2010 14:38:49 -0400

The version here
http://github.com/jtriley/StarCluster/blob/experimental/starcluster/cluster.py
may have those bugs fixed, as I remember that one as well.


Marc

On Sat, Oct 30, 2010 at 2:31 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org> wrote:
> I already did that. ;)
> Now I'm getting another error:
> In [1]: cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
>>>> Using private key /Users/dante/.ec2/bioc-keypair.pem.rsa (rsa)
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc
> in <module>()
> ----> 1
>       2
>       3
>       4
>       5
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
> in run_plugin(plugin_name, cluster_tag, cfg)
>     175     plugins = {}
>     176     plugins[plugin_name] = plug
> --> 177     plugins = cl.load_plugins(plugins)
>     178     master = cl.master_node
>     179     for p in plugins:
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
> in load_plugins(self, plugins)
>     314         plugs = []
>     315         for plugin in plugins:
> --> 316             setup_class = plugin.get('setup_class')
>     317             plugin_name = plugin.get('__name__')
>     318             mod_name = '.'.join(setup_class.split('.')[:-1])
> AttributeError: 'str' object has no attribute 'get'
> Hmm....
> Dan
>
> On Sat, Oct 30, 2010 at 11:27 AM, Marc Resnick <mresnick_at_mit.edu> wrote:
>>
>> Yes, that's the bug I told Justin about this Summer... I think I
>> patched it, but he may have forgotten to pull from my fork.
>>
>> You can fix it by editing the run_plugin method, and replacing
>>
>>  cl = get_cluster(cluster_tag)
>>
>> with
>>
>>  cl = get_cluster(cluster_tag, cfg)
>>
>>
>>
>> Marc
>>
>>
>> On Sat, Oct 30, 2010 at 2:20 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org> wrote:
>> > Hi Marc,
>> > I am running 0.91.
>> > I was doing a couple things wrong. I figured them out but I am still
>> > having
>> > this problem.
>> > One thing I did wrong was have the
>> > plugins=shortreadplugin
>> > line under the global section; it needed to be under the
>> > [cluster smallcluster]
>> > section.
>> > When I start the cluster now, I can see the output from my plugin:
>> >>>> Running plugin shortreadplugin
>> >>>> about to echo
>> > But, when I run the shell and put in this command:
>> > cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
>> > I get the same error described below.
>> > Dan
>> >
>> > On Sat, Oct 30, 2010 at 11:11 AM, Marc Resnick <mresnick_at_mit.edu> wrote:
>> >>
>> >> Hey Dan,
>> >>
>> >> Are you running the latest version? I saw that bug a few months ago, I
>> >> thought I remember Justin fixing it.
>> >>
>> >>
>> >> Marc
>> >>
>> >> On Sat, Oct 30, 2010 at 2:00 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org>
>> >> wrote:
>> >> > Hi,
>> >> > I am not much of a python programmer and I'm trying to figure out the
>> >> > plugin
>> >> > system.
>> >> > I created a ~/.starcluster/plugins directory and put a file in it
>> >> > called
>> >> > shortread.py.
>> >> > Here are its contents:
>> >> >
>> >> > from starcluster.clustersetup import ClusterSetup
>> >> > from starcluster.logger import log
>> >> > class ShortReadPlugin(ClusterSetup):
>> >> >      def __init__(self, arg1):
>> >> >           self.arg1 = arg1
>> >> >           log.info('arg1 = %s' % arg1)
>> >> >      def run(self, nodes, master, user, user_shell, volumes):
>> >> >          log.info("about to echo")
>> >> >          master.ssh.execute("echo 'hey'")
>> >> > As you can see, it's just a test.
>> >> > In my config, I put this at the bottom:
>> >> > [plugin shortreadplugin]
>> >> > SETUP_CLASS = ShortReadPlugin
>> >> > arg1=foo
>> >> > And above in the [global] section I put:
>> >> > plugins=shortreadplugin
>> >> > I'm a bit confused about whether this is right because in your
>> >> > example:
>> >> > http://web.mit.edu/stardev/cluster/docs/plugins.html
>> >> > ...you have the SETUP_CLASS as ubuntu.PackageInstaller. I'm not sure
>> >> > where
>> >> > the "ubuntu" fits into this. Is it like java where I need to have a
>> >> > folder
>> >> > called ubuntu where the class resides?
>> >> > Anyway, I fired up my cluster and went to test this with "starcluster
>> >> > shell"
>> >> > and got the following output:
>> >> > In [5]: cluster.run_plugin('shortreadplugin', 'myfirstcluster', cfg)
>> >> >
>> >> >
>> >> > ---------------------------------------------------------------------------
>> >> > TypeError                                 Traceback (most recent call
>> >> > last)
>> >> >
>> >> >
>> >> > /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc
>> >> > in <module>()
>> >> > ----> 1
>> >> >       2
>> >> >       3
>> >> >       4
>> >> >       5
>> >> >
>> >> >
>> >> > /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
>> >> > in run_plugin(plugin_name, cluster_tag, cfg)
>> >> >     170 def run_plugin(plugin_name, cluster_tag, cfg):
>> >> >     171     ec2 = cfg.get_easy_ec2()
>> >> > --> 172     cl = get_cluster(cluster_tag)
>> >> >     173     cl.load_receipt()
>> >> >     174     plug = cfg.get_plugin(plugin_name)
>> >> > TypeError: get_cluster() takes exactly 2 arguments (1 given)
>> >> >
>> >> > It's hard for me to understand what's happening here. I of course am
>> >> > not
>> >> > calling get_cluster directly. Is there a bug in run_plugin() or is
>> >> > there
>> >> > something wrong in my setup, or both?
>> >> > Thanks
>> >> > Dan
>> >> >
>> >> > _______________________________________________
>> >> > StarCluster mailing list
>> >> > StarCluster_at_mit.edu
>> >> > http://mailman.mit.edu/mailman/listinfo/starcluster
>> >> >
>> >> >
>> >>
>> >
>> >
>>
>
>
Received on Sat Oct 30 2010 - 14:39:12 EDT