OK, I patched it and got it working for now.
I will try the github code later, but for now here is what I did:
in run_plugin():
got rid of:
for p in plugins:
p.run(cl.nodes, cl.master_node, cl.cluster_user, cl.cluster_shell,
volumes)
replaced it with:
plugins[0][1].run(cl.nodes, cl.master_node, cl.cluster_user,
cl.cluster_shell, cl.volumes)
Thanks!
Dan
On Sat, Oct 30, 2010 at 3:32 PM, Justin Riley <jtriley_at_mit.edu> wrote:
> Dan,
>
> The next version uses a ClusterManager object rather than the cluster.*
> methods. The new ClusterManager is automatically loaded in the shell as
> variable 'cm'. So with the latest github code it would be:
>
> $ starcluster shell
> $ cm.run_plugin("shortreadplugin", "myfirstcluster")
>
> You can pretty much replace any cluster.* method call with cm.* if you're
> using the latest github code.
>
> ~Justin
>
>
> On 10/30/10 2:57 PM, Dan Tenenbaum wrote:
>
> Looks like there are a number of files in this fork that don't exist in the
> 0.91 that I downloaded.
>
> So just downloaded your whole fork and built that, then got this in the
> shell:
>
> In [1]: cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
> ---------------------------------------------------------------------------
> AttributeError Traceback (most recent call last)
>
> /Library/Python/2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/commands/shell.pyc
> in <module>()
> ----> 1
> 2
> 3
> 4
> 5
>
> AttributeError: 'module' object has no attribute 'run_plugin'
>
> In [2]:
>
> Dan
>
>
> On Sat, Oct 30, 2010 at 11:45 AM, Dan Tenenbaum <dtenenba_at_fhcrc.org>wrote:
>
>> Keeping this on the list.
>> BTW, the new cluster.py has no syntax errors, so something else must be
>> happening.
>> Dan
>>
>>
>> On Sat, Oct 30, 2010 at 11:44 AM, Dan Tenenbaum <dtenenba_at_fhcrc.org>wrote:
>>
>>> Thanks. That gave me a different error, below.
>>> The first time this happened, I did:
>>> sudo easy_install iptools, but it is still happening.
>>>
>>> Dan
>>>
>>> $ starcluster shell
>>> Leopard libedit detected.
>>> ERROR: An unexpected error occurred while tokenizing input
>>> The following traceback may be corrupted or invalid
>>> The error message is: ('EOF in multi-line statement', (405, 0))
>>>
>>>
>>> ---------------------------------------------------------------------------
>>> ImportError Traceback (most recent call
>>> last)
>>>
>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/EGG-INFO/scripts/starcluster
>>> in <module>()
>>> 3 __requires__ = 'StarCluster==0.91'
>>> 4 import pkg_resources
>>> ----> 5 pkg_resources.run_script('StarCluster==0.91', 'starcluster')
>>> 6
>>> 7
>>>
>>> /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc
>>> in run_script(self, requires, script_name)
>>> 440 ns.clear()
>>> 441 ns['__name__'] = name
>>> --> 442 self.require(requires)[0].run_script(script_name, ns)
>>> 443
>>> 444
>>>
>>> /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc
>>> in run_script(self, script_name, namespace)
>>> 1165 )
>>> 1166 script_code =
>>> compile(script_text,script_filename,'exec')
>>> -> 1167 exec script_code in namespace, namespace
>>> 1168
>>> 1169 def _has(self, path):
>>>
>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/EGG-INFO/scripts/starcluster
>>> in <module>()
>>> 1
>>> ----> 2 #!/usr/bin/python
>>>
>>> 3 # EASY-INSTALL-SCRIPT: 'StarCluster==0.91','starcluster'
>>>
>>> 4 __requires__ = 'StarCluster==0.91'
>>> 5 import pkg_resources
>>> 6 pkg_resources.run_script('StarCluster==0.91', 'starcluster')
>>>
>>> /Users/dante/.starcluster/build/bdist.macosx-10.6-universal/egg/starcluster/cli.py
>>> in <module>()
>>> 36
>>> 37
>>> ---> 38
>>> 39
>>> 40
>>>
>>> /Users/dante/.starcluster/build/bdist.macosx-10.6-universal/egg/starcluster/cluster.py
>>> in <module>()
>>> 10
>>> 11
>>> ---> 12
>>> 13
>>> 14
>>>
>>> ImportError: cannot import name iptools
>>>
>>>
>>> On Sat, Oct 30, 2010 at 11:38 AM, Marc Resnick <mresnick_at_mit.edu>wrote:
>>>
>>>> The version here
>>>>
>>>> http://github.com/jtriley/StarCluster/blob/experimental/starcluster/cluster.py
>>>> may have those bugs fixed, as I remember that one as well.
>>>>
>>>>
>>>> Marc
>>>>
>>>> On Sat, Oct 30, 2010 at 2:31 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org>
>>>> wrote:
>>>> > I already did that. ;)
>>>> > Now I'm getting another error:
>>>> > In [1]: cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
>>>> >>>> Using private key /Users/dante/.ec2/bioc-keypair.pem.rsa (rsa)
>>>> >
>>>> ---------------------------------------------------------------------------
>>>> > AttributeError Traceback (most recent call
>>>> last)
>>>> >
>>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc
>>>> > in <module>()
>>>> > ----> 1
>>>> > 2
>>>> > 3
>>>> > 4
>>>> > 5
>>>> >
>>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
>>>> > in run_plugin(plugin_name, cluster_tag, cfg)
>>>> > 175 plugins = {}
>>>> > 176 plugins[plugin_name] = plug
>>>> > --> 177 plugins = cl.load_plugins(plugins)
>>>> > 178 master = cl.master_node
>>>> > 179 for p in plugins:
>>>> >
>>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
>>>> > in load_plugins(self, plugins)
>>>> > 314 plugs = []
>>>> > 315 for plugin in plugins:
>>>> > --> 316 setup_class = plugin.get('setup_class')
>>>> > 317 plugin_name = plugin.get('__name__')
>>>> > 318 mod_name = '.'.join(setup_class.split('.')[:-1])
>>>> > AttributeError: 'str' object has no attribute 'get'
>>>> > Hmm....
>>>> > Dan
>>>> >
>>>> > On Sat, Oct 30, 2010 at 11:27 AM, Marc Resnick <mresnick_at_mit.edu>
>>>> wrote:
>>>> >>
>>>> >> Yes, that's the bug I told Justin about this Summer... I think I
>>>> >> patched it, but he may have forgotten to pull from my fork.
>>>> >>
>>>> >> You can fix it by editing the run_plugin method, and replacing
>>>> >>
>>>> >> cl = get_cluster(cluster_tag)
>>>> >>
>>>> >> with
>>>> >>
>>>> >> cl = get_cluster(cluster_tag, cfg)
>>>> >>
>>>> >>
>>>> >>
>>>> >> Marc
>>>> >>
>>>> >>
>>>> >> On Sat, Oct 30, 2010 at 2:20 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org>
>>>> wrote:
>>>> >> > Hi Marc,
>>>> >> > I am running 0.91.
>>>> >> > I was doing a couple things wrong. I figured them out but I am
>>>> still
>>>> >> > having
>>>> >> > this problem.
>>>> >> > One thing I did wrong was have the
>>>> >> > plugins=shortreadplugin
>>>> >> > line under the global section; it needed to be under the
>>>> >> > [cluster smallcluster]
>>>> >> > section.
>>>> >> > When I start the cluster now, I can see the output from my plugin:
>>>> >> >>>> Running plugin shortreadplugin
>>>> >> >>>> about to echo
>>>> >> > But, when I run the shell and put in this command:
>>>> >> > cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
>>>> >> > I get the same error described below.
>>>> >> > Dan
>>>> >> >
>>>> >> > On Sat, Oct 30, 2010 at 11:11 AM, Marc Resnick <mresnick_at_mit.edu>
>>>> wrote:
>>>> >> >>
>>>> >> >> Hey Dan,
>>>> >> >>
>>>> >> >> Are you running the latest version? I saw that bug a few months
>>>> ago, I
>>>> >> >> thought I remember Justin fixing it.
>>>> >> >>
>>>> >> >>
>>>> >> >> Marc
>>>> >> >>
>>>> >> >> On Sat, Oct 30, 2010 at 2:00 PM, Dan Tenenbaum <
>>>> dtenenba_at_fhcrc.org>
>>>> >> >> wrote:
>>>> >> >> > Hi,
>>>> >> >> > I am not much of a python programmer and I'm trying to figure
>>>> out the
>>>> >> >> > plugin
>>>> >> >> > system.
>>>> >> >> > I created a ~/.starcluster/plugins directory and put a file in
>>>> it
>>>> >> >> > called
>>>> >> >> > shortread.py.
>>>> >> >> > Here are its contents:
>>>> >> >> >
>>>> >> >> > from starcluster.clustersetup import ClusterSetup
>>>> >> >> > from starcluster.logger import log
>>>> >> >> > class ShortReadPlugin(ClusterSetup):
>>>> >> >> > def __init__(self, arg1):
>>>> >> >> > self.arg1 = arg1
>>>> >> >> > log.info('arg1 = %s' % arg1)
>>>> >> >> > def run(self, nodes, master, user, user_shell, volumes):
>>>> >> >> > log.info("about to echo")
>>>> >> >> > master.ssh.execute("echo 'hey'")
>>>> >> >> > As you can see, it's just a test.
>>>> >> >> > In my config, I put this at the bottom:
>>>> >> >> > [plugin shortreadplugin]
>>>> >> >> > SETUP_CLASS = ShortReadPlugin
>>>> >> >> > arg1=foo
>>>> >> >> > And above in the [global] section I put:
>>>> >> >> > plugins=shortreadplugin
>>>> >> >> > I'm a bit confused about whether this is right because in your
>>>> >> >> > example:
>>>> >> >> > http://web.mit.edu/stardev/cluster/docs/plugins.html
>>>> >> >> > ...you have the SETUP_CLASS as ubuntu.PackageInstaller. I'm not
>>>> sure
>>>> >> >> > where
>>>> >> >> > the "ubuntu" fits into this. Is it like java where I need to
>>>> have a
>>>> >> >> > folder
>>>> >> >> > called ubuntu where the class resides?
>>>> >> >> > Anyway, I fired up my cluster and went to test this with
>>>> "starcluster
>>>> >> >> > shell"
>>>> >> >> > and got the following output:
>>>> >> >> > In [5]: cluster.run_plugin('shortreadplugin', 'myfirstcluster',
>>>> cfg)
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> ---------------------------------------------------------------------------
>>>> >> >> > TypeError Traceback (most recent
>>>> call
>>>> >> >> > last)
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc
>>>> >> >> > in <module>()
>>>> >> >> > ----> 1
>>>> >> >> > 2
>>>> >> >> > 3
>>>> >> >> > 4
>>>> >> >> > 5
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
>>>> >> >> > in run_plugin(plugin_name, cluster_tag, cfg)
>>>> >> >> > 170 def run_plugin(plugin_name, cluster_tag, cfg):
>>>> >> >> > 171 ec2 = cfg.get_easy_ec2()
>>>> >> >> > --> 172 cl = get_cluster(cluster_tag)
>>>> >> >> > 173 cl.load_receipt()
>>>> >> >> > 174 plug = cfg.get_plugin(plugin_name)
>>>> >> >> > TypeError: get_cluster() takes exactly 2 arguments (1 given)
>>>> >> >> >
>>>> >> >> > It's hard for me to understand what's happening here. I of
>>>> course am
>>>> >> >> > not
>>>> >> >> > calling get_cluster directly. Is there a bug in run_plugin() or
>>>> is
>>>> >> >> > there
>>>> >> >> > something wrong in my setup, or both?
>>>> >> >> > Thanks
>>>> >> >> > Dan
>>>> >> >> >
>>>> >> >> > _______________________________________________
>>>> >> >> > StarCluster mailing list
>>>> >> >> > StarCluster_at_mit.edu
>>>> >> >> > http://mailman.mit.edu/mailman/listinfo/starcluster
>>>> >> >> >
>>>> >> >> >
>>>> >> >>
>>>> >> >
>>>> >> >
>>>> >>
>>>> >
>>>> >
>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
Received on Sat Oct 30 2010 - 18:45:42 EDT
This archive was generated by
hypermail 2.3.0.