Something else is going on....the shell doesn't seem to recognize any of the
methods in "cluster" as belonging to ClusterManager. I'm not enough of a
python developer to know why this might be.
Dan
On Sat, Oct 30, 2010 at 12:05 PM, Marc Resnick <mresnick_at_mit.edu> wrote:
> Got me stumped. If I remember correctly, that function worked for me
> the last time I tried it...
>
> On Sat, Oct 30, 2010 at 2:57 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org> wrote:
> > Looks like there are a number of files in this fork that don't exist in
> the
> > 0.91 that I downloaded.
> > So just downloaded your whole fork and built that, then got this in the
> > shell:
> > In [1]: cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
> >
> ---------------------------------------------------------------------------
> > AttributeError Traceback (most recent call
> last)
> >
> /Library/Python/2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/commands/shell.pyc
> > in <module>()
> > ----> 1
> > 2
> > 3
> > 4
> > 5
> > AttributeError: 'module' object has no attribute 'run_plugin'
> > In [2]:
> > Dan
> >
> > On Sat, Oct 30, 2010 at 11:45 AM, Dan Tenenbaum <dtenenba_at_fhcrc.org>
> wrote:
> >>
> >> Keeping this on the list.
> >> BTW, the new cluster.py has no syntax errors, so something else must be
> >> happening.
> >> Dan
> >>
> >> On Sat, Oct 30, 2010 at 11:44 AM, Dan Tenenbaum <dtenenba_at_fhcrc.org>
> >> wrote:
> >>>
> >>> Thanks. That gave me a different error, below.
> >>> The first time this happened, I did:
> >>> sudo easy_install iptools, but it is still happening.
> >>> Dan
> >>> $ starcluster shell
> >>> Leopard libedit detected.
> >>> ERROR: An unexpected error occurred while tokenizing input
> >>> The following traceback may be corrupted or invalid
> >>> The error message is: ('EOF in multi-line statement', (405, 0))
> >>>
> >>>
> ---------------------------------------------------------------------------
> >>> ImportError Traceback (most recent call
> >>> last)
> >>>
> >>>
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/EGG-INFO/scripts/starcluster
> >>> in <module>()
> >>> 3 __requires__ = 'StarCluster==0.91'
> >>> 4 import pkg_resources
> >>> ----> 5 pkg_resources.run_script('StarCluster==0.91', 'starcluster')
> >>> 6
> >>> 7
> >>>
> >>>
> /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc
> >>> in run_script(self, requires, script_name)
> >>> 440 ns.clear()
> >>> 441 ns['__name__'] = name
> >>> --> 442 self.require(requires)[0].run_script(script_name, ns)
> >>> 443
> >>> 444
> >>>
> >>>
> /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc
> >>> in run_script(self, script_name, namespace)
> >>> 1165 )
> >>> 1166 script_code =
> >>> compile(script_text,script_filename,'exec')
> >>> -> 1167 exec script_code in namespace, namespace
> >>> 1168
> >>> 1169 def _has(self, path):
> >>>
> >>>
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/EGG-INFO/scripts/starcluster
> >>> in <module>()
> >>> 1
> >>> ----> 2 #!/usr/bin/python
> >>> 3 # EASY-INSTALL-SCRIPT: 'StarCluster==0.91','starcluster'
> >>> 4 __requires__ = 'StarCluster==0.91'
> >>> 5 import pkg_resources
> >>> 6 pkg_resources.run_script('StarCluster==0.91', 'starcluster')
> >>>
> >>>
> /Users/dante/.starcluster/build/bdist.macosx-10.6-universal/egg/starcluster/cli.py
> >>> in <module>()
> >>> 36
> >>> 37
> >>> ---> 38
> >>> 39
> >>> 40
> >>>
> >>>
> /Users/dante/.starcluster/build/bdist.macosx-10.6-universal/egg/starcluster/cluster.py
> >>> in <module>()
> >>> 10
> >>> 11
> >>> ---> 12
> >>> 13
> >>> 14
> >>> ImportError: cannot import name iptools
> >>>
> >>> On Sat, Oct 30, 2010 at 11:38 AM, Marc Resnick <mresnick_at_mit.edu>
> wrote:
> >>>>
> >>>> The version here
> >>>>
> >>>>
> http://github.com/jtriley/StarCluster/blob/experimental/starcluster/cluster.py
> >>>> may have those bugs fixed, as I remember that one as well.
> >>>>
> >>>>
> >>>> Marc
> >>>>
> >>>> On Sat, Oct 30, 2010 at 2:31 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org>
> >>>> wrote:
> >>>> > I already did that. ;)
> >>>> > Now I'm getting another error:
> >>>> > In [1]: cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
> >>>> >>>> Using private key /Users/dante/.ec2/bioc-keypair.pem.rsa (rsa)
> >>>> >
> >>>> >
> ---------------------------------------------------------------------------
> >>>> > AttributeError Traceback (most recent
> call
> >>>> > last)
> >>>> >
> >>>> >
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc
> >>>> > in <module>()
> >>>> > ----> 1
> >>>> > 2
> >>>> > 3
> >>>> > 4
> >>>> > 5
> >>>> >
> >>>> >
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
> >>>> > in run_plugin(plugin_name, cluster_tag, cfg)
> >>>> > 175 plugins = {}
> >>>> > 176 plugins[plugin_name] = plug
> >>>> > --> 177 plugins = cl.load_plugins(plugins)
> >>>> > 178 master = cl.master_node
> >>>> > 179 for p in plugins:
> >>>> >
> >>>> >
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
> >>>> > in load_plugins(self, plugins)
> >>>> > 314 plugs = []
> >>>> > 315 for plugin in plugins:
> >>>> > --> 316 setup_class = plugin.get('setup_class')
> >>>> > 317 plugin_name = plugin.get('__name__')
> >>>> > 318 mod_name = '.'.join(setup_class.split('.')[:-1])
> >>>> > AttributeError: 'str' object has no attribute 'get'
> >>>> > Hmm....
> >>>> > Dan
> >>>> >
> >>>> > On Sat, Oct 30, 2010 at 11:27 AM, Marc Resnick <mresnick_at_mit.edu>
> >>>> > wrote:
> >>>> >>
> >>>> >> Yes, that's the bug I told Justin about this Summer... I think I
> >>>> >> patched it, but he may have forgotten to pull from my fork.
> >>>> >>
> >>>> >> You can fix it by editing the run_plugin method, and replacing
> >>>> >>
> >>>> >> cl = get_cluster(cluster_tag)
> >>>> >>
> >>>> >> with
> >>>> >>
> >>>> >> cl = get_cluster(cluster_tag, cfg)
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> Marc
> >>>> >>
> >>>> >>
> >>>> >> On Sat, Oct 30, 2010 at 2:20 PM, Dan Tenenbaum <dtenenba_at_fhcrc.org
> >
> >>>> >> wrote:
> >>>> >> > Hi Marc,
> >>>> >> > I am running 0.91.
> >>>> >> > I was doing a couple things wrong. I figured them out but I am
> >>>> >> > still
> >>>> >> > having
> >>>> >> > this problem.
> >>>> >> > One thing I did wrong was have the
> >>>> >> > plugins=shortreadplugin
> >>>> >> > line under the global section; it needed to be under the
> >>>> >> > [cluster smallcluster]
> >>>> >> > section.
> >>>> >> > When I start the cluster now, I can see the output from my
> plugin:
> >>>> >> >>>> Running plugin shortreadplugin
> >>>> >> >>>> about to echo
> >>>> >> > But, when I run the shell and put in this command:
> >>>> >> > cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)
> >>>> >> > I get the same error described below.
> >>>> >> > Dan
> >>>> >> >
> >>>> >> > On Sat, Oct 30, 2010 at 11:11 AM, Marc Resnick <mresnick_at_mit.edu
> >
> >>>> >> > wrote:
> >>>> >> >>
> >>>> >> >> Hey Dan,
> >>>> >> >>
> >>>> >> >> Are you running the latest version? I saw that bug a few months
> >>>> >> >> ago, I
> >>>> >> >> thought I remember Justin fixing it.
> >>>> >> >>
> >>>> >> >>
> >>>> >> >> Marc
> >>>> >> >>
> >>>> >> >> On Sat, Oct 30, 2010 at 2:00 PM, Dan Tenenbaum
> >>>> >> >> <dtenenba_at_fhcrc.org>
> >>>> >> >> wrote:
> >>>> >> >> > Hi,
> >>>> >> >> > I am not much of a python programmer and I'm trying to figure
> >>>> >> >> > out the
> >>>> >> >> > plugin
> >>>> >> >> > system.
> >>>> >> >> > I created a ~/.starcluster/plugins directory and put a file in
> >>>> >> >> > it
> >>>> >> >> > called
> >>>> >> >> > shortread.py.
> >>>> >> >> > Here are its contents:
> >>>> >> >> >
> >>>> >> >> > from starcluster.clustersetup import ClusterSetup
> >>>> >> >> > from starcluster.logger import log
> >>>> >> >> > class ShortReadPlugin(ClusterSetup):
> >>>> >> >> > def __init__(self, arg1):
> >>>> >> >> > self.arg1 = arg1
> >>>> >> >> > log.info('arg1 = %s' % arg1)
> >>>> >> >> > def run(self, nodes, master, user, user_shell, volumes):
> >>>> >> >> > log.info("about to echo")
> >>>> >> >> > master.ssh.execute("echo 'hey'")
> >>>> >> >> > As you can see, it's just a test.
> >>>> >> >> > In my config, I put this at the bottom:
> >>>> >> >> > [plugin shortreadplugin]
> >>>> >> >> > SETUP_CLASS = ShortReadPlugin
> >>>> >> >> > arg1=foo
> >>>> >> >> > And above in the [global] section I put:
> >>>> >> >> > plugins=shortreadplugin
> >>>> >> >> > I'm a bit confused about whether this is right because in your
> >>>> >> >> > example:
> >>>> >> >> > http://web.mit.edu/stardev/cluster/docs/plugins.html
> >>>> >> >> > ...you have the SETUP_CLASS as ubuntu.PackageInstaller. I'm
> not
> >>>> >> >> > sure
> >>>> >> >> > where
> >>>> >> >> > the "ubuntu" fits into this. Is it like java where I need to
> >>>> >> >> > have a
> >>>> >> >> > folder
> >>>> >> >> > called ubuntu where the class resides?
> >>>> >> >> > Anyway, I fired up my cluster and went to test this with
> >>>> >> >> > "starcluster
> >>>> >> >> > shell"
> >>>> >> >> > and got the following output:
> >>>> >> >> > In [5]: cluster.run_plugin('shortreadplugin',
> 'myfirstcluster',
> >>>> >> >> > cfg)
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> >
> ---------------------------------------------------------------------------
> >>>> >> >> > TypeError Traceback (most
> recent
> >>>> >> >> > call
> >>>> >> >> > last)
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> >
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc
> >>>> >> >> > in <module>()
> >>>> >> >> > ----> 1
> >>>> >> >> > 2
> >>>> >> >> > 3
> >>>> >> >> > 4
> >>>> >> >> > 5
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> >
> /Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc
> >>>> >> >> > in run_plugin(plugin_name, cluster_tag, cfg)
> >>>> >> >> > 170 def run_plugin(plugin_name, cluster_tag, cfg):
> >>>> >> >> > 171 ec2 = cfg.get_easy_ec2()
> >>>> >> >> > --> 172 cl = get_cluster(cluster_tag)
> >>>> >> >> > 173 cl.load_receipt()
> >>>> >> >> > 174 plug = cfg.get_plugin(plugin_name)
> >>>> >> >> > TypeError: get_cluster() takes exactly 2 arguments (1 given)
> >>>> >> >> >
> >>>> >> >> > It's hard for me to understand what's happening here. I of
> >>>> >> >> > course am
> >>>> >> >> > not
> >>>> >> >> > calling get_cluster directly. Is there a bug in run_plugin()
> or
> >>>> >> >> > is
> >>>> >> >> > there
> >>>> >> >> > something wrong in my setup, or both?
> >>>> >> >> > Thanks
> >>>> >> >> > Dan
> >>>> >> >> >
> >>>> >> >> > _______________________________________________
> >>>> >> >> > StarCluster mailing list
> >>>> >> >> > StarCluster_at_mit.edu
> >>>> >> >> > http://mailman.mit.edu/mailman/listinfo/starcluster
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >>
> >>>> >> >
> >>>> >> >
> >>>> >>
> >>>> >
> >>>> >
> >>>>
> >>>
> >>
> >
> >
>
>
Received on Sat Oct 30 2010 - 15:16:46 EDT
This archive was generated by
hypermail 2.3.0.