---------- SYSTEM INFO ----------
StarCluster: 0.9999
Python: 2.6.8 (unknown, Sep 17 2012, 03:13:50)  [GCC 4.6.2 20111027 (Red Hat 4.6.2-1)]
Platform: Linux-3.2.21-1.32.6.amzn1.x86_64-x86_64-with-glibc2.2.5
boto: 2.6.0
paramiko: 1.9.0
Crypto: 2.6

---------- CRASH DETAILS ----------
Command: starcluster reboot cluster

2012-12-05 16:51:31,268 PID: 10453 config.py:551 - DEBUG - Loading config
2012-12-05 16:51:31,268 PID: 10453 config.py:120 - DEBUG - Loading file: /home/ec2-user/.starcluster/config
2012-12-05 16:51:31,271 PID: 10453 awsutils.py:55 - DEBUG - creating self._conn w/ connection_authenticator kwargs = {'proxy_user': None, 'proxy_pass': None, 'proxy_port': None, 'proxy': None, 'is_secure': True, 'path': '/', 'region': None, 'port': None}
2012-12-05 16:51:31,732 PID: 10453 cluster.py:638 - DEBUG - existing nodes: {}
2012-12-05 16:51:31,732 PID: 10453 cluster.py:646 - DEBUG - adding node i-3ef91d40 to self._nodes list
2012-12-05 16:51:31,732 PID: 10453 cluster.py:646 - DEBUG - adding node i-3cf91d42 to self._nodes list
2012-12-05 16:51:31,732 PID: 10453 cluster.py:646 - DEBUG - adding node i-3af91d44 to self._nodes list
2012-12-05 16:51:31,733 PID: 10453 cluster.py:654 - DEBUG - returning self._nodes = [<Node: master (i-3ef91d40)>, <Node: node001 (i-3cf91d42)>, <Node: node002 (i-3af91d44)>]
2012-12-05 16:51:32,047 PID: 10453 cluster.py:638 - DEBUG - existing nodes: {u'i-3ef91d40': <Node: master (i-3ef91d40)>, u'i-3cf91d42': <Node: node001 (i-3cf91d42)>, u'i-3af91d44': <Node: node002 (i-3af91d44)>}
2012-12-05 16:51:32,047 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3ef91d40 in self._nodes
2012-12-05 16:51:32,047 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3cf91d42 in self._nodes
2012-12-05 16:51:32,048 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3af91d44 in self._nodes
2012-12-05 16:51:32,048 PID: 10453 cluster.py:654 - DEBUG - returning self._nodes = [<Node: master (i-3ef91d40)>, <Node: node001 (i-3cf91d42)>, <Node: node002 (i-3af91d44)>]
2012-12-05 16:51:32,048 PID: 10453 cluster.py:1308 - INFO - Rebooting cluster...
2012-12-05 16:51:32,558 PID: 10453 cluster.py:1312 - INFO - Sleeping for 20 seconds...
2012-12-05 16:51:52,579 PID: 10453 cluster.py:1233 - INFO - Waiting for cluster to come up... (updating every 30s)
2012-12-05 16:51:52,881 PID: 10453 cluster.py:638 - DEBUG - existing nodes: {u'i-3ef91d40': <Node: master (i-3ef91d40)>, u'i-3cf91d42': <Node: node001 (i-3cf91d42)>, u'i-3af91d44': <Node: node002 (i-3af91d44)>}
2012-12-05 16:51:52,881 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3ef91d40 in self._nodes
2012-12-05 16:51:52,881 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3cf91d42 in self._nodes
2012-12-05 16:51:52,882 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3af91d44 in self._nodes
2012-12-05 16:51:52,882 PID: 10453 cluster.py:654 - DEBUG - returning self._nodes = [<Node: master (i-3ef91d40)>, <Node: node001 (i-3cf91d42)>, <Node: node002 (i-3af91d44)>]
2012-12-05 16:51:52,882 PID: 10453 cluster.py:1201 - INFO - Waiting for all nodes to be in a 'running' state...
2012-12-05 16:51:52,946 PID: 10453 cluster.py:638 - DEBUG - existing nodes: {u'i-3ef91d40': <Node: master (i-3ef91d40)>, u'i-3cf91d42': <Node: node001 (i-3cf91d42)>, u'i-3af91d44': <Node: node002 (i-3af91d44)>}
2012-12-05 16:51:52,947 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3ef91d40 in self._nodes
2012-12-05 16:51:52,947 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3cf91d42 in self._nodes
2012-12-05 16:51:52,947 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3af91d44 in self._nodes
2012-12-05 16:51:52,947 PID: 10453 cluster.py:654 - DEBUG - returning self._nodes = [<Node: master (i-3ef91d40)>, <Node: node001 (i-3cf91d42)>, <Node: node002 (i-3af91d44)>]
2012-12-05 16:51:52,948 PID: 10453 cluster.py:1219 - INFO - Waiting for SSH to come up on all nodes...
2012-12-05 16:51:53,020 PID: 10453 cluster.py:638 - DEBUG - existing nodes: {u'i-3ef91d40': <Node: master (i-3ef91d40)>, u'i-3cf91d42': <Node: node001 (i-3cf91d42)>, u'i-3af91d44': <Node: node002 (i-3af91d44)>}
2012-12-05 16:51:53,021 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3ef91d40 in self._nodes
2012-12-05 16:51:53,021 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3cf91d42 in self._nodes
2012-12-05 16:51:53,021 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3af91d44 in self._nodes
2012-12-05 16:51:53,021 PID: 10453 cluster.py:654 - DEBUG - returning self._nodes = [<Node: master (i-3ef91d40)>, <Node: node001 (i-3cf91d42)>, <Node: node002 (i-3af91d44)>]
2012-12-05 16:51:53,026 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 3
2012-12-05 16:51:53,101 PID: 10453 __init__.py:67 - DEBUG - loading private key /home/ec2-user/.ssh/rsa-11702
2012-12-05 16:51:53,102 PID: 10453 __init__.py:159 - DEBUG - Using private key /home/ec2-user/.ssh/rsa-11702 (rsa)
2012-12-05 16:51:53,102 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-174-129-135-55.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:51:53,115 PID: 10453 __init__.py:67 - DEBUG - loading private key /home/ec2-user/.ssh/rsa-11702
2012-12-05 16:51:53,116 PID: 10453 __init__.py:159 - DEBUG - Using private key /home/ec2-user/.ssh/rsa-11702 (rsa)
2012-12-05 16:51:53,116 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:51:53,119 PID: 10453 __init__.py:67 - DEBUG - loading private key /home/ec2-user/.ssh/rsa-11702
2012-12-05 16:51:53,120 PID: 10453 __init__.py:159 - DEBUG - Using private key /home/ec2-user/.ssh/rsa-11702 (rsa)
2012-12-05 16:51:53,120 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:51:54,330 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 3
2012-12-05 16:51:54,518 PID: 10453 __init__.py:178 - DEBUG - creating sftp connection
2012-12-05 16:51:55,332 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:51:56,333 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:51:57,335 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:51:58,336 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:51:59,338 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:00,339 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:01,341 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:02,343 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:03,344 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:04,346 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:05,347 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:06,349 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:07,350 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:08,352 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:09,354 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:10,355 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:11,357 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:12,358 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:13,360 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:14,361 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:15,363 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:16,364 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:17,366 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:18,368 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:19,369 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:20,371 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:21,372 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:22,374 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:23,375 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:24,377 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:25,378 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:26,380 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:27,382 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:28,383 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:29,385 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:30,386 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:31,388 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:32,389 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:33,391 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:34,392 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:35,394 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:36,395 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:37,397 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:38,398 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:39,400 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:40,402 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:41,403 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:42,405 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:43,406 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:44,408 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:45,409 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:46,411 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:47,412 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:48,414 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:49,416 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:50,417 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:51,419 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:52,420 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:53,254 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:52:53,258 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:52:53,422 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:54,424 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:55,425 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:56,427 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:57,429 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:58,430 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:52:59,432 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:00,434 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:01,435 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:02,437 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:03,439 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:04,440 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:05,442 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:06,443 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:07,445 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:08,446 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:09,448 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:10,450 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:11,451 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:12,453 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:13,454 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:14,455 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:15,456 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:16,458 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:17,459 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:18,461 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:19,463 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:20,464 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:21,466 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:22,467 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:23,469 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:24,470 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:25,472 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:26,474 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:27,475 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:28,477 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:29,478 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:30,480 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:31,481 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:32,483 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:33,485 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:34,486 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:35,488 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:36,489 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:37,491 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:38,492 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:39,494 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:40,495 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:41,497 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:42,498 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:43,500 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:44,501 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:45,503 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:46,504 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:47,506 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:48,507 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:49,509 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:50,510 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:51,512 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:52,513 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:53,460 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:53:53,463 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:53:53,515 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:54,517 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:55,518 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:56,520 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:57,522 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:58,523 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:53:59,525 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:00,526 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:01,528 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:02,529 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:03,531 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:04,533 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:05,534 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:06,536 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:07,537 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:08,539 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:09,540 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:10,542 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:11,543 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:12,545 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:13,546 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:14,548 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:15,549 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:16,551 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:17,552 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:18,554 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:19,556 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:20,557 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:21,559 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:22,560 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:23,562 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:24,563 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:25,565 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:26,566 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:27,568 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:28,569 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:29,248 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:54:29,571 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:30,572 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:31,573 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:32,575 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:33,576 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:34,578 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:35,580 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:36,581 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:37,583 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:38,584 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:39,586 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:40,587 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:41,589 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:42,590 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:43,592 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:44,594 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:45,595 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:46,597 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:47,598 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:48,600 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:49,601 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:50,603 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:51,604 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:52,605 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:53,575 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:54:53,607 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:54,608 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:55,610 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:56,611 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:57,613 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:58,615 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:54:59,616 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:00,618 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:01,619 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:02,621 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:03,623 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:04,624 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:04,887 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:55:05,626 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:06,627 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:07,629 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:08,630 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:09,632 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:10,634 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:11,635 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:12,637 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:13,638 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:14,640 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:15,641 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:16,643 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:17,644 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:18,646 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:19,647 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:20,649 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:21,650 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:22,651 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:23,653 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:24,654 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:25,656 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:26,658 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:27,659 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:28,661 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:29,663 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:30,664 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:31,666 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:32,667 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:33,669 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:33,686 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:55:34,670 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:35,672 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:36,674 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:37,675 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:37,966 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:55:38,677 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:39,678 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:40,680 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:41,682 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:42,683 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:43,685 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:44,686 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:45,688 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:46,689 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:47,691 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:48,693 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:49,694 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:50,696 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:51,697 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:52,699 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:53,700 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:54,702 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:55,704 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:56,705 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:57,707 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:58,708 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:55:59,710 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:00,711 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:01,713 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:02,715 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:03,716 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:04,718 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:05,719 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:06,722 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:06,797 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:56:07,723 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:08,725 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:09,726 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:10,728 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:11,038 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-204-236-196-32.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:56:12,391 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:12,973 PID: 10453 __init__.py:178 - DEBUG - creating sftp connection
2012-12-05 16:56:13,392 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:14,394 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:15,395 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:16,397 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:17,398 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 2
2012-12-05 16:56:18,400 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:19,401 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:20,403 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:21,404 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:22,406 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:23,408 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:24,409 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:25,411 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:26,412 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:27,414 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:28,415 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:29,417 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:30,419 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:31,421 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:32,423 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:33,424 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:34,426 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:35,428 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:36,429 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:37,431 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:38,433 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:39,437 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:39,920 PID: 10453 __init__.py:89 - DEBUG - connecting to host ec2-54-242-199-98.compute-1.amazonaws.com on port 22 as user root
2012-12-05 16:56:41,348 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:41,825 PID: 10453 __init__.py:178 - DEBUG - creating sftp connection
2012-12-05 16:56:42,350 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:43,351 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:44,352 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:45,354 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:46,355 PID: 10453 utils.py:97 - INFO - Waiting for cluster to come up took 4.896 mins
2012-12-05 16:56:46,356 PID: 10453 cluster.py:1453 - INFO - The master node is ec2-174-129-135-55.compute-1.amazonaws.com
2012-12-05 16:56:46,356 PID: 10453 cluster.py:1454 - INFO - Setting up the cluster...
2012-12-05 16:56:46,436 PID: 10453 cluster.py:638 - DEBUG - existing nodes: {u'i-3ef91d40': <Node: master (i-3ef91d40)>, u'i-3cf91d42': <Node: node001 (i-3cf91d42)>, u'i-3af91d44': <Node: node002 (i-3af91d44)>}
2012-12-05 16:56:46,436 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3ef91d40 in self._nodes
2012-12-05 16:56:46,436 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3cf91d42 in self._nodes
2012-12-05 16:56:46,437 PID: 10453 cluster.py:641 - DEBUG - updating existing node i-3af91d44 in self._nodes
2012-12-05 16:56:46,437 PID: 10453 cluster.py:654 - DEBUG - returning self._nodes = [<Node: master (i-3ef91d40)>, <Node: node001 (i-3cf91d42)>, <Node: node002 (i-3af91d44)>]
2012-12-05 16:56:46,437 PID: 10453 clustersetup.py:90 - INFO - Configuring hostnames...
2012-12-05 16:56:46,442 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 3
2012-12-05 16:56:46,454 PID: 10453 __init__.py:519 - DEBUG - executing remote command: source /etc/profile && hostname -F /etc/hostname
2012-12-05 16:56:46,455 PID: 10453 __init__.py:519 - DEBUG - executing remote command: source /etc/profile && hostname -F /etc/hostname
2012-12-05 16:56:46,500 PID: 10453 __init__.py:543 - DEBUG - output of 'source /etc/profile && hostname -F /etc/hostname':

2012-12-05 16:56:46,549 PID: 10453 __init__.py:543 - DEBUG - output of 'source /etc/profile && hostname -F /etc/hostname':

2012-12-05 16:56:47,443 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:48,445 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:49,446 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:50,448 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:51,450 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:52,451 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:53,453 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:54,455 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:55,456 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:56,458 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:57,459 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:58,461 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:56:59,462 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:00,464 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:01,465 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:02,467 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:03,469 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:04,470 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:05,472 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:06,473 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:07,475 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:08,476 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:09,478 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:10,479 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:11,481 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:12,483 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:13,484 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:14,486 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:15,487 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:16,489 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:17,490 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:18,492 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:19,493 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:20,495 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:21,497 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:22,498 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:23,500 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:24,501 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:25,503 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:26,505 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:27,506 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:28,508 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:29,510 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:30,511 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:31,513 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:32,515 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:33,516 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:34,518 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:35,520 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:36,522 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:37,523 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:38,525 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:39,527 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:40,528 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:41,530 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:42,532 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:43,533 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:44,535 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:45,537 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:46,538 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:47,539 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:48,541 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:49,543 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:50,545 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:51,547 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:52,548 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:53,550 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:54,552 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:55,554 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:56,555 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:57,557 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:58,559 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:57:59,561 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:00,562 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:01,564 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:02,566 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:03,568 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:04,569 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:05,571 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:06,573 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:07,575 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:08,576 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:09,578 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:10,580 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:11,582 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:12,583 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:13,585 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:14,587 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:15,589 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:16,590 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:17,592 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:18,594 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:19,596 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:20,597 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:21,599 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:22,601 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:23,603 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:24,605 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:25,606 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:26,608 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:27,609 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:28,611 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:29,613 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:30,615 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:31,616 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:32,618 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:33,620 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:34,622 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:35,623 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:36,625 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:37,627 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:38,629 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:39,630 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:40,632 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:41,634 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:42,635 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:43,637 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:44,639 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:45,641 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:46,642 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:47,644 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:48,646 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:49,647 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:50,649 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:51,651 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:52,653 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:53,654 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:54,656 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:55,658 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:56,660 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:57,661 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:58,663 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:58:59,665 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:00,667 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:01,669 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:02,670 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:03,672 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:04,674 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:05,676 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:06,678 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:07,679 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:08,681 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:09,683 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:10,685 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:11,686 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:12,688 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:13,690 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:14,692 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:15,694 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:16,695 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:17,697 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:18,699 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:19,701 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:20,702 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:21,704 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:22,706 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:23,708 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:24,710 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:25,711 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:26,713 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:27,715 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:28,717 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:29,718 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:30,720 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:31,722 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:32,724 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:33,725 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:34,727 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:35,729 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:36,731 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:37,732 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:38,734 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:39,736 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:40,737 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:41,739 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:42,741 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:43,743 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:44,744 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:45,746 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:46,748 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:47,750 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:48,751 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:49,753 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:50,755 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:51,757 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:52,758 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:53,760 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:54,762 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:55,764 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:56,766 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:57,767 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:58,769 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 16:59:59,771 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:00,772 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:01,774 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:02,776 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:03,778 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:04,779 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:05,781 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:06,783 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:07,785 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:08,787 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:09,788 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:10,790 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:11,792 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:12,794 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:13,796 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:14,797 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:15,799 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:16,801 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:17,803 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:18,804 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:19,806 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:20,808 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:21,809 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:22,810 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:23,812 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:24,814 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:25,816 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:26,817 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:27,819 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:28,821 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:29,822 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:30,824 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:31,826 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:32,828 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:33,829 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:34,831 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:35,833 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:36,834 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:37,836 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:38,838 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:39,840 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:40,841 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:41,843 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:42,845 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:43,846 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:44,848 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:45,850 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:46,851 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:47,853 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:48,855 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:49,857 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:50,858 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:51,860 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:52,862 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:53,863 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:54,865 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:55,867 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:56,868 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:57,870 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:58,872 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:00:59,873 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:00,875 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:01,877 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:02,879 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:03,880 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:04,882 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:05,884 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:06,885 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:07,887 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:08,889 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:09,891 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:10,892 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:11,894 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:12,896 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:13,897 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:14,899 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:15,901 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:16,983 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 1
2012-12-05 17:01:17,985 PID: 10453 threadpool.py:123 - INFO - Shutting down threads...
2012-12-05 17:01:17,985 PID: 10453 threadpool.py:135 - DEBUG - unfinished_tasks = 20
2012-12-05 17:01:18,987 PID: 10453 cli.py:266 - ERROR - error occurred in job (id=master): Server connection dropped: 
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/threadpool.py", line 31, in run
    job.run()
  File "/usr/lib/python2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/threadpool.py", line 58, in run
    r = self.method(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/node.py", line 735, in set_hostname
    hostname_file = self.ssh.remote_file("/etc/hostname", "w")
  File "/usr/lib/python2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/sshutils/__init__.py", line 289, in remote_file
    rfile = self.sftp.open(file, mode)
  File "/usr/lib/python2.6/site-packages/paramiko-1.9.0-py2.6.egg/paramiko/sftp_client.py", line 245, in open
    t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
  File "/usr/lib/python2.6/site-packages/paramiko-1.9.0-py2.6.egg/paramiko/sftp_client.py", line 635, in _request
    return self._read_response(num)
  File "/usr/lib/python2.6/site-packages/paramiko-1.9.0-py2.6.egg/paramiko/sftp_client.py", line 667, in _read_response
    raise SSHException('Server connection dropped: %s' % (str(e),))
SSHException: Server connection dropped: