StarCluster - Mailing List Archive

strange root vs. user behavior for GPU nodes

From: Charles Cadieu <no email>
Date: Mon, 23 Jan 2012 16:47:10 -0500

Hi,

I'm noticing some unusual behavior between running as a user vs. as root
when running some GPU code. As far as I can tell (PYTHONPATH, PATH) are
pretty much identical between the user and root, and both are using the
same relevant binaries. Is there some obvious step that's necessary to
allow the user code to run on the GPU?

Here's some more info on the problem:
I'm running some tutorial code from theano:
http://deeplearning.net/software/theano/tutorial/using_gpu.html

when I run as root I get output like this:
[root_at_node002 ~]# THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32
python2.6 thing.py
Setup took 0.873317956924 seconds
Looping 1000 times took 36.3292770386 seconds
Result is [ 1.23178029 1.61879337 1.52278066 ..., 2.20771813 2.29967761
  1.62323284]
Used the cpu

[root_at_node002 ~]# THEANO_FLAGS=mode=FAST_RUN,device=gpu0,floatX=float32
python2.6 thing.py
Using gpu device 0: Tesla M2050
Setup took 6.41229701042 seconds
Looping 1000 times took 1.73471212387 seconds
Result is [ 1.23178029 1.61879349 1.52278066 ..., 2.20771813 2.29967761
  1.62323296]
Used the gpu

BUT, as a user, I get:
[render_at_node002 ~]$ THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32
python2.6 thing.py
Setup took 0.591382980347 seconds
Looping 1000 times took 36.3925471306 seconds
Result is [ 1.23178029 1.61879337 1.52278066 ..., 2.20771813 2.29967761
  1.62323284]
Used the cpu

[render_at_node002 ~]$ THEANO_FLAGS=mode=FAST_RUN,device=gpu0,floatX=float32
python2.6 thing.py
Using gpu device 0: Tesla M2050
Setup took 186.97786808 seconds
Looping 1000 times took 182.75831604 seconds
Result is [ 1.23178029 1.61879349 1.52278066 ..., 2.20771813 2.29967761
  1.62323296]
Used the gpu

So, major slowdown, but otherwise no obvious errors.

Let me know if there's more info or something else I should try,
Charles
Received on Mon Jan 23 2012 - 16:47:12 EST
This archive was generated by hypermail 2.3.0.

Search:

Sort all by:

Date

Month

Thread

Author

Subject