28 October 2005 3:53 PM (python)
For some reason Flumotion is running at 100% cpu when starting components. It didn't use to do this, and it doesn't really make much sense -- the rate of data input is limited by the capture devices, and a certain time's worth of data has to accumulate to fill all of the relevant buffers in the encoders, muxers, and streamers.
But I have no clue where this CPU usage is. I tried pressing control-C occasionally and getting backtraces in GDB, but the normal methods aren't working.
In the end this was all due to not having an idea of where Flumotion spends its time. I wanted to profile, but didn't want to use an instrumenting profiler, and there didn't exist a sampling profiler for Python. So I ported the one I hacked on for Guile to python; it's available here.
Usage goes like this:
>>> import statprof >>> statprof.start() >>> import test.pystone; test.pystone.pystones() (1.3200000000000001, 37878.78787878788) >>> statprof.stop() >>> statprof.display() % cumulative self time seconds seconds name 23.01 1.36 0.31 pystone.py:79:Proc0 15.04 0.60 0.20 pystone.py:133:Proc1 11.50 0.16 0.16 pystone.py:45:__init__ 10.62 0.14 0.14 pystone.py:208:Proc8 7.96 0.16 0.11 pystone.py:229:Func2 7.96 0.11 0.11 pystone.py:221:Func1 6.19 0.12 0.08 pystone.py:160:Proc3 5.31 0.07 0.07 pystone.py:203:Proc7 3.54 0.20 0.05 pystone.py:53:copy 2.65 0.05 0.04 pystone.py:184:Proc6 2.65 0.04 0.04 pystone.py:149:Proc2 1.77 0.02 0.02 pystone.py:170:Proc4 0.88 0.01 0.01 pystone.py:177:Proc5 0.88 0.01 0.01 pystone.py:246:Func3 0.00 1.36 0.00 pystone.py:67:pystones 0.00 1.36 0.00 <stdin>:1:? --- Sample count: 113 Total time: 1.360000 seconds
There's lots of info in help(statprof). Also it requires the itimer extension from http://www.cute.fi/~torppa/py-itimer/. Just unpack the tarball and run sudo python setup.py install.
It's not very well tested at this point, but it gives similar results as the stock profiler (10-20 times faster though). It's of course not the same, because an instrumenting profiler unfairly penalizes procedure calls; while they do have a cost it is much less than their cost as measured by the stock profiler or hotshot.
I'd be interested in hearing about bugs in it for the next few weeks; after that it will probably slip from my mind though. (This is of course the natural fate of profilers, to bitrot. I have a larger rant about this for later.)