Spiga

Tee'ing Python subprocess.Popen output

Thursday, February 28, 2008 at 11:11 Posted by KageSenshi

A little hack for python coders out there who wanted to have a functionality similar to the unix's tee command for redirecting output to multiple places.


import sys
from subprocess import Popen,PIPE
p = Popen(['put','command','and','arguments','here'],stdout=PIPE)

while True:
   o = p.stdout.readline()
   if o == '' and p.poll() != None: break
   # the 'o' variable stores a line from the command's stdout
   # do anything u wish with the 'o' variable here
   # this loop will break once theres a blank output 
   # from stdout and the subprocess have ended

Labels: , ,

10 comments:

Andrew Z

11:18 PM

Just what I was looking for! BTW, I use Fedora too. :)

miya

11:15 PM

This post has been removed by the author.
miya

11:16 PM

But... isn't the Popen command a sync. one ?
I mean, the while loop will only get executed once the Popen finishes...
Is this a feature or a bug ?

KageSenshi

10:01 AM

subprocess.Popen is async, command executed in subprocess.Popen will go to background.

The while loop, yeah, it'll only break after the command is done. If u want to make it async (for GUI use for example) u should think of another way, perhaps printing the output out through GTK callback or something.

miya

10:14 AM

Is it async?
I've been playing with this and there's sth. I just cannot figure out.

When I execute the following code, the main.py will only get into the p.poll() once the process finishes. It will stay blocked in the p.poll() line...

[code main.py]
import subprocess
import sys

def main():

p = subprocess.Popen(['python', 'test.py'], stdout = subprocess.PIPE)

while 1:
o = p.stdout.readline()
if o == '' and p.poll() != None:
print 'here!!!'
break
print o

if __name__ == "__main__":
main()
[/code]

and the test.py is

[code test.py]
import time

i = 0

while i != 5:
print "%s seconds" % i
time.sleep(1)
i += 1

[/code]

Anyway... thanx for answering =D

KageSenshi

10:54 AM

thats weird,

replacing ['python','test.py'] with ['ping','www.google.com'] works.

for ['python','test.py'], o.stdout.readline() seems to wait for something. No idea why. Sorry, couldnt help much here.

miya

10:35 PM

It's ok. Thanks for taking the time to answer.

cya!

Anonymous

6:41 AM

This is the code I ended up using. It includes killing the process on interrupt, as well as flushing the output in opportune moments, such that the display isn't distorted. The app I was tee'ing is mplayer. Also take a look at the excellent pexpect package if you're more frequently confronted with the subject!


import signal,os
p = Popen(args,stdout=PIPE)
def kill(bla,blo): os.kill(p.pid,9)
signal.signal(signal.SIGINT,kill)

while True:
____o = p.stdout.read(50)
____if o == '' and p.poll() != None: break
____logfile.write(o)
____f1 = o.rfind('\r')
____f2 = o.rfind('\n')
____if f1>=0 and (f2<0 or f2<f1):
________sys.stdout.write(o[:f1])
________sys.stdout.flush()
________sys.stdout.write(o[f1:])
____elif f2>=0:
________sys.stdout.write(o)
____else:
________sys.stdout.write(o)
________sys.stdout.flush()

Sorin Sbarnea

11:17 PM

This post has been removed by the author.
Sorin Sbarnea

11:20 PM

The above solutions are not working and I added a question on stackoverflow, maybe we'll be able to find a generic solution for this issue.

Post a Comment

Subscribe to: Post Comments (Atom)