1
2
3
4
5
6
7
8
9
10
11
12 r"""subprocess - Subprocesses with accessible I/O streams
13
14 This module allows you to spawn processes, connect to their
15 input/output/error pipes, and obtain their return codes. This module
16 intends to replace several other, older modules and functions, like:
17
18 os.system
19 os.spawn*
20 os.popen*
21 popen2.*
22 commands.*
23
24 Information about how the subprocess module can be used to replace these
25 modules and functions can be found below.
26
27
28
29 Using the subprocess module
30 ===========================
31 This module defines one class called Popen:
32
33 class Popen(args, bufsize=0, executable=None,
34 stdin=None, stdout=None, stderr=None,
35 preexec_fn=None, close_fds=False, shell=False,
36 cwd=None, env=None, universal_newlines=False,
37 startupinfo=None, creationflags=0):
38
39
40 Arguments are:
41
42 args should be a string, or a sequence of program arguments. The
43 program to execute is normally the first item in the args sequence or
44 string, but can be explicitly set by using the executable argument.
45
46 On UNIX, with shell=False (default): In this case, the Popen class
47 uses os.execvp() to execute the child program. args should normally
48 be a sequence. A string will be treated as a sequence with the string
49 as the only item (the program to execute).
50
51 On UNIX, with shell=True: If args is a string, it specifies the
52 command string to execute through the shell. If args is a sequence,
53 the first item specifies the command string, and any additional items
54 will be treated as additional shell arguments.
55
56 On Windows: the Popen class uses CreateProcess() to execute the child
57 program, which operates on strings. If args is a sequence, it will be
58 converted to a string using the list2cmdline method. Please note that
59 not all MS Windows applications interpret the command line the same
60 way: The list2cmdline is designed for applications using the same
61 rules as the MS C runtime.
62
63 bufsize, if given, has the same meaning as the corresponding argument
64 to the built-in open() function: 0 means unbuffered, 1 means line
65 buffered, any other positive value means use a buffer of
66 (approximately) that size. A negative bufsize means to use the system
67 default, which usually means fully buffered. The default value for
68 bufsize is 0 (unbuffered).
69
70 stdin, stdout and stderr specify the executed programs' standard
71 input, standard output and standard error file handles, respectively.
72 Valid values are PIPE, an existing file descriptor (a positive
73 integer), an existing file object, and None. PIPE indicates that a
74 new pipe to the child should be created. With None, no redirection
75 will occur; the child's file handles will be inherited from the
76 parent. Additionally, stderr can be STDOUT, which indicates that the
77 stderr data from the applications should be captured into the same
78 file handle as for stdout.
79
80 If preexec_fn is set to a callable object, this object will be called
81 in the child process just before the child is executed.
82
83 If close_fds is true, all file descriptors except 0, 1 and 2 will be
84 closed before the child process is executed.
85
86 if shell is true, the specified command will be executed through the
87 shell.
88
89 If cwd is not None, the current directory will be changed to cwd
90 before the child is executed.
91
92 If env is not None, it defines the environment variables for the new
93 process.
94
95 If universal_newlines is true, the file objects stdout and stderr are
96 opened as a text files, but lines may be terminated by any of '\n',
97 the Unix end-of-line convention, '\r', the Macintosh convention or
98 '\r\n', the Windows convention. All of these external representations
99 are seen as '\n' by the Python program. Note: This feature is only
100 available if Python is built with universal newline support (the
101 default). Also, the newlines attribute of the file objects stdout,
102 stdin and stderr are not updated by the communicate() method.
103
104 The startupinfo and creationflags, if given, will be passed to the
105 underlying CreateProcess() function. They can specify things such as
106 appearance of the main window and priority for the new process.
107 (Windows only)
108
109
110 This module also defines some shortcut functions:
111
112 call(*popenargs, **kwargs):
113 Run command with arguments. Wait for command to complete, then
114 return the returncode attribute.
115
116 The arguments are the same as for the Popen constructor. Example:
117
118 retcode = call(["ls", "-l"])
119
120 check_call(*popenargs, **kwargs):
121 Run command with arguments. Wait for command to complete. If the
122 exit code was zero then return, otherwise raise
123 CalledProcessError. The CalledProcessError object will have the
124 return code in the returncode attribute.
125
126 The arguments are the same as for the Popen constructor. Example:
127
128 check_call(["ls", "-l"])
129
130 check_output(*popenargs, **kwargs):
131 Run command with arguments and return its output as a byte string.
132
133 If the exit code was non-zero it raises a CalledProcessError. The
134 CalledProcessError object will have the return code in the returncode
135 attribute and output in the output attribute.
136
137 The arguments are the same as for the Popen constructor. Example:
138
139 output = check_output(["ls", "-l", "/dev/null"])
140
141
142 Exceptions
143 ----------
144 Exceptions raised in the child process, before the new program has
145 started to execute, will be re-raised in the parent. Additionally,
146 the exception object will have one extra attribute called
147 'child_traceback', which is a string containing traceback information
148 from the childs point of view.
149
150 The most common exception raised is OSError. This occurs, for
151 example, when trying to execute a non-existent file. Applications
152 should prepare for OSErrors.
153
154 A ValueError will be raised if Popen is called with invalid arguments.
155
156 check_call() and check_output() will raise CalledProcessError, if the
157 called process returns a non-zero return code.
158
159
160 Security
161 --------
162 Unlike some other popen functions, this implementation will never call
163 /bin/sh implicitly. This means that all characters, including shell
164 metacharacters, can safely be passed to child processes.
165
166
167 Popen objects
168 =============
169 Instances of the Popen class have the following methods:
170
171 poll()
172 Check if child process has terminated. Returns returncode
173 attribute.
174
175 wait()
176 Wait for child process to terminate. Returns returncode attribute.
177
178 communicate(input=None)
179 Interact with process: Send data to stdin. Read data from stdout
180 and stderr, until end-of-file is reached. Wait for process to
181 terminate. The optional input argument should be a string to be
182 sent to the child process, or None, if no data should be sent to
183 the child.
184
185 communicate() returns a tuple (stdout, stderr).
186
187 Note: The data read is buffered in memory, so do not use this
188 method if the data size is large or unlimited.
189
190 The following attributes are also available:
191
192 stdin
193 If the stdin argument is PIPE, this attribute is a file object
194 that provides input to the child process. Otherwise, it is None.
195
196 stdout
197 If the stdout argument is PIPE, this attribute is a file object
198 that provides output from the child process. Otherwise, it is
199 None.
200
201 stderr
202 If the stderr argument is PIPE, this attribute is file object that
203 provides error output from the child process. Otherwise, it is
204 None.
205
206 pid
207 The process ID of the child process.
208
209 returncode
210 The child return code. A None value indicates that the process
211 hasn't terminated yet. A negative value -N indicates that the
212 child was terminated by signal N (UNIX only).
213
214
215 Replacing older functions with the subprocess module
216 ====================================================
217 In this section, "a ==> b" means that b can be used as a replacement
218 for a.
219
220 Note: All functions in this section fail (more or less) silently if
221 the executed program cannot be found; this module raises an OSError
222 exception.
223
224 In the following examples, we assume that the subprocess module is
225 imported with "from subprocess import *".
226
227
228 Replacing /bin/sh shell backquote
229 ---------------------------------
230 output=`mycmd myarg`
231 ==>
232 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
233
234
235 Replacing shell pipe line
236 -------------------------
237 output=`dmesg | grep hda`
238 ==>
239 p1 = Popen(["dmesg"], stdout=PIPE)
240 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
241 output = p2.communicate()[0]
242
243
244 Replacing os.system()
245 ---------------------
246 sts = os.system("mycmd" + " myarg")
247 ==>
248 p = Popen("mycmd" + " myarg", shell=True)
249 pid, sts = os.waitpid(p.pid, 0)
250
251 Note:
252
253 * Calling the program through the shell is usually not required.
254
255 * It's easier to look at the returncode attribute than the
256 exitstatus.
257
258 A more real-world example would look like this:
259
260 try:
261 retcode = call("mycmd" + " myarg", shell=True)
262 if retcode < 0:
263 print >>sys.stderr, "Child was terminated by signal", -retcode
264 else:
265 print >>sys.stderr, "Child returned", retcode
266 except OSError, e:
267 print >>sys.stderr, "Execution failed:", e
268
269
270 Replacing os.spawn*
271 -------------------
272 P_NOWAIT example:
273
274 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
275 ==>
276 pid = Popen(["/bin/mycmd", "myarg"]).pid
277
278
279 P_WAIT example:
280
281 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
282 ==>
283 retcode = call(["/bin/mycmd", "myarg"])
284
285
286 Vector example:
287
288 os.spawnvp(os.P_NOWAIT, path, args)
289 ==>
290 Popen([path] + args[1:])
291
292
293 Environment example:
294
295 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
296 ==>
297 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
298
299
300 Replacing os.popen*
301 -------------------
302 pipe = os.popen("cmd", mode='r', bufsize)
303 ==>
304 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout
305
306 pipe = os.popen("cmd", mode='w', bufsize)
307 ==>
308 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin
309
310
311 (child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
312 ==>
313 p = Popen("cmd", shell=True, bufsize=bufsize,
314 stdin=PIPE, stdout=PIPE, close_fds=True)
315 (child_stdin, child_stdout) = (p.stdin, p.stdout)
316
317
318 (child_stdin,
319 child_stdout,
320 child_stderr) = os.popen3("cmd", mode, bufsize)
321 ==>
322 p = Popen("cmd", shell=True, bufsize=bufsize,
323 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
324 (child_stdin,
325 child_stdout,
326 child_stderr) = (p.stdin, p.stdout, p.stderr)
327
328
329 (child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
330 bufsize)
331 ==>
332 p = Popen("cmd", shell=True, bufsize=bufsize,
333 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
334 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
335
336 On Unix, os.popen2, os.popen3 and os.popen4 also accept a sequence as
337 the command to execute, in which case arguments will be passed
338 directly to the program without shell intervention. This usage can be
339 replaced as follows:
340
341 (child_stdin, child_stdout) = os.popen2(["/bin/ls", "-l"], mode,
342 bufsize)
343 ==>
344 p = Popen(["/bin/ls", "-l"], bufsize=bufsize, stdin=PIPE, stdout=PIPE)
345 (child_stdin, child_stdout) = (p.stdin, p.stdout)
346
347 Return code handling translates as follows:
348
349 pipe = os.popen("cmd", 'w')
350 ...
351 rc = pipe.close()
352 if rc is not None and rc % 256:
353 print "There were some errors"
354 ==>
355 process = Popen("cmd", 'w', shell=True, stdin=PIPE)
356 ...
357 process.stdin.close()
358 if process.wait() != 0:
359 print "There were some errors"
360
361
362 Replacing popen2.*
363 ------------------
364 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
365 ==>
366 p = Popen(["somestring"], shell=True, bufsize=bufsize
367 stdin=PIPE, stdout=PIPE, close_fds=True)
368 (child_stdout, child_stdin) = (p.stdout, p.stdin)
369
370 On Unix, popen2 also accepts a sequence as the command to execute, in
371 which case arguments will be passed directly to the program without
372 shell intervention. This usage can be replaced as follows:
373
374 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize,
375 mode)
376 ==>
377 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
378 stdin=PIPE, stdout=PIPE, close_fds=True)
379 (child_stdout, child_stdin) = (p.stdout, p.stdin)
380
381 The popen2.Popen3 and popen2.Popen4 basically works as subprocess.Popen,
382 except that:
383
384 * subprocess.Popen raises an exception if the execution fails
385 * the capturestderr argument is replaced with the stderr argument.
386 * stdin=PIPE and stdout=PIPE must be specified.
387 * popen2 closes all filedescriptors by default, but you have to specify
388 close_fds=True with subprocess.Popen.
389 """
390
391 import sys
392 mswindows = (sys.platform == "win32")
393
394 import os
395 import types
396 import traceback
397 import gc
398 import signal
399 import errno
400
401 try:
402 set
403 except NameError:
404 from sets import Set as set
405
406
407
408
410
411 """This exception is raised when a process run by check_call() or
412 check_output() returns a non-zero exit status.
413 The exit status will be stored in the returncode attribute;
414 check_output() will also store the output in the output attribute.
415 """
416
417 - def __init__(self, returncode, cmd, output=None):
418 self.returncode = returncode
419 self.cmd = cmd
420 self.output = output
421
423 return "Command '%s' returned non-zero exit status %d" % (
424 self.cmd, self.returncode)
425
426
427 if mswindows:
428 import threading
429 import msvcrt
430 import _subprocess
431
438
441 else:
442 import select
443 _has_poll = hasattr(select, 'poll')
444 import fcntl
445 import pickle
446
447
448
449
450 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
451
452
453 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call",
454 "check_output", "CalledProcessError"]
455
456 if mswindows:
457 from _subprocess import CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP, \
458 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, \
459 STD_ERROR_HANDLE, SW_HIDE, \
460 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW
461
462 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
463 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
464 "STD_ERROR_HANDLE", "SW_HIDE",
465 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
466 try:
467 MAXFD = os.sysconf("SC_OPEN_MAX")
468 except:
469 MAXFD = 256
470
471 _active = []
472
473
475 for inst in _active[:]:
476 res = inst._internal_poll(_deadstate=sys.maxint)
477 if res is not None:
478 try:
479 _active.remove(inst)
480 except ValueError:
481
482
483 pass
484
485 PIPE = -1
486 STDOUT = -2
487
488
490 while True:
491 try:
492 return func(*args)
493 except (OSError, IOError), e:
494 if e.errno == errno.EINTR:
495 continue
496 raise
497
498
499 -def call(*popenargs, **kwargs):
500 """Run command with arguments. Wait for command to complete, then
501 return the returncode attribute.
502
503 The arguments are the same as for the Popen constructor. Example:
504
505 retcode = call(["ls", "-l"])
506 """
507 return Popen(*popenargs, **kwargs).wait()
508
509
511 """Run command with arguments. Wait for command to complete. If
512 the exit code was zero then return, otherwise raise
513 CalledProcessError. The CalledProcessError object will have the
514 return code in the returncode attribute.
515
516 The arguments are the same as for the Popen constructor. Example:
517
518 check_call(["ls", "-l"])
519 """
520 retcode = call(*popenargs, **kwargs)
521 if retcode:
522 cmd = kwargs.get("args")
523 if cmd is None:
524 cmd = popenargs[0]
525 raise CalledProcessError(retcode, cmd)
526 return 0
527
528
530 r"""Run command with arguments and return its output as a byte string.
531
532 If the exit code was non-zero it raises a CalledProcessError. The
533 CalledProcessError object will have the return code in the returncode
534 attribute and output in the output attribute.
535
536 The arguments are the same as for the Popen constructor. Example:
537
538 >>> check_output(["ls", "-l", "/dev/null"])
539 'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
540
541 The stdout argument is not allowed as it is used internally.
542 To capture standard error in the result, use stderr=STDOUT.
543
544 >>> check_output(["/bin/sh", "-c",
545 ... "ls -l non_existent_file ; exit 0"],
546 ... stderr=STDOUT)
547 'ls: non_existent_file: No such file or directory\n'
548 """
549 if 'stdout' in kwargs:
550 raise ValueError('stdout argument not allowed, it will be overridden.')
551 process = Popen(stdout=PIPE, *popenargs, **kwargs)
552 output, unused_err = process.communicate()
553 retcode = process.poll()
554 if retcode:
555 cmd = kwargs.get("args")
556 if cmd is None:
557 cmd = popenargs[0]
558 raise CalledProcessError(retcode, cmd, output=output)
559 return output
560
561
563 """
564 Translate a sequence of arguments into a command line
565 string, using the same rules as the MS C runtime:
566
567 1) Arguments are delimited by white space, which is either a
568 space or a tab.
569
570 2) A string surrounded by double quotation marks is
571 interpreted as a single argument, regardless of white space
572 contained within. A quoted string can be embedded in an
573 argument.
574
575 3) A double quotation mark preceded by a backslash is
576 interpreted as a literal double quotation mark.
577
578 4) Backslashes are interpreted literally, unless they
579 immediately precede a double quotation mark.
580
581 5) If backslashes immediately precede a double quotation mark,
582 every pair of backslashes is interpreted as a literal
583 backslash. If the number of backslashes is odd, the last
584 backslash escapes the next double quotation mark as
585 described in rule 3.
586 """
587
588
589
590
591
592 result = []
593 needquote = False
594 for arg in seq:
595 bs_buf = []
596
597
598 if result:
599 result.append(' ')
600
601 needquote = (" " in arg) or ("\t" in arg) or not arg
602 if needquote:
603 result.append('"')
604
605 for c in arg:
606 if c == '\\':
607
608 bs_buf.append(c)
609 elif c == '"':
610
611 result.append('\\' * len(bs_buf) * 2)
612 bs_buf = []
613 result.append('\\"')
614 else:
615
616 if bs_buf:
617 result.extend(bs_buf)
618 bs_buf = []
619 result.append(c)
620
621
622 if bs_buf:
623 result.extend(bs_buf)
624
625 if needquote:
626 result.extend(bs_buf)
627 result.append('"')
628
629 return ''.join(result)
630
631
633
634 - def __init__(self, args, bufsize=0, executable=None,
635 stdin=None, stdout=None, stderr=None,
636 preexec_fn=None, close_fds=False, shell=False,
637 cwd=None, env=None, universal_newlines=False,
638 startupinfo=None, creationflags=0):
639 """Create new Popen instance."""
640 _cleanup()
641
642 self._child_created = False
643 if not isinstance(bufsize, (int, long)):
644 raise TypeError("bufsize must be an integer")
645
646 if mswindows:
647 if preexec_fn is not None:
648 raise ValueError("preexec_fn is not supported on Windows "
649 "platforms")
650 if close_fds and (stdin is not None or stdout is not None or
651 stderr is not None):
652 raise ValueError("close_fds is not supported on Windows "
653 "platforms if you redirect "
654 "stdin/stdout/stderr")
655 else:
656
657 if startupinfo is not None:
658 raise ValueError("startupinfo is only supported on Windows "
659 "platforms")
660 if creationflags != 0:
661 raise ValueError("creationflags is only supported on Windows "
662 "platforms")
663
664 self.stdin = None
665 self.stdout = None
666 self.stderr = None
667 self.pid = None
668 self.returncode = None
669 self.universal_newlines = universal_newlines
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686 (p2cread, p2cwrite,
687 c2pread, c2pwrite,
688 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
689
690 self._execute_child(args, executable, preexec_fn, close_fds,
691 cwd, env, universal_newlines,
692 startupinfo, creationflags, shell,
693 p2cread, p2cwrite,
694 c2pread, c2pwrite,
695 errread, errwrite)
696
697 if mswindows:
698 if p2cwrite is not None:
699 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
700 if c2pread is not None:
701 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
702 if errread is not None:
703 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
704
705 if p2cwrite is not None:
706 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)
707 if c2pread is not None:
708 if universal_newlines:
709 self.stdout = os.fdopen(c2pread, 'rU', bufsize)
710 else:
711 self.stdout = os.fdopen(c2pread, 'rb', bufsize)
712 if errread is not None:
713 if universal_newlines:
714 self.stderr = os.fdopen(errread, 'rU', bufsize)
715 else:
716 self.stderr = os.fdopen(errread, 'rb', bufsize)
717
722
724
725
726
727 if not getattr(self, '_child_created', False):
728
729 return
730
731 self._internal_poll(_deadstate=_maxint)
732 if self.returncode is None and _active is not None:
733
734 _active.append(self)
735
737 """Interact with process: Send data to stdin. Read data from
738 stdout and stderr, until end-of-file is reached. Wait for
739 process to terminate. The optional input argument should be a
740 string to be sent to the child process, or None, if no data
741 should be sent to the child.
742
743 communicate() returns a tuple (stdout, stderr)."""
744
745
746
747 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
748 stdout = None
749 stderr = None
750 if self.stdin:
751 if input:
752 try:
753 self.stdin.write(input)
754 except IOError, e:
755 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
756 raise
757 self.stdin.close()
758 elif self.stdout:
759 stdout = _eintr_retry_call(self.stdout.read)
760 self.stdout.close()
761 elif self.stderr:
762 stderr = _eintr_retry_call(self.stderr.read)
763 self.stderr.close()
764 self.wait()
765 return (stdout, stderr)
766
767 return self._communicate(input)
768
771
772 if mswindows:
773
774
775
777 """Construct and return tuple with IO objects:
778 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
779 """
780 if stdin is None and stdout is None and stderr is None:
781 return (None, None, None, None, None, None)
782
783 p2cread, p2cwrite = None, None
784 c2pread, c2pwrite = None, None
785 errread, errwrite = None, None
786
787 if stdin is None:
788 p2cread = _subprocess.GetStdHandle(
789 _subprocess.STD_INPUT_HANDLE)
790 if p2cread is None:
791 p2cread, _ = _subprocess.CreatePipe(None, 0)
792 elif stdin == PIPE:
793 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
794 elif isinstance(stdin, int):
795 p2cread = msvcrt.get_osfhandle(stdin)
796 else:
797
798 p2cread = msvcrt.get_osfhandle(stdin.fileno())
799 p2cread = self._make_inheritable(p2cread)
800
801 if stdout is None:
802 c2pwrite = _subprocess.GetStdHandle(
803 _subprocess.STD_OUTPUT_HANDLE)
804 if c2pwrite is None:
805 _, c2pwrite = _subprocess.CreatePipe(None, 0)
806 elif stdout == PIPE:
807 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
808 elif isinstance(stdout, int):
809 c2pwrite = msvcrt.get_osfhandle(stdout)
810 else:
811
812 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
813 c2pwrite = self._make_inheritable(c2pwrite)
814
815 if stderr is None:
816 errwrite = _subprocess.GetStdHandle(
817 _subprocess.STD_ERROR_HANDLE)
818 if errwrite is None:
819 _, errwrite = _subprocess.CreatePipe(None, 0)
820 elif stderr == PIPE:
821 errread, errwrite = _subprocess.CreatePipe(None, 0)
822 elif stderr == STDOUT:
823 errwrite = c2pwrite
824 elif isinstance(stderr, int):
825 errwrite = msvcrt.get_osfhandle(stderr)
826 else:
827
828 errwrite = msvcrt.get_osfhandle(stderr.fileno())
829 errwrite = self._make_inheritable(errwrite)
830
831 return (p2cread, p2cwrite,
832 c2pread, c2pwrite,
833 errread, errwrite)
834
836 """Return a duplicate of handle, which is inheritable"""
837 return _subprocess.DuplicateHandle(
838 _subprocess.GetCurrentProcess(),
839 handle,
840 _subprocess.GetCurrentProcess(),
841 0,
842 1,
843 _subprocess.DUPLICATE_SAME_ACCESS
844 )
845
847 """Find and return absolut path to w9xpopen.exe"""
848 w9xpopen = os.path.join(
849 os.path.dirname(_subprocess.GetModuleFileName(0)),
850 "w9xpopen.exe")
851 if not os.path.exists(w9xpopen):
852
853
854 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
855 "w9xpopen.exe")
856 if not os.path.exists(w9xpopen):
857 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
858 "needed for Popen to work with your "
859 "shell or platform.")
860 return w9xpopen
861
862 - def _execute_child(self, args, executable, preexec_fn, close_fds,
863 cwd, env, universal_newlines,
864 startupinfo, creationflags, shell,
865 p2cread, p2cwrite,
866 c2pread, c2pwrite,
867 errread, errwrite):
868 """Execute program (MS Windows version)"""
869
870 if not isinstance(args, types.StringTypes):
871 args = list2cmdline(args)
872
873
874 if startupinfo is None:
875 startupinfo = STARTUPINFO()
876 if None not in (p2cread, c2pwrite, errwrite):
877 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
878 startupinfo.hStdInput = p2cread
879 startupinfo.hStdOutput = c2pwrite
880 startupinfo.hStdError = errwrite
881
882 if shell:
883 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
884 startupinfo.wShowWindow = _subprocess.SW_HIDE
885 comspec = os.environ.get("COMSPEC", "cmd.exe")
886 args = '{} /c "{}"'.format(comspec, args)
887 if (_subprocess.GetVersion() >= 0x80000000 or
888 os.path.basename(comspec).lower() == "command.com"):
889
890
891
892
893 w9xpopen = self._find_w9xpopen()
894 args = '"%s" %s' % (w9xpopen, args)
895
896
897
898
899
900
901 creationflags |= _subprocess.CREATE_NEW_CONSOLE
902
903
904 try:
905 try:
906 hp, ht, pid, tid = _subprocess.CreateProcess(
907 executable, args,
908
909
910 None, None,
911 int(not close_fds),
912 creationflags,
913 env,
914 cwd,
915 startupinfo)
916 except pywintypes.error, e:
917
918
919
920
921 raise WindowsError(*e.args)
922 finally:
923
924
925
926
927
928
929 if p2cread is not None:
930 p2cread.Close()
931 if c2pwrite is not None:
932 c2pwrite.Close()
933 if errwrite is not None:
934 errwrite.Close()
935
936
937 self._child_created = True
938 self._handle = hp
939 self.pid = pid
940 ht.Close()
941
942 - def _internal_poll(
943 self, _deadstate=None,
944 _WaitForSingleObject=_subprocess.WaitForSingleObject,
945 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
946 _GetExitCodeProcess=_subprocess.GetExitCodeProcess
947 ):
948 """Check if child process has terminated. Returns returncode
949 attribute.
950
951 This method is called by __del__, so it can only refer to objects
952 in its local scope.
953
954 """
955 if self.returncode is None:
956 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
957 self.returncode = _GetExitCodeProcess(self._handle)
958 return self.returncode
959
961 """Wait for child process to terminate. Returns returncode
962 attribute."""
963 if self.returncode is None:
964 _subprocess.WaitForSingleObject(self._handle,
965 _subprocess.INFINITE)
966 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
967 return self.returncode
968
970 buffer.append(fh.read())
971
973 stdout = None
974 stderr = None
975
976 if self.stdout:
977 stdout = []
978 stdout_thread = threading.Thread(target=self._readerthread,
979 args=(self.stdout, stdout))
980 stdout_thread.setDaemon(True)
981 stdout_thread.start()
982 if self.stderr:
983 stderr = []
984 stderr_thread = threading.Thread(target=self._readerthread,
985 args=(self.stderr, stderr))
986 stderr_thread.setDaemon(True)
987 stderr_thread.start()
988
989 if self.stdin:
990 if input is not None:
991 try:
992 self.stdin.write(input)
993 except IOError, e:
994 if e.errno != errno.EPIPE:
995 raise
996 self.stdin.close()
997
998 if self.stdout:
999 stdout_thread.join()
1000 if self.stderr:
1001 stderr_thread.join()
1002
1003
1004 if stdout is not None:
1005 stdout = stdout[0]
1006 if stderr is not None:
1007 stderr = stderr[0]
1008
1009
1010
1011
1012
1013 if self.universal_newlines and hasattr(file, 'newlines'):
1014 if stdout:
1015 stdout = self._translate_newlines(stdout)
1016 if stderr:
1017 stderr = self._translate_newlines(stderr)
1018
1019 self.wait()
1020 return (stdout, stderr)
1021
1023 """Send a signal to the process
1024 """
1025 if sig == signal.SIGTERM:
1026 self.terminate()
1027 elif sig == signal.CTRL_C_EVENT:
1028 os.kill(self.pid, signal.CTRL_C_EVENT)
1029 elif sig == signal.CTRL_BREAK_EVENT:
1030 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
1031 else:
1032 raise ValueError("Unsupported signal: {}".format(sig))
1033
1035 """Terminates the process
1036 """
1037 _subprocess.TerminateProcess(self._handle, 1)
1038
1039 kill = terminate
1040
1041 else:
1042
1043
1044
1046 """Construct and return tuple with IO objects:
1047 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1048 """
1049 p2cread, p2cwrite = None, None
1050 c2pread, c2pwrite = None, None
1051 errread, errwrite = None, None
1052
1053 if stdin is None:
1054 pass
1055 elif stdin == PIPE:
1056 p2cread, p2cwrite = self.pipe_cloexec()
1057 elif isinstance(stdin, int):
1058 p2cread = stdin
1059 else:
1060
1061 p2cread = stdin.fileno()
1062
1063 if stdout is None:
1064 pass
1065 elif stdout == PIPE:
1066 c2pread, c2pwrite = self.pipe_cloexec()
1067 elif isinstance(stdout, int):
1068 c2pwrite = stdout
1069 else:
1070
1071 c2pwrite = stdout.fileno()
1072
1073 if stderr is None:
1074 pass
1075 elif stderr == PIPE:
1076 errread, errwrite = self.pipe_cloexec()
1077 elif stderr == STDOUT:
1078 errwrite = c2pwrite
1079 elif isinstance(stderr, int):
1080 errwrite = stderr
1081 else:
1082
1083 errwrite = stderr.fileno()
1084
1085 return (p2cread, p2cwrite,
1086 c2pread, c2pwrite,
1087 errread, errwrite)
1088
1090 try:
1091 cloexec_flag = fcntl.FD_CLOEXEC
1092 except AttributeError:
1093 cloexec_flag = 1
1094
1095 old = fcntl.fcntl(fd, fcntl.F_GETFD)
1096 if cloexec:
1097 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
1098 else:
1099 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~cloexec_flag)
1100
1102 """Create a pipe with FDs set CLOEXEC."""
1103
1104
1105
1106
1107
1108 r, w = os.pipe()
1109 self._set_cloexec_flag(r)
1110 self._set_cloexec_flag(w)
1111 return r, w
1112
1114 if hasattr(os, 'closerange'):
1115 os.closerange(3, but)
1116 os.closerange(but + 1, MAXFD)
1117 else:
1118 for i in xrange(3, MAXFD):
1119 if i == but:
1120 continue
1121 try:
1122 os.close(i)
1123 except:
1124 pass
1125
1126 - def _execute_child(self, args, executable, preexec_fn, close_fds,
1127 cwd, env, universal_newlines,
1128 startupinfo, creationflags, shell,
1129 p2cread, p2cwrite,
1130 c2pread, c2pwrite,
1131 errread, errwrite):
1132 """Execute program (POSIX version)"""
1133
1134 if isinstance(args, types.StringTypes):
1135 args = [args]
1136 else:
1137 args = list(args)
1138
1139 if shell:
1140 args = ["/bin/sh", "-c"] + args
1141 if executable:
1142 args[0] = executable
1143
1144 if executable is None:
1145 executable = args[0]
1146
1147
1148
1149
1150 errpipe_read, errpipe_write = self.pipe_cloexec()
1151 try:
1152 try:
1153 gc_was_enabled = gc.isenabled()
1154
1155
1156
1157 gc.disable()
1158 try:
1159 self.pid = os.fork()
1160 except:
1161 if gc_was_enabled:
1162 gc.enable()
1163 raise
1164 self._child_created = True
1165 if self.pid == 0:
1166
1167 try:
1168
1169 if p2cwrite is not None:
1170 os.close(p2cwrite)
1171 if c2pread is not None:
1172 os.close(c2pread)
1173 if errread is not None:
1174 os.close(errread)
1175 os.close(errpipe_read)
1176
1177
1178
1179
1180 if c2pwrite == 0:
1181 c2pwrite = os.dup(c2pwrite)
1182 if errwrite == 0 or errwrite == 1:
1183 errwrite = os.dup(errwrite)
1184
1185
1186 def _dup2(a, b):
1187
1188
1189
1190 if a == b:
1191 self._set_cloexec_flag(a, False)
1192 elif a is not None:
1193 os.dup2(a, b)
1194 _dup2(p2cread, 0)
1195 _dup2(c2pwrite, 1)
1196 _dup2(errwrite, 2)
1197
1198
1199
1200 closed = set([None])
1201 for fd in [p2cread, c2pwrite, errwrite]:
1202 if fd not in closed and fd > 2:
1203 os.close(fd)
1204 closed.add(fd)
1205
1206
1207 if close_fds:
1208 self._close_fds(but=errpipe_write)
1209
1210 if cwd is not None:
1211 os.chdir(cwd)
1212
1213 if preexec_fn:
1214 preexec_fn()
1215
1216 if env is None:
1217 os.execvp(executable, args)
1218 else:
1219 os.execvpe(executable, args, env)
1220
1221 except:
1222 exc_type, exc_value, tb = sys.exc_info()
1223
1224
1225 exc_lines = traceback.format_exception(exc_type,
1226 exc_value,
1227 tb)
1228 exc_value.child_traceback = ''.join(exc_lines)
1229 os.write(errpipe_write, pickle.dumps(exc_value))
1230
1231
1232
1233 os._exit(255)
1234
1235
1236 if gc_was_enabled:
1237 gc.enable()
1238 finally:
1239
1240 os.close(errpipe_write)
1241
1242 if p2cread is not None and p2cwrite is not None:
1243 os.close(p2cread)
1244 if c2pwrite is not None and c2pread is not None:
1245 os.close(c2pwrite)
1246 if errwrite is not None and errread is not None:
1247 os.close(errwrite)
1248
1249
1250
1251 data = _eintr_retry_call(os.read, errpipe_read, 1048576)
1252 finally:
1253
1254 os.close(errpipe_read)
1255
1256 if data != "":
1257 try:
1258 _eintr_retry_call(os.waitpid, self.pid, 0)
1259 except OSError, e:
1260 if e.errno != errno.ECHILD:
1261 raise
1262 child_exception = pickle.loads(data)
1263 for fd in (p2cwrite, c2pread, errread):
1264 if fd is not None:
1265 os.close(fd)
1266 raise child_exception
1267
1268 - def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1269 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1270 _WEXITSTATUS=os.WEXITSTATUS):
1271
1272
1273 if _WIFSIGNALED(sts):
1274 self.returncode = -_WTERMSIG(sts)
1275 elif _WIFEXITED(sts):
1276 self.returncode = _WEXITSTATUS(sts)
1277 else:
1278
1279 raise RuntimeError("Unknown child exit status!")
1280
1281 - def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1282 _WNOHANG=os.WNOHANG, _os_error=os.error):
1283 """Check if child process has terminated. Returns returncode
1284 attribute.
1285
1286 This method is called by __del__, so it cannot reference anything
1287 outside of the local scope (nor can any methods it calls).
1288
1289 """
1290 if self.returncode is None:
1291 try:
1292 pid, sts = _waitpid(self.pid, _WNOHANG)
1293 if pid == self.pid:
1294 self._handle_exitstatus(sts)
1295 except _os_error:
1296 if _deadstate is not None:
1297 self.returncode = _deadstate
1298 return self.returncode
1299
1301 """Wait for child process to terminate. Returns returncode
1302 attribute."""
1303 if self.returncode is None:
1304 try:
1305 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1306 except OSError, e:
1307 if e.errno != errno.ECHILD:
1308 raise
1309
1310
1311
1312 sts = 0
1313 self._handle_exitstatus(sts)
1314 return self.returncode
1315
1317 if self.stdin:
1318
1319
1320 self.stdin.flush()
1321 if not input:
1322 self.stdin.close()
1323
1324 if _has_poll:
1325 stdout, stderr = self._communicate_with_poll(input)
1326 else:
1327 stdout, stderr = self._communicate_with_select(input)
1328
1329
1330 if stdout is not None:
1331 stdout = ''.join(stdout)
1332 if stderr is not None:
1333 stderr = ''.join(stderr)
1334
1335
1336
1337
1338
1339 if self.universal_newlines and hasattr(file, 'newlines'):
1340 if stdout:
1341 stdout = self._translate_newlines(stdout)
1342 if stderr:
1343 stderr = self._translate_newlines(stderr)
1344
1345 self.wait()
1346 return (stdout, stderr)
1347
1349 stdout = None
1350 stderr = None
1351 fd2file = {}
1352 fd2output = {}
1353
1354 poller = select.poll()
1355
1356 def register_and_append(file_obj, eventmask):
1357 poller.register(file_obj.fileno(), eventmask)
1358 fd2file[file_obj.fileno()] = file_obj
1359
1360 def close_unregister_and_remove(fd):
1361 poller.unregister(fd)
1362 fd2file[fd].close()
1363 fd2file.pop(fd)
1364
1365 if self.stdin and input:
1366 register_and_append(self.stdin, select.POLLOUT)
1367
1368 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1369 if self.stdout:
1370 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1371 fd2output[self.stdout.fileno()] = stdout = []
1372 if self.stderr:
1373 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1374 fd2output[self.stderr.fileno()] = stderr = []
1375
1376 input_offset = 0
1377 while fd2file:
1378 try:
1379 ready = poller.poll()
1380 except select.error, e:
1381 if e.args[0] == errno.EINTR:
1382 continue
1383 raise
1384
1385 for fd, mode in ready:
1386 if mode & select.POLLOUT:
1387 chunk = input[input_offset: input_offset + _PIPE_BUF]
1388 try:
1389 input_offset += os.write(fd, chunk)
1390 except OSError, e:
1391 if e.errno == errno.EPIPE:
1392 close_unregister_and_remove(fd)
1393 else:
1394 raise
1395 else:
1396 if input_offset >= len(input):
1397 close_unregister_and_remove(fd)
1398 elif mode & select_POLLIN_POLLPRI:
1399 data = os.read(fd, 4096)
1400 if not data:
1401 close_unregister_and_remove(fd)
1402 fd2output[fd].append(data)
1403 else:
1404
1405 close_unregister_and_remove(fd)
1406
1407 return (stdout, stderr)
1408
1410 read_set = []
1411 write_set = []
1412 stdout = None
1413 stderr = None
1414
1415 if self.stdin and input:
1416 write_set.append(self.stdin)
1417 if self.stdout:
1418 read_set.append(self.stdout)
1419 stdout = []
1420 if self.stderr:
1421 read_set.append(self.stderr)
1422 stderr = []
1423
1424 input_offset = 0
1425 while read_set or write_set:
1426 try:
1427 rlist, wlist, xlist = select.select(
1428 read_set, write_set, [])
1429 except select.error, e:
1430 if e.args[0] == errno.EINTR:
1431 continue
1432 raise
1433
1434 if self.stdin in wlist:
1435 chunk = input[input_offset: input_offset + _PIPE_BUF]
1436 try:
1437 bytes_written = os.write(self.stdin.fileno(), chunk)
1438 except OSError, e:
1439 if e.errno == errno.EPIPE:
1440 self.stdin.close()
1441 write_set.remove(self.stdin)
1442 else:
1443 raise
1444 else:
1445 input_offset += bytes_written
1446 if input_offset >= len(input):
1447 self.stdin.close()
1448 write_set.remove(self.stdin)
1449
1450 if self.stdout in rlist:
1451 data = os.read(self.stdout.fileno(), 1024)
1452 if data == "":
1453 self.stdout.close()
1454 read_set.remove(self.stdout)
1455 stdout.append(data)
1456
1457 if self.stderr in rlist:
1458 data = os.read(self.stderr.fileno(), 1024)
1459 if data == "":
1460 self.stderr.close()
1461 read_set.remove(self.stderr)
1462 stderr.append(data)
1463
1464 return (stdout, stderr)
1465
1467 """Send a signal to the process
1468 """
1469 os.kill(self.pid, sig)
1470
1472 """Terminate the process with SIGTERM
1473 """
1474 self.send_signal(signal.SIGTERM)
1475
1477 """Kill the process with SIGKILL
1478 """
1479 self.send_signal(signal.SIGKILL)
1480
1481
1483
1484
1485
1486 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
1487 print "Process list:"
1488 print plist
1489
1490
1491
1492
1493 if os.getuid() == 0:
1494 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1495 p.wait()
1496
1497
1498
1499
1500 print "Looking for 'hda'..."
1501 p1 = Popen(["dmesg"], stdout=PIPE)
1502 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
1503 print repr(p2.communicate()[0])
1504
1505
1506
1507
1508 print
1509 print "Trying a weird file..."
1510 try:
1511 print Popen(["/this/path/does/not/exist"]).communicate()
1512 except OSError, e:
1513 if e.errno == errno.ENOENT:
1514 print "The file didn't exist. I thought so..."
1515 print "Child traceback:"
1516 print e.child_traceback
1517 else:
1518 print "Error", e.errno
1519 else:
1520 print >>sys.stderr, "Gosh. No error."
1521
1522
1524
1525
1526
1527 print "Looking for 'PROMPT' in set output..."
1528 p1 = Popen("set", stdout=PIPE, shell=True)
1529 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
1530 print repr(p2.communicate()[0])
1531
1532
1533
1534
1535 print "Executing calc..."
1536 p = Popen("calc")
1537 p.wait()
1538
1539
1540 if __name__ == "__main__":
1541 if mswindows:
1542 _demo_windows()
1543 else:
1544 _demo_posix()
1545