| \section{\module{subprocess} --- Subprocess management} |
| |
| \declaremodule{standard}{subprocess} |
| \modulesynopsis{Subprocess management.} |
| \moduleauthor{Peter \AA strand}{astrand@lysator.liu.se} |
| \sectionauthor{Peter \AA strand}{astrand@lysator.liu.se} |
| |
| \versionadded{2.4} |
| |
| The \module{subprocess} module allows you to spawn new processes, |
| connect to their input/output/error pipes, and obtain their return |
| codes. This module intends to replace several other, older modules |
| and functions, such as: |
| |
| % XXX Should add pointers to this module to at least the popen2 |
| % and commands sections. |
| |
| \begin{verbatim} |
| os.system |
| os.spawn* |
| os.popen* |
| popen2.* |
| commands.* |
| \end{verbatim} |
| |
| Information about how the \module{subprocess} module can be used to |
| replace these modules and functions can be found in the following |
| sections. |
| |
| \subsection{Using the subprocess Module} |
| |
| This module defines one class called \class{Popen}: |
| |
| \begin{classdesc}{Popen}{args, bufsize=0, executable=None, |
| stdin=None, stdout=None, stderr=None, |
| preexec_fn=None, close_fds=False, shell=False, |
| cwd=None, env=None, universal_newlines=False, |
| startupinfo=None, creationflags=0} |
| |
| Arguments are: |
| |
| \var{args} should be a string, or a sequence of program arguments. The |
| program to execute is normally the first item in the args sequence or |
| string, but can be explicitly set by using the executable argument. |
| |
| On \UNIX{}, with \var{shell=False} (default): In this case, the Popen |
| class uses \method{os.execvp()} to execute the child program. |
| \var{args} should normally be a sequence. A string will be treated as a |
| sequence with the string as the only item (the program to execute). |
| |
| On \UNIX{}, with \var{shell=True}: If args is a string, it specifies the |
| command string to execute through the shell. If \var{args} is a |
| sequence, the first item specifies the command string, and any |
| additional items will be treated as additional shell arguments. |
| |
| On Windows: the \class{Popen} class uses CreateProcess() to execute |
| the child program, which operates on strings. If \var{args} is a |
| sequence, it will be converted to a string using the |
| \method{list2cmdline} method. Please note that not all MS Windows |
| applications interpret the command line the same way: |
| \method{list2cmdline} is designed for applications using the same |
| rules as the MS C runtime. |
| |
| \var{bufsize}, if given, has the same meaning as the corresponding |
| argument to the built-in open() function: \constant{0} means unbuffered, |
| \constant{1} means line buffered, any other positive value means use a |
| buffer of (approximately) that size. A negative \var{bufsize} means to |
| use the system default, which usually means fully buffered. The default |
| value for \var{bufsize} is \constant{0} (unbuffered). |
| |
| The \var{executable} argument specifies the program to execute. It is |
| very seldom needed: Usually, the program to execute is defined by the |
| \var{args} argument. If \var{shell=True}, the \var{executable} |
| argument specifies which shell to use. On \UNIX{}, the default shell |
| is /bin/sh. On Windows, the default shell is specified by the COMSPEC |
| environment variable. |
| |
| \var{stdin}, \var{stdout} and \var{stderr} specify the executed |
| programs' standard input, standard output and standard error file |
| handles, respectively. Valid values are \code{PIPE}, an existing file |
| descriptor (a positive integer), an existing file object, and |
| \code{None}. \code{PIPE} indicates that a new pipe to the child |
| should be created. With \code{None}, no redirection will occur; the |
| child's file handles will be inherited from the parent. Additionally, |
| \var{stderr} can be \code{STDOUT}, which indicates that the stderr |
| data from the applications should be captured into the same file |
| handle as for stdout. |
| |
| If \var{preexec_fn} is set to a callable object, this object will be |
| called in the child process just before the child is executed. |
| |
| If \var{close_fds} is true, all file descriptors except \constant{0}, |
| \constant{1} and \constant{2} will be closed before the child process is |
| executed. |
| |
| If \var{shell} is \constant{True}, the specified command will be |
| executed through the shell. |
| |
| If \var{cwd} is not \code{None}, the current directory will be changed |
| to cwd before the child is executed. |
| |
| If \var{env} is not \code{None}, it defines the environment variables |
| for the new process. |
| |
| If \var{universal_newlines} is \constant{True}, the file objects stdout |
| and stderr are opened as a text files, but lines may be terminated by |
| any of \code{'\e n'}, the Unix end-of-line convention, \code{'\e r'}, |
| the Macintosh convention or \code{'\e r\e n'}, the Windows convention. |
| All of these external representations are seen as \code{'\e n'} by the |
| Python program. \note{This feature is only available if Python is built |
| with universal newline support (the default). Also, the newlines |
| attribute of the file objects \member{stdout}, \member{stdin} and |
| \member{stderr} are not updated by the communicate() method.} |
| |
| The \var{startupinfo} and \var{creationflags}, if given, will be |
| passed to the underlying CreateProcess() function. They can specify |
| things such as appearance of the main window and priority for the new |
| process. (Windows only) |
| \end{classdesc} |
| |
| \subsubsection{Convenience Functions} |
| |
| This module also defines one shortcut function: |
| |
| \begin{funcdesc}{call}{*args, **kwargs} |
| Run command with arguments. Wait for command to complete, then |
| return the \member{returncode} attribute. |
| |
| The arguments are the same as for the Popen constructor. Example: |
| |
| \begin{verbatim} |
| retcode = call(["ls", "-l"]) |
| \end{verbatim} |
| \end{funcdesc} |
| |
| |
| \subsubsection{Exceptions} |
| |
| Exceptions raised in the child process, before the new program has |
| started to execute, will be re-raised in the parent. Additionally, |
| the exception object will have one extra attribute called |
| \member{child_traceback}, which is a string containing traceback |
| information from the childs point of view. |
| |
| The most common exception raised is \exception{OSError}. This occurs, |
| for example, when trying to execute a non-existent file. Applications |
| should prepare for \exception{OSError} exceptions. |
| |
| A \exception{ValueError} will be raised if \class{Popen} is called |
| with invalid arguments. |
| |
| |
| \subsubsection{Security} |
| |
| Unlike some other popen functions, this implementation will never call |
| /bin/sh implicitly. This means that all characters, including shell |
| metacharacters, can safely be passed to child processes. |
| |
| |
| \subsection{Popen Objects} |
| |
| Instances of the \class{Popen} class have the following methods: |
| |
| \begin{methoddesc}{poll}{} |
| Check if child process has terminated. Returns returncode |
| attribute. |
| \end{methoddesc} |
| |
| \begin{methoddesc}{wait}{} |
| Wait for child process to terminate. Returns returncode attribute. |
| \end{methoddesc} |
| |
| \begin{methoddesc}{communicate}{input=None} |
| Interact with process: Send data to stdin. Read data from stdout and |
| stderr, until end-of-file is reached. Wait for process to terminate. |
| The optional \var{stdin} argument should be a string to be sent to the |
| child process, or \code{None}, if no data should be sent to the child. |
| |
| communicate() returns a tuple (stdout, stderr). |
| |
| \note{The data read is buffered in memory, so do not use this method |
| if the data size is large or unlimited.} |
| \end{methoddesc} |
| |
| The following attributes are also available: |
| |
| \begin{memberdesc}{stdin} |
| If the \var{stdin} argument is \code{PIPE}, this attribute is a file |
| object that provides input to the child process. Otherwise, it is |
| \code{None}. |
| \end{memberdesc} |
| |
| \begin{memberdesc}{stdout} |
| If the \var{stdout} argument is \code{PIPE}, this attribute is a file |
| object that provides output from the child process. Otherwise, it is |
| \code{None}. |
| \end{memberdesc} |
| |
| \begin{memberdesc}{stderr} |
| If the \var{stderr} argument is \code{PIPE}, this attribute is file |
| object that provides error output from the child process. Otherwise, |
| it is \code{None}. |
| \end{memberdesc} |
| |
| \begin{memberdesc}{pid} |
| The process ID of the child process. |
| \end{memberdesc} |
| |
| \begin{memberdesc}{returncode} |
| The child return code. A \code{None} value indicates that the process |
| hasn't terminated yet. A negative value -N indicates that the child |
| was terminated by signal N (\UNIX{} only). |
| \end{memberdesc} |
| |
| |
| \subsection{Replacing Older Functions with the subprocess Module} |
| |
| In this section, "a ==> b" means that b can be used as a replacement |
| for a. |
| |
| \note{All functions in this section fail (more or less) silently if |
| the executed program cannot be found; this module raises an |
| \exception{OSError} exception.} |
| |
| In the following examples, we assume that the subprocess module is |
| imported with "from subprocess import *". |
| |
| \subsubsection{Replacing /bin/sh shell backquote} |
| |
| \begin{verbatim} |
| output=`mycmd myarg` |
| ==> |
| output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0] |
| \end{verbatim} |
| |
| \subsubsection{Replacing shell pipe line} |
| |
| \begin{verbatim} |
| output=`dmesg | grep hda` |
| ==> |
| p1 = Popen(["dmesg"], stdout=PIPE) |
| p2 = Popen(["grep", "hda"], stdin=p1.stdout) |
| output = p2.communicate()[0] |
| \end{verbatim} |
| |
| \subsubsection{Replacing os.system()} |
| |
| \begin{verbatim} |
| sts = os.system("mycmd" + " myarg") |
| ==> |
| p = Popen("mycmd" + " myarg", shell=True) |
| sts = os.waitpid(p.pid, 0) |
| \end{verbatim} |
| |
| Notes: |
| |
| \begin{itemize} |
| \item Calling the program through the shell is usually not required. |
| \item It's easier to look at the \member{returncode} attribute than |
| the exit status. |
| \end{itemize} |
| |
| A more realistic example would look like this: |
| |
| \begin{verbatim} |
| try: |
| retcode = call("mycmd" + " myarg", shell=True) |
| if retcode < 0: |
| print >>sys.stderr, "Child was terminated by signal", -retcode |
| else: |
| print >>sys.stderr, "Child returned", retcode |
| except OSError, e: |
| print >>sys.stderr, "Execution failed:", e |
| \end{verbatim} |
| |
| \subsubsection{Replacing os.spawn*} |
| |
| P_NOWAIT example: |
| |
| \begin{verbatim} |
| pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") |
| ==> |
| pid = Popen(["/bin/mycmd", "myarg"]).pid |
| \end{verbatim} |
| |
| P_WAIT example: |
| |
| \begin{verbatim} |
| retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") |
| ==> |
| retcode = call(["/bin/mycmd", "myarg"]) |
| \end{verbatim} |
| |
| Vector example: |
| |
| \begin{verbatim} |
| os.spawnvp(os.P_NOWAIT, path, args) |
| ==> |
| Popen([path] + args[1:]) |
| \end{verbatim} |
| |
| Environment example: |
| |
| \begin{verbatim} |
| os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) |
| ==> |
| Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) |
| \end{verbatim} |
| |
| \subsubsection{Replacing os.popen*} |
| |
| \begin{verbatim} |
| pipe = os.popen(cmd, mode='r', bufsize) |
| ==> |
| pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout |
| \end{verbatim} |
| |
| \begin{verbatim} |
| pipe = os.popen(cmd, mode='w', bufsize) |
| ==> |
| pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin |
| \end{verbatim} |
| |
| \begin{verbatim} |
| (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) |
| ==> |
| p = Popen(cmd, shell=True, bufsize=bufsize, |
| stdin=PIPE, stdout=PIPE, close_fds=True) |
| (child_stdin, child_stdout) = (p.stdin, p.stdout) |
| \end{verbatim} |
| |
| \begin{verbatim} |
| (child_stdin, |
| child_stdout, |
| child_stderr) = os.popen3(cmd, mode, bufsize) |
| ==> |
| p = Popen(cmd, shell=True, bufsize=bufsize, |
| stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) |
| (child_stdin, |
| child_stdout, |
| child_stderr) = (p.stdin, p.stdout, p.stderr) |
| \end{verbatim} |
| |
| \begin{verbatim} |
| (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) |
| ==> |
| p = Popen(cmd, shell=True, bufsize=bufsize, |
| stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) |
| (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) |
| \end{verbatim} |
| |
| \subsubsection{Replacing popen2.*} |
| |
| \note{If the cmd argument to popen2 functions is a string, the command |
| is executed through /bin/sh. If it is a list, the command is directly |
| executed.} |
| |
| \begin{verbatim} |
| (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) |
| ==> |
| p = Popen(["somestring"], shell=True, bufsize=bufsize |
| stdin=PIPE, stdout=PIPE, close_fds=True) |
| (child_stdout, child_stdin) = (p.stdout, p.stdin) |
| \end{verbatim} |
| |
| \begin{verbatim} |
| (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) |
| ==> |
| p = Popen(["mycmd", "myarg"], bufsize=bufsize, |
| stdin=PIPE, stdout=PIPE, close_fds=True) |
| (child_stdout, child_stdin) = (p.stdout, p.stdin) |
| \end{verbatim} |
| |
| The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen, |
| except that: |
| |
| \begin{itemize} |
| \item subprocess.Popen raises an exception if the execution fails |
| |
| \item the \var{capturestderr} argument is replaced with the \var{stderr} |
| argument. |
| |
| \item stdin=PIPE and stdout=PIPE must be specified. |
| |
| \item popen2 closes all file descriptors by default, but you have to |
| specify close_fds=True with subprocess.Popen. |
| \end{itemize} |