bpo-41586: Add pipesize parameter to subprocess & F_GETPIPE_SZ and F_SETPIPE_SZ to fcntl. (GH-21921)
* Add F_SETPIPE_SZ and F_GETPIPE_SZ to fcntl module
* Add pipesize parameter for subprocess.Popen class
This will allow the user to control the size of the pipes.
On linux the default is 64K. When a pipe is full it blocks for writing.
When a pipe is empty it blocks for reading. On processes that are
very fast this can lead to a lot of wasted CPU cycles. On a typical
Linux system the max pipe size is 1024K which is much better.
For high performance-oriented libraries such as xopen it is nice to
be able to set the pipe size.
The workaround without this feature is to use my_popen_process.stdout.fileno() in
conjuction with fcntl and 1031 (value of F_SETPIPE_SZ) to acquire this behavior.
diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
index e37cc98..7993b10 100644
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -341,7 +341,7 @@
startupinfo=None, creationflags=0, restore_signals=True, \
start_new_session=False, pass_fds=(), \*, group=None, \
extra_groups=None, user=None, umask=-1, \
- encoding=None, errors=None, text=None)
+ encoding=None, errors=None, text=None, pipesize=-1)
Execute a child program in a new process. On POSIX, the class uses
:meth:`os.execvp`-like behavior to execute the child program. On Windows,
@@ -625,6 +625,14 @@
* :data:`CREATE_DEFAULT_ERROR_MODE`
* :data:`CREATE_BREAKAWAY_FROM_JOB`
+ *pipesize* can be used to change the size of the pipe when
+ :data:`PIPE` is used for *stdin*, *stdout* or *stderr*. The size of the pipe
+ is only changed on platforms that support this (only Linux at this time of
+ writing). Other platforms will ignore this parameter.
+
+ .. versionadded:: 3.10
+ The ``pipesize`` parameter was added.
+
Popen objects are supported as context managers via the :keyword:`with` statement:
on exit, standard file descriptors are closed, and the process is waited for.
::