blob: 3cca9c84a27f52a8aa403408ba90941d4fc0e6b1 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`shutil` --- High-level file operations
2============================================
3
4.. module:: shutil
5 :synopsis: High-level file operations, including copying.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
Christian Heimes5b5e81c2007-12-31 16:14:33 +00008.. partly based on the docstrings
Georg Brandl116aa622007-08-15 14:28:22 +00009
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040010**Source code:** :source:`Lib/shutil.py`
11
Georg Brandl116aa622007-08-15 14:28:22 +000012.. index::
13 single: file; copying
14 single: copying files
15
Raymond Hettinger4f707fd2011-01-10 19:54:11 +000016--------------
17
Georg Brandl116aa622007-08-15 14:28:22 +000018The :mod:`shutil` module offers a number of high-level operations on files and
19collections of files. In particular, functions are provided which support file
Guido van Rossum2cc30da2007-11-02 23:46:40 +000020copying and removal. For operations on individual files, see also the
21:mod:`os` module.
Georg Brandl116aa622007-08-15 14:28:22 +000022
Guido van Rossumda27fd22007-08-17 00:24:54 +000023.. warning::
Christian Heimes7f044312008-01-06 17:05:40 +000024
Senthil Kumaran7f728c12012-02-13 23:30:47 +080025 Even the higher-level file copying functions (:func:`shutil.copy`,
26 :func:`shutil.copy2`) cannot copy all file metadata.
Georg Brandl48310cd2009-01-03 21:18:54 +000027
Christian Heimes7f044312008-01-06 17:05:40 +000028 On POSIX platforms, this means that file owner and group are lost as well
Georg Brandlc575c902008-09-13 17:46:05 +000029 as ACLs. On Mac OS, the resource fork and other metadata are not used.
Christian Heimes7f044312008-01-06 17:05:40 +000030 This means that resources will be lost and file type and creator codes will
31 not be correct. On Windows, file owners, ACLs and alternate data streams
32 are not copied.
Georg Brandl116aa622007-08-15 14:28:22 +000033
Éric Araujo6e6cb8e2010-11-16 19:13:50 +000034
Éric Araujof2fbb9c2012-01-16 16:55:55 +010035.. _file-operations:
36
Tarek Ziadé396fad72010-02-23 05:30:31 +000037Directory and files operations
38------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +000039
Georg Brandl116aa622007-08-15 14:28:22 +000040.. function:: copyfileobj(fsrc, fdst[, length])
41
42 Copy the contents of the file-like object *fsrc* to the file-like object *fdst*.
43 The integer *length*, if given, is the buffer size. In particular, a negative
44 *length* value means to copy the data without looping over the source data in
45 chunks; by default the data is read in chunks to avoid uncontrolled memory
46 consumption. Note that if the current file position of the *fsrc* object is not
47 0, only the contents from the current file position to the end of the file will
48 be copied.
49
50
Larry Hastingsb4038062012-07-15 10:57:38 -070051.. function:: copyfile(src, dst, *, follow_symlinks=True)
Christian Heimesa342c012008-04-20 21:01:16 +000052
Senthil Kumaran7f728c12012-02-13 23:30:47 +080053 Copy the contents (no metadata) of the file named *src* to a file named
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020054 *dst* and return *dst* in the most efficient way possible.
55 *src* and *dst* are path names given as strings.
56
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +020057 *dst* must be the complete target file name; look at :func:`~shutil.copy`
Larry Hastings60eba572012-09-21 10:12:14 -070058 for a copy that accepts a target directory path. If *src* and *dst*
Hynek Schlawack48653762012-10-07 12:49:58 +020059 specify the same file, :exc:`SameFileError` is raised.
Senthil Kumaran1fd64822012-02-13 23:35:44 +080060
Larry Hastings60eba572012-09-21 10:12:14 -070061 The destination location must be writable; otherwise, an :exc:`OSError`
62 exception will be raised. If *dst* already exists, it will be replaced.
63 Special files such as character or block devices and pipes cannot be
64 copied with this function.
Christian Heimesa342c012008-04-20 21:01:16 +000065
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070066 If *follow_symlinks* is false and *src* is a symbolic link,
67 a new symbolic link will be created instead of copying the
68 file *src* points to.
Antoine Pitrou78091e62011-12-29 18:54:15 +010069
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020070 .. versionchanged:: 3.3
71 :exc:`IOError` used to be raised instead of :exc:`OSError`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070072 Added *follow_symlinks* argument.
73 Now returns *dst*.
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020074
Hynek Schlawack48653762012-10-07 12:49:58 +020075 .. versionchanged:: 3.4
Hynek Schlawack27ddb572012-10-28 13:59:27 +010076 Raise :exc:`SameFileError` instead of :exc:`Error`. Since the former is
77 a subclass of the latter, this change is backward compatible.
Hynek Schlawack48653762012-10-07 12:49:58 +020078
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020079 .. versionchanged:: 3.8
80 Platform-specific fast-copy syscalls may be used internally in order to
81 copy the file more efficiently. See
82 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Hynek Schlawack48653762012-10-07 12:49:58 +020083
84.. exception:: SameFileError
85
86 This exception is raised if source and destination in :func:`copyfile`
87 are the same file.
88
89 .. versionadded:: 3.4
90
91
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070092.. function:: copymode(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +000093
94 Copy the permission bits from *src* to *dst*. The file contents, owner, and
Larry Hastings60eba572012-09-21 10:12:14 -070095 group are unaffected. *src* and *dst* are path names given as strings.
96 If *follow_symlinks* is false, and both *src* and *dst* are symbolic links,
97 :func:`copymode` will attempt to modify the mode of *dst* itself (rather
98 than the file it points to). This functionality is not available on every
99 platform; please see :func:`copystat` for more information. If
100 :func:`copymode` cannot modify symbolic links on the local platform, and it
101 is asked to do so, it will do nothing and return.
Georg Brandl116aa622007-08-15 14:28:22 +0000102
Antoine Pitrou78091e62011-12-29 18:54:15 +0100103 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700104 Added *follow_symlinks* argument.
Georg Brandl116aa622007-08-15 14:28:22 +0000105
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700106.. function:: copystat(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000107
Larry Hastings60eba572012-09-21 10:12:14 -0700108 Copy the permission bits, last access time, last modification time, and
109 flags from *src* to *dst*. On Linux, :func:`copystat` also copies the
110 "extended attributes" where possible. The file contents, owner, and
111 group are unaffected. *src* and *dst* are path names given as strings.
112
113 If *follow_symlinks* is false, and *src* and *dst* both
114 refer to symbolic links, :func:`copystat` will operate on
115 the symbolic links themselves rather than the files the
Martin Panter357ed2e2016-11-21 00:15:20 +0000116 symbolic links refer to—reading the information from the
Larry Hastings60eba572012-09-21 10:12:14 -0700117 *src* symbolic link, and writing the information to the
118 *dst* symbolic link.
119
120 .. note::
121
122 Not all platforms provide the ability to examine and
123 modify symbolic links. Python itself can tell you what
124 functionality is locally available.
125
126 * If ``os.chmod in os.supports_follow_symlinks`` is
127 ``True``, :func:`copystat` can modify the permission
128 bits of a symbolic link.
129
130 * If ``os.utime in os.supports_follow_symlinks`` is
131 ``True``, :func:`copystat` can modify the last access
132 and modification times of a symbolic link.
133
134 * If ``os.chflags in os.supports_follow_symlinks`` is
135 ``True``, :func:`copystat` can modify the flags of
136 a symbolic link. (``os.chflags`` is not available on
137 all platforms.)
138
139 On platforms where some or all of this functionality
140 is unavailable, when asked to modify a symbolic link,
141 :func:`copystat` will copy everything it can.
142 :func:`copystat` never returns failure.
143
144 Please see :data:`os.supports_follow_symlinks`
145 for more information.
Georg Brandl116aa622007-08-15 14:28:22 +0000146
Antoine Pitrou78091e62011-12-29 18:54:15 +0100147 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700148 Added *follow_symlinks* argument and support for Linux extended attributes.
Georg Brandl116aa622007-08-15 14:28:22 +0000149
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700150.. function:: copy(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000151
Larry Hastings60eba572012-09-21 10:12:14 -0700152 Copies the file *src* to the file or directory *dst*. *src* and *dst*
153 should be strings. If *dst* specifies a directory, the file will be
154 copied into *dst* using the base filename from *src*. Returns the
155 path to the newly created file.
156
157 If *follow_symlinks* is false, and *src* is a symbolic link,
158 *dst* will be created as a symbolic link. If *follow_symlinks*
159 is true and *src* is a symbolic link, *dst* will be a copy of
160 the file *src* refers to.
161
Mariatta70ee0cd2017-03-10 18:17:21 -0800162 :func:`~shutil.copy` copies the file data and the file's permission
Larry Hastings60eba572012-09-21 10:12:14 -0700163 mode (see :func:`os.chmod`). Other metadata, like the
164 file's creation and modification times, is not preserved.
165 To preserve all file metadata from the original, use
166 :func:`~shutil.copy2` instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000167
Antoine Pitrou78091e62011-12-29 18:54:15 +0100168 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700169 Added *follow_symlinks* argument.
Larry Hastings60eba572012-09-21 10:12:14 -0700170 Now returns path to the newly created file.
Georg Brandl116aa622007-08-15 14:28:22 +0000171
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200172 .. versionchanged:: 3.8
173 Platform-specific fast-copy syscalls may be used internally in order to
174 copy the file more efficiently. See
175 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
176
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700177.. function:: copy2(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000178
Larry Hastings60eba572012-09-21 10:12:14 -0700179 Identical to :func:`~shutil.copy` except that :func:`copy2`
Zsolt Cserna4f399be2018-10-23 12:09:50 +0200180 also attempts to preserve file metadata.
Larry Hastings60eba572012-09-21 10:12:14 -0700181
182 When *follow_symlinks* is false, and *src* is a symbolic
183 link, :func:`copy2` attempts to copy all metadata from the
184 *src* symbolic link to the newly-created *dst* symbolic link.
185 However, this functionality is not available on all platforms.
186 On platforms where some or all of this functionality is
187 unavailable, :func:`copy2` will preserve all the metadata
188 it can; :func:`copy2` never returns failure.
189
190 :func:`copy2` uses :func:`copystat` to copy the file metadata.
191 Please see :func:`copystat` for more information
192 about platform support for modifying symbolic link metadata.
Georg Brandl116aa622007-08-15 14:28:22 +0000193
Antoine Pitrou78091e62011-12-29 18:54:15 +0100194 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700195 Added *follow_symlinks* argument, try to copy extended
196 file system attributes too (currently Linux only).
Larry Hastings60eba572012-09-21 10:12:14 -0700197 Now returns path to the newly created file.
Brian Curtin066dacf2012-06-19 10:03:05 -0500198
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200199 .. versionchanged:: 3.8
200 Platform-specific fast-copy syscalls may be used internally in order to
201 copy the file more efficiently. See
202 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
203
Georg Brandl86b2fb92008-07-16 03:43:04 +0000204.. function:: ignore_patterns(\*patterns)
205
206 This factory function creates a function that can be used as a callable for
207 :func:`copytree`\'s *ignore* argument, ignoring files and directories that
208 match one of the glob-style *patterns* provided. See the example below.
209
210
R David Murray6ffface2014-06-11 14:40:13 -0400211.. function:: copytree(src, dst, symlinks=False, ignore=None, \
jab9e00d9e2018-12-28 13:03:40 -0500212 copy_function=copy2, ignore_dangling_symlinks=False, \
213 dirs_exist_ok=False)
Georg Brandl116aa622007-08-15 14:28:22 +0000214
jab9e00d9e2018-12-28 13:03:40 -0500215 Recursively copy an entire directory tree rooted at *src* to a directory
216 named *dst* and return the destination directory. *dirs_exist_ok* dictates
217 whether to raise an exception in case *dst* or any missing parent directory
218 already exists.
219
220 Permissions and times of directories are copied with :func:`copystat`,
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200221 individual files are copied using :func:`~shutil.copy2`.
Georg Brandl116aa622007-08-15 14:28:22 +0000222
Georg Brandl86b2fb92008-07-16 03:43:04 +0000223 If *symlinks* is true, symbolic links in the source tree are represented as
Antoine Pitrou78091e62011-12-29 18:54:15 +0100224 symbolic links in the new tree and the metadata of the original links will
225 be copied as far as the platform allows; if false or omitted, the contents
226 and metadata of the linked files are copied to the new tree.
Georg Brandl86b2fb92008-07-16 03:43:04 +0000227
Tarek Ziadéfb437512010-04-20 08:57:33 +0000228 When *symlinks* is false, if the file pointed by the symlink doesn't
Martin Panter7462b6492015-11-02 03:37:02 +0000229 exist, an exception will be added in the list of errors raised in
230 an :exc:`Error` exception at the end of the copy process.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000231 You can set the optional *ignore_dangling_symlinks* flag to true if you
Tarek Ziadé8c26c7d2010-04-23 13:03:50 +0000232 want to silence this exception. Notice that this option has no effect
233 on platforms that don't support :func:`os.symlink`.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000234
Georg Brandl86b2fb92008-07-16 03:43:04 +0000235 If *ignore* is given, it must be a callable that will receive as its
236 arguments the directory being visited by :func:`copytree`, and a list of its
237 contents, as returned by :func:`os.listdir`. Since :func:`copytree` is
238 called recursively, the *ignore* callable will be called once for each
239 directory that is copied. The callable must return a sequence of directory
240 and file names relative to the current directory (i.e. a subset of the items
241 in its second argument); these names will then be ignored in the copy
242 process. :func:`ignore_patterns` can be used to create such a callable that
243 ignores names based on glob-style patterns.
244
245 If exception(s) occur, an :exc:`Error` is raised with a list of reasons.
246
Senthil Kumaran7f728c12012-02-13 23:30:47 +0800247 If *copy_function* is given, it must be a callable that will be used to copy
248 each file. It will be called with the source path and the destination path
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200249   as arguments. By default, :func:`~shutil.copy2` is used, but any function
250   that supports the same signature (like :func:`~shutil.copy`) can be used.
Georg Brandl116aa622007-08-15 14:28:22 +0000251
Steve Dower60419a72019-06-24 08:42:54 -0700252 .. audit-event:: shutil.copytree "src dst"
253
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700254 .. versionchanged:: 3.3
255 Copy metadata when *symlinks* is false.
256 Now returns *dst*.
257
Tarek Ziadé5340db32010-04-19 22:30:51 +0000258 .. versionchanged:: 3.2
259 Added the *copy_function* argument to be able to provide a custom copy
260 function.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000261 Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
262 errors when *symlinks* is false.
263
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200264 .. versionchanged:: 3.8
265 Platform-specific fast-copy syscalls may be used internally in order to
266 copy the file more efficiently. See
267 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Georg Brandl96acb732012-06-24 17:39:05 +0200268
jab9e00d9e2018-12-28 13:03:40 -0500269 .. versionadded:: 3.8
270 The *dirs_exist_ok* parameter.
271
Georg Brandl18244152009-09-02 20:34:52 +0000272.. function:: rmtree(path, ignore_errors=False, onerror=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000273
274 .. index:: single: directory; deleting
275
Christian Heimes9bd667a2008-01-20 15:14:11 +0000276 Delete an entire directory tree; *path* must point to a directory (but not a
277 symbolic link to a directory). If *ignore_errors* is true, errors resulting
278 from failed removals will be ignored; if false or omitted, such errors are
279 handled by calling a handler specified by *onerror* or, if that is omitted,
280 they raise an exception.
Georg Brandl116aa622007-08-15 14:28:22 +0000281
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000282 .. note::
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200283
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000284 On platforms that support the necessary fd-based functions a symlink
Georg Brandl96acb732012-06-24 17:39:05 +0200285 attack resistant version of :func:`rmtree` is used by default. On other
286 platforms, the :func:`rmtree` implementation is susceptible to a symlink
287 attack: given proper timing and circumstances, attackers can manipulate
288 symlinks on the filesystem to delete files they wouldn't be able to access
289 otherwise. Applications can use the :data:`rmtree.avoids_symlink_attacks`
290 function attribute to determine which case applies.
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200291
Christian Heimes9bd667a2008-01-20 15:14:11 +0000292 If *onerror* is provided, it must be a callable that accepts three
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200293 parameters: *function*, *path*, and *excinfo*.
294
295 The first parameter, *function*, is the function which raised the exception;
296 it depends on the platform and implementation. The second parameter,
297 *path*, will be the path name passed to *function*. The third parameter,
298 *excinfo*, will be the exception information returned by
299 :func:`sys.exc_info`. Exceptions raised by *onerror* will not be caught.
300
Steve Dower60419a72019-06-24 08:42:54 -0700301 .. audit-event:: shutil.rmtree path
302
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200303 .. versionchanged:: 3.3
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000304 Added a symlink attack resistant version that is used automatically
305 if platform supports fd-based functions.
Christian Heimes9bd667a2008-01-20 15:14:11 +0000306
Éric Araujo544e13d2012-06-24 13:53:48 -0400307 .. attribute:: rmtree.avoids_symlink_attacks
Hynek Schlawack2100b422012-06-23 20:28:32 +0200308
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000309 Indicates whether the current platform and implementation provides a
Georg Brandl96acb732012-06-24 17:39:05 +0200310 symlink attack resistant version of :func:`rmtree`. Currently this is
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000311 only true for platforms supporting fd-based directory access functions.
Hynek Schlawack2100b422012-06-23 20:28:32 +0200312
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000313 .. versionadded:: 3.3
Georg Brandl116aa622007-08-15 14:28:22 +0000314
Georg Brandl96acb732012-06-24 17:39:05 +0200315
R David Murray6ffface2014-06-11 14:40:13 -0400316.. function:: move(src, dst, copy_function=copy2)
Georg Brandl116aa622007-08-15 14:28:22 +0000317
Brian Curtin0d0a1de2012-06-18 18:41:07 -0500318 Recursively move a file or directory (*src*) to another location (*dst*)
319 and return the destination.
Georg Brandl116aa622007-08-15 14:28:22 +0000320
Benjamin Peterson218144a2015-03-22 10:11:54 -0400321 If the destination is an existing directory, then *src* is moved inside that
322 directory. If the destination already exists but is not a directory, it may
323 be overwritten depending on :func:`os.rename` semantics.
Éric Araujo14382dc2011-07-28 22:49:11 +0200324
325 If the destination is on the current filesystem, then :func:`os.rename` is
R David Murray6ffface2014-06-11 14:40:13 -0400326 used. Otherwise, *src* is copied to *dst* using *copy_function* and then
327 removed. In case of symlinks, a new symlink pointing to the target of *src*
328 will be created in or as *dst* and *src* will be removed.
329
330 If *copy_function* is given, it must be a callable that takes two arguments
331 *src* and *dst*, and will be used to copy *src* to *dest* if
332 :func:`os.rename` cannot be used. If the source is a directory,
333 :func:`copytree` is called, passing it the :func:`copy_function`. The
Mariatta70ee0cd2017-03-10 18:17:21 -0800334 default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the
R David Murray6ffface2014-06-11 14:40:13 -0400335 *copy_function* allows the move to succeed when it is not possible to also
336 copy the metadata, at the expense of not copying any of the metadata.
Antoine Pitrou0a08d7a2012-01-06 20:16:19 +0100337
338 .. versionchanged:: 3.3
339 Added explicit symlink handling for foreign filesystems, thus adapting
340 it to the behavior of GNU's :program:`mv`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700341 Now returns *dst*.
Brian Curtin066dacf2012-06-19 10:03:05 -0500342
R David Murray6ffface2014-06-11 14:40:13 -0400343 .. versionchanged:: 3.5
344 Added the *copy_function* keyword argument.
345
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200346 .. versionchanged:: 3.8
347 Platform-specific fast-copy syscalls may be used internally in order to
348 copy the file more efficiently. See
349 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
350
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200351.. function:: disk_usage(path)
352
Éric Araujoe4d5b8e2011-08-08 16:51:11 +0200353 Return disk usage statistics about the given path as a :term:`named tuple`
354 with the attributes *total*, *used* and *free*, which are the amount of
Joe Pamerc8c02492018-09-25 10:57:36 -0400355 total, used and free space, in bytes. *path* may be a file or a
356 directory.
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200357
358 .. versionadded:: 3.3
359
Joe Pamerc8c02492018-09-25 10:57:36 -0400360 .. versionchanged:: 3.8
361 On Windows, *path* can now be a file or directory.
362
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400363 .. availability:: Unix, Windows.
Georg Brandl116aa622007-08-15 14:28:22 +0000364
Sandro Tosid902a142011-08-22 23:28:27 +0200365.. function:: chown(path, user=None, group=None)
366
367 Change owner *user* and/or *group* of the given *path*.
368
369 *user* can be a system user name or a uid; the same applies to *group*. At
370 least one argument is required.
371
372 See also :func:`os.chown`, the underlying function.
373
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400374 .. availability:: Unix.
Sandro Tosid902a142011-08-22 23:28:27 +0200375
376 .. versionadded:: 3.3
377
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200378
Brian Curtinc57a3452012-06-22 16:00:30 -0500379.. function:: which(cmd, mode=os.F_OK | os.X_OK, path=None)
380
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200381 Return the path to an executable which would be run if the given *cmd* was
382 called. If no *cmd* would be called, return ``None``.
Brian Curtinc57a3452012-06-22 16:00:30 -0500383
Serhiy Storchaka6a7b3a72016-04-17 08:32:47 +0300384 *mode* is a permission mask passed to :func:`os.access`, by default
Brian Curtinc57a3452012-06-22 16:00:30 -0500385 determining if the file exists and executable.
386
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200387 When no *path* is specified, the results of :func:`os.environ` are used,
388 returning either the "PATH" value or a fallback of :attr:`os.defpath`.
Brian Curtinc57a3452012-06-22 16:00:30 -0500389
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200390 On Windows, the current directory is always prepended to the *path* whether
391 or not you use the default or provide your own, which is the behavior the
Donald Stufft8b852f12014-05-20 12:58:38 -0400392 command shell uses when finding executables. Additionally, when finding the
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200393 *cmd* in the *path*, the ``PATHEXT`` environment variable is checked. For
394 example, if you call ``shutil.which("python")``, :func:`which` will search
395 ``PATHEXT`` to know that it should look for ``python.exe`` within the *path*
396 directories. For example, on Windows::
Brian Curtinc57a3452012-06-22 16:00:30 -0500397
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200398 >>> shutil.which("python")
Serhiy Storchaka80c88f42013-01-22 10:31:36 +0200399 'C:\\Python33\\python.EXE'
Brian Curtinc57a3452012-06-22 16:00:30 -0500400
401 .. versionadded:: 3.3
Sandro Tosid902a142011-08-22 23:28:27 +0200402
Cheryl Sabella5680f652019-02-13 06:25:10 -0500403 .. versionchanged:: 3.8
404 The :class:`bytes` type is now accepted. If *cmd* type is
405 :class:`bytes`, the result type is also :class:`bytes`.
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200406
Georg Brandl116aa622007-08-15 14:28:22 +0000407.. exception:: Error
408
Éric Araujo14382dc2011-07-28 22:49:11 +0200409 This exception collects exceptions that are raised during a multi-file
410 operation. For :func:`copytree`, the exception argument is a list of 3-tuples
411 (*srcname*, *dstname*, *exception*).
Georg Brandl116aa622007-08-15 14:28:22 +0000412
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200413.. _shutil-platform-dependent-efficient-copy-operations:
414
415Platform-dependent efficient copy operations
416~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
417
418Starting from Python 3.8 all functions involving a file copy (:func:`copyfile`,
419:func:`copy`, :func:`copy2`, :func:`copytree`, and :func:`move`) may use
420platform-specific "fast-copy" syscalls in order to copy the file more
421efficiently (see :issue:`33671`).
422"fast-copy" means that the copying operation occurs within the kernel, avoiding
423the use of userspace buffers in Python as in "``outfd.write(infd.read())``".
424
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700425On macOS `fcopyfile`_ is used to copy the file content (not metadata).
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200426
Giampaolo Rodola413d9552019-05-30 14:05:41 +0800427On Linux :func:`os.sendfile` is used.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200428
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700429On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB
Inada Naoki4f190302019-03-02 13:31:01 +0900430instead of 64 KiB) and a :func:`memoryview`-based variant of
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700431:func:`shutil.copyfileobj` is used.
432
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200433If the fast-copy operation fails and no data was written in the destination
434file then shutil will silently fallback on using less efficient
435:func:`copyfileobj` function internally.
436
437.. versionchanged:: 3.8
Georg Brandl116aa622007-08-15 14:28:22 +0000438
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100439.. _shutil-copytree-example:
Georg Brandl116aa622007-08-15 14:28:22 +0000440
Tarek Ziadé396fad72010-02-23 05:30:31 +0000441copytree example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200442~~~~~~~~~~~~~~~~
Georg Brandl116aa622007-08-15 14:28:22 +0000443
444This example is the implementation of the :func:`copytree` function, described
445above, with the docstring omitted. It demonstrates many of the other functions
446provided by this module. ::
447
448 def copytree(src, dst, symlinks=False):
449 names = os.listdir(src)
450 os.makedirs(dst)
451 errors = []
452 for name in names:
453 srcname = os.path.join(src, name)
454 dstname = os.path.join(dst, name)
455 try:
456 if symlinks and os.path.islink(srcname):
457 linkto = os.readlink(srcname)
458 os.symlink(linkto, dstname)
459 elif os.path.isdir(srcname):
460 copytree(srcname, dstname, symlinks)
461 else:
462 copy2(srcname, dstname)
463 # XXX What about devices, sockets etc.?
Andrew Svetlov618c2e12012-12-15 22:59:24 +0200464 except OSError as why:
Georg Brandl116aa622007-08-15 14:28:22 +0000465 errors.append((srcname, dstname, str(why)))
466 # catch the Error from the recursive copytree so that we can
467 # continue with other files
468 except Error as err:
469 errors.extend(err.args[0])
470 try:
471 copystat(src, dst)
Georg Brandl116aa622007-08-15 14:28:22 +0000472 except OSError as why:
Andrew Svetlov2606a6f2012-12-19 14:33:35 +0200473 # can't copy file access times on Windows
474 if why.winerror is None:
475 errors.extend((src, dst, str(why)))
Georg Brandl116aa622007-08-15 14:28:22 +0000476 if errors:
Collin Winterc79461b2007-09-01 23:34:30 +0000477 raise Error(errors)
Georg Brandl116aa622007-08-15 14:28:22 +0000478
Tarek Ziadé396fad72010-02-23 05:30:31 +0000479Another example that uses the :func:`ignore_patterns` helper::
480
481 from shutil import copytree, ignore_patterns
482
483 copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))
484
485This will copy everything except ``.pyc`` files and files or directories whose
486name starts with ``tmp``.
487
488Another example that uses the *ignore* argument to add a logging call::
489
490 from shutil import copytree
491 import logging
492
493 def _logpath(path, names):
Vinay Sajipdd917f82016-08-31 08:22:29 +0100494 logging.info('Working in %s', path)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000495 return [] # nothing will be ignored
496
497 copytree(source, destination, ignore=_logpath)
498
499
Tim Golden78337792014-05-07 18:05:45 +0100500.. _shutil-rmtree-example:
501
502rmtree example
503~~~~~~~~~~~~~~
504
505This example shows how to remove a directory tree on Windows where some
506of the files have their read-only bit set. It uses the onerror callback
507to clear the readonly bit and reattempt the remove. Any subsequent failure
508will propagate. ::
509
510 import os, stat
511 import shutil
Tim Goldenba748852014-05-07 18:08:08 +0100512
Tim Golden78337792014-05-07 18:05:45 +0100513 def remove_readonly(func, path, _):
514 "Clear the readonly bit and reattempt the removal"
515 os.chmod(path, stat.S_IWRITE)
Tim Goldenba748852014-05-07 18:08:08 +0100516 func(path)
517
Tim Golden78337792014-05-07 18:05:45 +0100518 shutil.rmtree(directory, onerror=remove_readonly)
519
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000520.. _archiving-operations:
521
522Archiving operations
523--------------------
Tarek Ziadé396fad72010-02-23 05:30:31 +0000524
Georg Brandl03b9ad02012-06-24 18:09:40 +0200525.. versionadded:: 3.2
526
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200527.. versionchanged:: 3.5
528 Added support for the *xztar* format.
529
530
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100531High-level utilities to create and read compressed and archived files are also
532provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
533
Tarek Ziadé396fad72010-02-23 05:30:31 +0000534.. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]])
535
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000536 Create an archive file (such as zip or tar) and return its name.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000537
538 *base_name* is the name of the file to create, including the path, minus
539 any format-specific extension. *format* is the archive format: one of
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200540 "zip" (if the :mod:`zlib` module is available), "tar", "gztar" (if the
541 :mod:`zlib` module is available), "bztar" (if the :mod:`bz2` module is
542 available), or "xztar" (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000543
544 *root_dir* is a directory that will be the root directory of the
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000545 archive; for example, we typically chdir into *root_dir* before creating the
Tarek Ziadé396fad72010-02-23 05:30:31 +0000546 archive.
547
548 *base_dir* is the directory where we start archiving from;
Ezio Melotticb999a32010-04-20 11:26:51 +0000549 i.e. *base_dir* will be the common prefix of all files and
Tarek Ziadé396fad72010-02-23 05:30:31 +0000550 directories in the archive.
551
552 *root_dir* and *base_dir* both default to the current directory.
553
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100554 If *dry_run* is true, no archive is created, but the operations that would be
555 executed are logged to *logger*.
556
Tarek Ziadé396fad72010-02-23 05:30:31 +0000557 *owner* and *group* are used when creating a tar archive. By default,
558 uses the current owner and group.
559
Éric Araujo06c42a32011-11-07 17:31:07 +0100560 *logger* must be an object compatible with :pep:`282`, usually an instance of
561 :class:`logging.Logger`.
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000562
Georg Brandl36ac5102014-10-31 10:54:06 +0100563 The *verbose* argument is unused and deprecated.
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100564
Steve Dower60419a72019-06-24 08:42:54 -0700565 .. audit-event:: shutil.make_archive "base_name format root_dir base_dir"
566
CAM Gerlach89a89442019-04-06 23:47:49 -0500567 .. versionchanged:: 3.8
568 The modern pax (POSIX.1-2001) format is now used instead of
569 the legacy GNU format for archives created with ``format="tar"``.
570
Tarek Ziadé396fad72010-02-23 05:30:31 +0000571
572.. function:: get_archive_formats()
573
Éric Araujo14382dc2011-07-28 22:49:11 +0200574 Return a list of supported formats for archiving.
Martin Panterd21e0b52015-10-10 10:36:22 +0000575 Each element of the returned sequence is a tuple ``(name, description)``.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000576
577 By default :mod:`shutil` provides these formats:
578
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200579 - *zip*: ZIP file (if the :mod:`zlib` module is available).
CAM Gerlach89a89442019-04-06 23:47:49 -0500580 - *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200581 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
582 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
583 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000584
585 You can register new formats or provide your own archiver for any existing
586 formats, by using :func:`register_archive_format`.
587
Tarek Ziadé396fad72010-02-23 05:30:31 +0000588
589.. function:: register_archive_format(name, function, [extra_args, [description]])
590
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100591 Register an archiver for the format *name*.
592
593 *function* is the callable that will be used to unpack archives. The callable
594 will receive the *base_name* of the file to create, followed by the
595 *base_dir* (which defaults to :data:`os.curdir`) to start archiving from.
596 Further arguments are passed as keyword arguments: *owner*, *group*,
597 *dry_run* and *logger* (as passed in :func:`make_archive`).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000598
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000599 If given, *extra_args* is a sequence of ``(name, value)`` pairs that will be
Tarek Ziadé396fad72010-02-23 05:30:31 +0000600 used as extra keywords arguments when the archiver callable is used.
601
602 *description* is used by :func:`get_archive_formats` which returns the
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100603 list of archivers. Defaults to an empty string.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000604
Tarek Ziadé396fad72010-02-23 05:30:31 +0000605
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000606.. function:: unregister_archive_format(name)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000607
608 Remove the archive format *name* from the list of supported formats.
609
Tarek Ziadé396fad72010-02-23 05:30:31 +0000610
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000611.. function:: unpack_archive(filename[, extract_dir[, format]])
612
613 Unpack an archive. *filename* is the full path of the archive.
614
615 *extract_dir* is the name of the target directory where the archive is
616 unpacked. If not provided, the current working directory is used.
617
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200618 *format* is the archive format: one of "zip", "tar", "gztar", "bztar", or
619 "xztar". Or any other format registered with
620 :func:`register_unpack_format`. If not provided, :func:`unpack_archive`
621 will use the archive file name extension and see if an unpacker was
622 registered for that extension. In case none is found,
623 a :exc:`ValueError` is raised.
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000624
Jelle Zijlstraa12df7b2017-05-05 14:27:12 -0700625 .. versionchanged:: 3.7
626 Accepts a :term:`path-like object` for *filename* and *extract_dir*.
627
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000628
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000629.. function:: register_unpack_format(name, extensions, function[, extra_args[, description]])
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000630
631 Registers an unpack format. *name* is the name of the format and
632 *extensions* is a list of extensions corresponding to the format, like
633 ``.zip`` for Zip files.
634
635 *function* is the callable that will be used to unpack archives. The
636 callable will receive the path of the archive, followed by the directory
637 the archive must be extracted to.
638
639 When provided, *extra_args* is a sequence of ``(name, value)`` tuples that
640 will be passed as keywords arguments to the callable.
641
642 *description* can be provided to describe the format, and will be returned
643 by the :func:`get_unpack_formats` function.
644
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000645
646.. function:: unregister_unpack_format(name)
647
648 Unregister an unpack format. *name* is the name of the format.
649
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000650
651.. function:: get_unpack_formats()
652
653 Return a list of all registered formats for unpacking.
654 Each element of the returned sequence is a tuple
655 ``(name, extensions, description)``.
656
657 By default :mod:`shutil` provides these formats:
658
Martin Panter2f9171d2016-12-18 01:23:09 +0000659 - *zip*: ZIP file (unpacking compressed files works only if the corresponding
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200660 module is available).
661 - *tar*: uncompressed tar file.
662 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
663 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
664 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000665
666 You can register new formats or provide your own unpacker for any existing
667 formats, by using :func:`register_unpack_format`.
668
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000669
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100670.. _shutil-archiving-example:
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000671
Tarek Ziadé396fad72010-02-23 05:30:31 +0000672Archiving example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200673~~~~~~~~~~~~~~~~~
Tarek Ziadé396fad72010-02-23 05:30:31 +0000674
675In this example, we create a gzip'ed tar-file archive containing all files
676found in the :file:`.ssh` directory of the user::
677
678 >>> from shutil import make_archive
679 >>> import os
680 >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))
681 >>> root_dir = os.path.expanduser(os.path.join('~', '.ssh'))
682 >>> make_archive(archive_name, 'gztar', root_dir)
683 '/Users/tarek/myarchive.tar.gz'
684
Martin Panter1050d2d2016-07-26 11:18:21 +0200685The resulting archive contains:
686
687.. code-block:: shell-session
Tarek Ziadé396fad72010-02-23 05:30:31 +0000688
689 $ tar -tzvf /Users/tarek/myarchive.tar.gz
690 drwx------ tarek/staff 0 2010-02-01 16:23:40 ./
691 -rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys
692 -rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config
693 -rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa
694 -rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub
695 -rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa
696 -rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub
697 -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100698
699
700Querying the size of the output terminal
701----------------------------------------
702
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100703.. function:: get_terminal_size(fallback=(columns, lines))
704
705 Get the size of the terminal window.
706
707 For each of the two dimensions, the environment variable, ``COLUMNS``
708 and ``LINES`` respectively, is checked. If the variable is defined and
709 the value is a positive integer, it is used.
710
711 When ``COLUMNS`` or ``LINES`` is not defined, which is the common case,
712 the terminal connected to :data:`sys.__stdout__` is queried
713 by invoking :func:`os.get_terminal_size`.
714
715 If the terminal size cannot be successfully queried, either because
716 the system doesn't support querying, or because we are not
717 connected to a terminal, the value given in ``fallback`` parameter
718 is used. ``fallback`` defaults to ``(80, 24)`` which is the default
719 size used by many terminal emulators.
720
721 The value returned is a named tuple of type :class:`os.terminal_size`.
722
723 See also: The Single UNIX Specification, Version 2,
724 `Other Environment Variables`_.
725
Berker Peksag8e2bdc82016-12-27 15:09:11 +0300726 .. versionadded:: 3.3
727
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200728.. _`fcopyfile`:
729 http://www.manpagez.com/man/3/copyfile/
730
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100731.. _`Other Environment Variables`:
732 http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003