blob: bd24de720232140bf1a94b82b64f7db43e8f7337 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`shutil` --- High-level file operations
2============================================
3
4.. module:: shutil
5 :synopsis: High-level file operations, including copying.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
Christian Heimes5b5e81c2007-12-31 16:14:33 +00008.. partly based on the docstrings
Georg Brandl116aa622007-08-15 14:28:22 +00009
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040010**Source code:** :source:`Lib/shutil.py`
11
Georg Brandl116aa622007-08-15 14:28:22 +000012.. index::
13 single: file; copying
14 single: copying files
15
Raymond Hettinger4f707fd2011-01-10 19:54:11 +000016--------------
17
Georg Brandl116aa622007-08-15 14:28:22 +000018The :mod:`shutil` module offers a number of high-level operations on files and
19collections of files. In particular, functions are provided which support file
Guido van Rossum2cc30da2007-11-02 23:46:40 +000020copying and removal. For operations on individual files, see also the
21:mod:`os` module.
Georg Brandl116aa622007-08-15 14:28:22 +000022
Guido van Rossumda27fd22007-08-17 00:24:54 +000023.. warning::
Christian Heimes7f044312008-01-06 17:05:40 +000024
Senthil Kumaran7f728c12012-02-13 23:30:47 +080025 Even the higher-level file copying functions (:func:`shutil.copy`,
26 :func:`shutil.copy2`) cannot copy all file metadata.
Georg Brandl48310cd2009-01-03 21:18:54 +000027
Christian Heimes7f044312008-01-06 17:05:40 +000028 On POSIX platforms, this means that file owner and group are lost as well
Georg Brandlc575c902008-09-13 17:46:05 +000029 as ACLs. On Mac OS, the resource fork and other metadata are not used.
Christian Heimes7f044312008-01-06 17:05:40 +000030 This means that resources will be lost and file type and creator codes will
31 not be correct. On Windows, file owners, ACLs and alternate data streams
32 are not copied.
Georg Brandl116aa622007-08-15 14:28:22 +000033
Éric Araujo6e6cb8e2010-11-16 19:13:50 +000034
Éric Araujof2fbb9c2012-01-16 16:55:55 +010035.. _file-operations:
36
Tarek Ziadé396fad72010-02-23 05:30:31 +000037Directory and files operations
38------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +000039
Georg Brandl116aa622007-08-15 14:28:22 +000040.. function:: copyfileobj(fsrc, fdst[, length])
41
42 Copy the contents of the file-like object *fsrc* to the file-like object *fdst*.
43 The integer *length*, if given, is the buffer size. In particular, a negative
44 *length* value means to copy the data without looping over the source data in
45 chunks; by default the data is read in chunks to avoid uncontrolled memory
46 consumption. Note that if the current file position of the *fsrc* object is not
47 0, only the contents from the current file position to the end of the file will
48 be copied.
49
50
Larry Hastingsb4038062012-07-15 10:57:38 -070051.. function:: copyfile(src, dst, *, follow_symlinks=True)
Christian Heimesa342c012008-04-20 21:01:16 +000052
Senthil Kumaran7f728c12012-02-13 23:30:47 +080053 Copy the contents (no metadata) of the file named *src* to a file named
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020054 *dst* and return *dst* in the most efficient way possible.
Miss Islington (bot)0468a852019-09-09 10:18:51 -070055 *src* and *dst* are path-like objects or path names given as strings.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020056
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +020057 *dst* must be the complete target file name; look at :func:`~shutil.copy`
Larry Hastings60eba572012-09-21 10:12:14 -070058 for a copy that accepts a target directory path. If *src* and *dst*
Hynek Schlawack48653762012-10-07 12:49:58 +020059 specify the same file, :exc:`SameFileError` is raised.
Senthil Kumaran1fd64822012-02-13 23:35:44 +080060
Larry Hastings60eba572012-09-21 10:12:14 -070061 The destination location must be writable; otherwise, an :exc:`OSError`
62 exception will be raised. If *dst* already exists, it will be replaced.
63 Special files such as character or block devices and pipes cannot be
64 copied with this function.
Christian Heimesa342c012008-04-20 21:01:16 +000065
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070066 If *follow_symlinks* is false and *src* is a symbolic link,
67 a new symbolic link will be created instead of copying the
68 file *src* points to.
Antoine Pitrou78091e62011-12-29 18:54:15 +010069
Steve Dowera00b5be2020-02-13 08:30:27 +000070 .. audit-event:: shutil.copyfile src,dst shutil.copyfile
71
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020072 .. versionchanged:: 3.3
73 :exc:`IOError` used to be raised instead of :exc:`OSError`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070074 Added *follow_symlinks* argument.
75 Now returns *dst*.
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020076
Hynek Schlawack48653762012-10-07 12:49:58 +020077 .. versionchanged:: 3.4
Hynek Schlawack27ddb572012-10-28 13:59:27 +010078 Raise :exc:`SameFileError` instead of :exc:`Error`. Since the former is
79 a subclass of the latter, this change is backward compatible.
Hynek Schlawack48653762012-10-07 12:49:58 +020080
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020081 .. versionchanged:: 3.8
82 Platform-specific fast-copy syscalls may be used internally in order to
83 copy the file more efficiently. See
84 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Hynek Schlawack48653762012-10-07 12:49:58 +020085
86.. exception:: SameFileError
87
88 This exception is raised if source and destination in :func:`copyfile`
89 are the same file.
90
91 .. versionadded:: 3.4
92
93
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070094.. function:: copymode(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +000095
96 Copy the permission bits from *src* to *dst*. The file contents, owner, and
Miss Islington (bot)0468a852019-09-09 10:18:51 -070097 group are unaffected. *src* and *dst* are path-like objects or path names
98 given as strings.
Larry Hastings60eba572012-09-21 10:12:14 -070099 If *follow_symlinks* is false, and both *src* and *dst* are symbolic links,
100 :func:`copymode` will attempt to modify the mode of *dst* itself (rather
101 than the file it points to). This functionality is not available on every
102 platform; please see :func:`copystat` for more information. If
103 :func:`copymode` cannot modify symbolic links on the local platform, and it
104 is asked to do so, it will do nothing and return.
Georg Brandl116aa622007-08-15 14:28:22 +0000105
Steve Dowera00b5be2020-02-13 08:30:27 +0000106 .. audit-event:: shutil.copymode src,dst shutil.copymode
107
Antoine Pitrou78091e62011-12-29 18:54:15 +0100108 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700109 Added *follow_symlinks* argument.
Georg Brandl116aa622007-08-15 14:28:22 +0000110
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700111.. function:: copystat(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000112
Larry Hastings60eba572012-09-21 10:12:14 -0700113 Copy the permission bits, last access time, last modification time, and
114 flags from *src* to *dst*. On Linux, :func:`copystat` also copies the
115 "extended attributes" where possible. The file contents, owner, and
Miss Islington (bot)0468a852019-09-09 10:18:51 -0700116 group are unaffected. *src* and *dst* are path-like objects or path
117 names given as strings.
Larry Hastings60eba572012-09-21 10:12:14 -0700118
119 If *follow_symlinks* is false, and *src* and *dst* both
120 refer to symbolic links, :func:`copystat` will operate on
121 the symbolic links themselves rather than the files the
Martin Panter357ed2e2016-11-21 00:15:20 +0000122 symbolic links refer to—reading the information from the
Larry Hastings60eba572012-09-21 10:12:14 -0700123 *src* symbolic link, and writing the information to the
124 *dst* symbolic link.
125
126 .. note::
127
128 Not all platforms provide the ability to examine and
129 modify symbolic links. Python itself can tell you what
130 functionality is locally available.
131
132 * If ``os.chmod in os.supports_follow_symlinks`` is
133 ``True``, :func:`copystat` can modify the permission
134 bits of a symbolic link.
135
136 * If ``os.utime in os.supports_follow_symlinks`` is
137 ``True``, :func:`copystat` can modify the last access
138 and modification times of a symbolic link.
139
140 * If ``os.chflags in os.supports_follow_symlinks`` is
141 ``True``, :func:`copystat` can modify the flags of
142 a symbolic link. (``os.chflags`` is not available on
143 all platforms.)
144
145 On platforms where some or all of this functionality
146 is unavailable, when asked to modify a symbolic link,
147 :func:`copystat` will copy everything it can.
148 :func:`copystat` never returns failure.
149
150 Please see :data:`os.supports_follow_symlinks`
151 for more information.
Georg Brandl116aa622007-08-15 14:28:22 +0000152
Steve Dowera00b5be2020-02-13 08:30:27 +0000153 .. audit-event:: shutil.copystat src,dst shutil.copystat
154
Antoine Pitrou78091e62011-12-29 18:54:15 +0100155 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700156 Added *follow_symlinks* argument and support for Linux extended attributes.
Georg Brandl116aa622007-08-15 14:28:22 +0000157
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700158.. function:: copy(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000159
Larry Hastings60eba572012-09-21 10:12:14 -0700160 Copies the file *src* to the file or directory *dst*. *src* and *dst*
161 should be strings. If *dst* specifies a directory, the file will be
162 copied into *dst* using the base filename from *src*. Returns the
163 path to the newly created file.
164
165 If *follow_symlinks* is false, and *src* is a symbolic link,
166 *dst* will be created as a symbolic link. If *follow_symlinks*
167 is true and *src* is a symbolic link, *dst* will be a copy of
168 the file *src* refers to.
169
Mariatta70ee0cd2017-03-10 18:17:21 -0800170 :func:`~shutil.copy` copies the file data and the file's permission
Larry Hastings60eba572012-09-21 10:12:14 -0700171 mode (see :func:`os.chmod`). Other metadata, like the
172 file's creation and modification times, is not preserved.
173 To preserve all file metadata from the original, use
174 :func:`~shutil.copy2` instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000175
Steve Dowera00b5be2020-02-13 08:30:27 +0000176 .. audit-event:: shutil.copyfile src,dst shutil.copy
177
178 .. audit-event:: shutil.copymode src,dst shutil.copy
179
Antoine Pitrou78091e62011-12-29 18:54:15 +0100180 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700181 Added *follow_symlinks* argument.
Larry Hastings60eba572012-09-21 10:12:14 -0700182 Now returns path to the newly created file.
Georg Brandl116aa622007-08-15 14:28:22 +0000183
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200184 .. versionchanged:: 3.8
185 Platform-specific fast-copy syscalls may be used internally in order to
186 copy the file more efficiently. See
187 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
188
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700189.. function:: copy2(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000190
Larry Hastings60eba572012-09-21 10:12:14 -0700191 Identical to :func:`~shutil.copy` except that :func:`copy2`
Zsolt Cserna4f399be2018-10-23 12:09:50 +0200192 also attempts to preserve file metadata.
Larry Hastings60eba572012-09-21 10:12:14 -0700193
194 When *follow_symlinks* is false, and *src* is a symbolic
195 link, :func:`copy2` attempts to copy all metadata from the
196 *src* symbolic link to the newly-created *dst* symbolic link.
197 However, this functionality is not available on all platforms.
198 On platforms where some or all of this functionality is
199 unavailable, :func:`copy2` will preserve all the metadata
Miss Islington (bot)c27bcc32019-09-13 06:43:34 -0700200 it can; :func:`copy2` never raises an exception because it
201 cannot preserve file metadata.
Larry Hastings60eba572012-09-21 10:12:14 -0700202
203 :func:`copy2` uses :func:`copystat` to copy the file metadata.
204 Please see :func:`copystat` for more information
205 about platform support for modifying symbolic link metadata.
Georg Brandl116aa622007-08-15 14:28:22 +0000206
Steve Dowera00b5be2020-02-13 08:30:27 +0000207 .. audit-event:: shutil.copyfile src,dst shutil.copy2
208
209 .. audit-event:: shutil.copystat src,dst shutil.copy2
210
Antoine Pitrou78091e62011-12-29 18:54:15 +0100211 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700212 Added *follow_symlinks* argument, try to copy extended
213 file system attributes too (currently Linux only).
Larry Hastings60eba572012-09-21 10:12:14 -0700214 Now returns path to the newly created file.
Brian Curtin066dacf2012-06-19 10:03:05 -0500215
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200216 .. versionchanged:: 3.8
217 Platform-specific fast-copy syscalls may be used internally in order to
218 copy the file more efficiently. See
219 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
220
Georg Brandl86b2fb92008-07-16 03:43:04 +0000221.. function:: ignore_patterns(\*patterns)
222
223 This factory function creates a function that can be used as a callable for
224 :func:`copytree`\'s *ignore* argument, ignoring files and directories that
225 match one of the glob-style *patterns* provided. See the example below.
226
227
R David Murray6ffface2014-06-11 14:40:13 -0400228.. function:: copytree(src, dst, symlinks=False, ignore=None, \
jab9e00d9e2018-12-28 13:03:40 -0500229 copy_function=copy2, ignore_dangling_symlinks=False, \
230 dirs_exist_ok=False)
Georg Brandl116aa622007-08-15 14:28:22 +0000231
jab9e00d9e2018-12-28 13:03:40 -0500232 Recursively copy an entire directory tree rooted at *src* to a directory
233 named *dst* and return the destination directory. *dirs_exist_ok* dictates
234 whether to raise an exception in case *dst* or any missing parent directory
235 already exists.
236
237 Permissions and times of directories are copied with :func:`copystat`,
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200238 individual files are copied using :func:`~shutil.copy2`.
Georg Brandl116aa622007-08-15 14:28:22 +0000239
Georg Brandl86b2fb92008-07-16 03:43:04 +0000240 If *symlinks* is true, symbolic links in the source tree are represented as
Antoine Pitrou78091e62011-12-29 18:54:15 +0100241 symbolic links in the new tree and the metadata of the original links will
242 be copied as far as the platform allows; if false or omitted, the contents
243 and metadata of the linked files are copied to the new tree.
Georg Brandl86b2fb92008-07-16 03:43:04 +0000244
Tarek Ziadéfb437512010-04-20 08:57:33 +0000245 When *symlinks* is false, if the file pointed by the symlink doesn't
Martin Panter7462b6492015-11-02 03:37:02 +0000246 exist, an exception will be added in the list of errors raised in
247 an :exc:`Error` exception at the end of the copy process.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000248 You can set the optional *ignore_dangling_symlinks* flag to true if you
Tarek Ziadé8c26c7d2010-04-23 13:03:50 +0000249 want to silence this exception. Notice that this option has no effect
250 on platforms that don't support :func:`os.symlink`.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000251
Georg Brandl86b2fb92008-07-16 03:43:04 +0000252 If *ignore* is given, it must be a callable that will receive as its
253 arguments the directory being visited by :func:`copytree`, and a list of its
254 contents, as returned by :func:`os.listdir`. Since :func:`copytree` is
255 called recursively, the *ignore* callable will be called once for each
256 directory that is copied. The callable must return a sequence of directory
257 and file names relative to the current directory (i.e. a subset of the items
258 in its second argument); these names will then be ignored in the copy
259 process. :func:`ignore_patterns` can be used to create such a callable that
260 ignores names based on glob-style patterns.
261
262 If exception(s) occur, an :exc:`Error` is raised with a list of reasons.
263
Senthil Kumaran7f728c12012-02-13 23:30:47 +0800264 If *copy_function* is given, it must be a callable that will be used to copy
265 each file. It will be called with the source path and the destination path
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200266   as arguments. By default, :func:`~shutil.copy2` is used, but any function
267   that supports the same signature (like :func:`~shutil.copy`) can be used.
Georg Brandl116aa622007-08-15 14:28:22 +0000268
Miss Islington (bot)4fee28a2019-06-27 11:07:16 -0700269 .. audit-event:: shutil.copytree src,dst shutil.copytree
Miss Islington (bot)8763d432019-06-24 09:09:47 -0700270
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700271 .. versionchanged:: 3.3
272 Copy metadata when *symlinks* is false.
273 Now returns *dst*.
274
Tarek Ziadé5340db32010-04-19 22:30:51 +0000275 .. versionchanged:: 3.2
276 Added the *copy_function* argument to be able to provide a custom copy
277 function.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000278 Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
279 errors when *symlinks* is false.
280
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200281 .. versionchanged:: 3.8
282 Platform-specific fast-copy syscalls may be used internally in order to
283 copy the file more efficiently. See
284 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Georg Brandl96acb732012-06-24 17:39:05 +0200285
jab9e00d9e2018-12-28 13:03:40 -0500286 .. versionadded:: 3.8
287 The *dirs_exist_ok* parameter.
288
Georg Brandl18244152009-09-02 20:34:52 +0000289.. function:: rmtree(path, ignore_errors=False, onerror=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000290
291 .. index:: single: directory; deleting
292
Christian Heimes9bd667a2008-01-20 15:14:11 +0000293 Delete an entire directory tree; *path* must point to a directory (but not a
294 symbolic link to a directory). If *ignore_errors* is true, errors resulting
295 from failed removals will be ignored; if false or omitted, such errors are
296 handled by calling a handler specified by *onerror* or, if that is omitted,
297 they raise an exception.
Georg Brandl116aa622007-08-15 14:28:22 +0000298
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000299 .. note::
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200300
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000301 On platforms that support the necessary fd-based functions a symlink
Georg Brandl96acb732012-06-24 17:39:05 +0200302 attack resistant version of :func:`rmtree` is used by default. On other
303 platforms, the :func:`rmtree` implementation is susceptible to a symlink
304 attack: given proper timing and circumstances, attackers can manipulate
305 symlinks on the filesystem to delete files they wouldn't be able to access
306 otherwise. Applications can use the :data:`rmtree.avoids_symlink_attacks`
307 function attribute to determine which case applies.
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200308
Christian Heimes9bd667a2008-01-20 15:14:11 +0000309 If *onerror* is provided, it must be a callable that accepts three
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200310 parameters: *function*, *path*, and *excinfo*.
311
312 The first parameter, *function*, is the function which raised the exception;
313 it depends on the platform and implementation. The second parameter,
314 *path*, will be the path name passed to *function*. The third parameter,
315 *excinfo*, will be the exception information returned by
316 :func:`sys.exc_info`. Exceptions raised by *onerror* will not be caught.
317
Miss Islington (bot)4fee28a2019-06-27 11:07:16 -0700318 .. audit-event:: shutil.rmtree path shutil.rmtree
Miss Islington (bot)8763d432019-06-24 09:09:47 -0700319
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200320 .. versionchanged:: 3.3
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000321 Added a symlink attack resistant version that is used automatically
322 if platform supports fd-based functions.
Christian Heimes9bd667a2008-01-20 15:14:11 +0000323
Steve Dower9eb3d542019-08-21 15:52:42 -0700324 .. versionchanged:: 3.8
325 On Windows, will no longer delete the contents of a directory junction
326 before removing the junction.
327
Éric Araujo544e13d2012-06-24 13:53:48 -0400328 .. attribute:: rmtree.avoids_symlink_attacks
Hynek Schlawack2100b422012-06-23 20:28:32 +0200329
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000330 Indicates whether the current platform and implementation provides a
Georg Brandl96acb732012-06-24 17:39:05 +0200331 symlink attack resistant version of :func:`rmtree`. Currently this is
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000332 only true for platforms supporting fd-based directory access functions.
Hynek Schlawack2100b422012-06-23 20:28:32 +0200333
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000334 .. versionadded:: 3.3
Georg Brandl116aa622007-08-15 14:28:22 +0000335
Georg Brandl96acb732012-06-24 17:39:05 +0200336
R David Murray6ffface2014-06-11 14:40:13 -0400337.. function:: move(src, dst, copy_function=copy2)
Georg Brandl116aa622007-08-15 14:28:22 +0000338
Brian Curtin0d0a1de2012-06-18 18:41:07 -0500339 Recursively move a file or directory (*src*) to another location (*dst*)
340 and return the destination.
Georg Brandl116aa622007-08-15 14:28:22 +0000341
Benjamin Peterson218144a2015-03-22 10:11:54 -0400342 If the destination is an existing directory, then *src* is moved inside that
343 directory. If the destination already exists but is not a directory, it may
344 be overwritten depending on :func:`os.rename` semantics.
Éric Araujo14382dc2011-07-28 22:49:11 +0200345
346 If the destination is on the current filesystem, then :func:`os.rename` is
R David Murray6ffface2014-06-11 14:40:13 -0400347 used. Otherwise, *src* is copied to *dst* using *copy_function* and then
348 removed. In case of symlinks, a new symlink pointing to the target of *src*
349 will be created in or as *dst* and *src* will be removed.
350
351 If *copy_function* is given, it must be a callable that takes two arguments
352 *src* and *dst*, and will be used to copy *src* to *dest* if
353 :func:`os.rename` cannot be used. If the source is a directory,
354 :func:`copytree` is called, passing it the :func:`copy_function`. The
Mariatta70ee0cd2017-03-10 18:17:21 -0800355 default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the
R David Murray6ffface2014-06-11 14:40:13 -0400356 *copy_function* allows the move to succeed when it is not possible to also
357 copy the metadata, at the expense of not copying any of the metadata.
Antoine Pitrou0a08d7a2012-01-06 20:16:19 +0100358
Steve Dowera00b5be2020-02-13 08:30:27 +0000359 .. audit-event:: shutil.move src,dst shutil.move
360
Antoine Pitrou0a08d7a2012-01-06 20:16:19 +0100361 .. versionchanged:: 3.3
362 Added explicit symlink handling for foreign filesystems, thus adapting
363 it to the behavior of GNU's :program:`mv`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700364 Now returns *dst*.
Brian Curtin066dacf2012-06-19 10:03:05 -0500365
R David Murray6ffface2014-06-11 14:40:13 -0400366 .. versionchanged:: 3.5
367 Added the *copy_function* keyword argument.
368
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200369 .. versionchanged:: 3.8
370 Platform-specific fast-copy syscalls may be used internally in order to
371 copy the file more efficiently. See
372 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
373
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200374.. function:: disk_usage(path)
375
Éric Araujoe4d5b8e2011-08-08 16:51:11 +0200376 Return disk usage statistics about the given path as a :term:`named tuple`
377 with the attributes *total*, *used* and *free*, which are the amount of
Joe Pamerc8c02492018-09-25 10:57:36 -0400378 total, used and free space, in bytes. *path* may be a file or a
379 directory.
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200380
381 .. versionadded:: 3.3
382
Joe Pamerc8c02492018-09-25 10:57:36 -0400383 .. versionchanged:: 3.8
384 On Windows, *path* can now be a file or directory.
385
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400386 .. availability:: Unix, Windows.
Georg Brandl116aa622007-08-15 14:28:22 +0000387
Sandro Tosid902a142011-08-22 23:28:27 +0200388.. function:: chown(path, user=None, group=None)
389
390 Change owner *user* and/or *group* of the given *path*.
391
392 *user* can be a system user name or a uid; the same applies to *group*. At
393 least one argument is required.
394
395 See also :func:`os.chown`, the underlying function.
396
Steve Dowera00b5be2020-02-13 08:30:27 +0000397 .. audit-event:: shutil.chown path,user,group shutil.chown
398
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400399 .. availability:: Unix.
Sandro Tosid902a142011-08-22 23:28:27 +0200400
401 .. versionadded:: 3.3
402
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200403
Brian Curtinc57a3452012-06-22 16:00:30 -0500404.. function:: which(cmd, mode=os.F_OK | os.X_OK, path=None)
405
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200406 Return the path to an executable which would be run if the given *cmd* was
407 called. If no *cmd* would be called, return ``None``.
Brian Curtinc57a3452012-06-22 16:00:30 -0500408
Serhiy Storchaka6a7b3a72016-04-17 08:32:47 +0300409 *mode* is a permission mask passed to :func:`os.access`, by default
Brian Curtinc57a3452012-06-22 16:00:30 -0500410 determining if the file exists and executable.
411
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200412 When no *path* is specified, the results of :func:`os.environ` are used,
413 returning either the "PATH" value or a fallback of :attr:`os.defpath`.
Brian Curtinc57a3452012-06-22 16:00:30 -0500414
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200415 On Windows, the current directory is always prepended to the *path* whether
416 or not you use the default or provide your own, which is the behavior the
Donald Stufft8b852f12014-05-20 12:58:38 -0400417 command shell uses when finding executables. Additionally, when finding the
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200418 *cmd* in the *path*, the ``PATHEXT`` environment variable is checked. For
419 example, if you call ``shutil.which("python")``, :func:`which` will search
420 ``PATHEXT`` to know that it should look for ``python.exe`` within the *path*
421 directories. For example, on Windows::
Brian Curtinc57a3452012-06-22 16:00:30 -0500422
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200423 >>> shutil.which("python")
Serhiy Storchaka80c88f42013-01-22 10:31:36 +0200424 'C:\\Python33\\python.EXE'
Brian Curtinc57a3452012-06-22 16:00:30 -0500425
426 .. versionadded:: 3.3
Sandro Tosid902a142011-08-22 23:28:27 +0200427
Cheryl Sabella5680f652019-02-13 06:25:10 -0500428 .. versionchanged:: 3.8
429 The :class:`bytes` type is now accepted. If *cmd* type is
430 :class:`bytes`, the result type is also :class:`bytes`.
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200431
Georg Brandl116aa622007-08-15 14:28:22 +0000432.. exception:: Error
433
Éric Araujo14382dc2011-07-28 22:49:11 +0200434 This exception collects exceptions that are raised during a multi-file
435 operation. For :func:`copytree`, the exception argument is a list of 3-tuples
436 (*srcname*, *dstname*, *exception*).
Georg Brandl116aa622007-08-15 14:28:22 +0000437
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200438.. _shutil-platform-dependent-efficient-copy-operations:
439
440Platform-dependent efficient copy operations
441~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
442
443Starting from Python 3.8 all functions involving a file copy (:func:`copyfile`,
444:func:`copy`, :func:`copy2`, :func:`copytree`, and :func:`move`) may use
445platform-specific "fast-copy" syscalls in order to copy the file more
446efficiently (see :issue:`33671`).
447"fast-copy" means that the copying operation occurs within the kernel, avoiding
448the use of userspace buffers in Python as in "``outfd.write(infd.read())``".
449
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700450On macOS `fcopyfile`_ is used to copy the file content (not metadata).
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200451
Giampaolo Rodola413d9552019-05-30 14:05:41 +0800452On Linux :func:`os.sendfile` is used.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200453
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700454On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB
Inada Naoki4f190302019-03-02 13:31:01 +0900455instead of 64 KiB) and a :func:`memoryview`-based variant of
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700456:func:`shutil.copyfileobj` is used.
457
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200458If the fast-copy operation fails and no data was written in the destination
459file then shutil will silently fallback on using less efficient
460:func:`copyfileobj` function internally.
461
462.. versionchanged:: 3.8
Georg Brandl116aa622007-08-15 14:28:22 +0000463
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100464.. _shutil-copytree-example:
Georg Brandl116aa622007-08-15 14:28:22 +0000465
Tarek Ziadé396fad72010-02-23 05:30:31 +0000466copytree example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200467~~~~~~~~~~~~~~~~
Georg Brandl116aa622007-08-15 14:28:22 +0000468
469This example is the implementation of the :func:`copytree` function, described
470above, with the docstring omitted. It demonstrates many of the other functions
471provided by this module. ::
472
473 def copytree(src, dst, symlinks=False):
474 names = os.listdir(src)
475 os.makedirs(dst)
476 errors = []
477 for name in names:
478 srcname = os.path.join(src, name)
479 dstname = os.path.join(dst, name)
480 try:
481 if symlinks and os.path.islink(srcname):
482 linkto = os.readlink(srcname)
483 os.symlink(linkto, dstname)
484 elif os.path.isdir(srcname):
485 copytree(srcname, dstname, symlinks)
486 else:
487 copy2(srcname, dstname)
488 # XXX What about devices, sockets etc.?
Andrew Svetlov618c2e12012-12-15 22:59:24 +0200489 except OSError as why:
Georg Brandl116aa622007-08-15 14:28:22 +0000490 errors.append((srcname, dstname, str(why)))
491 # catch the Error from the recursive copytree so that we can
492 # continue with other files
493 except Error as err:
494 errors.extend(err.args[0])
495 try:
496 copystat(src, dst)
Georg Brandl116aa622007-08-15 14:28:22 +0000497 except OSError as why:
Andrew Svetlov2606a6f2012-12-19 14:33:35 +0200498 # can't copy file access times on Windows
499 if why.winerror is None:
500 errors.extend((src, dst, str(why)))
Georg Brandl116aa622007-08-15 14:28:22 +0000501 if errors:
Collin Winterc79461b2007-09-01 23:34:30 +0000502 raise Error(errors)
Georg Brandl116aa622007-08-15 14:28:22 +0000503
Tarek Ziadé396fad72010-02-23 05:30:31 +0000504Another example that uses the :func:`ignore_patterns` helper::
505
506 from shutil import copytree, ignore_patterns
507
508 copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))
509
510This will copy everything except ``.pyc`` files and files or directories whose
511name starts with ``tmp``.
512
513Another example that uses the *ignore* argument to add a logging call::
514
515 from shutil import copytree
516 import logging
517
518 def _logpath(path, names):
Vinay Sajipdd917f82016-08-31 08:22:29 +0100519 logging.info('Working in %s', path)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000520 return [] # nothing will be ignored
521
522 copytree(source, destination, ignore=_logpath)
523
524
Tim Golden78337792014-05-07 18:05:45 +0100525.. _shutil-rmtree-example:
526
527rmtree example
528~~~~~~~~~~~~~~
529
530This example shows how to remove a directory tree on Windows where some
531of the files have their read-only bit set. It uses the onerror callback
532to clear the readonly bit and reattempt the remove. Any subsequent failure
533will propagate. ::
534
535 import os, stat
536 import shutil
Tim Goldenba748852014-05-07 18:08:08 +0100537
Tim Golden78337792014-05-07 18:05:45 +0100538 def remove_readonly(func, path, _):
539 "Clear the readonly bit and reattempt the removal"
540 os.chmod(path, stat.S_IWRITE)
Tim Goldenba748852014-05-07 18:08:08 +0100541 func(path)
542
Tim Golden78337792014-05-07 18:05:45 +0100543 shutil.rmtree(directory, onerror=remove_readonly)
544
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000545.. _archiving-operations:
546
547Archiving operations
548--------------------
Tarek Ziadé396fad72010-02-23 05:30:31 +0000549
Georg Brandl03b9ad02012-06-24 18:09:40 +0200550.. versionadded:: 3.2
551
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200552.. versionchanged:: 3.5
553 Added support for the *xztar* format.
554
555
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100556High-level utilities to create and read compressed and archived files are also
557provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
558
Tarek Ziadé396fad72010-02-23 05:30:31 +0000559.. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]])
560
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000561 Create an archive file (such as zip or tar) and return its name.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000562
563 *base_name* is the name of the file to create, including the path, minus
564 any format-specific extension. *format* is the archive format: one of
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200565 "zip" (if the :mod:`zlib` module is available), "tar", "gztar" (if the
566 :mod:`zlib` module is available), "bztar" (if the :mod:`bz2` module is
567 available), or "xztar" (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000568
569 *root_dir* is a directory that will be the root directory of the
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000570 archive; for example, we typically chdir into *root_dir* before creating the
Tarek Ziadé396fad72010-02-23 05:30:31 +0000571 archive.
572
573 *base_dir* is the directory where we start archiving from;
Ezio Melotticb999a32010-04-20 11:26:51 +0000574 i.e. *base_dir* will be the common prefix of all files and
Tarek Ziadé396fad72010-02-23 05:30:31 +0000575 directories in the archive.
576
577 *root_dir* and *base_dir* both default to the current directory.
578
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100579 If *dry_run* is true, no archive is created, but the operations that would be
580 executed are logged to *logger*.
581
Tarek Ziadé396fad72010-02-23 05:30:31 +0000582 *owner* and *group* are used when creating a tar archive. By default,
583 uses the current owner and group.
584
Éric Araujo06c42a32011-11-07 17:31:07 +0100585 *logger* must be an object compatible with :pep:`282`, usually an instance of
586 :class:`logging.Logger`.
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000587
Georg Brandl36ac5102014-10-31 10:54:06 +0100588 The *verbose* argument is unused and deprecated.
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100589
Miss Islington (bot)4fee28a2019-06-27 11:07:16 -0700590 .. audit-event:: shutil.make_archive base_name,format,root_dir,base_dir shutil.make_archive
Miss Islington (bot)8763d432019-06-24 09:09:47 -0700591
CAM Gerlach89a89442019-04-06 23:47:49 -0500592 .. versionchanged:: 3.8
593 The modern pax (POSIX.1-2001) format is now used instead of
594 the legacy GNU format for archives created with ``format="tar"``.
595
Tarek Ziadé396fad72010-02-23 05:30:31 +0000596
597.. function:: get_archive_formats()
598
Éric Araujo14382dc2011-07-28 22:49:11 +0200599 Return a list of supported formats for archiving.
Martin Panterd21e0b52015-10-10 10:36:22 +0000600 Each element of the returned sequence is a tuple ``(name, description)``.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000601
602 By default :mod:`shutil` provides these formats:
603
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200604 - *zip*: ZIP file (if the :mod:`zlib` module is available).
CAM Gerlach89a89442019-04-06 23:47:49 -0500605 - *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200606 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
607 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
608 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000609
610 You can register new formats or provide your own archiver for any existing
611 formats, by using :func:`register_archive_format`.
612
Tarek Ziadé396fad72010-02-23 05:30:31 +0000613
614.. function:: register_archive_format(name, function, [extra_args, [description]])
615
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100616 Register an archiver for the format *name*.
617
618 *function* is the callable that will be used to unpack archives. The callable
619 will receive the *base_name* of the file to create, followed by the
620 *base_dir* (which defaults to :data:`os.curdir`) to start archiving from.
621 Further arguments are passed as keyword arguments: *owner*, *group*,
622 *dry_run* and *logger* (as passed in :func:`make_archive`).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000623
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000624 If given, *extra_args* is a sequence of ``(name, value)`` pairs that will be
Tarek Ziadé396fad72010-02-23 05:30:31 +0000625 used as extra keywords arguments when the archiver callable is used.
626
627 *description* is used by :func:`get_archive_formats` which returns the
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100628 list of archivers. Defaults to an empty string.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000629
Tarek Ziadé396fad72010-02-23 05:30:31 +0000630
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000631.. function:: unregister_archive_format(name)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000632
633 Remove the archive format *name* from the list of supported formats.
634
Tarek Ziadé396fad72010-02-23 05:30:31 +0000635
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000636.. function:: unpack_archive(filename[, extract_dir[, format]])
637
638 Unpack an archive. *filename* is the full path of the archive.
639
640 *extract_dir* is the name of the target directory where the archive is
641 unpacked. If not provided, the current working directory is used.
642
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200643 *format* is the archive format: one of "zip", "tar", "gztar", "bztar", or
644 "xztar". Or any other format registered with
645 :func:`register_unpack_format`. If not provided, :func:`unpack_archive`
646 will use the archive file name extension and see if an unpacker was
647 registered for that extension. In case none is found,
648 a :exc:`ValueError` is raised.
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000649
Steve Dowera00b5be2020-02-13 08:30:27 +0000650 .. audit-event:: shutil.unpack_archive filename,extract_dir,format shutil.unpack_archive
651
Jelle Zijlstraa12df7b2017-05-05 14:27:12 -0700652 .. versionchanged:: 3.7
653 Accepts a :term:`path-like object` for *filename* and *extract_dir*.
654
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000655
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000656.. function:: register_unpack_format(name, extensions, function[, extra_args[, description]])
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000657
658 Registers an unpack format. *name* is the name of the format and
659 *extensions* is a list of extensions corresponding to the format, like
660 ``.zip`` for Zip files.
661
662 *function* is the callable that will be used to unpack archives. The
663 callable will receive the path of the archive, followed by the directory
664 the archive must be extracted to.
665
666 When provided, *extra_args* is a sequence of ``(name, value)`` tuples that
667 will be passed as keywords arguments to the callable.
668
669 *description* can be provided to describe the format, and will be returned
670 by the :func:`get_unpack_formats` function.
671
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000672
673.. function:: unregister_unpack_format(name)
674
675 Unregister an unpack format. *name* is the name of the format.
676
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000677
678.. function:: get_unpack_formats()
679
680 Return a list of all registered formats for unpacking.
681 Each element of the returned sequence is a tuple
682 ``(name, extensions, description)``.
683
684 By default :mod:`shutil` provides these formats:
685
Martin Panter2f9171d2016-12-18 01:23:09 +0000686 - *zip*: ZIP file (unpacking compressed files works only if the corresponding
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200687 module is available).
688 - *tar*: uncompressed tar file.
689 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
690 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
691 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000692
693 You can register new formats or provide your own unpacker for any existing
694 formats, by using :func:`register_unpack_format`.
695
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000696
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100697.. _shutil-archiving-example:
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000698
Tarek Ziadé396fad72010-02-23 05:30:31 +0000699Archiving example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200700~~~~~~~~~~~~~~~~~
Tarek Ziadé396fad72010-02-23 05:30:31 +0000701
702In this example, we create a gzip'ed tar-file archive containing all files
703found in the :file:`.ssh` directory of the user::
704
705 >>> from shutil import make_archive
706 >>> import os
707 >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))
708 >>> root_dir = os.path.expanduser(os.path.join('~', '.ssh'))
709 >>> make_archive(archive_name, 'gztar', root_dir)
710 '/Users/tarek/myarchive.tar.gz'
711
Martin Panter1050d2d2016-07-26 11:18:21 +0200712The resulting archive contains:
713
714.. code-block:: shell-session
Tarek Ziadé396fad72010-02-23 05:30:31 +0000715
716 $ tar -tzvf /Users/tarek/myarchive.tar.gz
717 drwx------ tarek/staff 0 2010-02-01 16:23:40 ./
718 -rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys
719 -rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config
720 -rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa
721 -rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub
722 -rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa
723 -rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub
724 -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100725
726
727Querying the size of the output terminal
728----------------------------------------
729
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100730.. function:: get_terminal_size(fallback=(columns, lines))
731
732 Get the size of the terminal window.
733
734 For each of the two dimensions, the environment variable, ``COLUMNS``
735 and ``LINES`` respectively, is checked. If the variable is defined and
736 the value is a positive integer, it is used.
737
738 When ``COLUMNS`` or ``LINES`` is not defined, which is the common case,
739 the terminal connected to :data:`sys.__stdout__` is queried
740 by invoking :func:`os.get_terminal_size`.
741
742 If the terminal size cannot be successfully queried, either because
743 the system doesn't support querying, or because we are not
744 connected to a terminal, the value given in ``fallback`` parameter
745 is used. ``fallback`` defaults to ``(80, 24)`` which is the default
746 size used by many terminal emulators.
747
748 The value returned is a named tuple of type :class:`os.terminal_size`.
749
750 See also: The Single UNIX Specification, Version 2,
751 `Other Environment Variables`_.
752
Berker Peksag8e2bdc82016-12-27 15:09:11 +0300753 .. versionadded:: 3.3
754
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200755.. _`fcopyfile`:
756 http://www.manpagez.com/man/3/copyfile/
757
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100758.. _`Other Environment Variables`:
759 http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003