blob: 88c5f62dd4c99fdb011cd40c08d991713d50ebab [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`shutil` --- High-level file operations
2============================================
3
4.. module:: shutil
5 :synopsis: High-level file operations, including copying.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
Christian Heimes5b5e81c2007-12-31 16:14:33 +00008.. partly based on the docstrings
Georg Brandl116aa622007-08-15 14:28:22 +00009
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040010**Source code:** :source:`Lib/shutil.py`
11
Georg Brandl116aa622007-08-15 14:28:22 +000012.. index::
13 single: file; copying
14 single: copying files
15
Raymond Hettinger4f707fd2011-01-10 19:54:11 +000016--------------
17
Georg Brandl116aa622007-08-15 14:28:22 +000018The :mod:`shutil` module offers a number of high-level operations on files and
19collections of files. In particular, functions are provided which support file
Guido van Rossum2cc30da2007-11-02 23:46:40 +000020copying and removal. For operations on individual files, see also the
21:mod:`os` module.
Georg Brandl116aa622007-08-15 14:28:22 +000022
Guido van Rossumda27fd22007-08-17 00:24:54 +000023.. warning::
Christian Heimes7f044312008-01-06 17:05:40 +000024
Senthil Kumaran7f728c12012-02-13 23:30:47 +080025 Even the higher-level file copying functions (:func:`shutil.copy`,
26 :func:`shutil.copy2`) cannot copy all file metadata.
Georg Brandl48310cd2009-01-03 21:18:54 +000027
Christian Heimes7f044312008-01-06 17:05:40 +000028 On POSIX platforms, this means that file owner and group are lost as well
Georg Brandlc575c902008-09-13 17:46:05 +000029 as ACLs. On Mac OS, the resource fork and other metadata are not used.
Christian Heimes7f044312008-01-06 17:05:40 +000030 This means that resources will be lost and file type and creator codes will
31 not be correct. On Windows, file owners, ACLs and alternate data streams
32 are not copied.
Georg Brandl116aa622007-08-15 14:28:22 +000033
Éric Araujo6e6cb8e2010-11-16 19:13:50 +000034
Éric Araujof2fbb9c2012-01-16 16:55:55 +010035.. _file-operations:
36
Tarek Ziadé396fad72010-02-23 05:30:31 +000037Directory and files operations
38------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +000039
Georg Brandl116aa622007-08-15 14:28:22 +000040.. function:: copyfileobj(fsrc, fdst[, length])
41
42 Copy the contents of the file-like object *fsrc* to the file-like object *fdst*.
43 The integer *length*, if given, is the buffer size. In particular, a negative
44 *length* value means to copy the data without looping over the source data in
45 chunks; by default the data is read in chunks to avoid uncontrolled memory
46 consumption. Note that if the current file position of the *fsrc* object is not
47 0, only the contents from the current file position to the end of the file will
48 be copied.
49
50
Larry Hastingsb4038062012-07-15 10:57:38 -070051.. function:: copyfile(src, dst, *, follow_symlinks=True)
Christian Heimesa342c012008-04-20 21:01:16 +000052
Senthil Kumaran7f728c12012-02-13 23:30:47 +080053 Copy the contents (no metadata) of the file named *src* to a file named
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020054 *dst* and return *dst* in the most efficient way possible.
Boris Verhovsky9488a522019-09-09 09:51:56 -060055 *src* and *dst* are path-like objects or path names given as strings.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020056
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +020057 *dst* must be the complete target file name; look at :func:`~shutil.copy`
Larry Hastings60eba572012-09-21 10:12:14 -070058 for a copy that accepts a target directory path. If *src* and *dst*
Hynek Schlawack48653762012-10-07 12:49:58 +020059 specify the same file, :exc:`SameFileError` is raised.
Senthil Kumaran1fd64822012-02-13 23:35:44 +080060
Larry Hastings60eba572012-09-21 10:12:14 -070061 The destination location must be writable; otherwise, an :exc:`OSError`
62 exception will be raised. If *dst* already exists, it will be replaced.
63 Special files such as character or block devices and pipes cannot be
64 copied with this function.
Christian Heimesa342c012008-04-20 21:01:16 +000065
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070066 If *follow_symlinks* is false and *src* is a symbolic link,
67 a new symbolic link will be created instead of copying the
68 file *src* points to.
Antoine Pitrou78091e62011-12-29 18:54:15 +010069
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020070 .. versionchanged:: 3.3
71 :exc:`IOError` used to be raised instead of :exc:`OSError`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070072 Added *follow_symlinks* argument.
73 Now returns *dst*.
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020074
Hynek Schlawack48653762012-10-07 12:49:58 +020075 .. versionchanged:: 3.4
Hynek Schlawack27ddb572012-10-28 13:59:27 +010076 Raise :exc:`SameFileError` instead of :exc:`Error`. Since the former is
77 a subclass of the latter, this change is backward compatible.
Hynek Schlawack48653762012-10-07 12:49:58 +020078
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020079 .. versionchanged:: 3.8
80 Platform-specific fast-copy syscalls may be used internally in order to
81 copy the file more efficiently. See
82 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Hynek Schlawack48653762012-10-07 12:49:58 +020083
84.. exception:: SameFileError
85
86 This exception is raised if source and destination in :func:`copyfile`
87 are the same file.
88
89 .. versionadded:: 3.4
90
91
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070092.. function:: copymode(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +000093
94 Copy the permission bits from *src* to *dst*. The file contents, owner, and
Boris Verhovsky9488a522019-09-09 09:51:56 -060095 group are unaffected. *src* and *dst* are path-like objects or path names
96 given as strings.
Larry Hastings60eba572012-09-21 10:12:14 -070097 If *follow_symlinks* is false, and both *src* and *dst* are symbolic links,
98 :func:`copymode` will attempt to modify the mode of *dst* itself (rather
99 than the file it points to). This functionality is not available on every
100 platform; please see :func:`copystat` for more information. If
101 :func:`copymode` cannot modify symbolic links on the local platform, and it
102 is asked to do so, it will do nothing and return.
Georg Brandl116aa622007-08-15 14:28:22 +0000103
Antoine Pitrou78091e62011-12-29 18:54:15 +0100104 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700105 Added *follow_symlinks* argument.
Georg Brandl116aa622007-08-15 14:28:22 +0000106
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700107.. function:: copystat(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000108
Larry Hastings60eba572012-09-21 10:12:14 -0700109 Copy the permission bits, last access time, last modification time, and
110 flags from *src* to *dst*. On Linux, :func:`copystat` also copies the
111 "extended attributes" where possible. The file contents, owner, and
Boris Verhovsky9488a522019-09-09 09:51:56 -0600112 group are unaffected. *src* and *dst* are path-like objects or path
113 names given as strings.
Larry Hastings60eba572012-09-21 10:12:14 -0700114
115 If *follow_symlinks* is false, and *src* and *dst* both
116 refer to symbolic links, :func:`copystat` will operate on
117 the symbolic links themselves rather than the files the
Martin Panter357ed2e2016-11-21 00:15:20 +0000118 symbolic links refer to—reading the information from the
Larry Hastings60eba572012-09-21 10:12:14 -0700119 *src* symbolic link, and writing the information to the
120 *dst* symbolic link.
121
122 .. note::
123
124 Not all platforms provide the ability to examine and
125 modify symbolic links. Python itself can tell you what
126 functionality is locally available.
127
128 * If ``os.chmod in os.supports_follow_symlinks`` is
129 ``True``, :func:`copystat` can modify the permission
130 bits of a symbolic link.
131
132 * If ``os.utime in os.supports_follow_symlinks`` is
133 ``True``, :func:`copystat` can modify the last access
134 and modification times of a symbolic link.
135
136 * If ``os.chflags in os.supports_follow_symlinks`` is
137 ``True``, :func:`copystat` can modify the flags of
138 a symbolic link. (``os.chflags`` is not available on
139 all platforms.)
140
141 On platforms where some or all of this functionality
142 is unavailable, when asked to modify a symbolic link,
143 :func:`copystat` will copy everything it can.
144 :func:`copystat` never returns failure.
145
146 Please see :data:`os.supports_follow_symlinks`
147 for more information.
Georg Brandl116aa622007-08-15 14:28:22 +0000148
Antoine Pitrou78091e62011-12-29 18:54:15 +0100149 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700150 Added *follow_symlinks* argument and support for Linux extended attributes.
Georg Brandl116aa622007-08-15 14:28:22 +0000151
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700152.. function:: copy(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000153
Larry Hastings60eba572012-09-21 10:12:14 -0700154 Copies the file *src* to the file or directory *dst*. *src* and *dst*
155 should be strings. If *dst* specifies a directory, the file will be
156 copied into *dst* using the base filename from *src*. Returns the
157 path to the newly created file.
158
159 If *follow_symlinks* is false, and *src* is a symbolic link,
160 *dst* will be created as a symbolic link. If *follow_symlinks*
161 is true and *src* is a symbolic link, *dst* will be a copy of
162 the file *src* refers to.
163
Mariatta70ee0cd2017-03-10 18:17:21 -0800164 :func:`~shutil.copy` copies the file data and the file's permission
Larry Hastings60eba572012-09-21 10:12:14 -0700165 mode (see :func:`os.chmod`). Other metadata, like the
166 file's creation and modification times, is not preserved.
167 To preserve all file metadata from the original, use
168 :func:`~shutil.copy2` instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000169
Antoine Pitrou78091e62011-12-29 18:54:15 +0100170 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700171 Added *follow_symlinks* argument.
Larry Hastings60eba572012-09-21 10:12:14 -0700172 Now returns path to the newly created file.
Georg Brandl116aa622007-08-15 14:28:22 +0000173
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200174 .. versionchanged:: 3.8
175 Platform-specific fast-copy syscalls may be used internally in order to
176 copy the file more efficiently. See
177 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
178
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700179.. function:: copy2(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000180
Larry Hastings60eba572012-09-21 10:12:14 -0700181 Identical to :func:`~shutil.copy` except that :func:`copy2`
Zsolt Cserna4f399be2018-10-23 12:09:50 +0200182 also attempts to preserve file metadata.
Larry Hastings60eba572012-09-21 10:12:14 -0700183
184 When *follow_symlinks* is false, and *src* is a symbolic
185 link, :func:`copy2` attempts to copy all metadata from the
186 *src* symbolic link to the newly-created *dst* symbolic link.
187 However, this functionality is not available on all platforms.
188 On platforms where some or all of this functionality is
189 unavailable, :func:`copy2` will preserve all the metadata
190 it can; :func:`copy2` never returns failure.
191
192 :func:`copy2` uses :func:`copystat` to copy the file metadata.
193 Please see :func:`copystat` for more information
194 about platform support for modifying symbolic link metadata.
Georg Brandl116aa622007-08-15 14:28:22 +0000195
Antoine Pitrou78091e62011-12-29 18:54:15 +0100196 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700197 Added *follow_symlinks* argument, try to copy extended
198 file system attributes too (currently Linux only).
Larry Hastings60eba572012-09-21 10:12:14 -0700199 Now returns path to the newly created file.
Brian Curtin066dacf2012-06-19 10:03:05 -0500200
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200201 .. versionchanged:: 3.8
202 Platform-specific fast-copy syscalls may be used internally in order to
203 copy the file more efficiently. See
204 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
205
Georg Brandl86b2fb92008-07-16 03:43:04 +0000206.. function:: ignore_patterns(\*patterns)
207
208 This factory function creates a function that can be used as a callable for
209 :func:`copytree`\'s *ignore* argument, ignoring files and directories that
210 match one of the glob-style *patterns* provided. See the example below.
211
212
R David Murray6ffface2014-06-11 14:40:13 -0400213.. function:: copytree(src, dst, symlinks=False, ignore=None, \
jab9e00d9e2018-12-28 13:03:40 -0500214 copy_function=copy2, ignore_dangling_symlinks=False, \
215 dirs_exist_ok=False)
Georg Brandl116aa622007-08-15 14:28:22 +0000216
jab9e00d9e2018-12-28 13:03:40 -0500217 Recursively copy an entire directory tree rooted at *src* to a directory
218 named *dst* and return the destination directory. *dirs_exist_ok* dictates
219 whether to raise an exception in case *dst* or any missing parent directory
220 already exists.
221
222 Permissions and times of directories are copied with :func:`copystat`,
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200223 individual files are copied using :func:`~shutil.copy2`.
Georg Brandl116aa622007-08-15 14:28:22 +0000224
Georg Brandl86b2fb92008-07-16 03:43:04 +0000225 If *symlinks* is true, symbolic links in the source tree are represented as
Antoine Pitrou78091e62011-12-29 18:54:15 +0100226 symbolic links in the new tree and the metadata of the original links will
227 be copied as far as the platform allows; if false or omitted, the contents
228 and metadata of the linked files are copied to the new tree.
Georg Brandl86b2fb92008-07-16 03:43:04 +0000229
Tarek Ziadéfb437512010-04-20 08:57:33 +0000230 When *symlinks* is false, if the file pointed by the symlink doesn't
Martin Panter7462b6492015-11-02 03:37:02 +0000231 exist, an exception will be added in the list of errors raised in
232 an :exc:`Error` exception at the end of the copy process.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000233 You can set the optional *ignore_dangling_symlinks* flag to true if you
Tarek Ziadé8c26c7d2010-04-23 13:03:50 +0000234 want to silence this exception. Notice that this option has no effect
235 on platforms that don't support :func:`os.symlink`.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000236
Georg Brandl86b2fb92008-07-16 03:43:04 +0000237 If *ignore* is given, it must be a callable that will receive as its
238 arguments the directory being visited by :func:`copytree`, and a list of its
239 contents, as returned by :func:`os.listdir`. Since :func:`copytree` is
240 called recursively, the *ignore* callable will be called once for each
241 directory that is copied. The callable must return a sequence of directory
242 and file names relative to the current directory (i.e. a subset of the items
243 in its second argument); these names will then be ignored in the copy
244 process. :func:`ignore_patterns` can be used to create such a callable that
245 ignores names based on glob-style patterns.
246
247 If exception(s) occur, an :exc:`Error` is raised with a list of reasons.
248
Senthil Kumaran7f728c12012-02-13 23:30:47 +0800249 If *copy_function* is given, it must be a callable that will be used to copy
250 each file. It will be called with the source path and the destination path
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200251   as arguments. By default, :func:`~shutil.copy2` is used, but any function
252   that supports the same signature (like :func:`~shutil.copy`) can be used.
Georg Brandl116aa622007-08-15 14:28:22 +0000253
Steve Dower44f91c32019-06-27 10:47:59 -0700254 .. audit-event:: shutil.copytree src,dst shutil.copytree
Steve Dower60419a72019-06-24 08:42:54 -0700255
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700256 .. versionchanged:: 3.3
257 Copy metadata when *symlinks* is false.
258 Now returns *dst*.
259
Tarek Ziadé5340db32010-04-19 22:30:51 +0000260 .. versionchanged:: 3.2
261 Added the *copy_function* argument to be able to provide a custom copy
262 function.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000263 Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
264 errors when *symlinks* is false.
265
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200266 .. versionchanged:: 3.8
267 Platform-specific fast-copy syscalls may be used internally in order to
268 copy the file more efficiently. See
269 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Georg Brandl96acb732012-06-24 17:39:05 +0200270
jab9e00d9e2018-12-28 13:03:40 -0500271 .. versionadded:: 3.8
272 The *dirs_exist_ok* parameter.
273
Georg Brandl18244152009-09-02 20:34:52 +0000274.. function:: rmtree(path, ignore_errors=False, onerror=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000275
276 .. index:: single: directory; deleting
277
Christian Heimes9bd667a2008-01-20 15:14:11 +0000278 Delete an entire directory tree; *path* must point to a directory (but not a
279 symbolic link to a directory). If *ignore_errors* is true, errors resulting
280 from failed removals will be ignored; if false or omitted, such errors are
281 handled by calling a handler specified by *onerror* or, if that is omitted,
282 they raise an exception.
Georg Brandl116aa622007-08-15 14:28:22 +0000283
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000284 .. note::
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200285
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000286 On platforms that support the necessary fd-based functions a symlink
Georg Brandl96acb732012-06-24 17:39:05 +0200287 attack resistant version of :func:`rmtree` is used by default. On other
288 platforms, the :func:`rmtree` implementation is susceptible to a symlink
289 attack: given proper timing and circumstances, attackers can manipulate
290 symlinks on the filesystem to delete files they wouldn't be able to access
291 otherwise. Applications can use the :data:`rmtree.avoids_symlink_attacks`
292 function attribute to determine which case applies.
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200293
Christian Heimes9bd667a2008-01-20 15:14:11 +0000294 If *onerror* is provided, it must be a callable that accepts three
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200295 parameters: *function*, *path*, and *excinfo*.
296
297 The first parameter, *function*, is the function which raised the exception;
298 it depends on the platform and implementation. The second parameter,
299 *path*, will be the path name passed to *function*. The third parameter,
300 *excinfo*, will be the exception information returned by
301 :func:`sys.exc_info`. Exceptions raised by *onerror* will not be caught.
302
Steve Dower44f91c32019-06-27 10:47:59 -0700303 .. audit-event:: shutil.rmtree path shutil.rmtree
Steve Dower60419a72019-06-24 08:42:54 -0700304
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200305 .. versionchanged:: 3.3
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000306 Added a symlink attack resistant version that is used automatically
307 if platform supports fd-based functions.
Christian Heimes9bd667a2008-01-20 15:14:11 +0000308
Steve Dowerdf2d4a62019-08-21 15:27:33 -0700309 .. versionchanged:: 3.8
310 On Windows, will no longer delete the contents of a directory junction
311 before removing the junction.
312
Éric Araujo544e13d2012-06-24 13:53:48 -0400313 .. attribute:: rmtree.avoids_symlink_attacks
Hynek Schlawack2100b422012-06-23 20:28:32 +0200314
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000315 Indicates whether the current platform and implementation provides a
Georg Brandl96acb732012-06-24 17:39:05 +0200316 symlink attack resistant version of :func:`rmtree`. Currently this is
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000317 only true for platforms supporting fd-based directory access functions.
Hynek Schlawack2100b422012-06-23 20:28:32 +0200318
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000319 .. versionadded:: 3.3
Georg Brandl116aa622007-08-15 14:28:22 +0000320
Georg Brandl96acb732012-06-24 17:39:05 +0200321
R David Murray6ffface2014-06-11 14:40:13 -0400322.. function:: move(src, dst, copy_function=copy2)
Georg Brandl116aa622007-08-15 14:28:22 +0000323
Brian Curtin0d0a1de2012-06-18 18:41:07 -0500324 Recursively move a file or directory (*src*) to another location (*dst*)
325 and return the destination.
Georg Brandl116aa622007-08-15 14:28:22 +0000326
Benjamin Peterson218144a2015-03-22 10:11:54 -0400327 If the destination is an existing directory, then *src* is moved inside that
328 directory. If the destination already exists but is not a directory, it may
329 be overwritten depending on :func:`os.rename` semantics.
Éric Araujo14382dc2011-07-28 22:49:11 +0200330
331 If the destination is on the current filesystem, then :func:`os.rename` is
R David Murray6ffface2014-06-11 14:40:13 -0400332 used. Otherwise, *src* is copied to *dst* using *copy_function* and then
333 removed. In case of symlinks, a new symlink pointing to the target of *src*
334 will be created in or as *dst* and *src* will be removed.
335
336 If *copy_function* is given, it must be a callable that takes two arguments
337 *src* and *dst*, and will be used to copy *src* to *dest* if
338 :func:`os.rename` cannot be used. If the source is a directory,
339 :func:`copytree` is called, passing it the :func:`copy_function`. The
Mariatta70ee0cd2017-03-10 18:17:21 -0800340 default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the
R David Murray6ffface2014-06-11 14:40:13 -0400341 *copy_function* allows the move to succeed when it is not possible to also
342 copy the metadata, at the expense of not copying any of the metadata.
Antoine Pitrou0a08d7a2012-01-06 20:16:19 +0100343
344 .. versionchanged:: 3.3
345 Added explicit symlink handling for foreign filesystems, thus adapting
346 it to the behavior of GNU's :program:`mv`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700347 Now returns *dst*.
Brian Curtin066dacf2012-06-19 10:03:05 -0500348
R David Murray6ffface2014-06-11 14:40:13 -0400349 .. versionchanged:: 3.5
350 Added the *copy_function* keyword argument.
351
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200352 .. versionchanged:: 3.8
353 Platform-specific fast-copy syscalls may be used internally in order to
354 copy the file more efficiently. See
355 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
356
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200357.. function:: disk_usage(path)
358
Éric Araujoe4d5b8e2011-08-08 16:51:11 +0200359 Return disk usage statistics about the given path as a :term:`named tuple`
360 with the attributes *total*, *used* and *free*, which are the amount of
Joe Pamerc8c02492018-09-25 10:57:36 -0400361 total, used and free space, in bytes. *path* may be a file or a
362 directory.
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200363
364 .. versionadded:: 3.3
365
Joe Pamerc8c02492018-09-25 10:57:36 -0400366 .. versionchanged:: 3.8
367 On Windows, *path* can now be a file or directory.
368
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400369 .. availability:: Unix, Windows.
Georg Brandl116aa622007-08-15 14:28:22 +0000370
Sandro Tosid902a142011-08-22 23:28:27 +0200371.. function:: chown(path, user=None, group=None)
372
373 Change owner *user* and/or *group* of the given *path*.
374
375 *user* can be a system user name or a uid; the same applies to *group*. At
376 least one argument is required.
377
378 See also :func:`os.chown`, the underlying function.
379
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400380 .. availability:: Unix.
Sandro Tosid902a142011-08-22 23:28:27 +0200381
382 .. versionadded:: 3.3
383
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200384
Brian Curtinc57a3452012-06-22 16:00:30 -0500385.. function:: which(cmd, mode=os.F_OK | os.X_OK, path=None)
386
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200387 Return the path to an executable which would be run if the given *cmd* was
388 called. If no *cmd* would be called, return ``None``.
Brian Curtinc57a3452012-06-22 16:00:30 -0500389
Serhiy Storchaka6a7b3a72016-04-17 08:32:47 +0300390 *mode* is a permission mask passed to :func:`os.access`, by default
Brian Curtinc57a3452012-06-22 16:00:30 -0500391 determining if the file exists and executable.
392
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200393 When no *path* is specified, the results of :func:`os.environ` are used,
394 returning either the "PATH" value or a fallback of :attr:`os.defpath`.
Brian Curtinc57a3452012-06-22 16:00:30 -0500395
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200396 On Windows, the current directory is always prepended to the *path* whether
397 or not you use the default or provide your own, which is the behavior the
Donald Stufft8b852f12014-05-20 12:58:38 -0400398 command shell uses when finding executables. Additionally, when finding the
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200399 *cmd* in the *path*, the ``PATHEXT`` environment variable is checked. For
400 example, if you call ``shutil.which("python")``, :func:`which` will search
401 ``PATHEXT`` to know that it should look for ``python.exe`` within the *path*
402 directories. For example, on Windows::
Brian Curtinc57a3452012-06-22 16:00:30 -0500403
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200404 >>> shutil.which("python")
Serhiy Storchaka80c88f42013-01-22 10:31:36 +0200405 'C:\\Python33\\python.EXE'
Brian Curtinc57a3452012-06-22 16:00:30 -0500406
407 .. versionadded:: 3.3
Sandro Tosid902a142011-08-22 23:28:27 +0200408
Cheryl Sabella5680f652019-02-13 06:25:10 -0500409 .. versionchanged:: 3.8
410 The :class:`bytes` type is now accepted. If *cmd* type is
411 :class:`bytes`, the result type is also :class:`bytes`.
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200412
Georg Brandl116aa622007-08-15 14:28:22 +0000413.. exception:: Error
414
Éric Araujo14382dc2011-07-28 22:49:11 +0200415 This exception collects exceptions that are raised during a multi-file
416 operation. For :func:`copytree`, the exception argument is a list of 3-tuples
417 (*srcname*, *dstname*, *exception*).
Georg Brandl116aa622007-08-15 14:28:22 +0000418
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200419.. _shutil-platform-dependent-efficient-copy-operations:
420
421Platform-dependent efficient copy operations
422~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
423
424Starting from Python 3.8 all functions involving a file copy (:func:`copyfile`,
425:func:`copy`, :func:`copy2`, :func:`copytree`, and :func:`move`) may use
426platform-specific "fast-copy" syscalls in order to copy the file more
427efficiently (see :issue:`33671`).
428"fast-copy" means that the copying operation occurs within the kernel, avoiding
429the use of userspace buffers in Python as in "``outfd.write(infd.read())``".
430
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700431On macOS `fcopyfile`_ is used to copy the file content (not metadata).
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200432
Giampaolo Rodola413d9552019-05-30 14:05:41 +0800433On Linux :func:`os.sendfile` is used.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200434
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700435On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB
Inada Naoki4f190302019-03-02 13:31:01 +0900436instead of 64 KiB) and a :func:`memoryview`-based variant of
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700437:func:`shutil.copyfileobj` is used.
438
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200439If the fast-copy operation fails and no data was written in the destination
440file then shutil will silently fallback on using less efficient
441:func:`copyfileobj` function internally.
442
443.. versionchanged:: 3.8
Georg Brandl116aa622007-08-15 14:28:22 +0000444
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100445.. _shutil-copytree-example:
Georg Brandl116aa622007-08-15 14:28:22 +0000446
Tarek Ziadé396fad72010-02-23 05:30:31 +0000447copytree example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200448~~~~~~~~~~~~~~~~
Georg Brandl116aa622007-08-15 14:28:22 +0000449
450This example is the implementation of the :func:`copytree` function, described
451above, with the docstring omitted. It demonstrates many of the other functions
452provided by this module. ::
453
454 def copytree(src, dst, symlinks=False):
455 names = os.listdir(src)
456 os.makedirs(dst)
457 errors = []
458 for name in names:
459 srcname = os.path.join(src, name)
460 dstname = os.path.join(dst, name)
461 try:
462 if symlinks and os.path.islink(srcname):
463 linkto = os.readlink(srcname)
464 os.symlink(linkto, dstname)
465 elif os.path.isdir(srcname):
466 copytree(srcname, dstname, symlinks)
467 else:
468 copy2(srcname, dstname)
469 # XXX What about devices, sockets etc.?
Andrew Svetlov618c2e12012-12-15 22:59:24 +0200470 except OSError as why:
Georg Brandl116aa622007-08-15 14:28:22 +0000471 errors.append((srcname, dstname, str(why)))
472 # catch the Error from the recursive copytree so that we can
473 # continue with other files
474 except Error as err:
475 errors.extend(err.args[0])
476 try:
477 copystat(src, dst)
Georg Brandl116aa622007-08-15 14:28:22 +0000478 except OSError as why:
Andrew Svetlov2606a6f2012-12-19 14:33:35 +0200479 # can't copy file access times on Windows
480 if why.winerror is None:
481 errors.extend((src, dst, str(why)))
Georg Brandl116aa622007-08-15 14:28:22 +0000482 if errors:
Collin Winterc79461b2007-09-01 23:34:30 +0000483 raise Error(errors)
Georg Brandl116aa622007-08-15 14:28:22 +0000484
Tarek Ziadé396fad72010-02-23 05:30:31 +0000485Another example that uses the :func:`ignore_patterns` helper::
486
487 from shutil import copytree, ignore_patterns
488
489 copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))
490
491This will copy everything except ``.pyc`` files and files or directories whose
492name starts with ``tmp``.
493
494Another example that uses the *ignore* argument to add a logging call::
495
496 from shutil import copytree
497 import logging
498
499 def _logpath(path, names):
Vinay Sajipdd917f82016-08-31 08:22:29 +0100500 logging.info('Working in %s', path)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000501 return [] # nothing will be ignored
502
503 copytree(source, destination, ignore=_logpath)
504
505
Tim Golden78337792014-05-07 18:05:45 +0100506.. _shutil-rmtree-example:
507
508rmtree example
509~~~~~~~~~~~~~~
510
511This example shows how to remove a directory tree on Windows where some
512of the files have their read-only bit set. It uses the onerror callback
513to clear the readonly bit and reattempt the remove. Any subsequent failure
514will propagate. ::
515
516 import os, stat
517 import shutil
Tim Goldenba748852014-05-07 18:08:08 +0100518
Tim Golden78337792014-05-07 18:05:45 +0100519 def remove_readonly(func, path, _):
520 "Clear the readonly bit and reattempt the removal"
521 os.chmod(path, stat.S_IWRITE)
Tim Goldenba748852014-05-07 18:08:08 +0100522 func(path)
523
Tim Golden78337792014-05-07 18:05:45 +0100524 shutil.rmtree(directory, onerror=remove_readonly)
525
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000526.. _archiving-operations:
527
528Archiving operations
529--------------------
Tarek Ziadé396fad72010-02-23 05:30:31 +0000530
Georg Brandl03b9ad02012-06-24 18:09:40 +0200531.. versionadded:: 3.2
532
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200533.. versionchanged:: 3.5
534 Added support for the *xztar* format.
535
536
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100537High-level utilities to create and read compressed and archived files are also
538provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
539
Tarek Ziadé396fad72010-02-23 05:30:31 +0000540.. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]])
541
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000542 Create an archive file (such as zip or tar) and return its name.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000543
544 *base_name* is the name of the file to create, including the path, minus
545 any format-specific extension. *format* is the archive format: one of
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200546 "zip" (if the :mod:`zlib` module is available), "tar", "gztar" (if the
547 :mod:`zlib` module is available), "bztar" (if the :mod:`bz2` module is
548 available), or "xztar" (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000549
550 *root_dir* is a directory that will be the root directory of the
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000551 archive; for example, we typically chdir into *root_dir* before creating the
Tarek Ziadé396fad72010-02-23 05:30:31 +0000552 archive.
553
554 *base_dir* is the directory where we start archiving from;
Ezio Melotticb999a32010-04-20 11:26:51 +0000555 i.e. *base_dir* will be the common prefix of all files and
Tarek Ziadé396fad72010-02-23 05:30:31 +0000556 directories in the archive.
557
558 *root_dir* and *base_dir* both default to the current directory.
559
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100560 If *dry_run* is true, no archive is created, but the operations that would be
561 executed are logged to *logger*.
562
Tarek Ziadé396fad72010-02-23 05:30:31 +0000563 *owner* and *group* are used when creating a tar archive. By default,
564 uses the current owner and group.
565
Éric Araujo06c42a32011-11-07 17:31:07 +0100566 *logger* must be an object compatible with :pep:`282`, usually an instance of
567 :class:`logging.Logger`.
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000568
Georg Brandl36ac5102014-10-31 10:54:06 +0100569 The *verbose* argument is unused and deprecated.
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100570
Steve Dower44f91c32019-06-27 10:47:59 -0700571 .. audit-event:: shutil.make_archive base_name,format,root_dir,base_dir shutil.make_archive
Steve Dower60419a72019-06-24 08:42:54 -0700572
CAM Gerlach89a89442019-04-06 23:47:49 -0500573 .. versionchanged:: 3.8
574 The modern pax (POSIX.1-2001) format is now used instead of
575 the legacy GNU format for archives created with ``format="tar"``.
576
Tarek Ziadé396fad72010-02-23 05:30:31 +0000577
578.. function:: get_archive_formats()
579
Éric Araujo14382dc2011-07-28 22:49:11 +0200580 Return a list of supported formats for archiving.
Martin Panterd21e0b52015-10-10 10:36:22 +0000581 Each element of the returned sequence is a tuple ``(name, description)``.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000582
583 By default :mod:`shutil` provides these formats:
584
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200585 - *zip*: ZIP file (if the :mod:`zlib` module is available).
CAM Gerlach89a89442019-04-06 23:47:49 -0500586 - *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200587 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
588 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
589 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000590
591 You can register new formats or provide your own archiver for any existing
592 formats, by using :func:`register_archive_format`.
593
Tarek Ziadé396fad72010-02-23 05:30:31 +0000594
595.. function:: register_archive_format(name, function, [extra_args, [description]])
596
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100597 Register an archiver for the format *name*.
598
599 *function* is the callable that will be used to unpack archives. The callable
600 will receive the *base_name* of the file to create, followed by the
601 *base_dir* (which defaults to :data:`os.curdir`) to start archiving from.
602 Further arguments are passed as keyword arguments: *owner*, *group*,
603 *dry_run* and *logger* (as passed in :func:`make_archive`).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000604
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000605 If given, *extra_args* is a sequence of ``(name, value)`` pairs that will be
Tarek Ziadé396fad72010-02-23 05:30:31 +0000606 used as extra keywords arguments when the archiver callable is used.
607
608 *description* is used by :func:`get_archive_formats` which returns the
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100609 list of archivers. Defaults to an empty string.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000610
Tarek Ziadé396fad72010-02-23 05:30:31 +0000611
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000612.. function:: unregister_archive_format(name)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000613
614 Remove the archive format *name* from the list of supported formats.
615
Tarek Ziadé396fad72010-02-23 05:30:31 +0000616
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000617.. function:: unpack_archive(filename[, extract_dir[, format]])
618
619 Unpack an archive. *filename* is the full path of the archive.
620
621 *extract_dir* is the name of the target directory where the archive is
622 unpacked. If not provided, the current working directory is used.
623
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200624 *format* is the archive format: one of "zip", "tar", "gztar", "bztar", or
625 "xztar". Or any other format registered with
626 :func:`register_unpack_format`. If not provided, :func:`unpack_archive`
627 will use the archive file name extension and see if an unpacker was
628 registered for that extension. In case none is found,
629 a :exc:`ValueError` is raised.
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000630
Jelle Zijlstraa12df7b2017-05-05 14:27:12 -0700631 .. versionchanged:: 3.7
632 Accepts a :term:`path-like object` for *filename* and *extract_dir*.
633
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000634
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000635.. function:: register_unpack_format(name, extensions, function[, extra_args[, description]])
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000636
637 Registers an unpack format. *name* is the name of the format and
638 *extensions* is a list of extensions corresponding to the format, like
639 ``.zip`` for Zip files.
640
641 *function* is the callable that will be used to unpack archives. The
642 callable will receive the path of the archive, followed by the directory
643 the archive must be extracted to.
644
645 When provided, *extra_args* is a sequence of ``(name, value)`` tuples that
646 will be passed as keywords arguments to the callable.
647
648 *description* can be provided to describe the format, and will be returned
649 by the :func:`get_unpack_formats` function.
650
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000651
652.. function:: unregister_unpack_format(name)
653
654 Unregister an unpack format. *name* is the name of the format.
655
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000656
657.. function:: get_unpack_formats()
658
659 Return a list of all registered formats for unpacking.
660 Each element of the returned sequence is a tuple
661 ``(name, extensions, description)``.
662
663 By default :mod:`shutil` provides these formats:
664
Martin Panter2f9171d2016-12-18 01:23:09 +0000665 - *zip*: ZIP file (unpacking compressed files works only if the corresponding
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200666 module is available).
667 - *tar*: uncompressed tar file.
668 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
669 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
670 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000671
672 You can register new formats or provide your own unpacker for any existing
673 formats, by using :func:`register_unpack_format`.
674
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000675
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100676.. _shutil-archiving-example:
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000677
Tarek Ziadé396fad72010-02-23 05:30:31 +0000678Archiving example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200679~~~~~~~~~~~~~~~~~
Tarek Ziadé396fad72010-02-23 05:30:31 +0000680
681In this example, we create a gzip'ed tar-file archive containing all files
682found in the :file:`.ssh` directory of the user::
683
684 >>> from shutil import make_archive
685 >>> import os
686 >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))
687 >>> root_dir = os.path.expanduser(os.path.join('~', '.ssh'))
688 >>> make_archive(archive_name, 'gztar', root_dir)
689 '/Users/tarek/myarchive.tar.gz'
690
Martin Panter1050d2d2016-07-26 11:18:21 +0200691The resulting archive contains:
692
693.. code-block:: shell-session
Tarek Ziadé396fad72010-02-23 05:30:31 +0000694
695 $ tar -tzvf /Users/tarek/myarchive.tar.gz
696 drwx------ tarek/staff 0 2010-02-01 16:23:40 ./
697 -rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys
698 -rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config
699 -rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa
700 -rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub
701 -rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa
702 -rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub
703 -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100704
705
706Querying the size of the output terminal
707----------------------------------------
708
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100709.. function:: get_terminal_size(fallback=(columns, lines))
710
711 Get the size of the terminal window.
712
713 For each of the two dimensions, the environment variable, ``COLUMNS``
714 and ``LINES`` respectively, is checked. If the variable is defined and
715 the value is a positive integer, it is used.
716
717 When ``COLUMNS`` or ``LINES`` is not defined, which is the common case,
718 the terminal connected to :data:`sys.__stdout__` is queried
719 by invoking :func:`os.get_terminal_size`.
720
721 If the terminal size cannot be successfully queried, either because
722 the system doesn't support querying, or because we are not
723 connected to a terminal, the value given in ``fallback`` parameter
724 is used. ``fallback`` defaults to ``(80, 24)`` which is the default
725 size used by many terminal emulators.
726
727 The value returned is a named tuple of type :class:`os.terminal_size`.
728
729 See also: The Single UNIX Specification, Version 2,
730 `Other Environment Variables`_.
731
Berker Peksag8e2bdc82016-12-27 15:09:11 +0300732 .. versionadded:: 3.3
733
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200734.. _`fcopyfile`:
735 http://www.manpagez.com/man/3/copyfile/
736
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100737.. _`Other Environment Variables`:
738 http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003