blob: 59390d0e907eb9ede6310e40b12deb42eddc09bb [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`shutil` --- High-level file operations
2============================================
3
4.. module:: shutil
5 :synopsis: High-level file operations, including copying.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
Christian Heimes5b5e81c2007-12-31 16:14:33 +00008.. partly based on the docstrings
Georg Brandl116aa622007-08-15 14:28:22 +00009
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040010**Source code:** :source:`Lib/shutil.py`
11
Georg Brandl116aa622007-08-15 14:28:22 +000012.. index::
13 single: file; copying
14 single: copying files
15
Raymond Hettinger4f707fd2011-01-10 19:54:11 +000016--------------
17
Georg Brandl116aa622007-08-15 14:28:22 +000018The :mod:`shutil` module offers a number of high-level operations on files and
19collections of files. In particular, functions are provided which support file
Guido van Rossum2cc30da2007-11-02 23:46:40 +000020copying and removal. For operations on individual files, see also the
21:mod:`os` module.
Georg Brandl116aa622007-08-15 14:28:22 +000022
Guido van Rossumda27fd22007-08-17 00:24:54 +000023.. warning::
Christian Heimes7f044312008-01-06 17:05:40 +000024
Senthil Kumaran7f728c12012-02-13 23:30:47 +080025 Even the higher-level file copying functions (:func:`shutil.copy`,
26 :func:`shutil.copy2`) cannot copy all file metadata.
Georg Brandl48310cd2009-01-03 21:18:54 +000027
Christian Heimes7f044312008-01-06 17:05:40 +000028 On POSIX platforms, this means that file owner and group are lost as well
Georg Brandlc575c902008-09-13 17:46:05 +000029 as ACLs. On Mac OS, the resource fork and other metadata are not used.
Christian Heimes7f044312008-01-06 17:05:40 +000030 This means that resources will be lost and file type and creator codes will
31 not be correct. On Windows, file owners, ACLs and alternate data streams
32 are not copied.
Georg Brandl116aa622007-08-15 14:28:22 +000033
Éric Araujo6e6cb8e2010-11-16 19:13:50 +000034
Éric Araujof2fbb9c2012-01-16 16:55:55 +010035.. _file-operations:
36
Tarek Ziadé396fad72010-02-23 05:30:31 +000037Directory and files operations
38------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +000039
Georg Brandl116aa622007-08-15 14:28:22 +000040.. function:: copyfileobj(fsrc, fdst[, length])
41
42 Copy the contents of the file-like object *fsrc* to the file-like object *fdst*.
43 The integer *length*, if given, is the buffer size. In particular, a negative
44 *length* value means to copy the data without looping over the source data in
45 chunks; by default the data is read in chunks to avoid uncontrolled memory
46 consumption. Note that if the current file position of the *fsrc* object is not
47 0, only the contents from the current file position to the end of the file will
48 be copied.
49
50
Larry Hastingsb4038062012-07-15 10:57:38 -070051.. function:: copyfile(src, dst, *, follow_symlinks=True)
Christian Heimesa342c012008-04-20 21:01:16 +000052
Senthil Kumaran7f728c12012-02-13 23:30:47 +080053 Copy the contents (no metadata) of the file named *src* to a file named
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020054 *dst* and return *dst* in the most efficient way possible.
Boris Verhovsky9488a522019-09-09 09:51:56 -060055 *src* and *dst* are path-like objects or path names given as strings.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020056
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +020057 *dst* must be the complete target file name; look at :func:`~shutil.copy`
Larry Hastings60eba572012-09-21 10:12:14 -070058 for a copy that accepts a target directory path. If *src* and *dst*
Hynek Schlawack48653762012-10-07 12:49:58 +020059 specify the same file, :exc:`SameFileError` is raised.
Senthil Kumaran1fd64822012-02-13 23:35:44 +080060
Larry Hastings60eba572012-09-21 10:12:14 -070061 The destination location must be writable; otherwise, an :exc:`OSError`
62 exception will be raised. If *dst* already exists, it will be replaced.
63 Special files such as character or block devices and pipes cannot be
64 copied with this function.
Christian Heimesa342c012008-04-20 21:01:16 +000065
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070066 If *follow_symlinks* is false and *src* is a symbolic link,
67 a new symbolic link will be created instead of copying the
68 file *src* points to.
Antoine Pitrou78091e62011-12-29 18:54:15 +010069
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020070 .. versionchanged:: 3.3
71 :exc:`IOError` used to be raised instead of :exc:`OSError`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070072 Added *follow_symlinks* argument.
73 Now returns *dst*.
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020074
Hynek Schlawack48653762012-10-07 12:49:58 +020075 .. versionchanged:: 3.4
Hynek Schlawack27ddb572012-10-28 13:59:27 +010076 Raise :exc:`SameFileError` instead of :exc:`Error`. Since the former is
77 a subclass of the latter, this change is backward compatible.
Hynek Schlawack48653762012-10-07 12:49:58 +020078
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +020079 .. versionchanged:: 3.8
80 Platform-specific fast-copy syscalls may be used internally in order to
81 copy the file more efficiently. See
82 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Hynek Schlawack48653762012-10-07 12:49:58 +020083
84.. exception:: SameFileError
85
86 This exception is raised if source and destination in :func:`copyfile`
87 are the same file.
88
89 .. versionadded:: 3.4
90
91
Larry Hastings7aa2c8b2012-07-15 16:58:29 -070092.. function:: copymode(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +000093
94 Copy the permission bits from *src* to *dst*. The file contents, owner, and
Boris Verhovsky9488a522019-09-09 09:51:56 -060095 group are unaffected. *src* and *dst* are path-like objects or path names
96 given as strings.
Larry Hastings60eba572012-09-21 10:12:14 -070097 If *follow_symlinks* is false, and both *src* and *dst* are symbolic links,
98 :func:`copymode` will attempt to modify the mode of *dst* itself (rather
99 than the file it points to). This functionality is not available on every
100 platform; please see :func:`copystat` for more information. If
101 :func:`copymode` cannot modify symbolic links on the local platform, and it
102 is asked to do so, it will do nothing and return.
Georg Brandl116aa622007-08-15 14:28:22 +0000103
Antoine Pitrou78091e62011-12-29 18:54:15 +0100104 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700105 Added *follow_symlinks* argument.
Georg Brandl116aa622007-08-15 14:28:22 +0000106
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700107.. function:: copystat(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000108
Larry Hastings60eba572012-09-21 10:12:14 -0700109 Copy the permission bits, last access time, last modification time, and
110 flags from *src* to *dst*. On Linux, :func:`copystat` also copies the
111 "extended attributes" where possible. The file contents, owner, and
Boris Verhovsky9488a522019-09-09 09:51:56 -0600112 group are unaffected. *src* and *dst* are path-like objects or path
113 names given as strings.
Larry Hastings60eba572012-09-21 10:12:14 -0700114
115 If *follow_symlinks* is false, and *src* and *dst* both
116 refer to symbolic links, :func:`copystat` will operate on
117 the symbolic links themselves rather than the files the
Martin Panter357ed2e2016-11-21 00:15:20 +0000118 symbolic links refer to—reading the information from the
Larry Hastings60eba572012-09-21 10:12:14 -0700119 *src* symbolic link, and writing the information to the
120 *dst* symbolic link.
121
122 .. note::
123
124 Not all platforms provide the ability to examine and
125 modify symbolic links. Python itself can tell you what
126 functionality is locally available.
127
128 * If ``os.chmod in os.supports_follow_symlinks`` is
129 ``True``, :func:`copystat` can modify the permission
130 bits of a symbolic link.
131
132 * If ``os.utime in os.supports_follow_symlinks`` is
133 ``True``, :func:`copystat` can modify the last access
134 and modification times of a symbolic link.
135
136 * If ``os.chflags in os.supports_follow_symlinks`` is
137 ``True``, :func:`copystat` can modify the flags of
138 a symbolic link. (``os.chflags`` is not available on
139 all platforms.)
140
141 On platforms where some or all of this functionality
142 is unavailable, when asked to modify a symbolic link,
143 :func:`copystat` will copy everything it can.
144 :func:`copystat` never returns failure.
145
146 Please see :data:`os.supports_follow_symlinks`
147 for more information.
Georg Brandl116aa622007-08-15 14:28:22 +0000148
Antoine Pitrou78091e62011-12-29 18:54:15 +0100149 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700150 Added *follow_symlinks* argument and support for Linux extended attributes.
Georg Brandl116aa622007-08-15 14:28:22 +0000151
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700152.. function:: copy(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000153
Larry Hastings60eba572012-09-21 10:12:14 -0700154 Copies the file *src* to the file or directory *dst*. *src* and *dst*
155 should be strings. If *dst* specifies a directory, the file will be
156 copied into *dst* using the base filename from *src*. Returns the
157 path to the newly created file.
158
159 If *follow_symlinks* is false, and *src* is a symbolic link,
160 *dst* will be created as a symbolic link. If *follow_symlinks*
161 is true and *src* is a symbolic link, *dst* will be a copy of
162 the file *src* refers to.
163
Mariatta70ee0cd2017-03-10 18:17:21 -0800164 :func:`~shutil.copy` copies the file data and the file's permission
Larry Hastings60eba572012-09-21 10:12:14 -0700165 mode (see :func:`os.chmod`). Other metadata, like the
166 file's creation and modification times, is not preserved.
167 To preserve all file metadata from the original, use
168 :func:`~shutil.copy2` instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000169
Antoine Pitrou78091e62011-12-29 18:54:15 +0100170 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700171 Added *follow_symlinks* argument.
Larry Hastings60eba572012-09-21 10:12:14 -0700172 Now returns path to the newly created file.
Georg Brandl116aa622007-08-15 14:28:22 +0000173
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200174 .. versionchanged:: 3.8
175 Platform-specific fast-copy syscalls may be used internally in order to
176 copy the file more efficiently. See
177 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
178
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700179.. function:: copy2(src, dst, *, follow_symlinks=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000180
Larry Hastings60eba572012-09-21 10:12:14 -0700181 Identical to :func:`~shutil.copy` except that :func:`copy2`
Zsolt Cserna4f399be2018-10-23 12:09:50 +0200182 also attempts to preserve file metadata.
Larry Hastings60eba572012-09-21 10:12:14 -0700183
184 When *follow_symlinks* is false, and *src* is a symbolic
185 link, :func:`copy2` attempts to copy all metadata from the
186 *src* symbolic link to the newly-created *dst* symbolic link.
187 However, this functionality is not available on all platforms.
188 On platforms where some or all of this functionality is
189 unavailable, :func:`copy2` will preserve all the metadata
Windson yang9585f462019-09-13 21:36:09 +0800190 it can; :func:`copy2` never raises an exception because it
191 cannot preserve file metadata.
Larry Hastings60eba572012-09-21 10:12:14 -0700192
193 :func:`copy2` uses :func:`copystat` to copy the file metadata.
194 Please see :func:`copystat` for more information
195 about platform support for modifying symbolic link metadata.
Georg Brandl116aa622007-08-15 14:28:22 +0000196
Antoine Pitrou78091e62011-12-29 18:54:15 +0100197 .. versionchanged:: 3.3
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700198 Added *follow_symlinks* argument, try to copy extended
199 file system attributes too (currently Linux only).
Larry Hastings60eba572012-09-21 10:12:14 -0700200 Now returns path to the newly created file.
Brian Curtin066dacf2012-06-19 10:03:05 -0500201
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200202 .. versionchanged:: 3.8
203 Platform-specific fast-copy syscalls may be used internally in order to
204 copy the file more efficiently. See
205 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
206
Georg Brandl86b2fb92008-07-16 03:43:04 +0000207.. function:: ignore_patterns(\*patterns)
208
209 This factory function creates a function that can be used as a callable for
210 :func:`copytree`\'s *ignore* argument, ignoring files and directories that
211 match one of the glob-style *patterns* provided. See the example below.
212
213
R David Murray6ffface2014-06-11 14:40:13 -0400214.. function:: copytree(src, dst, symlinks=False, ignore=None, \
jab9e00d9e2018-12-28 13:03:40 -0500215 copy_function=copy2, ignore_dangling_symlinks=False, \
216 dirs_exist_ok=False)
Georg Brandl116aa622007-08-15 14:28:22 +0000217
jab9e00d9e2018-12-28 13:03:40 -0500218 Recursively copy an entire directory tree rooted at *src* to a directory
219 named *dst* and return the destination directory. *dirs_exist_ok* dictates
220 whether to raise an exception in case *dst* or any missing parent directory
221 already exists.
222
223 Permissions and times of directories are copied with :func:`copystat`,
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200224 individual files are copied using :func:`~shutil.copy2`.
Georg Brandl116aa622007-08-15 14:28:22 +0000225
Georg Brandl86b2fb92008-07-16 03:43:04 +0000226 If *symlinks* is true, symbolic links in the source tree are represented as
Antoine Pitrou78091e62011-12-29 18:54:15 +0100227 symbolic links in the new tree and the metadata of the original links will
228 be copied as far as the platform allows; if false or omitted, the contents
229 and metadata of the linked files are copied to the new tree.
Georg Brandl86b2fb92008-07-16 03:43:04 +0000230
Tarek Ziadéfb437512010-04-20 08:57:33 +0000231 When *symlinks* is false, if the file pointed by the symlink doesn't
Martin Panter7462b6492015-11-02 03:37:02 +0000232 exist, an exception will be added in the list of errors raised in
233 an :exc:`Error` exception at the end of the copy process.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000234 You can set the optional *ignore_dangling_symlinks* flag to true if you
Tarek Ziadé8c26c7d2010-04-23 13:03:50 +0000235 want to silence this exception. Notice that this option has no effect
236 on platforms that don't support :func:`os.symlink`.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000237
Georg Brandl86b2fb92008-07-16 03:43:04 +0000238 If *ignore* is given, it must be a callable that will receive as its
239 arguments the directory being visited by :func:`copytree`, and a list of its
240 contents, as returned by :func:`os.listdir`. Since :func:`copytree` is
241 called recursively, the *ignore* callable will be called once for each
242 directory that is copied. The callable must return a sequence of directory
243 and file names relative to the current directory (i.e. a subset of the items
244 in its second argument); these names will then be ignored in the copy
245 process. :func:`ignore_patterns` can be used to create such a callable that
246 ignores names based on glob-style patterns.
247
248 If exception(s) occur, an :exc:`Error` is raised with a list of reasons.
249
Senthil Kumaran7f728c12012-02-13 23:30:47 +0800250 If *copy_function* is given, it must be a callable that will be used to copy
251 each file. It will be called with the source path and the destination path
Aurelio Jargasf6e17dd2019-05-11 04:51:45 +0200252   as arguments. By default, :func:`~shutil.copy2` is used, but any function
253   that supports the same signature (like :func:`~shutil.copy`) can be used.
Georg Brandl116aa622007-08-15 14:28:22 +0000254
Steve Dower44f91c32019-06-27 10:47:59 -0700255 .. audit-event:: shutil.copytree src,dst shutil.copytree
Steve Dower60419a72019-06-24 08:42:54 -0700256
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700257 .. versionchanged:: 3.3
258 Copy metadata when *symlinks* is false.
259 Now returns *dst*.
260
Tarek Ziadé5340db32010-04-19 22:30:51 +0000261 .. versionchanged:: 3.2
262 Added the *copy_function* argument to be able to provide a custom copy
263 function.
Tarek Ziadéfb437512010-04-20 08:57:33 +0000264 Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
265 errors when *symlinks* is false.
266
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200267 .. versionchanged:: 3.8
268 Platform-specific fast-copy syscalls may be used internally in order to
269 copy the file more efficiently. See
270 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
Georg Brandl96acb732012-06-24 17:39:05 +0200271
jab9e00d9e2018-12-28 13:03:40 -0500272 .. versionadded:: 3.8
273 The *dirs_exist_ok* parameter.
274
Georg Brandl18244152009-09-02 20:34:52 +0000275.. function:: rmtree(path, ignore_errors=False, onerror=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000276
277 .. index:: single: directory; deleting
278
Christian Heimes9bd667a2008-01-20 15:14:11 +0000279 Delete an entire directory tree; *path* must point to a directory (but not a
280 symbolic link to a directory). If *ignore_errors* is true, errors resulting
281 from failed removals will be ignored; if false or omitted, such errors are
282 handled by calling a handler specified by *onerror* or, if that is omitted,
283 they raise an exception.
Georg Brandl116aa622007-08-15 14:28:22 +0000284
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000285 .. note::
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200286
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000287 On platforms that support the necessary fd-based functions a symlink
Georg Brandl96acb732012-06-24 17:39:05 +0200288 attack resistant version of :func:`rmtree` is used by default. On other
289 platforms, the :func:`rmtree` implementation is susceptible to a symlink
290 attack: given proper timing and circumstances, attackers can manipulate
291 symlinks on the filesystem to delete files they wouldn't be able to access
292 otherwise. Applications can use the :data:`rmtree.avoids_symlink_attacks`
293 function attribute to determine which case applies.
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200294
Christian Heimes9bd667a2008-01-20 15:14:11 +0000295 If *onerror* is provided, it must be a callable that accepts three
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200296 parameters: *function*, *path*, and *excinfo*.
297
298 The first parameter, *function*, is the function which raised the exception;
299 it depends on the platform and implementation. The second parameter,
300 *path*, will be the path name passed to *function*. The third parameter,
301 *excinfo*, will be the exception information returned by
302 :func:`sys.exc_info`. Exceptions raised by *onerror* will not be caught.
303
Steve Dower44f91c32019-06-27 10:47:59 -0700304 .. audit-event:: shutil.rmtree path shutil.rmtree
Steve Dower60419a72019-06-24 08:42:54 -0700305
Hynek Schlawack67be92b2012-06-23 17:58:42 +0200306 .. versionchanged:: 3.3
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000307 Added a symlink attack resistant version that is used automatically
308 if platform supports fd-based functions.
Christian Heimes9bd667a2008-01-20 15:14:11 +0000309
Steve Dowerdf2d4a62019-08-21 15:27:33 -0700310 .. versionchanged:: 3.8
311 On Windows, will no longer delete the contents of a directory junction
312 before removing the junction.
313
Éric Araujo544e13d2012-06-24 13:53:48 -0400314 .. attribute:: rmtree.avoids_symlink_attacks
Hynek Schlawack2100b422012-06-23 20:28:32 +0200315
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000316 Indicates whether the current platform and implementation provides a
Georg Brandl96acb732012-06-24 17:39:05 +0200317 symlink attack resistant version of :func:`rmtree`. Currently this is
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000318 only true for platforms supporting fd-based directory access functions.
Hynek Schlawack2100b422012-06-23 20:28:32 +0200319
Nick Coghlan5b0eca12012-06-24 16:43:06 +1000320 .. versionadded:: 3.3
Georg Brandl116aa622007-08-15 14:28:22 +0000321
Georg Brandl96acb732012-06-24 17:39:05 +0200322
R David Murray6ffface2014-06-11 14:40:13 -0400323.. function:: move(src, dst, copy_function=copy2)
Georg Brandl116aa622007-08-15 14:28:22 +0000324
Brian Curtin0d0a1de2012-06-18 18:41:07 -0500325 Recursively move a file or directory (*src*) to another location (*dst*)
326 and return the destination.
Georg Brandl116aa622007-08-15 14:28:22 +0000327
Benjamin Peterson218144a2015-03-22 10:11:54 -0400328 If the destination is an existing directory, then *src* is moved inside that
329 directory. If the destination already exists but is not a directory, it may
330 be overwritten depending on :func:`os.rename` semantics.
Éric Araujo14382dc2011-07-28 22:49:11 +0200331
332 If the destination is on the current filesystem, then :func:`os.rename` is
R David Murray6ffface2014-06-11 14:40:13 -0400333 used. Otherwise, *src* is copied to *dst* using *copy_function* and then
334 removed. In case of symlinks, a new symlink pointing to the target of *src*
335 will be created in or as *dst* and *src* will be removed.
336
337 If *copy_function* is given, it must be a callable that takes two arguments
338 *src* and *dst*, and will be used to copy *src* to *dest* if
339 :func:`os.rename` cannot be used. If the source is a directory,
340 :func:`copytree` is called, passing it the :func:`copy_function`. The
Mariatta70ee0cd2017-03-10 18:17:21 -0800341 default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the
R David Murray6ffface2014-06-11 14:40:13 -0400342 *copy_function* allows the move to succeed when it is not possible to also
343 copy the metadata, at the expense of not copying any of the metadata.
Antoine Pitrou0a08d7a2012-01-06 20:16:19 +0100344
345 .. versionchanged:: 3.3
346 Added explicit symlink handling for foreign filesystems, thus adapting
347 it to the behavior of GNU's :program:`mv`.
Larry Hastings7aa2c8b2012-07-15 16:58:29 -0700348 Now returns *dst*.
Brian Curtin066dacf2012-06-19 10:03:05 -0500349
R David Murray6ffface2014-06-11 14:40:13 -0400350 .. versionchanged:: 3.5
351 Added the *copy_function* keyword argument.
352
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200353 .. versionchanged:: 3.8
354 Platform-specific fast-copy syscalls may be used internally in order to
355 copy the file more efficiently. See
356 :ref:`shutil-platform-dependent-efficient-copy-operations` section.
357
Maxwell A McKinnoncf57cab2019-09-30 19:41:16 -0700358 .. versionchanged:: 3.9
359 Accepts a :term:`path-like object` for both *src* and *dst*.
360
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200361.. function:: disk_usage(path)
362
Éric Araujoe4d5b8e2011-08-08 16:51:11 +0200363 Return disk usage statistics about the given path as a :term:`named tuple`
364 with the attributes *total*, *used* and *free*, which are the amount of
Joe Pamerc8c02492018-09-25 10:57:36 -0400365 total, used and free space, in bytes. *path* may be a file or a
366 directory.
Giampaolo Rodola'210e7ca2011-07-01 13:55:36 +0200367
368 .. versionadded:: 3.3
369
Joe Pamerc8c02492018-09-25 10:57:36 -0400370 .. versionchanged:: 3.8
371 On Windows, *path* can now be a file or directory.
372
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400373 .. availability:: Unix, Windows.
Georg Brandl116aa622007-08-15 14:28:22 +0000374
Sandro Tosid902a142011-08-22 23:28:27 +0200375.. function:: chown(path, user=None, group=None)
376
377 Change owner *user* and/or *group* of the given *path*.
378
379 *user* can be a system user name or a uid; the same applies to *group*. At
380 least one argument is required.
381
382 See also :func:`os.chown`, the underlying function.
383
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400384 .. availability:: Unix.
Sandro Tosid902a142011-08-22 23:28:27 +0200385
386 .. versionadded:: 3.3
387
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200388
Brian Curtinc57a3452012-06-22 16:00:30 -0500389.. function:: which(cmd, mode=os.F_OK | os.X_OK, path=None)
390
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200391 Return the path to an executable which would be run if the given *cmd* was
392 called. If no *cmd* would be called, return ``None``.
Brian Curtinc57a3452012-06-22 16:00:30 -0500393
Serhiy Storchaka6a7b3a72016-04-17 08:32:47 +0300394 *mode* is a permission mask passed to :func:`os.access`, by default
Brian Curtinc57a3452012-06-22 16:00:30 -0500395 determining if the file exists and executable.
396
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200397 When no *path* is specified, the results of :func:`os.environ` are used,
398 returning either the "PATH" value or a fallback of :attr:`os.defpath`.
Brian Curtinc57a3452012-06-22 16:00:30 -0500399
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200400 On Windows, the current directory is always prepended to the *path* whether
401 or not you use the default or provide your own, which is the behavior the
Donald Stufft8b852f12014-05-20 12:58:38 -0400402 command shell uses when finding executables. Additionally, when finding the
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200403 *cmd* in the *path*, the ``PATHEXT`` environment variable is checked. For
404 example, if you call ``shutil.which("python")``, :func:`which` will search
405 ``PATHEXT`` to know that it should look for ``python.exe`` within the *path*
406 directories. For example, on Windows::
Brian Curtinc57a3452012-06-22 16:00:30 -0500407
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200408 >>> shutil.which("python")
Serhiy Storchaka80c88f42013-01-22 10:31:36 +0200409 'C:\\Python33\\python.EXE'
Brian Curtinc57a3452012-06-22 16:00:30 -0500410
411 .. versionadded:: 3.3
Sandro Tosid902a142011-08-22 23:28:27 +0200412
Cheryl Sabella5680f652019-02-13 06:25:10 -0500413 .. versionchanged:: 3.8
414 The :class:`bytes` type is now accepted. If *cmd* type is
415 :class:`bytes`, the result type is also :class:`bytes`.
Georg Brandl4a7e25f2012-06-24 17:37:07 +0200416
Georg Brandl116aa622007-08-15 14:28:22 +0000417.. exception:: Error
418
Éric Araujo14382dc2011-07-28 22:49:11 +0200419 This exception collects exceptions that are raised during a multi-file
420 operation. For :func:`copytree`, the exception argument is a list of 3-tuples
421 (*srcname*, *dstname*, *exception*).
Georg Brandl116aa622007-08-15 14:28:22 +0000422
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200423.. _shutil-platform-dependent-efficient-copy-operations:
424
425Platform-dependent efficient copy operations
426~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
427
428Starting from Python 3.8 all functions involving a file copy (:func:`copyfile`,
429:func:`copy`, :func:`copy2`, :func:`copytree`, and :func:`move`) may use
430platform-specific "fast-copy" syscalls in order to copy the file more
431efficiently (see :issue:`33671`).
432"fast-copy" means that the copying operation occurs within the kernel, avoiding
433the use of userspace buffers in Python as in "``outfd.write(infd.read())``".
434
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700435On macOS `fcopyfile`_ is used to copy the file content (not metadata).
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200436
Giampaolo Rodola413d9552019-05-30 14:05:41 +0800437On Linux :func:`os.sendfile` is used.
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200438
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700439On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB
Inada Naoki4f190302019-03-02 13:31:01 +0900440instead of 64 KiB) and a :func:`memoryview`-based variant of
Giampaolo Rodolac7f02a92018-06-19 08:27:29 -0700441:func:`shutil.copyfileobj` is used.
442
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200443If the fast-copy operation fails and no data was written in the destination
444file then shutil will silently fallback on using less efficient
445:func:`copyfileobj` function internally.
446
447.. versionchanged:: 3.8
Georg Brandl116aa622007-08-15 14:28:22 +0000448
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100449.. _shutil-copytree-example:
Georg Brandl116aa622007-08-15 14:28:22 +0000450
Tarek Ziadé396fad72010-02-23 05:30:31 +0000451copytree example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200452~~~~~~~~~~~~~~~~
Georg Brandl116aa622007-08-15 14:28:22 +0000453
454This example is the implementation of the :func:`copytree` function, described
455above, with the docstring omitted. It demonstrates many of the other functions
456provided by this module. ::
457
458 def copytree(src, dst, symlinks=False):
459 names = os.listdir(src)
460 os.makedirs(dst)
461 errors = []
462 for name in names:
463 srcname = os.path.join(src, name)
464 dstname = os.path.join(dst, name)
465 try:
466 if symlinks and os.path.islink(srcname):
467 linkto = os.readlink(srcname)
468 os.symlink(linkto, dstname)
469 elif os.path.isdir(srcname):
470 copytree(srcname, dstname, symlinks)
471 else:
472 copy2(srcname, dstname)
473 # XXX What about devices, sockets etc.?
Andrew Svetlov618c2e12012-12-15 22:59:24 +0200474 except OSError as why:
Georg Brandl116aa622007-08-15 14:28:22 +0000475 errors.append((srcname, dstname, str(why)))
476 # catch the Error from the recursive copytree so that we can
477 # continue with other files
478 except Error as err:
479 errors.extend(err.args[0])
480 try:
481 copystat(src, dst)
Georg Brandl116aa622007-08-15 14:28:22 +0000482 except OSError as why:
Andrew Svetlov2606a6f2012-12-19 14:33:35 +0200483 # can't copy file access times on Windows
484 if why.winerror is None:
485 errors.extend((src, dst, str(why)))
Georg Brandl116aa622007-08-15 14:28:22 +0000486 if errors:
Collin Winterc79461b2007-09-01 23:34:30 +0000487 raise Error(errors)
Georg Brandl116aa622007-08-15 14:28:22 +0000488
Tarek Ziadé396fad72010-02-23 05:30:31 +0000489Another example that uses the :func:`ignore_patterns` helper::
490
491 from shutil import copytree, ignore_patterns
492
493 copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))
494
495This will copy everything except ``.pyc`` files and files or directories whose
496name starts with ``tmp``.
497
498Another example that uses the *ignore* argument to add a logging call::
499
500 from shutil import copytree
501 import logging
502
503 def _logpath(path, names):
Vinay Sajipdd917f82016-08-31 08:22:29 +0100504 logging.info('Working in %s', path)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000505 return [] # nothing will be ignored
506
507 copytree(source, destination, ignore=_logpath)
508
509
Tim Golden78337792014-05-07 18:05:45 +0100510.. _shutil-rmtree-example:
511
512rmtree example
513~~~~~~~~~~~~~~
514
515This example shows how to remove a directory tree on Windows where some
516of the files have their read-only bit set. It uses the onerror callback
517to clear the readonly bit and reattempt the remove. Any subsequent failure
518will propagate. ::
519
520 import os, stat
521 import shutil
Tim Goldenba748852014-05-07 18:08:08 +0100522
Tim Golden78337792014-05-07 18:05:45 +0100523 def remove_readonly(func, path, _):
524 "Clear the readonly bit and reattempt the removal"
525 os.chmod(path, stat.S_IWRITE)
Tim Goldenba748852014-05-07 18:08:08 +0100526 func(path)
527
Tim Golden78337792014-05-07 18:05:45 +0100528 shutil.rmtree(directory, onerror=remove_readonly)
529
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000530.. _archiving-operations:
531
532Archiving operations
533--------------------
Tarek Ziadé396fad72010-02-23 05:30:31 +0000534
Georg Brandl03b9ad02012-06-24 18:09:40 +0200535.. versionadded:: 3.2
536
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200537.. versionchanged:: 3.5
538 Added support for the *xztar* format.
539
540
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100541High-level utilities to create and read compressed and archived files are also
542provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
543
Tarek Ziadé396fad72010-02-23 05:30:31 +0000544.. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]])
545
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000546 Create an archive file (such as zip or tar) and return its name.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000547
548 *base_name* is the name of the file to create, including the path, minus
549 any format-specific extension. *format* is the archive format: one of
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200550 "zip" (if the :mod:`zlib` module is available), "tar", "gztar" (if the
551 :mod:`zlib` module is available), "bztar" (if the :mod:`bz2` module is
552 available), or "xztar" (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000553
554 *root_dir* is a directory that will be the root directory of the
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000555 archive; for example, we typically chdir into *root_dir* before creating the
Tarek Ziadé396fad72010-02-23 05:30:31 +0000556 archive.
557
558 *base_dir* is the directory where we start archiving from;
Ezio Melotticb999a32010-04-20 11:26:51 +0000559 i.e. *base_dir* will be the common prefix of all files and
Tarek Ziadé396fad72010-02-23 05:30:31 +0000560 directories in the archive.
561
562 *root_dir* and *base_dir* both default to the current directory.
563
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100564 If *dry_run* is true, no archive is created, but the operations that would be
565 executed are logged to *logger*.
566
Tarek Ziadé396fad72010-02-23 05:30:31 +0000567 *owner* and *group* are used when creating a tar archive. By default,
568 uses the current owner and group.
569
Éric Araujo06c42a32011-11-07 17:31:07 +0100570 *logger* must be an object compatible with :pep:`282`, usually an instance of
571 :class:`logging.Logger`.
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000572
Georg Brandl36ac5102014-10-31 10:54:06 +0100573 The *verbose* argument is unused and deprecated.
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100574
Steve Dower44f91c32019-06-27 10:47:59 -0700575 .. audit-event:: shutil.make_archive base_name,format,root_dir,base_dir shutil.make_archive
Steve Dower60419a72019-06-24 08:42:54 -0700576
CAM Gerlach89a89442019-04-06 23:47:49 -0500577 .. versionchanged:: 3.8
578 The modern pax (POSIX.1-2001) format is now used instead of
579 the legacy GNU format for archives created with ``format="tar"``.
580
Tarek Ziadé396fad72010-02-23 05:30:31 +0000581
582.. function:: get_archive_formats()
583
Éric Araujo14382dc2011-07-28 22:49:11 +0200584 Return a list of supported formats for archiving.
Martin Panterd21e0b52015-10-10 10:36:22 +0000585 Each element of the returned sequence is a tuple ``(name, description)``.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000586
587 By default :mod:`shutil` provides these formats:
588
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200589 - *zip*: ZIP file (if the :mod:`zlib` module is available).
CAM Gerlach89a89442019-04-06 23:47:49 -0500590 - *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200591 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
592 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
593 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000594
595 You can register new formats or provide your own archiver for any existing
596 formats, by using :func:`register_archive_format`.
597
Tarek Ziadé396fad72010-02-23 05:30:31 +0000598
599.. function:: register_archive_format(name, function, [extra_args, [description]])
600
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100601 Register an archiver for the format *name*.
602
603 *function* is the callable that will be used to unpack archives. The callable
604 will receive the *base_name* of the file to create, followed by the
605 *base_dir* (which defaults to :data:`os.curdir`) to start archiving from.
606 Further arguments are passed as keyword arguments: *owner*, *group*,
607 *dry_run* and *logger* (as passed in :func:`make_archive`).
Tarek Ziadé396fad72010-02-23 05:30:31 +0000608
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000609 If given, *extra_args* is a sequence of ``(name, value)`` pairs that will be
Tarek Ziadé396fad72010-02-23 05:30:31 +0000610 used as extra keywords arguments when the archiver callable is used.
611
612 *description* is used by :func:`get_archive_formats` which returns the
Georg Brandl9b1b0e52014-10-31 10:02:40 +0100613 list of archivers. Defaults to an empty string.
Tarek Ziadé396fad72010-02-23 05:30:31 +0000614
Tarek Ziadé396fad72010-02-23 05:30:31 +0000615
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000616.. function:: unregister_archive_format(name)
Tarek Ziadé396fad72010-02-23 05:30:31 +0000617
618 Remove the archive format *name* from the list of supported formats.
619
Tarek Ziadé396fad72010-02-23 05:30:31 +0000620
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000621.. function:: unpack_archive(filename[, extract_dir[, format]])
622
623 Unpack an archive. *filename* is the full path of the archive.
624
625 *extract_dir* is the name of the target directory where the archive is
626 unpacked. If not provided, the current working directory is used.
627
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200628 *format* is the archive format: one of "zip", "tar", "gztar", "bztar", or
629 "xztar". Or any other format registered with
630 :func:`register_unpack_format`. If not provided, :func:`unpack_archive`
631 will use the archive file name extension and see if an unpacker was
632 registered for that extension. In case none is found,
633 a :exc:`ValueError` is raised.
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000634
Jelle Zijlstraa12df7b2017-05-05 14:27:12 -0700635 .. versionchanged:: 3.7
636 Accepts a :term:`path-like object` for *filename* and *extract_dir*.
637
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000638
Raymond Hettinger0929b1f2011-01-23 11:29:08 +0000639.. function:: register_unpack_format(name, extensions, function[, extra_args[, description]])
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000640
641 Registers an unpack format. *name* is the name of the format and
642 *extensions* is a list of extensions corresponding to the format, like
643 ``.zip`` for Zip files.
644
645 *function* is the callable that will be used to unpack archives. The
646 callable will receive the path of the archive, followed by the directory
647 the archive must be extracted to.
648
649 When provided, *extra_args* is a sequence of ``(name, value)`` tuples that
650 will be passed as keywords arguments to the callable.
651
652 *description* can be provided to describe the format, and will be returned
653 by the :func:`get_unpack_formats` function.
654
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000655
656.. function:: unregister_unpack_format(name)
657
658 Unregister an unpack format. *name* is the name of the format.
659
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000660
661.. function:: get_unpack_formats()
662
663 Return a list of all registered formats for unpacking.
664 Each element of the returned sequence is a tuple
665 ``(name, extensions, description)``.
666
667 By default :mod:`shutil` provides these formats:
668
Martin Panter2f9171d2016-12-18 01:23:09 +0000669 - *zip*: ZIP file (unpacking compressed files works only if the corresponding
Serhiy Storchaka20cdffd2016-12-16 18:58:33 +0200670 module is available).
671 - *tar*: uncompressed tar file.
672 - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
673 - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
674 - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000675
676 You can register new formats or provide your own unpacker for any existing
677 formats, by using :func:`register_unpack_format`.
678
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000679
Éric Araujof2fbb9c2012-01-16 16:55:55 +0100680.. _shutil-archiving-example:
Tarek Ziadé6ac91722010-04-28 17:51:36 +0000681
Tarek Ziadé396fad72010-02-23 05:30:31 +0000682Archiving example
Georg Brandl03b9ad02012-06-24 18:09:40 +0200683~~~~~~~~~~~~~~~~~
Tarek Ziadé396fad72010-02-23 05:30:31 +0000684
685In this example, we create a gzip'ed tar-file archive containing all files
686found in the :file:`.ssh` directory of the user::
687
688 >>> from shutil import make_archive
689 >>> import os
690 >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))
691 >>> root_dir = os.path.expanduser(os.path.join('~', '.ssh'))
692 >>> make_archive(archive_name, 'gztar', root_dir)
693 '/Users/tarek/myarchive.tar.gz'
694
Martin Panter1050d2d2016-07-26 11:18:21 +0200695The resulting archive contains:
696
697.. code-block:: shell-session
Tarek Ziadé396fad72010-02-23 05:30:31 +0000698
699 $ tar -tzvf /Users/tarek/myarchive.tar.gz
700 drwx------ tarek/staff 0 2010-02-01 16:23:40 ./
701 -rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys
702 -rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config
703 -rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa
704 -rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub
705 -rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa
706 -rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub
707 -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100708
709
710Querying the size of the output terminal
711----------------------------------------
712
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100713.. function:: get_terminal_size(fallback=(columns, lines))
714
715 Get the size of the terminal window.
716
717 For each of the two dimensions, the environment variable, ``COLUMNS``
718 and ``LINES`` respectively, is checked. If the variable is defined and
719 the value is a positive integer, it is used.
720
721 When ``COLUMNS`` or ``LINES`` is not defined, which is the common case,
722 the terminal connected to :data:`sys.__stdout__` is queried
723 by invoking :func:`os.get_terminal_size`.
724
725 If the terminal size cannot be successfully queried, either because
726 the system doesn't support querying, or because we are not
727 connected to a terminal, the value given in ``fallback`` parameter
728 is used. ``fallback`` defaults to ``(80, 24)`` which is the default
729 size used by many terminal emulators.
730
731 The value returned is a named tuple of type :class:`os.terminal_size`.
732
733 See also: The Single UNIX Specification, Version 2,
734 `Other Environment Variables`_.
735
Berker Peksag8e2bdc82016-12-27 15:09:11 +0300736 .. versionadded:: 3.3
737
Giampaolo Rodola4a172cc2018-06-12 23:04:50 +0200738.. _`fcopyfile`:
739 http://www.manpagez.com/man/3/copyfile/
740
Antoine Pitroubcf2b592012-02-08 23:28:36 +0100741.. _`Other Environment Variables`:
742 http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003