blob: 3dca40f6c4b82c9aa9f4f1fd91550ba712459828 [file] [log] [blame]
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +00001\section{\module{mmap} ---
Tim Peters5ebfd362001-11-13 23:11:19 +00002Memory-mapped file support}
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +00003
4\declaremodule{builtin}{mmap}
Fred Drakec37b65e2001-11-28 07:26:15 +00005\modulesynopsis{Interface to memory-mapped files for \UNIX\ and Windows.}
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +00006
Fred Drake1722e4a2001-12-03 18:27:22 +00007Memory-mapped file objects behave like both strings and like
8file objects. Unlike normal string objects, however, these are
9mutable. You can use mmap objects in most places where strings
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000010are expected; for example, you can use the \module{re} module to
11search through a memory-mapped file. Since they're mutable, you can
Fred Drake8ff4cd72000-09-05 13:50:21 +000012change a single character by doing \code{obj[\var{index}] = 'a'}, or
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000013change a substring by assigning to a slice:
Fred Drake8ff4cd72000-09-05 13:50:21 +000014\code{obj[\var{i1}:\var{i2}] = '...'}. You can also read and write
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000015data starting at the current file position, and \method{seek()}
16through the file to different positions.
17
Fred Drake50555452001-09-25 19:00:08 +000018A memory-mapped file is created by the \function{mmap()} function,
19which is different on \UNIX{} and on Windows. In either case you must
20provide a file descriptor for a file opened for update.
21If you wish to map an existing Python file object, use its
22\method{fileno()} method to obtain the correct value for the
23\var{fileno} parameter. Otherwise, you can open the file using the
24\function{os.open()} function, which returns a file descriptor
25directly (the file still needs to be closed when done).
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000026
Fred Drake1722e4a2001-12-03 18:27:22 +000027For both the \UNIX{} and Windows versions of the function,
28\var{access} may be specified as an optional keyword parameter.
29\var{access} accepts one of three values: \constant{ACCESS_READ},
30\constant{ACCESS_WRITE}, or \constant{ACCESS_COPY} to specify
31readonly, write-through or copy-on-write memory respectively.
32\var{access} can be used on both \UNIX{} and Windows. If
33\var{access} is not specified, Windows mmap returns a write-through
34mapping. The initial memory values for all three access types are
35taken from the specified file. Assignment to an
36\constant{ACCESS_READ} memory map raises a \exception{TypeError}
37exception. Assignment to an \constant{ACCESS_WRITE} memory map
Fred Drakeb184ae82005-01-19 03:39:17 +000038affects both memory and the underlying file. Assignment to an
Fred Drake1722e4a2001-12-03 18:27:22 +000039\constant{ACCESS_COPY} memory map affects memory but does not update
Neal Norwitz0e6bc8c2006-02-05 05:45:43 +000040the underlying file. \versionchanged[To map anonymous memory,
41-1 should be passed as the fileno along with the length]{2.5}
Fred Drake1722e4a2001-12-03 18:27:22 +000042
Tim Peters5ebfd362001-11-13 23:11:19 +000043\begin{funcdesc}{mmap}{fileno, length\optional{, tagname\optional{, access}}}
44 \strong{(Windows version)} Maps \var{length} bytes from the file
45 specified by the file handle \var{fileno}, and returns a mmap
Tim Peters0b4d1ee2004-06-06 16:51:46 +000046 object. If \var{length} is larger than the current size of the file,
47 the file is extended to contain \var{length} bytes. If \var{length}
48 is \code{0}, the maximum length of the map is the current size
49 of the file, except that if the file is empty Windows raises an
50 exception (you cannot create an empty mapping on Windows).
51
Tim Peters5ebfd362001-11-13 23:11:19 +000052 \var{tagname}, if specified and not \code{None}, is a string giving
53 a tag name for the mapping. Windows allows you to have many
54 different mappings against the same file. If you specify the name
55 of an existing tag, that tag is opened, otherwise a new tag of this
56 name is created. If this parameter is omitted or \code{None}, the
57 mapping is created without a name. Avoiding the use of the tag
58 parameter will assist in keeping your code portable between \UNIX{}
59 and Windows.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000060\end{funcdesc}
61
Fred Drake1722e4a2001-12-03 18:27:22 +000062\begin{funcdescni}{mmap}{fileno, length\optional{, flags\optional{,
63 prot\optional{, access}}}}
Tim Peters5ebfd362001-11-13 23:11:19 +000064 \strong{(\UNIX{} version)} Maps \var{length} bytes from the file
65 specified by the file descriptor \var{fileno}, and returns a mmap
Martin v. Löwis7fe60c02005-03-03 11:22:44 +000066 object. If \var{length} is \code{0}, the maximum length of the map
Georg Brandl296152e2006-01-23 21:33:03 +000067 will be the current size of the file when \function{mmap()} is
Martin v. Löwis7fe60c02005-03-03 11:22:44 +000068 called.
69
Tim Peters5ebfd362001-11-13 23:11:19 +000070 \var{flags} specifies the nature of the mapping.
71 \constant{MAP_PRIVATE} creates a private copy-on-write mapping, so
72 changes to the contents of the mmap object will be private to this
73 process, and \constant{MAP_SHARED} creates a mapping that's shared
74 with all other processes mapping the same areas of the file. The
75 default value is \constant{MAP_SHARED}.
Tim Peters0b4d1ee2004-06-06 16:51:46 +000076
Tim Peters5ebfd362001-11-13 23:11:19 +000077 \var{prot}, if specified, gives the desired memory protection; the
78 two most useful values are \constant{PROT_READ} and
79 \constant{PROT_WRITE}, to specify that the pages may be read or
80 written. \var{prot} defaults to \constant{PROT_READ | PROT_WRITE}.
Tim Peters0b4d1ee2004-06-06 16:51:46 +000081
Tim Peters5ebfd362001-11-13 23:11:19 +000082 \var{access} may be specified in lieu of \var{flags} and \var{prot}
Fred Drake1722e4a2001-12-03 18:27:22 +000083 as an optional keyword parameter. It is an error to specify both
84 \var{flags}, \var{prot} and \var{access}. See the description of
85 \var{access} above for information on how to use this parameter.
86\end{funcdescni}
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000087
Fred Drake50555452001-09-25 19:00:08 +000088
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000089Memory-mapped file objects support the following methods:
90
91
92\begin{methoddesc}{close}{}
Tim Peters5ebfd362001-11-13 23:11:19 +000093 Close the file. Subsequent calls to other methods of the object
94 will result in an exception being raised.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +000095\end{methoddesc}
96
Fred Drake8ff4cd72000-09-05 13:50:21 +000097\begin{methoddesc}{find}{string\optional{, start}}
Tim Peters5ebfd362001-11-13 23:11:19 +000098 Returns the lowest index in the object where the substring
99 \var{string} is found. Returns \code{-1} on failure. \var{start}
100 is the index at which the search begins, and defaults to zero.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000101\end{methoddesc}
102
Fred Drake8ff4cd72000-09-05 13:50:21 +0000103\begin{methoddesc}{flush}{\optional{offset, size}}
Tim Peters5ebfd362001-11-13 23:11:19 +0000104 Flushes changes made to the in-memory copy of a file back to disk.
105 Without use of this call there is no guarantee that changes are
106 written back before the object is destroyed. If \var{offset} and
107 \var{size} are specified, only changes to the given range of bytes
108 will be flushed to disk; otherwise, the whole extent of the mapping
109 is flushed.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000110\end{methoddesc}
111
112\begin{methoddesc}{move}{\var{dest}, \var{src}, \var{count}}
Tim Peters5ebfd362001-11-13 23:11:19 +0000113 Copy the \var{count} bytes starting at offset \var{src} to the
114 destination index \var{dest}. If the mmap was created with
115 \constant{ACCESS_READ}, then calls to move will throw a
116 \exception{TypeError} exception.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000117\end{methoddesc}
118
119\begin{methoddesc}{read}{\var{num}}
Tim Peters5ebfd362001-11-13 23:11:19 +0000120 Return a string containing up to \var{num} bytes starting from the
121 current file position; the file position is updated to point after the
122 bytes that were returned.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000123\end{methoddesc}
124
125\begin{methoddesc}{read_byte}{}
Tim Peters5ebfd362001-11-13 23:11:19 +0000126 Returns a string of length 1 containing the character at the current
127 file position, and advances the file position by 1.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000128\end{methoddesc}
129
130\begin{methoddesc}{readline}{}
Tim Peters0b4d1ee2004-06-06 16:51:46 +0000131 Returns a single line, starting at the current file position and up to
Tim Peters5ebfd362001-11-13 23:11:19 +0000132 the next newline.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000133\end{methoddesc}
134
135\begin{methoddesc}{resize}{\var{newsize}}
Georg Brandl38387b82005-08-24 07:17:40 +0000136 Resizes the map and the underlying file, if any.
Tim Peters5ebfd362001-11-13 23:11:19 +0000137 If the mmap was created with \constant{ACCESS_READ} or
138 \constant{ACCESS_COPY}, resizing the map will throw a \exception{TypeError} exception.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000139\end{methoddesc}
140
Fred Drake8ff4cd72000-09-05 13:50:21 +0000141\begin{methoddesc}{seek}{pos\optional{, whence}}
Tim Peters5ebfd362001-11-13 23:11:19 +0000142 Set the file's current position. \var{whence} argument is optional
143 and defaults to \code{0} (absolute file positioning); other values
144 are \code{1} (seek relative to the current position) and \code{2}
145 (seek relative to the file's end).
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000146\end{methoddesc}
147
148\begin{methoddesc}{size}{}
Tim Peters5ebfd362001-11-13 23:11:19 +0000149 Return the length of the file, which can be larger than the size of
150 the memory-mapped area.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000151\end{methoddesc}
152
153\begin{methoddesc}{tell}{}
Tim Peters5ebfd362001-11-13 23:11:19 +0000154 Returns the current position of the file pointer.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000155\end{methoddesc}
156
157\begin{methoddesc}{write}{\var{string}}
Tim Peters5ebfd362001-11-13 23:11:19 +0000158 Write the bytes in \var{string} into memory at the current position
159 of the file pointer; the file position is updated to point after the
160 bytes that were written. If the mmap was created with
161 \constant{ACCESS_READ}, then writing to it will throw a
162 \exception{TypeError} exception.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000163\end{methoddesc}
164
165\begin{methoddesc}{write_byte}{\var{byte}}
Tim Peters5ebfd362001-11-13 23:11:19 +0000166 Write the single-character string \var{byte} into memory at the
167 current position of the file pointer; the file position is advanced
Fredrik Lundh7c2ea7f2006-01-15 10:17:59 +0000168 by \code{1}. If the mmap was created with \constant{ACCESS_READ},
Tim Peters5ebfd362001-11-13 23:11:19 +0000169 then writing to it will throw a \exception{TypeError} exception.
Andrew M. Kuchlingb8050692000-06-17 22:39:05 +0000170\end{methoddesc}