Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 1 | \section{\module{mmap} --- |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 2 | Memory-mapped file support} |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 3 | |
| 4 | \declaremodule{builtin}{mmap} |
Fred Drake | c37b65e | 2001-11-28 07:26:15 +0000 | [diff] [blame] | 5 | \modulesynopsis{Interface to memory-mapped files for \UNIX\ and Windows.} |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 6 | |
Fred Drake | 1722e4a | 2001-12-03 18:27:22 +0000 | [diff] [blame] | 7 | Memory-mapped file objects behave like both strings and like |
| 8 | file objects. Unlike normal string objects, however, these are |
| 9 | mutable. You can use mmap objects in most places where strings |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 10 | are expected; for example, you can use the \module{re} module to |
| 11 | search through a memory-mapped file. Since they're mutable, you can |
Fred Drake | 8ff4cd7 | 2000-09-05 13:50:21 +0000 | [diff] [blame] | 12 | change a single character by doing \code{obj[\var{index}] = 'a'}, or |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 13 | change a substring by assigning to a slice: |
Fred Drake | 8ff4cd7 | 2000-09-05 13:50:21 +0000 | [diff] [blame] | 14 | \code{obj[\var{i1}:\var{i2}] = '...'}. You can also read and write |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 15 | data starting at the current file position, and \method{seek()} |
| 16 | through the file to different positions. |
| 17 | |
Fred Drake | 5055545 | 2001-09-25 19:00:08 +0000 | [diff] [blame] | 18 | A memory-mapped file is created by the \function{mmap()} function, |
| 19 | which is different on \UNIX{} and on Windows. In either case you must |
| 20 | provide a file descriptor for a file opened for update. |
| 21 | If you wish to map an existing Python file object, use its |
| 22 | \method{fileno()} method to obtain the correct value for the |
| 23 | \var{fileno} parameter. Otherwise, you can open the file using the |
| 24 | \function{os.open()} function, which returns a file descriptor |
| 25 | directly (the file still needs to be closed when done). |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 26 | |
Fred Drake | 1722e4a | 2001-12-03 18:27:22 +0000 | [diff] [blame] | 27 | For both the \UNIX{} and Windows versions of the function, |
| 28 | \var{access} may be specified as an optional keyword parameter. |
| 29 | \var{access} accepts one of three values: \constant{ACCESS_READ}, |
| 30 | \constant{ACCESS_WRITE}, or \constant{ACCESS_COPY} to specify |
| 31 | readonly, write-through or copy-on-write memory respectively. |
| 32 | \var{access} can be used on both \UNIX{} and Windows. If |
| 33 | \var{access} is not specified, Windows mmap returns a write-through |
| 34 | mapping. The initial memory values for all three access types are |
| 35 | taken from the specified file. Assignment to an |
| 36 | \constant{ACCESS_READ} memory map raises a \exception{TypeError} |
| 37 | exception. Assignment to an \constant{ACCESS_WRITE} memory map |
Fred Drake | b184ae8 | 2005-01-19 03:39:17 +0000 | [diff] [blame] | 38 | affects both memory and the underlying file. Assignment to an |
Fred Drake | 1722e4a | 2001-12-03 18:27:22 +0000 | [diff] [blame] | 39 | \constant{ACCESS_COPY} memory map affects memory but does not update |
Neal Norwitz | 0e6bc8c | 2006-02-05 05:45:43 +0000 | [diff] [blame] | 40 | the underlying file. \versionchanged[To map anonymous memory, |
| 41 | -1 should be passed as the fileno along with the length]{2.5} |
Fred Drake | 1722e4a | 2001-12-03 18:27:22 +0000 | [diff] [blame] | 42 | |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 43 | \begin{funcdesc}{mmap}{fileno, length\optional{, tagname\optional{, access}}} |
| 44 | \strong{(Windows version)} Maps \var{length} bytes from the file |
| 45 | specified by the file handle \var{fileno}, and returns a mmap |
Tim Peters | 0b4d1ee | 2004-06-06 16:51:46 +0000 | [diff] [blame] | 46 | object. If \var{length} is larger than the current size of the file, |
| 47 | the file is extended to contain \var{length} bytes. If \var{length} |
| 48 | is \code{0}, the maximum length of the map is the current size |
| 49 | of the file, except that if the file is empty Windows raises an |
| 50 | exception (you cannot create an empty mapping on Windows). |
| 51 | |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 52 | \var{tagname}, if specified and not \code{None}, is a string giving |
| 53 | a tag name for the mapping. Windows allows you to have many |
| 54 | different mappings against the same file. If you specify the name |
| 55 | of an existing tag, that tag is opened, otherwise a new tag of this |
| 56 | name is created. If this parameter is omitted or \code{None}, the |
| 57 | mapping is created without a name. Avoiding the use of the tag |
| 58 | parameter will assist in keeping your code portable between \UNIX{} |
| 59 | and Windows. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 60 | \end{funcdesc} |
| 61 | |
Fred Drake | 1722e4a | 2001-12-03 18:27:22 +0000 | [diff] [blame] | 62 | \begin{funcdescni}{mmap}{fileno, length\optional{, flags\optional{, |
| 63 | prot\optional{, access}}}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 64 | \strong{(\UNIX{} version)} Maps \var{length} bytes from the file |
| 65 | specified by the file descriptor \var{fileno}, and returns a mmap |
Martin v. Löwis | 7fe60c0 | 2005-03-03 11:22:44 +0000 | [diff] [blame] | 66 | object. If \var{length} is \code{0}, the maximum length of the map |
Georg Brandl | 296152e | 2006-01-23 21:33:03 +0000 | [diff] [blame] | 67 | will be the current size of the file when \function{mmap()} is |
Martin v. Löwis | 7fe60c0 | 2005-03-03 11:22:44 +0000 | [diff] [blame] | 68 | called. |
| 69 | |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 70 | \var{flags} specifies the nature of the mapping. |
| 71 | \constant{MAP_PRIVATE} creates a private copy-on-write mapping, so |
| 72 | changes to the contents of the mmap object will be private to this |
| 73 | process, and \constant{MAP_SHARED} creates a mapping that's shared |
| 74 | with all other processes mapping the same areas of the file. The |
| 75 | default value is \constant{MAP_SHARED}. |
Tim Peters | 0b4d1ee | 2004-06-06 16:51:46 +0000 | [diff] [blame] | 76 | |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 77 | \var{prot}, if specified, gives the desired memory protection; the |
| 78 | two most useful values are \constant{PROT_READ} and |
| 79 | \constant{PROT_WRITE}, to specify that the pages may be read or |
| 80 | written. \var{prot} defaults to \constant{PROT_READ | PROT_WRITE}. |
Tim Peters | 0b4d1ee | 2004-06-06 16:51:46 +0000 | [diff] [blame] | 81 | |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 82 | \var{access} may be specified in lieu of \var{flags} and \var{prot} |
Fred Drake | 1722e4a | 2001-12-03 18:27:22 +0000 | [diff] [blame] | 83 | as an optional keyword parameter. It is an error to specify both |
| 84 | \var{flags}, \var{prot} and \var{access}. See the description of |
| 85 | \var{access} above for information on how to use this parameter. |
| 86 | \end{funcdescni} |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 87 | |
Fred Drake | 5055545 | 2001-09-25 19:00:08 +0000 | [diff] [blame] | 88 | |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 89 | Memory-mapped file objects support the following methods: |
| 90 | |
| 91 | |
| 92 | \begin{methoddesc}{close}{} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 93 | Close the file. Subsequent calls to other methods of the object |
| 94 | will result in an exception being raised. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 95 | \end{methoddesc} |
| 96 | |
Fred Drake | 8ff4cd7 | 2000-09-05 13:50:21 +0000 | [diff] [blame] | 97 | \begin{methoddesc}{find}{string\optional{, start}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 98 | Returns the lowest index in the object where the substring |
| 99 | \var{string} is found. Returns \code{-1} on failure. \var{start} |
| 100 | is the index at which the search begins, and defaults to zero. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 101 | \end{methoddesc} |
| 102 | |
Fred Drake | 8ff4cd7 | 2000-09-05 13:50:21 +0000 | [diff] [blame] | 103 | \begin{methoddesc}{flush}{\optional{offset, size}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 104 | Flushes changes made to the in-memory copy of a file back to disk. |
| 105 | Without use of this call there is no guarantee that changes are |
| 106 | written back before the object is destroyed. If \var{offset} and |
| 107 | \var{size} are specified, only changes to the given range of bytes |
| 108 | will be flushed to disk; otherwise, the whole extent of the mapping |
| 109 | is flushed. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 110 | \end{methoddesc} |
| 111 | |
| 112 | \begin{methoddesc}{move}{\var{dest}, \var{src}, \var{count}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 113 | Copy the \var{count} bytes starting at offset \var{src} to the |
| 114 | destination index \var{dest}. If the mmap was created with |
| 115 | \constant{ACCESS_READ}, then calls to move will throw a |
| 116 | \exception{TypeError} exception. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 117 | \end{methoddesc} |
| 118 | |
| 119 | \begin{methoddesc}{read}{\var{num}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 120 | Return a string containing up to \var{num} bytes starting from the |
| 121 | current file position; the file position is updated to point after the |
| 122 | bytes that were returned. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 123 | \end{methoddesc} |
| 124 | |
| 125 | \begin{methoddesc}{read_byte}{} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 126 | Returns a string of length 1 containing the character at the current |
| 127 | file position, and advances the file position by 1. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 128 | \end{methoddesc} |
| 129 | |
| 130 | \begin{methoddesc}{readline}{} |
Tim Peters | 0b4d1ee | 2004-06-06 16:51:46 +0000 | [diff] [blame] | 131 | Returns a single line, starting at the current file position and up to |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 132 | the next newline. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 133 | \end{methoddesc} |
| 134 | |
| 135 | \begin{methoddesc}{resize}{\var{newsize}} |
Georg Brandl | 38387b8 | 2005-08-24 07:17:40 +0000 | [diff] [blame] | 136 | Resizes the map and the underlying file, if any. |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 137 | If the mmap was created with \constant{ACCESS_READ} or |
| 138 | \constant{ACCESS_COPY}, resizing the map will throw a \exception{TypeError} exception. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 139 | \end{methoddesc} |
| 140 | |
Fred Drake | 8ff4cd7 | 2000-09-05 13:50:21 +0000 | [diff] [blame] | 141 | \begin{methoddesc}{seek}{pos\optional{, whence}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 142 | Set the file's current position. \var{whence} argument is optional |
| 143 | and defaults to \code{0} (absolute file positioning); other values |
| 144 | are \code{1} (seek relative to the current position) and \code{2} |
| 145 | (seek relative to the file's end). |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 146 | \end{methoddesc} |
| 147 | |
| 148 | \begin{methoddesc}{size}{} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 149 | Return the length of the file, which can be larger than the size of |
| 150 | the memory-mapped area. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 151 | \end{methoddesc} |
| 152 | |
| 153 | \begin{methoddesc}{tell}{} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 154 | Returns the current position of the file pointer. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 155 | \end{methoddesc} |
| 156 | |
| 157 | \begin{methoddesc}{write}{\var{string}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 158 | Write the bytes in \var{string} into memory at the current position |
| 159 | of the file pointer; the file position is updated to point after the |
| 160 | bytes that were written. If the mmap was created with |
| 161 | \constant{ACCESS_READ}, then writing to it will throw a |
| 162 | \exception{TypeError} exception. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 163 | \end{methoddesc} |
| 164 | |
| 165 | \begin{methoddesc}{write_byte}{\var{byte}} |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 166 | Write the single-character string \var{byte} into memory at the |
| 167 | current position of the file pointer; the file position is advanced |
Fredrik Lundh | 7c2ea7f | 2006-01-15 10:17:59 +0000 | [diff] [blame] | 168 | by \code{1}. If the mmap was created with \constant{ACCESS_READ}, |
Tim Peters | 5ebfd36 | 2001-11-13 23:11:19 +0000 | [diff] [blame] | 169 | then writing to it will throw a \exception{TypeError} exception. |
Andrew M. Kuchling | b805069 | 2000-06-17 22:39:05 +0000 | [diff] [blame] | 170 | \end{methoddesc} |