Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 1 | :mod:`venv` --- Creation of virtual environments |
| 2 | ================================================ |
| 3 | |
| 4 | .. module:: venv |
| 5 | :synopsis: Creation of virtual environments. |
Terry Jan Reedy | fa089b9 | 2016-06-11 15:02:54 -0400 | [diff] [blame] | 6 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 7 | .. moduleauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk> |
| 8 | .. sectionauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk> |
| 9 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 10 | .. versionadded:: 3.3 |
| 11 | |
Terry Jan Reedy | fa089b9 | 2016-06-11 15:02:54 -0400 | [diff] [blame] | 12 | **Source code:** :source:`Lib/venv/` |
| 13 | |
| 14 | .. index:: pair: Environments; virtual |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 15 | |
| 16 | -------------- |
| 17 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 18 | The :mod:`venv` module provides support for creating lightweight "virtual |
| 19 | environments" with their own site directories, optionally isolated from system |
| 20 | site directories. Each virtual environment has its own Python binary (allowing |
| 21 | creation of environments with various Python versions) and can have its own |
| 22 | independent set of installed Python packages in its site directories. |
| 23 | |
Vinay Sajip | a704582 | 2013-09-06 09:50:43 +0100 | [diff] [blame] | 24 | See :pep:`405` for more information about Python virtual environments. |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 25 | |
Brett Cannon | f8c1505 | 2016-10-21 12:53:40 -0700 | [diff] [blame] | 26 | .. note:: |
Zachary Ware | bb44477 | 2016-10-25 21:35:22 -0500 | [diff] [blame] | 27 | The ``pyvenv`` script has been deprecated as of Python 3.6 in favor of using |
Brett Cannon | f8c1505 | 2016-10-21 12:53:40 -0700 | [diff] [blame] | 28 | ``python3 -m venv`` to help prevent any potential confusion as to which |
| 29 | Python interpreter a virtual environment will be based on. |
| 30 | |
| 31 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 32 | Creating virtual environments |
| 33 | ----------------------------- |
| 34 | |
Vinay Sajip | c4618e3 | 2012-07-10 08:21:07 +0100 | [diff] [blame] | 35 | .. include:: /using/venv-create.inc |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 36 | |
Vinay Sajip | a945ad1 | 2012-07-09 09:24:59 +0100 | [diff] [blame] | 37 | |
Vinay Sajip | cd9b746 | 2012-07-09 10:37:01 +0100 | [diff] [blame] | 38 | .. _venv-def: |
| 39 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 40 | .. note:: A virtual environment is a Python environment such that the Python |
| 41 | interpreter, libraries and scripts installed into it are isolated from those |
| 42 | installed in other virtual environments, and (by default) any libraries |
| 43 | installed in a "system" Python, i.e., one which is installed as part of your |
| 44 | operating system. |
Vinay Sajip | a945ad1 | 2012-07-09 09:24:59 +0100 | [diff] [blame] | 45 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 46 | A virtual environment is a directory tree which contains Python executable |
| 47 | files and other files which indicate that it is a virtual environment. |
Vinay Sajip | a945ad1 | 2012-07-09 09:24:59 +0100 | [diff] [blame] | 48 | |
Jason R. Coombs | 13266fb | 2014-05-12 22:40:49 -0400 | [diff] [blame] | 49 | Common installation tools such as ``Setuptools`` and ``pip`` work as |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 50 | expected with virtual environments. In other words, when a virtual |
| 51 | environment is active, they install Python packages into the virtual |
| 52 | environment without needing to be told to do so explicitly. |
Vinay Sajip | a945ad1 | 2012-07-09 09:24:59 +0100 | [diff] [blame] | 53 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 54 | When a virtual environment is active (i.e., the virtual environment's Python |
| 55 | interpreter is running), the attributes :attr:`sys.prefix` and |
| 56 | :attr:`sys.exec_prefix` point to the base directory of the virtual |
| 57 | environment, whereas :attr:`sys.base_prefix` and |
| 58 | :attr:`sys.base_exec_prefix` point to the non-virtual environment Python |
| 59 | installation which was used to create the virtual environment. If a virtual |
| 60 | environment is not active, then :attr:`sys.prefix` is the same as |
| 61 | :attr:`sys.base_prefix` and :attr:`sys.exec_prefix` is the same as |
| 62 | :attr:`sys.base_exec_prefix` (they all point to a non-virtual environment |
| 63 | Python installation). |
Vinay Sajip | a945ad1 | 2012-07-09 09:24:59 +0100 | [diff] [blame] | 64 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 65 | When a virtual environment is active, any options that change the |
| 66 | installation path will be ignored from all distutils configuration files to |
| 67 | prevent projects being inadvertently installed outside of the virtual |
| 68 | environment. |
Georg Brandl | 521ed52 | 2013-05-12 12:36:07 +0200 | [diff] [blame] | 69 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 70 | When working in a command shell, users can make a virtual environment active |
| 71 | by running an ``activate`` script in the virtual environment's executables |
| 72 | directory (the precise filename is shell-dependent), which prepends the |
| 73 | virtual environment's directory for executables to the ``PATH`` environment |
| 74 | variable for the running shell. There should be no need in other |
| 75 | circumstances to activate a virtual environment—scripts installed into |
| 76 | virtual environments have a "shebang" line which points to the virtual |
| 77 | environment's Python interpreter. This means that the script will run with |
| 78 | that interpreter regardless of the value of ``PATH``. On Windows, "shebang" |
| 79 | line processing is supported if you have the Python Launcher for Windows |
| 80 | installed (this was added to Python in 3.3 - see :pep:`397` for more |
| 81 | details). Thus, double-clicking an installed script in a Windows Explorer |
| 82 | window should run the script with the correct interpreter without there |
| 83 | needing to be any reference to its virtual environment in ``PATH``. |
Vinay Sajip | a704582 | 2013-09-06 09:50:43 +0100 | [diff] [blame] | 84 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 85 | |
Larry Hastings | 3732ed2 | 2014-03-15 21:13:56 -0700 | [diff] [blame] | 86 | .. _venv-api: |
| 87 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 88 | API |
| 89 | --- |
| 90 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 91 | .. highlight:: python |
| 92 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 93 | The high-level method described above makes use of a simple API which provides |
| 94 | mechanisms for third-party virtual environment creators to customize environment |
| 95 | creation according to their needs, the :class:`EnvBuilder` class. |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 96 | |
Nick Coghlan | 8fbdb09 | 2013-11-23 00:30:34 +1000 | [diff] [blame] | 97 | .. class:: EnvBuilder(system_site_packages=False, clear=False, \ |
Vinay Sajip | fd0f84b | 2016-08-06 10:43:44 +0100 | [diff] [blame] | 98 | symlinks=False, upgrade=False, with_pip=False, \ |
| 99 | prompt=None) |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 100 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 101 | The :class:`EnvBuilder` class accepts the following keyword arguments on |
| 102 | instantiation: |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 103 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 104 | * ``system_site_packages`` -- a Boolean value indicating that the system Python |
| 105 | site-packages should be available to the environment (defaults to ``False``). |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 106 | |
Serhiy Storchaka | 0e90e99 | 2013-11-29 12:19:53 +0200 | [diff] [blame] | 107 | * ``clear`` -- a Boolean value which, if true, will delete the contents of |
Vinay Sajip | bd40d3e | 2012-10-11 17:22:45 +0100 | [diff] [blame] | 108 | any existing target directory, before creating the environment. |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 109 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 110 | * ``symlinks`` -- a Boolean value indicating whether to attempt to symlink the |
| 111 | Python binary (and any necessary DLLs or other binaries, |
| 112 | e.g. ``pythonw.exe``), rather than copying. Defaults to ``True`` on Linux and |
Vinay Sajip | 90db661 | 2012-07-17 17:33:46 +0100 | [diff] [blame] | 113 | Unix systems, but ``False`` on Windows. |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 114 | |
Serhiy Storchaka | fbc1c26 | 2013-11-29 12:17:13 +0200 | [diff] [blame] | 115 | * ``upgrade`` -- a Boolean value which, if true, will upgrade an existing |
Vinay Sajip | a945ad1 | 2012-07-09 09:24:59 +0100 | [diff] [blame] | 116 | environment with the running Python - for use when that Python has been |
| 117 | upgraded in-place (defaults to ``False``). |
| 118 | |
Serhiy Storchaka | 0e90e99 | 2013-11-29 12:19:53 +0200 | [diff] [blame] | 119 | * ``with_pip`` -- a Boolean value which, if true, ensures pip is |
Nick Coghlan | aa029da | 2014-02-09 10:10:24 +1000 | [diff] [blame] | 120 | installed in the virtual environment. This uses :mod:`ensurepip` with |
| 121 | the ``--default-pip`` option. |
Nick Coghlan | 8fbdb09 | 2013-11-23 00:30:34 +1000 | [diff] [blame] | 122 | |
Vinay Sajip | fd0f84b | 2016-08-06 10:43:44 +0100 | [diff] [blame] | 123 | * ``prompt`` -- a String to be used after virtual environment is activated |
| 124 | (defaults to ``None`` which means directory name of the environment would |
| 125 | be used). |
| 126 | |
Nick Coghlan | 8fbdb09 | 2013-11-23 00:30:34 +1000 | [diff] [blame] | 127 | .. versionchanged:: 3.4 |
| 128 | Added the ``with_pip`` parameter |
| 129 | |
Vinay Sajip | fd0f84b | 2016-08-06 10:43:44 +0100 | [diff] [blame] | 130 | .. versionadded:: 3.6 |
| 131 | Added the ``prompt`` parameter |
| 132 | |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 133 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 134 | Creators of third-party virtual environment tools will be free to use the |
| 135 | provided ``EnvBuilder`` class as a base class. |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 136 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 137 | The returned env-builder is an object which has a method, ``create``: |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 138 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 139 | .. method:: create(env_dir) |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 140 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 141 | This method takes as required argument the path (absolute or relative to |
| 142 | the current directory) of the target directory which is to contain the |
| 143 | virtual environment. The ``create`` method will either create the |
| 144 | environment in the specified directory, or raise an appropriate |
| 145 | exception. |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 146 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 147 | The ``create`` method of the ``EnvBuilder`` class illustrates the hooks |
| 148 | available for subclass customization:: |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 149 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 150 | def create(self, env_dir): |
| 151 | """ |
| 152 | Create a virtualized Python environment in a directory. |
| 153 | env_dir is the target directory to create an environment in. |
| 154 | """ |
| 155 | env_dir = os.path.abspath(env_dir) |
Georg Brandl | a0b7923 | 2013-10-06 12:52:49 +0200 | [diff] [blame] | 156 | context = self.ensure_directories(env_dir) |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 157 | self.create_configuration(context) |
| 158 | self.setup_python(context) |
| 159 | self.setup_scripts(context) |
| 160 | self.post_setup(context) |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 161 | |
Georg Brandl | a0b7923 | 2013-10-06 12:52:49 +0200 | [diff] [blame] | 162 | Each of the methods :meth:`ensure_directories`, |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 163 | :meth:`create_configuration`, :meth:`setup_python`, |
| 164 | :meth:`setup_scripts` and :meth:`post_setup` can be overridden. |
| 165 | |
Vinay Sajip | 577d4ff | 2013-07-12 21:52:51 +0100 | [diff] [blame] | 166 | .. method:: ensure_directories(env_dir) |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 167 | |
| 168 | Creates the environment directory and all necessary directories, and |
| 169 | returns a context object. This is just a holder for attributes (such as |
Vinay Sajip | 577d4ff | 2013-07-12 21:52:51 +0100 | [diff] [blame] | 170 | paths), for use by the other methods. The directories are allowed to |
| 171 | exist already, as long as either ``clear`` or ``upgrade`` were |
| 172 | specified to allow operating on an existing environment directory. |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 173 | |
| 174 | .. method:: create_configuration(context) |
| 175 | |
| 176 | Creates the ``pyvenv.cfg`` configuration file in the environment. |
| 177 | |
| 178 | .. method:: setup_python(context) |
| 179 | |
| 180 | Creates a copy of the Python executable (and, under Windows, DLLs) in |
Vinay Sajip | 577d4ff | 2013-07-12 21:52:51 +0100 | [diff] [blame] | 181 | the environment. On a POSIX system, if a specific executable |
| 182 | ``python3.x`` was used, symlinks to ``python`` and ``python3`` will be |
| 183 | created pointing to that executable, unless files with those names |
| 184 | already exist. |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 185 | |
| 186 | .. method:: setup_scripts(context) |
| 187 | |
| 188 | Installs activation scripts appropriate to the platform into the virtual |
| 189 | environment. |
| 190 | |
| 191 | .. method:: post_setup(context) |
| 192 | |
| 193 | A placeholder method which can be overridden in third party |
| 194 | implementations to pre-install packages in the virtual environment or |
| 195 | perform other post-creation steps. |
| 196 | |
| 197 | In addition, :class:`EnvBuilder` provides this utility method that can be |
| 198 | called from :meth:`setup_scripts` or :meth:`post_setup` in subclasses to |
| 199 | assist in installing custom scripts into the virtual environment. |
| 200 | |
| 201 | .. method:: install_scripts(context, path) |
| 202 | |
| 203 | *path* is the path to a directory that should contain subdirectories |
| 204 | "common", "posix", "nt", each containing scripts destined for the bin |
| 205 | directory in the environment. The contents of "common" and the |
| 206 | directory corresponding to :data:`os.name` are copied after some text |
| 207 | replacement of placeholders: |
| 208 | |
| 209 | * ``__VENV_DIR__`` is replaced with the absolute path of the environment |
| 210 | directory. |
| 211 | |
| 212 | * ``__VENV_NAME__`` is replaced with the environment name (final path |
| 213 | segment of environment directory). |
| 214 | |
Vinay Sajip | dff9e25 | 2013-10-02 11:36:16 +0100 | [diff] [blame] | 215 | * ``__VENV_PROMPT__`` is replaced with the prompt (the environment |
| 216 | name surrounded by parentheses and with a following space) |
| 217 | |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 218 | * ``__VENV_BIN_NAME__`` is replaced with the name of the bin directory |
| 219 | (either ``bin`` or ``Scripts``). |
| 220 | |
| 221 | * ``__VENV_PYTHON__`` is replaced with the absolute path of the |
| 222 | environment's executable. |
| 223 | |
Vinay Sajip | 577d4ff | 2013-07-12 21:52:51 +0100 | [diff] [blame] | 224 | The directories are allowed to exist (for when an existing environment |
| 225 | is being upgraded). |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 226 | |
| 227 | There is also a module-level convenience function: |
| 228 | |
Nick Coghlan | 8fbdb09 | 2013-11-23 00:30:34 +1000 | [diff] [blame] | 229 | .. function:: create(env_dir, system_site_packages=False, clear=False, \ |
| 230 | symlinks=False, with_pip=False) |
Georg Brandl | dbab58f | 2012-06-24 16:37:59 +0200 | [diff] [blame] | 231 | |
| 232 | Create an :class:`EnvBuilder` with the given keyword arguments, and call its |
| 233 | :meth:`~EnvBuilder.create` method with the *env_dir* argument. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 234 | |
Nick Coghlan | 8fbdb09 | 2013-11-23 00:30:34 +1000 | [diff] [blame] | 235 | .. versionchanged:: 3.4 |
| 236 | Added the ``with_pip`` parameter |
| 237 | |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 238 | An example of extending ``EnvBuilder`` |
| 239 | -------------------------------------- |
| 240 | |
| 241 | The following script shows how to extend :class:`EnvBuilder` by implementing a |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 242 | subclass which installs setuptools and pip into a created virtual environment:: |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 243 | |
| 244 | import os |
| 245 | import os.path |
| 246 | from subprocess import Popen, PIPE |
| 247 | import sys |
| 248 | from threading import Thread |
| 249 | from urllib.parse import urlparse |
| 250 | from urllib.request import urlretrieve |
| 251 | import venv |
| 252 | |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 253 | class ExtendedEnvBuilder(venv.EnvBuilder): |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 254 | """ |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 255 | This builder installs setuptools and pip so that you can pip or |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 256 | easy_install other packages into the created virtual environment. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 257 | |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 258 | :param nodist: If True, setuptools and pip are not installed into the |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 259 | created virtual environment. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 260 | :param nopip: If True, pip is not installed into the created |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 261 | virtual environment. |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 262 | :param progress: If setuptools or pip are installed, the progress of the |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 263 | installation can be monitored by passing a progress |
| 264 | callable. If specified, it is called with two |
| 265 | arguments: a string indicating some progress, and a |
| 266 | context indicating where the string is coming from. |
| 267 | The context argument can have one of three values: |
| 268 | 'main', indicating that it is called from virtualize() |
| 269 | itself, and 'stdout' and 'stderr', which are obtained |
| 270 | by reading lines from the output streams of a subprocess |
| 271 | which is used to install the app. |
| 272 | |
| 273 | If a callable is not specified, default progress |
| 274 | information is output to sys.stderr. |
| 275 | """ |
| 276 | |
| 277 | def __init__(self, *args, **kwargs): |
| 278 | self.nodist = kwargs.pop('nodist', False) |
| 279 | self.nopip = kwargs.pop('nopip', False) |
| 280 | self.progress = kwargs.pop('progress', None) |
| 281 | self.verbose = kwargs.pop('verbose', False) |
| 282 | super().__init__(*args, **kwargs) |
| 283 | |
| 284 | def post_setup(self, context): |
| 285 | """ |
| 286 | Set up any packages which need to be pre-installed into the |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 287 | virtual environment being created. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 288 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 289 | :param context: The information for the virtual environment |
| 290 | creation request being processed. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 291 | """ |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 292 | os.environ['VIRTUAL_ENV'] = context.env_dir |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 293 | if not self.nodist: |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 294 | self.install_setuptools(context) |
| 295 | # Can't install pip without setuptools |
| 296 | if not self.nopip and not self.nodist: |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 297 | self.install_pip(context) |
| 298 | |
| 299 | def reader(self, stream, context): |
| 300 | """ |
| 301 | Read lines from a subprocess' output stream and either pass to a progress |
| 302 | callable (if specified) or write progress information to sys.stderr. |
| 303 | """ |
| 304 | progress = self.progress |
| 305 | while True: |
| 306 | s = stream.readline() |
| 307 | if not s: |
| 308 | break |
| 309 | if progress is not None: |
| 310 | progress(s, context) |
| 311 | else: |
| 312 | if not self.verbose: |
| 313 | sys.stderr.write('.') |
| 314 | else: |
| 315 | sys.stderr.write(s.decode('utf-8')) |
| 316 | sys.stderr.flush() |
Vinay Sajip | ad6bb03 | 2013-07-12 21:44:35 +0100 | [diff] [blame] | 317 | stream.close() |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 318 | |
| 319 | def install_script(self, context, name, url): |
| 320 | _, _, path, _, _, _ = urlparse(url) |
| 321 | fn = os.path.split(path)[-1] |
| 322 | binpath = context.bin_path |
| 323 | distpath = os.path.join(binpath, fn) |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 324 | # Download script into the virtual environment's binaries folder |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 325 | urlretrieve(url, distpath) |
| 326 | progress = self.progress |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 327 | if self.verbose: |
| 328 | term = '\n' |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 329 | else: |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 330 | term = '' |
| 331 | if progress is not None: |
| 332 | progress('Installing %s ...%s' % (name, term), 'main') |
| 333 | else: |
| 334 | sys.stderr.write('Installing %s ...%s' % (name, term)) |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 335 | sys.stderr.flush() |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 336 | # Install in the virtual environment |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 337 | args = [context.env_exe, fn] |
| 338 | p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath) |
| 339 | t1 = Thread(target=self.reader, args=(p.stdout, 'stdout')) |
| 340 | t1.start() |
| 341 | t2 = Thread(target=self.reader, args=(p.stderr, 'stderr')) |
| 342 | t2.start() |
| 343 | p.wait() |
| 344 | t1.join() |
| 345 | t2.join() |
| 346 | if progress is not None: |
| 347 | progress('done.', 'main') |
| 348 | else: |
| 349 | sys.stderr.write('done.\n') |
| 350 | # Clean up - no longer needed |
| 351 | os.unlink(distpath) |
| 352 | |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 353 | def install_setuptools(self, context): |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 354 | """ |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 355 | Install setuptools in the virtual environment. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 356 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 357 | :param context: The information for the virtual environment |
| 358 | creation request being processed. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 359 | """ |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 360 | url = 'https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py' |
| 361 | self.install_script(context, 'setuptools', url) |
| 362 | # clear up the setuptools archive which gets downloaded |
| 363 | pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz') |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 364 | files = filter(pred, os.listdir(context.bin_path)) |
| 365 | for f in files: |
| 366 | f = os.path.join(context.bin_path, f) |
| 367 | os.unlink(f) |
| 368 | |
| 369 | def install_pip(self, context): |
| 370 | """ |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 371 | Install pip in the virtual environment. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 372 | |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 373 | :param context: The information for the virtual environment |
| 374 | creation request being processed. |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 375 | """ |
| 376 | url = 'https://raw.github.com/pypa/pip/master/contrib/get-pip.py' |
| 377 | self.install_script(context, 'pip', url) |
| 378 | |
| 379 | def main(args=None): |
| 380 | compatible = True |
| 381 | if sys.version_info < (3, 3): |
| 382 | compatible = False |
| 383 | elif not hasattr(sys, 'base_prefix'): |
| 384 | compatible = False |
| 385 | if not compatible: |
| 386 | raise ValueError('This script is only for use with ' |
| 387 | 'Python 3.3 or later') |
| 388 | else: |
| 389 | import argparse |
| 390 | |
| 391 | parser = argparse.ArgumentParser(prog=__name__, |
| 392 | description='Creates virtual Python ' |
| 393 | 'environments in one or ' |
| 394 | 'more target ' |
| 395 | 'directories.') |
| 396 | parser.add_argument('dirs', metavar='ENV_DIR', nargs='+', |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 397 | help='A directory in which to create the |
| 398 | 'virtual environment.') |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 399 | parser.add_argument('--no-setuptools', default=False, |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 400 | action='store_true', dest='nodist', |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 401 | help="Don't install setuptools or pip in the " |
| 402 | "virtual environment.") |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 403 | parser.add_argument('--no-pip', default=False, |
| 404 | action='store_true', dest='nopip', |
| 405 | help="Don't install pip in the virtual " |
| 406 | "environment.") |
| 407 | parser.add_argument('--system-site-packages', default=False, |
| 408 | action='store_true', dest='system_site', |
| 409 | help='Give the virtual environment access to the ' |
| 410 | 'system site-packages dir.') |
| 411 | if os.name == 'nt': |
| 412 | use_symlinks = False |
| 413 | else: |
| 414 | use_symlinks = True |
| 415 | parser.add_argument('--symlinks', default=use_symlinks, |
| 416 | action='store_true', dest='symlinks', |
| 417 | help='Try to use symlinks rather than copies, ' |
| 418 | 'when symlinks are not the default for ' |
| 419 | 'the platform.') |
| 420 | parser.add_argument('--clear', default=False, action='store_true', |
| 421 | dest='clear', help='Delete the contents of the ' |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 422 | 'virtual environment ' |
| 423 | 'directory if it already ' |
| 424 | 'exists, before virtual ' |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 425 | 'environment creation.') |
| 426 | parser.add_argument('--upgrade', default=False, action='store_true', |
Brett Cannon | 15552c3 | 2016-07-08 10:46:21 -0700 | [diff] [blame] | 427 | dest='upgrade', help='Upgrade the virtual ' |
| 428 | 'environment directory to ' |
| 429 | 'use this version of ' |
| 430 | 'Python, assuming Python ' |
| 431 | 'has been upgraded ' |
| 432 | 'in-place.') |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 433 | parser.add_argument('--verbose', default=False, action='store_true', |
| 434 | dest='verbose', help='Display the output ' |
| 435 | 'from the scripts which ' |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 436 | 'install setuptools and pip.') |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 437 | options = parser.parse_args(args) |
| 438 | if options.upgrade and options.clear: |
| 439 | raise ValueError('you cannot supply --upgrade and --clear together.') |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 440 | builder = ExtendedEnvBuilder(system_site_packages=options.system_site, |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 441 | clear=options.clear, |
| 442 | symlinks=options.symlinks, |
| 443 | upgrade=options.upgrade, |
| 444 | nodist=options.nodist, |
| 445 | nopip=options.nopip, |
| 446 | verbose=options.verbose) |
| 447 | for d in options.dirs: |
| 448 | builder.create(d) |
| 449 | |
| 450 | if __name__ == '__main__': |
| 451 | rc = 1 |
| 452 | try: |
| 453 | main() |
| 454 | rc = 0 |
| 455 | except Exception as e: |
| 456 | print('Error: %s' % e, file=sys.stderr) |
| 457 | sys.exit(rc) |
| 458 | |
Vinay Sajip | 3c557f2 | 2013-07-12 20:54:25 +0100 | [diff] [blame] | 459 | |
Vinay Sajip | 2b4fcfb | 2013-01-30 13:44:00 +0000 | [diff] [blame] | 460 | This script is also available for download `online |
Sanyam Khurana | 338cd83 | 2018-01-20 05:55:37 +0530 | [diff] [blame] | 461 | <https://gist.github.com/vsajip/4673395>`_. |