Masood Malekghassemi | 7e660d7 | 2016-05-05 20:09:33 -0700 | [diff] [blame] | 1 | gRPC Python Tools |
| 2 | ================= |
| 3 | |
| 4 | Package for gRPC Python tools. |
| 5 | |
| 6 | Installation |
| 7 | ------------ |
| 8 | |
| 9 | The gRPC Python tools package is available for Linux, Mac OS X, and Windows |
| 10 | running Python 2.7. |
| 11 | |
| 12 | From PyPI |
| 13 | ~~~~~~~~~ |
| 14 | |
| 15 | If you are installing locally... |
| 16 | |
| 17 | :: |
| 18 | |
| 19 | $ pip install grpcio-tools |
| 20 | |
| 21 | Else system wide (on Ubuntu)... |
| 22 | |
| 23 | :: |
| 24 | |
| 25 | $ sudo pip install grpcio-tools |
| 26 | |
| 27 | If you're on Windows make sure that you installed the :code:`pip.exe` component |
| 28 | when you installed Python (if not go back and install it!) then invoke: |
| 29 | |
| 30 | :: |
| 31 | |
| 32 | $ pip.exe install grpcio-tools |
| 33 | |
| 34 | Windows users may need to invoke :code:`pip.exe` from a command line ran as |
| 35 | administrator. |
| 36 | |
| 37 | n.b. On Windows and on Mac OS X one *must* have a recent release of :code:`pip` |
| 38 | to retrieve the proper wheel from PyPI. Be sure to upgrade to the latest |
| 39 | version! |
| 40 | |
| 41 | You might also need to install Cython to handle installation via the source |
| 42 | distribution if gRPC Python's system coverage with wheels does not happen to |
| 43 | include your system. |
| 44 | |
| 45 | From Source |
| 46 | ~~~~~~~~~~~ |
| 47 | |
| 48 | Building from source requires that you have the Python headers (usually a |
Masood Malekghassemi | b4e82f7 | 2016-05-06 12:58:30 -0700 | [diff] [blame] | 49 | package named :code:`python-dev`) and Cython installed. It further requires a |
| 50 | GCC-like compiler to go smoothly; you can probably get it to work without |
| 51 | GCC-like stuff, but you may end up having a bad time. |
Masood Malekghassemi | 7e660d7 | 2016-05-05 20:09:33 -0700 | [diff] [blame] | 52 | |
| 53 | :: |
| 54 | |
| 55 | $ export REPO_ROOT=grpc # REPO_ROOT can be any directory of your choice |
Mehrdad Afshari | 976eb7c | 2017-07-10 21:54:56 +0000 | [diff] [blame] | 56 | $ git clone -b $(curl -L https://grpc.io/release) https://github.com/grpc/grpc $REPO_ROOT |
Masood Malekghassemi | 7e660d7 | 2016-05-05 20:09:33 -0700 | [diff] [blame] | 57 | $ cd $REPO_ROOT |
| 58 | $ git submodule update --init |
| 59 | |
| 60 | $ cd tools/distrib/python/grpcio_tools |
| 61 | $ python ../make_grpcio_tools.py |
| 62 | |
| 63 | # For the next command do `sudo pip install` if you get permission-denied errors |
Feng Li | d1ffa61 | 2018-01-26 11:55:14 -0800 | [diff] [blame] | 64 | $ GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install . |
Masood Malekghassemi | 7e660d7 | 2016-05-05 20:09:33 -0700 | [diff] [blame] | 65 | |
| 66 | You cannot currently install Python from source on Windows. Things might work |
| 67 | out for you in MSYS2 (follow the Linux instructions), but it isn't officially |
| 68 | supported at the moment. |
| 69 | |
| 70 | Troubleshooting |
| 71 | ~~~~~~~~~~~~~~~ |
| 72 | |
| 73 | Help, I ... |
| 74 | |
| 75 | * **... see a** :code:`pkg_resources.VersionConflict` **when I try to install |
| 76 | grpc** |
| 77 | |
| 78 | This is likely because :code:`pip` doesn't own the offending dependency, |
| 79 | which in turn is likely because your operating system's package manager owns |
| 80 | it. You'll need to force the installation of the dependency: |
| 81 | |
| 82 | :code:`pip install --ignore-installed $OFFENDING_DEPENDENCY` |
| 83 | |
| 84 | For example, if you get an error like the following: |
| 85 | |
| 86 | :: |
| 87 | |
| 88 | Traceback (most recent call last): |
| 89 | File "<string>", line 17, in <module> |
| 90 | ... |
| 91 | File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 509, in find |
| 92 | raise VersionConflict(dist, req) |
| 93 | pkg_resources.VersionConflict: (six 1.8.0 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10')) |
| 94 | |
| 95 | You can fix it by doing: |
| 96 | |
| 97 | :: |
| 98 | |
| 99 | sudo pip install --ignore-installed six |
| 100 | |
| 101 | * **... see compiler errors on some platforms when either installing from source or from the source distribution** |
| 102 | |
| 103 | If you see |
| 104 | |
| 105 | :: |
| 106 | |
| 107 | /tmp/pip-build-U8pSsr/cython/Cython/Plex/Scanners.c:4:20: fatal error: Python.h: No such file or directory |
| 108 | #include "Python.h" |
| 109 | ^ |
| 110 | compilation terminated. |
| 111 | |
| 112 | You can fix it by installing `python-dev` package. i.e |
| 113 | |
| 114 | :: |
| 115 | |
| 116 | sudo apt-get install python-dev |
| 117 | |
| 118 | If you see something similar to: |
| 119 | |
| 120 | :: |
| 121 | |
| 122 | third_party/protobuf/src/google/protobuf/stubs/mathlimits.h:173:31: note: in expansion of macro 'SIGNED_INT_MAX' |
| 123 | static const Type kPosMax = SIGNED_INT_MAX(Type); \\ |
| 124 | ^ |
Masood Malekghassemi | 165f9a7 | 2016-07-12 16:17:46 -0700 | [diff] [blame] | 125 | |
Masood Malekghassemi | 7e660d7 | 2016-05-05 20:09:33 -0700 | [diff] [blame] | 126 | And your toolchain is GCC (at the time of this writing, up through at least |
| 127 | GCC 6.0), this is probably a bug where GCC chokes on constant expressions |
| 128 | when the :code:`-fwrapv` flag is specified. You should consider setting your |
| 129 | environment with :code:`CFLAGS=-fno-wrapv` or using clang (:code:`CC=clang`). |
Masood Malekghassemi | 82c3dd1 | 2016-06-08 16:56:00 -0700 | [diff] [blame] | 130 | |
| 131 | Usage |
| 132 | ----- |
| 133 | |
| 134 | Given protobuf include directories :code:`$INCLUDE`, an output directory |
| 135 | :code:`$OUTPUT`, and proto files :code:`$PROTO_FILES`, invoke as: |
| 136 | |
| 137 | :: |
| 138 | |
| 139 | $ python -m grpc.tools.protoc -I$INCLUDE --python_out=$OUTPUT --grpc_python_out=$OUTPUT $PROTO_FILES |
Masood Malekghassemi | 34c9a73 | 2016-07-14 17:22:01 -0700 | [diff] [blame] | 140 | |
| 141 | To use as a build step in distutils-based projects, you may use the provided |
| 142 | command class in your :code:`setup.py`: |
| 143 | |
| 144 | :: |
| 145 | |
| 146 | setuptools.setup( |
| 147 | # ... |
| 148 | cmdclass={ |
| 149 | 'build_proto_modules': grpc.tools.command.BuildPackageProtos, |
| 150 | } |
| 151 | # ... |
| 152 | ) |
| 153 | |
| 154 | Invocation of the command will walk the project tree and transpile every |
| 155 | :code:`.proto` file into a :code:`_pb2.py` file in the same directory. |
| 156 | |
| 157 | Note that this particular approach requires :code:`grpcio-tools` to be |
| 158 | installed on the machine before the setup script is invoked (i.e. no |
| 159 | combination of :code:`setup_requires` or :code:`install_requires` will provide |
| 160 | access to :code:`grpc.tools.command.BuildPackageProtos` if it isn't already |
| 161 | installed). One way to work around this can be found in our |
| 162 | :code:`grpcio-health-checking` |
| 163 | `package <https://pypi.python.org/pypi/grpcio-health-checking>`_: |
| 164 | |
| 165 | :: |
| 166 | |
| 167 | class BuildPackageProtos(setuptools.Command): |
| 168 | """Command to generate project *_pb2.py modules from proto files.""" |
| 169 | # ... |
| 170 | def run(self): |
| 171 | from grpc.tools import command |
| 172 | command.build_package_protos(self.distribution.package_dir['']) |
| 173 | |
| 174 | Now including :code:`grpcio-tools` in :code:`setup_requires` will provide the |
| 175 | command on-setup as desired. |
| 176 | |
| 177 | For more information on command classes, consult :code:`distutils` and |
| 178 | :code:`setuptools` documentation. |