Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 1 | .. _module-pw_build: |
Alexei Frolov | 9d169d5 | 2020-03-03 17:20:06 -0800 | [diff] [blame] | 2 | |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 3 | -------- |
| 4 | pw_build |
| 5 | -------- |
Alexei Frolov | 9d169d5 | 2020-03-03 17:20:06 -0800 | [diff] [blame] | 6 | Pigweed's modules aim to be easily integratable into both new and existing |
| 7 | embedded projects. To that goal, the ``pw_build`` module provides support for |
| 8 | multiple build systems. Our personal favorite is `GN`_/`Ninja`_, which is used |
| 9 | by upstream developers for its speed and flexibility. `CMake`_ and `Bazel`_ |
| 10 | build files are also provided by all modules, allowing Pigweed to be added to a |
| 11 | project with minimal effort. |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 12 | |
| 13 | .. _GN: https://gn.googlesource.com/gn/ |
| 14 | .. _Ninja: https://ninja-build.org/ |
Alexei Frolov | 9d169d5 | 2020-03-03 17:20:06 -0800 | [diff] [blame] | 15 | .. _CMake: https://cmake.org/ |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 16 | .. _Bazel: https://bazel.build/ |
| 17 | |
Alexei Frolov | 9d169d5 | 2020-03-03 17:20:06 -0800 | [diff] [blame] | 18 | Beyond just compiling code, Pigweed’s GN build system can also: |
| 19 | |
| 20 | * Generate HTML documentation, via our Sphinx integration (with ``pw_docgen``) |
Armando Montanez | 0054a9b | 2020-03-13 13:06:24 -0700 | [diff] [blame] | 21 | * Display memory usage report cards (with ``pw_bloat``) |
Alexei Frolov | 9d169d5 | 2020-03-03 17:20:06 -0800 | [diff] [blame] | 22 | * Incrementally run unit tests after code changes (with ``pw_target_runner``) |
| 23 | * And more! |
| 24 | |
| 25 | These are only supported in the GN build, so we recommend using it if possible. |
| 26 | |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 27 | GN / Ninja |
| 28 | ========== |
Armando Montanez | bcc194b | 2020-03-10 10:23:18 -0700 | [diff] [blame] | 29 | The GN / Ninja build system is the primary build system used for upstream |
| 30 | Pigweed development, and is the most tested and feature-rich build system |
| 31 | Pigweed offers. |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 32 | |
Armando Montanez | bcc194b | 2020-03-10 10:23:18 -0700 | [diff] [blame] | 33 | This module's ``build.gn`` file contains a number of C/C++ ``config`` |
| 34 | declarations that are used by upstream Pigweed to set some architecture-agnostic |
| 35 | compiler defaults. (See Pigweed's ``//BUILDCONFIG.gn``) |
| 36 | |
Armando Montanez | 0054a9b | 2020-03-13 13:06:24 -0700 | [diff] [blame] | 37 | ``pw_build`` also provides several useful GN templates that are used throughout |
Armando Montanez | bcc194b | 2020-03-10 10:23:18 -0700 | [diff] [blame] | 38 | Pigweed. |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 39 | |
Alexei Frolov | edd2f14 | 2020-06-09 19:11:27 -0700 | [diff] [blame] | 40 | Target types |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 41 | ------------ |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 42 | .. code-block:: |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 43 | |
Alexei Frolov | edd2f14 | 2020-06-09 19:11:27 -0700 | [diff] [blame] | 44 | import("$dir_pw_build/target_types.gni") |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 45 | |
Alexei Frolov | edd2f14 | 2020-06-09 19:11:27 -0700 | [diff] [blame] | 46 | pw_source_set("my_library") { |
| 47 | sources = [ "lib.cc" ] |
| 48 | } |
| 49 | |
| 50 | Pigweed defines wrappers around the four basic GN binary types ``source_set``, |
| 51 | ``executable``, ``static_library``, and ``shared_library``. These wrappers apply |
| 52 | default arguments to each target as specified in the ``default_configs`` and |
| 53 | ``default_public_deps`` build args. Additionally, they allow defaults to be |
| 54 | removed on a per-target basis using ``remove_configs`` and |
| 55 | ``remove_public_deps`` variables, respectively. |
| 56 | |
| 57 | The ``pw_executable`` template provides additional functionality around building |
| 58 | complete binaries. As Pigweed is a collection of libraries, it does not know how |
| 59 | its final targets are built. ``pw_executable`` solves this by letting each user |
| 60 | of Pigweed specify a global executable template for their target, and have |
| 61 | Pigweed build against it. This is controlled by the build variable |
| 62 | ``pw_executable_config.target_type``, specifying the name of the executable |
| 63 | template for a project. |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 64 | |
Armando Montanez | 256fb50 | 2021-06-28 10:47:49 -0700 | [diff] [blame] | 65 | In some uncommon cases, a project's ``pw_executable`` template definition may |
| 66 | need to stamp out some ``pw_source_set``s. Since a pw_executable template can't |
| 67 | import ``$dir_pw_build/target_types.gni`` due to circular imports, it should |
| 68 | import ``$dir_pw_build/cc_library.gni`` instead. |
| 69 | |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 70 | .. tip:: |
| 71 | |
| 72 | Prefer to use ``pw_executable`` over plain ``executable`` targets to allow |
| 73 | cleanly building the same code for multiple target configs. |
| 74 | |
| 75 | **Arguments** |
| 76 | |
Alexei Frolov | edd2f14 | 2020-06-09 19:11:27 -0700 | [diff] [blame] | 77 | All of the ``pw_*`` target type overrides accept any arguments, as they simply |
| 78 | forward them through to the underlying target. |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 79 | |
Wyatt Hepler | 8bd4fb0 | 2021-05-03 15:30:58 -0700 | [diff] [blame] | 80 | .. _module-pw_build-link-deps: |
| 81 | |
| 82 | Link-only deps |
| 83 | -------------- |
| 84 | It may be necessary to specify additional link-time dependencies that may not be |
| 85 | explicitly depended on elsewhere in the build. One example of this is a |
| 86 | ``pw_assert`` backend, which may need to leave out dependencies to avoid |
| 87 | circular dependencies. Its dependencies need to be linked for executables and |
| 88 | libraries, even if they aren't pulled in elsewhere. |
| 89 | |
| 90 | The ``pw_build_LINK_DEPS`` build arg is a list of dependencies to add to all |
| 91 | ``pw_executable``, ``pw_static_library``, and ``pw_shared_library`` targets. |
| 92 | This should only be used as a last resort when dependencies cannot be properly |
| 93 | expressed in the build. |
| 94 | |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 95 | Python packages |
| 96 | --------------- |
| 97 | GN templates for :ref:`Python build automation <docs-python-build>` are |
| 98 | described in :ref:`module-pw_build-python`. |
| 99 | |
| 100 | .. toctree:: |
| 101 | :hidden: |
| 102 | |
| 103 | python |
| 104 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 105 | .. _module-pw_build-facade: |
| 106 | |
| 107 | pw_facade |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 108 | --------- |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 109 | In their simplest form, a :ref:`facade<docs-module-structure-facades>` is a GN |
| 110 | build arg used to change a dependency at compile time. Pigweed targets configure |
| 111 | these facades as needed. |
| 112 | |
| 113 | The ``pw_facade`` template bundles a ``pw_source_set`` with a facade build arg. |
| 114 | This allows the facade to provide header files, compilation options or anything |
| 115 | else a GN ``source_set`` provides. |
| 116 | |
| 117 | The ``pw_facade`` template declares two targets: |
| 118 | |
| 119 | * ``$target_name``: the public-facing ``pw_source_set``, with a ``public_dep`` |
| 120 | on the backend |
| 121 | * ``$target_name.facade``: target used by the backend to avoid circular |
| 122 | dependencies |
| 123 | |
| 124 | .. code-block:: |
| 125 | |
| 126 | # Declares ":foo" and ":foo.facade" GN targets |
| 127 | pw_facade("foo") { |
| 128 | backend = pw_log_BACKEND |
| 129 | public_configs = [ ":public_include_path" ] |
| 130 | public = [ "public/pw_foo/foo.h" ] |
| 131 | } |
| 132 | |
Wyatt Hepler | 6166322 | 2021-05-06 10:57:43 -0700 | [diff] [blame] | 133 | Low-level facades like ``pw_assert`` cannot express all of their dependencies |
| 134 | due to the potential for dependency cycles. Facades with this issue may require |
| 135 | backends to place their implementations in a separate build target to be listed |
| 136 | in ``pw_build_LINK_DEPS`` (see :ref:`module-pw_build-link-deps`). The |
| 137 | ``require_link_deps`` variable in ``pw_facade`` asserts that all specified build |
| 138 | targets are present in ``pw_build_LINK_DEPS`` if the facade's backend variable |
| 139 | is set. |
| 140 | |
Wyatt Hepler | 0d32d1d | 2020-10-23 08:05:23 -0700 | [diff] [blame] | 141 | .. _module-pw_build-python-action: |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 142 | |
Wyatt Hepler | c8e05a4 | 2020-10-19 14:49:39 -0700 | [diff] [blame] | 143 | pw_python_action |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 144 | ---------------- |
Wyatt Hepler | c8e05a4 | 2020-10-19 14:49:39 -0700 | [diff] [blame] | 145 | The ``pw_python_action`` template is a convenience wrapper around ``action`` for |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 146 | running Python scripts. The main benefit it provides is resolution of GN target |
| 147 | labels to compiled binary files. This allows Python scripts to be written |
| 148 | independently of GN, taking only filesystem paths as arguments. |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 149 | |
| 150 | Another convenience provided by the template is to allow running scripts without |
| 151 | any outputs. Sometimes scripts run in a build do not directly produce output |
Wyatt Hepler | c8e05a4 | 2020-10-19 14:49:39 -0700 | [diff] [blame] | 152 | files, but GN requires that all actions have an output. ``pw_python_action`` |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 153 | solves this by accepting a boolean ``stamp`` argument which tells it to create a |
Rob Mohr | 463700d | 2021-05-22 10:31:03 -0700 | [diff] [blame] | 154 | placeholder output file for the action. |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 155 | |
| 156 | **Arguments** |
| 157 | |
Wyatt Hepler | c8e05a4 | 2020-10-19 14:49:39 -0700 | [diff] [blame] | 158 | ``pw_python_action`` accepts all of the arguments of a regular ``action`` |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 159 | target. Additionally, it has some of its own arguments: |
| 160 | |
Wyatt Hepler | 79d983f | 2020-10-12 08:46:34 -0700 | [diff] [blame] | 161 | * ``module``: Run the specified Python module instead of a script. Either |
| 162 | ``script`` or ``module`` must be specified, but not both. |
Wyatt Hepler | a74f7b0 | 2020-07-23 14:10:56 -0700 | [diff] [blame] | 163 | * ``capture_output``: Optional boolean. If true, script output is hidden unless |
| 164 | the script fails with an error. Defaults to true. |
| 165 | * ``stamp``: Optional variable indicating whether to automatically create a |
Rob Mohr | 463700d | 2021-05-22 10:31:03 -0700 | [diff] [blame] | 166 | placeholder output file for the script. This allows running scripts without |
Wyatt Hepler | a74f7b0 | 2020-07-23 14:10:56 -0700 | [diff] [blame] | 167 | specifying ``outputs``. If ``stamp`` is true, a generic output file is |
| 168 | used. If ``stamp`` is a file path, that file is used as a stamp file. Like any |
| 169 | output file, ``stamp`` must be in the build directory. Defaults to false. |
Wyatt Hepler | 79d983f | 2020-10-12 08:46:34 -0700 | [diff] [blame] | 170 | * ``environment``: Optional list of strings. Environment variables to set, |
| 171 | passed as NAME=VALUE strings. |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 172 | |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 173 | **Expressions** |
| 174 | |
Wyatt Hepler | c8e05a4 | 2020-10-19 14:49:39 -0700 | [diff] [blame] | 175 | ``pw_python_action`` evaluates expressions in ``args``, the arguments passed to |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 176 | the script. These expressions function similarly to generator expressions in |
| 177 | CMake. Expressions may be passed as a standalone argument or as part of another |
| 178 | argument. A single argument may contain multiple expressions. |
| 179 | |
| 180 | Generally, these expressions are used within templates rather than directly in |
| 181 | BUILD.gn files. This allows build code to use GN labels without having to worry |
| 182 | about converting them to files. |
| 183 | |
Wyatt Hepler | a2ddc41 | 2021-03-16 13:47:29 -0700 | [diff] [blame] | 184 | .. note:: |
| 185 | |
| 186 | We intend to replace these expressions with native GN features when possible. |
| 187 | See `pwbug/347 <http://bugs.pigweed.dev/347>`_. |
| 188 | |
Wyatt Hepler | 53a06fb | 2020-07-31 13:04:56 -0700 | [diff] [blame] | 189 | The following expressions are supported: |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 190 | |
| 191 | .. describe:: <TARGET_FILE(gn_target)> |
| 192 | |
| 193 | Evaluates to the output file of the provided GN target. For example, the |
| 194 | expression |
| 195 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 196 | .. code-block:: |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 197 | |
| 198 | "<TARGET_FILE(//foo/bar:static_lib)>" |
| 199 | |
| 200 | might expand to |
| 201 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 202 | .. code-block:: |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 203 | |
| 204 | "/home/User/project_root/out/obj/foo/bar/static_lib.a" |
| 205 | |
| 206 | ``TARGET_FILE`` parses the ``.ninja`` file for the GN target, so it should |
| 207 | always find the correct output file, regardless of the toolchain's or target's |
| 208 | configuration. Some targets, such as ``source_set`` and ``group`` targets, do |
| 209 | not have an output file, and attempting to use ``TARGET_FILE`` with them |
| 210 | results in an error. |
| 211 | |
| 212 | ``TARGET_FILE`` only resolves GN target labels to their outputs. To resolve |
| 213 | paths generally, use the standard GN approach of applying the |
Michael Spang | c8b9390 | 2021-05-30 15:53:56 -0400 | [diff] [blame] | 214 | ``rebase_path(path, root_build_dir)`` function. This function |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 215 | converts the provided GN path or list of paths to be relative to the build |
| 216 | directory, from which all build commands and scripts are executed. |
| 217 | |
Wyatt Hepler | 53a06fb | 2020-07-31 13:04:56 -0700 | [diff] [blame] | 218 | .. describe:: <TARGET_FILE_IF_EXISTS(gn_target)> |
| 219 | |
| 220 | ``TARGET_FILE_IF_EXISTS`` evaluates to the output file of the provided GN |
| 221 | target, if the output file exists. If the output file does not exist, the |
| 222 | entire argument that includes this expression is omitted, even if there is |
| 223 | other text or another expression. |
| 224 | |
| 225 | For example, consider this expression: |
| 226 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 227 | .. code-block:: |
Wyatt Hepler | 53a06fb | 2020-07-31 13:04:56 -0700 | [diff] [blame] | 228 | |
| 229 | "--database=<TARGET_FILE_IF_EXISTS(//alpha/bravo)>" |
| 230 | |
| 231 | If the ``//alpha/bravo`` target file exists, this might expand to the |
| 232 | following: |
| 233 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 234 | .. code-block:: |
Wyatt Hepler | 53a06fb | 2020-07-31 13:04:56 -0700 | [diff] [blame] | 235 | |
| 236 | "--database=/home/User/project/out/obj/alpha/bravo/bravo.elf" |
| 237 | |
| 238 | If the ``//alpha/bravo`` target file does not exist, the entire |
| 239 | ``--database=`` argument is omitted from the script arguments. |
| 240 | |
| 241 | .. describe:: <TARGET_OBJECTS(gn_target)> |
| 242 | |
| 243 | Evaluates to the object files of the provided GN target. Expands to a separate |
| 244 | argument for each object file. If the target has no object files, the argument |
| 245 | is omitted entirely. Because it does not expand to a single expression, the |
| 246 | ``<TARGET_OBJECTS(...)>`` expression may not have leading or trailing text. |
| 247 | |
| 248 | For example, the expression |
| 249 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 250 | .. code-block:: |
Wyatt Hepler | 53a06fb | 2020-07-31 13:04:56 -0700 | [diff] [blame] | 251 | |
| 252 | "<TARGET_OBJECTS(//foo/bar:a_source_set)>" |
| 253 | |
| 254 | might expand to multiple separate arguments: |
| 255 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 256 | .. code-block:: |
Wyatt Hepler | 53a06fb | 2020-07-31 13:04:56 -0700 | [diff] [blame] | 257 | |
| 258 | "/home/User/project_root/out/obj/foo/bar/a_source_set.file_a.cc.o" |
| 259 | "/home/User/project_root/out/obj/foo/bar/a_source_set.file_b.cc.o" |
| 260 | "/home/User/project_root/out/obj/foo/bar/a_source_set.file_c.cc.o" |
| 261 | |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 262 | **Example** |
| 263 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 264 | .. code-block:: |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 265 | |
Wyatt Hepler | 51ded74 | 2020-10-19 14:45:27 -0700 | [diff] [blame] | 266 | import("$dir_pw_build/python_action.gni") |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 267 | |
Wyatt Hepler | c8e05a4 | 2020-10-19 14:49:39 -0700 | [diff] [blame] | 268 | pw_python_action("postprocess_main_image") { |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 269 | script = "py/postprocess_binary.py" |
| 270 | args = [ |
| 271 | "--database", |
Michael Spang | c8b9390 | 2021-05-30 15:53:56 -0400 | [diff] [blame] | 272 | rebase_path("my/database.csv", root_build_dir), |
Wyatt Hepler | 8224a64 | 2020-07-29 08:55:56 -0700 | [diff] [blame] | 273 | "--binary=<TARGET_FILE(//firmware/images:main)>", |
| 274 | ] |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 275 | stamp = true |
| 276 | } |
| 277 | |
| 278 | pw_input_group |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 279 | -------------- |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 280 | ``pw_input_group`` defines a group of input files which are not directly |
| 281 | processed by the build but are still important dependencies of later build |
| 282 | steps. This is commonly used alongside metadata to propagate file dependencies |
| 283 | through the build graph and force rebuilds on file modifications. |
| 284 | |
| 285 | For example ``pw_docgen`` defines a ``pw_doc_group`` template which outputs |
| 286 | metadata from a list of input files. The metadata file is not actually part of |
| 287 | the build, and so changes to any of the input files do not trigger a rebuild. |
| 288 | This is problematic, as targets that depend on the metadata should rebuild when |
| 289 | the inputs are modified but GN cannot express this dependency. |
| 290 | |
| 291 | ``pw_input_group`` solves this problem by allowing a list of files to be listed |
| 292 | in a target that does not output any build artifacts, causing all dependent |
| 293 | targets to correctly rebuild. |
| 294 | |
| 295 | **Arguments** |
| 296 | |
| 297 | ``pw_input_group`` accepts all arguments that can be passed to a ``group`` |
| 298 | target, as well as requiring one extra: |
| 299 | |
| 300 | * ``inputs``: List of input files. |
| 301 | |
| 302 | **Example** |
| 303 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 304 | .. code-block:: |
Alexei Frolov | 69ad192 | 2019-12-13 13:11:32 -0800 | [diff] [blame] | 305 | |
| 306 | import("$dir_pw_build/input_group.gni") |
| 307 | |
| 308 | pw_input_group("foo_metadata") { |
| 309 | metadata = { |
| 310 | files = [ |
| 311 | "x.foo", |
| 312 | "y.foo", |
| 313 | "z.foo", |
| 314 | ] |
| 315 | } |
| 316 | inputs = metadata.files |
| 317 | } |
| 318 | |
| 319 | Targets depending on ``foo_metadata`` will rebuild when any of the ``.foo`` |
| 320 | files are modified. |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 321 | |
Sam McCauley | 0da412c | 2020-10-02 10:34:56 -0700 | [diff] [blame] | 322 | pw_zip |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 323 | ------ |
Sam McCauley | 0da412c | 2020-10-02 10:34:56 -0700 | [diff] [blame] | 324 | ``pw_zip`` is a target that allows users to zip up a set of input files and |
| 325 | directories into a single output ``.zip`` file—a simple automation of a |
| 326 | potentially repetitive task. |
| 327 | |
| 328 | **Arguments** |
| 329 | |
| 330 | * ``inputs``: List of source files as well as the desired relative zip |
| 331 | destination. See below for the input syntax. |
| 332 | * ``dirs``: List of entire directories to be zipped as well as the desired |
| 333 | relative zip destination. See below for the input syntax. |
| 334 | * ``output``: Filename of output ``.zip`` file. |
| 335 | * ``deps``: List of dependencies for the target. |
| 336 | |
| 337 | **Input Syntax** |
| 338 | |
| 339 | Inputs all need to follow the correct syntax: |
| 340 | |
| 341 | #. Path to source file or directory. Directories must end with a ``/``. |
| 342 | #. The delimiter (defaults to ``>``). |
| 343 | #. The desired destination of the contents within the ``.zip``. Must start |
| 344 | with ``/`` to indicate the zip root. Any number of subdirectories are |
| 345 | allowed. If the source is a file it can be put into any subdirectory of the |
| 346 | root. If the source is a file, the zip copy can also be renamed by ending |
| 347 | the zip destination with a filename (no trailing ``/``). |
| 348 | |
| 349 | Thus, it should look like the following: ``"[source file or dir] > /"``. |
| 350 | |
| 351 | **Example** |
| 352 | |
| 353 | Let's say we have the following structure for a ``//source/`` directory: |
| 354 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 355 | .. code-block:: |
Sam McCauley | 0da412c | 2020-10-02 10:34:56 -0700 | [diff] [blame] | 356 | |
| 357 | source/ |
| 358 | ├── file1.txt |
| 359 | ├── file2.txt |
| 360 | ├── file3.txt |
| 361 | └── some_dir/ |
| 362 | ├── file4.txt |
| 363 | └── some_other_dir/ |
| 364 | └── file5.txt |
| 365 | |
| 366 | And we create the following build target: |
| 367 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 368 | .. code-block:: |
Sam McCauley | 0da412c | 2020-10-02 10:34:56 -0700 | [diff] [blame] | 369 | |
| 370 | import("$dir_pw_build/zip.gni") |
| 371 | |
| 372 | pw_zip("target_name") { |
| 373 | inputs = [ |
| 374 | "//source/file1.txt > /", # Copied to the zip root dir. |
| 375 | "//source/file2.txt > /renamed.txt", # File renamed. |
| 376 | "//source/file3.txt > /bar/", # File moved to the /bar/ dir. |
| 377 | ] |
| 378 | |
| 379 | dirs = [ |
| 380 | "//source/some_dir/ > /bar/some_dir/", # All /some_dir/ contents copied |
| 381 | # as /bar/some_dir/. |
| 382 | ] |
| 383 | |
| 384 | # Note on output: if the specific output directory isn't defined |
| 385 | # (such as output = "zoo.zip") then the .zip will output to the |
| 386 | # same directory as the BUILD.gn file that called the target. |
| 387 | output = "//$target_out_dir/foo.zip" # Where the foo.zip will end up |
| 388 | } |
| 389 | |
| 390 | This will result in a ``.zip`` file called ``foo.zip`` stored in |
| 391 | ``//$target_out_dir`` with the following structure: |
| 392 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 393 | .. code-block:: |
Sam McCauley | 0da412c | 2020-10-02 10:34:56 -0700 | [diff] [blame] | 394 | |
| 395 | foo.zip |
| 396 | ├── bar/ |
| 397 | │  ├── file3.txt |
| 398 | │  └── some_dir/ |
| 399 | │  ├── file4.txt |
| 400 | │  └── some_other_dir/ |
| 401 | │  └── file5.txt |
| 402 | ├── file1.txt |
| 403 | └── renamed.txt |
| 404 | |
Wyatt Hepler | b59d2c9 | 2021-05-11 11:20:43 -0700 | [diff] [blame] | 405 | CMake |
| 406 | ===== |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 407 | Pigweed's `CMake`_ support is provided primarily for projects that have an |
| 408 | existing CMake build and wish to integrate Pigweed without switching to a new |
| 409 | build system. |
Armando Montanez | bcc194b | 2020-03-10 10:23:18 -0700 | [diff] [blame] | 410 | |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 411 | The following command generates Ninja build files for a host build in the |
| 412 | ``out/cmake_host`` directory: |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 413 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 414 | .. code-block:: sh |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 415 | |
Wyatt Hepler | dcfa92b | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 416 | cmake -B out/cmake_host -S "$PW_ROOT" -G Ninja -DCMAKE_TOOLCHAIN_FILE=$PW_ROOT/pw_toolchain/host_clang/toolchain.cmake |
| 417 | |
| 418 | The ``PW_ROOT`` environment variable must point to the root of the Pigweed |
| 419 | directory. This variable is set by Pigweed's environment setup. |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 420 | |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 421 | Tests can be executed with the ``pw_run_tests.GROUP`` targets. To run Pigweed |
| 422 | module tests, execute ``pw_run_tests.modules``: |
| 423 | |
| 424 | .. code-block:: sh |
| 425 | |
| 426 | ninja -C out/cmake_host pw_run_tests.modules |
| 427 | |
| 428 | :ref:`module-pw_watch` supports CMake, so you can also run |
| 429 | |
| 430 | .. code-block:: sh |
| 431 | |
Wyatt Hepler | 00efe18 | 2020-11-23 08:25:14 -0800 | [diff] [blame] | 432 | pw watch -C out/cmake_host pw_run_tests.modules |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 433 | |
| 434 | CMake functions |
| 435 | --------------- |
| 436 | CMake convenience functions are defined in ``pw_build/pigweed.cmake``. |
| 437 | |
| 438 | * ``pw_auto_add_simple_module`` -- For modules with only one library, |
| 439 | automatically declare the library and its tests. |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 440 | * ``pw_auto_add_module_tests`` -- Create test targets for all tests in a module. |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 441 | * ``pw_add_facade`` -- Declare a module facade. |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 442 | * ``pw_set_backend`` -- Set the backend library to use for a facade. |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 443 | * ``pw_add_module_library`` -- Add a library that is part of a module. |
| 444 | * ``pw_add_test`` -- Declare a test target. |
| 445 | |
| 446 | See ``pw_build/pigweed.cmake`` for the complete documentation of these |
| 447 | functions. |
| 448 | |
| 449 | Special libraries that do not fit well with these functions are created with the |
| 450 | standard CMake functions, such as ``add_library`` and ``target_link_libraries``. |
| 451 | |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 452 | Facades and backends |
| 453 | -------------------- |
| 454 | The CMake build uses CMake cache variables for configuring |
| 455 | :ref:`facades<docs-module-structure-facades>` and backends. Cache variables are |
| 456 | similar to GN's build args set with ``gn args``. Unlike GN, CMake does not |
| 457 | support multi-toolchain builds, so these variables have a single global value |
| 458 | per build directory. |
| 459 | |
| 460 | The ``pw_add_facade`` function declares a cache variable named |
| 461 | ``<module_name>_BACKEND`` for each facade. Cache variables can be awkward to |
| 462 | work with, since their values only change when they're assigned, but then |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 463 | persist accross CMake invocations. These variables should be set in one of the |
| 464 | following ways: |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 465 | |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 466 | * Call ``pw_set_backend`` to set backends appropriate for the target in the |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 467 | target's toolchain file. The toolchain file is provided to ``cmake`` with |
| 468 | ``-DCMAKE_TOOLCHAIN_FILE=<toolchain file>``. |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 469 | * Call ``pw_set_backend`` in the top-level ``CMakeLists.txt`` before other |
| 470 | CMake code executes. |
| 471 | * Set the backend variable at the command line with the ``-D`` option. |
| 472 | |
| 473 | .. code-block:: sh |
| 474 | |
| 475 | cmake -B out/cmake_host -S "$PW_ROOT" -G Ninja \ |
| 476 | -DCMAKE_TOOLCHAIN_FILE=$PW_ROOT/pw_toolchain/host_clang/toolchain.cmake \ |
| 477 | -Dpw_log_BACKEND=pw_log_basic |
| 478 | |
| 479 | * Temporarily override a backend by setting it interactively with ``ccmake`` or |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 480 | ``cmake-gui``. |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 481 | |
Wyatt Hepler | 8026d8d | 2021-05-11 11:37:29 -0700 | [diff] [blame] | 482 | If the backend is set to a build target that does not exist, there will be an |
| 483 | error message like the following: |
| 484 | |
| 485 | .. code-block:: |
| 486 | |
| 487 | CMake Error at pw_build/pigweed.cmake:244 (add_custom_target): |
| 488 | Error evaluating generator expression: |
| 489 | |
| 490 | $<TARGET_PROPERTY:my_backend_that_does_not_exist,TYPE> |
| 491 | |
| 492 | Target "my_backend_that_does_not_exist" not found. |
| 493 | |
Wyatt Hepler | 73439f7 | 2020-11-03 14:28:11 -0800 | [diff] [blame] | 494 | Toolchain setup |
| 495 | --------------- |
| 496 | In CMake, the toolchain is configured by setting CMake variables, as described |
| 497 | in the `CMake documentation <https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html>`_. |
| 498 | These variables are typically set in a toolchain CMake file passed to ``cmake`` |
| 499 | with the ``-D`` option (``-DCMAKE_TOOLCHAIN_FILE=path/to/file.cmake``). |
| 500 | For Pigweed embedded builds, set ``CMAKE_SYSTEM_NAME`` to the empty string |
| 501 | (``""``). |
| 502 | |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 503 | Third party libraries |
| 504 | --------------------- |
| 505 | The CMake build includes third-party libraries similarly to the GN build. A |
| 506 | ``dir_pw_third_party_<library>`` cache variable is defined for each third-party |
Wyatt Hepler | 8f357f4 | 2021-03-17 15:37:09 -0700 | [diff] [blame] | 507 | dependency. The variable must be set to the absolute path of the library in |
| 508 | order to use it. If the variable is empty |
| 509 | (``if("${dir_pw_third_party_<library>}" STREQUAL "")``), the dependency is not |
| 510 | available. |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 511 | |
Wyatt Hepler | 8f357f4 | 2021-03-17 15:37:09 -0700 | [diff] [blame] | 512 | Third-party dependencies are not automatically added to the build. They can be |
| 513 | manually added with ``add_subdirectory`` or by setting the |
| 514 | ``pw_third_party_<library>_ADD_SUBDIRECTORY`` option to ``ON``. |
Wyatt Hepler | c9e51d2 | 2020-10-29 09:12:37 -0700 | [diff] [blame] | 515 | |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 516 | Third party variables are set like any other cache global variable in CMake. It |
| 517 | is recommended to set these in one of the following ways: |
Wyatt Hepler | dcfa92b | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 518 | |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 519 | * Set with the CMake ``set`` function in the toolchain file or a |
| 520 | ``CMakeLists.txt`` before other CMake code executes. |
Wyatt Hepler | dcfa92b | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 521 | |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 522 | .. code-block:: cmake |
Wyatt Hepler | dcfa92b | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 523 | |
Wyatt Hepler | 8f357f4 | 2021-03-17 15:37:09 -0700 | [diff] [blame] | 524 | set(dir_pw_third_party_nanopb ${CMAKE_CURRENT_SOURCE_DIR}/external/nanopb CACHE PATH "" FORCE) |
Wyatt Hepler | d9336a4 | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 525 | |
| 526 | * Set the variable at the command line with the ``-D`` option. |
| 527 | |
| 528 | .. code-block:: sh |
| 529 | |
| 530 | cmake -B out/cmake_host -S "$PW_ROOT" -G Ninja \ |
| 531 | -DCMAKE_TOOLCHAIN_FILE=$PW_ROOT/pw_toolchain/host_clang/toolchain.cmake \ |
| 532 | -Ddir_pw_third_party_nanopb=/path/to/nanopb |
| 533 | |
| 534 | * Set the variable interactively with ``ccmake`` or ``cmake-gui``. |
Wyatt Hepler | dcfa92b | 2020-11-10 09:47:30 -0800 | [diff] [blame] | 535 | |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 536 | Use Pigweed from an existing CMake project |
| 537 | ------------------------------------------ |
| 538 | To use Pigweed libraries form a CMake-based project, simply include the Pigweed |
| 539 | repository from a ``CMakeLists.txt``. |
| 540 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 541 | .. code-block:: cmake |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 542 | |
| 543 | add_subdirectory(path/to/pigweed pigweed) |
| 544 | |
| 545 | All module libraries will be available as ``module_name`` or |
| 546 | ``module_name.sublibrary``. |
| 547 | |
| 548 | If desired, modules can be included individually. |
| 549 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 550 | .. code-block:: cmake |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 551 | |
Wyatt Hepler | 0fbcdfc | 2020-01-02 07:53:39 -0800 | [diff] [blame] | 552 | add_subdirectory(path/to/pigweed/pw_some_module pw_some_module) |
| 553 | add_subdirectory(path/to/pigweed/pw_another_module pw_another_module) |
| 554 | |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 555 | Bazel |
| 556 | ===== |
Nathaniel Brough | 82cf687 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 557 | Bazel is currently very experimental, and only builds for host and ARM Cortex-M |
| 558 | microcontrollers. |
Armando Montanez | bcc194b | 2020-03-10 10:23:18 -0700 | [diff] [blame] | 559 | |
Rob Mohr | 84f234e | 2019-12-06 09:16:50 -0800 | [diff] [blame] | 560 | The common configuration for Bazel for all modules is in the ``pigweed.bzl`` |
| 561 | file. The built-in Bazel rules ``cc_binary``, ``cc_library``, and ``cc_test`` |
| 562 | are wrapped with ``pw_cc_binary``, ``pw_cc_library``, and ``pw_cc_test``. |
| 563 | These wrappers add parameters to calls to the compiler and linker. |
| 564 | |
Nathaniel Brough | 82cf687 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 565 | Currently Pigweed is making use of a set of |
| 566 | [open source](https://github.com/silvergasp/bazel-embedded) toolchains. The host |
| 567 | builds are only supported on Linux/Mac based systems. Additionally the host |
| 568 | builds are not entirely hermetic, and will make use of system |
| 569 | libraries and headers. This is close to the default configuration for Bazel, |
| 570 | though slightly more hermetic. The host toolchain is based around clang-11 which |
| 571 | has a system dependency on 'libtinfo.so.5' which is often included as part of |
| 572 | the libncurses packages. On Debian based systems this can be installed using the |
| 573 | command below: |
| 574 | |
| 575 | .. code-block:: sh |
| 576 | |
| 577 | sudo apt install libncurses5 |
| 578 | |
| 579 | The host toolchain does not currently support native Windows, though using WSL |
| 580 | is a viable alternative. |
| 581 | |
| 582 | The ARM Cortex-M Bazel toolchains are based around gcc-arm-non-eabi and are |
| 583 | entirely hermetic. You can target Cortex-M, by using the platforms command line |
| 584 | option. This set of toolchains is supported from hosts; Windows, Mac and Linux. |
Nathaniel Brough | ce91098 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 585 | The platforms that are currently supported are listed below: |
Nathaniel Brough | 82cf687 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 586 | |
| 587 | .. code-block:: sh |
| 588 | |
Nathaniel Brough | ce91098 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 589 | bazel build //:your_target --platforms=@pigweed//pw_build/platforms:cortex_m0 |
| 590 | bazel build //:your_target --platforms=@pigweed//pw_build/platforms:cortex_m1 |
| 591 | bazel build //:your_target --platforms=@pigweed//pw_build/platforms:cortex_m3 |
| 592 | bazel build //:your_target --platforms=@pigweed//pw_build/platforms:cortex_m4 |
| 593 | bazel build //:your_target --platforms=@pigweed//pw_build/platforms:cortex_m7 |
Nathaniel Brough | 82cf687 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 594 | bazel build //:your_target \ |
Nathaniel Brough | ce91098 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 595 | --platforms=@pigweed//pw_build/platforms:cortex_m4_fpu |
Nathaniel Brough | 82cf687 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 596 | bazel build //:your_target \ |
Nathaniel Brough | ce91098 | 2021-02-16 15:51:57 +0800 | [diff] [blame] | 597 | --platforms=@pigweed//pw_build/platforms:cortex_m7_fpu |
| 598 | |
| 599 | |
| 600 | The above examples are cpu/fpu oriented platforms and can be used where |
| 601 | applicable for your application. There some more specific platforms for the |
| 602 | types of boards that are included as examples in Pigweed. It is strongly |
| 603 | encouraged that you create your own set of platforms specific for your project, |
| 604 | that implement the constraint_settings in this repository. e.g. |
| 605 | |
| 606 | New board constraint_value: |
| 607 | |
| 608 | .. code-block:: python |
| 609 | |
| 610 | #your_repo/build_settings/constraints/board/BUILD |
| 611 | constraint_value( |
| 612 | name = "nucleo_l432kc", |
| 613 | constraint_setting = "@pigweed//pw_build/constraints/board", |
| 614 | ) |
| 615 | |
| 616 | New chipset constraint_value: |
| 617 | |
| 618 | .. code-block:: python |
| 619 | |
| 620 | # your_repo/build_settings/constraints/chipset/BUILD |
| 621 | constraint_value( |
| 622 | name = "stm32l432kc", |
| 623 | constraint_setting = "@pigweed//pw_build/constraints/chipset", |
| 624 | ) |
| 625 | |
| 626 | New platforms for chipset and board: |
| 627 | |
| 628 | .. code-block:: python |
| 629 | |
| 630 | #your_repo/build_settings/platforms/BUILD |
| 631 | # Works with all stm32l432kc |
| 632 | platforms( |
| 633 | name = "stm32l432kc", |
| 634 | parents = ["@pigweed//pw_build/platforms:cortex_m4"], |
| 635 | constraint_values = |
| 636 | ["@your_repo//build_settings/constraints/chipset:stm32l432kc"], |
| 637 | ) |
| 638 | |
| 639 | # Works with only the nucleo_l432kc |
| 640 | platforms( |
| 641 | name = "nucleo_l432kc", |
| 642 | parents = [":stm32l432kc"], |
| 643 | constraint_values = |
| 644 | ["@your_repo//build_settings/constraints/board:nucleo_l432kc"], |
| 645 | ) |
| 646 | |
| 647 | In the above example you can build your code with the command line: |
| 648 | |
| 649 | .. code-block:: python |
| 650 | |
| 651 | bazel build //:your_target_for_nucleo_l432kc \ |
| 652 | --platforms=@your_repo//build_settings:nucleo_l432kc |
| 653 | |
| 654 | |
| 655 | You can also specify that a specific target is only compatible with one |
| 656 | platform: |
| 657 | |
| 658 | .. code-block:: python |
| 659 | |
| 660 | cc_library( |
| 661 | name = "compatible_with_all_stm32l432kc", |
| 662 | srcs = ["tomato_src.c"], |
| 663 | target_compatible_with = |
| 664 | ["@your_repo//build_settings/constraints/chipset:stm32l432kc"], |
| 665 | ) |
| 666 | |
| 667 | cc_library( |
| 668 | name = "compatible_with_only_nucleo_l432kc", |
| 669 | srcs = ["bbq_src.c"], |
| 670 | target_compatible_with = |
| 671 | ["@your_repo//build_settings/constraints/board:nucleo_l432kc"], |
| 672 | ) |
| 673 | |