Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 1 | .. _docs-build-system: |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 2 | |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 3 | ============ |
| 4 | Build system |
| 5 | ============ |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 6 | Building software for embedded devices is a complex process. Projects often have |
| 7 | custom toolchains, target different hardware platforms, and require additional |
| 8 | configuration and post-processing of artifacts. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 9 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 10 | As a modern embedded framework, Pigweed's goal is to collect these embedded use |
| 11 | cases into a powerful and flexible build system, then extend it with support for |
| 12 | modern software development practices. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 13 | |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 14 | See :ref:`docs-python-build` for information about Python build automation with |
| 15 | Pigweed. |
| 16 | |
| 17 | .. toctree:: |
| 18 | :hidden: |
| 19 | |
| 20 | python_build |
| 21 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 22 | What's in a build system? |
| 23 | ========================= |
| 24 | A quality build system provides a variety of features beyond compiling code. |
| 25 | Throughout our experience with embedded development, we've found several build |
| 26 | features to be especially useful, and designed Pigweed's build system with them |
| 27 | in mind. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 28 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 29 | Simple toolchain configuration |
| 30 | ------------------------------ |
| 31 | Embedded projects often use custom build toolchains for their specific hardware. |
| 32 | Configuring these should be a simple process, both in their initial setup and |
| 33 | later adjustments. |
| 34 | |
| 35 | Multi-target builds |
| 36 | ------------------- |
| 37 | Virtually every consumer product has firmware that targets different boards or |
| 38 | MCUs during development. While building for a single board is simple enough, the |
| 39 | complexity of supporting different targets ranges from changing compiler flags |
| 40 | to swapping out entire libraries of firmware and drivers. This is often done by |
| 41 | running multiple builds, configuring each one accordingly. In Pigweed, we've |
| 42 | designed our build system with first-class multi-target support in mind, |
| 43 | allowing any number of target configurations to be built simultaneously. |
| 44 | |
| 45 | Multi-language support |
| 46 | ---------------------- |
| 47 | Embedded projects are typically written in C, C++, and assembly. However, it is |
| 48 | possible to have firmware written in other languages, such as Rust. |
| 49 | Additionally, projects may have host-side tooling written in a wide variety of |
| 50 | languages. Having all of these build together proves to be a large time saver. |
| 51 | |
| 52 | Custom scripting |
| 53 | ---------------- |
| 54 | Embedded projects often require post-processing of build artifacts; these may |
| 55 | include: |
| 56 | |
| 57 | * Extracting ELF sections into a different container |
| 58 | * Injecting metadata into firmware images |
| 59 | * Image signing |
| 60 | * Creating databases of symbols for debugging |
| 61 | * Extracting string tokens into a database (for example, with |
Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 62 | :ref:`module-pw_tokenizer`) |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 63 | |
| 64 | These are run as steps during a build, facilitated by the build system. |
| 65 | |
| 66 | See also |
| 67 | ^^^^^^^^ |
| 68 | |
Wyatt Hepler | 0d32d1d | 2020-10-23 08:05:23 -0700 | [diff] [blame] | 69 | * :ref:`module-pw_build-python-action` |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 70 | |
Wyatt Hepler | b3ea980 | 2021-02-23 09:46:09 -0800 | [diff] [blame] | 71 | Python |
| 72 | ------ |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 73 | Python is a favorite scripting language of many development teams, and here at |
| 74 | Pigweed, we're no exception. Much of Pigweed's host-side tooling is written in |
| 75 | Python. While Python works great for local development, problems can arise when |
| 76 | scripts need to be packaged and distributed for vendors or factory teams. Having |
| 77 | proper support for packaging Python within a build system allows teams to focus |
| 78 | on writing code instead of worrying about distribution. |
| 79 | |
| 80 | Size reporting |
| 81 | -------------- |
| 82 | On embedded devices, memory is everything. Most projects have some sort of |
| 83 | custom tooling to determine how much flash and RAM space their firmware uses. |
| 84 | Being able to run size reports as part of a build ensures that they are always |
| 85 | up-to-date and allows space usage to be tracked over time. |
| 86 | |
| 87 | See also |
| 88 | ^^^^^^^^ |
| 89 | |
Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 90 | * :ref:`module-pw_bloat` |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 91 | |
| 92 | Documentation |
| 93 | ------------- |
| 94 | An oft-neglected part of software development, documentation is invaluable for |
| 95 | future maintainers of a project. As such, Pigweed has integrated documentation |
| 96 | which builds alongside its code and combines with other build features, such as |
| 97 | size reports, to provide high quality, up-to-date references for developers. |
| 98 | |
| 99 | See also |
| 100 | ^^^^^^^^ |
| 101 | |
Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 102 | * :ref:`module-pw_docgen` |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 103 | |
| 104 | Unit testing |
| 105 | ------------ |
| 106 | Unit tests are essential to ensure that the functionality of code remains |
| 107 | consistent as changes are made to avoid accidental regressions. Running unit |
| 108 | tests as part of a build keeps developers constantly aware of the impact of |
| 109 | their changes. |
| 110 | |
| 111 | Host-side unit tests |
| 112 | ^^^^^^^^^^^^^^^^^^^^ |
| 113 | Though Pigweed targets embedded devices, a lot of its code can be run and tested |
| 114 | on a host desktop by swapping out backends to host platform libraries. This is |
| 115 | highly beneficial during development, as it allows tests to consistently run |
| 116 | without having to go through the process of flashing a device. |
| 117 | |
| 118 | Device-side unit tests |
| 119 | ^^^^^^^^^^^^^^^^^^^^^^ |
| 120 | As useful as host-side tests are, they are not sufficient for developing actual |
| 121 | firmware, and it is critical to run tests on the actual hardware. Pigweed has |
| 122 | invested into creating a test framework and build integration for running tests |
| 123 | across physical devices as part of a build. |
| 124 | |
| 125 | See also |
| 126 | ^^^^^^^^ |
| 127 | |
Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 128 | * :ref:`module-pw_unit_test` |
| 129 | * :ref:`module-pw_target_runner` |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 130 | |
| 131 | Bonus: pw watch |
| 132 | --------------- |
| 133 | In web development, it is common to have a file system watcher listening for |
| 134 | source file changes and triggering a build for quick iteration. When combined |
| 135 | with a fast incremental build system, this becomes a powerful feature, allowing |
| 136 | things such as unit tests and size reports to re-run whenever any dependent |
| 137 | code is modified. |
| 138 | |
| 139 | While initially seen as somewhat of a gimmick, Pigweed's watcher has become a |
| 140 | staple of Pigweed development, with most Pigweed users having it permanently |
| 141 | running in a terminal window. |
| 142 | |
| 143 | See also |
| 144 | ^^^^^^^^ |
| 145 | |
Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 146 | * :ref:`module-pw_watch` |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 147 | |
| 148 | Pigweed's build systems |
| 149 | ======================= |
| 150 | Pigweed can be used either as a monolith or à la carte, slotting into an |
| 151 | existing project. To this end, Pigweed supports multiple build systems, allowing |
| 152 | Pigweed-based projects to choose the most suitable one for them. |
| 153 | |
| 154 | Of the supported build systems, GN is the most full-featured, followed by CMake, |
| 155 | and finally Bazel. |
| 156 | |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 157 | .. note:: |
| 158 | A quick note on terminology: the word "target" is overloaded within GN/Bazel (and |
| 159 | Pigweed)---it can refer to either a GN/Bazel build target, such as a ``source_set`` |
| 160 | or ``executable``, or to an output platform (e.g. a specific board, device, or |
| 161 | system). |
| 162 | |
| 163 | To avoid confusing the two, we refer to the former as "GN/Bazel targets" and the |
| 164 | latter as "Pigweed targets". |
| 165 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 166 | GN |
| 167 | -- |
| 168 | A perhaps unfamiliar name, `GN (Generate Ninja)`_ is a meta-build system that |
| 169 | outputs `Ninja`_ build files, originally designed for use in Chromium. Pigweed |
| 170 | first experimented with GN after hearing about it from another team, and we |
| 171 | quickly came to appreciate its speed and simplicity. GN has become Pigweed's |
| 172 | primary build system; it is used for all upstream development and strongly |
| 173 | recommended for Pigweed-based projects where possible. |
| 174 | |
| 175 | .. _CMake: https://cmake.org/ |
| 176 | .. _Bazel: https://bazel.build/ |
| 177 | .. _GN (Generate Ninja): https://gn.googlesource.com/gn |
| 178 | .. _Ninja: https://ninja-build.org/ |
| 179 | |
| 180 | The GN build |
| 181 | ============ |
| 182 | This section describes Pigweed's GN build structure, how it is used upstream, |
| 183 | build conventions, and recommendations for Pigweed-based projects. While |
| 184 | containing some details about how GN works in general, this section is not |
| 185 | intended to be a guide on how to use GN. To learn more about the tool itself, |
| 186 | refer to the official `GN reference`_. |
| 187 | |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 188 | .. _GN reference: https://gn.googlesource.com/gn/+/HEAD/docs/reference.md |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 189 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 190 | Entrypoint: .gn |
| 191 | --------------- |
| 192 | The entrypoint to a GN build is the ``.gn`` file, which defines a project's root |
| 193 | directory (henceforth ``//``). |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 194 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 195 | ``.gn`` must point to the location of a ``BUILDCONFIG.gn`` file for the project. |
| 196 | In Pigweed upstream, this is its only purpose. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 197 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 198 | Downstream projects may additionally use ``.gn`` to set global overrides for |
| 199 | Pigweed's build arguments, which apply across all Pigweed targets. For example, |
| 200 | a project could configure the protobuf libraries that it uses. This is done by |
| 201 | defining a ``default_args`` scope containing the overrides. |
| 202 | |
| 203 | .. code:: |
| 204 | |
| 205 | # The location of the BUILDCONFIG file. |
| 206 | buildconfig = "//BUILDCONFIG.gn" |
| 207 | |
| 208 | # Build arguments set across all Pigweed targets. |
| 209 | default_args = { |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 210 | dir_pw_third_party_nanopb = "//third_party/nanopb-0.4.2" |
| 211 | } |
| 212 | |
| 213 | Configuration: BUILDCONFIG.gn |
| 214 | ----------------------------- |
| 215 | The file ``BUILDCONFIG.gn`` configures the GN build by defining any desired |
| 216 | global variables/options. It can be located anywhere in the build tree, but is |
| 217 | conventionally placed at the root. ``.gn`` points GN to this file. |
| 218 | |
| 219 | ``BUILDCONFIG.gn`` is evaluated before any other GN files, and variables defined |
| 220 | within it are placed into GN's global scope, becoming available in every file |
| 221 | without requiring imports. |
| 222 | |
| 223 | The options configured in this file differ from those in ``.gn`` in two ways: |
| 224 | |
| 225 | 1. ``BUILDCONFIG.gn`` is evaluated for every GN toolchain (and Pigweed target), |
| 226 | whereas ``.gn`` is only evaluated once. This allows ``BUILDCONFIG.gn`` to set |
| 227 | different options for each Pigweed target. |
| 228 | 2. In ``.gn``, only GN build arguments can be overridden. ``BUILDCONFIG.gn`` |
| 229 | allows defining arbitrary variables. |
| 230 | |
| 231 | Generally, it is preferable to expose configuration options through build args |
| 232 | instead of globals in ``BUILDCONFIG.gn`` (something Pigweed's build previously |
| 233 | did), as they are more flexible, greppable, and easier to manage. However, it |
| 234 | may make sense to define project-specific constants in ``BUILDCONFIG.gn``. |
| 235 | |
| 236 | Pigweed's upstream ``BUILDCONFIG.gn`` does not define any variables; it just |
| 237 | sets Pigweed's default toolchain, which GN requires. |
| 238 | |
| 239 | .. _top-level-build: |
| 240 | |
| 241 | Top-level GN targets: //BUILD.gn |
| 242 | -------------------------------- |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 243 | The root ``BUILD.gn`` file defines all of the libraries, images, tests, and |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 244 | binaries built by a Pigweed project. This file is evaluated immediately after |
| 245 | ``BUILDCONFIG.gn``, with the active toolchain (which is the default toolchain |
| 246 | at the start of a build). |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 247 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 248 | ``//BUILD.gn`` is responsible for enumerating each of the Pigweed targets built |
| 249 | by a project. This is done by instantiating a version of each of the project's |
| 250 | GN target groups with each Pigweed target's toolchain. For example, in upstream, |
| 251 | all of Pigweed's GN targets are contained within the ``pigweed_default`` group. |
| 252 | This group is instantiated multiple times, with different Pigweed target |
| 253 | toolchains. |
| 254 | |
Wyatt Hepler | 38c398d | 2020-07-14 11:15:30 -0700 | [diff] [blame] | 255 | These groups include the following: |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 256 | |
Wyatt Hepler | 38c398d | 2020-07-14 11:15:30 -0700 | [diff] [blame] | 257 | * ``host`` -- builds ``pigweed_default`` with Clang or GCC, depending on the |
| 258 | platform |
| 259 | * ``host_clang`` -- builds ``pigweed_default`` for the host with Clang |
| 260 | * ``host_gcc`` -- builds ``pigweed_default`` for the host with GCC |
| 261 | * ``stm32f429i`` -- builds ``pigweed_default`` for STM32F429i Discovery board |
| 262 | * ``docs`` -- builds the Pigweed documentation and size reports |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 263 | |
| 264 | Pigweed projects are recommended to follow this pattern, creating a top-level |
Wyatt Hepler | 38c398d | 2020-07-14 11:15:30 -0700 | [diff] [blame] | 265 | group for each of their Pigweed targets that builds a common GN target with the |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 266 | appropriate toolchain. |
| 267 | |
| 268 | It is important that no dependencies are listed under the default toolchain |
| 269 | within ``//BUILD.gn``, as it does not configure any build parameters, and |
| 270 | therefore should not evaluate any other GN files. The pattern that Pigweed uses |
| 271 | to achieve this is to wrap all dependencies within a condition checking the |
| 272 | toolchain. |
| 273 | |
| 274 | .. code:: |
| 275 | |
| 276 | group("my_application_images") { |
| 277 | deps = [] # Empty in the default toolchain. |
| 278 | |
| 279 | if (current_toolchain != default_toolchain) { |
| 280 | # This is only evaluated by Pigweed target toolchains, which configure |
| 281 | # all of the required options to build Pigweed code. |
| 282 | deps += [ "//images:evt" ] |
| 283 | } |
| 284 | } |
| 285 | |
| 286 | # The images group is instantiated for each of the project's Pigweed targets. |
Wyatt Hepler | 38c398d | 2020-07-14 11:15:30 -0700 | [diff] [blame] | 287 | group("my_pigweed_target") { |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 288 | deps = [ ":my_application_images(//toolchains:my_pigweed_target)" ] |
| 289 | } |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 290 | |
| 291 | .. warning:: |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 292 | Pigweed's default toolchain is never used, so it is set to an empty toolchain |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 293 | which doesn't define any tools. ``//BUILD.gn`` contains conditions which check |
| 294 | that the current toolchain is not the default before declaring any GN target |
| 295 | dependencies to prevent the default toolchain from evaluating any other BUILD |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 296 | files. All GN targets added to the build must be placed under one of these |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 297 | conditional scopes. |
| 298 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 299 | "default" group |
| 300 | ^^^^^^^^^^^^^^^ |
| 301 | The root ``BUILD.gn`` file can define a special group named ``default``. If |
| 302 | present, Ninja will build this group when invoked without arguments. |
| 303 | |
| 304 | .. tip:: |
| 305 | Defining a ``default`` group makes using ``pw watch`` simple! |
| 306 | |
Wyatt Hepler | 38c398d | 2020-07-14 11:15:30 -0700 | [diff] [blame] | 307 | Optimization levels |
| 308 | ^^^^^^^^^^^^^^^^^^^ |
| 309 | Pigweed's ``//BUILD.gn`` defines the ``pw_default_optimization_level`` build |
| 310 | arg, which specifies the optimization level to use for the default groups |
| 311 | (``host``, ``stm32f429i``, etc.). The supported values for |
| 312 | ``pw_default_optimization_level`` are: |
| 313 | |
| 314 | * ``debug`` -- create debugging-friendly binaries (``-Og``) |
| 315 | * ``size_optimized`` -- optimize for size (``-Os``) |
| 316 | * ``speed_optimized`` -- optimized for speed, without increasing code size |
| 317 | (``-O2``) |
| 318 | |
| 319 | Pigweed defines versions of its groups in ``//BUILD.gn`` for each optimization |
| 320 | level. Rather than relying on ``pw_default_optimization_level``, you may |
| 321 | directly build a group at the desired optimization level: |
| 322 | ``<group>_<optimization>``. Examples include ``host_clang_debug``, |
| 323 | ``host_gcc_size_optimized``, and ``stm32f429i_speed_optimized``. |
| 324 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 325 | Upstream GN target groups |
| 326 | ^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 327 | In upstream, Pigweed splits its top-level GN targets into a few logical groups, |
| 328 | which are described below. In order to build a GN target, it *must* be listed in |
| 329 | one of the groups in this file. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 330 | |
Wyatt Hepler | 70a6210 | 2022-02-11 15:14:35 -0800 | [diff] [blame] | 331 | .. important:: |
| 332 | |
| 333 | Pigweed's top-level ``BUILD.gn`` file should not be used by downstream |
| 334 | projects. Projects that wish to pull all of Pigweed's code into their build |
| 335 | may use the ``pw_modules`` and ``pw_module_tests`` variables in |
| 336 | ``modules.gni``. |
| 337 | |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 338 | apps |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 339 | ~~~~ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 340 | This group defines the application images built in Pigweed. It lists all of the |
| 341 | common images built across all Pigweed targets, such as modules' example |
| 342 | executables. Each Pigweed target can additionally provide its own specific |
| 343 | images through the ``pw_TARGET_APPLICATIONS`` build arg, which is included by |
| 344 | this group. |
| 345 | |
| 346 | host_tools |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 347 | ~~~~~~~~~~ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 348 | This group defines host-side tooling binaries built for Pigweed. |
| 349 | |
Dmitry Yatsushkevich | f3b9d0d | 2021-10-20 13:49:36 -0700 | [diff] [blame] | 350 | runtime_sanitizers |
| 351 | ~~~~~~~~~~~~~~~~~~ |
| 352 | This group defines host-side build targets for Clang runtime sanitizers. |
| 353 | Next runtime sanitizers supported: |
| 354 | |
| 355 | * ``asan`` -- `AddressSanitizer`_ is a fast memory error detector. |
| 356 | * ``msan`` -- `MemorySanitizer`_ is a detector of uninitialized reads. |
| 357 | * ``ubsan`` -- `UndefinedBehaviorSanitizer`_ is a fast undefined behavior detector. |
Ted Pudlik | 4c8206b | 2021-11-04 22:26:31 +0000 | [diff] [blame] | 358 | * ``ubsan_heuristic`` -- `UndefinedBehaviorSanitizer`_ with the following |
| 359 | additional checks enabled: |
| 360 | |
| 361 | * ``integer``: Checks for undefined or suspicious integer behavior. |
| 362 | * ``float-divide-by-zero``: Checks for floating point division by zero. |
| 363 | * ``implicit-conversion"``: Checks for suspicious behavior of implicit conversions. |
| 364 | * ``nullability``: Checks for null as function arg, lvalue and return type. |
| 365 | |
| 366 | These additional checks are heuristic and may not correspond to undefined |
| 367 | behavior. |
Dmitry Yatsushkevich | f3b9d0d | 2021-10-20 13:49:36 -0700 | [diff] [blame] | 368 | * ``tsan`` -- `ThreadSanitizer`_ is a tool that detects data races. |
| 369 | |
| 370 | Results of building this group are ``host_clang_<sanitizer>`` build directories |
| 371 | with ``pw_module_tests`` per supported sanitizer. |
| 372 | |
| 373 | .. _AddressSanitizer: https://clang.llvm.org/docs/AddressSanitizer.html |
| 374 | .. _MemorySanitizer: https://clang.llvm.org/docs/MemorySanitizer.html |
| 375 | .. _UndefinedBehaviorSanitizer: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html |
| 376 | .. _ThreadSanitizer: https://clang.llvm.org/docs/ThreadSanitizer.html |
| 377 | |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 378 | pw_modules |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 379 | ~~~~~~~~~~ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 380 | This group lists the main libraries for all of Pigweed's modules. |
| 381 | |
Wyatt Hepler | 70a6210 | 2022-02-11 15:14:35 -0800 | [diff] [blame] | 382 | The modules in the ``pw_modules`` group are listed in the ``pw_modules`` |
| 383 | variable, which is provided by ``modules.gni``. |
| 384 | |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 385 | pw_module_tests |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 386 | ~~~~~~~~~~~~~~~ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 387 | All modules' unit tests are collected here, so that they can all be run at once. |
| 388 | |
Wyatt Hepler | 70a6210 | 2022-02-11 15:14:35 -0800 | [diff] [blame] | 389 | The test groups in ``pw_module_tests`` group are listed in the |
| 390 | ``pw_module_tests`` variable, which is provided by ``modules.gni``. |
| 391 | |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 392 | pigweed_default |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 393 | ~~~~~~~~~~~~~~~ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 394 | This group defines everything built in a Pigweed build invocation by collecting |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 395 | the above groups and conditionally depending on them based on the active Pigweed |
| 396 | target's configuration. Generally, new dependencies should not be added here; |
| 397 | instead, use one of the groups listed above. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 398 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 399 | The ``pigweed_default`` group is instantiated for each upstream Pigweed target's |
| 400 | toolchain. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 401 | |
| 402 | Pigweed target instantiations |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 403 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 404 | These groups wrap ``pigweed_default`` with a specific target toolchain. They are |
| 405 | named after the Pigweed target, e.g. ``host_clang``, ``stm32f429i``, etc. |
| 406 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 407 | Other BUILD files: //\*\*/BUILD.gn |
| 408 | ---------------------------------- |
| 409 | The rest of the ``BUILD.gn`` files in the tree define libraries, configs, and |
| 410 | build args for each of the modules in a Pigweed project. |
| 411 | |
| 412 | Project configuration: //build_overrides/pigweed.gni |
| 413 | ---------------------------------------------------- |
| 414 | Each Pigweed project must contain a Pigweed configuration file at a known |
| 415 | location in the GN build tree. Currently, this file only contains a single build |
| 416 | argument, which must be set to the GN build path to the root of the Pigweed |
| 417 | repository within the project. |
| 418 | |
| 419 | Module variables |
| 420 | ---------------- |
Ali Zhang | f22f1f1 | 2021-04-02 18:59:28 -0700 | [diff] [blame] | 421 | As Pigweed is intended to be a subcomponent of a larger project, it cannot assume |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 422 | where it or its modules is located. Therefore, Pigweed's upstream BUILD.gn files |
| 423 | do not use absolute paths; instead, variables are defined pointing to each of |
| 424 | Pigweed's modules, set relative to a project-specific ``dir_pigweed``. |
| 425 | |
| 426 | To depend on Pigweed modules from GN code, import Pigweed's overrides file and |
| 427 | reference these module variables. |
| 428 | |
| 429 | .. code:: |
| 430 | |
Wyatt Hepler | e0575f7 | 2020-10-16 10:47:03 -0700 | [diff] [blame] | 431 | # This must be imported before .gni files from any other Pigweed modules. To |
| 432 | # prevent gn format from reordering this import, it must be separated by a |
| 433 | # blank line from other imports. |
| 434 | |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 435 | import("//build_overrides/pigweed.gni") |
| 436 | |
| 437 | GN target type wrappers |
| 438 | ----------------------- |
Ali Zhang | f22f1f1 | 2021-04-02 18:59:28 -0700 | [diff] [blame] | 439 | To facilitate injecting global configuration options, Pigweed defines wrappers |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 440 | around builtin GN target types such as ``source_set`` and ``executable``. These |
| 441 | are defined within ``$dir_pw_build/target_types.gni``. |
| 442 | |
| 443 | .. note:: |
| 444 | To take advantage of Pigweed's flexible target configuration system, use |
| 445 | ``pw_*`` target types (e.g. ``pw_source_set``) in your BUILD.gn files instead |
| 446 | of GN builtins. |
| 447 | |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 448 | Pigweed targets |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 449 | --------------- |
| 450 | To build for a specific hardware platform, builds define Pigweed targets. These |
| 451 | are essentially GN toolchains which set special arguments telling Pigweed how to |
| 452 | build. For information on Pigweed's target system, refer to |
Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 453 | :ref:`docs-targets`. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 454 | |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 455 | The empty toolchain |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 456 | ------------------- |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 457 | Pigweed's ``BUILDCONFIG.gn`` sets the project's default toolchain to a "empty" |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 458 | toolchain which does not specify any compilers or override any build arguments. |
| 459 | Downstream projects are recommended to do the same, following the steps |
| 460 | described in :ref:`top-level-build` to configure builds for each of their |
| 461 | Pigweed targets. |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 462 | |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 463 | .. admonition:: Why use an empty toolchain? |
Alexei Frolov | 199045a | 2020-08-28 13:02:30 -0700 | [diff] [blame] | 464 | |
| 465 | To support some of its advanced (and useful!) build features, Pigweed requires |
| 466 | the ability to generate new toolchains on the fly. This requires having |
| 467 | knowledge of the full configuration of the current toolchain (which is easy if |
| 468 | it's all defined within a scope), something which is impractical to achieve |
| 469 | using the default toolchain. |
| 470 | |
| 471 | Additionally, there are some cases where GN treats default and non-default |
| 472 | toolchains differently. By not using the default toolchain, we avoid having |
| 473 | to deal with these inconsistencies. |
| 474 | |
| 475 | It is possible to build Pigweed using only the default toolchain, but it |
| 476 | requires a more complicated setup to get everything working and should be |
| 477 | avoided unless necessary (for example, when integrating with a large existing |
| 478 | GN-based project). |
| 479 | |
| 480 | Upstream development examples |
| 481 | ----------------------------- |
| 482 | If developing for upstream Pigweed, some common build use cases are described |
| 483 | below. |
| 484 | |
| 485 | Building a custom executable/app image |
| 486 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
Alexei Frolov | f675390 | 2020-07-08 11:01:45 -0700 | [diff] [blame] | 487 | |
| 488 | 1. Define your executable GN target using the ``pw_executable`` template. |
| 489 | |
| 490 | .. code:: |
| 491 | |
| 492 | # //foo/BUILD.gn |
| 493 | pw_executable("foo") { |
| 494 | sources = [ "main.cc" ] |
| 495 | deps = [ ":libfoo" ] |
| 496 | } |
| 497 | |
| 498 | 2. In the root ``BUILD.gn`` file, add the executable's GN target to the ``apps`` |
| 499 | group. |
| 500 | |
| 501 | .. code:: |
| 502 | |
| 503 | # //BUILD.gn |
| 504 | group("apps") { |
| 505 | deps = [ |
| 506 | # ... |
| 507 | "//foo", # Shorthand for //foo:foo |
| 508 | ] |
| 509 | } |
| 510 | |
| 511 | 3. Run the ninja build to compile your executable. The apps group is built by |
| 512 | default, so there's no need to provide a target. The executable will be |
| 513 | compiled for every supported Pigweed target. |
| 514 | |
| 515 | .. code:: |
| 516 | |
| 517 | ninja -C out |
| 518 | |
| 519 | Alternatively, build your executable by itself by specifying its path to |
| 520 | Ninja. When building a GN target manually, the Pigweed target for which it |
| 521 | is built must be specified on the Ninja command line. |
| 522 | |
| 523 | For example, to build for the Pigweed target ``host_gcc_debug``: |
| 524 | |
| 525 | .. code:: |
| 526 | |
| 527 | ninja -C out host_gcc_debug/obj/foo/bin/foo |
| 528 | |
| 529 | .. note:: |
| 530 | |
| 531 | The path passed to Ninja is a filesystem path within the ``out`` directory, |
| 532 | rather than a GN path. This path can be found by running ``gn outputs``. |
| 533 | |
| 534 | 4. Retrieve your compiled binary from the out directory. It is located at the |
| 535 | path |
| 536 | |
| 537 | .. code:: |
| 538 | |
| 539 | out/<pw_target>/obj/<gn_path>/{bin,test}/<executable> |
| 540 | |
| 541 | where ``pw_target`` is the Pigweed target for which the binary was built, |
| 542 | ``gn_path`` is the GN path to the BUILD.gn file defining the executable, |
| 543 | and ``executable`` is the executable's GN target name (potentially with an |
| 544 | extension). Note that the executable is located within a ``bin`` subdirectory |
| 545 | in the module (or ``test`` for unit tests defined with ``pw_test``). |
| 546 | |
| 547 | For example, the ``foo`` executable defined above and compiled for the |
| 548 | Pigweed target stm32f429i_disc1_debug is found at: |
| 549 | |
| 550 | .. code:: |
| 551 | |
| 552 | out/stm32f429i_disc1_debug/obj/foo/bin/foo |
Nathaniel Brough | 3c81320 | 2021-03-23 16:30:31 +0800 | [diff] [blame] | 553 | |
Nathaniel Brough | 3c81320 | 2021-03-23 16:30:31 +0800 | [diff] [blame] | 554 | CMake |
| 555 | ----- |
| 556 | A well-known name in C/C++ development, `CMake`_ is widely used by all kinds of |
| 557 | projects, including embedded devices. Pigweed's CMake support is provided |
| 558 | primarily for projects that have an existing CMake build and wish to integrate |
| 559 | Pigweed modules. |
| 560 | |
| 561 | Bazel |
| 562 | ----- |
| 563 | The open source version of Google's internal build system. `Bazel`_ has been |
| 564 | growing in popularity within the open source world, as well as being adopted by |
| 565 | various proprietary projects. Its modular structure makes it a great fit for |
| 566 | à la carte usage. |
| 567 | |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 568 | .. note:: |
| 569 | Bazel support is experimental and only for the brave for now. If you are |
| 570 | looking for stable set of build API's please use GN. |
| 571 | |
Nathaniel Brough | 3c81320 | 2021-03-23 16:30:31 +0800 | [diff] [blame] | 572 | The Bazel build |
| 573 | =============== |
| 574 | This section describes Pigweed's Bazel build structure, how it is used upstream, |
| 575 | build conventions, and recommendations for Pigweed-based projects. While |
| 576 | containing some details about how Bazel works in general, this section is not |
| 577 | intended to be a guide on how to use Bazel. To learn more about the tool itself, |
| 578 | refer to the official `Bazel reference`_. |
| 579 | |
Ted Pudlik | 12fa59f | 2022-01-28 01:54:06 +0000 | [diff] [blame] | 580 | .. _Bazel reference: https://www.bazel.build/ |
Nathaniel Brough | 3c81320 | 2021-03-23 16:30:31 +0800 | [diff] [blame] | 581 | |
| 582 | General usage |
| 583 | ------------- |
| 584 | While described in more detail in the Bazel docs there a few Bazel features that |
| 585 | are of particular importance when targeting embedded platforms. The most |
| 586 | commonly used commands used in bazel are; |
| 587 | |
| 588 | .. code:: sh |
| 589 | |
| 590 | bazel build //your:target |
| 591 | bazel test //your:target |
| 592 | bazel coverage //your:target |
| 593 | |
| 594 | .. note:: Code coverage support is only available on the host for now. |
| 595 | |
| 596 | Building |
| 597 | ^^^^^^^^ |
| 598 | When it comes to building/testing your Bazel target for a specific Pigweed |
| 599 | target (e.g. stm32f429i-discovery) a slight variation is required. |
| 600 | |
| 601 | .. code:: sh |
| 602 | |
| 603 | bazel build //your:target \ |
| 604 | --platforms=@pigweed//pw_build/platforms:stm32f429i-disc1 |
| 605 | |
| 606 | For more information on how to create your own platforms refer to the official |
| 607 | `Bazel platforms reference`_. You may also find helpful examples of constraints |
| 608 | and platforms in the '//pw_build/platforms' and '//pw_build/constraints' |
| 609 | directories. |
| 610 | |
Rob Mohr | dfbdd1f | 2021-06-11 08:21:17 -0700 | [diff] [blame] | 611 | .. _Bazel platforms reference: https://docs.bazel.build/versions/main/platforms.html |
Nathaniel Brough | 3c81320 | 2021-03-23 16:30:31 +0800 | [diff] [blame] | 612 | |
| 613 | Testing |
| 614 | ^^^^^^^ |
| 615 | Running tests on an embedded target with Bazel is possible although support for |
| 616 | this is experimental. The easiest way of achieving this at the moment is to use |
| 617 | Bazel's '--run_under' flag. To make this work create a Bazel target |
| 618 | ('//your_handler') that; |
| 619 | |
| 620 | 1. Takes a single argument (the path to the elf) and uploads the elf to your |
| 621 | Pigweed target. |
| 622 | 2. Connects with your target using serial or other communication method. |
| 623 | 3. Listens to the communication transport for the keywords ("PASSED", "FAIL") |
| 624 | and returns (0, 1) respectively if one of the keywords is intercepted. (This |
| 625 | step assumes you are using the pw_unit_test package and it is configured for |
| 626 | your target). |
| 627 | 4. Run; |
| 628 | |
| 629 | .. code:: sh |
| 630 | |
| 631 | bazel test //your:test --platforms=//your/platform --run_under=//your_handler |
| 632 | |
| 633 | Code Coverage |
| 634 | ^^^^^^^^^^^^^ |
| 635 | Making use of the code coverage functionality in Bazel is straightforward. |
| 636 | |
| 637 | 1. Add the following lines to your '.bazelrc'. |
| 638 | |
| 639 | .. code:: sh |
| 640 | |
| 641 | coverage --experimental_generate_llvm_lcov |
| 642 | coverage --combined_report=lcov |
| 643 | |
| 644 | 2. Generate a combined lcov coverage report. This will produce a combined lcov |
| 645 | coverage report at the path 'bazel-out/_coverage/_coverage_report.dat'. e.g. |
| 646 | |
| 647 | .. code:: sh |
| 648 | |
| 649 | bazel coverage //pw_log/... |
| 650 | |
| 651 | 3. View the results using the command line utility 'lcov'. |
| 652 | |
| 653 | .. code:: sh |
| 654 | |
| 655 | lcov --list bazel-out/_coverage/_coverage_report.dat |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 656 | |
| 657 | Configuration |
| 658 | ------------- |
| 659 | Generally speaking there are three primary concepts that make up Bazel's |
| 660 | configuration API. |
| 661 | |
| 662 | 1. Selects |
| 663 | 2. Compatibility lists |
| 664 | 3. Flags/Build settings |
| 665 | |
| 666 | Selects |
| 667 | ^^^^^^^ |
| 668 | Selects are useful for specifying different dependencies/source depending on the |
| 669 | platform that is currently being targeted. For more information on this please |
| 670 | see the `Bazel selects reference`_. e.g. |
| 671 | |
| 672 | .. code:: py |
| 673 | |
| 674 | pw_cc_library( |
| 675 | name = "some_platform_dependant_library", |
| 676 | deps = select({ |
| 677 | "@platforms//cpu:armv7e-m": [":arm_libs"], |
| 678 | "//conditions:default": [":host_libs"], |
| 679 | }), |
| 680 | ) |
| 681 | |
| 682 | Compatibility lists |
| 683 | ^^^^^^^^^^^^^^^^^^^ |
| 684 | Compatibility lists allow you to specify which platforms your targets are |
| 685 | compatible with. Consider an example where you want to specify that a target is |
| 686 | compatible with only a host os; |
| 687 | |
| 688 | .. code:: py |
| 689 | |
| 690 | pw_cc_library( |
| 691 | name = "some_host_only_lib", |
| 692 | srcs = ["host.cc"], |
| 693 | target_compatible_with = select({ |
| 694 | "@platforms//os:windows": [], |
| 695 | "@platforms//os:linux": [], |
| 696 | "@platforms//os:macos": [], |
| 697 | "//conditions:default": ["@platforms//:incompatible"], |
| 698 | }), |
| 699 | ) |
| 700 | |
| 701 | In this case building from or for either Windows/Linux/Mac will be supported, but |
| 702 | other OS's will fail if this target is explicitly depended on. However if |
| 703 | building with a wild card for a non-host platform this target will be skipped |
| 704 | and the build will continue. e.g. |
| 705 | |
| 706 | .. code:: sh |
| 707 | |
| 708 | bazel build //... --platforms=@pigweed//pw_build/platforms:cortex_m0 |
| 709 | |
| 710 | This allows for you to easily create compatibility matricies without adversely |
| 711 | affecting your ability build your entire repo for a given Pigweed target. |
| 712 | For more detailed information on how to use the target_compatible_with attribute |
| 713 | please see `Bazel target_compatible_with reference`_. |
| 714 | |
| 715 | Flags/build settings |
| 716 | ^^^^^^^^^^^^^^^^^^^^ |
| 717 | Flags/build settings are particularly useful in scenarios where you may want to |
| 718 | be able to quickly inject a dependency from the command line but don't |
| 719 | necessarily want to create an entirely new set of constraints to use with a |
| 720 | select statement. |
| 721 | |
| 722 | .. note:: |
| 723 | The scope for what is possible with build flags/settings goes well beyond |
| 724 | what will be described here. For more detailed information on flags/settings |
| 725 | please see `Bazel config reference`_. |
| 726 | |
| 727 | A simple example of when it is useful to use a label_flag is when you want to |
| 728 | swap out a single dependency from the command line. e.g. |
| 729 | |
| 730 | .. code:: py |
| 731 | |
| 732 | pw_cc_library( |
| 733 | name = "some_default_io", |
| 734 | srcs = ["default_io.cc"], |
| 735 | ) |
| 736 | |
| 737 | pw_cc_library( |
| 738 | name = "some_other_io", |
| 739 | srcs = ["other_io.cc"], |
| 740 | ) |
| 741 | |
| 742 | label_flag( |
| 743 | name = "io", |
| 744 | default_build_setting = ":some_default_io", |
| 745 | ) |
| 746 | |
| 747 | pw_cc_library( |
| 748 | name = "some_target_that_needs_io", |
| 749 | deps = [":io"], |
| 750 | ) |
| 751 | |
| 752 | From here the label_flag by default redirects to the target ":some_default_io", |
| 753 | however it is possible to override this from the command line. e.g. |
| 754 | |
| 755 | .. code:: sh |
| 756 | |
| 757 | bazel build //:some_target_that_needs_io --//:io=//:some_other_io |
| 758 | |
| 759 | |
| 760 | |
Rob Mohr | dfbdd1f | 2021-06-11 08:21:17 -0700 | [diff] [blame] | 761 | .. _Bazel selects reference: https://docs.bazel.build/versions/main/configurable-attributes.html#select-and-dependencies |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 762 | |
Rob Mohr | dfbdd1f | 2021-06-11 08:21:17 -0700 | [diff] [blame] | 763 | .. _Bazel target_compatible_with reference: https://docs.bazel.build/versions/main/platforms.html#skipping-incompatible-targets |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 764 | |
Rob Mohr | dfbdd1f | 2021-06-11 08:21:17 -0700 | [diff] [blame] | 765 | .. _Bazel config reference: https://docs.bazel.build/versions/main/skylark/config.html |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 766 | |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 767 | |
| 768 | Pigweeds configuration |
| 769 | ^^^^^^^^^^^^^^^^^^^^^^ |
| 770 | Pigweeds Bazel configuration API is designed to be distributed across the |
| 771 | Pigweed repository and/or your downstream repository. If you are coming from |
| 772 | GN's centralized configuration API it might be useful to think about |
| 773 | Pigweed+Bazel's configuration as the transpose of GN's configuration. The |
| 774 | configuration interface that is supported by Pigweed is designed to start simple |
| 775 | and then grow with your project. |
| 776 | |
| 777 | .. note:: |
| 778 | There are plans to extend the configuration API for Bazel. However, |
| 779 | currently the only configurations that are available under the Bazel+Pigweed |
| 780 | configuration API is the ability to switch facade backends. For more |
| 781 | information on what this is please see the |
| 782 | :ref:`docs-module-structure-facades` section of :ref:`docs-module-structure`. |
| 783 | |
| 784 | Consider a scenario that you are building a flight controller for a |
| 785 | spacecraft. But have very little experience with Pigweed and you have just |
| 786 | landed here. First things first you would; |
| 787 | |
| 788 | 1. Set up your WORKSPACE to fetch the Pigweeds repository. Then add the |
| 789 | dependencies that you need from Pigweeds WORKSPACE. |
| 790 | |
| 791 | 2. Add a pigweed_config rule to your WORKSPACE, using Pigweed's default |
| 792 | configuration. |
| 793 | |
| 794 | .. code:: py |
| 795 | |
| 796 | # WORKSPACE ... |
| 797 | load("//pw_build:target_config.bzl", "pigweed_config") |
| 798 | |
| 799 | # Configure Pigweeds backend. |
| 800 | pigweed_config( |
| 801 | name = "pigweed_config", |
| 802 | build_file = "@pigweed//targets:default_config.BUILD", |
| 803 | ) |
| 804 | |
| 805 | .. note:: |
| 806 | We are aware, that the experience of setting up your WORKSPACE file to work |
| 807 | with pigweed is less than ideal. This API is under construction, but we are |
| 808 | working on this! |
| 809 | |
| 810 | .. |
| 811 | TODO: Add in a better way to pull down WORKSPACE dependencies in Bazel then |
| 812 | add docs in here. |
| 813 | |
| 814 | Now to explain what is going on here. Housed under the "pigweed_config" remote |
| 815 | repository is a set of configuration flags. These can be used to inject |
| 816 | dependencies into facades to override the default backend. |
| 817 | |
| 818 | Continuing on with our scenario, consider that you maybe want to try using the |
| 819 | '//pw_chrono' module. So you create a target in your repository like so; |
| 820 | |
| 821 | .. code:: |
| 822 | |
| 823 | # BUILD |
| 824 | pw_cc_library( |
| 825 | name = "time_is_relative", |
| 826 | srcs = ["relative_time_on_earth.cc"], |
| 827 | deps = ["@pigweed//pw_chrono"], |
| 828 | ) |
| 829 | |
| 830 | Now this should work out of the box for any host operating system. e.g. Running; |
| 831 | |
| 832 | .. code:: |
| 833 | |
| 834 | bazel build //:time_is_relative |
| 835 | |
| 836 | will produce a working library. But as your probably here because Pigweed offers |
| 837 | a set of embedded libraries you might be interested in running your code on some |
| 838 | random micro-controller/FPGA combined with an RTOS. For now let's assume that by |
| 839 | some coincidence you are using FreeRTOS and are happy to make use |
| 840 | of our default '//pw_chrono' backend for FreeRTOS. You could build the following |
| 841 | with; |
| 842 | |
| 843 | .. code:: sh |
| 844 | |
| 845 | bazel build //:time_is_relative \ |
| 846 | --platforms=@pigweed//pw_build/platforms:freertos |
| 847 | |
| 848 | There is a fair bit to unpack here in terms of how our configuration system |
| 849 | is determining which dependencies to choose for your build. The dependency |
| 850 | tree (that is important for configuration) in a project such as this would |
| 851 | look like. |
| 852 | |
| 853 | .. code:: |
| 854 | |
| 855 | @pigweed//pw_chrono:pw_chrono_facade <-----------. |
| 856 | ^ | |
| 857 | | @pigweed//pw_chrono_freertos:system_clock |
| 858 | | (Actual backend) |
| 859 | | ^ |
| 860 | | | |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 861 | | @pigweed//pw_chrono:system_clock_backend_multiplexer |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 862 | | Select backend based on OS: |
| 863 | | [FreeRTOS (X), Embos ( ), STL ( ), Threadx ( )] |
| 864 | | ^ |
| 865 | | | |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 866 | @pigweed//pw_chrono -------> @pigweed_config//:pw_chrono_system_clock_backend |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 867 | ^ (Injectable) |
| 868 | | |
| 869 | //:time_is_relative |
| 870 | |
| 871 | So when evaluating this setup Bazel checks the dependencies for '//pw_chrono' |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 872 | and finds that it depends on "@pigweed_config//:pw_chrono_system_clock_backend" which looks |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 873 | like this; |
| 874 | |
| 875 | .. code:: py |
| 876 | |
| 877 | # pw_chrono config. |
| 878 | label_flag( |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 879 | name = "pw_chrono_system_clock_backend", |
| 880 | build_setting_default = "@pigweed//pw_chrono:system_clock_backend_multiplexer", |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 881 | ) |
| 882 | |
| 883 | Looking at the 'build_setting_default' we can see that by default it depends |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 884 | back on the target "@pigweed//pw_chrono:system_clock_backend_multiplexer". If |
| 885 | you only had one backend you could actually just change the |
| 886 | 'build_setting_default' to point directly to your backend. However because we |
| 887 | have four different backends we have to use the select semantics to choose the |
| 888 | right one. In this case it looks like; |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 889 | |
| 890 | .. code:: py |
| 891 | |
| 892 | pw_cc_library( |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 893 | name = "system_clock_backend_multiplexer", |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 894 | visibility = ["@pigweed_config//:__pkg__"], |
| 895 | deps = select({ |
| 896 | "@pigweed//pw_build/constraints/rtos:freertos": |
| 897 | ["//pw_chrono_freertos:system_clock"], |
| 898 | "@pigweed//pw_build/constraints/rtos:embos": |
| 899 | ["//pw_chrono_embos:system_clock"], |
| 900 | "@pigweed//pw_build/constraints/rtos:threadx": |
| 901 | ["//pw_chrono_threadx:system_clock"], |
| 902 | "//conditions:default": ["//pw_chrono_stl:system_clock"], |
| 903 | }), |
| 904 | ) |
| 905 | |
| 906 | Intuitively you can see that the first option was selected, which terminates |
| 907 | the configuration chain. |
| 908 | |
| 909 | Continuing on with our scenario let's say that you have read |
| 910 | :ref:`docs-module-structure` and now want to implement your own backend for |
| 911 | '//pw_chrono' using a hardware RTC. In this case you would create a new |
| 912 | directory 'pw_chrono_my_hardware_rtc'. To ensure that your new backend compiles |
| 913 | with the facade an easy and temporary way to override the dependency tree is |
| 914 | to override the label flag in '@pigweed_config'. For example; |
| 915 | |
| 916 | .. code:: sh |
| 917 | |
| 918 | bazel build //:time_is_relative \ |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 919 | --@pigweed_config//pw_chrono_system_clock_backend=//pw_chrono_my_hardware_rtc:system_clock |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 920 | |
| 921 | This temporarily modifies the build graph to look something like this; |
| 922 | |
| 923 | .. code:: |
| 924 | |
| 925 | @pigweed//pw_chrono:pw_chrono_facade <-----. |
| 926 | ^ | |
| 927 | | @your_workspace//pw_chrono_my_hardware_rtc:system_clock |
| 928 | | (Actual backend) |
| 929 | | ^ |
| 930 | | | |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 931 | @pigweed//pw_chrono -> @pigweed_config//:pw_chrono_system_clock_backend |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 932 | ^ (Injectable) |
| 933 | | |
| 934 | //:time_is_relative |
| 935 | |
| 936 | Now while this is a nice temporary change, but you might find yourself in need |
| 937 | of a more permanent configuration. Particularly if you want to override multiple |
| 938 | different backends. In other words if you had several backends to override, that |
| 939 | would translate to several different command line flags (one for each override). |
| 940 | This problem further compounds as you have multiple Pigweed targets all |
| 941 | requiring different combinations of different backends as you can't even reuse |
| 942 | your command line entries. Instead you would have to memorize the correct |
| 943 | combination of backends for each of your targets. |
| 944 | |
| 945 | So continuing on with our scenario, let's say we add a backup micro-controller, |
| 946 | to our spacecraft. But this backup computer doesn't have a hardware RTC. We |
| 947 | still want to share the bulk of the code between the two computers but now we |
| 948 | need two separate implementations for our pw_chrono facade. Let's say we choose |
| 949 | to keep the primary flight computer using the hardware RTC and switch the backup |
| 950 | computer over to use Pigweeds default FreeRTOS backend. In this case we might, |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 951 | want to do something similar to |
| 952 | '@pigweed//pw_chrono:system_clock_backend_multiplexer' and create selectable |
| 953 | dependencies for the two different computers. Now because there are no default |
| 954 | constraint_setting's that meet our requirements we are going to have to; |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 955 | |
| 956 | 1. Create a constraint_setting and a set of constraint_value's for the flight |
| 957 | computer. For example; |
| 958 | |
| 959 | .. code:: py |
| 960 | |
| 961 | # //platforms/flight_computer/BUILD |
| 962 | constraint_setting( |
| 963 | name = "flight_computer", |
| 964 | ) |
| 965 | |
| 966 | constraint_value( |
| 967 | name = "primary", |
| 968 | constraint_setting = ":flight_computer", |
| 969 | ) |
| 970 | |
| 971 | constraint_value( |
| 972 | name = "backup", |
| 973 | constraint_setting = ":flight_computer", |
| 974 | ) |
| 975 | |
| 976 | 2. Create a set of platforms that can be used to switch constraint_value's. |
| 977 | For example; |
| 978 | |
| 979 | .. code:: py |
| 980 | |
| 981 | # //platforms/BUILD |
| 982 | platform( |
| 983 | name = "primary_computer", |
| 984 | constraint_values = ["//platforms/flight_computer:primary"], |
| 985 | ) |
| 986 | |
| 987 | platform( |
| 988 | name = "backup_computer", |
| 989 | constraint_values = ["//platforms/flight_computer:backup"], |
| 990 | ) |
| 991 | |
| 992 | 3. Create a target multiplexer that will select the right backend depending on |
| 993 | which computer you are using. For example; |
| 994 | |
| 995 | .. code:: py |
| 996 | |
| 997 | # //pw_chrono/BUILD |
| 998 | load("//pw_build:pigweed.bzl", "pw_cc_library") |
| 999 | |
| 1000 | pw_cc_library( |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1001 | name = "system_clock_backend_multiplexer", |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1002 | deps = select({ |
| 1003 | "//platforms/flight_computer:primary": [ |
| 1004 | "//pw_chrono_my_hardware_rtc:system_clock", |
| 1005 | ], |
| 1006 | "//platforms/flight_computer:backup": [ |
| 1007 | "@pigweed//pw_chrono_freertos:system_clock", |
| 1008 | ], |
| 1009 | "//conditions:default": [ |
| 1010 | "@pigweed//pw_chrono_stl:system_clock", |
| 1011 | ], |
| 1012 | }), |
| 1013 | ) |
| 1014 | |
| 1015 | 4. Copy and paste across the target/default_config.BUILD across from the |
| 1016 | Pigweed repository and modifying the build_setting_default for the target |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1017 | 'pw_chrono_system_clock_backend' to point to your new system_clock_backend_multiplexer |
| 1018 | target. For example; |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1019 | |
| 1020 | This; |
| 1021 | |
| 1022 | .. code:: py |
| 1023 | |
| 1024 | # @pigweed//target:default_config.BUILD |
| 1025 | label_flag( |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1026 | name = "pw_chrono_system_clock_backend", |
| 1027 | build_setting_default = "@pigweed//pw_chrono:system_clock_backend_multiplexer", |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1028 | ) |
| 1029 | |
| 1030 | Becomes this; |
| 1031 | |
| 1032 | .. code:: py |
| 1033 | |
| 1034 | # @your_workspace//target:your_config.BUILD |
| 1035 | label_flag( |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1036 | name = "pw_chrono_system_clock_backend", |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1037 | build_setting_default = |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1038 | "@your_workspace//pw_chrono:system_clock_backend_multiplexer", |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1039 | ) |
| 1040 | |
| 1041 | 5. Switch your workspace 'pigweed_config' rule over to use your custom config. |
| 1042 | |
| 1043 | .. code:: py |
| 1044 | |
| 1045 | # WORKSPACE |
| 1046 | pigweed_config( |
| 1047 | name = "pigweed_config", |
| 1048 | build_file = "//target/your_config.BUILD", |
| 1049 | ) |
| 1050 | |
| 1051 | Building your target now will result in slightly different build graph. For |
| 1052 | example, running; |
| 1053 | |
| 1054 | .. code:: sh |
| 1055 | |
| 1056 | bazel build //:time_is_relative --platforms=//platforms:primary_computer |
| 1057 | |
| 1058 | Will result in a build graph that looks like; |
| 1059 | |
| 1060 | .. code:: |
| 1061 | |
| 1062 | @pigweed//pw_chrono:pw_chrono_facade <---. |
| 1063 | ^ | |
| 1064 | | @your_workspace//pw_chrono_my_hardware_rtc:system_clock |
| 1065 | | (Actual backend) |
| 1066 | | ^ |
| 1067 | | | |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1068 | | @your_workspace//pw_chrono:system_clock_backend_multiplexer |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1069 | | Select backend based on OS: |
| 1070 | | [Primary (X), Backup ( ), Host only default ( )] |
| 1071 | | ^ |
| 1072 | | | |
Ewout van Bekkum | 2140484 | 2021-06-08 11:06:17 -0700 | [diff] [blame] | 1073 | @pigweed//pw_chrono -> @pigweed_config//:pw_chrono_system_clock_backend |
Nathaniel Brough | c2d5781 | 2021-04-18 22:52:00 +0800 | [diff] [blame] | 1074 | ^ (Injectable) |
| 1075 | | |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 1076 | //:time_is_relative |