Keir Mierle | 72cae43 | 2021-04-10 02:03:40 -0700 | [diff] [blame] | 1 | .. _docs-getting-started: |
| 2 | |
| 3 | =============== |
| 4 | Getting Started |
| 5 | =============== |
| 6 | This guide will walk you through the typical upstream development workflow. |
| 7 | |
| 8 | .. note:: |
| 9 | |
| 10 | We don't yet have thorough documentation for leveraging Pigweed in a separate |
| 11 | project (our intended use case!). The `sample project |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 12 | <https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md>`_ |
Keir Mierle | 72cae43 | 2021-04-10 02:03:40 -0700 | [diff] [blame] | 13 | shows how to use Pigweed as a library in your broader project, but you may |
| 14 | need further guidance. |
| 15 | |
| 16 | We're happy to help you get your project setup; just drop in our `chat room |
| 17 | <https://discord.gg/M9NSeTA>`_ or send a note to the `mailing list |
| 18 | <https://groups.google.com/forum/#!forum/pigweed>`_. |
| 19 | |
| 20 | Express setup |
| 21 | ============= |
| 22 | If you'd like to skip the detailed explanations, below is the shorter version |
| 23 | of getting setup for Pigweed. If you run into trouble, look at the more |
| 24 | in-depth guide below, starting at :ref:`prerequisites`. The express setup |
| 25 | configures Pigweed's watcher for three targets to give a taste of Pigweed: |
| 26 | |
| 27 | #. **Host** - Mac, Linux, or Windows. Builds and runs tests |
| 28 | #. **Device/STM32F429** - Build only; Optionally, the STM32F429I-DISC1 kit to |
| 29 | follow along later in the guide to run tests directly on said device(s) |
| 30 | #. **Docs** - Builds the Pigweed docs |
| 31 | |
| 32 | To get setup: |
| 33 | |
| 34 | #. Make sure you have Git and Python installed and on your path. |
| 35 | |
| 36 | #. Clone Pigweed and bootstrap the environment (compiler setup & more). **Be |
| 37 | patient, this step downloads ~1GB of LLVM, GCC, and other tooling**. |
| 38 | |
| 39 | .. code:: bash |
| 40 | |
| 41 | $ cd ~ |
| 42 | $ git clone https://pigweed.googlesource.com/pigweed/pigweed |
| 43 | ... |
| 44 | $ cd pigweed |
| 45 | $ source ./bootstrap.sh (On Linux & Mac) |
| 46 | $ bootstrap.bat (On Windows) |
| 47 | ... |
| 48 | |
| 49 | #. Configure the GN build. |
| 50 | |
| 51 | .. code:: bash |
| 52 | |
| 53 | $ gn gen out |
| 54 | Done. Made 1047 targets from 91 files in 114ms |
| 55 | |
| 56 | #. Start the watcher. The watcher will invoke Ninja to build all the targets |
| 57 | |
| 58 | .. code:: bash |
| 59 | |
| 60 | $ pw watch |
| 61 | |
| 62 | ▒█████▄ █▓ ▄███▒ ▒█ ▒█ ░▓████▒ ░▓████▒ ▒▓████▄ |
| 63 | ▒█░ █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█ ▒█ ▀ ▒█ ▀ ▒█ ▀█▌ |
| 64 | ▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█ ▒███ ▒███ ░█ █▌ |
| 65 | ▒█▀ ░█░ ▓█ █▓ ░█░ █ ▒█ ▒█ ▄ ▒█ ▄ ░█ ▄█▌ |
| 66 | ▒█ ░█░ ░▓███▀ ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀ |
| 67 | |
| 68 | 20200707 17:24:06 INF Starting Pigweed build watcher |
| 69 | 20200707 17:24:06 INF Will build [1/1]: out |
| 70 | 20200707 17:24:06 INF Attaching filesystem watcher to $HOME/wrk/pigweed/... |
| 71 | 20200707 17:24:06 INF Triggering initial build... |
| 72 | ... |
| 73 | |
| 74 | #. **Congratulations, you're ready to go!** Now take Pigweed for a spin by |
| 75 | making a test fail. |
| 76 | |
| 77 | #. With the watcher running in a separate window, edit |
| 78 | ``pw_status/status_test.cc`` to make an expectation fail; for example, add |
| 79 | ``EXPECT_EQ(0, 1);`` in a test. |
| 80 | |
| 81 | #. Save the file. Observe the watcher rebuild & retest, and fail. Restore the |
| 82 | test if you feel like it. |
| 83 | |
| 84 | #. Open the generated docs in ``out/docs/gen/docs/html/index.html`` in your |
| 85 | browser. |
| 86 | |
| 87 | #. Edit ``docs/getting_started.rst`` (this file!) and make any change. Save. |
| 88 | See the watcher rebuild the docs. Reload your browser, and see the changes. |
| 89 | |
| 90 | See below for equivalent Windows commands, and for more details on what each |
| 91 | part does. |
| 92 | |
| 93 | **Note:** After running bootstrap once, use ``source ./activate.sh`` (or |
| 94 | ``activate.bat`` on Windows) to re-activate the environment without |
| 95 | re-bootstrapping. |
| 96 | |
| 97 | .. _prerequisites: |
| 98 | |
| 99 | Prerequisites |
| 100 | ------------- |
| 101 | **Linux** |
| 102 | |
| 103 | Most Linux installations should work out of box, and not require any manual |
| 104 | installation of prerequisites beyond basics like ``git`` and |
| 105 | ``build-essential``. Make sure gcc is set to gcc-8. |
| 106 | |
| 107 | **macOS** |
| 108 | |
| 109 | To start using Pigweed on MacOS, you'll need to install XCode. Download it |
| 110 | via the App Store, then install the relevant tools from the command line. |
| 111 | |
| 112 | .. code:: none |
| 113 | |
| 114 | $ xcode-select --install |
| 115 | |
| 116 | On macOS you may get SSL certificate errors with the system Python |
| 117 | installation. Run ``sudo pip install certifi`` to fix this. If you get SSL |
| 118 | errors with the Python from `Homebrew <https://brew.sh>`_ try running the |
| 119 | following commands to ensure Python knows how to use OpenSSL. |
| 120 | |
| 121 | .. code:: none |
| 122 | |
| 123 | $ brew install openssl |
| 124 | $ brew uninstall python |
| 125 | $ brew install python |
| 126 | |
| 127 | To flash firmware to a STM32 Discovery development board (and run ``pw test``) |
| 128 | from macOS, you will need to install OpenOCD. Install |
| 129 | [Homebrew](https://brew.sh), then install OpenOCD with `brew install openocd`. |
| 130 | |
| 131 | **Windows** |
| 132 | |
Wyatt Hepler | c11c5ac | 2021-07-20 17:44:13 -0700 | [diff] [blame] | 133 | To start using Pigweed on Windows, you'll need to do the following: |
| 134 | |
| 135 | * Install `Git <https://git-scm.com/download/win>`_. Git must be installed to |
| 136 | run from the command line and third-party software or be added to ``PATH``. |
| 137 | Also, ensure that the **Enable symbolic links** option is selected. |
| 138 | * Install `Python <https://www.python.org/downloads/windows/>`_. |
| 139 | * Ensure that `Developer Mode |
| 140 | <https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development>`_ |
| 141 | is enabled. |
Keir Mierle | 72cae43 | 2021-04-10 02:03:40 -0700 | [diff] [blame] | 142 | |
| 143 | If you plan to flash devices with firmware, you'll need to install OpenOCD and |
| 144 | ensure it's on your system path. |
| 145 | |
| 146 | Bootstrap |
| 147 | ========= |
| 148 | |
| 149 | Once you satisfied the prerequisites, you will be able to clone Pigweed and |
| 150 | run the bootstrap that initializes the Pigweed virtual environment. The |
| 151 | bootstrap may take several minutes to complete, so please be patient. |
| 152 | |
| 153 | **Linux & macOS** |
| 154 | |
| 155 | .. code:: bash |
| 156 | |
| 157 | $ git clone https://pigweed.googlesource.com/pigweed/pigweed ~/pigweed |
| 158 | $ cd ~/pigweed |
| 159 | $ source ./bootstrap.sh |
| 160 | |
| 161 | **Windows** |
| 162 | |
| 163 | .. code:: batch |
| 164 | |
| 165 | :: Run git commands from the shell you set up to use with Git during install. |
| 166 | > git clone https://pigweed.googlesource.com/pigweed/pigweed %HOMEPATH%\pigweed |
| 167 | > cd %HOMEPATH%\pigweed |
| 168 | > bootstrap.bat |
| 169 | |
| 170 | Below is a real-time demo with roughly what you should expect to see as output: |
| 171 | |
| 172 | .. image:: images/pw_env_setup_demo.gif |
| 173 | :width: 800 |
| 174 | :alt: build example using pw watch |
| 175 | |
| 176 | Congratulations, you are now set up to start using Pigweed! |
| 177 | |
| 178 | Pigweed Environment |
| 179 | =================== |
| 180 | After going through the initial setup process, your current terminal will be in |
| 181 | the Pigweed development environment that provides all the tools you should need |
| 182 | to develop on Pigweed. If you leave that session, you can activate the |
| 183 | environment in a new session with the following command: |
| 184 | |
| 185 | **Linux & macOS** |
| 186 | |
| 187 | .. code:: bash |
| 188 | |
| 189 | $ source ./activate.sh |
| 190 | |
| 191 | **Windows** |
| 192 | |
| 193 | .. code:: batch |
| 194 | |
| 195 | > activate.bat |
| 196 | |
| 197 | Some major changes may require triggering the bootstrap again, so if you run |
| 198 | into host tooling changes after a pull it may be worth re-running bootstrap. |
| 199 | |
| 200 | Build Pigweed for Host |
| 201 | ====================== |
| 202 | Pigweed's primary build system is GN/Ninja based. There are CMake and Bazel |
| 203 | builds in-development, but they are incomplete and don't have feature parity |
| 204 | with the GN build. We strongly recommend you stick to the GN build system. |
| 205 | |
| 206 | GN (Generate Ninja) just does what it says on the tin; GN generates |
| 207 | `Ninja <https://ninja-build.org/>`_ build files. |
| 208 | |
| 209 | The default GN configuration generates build files that allow you to build host |
| 210 | binaries, device binaries, and upstream documentation all in one Ninja |
| 211 | invocation. |
| 212 | |
| 213 | Run GN as seen below: |
| 214 | |
| 215 | .. code:: bash |
| 216 | |
| 217 | $ gn gen out |
| 218 | |
| 219 | Note that ``out`` is simply the directory the build files are saved to. Unless |
| 220 | this directory is deleted or you desire to do a clean build, there's no need to |
| 221 | run GN again; just rebuild using Ninja directly. |
| 222 | |
| 223 | Now that we have build files, it's time to build Pigweed! |
| 224 | |
| 225 | Now you *could* manually invoke the host build using ``ninja -C out`` every |
| 226 | time you make a change, but that's tedious. Instead, let's use ``pw_watch``. |
| 227 | |
| 228 | Go ahead and start ``pw_watch``: |
| 229 | |
| 230 | .. code:: bash |
| 231 | |
| 232 | $ pw watch |
| 233 | |
| 234 | When ``pw_watch`` starts up, it will automatically build the directory we |
| 235 | generated in ``out``. Additionally, ``pw_watch`` watches source code files for |
| 236 | changes, and triggers a Ninja build whenever it notices a file has been saved. |
| 237 | You might be surprised how much time it can save you! |
| 238 | |
| 239 | With ``pw watch`` running, try modifying |
| 240 | ``pw_status/public/pw_status/status.h`` and watch the build re-trigger when you |
| 241 | save the file. |
| 242 | |
| 243 | See below for a demo of this in action: |
| 244 | |
| 245 | .. image:: images/pw_watch_build_demo.gif |
| 246 | :width: 800 |
| 247 | :alt: build example using pw watch |
| 248 | |
| 249 | Running Unit Tests |
| 250 | ================== |
| 251 | Fun fact, you've been running the unit tests already! Ninja builds targeting |
| 252 | the host automatically build and run the unit tests. Unit tests err on the side |
| 253 | of being quiet in the success case, and only output test results when there's a |
| 254 | failure. |
| 255 | |
| 256 | To see the a test failure, modify ``pw_status/status_test.cc`` to fail by |
| 257 | changing one of the strings in the "KnownString" test. |
| 258 | |
| 259 | .. image:: images/pw_watch_test_demo.gif |
| 260 | :width: 800 |
| 261 | :alt: example test failure using pw watch |
| 262 | |
| 263 | Running tests as part of the build isn't particularly expensive because GN |
| 264 | caches passing tests. Each time you build, only the tests that are affected |
| 265 | (whether directly or transitively) by the code changes since the last build |
| 266 | will be re-built and re-run. |
| 267 | |
| 268 | Try running the ``pw_status`` test manually: |
| 269 | |
| 270 | .. code:: bash |
| 271 | |
| 272 | $ ./out/host_{clang,gcc}_debug/obj/pw_status/test/status_test |
| 273 | |
| 274 | Depending on your host OS, the compiler will default to either ``clang`` or |
| 275 | ``gcc``. |
| 276 | |
| 277 | Building for a Device |
| 278 | ===================== |
| 279 | A Pigweed "target" is a build configuration that includes a toolchain, default |
| 280 | library configurations, and more to result in binaries that run natively on the |
| 281 | target. With the default build invocation, you're already building for a device |
| 282 | target (the STMicroelectronics STM32F429I-DISC1) in parallel with the host |
| 283 | build! |
| 284 | |
| 285 | If you want to build JUST for the device, you can kick of watch with: |
| 286 | |
| 287 | .. code:: bash |
| 288 | |
| 289 | $ pw watch stm32f429i |
| 290 | |
| 291 | This is equivalent to the following Ninja invocation: |
| 292 | |
| 293 | .. code:: bash |
| 294 | |
| 295 | $ ninja -C out stm32f429i |
| 296 | |
| 297 | Running Tests on a Device |
| 298 | ========================= |
| 299 | While tests run automatically on the host, it takes a few more steps to get |
| 300 | tests to run automatically on a device, too. Even though we've verified tests |
| 301 | pass on the host, it's crucial to verify the same with on-device testing. We've |
| 302 | encountered some unexpected bugs that can only be found by running the unit |
| 303 | tests directly on the device. |
| 304 | |
| 305 | 1. Connect Device(s) |
| 306 | -------------------- |
| 307 | Connect any number of STM32F429I-DISC1 boards to your computer using the mini |
| 308 | USB port on the board (**not** the micro USB). Pigweed will automatically |
| 309 | detect the boards and distribute the tests across the devices. More boards = |
| 310 | faster tests! Keep in mind that you may have to make some environment specific |
| 311 | updates to ensure you have permissions to use the USB device. For example, on |
| 312 | Linux you may need to update your udev rules and ensure you're in the plugdev |
| 313 | and dialout groups. |
| 314 | |
| 315 | .. image:: images/stm32f429i-disc1_connected.jpg |
| 316 | :width: 800 |
| 317 | :alt: development boards connected via USB |
| 318 | |
| 319 | 2. Launch Test Server |
| 320 | --------------------- |
| 321 | To allow Ninja to run tests on an arbitrary number of devices, Ninja will send |
| 322 | test requests to a server running in the background. Launch the server in |
| 323 | another window using the command below (remember, you'll need to activate the |
| 324 | Pigweed environment first). |
| 325 | |
| 326 | .. code:: bash |
| 327 | |
| 328 | $ stm32f429i_disc1_test_server |
| 329 | |
| 330 | **Note:** If you attach or detach any more boards to your workstation you'll |
| 331 | need to relaunch this server. |
| 332 | |
| 333 | 3. Configure GN |
| 334 | --------------- |
| 335 | Tell GN to use the testing server by enabling a build arg specific to the |
| 336 | stm32f429i-disc1 target. |
| 337 | |
| 338 | .. code:: bash |
| 339 | |
| 340 | $ gn args out |
| 341 | # Append this line to the file that opens in your editor to tell GN to run |
| 342 | # on-device unit tests. |
| 343 | pw_use_test_server = true |
| 344 | |
| 345 | Done! |
| 346 | ----- |
| 347 | Whenever you make code changes and trigger a build, all the affected unit tests |
| 348 | will be run across the attached boards! |
| 349 | |
| 350 | See the demo below for an example of what this all looks like put together: |
| 351 | |
| 352 | .. image:: images/pw_watch_on_device_demo.gif |
| 353 | :width: 800 |
| 354 | :alt: pw watch running on-device tests |
| 355 | |
| 356 | Building the Documentation |
| 357 | ========================== |
| 358 | In addition to the markdown documentation, Pigweed has a collection of |
| 359 | information-rich RST files that are used to generate HTML documentation. All |
| 360 | the docs are hosted at https://pigweed.dev/, and are built as a part of the |
| 361 | default build invocation. This makes it easier to make changes and see how they |
| 362 | turn out. Once built, you can find the rendered HTML documentation at |
| 363 | ``out/docs/gen/docs/html``. |
| 364 | |
| 365 | You can explicitly build just the documentation with the command below. |
| 366 | |
| 367 | .. code:: bash |
| 368 | |
| 369 | $ ninja -C out docs |
| 370 | |
| 371 | This concludes the introduction to developing for upstream Pigweed. |
| 372 | |
| 373 | Next steps |
| 374 | ========== |
| 375 | |
| 376 | Check out other modules |
| 377 | ----------------------- |
| 378 | If you'd like to see more of what Pigweed has to offer, dive into the |
| 379 | :ref:`docs-module-guides`. |
| 380 | |
| 381 | Check out the sample project |
| 382 | ---------------------------- |
| 383 | We have a `sample project |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 384 | <https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md>`_ |
Keir Mierle | 72cae43 | 2021-04-10 02:03:40 -0700 | [diff] [blame] | 385 | that demonstrates how to use Pigweed in your own project. Note that there are |
| 386 | many ways to leverage Pigweed and the sample project is one approach. |
| 387 | |
| 388 | Check out the Hackaday Supercon talk about Pigweed |
| 389 | -------------------------------------------------- |
| 390 | We gave a talk at Hackaday's 2021 supercon, `Give Pigweed a Whirl |
| 391 | <https://hackaday.com/2021/01/13/remoticon-video-pigweed-brings-embedded-unit-testing-library-integration-to-commandline/>`_ |
| 392 | |
| 393 | We've made improvements since we gave the talk; for example, we now have RTOS |
| 394 | primitives. |
| 395 | |
| 396 | Set up Pigweed for your own project |
| 397 | ------------------------------------ |
| 398 | We don't yet have thorough documentation for leveraging Pigweed in a separate |
| 399 | project (our intended use case!). The `sample project |
Rob Mohr | 640c75c | 2021-05-26 07:22:54 -0700 | [diff] [blame] | 400 | <https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md>`_ |
Keir Mierle | 72cae43 | 2021-04-10 02:03:40 -0700 | [diff] [blame] | 401 | shows how to use Pigweed as a library in your broader project, but you may need |
| 402 | further guidance. |
| 403 | |
| 404 | Dropping into our `chat room <https://discord.gg/M9NSeTA>`_ is the most |
| 405 | immediate way to get help. Alternatively, you can send a note to the `mailing |
| 406 | list <https://groups.google.com/forum/#!forum/pigweed>`_. |