Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 1 | ================================= |
| 2 | HOWTO interact with BPF subsystem |
| 3 | ================================= |
| 4 | |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 5 | This document provides information for the BPF subsystem about various |
| 6 | workflows related to reporting bugs, submitting patches, and queueing |
| 7 | patches for stable kernels. |
| 8 | |
| 9 | For general information about submitting patches, please refer to |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 10 | `Documentation/process/`_. This document only describes additional specifics |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 11 | related to BPF. |
| 12 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 13 | .. contents:: |
| 14 | :local: |
| 15 | :depth: 2 |
| 16 | |
| 17 | Reporting bugs |
| 18 | ============== |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 19 | |
| 20 | Q: How do I report bugs for BPF kernel code? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 21 | -------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 22 | A: Since all BPF kernel development as well as bpftool and iproute2 BPF |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 23 | loader development happens through the netdev kernel mailing list, |
| 24 | please report any found issues around BPF to the following mailing |
| 25 | list: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 26 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 27 | netdev@vger.kernel.org |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 28 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 29 | This may also include issues related to XDP, BPF tracing, etc. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 30 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 31 | Given netdev has a high volume of traffic, please also add the BPF |
| 32 | maintainers to Cc (from kernel MAINTAINERS_ file): |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 33 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 34 | * Alexei Starovoitov <ast@kernel.org> |
| 35 | * Daniel Borkmann <daniel@iogearbox.net> |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 36 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 37 | In case a buggy commit has already been identified, make sure to keep |
| 38 | the actual commit authors in Cc as well for the report. They can |
| 39 | typically be identified through the kernel's git tree. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 40 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 41 | **Please do NOT report BPF issues to bugzilla.kernel.org since it |
| 42 | is a guarantee that the reported issue will be overlooked.** |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 43 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 44 | Submitting patches |
| 45 | ================== |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 46 | |
| 47 | Q: To which mailing list do I need to submit my BPF patches? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 48 | ------------------------------------------------------------ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 49 | A: Please submit your BPF patches to the netdev kernel mailing list: |
| 50 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 51 | netdev@vger.kernel.org |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 52 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 53 | Historically, BPF came out of networking and has always been maintained |
| 54 | by the kernel networking community. Although these days BPF touches |
| 55 | many other subsystems as well, the patches are still routed mainly |
| 56 | through the networking community. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 57 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 58 | In case your patch has changes in various different subsystems (e.g. |
| 59 | tracing, security, etc), make sure to Cc the related kernel mailing |
| 60 | lists and maintainers from there as well, so they are able to review |
| 61 | the changes and provide their Acked-by's to the patches. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 62 | |
| 63 | Q: Where can I find patches currently under discussion for BPF subsystem? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 64 | ------------------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 65 | A: All patches that are Cc'ed to netdev are queued for review under netdev |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 66 | patchwork project: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 67 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 68 | http://patchwork.ozlabs.org/project/netdev/list/ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 69 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 70 | Those patches which target BPF, are assigned to a 'bpf' delegate for |
| 71 | further processing from BPF maintainers. The current queue with |
| 72 | patches under review can be found at: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 73 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 74 | https://patchwork.ozlabs.org/project/netdev/list/?delegate=77147 |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 75 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 76 | Once the patches have been reviewed by the BPF community as a whole |
| 77 | and approved by the BPF maintainers, their status in patchwork will be |
| 78 | changed to 'Accepted' and the submitter will be notified by mail. This |
| 79 | means that the patches look good from a BPF perspective and have been |
| 80 | applied to one of the two BPF kernel trees. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 81 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 82 | In case feedback from the community requires a respin of the patches, |
| 83 | their status in patchwork will be set to 'Changes Requested', and purged |
| 84 | from the current review queue. Likewise for cases where patches would |
| 85 | get rejected or are not applicable to the BPF trees (but assigned to |
| 86 | the 'bpf' delegate). |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 87 | |
| 88 | Q: How do the changes make their way into Linux? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 89 | ------------------------------------------------ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 90 | A: There are two BPF kernel trees (git repositories). Once patches have |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 91 | been accepted by the BPF maintainers, they will be applied to one |
| 92 | of the two BPF trees: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 93 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 94 | * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/ |
| 95 | * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 96 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 97 | The bpf tree itself is for fixes only, whereas bpf-next for features, |
| 98 | cleanups or other kind of improvements ("next-like" content). This is |
| 99 | analogous to net and net-next trees for networking. Both bpf and |
| 100 | bpf-next will only have a master branch in order to simplify against |
| 101 | which branch patches should get rebased to. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 102 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 103 | Accumulated BPF patches in the bpf tree will regularly get pulled |
| 104 | into the net kernel tree. Likewise, accumulated BPF patches accepted |
| 105 | into the bpf-next tree will make their way into net-next tree. net and |
| 106 | net-next are both run by David S. Miller. From there, they will go |
| 107 | into the kernel mainline tree run by Linus Torvalds. To read up on the |
| 108 | process of net and net-next being merged into the mainline tree, see |
| 109 | the `netdev FAQ`_ under: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 110 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 111 | `Documentation/networking/netdev-FAQ.txt`_ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 112 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 113 | Occasionally, to prevent merge conflicts, we might send pull requests |
| 114 | to other trees (e.g. tracing) with a small subset of the patches, but |
| 115 | net and net-next are always the main trees targeted for integration. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 116 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 117 | The pull requests will contain a high-level summary of the accumulated |
| 118 | patches and can be searched on netdev kernel mailing list through the |
| 119 | following subject lines (``yyyy-mm-dd`` is the date of the pull |
| 120 | request):: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 121 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 122 | pull-request: bpf yyyy-mm-dd |
| 123 | pull-request: bpf-next yyyy-mm-dd |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 124 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 125 | Q: How do I indicate which tree (bpf vs. bpf-next) my patch should be applied to? |
| 126 | --------------------------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 127 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 128 | A: The process is the very same as described in the `netdev FAQ`_, so |
| 129 | please read up on it. The subject line must indicate whether the |
| 130 | patch is a fix or rather "next-like" content in order to let the |
| 131 | maintainers know whether it is targeted at bpf or bpf-next. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 132 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 133 | For fixes eventually landing in bpf -> net tree, the subject must |
| 134 | look like:: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 135 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 136 | git format-patch --subject-prefix='PATCH bpf' start..finish |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 137 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 138 | For features/improvements/etc that should eventually land in |
| 139 | bpf-next -> net-next, the subject must look like:: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 140 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 141 | git format-patch --subject-prefix='PATCH bpf-next' start..finish |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 142 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 143 | If unsure whether the patch or patch series should go into bpf |
| 144 | or net directly, or bpf-next or net-next directly, it is not a |
| 145 | problem either if the subject line says net or net-next as target. |
| 146 | It is eventually up to the maintainers to do the delegation of |
| 147 | the patches. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 148 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 149 | If it is clear that patches should go into bpf or bpf-next tree, |
| 150 | please make sure to rebase the patches against those trees in |
| 151 | order to reduce potential conflicts. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 152 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 153 | In case the patch or patch series has to be reworked and sent out |
| 154 | again in a second or later revision, it is also required to add a |
| 155 | version number (``v2``, ``v3``, ...) into the subject prefix:: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 156 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 157 | git format-patch --subject-prefix='PATCH net-next v2' start..finish |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 158 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 159 | When changes have been requested to the patch series, always send the |
| 160 | whole patch series again with the feedback incorporated (never send |
| 161 | individual diffs on top of the old series). |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 162 | |
| 163 | Q: What does it mean when a patch gets applied to bpf or bpf-next tree? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 164 | ----------------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 165 | A: It means that the patch looks good for mainline inclusion from |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 166 | a BPF point of view. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 167 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 168 | Be aware that this is not a final verdict that the patch will |
| 169 | automatically get accepted into net or net-next trees eventually: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 170 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 171 | On the netdev kernel mailing list reviews can come in at any point |
| 172 | in time. If discussions around a patch conclude that they cannot |
| 173 | get included as-is, we will either apply a follow-up fix or drop |
| 174 | them from the trees entirely. Therefore, we also reserve to rebase |
| 175 | the trees when deemed necessary. After all, the purpose of the tree |
| 176 | is to: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 177 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 178 | i) accumulate and stage BPF patches for integration into trees |
| 179 | like net and net-next, and |
| 180 | |
| 181 | ii) run extensive BPF test suite and |
| 182 | workloads on the patches before they make their way any further. |
| 183 | |
| 184 | Once the BPF pull request was accepted by David S. Miller, then |
| 185 | the patches end up in net or net-next tree, respectively, and |
| 186 | make their way from there further into mainline. Again, see the |
| 187 | `netdev FAQ`_ for additional information e.g. on how often they are |
| 188 | merged to mainline. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 189 | |
| 190 | Q: How long do I need to wait for feedback on my BPF patches? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 191 | ------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 192 | A: We try to keep the latency low. The usual time to feedback will |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 193 | be around 2 or 3 business days. It may vary depending on the |
| 194 | complexity of changes and current patch load. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 195 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 196 | Q: How often do you send pull requests to major kernel trees like net or net-next? |
| 197 | ---------------------------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 198 | |
| 199 | A: Pull requests will be sent out rather often in order to not |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 200 | accumulate too many patches in bpf or bpf-next. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 201 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 202 | As a rule of thumb, expect pull requests for each tree regularly |
| 203 | at the end of the week. In some cases pull requests could additionally |
| 204 | come also in the middle of the week depending on the current patch |
| 205 | load or urgency. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 206 | |
| 207 | Q: Are patches applied to bpf-next when the merge window is open? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 208 | ----------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 209 | A: For the time when the merge window is open, bpf-next will not be |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 210 | processed. This is roughly analogous to net-next patch processing, |
| 211 | so feel free to read up on the `netdev FAQ`_ about further details. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 212 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 213 | During those two weeks of merge window, we might ask you to resend |
| 214 | your patch series once bpf-next is open again. Once Linus released |
| 215 | a ``v*-rc1`` after the merge window, we continue processing of bpf-next. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 216 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 217 | For non-subscribers to kernel mailing lists, there is also a status |
| 218 | page run by David S. Miller on net-next that provides guidance: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 219 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 220 | http://vger.kernel.org/~davem/net-next.html |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 221 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 222 | Q: Verifier changes and test cases |
| 223 | ---------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 224 | Q: I made a BPF verifier change, do I need to add test cases for |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 225 | BPF kernel selftests_? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 226 | |
| 227 | A: If the patch has changes to the behavior of the verifier, then yes, |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 228 | it is absolutely necessary to add test cases to the BPF kernel |
| 229 | selftests_ suite. If they are not present and we think they are |
| 230 | needed, then we might ask for them before accepting any changes. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 231 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 232 | In particular, test_verifier.c is tracking a high number of BPF test |
| 233 | cases, including a lot of corner cases that LLVM BPF back end may |
| 234 | generate out of the restricted C code. Thus, adding test cases is |
| 235 | absolutely crucial to make sure future changes do not accidentally |
| 236 | affect prior use-cases. Thus, treat those test cases as: verifier |
| 237 | behavior that is not tracked in test_verifier.c could potentially |
| 238 | be subject to change. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 239 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 240 | Q: samples/bpf preference vs selftests? |
| 241 | --------------------------------------- |
| 242 | Q: When should I add code to `samples/bpf/`_ and when to BPF kernel |
| 243 | selftests_ ? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 244 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 245 | A: In general, we prefer additions to BPF kernel selftests_ rather than |
| 246 | `samples/bpf/`_. The rationale is very simple: kernel selftests are |
| 247 | regularly run by various bots to test for kernel regressions. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 248 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 249 | The more test cases we add to BPF selftests, the better the coverage |
| 250 | and the less likely it is that those could accidentally break. It is |
| 251 | not that BPF kernel selftests cannot demo how a specific feature can |
| 252 | be used. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 253 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 254 | That said, `samples/bpf/`_ may be a good place for people to get started, |
| 255 | so it might be advisable that simple demos of features could go into |
| 256 | `samples/bpf/`_, but advanced functional and corner-case testing rather |
| 257 | into kernel selftests. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 258 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 259 | If your sample looks like a test case, then go for BPF kernel selftests |
| 260 | instead! |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 261 | |
| 262 | Q: When should I add code to the bpftool? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 263 | ----------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 264 | A: The main purpose of bpftool (under tools/bpf/bpftool/) is to provide |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 265 | a central user space tool for debugging and introspection of BPF programs |
| 266 | and maps that are active in the kernel. If UAPI changes related to BPF |
| 267 | enable for dumping additional information of programs or maps, then |
| 268 | bpftool should be extended as well to support dumping them. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 269 | |
| 270 | Q: When should I add code to iproute2's BPF loader? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 271 | --------------------------------------------------- |
| 272 | A: For UAPI changes related to the XDP or tc layer (e.g. ``cls_bpf``), |
| 273 | the convention is that those control-path related changes are added to |
| 274 | iproute2's BPF loader as well from user space side. This is not only |
| 275 | useful to have UAPI changes properly designed to be usable, but also |
| 276 | to make those changes available to a wider user base of major |
| 277 | downstream distributions. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 278 | |
| 279 | Q: Do you accept patches as well for iproute2's BPF loader? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 280 | ----------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 281 | A: Patches for the iproute2's BPF loader have to be sent to: |
| 282 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 283 | netdev@vger.kernel.org |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 284 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 285 | While those patches are not processed by the BPF kernel maintainers, |
| 286 | please keep them in Cc as well, so they can be reviewed. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 287 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 288 | The official git repository for iproute2 is run by Stephen Hemminger |
| 289 | and can be found at: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 290 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 291 | https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 292 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 293 | The patches need to have a subject prefix of '``[PATCH iproute2 |
| 294 | master]``' or '``[PATCH iproute2 net-next]``'. '``master``' or |
| 295 | '``net-next``' describes the target branch where the patch should be |
| 296 | applied to. Meaning, if kernel changes went into the net-next kernel |
| 297 | tree, then the related iproute2 changes need to go into the iproute2 |
| 298 | net-next branch, otherwise they can be targeted at master branch. The |
| 299 | iproute2 net-next branch will get merged into the master branch after |
| 300 | the current iproute2 version from master has been released. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 301 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 302 | Like BPF, the patches end up in patchwork under the netdev project and |
| 303 | are delegated to 'shemminger' for further processing: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 304 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 305 | http://patchwork.ozlabs.org/project/netdev/list/?delegate=389 |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 306 | |
| 307 | Q: What is the minimum requirement before I submit my BPF patches? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 308 | ------------------------------------------------------------------ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 309 | A: When submitting patches, always take the time and properly test your |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 310 | patches *prior* to submission. Never rush them! If maintainers find |
| 311 | that your patches have not been properly tested, it is a good way to |
| 312 | get them grumpy. Testing patch submissions is a hard requirement! |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 313 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 314 | Note, fixes that go to bpf tree *must* have a ``Fixes:`` tag included. |
| 315 | The same applies to fixes that target bpf-next, where the affected |
| 316 | commit is in net-next (or in some cases bpf-next). The ``Fixes:`` tag is |
| 317 | crucial in order to identify follow-up commits and tremendously helps |
| 318 | for people having to do backporting, so it is a must have! |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 319 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 320 | We also don't accept patches with an empty commit message. Take your |
| 321 | time and properly write up a high quality commit message, it is |
| 322 | essential! |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 323 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 324 | Think about it this way: other developers looking at your code a month |
| 325 | from now need to understand *why* a certain change has been done that |
| 326 | way, and whether there have been flaws in the analysis or assumptions |
| 327 | that the original author did. Thus providing a proper rationale and |
| 328 | describing the use-case for the changes is a must. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 329 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 330 | Patch submissions with >1 patch must have a cover letter which includes |
| 331 | a high level description of the series. This high level summary will |
| 332 | then be placed into the merge commit by the BPF maintainers such that |
| 333 | it is also accessible from the git log for future reference. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 334 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 335 | Q: Features changing BPF JIT and/or LLVM |
| 336 | ---------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 337 | Q: What do I need to consider when adding a new instruction or feature |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 338 | that would require BPF JIT and/or LLVM integration as well? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 339 | |
| 340 | A: We try hard to keep all BPF JITs up to date such that the same user |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 341 | experience can be guaranteed when running BPF programs on different |
| 342 | architectures without having the program punt to the less efficient |
| 343 | interpreter in case the in-kernel BPF JIT is enabled. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 344 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 345 | If you are unable to implement or test the required JIT changes for |
| 346 | certain architectures, please work together with the related BPF JIT |
| 347 | developers in order to get the feature implemented in a timely manner. |
| 348 | Please refer to the git log (``arch/*/net/``) to locate the necessary |
| 349 | people for helping out. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 350 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 351 | Also always make sure to add BPF test cases (e.g. test_bpf.c and |
| 352 | test_verifier.c) for new instructions, so that they can receive |
| 353 | broad test coverage and help run-time testing the various BPF JITs. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 354 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 355 | In case of new BPF instructions, once the changes have been accepted |
| 356 | into the Linux kernel, please implement support into LLVM's BPF back |
| 357 | end. See LLVM_ section below for further information. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 358 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 359 | Stable submission |
| 360 | ================= |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 361 | |
| 362 | Q: I need a specific BPF commit in stable kernels. What should I do? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 363 | -------------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 364 | A: In case you need a specific fix in stable kernels, first check whether |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 365 | the commit has already been applied in the related ``linux-*.y`` branches: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 366 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 367 | https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 368 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 369 | If not the case, then drop an email to the BPF maintainers with the |
| 370 | netdev kernel mailing list in Cc and ask for the fix to be queued up: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 371 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 372 | netdev@vger.kernel.org |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 373 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 374 | The process in general is the same as on netdev itself, see also the |
| 375 | `netdev FAQ`_ document. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 376 | |
| 377 | Q: Do you also backport to kernels not currently maintained as stable? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 378 | ---------------------------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 379 | A: No. If you need a specific BPF commit in kernels that are currently not |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 380 | maintained by the stable maintainers, then you are on your own. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 381 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 382 | The current stable and longterm stable kernels are all listed here: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 383 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 384 | https://www.kernel.org/ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 385 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 386 | Q: The BPF patch I am about to submit needs to go to stable as well |
| 387 | ------------------------------------------------------------------- |
| 388 | What should I do? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 389 | |
| 390 | A: The same rules apply as with netdev patch submissions in general, see |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 391 | `netdev FAQ`_ under: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 392 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 393 | `Documentation/networking/netdev-FAQ.txt`_ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 394 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 395 | Never add "``Cc: stable@vger.kernel.org``" to the patch description, but |
| 396 | ask the BPF maintainers to queue the patches instead. This can be done |
| 397 | with a note, for example, under the ``---`` part of the patch which does |
| 398 | not go into the git log. Alternatively, this can be done as a simple |
| 399 | request by mail instead. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 400 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 401 | Q: Queue stable patches |
| 402 | ----------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 403 | Q: Where do I find currently queued BPF patches that will be submitted |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 404 | to stable? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 405 | |
| 406 | A: Once patches that fix critical bugs got applied into the bpf tree, they |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 407 | are queued up for stable submission under: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 408 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 409 | http://patchwork.ozlabs.org/bundle/bpf/stable/?state=* |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 410 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 411 | They will be on hold there at minimum until the related commit made its |
| 412 | way into the mainline kernel tree. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 413 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 414 | After having been under broader exposure, the queued patches will be |
| 415 | submitted by the BPF maintainers to the stable maintainers. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 416 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 417 | Testing patches |
| 418 | =============== |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 419 | |
Jesper Dangaard Brouer | b7a27c3 | 2018-05-14 15:42:32 +0200 | [diff] [blame] | 420 | Q: How to run BPF selftests |
| 421 | --------------------------- |
| 422 | A: After you have booted into the newly compiled kernel, navigate to |
| 423 | the BPF selftests_ suite in order to test BPF functionality (current |
| 424 | working directory points to the root of the cloned git tree):: |
| 425 | |
| 426 | $ cd tools/testing/selftests/bpf/ |
| 427 | $ make |
| 428 | |
| 429 | To run the verifier tests:: |
| 430 | |
| 431 | $ sudo ./test_verifier |
| 432 | |
| 433 | The verifier tests print out all the current checks being |
| 434 | performed. The summary at the end of running all tests will dump |
| 435 | information of test successes and failures:: |
| 436 | |
| 437 | Summary: 418 PASSED, 0 FAILED |
| 438 | |
| 439 | In order to run through all BPF selftests, the following command is |
| 440 | needed:: |
| 441 | |
| 442 | $ sudo make run_tests |
| 443 | |
| 444 | See the kernels selftest `Documentation/dev-tools/kselftest.rst`_ |
| 445 | document for further documentation. |
| 446 | |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 447 | Q: Which BPF kernel selftests version should I run my kernel against? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 448 | --------------------------------------------------------------------- |
| 449 | A: If you run a kernel ``xyz``, then always run the BPF kernel selftests |
| 450 | from that kernel ``xyz`` as well. Do not expect that the BPF selftest |
| 451 | from the latest mainline tree will pass all the time. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 452 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 453 | In particular, test_bpf.c and test_verifier.c have a large number of |
| 454 | test cases and are constantly updated with new BPF test sequences, or |
| 455 | existing ones are adapted to verifier changes e.g. due to verifier |
| 456 | becoming smarter and being able to better track certain things. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 457 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 458 | LLVM |
| 459 | ==== |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 460 | |
| 461 | Q: Where do I find LLVM with BPF support? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 462 | ----------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 463 | A: The BPF back end for LLVM is upstream in LLVM since version 3.7.1. |
| 464 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 465 | All major distributions these days ship LLVM with BPF back end enabled, |
| 466 | so for the majority of use-cases it is not required to compile LLVM by |
| 467 | hand anymore, just install the distribution provided package. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 468 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 469 | LLVM's static compiler lists the supported targets through |
| 470 | ``llc --version``, make sure BPF targets are listed. Example:: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 471 | |
| 472 | $ llc --version |
| 473 | LLVM (http://llvm.org/): |
| 474 | LLVM version 6.0.0svn |
| 475 | Optimized build. |
| 476 | Default target: x86_64-unknown-linux-gnu |
| 477 | Host CPU: skylake |
| 478 | |
| 479 | Registered Targets: |
| 480 | bpf - BPF (host endian) |
| 481 | bpfeb - BPF (big endian) |
| 482 | bpfel - BPF (little endian) |
| 483 | x86 - 32-bit X86: Pentium-Pro and above |
| 484 | x86-64 - 64-bit X86: EM64T and AMD64 |
| 485 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 486 | For developers in order to utilize the latest features added to LLVM's |
| 487 | BPF back end, it is advisable to run the latest LLVM releases. Support |
| 488 | for new BPF kernel features such as additions to the BPF instruction |
| 489 | set are often developed together. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 490 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 491 | All LLVM releases can be found at: http://releases.llvm.org/ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 492 | |
| 493 | Q: Got it, so how do I build LLVM manually anyway? |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 494 | -------------------------------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 495 | A: You need cmake and gcc-c++ as build requisites for LLVM. Once you have |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 496 | that set up, proceed with building the latest LLVM and clang version |
| 497 | from the git repositories:: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 498 | |
| 499 | $ git clone http://llvm.org/git/llvm.git |
| 500 | $ cd llvm/tools |
| 501 | $ git clone --depth 1 http://llvm.org/git/clang.git |
| 502 | $ cd ..; mkdir build; cd build |
| 503 | $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86" \ |
| 504 | -DBUILD_SHARED_LIBS=OFF \ |
| 505 | -DCMAKE_BUILD_TYPE=Release \ |
| 506 | -DLLVM_BUILD_RUNTIME=OFF |
| 507 | $ make -j $(getconf _NPROCESSORS_ONLN) |
| 508 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 509 | The built binaries can then be found in the build/bin/ directory, where |
| 510 | you can point the PATH variable to. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 511 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 512 | Q: Reporting LLVM BPF issues |
| 513 | ---------------------------- |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 514 | Q: Should I notify BPF kernel maintainers about issues in LLVM's BPF code |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 515 | generation back end or about LLVM generated code that the verifier |
| 516 | refuses to accept? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 517 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 518 | A: Yes, please do! |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 519 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 520 | LLVM's BPF back end is a key piece of the whole BPF |
| 521 | infrastructure and it ties deeply into verification of programs from the |
| 522 | kernel side. Therefore, any issues on either side need to be investigated |
| 523 | and fixed whenever necessary. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 524 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 525 | Therefore, please make sure to bring them up at netdev kernel mailing |
| 526 | list and Cc BPF maintainers for LLVM and kernel bits: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 527 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 528 | * Yonghong Song <yhs@fb.com> |
| 529 | * Alexei Starovoitov <ast@kernel.org> |
| 530 | * Daniel Borkmann <daniel@iogearbox.net> |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 531 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 532 | LLVM also has an issue tracker where BPF related bugs can be found: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 533 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 534 | https://bugs.llvm.org/buglist.cgi?quicksearch=bpf |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 535 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 536 | However, it is better to reach out through mailing lists with having |
| 537 | maintainers in Cc. |
| 538 | |
| 539 | Q: New BPF instruction for kernel and LLVM |
| 540 | ------------------------------------------ |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 541 | Q: I have added a new BPF instruction to the kernel, how can I integrate |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 542 | it into LLVM? |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 543 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 544 | A: LLVM has a ``-mcpu`` selector for the BPF back end in order to allow |
| 545 | the selection of BPF instruction set extensions. By default the |
| 546 | ``generic`` processor target is used, which is the base instruction set |
| 547 | (v1) of BPF. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 548 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 549 | LLVM has an option to select ``-mcpu=probe`` where it will probe the host |
| 550 | kernel for supported BPF instruction set extensions and selects the |
| 551 | optimal set automatically. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 552 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 553 | For cross-compilation, a specific version can be select manually as well :: |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 554 | |
| 555 | $ llc -march bpf -mcpu=help |
| 556 | Available CPUs for this target: |
| 557 | |
| 558 | generic - Select the generic processor. |
| 559 | probe - Select the probe processor. |
| 560 | v1 - Select the v1 processor. |
| 561 | v2 - Select the v2 processor. |
| 562 | [...] |
| 563 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 564 | Newly added BPF instructions to the Linux kernel need to follow the same |
| 565 | scheme, bump the instruction set version and implement probing for the |
| 566 | extensions such that ``-mcpu=probe`` users can benefit from the |
| 567 | optimization transparently when upgrading their kernels. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 568 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 569 | If you are unable to implement support for the newly added BPF instruction |
| 570 | please reach out to BPF developers for help. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 571 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 572 | By the way, the BPF kernel selftests run with ``-mcpu=probe`` for better |
| 573 | test coverage. |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 574 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 575 | Q: clang flag for target bpf? |
| 576 | ----------------------------- |
| 577 | Q: In some cases clang flag ``-target bpf`` is used but in other cases the |
| 578 | default clang target, which matches the underlying architecture, is used. |
| 579 | What is the difference and when I should use which? |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 580 | |
| 581 | A: Although LLVM IR generation and optimization try to stay architecture |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 582 | independent, ``-target <arch>`` still has some impact on generated code: |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 583 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 584 | - BPF program may recursively include header file(s) with file scope |
| 585 | inline assembly codes. The default target can handle this well, |
| 586 | while ``bpf`` target may fail if bpf backend assembler does not |
| 587 | understand these assembly codes, which is true in most cases. |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 588 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 589 | - When compiled without ``-g``, additional elf sections, e.g., |
| 590 | .eh_frame and .rela.eh_frame, may be present in the object file |
| 591 | with default target, but not with ``bpf`` target. |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 592 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 593 | - The default target may turn a C switch statement into a switch table |
| 594 | lookup and jump operation. Since the switch table is placed |
| 595 | in the global readonly section, the bpf program will fail to load. |
| 596 | The bpf target does not support switch table optimization. |
| 597 | The clang option ``-fno-jump-tables`` can be used to disable |
| 598 | switch table generation. |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 599 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 600 | - For clang ``-target bpf``, it is guaranteed that pointer or long / |
| 601 | unsigned long types will always have a width of 64 bit, no matter |
| 602 | whether underlying clang binary or default target (or kernel) is |
| 603 | 32 bit. However, when native clang target is used, then it will |
| 604 | compile these types based on the underlying architecture's conventions, |
| 605 | meaning in case of 32 bit architecture, pointer or long / unsigned |
| 606 | long types e.g. in BPF context structure will have width of 32 bit |
| 607 | while the BPF LLVM back end still operates in 64 bit. The native |
| 608 | target is mostly needed in tracing for the case of walking ``pt_regs`` |
| 609 | or other kernel structures where CPU's register width matters. |
| 610 | Otherwise, ``clang -target bpf`` is generally recommended. |
Daniel Borkmann | 78262f4 | 2018-03-20 00:21:15 +0100 | [diff] [blame] | 611 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 612 | You should use default target when: |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 613 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 614 | - Your program includes a header file, e.g., ptrace.h, which eventually |
| 615 | pulls in some header files containing file scope host assembly codes. |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 616 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 617 | - You can add ``-fno-jump-tables`` to work around the switch table issue. |
John Fastabend | 514d6c1 | 2018-04-23 12:11:02 -0700 | [diff] [blame] | 618 | |
Jesper Dangaard Brouer | 5422283 | 2018-05-14 15:42:27 +0200 | [diff] [blame] | 619 | Otherwise, you can use ``bpf`` target. Additionally, you *must* use bpf target |
| 620 | when: |
| 621 | |
| 622 | - Your program uses data structures with pointer or long / unsigned long |
| 623 | types that interface with BPF helpers or context data structures. Access |
| 624 | into these structures is verified by the BPF verifier and may result |
| 625 | in verification failures if the native architecture is not aligned with |
| 626 | the BPF architecture, e.g. 64-bit. An example of this is |
| 627 | BPF_PROG_TYPE_SK_MSG require ``-target bpf`` |
| 628 | |
| 629 | |
| 630 | .. Links |
| 631 | .. _Documentation/process/: https://www.kernel.org/doc/html/latest/process/ |
| 632 | .. _MAINTAINERS: ../../MAINTAINERS |
| 633 | .. _Documentation/networking/netdev-FAQ.txt: ../networking/netdev-FAQ.txt |
| 634 | .. _netdev FAQ: ../networking/netdev-FAQ.txt |
| 635 | .. _samples/bpf/: ../../samples/bpf/ |
| 636 | .. _selftests: ../../tools/testing/selftests/bpf/ |
Jesper Dangaard Brouer | b7a27c3 | 2018-05-14 15:42:32 +0200 | [diff] [blame] | 637 | .. _Documentation/dev-tools/kselftest.rst: |
| 638 | https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html |
Yonghong Song | 6215ea6 | 2018-02-01 23:00:11 -0800 | [diff] [blame] | 639 | |
Daniel Borkmann | 34f15bf | 2017-12-06 01:12:41 +0100 | [diff] [blame] | 640 | Happy BPF hacking! |