Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 1 | Remote Processor Framework |
| 2 | |
| 3 | 1. Introduction |
| 4 | |
| 5 | Modern SoCs typically have heterogeneous remote processor devices in asymmetric |
| 6 | multiprocessing (AMP) configurations, which may be running different instances |
| 7 | of operating system, whether it's Linux or any other flavor of real-time OS. |
| 8 | |
| 9 | OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. |
| 10 | In a typical configuration, the dual cortex-A9 is running Linux in a SMP |
| 11 | configuration, and each of the other three cores (two M3 cores and a DSP) |
| 12 | is running its own instance of RTOS in an AMP configuration. |
| 13 | |
| 14 | The remoteproc framework allows different platforms/architectures to |
| 15 | control (power on, load firmware, power off) those remote processors while |
| 16 | abstracting the hardware differences, so the entire driver doesn't need to be |
| 17 | duplicated. In addition, this framework also adds rpmsg virtio devices |
| 18 | for remote processors that supports this kind of communication. This way, |
| 19 | platform-specific remoteproc drivers only need to provide a few low-level |
| 20 | handlers, and then all rpmsg drivers will then just work |
| 21 | (for more information about the virtio-based rpmsg bus and its drivers, |
| 22 | please read Documentation/rpmsg.txt). |
| 23 | |
| 24 | 2. User API |
| 25 | |
| 26 | int rproc_boot(struct rproc *rproc) |
| 27 | - Boot a remote processor (i.e. load its firmware, power it on, ...). |
| 28 | If the remote processor is already powered on, this function immediately |
| 29 | returns (successfully). |
| 30 | Returns 0 on success, and an appropriate error value otherwise. |
| 31 | Note: to use this function you should already have a valid rproc |
| 32 | handle. There are several ways to achieve that cleanly (devres, pdata, |
| 33 | the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we |
| 34 | might also consider using dev_archdata for this). See also |
| 35 | rproc_get_by_name() below. |
| 36 | |
| 37 | void rproc_shutdown(struct rproc *rproc) |
| 38 | - Power off a remote processor (previously booted with rproc_boot()). |
| 39 | In case @rproc is still being used by an additional user(s), then |
| 40 | this function will just decrement the power refcount and exit, |
| 41 | without really powering off the device. |
| 42 | Every call to rproc_boot() must (eventually) be accompanied by a call |
| 43 | to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. |
| 44 | Notes: |
| 45 | - we're not decrementing the rproc's refcount, only the power refcount. |
| 46 | which means that the @rproc handle stays valid even after |
| 47 | rproc_shutdown() returns, and users can still use it with a subsequent |
| 48 | rproc_boot(), if needed. |
| 49 | - don't call rproc_shutdown() to unroll rproc_get_by_name(), exactly |
| 50 | because rproc_shutdown() _does not_ decrement the refcount of @rproc. |
| 51 | To decrement the refcount of @rproc, use rproc_put() (but _only_ if |
| 52 | you acquired @rproc using rproc_get_by_name()). |
| 53 | |
| 54 | struct rproc *rproc_get_by_name(const char *name) |
| 55 | - Find an rproc handle using the remote processor's name, and then |
| 56 | boot it. If it's already powered on, then just immediately return |
| 57 | (successfully). Returns the rproc handle on success, and NULL on failure. |
| 58 | This function increments the remote processor's refcount, so always |
| 59 | use rproc_put() to decrement it back once rproc isn't needed anymore. |
| 60 | Note: currently rproc_get_by_name() and rproc_put() are not used anymore |
| 61 | by the rpmsg bus and its drivers. We need to scrutinize the use cases |
| 62 | that still need them, and see if we can migrate them to use the non |
| 63 | name-based boot/shutdown interface. |
| 64 | |
| 65 | void rproc_put(struct rproc *rproc) |
| 66 | - Decrement @rproc's power refcount and shut it down if it reaches zero |
| 67 | (essentially by just calling rproc_shutdown), and then decrement @rproc's |
| 68 | validity refcount too. |
| 69 | After this function returns, @rproc may _not_ be used anymore, and its |
| 70 | handle should be considered invalid. |
| 71 | This function should be called _iff_ the @rproc handle was grabbed by |
| 72 | calling rproc_get_by_name(). |
| 73 | |
| 74 | 3. Typical usage |
| 75 | |
| 76 | #include <linux/remoteproc.h> |
| 77 | |
| 78 | /* in case we were given a valid 'rproc' handle */ |
| 79 | int dummy_rproc_example(struct rproc *my_rproc) |
| 80 | { |
| 81 | int ret; |
| 82 | |
| 83 | /* let's power on and boot our remote processor */ |
| 84 | ret = rproc_boot(my_rproc); |
| 85 | if (ret) { |
| 86 | /* |
| 87 | * something went wrong. handle it and leave. |
| 88 | */ |
| 89 | } |
| 90 | |
| 91 | /* |
| 92 | * our remote processor is now powered on... give it some work |
| 93 | */ |
| 94 | |
| 95 | /* let's shut it down now */ |
| 96 | rproc_shutdown(my_rproc); |
| 97 | } |
| 98 | |
| 99 | 4. API for implementors |
| 100 | |
| 101 | struct rproc *rproc_alloc(struct device *dev, const char *name, |
| 102 | const struct rproc_ops *ops, |
| 103 | const char *firmware, int len) |
| 104 | - Allocate a new remote processor handle, but don't register |
| 105 | it yet. Required parameters are the underlying device, the |
| 106 | name of this remote processor, platform-specific ops handlers, |
| 107 | the name of the firmware to boot this rproc with, and the |
| 108 | length of private data needed by the allocating rproc driver (in bytes). |
| 109 | |
| 110 | This function should be used by rproc implementations during |
| 111 | initialization of the remote processor. |
| 112 | After creating an rproc handle using this function, and when ready, |
| 113 | implementations should then call rproc_register() to complete |
| 114 | the registration of the remote processor. |
| 115 | On success, the new rproc is returned, and on failure, NULL. |
| 116 | |
| 117 | Note: _never_ directly deallocate @rproc, even if it was not registered |
| 118 | yet. Instead, if you just need to unroll rproc_alloc(), use rproc_free(). |
| 119 | |
| 120 | void rproc_free(struct rproc *rproc) |
| 121 | - Free an rproc handle that was allocated by rproc_alloc. |
| 122 | This function should _only_ be used if @rproc was only allocated, |
| 123 | but not registered yet. |
| 124 | If @rproc was already successfully registered (by calling |
| 125 | rproc_register()), then use rproc_unregister() instead. |
| 126 | |
| 127 | int rproc_register(struct rproc *rproc) |
| 128 | - Register @rproc with the remoteproc framework, after it has been |
| 129 | allocated with rproc_alloc(). |
| 130 | This is called by the platform-specific rproc implementation, whenever |
| 131 | a new remote processor device is probed. |
| 132 | Returns 0 on success and an appropriate error code otherwise. |
| 133 | Note: this function initiates an asynchronous firmware loading |
| 134 | context, which will look for virtio devices supported by the rproc's |
| 135 | firmware. |
| 136 | If found, those virtio devices will be created and added, so as a result |
| 137 | of registering this remote processor, additional virtio drivers might get |
| 138 | probed. |
| 139 | Currently, though, we only support a single RPMSG virtio vdev per remote |
| 140 | processor. |
| 141 | |
| 142 | int rproc_unregister(struct rproc *rproc) |
| 143 | - Unregister a remote processor, and decrement its refcount. |
| 144 | If its refcount drops to zero, then @rproc will be freed. If not, |
| 145 | it will be freed later once the last reference is dropped. |
| 146 | |
| 147 | This function should be called when the platform specific rproc |
| 148 | implementation decides to remove the rproc device. it should |
| 149 | _only_ be called if a previous invocation of rproc_register() |
| 150 | has completed successfully. |
| 151 | |
| 152 | After rproc_unregister() returns, @rproc is _not_ valid anymore and |
| 153 | it shouldn't be used. More specifically, don't call rproc_free() |
| 154 | or try to directly free @rproc after rproc_unregister() returns; |
| 155 | none of these are needed, and calling them is a bug. |
| 156 | |
| 157 | Returns 0 on success and -EINVAL if @rproc isn't valid. |
| 158 | |
| 159 | 5. Implementation callbacks |
| 160 | |
| 161 | These callbacks should be provided by platform-specific remoteproc |
| 162 | drivers: |
| 163 | |
| 164 | /** |
| 165 | * struct rproc_ops - platform-specific device handlers |
| 166 | * @start: power on the device and boot it |
| 167 | * @stop: power off the device |
| 168 | * @kick: kick a virtqueue (virtqueue id given as a parameter) |
| 169 | */ |
| 170 | struct rproc_ops { |
| 171 | int (*start)(struct rproc *rproc); |
| 172 | int (*stop)(struct rproc *rproc); |
| 173 | void (*kick)(struct rproc *rproc, int vqid); |
| 174 | }; |
| 175 | |
| 176 | Every remoteproc implementation should at least provide the ->start and ->stop |
| 177 | handlers. If rpmsg functionality is also desired, then the ->kick handler |
| 178 | should be provided as well. |
| 179 | |
| 180 | The ->start() handler takes an rproc handle and should then power on the |
| 181 | device and boot it (use rproc->priv to access platform-specific private data). |
| 182 | The boot address, in case needed, can be found in rproc->bootaddr (remoteproc |
| 183 | core puts there the ELF entry point). |
| 184 | On success, 0 should be returned, and on failure, an appropriate error code. |
| 185 | |
| 186 | The ->stop() handler takes an rproc handle and powers the device down. |
| 187 | On success, 0 is returned, and on failure, an appropriate error code. |
| 188 | |
| 189 | The ->kick() handler takes an rproc handle, and an index of a virtqueue |
| 190 | where new message was placed in. Implementations should interrupt the remote |
| 191 | processor and let it know it has pending messages. Notifying remote processors |
| 192 | the exact virtqueue index to look in is optional: it is easy (and not |
| 193 | too expensive) to go through the existing virtqueues and look for new buffers |
| 194 | in the used rings. |
| 195 | |
| 196 | 6. Binary Firmware Structure |
| 197 | |
| 198 | At this point remoteproc only supports ELF32 firmware binaries. However, |
| 199 | it is quite expected that other platforms/devices which we'd want to |
| 200 | support with this framework will be based on different binary formats. |
| 201 | |
| 202 | When those use cases show up, we will have to decouple the binary format |
| 203 | from the framework core, so we can support several binary formats without |
| 204 | duplicating common code. |
| 205 | |
| 206 | When the firmware is parsed, its various segments are loaded to memory |
| 207 | according to the specified device address (might be a physical address |
| 208 | if the remote processor is accessing memory directly). |
| 209 | |
| 210 | In addition to the standard ELF segments, most remote processors would |
| 211 | also include a special section which we call "the resource table". |
| 212 | |
| 213 | The resource table contains system resources that the remote processor |
| 214 | requires before it should be powered on, such as allocation of physically |
| 215 | contiguous memory, or iommu mapping of certain on-chip peripherals. |
| 216 | Remotecore will only power up the device after all the resource table's |
| 217 | requirement are met. |
| 218 | |
| 219 | In addition to system resources, the resource table may also contain |
| 220 | resource entries that publish the existence of supported features |
| 221 | or configurations by the remote processor, such as trace buffers and |
| 222 | supported virtio devices (and their configurations). |
| 223 | |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 224 | The resource table begins with this header: |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 225 | |
| 226 | /** |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 227 | * struct resource_table - firmware resource table header |
| 228 | * @ver: version number |
| 229 | * @num: number of resource entries |
| 230 | * @reserved: reserved (must be zero) |
| 231 | * @offset: array of offsets pointing at the various resource entries |
| 232 | * |
| 233 | * The header of the resource table, as expressed by this structure, |
| 234 | * contains a version number (should we need to change this format in the |
| 235 | * future), the number of available resource entries, and their offsets |
| 236 | * in the table. |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 237 | */ |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 238 | struct resource_table { |
| 239 | u32 ver; |
| 240 | u32 num; |
| 241 | u32 reserved[2]; |
| 242 | u32 offset[0]; |
| 243 | } __packed; |
| 244 | |
| 245 | Immediately following this header are the resource entries themselves, |
| 246 | each of which begins with the following resource entry header: |
| 247 | |
| 248 | /** |
| 249 | * struct fw_rsc_hdr - firmware resource entry header |
| 250 | * @type: resource type |
| 251 | * @data: resource data |
| 252 | * |
| 253 | * Every resource entry begins with a 'struct fw_rsc_hdr' header providing |
| 254 | * its @type. The content of the entry itself will immediately follow |
| 255 | * this header, and it should be parsed according to the resource type. |
| 256 | */ |
| 257 | struct fw_rsc_hdr { |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 258 | u32 type; |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 259 | u8 data[0]; |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 260 | } __packed; |
| 261 | |
| 262 | Some resources entries are mere announcements, where the host is informed |
| 263 | of specific remoteproc configuration. Other entries require the host to |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 264 | do something (e.g. allocate a system resource). Sometimes a negotiation |
| 265 | is expected, where the firmware requests a resource, and once allocated, |
| 266 | the host should provide back its details (e.g. address of an allocated |
| 267 | memory region). |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 268 | |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 269 | Here are the various resource types that are currently supported: |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 270 | |
| 271 | /** |
| 272 | * enum fw_resource_type - types of resource entries |
| 273 | * |
| 274 | * @RSC_CARVEOUT: request for allocation of a physically contiguous |
| 275 | * memory region. |
| 276 | * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. |
| 277 | * @RSC_TRACE: announces the availability of a trace buffer into which |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 278 | * the remote processor will be writing logs. |
| 279 | * @RSC_VDEV: declare support for a virtio device, and serve as its |
| 280 | * virtio header. |
| 281 | * @RSC_LAST: just keep this one at the end |
| 282 | * |
| 283 | * Please note that these values are used as indices to the rproc_handle_rsc |
| 284 | * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to |
| 285 | * check the validity of an index before the lookup table is accessed, so |
| 286 | * please update it as needed. |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 287 | */ |
| 288 | enum fw_resource_type { |
| 289 | RSC_CARVEOUT = 0, |
| 290 | RSC_DEVMEM = 1, |
| 291 | RSC_TRACE = 2, |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 292 | RSC_VDEV = 3, |
| 293 | RSC_LAST = 4, |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 294 | }; |
| 295 | |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 296 | For more details regarding a specific resource type, please see its |
| 297 | dedicated structure in include/linux/remoteproc.h. |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 298 | |
| 299 | We also expect that platform-specific resource entries will show up |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 300 | at some point. When that happens, we could easily add a new RSC_PLATFORM |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 301 | type, and hand those resources to the platform-specific rproc driver to handle. |
| 302 | |
| 303 | 7. Virtio and remoteproc |
| 304 | |
| 305 | The firmware should provide remoteproc information about virtio devices |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 306 | that it supports, and their configurations: a RSC_VDEV resource entry |
| 307 | should specify the virtio device id (as in virtio_ids.h), virtio features, |
| 308 | virtio config space, vrings information, etc. |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 309 | |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 310 | When a new remote processor is registered, the remoteproc framework |
| 311 | will look for its resource table and will register the virtio devices |
| 312 | it supports. A firmware may support any number of virtio devices, and |
| 313 | of any type (a single remote processor can also easily support several |
| 314 | rpmsg virtio devices this way, if desired). |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 315 | |
Ohad Ben-Cohen | fd2c15e | 2012-02-01 21:56:16 +0200 | [diff] [blame] | 316 | Of course, RSC_VDEV resource entries are only good enough for static |
Ohad Ben-Cohen | 400e64d | 2011-10-20 16:52:46 +0200 | [diff] [blame] | 317 | allocation of virtio devices. Dynamic allocations will also be made possible |
| 318 | using the rpmsg bus (similar to how we already do dynamic allocations of |
| 319 | rpmsg channels; read more about it in rpmsg.txt). |