Srinivas Pandruvada | 17e2adf | 2016-08-07 02:25:33 -0700 | [diff] [blame] | 1 | Intel Integrated Sensor Hub (ISH) |
| 2 | =============================== |
| 3 | |
| 4 | A sensor hub enables the ability to offload sensor polling and algorithm |
| 5 | processing to a dedicated low power co-processor. This allows the core |
| 6 | processor to go into low power modes more often, resulting in the increased |
| 7 | battery life. |
| 8 | |
| 9 | There are many vendors providing external sensor hubs confirming to HID |
| 10 | Sensor usage tables, and used in several tablets, 2 in 1 convertible laptops |
| 11 | and embedded products. Linux had this support since Linux 3.9. |
| 12 | |
| 13 | Intel® introduced integrated sensor hubs as a part of the SoC starting from |
| 14 | Cherry Trail and now supported on multiple generations of CPU packages. There |
| 15 | are many commercial devices already shipped with Integrated Sensor Hubs (ISH). |
| 16 | These ISH also comply to HID sensor specification, but the difference is the |
| 17 | transport protocol used for communication. The current external sensor hubs |
| 18 | mainly use HID over i2C or USB. But ISH doesn't use either i2c or USB. |
| 19 | |
| 20 | 1. Overview |
| 21 | |
| 22 | Using a analogy with a usbhid implementation, the ISH follows a similar model |
| 23 | for a very high speed communication: |
| 24 | |
| 25 | ----------------- ---------------------- |
| 26 | | USB HID | --> | ISH HID | |
| 27 | ----------------- ---------------------- |
| 28 | ----------------- ---------------------- |
| 29 | | USB protocol | --> | ISH Transport | |
| 30 | ----------------- ---------------------- |
| 31 | ----------------- ---------------------- |
| 32 | | EHCI/XHCI | --> | ISH IPC | |
| 33 | ----------------- ---------------------- |
| 34 | PCI PCI |
| 35 | ----------------- ---------------------- |
| 36 | |Host controller| --> | ISH processor | |
| 37 | ----------------- ---------------------- |
| 38 | USB Link |
| 39 | ----------------- ---------------------- |
| 40 | | USB End points| --> | ISH Clients | |
| 41 | ----------------- ---------------------- |
| 42 | |
| 43 | Like USB protocol provides a method for device enumeration, link management |
| 44 | and user data encapsulation, the ISH also provides similar services. But it is |
| 45 | very light weight tailored to manage and communicate with ISH client |
| 46 | applications implemented in the firmware. |
| 47 | |
| 48 | The ISH allows multiple sensor management applications executing in the |
| 49 | firmware. Like USB endpoints the messaging can be to/from a client. As part of |
| 50 | enumeration process, these clients are identified. These clients can be simple |
| 51 | HID sensor applications, sensor calibration application or senor firmware |
| 52 | update application. |
| 53 | |
| 54 | The implementation model is similar, like USB bus, ISH transport is also |
| 55 | implemented as a bus. Each client application executing in the ISH processor |
| 56 | is registered as a device on this bus. The driver, which binds each device |
| 57 | (ISH HID driver) identifies the device type and registers with the hid core. |
| 58 | |
| 59 | 2. ISH Implementation: Block Diagram |
| 60 | |
| 61 | --------------------------- |
| 62 | | User Space Applications | |
| 63 | --------------------------- |
| 64 | |
| 65 | ----------------IIO ABI---------------- |
| 66 | -------------------------- |
| 67 | | IIO Sensor Drivers | |
| 68 | -------------------------- |
| 69 | -------------------------- |
| 70 | | IIO core | |
| 71 | -------------------------- |
| 72 | -------------------------- |
| 73 | | HID Sensor Hub MFD | |
| 74 | -------------------------- |
| 75 | -------------------------- |
| 76 | | HID Core | |
| 77 | -------------------------- |
| 78 | -------------------------- |
| 79 | | HID over ISH Client | |
| 80 | -------------------------- |
| 81 | -------------------------- |
| 82 | | ISH Transport (ISHTP) | |
| 83 | -------------------------- |
| 84 | -------------------------- |
| 85 | | IPC Drivers | |
| 86 | -------------------------- |
| 87 | OS |
| 88 | ---------------- PCI ----------------- |
| 89 | Hardware + Firmware |
| 90 | ---------------------------- |
| 91 | | ISH Hardware/Firmware(FW) | |
| 92 | ---------------------------- |
| 93 | |
| 94 | 3. High level processing in above blocks |
| 95 | |
| 96 | 3.1 Hardware Interface |
| 97 | |
| 98 | The ISH is exposed as "Non-VGA unclassified PCI device" to the host. The PCI |
| 99 | product and vendor IDs are changed from different generations of processors. So |
| 100 | the source code which enumerate drivers needs to update from generation to |
| 101 | generation. |
| 102 | |
| 103 | 3.2 Inter Processor Communication (IPC) driver |
| 104 | Location: drivers/hid/intel-ish-hid/ipc |
| 105 | |
| 106 | The IPC message used memory mapped I/O. The registers are defined in |
| 107 | hw-ish-regs.h. |
| 108 | |
| 109 | 3.2.1 IPC/FW message types |
| 110 | |
| 111 | There are two types of messages, one for management of link and other messages |
| 112 | are to and from transport layers. |
| 113 | |
| 114 | TX and RX of Transport messages |
| 115 | |
| 116 | A set of memory mapped register offers support of multi byte messages TX and |
| 117 | RX (E.g.IPC_REG_ISH2HOST_MSG, IPC_REG_HOST2ISH_MSG). The IPC layer maintains |
| 118 | internal queues to sequence messages and send them in order to the FW. |
| 119 | Optionally the caller can register handler to get notification of completion. |
| 120 | A door bell mechanism is used in messaging to trigger processing in host and |
| 121 | client firmware side. When ISH interrupt handler is called, the ISH2HOST |
| 122 | doorbell register is used by host drivers to determine that the interrupt |
| 123 | is for ISH. |
| 124 | |
| 125 | Each side has 32 32-bit message registers and a 32-bit doorbell. Doorbell |
| 126 | register has the following format: |
| 127 | Bits 0..6: fragment length (7 bits are used) |
| 128 | Bits 10..13: encapsulated protocol |
| 129 | Bits 16..19: management command (for IPC management protocol) |
| 130 | Bit 31: doorbell trigger (signal H/W interrupt to the other side) |
| 131 | Other bits are reserved, should be 0. |
| 132 | |
| 133 | 3.2.2 Transport layer interface |
| 134 | |
| 135 | To abstract HW level IPC communication, a set of callbacks are registered. |
| 136 | The transport layer uses them to send and receive messages. |
| 137 | Refer to struct ishtp_hw_ops for callbacks. |
| 138 | |
| 139 | 3.3 ISH Transport layer |
| 140 | Location: drivers/hid/intel-ish-hid/ishtp/ |
| 141 | |
| 142 | 3.3.1 A Generic Transport Layer |
| 143 | |
| 144 | The transport layer is a bi-directional protocol, which defines: |
| 145 | - Set of commands to start, stop, connect, disconnect and flow control |
| 146 | (ishtp/hbm.h) for details |
| 147 | - A flow control mechanism to avoid buffer overflows |
| 148 | |
| 149 | This protocol resembles bus messages described in the following document: |
| 150 | http://www.intel.com/content/dam/www/public/us/en/documents/technical-\ |
| 151 | specifications/dcmi-hi-1-0-spec.pdf "Chapter 7: Bus Message Layer" |
| 152 | |
| 153 | 3.3.2 Connection and Flow Control Mechanism |
| 154 | |
| 155 | Each FW client and a protocol is identified by an UUID. In order to communicate |
| 156 | to a FW client, a connection must be established using connect request and |
| 157 | response bus messages. If successful, a pair (host_client_id and fw_client_id) |
| 158 | will identify the connection. |
| 159 | |
| 160 | Once connection is established, peers send each other flow control bus messages |
| 161 | independently. Every peer may send a message only if it has received a |
| 162 | flow-control credit before. Once it sent a message, it may not send another one |
| 163 | before receiving the next flow control credit. |
| 164 | Either side can send disconnect request bus message to end communication. Also |
| 165 | the link will be dropped if major FW reset occurs. |
| 166 | |
| 167 | 3.3.3 Peer to Peer data transfer |
| 168 | |
| 169 | Peer to Peer data transfer can happen with or without using DMA. Depending on |
| 170 | the sensor bandwidth requirement DMA can be enabled by using module parameter |
| 171 | ishtp_use_dma under intel_ishtp. |
| 172 | |
| 173 | Each side (host and FW) manages its DMA transfer memory independently. When an |
| 174 | ISHTP client from either host or FW side wants to send something, it decides |
| 175 | whether to send over IPC or over DMA; for each transfer the decision is |
| 176 | independent. The sending side sends DMA_XFER message when the message is in |
| 177 | the respective host buffer (TX when host client sends, RX when FW client |
| 178 | sends). The recipient of DMA message responds with DMA_XFER_ACK, indicating |
| 179 | the sender that the memory region for that message may be reused. |
| 180 | |
| 181 | DMA initialization is started with host sending DMA_ALLOC_NOTIFY bus message |
| 182 | (that includes RX buffer) and FW responds with DMA_ALLOC_NOTIFY_ACK. |
| 183 | Additionally to DMA address communication, this sequence checks capabilities: |
| 184 | if thw host doesn't support DMA, then it won't send DMA allocation, so FW can't |
| 185 | send DMA; if FW doesn't support DMA then it won't respond with |
| 186 | DMA_ALLOC_NOTIFY_ACK, in which case host will not use DMA transfers. |
| 187 | Here ISH acts as busmaster DMA controller. Hence when host sends DMA_XFER, |
| 188 | it's request to do host->ISH DMA transfer; when FW sends DMA_XFER, it means |
| 189 | that it already did DMA and the message resides at host. Thus, DMA_XFER |
| 190 | and DMA_XFER_ACK act as ownership indicators. |
| 191 | |
| 192 | At initial state all outgoing memory belongs to the sender (TX to host, RX to |
| 193 | FW), DMA_XFER transfers ownership on the region that contains ISHTP message to |
| 194 | the receiving side, DMA_XFER_ACK returns ownership to the sender. A sender |
| 195 | needs not wait for previous DMA_XFER to be ack'ed, and may send another message |
| 196 | as long as remaining continuous memory in its ownership is enough. |
| 197 | In principle, multiple DMA_XFER and DMA_XFER_ACK messages may be sent at once |
| 198 | (up to IPC MTU), thus allowing for interrupt throttling. |
| 199 | Currently, ISH FW decides to send over DMA if ISHTP message is more than 3 IPC |
| 200 | fragments and via IPC otherwise. |
| 201 | |
| 202 | 3.3.4 Ring Buffers |
| 203 | |
| 204 | When a client initiate a connection, a ring or RX and TX buffers are allocated. |
| 205 | The size of ring can be specified by the client. HID client set 16 and 32 for |
| 206 | TX and RX buffers respectively. On send request from client, the data to be |
| 207 | sent is copied to one of the send ring buffer and scheduled to be sent using |
| 208 | bus message protocol. These buffers are required because the FW may have not |
| 209 | have processed the last message and may not have enough flow control credits |
| 210 | to send. Same thing holds true on receive side and flow control is required. |
| 211 | |
| 212 | 3.3.5 Host Enumeration |
| 213 | |
| 214 | The host enumeration bus command allow discovery of clients present in the FW. |
| 215 | There can be multiple sensor clients and clients for calibration function. |
| 216 | |
| 217 | To ease in implantation and allow independent driver handle each client |
| 218 | this transport layer takes advantage of Linux Bus driver model. Each |
| 219 | client is registered as device on the the transport bus (ishtp bus). |
| 220 | |
| 221 | Enumeration sequence of messages: |
| 222 | - Host sends HOST_START_REQ_CMD, indicating that host ISHTP layer is up. |
| 223 | - FW responds with HOST_START_RES_CMD |
| 224 | - Host sends HOST_ENUM_REQ_CMD (enumerate FW clients) |
| 225 | - FW responds with HOST_ENUM_RES_CMD that includes bitmap of available FW |
| 226 | client IDs |
| 227 | - For each FW ID found in that bitmap host sends |
| 228 | HOST_CLIENT_PROPERTIES_REQ_CMD |
| 229 | - FW responds with HOST_CLIENT_PROPERTIES_RES_CMD. Properties include UUID, |
| 230 | max ISHTP message size, etc. |
| 231 | - Once host received properties for that last discovered client, it considers |
| 232 | ISHTP device fully functional (and allocates DMA buffers) |
| 233 | |
| 234 | 3.4 HID over ISH Client |
| 235 | Location: drivers/hid/intel-ish-hid |
| 236 | |
| 237 | The ISHTP client driver is responsible for: |
| 238 | - enumerate HID devices under FW ISH client |
| 239 | - Get Report descriptor |
| 240 | - Register with HID core as a LL driver |
| 241 | - Process Get/Set feature request |
| 242 | - Get input reports |
| 243 | |
| 244 | 3.5 HID Sensor Hub MFD and IIO sensor drivers |
| 245 | |
| 246 | The functionality in these drivers is the same as an external sensor hub. |
| 247 | Refer to |
| 248 | Documentation/hid/hid-sensor.txt for HID sensor |
| 249 | Documentation/ABI/testing/sysfs-bus-iio for IIO ABIs to user space |
| 250 | |
| 251 | 3.6 End to End HID transport Sequence Diagram |
| 252 | |
| 253 | HID-ISH-CLN ISHTP IPC HW |
| 254 | | | | | |
| 255 | | | |-----WAKE UP------------------>| |
| 256 | | | | | |
| 257 | | | |-----HOST READY--------------->| |
| 258 | | | | | |
| 259 | | | |<----MNG_RESET_NOTIFY_ACK----- | |
| 260 | | | | | |
| 261 | | |<----ISHTP_START------ | | |
| 262 | | | | | |
| 263 | | |<-----------------HOST_START_RES_CMD-------------------| |
| 264 | | | | | |
| 265 | | |------------------QUERY_SUBSCRIBER-------------------->| |
| 266 | | | | | |
| 267 | | |------------------HOST_ENUM_REQ_CMD------------------->| |
| 268 | | | | | |
| 269 | | |<-----------------HOST_ENUM_RES_CMD--------------------| |
| 270 | | | | | |
| 271 | | |------------------HOST_CLIENT_PROPERTIES_REQ_CMD------>| |
| 272 | | | | | |
| 273 | | |<-----------------HOST_CLIENT_PROPERTIES_RES_CMD-------| |
| 274 | | Create new device on in ishtp bus | | |
| 275 | | | | | |
| 276 | | |------------------HOST_CLIENT_PROPERTIES_REQ_CMD------>| |
| 277 | | | | | |
| 278 | | |<-----------------HOST_CLIENT_PROPERTIES_RES_CMD-------| |
| 279 | | Create new device on in ishtp bus | | |
| 280 | | | | | |
| 281 | | |--Repeat HOST_CLIENT_PROPERTIES_REQ_CMD-till last one--| |
| 282 | | | | | |
| 283 | probed() |
| 284 | |----ishtp_cl_connect-->|----------------- CLIENT_CONNECT_REQ_CMD-------------->| |
| 285 | | | | | |
| 286 | | |<----------------CLIENT_CONNECT_RES_CMD----------------| |
| 287 | | | | | |
| 288 | |register event callback| | | |
| 289 | | | | | |
| 290 | |ishtp_cl_send( |
| 291 | HOSTIF_DM_ENUM_DEVICES) |----------fill ishtp_msg_hdr struct write to HW----- >| |
| 292 | | | | | |
| 293 | | | |<-----IRQ(IPC_PROTOCOL_ISHTP---| |
| 294 | | | | | |
| 295 | |<--ENUM_DEVICE RSP-----| | | |
| 296 | | | | | |
| 297 | for each enumerated device |
| 298 | |ishtp_cl_send( |
| 299 | HOSTIF_GET_HID_DESCRIPTOR |----------fill ishtp_msg_hdr struct write to HW--- >| |
| 300 | | | | | |
| 301 | ...Response |
| 302 | | | | | |
| 303 | for each enumerated device |
| 304 | |ishtp_cl_send( |
| 305 | HOSTIF_GET_REPORT_DESCRIPTOR |----------fill ishtp_msg_hdr struct write to HW- >| |
| 306 | | | | | |
| 307 | | | | | |
| 308 | hid_allocate_device |
| 309 | | | | | |
| 310 | hid_add_device | | | |
| 311 | | | | | |
| 312 | |
| 313 | |
| 314 | 3.7 ISH Debugging |
| 315 | |
| 316 | To debug ISH, event tracing mechanism is used. To enable debug logs |
| 317 | echo 1 > /sys/kernel/debug/tracing/events/intel_ish/enable |
| 318 | cat sys/kernel/debug/tracing/trace |
| 319 | |
| 320 | 3.8 ISH IIO sysfs Example on Lenovo thinkpad Yoga 260 |
| 321 | |
| 322 | root@otcpl-ThinkPad-Yoga-260:~# tree -l /sys/bus/iio/devices/ |
| 323 | /sys/bus/iio/devices/ |
| 324 | ├── iio:device0 -> ../../../devices/0044:8086:22D8.0001/HID-SENSOR-200073.9.auto/iio:device0 |
| 325 | │ ├── buffer |
| 326 | │ │ ├── enable |
| 327 | │ │ ├── length |
| 328 | │ │ └── watermark |
| 329 | ... |
| 330 | │ ├── in_accel_hysteresis |
| 331 | │ ├── in_accel_offset |
| 332 | │ ├── in_accel_sampling_frequency |
| 333 | │ ├── in_accel_scale |
| 334 | │ ├── in_accel_x_raw |
| 335 | │ ├── in_accel_y_raw |
| 336 | │ ├── in_accel_z_raw |
| 337 | │ ├── name |
| 338 | │ ├── scan_elements |
| 339 | │ │ ├── in_accel_x_en |
| 340 | │ │ ├── in_accel_x_index |
| 341 | │ │ ├── in_accel_x_type |
| 342 | │ │ ├── in_accel_y_en |
| 343 | │ │ ├── in_accel_y_index |
| 344 | │ │ ├── in_accel_y_type |
| 345 | │ │ ├── in_accel_z_en |
| 346 | │ │ ├── in_accel_z_index |
| 347 | │ │ └── in_accel_z_type |
| 348 | ... |
| 349 | │ │ ├── devices |
| 350 | │ │ │ │ ├── buffer |
| 351 | │ │ │ │ │ ├── enable |
| 352 | │ │ │ │ │ ├── length |
| 353 | │ │ │ │ │ └── watermark |
| 354 | │ │ │ │ ├── dev |
| 355 | │ │ │ │ ├── in_intensity_both_raw |
| 356 | │ │ │ │ ├── in_intensity_hysteresis |
| 357 | │ │ │ │ ├── in_intensity_offset |
| 358 | │ │ │ │ ├── in_intensity_sampling_frequency |
| 359 | │ │ │ │ ├── in_intensity_scale |
| 360 | │ │ │ │ ├── name |
| 361 | │ │ │ │ ├── scan_elements |
| 362 | │ │ │ │ │ ├── in_intensity_both_en |
| 363 | │ │ │ │ │ ├── in_intensity_both_index |
| 364 | │ │ │ │ │ └── in_intensity_both_type |
| 365 | │ │ │ │ ├── trigger |
| 366 | │ │ │ │ │ └── current_trigger |
| 367 | ... |
| 368 | │ │ │ │ ├── buffer |
| 369 | │ │ │ │ │ ├── enable |
| 370 | │ │ │ │ │ ├── length |
| 371 | │ │ │ │ │ └── watermark |
| 372 | │ │ │ │ ├── dev |
| 373 | │ │ │ │ ├── in_magn_hysteresis |
| 374 | │ │ │ │ ├── in_magn_offset |
| 375 | │ │ │ │ ├── in_magn_sampling_frequency |
| 376 | │ │ │ │ ├── in_magn_scale |
| 377 | │ │ │ │ ├── in_magn_x_raw |
| 378 | │ │ │ │ ├── in_magn_y_raw |
| 379 | │ │ │ │ ├── in_magn_z_raw |
| 380 | │ │ │ │ ├── in_rot_from_north_magnetic_tilt_comp_raw |
| 381 | │ │ │ │ ├── in_rot_hysteresis |
| 382 | │ │ │ │ ├── in_rot_offset |
| 383 | │ │ │ │ ├── in_rot_sampling_frequency |
| 384 | │ │ │ │ ├── in_rot_scale |
| 385 | │ │ │ │ ├── name |
| 386 | ... |
| 387 | │ │ │ │ ├── scan_elements |
| 388 | │ │ │ │ │ ├── in_magn_x_en |
| 389 | │ │ │ │ │ ├── in_magn_x_index |
| 390 | │ │ │ │ │ ├── in_magn_x_type |
| 391 | │ │ │ │ │ ├── in_magn_y_en |
| 392 | │ │ │ │ │ ├── in_magn_y_index |
| 393 | │ │ │ │ │ ├── in_magn_y_type |
| 394 | │ │ │ │ │ ├── in_magn_z_en |
| 395 | │ │ │ │ │ ├── in_magn_z_index |
| 396 | │ │ │ │ │ ├── in_magn_z_type |
| 397 | │ │ │ │ │ ├── in_rot_from_north_magnetic_tilt_comp_en |
| 398 | │ │ │ │ │ ├── in_rot_from_north_magnetic_tilt_comp_index |
| 399 | │ │ │ │ │ └── in_rot_from_north_magnetic_tilt_comp_type |
| 400 | │ │ │ │ ├── trigger |
| 401 | │ │ │ │ │ └── current_trigger |
| 402 | ... |
| 403 | │ │ │ │ ├── buffer |
| 404 | │ │ │ │ │ ├── enable |
| 405 | │ │ │ │ │ ├── length |
| 406 | │ │ │ │ │ └── watermark |
| 407 | │ │ │ │ ├── dev |
| 408 | │ │ │ │ ├── in_anglvel_hysteresis |
| 409 | │ │ │ │ ├── in_anglvel_offset |
| 410 | │ │ │ │ ├── in_anglvel_sampling_frequency |
| 411 | │ │ │ │ ├── in_anglvel_scale |
| 412 | │ │ │ │ ├── in_anglvel_x_raw |
| 413 | │ │ │ │ ├── in_anglvel_y_raw |
| 414 | │ │ │ │ ├── in_anglvel_z_raw |
| 415 | │ │ │ │ ├── name |
| 416 | │ │ │ │ ├── scan_elements |
| 417 | │ │ │ │ │ ├── in_anglvel_x_en |
| 418 | │ │ │ │ │ ├── in_anglvel_x_index |
| 419 | │ │ │ │ │ ├── in_anglvel_x_type |
| 420 | │ │ │ │ │ ├── in_anglvel_y_en |
| 421 | │ │ │ │ │ ├── in_anglvel_y_index |
| 422 | │ │ │ │ │ ├── in_anglvel_y_type |
| 423 | │ │ │ │ │ ├── in_anglvel_z_en |
| 424 | │ │ │ │ │ ├── in_anglvel_z_index |
| 425 | │ │ │ │ │ └── in_anglvel_z_type |
| 426 | │ │ │ │ ├── trigger |
| 427 | │ │ │ │ │ └── current_trigger |
| 428 | ... |
| 429 | │ │ │ │ ├── buffer |
| 430 | │ │ │ │ │ ├── enable |
| 431 | │ │ │ │ │ ├── length |
| 432 | │ │ │ │ │ └── watermark |
| 433 | │ │ │ │ ├── dev |
| 434 | │ │ │ │ ├── in_anglvel_hysteresis |
| 435 | │ │ │ │ ├── in_anglvel_offset |
| 436 | │ │ │ │ ├── in_anglvel_sampling_frequency |
| 437 | │ │ │ │ ├── in_anglvel_scale |
| 438 | │ │ │ │ ├── in_anglvel_x_raw |
| 439 | │ │ │ │ ├── in_anglvel_y_raw |
| 440 | │ │ │ │ ├── in_anglvel_z_raw |
| 441 | │ │ │ │ ├── name |
| 442 | │ │ │ │ ├── scan_elements |
| 443 | │ │ │ │ │ ├── in_anglvel_x_en |
| 444 | │ │ │ │ │ ├── in_anglvel_x_index |
| 445 | │ │ │ │ │ ├── in_anglvel_x_type |
| 446 | │ │ │ │ │ ├── in_anglvel_y_en |
| 447 | │ │ │ │ │ ├── in_anglvel_y_index |
| 448 | │ │ │ │ │ ├── in_anglvel_y_type |
| 449 | │ │ │ │ │ ├── in_anglvel_z_en |
| 450 | │ │ │ │ │ ├── in_anglvel_z_index |
| 451 | │ │ │ │ │ └── in_anglvel_z_type |
| 452 | │ │ │ │ ├── trigger |
| 453 | │ │ │ │ │ └── current_trigger |
| 454 | ... |