| Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 1 | Chelsio S3 iSCSI Driver for Linux |
| 2 | |
| 3 | Introduction |
| 4 | ============ |
| 5 | |
| 6 | The Chelsio T3 ASIC based Adapters (S310, S320, S302, S304, Mezz cards, etc. |
| 7 | series of products) supports iSCSI acceleration and iSCSI Direct Data Placement |
| 8 | (DDP) where the hardware handles the expensive byte touching operations, such |
| 9 | as CRC computation and verification, and direct DMA to the final host memory |
| 10 | destination: |
| 11 | |
| 12 | - iSCSI PDU digest generation and verification |
| 13 | |
| 14 | On transmitting, Chelsio S3 h/w computes and inserts the Header and |
| 15 | Data digest into the PDUs. |
| 16 | On receiving, Chelsio S3 h/w computes and verifies the Header and |
| 17 | Data digest of the PDUs. |
| 18 | |
| 19 | - Direct Data Placement (DDP) |
| 20 | |
| 21 | S3 h/w can directly place the iSCSI Data-In or Data-Out PDU's |
| 22 | payload into pre-posted final destination host-memory buffers based |
| 23 | on the Initiator Task Tag (ITT) in Data-In or Target Task Tag (TTT) |
| 24 | in Data-Out PDUs. |
| 25 | |
| 26 | - PDU Transmit and Recovery |
| 27 | |
| 28 | On transmitting, S3 h/w accepts the complete PDU (header + data) |
| 29 | from the host driver, computes and inserts the digests, decomposes |
| 30 | the PDU into multiple TCP segments if necessary, and transmit all |
| 31 | the TCP segments onto the wire. It handles TCP retransmission if |
| 32 | needed. |
| 33 | |
| 34 | On receving, S3 h/w recovers the iSCSI PDU by reassembling TCP |
| 35 | segments, separating the header and data, calculating and verifying |
| 36 | the digests, then forwards the header to the host. The payload data, |
| 37 | if possible, will be directly placed into the pre-posted host DDP |
| 38 | buffer. Otherwise, the payload data will be sent to the host too. |
| 39 | |
| 40 | The cxgb3i driver interfaces with open-iscsi initiator and provides the iSCSI |
| 41 | acceleration through Chelsio hardware wherever applicable. |
| 42 | |
| 43 | Using the cxgb3i Driver |
| 44 | ======================= |
| 45 | |
| 46 | The following steps need to be taken to accelerates the open-iscsi initiator: |
| 47 | |
| 48 | 1. Load the cxgb3i driver: "modprobe cxgb3i" |
| 49 | |
| 50 | The cxgb3i module registers a new transport class "cxgb3i" with open-iscsi. |
| 51 | |
| 52 | * in the case of recompiling the kernel, the cxgb3i selection is located at |
| 53 | Device Drivers |
| 54 | SCSI device support ---> |
| 55 | [*] SCSI low-level drivers ---> |
| 56 | <M> Chelsio S3xx iSCSI support |
| 57 | |
| 58 | 2. Create an interface file located under /etc/iscsi/ifaces/ for the new |
| 59 | transport class "cxgb3i". |
| 60 | |
| 61 | The content of the file should be in the following format: |
| 62 | iface.transport_name = cxgb3i |
| 63 | iface.net_ifacename = <ethX> |
| 64 | iface.ipaddress = <iscsi ip address> |
| 65 | |
| 66 | * if iface.ipaddress is specified, <iscsi ip address> needs to be either the |
| 67 | same as the ethX's ip address or an address on the same subnet. Make |
| 68 | sure the ip address is unique in the network. |
| 69 | |
| 70 | 3. edit /etc/iscsi/iscsid.conf |
| 71 | The default setting for MaxRecvDataSegmentLength (131072) is too big, |
| 72 | replace "node.conn[0].iscsi.MaxRecvDataSegmentLength" to be a value no |
| 73 | bigger than 15360 (for example 8192): |
| 74 | |
| 75 | node.conn[0].iscsi.MaxRecvDataSegmentLength = 8192 |
| 76 | |
| 77 | * The login would fail for a normal session if MaxRecvDataSegmentLength is |
| 78 | too big. A error message in the format of |
| 79 | "cxgb3i: ERR! MaxRecvSegmentLength <X> too big. Need to be <= <Y>." |
| 80 | would be logged to dmesg. |
| 81 | |
| 82 | 4. To direct open-iscsi traffic to go through cxgb3i's accelerated path, |
| 83 | "-I <iface file name>" option needs to be specified with most of the |
| 84 | iscsiadm command. <iface file name> is the transport interface file created |
| 85 | in step 2. |