Ravinandan Arakali | 571de88 | 2005-11-01 15:24:25 -0500 | [diff] [blame^] | 1 | Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver. |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 2 | |
Ravinandan Arakali | 571de88 | 2005-11-01 15:24:25 -0500 | [diff] [blame^] | 3 | Contents |
| 4 | ======= |
| 5 | - 1. Introduction |
| 6 | - 2. Identifying the adapter/interface |
| 7 | - 3. Features supported |
| 8 | - 4. Command line parameters |
| 9 | - 5. Performance suggestions |
| 10 | - 6. Available Downloads |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 11 | |
Ravinandan Arakali | 571de88 | 2005-11-01 15:24:25 -0500 | [diff] [blame^] | 12 | |
| 13 | 1. Introduction: |
| 14 | This Linux driver supports Neterion's Xframe I PCI-X 1.0 and |
| 15 | Xframe II PCI-X 2.0 adapters. It supports several features |
| 16 | such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. |
| 17 | See below for complete list of features. |
| 18 | All features are supported for both IPv4 and IPv6. |
| 19 | |
| 20 | 2. Identifying the adapter/interface: |
| 21 | a. Insert the adapter(s) in your system. |
| 22 | b. Build and load driver |
| 23 | # insmod s2io.ko |
| 24 | c. View log messages |
| 25 | # dmesg | tail -40 |
| 26 | You will see messages similar to: |
| 27 | eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA |
| 28 | eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA |
| 29 | eth4: Device is on 64 bit 133MHz PCIX(M1) bus |
| 30 | |
| 31 | The above messages identify the adapter type(Xframe I/II), adapter revision, |
| 32 | driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). |
| 33 | In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed |
| 34 | as well. |
| 35 | |
| 36 | To associate an interface with a physical adapter use "ethtool -p <ethX>". |
| 37 | The corresponding adapter's LED will blink multiple times. |
| 38 | |
| 39 | 3. Features supported: |
| 40 | a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes, |
| 41 | modifiable using ifconfig command. |
| 42 | |
| 43 | b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit |
| 44 | and receive, TSO. |
| 45 | |
| 46 | c. Multi-buffer receive mode. Scattering of packet across multiple |
| 47 | buffers. Currently driver supports 2-buffer mode which yields |
| 48 | significant performance improvement on certain platforms(SGI Altix, |
| 49 | IBM xSeries). |
| 50 | |
| 51 | d. MSI/MSI-X. Can be enabled on platforms which support this feature |
| 52 | (IA64, Xeon) resulting in noticeable performance improvement(upto 7% |
| 53 | on certain platforms). |
| 54 | |
| 55 | e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt |
| 56 | moderation. |
| 57 | |
| 58 | f. Statistics. Comprehensive MAC-level and software statistics displayed |
| 59 | using "ethtool -S" option. |
| 60 | |
| 61 | g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, |
| 62 | with multiple steering options. |
| 63 | |
| 64 | 4. Command line parameters |
| 65 | a. tx_fifo_num |
| 66 | Number of transmit queues |
| 67 | Valid range: 1-8 |
| 68 | Default: 1 |
| 69 | |
| 70 | b. rx_ring_num |
| 71 | Number of receive rings |
| 72 | Valid range: 1-8 |
| 73 | Default: 1 |
| 74 | |
| 75 | c. tx_fifo_len |
| 76 | Size of each transmit queue |
| 77 | Valid range: Total length of all queues should not exceed 8192 |
| 78 | Default: 4096 |
| 79 | |
| 80 | d. rx_ring_sz |
| 81 | Size of each receive ring(in 4K blocks) |
| 82 | Valid range: Limited by memory on system |
| 83 | Default: 30 |
| 84 | |
| 85 | e. intr_type |
| 86 | Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X) |
| 87 | Valid range: 1-3 |
| 88 | Default: 1 |
| 89 | |
| 90 | 5. Performance suggestions |
| 91 | General: |
| 92 | a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) |
| 93 | b. Set TCP windows size to optimal value. |
| 94 | For instance, for MTU=1500 a value of 210K has been observed to result in |
| 95 | good performance. |
| 96 | # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000" |
| 97 | # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000" |
| 98 | For MTU=9000, TCP window size of 10 MB is recommended. |
| 99 | # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000" |
| 100 | # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000" |
| 101 | |
| 102 | Transmit performance: |
| 103 | a. By default, the driver respects BIOS settings for PCI bus parameters. |
| 104 | However, you may want to experiment with PCI bus parameters |
| 105 | max-split-transactions(MOST) and MMRBC (use setpci command). |
| 106 | A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. |
| 107 | It could be different for your hardware. |
| 108 | Set MMRBC to 4K**. |
| 109 | |
| 110 | For example you can set |
| 111 | For opteron |
| 112 | #setpci -d 17d5:* 62=1d |
| 113 | For Itanium |
| 114 | #setpci -d 17d5:* 62=3d |
| 115 | |
| 116 | For detailed description of the PCI registers, please see Xframe User Guide. |
| 117 | |
| 118 | b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this |
| 119 | parameter. |
| 120 | c. Turn on TSO(using "ethtool -K") |
| 121 | # ethtool -K <ethX> tso on |
| 122 | |
| 123 | Receive performance: |
| 124 | a. By default, the driver respects BIOS settings for PCI bus parameters. |
| 125 | However, you may want to set PCI latency timer to 248. |
| 126 | #setpci -d 17d5:* LATENCY_TIMER=f8 |
| 127 | For detailed description of the PCI registers, please see Xframe User Guide. |
| 128 | b. Use 2-buffer mode. This results in large performance boost on |
| 129 | on certain platforms(eg. SGI Altix, IBM xSeries). |
| 130 | c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to |
| 131 | set/verify this option. |
| 132 | d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network |
| 133 | device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to |
| 134 | bring down CPU utilization. |
| 135 | |
| 136 | ** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are |
| 137 | recommended as safe parameters. |
| 138 | For more information, please review the AMD8131 errata at |
| 139 | http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf |
| 140 | |
| 141 | 6. Available Downloads |
| 142 | Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up |
| 143 | to date, also the latest "s2io" code (including support for 2.4 kernels) is |
| 144 | available via "Support" link on the Neterion site: http://www.neterion.com. |
| 145 | |
| 146 | For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com, |
| 147 | user: linuxdocs password: HALdocs |
| 148 | |
| 149 | 7. Support |
| 150 | For further support please contact either your 10GbE Xframe NIC vendor (IBM, |
| 151 | HP, SGI etc.) or click on the "Support" link on the Neterion site: |
| 152 | http://www.neterion.com. |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 153 | |