Ed Cashin | eecdf22 | 2012-10-04 17:16:39 -0700 | [diff] [blame^] | 1 | The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ... |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 2 | |
Ed Cashin | eecdf22 | 2012-10-04 17:16:39 -0700 | [diff] [blame^] | 3 | http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 4 | |
Ed Cashin | eecdf22 | 2012-10-04 17:16:39 -0700 | [diff] [blame^] | 5 | It has many tips and hints! Please see, especially, recommended |
| 6 | tunings for virtual memory: |
| 7 | |
| 8 | http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19 |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 9 | |
Ed L Cashin | 67d9f84 | 2005-04-29 10:24:18 -0400 | [diff] [blame] | 10 | The aoetools are userland programs that are designed to work with this |
| 11 | driver. The aoetools are on sourceforge. |
| 12 | |
| 13 | http://aoetools.sourceforge.net/ |
| 14 | |
| 15 | The scripts in this Documentation/aoe directory are intended to |
| 16 | document the use of the driver and are not necessary if you install |
| 17 | the aoetools. |
| 18 | |
| 19 | |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 20 | CREATING DEVICE NODES |
| 21 | |
| 22 | Users of udev should find the block device nodes created |
| 23 | automatically, but to create all the necessary device nodes, use the |
| 24 | udev configuration rules provided in udev.txt (in this directory). |
| 25 | |
| 26 | There is a udev-install.sh script that shows how to install these |
| 27 | rules on your system. |
| 28 | |
| 29 | If you are not using udev, two scripts are provided in |
| 30 | Documentation/aoe as examples of static device node creation for |
| 31 | using the aoe driver. |
| 32 | |
| 33 | rm -rf /dev/etherd |
| 34 | sh Documentation/aoe/mkdevs.sh /dev/etherd |
| 35 | |
| 36 | ... or to make just one shelf's worth of block device nodes ... |
| 37 | |
| 38 | sh Documentation/aoe/mkshelf.sh /dev/etherd 0 |
| 39 | |
| 40 | There is also an autoload script that shows how to edit |
Lucas De Marchi | 970e248 | 2012-03-30 13:37:16 -0700 | [diff] [blame] | 41 | /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 42 | necessary. |
| 43 | |
| 44 | USING DEVICE NODES |
| 45 | |
| 46 | "cat /dev/etherd/err" blocks, waiting for error diagnostic output, |
| 47 | like any retransmitted packets. |
| 48 | |
| 49 | "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to |
| 50 | limit ATA over Ethernet traffic to eth2 and eth4. AoE traffic from |
Ed L Cashin | 67d9f84 | 2005-04-29 10:24:18 -0400 | [diff] [blame] | 51 | untrusted networks should be ignored as a matter of security. See |
| 52 | also the aoe_iflist driver option described below. |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 53 | |
| 54 | "echo > /dev/etherd/discover" tells the driver to find out what AoE |
| 55 | devices are available. |
| 56 | |
| 57 | These character devices may disappear and be replaced by sysfs |
Ed L Cashin | 67d9f84 | 2005-04-29 10:24:18 -0400 | [diff] [blame] | 58 | counterparts. Using the commands in aoetools insulates users from |
| 59 | these implementation details. |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 60 | |
| 61 | The block devices are named like this: |
| 62 | |
| 63 | e{shelf}.{slot} |
| 64 | e{shelf}.{slot}p{part} |
| 65 | |
| 66 | ... so that "e0.2" is the third blade from the left (slot 2) in the |
| 67 | first shelf (shelf address zero). That's the whole disk. The first |
| 68 | partition on that disk would be "e0.2p1". |
| 69 | |
| 70 | USING SYSFS |
| 71 | |
| 72 | Each aoe block device in /sys/block has the extra attributes of |
| 73 | state, mac, and netif. The state attribute is "up" when the device |
| 74 | is ready for I/O and "down" if detected but unusable. The |
| 75 | "down,closewait" state shows that the device is still open and |
| 76 | cannot come up again until it has been closed. |
| 77 | |
| 78 | The mac attribute is the ethernet address of the remote AoE device. |
| 79 | The netif attribute is the network interface on the localhost |
| 80 | through which we are communicating with the remote AoE device. |
| 81 | |
| 82 | There is a script in this directory that formats this information |
Ed L Cashin | 67d9f84 | 2005-04-29 10:24:18 -0400 | [diff] [blame] | 83 | in a convenient way. Users with aoetools can use the aoe-stat |
| 84 | command. |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 85 | |
| 86 | root@makki root# sh Documentation/aoe/status.sh |
| 87 | e10.0 eth3 up |
| 88 | e10.1 eth3 up |
| 89 | e10.2 eth3 up |
| 90 | e10.3 eth3 up |
| 91 | e10.4 eth3 up |
| 92 | e10.5 eth3 up |
| 93 | e10.6 eth3 up |
| 94 | e10.7 eth3 up |
| 95 | e10.8 eth3 up |
| 96 | e10.9 eth3 up |
| 97 | e4.0 eth1 up |
| 98 | e4.1 eth1 up |
| 99 | e4.2 eth1 up |
| 100 | e4.3 eth1 up |
| 101 | e4.4 eth1 up |
| 102 | e4.5 eth1 up |
| 103 | e4.6 eth1 up |
| 104 | e4.7 eth1 up |
| 105 | e4.8 eth1 up |
| 106 | e4.9 eth1 up |
Ed L Cashin | 03c41c4 | 2005-04-29 10:24:03 -0400 | [diff] [blame] | 107 | |
| 108 | Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver |
| 109 | option discussed below) instead of /dev/etherd/interfaces to limit |
| 110 | AoE traffic to the network interfaces in the given |
| 111 | whitespace-separated list. Unlike the old character device, the |
| 112 | sysfs entry can be read from as well as written to. |
| 113 | |
| 114 | It's helpful to trigger discovery after setting the list of allowed |
Ed L Cashin | 67d9f84 | 2005-04-29 10:24:18 -0400 | [diff] [blame] | 115 | interfaces. The aoetools package provides an aoe-discover script |
| 116 | for this purpose. You can also directly use the |
| 117 | /dev/etherd/discover special file described above. |
Ed L Cashin | 03c41c4 | 2005-04-29 10:24:03 -0400 | [diff] [blame] | 118 | |
| 119 | DRIVER OPTIONS |
| 120 | |
| 121 | There is a boot option for the built-in aoe driver and a |
| 122 | corresponding module parameter, aoe_iflist. Without this option, |
| 123 | all network interfaces may be used for ATA over Ethernet. Here is a |
| 124 | usage example for the module parameter. |
| 125 | |
| 126 | modprobe aoe_iflist="eth1 eth3" |