blob: c71487d399d14688206afbe818be88c093ffde29 [file] [log] [blame]
Ed Cashin5b8c4be2012-10-04 17:16:49 -07001ATA over Ethernet is a network protocol that provides simple access to
2block storage on the LAN.
3
4 http://support.coraid.com/documents/AoEr11.txt
5
Ed Cashineecdf222012-10-04 17:16:39 -07006The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ...
Linus Torvalds1da177e2005-04-16 15:20:36 -07007
Ed Cashineecdf222012-10-04 17:16:39 -07008 http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html
Linus Torvalds1da177e2005-04-16 15:20:36 -07009
Ed Cashineecdf222012-10-04 17:16:39 -070010It has many tips and hints! Please see, especially, recommended
11tunings for virtual memory:
12
13 http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19
Linus Torvalds1da177e2005-04-16 15:20:36 -070014
Ed L Cashin67d9f842005-04-29 10:24:18 -040015The aoetools are userland programs that are designed to work with this
16driver. The aoetools are on sourceforge.
17
18 http://aoetools.sourceforge.net/
19
20The scripts in this Documentation/aoe directory are intended to
21document the use of the driver and are not necessary if you install
22the aoetools.
23
24
Linus Torvalds1da177e2005-04-16 15:20:36 -070025CREATING DEVICE NODES
26
27 Users of udev should find the block device nodes created
28 automatically, but to create all the necessary device nodes, use the
29 udev configuration rules provided in udev.txt (in this directory).
30
31 There is a udev-install.sh script that shows how to install these
32 rules on your system.
33
Linus Torvalds1da177e2005-04-16 15:20:36 -070034 There is also an autoload script that shows how to edit
Lucas De Marchi970e2482012-03-30 13:37:16 -070035 /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when
Ed Cashin5b8c4be2012-10-04 17:16:49 -070036 necessary. Preloading the aoe module is preferable to autoloading,
37 however, because AoE discovery takes a few seconds. It can be
38 confusing when an AoE device is not present the first time the a
39 command is run but appears a second later.
Linus Torvalds1da177e2005-04-16 15:20:36 -070040
41USING DEVICE NODES
42
43 "cat /dev/etherd/err" blocks, waiting for error diagnostic output,
44 like any retransmitted packets.
45
46 "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to
47 limit ATA over Ethernet traffic to eth2 and eth4. AoE traffic from
Ed L Cashin67d9f842005-04-29 10:24:18 -040048 untrusted networks should be ignored as a matter of security. See
49 also the aoe_iflist driver option described below.
Linus Torvalds1da177e2005-04-16 15:20:36 -070050
51 "echo > /dev/etherd/discover" tells the driver to find out what AoE
52 devices are available.
53
Ed Cashin5b8c4be2012-10-04 17:16:49 -070054 In the future these character devices may disappear and be replaced
55 by sysfs counterparts. Using the commands in aoetools insulates
56 users from these implementation details.
Linus Torvalds1da177e2005-04-16 15:20:36 -070057
58 The block devices are named like this:
59
60 e{shelf}.{slot}
61 e{shelf}.{slot}p{part}
62
63 ... so that "e0.2" is the third blade from the left (slot 2) in the
64 first shelf (shelf address zero). That's the whole disk. The first
65 partition on that disk would be "e0.2p1".
66
67USING SYSFS
68
69 Each aoe block device in /sys/block has the extra attributes of
70 state, mac, and netif. The state attribute is "up" when the device
71 is ready for I/O and "down" if detected but unusable. The
72 "down,closewait" state shows that the device is still open and
73 cannot come up again until it has been closed.
74
75 The mac attribute is the ethernet address of the remote AoE device.
76 The netif attribute is the network interface on the localhost
77 through which we are communicating with the remote AoE device.
78
Ed Cashin5b8c4be2012-10-04 17:16:49 -070079 There is a script in this directory that formats this information in
80 a convenient way. Users with aoetools should use the aoe-stat
Ed L Cashin67d9f842005-04-29 10:24:18 -040081 command.
Linus Torvalds1da177e2005-04-16 15:20:36 -070082
83 root@makki root# sh Documentation/aoe/status.sh
84 e10.0 eth3 up
85 e10.1 eth3 up
86 e10.2 eth3 up
87 e10.3 eth3 up
88 e10.4 eth3 up
89 e10.5 eth3 up
90 e10.6 eth3 up
91 e10.7 eth3 up
92 e10.8 eth3 up
93 e10.9 eth3 up
94 e4.0 eth1 up
95 e4.1 eth1 up
96 e4.2 eth1 up
97 e4.3 eth1 up
98 e4.4 eth1 up
99 e4.5 eth1 up
100 e4.6 eth1 up
101 e4.7 eth1 up
102 e4.8 eth1 up
103 e4.9 eth1 up
Ed L Cashin03c41c42005-04-29 10:24:03 -0400104
105 Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver
106 option discussed below) instead of /dev/etherd/interfaces to limit
107 AoE traffic to the network interfaces in the given
108 whitespace-separated list. Unlike the old character device, the
109 sysfs entry can be read from as well as written to.
110
111 It's helpful to trigger discovery after setting the list of allowed
Ed L Cashin67d9f842005-04-29 10:24:18 -0400112 interfaces. The aoetools package provides an aoe-discover script
113 for this purpose. You can also directly use the
114 /dev/etherd/discover special file described above.
Ed L Cashin03c41c42005-04-29 10:24:03 -0400115
116DRIVER OPTIONS
117
118 There is a boot option for the built-in aoe driver and a
119 corresponding module parameter, aoe_iflist. Without this option,
120 all network interfaces may be used for ATA over Ethernet. Here is a
121 usage example for the module parameter.
122
123 modprobe aoe_iflist="eth1 eth3"
Ed Cashin5b8c4be2012-10-04 17:16:49 -0700124
125 The aoe_deadsecs module parameter determines the maximum number of
126 seconds that the driver will wait for an AoE device to provide a
127 response to an AoE command. After aoe_deadsecs seconds have
Ed Cashinc450ba02012-12-17 16:04:14 -0800128 elapsed, the AoE device will be marked as "down". A value of zero
129 is supported for testing purposes and makes the aoe driver keep
130 trying AoE commands forever.
Ed Cashin5b8c4be2012-10-04 17:16:49 -0700131
132 The aoe_maxout module parameter has a default of 128. This is the
133 maximum number of unresponded packets that will be sent to an AoE
134 target at one time.
135
136 The aoe_dyndevs module parameter defaults to 1, meaning that the
137 driver will assign a block device minor number to a discovered AoE
138 target based on the order of its discovery. With dynamic minor
139 device numbers in use, a greater range of AoE shelf and slot
140 addresses can be supported. Users with udev will never have to
141 think about minor numbers. Using aoe_dyndevs=0 allows device nodes
142 to be pre-created using a static minor-number scheme with the
143 aoe-mkshelf script in the aoetools.