ATA over Ethernet (AoE)

AoE is a low overhead, non-routable storage protocol developed by Coraid.

http://www.coraid.com/

Coraid has a full line of storage products for SAN and virtualization projects.

Starwind also offers products and services with AoE support.

http://www.starwindsoftware.com/

http://en.wikipedia.org/wiki/ATA_over_Ethernet

The AoE protocol is open source and is widely included in various Linux distributions.

There is Windows driver support as well:

• http://winaoe.org/
• http://www.starwindsoftware.com/aoe-ataoverethernet-initiator

http://www.howtoforge.com/low-cost-san

http://www.linuxjournal.com/article/10780

http://www.howtoforge.com/ata_over_ethernet_debian_etch

vblade - user space AoE server kvblade - kernel space server

Performance difference?

Xen Virtualization - Mount AoE storage in Dom0 then share to DomU's

Linux Software RAID 1 to mirror two RAID 5 arrays

Begin discovery of AoE targets:

modprobe aoe

No partition necessary - put filesystem on block device:

/dev/etherd/e<shelfnum>.<slotnum>

mkfs.ext3 /dev/etherd/e0.1

Partitions can be used:

/dev/etherd/e0.3p1

Custom init script to bring up and shut down AoE storage (FAQ)

Admin Console uses port 6936

Install compiler:

yum install gcc gcc-c++ autoconf automake

Install aoetools (domU):

wget http://superb-west.dl.sourceforge.net/sourceforge/aoetools/aoetools-16.tar.gz
make install

Install vblade (dom0) to export storage:

wget http://internap.dl.sourceforge.net/sourceforge/aoetools/vblade-14.tgz
make 
make install

vi /etc/udev/rules.d/60-aoe.rules

# These rules tell udev what device nodes to create for [[aoe]] support.
# They may be installed along the following lines.  Check the section
# 8 udev manpage to see whether your udev supports SUBSYSTEM, and 
# whether it uses one or two equal signs for SUBSYSTEM and KERNEL.

# [[aoe]] char devices
SUBSYSTEM=="[[aoe]]", KERNEL=="discover",   NAME="etherd/%k", GROUP="disk", MODE="0220"
SUBSYSTEM=="[[aoe]]", KERNEL=="err",        NAME="etherd/%k", GROUP="disk", MODE="0440"
SUBSYSTEM=="[[aoe]]", KERNEL=="interfaces", NAME="etherd/%k", GROUP="disk", MODE="0220"
SUBSYSTEM=="[[aoe]]", KERNEL=="revalidate", NAME="etherd/%k", GROUP="disk", MODE="0220"
SUBSYSTEM=="[[aoe]]", KERNEL=="flush",      NAME="etherd/%k", GROUP="disk", MODE="0220"

# [[aoe]] block devices     
KERNEL=="etherd*",       NAME="%k", GROUP="disk"

Install kernel source:

yum install kernel-xen-devel