SCST - Infiniband, SRP Target and Ubuntu Lucid

This Wiki will describe how to setup an Infiniband based SRP target on Ubuntu Lucid (10.04).

A Word of Warning

Configuring, designing and implementing an Infiniband based setup is not for the faint-hearted. It will take days or even weeks to understand its quirks, bottlenecks, complexities and you'll even end up doubting yourself why you had chosen Infiniband over 10GB ethernet.

Before we jump ahead

First of all: Check your BIOS settings! You may end up with really bad PCIe performance if not configured properly. Especially server boards tend to be configured very conservatively. "Optimized Defaults" may be your friend but your mileage may vary.

Ubuntu already has the most needed packages from OFED 1.4.2 (openfabrics.org) but sadly lacks some management and diagnostic utilities needed for an Infiniband setup.

You may try installing Debian packages which are also based on OFED 1.4.2: http://pkg-ofed.alioth.debian.org/apt/ofed-1.4.2/

Where do I start?

The kernel module for your Infiniband adapter will already be loaded (mine was ib_mtcha). Now you need to install the corresponding Infiniband-libs:

* ""apt-get install libibcm1 libibcommon1 libibmad1 libibumad1 libibverbs1 librdmacm1 rdmacm-utils ibverbs-utils""

Where do I get the appropriate Ubuntu packages?

You can find patched Kernels and SCST-Admin in this PPA:

https://launchpad.net/~ast/+archive/scst

CategoryDocumentation