iSCSI on AIX

iSCSI (Internet Small Computer Systems Interface) is a transport protocol for sending SCSI packets over TCP/IP networks. iSCSI initiators and iSCSI targets are the key components in iSCSI architecture. Initiators and targets are devices (software or hardware) that package and transfer SCSI information over an IP network. An iSCSI initiator encapsulates SCSI commands, data, and status information in iSCSI packets and sends the packets to an iSCSI target residing on a storage device.

Terminology:-

iQN : IQN (iSCSI qualified name): A naming standard supported by the iSCSI protocol. IQN names are globally unique and in the form of iqn. followed by a date and a Reversed domain name.

Initiator : iSCSI initiator: An iSCSI endpoint, identified by a unique iSCSI name, which begins an iSCSI session by issuing a command to the other endpoint (the target). iSCSI Initiators could be hardware (HBA) or a software.

Target : iSCSI target: An iSCSI endpoint, identified commands issued by the iSCSI initiator. iSCSI target runs at the storage end identified a iQN.


AIX iSCSI Driver ( Initiator )

AIX has an inbuilt iSCSI driver which can be used to configure targets and thus allows the system to use iSCSI LUNs configured on the storage device. Please consult with the Storage manufacturers documentation to check what is the supported configuration. EMC documents that only software iSCSI initiator is supported on AIX for EMC products. The first iSCSI protocol adapter can be identified as /dev/iscsi0 on AIX.

The AIX iSCSI driver can use the following authentication methods while connecting to a target.

* CHAP ( Challenge Handshake Authentication Protocol )
* MD5 ( Message Digest )

/etc/iscsi/targets file configures each configured target that an AIX system’s iSCSI driver will use. There is also an option to store this information in the ODM.


Configuring iSCSI on AIX


AIX iSCSI driver can be configured using the smitty fast path “smitty iscsi” .

Select iSCSI Protocol Device è

è Change / Show Characteristics of an iSCSI Protocol Device

Select the iscsi adapter

iSCSI Protocol Device iscsi0

Description iSCSI Protocol Device

Status Available

iSCSI Initiator Name [iqn.com.xxxxxxx]

Maximum number of commands to queue to driver [200]

Discovery Policy file

Maximum Targets Allowed [16]

Apply change to DATABASE only no


Use the following parameters based on the standards and practices we follow. Use the default name (IQN) for the iSCSI initiator. Use the number of targets that we intend to configure to this system.

Once done, Send the initiator IQN name to the SAN administrator so that it can be used to define the targets they create on the storage box.

Configuring the LUNs on AIX.

We need the following details from the storage team to configure the iSCSI LUNs.

iSCSI target iQN name # Use “iscsi” to detect all the targets configured

IP Address of the storage unit # IP Address of the storage unit.

Valid Port # Default is 3260 but we need to confirm this one

Create a new line in /etc/iscsi/targets file in the following syntax

ip_addr_storage valid_port iqn_name_of_target

Once this configuration is done , save the file. Please do not change the permission of /etc/iscsi directory or /etc/iscsi/targets file.

Run cfgmgr to define the iSCSI LUNs

cfgmgr –vl iscsi0

A new iSCSI LUN should look like the following

/root>lsdev -Ccdisk | grep -i iscsi

hdisk2 Available Other iSCSI Disk Drive
/root>lsattr -El hdisk2

clr_q no Device CLEARS its Queue on error True

host_addr xx.xx.xx.xx Hostname or IP Address False

location Location Label True

lun_id 0x0 Logical Unit Number ID False

max_transfer 0x40000 Maximum TRANSFER Size True

port_num 0xcbc PORT Number False

pvid 000c7ef2a7f1ed190000000000000000 Physical volume identifier False

q_err yes Use QERR bit True

q_type simple Queuing TYPE True

queue_depth 1 Queue DEPTH True

reassign_to 120 REASSIGN time out value True

rw_timeout 30 READ/WRITE time out value True

start_timeout 60 START unit time out value True

target_name xxxxxx.com.xxxxxxx:volume-1 Target NAME False



AIX Recommendations ( iSCSI )

Create LVM volume groups with ‘Auto Varyon = No ‘. The reason being the varyonvg function is performed much before the tcp/ip subsystem is configured. iSCSI being dependant on tcp/ip it needs to be varied on after the tcp/ip subsystem is started.

chvg –a n iscsivg01

Filesystems be configured “AutoMount = False”.

crfs –v jfs2 …….. –A no

The best way to do this would be to have a “Type” allocated for the iSCSI filesystems and have a corresponding /etc/inittab entry for mounting the filesystems once the device is ready.

crfs –v jfs2 –d iscsilv01 –m /iscsifs01 –A no –u iscsi –a log=INLINE

/etc/inittab calls a script after the tcpip subsystem started ,

cfgmr –l iscso0

varyonvg iscsivg01

Mount –t iscsi

Never span volume groups across non-iSCSI drives

Configuring iSCSI volumes during system startup

I was not able to locate a documented standard process for automatically activating iSCSI volumes from IBM. So I have implemented a *close to standard procedure to get the iSCSI volumes are configured while the system boot.

The following script will check and configure the iSCSI volumes. It has to be configured in the inittab and the script be ported in the correct location.

mkitab "iscsivg:2:boot:/etc/rc.iscsivg >/dev/null 2>/dev/null"

This makes it run only once by the init during system start up. It runs when the system moves to Run Level 2. init does not wait for it’s completion. The script is intelligent enough to send out an alert should anything go wrong. The detailed log message is printed out to /var/tmp/iscsivg.log.

cat /etc/rc.iscsivg

#!/usr/bin/ksh
#
# Configure iSCSI Volume groups
# Author KapilRaj
# Date 2/14/2007
#
# Who When What

export DEBUG=No
function set_env {
[ ${DEBUG} = "Yes" ] && set -x
target=`grep -v ^# /etc/iscsi/targets | awk '{print $1}'`
export status=1
export tmpfile1=/tmp/iscsivg1.$$
export tmpfile2=/tmp/iscsivg2.$$
export logfile=/var/tmp/iscsivg.log
export email_id="kkoroth@domain_name.com"
export err_cond=0
[ -f ${logfile} ] && mv ${logfile} ${logfile}.`date +%H%M%S%m%d%y` # Rotate the log
}

function check_target {
[ ${DEBUG} = "Yes" ] && set -x
until [ ${status} -eq 0 ]
do
ping -c1 ${target} 1>/dev/null 2>/dev/null
export status=$?
sleep 30
done
}

function cfg_iscsi {
[ ${DEBUG} = "Yes" ] && set -x
cfgmgr -vl iscsi0 1>>${logfile} 2>>${logfile}
sleep 10
lsdev -Ccdisk | grep -i iscsi |awk '{print $1}' > ${tmpfile1}
for iscsivg in `lspv | grep -wf ${tmpfile1} | awk '{print $3}'|sort -u`
do
varyonvg ${iscsivg} 1>>${logfile} 2>>${logfile}
if [ $? -ne 0 ]
then
echo "${iscsivg} can not be varied on !! " >> ${logfile}
export err_cond=1
else
lsvgfs ${iscsivg} >> ${tmpfile2}
fi
done
for iscsifs in `cat ${tmpfile2} | sort`
do
mount ${iscsifs} 1>>${logfile} 2>>${logfile}
if [ $? -ne 0 ]
then
echo "${iscsifs} can not be mounted !! " >> ${logfile}
export err_cond=1
fi
done
}
function cleanup_alert {
[ ${DEBUG} = "Yes" ] && set -x
[ ${err_cond} -eq 1 ] && cat ${logfile} | mail -s "Errors while configuring iSCSI on `uname -n`" ${email_id}
rm -rf ${tmpfile1}
rm -rf ${tmpfile2}
}
# Do the stuff
[ ${DEBUG} = "Yes" ] && set -x
set_env
check_target
cfg_iscsi
cleanup_alert


Security Related information

The /etc/iscsi/directory file and the /etc/iscsi/targets configuration file are protected from non-privileged users through file permission and ownership. CHAP secrets are saved in the /etc/iscsi/targets file as clear text.

Note:

Do not to change the original file permission and ownership of these files.

Network tuning

To ensure the best performance:

* Enable the TCP Large Send, TCP send and receive flow control, and Jumbo Frame features of the AIX Gigabit Ethernet Adapter and the iSCSI Target interface.
* Tune network options and interface parameters for maximum iSCSI I/O throughput on the AIX system

o Enable the RFC 1323 network option.
o Set up the tcp_sendspace, tcp_recvspace, sb_max, and mtu_size network options and network interface options to appropriate values.

The iSCSI Software Initiator's maximum transfer size is 256KB. Assuming that the system maximums for tcp_sendspace and tcp_recvspace are set to 262144 bytes, an ifconfig command used to configure a gigabit Ethernet interface might look like the following:

ifconfig en2 xx.xx.xx.xx mtu 9000 tcp_sendspace 262144 tcp_recvspace 262144

o Set the sb_max network option to at least 524288, and preferably 1048576.
o Set the mtu_size to 9000.