IT - Linux:iSCSI discovering new LUNs for the same client

Server

Add new disk (insert into USB cage with slot power-off)

Power-on the slot. Wait few seconds and check the new disk is visible

$ lsblk --scsi
NAME
    HCTL       TYPE VENDOR   MODEL                          REV SERIAL          TRAN
sda 0:0:0:0    disk ATA      Samsung SSD 850 EVO 500GB EMT02B6Q S21JNXAGC13723P sata
sdb 6:0:0:0    disk ASMT     ST16000NM001G-2KK103          SN02 ZL22TA9C        usb
sdc 8:0:0:0    disk ASMT     ST12000NM0007-2A1101          SN02 ZJV3CH1V        usb
sdd 9:0:0:0    disk ASMT     WDC WD40EZRX-00SPEB0      80.00A80 WD-WCC4E0486171 usb
sde 10:0:0:0   disk ASMT     TOSHIBA MG07ACA12TE           0101 3010A06GF95G    usb
sdf 7:0:0:0    disk ASMT     ST18000NM003D-3DL103          SN03 ZVT8NE6T        usb

Get disk WWN. This is important because at every system reboot, the device can change. This onyl affect USB devices

$ ls -lF /dev/disk/by-id/|grep 'wwn.*sdf'
lrwxrwxrwx. 1 root root  9 Feb 22 21:18 wwn-0x5000c500e66beeca -> ../../sdf

Update the CreateTarget_?.sh script

...
#----- Add the backstores
targetcli /backstores/block create t15-A-disk0 /dev/disk/by-id/wwn-0x5000c500e66beeca
...
#----- Add the LUNs from this "server" that will be shared trough the "access point"
targetcli /iscsi/${sThis}/tpg1/luns create /backstores/block/t15-A-disk0
...

Manually update the iSCSI setup, executing the commands above

$ sudo targetcli
[sudo] password for afberendsen: 
targetcli shell version 2.1.57
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls backstores/block
o- block ...................................................................................................... [Storage Objects: 4]
  o- t15-A-disk1 ........................................... [/dev/disk/by-id/wwn-0x5000c500b4063f47 (10.9TiB) write-thru activated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk2 ............................................ [/dev/disk/by-id/wwn-0x50014ee25ea449c0 (3.6TiB) write-thru activated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk3 ........................................... [/dev/disk/by-id/wwn-0x5000039a08c82220 (10.9TiB) write-thru activated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk4 ........................................... [/dev/disk/by-id/wwn-0x5000c500c4248047 (14.6TiB) write-thru activated]
    o- alua ....................................................................................................... [ALUA Groups: 1]
      o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
/> /backstores/block create t15-A-disk0 /dev/disk/by-id/wwn-0x5000c500e66beeca
Created block storage object t15-A-disk0 using /dev/disk/by-id/wwn-0x5000c500e66beeca.
/> ls backstores/block
o- block ...................................................................................................... [Storage Objects: 5]
  o- t15-A-disk0 ......................................... [/dev/disk/by-id/wwn-0x5000c500e66beeca (16.4TiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk1 ........................................... [/dev/disk/by-id/wwn-0x5000c500b4063f47 (10.9TiB) write-thru activated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk2 ............................................ [/dev/disk/by-id/wwn-0x50014ee25ea449c0 (3.6TiB) write-thru activated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk3 ........................................... [/dev/disk/by-id/wwn-0x5000039a08c82220 (10.9TiB) write-thru activated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- t15-A-disk4 ........................................... [/dev/disk/by-id/wwn-0x5000c500c4248047 (14.6TiB) write-thru activated]
    o- alua ....................................................................................................... [ALUA Groups: 1]
      o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
/> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/luns
o- luns .................................................................................................................. [LUNs: 4]
  o- lun0 .......................................... [block/t15-A-disk1 (/dev/disk/by-id/wwn-0x5000c500b4063f47) (default_tg_pt_gp)]
  o- lun1 .......................................... [block/t15-A-disk2 (/dev/disk/by-id/wwn-0x50014ee25ea449c0) (default_tg_pt_gp)]
  o- lun2 .......................................... [block/t15-A-disk3 (/dev/disk/by-id/wwn-0x5000039a08c82220) (default_tg_pt_gp)]
  o- lun3 .......................................... [block/t15-A-disk4 (/dev/disk/by-id/wwn-0x5000c500c4248047) (default_tg_pt_gp)]
/> /iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/luns create /backstores/block/t15-A-disk0
Created LUN 4.
/> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/luns
o- luns .................................................................................................................. [LUNs: 5]
  o- lun0 .......................................... [block/t15-A-disk1 (/dev/disk/by-id/wwn-0x5000c500b4063f47) (default_tg_pt_gp)]
  o- lun1 .......................................... [block/t15-A-disk2 (/dev/disk/by-id/wwn-0x50014ee25ea449c0) (default_tg_pt_gp)]
  o- lun2 .......................................... [block/t15-A-disk3 (/dev/disk/by-id/wwn-0x5000039a08c82220) (default_tg_pt_gp)]
  o- lun3 .......................................... [block/t15-A-disk4 (/dev/disk/by-id/wwn-0x5000c500c4248047) (default_tg_pt_gp)]
  o- lun4 .......................................... [block/t15-A-disk0 (/dev/disk/by-id/wwn-0x5000c500e66beeca) (default_tg_pt_gp)]
/> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/acls/iqn.1994-05.com.redhat:ecdd17858d7 create 4 lun4
o- iqn.1994-05.com.redhat:ecdd17858d7 ............................................................................. [Mapped LUNs: 4]
  o- mapped_lun0 ..................................................................................... [lun0 block/t15-A-disk1 (rw)]
  o- mapped_lun1 ..................................................................................... [lun1 block/t15-A-disk2 (rw)]
  o- mapped_lun2 ..................................................................................... [lun2 block/t15-A-disk3 (rw)]
  o- mapped_lun3 ..................................................................................... [lun3 block/t15-A-disk4 (rw)]
/> /iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/acls/iqn.1994-05.com.redhat:ecdd17858d7 create 4 lun4
Created Mapped LUN 4.
/> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/acls/iqn.1994-05.com.redhat:ecdd17858d7 create 4 lun4
o- iqn.1994-05.com.redhat:ecdd17858d7 ............................................................................. [Mapped LUNs: 5]
  o- mapped_lun0 ..................................................................................... [lun0 block/t15-A-disk1 (rw)]
  o- mapped_lun1 ..................................................................................... [lun1 block/t15-A-disk2 (rw)]
  o- mapped_lun2 ..................................................................................... [lun2 block/t15-A-disk3 (rw)]
  o- mapped_lun3 ..................................................................................... [lun3 block/t15-A-disk4 (rw)]
  o- mapped_lun4 ..................................................................................... [lun4 block/t15-A-disk0 (rw)]
/> saveconfig
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json
/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json
$

Client

Check that the new disk is still not visible on the client side

$ lsblk --scsi -t|grep t15
sdk  11:0:0:0   disk LIO-ORG  t15-A-disk1              4.0 2c0dbded-a792-47dc-9b34-5bb4915a945c iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdm  11:0:0:3   disk LIO-ORG  t15-A-disk4              4.0 263f33ef-3dab-479c-9ae9-6138ffbbee53 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdp  11:0:0:2   disk LIO-ORG  t15-A-disk3              4.0 fa7bfaa5-62f3-4d8b-a4ab-a786e7e91245 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdr  11:0:0:1   disk LIO-ORG  t15-A-disk2              4.0 be4f58cd-b41a-40b6-84d5-e128a6566d60 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B

Rescan iSCSI bus for new targets

$ sudo iscsiadm -m session --rescan
[sudo] password for afberendsen: 
Rescanning session [sid: 2, target: iqn.1994-05.com.redhat:4aa4ea84d4aa, portal: 192.168.0.15,3260]
Rescanning session [sid: 3, target: iqn.1994-05.com.redhat:1bf353c196ef, portal: 192.168.0.14,3260]
Rescanning session [sid: 1, target: iqn.1994-05.com.redhat:4aa4ea84d4aa, portal: 192.168.0.16,3260]

Confirm the new target is available

$ lsblk --scsi -t|grep t15
sdk  11:0:0:0   disk LIO-ORG  t15-A-disk1              4.0 2c0dbded-a792-47dc-9b34-5bb4915a945c iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdm  11:0:0:3   disk LIO-ORG  t15-A-disk4              4.0 263f33ef-3dab-479c-9ae9-6138ffbbee53 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdp  11:0:0:2   disk LIO-ORG  t15-A-disk3              4.0 fa7bfaa5-62f3-4d8b-a4ab-a786e7e91245 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdr  11:0:0:1   disk LIO-ORG  t15-A-disk2              4.0 be4f58cd-b41a-40b6-84d5-e128a6566d60 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B
sdad 11:0:0:4   disk LIO-ORG  t15-A-disk0              4.0 a5af33b8-5d79-47c4-8d06-7697a4038002 iscsi          0   4096   524288    4096     512    1 bfq       226  1024    0B

To add the new disk into a ZFS pool

Call fdisk and confirm the disk is using GPT. If not, re-partition the disk with GPT.

$ sudo fdisk  /dev/sdad

Welcome to fdisk (util-linux 2.39.3).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
The size of this disk is 16.4 TiB (18000207937536 bytes). DOS partition table format cannot be used on drives for volumes larger than 2199023255040 bytes for 512-byte sectors. Use GUID partition table format (GPT).

Created a new DOS (MBR) disklabel with disk identifier 0x10e272d4.

Command (m for help): p
Disk /dev/sdad: 16.37 TiB, 18000207937536 bytes, 35156656128 sectors
Disk model: t15-A-disk0     
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 524288 bytes
Disklabel type: dos
Disk identifier: 0x10e272d4

Command (m for help): g
Created a new GPT disklabel (GUID: A6AAFA30-00DA-4082-BA47-3F348BDA804A).

Command (m for help): n
Partition number (1-128, default 1): 
First sector (2048-35156656094, default 2048): 
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-35156656094, default 35156654079): 

Created a new partition 1 of type 'Linux filesystem' and of size 16.4 TiB.

Command (m for help): p
Disk /dev/sdad: 16.37 TiB, 18000207937536 bytes, 35156656128 sectors
Disk model: t15-A-disk0     
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 524288 bytes
Disklabel type: gpt
Disk identifier: A6AAFA30-00DA-4082-BA47-3F348BDA804A

Device     Start         End     Sectors  Size Type
/dev/sdad1  2048 35156654079 35156652032 16.4T Linux filesystem

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Identify the disk partition UUID

$ ls -laF /dev/disk/by-partuuid/|grep sdad
lrwxrwxrwx 1 root root  11 Feb 22 22:03 77c5a28a-c107-43fc-8f0b-bce8b7382565 -> ../../sdad1

Add the new partition to the ZFS pool. This can take some time...

$ sudo zpool list
NAME          SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
pool_videos   382T   368T  14.2T        -         -    71%    96%  1.00x    ONLINE  -
$ time sudo zpool add pool_videos  /dev/disk/by-partuuid/77c5a28a-c107-43fc-8f0b-bce8b7382565

real	2m12.278s
user	0m0.017s
sys	0m0.019s
$ sudo zpool list
NAME          SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
pool_videos   398T   368T  30.6T

ATA Drive Capacity Limitations

ATA interface versions up through ATA-5 suffered from a drive capacity limitation of about 137GB (billion bytes). Depending on the BIOS used, you can further reduce this limitation to 8.4GB, or even as low as 528MB (million bytes). This is due to limitations in both the BIOS and the ATA interface, which when combined create even further limitations. To understand these limits, you have to look at the BIOS (software) and ATA (hardware) interfaces together. NOTE In addition to the BIOS/ATA limitations discussed in this section, various operating system limitations exist. These are described later in this chapter. The limitations when dealing with ATA drives are those of the ATA interface as well as the BIOS interface used to talk to the drive. A summary of the limitations is shown in Table 7.12. Table 7.12. ATA/IDE Capacity Limitations for Various Sector Addressing Methods Sector Addressing Method Total Sectors Calculation Maximum Total Sectors Maximum Capacity (Byte...

My things

Search This Blog