Server
- Add new disk (insert into USB cage with slot power-off)
- Power-on the slot. Wait few seconds and check the new disk is visible
$ lsblk --scsi NAME HCTL TYPE VENDOR MODEL REV SERIAL TRAN sda 0:0:0:0 disk ATA Samsung SSD 850 EVO 500GB EMT02B6Q S21JNXAGC13723P sata sdb 6:0:0:0 disk ASMT ST16000NM001G-2KK103 SN02 ZL22TA9C usb sdc 8:0:0:0 disk ASMT ST12000NM0007-2A1101 SN02 ZJV3CH1V usb sdd 9:0:0:0 disk ASMT WDC WD40EZRX-00SPEB0 80.00A80 WD-WCC4E0486171 usb sde 10:0:0:0 disk ASMT TOSHIBA MG07ACA12TE 0101 3010A06GF95G usb sdf 7:0:0:0 disk ASMT ST18000NM003D-3DL103 SN03 ZVT8NE6T usb - Get disk WWN. This is important because at every system reboot, the device can change. This onyl affect USB devices
$ ls -lF /dev/disk/by-id/|grep 'wwn.*sdf' lrwxrwxrwx. 1 root root 9 Feb 22 21:18 wwn-0x5000c500e66beeca -> ../../sdf - Update the CreateTarget_?.sh script
... #----- Add the backstores targetcli /backstores/block create t15-A-disk0 /dev/disk/by-id/wwn-0x5000c500e66beeca ... #----- Add the LUNs from this "server" that will be shared trough the "access point" targetcli /iscsi/${sThis}/tpg1/luns create /backstores/block/t15-A-disk0 ... - Manually update the iSCSI setup, executing the commands above
$ sudo targetcli [sudo] password for afberendsen: targetcli shell version 2.1.57 Copyright 2011-2013 by Datera, Inc and others. For help on commands, type 'help'. /> ls backstores/block o- block ...................................................................................................... [Storage Objects: 4] o- t15-A-disk1 ........................................... [/dev/disk/by-id/wwn-0x5000c500b4063f47 (10.9TiB) write-thru activated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk2 ............................................ [/dev/disk/by-id/wwn-0x50014ee25ea449c0 (3.6TiB) write-thru activated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk3 ........................................... [/dev/disk/by-id/wwn-0x5000039a08c82220 (10.9TiB) write-thru activated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk4 ........................................... [/dev/disk/by-id/wwn-0x5000c500c4248047 (14.6TiB) write-thru activated] o- alua ....................................................................................................... [ALUA Groups: 1] o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] /> /backstores/block create t15-A-disk0 /dev/disk/by-id/wwn-0x5000c500e66beeca Created block storage object t15-A-disk0 using /dev/disk/by-id/wwn-0x5000c500e66beeca. /> ls backstores/block o- block ...................................................................................................... [Storage Objects: 5] o- t15-A-disk0 ......................................... [/dev/disk/by-id/wwn-0x5000c500e66beeca (16.4TiB) write-thru deactivated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk1 ........................................... [/dev/disk/by-id/wwn-0x5000c500b4063f47 (10.9TiB) write-thru activated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk2 ............................................ [/dev/disk/by-id/wwn-0x50014ee25ea449c0 (3.6TiB) write-thru activated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk3 ........................................... [/dev/disk/by-id/wwn-0x5000039a08c82220 (10.9TiB) write-thru activated] | o- alua ....................................................................................................... [ALUA Groups: 1] | o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] o- t15-A-disk4 ........................................... [/dev/disk/by-id/wwn-0x5000c500c4248047 (14.6TiB) write-thru activated] o- alua ....................................................................................................... [ALUA Groups: 1] o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized] /> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/luns o- luns .................................................................................................................. [LUNs: 4] o- lun0 .......................................... [block/t15-A-disk1 (/dev/disk/by-id/wwn-0x5000c500b4063f47) (default_tg_pt_gp)] o- lun1 .......................................... [block/t15-A-disk2 (/dev/disk/by-id/wwn-0x50014ee25ea449c0) (default_tg_pt_gp)] o- lun2 .......................................... [block/t15-A-disk3 (/dev/disk/by-id/wwn-0x5000039a08c82220) (default_tg_pt_gp)] o- lun3 .......................................... [block/t15-A-disk4 (/dev/disk/by-id/wwn-0x5000c500c4248047) (default_tg_pt_gp)] /> /iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/luns create /backstores/block/t15-A-disk0 Created LUN 4. /> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/luns o- luns .................................................................................................................. [LUNs: 5] o- lun0 .......................................... [block/t15-A-disk1 (/dev/disk/by-id/wwn-0x5000c500b4063f47) (default_tg_pt_gp)] o- lun1 .......................................... [block/t15-A-disk2 (/dev/disk/by-id/wwn-0x50014ee25ea449c0) (default_tg_pt_gp)] o- lun2 .......................................... [block/t15-A-disk3 (/dev/disk/by-id/wwn-0x5000039a08c82220) (default_tg_pt_gp)] o- lun3 .......................................... [block/t15-A-disk4 (/dev/disk/by-id/wwn-0x5000c500c4248047) (default_tg_pt_gp)] o- lun4 .......................................... [block/t15-A-disk0 (/dev/disk/by-id/wwn-0x5000c500e66beeca) (default_tg_pt_gp)] /> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/acls/iqn.1994-05.com.redhat:ecdd17858d7 create 4 lun4 o- iqn.1994-05.com.redhat:ecdd17858d7 ............................................................................. [Mapped LUNs: 4] o- mapped_lun0 ..................................................................................... [lun0 block/t15-A-disk1 (rw)] o- mapped_lun1 ..................................................................................... [lun1 block/t15-A-disk2 (rw)] o- mapped_lun2 ..................................................................................... [lun2 block/t15-A-disk3 (rw)] o- mapped_lun3 ..................................................................................... [lun3 block/t15-A-disk4 (rw)] /> /iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/acls/iqn.1994-05.com.redhat:ecdd17858d7 create 4 lun4 Created Mapped LUN 4. /> ls iscsi/iqn.1994-05.com.redhat:4aa4ea84d4aa/tpg1/acls/iqn.1994-05.com.redhat:ecdd17858d7 create 4 lun4 o- iqn.1994-05.com.redhat:ecdd17858d7 ............................................................................. [Mapped LUNs: 5] o- mapped_lun0 ..................................................................................... [lun0 block/t15-A-disk1 (rw)] o- mapped_lun1 ..................................................................................... [lun1 block/t15-A-disk2 (rw)] o- mapped_lun2 ..................................................................................... [lun2 block/t15-A-disk3 (rw)] o- mapped_lun3 ..................................................................................... [lun3 block/t15-A-disk4 (rw)] o- mapped_lun4 ..................................................................................... [lun4 block/t15-A-disk0 (rw)] /> saveconfig Last 10 configs saved in /etc/target/backup/. Configuration saved to /etc/target/saveconfig.json /> exit Global pref auto_save_on_exit=true Last 10 configs saved in /etc/target/backup/. Configuration saved to /etc/target/saveconfig.json $
Client
- Check that the new disk is still not visible on the client side
$ lsblk --scsi -t|grep t15 sdk 11:0:0:0 disk LIO-ORG t15-A-disk1 4.0 2c0dbded-a792-47dc-9b34-5bb4915a945c iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdm 11:0:0:3 disk LIO-ORG t15-A-disk4 4.0 263f33ef-3dab-479c-9ae9-6138ffbbee53 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdp 11:0:0:2 disk LIO-ORG t15-A-disk3 4.0 fa7bfaa5-62f3-4d8b-a4ab-a786e7e91245 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdr 11:0:0:1 disk LIO-ORG t15-A-disk2 4.0 be4f58cd-b41a-40b6-84d5-e128a6566d60 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B
- Rescan iSCSI bus for new targets
$ sudo iscsiadm -m session --rescan [sudo] password for afberendsen: Rescanning session [sid: 2, target: iqn.1994-05.com.redhat:4aa4ea84d4aa, portal: 192.168.0.15,3260] Rescanning session [sid: 3, target: iqn.1994-05.com.redhat:1bf353c196ef, portal: 192.168.0.14,3260] Rescanning session [sid: 1, target: iqn.1994-05.com.redhat:4aa4ea84d4aa, portal: 192.168.0.16,3260]
- Confirm the new target is available
$ lsblk --scsi -t|grep t15 sdk 11:0:0:0 disk LIO-ORG t15-A-disk1 4.0 2c0dbded-a792-47dc-9b34-5bb4915a945c iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdm 11:0:0:3 disk LIO-ORG t15-A-disk4 4.0 263f33ef-3dab-479c-9ae9-6138ffbbee53 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdp 11:0:0:2 disk LIO-ORG t15-A-disk3 4.0 fa7bfaa5-62f3-4d8b-a4ab-a786e7e91245 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdr 11:0:0:1 disk LIO-ORG t15-A-disk2 4.0 be4f58cd-b41a-40b6-84d5-e128a6566d60 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B sdad 11:0:0:4 disk LIO-ORG t15-A-disk0 4.0 a5af33b8-5d79-47c4-8d06-7697a4038002 iscsi 0 4096 524288 4096 512 1 bfq 226 1024 0B - To add the new disk into a ZFS pool
- Call fdisk and confirm the disk is using GPT. If not, re-partition the disk with GPT.
$ sudo fdisk /dev/sdad Welcome to fdisk (util-linux 2.39.3). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table. The size of this disk is 16.4 TiB (18000207937536 bytes). DOS partition table format cannot be used on drives for volumes larger than 2199023255040 bytes for 512-byte sectors. Use GUID partition table format (GPT). Created a new DOS (MBR) disklabel with disk identifier 0x10e272d4. Command (m for help): p Disk /dev/sdad: 16.37 TiB, 18000207937536 bytes, 35156656128 sectors Disk model: t15-A-disk0 Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 524288 bytes Disklabel type: dos Disk identifier: 0x10e272d4 Command (m for help): g Created a new GPT disklabel (GUID: A6AAFA30-00DA-4082-BA47-3F348BDA804A). Command (m for help): n Partition number (1-128, default 1): First sector (2048-35156656094, default 2048): Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-35156656094, default 35156654079): Created a new partition 1 of type 'Linux filesystem' and of size 16.4 TiB. Command (m for help): p Disk /dev/sdad: 16.37 TiB, 18000207937536 bytes, 35156656128 sectors Disk model: t15-A-disk0 Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 524288 bytes Disklabel type: gpt Disk identifier: A6AAFA30-00DA-4082-BA47-3F348BDA804A Device Start End Sectors Size Type /dev/sdad1 2048 35156654079 35156652032 16.4T Linux filesystem Command (m for help): w The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks. - Identify the disk partition UUID
$ ls -laF /dev/disk/by-partuuid/|grep sdad lrwxrwxrwx 1 root root 11 Feb 22 22:03 77c5a28a-c107-43fc-8f0b-bce8b7382565 -> ../../sdad1 - Add the new partition to the ZFS pool. This can take some time...
$ sudo zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT pool_videos 382T 368T 14.2T - - 71% 96% 1.00x ONLINE - $ time sudo zpool add pool_videos /dev/disk/by-partuuid/77c5a28a-c107-43fc-8f0b-bce8b7382565 real 2m12.278s user 0m0.017s sys 0m0.019s $ sudo zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT pool_videos 398T 368T 30.6T
- Call fdisk and confirm the disk is using GPT. If not, re-partition the disk with GPT.
Comments
Post a Comment