Home / server / Questions / 1031580
Accepted
CodyMouse
Asked: 2020-08-27 04:59:09 +0800 CST

TRIM/UNMAP Zvol over iSCSI

  • 772

I am currently setting up a SAN for diskless boot. My backend consists of ZFS-Vol shared via iSCSI. So far everything is working just fine except for TRIM/UNMAP. For test puposes I setup two VMs running Ubuntu20.04 in VirtualBox networked together via an internal network with static IPv4 addresses. On the target (tgt) got a second virtual drive formatted with ZFS. On this zpool I created a zVol and formatted it with GPT and ext4.

/etc/tgt/conf.d/iscsi.conf
<target example.com:lun1>
    <backing-store /dev/zvol/tank/iscsi_share>
        params thin_provisioning=1
    </backing-store>
    initiator-address 192.168.0.2
</target>

On the initiator (open-iscsi) I use this command to provoke a TRIM operation:

sudo mount /dev/sdb1 /iscsi-share
sudo dd if=/dev/zero of=/iscsi-share/zero bs=1M count=512
sudo rm /iscsi-share/zero
sudo fstrim /iscsi-share

but the shell responds with "fstrim: /iscsi-share: the discard option is not supported". If I issue those commands on the target machine the "REFER" property of the zVol decreases as expected.

As I found nothing while searching the web I found no hint as to why this is not working or if this is even possible at all.

Edit: As I got the advice to use the option thin_provisioning.

After I repartitioned the drive and mounted it on the initiator I got error message blk_update_request: critical target error, dev sdb, sector 23784 op 0x9:(WRITE_ZEROES) flags 0x800 phys_seg 0 prio class 0 for several sectors and after creating and deleting my testfile, fstrim send the message

blk_update_request: I/O error, dev sdb, sector 68968 op 0x3:(DISCARD) flags 0x800 phys_seg 1 prio class 0
fstrim: iscsi-share: FITRIM ioctl failed: Input/output error

Edit: As there were Answers refering to LIO I now also tried targetcli. There I setup a target with my zVol under /backstores/block/iscsi and set attribute emultate_tpu=1. After importing this into my initiator I repartitioned, formatted and mounted it on the initiator. Then I created my test file, deletetd it and issued the fstrim command and it worked. Thanks for the help.

zfsonlinux open-iscsi

3 Answers

  1. What you're asking is highly iSCSI target implementation specific. Most of them don't do 1:1 SCSI command mapping, so if iSCSI target emulates hard disk - it won't bypass unrecognized commands (incl. UNMAP of course) to underlying storage @ back-end UNLESS you'll explicitly ask iSCSI target to do so. With TGT you ment'd you specify "thin_provisioning=1" for your virtual LUN in config file.

    • 6

  2. TRIM is enabled by default to run weekly in Ubuntu 20.04 so there should be no problem even if you run it manually. You can check this in the fstrim.service and fstrim.timer just in case: https://askubuntu.com/questions/1034169/is-trim-enabled-on-my-ubuntu-18-04-installation

    However, this indeed looks like you need to enable UNMAP on the target as already mentioned. Since many SATA SSDs had issues with UNMAP, it was disabled in LIO by default: http://www.linux-iscsi.org/Doc/LIO%20Admin%20Manual.pdf.

    On the side note, of course, things are getting a lot easier when an iSCSI target natively supports TRIM/UNMAP with no further tinkering. Here is an example: https://forums.starwindsoftware.com/viewtopic.php?f=5&t=5343

    • 5
  3. Best Answer

    LIO disables UNMAP by default. If you want to enable it you should set the emulate_tpu attribute on the target.

    • 2