Home / server / Questions / 654933
Accepted
Jasmine Lognnes
Asked: 2014-12-27 16:34:06 +0800 CST

How to migrate a ZFS filesystem to use compression?

  • 772

Question at the bottom.

I have several ZFS file system I would like to use ZFS compression, but since enabling the compression will only affect new data written to the file system, I would like to write a script that can migrate file systems, so all data is compressed.

This is my test attempt

du -h /tmp/dump.txt

zfs create -p tank3/xtest1/fs
cp /tmp/dump.txt /tank3/xtest1/fs
zfs list | grep xtest

zfs create tank3/xtest2
zfs set compression=lzjb tank3/xtest2
zfs inherit compression tank3/xtest2

zfs snapshot tank3/xtest1/fs@snap
zfs send tank3/xtest1/fs@snap | zfs receive tank3/xtest2/fs
zfs get compression tank3/xtest2/fs
zfs list | grep xtest

zfs destroy -r tank3/xtest1
zfs destroy -r tank3/xtest2


echo "test 2"

zfs create tank3/xtest2
zfs set compression=lzjb tank3/xtest2
zfs list | grep xtest

cp /tmp/dump.txt /tank3/xtest2
zfs list | grep xtest
zfs get compressratio tank3/xtest2

zfs destroy -r tank3/xtest2

which gives

344M    /tmp/dump.txt
tank3/xtest1                          575K  6.38T   288K  /tank3/xtest1
tank3/xtest1/fs                       288K  6.38T   288K  /tank3/xtest1/fs
NAME             PROPERTY     VALUE     SOURCE
tank3/xtest2/fs  compression  off       default
tank3/xtest1                          344M  6.38T   304K  /tank3/xtest1
tank3/xtest1/fs                       344M  6.38T   344M  /tank3/xtest1/fs
tank3/xtest2                          344M  6.38T   288K  /tank3/xtest2
tank3/xtest2/fs                       344M  6.38T   344M  /tank3/xtest2/fs
test 2
tank3/xtest2                          288K  6.38T   288K  /tank3/xtest2
tank3/xtest2                          288K  6.38T   288K  /tank3/xtest2
NAME          PROPERTY       VALUE  SOURCE
tank3/xtest2  compressratio  1.00x  -

In the first test would I have expected the replication would compress the data when creating tank3/xtest2/fs but newly created file systems does not inherit compression when using send/receive it seams.

In test 2 I can't see the 344MB file takes any space.

From what I can tell compression doesn't work.

Question

Why do I see these weird results?

And how should I migrate a not compressed file system to be compressed?

Update

Added compressratio property which shows that no compression have been done. dump.txt can be compressed to 190MB.

filesystems

2 Answers

  1. Copy the data.

    That is all.

    There's a bunch of missing output in your examples, but it doesn't matter. The only thing you need to do to move data into a compressed filesystem is to rewrite the data either by copying it, moving it or populating a new filesystem.

    Avoid the problem in future situations by enabling compression at the pool level so that it's inherited by new filesystems. Also, use lz4 compression flag. It's generally better.

    • 5
  2. Best Answer

    Even though dump.txt is highly compressible it turns out, that zfs won't compress it for some reason.

    Redoing my experiments again with other data, it turns out that send/receive will compress the data. Here I have set the compression on the zpool, so all file systems would inherit this. I also used Linux here, but that shouldn't matter.

    # dd if=/dev/zero of=/mnt/dat0/zpool1 bs=1M count=1000
# dd if=/dev/zero of=/mnt/dat0/zpool2 bs=1M count=1000
# service zfs-fuse restart
# zpool create zpool1 /mnt/dat0/zpool1 
# zpool create zpool2 /mnt/dat0/zpool2
# zfs set compression=on zpool1
# zfs create zpool1/fs1
# zfs create zpool2/fs2
# cp -r /etc/ /var/log /zpool2/fs2/
# zfs snapshot zpool2/fs2@snap
# zfs send zpool2/fs2@snap | zfs receive zpool1/fs2
# zfs get compressratio zpool1/fs2
NAME        PROPERTY       VALUE  SOURCE
zpool1/fs2  compressratio  4.75x  -
    • 1