Home / user-55114

Ketan Maheshwari's questions

Martin Hope
Ketan Maheshwari
Asked: 2017-08-01 06:57:39 +0800 CST
  • 4

I have a 500T ZFS server with 8 zpools. The usage overall is pretty moderate. However, I noticed that on one of the zpools the usedbydataset value is as large as 24.2T. The actual user data on this zpool is just about 4T. The other zpools have pretty moderate values. See below:

$ zfs list|egrep '(^jbodp4 |^NAME)'
NAME                   USED  AVAIL  REFER  MOUNTPOINT
jbodp4                28.0T  20.0T  24.2T  /zfs/4

$ zfs list|egrep '^jbodp4\/'
jbodp4/user1        219K  5.00T   219K  /zfs/4/user1
jbodp4/user2          144G  4.86T   144G  /zfs/4/user2
jbodp4/user3         4.87G  5.00T  4.87G  /zfs/4/user3
jbodp4/user4       3.69T  1.31T  3.69T  /zfs/4/user4

$ zfs get all|egrep '(^jbodp[0-9] |^intp[0-9] )' | grep 'usedbydataset'
intp1                 usedbydataset         219K                   -
intp2                 usedbydataset         252G                   -
intp3                 usedbydataset         219K                   -
jbodp4                usedbydataset         24.2T                  -
jbodp5                usedbydataset         16.1G                  -
jbodp6                usedbydataset         1.42T                  -
jbodp7                usedbydataset         901G                   -
jbodp8                usedbydataset         329K                   -

Could someone help me identify why this particular jbod has such a high usage for usedbydataset and how could I find more about this dataset.

zfs
Martin Hope
Ketan Maheshwari
Asked: 2016-10-01 11:12:40 +0800 CST
  • 0

We are running a cluster for about 50 user-groups mapped as Linux groups. Each group has a filesystem allocated from a ZFS storage server which is exported over NFS. This results in a long list of mountpoints that are required to be mounted over the NFS client nodes. Similarly the output of df -h is also very long listing all those mountpoints for each user group. Is there a way to avoid this by somehow restructuring or reconfiguring the ZFS server while still managing user-groups efficiently such as quotas, access rights etc.

Below is the partial output of zfs list command:

~]# zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
intp1               43.7G  47.9T   219K  /zfs/1
intp1/sam           43.7G  47.9T   219K  /zfs/1/sam
intp1/harry         219K  4.00T   219K  /zfs/1/harry
intp1/rick          219K  4.00T   219K  /zfs/1/rick
intp1/kim           43.7G  3.96T  43.7G  /zfs/1/kim
intp2                252G  47.7T   252G  /zfs/2
intp2/johnson       219K  8.00T   219K  /zfs/2/johnson
intp2/hoekstra        219K  8.00T   219K  /zfs/2/hoekstra

Shown below is the output of zfs get all command for one filesystem:

# zfs get all intp1/sam
NAME       PROPERTY              VALUE                  SOURCE
intp1/sam  type                  filesystem             -
intp1/sam  creation              Fri Sep 23  9:56 2016  -
intp1/sam  used                  43.7G                  -
intp1/sam  available             47.9T                  -
intp1/sam  referenced            219K                   -
intp1/sam  compressratio         5.42x                  -
intp1/sam  mounted               yes                    -
intp1/sam  quota                 none                   default
intp1/sam  reservation           none                   default
intp1/sam  recordsize            128K                   default
intp1/sam  mountpoint            /zfs/1/sam             inherited from intp1
intp1/sam  sharenfs              on                     inherited from intp1
intp1/sam  checksum              on                     default
intp1/sam  compression           lz4                    inherited from intp1
intp1/sam  atime                 on                     default
intp1/sam  devices               on                     default
intp1/sam  exec                  on                     default
intp1/sam  setuid                on                     default
intp1/sam  readonly              off                    default
intp1/sam  zoned                 off                    default
intp1/sam  snapdir               hidden                 default
intp1/sam  aclinherit            restricted             default
intp1/sam  canmount              on                     default
intp1/sam  xattr                 on                     default
intp1/sam  copies                1                      default
intp1/sam  version               5                      -
intp1/sam  utf8only              off                    -
intp1/sam  normalization         none                   -
intp1/sam  casesensitivity       sensitive              -
intp1/sam  vscan                 off                    default
intp1/sam  nbmand                off                    default
intp1/sam  sharesmb              off                    default
intp1/sam  refquota              none                   default
intp1/sam  refreservation        none                   default
intp1/sam  primarycache          all                    default
intp1/sam  secondarycache        all                    default
intp1/sam  usedbysnapshots       0                      -
intp1/sam  usedbydataset         219K                   -
intp1/sam  usedbychildren        43.7G                  -
intp1/sam  usedbyrefreservation  0                      -
intp1/sam  logbias               latency                default
intp1/sam  dedup                 off                    default
intp1/sam  mlslabel              none                   default
intp1/sam  sync                  standard               default
intp1/sam  refcompressratio      1.00x                  -
intp1/sam  written               219K                   -
intp1/sam  logicalused           198G                   -
intp1/sam  logicalreferenced     40K                    -
intp1/sam  filesystem_limit      none                   default
intp1/sam  snapshot_limit        none                   default
intp1/sam  filesystem_count      none                   default
intp1/sam  snapshot_count        none                   default
intp1/sam  snapdev               hidden                 default
intp1/sam  acltype               off                    default
intp1/sam  context               none                   default
intp1/sam  fscontext             none                   default
intp1/sam  defcontext            none                   default
intp1/sam  rootcontext           none                   default
intp1/sam  relatime              off                    default
intp1/sam  redundant_metadata    all                    default
intp1/sam  overlay               off                    default
zfs zfsonlinux
Martin Hope
Ketan Maheshwari
Asked: 2016-08-15 15:57:37 +0800 CST
  • 17

I have a ZFS server with 8 zpools. Each pool is 12 6TB disks in a 10+2 RAIDZ configuration. So, each pool has a raw storage space of 12*6=72 TB and usable space of 10*6=60 TB. However, I see different results when I query it using zfs list vs. using zpool list as below:

# zfs list
NAME     USED  AVAIL  REFER  MOUNTPOINT
intp1    631K  48.0T   219K  /intp1
intp2    631K  48.0T   219K  /intp2
intp3    631K  48.0T   219K  /intp3
jbodp4   631K  48.0T   219K  /jbodp4

# zpool list
NAME     SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
intp1     65T  1.02M  65.0T         -     0%     0%  1.00x  ONLINE  -
intp2     65T  1020K  65.0T         -     0%     0%  1.00x  ONLINE  -
intp3     65T  1.02M  65.0T         -     0%     0%  1.00x  ONLINE  -
jbodp4    65T  1.02M  65.0T         -     0%     0%  1.00x  ONLINE  -

Could someone help me understand why is this discrepancy?

zfs zfsonlinux