Tom Shaw's questions

Is there a good way to prime a ZFS L2ARC cache on Solaris 11.3?

The L2ARC is designed to ignore blocks that have been read sequentially from a file. This makes sense for ongoing operation but makes it hard to prime the cache for initial warm-up or benchmarking.

In addition, highly-fragmented files may benefit greatly from sequential reads being cached in the L2ARC (because on-disk they are random reads), but with the current heuristics these files will never get cached even if the L2ARC is only 10% full.

In previous releases of Solaris 10 and 11, I had success in using dd twice in a row on each file. The first dd read the file into the ARC, and the second dd seemed to tickle the buffers so they became eligible for L2ARC caching. The same technique does not appear to work in Solaris 11.3.

I have confirmed that the files in question have an 8k recordsize, and I have tried setting zfs_prefetch_disable but this had no impact on the L2ARC behaviour UPDATE: zfs_prefetch_disable turns out to be important, see my answer below.

If there is no good way to do it, I would consider using a tool that generates random reads over 100% of a file. This might be worth the time given that the cache is persistent now in 11.3. Do any tools like this exist?

I have a stack of old LTO-2 tapes, and my task is to save the contents as large binary files on disk for eventual consolidation on newer media. (The contents are in a custom format specific to this site, not relevant to this question. I don't need to preserve the blocking.)

I'd like some advice on how to read the contents as fast as possible. Current performance using dd is about 2MB/s, and I believe this is due to the low and variable block size of the files on tape. The LTO-2 specs say I should be able to get 40MB/s native.

Using Solaris, I can see with the tcopy utility that the files are stored with a variable block size:

# tcopy /dev/rmt/1cbn
file 1: record 1: size 40
file 1: record 2: size 1024
file 1: record 3: size 10240
file 1: record 4: size 7168
file 1: record 5: size 1024
file 1: records 6 to 7: size 10240
[...]

Test read from tape to /dev/null:

# dd if=/dev/rmt/1cbn of=/dev/null bs=128k

(Note that the block size of 128k specified here is the maximum block size. If the actual size of the block on tape is smaller than this, that smaller amount of data will be returned for each IO.)

iostat -Mzcnx 1 shows:

    r/s    w/s   Mr/s   Mw/s wait actv wsvc_t asvc_t  %w  %b device
  304.2    0.0    1.9    0.0  0.0  1.0    0.0    3.2   0  97 rmt/1

This says to me that it's reading at 1.9MB/s, with an average IO size of about 6500 bytes and an average of exactly 1 IO outstanding at any one time.

So: given that I can't go back in time and change the block size that was written to tape, please let me know if there's some way to read the existing data faster.

I have ipfilter running on a Solaris 11.2 server with rules similar to:

pass in log first quick proto tcp from any to any port = 22 flags S keep state

I would like to be able to edit and reload the ipfilter configuration from /etc/ipf/ipf.conf. However, when I run svcadm refresh ipfilter, the dynamic state is dropped and I lose any active sessions that rely on state.

It appears that refreshing the service in Solaris takes the simple but brutal approach of:

1. ipf -D
2. ipf -E
3. Load the rules from the configuration file

Is there a better way to refresh ipfilter which (a) does not drop the state, and (b) ensures that the running configuration is identical to loading the configuration from scratch?

I'd like to collect filesystem space information using SNMP on Solaris 10 and Ubuntu 11.10. The problem is that SNMP doesn't seem to have an attribute that corresponds to the "Available" column in df.

For example, here is the df -k output of my local filesystem:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb1             36841040  13489312  21480268  39% /

And here are the relevant attributes from the SNMP daemon (as per the Net-SNMP documentation):

HOST-RESOURCES-MIB::hrStorageDescr.31 = STRING: /
HOST-RESOURCES-MIB::hrStorageSize.31 = INTEGER: 9210260
HOST-RESOURCES-MIB::hrStorageUsed.31 = INTEGER: 3372328
HOST-RESOURCES-MIB::hrStorageAllocationUnits.31 = INTEGER: 4096 Bytes

Now 9210260*4 = 36841040, and 3372328*4 = 13489312, which match the columns in the df output. However, I can't see any attribute that corresponds to the "Available" column. Note that it's not just a matter of subtracting the two columns - the "Available" column seems to take into account some hidden overheads which vary per filesystem.

Is there a way to collect or calculate the "Available" column information using SNMP?

It is possible to configure remote data replication at many layers of the storage stack. Some examples to explain what I mean by layers:

• Physical Volume Layer (TruCopy, SRDF/MirrorView, SnapMirror)
• Virtual Server Layer (vReplicator, Veeam)
• Logical Volume Layer (VVR, AVS)
• Filesystem Layer (Rsync, ZFS Send/Recv)
• Application Layer (Data Guard, MySQL/PostgreSQL Replication, AD Replication)

Replication at each layer has a unique value proposition but it would be crazy to use them all. My particular problem is the proliferation of different solutions; I'd like to standardize on one or two solutions in order to keep down license costs, complexity, and expertise requirements.

My question is: ignoring the individual product examples, which layer or combination of layers of replication provide the most value in a typical corporate environment, and which are becoming more valuable?

Possible metrics include ease of administration, availability of expertise, protection from data corruption, recovery time/complexity in a DR scenario, reduction in planned outage windows, application performance, application flexibility, and anything else that might be valuable.

This is a question I encounter often as a system administrator so I would appreciate any advice.