Home / server / Questions / 357471
In Process
Edilaic
Asked: 2012-02-07 18:22:38 +0800 CST

What would cause huge amounts of snapshot lag with NetApp Snapvault?

  • 772

Our NetApp filer shows huge amounts of lag for snapvault operations. What could be possible causes of such large snapshot lag?

Snapvault is ON.
Source                Destination                     State          Lag        Status
home:/vol/home/       vault1:/vol/home_backup/        Source         1602:06:04  Idle
home:/vol/h_root/-    vault1:/vol/home_backup/h_root  Source         1578:07:40  Idle

Listing of the .snapshot directory reveals an mtime of Dec 2, even on directories where the live filesystem has had modified files since that date.

drwx--x--x  13 user  staff       8192 Dec  2 17:35 Feb-01

A snapvault status -l reveals the following information:

Snapvault is ON.

Source:                 home:/vol/home/
Destination:            vault1:/vol/home_backup/
Status:                 Idle
Progress:               -
State:                  Source
Lag:                    1602:15:24
Mirror Timestamp:       Fri Dec  2 00:01:56 PST 2011
Base Snapshot:          sv.7
Current Transfer Type:  -
Current Transfer Error: could not read from socket
Contents:               -
Last Transfer Type:     -
Last Transfer Size:     28618316 KB
Last Transfer Duration: 04:02:46
Last Transfer From:     -

Source:                 home:/vol/h_root/-
Destination:            vault1:/vol/home_backup/h_root
Status:                 Idle
Progress:               -
State:                  Source
Lag:                    1578:17:00
Mirror Timestamp:       Sat Dec  3 00:00:20 PST 2011
Base Snapshot:          sv.7
Current Transfer Type:  -
Current Transfer Error: could not read from socket
Contents:               -
Last Transfer Type:     -
Last Transfer Size:     1488 KB
Last Transfer Duration: 00:00:04
Last Transfer From:     -

On the destination end, we have this output for snapvault status:

Source                               Destination                                State          Lag        Status
home:/vol/home/                      vault1:/vol/home_backup/                   Snapvaulted    1602:24:40  Quiescing
home:/vol/h_root/-                   vault1:/vol/home_backup/h_root             Snapvaulted    1578:26:16  Quiescing

Both source and destination filers are running NetApp Release 8.0.2.

netapp snapshot filer

1 Answers

  1. The destination status says that it's quiescing, meaning that it's trying to stop the transfer cleanly... It has clearly been like this for quite a while, unless you recently tried to quiesce the transfer.

    First I would try to create a new transfer with dummy volumes and see if you can get a new one to transfer correctly, that'll help troubleshoot some... Assuming that works I would break the snapvault transfers and then try to reinitialize them one at a time.

    You may also want to check to see if you have any throttling set up, as sometimes the throttle config can be confusing and you may have accidentally set it much lower than expected.

    Did anything in your network change that may prevent the source from talking to the destination? I've run into a bug with Cisco code that caused our snapmirror traffic to show up as asynchronous routing where our firewall kept dropping the packets erroneously.

    Is the CPU high on the system? a 3050 is a pretty old system, it's possible that you're overloading it and it's unable to keep up with the snapvault overhead while serving user data. You would only see this if you're up in the high 90s consistently, it's not a very common issue.

    • 1