I can DIR a file and it doesn't exist, but DIRing its directory, or a file in the same directory as it, works fine.
Is this a hardware or software problem? If it's hardware, is it the drive or something worse?
2013-06-13 9:35 C:\>dir E:\Shares\Users\Test\Desktop\XAV-1.htm
Volume in drive E is BKUP2013-1
Volume Serial Number is 1AEA-6007
Directory of E:\Shares\Users\Test\Desktop
File Not Found
Yet:
2013-06-13 9:36 C:\>dir E:\Shares\Users\Test\Desktop
Volume in drive E is BKUP2013-1
Volume Serial Number is 1AEA-6007
Directory of E:\Shares\Users\Test\Desktop
2013-06-12 07:15 PM <DIR> .
2013-06-12 07:15 PM <DIR> ..
[snip]
2006-05-04 03:48 PM 1,232 Customer Files.lnk
2009-09-03 09:06 AM 767 Internet Explorer.lnk
2010-03-11 09:49 AM 104 My Computer.lnk
2004-01-02 04:50 PM 1,221 Reference Material.lnk
2011-11-18 05:08 PM 482 Shortcut to Downloads.lnk
2013-04-27 03:07 AM <DIR> Unused Desktop Shortcuts
2013-06-12 02:41 PM 2,710 XAV-1.htm
2011-01-10 03:31 PM 21,637,284 XP_EzTrends_1.0.3835.exe
11 File(s) 27,977,417 bytes
3 Dir(s) 670,902,140,928 bytes free
And a sibling:
2013-06-13 9:36 C:\>dir E:\Shares\Users\Test\Desktop\XP_EzTrends_1.0.3835.exe
Volume in drive E is BKUP2013-1
Volume Serial Number is 1AEA-6007
Directory of E:\Shares\Users\Test\Desktop
2011-01-10 03:31 PM 21,637,284 XP_EzTrends_1.0.3835.exe
1 File(s) 21,637,284 bytes
0 Dir(s) 670,902,140,928 bytes free
Furthermore, I can open XAV-1.htm no problem, and it has the same ACL profile as XP_EzTrends...exe.
Regarding ACL, if I right-click either C: or E: drives in My Computer, and hit Security, both of them show SYSTEM as having Full Control. As I understand it, this refers to NT AUTHORITY\SYSTEM
. UPDATE: Furthermore, the ACL is the roughly same as the old drive we used to use for E: which is now too small for us. I plugged it in and checked, and if anything the ACL is more restrictive on the old drive, with read-only for Users. However, for SYSTEM, Everyone, and Administrators, both old and new allow full access. (Which maybe is too loose, but then that's even more on the side of "it should be working.") UPDATE 2: I turned on auditing and checked the event log, which if it's like 2008 should make an event in the Security log with category Object Access
, but my Security log does not have any like that. I wonder if the footnote here about domain policies trumping applies here, even though this is on a domain controller. UPDATE 3: Okay, I have managed to get Object Access
entries to show up, by enabling Object Access logging in general in the domain / domain controller policy areas, in addition to requesting it on the specific folder, for Everyone, me, and SYSTEM, for any kind of access whatsoever. However, running a backup does not generate any entries, they come from using the MMC snap-on.
UPDATE 4: I modified the backup script to manual byte-for-byte verify all files in the Desktop directory in addition to the random statistical sample it normally does, and reformatted the drive in exFAT, so every file would be copied from scratch. I only got Success Audit
Object Access
entries when those files were touched by the verify step. I'm now trying the same thing on the second of the two drives, which was the flakier-seeming one, so we'll see what it says.
UPDATE 5: Oddly, the Test and SYSTEM users both have alternating Success Audit
entries from during the backup for the Desktop directory. No failures though. Meanwhile, I'm watching ROBOCOPY do its thing, and it has had no trouble for a good chunk of the backup. Now it's failing on every file, starting with this (last successful file and first two fails--since then it seems to do runs of successes and failures):
\C\Rebuild\Apache\httpd-2.2.24-win32\Apache2\modules\mod_version.so
New File 11776 2013/02/25 20:09:27 C:\scheduled\res \C\Rebuild\Apache\httpd-2.2.24-win32\Apache2\modules\mod_vhost_alias.so
New Dir 4 C:\scheduled\res\C\Rebuild\Backup\
New File 14.7 m 2010/12/01 17:09:00 C:\scheduled\res \C\Rebuild\Backup\cbSetup.exe 2013/07/10 17:37:04 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\cbSetup.exe The system cannot find the path specified. Waiting 1 seconds... Retrying...
New File 14.7 m 2010/12/01 17:09:00 C:\scheduled\res \C\Rebuild\Backup\cbSetup.exe 2013/07/10 17:37:06 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\cbSetup.exe The system cannot find the path specified. Waiting 1 seconds... Retrying...
New File 14.7 m 2010/12/01 17:09:00 C:\scheduled\res \C\Rebuild\Backup\cbSetup.exe 2013/07/10 17:37:08 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\cbSetup.exe The system cannot find the path specified. Waiting 1 seconds... Retrying...
New File 14.7 m 2010/12/01 17:09:00 C:\scheduled\res \C\Rebuild\Backup\cbSetup.exe 2013/07/10 17:37:09 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\cbSetup.exe The system cannot find the path specified.
ERROR: RETRY LIMIT EXCEEDED.
New File 2.0 m 2010/03/04 16:18:28 C:\scheduled\res \C\Rebuild\Backup\diffutils-2.8.7-1.exe 2013/07/10 17:37:10 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\diffutils-2.8.7-1.exe The system cannot find the path specified. Waiting 1 seconds... Retrying...
New File 2.0 m 2010/03/04 16:18:28 C:\scheduled\res \C\Rebuild\Backup\diffutils-2.8.7-1.exe 2013/07/10 17:37:11 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\diffutils-2.8.7-1.exe The system cannot find the path specified. Waiting 1 seconds... Retrying...
New File 2.0 m 2010/03/04 16:18:28 C:\scheduled\res \C\Rebuild\Backup\diffutils-2.8.7-1.exe 2013/07/10 17:37:12 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\diffutils-2.8.7-1.exe The system cannot find the path specified. Waiting 1 seconds... Retrying...
New File 2.0 m 2010/03/04 16:18:28 C:\scheduled\res \C\Rebuild\Backup\diffutils-2.8.7-1.exe 2013/07/10 17:37:13 ERROR 3 (0x00000003) Copying File C:\scheduled\res\C\Rebuild \Backup\diffutils-2.8.7-1.exe The system cannot find the path specified.
ERROR: RETRY LIMIT EXCEEDED.
And yet:
2013-07-1017:45 C:\>dir C:\scheduled\res\C\Rebuild\Backup\cbSetup.ex
Volume in drive C has no label.
Volume Serial Number is 3C18-E114
Directory of C:\scheduled\res\C\Rebuild\Backup
2010-12-01 01:09 PM 15,492,608 cbSetup.exe
1 File(s) 15,492,608 bytes
0 Dir(s) 240,190,017,536 bytes free
2013-07-1017:45 C:\>dir C:\Rebuild\Backup\cbSetup.exe
Volume in drive C has no label.
Volume Serial Number is 3C18-E114
Directory of C:\Rebuild\Backup
2010-12-01 01:09 PM 15,492,608 cbSetup.exe
1 File(s) 15,492,608 bytes
0 Dir(s) 239,572,938,752 bytes free
2013-07-1017:46 C:\>
UPDATE 6: Stranger still, it appears to have actually copied the file it supposedly gave up on, and with a strange datestamp (which it did for its sibling files in the same dir):
2013-07-1017:46 C:\>dir e:\IN\Rebuild\Backup\cbSetup.exe
Volume in drive E is BAERO2013-2
Volume Serial Number is 76F0-668E
Directory of e:\IN\Rebuild\Backup
1980-01-01 08:00 PM 15,492,608 cbSetup.exe
1 File(s) 15,492,608 bytes
0 Dir(s) 822,191,718,400 bytes free
2013-07-1018:05 C:\>
And, for ones I'm pretty sure didn't error out (it's past what I can scroll back to in the command window now), a random smattering of them have their timestamp messed up (ROBOCOPY's way of marking them incomplete):
2013-07-1018:05 C:\>dir e:\IN\Rebuild\Apache
Volume in drive E is BAERO2013-2
Volume Serial Number is 76F0-668E
Directory of e:\IN\Rebuild\Apache
2013-07-10 05:36 PM <DIR> .
2013-07-10 05:36 PM <DIR> ..
1980-01-01 08:00 PM 6,042,112 httpd-2.2.15-win32-x86-openssl-0.9.8m-r2.msi
2013-07-10 05:36 PM 80 httpd-2.2.15-win32-x86-openssl-0.9.8m-r2.msi.md5
1980-01-01 08:00 PM 6,085,632 httpd-2.2.17-win32-x86-openssl-0.9.8o.msi
2013-07-10 05:36 PM 76 httpd-2.2.17-win32-x86-openssl-0.9.8o.msi.md5
1980-01-01 08:00 PM 9,097,535 httpd-2.2.23-win32-ssl_0.9.8.zip
1980-01-01 08:00 PM 9,257,408 httpd-2.2.24-win32.zip
2012-11-06 05:23 AM 2,251 service-dependencies--unused.reg
2010-10-20 07:51 AM 2,410 services.bat
2013-05-28 06:57 AM 2,665 services-splittest.bat
2012-11-06 05:58 AM 2,448 services-upgradetest.bat
2013-07-10 05:36 PM <DIR> httpd-2.2.24-win32
10 File(s) 30,492,617 bytes
3 Dir(s) 821,313,273,856 bytes free
2013-07-1018:07 C:\>
Yet, as you can see, the files on C, in the volume shadow copy of C being backed up by ROBOCOPY, and E, which was formatted just before ROBOCOPY began, are byte-for-byte identical using cmp.exe:
2013-07-1018:13 C:\>c:\scheduled\res\cmp.exe --verbose c:\scheduled\res\C\Rebuild\Apache\httpd-2.2.24-win32.zip e:\IN\Rebuild\Apache\httpd-2.2.24-win32.zip
2013-07-1018:13 C:\>c:\scheduled\res\cmp.exe --verbose c:\Rebuild\Apache\httpd-2.2.24-win32.zip e:\IN\Rebuild\Apache\httpd-2.2.24-win32.zip
2013-07-1018:13 C:\>
UPDATE 7: Now I see some files going by saying The process cannot access the file because it is being used by another process.
and others saying The system cannot find the path specified.
. Both messages seem to be red herrings, because I'm running ROBOCOPY with /B which means it runs as Backup Operator and should have access to all local files, which is what these are. Also I'm looking at the Open Files MMC snap-in and they aren't. I even used LockHunter on one of the "being used by another process" files right after it complained, and LockHunter has never failed in my experience, yet it says nothing is using it. I've kicked all users off and I'm the only one logged in, and I'm certainly not touching random groups of files like this. I've disabled the task scheduler, and verified that my robocopy is the only robocopy process running right now.
UPDATE 8: Since reformatting to exFAT, I hadn't tried a CHKDSK /R
, so I did that. 0 KB in bad sectors, it says, after 7 hours (it's a 1 TB drive.)
This is on a weeks-old WD USB external drive. Time to take it back? Or is our USB port going? Or what?
UPDATE 9: We just tried a loaner 1 TB drive from a local business which is USB 2.0 instead of USB 3.0, just like our former drives were (but bigger). It doesn't exibit the exact same behaviour as the original question, but during the backup the verify step did fail saying the file didn't exist on E:, even though, afterwards, it does and I can DIR it alone unlike above.
0 Answers