The C:
drive on our SBS 2008 server just ran out of space. The culprit appears to be IIS, which is logging an incredible amount of activity in one particular log folder. Most of the IIS log folders look normal, but each of the daily files in C:\inetpub\logs\LogFiles\W3SVC1372222313
is at least 4.4MB, with the largest being yesterday's, at 1.67 GB!
The largest ones I can't even open on the server, but I've examined several of the smaller ones. They all show several dozen entries being made every few minutes that look like this:
2009-07-11 00:00:02 fe80::5558:434c:a610:405a%10 POST /ApiRemoting30/WebService.asmx - 8530 [DOMAINNAME]\[SERVERNAME$] fe80::5558:434c:a610:405a%10 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.4016) 200 0 0 3
Typically 40 or 50 of these entries will be made in the same second, with gaps of 2-5 minutes between each batch of entries. The other 1 percent of the entries in the file appear to involve WSUS.
I'm going to delete most of these files, because I don't really have a choice, but I'd like to know what's causing this out-of-control logging and how to put a lid on it in the future.
UPDATE: Okay, I've been able to examine a few more files. The bloat is apparently being caused by something going wrong when someone (i.e., me or another admin) logs in to WSUS interactively:
The trouble begins with a single log entry with no username (just "
-
"). It gets an HTTP status of401.2
and asc-win32-status
code of5
.This is followed by a long stretch of entries that alternate between no username and my own username. The ones with no username have an HTTP status of
401.1
and asc-win32-status
of2148074254
. The ones with my username are normal HTTP200
entries.
So as far as I can tell what appears to be happening is that when I log in to administer WSUS via the SBS console, NTLM authentication is not persisting behind the scenes, causing continual reauthentication attempts throughout the session, transparently to me. Hundreds of these entries are being created every second, adding about 70MB per hour to the log file. I have no idea why this is happening.
That's IPv6-based access to WSUS that you're seeing there.
Temporarily disable logging so that you don't fill the drive again:
That'll stop the logs from building up.
I can't say that I've seen WSUS-related traffic build up logs that big before. 4.4MB in a day isn't unheard of, but the 1.67GB in a day means that something has gone wrong.
Yesterday's log file is going to tell you lots about what was occurring. I find it hard to believe that it was all WSUS traffic. I wonder if something else didn't start banging on the server computer. Get that larger log file off of the machine and have a look at it.
Your log looks like it's in the W3C extended format. The format of that log file appears to be:
Date, Time, source IP address, HTTP request method, URI stem, probably URI query, server port, username, server IP address, user agent, HTTP result, probably Win32 status, and probably time taken
(The "probably" fields are because I can't be sure without seeing more of the file.) The header on the file will tell you the format for sure.
You need to get a look at that 1.67GB file-- it's gonna tell you what's up. Logging disabled on the site will prevent the hard drive from filling up again, but you want to know what's happening, behind the scenes, since it's going to be impacting server performance in some manner. Ultimately, you want to get to the bottom of the cause and then get logging enabled again (so that you have an audit trail if you have to track down strangeness again in the future).
Solution:
SBS console is behaving normally. Apparently it's designed to do a crazy amount of logging. Go figure.
Here is the SBS Blog article that gives more detail on the solution: Recovering Disk Space on the C: Drive in Small Business Server 2008
Here is the SBS 2008 forum thread explaining why this is the answer: SBS 2008 console causing disk space issue (IIS log is going haywire)
I thank you.
Is WSUS working ok? You could try running the WSUS diagnostic tool.
Microsoft Windows Server Update Services Tools and Utilities
On one side of the equation, you still want to figure out why you are having so much log traffice being generated (no suggestions there...), but I have found that the log folder is a good candidate for NTFS compression. Those text files compress nicely, and since you rarely open the log files, you will likely not even notice that they are compressed.
On a related note, you may want to consider turning off logging permanently for the WSUS site. Granted it may be useful for troubleshooting WSUS problems, but you can always turn it on as needed.
Having said that, I would look into the root cause of all the log entries and address that issue, which will then make my previous sentence a moot point. ;)
I'm experiencing exactely the same problem and just stumbled over it because the system partition was filling up. Since the day I started the server this very error is written into the log files multiple times per second in the manner discribed above. The peak was a file with about 270mb.
I find it very strange that the source AND destiniation IP addresses are always the same. this leads to the thought that the server has a problem talking to itself. I already found an article addressing this issue but it didn't help in my case: http://verbalprocessor.com/2008/06/03/sccm-and-wsus-on-server-2008/
did anybody find out anything new to this?