Richard Beier's questions

We have a Windows Server 2003 SP2 VM running on Hyper-V (Server 2008 R2 host). The VM has Hyper-V time synchronization enabled.

I noticed that the time on the VM was fast by around 25 minutes. I saw the following in the event log:

• The time provider NtpClient is configured to acquire time from one or more time sources, however none of the sources are currently accessible. No attempt to contact a source will be made for 15 minutes. NtpClient has no source of accurate time.

• The time provider NtpClient cannot reach or is currently receiving invalid time data from ourdc.ourdomain.local (ntp.d|192.168.2.18:123->192.168.2.2:123).

• Time Provider NtpClient: No valid response has been received from domain controller ourdc.ourdomain.local after 8 attempts to contact it. This domain controller will be discarded as a time source and NtpClient will attempt to discover a new domain controller from which to synchronize.

I had been logged in as a local user. (We have an old app that runs on this VM - it requires a user to be logged in at all times, and we use a non-domain user account for this.)

When I logged in as a domain user, the clock almost immediately corrected itself. Running "w32tm /monitor" and "net time" as the domain user showed no errors, and indicated that our domain controller was the time source.

Does anyone know what might cause this, and why logging in under a domain account fixes the problem? I'm wondering if the time will start to drift again.

Thanks for your help,
Richard

We have several files on a Windows server that show up in directory listings but can't be accessed. I can take ownership of the files, and the ACL shows that Everyone has read/write/execute access; but I can't open the files. chkdsk finds no problems on the volume.

The server is 64-bit Win 2008 R2 Standard SP1.

We had a BSOD recently on this server, and the files were modified shortly before the crash. The crash wasn't in anything filesystem-related, but I'm thinking it corrupted something in kernel memory and that corrupted the filesystem.

Here is some of what I'm seeing when trying to access file "com2.log":

D:\Logs\Application>dir com2*
Volume in drive D is Data
Volume Serial Number is 84D5-FD84

Directory of D:\Logs\Application

09/07/2011  06:57 PM           169,025 com2.log
       1 File(s)        169,025 bytes
       0 Dir(s)  39,320,612,864 bytes free

D:\Logs\Application>dir com2.log

 Directory of \\.

File Not Found

D:\Logs\Application>dir com2.*

Directory of \\.

File Not Found

D:\Logs\Application>cacls com2.log
The filename, directory name, or volume label syntax is incorrect.

D:\Logs\Application>more com2.log
Cannot access file \\.\com2

D:\Logs\Application>more com2*
Cannot access file D:\Logs\Application\com2.log

(cacls doesn't show the ACL, but I can see it in Windows Explorer.)

It seems like it might be a problem with the extension. dir com2* finds it but dir com2.* does not.

The file shows up as com2.log in Windows explorer, but when I double-click it, I get this error in Notepad referring to a .txt file extension:

Cannot find the \\.\com2.txt file.
Do you want to create a new file?

Has anyone seen anything like this before? Any suggestions for troubleshooting?

We're having a problem with SVN... Subversion clients such as TortoiseSVN hang when we commit any more than a few files at a time to our server. Everything appears to actually be committed successfully to the repository; but the client hangs after all the data has been transmitted.

We're using version 1.4.4 of the SVN server. We use the svn:// protocol rather than http to connect.

We've reproduced this problem with several clients: TortoiseSVN (1.6.10), AnkhSVN (2.1), and the Silk command-line client (1.6.12). This is happening for everyone on the team, though some people seem to be more affected than others. If someone commits only a few files, it often works; but with more than half a dozen files, it usually hangs.

Does anyone have troubleshooting suggestions? This has been happening sporadically for a while, but it's become pretty consistent lately. We've been working around the issue by killing the hung SVN client, doing "svn cleanup", and then doing "svn up"; but sometimes that causes tree conflicts. Another workaround is to blow away the workspace and check it out again after every commit; but of course that's pretty annoying.

Are there any diagnostics that could help us troubleshoot this? We're considering upgrading to SVN 1.6 server, and installing the server on a new machine; but we're wondering if there's an easier solution.

Thanks for your help,
Richard

We have a user with two Outlook accounts. [email protected] is of type Exchange; [email protected] is of type IMAP/SMTP. Both are actually on our Exchange server; but since an Outlook profile can only have one Exchange account, the second one is set up as IMAP.

The user would like to send a meeting request from her xyz.com account, so the "from" address appears as [email protected]. Unfortunately that doesn't work. If she creates the meeting in her xyz.com calendar, the meeting request still goes out through her Exchange account, [email protected]. The meeting request "compose message" window has an Account dropdown below the Send button, but this has no effect.

Before she sends the invitation, a warning appears:

"Responses to this meeting request will not be tallied because this meeting is not in your main Calendar folder. Is this OK?"

Is there any workaround for this? We're using Outlook 2007 and Exchange 2003 SP2.

Thanks,
Richard

We're setting up an IIS7 web farm with two servers. Should each server have its own local copy of the content, or should they pull content directly from a UNC share? What are the pros and cons of each approach?

We currently have a single live server WEB1, with content stored locally on a separate partition. A job periodically syncs WEB1 to a standby server WEB2, using robocopy for content and msdeploy for config. If WEB1 goes down, Nagios notifies us, and we manually run a script to move the IP addresses to WEB2's network interface. Both servers are actually VMs running on separate VMWare ESX 4 hosts. The servers are domain-joined.

We have around 50-60 live sites on WEB1 - mostly ASP.NET, with a few that are just static HTML. Most are low-traffic "microsites". A few have moderate traffic, but none are massive.

We'd like to change this so both WEB1 and WEB2 are actively serving content. This is mainly for reliability - if WEB1 goes down, we don't want to have to manually intervene to fail things over. Spreading the load is also nice, but the load is not high enough right now for us to need this.

We're planning to configure our firewall to balance traffic across the two servers. It will detect when a server goes down and will send all the traffic to the remaining live server. We're planning to use sticky sessions for now... eventually we may move to SQL Server session state and stateless load balancing.

But we need a way for the servers to share content. We were originally planning to move all the content to a UNC share. Our storage provider says they can set up a highly available SMB share for us. So if we go the UNC route, the storage shouldn't be a single point of failure. But we're wondering about the downsides to this approach:

• We'll need to change the physical paths for each site and virtual directory. There are also some projects that have absolute paths in their web.config files - we'll have to update those as well.

• We'll need to create a domain user for the web servers to access the share, and grant that user appropriate permissions. I haven't looked into this yet - I'm not sure if the application pool identity needs to be changed to this user, or if there's another way to tell IIS to use this account when connecting to the share.

• Sites will no longer be able to access their content if there's ever an Active Directory problem.

• In general, it just seems a lot more complicated, with more moving parts that could break. Our storage provider would create a volume for us on their redundant SAN. If I understand correctly, this SAN volume would be mounted on a VM running in their redundant VMWare environment; this VM would then expose the SMB share to our web servers.

On the other hand, a benefit of the shared content approach is that we'd only need to deploy code to one place, and there would never be a temporary inconsistency between multiple copies of the content.

This thread is pretty interesting, though some of these people are working at a much larger scale.

I've just been discussing content so far, but we also need to think about configuration. I don't know if we can just use DFS replication for the applicationHost.config and other files, or if it's best to use the shared configuration feature with the config on a UNC share.

What do you think?

We're installing Microsoft Data Protection Manager 2007 - we'll be using it to back up Exchange and SQL Server among other things.

Does anyone know if DPM can also back up Active Directory?

It sounds like the answer is "not really". You can install the DPM agent on a domain controller and make system state backups. But if your Active Directory is out of commission, there will be no way to restore the backups, since DPM depends on AD.

Currently we're just using Windows Backup (ntbackup) to take system state backups on one of the DCs. Should we just continue with that?

Thanks,
Richard

We have a two-domain Active Directory forest: ourcompany.com at the root, and prod.ourcompany.com for production servers. Time is propagating properly through the root domain, but servers in the child domain are unable to sync via NTP. So the time on these servers is starting to drift, since they're relying only on the hardware clock.

WHen I type "net time" on one of the production servers, I get the following error:

Could not locate a time-server.
More help is available by typing NET HELPMSG 3912.

When I type "w32tm /resync", i get the following:

Sending resync command to local computer
The computer did not resync because no time data was available.

"w32tm /query /source" shows the following:

Free-running System Clock

We have three domain controllers in the prod.ourcompany.com subdomain (overkill, but the result of a migration - we haven't gotten rid of one of the old ones yet.) To complicate matters, the domain controllers are all virtualized, running on two different physical hosts. But the time on the domain controllers themselves is accurate - the servers that aren't DCs are the ones having problems.

Two of the DCs are running Server 2003, including the PDC emulator. The third DC is running Server 2008. (I could move the PDC emulator role to the 2008 machine if that would help.) The non-DC servers are all running Server 2008.

All other Active Directory functionality works fine in the production domain - we're only seeing problems with NTP.

I can manually sync each machine to the time source (the PDC emulator) by doing the following:

net time \\dc1.prod.ourcompany.com /set /y

But this is just a one-off, and it doesn't cause automated time syncing to start working. I guess I could create a scheduled task which runs the above command periodically, but I'm hoping there's a better way.

Does anyone have any ideas as to why this isn't working, and what we can do to fix it?

Thanks for your help,
Richard

We host a bunch of ASP.NET sites on an IIS7 server. Occasionally, we'd like to be able to log HTTP POST data to troubleshoot problems. IIS lets us log the query string, but not the POST data - at least, we haven't found a way.

Do you think it's safe to use Wireshark (or Netmon or another sniffer) on a production server? My gut feeling says "no" but I'd like to hear what others think.

It would be better to use port mirroring, and run the sniffer on a different box on the same switch. Unfortunately though, all the servers on that switch are production servers... so we'd have to affect one of them.

Thanks for your help,
Richard

We're running an IIS7 server hosting several dozen websites. Several of these websites are all part of the same legacy app we've developed. These sites all run the same code and run in the same app pool.

Roughly once a month over the past few months, we've found that all requests for this app pool start hanging indefinitely. When this happens, we receive an alert and we recycle the app pool. After that, the sites start working again.

This only ever affects this one app pool - never any others on the same server. A couple times, before recycling the pool, I've looked at the currently-executing requests in the worker process. They all show up as executing inside the WindowsAuthenticationModule. Which is strange, because the vast majority of the application does not require authentication. There is a small admin section which uses Windows auth... but all the other requests should be anonymous.

Does anyone have any idea as to what might be causing this?

There are several unusual things about the way these sites are set up. As I mentioned, they all run the same code - multiple sites point at the same physical directory. The only difference is the host header bindings. I'm not sure why there isn't just one site with all the host headers, but that's how it works.

In several of these sites, the same physical directory is mapped at two levels - as the root of the site and again as an application within the site. So if a user goes to http://oursite.com/index.aspx, that maps to c:\files\oursite\index.aspx. If a user goes to http://oursite.com/foo/index.aspx, that also maps to c:\files\oursite\index.aspx. I think there is code which looks at the request URL and handles the two requests differently.

This is strange because the same web.config ends up being interpreted as a site config file, and also as an application config file within the site. I don't know if this might be related to the authentication problem.

If we can't find the cause, we're thinking of a few workarounds we could try:

• Move the admin section into a separate site, and give the client a new admin URL. Run that separate site in its own app pool. Then in the web.config shared by all the other sites, remove the WindowsAuthenticationModule. That way there should be no possibility of a hang within the WindowsAuthenticationModule.

• Try running all these sites in the classic pipeline instead of the integrated pipeline. They were working fine on our old IIS6 server...

• (If we get desperate) Set up a watchdog script which monitors the sites and auto-recycles the app pool when it detects that requests are getting stuck.

What do you think?

Thanks for your help,
Richard

We're a small web development company. Our domain has two DCs: a main one (BEEHIVE, 192.168.3.20) in the datacenter and a second one (SPHERE2, 10.0.66.19) in the office. The office is connected to the datacenter via a VPN.

We recently had a brief network outage in the office. During this outage, we weren't able to access the domain from our office machines. I had hoped that they would fail over to the DC in the office, but that didn't happen. So I'm trying to figure out why. I'm not an expert on Active Directory so maybe I'm missing something obvious.

Both domain controllers are running a DNS server. Each office workstation is configured to use the datacenter DC as its primary DNS server, and the office DC as its secondary:

  DNS Servers . . . . . . . . . . . : 192.168.3.20
                                      10.0.66.19

Both DNS servers are working, and both domain controllers are working (at least, I can connect to them both using AD Users + Computers).

Here are the SRV records that point to the domain controllers (I've changed the domain name but I've left the rest alone):

C:\>nslookup
Default Server:  beehive.ourcorp.com
Address:  192.168.3.20

> set type=srv
> _ldap._tcp.ourcorp.com
Server:  beehive.ourcorp.com
Address:  192.168.3.20

_ldap._tcp.ourcorp.com  SRV service location:
          priority       = 0
          weight         = 100
          port           = 389
          svr hostname   = beehive.ourcorp.com
_ldap._tcp.ourcorp.com  SRV service location:
          priority       = 0
          weight         = 100
          port           = 389
          svr hostname   = sphere2.ourcorp.com
beehive.ourcorp.com     internet address = 192.168.3.20
sphere2.ourcorp.com     internet address = 10.0.66.19

Does anyone have any ideas?


Thanks,
Richard

I didn't get any help on the TechNet forum so I'm going to try my luck here :).


Our web server is running Server 2008 SP1, 64-bit Standard Edition. For the past week or so, we've been experiencing repeated corruption in the Application event log. Each time, we can fix the corruption by clearing the log. But the corruption always comes back, sometimes within an hour or two.

We have a PowerShell script which checks the event log periodically and emails us about error events. Once the log has become corrupt, running "get-eventlog Application" from PowerShell produces the following output:

Get-EventLog : Cannot read log entry number 696. The event log may be corrupt. At line:1 char:13
+ get-eventlog <<<< Application

Usually there are several valid event log entries displayed before the corruption error.

The other event logs (System, Security, and so on) are not experiencing this corruption.

If I look at the Application log in the event viewer, I see several events with no information at all. In the list view, they show the "information" icon, but all other columns are blank. In the "general" and "details" tabs, everything is blank.


I thought about disk corruption. But since this is only affecting the event log, I'm skeptical that the disks are the cause. The server has a single RAID-1 array which shows a status of healthy. Write caching is disabled. All applications and sites hosted on the server appear to be working fine; only the event log is having problems.

On TechNet, someone replied saying it sounded like a PowerShell problem - but it isn't, since the corruption is also visible through the Event Viewer GUI.


I'd appreciate any ideas you might have for finding the cause of this problem...

Thanks,
Richard