Oliver Salzburg's questions

We're using a 3rd-party service provider to send transactional email. I recently noticed increased failure rates for a given receiving domain.

The sends fail with the error "498 No MX for example.com".

The sends are retried after a given delay and then usually succeed after a couple retries. But sometimes, they exceed the retry limit and are dropped permanently.

I contacted the support of the provider and they told me that this is due to the receiving domain declaring MX from different providers.

$ dig mx example.com
;; ANSWER SECTION:
example.com.        859     IN      MX      25 mail05.example.com.
example.com.        859     IN      MX      20 mail11.example.net.

They are referring to the fact that one MX is using example.com and the other is using example.net and that is apparently bad practice and can lead to the error described above.

This is the first time I'm hearing something like that and I would instantly call BS on it, but I thought I'd give them the benefit of the doubt and hear what others have to say on the topic.

After migrating to Office 365, our Outlook clients in the office still connect to the on-premise exchange. Mobile and remote clients already work fine.

What's going on? How do I fix this?

I went through the Hybrid Setup configuration, unaware that this would enable DirSync. I now want to disable AD synchronization again.

When I go to the Azure AD Admin Center and disable directory synchronization for our domain, the switch will go back to Activated right after it was deactivated.

I also tried disabling DirSync through PowerShell:

Set-MsolDirSyncEnabled -EnableDirSync $false -Verbose

This only results in the unhelpful error message:

Set-MsolDirSyncEnabled : You cannot turn off Active Directory synchronization.

I'm battling with a piece of UniFi hardware that refuses to be adopted by a controller.

Every attempt is rejected with the unhelpful message "Adoption failed". No other indication or information is available.

How can I determine what the actual problem is?

I've seen people use excessive quotes:

add_header 'Access-Control-Allow-Origin' '*';

I've seen people use no quotes:

add_header Access-Control-Allow-Origin *;

Both work fine as far as I know, so when do you actually have to use quotes?

When I try to synchronize my OneDrive for Business account, I get the error message:

The server you are trying to access is using an authentication protocol not supported by this version of Office

How can I get OneDrive to synchronize again?

I had a * record on one of our zones and I changed it. But when I queried certain, random names on that zone from public DNS servers, I still received the old IP address.

This makes no sense to me. The server I queried could not know the correct IP address for the name, unless they know there was a wildcard record. Otherwise, they would have to ask our name servers.

When I query our own name servers for the same name, they would return the new IP address.

So, what's going on?

We're currently implementing a backup solution for a client and their ERP solution uses SQL Server.

The ERP solution was set up by a different company. And they are telling me that it is super important to back up and truncate the transaction log.

I've been reading up a bit on this transaction log and I don't get why this is so important when I'm already backing up the whole machine anyway (We're using ArcServe UDP, which is aware of SQL Server and uses VSS). It is my understanding that cleanup tasks on the SQL Server VM are already taking care of truncating the log, however, UDP also allows SQL Server log truncation.

It is my understanding that the transaction log can be used to restore corrupted databases, because, well, it's a log of all transactions. But I already have an hourly backup of the whole database, so, why would I care?

We've recently upgraded the RAM on a Windows Storage Server 2008 R2 Standard to 32 GB.

When I open the System panel, it indicates Installed memory: 32.0 GB (16.0 GB usable). Why is that?

I've been trying to dig up documentation that would indicate that this is a limit of the operating system version, but I've come up short. Looking at the version comparison on Wikipedia shows no version with 16 GB, but Storage Server isn't listed and Memory Limits for Windows and Windows Server Releases doesn't list Storage Serer 2008 R2.

I've since found an infographic in the blog post titled Windows Storage Server 2008 R2 is now available!, which indicates that the limit would be 32 GB.

As suggested by @Nico, opening perfmon shows that 16 GB of memory are "reserved for hardware".

The board of the server is an Intel S3420GPLX set up with an Intel Xeon SP X3430, both of which are supposed to support 32 GB of memory.

The installed RAM modules are 4 x 8 GB Hynix HMT31GR7CFR8A-H9 RDIMM.

Support advised us to perform a firmware upgrade of the mainboard. We performed the firmware upgrade, but it had no effect on the issue

On our main domain, example.com we're hosting a lot of websites. So we just added a * A/AAAA record to our zone and pointed it at our webserver.

Sadly, this also causes Outlook to constantly hammer our webserver, because now autodiscover.example.com points at our webserver. Not only will Outlook constantly try to connect to it, it will also pop open a warning, that the SSL certificate served by autodiscover.example.com isn't valid for that domain.

To counter that issue, I added an explicit autodiscover A record and pointed it to 127.0.0.1, but that just causes other issues (especially when I'm running a webserver locally).

So, ideally, I would want to mark autodiscover as nonexistent, so that Outlook will skip to a different auto-discovery mechanism. I realize that I could simply maintain a proper list of valid names in our zone, but I was hoping for something that requires less maintenance.

I'm using some Perl-based scripts for service checks in Nagios and I get (Service check did not exit properly) and (null) as the result in Nagios, but the script works great on the command line.

I've seen solutions online suggesting to disable the internal Perl interpreter by setting enable_embedded_perl=0 in the Nagios configuration or by specifying the path to an interpreter explicitly. This did not help with the issue.

What else could it be?

So far, I've used a user group policy object utilizing Internet Explorer maintenance to set a proxy for the user in IE. We have now deployed the Enterprise Client (EC) starter group policy to our domain and this policy affects this behavior.

The EC group policy uses the policy Make proxy settings per-machine (rather than per-user). This policy describes itself as:

This policy is intended to ensure that proxy settings apply uniformly to the same computer and do not vary from user to user.

Great! This seems to be an improvement over my previous setup.

If you enable this policy, users cannot set user-specific proxy settings. They must use the zones created for all users of the computer.

What zones and where do I configure the proxy settings for them?

I assumed the policy would simply take the user settings and apply them, but that's not what's happening. Now no proxy server is set at all. So my previous settings obviously no longer have any effect.

So far, I've only come up with solutions that involved direct manipulation of the Windows registry. Which is fine, I guess, but the way the proxy is configured for users makes it appear as if there could be a higher level approach.

fail2ban just locked me out of our website because something from my desktop was hammering port 443 on the server (which is not in use).

I saw my IP also requesting "GET /autodiscover/autodiscover.xml HTTP/1.1", so I assume that's what's going on on port 443 as well.

But I only have 1 email account configured in Outlook and it's working just fine.

The account is for the address [email protected] and said server will answer for example.com, but that server is not our MX and it is also not configured as an Exchange server in my mail account.

So, why is Outlook still trying to retrieve those auto-configuration settings?

Since installing updates yesterday, one of our 2008 R2 servers refuses to connect to our WSUS again and, instead, reports the unknown error 0x80244019. Connecting to the official Windows Update repository works without any issues. The problem happens only with our local WSUS.

The Windows support website on the error suggests:

If you receive one of these errors while downloading updates, the most common cause is a computer virus that has turned off Windows Update, or another service on your computer that is needed by Windows Update.

I'm gonna vote No on that one.

My WindowsUpdate.log shows the following:

2012-09-13  13:00:52:738     892    5c0 PT  +++++++++++  PT: Synchronizing server updates  +++++++++++
2012-09-13  13:00:52:738     892    5c0 PT    + ServiceId = {3DA21691-E39D-4DA6-8A4B-B43877BCB1B7}, Server URL = http://SRV-PDC/ClientWebService/client.asmx
2012-09-13  13:00:52:769     892    5c0 PT  WARNING: Cached cookie has expired or new PID is available
2012-09-13  13:00:52:769     892    5c0 PT  Initializing simple targeting cookie, clientId = a6c96caf-d9ca-4f31-a003-827e7089ff64, target group = Server, DNS name = srv-exchange.porta.haseke.de
2012-09-13  13:00:52:769     892    5c0 PT    Server URL = http://SRV-PDC/SimpleAuthWebService/SimpleAuth.asmx
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: GetAuthorizationCookie failure, error = 0x80244019, soap client error = 10, soap error code = 0, HTTP status code = 404
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: Failed to initialize Simple Targeting Cookie: 0x80244019
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: PopulateAuthCookies failed: 0x80244019
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: RefreshCookie failed: 0x80244019
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: RefreshPTState failed: 0x80244019
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: Sync of Updates: 0x80244019
2012-09-13  13:00:52:957     892    5c0 PT  WARNING: SyncServerUpdatesInternal failed: 0x80244019
2012-09-13  13:00:52:957     892    5c0 Agent     * WARNING: Failed to synchronize, error = 0x80244019
2012-09-13  13:00:52:957     892    5c0 Agent     * WARNING: Exit code = 0x80244019

And, actually, another support article caught my eye: You cannot download updates when you access the Windows Update Web site from a Windows XP-based computer that is behind a firewall or a proxy server

That article refers to XP, but I had to recently tweak with the WinHTTP settings on that server to resolve yet another issue (Please see the support article Error message when you import a third-party certificate into Exchange Server 2010: "The certificate status could not be determined because the revocation check failed" for further details).

I fixed that access problem by running:

netsh winhttp set proxy proxy-server="http=www-proxy:8080" bypass-list="*.domain.example.com"

So, I'm assuming Windows Update uses WinHTTP as well, and my changes prevent it from working properly. But even after adjusting my bypass-list to "*.domain.example.com;<local>", Windows Update refuses to function.

I have a database in which there is one table where one column has invalid data.

I also have a backup of said database of a few days before. In that backup, that column still has the valid data. But other tables in that database are already incredibly outdated.

So, a full rollback to that backup is not an option. Rather, I would like to only import that single, corrupted column from the backup into the current database.

The live database has also accumulated a few new, valid rows in that table. Obviously, we would like those to be preserved.

I can't think of a way that doesn't involve heavy, manual editing. Can this be automated?

We have a Windows domain network with 2 Hyper-V hosts. On these hosts, there are several Hyper-V VM clients. Two of these clients are Domain Controllers.

The Hyper-V hosts are both clients of that domain. This brings one to a common time synchronization issue, as the hosts will try to sync their time with the PDC emulator and the domain controllers sync their time with the Hyper-V host.

While working on that issue, I didn't realize that the firewall in the network does not allow outgoing traffic on port 123. Which lead to several (failed) reconfiguration attempts.

Switching the time source

Once the problem was understood, I set the hosts up to sync to a reliable, on-site time source and wanted the DCs to simply use the VM IC Time Synchronization Provider again (so that they would sync the time with the host).

But one of the DCs is refusing to switch to the VM IC Time Synchronization Provider, it still uses the Local CMS Clock source (which, to my understanding, is not desirable).

What failed so far?

• I tried disabling the Time Synchronization component for that VM in the Hyper-V Manager, then rebooting the machine, restarting the w32time service a couple of times, re-enabling the Time Synchronization component. That didn't work. I also tried other permutations of this process that didn't help either.

• I tried changing the Enabled values in the registry (HKLM\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders) for the different time providers.

• I re-installed the Hyper-V integration package in the client.

I'm currently in the process of moving all of our domains to our own nameservers. Which wasn't an issue until I hit our own .de domain. I (think I) understand the implications of having the NS inside it's own domain, hence the need for glue records.

Until yesterday, I would have assumed I have a pretty good understanding of Bind and DNS zones until I was presented with this error from the Denic nameserver predelegation check:

Inconsistent set of nameserver IP addresses (NS, provided glues, determined glues)

• ns2.hartwig-at.de
• [88.198.242.190/88.198.242.190]
• Default resolver determined: [], other resolvers determined: {88.198.242.190/88.198.242.190=[/2a01:4f8:d13:3c85:0:0:0:2, /88.198.242.190]}

Inconsistent set of nameserver IP addresses (NS, provided glues, determined glues)

• ns1.hartwig-at.de
• [cloud.hartwig-at.de/176.221.46.23]
• Default resolver determined: [], other resolvers determined: {cloud.hartwig-at.de/176.221.46.23=[/2a00:1158:3:0:0:0:0:b6, /176.221.46.23]}

Screenshot of the result

The support of my registrar is either far better educated than me or doesn't have a clue. Either way, they're avoiding my questions in regards to what this error means. They just tell me

Your nameserver has to return your own nameservers as the default resolver.

But that doesn't make any sense to me and they refuse to try to explain it any other way.

This is the head of my current zone file:

@               86400   IN SOA          ns1.hartwig-at.de. hostmaster.hartwig-at.de. (
                                        2012070505 ; serial
                                        1d         ; refresh
                                        3h         ; retry
                                        4w         ; expiry
                                        1h )       ; minimum

                3600    IN NS           ns1.hartwig-at.de.
                3600    IN NS           ns2.hartwig-at.de.

                3600    IN MX 10        remote.hartwig-at.de.
                3600    IN MX 20        mx1.hartwig-at.de.
                3600    IN MX 30        mx2.hartwig-at.de.

localhost       3600    IN A            127.0.0.1
localhost       3600    IN AAAA         ::1
@               3600    IN A            176.221.46.23
                3600    IN AAAA         2a00:1158:3::b6
*               3600    IN A            176.221.46.23
                3600    IN AAAA         2a00:1158:3::b6

hetzner         3600    IN A            88.198.242.190
hetzner         3600    IN AAAA         2a01:4f8:d13:3c85::2
cloud           3600    IN A            176.221.46.23
cloud           3600    IN AAAA         2a00:1158:3::b6

; List all NS as A/AAAA record
ns              3600    IN A            176.221.46.23
ns              3600    IN AAAA         2a00:1158:3::b6
ns1             3600    IN A            176.221.46.23
ns1             3600    IN AAAA         2a00:1158:3::b6
ns2             3600    IN A            88.198.242.190
ns2             3600    IN AAAA         2a01:4f8:d13:3c85::2

So, what is the problem with my zone? And what is the "default resolver"?

I am currently setting up a new Debian (6.0.5) server. I put Cacti (0.8.7g) on it yesterday and have been battling with it ever since.

Initial issue

The initial issue I was observing, was that my graphs weren't updating. So I checked my cacti.log and found this concerning message:

POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.

That can't be good, right? So I went checking and started poller.php myself (via sudo -u www-data php poller.php --force). It will pump out a lot of messages (which all look like what I would expect) and then hang for a minute. After that 1 minute, it will loop the following message:

Waiting on 1 of 1 pollers.

This goes on for 4 more minutes until the process is forcefully ended for running longer than 298s.

So far so good

I went on for a good hour trying to determine what poller might still be running, until I got to the conclusion that there simply is no running poller.

Debugging

I checked poller.php to see how that warning is issued and why. On line 368, Cacti will retrieve the number of finished processes from the database and use that value to calculate how many processes are still running. So, let's see that value!

I added the following debug code into poller.php:

$finished_processes = db_fetch_cell("SELECT count(*) FROM cacti.poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'");
print "Finished: " . $finished_processes . " - Started: " . $started_processes . "\n";

Result

This will print the following within a second of starting poller.php:

Finished: 0 - Started: 1
Waiting on 1 of 1 pollers.
Finished: 1 - Started: 1

So the values are being read and are valid. Until we get to the part where it keeps looping:

Finished:  - Started: 1
Waiting on 1 of 1 pollers.

Suddenly, the value is gone. Why? Putting var_dump() in there confirms the issue:

NULL
Finished:  - Started: 1
Waiting on 1 of 1 pollers.

The return value is NULL. How can that be when querying SELECT COUNT()...? (SELECT COUNT() should always return one result row, shouldn't it?)

More debugging

So I went into lib\database.php and had a look at that db_fetch_cell(). A bit of testing confirmed, that the result set is actually empty.

So I added my own database query code in there to see what that would do:

$finished_processes = db_fetch_cell("SELECT count(*) FROM poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'");
print "Finished: " . $finished_processes . " - Started: " . $started_processes . "\n";

$mysqli = new mysqli("localhost","cacti","cacti","cacti");
$result = $mysqli->query("SELECT COUNT(*) FROM poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00';");
$row = $result->fetch_assoc();
var_dump( $row );

This will output

Finished:  - Started: 1
array(1) {
  ["COUNT(*)"]=>
  string(1) "2"
}
Waiting on 1 of 1 pollers.

So, the data is there and can be accessed without any problems, just not with the method Cacti is using?

Double-check that!

I enabled MySQL logging to make sure I'm not imagining things. Sure enough, when the error message is looped, the cacti.log reads as if it was querying like mad:

06/29/2012 08:44:00 PM - CMDPHP: Poller[0] DEVEL: SQL Cell: "SELECT count(*) FROM cacti.poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'"
06/29/2012 08:44:01 PM - CMDPHP: Poller[0] DEVEL: SQL Cell: "SELECT count(*) FROM cacti.poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'"
06/29/2012 08:44:02 PM - CMDPHP: Poller[0] DEVEL: SQL Cell: "SELECT count(*) FROM cacti.poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'"

But none of these queries are logged by MySQL. Yet, when I add my own database query code, it shows up just fine.

^{click to enlarge}

What the heck is going on here?

Digging deeper...

I concluded that the database connection must be lost somewhere in the process and adodb simply doesn't care.

So after a bit of digging around, I finally placed the debug message in drivers/adodb-mysql.inc.php, line 529, in the _close function. I wanted to see when the connection is closed.

^{I actually (finally) turned on PHP debugging and realized mysql_query() was called with a boolean connection id (an indicator of an intentionally closed connection).}

// returns true or false
function _close()
{
    @mysql_close($this->_connectionID);
    echo "!!!! CLOSED !!!!\n";
    debug_print_backtrace();
    $this->_connectionID = false;
}

What does that print?

oliver@j27773:/etc/php5/conf.d$ sudo -u www-data php /usr/share/cacti/site/poller.php --force
06/30/2012 01:33:49 AM - POLLER: Poller[0] NOTE: Poller Int: '60', Cron Int: '300', Time Since Last: '61', Max Runtime '298', Poller Runs: '5'
06/30/2012 01:33:49 AM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Process [CMD: /usr/bin/php, ARGS: -q "/usr/share/cacti/site/cmd.php" 0 3]
Finished: 0 - Started: 1
Waiting on 1 of 1 pollers.
Finished: 1 - Started: 1
06/30/2012 01:33:50 AM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/lib/cacti/rra/j27773_servers_jiffybox_net_apache_sb_keepalive_100.rrd --template apache_sb_keepalive 1341012829:0
06/30/2012 01:33:50 AM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/lib/cacti/rra/j27773_servers_jiffybox_net_apache_reqpersec_95.rrd --template apache_reqpersec 1341012829:.0228409
06/30/2012 01:33:50 AM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/lib/cacti/rra/j27773_servers_jiffybox_net_apache_bytesperreq_90.rrd --template apache_bytesperreq 1341012829:13925.7
06/30/2012 01:33:50 AM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/lib/cacti/rra/j27773_servers_jiffybox_net_cpu_85.rrd --template cpu 1341012829:1
OK u:0.00 s:0.00 r:1.00
06/30/2012 01:33:50 AM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/lib/cacti/rra/j27773_servers_jiffybox_net_hdd_used_80.rrd --template hdd_used:hdd_total 1341012829:924741632:2677886976
OK u:0.00 s:0.00 r:1.00
OK u:0.00 s:0.01 r:1.00
OK u:0.00 s:0.01 r:1.00
06/30/2012 01:33:50 AM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/lib/cacti/rra/j27773_servers_jiffybox_net_apache_sb_wait_105.rrd --template apache_sb_wait 1341012829:9
OK u:0.00 s:0.01 r:1.00
OK u:0.00 s:0.01 r:1.00
06/30/2012 01:33:50 AM - SYSTEM STATS: Time:1.1261 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:7 RRDsProcessed:6
Loop  Time is: 1.1291718482971
Sleep Time is: 58.867464065552
Total Time is: 1.1325359344482
!!!! CLOSED !!!!
#0  ADODB_mysql->_close() called at [/usr/share/php/adodb/adodb.inc.php:2141]
#1  ADOConnection->Close() called at [/usr/share/cacti/site/lib/database.php:68]
#2  db_close() called at [/usr/share/cacti/site/poller.php:455]
^C06/30/2012 01:33:55 AM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user

And now I'm too tired to investigate that...

I recently installed WSUS in a network and am now waiting for it to download all the updates I want to provide to other clients.

The updates are already approved. The installation is set up to only download approved updates. That is what it is doing right now.

So, this is going to take a while. In the meantime, I set up my group policies to make my clients aware of this new service.

Sadly, my Windows 7 test client (freshly installed), doesn't seem to think there are any updates available:

So, now I'm asking myself, is my setup broken? Or do I simply have to wait for those 140GB to download before it will all start to function properly.

The WindowsUpdate.log on my test client seems to tell that my correct server is being used:

2012-05-19  15:44:25:523     944    43c AU  ###########  AU: Initializing Automatic Updates  ###########
2012-05-19  15:44:25:523     944    43c AU    # WSUS server: http://SRV-PDC

The log is also pretty clear on the number of updates that should be applied:

2012-05-19  15:44:27:133     944    85c Agent     * Found 0 updates and 61 categories in search; evaluated appl. rules of 174 out of 895 deployed entities
...
2012-05-19  15:44:27:149     944    754 AU    # 0 updates detected

The reportingevents.log also indicates that no updates are available:

{6DB29978-5D20-4D80-9088-5F87D0CC3417}  2012-05-19 20:13:23:388+0200    1   147 101 {00000000-0000-0000-0000-000000000000}  0   0   AutomaticUpdates    Success Software Synchronization    Windows Update Client successfully detected 0 updates.
{03B0D7A7-A599-41CE-B583-F2870CEF4225}  2012-05-19 20:13:23:388+0200    1   156 101 {00000000-0000-0000-0000-000000000000}  0   0   AutomaticUpdates    Success Pre-Deployment Check    Reporting client status.

I started setting up a deb repository today for some of our applications. Given that they are interpreted scripts, I went with "all" architecture (from conf/distributions):

Codename: lucid
Architectures: all

But reprepro will print a warning when I try to add a .deb to the repository:

WARNING: Distribution lucid contains an architecture called 'all'.

Now I wonder why it warns about that? What's the problem?