T.J. Crowder's questions

In an AD environment as a local admin to a server, I went to give a colleague full permissions to a drive (right-click the drive, Security, Edit, add him and give him some rights) temporarily while waiting for him to be added to the correct AD group, and got this warning:

You are about to change permissions on the root directory of the startup disk. This can reduce the security of your computer and cause users to have problems accessing files. Do you want to continue?

I get the first part (security), but what is "cause users to have problems accessing files" talking about? Is that a catch-all in case the change I'm making reduces someone's access to the drive, or...?

One of my server's drives failed and so I removed the failed drive from all three relevant arrays, had the drive swapped out, and then added the new drive to the arrays. Two of the arrays worked perfectly. The third added the drive back as a spare, and there's an odd "removed" entry in the mdadm details.

I tried both

mdadm /dev/md2 --remove failed

and

mdadm /dev/md2 --remove detached

as suggested here and here, neither of which complained, but neither of which had any effect, either.

Does anyone know how I can get rid of that entry and get the drive added back properly? (Ideally without resyncing a third time, I've already had to do it twice and it takes hours. But if that's what it takes, that's what it takes.) The new drive is /dev/sda, the relevant partition is /dev/sda3.

Here's the detail on the array:

# mdadm --detail /dev/md2
/dev/md2:
        Version : 0.90
  Creation Time : Wed Oct 26 12:27:49 2011
     Raid Level : raid1
     Array Size : 729952192 (696.14 GiB 747.47 GB)
  Used Dev Size : 729952192 (696.14 GiB 747.47 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Tue Nov 12 17:48:53 2013
          State : clean, degraded 
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1

           UUID : 2fdbf68c:d572d905:776c2c25:004bd7b2 (local to host blah)
         Events : 0.34665

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       19        1      active sync   /dev/sdb3

       2       8        3        -      spare   /dev/sda3

If it's relevant, it's a 64-bit server. It normally runs Ubuntu, but right now I'm in the data centre's "rescue" OS, which is Debian 7 (wheezy). The "removed" entry was there the last time I was in Ubuntu (it won't, currently, boot from the disk), so I don't think that's not some Ubuntu/Debian conflict (and they are, of course, closely related).

Update:

Having done extensive tests with test devices on a local machine, I'm just plain getting anomalous behavior from mdadm with this array. For instance, with /dev/sda3 removed from the array again, I did this:

mdadm /dev/md2 --grow --force --raid-devices=1

And that got rid of the "removed" device, leaving me just with /dev/sdb3. Then I nuked /dev/sda3 (wrote a file system to it, so it didn't have the raid fs anymore), then:

mdadm /dev/md2 --grow --raid-devices=2

...which gave me an array with /dev/sdb3 in slot 0 and "removed" in slot 1 as you'd expect. Then

mdadm /dev/md2 --add /dev/sda3

...added it — as a spare again. (Another 3.5 hours down the drain.)

So with the rebuilt spare in the array, given that mdadm's man page says

RAID-DEVICES CHANGES

...

When the number of devices is increased, any hot spares that are present will be activated immediately.

...I grew the array to three devices, to try to activate the "spare":

mdadm /dev/md2 --grow --raid-devices=3

What did I get? Two "removed" devices, and the spare. And yet when I do this with a test array, I don't get this behavior.

So I nuked /dev/sda3 again, used it to create a brand-new array, and am copying the data from the old array to the new one:

rsync -r -t -v --exclude 'lost+found' --progress /mnt/oldarray/* /mnt/newarray

This will, of course, take hours. Hopefully when I'm done, I can stop the old array entirely, nuke /dev/sdb3, and add it to the new array. Hopefully, it won't get added as a spare!

One of the HDDs in my server's RAID config failed, so I took it out of the array and had the data center hot-swap it. They've done that, but now the new drive is /dev/sdc rather than /dev/sda. I suspect that if I reboot the server, it will be /dev/sda again, so I'm hesitant to add it back to the array as /dev/sdc because I don't want to lay a trap for myself to fall into on the next reboot. I'd just as soon not reboot the server if I don't need to (if I do need to, well, too bad for me).

If I add it as /dev/sdc, will there be a problem on reboot? Or is there some way to change the device name from /dev/sdc to /dev/sda without rebooting?

This is on Ubuntu 10.04 LTS. It's an md array ("Linux Software RAID"), where currently one of the devices (there are a couple of them) looks like this ("degraded" because I've removed the old /dev/sda from it):

# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Sun Oct 11 21:07:54 2009
     Raid Level : raid1
     Array Size : 97536 (95.27 MiB 99.88 MB)
  Used Dev Size : 97536 (95.27 MiB 99.88 MB)
   Raid Devices : 2
  Total Devices : 1
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Jun 30 09:31:16 2011
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           UUID : 496be7a5:ab9177ed:7792c71e:7dc17aa4
         Events : 0.112

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       0        0        1      removed

In many ways, this is the small gigabit switch version of my question about server RAM here. Basically, we have a few servers we want to connect through a gigabit switch (few = 4 at present, unlikely to grow at all; if it doubled I'd be truly shocked). When I was doing initial costings on the project, I found a variety of name-brand rackmount switches (such as this Cisco 48-port model) in the £200-500 range (ex-VAT). Our vendor is quoting us one for more than £2,000 (ex-VAT). They haven't given us the model number, but from what they have told us I suspect it's one of these, a "Cisco WS-C2960G-48TC-L - Catalyst 2960 48 10/100/1000, 4 T/SFP LAN Base Image" — for which their price is high, but not ridiculous.

This switch is to go into a rack that we will not grow our presence in, hooking into a network that the vendor is not in charge of (they're in charge of other networks in our deployment, but not this one, it's in a backup data centre). As far as I'm aware, there are no significant routing issues involved, the goal is simply that these four servers will talk fairly quickly to each other and the switch will have an upstream connection to the corporate net (not the internet); data that gets to the switch from the network at all need only be directed at the right server, not further clamped down, shaped, etc. I'm not even sure we need a managed switch.

I'm more a development guy than an admin guy, and I'm fairly aware of my limitations outside my area of expertise. (I definitely know a lot less about network switches than about RAM, vis-a-vis my other question.) Am I missing something? Is there a really solid justification for buying something so much larger and four times as expensive? Totally willing to believe it, but it seems worth asking the question. Naturally I'll be discussing it with the vendor, I just want to get some input from people who know more than I do first.

Edit And, of course, if you have a good experience with a particular brand or model for simple scenarios such as the above (must be rackmount), I'd love to hear them.

We're repurposing some old servers and need to put new RAM in them. There's a disconnect between the quote I'm getting from our usual supplier and what it seems like things should cost. (This is kind of an ongoing thing.) I'd like to know if, as more of a development guy than an admin guy, I'm just missing something. Wouldn't be the first time (but it wouldn't be the first time I'd pushed back on a similar point and been right, either).

The servers are old HP ProLiant DL380 G4 machines. The supplier is quoting us £275/2GB module of PC2-3200 DDR2 RAM, so bringing a server up to 4GB (which is all we really need) would be £550/machine.

But a quick check at crucial.com, walking through their selector and choosing that specific server model, tells me a two-pack of 2GB PC2-5300 DDR2 modules for £125 should do the trick at less than a quarter the cost. Edit: Yes, it's ECC, and yes, it's registered. Specifically, the link above tells us it's "DDR2 PC2-5300 • CL=5 • Single Ranked • Registered • ECC • DDR2-667 • 1.8V • 256Meg x 72". Edit: And as pipTheGeek points out, scan.co.uk has a very similar price (£140) on the Corsair-branded equivalent. (All of these are ex-VAT.)

The overview specs on the DL380 say "PC2-3200R DDR2", the quick technical specs say "Six PC2-3200R 400MHz DDR2 Sockets". It's my understanding — and apparently Crucial's — that I can run the PC2-5300s in the machine, albeit at PC2-3200 speeds.

So am I missing something? Is Crucial's stuff (which I've used for years without any trouble whatsoever) consumer-grade or dodgy or whatever in some way that makes it genuinely worth paying four times as much for something special? Or is the supplier being pedantic and sourcing PC-3200 RAM (which has to be relatively hard to find these days) because that's what the DL380 specs say, failing to consider that PC-5300 should work just fine? Or are they right that we shouldn't put PC-5300 stuff in the machines?

If I stick to one of the standard, well-supported VM disk images (like a raw image, or VDI, VMDK, ...), are Linux VMs typically easy to move between VM environments? E.g., between (say) VirtualBox and KVM, or VMWare and Xen? I'm talking here of fully virtualized environments, not paravirtualization requiring support within the guest OS.

It seems to me that the kernels in most Linux distributions these days are configured to...keep an open mind and detect things at boot time, so you don't have the issue that you sometimes have moving a Windows VM from one virtualization system to another (I'm thinking particularly of HAL issues that Windows has, like ACPI vs. non-ACPI; I've also just had Windows VMs generally acting strangely when moved from VMWare to VirtualBox, for instance).

I'm looking for a general answer, but if it helps, specifically I'm mostly going to be doing this with Ubuntu 8.04 LTS and 10.04 LTS guests. But that could change.

Summary

Freshly-installed (via apt-get) Tomcat on a freshly-installed Ubuntu 8.04 LTS 64-bit works, but doesn't (seem to) log anything. See also "What I've Tried" and "Closing In On It" below.

More Detail

Because the package maintainers have done a HUGE amount of the work, getting basic things working has been a fairly straightfoward matter of running apt-get and pointing Tomcat at the right JDK. And it works. But it doesn't appear to log anything.

As far as I can tell, the logging infrastructure is controlled via logging.properties files (since this is Tomcat 5.5, not Tomcat 5.0), including a central one (which you can override per web app) at /etc/tomcat55/logging.properties (/etc/tomcat55 is symlinked as /var/lib/tomcat55/conf). I'm not overriding it, and I can't see why I'm not seeing something. Here's the file:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

handlers = 1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli.FileHandler, 3manager.org.apache.juli.FileHandler, 4admin.org.apache.juli.FileHandler, 5host-manager.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler

.handlers = 1catalina.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler

############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################

1catalina.org.apache.juli.FileHandler.level = FINE
1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.FileHandler.prefix = catalina.

2localhost.org.apache.juli.FileHandler.level = FINE
2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.FileHandler.prefix = localhost.

3manager.org.apache.juli.FileHandler.level = FINE
3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.FileHandler.prefix = manager.

4admin.org.apache.juli.FileHandler.level = FINE
4admin.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
4admin.org.apache.juli.FileHandler.prefix = admin.

5host-manager.org.apache.juli.FileHandler.level = FINE
5host-manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
5host-manager.org.apache.juli.FileHandler.prefix = host-manager.

java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter


############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = DEBUG
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = 2localhost.org.apache.juli.FileHandler

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers = 3manager.org.apache.juli.FileHandler

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/admin].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/admin].handlers = 4admin.org.apache.juli.FileHandler

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].handlers = 5host-manager.org.apache.juli.FileHandler

# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
#org.apache.catalina.startup.ContextConfig.level = DEBUG
#org.apache.catalina.startup.HostConfig.level = DEBUG
#org.apache.catalina.session.ManagerBase.level = DEBUG

The only change I've made is:

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = DEBUG

...which was set to INFO like all the others, but I wanted to get it to log something, so I upped it to DEBUG. Naturally I've restarted Tomcat.

I can compile and view a JSP, so again Tomcat is working, but /var/lib/tomcat5.5/logs is stubbornly empty. Even if I force an error by having an invalid JSP or some such, I'm getting nothing.

What I've Tried

• I've been through the steps listed in the Tomcat FAQ for ensuring that the distro hasn't messed things up (they hadn't).
• I've gone looking for a logs directory elsewhere in case it's not /var/lib/tomcat5.5/logs (although that was created by the install).
• Naturally I've restarted Tomcat after making any changes.

One thing I haven't done is to chown everything to be owned by the tomcat55 user that the install created. logs is owned by tomcat55, but a number of the other directories and files are owned by root, though they're world-readable. I haven't done this because (hangs head) I can't quite figure out how to fully back-up the existing tree so I can readily undo the change. The blasted tree has absolute symlinks interweaving /etc/tomcat55, /var/lib/tomcat55, and /usr/share/tomcat55 that I know of. I could tar up each of those, but I don't know for certain I'm not missing something.

EDIT:

Closing In On It

Okay, so I've found that it's writing console-style stuff to /var/log/daemon.log. Whew! At least that gives me a place to start. And it's issuing complaints like:

jsvc.exec[18819]: Can't load log handler "2localhost.org.apache.juli.FileHandler"
jsvc.exec[18819]: java.lang.ClassNotFoundException: 2localhost.org.apache.juli.FileHandler

And I'm not surprised, 2localhost.org.apache.juli.FileHandler isn't a valid class name (packages can't start with digits!). So I'm suspecting that the default logging.properties file is...not good. Will report back.

Preface:

Server admin n00b here. I'm setting up a web server for our site (Ubuntu 8.04 LTS 64-bit). The web server is not used for processing inbound mail or outbound mail sent by actual mail users for the domain; all of that is handled separately (by a hosting service; I have neither the time nor the expertise to keep a mail system running properly).

So, three-part question:

1. Is there any need for sendmail, postfix, qmail, exim, etc. on the web server? E.g., for logwatch or other processes to send their notifications? This is a hosted dedicated server and came with mail pre-configured, but naturally if I don't need it, I want to reduce the attack surface -- it's a publicly-facing server, after all.

2. If I don't need any of those installed, presumably I have to tell the server somewhere what SMTP server to use when sending mail? Or would that be per each package doing so (apticron, logwatch, etc.).

3. Again assuming I don't need mail services, what are the most popular config files in which I'd want to change the email address from "root" to something more appropriate to ensure that we do receive notifications? E.g.: /etc/apticron/apticron.conf, /etc/logwatch/conf/logwatch.conf, ...

Links very welcome indeed.

Thanks in advance!

Obviously it's just one factor in the mix, but if I host a site here in the UK knowing that the majority of my users are in the U.S., how much weight should I give concerns about access speed for those users? (Currently ~70% U.S. users, ~20% UK, remainder are all over.) Trans-Atlantic hops used to be a bit of a big deal, but is it really much of a factor in today's world?

Again, I recognise this is only one factor in the overall mix, just looking to see how much weight the community thinks I should give it.

(Before you suggest it, using one of the big CDNs/clouds -- Amazon s3+ec2, Google AppEngine, etc. -- isn't an option at the moment.)

EDIT: All answers welcome; any stories of direct experience with this would be especially helpful.