Jim Miller's questions

I'm in the process of shutting down a site, and have replaced the old site with a single "nobody home" page at the root level of the site. Now I need to set up some redirection, so that any request to any part of the site, no matter how complicated, ends up at the root page.

I've tried what (I thought) ought to work: Creating an .htaccess file containing:

RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://www.example.com/ [L,R=301,NE]

but it mostly fails: Requests to http://www.example.com still get through, but https://www.example.com/doesnotexist.html throws a 404. (If there was no redirection going on, this would be correct, since that page doesn't exist on the site, but that's the point of the redirection: I want this request to be sent to https://www.example.com.)

Arggh. The answer to this is probably obvious to everyone but me; can anyone help out?

PS: I'm in a shared hosting situation, so I have to do this with a .htaccess file rather than hacking a full Apache configuration file.

I have fail2ban running on a centos 6.8 server; all's well. I've written a simple filter to watch for script kiddies, such as:

[Definition]
failregex = <HOST> .*GET \/admin/config.php
        <HOST> .*GET \/blog\/
        <HOST> .*GET \/backup.sql.gz
        <HOST> .*GET \/backup.sql.bz2
        ...

This works fine, and produces messages in /var/log/fail2ban.log like:

2017-04-19 11:12:40,333 fail2ban.filter         [7181]: INFO    [poison-attempts] Found 156.205.xxx.xxx
2017-04-19 11:12:40,900 fail2ban.actions        [7181]: NOTICE  [poison-attempts] Ban 156.205.xxx.xxx

So: Is it possible to change those log messages so that they include the pattern that caused the rule to fire? Perhaps something like:

2017-04-19 11:12:40,333 fail2ban.filter         [7181]: INFO    [poison-attempts] Found 156.205.xxx.xxx requesting /backup.sql.gz

or something similar? Not a big deal, but it'd be kinda nice. Thanks!

I'd like to do the following: using Apache, redirect all of the following URLs:

http://example.com
http://www.example.com
http://beta.example.com
https://example.com
https://www.example.com
https://beta.example.com

to the same place: https://beta.example.com. I'm close, but not quite there. Here's what I've got:

NameVirtualHost xxx.xxx.xxx.xxx:80
NameVirtualHost xxx.xxx.xxx.xxx:443

### rewrite scheme via http://stackoverflow.com/questions/1100343/apache-redirect-from-non-www-to-www                                    
<VirtualHost xxx.xxx.xxx.xxx:80>
    ServerName example.com
    ServerAlias www.example.com
    DocumentRoot /var/www/html/example
    CustomLog /var/log/httpd/example-log combined
    ErrorLog /var/log/httpd/example-log
</VirtualHost>

<Directory /var/www/html/example>
    RewriteEngine on
    RewriteCond %{HTTP_HOST} !^www\. [NC]
    RewriteRule ^(.*)$ https://beta.%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
    Options -Indexes
    Options +FollowSymLinks
    DirectoryIndex index.php
</Directory>

<VirtualHost xxx.xxx.xxx.xxx:80>
    ServerName beta.example.com
    DocumentRoot /var/www/html/example-beta
    CustomLog /var/log/httpd/example-beta-log combined
    ErrorLog /var/log/httpd/example-beta-log

    <IfModule mod_rewrite.c>
              RewriteEngine on
              # redirect all http -> https                                                                                             
              RewriteCond %{HTTPS} off
              RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
    </IfModule>
</VirtualHost>

<VirtualHost xxx.xxx.xxx.xxx:443>
    ServerName beta.example.com
    DocumentRoot /var/www/html/example-beta
    ErrorLog /var/log/httpd/example-beta-log
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %{SSL_PROTOCOL}x %{SSL_CIPHER}x" combined_ssl
    CustomLog /var/log/httpd/example-beta-log combined_ssl

    SSLEngine on
    ### SSL details omitted for clarity.
</VirtualHost>

What I get from this:

http://example.com => https://beta.example.com
http://www.example.com => Content for www.example.com (/var/www/html/example)
http://beta.example.com => https://beta.example.com
https://example.com => https://example.com (correct content but wrong url)
https://www.example.com => https://www.example.com (correct content but wrong url)
https://beta.example.com => https://beta.example.com

It's probably worth noting that my ssl certificate is fully wildcard-able, so I should be good there, and all the SSL stuff seems to be working (even when the pages are being served up under https://example.com). Meanwhile, there are no .htaccess files in place -- I'm trying to do everything through apache config.

So the problem is getting https://example.com to redirect correctly, with and without the www prefix. Any advice out there? Thanks!

One of my sites has been getting queries from googlebot, on the order of:

example-log:66.249.79.216 - - [06/Apr/2016:15:36:56 -0700] "GET /vvytnoxvontwusz.html HTTP/1.1" 404 15136 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" TLSv1.2 AES128-GCM-SHA256

I've gotten a few of these in the past couple of days. The address seems to be authentically in Google's IP space, but I have no clue what's going on, other than that it's properly 404'ing: I was not surprised to find that the referenced page is nowhere on my site, nor elsewhere on the internet (as you'd expect if someone is generating a random string of 15 characters).

I suppose I'm not overly worried about this in the grand scheme of things, but does anybody have a clue what's going on? Thanks!

I'm running an extremely typical CentOS 6.5 server at DigitalOcean. As part of the site setup, I've changed /etc/localtime to something appropriate for me:

rm /etc/localtime
ln -sf /usr/share/zoneinfo/America/Los_Angeles /etc/localtime

This works fine, except somebody/something is resetting it: I'll check back in with the server at some random (I think) point, and find that my link has been replaced with the original localtime file, putting the server back in UTC. I've looked in /etc/cron* on the chance that there's a script in there doing it, but I can't find anything. Has anyone else run into this before? Thanks!

I'm having problems with Google rejecting mail because of SPF problems. I thought I had this fixed, but evidently not...

The mail is being sent from a Drupal site running mimemail. A message that shows the problem looks like:

Delivered-To: [email protected]
Received: by 10.112.47.229 with SMTP id g5csp300564lbn;
        Wed, 2 Apr 2014 10:21:06 -0700 (PDT)
X-Received: by 10.66.249.233 with SMTP id yx9mr1407538pac.3.1396459264202;
        Wed, 02 Apr 2014 10:21:04 -0700 (PDT)
Return-Path: <[email protected]>
Received: from sender.com (sender.com. [xxx.xxx.xxx.xxx])
        by mx.google.com with ESMTP id m8si1612133pbd.503.2014.04.02.10.21.03
        for <[email protected]>;
        Wed, 02 Apr 2014 10:21:04 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning [email protected] does not designate xxx.xxx.xxx.xxx as permitted sender) client-ip=xxx.xxx.xxx.xxx;
Authentication-Results: mx.google.com;
       spf=softfail (google.com: domain of transitioning [email protected] does not designate xxx.xxx.xxx.xxx as permitted sender) [email protected]
Received: by sender.com (Postfix, from userid 48)
    id 3E486101774; Wed,  2 Apr 2014 17:21:02 +0000 (UTC)
To: "reg" <[email protected]>
Subject: Message Subject here
X-PHP-Originating-Script: 501:mimemail.module
MIME-Version:1.0
Content-Type:multipart/mixed;
 boundary="ca08a4eb4d5f903d3ee41d8632611e3ff9f7e67713"
Content-Transfer-Encoding:8Bit
X-Mailer:Drupal
Sender:Sender <[email protected]>
From:Sender <[email protected]>
Message-Id: <[email protected]>
Date: Wed,  2 Apr 2014 17:21:02 +0000 (UTC)

This is a multi-part message in MIME format.

--ca08a4eb4d5f903d3ee41d8632611e3ff9f7e67713
Content-Type:multipart/alternative;
 boundary="2db314e2a942d712c21894936b800a2c7998463d12"
Content-Transfer-Encoding:8bit


--2db314e2a942d712c21894936b800a2c7998463d12
Content-Type:text/plain; charset=utf-8
Content-Disposition:inline
Content-Transfer-Encoding:8bit

Message text goes here...

--2db314e2a942d712c21894936b800a2c7998463d12
Content-Type:text/html; charset=utf-8
Content-Disposition:inline
Content-Transfer-Encoding:8Bit

<html>
<html text goes here>
</html>

--2db314e2a942d712c21894936b800a2c7998463d12--

--ca08a4eb4d5f903d3ee41d8632611e3ff9f7e67713--

[Apologies for the anonomyzing.]

The SPF records for the domain, obtained from http://www.whatsmyip.us/txt, look like:

v=spf1 ip4:xxx.xxx.xxx.xxx ~all , ttl : 14144
v=spf1 include:_spf.google.com ~all , ttl : 14144
v=spf1 a -all , ttl : 14144
v=spf1 ip4:xxx.xxx.xxx.yyy ~all , ttl : 14144

similarly anonymized.

I can't figure out what's wrong -- the appropriate SPF records seem to be in place. The only issue I can see is that I have indeed set up (or tried to set up) two servers as legal senders for the domain; this is meant to cover both a production and a development server (let's call them sender.com and dev.sender.com). Is this an issue?

Any thoughts about what's wrong? Again, I thought this was once working correctly, but it's definitely not now. Thanks!

I'm running a site on DigitalOcean with CentOS 6.5.

As has been noted elsewhere, DigitalOcean servers are by default configured without any sort of swap, and I'm wondering if I should add one.

I know just enough about this stuff to think that the answer is "well, of course", but I don't really have a foundation for going beyond that.

• Will my system just die the first time it gets seriously loaded up with users and memory demands?
• Will swap save me from that, but only by imposing an unpleasant performance hit?

Any advice in this area, specific to DigitalOcean or not, would be greatly appreciated.

I've got a working monit installation on CentOS 6.5. Events are getting detected and alert mail is getting sent, but I'm trying to get other things to happen when an event fires, via the exec feature, and none of these are working.

For instance, I have a rule like:

if cpu usage (user) > 10% then 
   exec "/bin/echo 'user limit' >> /tmp/monit-status-log" as uid someuser and gid somegroup

The rule fires appropriately under cpu load, but nothing is written into the file (whose protection setting is rw-rw-rw).

The log file for an execution looks correct as far as it goes, I guess:

[PST Feb 20 09:31:04] info     : 'linux2.example.com' Monit reloaded
[PST Feb 20 09:31:31] info     : Reinitializing monit daemon
[PST Feb 20 09:31:31] info     : Awakened by the SIGHUP signal
[PST Feb 20 09:31:31] info     : Reinitializing monit - Control file '/etc/monit.conf'
[PST Feb 20 09:31:31] info     : Shutting down monit HTTP server
[PST Feb 20 09:31:32] info     : monit HTTP server stopped
[PST Feb 20 09:31:32] info     : Starting monit HTTP server at [*:2812]
[PST Feb 20 09:31:32] info     : monit HTTP server started
[PST Feb 20 09:31:32] info     : 'linux2.example.com' Monit reloaded
[PST Feb 20 09:32:32] error    : 'linux2.example.com' cpu user usage of 24.4% matches resource limit [cpu user usage>10.0%]
[PST Feb 20 09:32:32] info     : 'linux2.example.com' exec: /bin/echo
[PST Feb 20 09:33:32] info     : 'linux2.example.com' 'linux2.example.com' cpu user usage check succeeded [current cpu user usage=0.0%]

Any thoughts or advice?

I've got a number of php processes that I'm running in the background, with monit as the manager for them. My current config for monit looks like:

check process myprocess1
      matching "process1.php"
      start program = "/usr/bin/php /path/to/process1.php > /var/log/process1.log"
      stop program = "/usr/bin/pkill /path/to/process1.php"

(Again, remember that there are several processes like this one.) This is working for starting processes, but not for stopping them -- the stop attempt fails with a monat "failed to stop" message. My current diagnosis of this situation is that it's failing because pkill isn't finding the process: doing the analogous pgrep /path/to/process1.php fails to find anything. ps ax shows, among other things:

5307 ? S 0:00 /usr/bin/php /path/to/process1.php

Meanwhile, pgrep php succeeds in finding pids of all the php processes, but I only want to kill one of them.

So: Is there a way to get what I want here? I'm not wedded to using pkill, but it seems to be well thought-of...

I'm building a page (using video.js, should it matter) that holds players for a reasonably large number of videos -- click a button on a thumbnail of the image and a modal player opens up, playing the video. Works fine; no big deal. My server is Apache 2.2.15, fwiw.

The question: when I look at my server logs, I see entries for each of the videos with an HTTP code of 206 (partial content), such as:

GET /videos/a_video.mp4 HTTP/1.1" 206 1130496 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.73.11 (KHTML, like Gecko) Version/7.0.1 Safari/537.73.11"

I'm not sure what these entries mean, exactly. When this page loaded, was 1.1 MB of stuff really pushed over the network to the client, or is this just information that the client is meant to use when/if the file is really requested by the user? Dumping all this stuff onto the user/client would be a pretty piggish thing to do to somebody's bandwidth (let along my site's), especially on a mobile connection. (Based on some additional log analysis, it looks like those bits are really getting pushed, but checking this with other more knowledgeable people surely seems like the right thing to do...)

I've got a server happily running Fedora 19, but am thinking about living in a calmer, more stable, less bleeding-edge world and moving to CentOS 6. Is it possible to make this move in a straightforward way -- just sorta run the CentOS installer over the Fedora installation, tweak a package here and there, and come away running CentOS? Or is this a hairy, unreasonable thing to even think about doing, and that starting over with a clean disk is, in the long term, faster and easier? Thanks!

I'm working on a site that is probably destined for AWS hosting. This leads to a question about access rights to stuff that would be stored in S3 buckets:

• I set up an EC2 server for the site, and an S3 bucket to hold some of the site's stuff.
• Alice and Bob create accounts on my site.
• Alice and Bob both upload files that end up in the bucket.
• Alice should be able to see her file but not Bob's, and vice versa.
• Anonymous users of the site should be able to see neither file.

Is this possible with S3? My impression is that this is what AWS IAM is for; is this correct? I'm slowly getting past my AWS newbie-ism, but I'm not quite there yet -- thanks!

I'm in the process of moving a webapp from the local server that's been its development home to somewhere else for eventual deployment. I'm currently evaluating AWS and Rackspace, and am in the middle of trying the site out on AWS. I've tried to keep the two environments as similar as possible; both are running a pretty straightforward LAMP stack on top of Fedora 17, with the same versions of apache, php, and such. My development machine is a homebrew box based on an i7 860 chip with 32 GB of memory; at AWS, I'm typically using an m1.small instance, built from their "standard" fedora 17 instance, which gets described as "Intel Xeon E5-2650 0 @ 1.80GHz (1 Core), Memory: 2048MB" by some benchmarking software I've been using. The root device of my AWS instance is set up as an EBS volume.

The site is up and running on both boxes, and I was pleased to see that site performance is roughly comparable, with AWS being a little slower. However, I'm also doing some video encoding as part of the site's work, via a version of ffmpeg I've built from source on both sites. Here, I'm getting a huge performance difference, with my development server being about a factor of 10 faster than AWS. I've run a few benchmarks, and they show a similar difference: the Phoronix "apache" benchmark shows my server running about 12x the AWS instance.

So, I'm puzzled. I understand that the "E5-2650" description of the AWS instance is just for descriptive purposes, and that I don't really have a machine with an E5-2650 all to myself. But what's the right way to think about this? The E5-2650 seems to be a crazy-fast 8-core chip, by some measures about twice as fast as my i7; maybe I should be thinking that I effectively have 1/8 of such a machine (1 core out of 8)? That still doesn't get me to 10x, but maybe that's attributable to the (much) larger amount of memory of my development machine? Or have I screwed up something with my AWS installation -- I'm about one step above a complete AWS newbie, but not much more, so screwing something up is quite possible. Any advice out there?

I'm running a pretty standard LAMP stack on Fedora 15, mostly in support of some Drupal sites. Things generally seem to be running fine, but occasionally the disk on the machine will start to work noticeably harder than it usually does. My limited system debugging skills have found this:

• 'top' reports that httpd and mysqld are indeed active, taking up maybe 3-4% of what is otherwise a mostly idle machine.

• There are no changes in the httpd log files, and nothing out of the ordinary is showing up on other system logs. So the obvious answer of "somebody's hitting your websites, dummy" doesn't seem to be the case.

• resetting httpd with "apachectl graceful" has no effect. However, "apachectl stop" followed by "apachectl start" generally calms the machine down -- the disk churning stops.

• I believe the machine is secure; I have no reason to believe that there's any hackery going on. (Of course, I suppose that's usually the case...)

I'd be surprised (but happy!) if that was enough for somebody to tell me what's going on, but is there any advice out there on how I might dig into the machine a bit more and figure out what's going on? Thanks!

I have what doesn't seem like all that complicated a network in my home: I have a block of 8 (= 5 usable) static IP addresses, which are assigned to five externally-visible servers; I also have a bunch of Macs and PCs and other connected devices that get their IP addresses via NAT. I'm currently using a Netopia 3346-ENT, but its time has kinda passed, and am looking for a new modem/router. I've been considering the Netgear DGN3500, but, after looking its manual, I can't figure out how to set up the external address - internal address mappings, or if the router will even let me do this (I've had one guy on another board say that it can't, and that I have to to go some unspecified Netgear ProSafe product).

Any advice out there? I understand that I may need something more heavy-duty than a basic home modem/router, but finding the next step up can't be THAT hard... Thanks!

I'm trying to get a mail server to support IMAP-over-SSL services. Everything is fine inside my network: Clients can specify SSL and port 993 in their email client configuration, and mail gets through. However, it's not working from outside: Somewhere, the port is getting blocked.

Nmap tells me this for the internal network:

[root@linux2 ~]# nmap -p993 192.168.0.4
Starting Nmap 5.21 ( http://nmap.org ) at 2011-04-18 09:54 PDT
Nmap scan report for example.com (192.168.0.4)
Host is up (0.00022s latency).
PORT    STATE SERVICE
993/tcp open  imaps
MAC Address: 00:1F:5B:35:F4:C8 (Apple)

And for the outside network:

[root@linux2 ~]# nmap -p993 xxx.xxx.xxx.xxx 
Starting Nmap 5.21 ( http://nmap.org ) at 2011-04-18 09:54
PDT Nmap scan report for example.com (xxx.xxx.xxx.xxx) 
Host is up (0.0023s latency). 
PORT    STATE SERVICE 
993/tcp filtered imaps

So, somebody is doing some filtering; the question is who. My router's firewall has 993 open; I've also experimented with (briefly) disabling the router firewall altogether, but with no improvement. I've checked with my service provider (AT&T DSL), but they say they're not blocking it.

I'm traveling at the moment, and have moved one of the websites I'm working on to my MBP so I can work on it without a network connection. I've made an addition to the Mac's /etc/hosts file pointing the domain name to 127.0.0.1, and all's well.

I now want to get into Parallels and check the site from Windows browsers. How do I get things so that the Windows browser will understand the domain name and access the site? The Windows image obviously doesn't recognize / can't find the Mac's /etc/hosts file, and references to 127.0.0.1 in the Windows hosts file just as obviously point to Windows, not the Mac. Any advice out there? Thanks!