Eric's questions

I get these warnings every single day from Rootkit Hunter:

---------------------- Start Rootkit Hunter Scan ----------------------
Warning: Package manager verification has failed:
         File: /usr/sbin/tcpd
         The file permissions have changed
Warning: Package manager verification has failed:
         File: /usr/bin/less
         The symbolic link target has changed
Warning: Package manager verification has failed:
         File: /usr/bin/telnet
         The file permissions have changed

----------------------- End Rootkit Hunter Scan -----------------------

And yes :-) I have run rkhunter --propupd before and after this problem started. No change, still get these warnings every day. Why and how can I stop them?

I have a large, busy site; it currently runs completely on a dedicated server that I rent each month for ~$700.

It has three parts that I think I could carve off to a cloud solution:

• Media (image/video) file hosting. Currently I have something like 236 GB of static images, currently all just parked on my server. If I moved these to the cloud I would probably combine with a CDN (to minimize the cost of data transfer out of the cloud service for every image request).

• Database. Currently running MySQL with about 3 GB of data on my server.

• Web server. The same server runs nginx serving static files and PHP.

I'm not having any production issues now but I expect my site to double in traffic/server load next year. So I want to think about scalability now.

My question is this: how can I figure out if it would be cost-effective to move any/all of these onto a cloud platform, instead of keeping them on my current server?

(I already know some of the other factors in place: it would be easier to do backups with cloud, I wouldn't have one point of failure like I do now with my single server, etc. But I have no sense of how much more/less it would cost to carve off one of these services. How can I compute that?)

EDIT - thank you all for these amazing answers and comments. A few folks have asked for more info so I'm summarizing all below and adding a little more data:

Data Transfer ("Bandwidth") Used - the site sends ~17 TB outbound data per month (!) and I am planning to double that figure next year (!!). Almost all of this outbound is static media (pics and video clips), so perhaps a CDN would be a good idea, not only for better discoverability but to move the burden of transmitting all that data to the CDN network, so the media storage server doesn't have so much data transfer directly. --EDIT: it seems that CDNs are damn expensive for this much data transfer. So maybe the static media stays on a simple server that gives me a very high bandwidth cap (hello OVH!) and if I can find a cost-effective way to put a CDN in front of it, terrific.

Traffic Not Spiky - my traffic is fairly steady; my goal with moving to a more cloud-based solution is to be able to easily scale up. I.e. my current setup has everything on one hard drive and the drive is 60% full; this infrastructure literally couldn't deal with double the amount of data (and I'm not sure it would have enough computing power to run the web server and DB server at double the traffic, either).

Static Media - As I mentioned above, I have about 236 GB of static media, mostly all images and video clips. This seems like the most obvious (and maybe easiest?) piece to carve off first and put in the cloud.

Database - while the DB runs fine now, I will be having some more complex queries soon and like the idea of something a little more powerful there. So while I don't think my current needs (power and amount of data) dictate that I should move the DB server into the cloud, it's all about being able to scale up.

Busy Hours - I always have at least 1,000 users on the site 24/7, voraciously consuming media. The server is never idle.

Currently Dedicated Server - I misspoke earlier and said it was colo (implying that I owned the hardware). That was wrong. I have a dedicated server (owned by my hosting company) that I rent each month. Not a big distinction but just want to mention.

I have an Amazon S3 bucket that contains thousands of JPGs and similar media assets. (It's the storage for my forums.) I don't host a static site in this bucket (no HTML of any kind) and it's completely open for the public to read (so the forum software can just link to the images).

However I'm paying through the nose on S3 for bandwidth. Someone said I should stick CloudFlare in front of the S3 bucket and it would save a lot of $. Cool! But I don't get the process. CloudFlare seems to want to take over all the DNS for my main site, which is very much not something I want.

I just need https://my-bucket.s3.us-east-1.amazonaws.com to be cached/CDN'd by CloudFlare. And obviously I can't change Amazon's DNS ;-)

I don't think this would be difficult but I can't seem to find a relatively simple explanation of the right way to set this up. (In other words: I am lacking the "big picture"/main steps.)

(This is covered a bit on the CloudFlare site but I'm confused by their instructions and my use case is also different than what they talk about.)

I'm running a production server with CentOS 7 and PHP 7.1. I'd LOVE to upgrade to PHP 7.4 but I am nervous about doing this on a production server. I am not the greatest sysadmin in the world, so I have a 3 questions, apologies if they are dumb/obvious to others :-)

1. I have to upgrade all my PHP extensions (like openssl, pdo_mysql, iconv, json, gd, phar, pdo, libmagic, and a zillion others) at the same time, right? I can't use the existing extensions? So does this mean that I need to make a list of all those extensions, and then source the 7.4 versions to be sure I can upgrade all of them at the same time I upgrade the core PHP files? (Or is there some automated way to upgrade PHP and all existing extensions? Doing it "by hand" feels like a lot of work, and a lot of potential to screw up something. I'm just wondering if there's a more elegant solution that I'm overlooking.)

2. This is all on a production server (where PHP 7.1 runs as php-fpm under nginx). AFAIK the only option I have for upgrading is to take my site offline for maintenance, and then try to do the upgrade and hope nothing goes terribly wrong. Is that right? If so, how can I minimize the impact? I'd much prefer to get PHP 7.4 running perfectly alongside the existing 7.1 and then flip over to 7.4 on one quick switch, but can't figure out how. (EDIT: I should add that I am not using Docker or any other kind of virtualization. When I do whereis php I get: php: /usr/bin/php /usr/lib64/php /etc/php.d /etc/php.ini /usr/share/php /usr/share/man/man1/php.1.gz)

3. And then the best way to upgrade is... to use pre-built images that someone offers, like Remi's RPM Repo?

Thanks.

I'm using the AWS CLI to use Amazon's S3 storage for backing up files:

aws s3 sync /backup-these-files s3://my-backup

How can I specify the x-amz-storage-class header, to tell it to put these files into the STANDARD-IA storage tier?

I can't figure the options with the AWS CLI to add a header or otherwise specify this option.

Reference: https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html#sc-howtoset

I have a simple alias in my /etc/aliases file that is failing. The alias is simply support4 and it should point to [email protected].

When I send a message to [email protected], it bounces with a 5.1.1 Address error.

The main [email protected] address works perfectly and can receive mail with no problems.

How can I log/debug what is going on?

I recently updated my (busy) site to a new chat room and I think it's taxing the server, but I'm not sure why or how. I'm receiving an email like this from my server about once a day, which is concerning. I'm running nginx as the web server that calls php-fpm.

Subject line: [abrt] php71w-fpm: php-fpm killed by SIGSEGV

reason:         php-fpm killed by SIGSEGV
cmdline:        'php-fpm: pool www' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' ''
executable:     /usr/sbin/php-fpm
package:        php71w-fpm-7.1.32-1.w7
component:      php71w
pid:            21409
pwd:            /usr/share/nginx/www.example.com/chatroom
hostname:       main.example.com
count:          63
abrt_version:   2.1.11
analyzer:       CCpp
architecture:   x86_64
event_log:      
global_pid:     21409
kernel:         3.10.0-957.21.3.el7.x86_64
last_occurrence: 1572875501
os_release:     CentOS Linux release 7.6.1810 (Core) 
pkg_arch:       x86_64
pkg_epoch:      0
pkg_fingerprint: E87F D236 62E7 4CA5
pkg_name:       php71w-fpm
pkg_release:    1.w7
pkg_vendor:     Webtatic
pkg_version:    7.1.32
runlevel:       N 3
time:           Mon 14 Oct 2019 12:09:23 PM GMT
type:           CCpp
uid:            995
username:       nginx
uuid:           b008d2dabbed483b51d5bc88d507b2d0288f898d

core_backtrace:
:{   "signal": 11
:,   "executable": "/usr/sbin/php-fpm"
:,   "stacktrace":
:      [ {   "crash_thread": true
:        ,   "frames":
:              [ {   "address": 94687732525503
:                ,   "build_id": "7c6a08e78d7575f9b7120f9bba3686286da0c5c6"
:                ,   "build_id_offset": 3279295
:                ,   "function_name": "ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER"
:                ,   "file_name": "/usr/sbin/php-fpm"
:                }

(and then about a million of those "frames" that all look similar)

Is this a case where php-fpm is running out of memory (or something)? How can I fix this? Is there a blog post or something that can help me "tune" my PHP to solve? Thank you...

The latest version of the nano editor is 4.5 (https://www.nano-editor.org/)... I'm running 2.3.1 on my CentOS 7 server and want to upgrade. But no repos (EPEL, Remi, etc.) seem to have anything newer than 2.3.

How can I get the latest nano? Do I need to build it from scratch on the server? I'd rather be able to use yum to manage installs/upgrades, if possible.

I see lots of download options (packages, Open SUSE[?], distributions, blah blah) on the nano page but none of them seem to be the right thing for yum. I think.

I have these entries in my /etc/aliases file on my main site, example.com:

scott:             [email protected]
payment_support:   scott

If I email [email protected], it is correctly delivered to [email protected]

If I email [email protected], it bounces back to me with 500 5.1.1 [email protected] Address Error

What the-- ?

When I do nginx -V it only shows me those modules that were compiled in... I want to confirm that my load_module directives in nginx.conf are actually working.

I am running a server with nginx on port 80 and Apache on 8080. I want the home page of my site to be served with nginx, and every other request passed through to Apache. I found this great article and understand the nginx proxy_pass directive, but I can't figure out the right regex to tell nginx to only serve the home page of my site. Since users will come to the site by just visiting http://mysite.com (without /index.htm), I don't know what "location" value I should use.

Here's an example config file that demonstrates how to have all pages sent to Apache (like I want) except the /css folder, and image files. As you can see nginx uses a simple regex to specify what should be served by nginx. What regex would I use to specify only the home page should be served by nginx? Or should I be using try_files somehow?

server {
      root /usr/local/www/mydomain.com;
      server_name mydomain.com www.mydomain.com;

      # next line says anything that matches this regex should be sent to Apache     
      location / {
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $remote_addr;
            proxy_set_header Host $host;
            # Apache is listening on port 8080
            proxy_pass http://127.0.0.1:8080;
      }

      # Example 1 - Specify a folder and its contents
      # To serve everything under /css from nginx only
      location /css { }

      # Example 2 - Specify a RegEx pattern such as file extensions
      # to serve only image files directly from Nginx
      location ~* ^.+\.(jpg|jpeg|gif|png)$
      {
            # this will match any file of the above extensions
      }
}

I have an Amazon EC2 instance that has two EBS drives associated with it: an 8 GB drive at /dev/sda1, and a 20 GB drive at /dev/sdf. I didn't set this up so I'm not sure why this instance needs two drives of those sizes.

I'm trying to figure out which of these two EBS drives contains the root (/) volume. (My goal is to disconnect that drive and attach it to another instance, where I can gain access to a key file I need at /home/ec2-user/.ssh/authorized_keys, per this excellent answer.)

How can I tell? (What is /sda1 and /sdf?)

Per this excellent answer, I'm trying to disconnect an EBS drive from one instance (Server-01) and connect it to another instance (Server-02), where I will mount it and get access to a file I need. Then I'll disconnect it from Server-02 and move it back to Server-01.

Here's the problem. Server-02, the server where I want to add the drive, won't let me associate the EBS drive that I've disconnected from Server-01. Could it be because Server-02 already has two EBS drives attached to it (an 8 GB drive at /dev/sda1, and a 20 GB drive at /dev/sdf)?

(Please pardon me if I don't have my terminology exactly correct.)

I have an m1.medium Amazon EC2 instance running Apache and hosting a Wordpress blog. In turn, Wordpress works with a MySQL database over on a different EC2 instance. The Wordpress site has W3 Total Cache set up and working well, and much of the static content on the site is served from a CDN. The site runs a low amount of traffic regularly and then occasionally gets some huge traffic spikes.... and when those spikes occur (more than ~150 people accessing the site), the site goes down. I can also make this happen every time with with some load testing tools.

Here is the 'top' when the main server is idle:

top - 23:21:23 up 103 days, 19:40,  3 users,  load average: 0.91, 0.60, 0.62
Tasks:  93 total,   1 running,  92 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.9%sy,  0.0%ni, 99.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3844856k total,  1756268k used,  2088588k free,   150132k buffers
Swap:        0k total,        0k used,        0k free,   833740k cached

However, if I do some load tests to simulate hundreds of users accessing a static graphic file (which obviously doesn't trigger Wordpress, PHP or the database), everything is fine: the server load stays low, the graphic file is served quickly, etc.

My Apache settings (3.1G of memory in server / ~8100k per httpd instance = ~400 MaxClients):

    StartServers         5
    MinSpareServers      5
    MaxSpareServers     10
    ServerLimit        400
    MaxClients         400
    MaxRequestsPerChild  0

So based on all that, it seems like the problem has to do with when PHP or MySQL are used.

Over on the MySQL server, no matter what I do, the load stays pretty much at 0, and the slow query log stays empty... so I think things are healthy there. Here is my 'top' for the SQL server:

top - 23:20:21 up 103 days, 19:12,  5 users,  load average: 0.08, 0.03, 0.05
Tasks: 115 total,   1 running, 114 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3844856k total,  1076912k used,  2767944k free,   158412k buffers
Swap:        0k total,        0k used,        0k free,   638092k cached

This all leads me to think that one of these unlikely scenarios is occurring:

• The machine simply doesn't have enough raw horsepower to serve 150+ concurrent users, and I should move up to an L or XL EC2 instance. Maybe, but... really? 150 users is too many for a relatively powerful m1.medium server?
• Apache simply isn't built to handle this kind of traffic. Doubtful.
• There is a problem in the communication between the web and database servers. But I doubt that since this is between two Amazon EC2 instances.

I feel like I've checked everything I can and still no luck. What else should I check? What else can I try?

I am working with some Amazon EC2 servers that are up and running, and I need to SSH into the servers. I don't have any keys that were generated when the servers were first set up (someone else did it long before I got here). Can I still get into the servers without the key files?

FWIW I've tried a lot of things to SSH into the box so far, including generating new key pairs in the EC2 dashboard, and nothing seems to be working. This Amazon AWS support post and this answer seem to indicate that I'm out of luck unless I want to make an AMI of my current server and then use it to instantiate a whole new EC2 server instance (just to get the .pem file generated at that time). Is that really the only way I can get into the box at this point?!

I have a web site that runs on an Amazon EC2 Linux-based instance, and is mapped to a regular domain name (x.com). This site talks to a database on a second EC2 instance.

I need to shut down this instance and increase the computing power behind it... if I do that, how long will it take, and then when I spin the machine up again, will my site just come back online? Will the IP address, DNS info etc. all be preserved so the domain name will keep working, and it will still be able to talk to the DB?

FYI this is what I currently see for the web server in the AWS EC2 info panel (exact IP addresses changed slightly but you get the idea):

Public DNS: ec2-54-1-1-1.us-west-1.compute.amazonaws.com
Private DNS: ip-10-1-1-1.us-west-1.compute.internal
Private IPs: 10.1.1.1

I have a client with a popular Wordpress blog that experiences occasional short-term traffic spikes (when the site is promoted on national TV). The blog is already hosted on an Amazon EC2 instance, and we're exploring using Amazon's load balancer to help with the spikes.

Two general "big picture" questions about this:

1. To set this up, would I just create a machine image (AMI) from the existing server, and then duplicate it into a new instance, and tell the load balancer to split the traffic between the two instances? (Or more than two, I suppose...)

2. When my client needs to edit the content on the blog, it seems like I would need a way to point them to one particular "master" instance so they could use the WordPress admin tool on that server, and then copy (rsync) the content from that instance to the other instances, so everyone has the same data. Is this the right approach?

Any other tips/suggestions would be much appreciated. I'm new to Amazon's load balancer so I'm just trying to wrap my head around the big picture before I dive into specifics.

UPDATE: some static content lives on Amazon CloudFront CDN. I wasn't familiar with APC but I'm looking into it now (thank you). WP-SuperCache is next on the list to implement. I believe the database is on a separate server instance.

Thank you, Eric

I'm running this linux:

Linux host.themepark.com 2.6.32-220.4.1.el6.x86_64 #1 SMP Tue Jan 24 02:13:44 GMT 2012 x86_64 x86_64 x86_64 GNU/Linux

And I run the Selenium stand-alone server on my box with this command:

java -jar /home/l/cron/selenium-server-standalone-2.24.1.jar > /logs/selenium.log 2>&1 &

Here's the problem: as soon as I do that, the server load starts skyrocketing. I even went back and downloaded older versions of the Selenium server, but same results with 2.23.1, 2.23.0, and 2.19.0.

Note that the server load starts going nuts before I issue ANY commands to Selenium or do anything else. All I'm doing is firing up the server, per the command above.

This used to work perfectly on my server without causing massive server load, so something has changed, but I'm not sure what. My server is a managed VPS so I don't know if there is some kind of auto-update script that kicked in or what... but it's a problem.

(Incidentally, even though the server load climbs like crazy, everything still works: after firing up Selenium, my server creates a screen with Xvfb so Firefox will be happy, then a PHP script talks to Selenium to do what it needs to do before shutting everything down. It takes a LONG time, and the load gets all the way up to 8 [!!!] before it is finished, which kills my web server and makes the main site horribly unresponsive... but it does get everything done.)

Any suggestions for what is going on, why it's started doing this and/or, most importantly, how I can make Selenium not kill the server when it starts up... would be GREATLY appreciated!

I have a VPS server from LiquidWeb... it feels like sometimes during the day it's laggy. Since it's 100% virtual, I can't just check "top" to gauge the performance since that is measuring within the VPS. (In other words: if I am never running anything on the VPS, top will always show the same memory use and server load of zero, but because the VPS lives on a shared physical box, it may still fluctuate in speed.)

Is there a way to measure the "real" speed of how the VPS is performing? I was thinking about just hacking a little script that runs a loop (doing something-- file actions, or ??) and logs the time, so I could start to build history... but I'd prefer a more reliable/fundamental stat.

I must be missing something obvious.

This works:

[root@host2 /]# cd /home/mysite/public_html/../logs 
[root@host2 /home/mysite/logs]# touch x

Why doesn't this?

[root@host2 /]# touch /home/mysite/public_html/../logs/x
touch: cannot touch `/home/mysite/public_html/../logs/x': No such file or directory