I use packer to build immutable Ubuntu 20.04 servers.
How can it work smoothly with unattended upgrades?
Since the image is not bundled like it was in the past the updates do not apply to new instances. It means that when a server comes up unattended upgrades will need to run full upgrades. This is problematic because some of them requires reboot + it prolongs the server get-up time.
What is the best practice for unattended upgrades on immutable servers?
I have rewrite rules and get many logs like:
[Thu Mar 26 11:41:00.356526 2020] [rewrite:trace1] [pid 18095] mod_rewrite.c(483): [client xy.xy.220.72:56702] xy.xt.220.72 - - [app.xyz.com/sid#7f7996613868][rid#7f79964eb0a0/initial] [perdir /var/www/app/] initial URL equal rewritten URL: /var/www/app/assets/css/management.css [IGNORING REWRITE], referer: https://app.xyz.com/management/content_page
Things work well and I dont want to change redirect rules. How can I get rid of this notice? As far as I understand it just means the rewrite rules did not change the url.
I'm setting up email alert for unattended upgrades on ubuntu 18.04. It looks like unattended-upgrades cannot set the 'FROM' address and rely on mailx.
On mailx I'm using SMTP so the from address must be qualified. I get error that the from address is root and not [email protected]
To change this I need to set the 5th parameter in the root user in /etc/passwd file.
Is there no other less intrusive option?
EDIT: Adding ssmtp definitions (changed domain and user/pass for security)
#
# Config file for sSMTP sendmail
#
# The person who gets all mail for userids < 1000
# Make this empty to disable rewriting.
[email protected]
# The place where the mail goes. The actual machine name is required no
# MX records are consulted. Commonly mailhosts are named mail.domain.com
mailhub=email-smtp.eu-west-1.amazonaws.com:587
# Where will the mail seem to come from?
rewriteDomain=domain.com
# The full hostname
hostname=domain.com
# Are users allowed to set their own From: address?
# YES - Allow the user to specify their own From: address
# NO - Use the system generated From: address
FromLineOverride=YES
# Use SSL/TLS before starting negotiation
TLS_CA_FILE=/etc/ssl/certs/ca-certificates.crt
UseTLS=NO
UseSTARTTLS=YES
AuthUser=XXX
AuthPass=YYY
AuthMethod=LOGIN
# sSMTP aliases
#
# Format: local_account:outgoing_address:mailhub
#
# Example: root:[email protected]:mailhub.your.domain[:port]
# where [:port] is an optional port number that defaults to 25.
root:[email protected]:email-smtp.eu-west-1.amazonaws.com:587
SPF allows up to 10 DNS calls. We use multiple email providers and overall get 13 DNS calls for SPF.
The solutions I've found so far:
Manually collecting and "inlining" the IP addresses to reduce DNS lookups. Unfortunately this makes the entry static and will not be up-to-date when providers update their SPF records.
Creating subdomains for one/few of the SPF providers - won't work for business reasons.
Using an "SPF proxy" - I found only spfproxy.org (now defunct?) and I'm afraid to use them because I can't find who is behind it and they don't answer emails.
What options do I have? Ideally a solution like spfproxy which I can trust would work for me but I could not find any.
On AWS I have a few dedicated servers that do image processing, and they seem to get high traffic and fail.
When running
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
I get that the most active IP is an IP of an nginx load balancer server that is not supposed to route traffic to these servers.
Does it mean that my servers are exploited in some way? How can I tell what is the traffic reason and check the load balance server (nginx over ubuntu)?
I have an ubuntu apache/php server running php doing appx 100 hits/sec and a PHP cron running in the background.
I get occasionally high CPU load on one of the Apache processes which stays high regardless of traffic or cron activity. It seems to me that its stuck in some kind of loop or something.
Below you will find the top and strace info.
How can I find where the bad code is and what causes this?
top - 14:45:24 up 3 days, 3:38, 1 user, load average: 5.10, 5.88, 5.85
Tasks: 163 total, 5 running, 158 sleeping, 0 stopped, 0 zombie
Cpu(s): 47.8%us, 18.5%sy, 0.0%ni, 10.2%id, 0.0%wa, 0.0%hi, 1.8%si, 21.6%st
Mem: 7885012k total, 3858484k used, 4026528k free, 177444k buffers
Swap: 0k total, 0k used, 0k free, 1037868k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10736 www-data 20 0 769m 559m 478m R 69 7.3 29:08.30 apache2
10844 www-data 20 0 824m 601m 492m S 17 7.8 4:37.90 apache2
1016 root 20 0 242m 25m 4628 S 6 0.3 162:07.93 scalarizr
9030 www-data 20 0 879m 619m 492m S 4 8.0 5:06.82 apache2
20216 www-data 20 0 747m 228m 170m S 4 3.0 0:01.94 apache2
10807 www-data 20 0 814m 584m 492m S 3 7.6 4:54.10 apache2
10455 www-data 20 0 831m 574m 492m S 3 7.5 4:32.65 apache2
10495 www-data 20 0 849m 592m 492m S 3 7.7 4:41.10 apache2
10884 www-data 20 0 840m 581m 492m S 3 7.6 4:25.06 apache2
^CProcess 10736 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
74.55 0.148052 1 109755 gettimeofday
25.36 0.050370 0 164634 clock_gettime
0.09 0.000178 0 54878 poll
------ ----------- ----------- --------- --------- ----------------
100.00 0.198600 329267 total
root@ec2-67-202-54-36:~# ^C
I installed new relic according to their quick install instructions.
If I start the daemon manually after boot with /etc/init.d/newrelic-daemon restart it works, but it doesn't run automatically after boot. How can I find out why?
The servers are Scalr/AWS PHP/5.3.2-1ubuntu4.15
I installed the php5-geoip module on Ubuntu 10.04 via apt-get install php5-geoip. I've updated the geocities.dat library
The production server performance decreased and load average is soaring. Before the install it was less than 1 for the same traffic level. I suspect that the geoip data is not memory cached.
How can I activate memory caching for this module?
I need to test a saas service load by sending 1000s requests/sec from multiple remote sources. The urls need to be from my access log.
Is there a service that can simulate high traffic using the data from my log files?
I'm using httperf from a large ec2 instance with ubuntu 10.04 (no other software installed).
When I test without --hog the tests run however when I use --hog the tests never end
I have a rapidly growing (10Ms calls/day) web service currently on one EC2 app instance running ubuntu10.04+ apache + APC + memcache (+ several master/slaves mysql farms on other instances).
I consider to upgrade the app instance in one of the following ways:
1) Increase to larger instance type
2) Split to several app servers
I can assess pretty well my memory usage and cpu usage based on the current server graphs.
My fear is that if I go for the larger EC2 instance type the network i/o may become a bottleneck as traffic grows.
My questions: How can I estimate network i/o usage and constraints? How can I tell how much spare traffic I can handle until it becomes overloaded. (in terms of the network capacity not the server ram/cpu)
I used to have CentOS/plesk server on Mediatemple and there my bottleneck was kmemsize which is a limited amount of nonswappable memory which became full when the network traffic increased. Is there something similar to this on EC2 Instances with Ubuntu that can be monitored?
Is it possible to save a static html file with built in http headers ? So that putting it on any server will work including the http headers?
If so how?
I need to allow access to port 11211 from specific IP to existing rules (In a script that runs after the server loads)
The line I want to add:
iptables -A INPUT -p tcp -s xx.xx.xx.xx --dport 11211 -j ACCEPT
But the iptables already contains a drop line for this port so my line will append after and will never work. The drop line:
DROP tcp -- anywhere anywhere tcp dpt:11211
How can I do it?
I have a web site which will be featured in a place which will send a large peak of one-time traffic in a couple of weeks.
The server hosting the website also hosts the control panel my customers use.
The website part is simple and mostly static. But it includes a wordpress blog.
My question How and where can I put or cache the website and blog so that it will make it through traffic peak? For example, is it possible to host the pages on Amazon s3 so that they will be accessible via the regular urls on my domains without going through the server ?
EDIT:
I can change the urls of the control panel so that it will have its own subdomain and will remain on the main server. And force www.mysite.com for access to the company website and blog. Now, how can I make www.mysite.com/page1 read from amazon s3 bucket containing page1 html?
I am in the process of setting up mass email service.
My question is: what are the best practices to achieve maximum deliver-ability.
More precisely - what should I do/know to prevent spam filters from blocking the emails (the emails are not spam).
for example- how can I tell if my IP address is blacklisted somewhere and how can I prevent it from becoming blacklisted. Is amazon web services a suitable platform due to dynamic IP addresses, what are the restrictions on the from address, can it be different from the mail server domain.... you get it....
I'm experiencing a strange issue with Zabbix monitoring a MySQL server.
Most of the data from the server such as MySQL queries per second and MySQL uptime , Buffers memory etc. update nicely while some data like CPU iowait time (avg1) , Host local time ,MySQL number of threads and other items which were monitored in the past has last check time of about a week ago.
I can't find any logic in this, for example Mysql number of threads and Mysql queries per second are obtained in a similar way so it does not make sense one of them is monitored and one is not.
Please help- how can I fix this?
Update - I used zabbix_get from the zabbix server to check one of the items on the zabbix client and it works so the problem must be on the zabbix server side
On Ubuntu server load graphs I see 4 types of CPU consumption: User, System, Nice and Idle.
What does Nice type mean?
I have a server on AMAZON EC2 running Nginx +PHP with PHP FASTCGI via port 9000.
The server runs fine for a few minutes and after a while (several thousands of hits in this case) FastCGI Dies and Nginx returns 502 Error.
Nginx log shows
2010/01/12 16:49:24 [error] 1093#0: *9965 connect() failed (111: Connection refused) while connecting to upstream, client: 79.180.27.241, server: localhost, request: "GET /data.php?data=7781 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "site1.mysite.com", referrer: "http://www.othersite.com/subc.asp?t=10"
How can I debug what is causing FastCGI to die?
I'm writing a script that backup some files using s3cmd amazon s3 command line utility.
When I run a s3cmd shell command from shell it backs up the file nicely. When I run the same command from PHP script (using backticks or shell_exec) it does not do anything and returns no output to the PHP script.
Other shell commands run successfully in similar way (backticks) such as mysqldump and tar. using whoami in the script and shell I found the shell user is root and PHP script user is 'apache'.
How can I make the PHP script run the command successfully? (I know the command should work because copying it as is to the shell does work).
I'm trying to compile memcache for Dreamhost PS.
The library is created but not shown in phpinfo. Apache logs says the library isn't loaded because "undefined symbol: OnUpdateInt in Unknown on line 0"
I read that OnUpdateInt is defined in PHP4 and OnUpdateLong is defined in PHP5. Therefore the problem is that the compilation config thinks it should compile with PHP4.
The SAPI version of PHP is 5.2.10 (shown in phpinfo). The CLI is 4.4.9 (shown by php -v)
My question: How do I make the compilation of the libraries be compatible to the PHP5 on my server and not the php4.