My CentOS server's munin graph shows high swap, but the amount of apps memory seems low. Does this still mean I need to upgrade my memory? What is causing all that swap? Is it because of the committed memory? What does it mean when swap_cache is high?
What is the best determining factor when I need to upgrade the memory?
Company is expanding from a 30 person office to a 100 person office. We currently use SOHO routers and with the new expansion will probably need to look into enterprise-level routers.
What specs should I be focusing on that will ensure the stability and reliability of the network and internet speeds? Other bells and whistles such as VPN is second priority.
I have a website with domain mysite.com. I would like to have it so that if a user types in "mysite.com" it will redirect to "www.mysite.com". Additionally, "mysite.com/subdir" will also redirect to "www.mysite.com/subdir". I've looked and made changes with vhosts and also rewrites, but it's not working for the "mysite.com/subdir"->"www.mysite.com/subdir" case. Every time I type in "mysite.com/subdir", it will redirect to "www.mysite.com". Only "www.mysite.com/subdir" works.
Not sure how to troubleshoot this. I turned on rewrite logs and didn't see anything obvious, yet.
This is my config files so far. Please let me know what I'm missing. Thanks!
FYI: I'm using CentOS 5.4, nginx 1.2.0 on top of Apache 2.2.3. The site itself is built with CodeIgniter framework.
http.conf:
ServerTokens Prod
ServerRoot "/etc/httpd"
PidFile run/httpd.pid
Timeout 120
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 15
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 10
ServerLimit 256
MaxClients 60
MaxRequestsPerChild 10000
#StartServers 8
#MinSpareServers 5
#MaxSpareServers 20
#ServerLimit 256
#MaxClients 256
#MaxRequestsPerChild 4000
</IfModule>
<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
Listen 69
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_alias_module modules/mod_authn_alias.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule ldap_module modules/mod_ldap.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule cache_module modules/mod_cache.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule file_cache_module modules/mod_file_cache.so
LoadModule mem_cache_module modules/mod_mem_cache.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule version_module modules/mod_version.so
#rpaf settings
LoadModule rpaf_module modules/mod_rpaf-2.0.so
RPAFenable On
RPAFproxy_ips 127.0.0.1
RPAFsethostname On
# The header where the real client IP address is stored.
RPAFheader X-Forwarded-For
Include conf.d/*.conf
User apache
Group apache
ServerAdmin root@localhost
ServerName www.mysite.com:80
UseCanonicalName Off
DocumentRoot "/var/www/html"
<Directory />
Options FollowSymLinks
AllowOverride All
</Directory>
<Directory "/var/www/html">
Options Indexes FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all
</Directory>
<IfModule mod_userdir.c>
UserDir disable
</IfModule>
DirectoryIndex index.html index.html.var
AccessFileName .htaccess
<Files ~ "^\.ht">
Order allow,deny
Deny from all
</Files>
<DirectoryMatch "^/.*/\.svn/">
Order deny,allow
Deny from all
</DirectoryMatch>
TypesConfig /etc/mime.types
DefaultType text/plain
<IfModule mod_mime_magic.c>
MIMEMagicFile conf/magic
</IfModule>
HostnameLookups Off
ErrorLog logs/error_log
LogLevel warn
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog logs/access_log combined
ServerSignature Off
Alias /icons/ "/var/www/icons/"
<Directory "/var/www/icons">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>
<IfModule mod_dav_fs.c>
DAVLockDB /var/lib/dav/lockdb
</IfModule>
ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
<Directory "/var/www/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
IndexOptions FancyIndexing VersionSort NameWidth=* HTMLTable
AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip
AddIconByType (TXT,/icons/text.gif) text/*
AddIconByType (IMG,/icons/image2.gif) image/*
AddIconByType (SND,/icons/sound2.gif) audio/*
AddIconByType (VID,/icons/movie.gif) video/*
AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core
AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^
DefaultIcon /icons/unknown.gif
#AddDescription "GZIP compressed document" .gz
#AddDescription "tar archive" .tar
#AddDescription "GZIP compressed tar archive" .tgz
ReadmeName README.html
HeaderName HEADER.html
IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t
AddLanguage ca .ca
AddLanguage cs .cz .cs
AddLanguage da .dk
AddLanguage de .de
AddLanguage el .el
AddLanguage en .en
AddLanguage eo .eo
AddLanguage es .es
AddLanguage et .et
AddLanguage fr .fr
AddLanguage he .he
AddLanguage hr .hr
AddLanguage it .it
AddLanguage ja .ja
AddLanguage ko .ko
AddLanguage ltz .ltz
AddLanguage nl .nl
AddLanguage nn .nn
AddLanguage no .no
AddLanguage pl .po
AddLanguage pt .pt
AddLanguage pt-BR .pt-br
AddLanguage ru .ru
AddLanguage sv .sv
AddLanguage zh-CN .zh-cn
AddLanguage zh-TW .zh-tw
LanguagePriority en zh-CN zh-TW ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv
ForceLanguagePriority Prefer Fallback
AddDefaultCharset UTF-8
AddType text/x-component .htc
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddHandler type-map var
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
Alias /error/ "/var/www/error/"
<IfModule mod_negotiation.c>
<IfModule mod_include.c>
<Directory "/var/www/error">
AllowOverride None
Options IncludesNoExec
AddOutputFilter Includes html
AddHandler type-map var
Order allow,deny
Allow from all
LanguagePriority en es de fr
ForceLanguagePriority Prefer Fallback
</Directory>
</IfModule>
</IfModule>
BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0
BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
BrowserMatch "MS FrontPage" redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS/1.[0123]" redirect-carefully
BrowserMatch "^gnome-vfs/1.0" redirect-carefully
BrowserMatch "^XML Spy" redirect-carefully
BrowserMatch "^Dreamweaver-WebDAV-SCM1" redirect-carefully
vhost.conf:
NameVirtualHost *:69
<VirtualHost *:69>
ServerName mysite.com
ServerAlias vip.mysite.com
Redirect / http://www.mysite.com/
</VirtualHost>
<VirtualHost *:69>
DocumentRoot /home/mysite/mysite/www
ServerName www.mysite.com
</VirtualHost>
<VirtualHost *:69>
DocumentRoot /home/mysite/mysite/www/assets
ServerName static.mysite.com
</VirtualHost>
<VirtualHost *:69>
DocumentRoot /home/mysite/admin/www
ServerName admin.mysite.com
</VirtualHost>
<VirtualHost *:69>
DocumentRoot /home/other/trunk/www
ServerName othersite.com
ServerAlias www.othersite.com
</VirtualHost>
<VirtualHost *:69>
DocumentRoot /var/www/html
ServerName test.mysite.com
ServerAlias test2.mysite.com
</VirtualHost>
/home/mysite/mysite/www/.htaccess:
RewriteEngine on
# In my case all CI files are outside this web root, so we can
# allow any files or directories that exist to be displayed directly
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
# hide index.php
RewriteRule .* index.php/$0 [L]
# BEGIN Compress text files
<ifModule mod_deflate.c>
<filesMatch "\.(css|js|x?html?|php)$">
SetOutputFilter DEFLATE
</filesMatch>
</ifModule>
# END Compress text files
# BEGIN Expire headers
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 1 seconds"
ExpiresByType image/x-icon "access plus 2592000 seconds"
ExpiresByType image/jpeg "access plus 2592000 seconds"
ExpiresByType image/png "access plus 2592000 seconds"
ExpiresByType image/gif "access plus 2592000 seconds"
ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
ExpiresByType text/css "access plus 604800 seconds"
ExpiresByType text/javascript "access plus 604800 seconds"
ExpiresByType application/javascript "access plus 604800 seconds"
ExpiresByType application/x-javascript "access plus 604800 seconds"
ExpiresByType application/xhtml+xml "access plus 600 seconds"
</ifModule>
# END Expire headers
# BEGIN Cache-Control Headers
<ifModule mod_headers.c>
<filesMatch "\.(ico|jpe?g|png|gif|swf)$">
Header set Cache-Control "max-age=2592000, public"
</filesMatch>
<filesMatch "\.(css)$">
Header set Cache-Control "max-age=604800, public"
</filesMatch>
<filesMatch "\.(js)$">
Header set Cache-Control "max-age=604800, private"
</filesMatch>
</ifModule>
# END Cache-Control Headers
# BEGIN Turn ETags Off
<ifModule mod_headers.c>
Header unset ETag
</ifModule>
FileETag None
# END Turn ETags Off
/etc/nginx/conf.d/default.conf:
server {
listen 80;
server_name static.mysite.com;
location / {
root /home/mysite/mysite/www/assets;
index index.html index.htm;
expires max;
}
}
server {
listen 80;
server_name *.mysite.com www.mysite.com vip.mysite.com;
#Set this larger if uploading big files
client_max_body_size 5m;
location /
{
proxy_pass http://127.0.0.1:69;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
#client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
}
}
I've been noticing that my server has been using swap a quite more often now but I can't seem to figure out why. The munin graph shows that the apps memory usage is about 0.7G out of the 1.5G. The concern I have is the green line (committed memory). What is committed memory? Is it normal that the committed memory is high compared to the other memory stats, such as apps?
I want to know if I can optimize my apps a bit more or do I really need to just get more RAM for the server.
Currently running daemon apps on this one little server:
I've got a bare central repo that has this option turned on:
git config core.sharedRepository true
For some reason, the other users on this CentOS machine that pulls/pushes from the central repo always encounter a permission error when pushing onto a branch that someone already pushed.
I looked into the objects dir and realized that whoever pushed first gets their username set as the owner AND the group.
I've set it up so that the users all belong under a "developers" group and then chgrp -R the bare repo to be under the group ownership of "developers", but this has not fixed the issue yet.
Any other suggestions?
MORE INFO:
This is what happens when I push:
[dave@localhost wd]$ git push
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
error: insufficient permission for adding an object to repository database ./objects
fatal: failed to write object
error: pack-objects died of signal 13
error: failed to push some refs to '/gitrepos/repo.git'
As you can see, the permissions inside the objects folder on the bare central repo have dirs with different permissions that belong to another user, jane (the list has been shortened for readability):
[dave@localhost objects]$ ls -la
total 248
drwxrwxr-x 62 root developers 4096 Dec 13 15:37 .
drwxrwxr-x 9 root developers 4096 Dec 13 15:34 ..
drwxrwxr-x 2 root developers 4096 Dec 12 16:05 01
drwxrwxr-x 2 root developers 4096 Dec 12 16:05 2f
drwxrwsr-x 2 jane jane 4096 Dec 13 15:37 30
drwxrwxr-x 2 root developers 4096 Dec 12 16:19 33
drwxrwxr-x 2 root developers 4096 Dec 12 16:05 39
drwxrwxr-x 2 root developers 4096 Dec 12 16:19 6c
drwxrwxr-x 2 root developers 4096 Dec 12 16:05 89
drwxrwsr-x 2 jane jane 4096 Dec 13 15:37 8b
drwxrwxr-x 2 root developers 4096 Dec 13 05:41 8e
drwxrwxr-x 2 root developers 4096 Dec 12 16:19 90
drwxrwxr-x 2 root developers 4096 Dec 12 14:32 info
drwxrwxr-x 2 root developers 4096 Dec 12 14:32 pack
Currently have a MySQL replication setup in which Master A is replicated on another server db Slave B.
I'm shutting down the server that hosts Master A and need to start a new server that houses Master C that will be the replacement for Master A and maintaining all the data.
Slave B will then need to replicate from Master C. Master A will then be deleted.
How do I make this change with minimal or no downtime?
I see this process but can't seem to find anything about it on Google:
init─┬─crond
├─dbus-daemon
├─events/0
├─events/1
├─httpd───8*[httpd]
├─khelper
├─khostd───khostd
├─klogd
├─ksoftirqd/0
What is khostd? Is it useful?
I'm using a Centos 5.4 64bit system.
More Info after getting pidof khostd:
/proc/28069:
total 0
dr-xr-xr-x 2 root root 0 Oct 19 18:44 attr
-r-------- 1 root root 0 Oct 19 18:46 auxv
-r--r--r-- 1 root root 0 Oct 19 18:44 cmdline
-rw-r--r-- 1 root root 0 Oct 19 18:46 coredump_filter
-r--r--r-- 1 root root 0 Oct 19 18:46 cpuset
lrwxrwxrwx 1 root root 0 Oct 19 18:46 cwd -> /tmp
-r-------- 1 root root 0 Oct 19 18:46 environ
lrwxrwxrwx 1 root root 0 Oct 19 18:44 exe -> /usr/lib/.khostd/khostd
dr-x------ 2 root root 0 Oct 19 18:44 fd
dr-x------ 2 root root 0 Oct 19 18:46 fdinfo
-r--r--r-- 1 root root 0 Oct 19 18:46 io
-r--r--r-- 1 root root 0 Oct 19 18:46 limits
-rw-r--r-- 1 root root 0 Oct 19 18:46 loginuid
-r--r--r-- 1 root root 0 Oct 19 18:46 maps
-rw------- 1 root root 0 Oct 19 18:46 mem
-r--r--r-- 1 root root 0 Oct 19 18:46 mounts
-r-------- 1 root root 0 Oct 19 18:46 mountstats
-r--r--r-- 1 root root 0 Oct 19 18:46 numa_maps
-rw-r--r-- 1 root root 0 Oct 19 18:46 oom_adj
-r--r--r-- 1 root root 0 Oct 19 18:46 oom_score
lrwxrwxrwx 1 root root 0 Oct 19 18:46 root -> /
-r--r--r-- 1 root root 0 Oct 19 18:46 schedstat
-r--r--r-- 1 root root 0 Oct 19 18:46 smaps
-r--r--r-- 1 root root 0 Oct 19 18:44 stat
-r--r--r-- 1 root root 0 Oct 19 18:44 statm
-r--r--r-- 1 root root 0 Oct 19 18:44 status
dr-xr-xr-x 3 root root 0 Oct 19 18:44 task
-r--r--r-- 1 root root 0 Oct 19 18:46 wchan
ls -l fd
total 0
lr-x------ 1 root root 64 Oct 19 18:44 0 -> /dev/null
l-wx------ 1 root root 64 Oct 19 18:44 1 -> /dev/null
l-wx------ 1 root root 64 Oct 19 18:44 2 -> /dev/null
lrwx------ 1 root root 64 Oct 19 18:44 3 -> socket:[243807]
lsof -a -p 28069
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
khostd 28069 root cwd DIR 3,1 4096 6717441 /tmp
khostd 28069 root rtd DIR 3,1 4096 2 /
khostd 28069 root txt REG 3,1 2976132 6717448 /usr/lib/.khostd/khostd
khostd 28069 root mem REG 3,1 125736 9110591 /lib/ld-2.5.so
khostd 28069 root mem REG 3,1 1611564 9109521 /lib/libc-2.5.so
khostd 28069 root mem REG 3,1 208352 9109572 /lib/libm-2.5.so
khostd 28069 root mem REG 3,1 129716 9109534 /lib/libpthread-2.5.so
khostd 28069 root mem REG 3,1 16428 9109528 /lib/libdl-2.5.so
khostd 28069 root mem REG 3,1 101404 9110587 /lib/libnsl-2.5.so
khostd 28069 root mem REG 3,1 127661 6717504 /tmp/pdk-root/e6435b00fc79422519aa88bd9ce23223/POSIX.so
khostd 28069 root mem REG 3,1 18503 6717495 /tmp/pdk-root/34a1a6c9d35316e363f0994128ef61e6/Fcntl.so
khostd 28069 root mem REG 3,1 56454896 1118201 /usr/lib/locale/locale-archive
khostd 28069 root mem REG 3,1 1264090 6717493 /tmp/pdk-root/fcb734befe617ec3ae1edc38da810a5a/libperl.so
khostd 28069 root mem REG 3,1 46680 9109544 /lib/libnss_files-2.5.so
khostd 28069 root mem REG 3,1 13420 9109560 /lib/libutil-2.5.so
khostd 28069 root mem REG 3,1 45288 9109538 /lib/libcrypt-2.5.so
khostd 28069 root mem REG 3,1 26835 6717512 /tmp/pdk-root/3760d3688c78b22765b55d36a88382f4/FastCalc.so
khostd 28069 root mem REG 3,1 20493 6717510 /tmp/pdk-root/9319229253f468feb2a6076b8f5b0492/IO.so
khostd 28069 root mem REG 3,1 28572 6717506 /tmp/pdk-root/ff58a81c4ba367275c0ac887821ec093/Socket.so
khostd 28069 root 0r CHR 1,3 1201 /dev/null
khostd 28069 root 1w CHR 1,3 1201 /dev/null
khostd 28069 root 2w CHR 1,3 1201 /dev/null
khostd 28069 root 3u IPv4 243807 TCP *:etlservicemgr (LISTEN)
More info after looking into the .khostd directory:
ls -la
total 4188
drwxr-xr-x 2 root root 4096 Oct 13 16:30 .
drwxr-xr-x 59 root root 36864 Oct 18 16:47 ..
-rwxr-xr-x 1 root root 13096 Sep 4 2009 chat
-rwxr-xr-x 1 root root 157760 Sep 4 2009 find
-rwxr-xr-x 1 root root 711660 Mar 29 2011 hi
-rw-r--r-- 1 root root 334 Aug 16 17:07 .hostconf
-rwxr-xr-x 1 root root 60920 Sep 4 2009 iptables
-rwxr-xr-x 1 root root 2976132 Aug 23 13:59 khostd
-rwxr-xr-x 1 root root 14864 Sep 4 2009 kill
-rwxr-xr-x 1 root root 125920 May 25 2008 nstat
-r-xr-xr-x 1 root root 83696 Jan 21 2009 ps
-rwx--s--x 1 root slocate 28184 Sep 4 2009 slocate
cat .hostconf
bindport=9001
trustip=[Lots of comma separated IP addresses here]
heartserver=open.hichina.com
heartserver_port=3001
reportserver=open.hichina.com
reportserver_port=3001
version=Unix2.01
Recently moved to a new server with CentOS 5.4 and is 64 bit as opposed to our old 32-bit. It's got dual processors now and a bit higher memory.
I realized though, that apache has been heavily eating away at the memory and sometimes it would cause the system to swap.
The current setup has Nginx 1.0.8 in front to serve static content and Apache 2.2 to serve dynamic content (PHP). I also have APC installed. What could be causing this memory usage spike? I tried tuning the config file for apache but it would work if I turn certain directives to a absurdly low number. Before on the old machine I was able to use the default settings with no more than ~260MB of memory usage for all apache processes:
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
</IfModule>
But now the new machine is using this with high memory usage, it can get as high has 600MB+:
<IfModule prefork.c>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 20
MaxRequestsPerChild 500
</IfModule>
APC settings are as follows:
extension = apc.so
apc.enabled=1
apc.shm_segments=1
apc.optimization=0
apc.shm_size=64M
apc.ttl=0
apc.user_ttl=7200
apc.num_files_hint=1024
apc.mmap_file_mask=/tmp/apc.XXXXXX
apc.enable_cli=1
apc.cache_by_default=1
Any ideas why this is happening?
Edit: Some more process info
top - 21:18:45 up 10 days, 6:20, 4 users, load average: 0.00, 0.04, 0.06
Tasks: 103 total, 2 running, 101 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.3%us, 0.0%sy, 0.0%ni, 98.8%id, 0.2%wa, 0.0%hi, 0.7%si, 0.0%st
Mem: 1541280k total, 1479804k used, 61476k free, 99812k buffers
Swap: 10482404k total, 25548k used, 10456856k free, 725180k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5613 jenkins 17 0 1144m 184m 13m S 0.7 12.3 2:05.94 java
25306 apache 15 0 541m 125m 69m S 0.0 8.3 0:10.44 httpd
25308 apache 15 0 533m 118m 67m S 0.0 7.9 0:11.01 httpd
25356 apache 16 0 524m 112m 69m S 0.0 7.5 0:08.84 httpd
25305 apache 15 0 510m 103m 69m S 0.0 6.9 0:11.58 httpd
25309 apache 16 0 498m 92m 69m S 0.0 6.1 0:10.35 httpd
25307 apache 15 0 499m 90m 67m S 0.0 6.0 0:09.59 httpd
9759 mysql 15 0 428m 83m 6920 S 0.0 5.6 11:10.01 mysqld
16121 apache 15 0 494m 68m 48m S 0.0 4.6 0:05.50 httpd
2945 user 15 0 659m 18m 6364 S 0.0 1.2 0:00.53 node
25303 root 15 0 425m 12m 6176 S 0.0 0.8 0:00.07 httpd
2944 user 15 0 636m 10m 2288 S 0.0 0.7 0:00.11 node
[user@server~]# ps o rss,command -u apache
RSS COMMAND
70236 /usr/sbin/httpd
105860 /usr/sbin/httpd
128096 /usr/sbin/httpd
92888 /usr/sbin/httpd
121288 /usr/sbin/httpd
94768 /usr/sbin/httpd
115644 /usr/sbin/httpd
If I have a url that is used for getting messages and I create it like so: http://www.mydomain.com/somelonghash123456etcetc and this URL allows for other services to POST messages to. Is it possible for a search engine robot to find it? I don't want to have it in my robots.txt because that will expose it already to anybody that sees the robots file.
Of course I will put in other authentication in the app, but not having anyone discovering that URL is the first step.
Any common methods?
When users sign up to my site, the connection from my server to our 3rd-party SMTP provider is very bad and the app takes from a few seconds to a few minutes to connect. While it is trying to connect and send, users are forced to wait there and sometimes they would just give up and leave the site.
This is horrible user experience. What is a good way to make this seem instantaneous (having to host our own mailserver is not an option at this point)? Some way to execute this in the background? Maybe using some type of queuing mechanism?
I'm using the LNAMP stack on a CentOS 5 x64 machine. App is using CodeIgniter.
EDIT:
It seems using a local mailserver to use as a relay is a common solution. But how does one log or track whether or not email was delivered correctly by using this method?
Sidenote: Some ESPs provide a REST API for email delivery (ours do too). We currently use SMTP and not their REST API.
If setting up a mail server. What is the best domain to use for the IP's PTR record?
Is it usually better to use "mail.domain.com" or just "domain.com" ?
I understand that If the PTR record is set, some mail providers will attempt to get a HELO response from the origin address? Because I'm sending mail from my own server only AND is done via my code, I didn't think it was necessary to have port 25 open on the iptables settings. But it seems like I might need to for the mail providers to do the HELO, right? Are there any security implications to opening up port 25 even if I'm sending mail from my own server?
I installed mysql and server via yum on my 64bit CentOS 5 server. It starts up fine but when I try to stop it it stalls out and then I have to "Ctrl-C" it. Then I run a "service mysqld status" and it shows:
mysqld dead but subsys locked
I run ps aux and mysql is nowhere to be found. Starting mysqld again via "service mysqld start" works fine. Attempting to stop it creates the same problem.
I then realized that /var/lock/subsys/mysqld still exist. When running mysqld I checked /var/run/mysqld/mysqld.pid and it matched with the pid of the running service.
I tried reinstalling mysql and deleting all files and configs but to no avail.
What to do?
EDIT:
I added some echo statements in the /etc/init.d/mysqld file, particularly in the stop function:
stop(){
if [ ! -f "$mypidfile" ]; then
# not running; per LSB standards this is "ok"
action $"Stopping $prog: " /bin/true
return 0
fi
echo "beginning stop sequence"
MYSQLPID=`cat "$mypidfile"`
if [ -n "$MYSQLPID" ]; then
/bin/kill "$MYSQLPID" >/dev/null 2>&1
echo "killing pid $MYSQLPID"
ret=$?
if [ $ret -eq 0 ]; then
echo "return code $ret after kill attempt"
TIMEOUT="$STOPTIMEOUT"
echo "timeout is set to $STOPTIMEOUT"
while [ $TIMEOUT -gt 0 ]; do
/bin/kill -0 "$MYSQLPID" >/dev/null 2>&1 || break
sleep 1
let TIMEOUT=${TIMEOUT}-1
echo "timeout is now $TIMEOUT"
done
if [ $TIMEOUT -eq 0 ]; then
echo "Timeout error occurred trying to stop MySQL Daemon."
ret=1
action $"Stopping $prog: " /bin/false
else
echo "attempting to del lockfile: $lockfile"
rm -f $lockfile
rm -f "$socketfile"
action $"Stopping $prog: " /bin/true
fi
else
action $"Stopping $prog: " /bin/false
fi
else
# failed to read pidfile, probably insufficient permissions
action $"Stopping $prog: " /bin/false
ret=4
fi
return $ret
}
This is the result I get when I try to stop the service:
[root@server]# service mysqld stop
beginning stop sequence
killing pid 9145
return code 0 after kill attempt
timeout is set to 60
timeout is now 59
timeout is now 58
timeout is now 57
timeout is now 56
timeout is now 55
timeout is now 54
timeout is now 53
timeout is now 52
timeout is now 51
timeout is now 50
timeout is now 49
From looking at the code it seems to me that it will never break out of that while loop and will not be able to delete the lock file. Am I interpreting this wrong? I checked the same file on my other server and it uses the same code. I am dumbfounded.
EDIT: In the while loop part
/bin/kill -0 "$MYSQLPID" >/dev/null 2>&1 || break
For some reason it is not recognizing the return code. When service mysqld stop is called, the process has already been killed but not sure why it isn't allowing the loop to break out.
EDIT:
Further testing shows some weird behavior between calling /bin/kill and just calling kill, they apparently return different codes, why??????:
[root@server]# /bin/kill 25200
kill 25200: No such process
[user@server]# echo ${?}
0
[root@server]# kill 25200
-bash: kill: (25200) - No such process
[root@server]# echo ${?}
1
EDIT: I logged on as a non-root user and tried executing "kill" and "/bin/kill" with a surprising results:
[notroot@server ~]$ kill -0 23232
-bash: kill: (23232) - No such process
[notroot@server ~]$ echo $?
1
[notroot@server ~]$ /bin/kill -0 23232
kill 23232: No such process
(No info could be read for "-p": geteuid()=501 but you should be root.)
[notroot@server ~]$ echo $?
0
The "No info could be read" error does not show up in my other servers when executing kill and bin/kill as a nonroot user.
EDIT: Added logging described by quanta and also checked out the mysql log:
After a start and stop, mysql log shows this:
110918 00:11:28 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
110918 0:11:28 [Note] Plugin 'FEDERATED' is disabled.
110918 0:11:28 InnoDB: Initializing buffer pool, size = 16.0M
110918 0:11:28 InnoDB: Completed initialization of buffer pool
110918 0:11:29 InnoDB: Started; log sequence number 0 44233
110918 0:11:29 [Warning] 'user' entry 'root@server' ignored in --skip-name-resolve mode.
110918 0:11:29 [Warning] 'user' entry '@server' ignored in --skip-name-resolve mode.
110918 0:11:29 [Note] Event Scheduler: Loaded 0 events
110918 0:11:29 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.58-ius' socket: '/var/lib/mysql/mysql.sock' port: 3306 Distributed by The IUS Community Project
110918 0:11:34 [Note] /usr/libexec/mysqld: Normal shutdown
110918 0:11:34 [Note] Event Scheduler: Purging the queue. 0 events
110918 0:11:34 InnoDB: Starting shutdown...
110918 0:11:39 InnoDB: Shutdown completed; log sequence number 0 44233
110918 0:11:39 [Note] /usr/libexec/mysqld: Shutdown complete
110918 00:11:39 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Then in the tmp/mysql.log:
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
kill 23080: No such process
I stopped the stop process midway so I don't have to wait for the timeout. Looks like the process was killed. The problem, I think, is still with the different return codes from "kill" and "/bin/kill"
I installed nginx to handle requests alongside apache. Before, apache listened on port 80 and I've now switched to nginx to listen on port 80 and apache on some obscure port and have nginx proxy_pass to apache if the request is for non-static content.
My nginx config contains the following:
server {
listen 80;
server_name static.test.domain.com;
location / {
root /home/test/www/static;
index index.html index.htm;
}
}
server {
listen 80;
server_name domain.com *.domain.com;
location /
{
proxy_set_header Server "testserver";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://localhost:8800;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
The apache vhost config has the following:
NameVirtualHost *:8800
<VirtualHost *:8800>
DocumentRoot /var/www/html
ServerName domain.com
ServerAlias www.domain.com
</VirtualHost>
<VirtualHost *:8800>
DocumentRoot /home/test/www
ServerName test.domain.com
</VirtualHost>
...
I've noticed that the requests are faster now but I've also noticed that nginx is showing up at all the request header in the Server field, even if the request is for non-static pages. Is this a potential problem? I've seen some servers that use nginx on the same IP like my setup but the Server field is different (Apache shows up if it is a non-static content request, nginx show up when static).
Additionally, I'm using APC for opcode caching and I'm using .htaccess with a few redirect rules within my site directory (I'm thinking I need to port some apache rules over to nginx? Is that necessary?). I also have a few Java cron scripts that run (will this impede the nginx process?) Can this new setup lead to potential problems?
Lot's of questions, I know. But thanks in advance!
More info: using nginx 1.0.6 with apache 2.2 running on Centos 5 32bit.
My .htaccess file (does some of this need to be ported over to apache?):
# BEGIN Compress text files
<ifModule mod_deflate.c>
<filesMatch "\.(css|js|x?html?|php)$">
SetOutputFilter DEFLATE
</filesMatch>
</ifModule>
# END Compress text files
# BEGIN Expire headers
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 1 seconds"
ExpiresByType image/x-icon "access plus 2592000 seconds"
ExpiresByType image/jpeg "access plus 2592000 seconds"
ExpiresByType image/png "access plus 2592000 seconds"
ExpiresByType image/gif "access plus 2592000 seconds"
ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
ExpiresByType text/css "access plus 604800 seconds"
ExpiresByType text/javascript "access plus 604800 seconds"
ExpiresByType application/javascript "access plus 604800 seconds"
ExpiresByType application/x-javascript "access plus 604800 seconds"
ExpiresByType application/xhtml+xml "access plus 600 seconds"
</ifModule>
# END Expire headers
# BEGIN Cache-Control Headers
<ifModule mod_headers.c>
<filesMatch "\.(ico|jpe?g|png|gif|swf)$">
Header set Cache-Control "max-age=2592000, public"
</filesMatch>
<filesMatch "\.(css)$">
Header set Cache-Control "max-age=604800, public"
</filesMatch>
<filesMatch "\.(js)$">
Header set Cache-Control "max-age=604800, private"
</filesMatch>
</ifModule>
# END Cache-Control Headers
# BEGIN Turn ETags Off
<ifModule mod_headers.c>
Header unset ETag
</ifModule>
FileETag None
# END Turn ETags Off
I recently mounted a new partition and moved /home and /var/lib/mysql to the new drive and did a symlink to them.
Now I've noticed that the entropy (using munin) has dropped from around 3000 to around 100-200. Could the data mount and move caused this?
I use this as an apache webserver to host a site facing public users and I've noticed an increase in load as well (could the low entropy caused this?)
If this is a big issue, I would like to figure out a way to solve the low-entropy problem.
I've read that I can add entropy by using rng-tools and switching to using urandom. People say this is not a good approach for security-conscious systems. But would this be fine for a web app? How much less secure would that make my web app (we utilize encryption (hashing) for things like passwords and confirmation links and such, but I'm not sure how much of that falls under as "security-conscious")
Any ideas?
I'm using a cloud instance CentOS 5 (virtual machine).
I'm using a cloud CentOS 5 instance and I realized that I left the default partitioning in the beginning. Everything on / is mounted already on hda that is about 10GB. The other drive (sda) is about 90GB used for backups but I can free that up if needed.
My MySQL DB is now getting bigger and I'm thinking about mounting one of the sda drives onto /var/lib/mysql. Is this the usual practice for mysql (to give it its own partition)?
What do I need to do to make sure so that this move is painless and without data-loss? The server is live with users reading and writing to the DB, so I figure I will do it late night and turn off mysqld first. Then I'm planning to move everything in /var/lib/mysql into some temp dir and then mounting sda1 onto /var/lib/mysql and then moving the contents from the temp dir back into the new mounted /var/lib/mysql dir. Is this the right approach? Are there some other ways to give MySQL more space that I have not thought of?
I'm paranoid that a simple mv might miss some hidden files inside the dir (I apologize for my ignorance of how mv works).
My Centos 5 VM is drifting time.
I have scoured different answers to solve this problem.
I've looked at the VMWare's tips and added tinker panic 0 on top of my ntp.conf file. I also changed the kernel parameters to:
kernel /vmlinuz-2.6.18-194.32.1.el5 ro root=/dev/VolGroup00/LogVol00 notsc divider=10 clocksource=acpi_pm
It worked for a while and then I recently restarted my machine and it is back to its old ways again. I'm not sure how to make it work again even though I followed the same procedure as last time.
Any ideas?
Here is how my ntpq -pn looks like:
remote refid st t when poll reach delay offset jitter
==============================================================================
122.226.192.4 204.152.184.72 2 u 32 64 377 68.894 38873.7 5711.48
116.193.170.16 192.93.2.20 2 u 35 64 177 135.991 45598.7 10745.4
116.193.83.174 192.43.244.18 2 u 38 64 377 76.782 43291.2 8170.13
61.153.197.226 209.81.9.7 2 u 32 64 377 83.523 34288.8 6679.24
Sometimes if a database gets corrupted, a replicated database could also replicate that same corruption.
What is the best way to replicate/backup a database and prevent something like this from happening?
(specifically for MySQL)
I'm trying to setup passwordless login for from my mac to my remote server running CentOS with public key authentication for "user1".
I used $ ssh-keygen -t rsa to setup a public key on my mac and then copied the mykey.pub file over to the CentOS user1's .ssh dir and then did a
cat mykey.pub >> authorized_keys
in the .ssh dir.
I've also set the permissions for the .ssh dir to 700 and the authorized_keys to 600.
when I do a:
ssh [email protected]
It still prompts me for my password. What's going on?
Here is a copy of my sshd_config file on the remote machine:
Protocol 2
SyslogFacility AUTHPRIV
PermitRootLogin no
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PasswordAuthentication yes
PermitEmptyPasswords no
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
UsePAM no
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL
X11Forwarding yes
Subsystem sftp /usr/libexec/openssh/sftp-server
More info:
Here is my output for ssh -v. I think the system defaulted to using the id_rsa.pub instead of the mykey.pub and the mykey (private key file) that I named.
OpenSSH_5.2p1, OpenSSL 0.9.8r 8 Feb 2011
debug1: Reading configuration data /etc/ssh_config
debug1: Connecting to myremoteserver.com [1.1.1.1 (fake IP)] port 22.
debug1: Connection established.
debug1: identity file /Users/LocalUser/.ssh/identity type -1
debug1: identity file /Users/LocalUser/.ssh/id_rsa type -1
debug1: identity file /Users/LocalUser/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_4*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.2
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'myremoteserver.com' is known and matches the RSA host key.
debug1: Found key in /Users/LocalUser/.ssh/known_hosts:11
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/LocalUser/.ssh/identity
debug1: Trying private key: /Users/LocalUser/.ssh/id_rsa
debug1: Trying private key: /Users/LocalUser/.ssh/id_dsa
debug1: Next authentication method: password
How does one tell the system to use mykey instead of the id_rsa?
I'm currently looking at the apache documentation regarding virtual host. It gives an example on this page: http://httpd.apache.org/docs/2.0/vhosts/name-based.html
The example they show is:
<VirtualHost *:80>
ServerName www.domain.tld
ServerAlias domain.tld *.domain.tld
DocumentRoot /www/domain
</VirtualHost>
Wouldn't it work also with this type of config?:
<VirtualHost *:80>
ServerName *.domain.tld
DocumentRoot /www/domain
</VirtualHost>
Or this:
<VirtualHost *:80>
ServerName domain.tld
ServerAlias *.domain.tld
DocumentRoot /www/domain
</VirtualHost>
I currently have my web app on a single cloud server. Been doing this for a few months now and I have EVERYTHING on this instance (app, cron scripts, mysql, svn, dev environment, etc).
The site is getting bigger now and I'm going to open up a new server instance. What is the best way to utilize this? I have a few options:
Move my development environment over to the new instance along with production cron scripts (the scripts will do operations on the database). The production app and the database itself would still stay in the old server. This option will help minimize any accidental crashes to the production server if I want to do anything on the development server. But it won't be a pure production/development separation because I will still be running cron scripts and maybe thinking of hosting some static assets on the new instance (trying to utilize the most out of this setup).
Move database over to the new instance. This seems like a standard option from the research I've done. But a few questions come to mind: What should be done about the dev/prod separation (do I even need to separate the two)? Static assets--keep in app server or the db server?
I can only afford the total of two servers for now. I would love to get 3, but this is what I have to work with in the meantime.
Thanks for your feedback!