Home / user-77195

Max's questions

Martin Hope
Max
Asked: 2012-05-04 05:34:38 +0800 CST
  • 2

I have a deploy script where I want all of my webservers to simultaneously perform a git pull from the same place. When only two servers do the pull simultaneously, there are no problems. Once I have three or four servers, then it often happens that one or more servers 'times out' and doesn't finish the pull. I am using git over ssh.

Is there any server software I can use that will manage the incoming requests and and deal with each one individually or something?

Thanks

deployment git
Martin Hope
Max
Asked: 2012-01-04 23:45:51 +0800 CST
  • 13

knife ssh isn't finding my nodes. I know it should be able to because when I can search for them I find them

# knife search node name:*
2 items found

Node Name:   web_01
...

Node Name:   admin
...

However, when I run knife ssh (I'll show it with the 'debug' flag) I get

# knife ssh "node:*" "uptime" -VV
DEBUG: Using configuration from /root/.chef/knife.rb
DEBUG: Signing the request as dev
DEBUG: Sending HTTP Request via GET to ec2-xx-xx-xx-xx.compute-1.amazonaws.com:4000/search/node
FATAL: No nodes returned from search!

(yes the host is correct, I just censored it for the post).

I've tried modifying the QUERY parameter and always get the same results. I've tried:

  • node:*
  • role:*
  • *
  • *:*

Any ideas?

chef knife
Martin Hope
Max
Asked: 2011-11-28 06:47:06 +0800 CST
  • 1

We've got an ec2 m2.4xlarge running Magento (centos 5.6, httpd 2.2, php 5.2.17 with eaccelerator 0.9.5.3, mysql 5.1.52). Right now we're getting a large traffic spike, and our top looks like this:

top - 09:41:29 up 31 days,  1:12,  1 user,  load average: 120.01, 129.03, 113.23
Tasks: 1190 total,  18 running, 1172 sleeping,   0 stopped,   0 zombie
Cpu(s): 97.3%us,  1.8%sy,  0.0%ni,  0.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.4%st
Mem:  71687720k total, 36898928k used, 34788792k free,    49692k buffers
Swap: 880737784k total,        0k used, 880737784k free,  1586524k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2433 mysql     15   0 23.6g 4.5g 7112 S 564.7  6.6  33607:34 mysqld
24046 apache    16   0  411m  65m  28m S 26.4  0.1   0:09.05 httpd
24360 apache    15   0  410m  60m  25m S 26.4  0.1   0:03.65 httpd
24993 apache    16   0  410m  57m  21m S 26.1  0.1   0:01.41 httpd
24838 apache    16   0  428m  74m  20m S 24.8  0.1   0:02.37 httpd
24359 apache    16   0  411m  62m  26m R 22.3  0.1   0:08.12 httpd
23850 apache    15   0  411m  64m  27m S 16.8  0.1   0:14.54 httpd
25229 apache    16   0  404m  46m  17m R 10.2  0.1   0:00.71 httpd
14594 apache    15   0  404m  63m  34m S  8.4  0.1   1:10.26 httpd
24955 apache    16   0  404m  50m  21m R  8.4  0.1   0:01.66 httpd
24313 apache    16   0  399m  46m  22m R  8.1  0.1   0:02.30 httpd
25119 apache    16   0  411m  59m  23m S  6.8  0.1   0:01.45 httpd

Questions:

  • Would giving msyqld more memory help it cache queries and react faster?
  • If so, how?
  • Other than splitting mysql and php to separate servers (which we're about to do) is there anything else we could/should be doing?

Thanks!

UPDATE:

Here's our my.cnf along with the output of mysqltuner. It looks like a cache problem. Thanks again!

# cat /etc/my.cnf
[client]
port            = ****
socket          = /var/lib/mysql/mysql.sock

[mysqld]
datadir=/mnt/persistent/mysql
port=****
socket=/var/lib/mysql/mysql.sock
key_buffer = 512M
max_allowed_packet = 64M
table_cache = 1024
sort_buffer_size = 8M
read_buffer_size = 4M
read_rnd_buffer_size = 2M
myisam_sort_buffer_size = 64M
thread_cache_size = 128M
tmp_table_size = 128M
join_buffer_size = 1M
query_cache_limit = 2M
query_cache_size= 64M
query_cache_type = 1
max_connections = 1000
thread_stack = 128K
thread_concurrency = 48
log-bin=mysql-bin
server-id       = 1
wait_timeout = 300
innodb_data_home_dir = /mnt/persistent/mysql/
innodb_data_file_path = ibdata1:10M:autoextend
innodb_buffer_pool_size = 20G
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 64M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
innodb_thread_concurrency = 48
ft_min_word_len=3

[myisamchk]
ft_min_word_len=3
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

# ./mysqltuner.pl

 >>  MySQLTuner 1.2.0 - Major Hayden <[email protected]>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.52-log
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB +Federated +InnoDB -ISAM -NDBCluster 
[--] Data in MyISAM tables: 2G (Tables: 26)
[--] Data in InnoDB tables: 749M (Tables: 250)
[!!] Total fragmented tables: 262

-------- Security Recommendations  -------------------------------------------

-------- Performance Metrics -------------------------------------------------
[--] Up for: 31d 2h 30m 38s (680M q [253.371 qps], 2M conn, TX: 4825B, RX: 236B)
[--] Reads / Writes: 89% / 11%
[--] Total buffers: 20.6G global + 15.1M per thread (1000 max threads)
[OK] Maximum possible memory usage: 35.4G (51% of installed RAM)
[OK] Slow queries: 0% (35K/680M)
[OK] Highest usage of available connections: 53% (537/1000)
[OK] Key buffer size / total MyISAM indexes: 512.0M/457.2M
[OK] Key buffer hit rate: 100.0% (9B cached / 264K reads)
[OK] Query cache efficiency: 42.3% (260M cached / 615M selects)
[!!] Query cache prunes per day: 4384652
[OK] Sorts requiring temporary tables: 0% (1K temp sorts / 38M sorts)
[!!] Joins performed without indexes: 100404
[OK] Temporary tables created on disk: 17% (7M on disk / 45M total)
[OK] Thread cache hit rate: 99% (537 created / 2M connections)
[!!] Table cache hit rate: 0% (1K open / 946K opened)
[OK] Open file limit used: 9% (453/5K)
[OK] Table locks acquired immediately: 99% (758M immediate / 758M locks)
[OK] InnoDB data size / buffer pool: 749.3M/20.0G

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    Enable the slow query log to troubleshoot bad queries
    Adjust your join queries to always utilize indexes
    Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
    query_cache_size (> 64M)
    join_buffer_size (> 1.0M, or always use indexes with joins)
    table_cache (> 1024)
centos mysql php httpd amazon-web-services
Martin Hope
Max
Asked: 2011-10-07 03:56:14 +0800 CST
  • 1

Newb question: Running CentOS 5.x I had an older version of PHP (about 5.2.9 or something) and I downloaded the source for and installed the latest (5.2.17). Now, I checked yum and it had PHP 5.2.17 so I installed it. So now I have php installed by yum in /usr/bin and my old manual install in /usr/local/bin. I'm getting an exception 'pdo_mysql extension not installed' when I try to run php from the command line for some users and I think that the problem is that I have two PHPs running around. How can I get rid of the manually installed one?

Thanks

centos php
Martin Hope
Max
Asked: 2011-06-10 03:03:08 +0800 CST
  • 0

I'm having trouble gaining SSH access to my new server. I currently have one server set up working the way I want it to, and I'm trying to get a different one to work the same way. I've copied my users files (/etc/passwd, /etc/shadow, /etc/group, and /etc/gshadow) and the /home folders (including /home/*/.ssh). I even added a new user (testuser) on the new machine and am using that to test the ssh settings. Both /etc/ssh/sshd_config and /etc/ssh/ssh_config are the same, as are the iptables rules.

When I try to use PuTTY to log in to the new server using a copy of the settings from the old server (except for address, of course) I get an error:

Disconnected: No supported authentication methods available

And in the /var/log/secure file I get the line:

sshd[6562]: Received disconnect from xxx.xxx.xxx.xxx: 14: No supported authentication methods available

Both old and new server are running centOS

Where else can I find configurations that might be off?

Thanks.

centos ssh iptables
Martin Hope
Max
Asked: 2011-04-06 06:46:56 +0800 CST
  • 0

I upgraded MySQL on my CentOS server using yum

# mysql --version
mysql  Ver 14.14 Distrib 5.1.52, for redhat-linux-gnu (x86_64) using readline 5.1

I run service mysqld start and get the following error in the error logs (after it telling me that MySQL couldn't start):

110405 10:10:51 mysqld_safe Starting mysqld daemon with databases from /mnt/persistent/mysql
/usr/libexec/mysqld: Table 'mysql.plugin' doesn't exist
110405 10:10:51 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
110405 10:10:51  InnoDB: Started; log sequence number 30 391318478
110405 10:10:51 [ERROR] /usr/libexec/mysqld: unknown variable 'innodb_log_arch_dir=/mnt/persistent/mysql/'
110405 10:10:51 [ERROR] Aborting

110405 10:10:51  InnoDB: Starting shutdown...
110405 10:10:56  InnoDB: Shutdown completed; log sequence number 30 391318478
110405 10:10:57 [Note] /usr/libexec/mysqld: Shutdown complete

110405 10:10:57 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

So, I ran mysql_upgrade like the 3rd line says. However, that gives the error:

# mysql_upgrade
Looking for 'mysql' as: mysql
Looking for 'mysqlcheck' as: mysqlcheck
Running 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/lib/mysql/mysql.sock' mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) when trying to connect
FATAL ERROR: Upgrade failed

I think there's something circular going on: I can't start mysqld without running mysql_upgrade and I can't run mysql_upgrade without mysqld running.

Any ideas?

mysql