I'm trying to chase a bug on a Rails site and I've noticed that there are recurring Ruby processes which are hogging CPU and won't go away. I'd like to suss out what's starting them and how to either stop them or give them the resources they need to complete and get their work done.
Background: The server is Ubuntu 10.04 LTS. Ruby is 1.9.2-p290. The site is Rails 3.1.4, and Passenger 3.0.9 is on Nginx 1.0.8.
The site code is deployed and owned by a non-wheel user named site-runner
, so Passenger processes should be running as that user. That user has no crontab file.
This is what makes me suspicious:
top - 13:41:05 up 73 days, 20:26, 2 users, load average: 2.11, 2.06, 2.28 Mem: 508272k total, 295660k used, 212612k free, 12608k buffers Swap: 1048572k total, 32020k used, 1016552k free, 42580k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 26911 site-run 20 0 227m 116m 2656 R 101 23.4 18:08.55 ruby 26919 site-run 20 0 227m 116m 2656 R 100 23.4 18:08.50 ruby [...]
I wanted to confirm this wasn't Passenger, so I used pstree
:
$ pstree -apu init,1 |-PassengerWatchd,17903 | |-PassengerHelper,17908 | | |-ruby,17912 ... | | | |-{ruby},17938 | | | `-{ruby},26888 | | |-{PassengerHelpe},17913 [Collapsed a bunch of Passenger processes] | | `-{PassengerHelpe},17926 | |-PassengerLoggin,17915,nobody | | `-{PassengerLoggi},17928 | |-{PassengerWatch},17907 | |-{PassengerWatch},17929 | `-{PassengerWatch},17930 |-cron,2331 |-getty,7533 -8 38400 tty1 |-master,2536 | |-pickup,26864,postfix -l -t fifo -u -c | |-qmgr,2543,postfix -l -t fifo -u | `-tlsmgr,19889,postfix -l -t unix -u -c |-mysqld,23916,mysql | |-{mysqld},23922 [collapsed a bunch of mysql processes] | `-{mysqld},15541 |-nginx,17931 | `-nginx,17932,site-runner |-rsyslogd,2297,syslog -c4 | |-{rsyslogd},2303 | `-{rsyslogd},2304 |-ruby,26911,site-runner ... | `-{ruby},26913 |-ruby,26919,site-runner ... | `-{ruby},26921 |-sshd,2329 | `-sshd,27099 | `-sshd,27110,parker | `-bash,27111 | `-pstree,27218 -apu |-udevd,2108 --daemon | |-udevd,2139 --daemon | `-udevd,2142 --daemon `-upstart-udev-br,2066 --daemon
...and there are those ruby processes, 26911 and 26919, not children of Nginx or Passenger.
I've killed the processes, but they restart.
How can I figure out what these processes are doing and either urge them to completion or make them go away and stay away?
In
top
, you can toggle the display of the full command by hitting the "c" key.If you're out of
top
, you can also get the full command using someps
options:ps -fp 26911
So, "f" for the full command, and "p" to specify the process ID you're interested in.
You can also look at the proc filesystem on Linux.
cat /proc/26911/cmdline
will show the command line used for process 26911.