jacob_g's questions

Background

Hello,

I have a server I'm running on a free EC2 instance. I'm using nginx and passenger/rails as my web server and application server. The server receives little traffic (still in development), but a reasonable amount of traffic from random bots. The server also serves images from S3. The front end is served statically at mywebsite.com defined in one server block, and the backend is served using passenger at api.mywebsite.com.

The Problem

Seemingly randomly, the CPU usage on the server goes to 100%. The CPU usage spikes are correlated with network out spikes, although the network out spikes still seem to be relatively small. When this happens, the front end can no longer be served, and I can't even SSH into the server to check what processes are running.

What I've tried

• Blocking malicious bots using this bad-bot-blocker.
• Correlating network spikes to requests in the nginx access log at /var/logs/nginx/access.log. Usually the correlation is pretty unclear.
• Looking at /var/logs/nginx/error.log for anything relevant.

When the CPU does this, I often end up rebooting the server on the EC2 console which seems to work, but obviously isn't sustainable.

I'm new to deployment stuff/DevOps, so I was wondering if there's anything obvious I might be missing based on this information. I'm not even sure what layer is causing the problem (AWS/nginx/my rails backend/vanilla HTML/JS frontend). If there's any other information I can provide, please let me know.

Thanks,

Jacob

I'm trying to deploy a Rails app using capistrano. The deployment script, initiated by cap production deploy runs fine, until this command:

$HOME/.rbenv/bin/rbenv exec bundle install --path /home/deploy/app/shared/bundle --jobs 4 --without development test --deployment --debug --backtrace --quiet

I ssh'd into the server and run the same command, but replaced the --quiet flag with --verbose, and see the bundle is frozen on sassc installation:

Using activestorage 6.0.3.2
Using actionmailbox 6.0.3.2
Using webpacker 4.2.2
Using actiontext 6.0.3.2
Using rails 6.0.3.2
Installing sassc 2.4.0 with native extensions

From my extensive research, I understand that installing with native extensions may take very long, but no matter how long I let it run, nothing happens. I've manually installed the gem in verbose mode after leaving it running all night, and it appeared to have succeeded:

compiling ./libsass/src/position.cpp
compiling ./libsass/src/prelexer.cpp
compiling ./libsass/src/remove_placeholders.cpp
compiling ./libsass/src/sass.cpp
compiling ./libsass/src/sass2scss.cpp
compiling ./libsass/src/sass_context.cpp
compiling ./libsass/src/sass_functions.cpp
compiling ./libsass/src/sass_values.cpp
compiling ./libsass/src/source.cpp
compiling ./libsass/src/source_map.cpp
compiling ./libsass/src/stylesheet.cpp
compiling ./libsass/src/to_value.cpp
compiling ./libsass/src/units.cpp
compiling ./libsass/src/utf8_string.cpp
compiling ./libsass/src/util.cpp
compiling ./libsass/src/util_string.cpp
compiling ./libsass/src/values.cpp
linking shared-object sassc/libsass.so
current directory: /home/deploy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/sassc-2.4.0/ext
"make \"DESTDIR=\" install"
/usr/bin/install -c -m 0755 libsass.so ./.gem.20200720-1092-b4v9ch/sassc
current directory: /home/deploy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/sassc-2.4.0/ext
/home/deploy/.rbenv/versions/2.6.3/bin/ruby -I /home/deploy/.rbenv/versions/2.6.3/lib/ruby/2.6.0 -r ./siteconf20200720-1092-1xt3w4b.rb extconf.rb
current directory: /home/deploy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/sassc-2.4.0/ext
make "DESTDIR=" clean
current directory: /home/deploy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/sassc-2.4.0/ext
make "DESTDIR="
current directory: /home/deploy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/sassc-2.4.0/ext
make "DESTDIR=" install
Successfully installed sassc-2.4.0
Parsing documentation for sassc-2.4.0
Parsing sources...
100% [261/261]  lib/sassc/version.rb
Done installing documentation for sassc after 1 seconds
1 gem installed

but the deployment still freezes on sassc.

I am deploying on an AWS EC2 instance and noticed all my CPU Credits were depleted after a few hours of letting the deploy script run, so I even turned on unlimited mode to remedy this by paying for the extra CPU cycles. However, I've let the script run for days to no avail. I'm truly at a loss, and this is very frustrating.

Any help would be greatly appreciated.