Home / server / Questions / 929823
In Process
aidan
Asked: 2018-09-07 20:57:08 +0800 CST

Should I expect Google App Engine to randomly restart instances?

  • 772

I had one instance of the app (shame on me), running in the australia-southeast1 region yesterday.

Last night was a critical time for the app; people were relying on it working flawlessly, and then halfway through the event, the instance just disappeared. About 8 minutes later, a new instance was created and continued to serve traffic.

Is this something I should expect? Instances randomly disappearing and restarting? I realise I should have had a couple of versions running, but what if both versions restart at the same time?

The logs don't show any errors at the point the app (a standard, simple Rails app) restarts. The logs just have an 8 minute gap, followed by Puma booting up in the new instance.

CPU usage was constant, and rarely above about 5%. Memory usage was around 840MB, with a slight peak of 880MB just before the restart. "Disk Bytes" was a constant ~15KB/sec, with a large spike of 40MB/sec around the time of the restart.

app.yaml

entrypoint: bundle exec rackup -p $PORT
env: flex
runtime: ruby

handlers:
- url: .*
  script: main.app
  secure: always

automatic_scaling:
  min_num_instances: 1
  max_num_instances: 4
  cpu_utilization:
    target_utilization: 0.75
google-cloud-platform

2 Answers

  1. According to the documentation there are 2 main reasons for an unexpected restart:

    • Instances restarted on a weekly basis [1].
    • A health check giving an HTTP code different from 200 [2].

    If you provide some additional info we can discuss it.

    • 2

  2. Similar problem with me; new health checks do not check the application, only the VM and Docker (unless you set them up to check your application, which I have not), and so it seems pretty clear that the problem is with app engine itself.

    Edit: Unsure why the neg, but to make this more clear.

    Yes, you should expect them to randomly restart instances. The behaviour you're seeing is not due to your app.

    • -1