Home / server / Questions / 772172
Accepted
yonran
Asked: 2016-04-23 00:26:26 +0800 CST

How to avoid duplicate scale-in with step scaling policy?

  • 772

With EC2 auto scaling, when using a step scaling policy (as opposed to a simple scaling policy) that scales in due to an alarm that is based on an AWS/EC2 metric (e.g. CPUUtilization <= 30%) with detailed CloudWatch monitoring disabled, when my auto scaling group scales in, it scales in two instances in short succession, without waiting for the metric to update. How can I prevent the auto scaling group from scaling in too fast for the metric to update?

activity history showing two terminations too close together

Edit: here was last night’s scaling history. At 5:15, 5:17, 5:19, 5:21 UTC, auto scaling scaled in due to low CPU Utilization, even though CPU Utilization only has data points at 5:10, 5:15, 5:20, and the scaling event should have ceased after the 5:15 datapoint. There appears to be no way to adjust the cooldown duration of step scaling policy scale ins (step scaling policies ignore the Default Cooldown (=600s) and only scale out policies have estimated instance warmup).

activity history showing 4 terminations within 6 minutes healthy host count showing rapid scale ins and gradual scale outsCPUUUtilization with step scaling policies showing that only scale outs are allowed to have an estimated warmup time

amazon-ec2 autoscaling

2 Answers

  1. Best Answer

    I had exactly the same problem. What i ended up doing was changing my scale down policy to SimpleScaling reducing by one instance at a time and setting a 10m cooldown. I've also changed my scale down alarm condition to trigger when there are 10 periods of 60 seconds below my threshold of 35% CPU. ( I have detailed cloudwatch metrics enabled ) The idea being that as soon as a scale down occurs then the alarm will turn off faster than if there was fewer periods and a longer evaluation time.

    I've still got StepScaling to go up so I scale up fast, but with SimpleScaling and a cooldown to scale in I scale down much more slowly.

    • 4

  2. You'll want to adjust your autoscaling cooldown settings. By default, it's five minutes, but you can customize it in the console or via the API/CLI with PutScalingPolicy.

    • 0