Home / server / Questions / 348345
Accepted
Jeevan Dongre
Asked: 2012-01-10 21:21:34 +0800 CST

How to set the auto scaling policy based upon memory

  • 772

I was able to set the auto scaling policy and also tested works fine, I am using EC2 small instance to run Ruby on Rails app, since Rails is more of Memory Intense platform I wanna scale the EC2 instances based upon the memory not on CPU Utilization. I googled yesterday but did not fetched any results which would actually helped me.

Any suggestions would be help ful.

ubuntu amazon-ec2

2 Answers

  1. Best Answer

    Auto scaling on EC2 is based on triggers from Cloudwatch. By default, Cloudwatch does not collect data about memory usage (the official reason being something to the effect of such metrics requiring 'a look into the OS running in the instance')

    The solution, therefore, is to setup a custom metric to monitor memory usage attach an alarm to that metric, and then base your scaling policy off that alarm.

    Amazon has described the procedure fairly well in this forum post.

    Firstly, you have a script that will gather the data from 'free' (copied from the above page):

    #!/bin/bash

export AWS_CLOUDWATCH_HOME=/home/ec2-user/CloudWatch-1.0.12.1
export AWS_CREDENTIAL_FILE=$AWS_CLOUDWATCH_HOME/credentials
export AWS_CLOUDWATCH_URL=https://monitoring.amazonaws.com
export PATH=$AWS_CLOUDWATCH_HOME/bin:$PATH
export JAVA_HOME=/usr/lib/jvm/jre

# get ec2 instance id
instanceid=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`

memtotal=`free -m | grep 'Mem' | tr -s ' ' | cut -d ' ' -f 2`
memfree=`free -m | grep 'buffers/cache' | tr -s ' ' | cut -d ' ' -f 4`
let "memused=100-memfree*100/memtotal"

mon-put-data --metric-name "FreeMemoryMBytes" --namespace "System/Linux" --dimensions "InstanceId=$instanceid" --value "$memfree" --unit "Megabytes"

mon-put-data --metric-name "UsedMemoryPercent" --namespace "System/Linux" --dimensions "InstanceId=$instanceid" --value "$memused" --unit "Percent"

    The script takes the number from the '-/+ buffers/cache' row under the 'free' column, as a percent of 'total' (under the 'Mem' row), and sets up 2 metrics - the percent of memory used, and the total memory free in MB.

    All of the AWS API tools are very slow (relatively speaking) - if possible, use the API directly from some supported language (e.g. Ruby) and you will get much better performance than the script above.

    Modify the above script to suit your needs (you probably don't need both metrics, etc) and set it up to run every few minutes via cron. Keep in mind that you get a limited number of custom/detailed metrics and alarms for free, after which their is a monthly cost.

    There is also a Google Code project - 'Aws Missing Tools' that has scripts for monitoring memory usage and a few other metrics that may be helpful.

    Once you have your metric setup and functioning, create an alarm for it and proceed with autoscaling (as-put-scaling-policy, etc) as you would for any of the pre-defined metrics.

    • 6

  2. Amazon just updated (Feb, 2013) their CloudWatch's scripts, now you can choose to watch quite bit of information:

    • Memory Utilization - Memory allocated by applications and the operating system, exclusive of caches and buffers, in percentages.
    • Memory Used - Memory allocated by applications and the operating system, exclusive of caches and buffers, in megabytes.
    • Memory Available - System memory available for applications and the operating system, in megabytes.
    • Disk Space Utilization - Disk space usage as percentages.
    • Disk Space Used - Disk space usage in gigabytes.
    • Disk Space Available - Available disk space in gigabytes.
    • Swap Space Utilization - Swap space usage as a percentage.
    • Swap Space Used - Swap space usage in megabytes.

    You can download the CloudWatch Monitoring Scripts for Linux and also check the full documentation.

    Source of full AWS PR: http://aws.typepad.com/aws/2013/02/cloudwatch-monitoring-scripts-updated.html

    • 0