Home / server / Questions / 745144
Accepted
berkes
Asked: 2015-12-25 05:51:40 +0800 CST

Deploy or shut-down nodes based on load with Chef

  • 772

AWS has OpsWorks, which is their own take on Chef.

We have several reasons[1] why we want to use a "normal" chef-server/-client setup.

But OpsWorks has one feature that I cannot find in chef-server/-client setups: load based instances.

They allow a stack to handle variable loads by starting additional instances when traffic is high and stopping instances when traffic is low, based on any of several load metrics.

I could not find much wrt to such a feature for a normal Opscode Chef Server/client setup. Just a few blogposts explaining how to user the AWS api and Chef-server API to monitor AWS instances and trigger node bootstraps on certain conditions. Self-built scripts.

How is this typically solved with Chef, if at all? Is this the task of Chef at all, or should a third tool manage such scaling? If so, are there any that tie well into the chef server setup?

[1]: Reasons are:

  • It ties us to AWS: which is what Amazon wants, sure, but we'd rather choose a setup that is more neutral.
  • It does not allow to run some nodes outside of AWS, e.g. a fallback-server on another vm-provider, or our existing gateway servers and so on.
  • We already have a tailored set of recipes that need to be slightly rewritten to work within the OpsWorks environment, such as the use of searches, databags and such.
chef

1 Answers

  1. Best Answer

    Chef is (mostly) not a provisioning tool. You would use the auto-scaling features of your cloud provider of choice (AWS ASGs, OpenStack Heat, etc). Scaling requires an overall view of load of every server in the group, while Chef is operates on one node at a time. There is a tool called chef-provisioning to manage creating servers (and other things) from the Chef DSL, but you would have to write something to feed it load data (possibly via the CloudWatch API) and come up with your own scaling rules (thresholds, predictive analysis, etc). When using Chef Client/Server with Autoscaling groups, you would generally use Packer to build yourself a customized AMI with your Chef config and validator key baked in so they register themselves with the Chef Server when they come up.

    • 2