Home / server / Questions / 690175
Accepted
Amos Shapira
Asked: 2015-05-07 17:21:37 +0800 CST

What's a good load balancer check for puppet master

  • 772

We are running a couple of Puppet Master version 3.7 (Not PE) on AWS EC2 instances behind an Elastic Load Balancer (mainly for high availability).

Currently the load balancer's check is just TCP connect on port 8140. But this didn't detect that one of the instances was hung (it was still accepting the initial TCP connection but not doing anything with it).

I'm looking for way to actually send a meaningful "no-op" request to the puppet master and have it send back a result.

More specific requirements:

  1. It shouldn't trigger heavy processing (the checks happen every few seconds)
  2. It should succeed if the puppet master is able to actually handle "real request" and fail otherwise.
  3. It should be possible to execute from ELB (i.e. basically it should return "200 OK" on successful https requests on a static path.

Is there a good static path I can give the health check to use? So far I haven't found any.

amazon-web-services

1 Answers

  1. Best Answer

    What you could use as a check if puppet master is working correctly is to set up some kind of API query, for example:

    curl -k -H "Accept: pson" https://puppetmaster:8140/production/status/no_key

    Offcourse, API access won't be available by default, meaning you will have to use the SSL certificates of a signed client to be able to access the REST API. Also, all certificates won't have all the needed permissions.

    This is an example how to do a full manifest compilation:

    curl -k -H 'Accept: pson' \
--cert /var/lib/puppet/ssl/certs/node.example.com.pem \
--key /var/lib/puppet/ssl/private_keys/node.example.com.pem \
--cacert /var/lib/puppet/ssl/ca/ca_crt.pem \
https://puppetmaster:8140/production/status/no_key

    Don't know if it's possible to integrate certificates into ELB checks, but if it's not, maybe you can play around REST API access controls and set up something along these lines in auth.conf:

    # Allow ELB to access REST endpoint
path ~ ^/status/no_key$
auth off
allow_ip <ip_of_loadbalancer>

    Restart puppetmaster, and try simple check:

    curl -k -H "Accept: pson" https://puppetmaster:8140/production/status/no_key

    This should work. Just be carefull with auth.conf not to overextend yourself!

    To see all the available API keys, take a look at REST API documentation on PuppetLabs site.

    • 4