Home / server / Questions / 1091538
Accepted
agsamek
Asked: 2022-01-30 00:06:24 +0800 CST

how to automate servers installation with virtualization/LAMP (small IaC)?

  • 772

How to automate server installation? What infrastructure/systems fit our needs?

We have over 10 phisical servers with over 100 VMs, mostly LAMP. Not too many but also not anything that can keep working by itself. All servers have a KVM virtualization. We host various custom made applications for our customers that we modify frequently. So this is not a single application that needs to scale. Project teams work directly with customers and with VMs and this is a requirement.

The problem we routinely get into is that reinstallation of a VM/machine gets tricky since nobody controls recent changes to them, it poses a risk of instability, people defere it as long as possible and the problems get bigger over time.

I would like to establish a reinstallation procedure at least every 3 months and on demand. Reinstallation should start with a code repository and a backup. This should not be a VM duplication.

We estimate that this will cost us about 6 manmonths every 3 months in the current setup if done by hand. How can we reduce this time and also increase repetability of the process?

The question is what software can help us with automating this task. It should be as lightweight as possible. We do not need automated node allocation. We just need to have reinstallations to be as automated as possible. We assume human oversight of each reinstallation.

Our nodes are mostly LAMP with two systems having over 1TB structured DB data, several >200GB and around 50 small, custom VMs. We can schedule a planned downtime of each system easily, so this is not a question about high availability during reinstallation.

We just need to automate reinstallation as much as possible.

scripting debian installation

2 Answers

  1. Best Answer

    You need a configuration management framework like Ansible, Saltstack, Puppet or Chef.

    That is the tool to use for managing and applying configurations. However, selecting the actual tool is a minor part of the project.

    You need to design and deploy the process, where all changes to infrastructure are managed via this tool.

    This requires:

    • training people to use the tool
    • changing system architecture so that changes cannot be made without the tool
    • dealing with the resistance to changing fundamental operation mode of people

    You should also consider unifying the software stack infrastructure.

    If each VM has a bit different software configuration, you will either have

    • a lot of duplicate code in your configuration management code for different installations
    • complex logic to choose how to apply which configuration

    Either of these factors add a lot of maintenance overhead, and the end result might become as painful as your current situation.

    • 3

  2. Does your cloud have an API key?

    Most clouds have one-click LAMP images, e.g.

    Vultr: https://www.vultr.com/apps/lamp/

    Digital Ocean: https://marketplace.digitalocean.com/apps/lamp

    Linode: https://www.linode.com/docs/guides/lamp-stack-marketplace-app/

    If not, my absolute go-to is VestaCP https://vestacp.com/install/

    And https://hestiacp.com/

    Both add apache backend, nginx front-end, MySQL/mariadb/postgres, and php/php-fpm. All optionals too.

    For updates, I would create a separate server, drop a common SSH public key in /root/.ssh/authorized_keys that the separated server has, which can automatically log into each server, something like:

    INSTANCES=(
10.0.0.5
10.0.0.6
10.0.0.7
...
)


for INSTANCE in "${INSTANCES[@]}"; do
    ssh root@${INSTANCE} /bin/bash -c "apt update -y && apt upgrade -y"
done

    This is pretty much Terraform, Kubernetes etc. DIY.

    Also on GitHub, there's loads of cloud-specific guides, for example:

    Another Digital Ocean one: https://github.com/ethanbeyer/DigitalOcean-Droplet-Setup

    An AWS one: https://github.com/elionaz/aws-LAMP/blob/master/install_lamp.sh

    • 0