Home / server / Questions / 221835
In Process
Dan
Asked: 2011-01-13 17:10:38 +0800 CST

CentOS: update a package from a repository safely on a production server

  • 772

I have a CentOS server on a production environment. I need to update the PHP package that I installed using the REMI repository.

Quite easy:

yum update php

But what is it going to happen if something goes wrong during the update? How can I rollback?

What's the best technique to make sure not to compromise a production server due to an update?

Is it maybe better to compile PHP from the source, rather than using a binary package?

EDIT: I am not afraid of incompatibility between my code and the new PHP version (I have well tested that on development). I am more afraid of something going wrong while CentOS updated the binary (power cut, lost connection, unexpected conflit)

Thanks,
Dan

centos

3 Answers

  1. The binary package has been built by your vendor and has generally had a fair amount of testing. By building from source, you lose most of the advtantages of your packaging system, specifically:

    • The packaging system lets you verify the integrity of installed packages using checksum.
    • The packaging system is aware of dependencies between packages and can automatically install the necessary additional software to make things work.
    • The packaging system lets you conveniently remove a package, whereas most source installs don't really provide a good uninstall mechanism.
    • Using packages means it's very easy to install an older version of the package of there are problems with the new version.

    There are certainly some advantages to building from source -- for example, you may require a different configuration than provided in the vendor package. However, even if you elect to compile the code locally, it's almost always a better idea to create a package from it and install it using the normal packaging tools for above reasons (rather than running make install directly into your filesystem).

    What's the best technique to make sure not to compromise a production server due to an update?

    Test the updates in your development environment to make sure they work as expected before deploying them on your production system. Ensure that your development environment accurately reflects your production environment.

    This is true regardless of how you choose to install the updated software.

    • 1

  2. The absolute best thing you can do in this case is thoroughly test your set of packages as a release on a test server, including rollback to the previous set or packages. I work in release management in a very large internet company and we do exactly this.

    By doing thorough installs and rollbacks before going to production you eliminate nasty surprises (make sure to review your install logs!). You also need to consider doing bare metal restores, where you take a host with a basic OS installed, and install your release on top of that. You will often find many hidden surprises when you install all the packages together simultaneously this way.

    Try to take hosts out of rotation if possible when doing production installs, and do an automated healthcheck to determine they are working properly when the install completes. That way if something goes catastrophically wrong on one host, you can leave it our of rotation. Then come back after your software push, wipe the host, and reinstall. Obviously that approach is dependent on having redundant servers set up.

    • 1

  3. You're probably best off using the binary package, assuming the current version was installed as a binary. Do you have any test environment in which to run a trial of this new PHP package? What version of PHP is currently installed? Run rpm -qi php. What is the version that you're looking to install?

    • 0