Home / server / Questions / 137775
In Process
intuited
Asked: 2010-05-02 13:43:51 +0800 CST

Generating a record of the full(-ish) package management state

  • 772

I'm about to make some system changes and I'd like to have a record of my current happy system state. Is there a convenient way to create a record of this? I'd like to keep track of info like

  • currently installed packages and their versions
  • which packages are pinned at what version
  • which source (as in /etc/apt/sources.list) they were installed from
  • whether they were installed directly or automatically installed as a dependency of a different package
  • "unknown unknowns": ie stuff that I don't know that I should be keeping track of but which may be important when trying to figure out why something doesn't work

In short, I'd like to keep as much of the aptitude database as possible. What's the best way to do this? It would be nice if the resulting records were easily readable, though this is not really essential. It would be extra nice if it were readily versionable through an SCM tool like git.

There is a superuser question that partially answers this, but it only provides the list of currently installed packages.

update

Based on @ptman's answer, I've determined the following:

  • @ptman's last suggestion, backing up /etc/apt and /var/lib/..., would accomplish a subset of what aptitude-create-state-bundle does.
  • apt-cache policy just seems to give priority info, so I need something more comprehensive; aptitude-create-state-bundle would seem to be the ticket. However it would be useful to have some way to analyze the bundle to get information such as that provided by apt-cache policy.

I've started a git repo (using the setgitperms.perl hook util) that contains the extracted contents of aptitude-create-state-bundle, eg I do

sudo aptitude-create-state-bundle --force-gzip /dev/stdout | sudo tar xzv

to create the directory contents before running git add . && git add -u .. It's kind of slow since it's compressing and decompressing everything, I'll probably change it to just get the file listing with the --print-inputs option and then copy or hard link those files into the git repo.

This approach works pretty well: after running git gc --aggressive, I end up with a .git directory that's 2/3 the size of the original tarball. This was done after a second commit that recorded the state after installing puppet and its dependencies. It's pretty easy to see what changed by just viewing the diff, which is a neat way to find out what aptitude actually does when performing operations. I think there's some way to lay down hooks in apt itself (à la etckeeper), so I may end up using that facility to keep the repo updated with every apt operation.

One minor concern is that even with the setgitperms hook enabled, timestamps are not recorded by this system. So I added a second command to my pre-commit hook to dump a long directory listing to a file named .ls in the repo root directory:

find * | xargs -d \\n ls -ld >.ls

I'm not sure if this is really necessary, I'm only doing it in case aptitude-run-state-bundle uses the timestamps in some relevant way. Anyone know if this is true? Even then it seems unlikely to be necessary, since the timestamps of checked out files will always be at least as new as the original timestamps of those files when they were committed.

I'm going to keep doing things this way at least until I get a chance to figure out what puppet does and how to use it.

update 2

I've been using this system, but it does seem to be getting a bit unwieldy, in that the .git repo is growing quite rapidly. After 12 commits, the .git directory (after compression using git gc --aggressive) has grown in size from an initial size of (IIRC) 15-20 MB to about 60 MB.

It appears that that most of the bulk seems to be due to diffs in two large (~15MB) binary files, and I'm wondering if these files are necessary to the preservation of the system packaging state. Those files are:

var/cache/apt/pkgcache.bin
var/cache/apt/srcpkgcache.bin

Will it be possible to restore the repo using aptitude-run-state-bundle if these files are not present in the tarball? Would it be possible to add the current system versions to the tarball to enable that tarball to be used by aptitude-run-state-bundle (assuming that the current system versions are not corrupt, of course)?

ubuntu debian apt aptitude

2 Answers

  1. Check out the following tools:

    dpkg --get-selections
aptitude-create-state-bundle
dpkg -l
dpkg -l |tail -n +6 | awk '{ print $2 }' | xargs apt-cache policy

    Backup /etc/apt and /var/lib/{apt,dpkg,aptitude}.

    • 3

  2. Use puppet. Using that tool you can query a system to and make a list of it's current state. That list can then be used to clone the system. It's like buildout, except for servers.

    • 2