I'm experimenting with chef and moving my server configuration over to it. One hurdle I'm encountering is how to deal with packages that must be compiled and installed manually (configure, make, make install) but take forever to compile. I don't want to hinder a server start up time just because a dependency takes forever to build.
Specific usecase: wkhtmltopdf. Wkhtmltopdf with full features requires a custom/patched qt that takes FOREVER to compile.
Here are what I see as my options: 1) Compile the package using chef on each startup (Con: takes forever to start a server) 2) Compile the packages once, tar and them up. Then just download the package and install it. (Con: won't work for machines with different OSes/hardware) 3) Compile the package once and just keep it in a base image and out of chef (Chef: package is not in chef)
Any ideas?
The proper approach wrt chef/puppet et al is to package that software in your distro(')s native packaging format yourself and include it in your repositories and then to pull that package.
While many of the shared Chef cookbooks do build from source, that is not because it is necessarily the best route to get software on the systems. It is often indicative of the "simplest thing that could possibly work (but isn't optimized)," or "the (cookbook) author doesn't have time / resources to maintain a public repository of packages."
The "common practice"(*) is to build and install native packages for your platform, and host them on an internal repository. If you work at a company that is amicable to sharing, contributing those to upstream or using a publicly hosted repository (like Ubuntu PPAs) may be an option too.
The problem with the approach is every major distribution has different opinions on package management, so it all works slightly differently. That's where Jordan Sissel's "fpm" is really handy. While it only works on RPM, DEB and Solaris now, support for other packaging systems is planned. You can further hook fpm into a continuous integration server like Jenkins by calling the fpm command you want from a build step in Jenkins.
Once you have built the package(s) you want to deploy, you'll need to put them in a repository. If you can't share this publicly through a PPA, then I suggest building the package repo host with Chef. If you're using Debian/Ubuntu, reprepro is common for making an apt repository. For RPM/Yum, mrepo is common, though I don't know of a cookbook that sets it up. Then you could use apt or yum cookbooks to set up the repository on your other nodes.
Another route that might be a simpler option is to build the software, tar it up and stick it on an HTTP server your nodes can get to, and then use Bryan Berry's ark cookbook's resource for retrieving and unpacking it.
(*) I say common because I dispise "best" here :)