woky's questions

I have a storage where files are stored with filename corresponding to their MD5 hash. The files are stored like this:

...
/srv/storage/25/
/srv/storage/25/25D6D1AD130E4EB7D11F6AB48C09414E
...
/srv/storage/DF/
/srv/storage/DF/DF35407C5E433B5644538B41C957ABCE
...

The extra directories are just to prevent single root directory size blowing up.

These files are of various types; text files, PDF files, XLS files, anything goes. I'm serving these files over web interface that lists the files with real human file names that link to the HTTP file server. To make it easy for users to open these files from browser (Firefox) in appropriate application the web application appends file extension to the MD5 file name in file server URL so that browser can infer appropriate handler based on the filename extension. So for instance file report2017.pdf in the web interface links to

http://corp.biz/files/DF35407C5E433B5644538B41C957ABCE.pdf

which with the help of rewriting defined below will serve file:

/srv/storage/DF/DF35407C5E433B5644538B41C957ABCE

I use this location in my Nginx site to rewrite the URL to the correct path in the filesystem:

    location /files {
        root /srv/storage;
        rewrite '^/files/([0-9A-F]{2})([0-9A-F]{30}).*$' /$1/$1$2 break;
    }

Awesome! It works. Except that now Nginx replies with Content-Type: application/octet-stream for any URL under /files. Is there any way to make Nginx resolve MIME type before the rewrite?

The problem with Content-Type: application/octet-stream is that for PDF files even though the filename in URL ends with .pdf, Firefox for some reason goes straight to Save File dialog instead of letting users open it in a PDF viewer. This only happens on boxes where PDF file type is set to Preview in Firefox.

Thing that confuses me about configuration management tools such as Puppet is that they're great and straightforward for setting up server from scratch, but what every documentation neglects is the case where I have existing data (e.g. PostgreSQL database) that I want to use on the newly provisioned server.

The perfect example is migration between hosting providers. Sure Puppet will set up the new server with (almost) same configuration as the previous one, but the setup code (Puppet manifests) assumes that this new database server is at its genesis whereas there are precious business data that needs to be restored.

Setting up infrastructure for the first time is surely a task worth automating, but accidents happen and when the need for restoring the server arises, I'm left with semi-manual setup:

1. Puppet: Setup the server up to the point before data directories are initialized and services are started
2. Me at shell: Restore data from backups to proper places
3. Puppet: Continue with the rest, i.e. start the services

This cannot be that uncommon. Setting up servers for the first time with no existing data is one time event but recovering them can happen many times, and I execpt it will happen at least once (that's why we backup precious data).

What's the common practice in writing Puppet manifests to address the scenario that the node can be setup for second time and needs to restore data?

How do you deal with migration of servers managed by Puppet?

Googling this is hopeless. You mostly encounter pages describing how to restore Puppet DB/server but that piece of infrastructure mostly is not what your business generates profits from, right?

I realize that Puppet server has a fileserver, but keeping the backup data available there and setting proper access control feels very tedious to maintain compared to managing configuration with Puppet manifests.

I assume enterprises use full network backup solutions like Bacula or Amanda, or perhaps custom backup solutions based on Borg, Restic, Burp, Duplicati or plain rsync. Do you integrate these into your Puppet manifests? How?

I'd like to make an universal automated install script using Kickstart, which would be placed on a CD and used in conjunction with standard DVD installer, i.e. the user of this script would need to append inst.ks=cdrom to boot options and then it'd install e.g. Fedora on the machine asking only for passwords.

It seems that standard practice is to generate Kickstart scripts with passwords (or password hashes) in them, which doesn't seem ideal if you intend to distribute the script to third parties. I also want to enable sshd service in the Kickstart script, which would make it possible for anyone on LAN with knowledge of default passwords to log in into the machine. Of course I can just disable password authentication in sshd but now it's getting messy, i.e. I'm trying to workaround public knowledge of default passwords (and I may very well be forgetting something).

I'm new to Kickstart, I'd like to make the install script to ask/prompt/input for site-specific passwords, and so far I figured out two options:

1. Ask for passwords in %pre section and then generate e.g. accounts.ks with rootpw and user commands which will be included from main command section.
2. Use static default passwords and run interactive script on first boot that'll force user to change default passwords. (Or possibly use chage in %post.)

My worry is that, based on my Google searches, no one is doing that, no one is even asking for that. :-) Everyone seem to be placing final passwords/hashes directly into Kickstart scripts. So this leaves me with making my own interactive script for both options, which, I fear, will end up bad. Or perhaps, I'm getting the whole concept of (semi-) automated installs with Kickstart wrong.

What is the standard practice for creating universal Kickstart scripts for scenarios like this, where you need to distribute the script and not force users to edit it?

I'm currently responsible for setting up some infrastructure integration testing in my company. We'll definitely need to use virtual machines. I considered using libvirt for managing VMs in tests however it's likely we'll have more servers. So what I'm looking for now is something like private cloud. I'm currently considering to look at OpenStack, OpenNebula and Nimbus. I briefly looked at OpenStack documentation but it seems too complex. All I need above libvirt is some kind of load balancing of virtual machines across servers (but not transfer of running VM!) and the possibility of spawning multiple instances of one virtual machine.

Is there some other solution? What would you recommend to me?

Unfortunately paying for a cloud is not an option.