Home / server / Questions / 351304
Accepted
Tom
Asked: 2012-01-19 16:01:10 +0800 CST

projects or packages to emulate a tape drive with a Amazon S3 or OpenStack Swift compatible backend?

  • 772

I've got some data that is being backed up to tape via bacula, and at some point soon I am going to need to migrate to non-tape storage.

Bacula supports to-disk archiving, but it occurred to me that if only there was some virtual tape library interface to S3 or openstack Swift, or some other object store service, I could just swap out the configurations and maintain the pre-existing archiving, ageing and scheduling configurations seamlessly.

I had a google, and I can see that Amanda (using a Device API) and a bunch of backup products have S3 object store back-ends, but I don't see any projects that would drop-in replace using *nix /dev/st0 device interface semantics.

This may well be because I have no idea, and that the idea is preposterous and impossible, etc, however I'd be interested to know if I have missed any obvious things before I delve into some hacked up replacement.

So clarify the the platform options - its not hosted on AWS and there is at present no EBS available and the target is an in-house S3 compatible openStack Swift object store, so bandwidth and storage costs are internal.

backup amazon-s3 bacula openstack cloud-storage

3 Answers

  1. Best Answer

    This is more than a questionable approach for anything anyone could consider a backup but you should be able to give it a try. The first thing you need would be anything that presents a block interface on top of S3. Something like a NBD server with a S3 backend - have a look at this.

    After you got that working you need something that presents a file on top of a block device and offers a SCSI target/virtual HBA to your system with a changer and a tape drive. Have a look at this project here.

    See the S3NBD project (first link) for additional rationale why running your backup and recovery with S3 might be a bad idea (with the current S3 pricing scheme).

    • 1

  2. The bad thing about emulating a tape on EC2 is that in order to emulate a tape, you must emulate all the aspects. That includes the idea that when you load a tape and aren't planning to write at the start, you must seek through all that data.

    I believe a simpler course of action would be to interface EBS volumes as block devices in Bacula.

    What might be an even simpler course of action would be to use Media Type = Fifo and link that to a program that writes out to EBS. For restores, you'll have to work something else out, but it'll get your data moved.

    • 1

  3. You might try MHVTL it emulates a virtual tape library and uses files/directories as storage. You could S3FS or other mechanism to emulate files/directories on which MHVTL could sit. You also could modify MHTL to just move its files to/from S3/Swift like target on mount/unmount. It looks like /dev/tapedevice and a scsi media changer, so backup products might work unchanged. Dont know about support.

    • 1