Home / ubuntu / Questions / 164984
In Process
ECII
Asked: 2012-07-18 23:55:26 +0800 CST

Backup with duplicity on Google Cloud Storage

  • 772

I have already set a backup routine that backups all my files to Amazon S3 using duplicity. I would like to create a similar backup routine with Google Cloud Storage. Is it possible to get duplicity to work with GCS?

backup

5 Answers

  1. There is now (since 2013-07-19 - version 0.6.22) support to use Google Cloud Storage directly in duplicity.

    You should use gsutil config -a to create a ~/.boto config file. And then you can use duplicity like this:

    duplicity local_dir gs://yourbucket

    Also of note is that only the old authentication mechanism is supported. You should enable "Interoperable Access" from your Google Storage Console.

    Detailed steps on http://www.willdurness.com/back-up-your-vm-to-google-cloud-storage-using-duplicity.

    • 8
    • As of August 2012, Duplicity does not support Google Cloud Storage (GCS); a search of the duplicity-talk mailing list archives shows that there has been no discussion, even at a wishlist stage, of adding such support.
      • Google Docs is supported, but that API is totally different from GCS.
      • Wishlist bug filed recently
    • While the Python boto module used as a back-end by Duplicity to provide S3 support also supports GCS, adding GCS support to duplicity would require non-trivial modifications to its code; the S3 and GCS APIs are not identical.
      • If anyone wants to give adding GCS support a go, it would be simplest to start with and modify the _boto_single.py file which provides the backend for single-threaded S3 transactions.
    • That GCS isn't widely supported shouldn't come as a surprise. It is nowhere near as universal as S3; for example, you must register as a developer with Google and the service is still considered part of the "beta" Google Labs.
      • The only backup solution I could find supporting GCS is Zmanda (commercial).
      • As Google rolls out GCS on a wider basis--beyond just developers--I think we should start seeing support for it in FOSS apps.
    • For now, to use Duplicity with GCS, you must either backup to S3 and then use Google's gsutil tool to sync S3 with GCS, as Jorge's answer indicates; alternately, you could use Duplicity to backup locally and then sync that with GCS using gsutil.
    • 7

  3. This is a partial-answer that might help the next person:

    So far it seems that copying things to Google Drive/Docs is supported:

    duplicity localdir gdocs://username:[email protected]/remotedir/

    However this is different from Google Cloud Storage. I think this should be doable since duplicity and gsutil both use boto, but I'm not sure if this requires work to be done in duplicity itself though.

    As a work around until you sort this, you can certainly copy directly from the s3 to the GCS buckets:

    gsutil -m cp -R s3://yourbucket gs://yourbucket
    • 6

  4. Version v0.6.22 (2013/08/22) of Duplicity added Google Cloud Storage support.

    • 2

  5. deja-dup 34.1 (duplicity's GUI) now has "experimental support for Google Cloud Storage" (https://launchpad.net/deja-dup/+milestone/34.1)

    (I tried enabling interoperability for HMAC authentication on my GCS buckets and specifying an S3 Bucket in earlier versions of Deja-Dup - similar to how duplicity does it - but to no avail)

    • 2