How to backup 20+TB of data?

You need to take a step back and stop thinking "I've got 20TB on my NAS I need to back up!" and develop a storage strategy that takes into account the nature of your data:

• Where is it coming from and how much new data are you getting? (you've got this in your question)
• How is the data used once you have it? Are people editing the pictures? Do you keep the originals and generate edited versions?
• How long do you need to keep all the data? Are people still making changes to pictures from 2 years ago?

Depending on the answers to the last two questions, you probably need more of a Archiving System than a radically different backup system.

Data that is static (e.g. 2 year old pictures that you retain "just in case") doesn't need to be backed up every night, or even every week, it needs to be archived. What you actually do might be more complex, but conceptually, all the old pictures can be written off to tape (multiple copies!) and not backed up any more.

Based on your comments, some additional thoughts:

• Since you keep the originals of each shoot untouched and work on a copy, and assuming that at least some of the original pictures are duds, you might be able to cut the amount of data that needs to be backed up in half.

• If you still can't finish a full backup within whatever window of time you have, a common way to speed things up is to do a disk-to-disk backup first and then later copy the backup set off to tape.

You have two options:

Option 1:

1. Buy another NAS
2. Give your users RO access to the new_NAS
3. Move all files older than 2 years to new_NAS
4. Keep backing up old_NAS as usual
5. Every 6 months move files older than 2 years to new_NAS

Option 2:

1. Buy another NAS

2. Run rsync every hour: old_NAS -> new_NAS

   or, better use something like rdiff-backup which does rsync + keeps deltas with file changes (you can restore older versions of the files)

   rdiff-backup  user1@old_NAS::/source-dir    user2@new_NAS::/dest-dir
   

3. Every 6 months clean old files running something like:

   rdiff-backup --remove-older-than 2Y    old_NAS::/dest-dir
   

Why do your backups have to complete overnight? Fileserver performance? You might be able to constrain the bandwidth of your backup software to limit impact during the day. Or dedicate an interface on your NAS to talk to the tape drive to limit impact on other traffic.

Can you run full dumps on weekends and only do incrementals during the week? If the problem is changing tapes on the weekend when no one is around, a cheap tape library/autochanger costs a lot less than paying someone to change tapes.

Can you segment your data into multiple groups that are small enough to complete within your backup window?

We have about 50TB of data on a our NAS and it takes over a week to get a full dump of the entire thing using 2 tape drives (one volume takes nearly a week itself because it contains many tiny files). What we do is replicate our data to a second NAS. Our secondary NAS is on-site (but in a different datacenter from the primary), so we still spool data off to tape for off-site backup. We run backups from that secondary NAS so backups don't slow anyone down.

If you can colocate your secondary NAS far enough away, then it can be your backup, no tapes needed.

I'm just in doubt about the size of each shooting session, is it really 100gb / session? How many sessions does your company do each month?

Since you're mostly storing old sessions that won't be used frequently, etc, and probably don't need to recover that information that frequently, I would suggest you to use the services from some company to take care of that task for you.

Just for example, storing those 20TB using an online service like Amazon Glacier would cost a bit more than $200/month. If you need to retrieve those archives frequently, or even recover then in full, it would hit some time / cost constraint. If you just store those things "to be sure they are stored", perhaps using a third-part could make your life easier (and even cheaper than buying another NAS, tapes, etc)

full backups of this server are not completing overnight
Then try incremental backups? One full backup every xx days, incremental the rest.

Harddisks are inexpensive, faster than tapes and can be used for backup.

Also there are good alternatives for cloud backups now so its not required to keep adding more and faster tapes.
For example:

• Red-Gate Cloud Services
• Amazon backup storage
• Support for Amazon Glacier (cheap long-term storage)

I think the best solution for this is what we do with our payroll data, which should take a minimal effort for you to implement.

• Initially, it's kept with the rest of the server data that's backed up daily. Our retention period on those backups is 13 months.

• Once we no longer expect that the data will need to be modified, (two pay periods later, IIRC) the data is (via script) saved off to an archive volume that's excluded from the regular backups.

• The archive volume is backed up to tape yearly, and the tapes are sent off to Cintas for indefinite storage.

This allows us to have easy, online access to that unchanging data (so we don't have to call in a tape anytime an accountant wants to look at something), while maintaining indefinite off-site archives of data we may need to keep forever, and without crushing our backup system. Sounds like the same type of setup could work for you, though you might want to adjust the amount of data you keep online, depending on your needs to access this data in a timely fashion - 20TB of enterprise-grade storage is a lot more expensive than archiving it to two or three sets of LTO5 tapes that you store in off-site vaults.

Maybe you can build your own Backblaze Pod: 135Tb for 7384$
Click here for more information: Backblaze Pod building info

You can buy the needed pieces and build it by yourself.

Maybe you can build 3 of them, and keep 2 onsite, and 1 offsite. Then you can use one pod as the "online data", the second onsite pod as a backup of the first pod, and the third offsite pod as an emergency offsite backup.

With 135Tb of storage for each pod you can even think about keeping some history of the change...
135Tb / 20Tb = 19 full backup copy.
Alternatively you can keep 10 full backup plus a ridiculous amount of differential backup.

Naturally, if you want an offsite backup, you'll need some kind of big bandwidth... :-)

My coworker purchased a Synology 8-disk NAS. It runs a hybrid RAID. He purchased eight 3TB Seagate Barracuda from NewEgg a few weeks ago for $89 each. You could rsync mirror from the production NAS to this new NAS over GigaBit. Since you are only transferring the differences, the transfer will take a shorter time. Then you can use the backup NAS to perform full or incrementals. Cost to you would be under $2000 out the door for a backup NAS.