Home / server / Questions / 158781
Accepted
Jacko
Asked: 2010-07-09 16:53:36 +0800 CST

Need simple, fast way to store large gobs of xml to a server

  • 772

Our client software is currently sending large amounts of xml to our SQL Server db, stored as NTEXT, and the db is getting huge and performance is suffering. This xml doesn't need to be indexed, just needs to be persisted somehow to disk. Is there an easy, robust way of storing this stuff to disk on a remote server? I looked into CouchDB, but I'm sure there is an easier way of doing this. I prefer unix solution, but anything goes.

Thanks!

database couchdb

2 Answers

  1. Best Answer

    I would use rsync and store it as flat files; if you don't need indexing there's no need for a database. That way when you need to add more data to what's there, you don't have to send the entire file over again.

    It should be as simple as rsync -avz source/dir/ remote:/path/to/dest/dir, if the remote server has SSH running; the manpage can tell you more (be careful with the closing slash thing, I can help you more with that if I'm given more specific details if you like).

    • 1

  2. Since there is no need to use SQL features (like XML type in Microsoft SQL Server), I would suggest storing the files directly to the hard disk.

    Depending on the context, you can either create a shared directory with write-only access (it was possible in Windows XP, so it is probably possible with Unix solution).

    Or if the client software is used outside local network, you can receive XML data through Web services and store it to the hard disk. Or if performance is a major concern, you can use HTTP protocol directly (POST/PUT), instead of using Web services.

    • 0