Questions[s3fs](server)

How do I allow multiple SFTP Users with S3FS and OpenSSH?

Everything works, except SFTP Users don't have permission to write to their Chrooted Home Directory: remote open("/some_file"): Permission denied

Setup

I've got an Amazon EC2 instance running Amazon Linux. I've installed S3FS and mounted an S3 bucket. I've also configured OpenSSH to allow SFTP Users to access a Chrooted Home directory inside the mounted S3 Bucket /s3_mounted_folder/user_folder/. I've successfully used the SFTP connection on a non S3 mounted directory. I've successfully used the S3 bucket to create and download files from S3 as root on the EC2 instance via SSH. My SFTP users can successfully download files from their /s3_mounted_folder/user_folder/ directory. The problem is that the SFTP users cannot put files into the S3 mounted folder.

The Problem ... I think

I am only able to configure all folders (/s3_mounted_folder/ and /s3_mounted_folder/user_folder/)with the same user:group and same permissions, thus, I can't give the user access to write to his/her home directory (/s3_mounted_folder/user_folder/). If I mount the bucket with the user or group and give either write permissions, then OpenSSH SFTP won't let users connect because it believes the user permissions are misconfigured (example: drwxr-xr-x 1 root root vs. drwxrwxr-x 1 root usergroup).

S3FS Commands

Here are the two different commands to launch S3FS in these two modes (where user 501 and group 501 are the SFTP user and group):

root user permissions (drwxr-xr-x 1 root root): sudo s3fs nwd-sftp /sftp/ -o iam_role=sftp-server -o allow_other -o umask=022

sftp user permissions (drwxrwxr-x 1 root usergroup): sudo s3fs nwd-sftp /sftp/ -o iam_role=sftp-server -o allow_other -o umask=002 -o gid=501

In that second scenario, the user would theoretically be able to put files into their home directory via SFTP, but SFTP won't let them connect because their Chrooted home directory has write permissions for a group that isn't root.

I've been trying to use s3fs on an EC2 server but it's really slow. I spent the last 5 hours uploading 100MB of small files.

Is there anything I can do to improve the performance or is it something related to S3fs? If not, what's an alternative I could use?

Building my web service onto EC2 right now and have a single instance behind a load balancer. I will of course cater for multiple instances.

My initial idea was to run all the instances a dumb slaves, and use S3 as local storage. For this, I've begun using S3FS but it's not really ready, from what I've seen, for production use in a web serving environment. Writing of logs seems to appear very late, if not never. Numerous issues with odd caching, even with no cache flags etc. Just generally nightmare to develop on.

But, the alternatives look few. One is obviously EBS volumes, which can be attached to a single instance. Some solutions to sharing this:-

• SMB sharing to other instances. Having one master and the rest slaves - obv needs redundancy built in here with multiple EBS volumes perhaps?
• Rsync sharing to other boxes. This seems painful, considering its not persistent and will updated periodically. Potentially ok, if there are forcing scripts to update when major changes have happened.

Question is... what DO people DO? It seems an entirely common use case, but the variety of answers found in forums and even here on SF, seems to suggest there's not a concise answer... help wanted!

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data.

I first tried s3fs, but it downlooads the entire file first, which takes a really long time.

Are there any s3 backed file systems that make use of the S3 API bytes parameter, so that internal read (and seek) commands only read the desired part of the file?

As a practical example if I run:

tail -c 1024 huge_file_on_s3

only the last 1kb should be requested (via the bytes parameter), meaning I should get the result really fast.

(I am not concerned with writing back to S3; only reading from it)

I'm using s3fs 1.33 on Ubuntu 9.10 x86.

I mount it manually with this command:

sudo /usr/bin/s3fs mybucket /mnt/s3/mybucket -ouse_cache=/tmp

How can I force the mount on boot?

I guess that I have to write something to /etc/fstab. But what? And how do I test it without rebooting the server?