I got data on a private FTP server I want to mirror it, but "on demand". My frontend is an Apache/PHP server...
Here's the flow :
- Clients asks the apache server for a file
- if file is not on hdd, download it from the ftp server. (stream it to client, while saving it to hdd)
- send file to client (from apache)
I don't know exactly how to proceed...
Is there a way to stream a file while downloading it ?
Maybe we could do something with php/curl...
Try LftpFS: http://lftpfs.sourceforge.net/
It's a FUSE filesystem that relies on lftp as a backend and does exactly what you need. It requires a basic lftp script, but it's easy to write one. The official site provides sufficient info.
You can also try and turn your Apache into a caching proxy, that would work too (and probably faster than FUSE).