Considering that I run a low memory VPS, should I configure Apache to use mod_fcgi, PHP APC or both? As I understand it, the APC will not be shared between processes so I wonder what would produce the best result on busy sites: mod_fcgi, APC or a combination?
You should use both fcgi and APC -- they don't do the same thing at all. But 512MB of RAM is not enough for a busy site.