OS - Ubuntu, PHP version - 5.3, MTA - Exim.
How you can limit number of email messages virtual host can send using standart PHP mail() function per interval of time (preffered would be hours, but days are ok).
PHP processes are running with seperate user privileges, different for each vhost.
Are they any other ways to prevent extensive spamming by shared hosting clients, but allow senseful mail sending from PHP ?
Well, you can probably hack up something so Exim only allows a limited number of mails per user & hour. There is apparently nothing directly in Exim, but it should be doable:
http://osdir.com/ml/debian.user.isp/2002-12/msg00094.html
That said, users could still directly send mail via an SMTP server of their choice. To prevent that, you'd have to block port 25 outgoing (firewall, iptables or similar). But arguably that is not your problem, but that of the STMP server's admin...
You should be doing this at the router. (Or firewall).
The VM server itself might be a router, depending on your setup.
Unfortunately you'll probably be counting TCP connections per minute, not RCPT TO's per minute. But this doesn't really matter considering the ratio between email rates that a spammer must commit vs someone legit. (far greater than the number of RCPT TO's they can reliably put in one email).
You could require them to use your SMTP server. If they're customers, then it's probably not worth the decreased level of service you're providing even though you're increasing your cost. But if you have a significant spam problem despite reasonable rate limiting, and you switch to extreme rate limiting unless they use your SMTP server (which can do much more in-depth analysis than a firewall), I expect customers to be understanding. If customers are being NATted to the same IP address, asking them to share a single SMTP server is perfectly reasonable (they share an inbound address anyway, they share anti-spam protection risk).