Pretty sure it doesn't matter but I've had about a dozen OpenVz vps' and never seen this until I bought a Xen server for the first time.
Special characters like « get all funky and look like this: �
What can be done to so the characters display properly? Using Debian
How are you connecting to the VPS, via PuTTY for example? If yes, set the encoding to UTF-8 in the settings ("Translations").
If you're connecting from a client via ssh command directly, please show us the output of
echo $LANG(both on the client and the server).