I am lucky that hard disk space is not an issue however there are applications that come with the Ubuntu distro that I will never use and I have preferences for other packa ges. For example I use K3B not Brasero for burning DVDs. Are there any LTS support or stability issues if I remove unwanted/unused packages from my system?
For example Brasero is part of the meta package Ubuntu-desktop, http://packages.ubuntu.com/oneiric/ubuntu-desktop, Removing packages part of this meta-package will uninstall the ubuntu-desktop meta-package, while this dosn't really do much of anything to your desktop.
Customization is what linux is all about, use the program best for you, and remove the ones you don't want.
As long as the packages are from the Ubuntu repositories you should be good to go. You can always encounter a bug of course, but nothing wrong with your general plan.
If you are going to do customization, sometimes it is easier to start with a minimal install and build up.
https://help.ubuntu.com/community/Installation/MinimalCD
There is a learning curve with building up from a minimal install, but once you get the hang of it, many people prefer minimal installs.