I'm doing Web Applications that are always hosted in IIS (from 6 to 7), and I always have problems in deploying.
The applications work most of the time "out-of-the-box" but sometimes it's just terrible.
Last time, installing it on a IIS 7 where Share Point Server was installed I found out that, even SPS is running in a different WebSite, I needed to remove SPS compression. Other time was about application.config had with 32bit application running as it stopped the AppPool on the 2st request (this article helped) .
I wonder... because the web applications always work with no problems in Visual Studio, can't I aggregate Cassini Web Server and deploy my Web Application just like any other windows application?
Cassini or anything else...
What are your thoughts about it and what can I do to stop having so many problems with some installations?
You need to learn IIS, or else you need to leave IIS to the professionals. I recommend you don't look for a technological solution to a training problem.
Step 1 you need to stand up a test enviornment so you can practice a deployment before going to production.
Step 2 you need to become very familer with exactly what you are deploying.
Typically ASP.Net applications are simple to deploy since they usually just require an XCopy deployment.
I noticed you mentioned SharePoint. If you are deploying SharePoint you need to follow standard best practices which include using a SharePoint Solution (WSP). You should not have to touch web.config files since SharePoint has an API for doing these deployments. In general if you are having to do any manual steps with a SharePoint deployment then something is not normal.
Good luck and just take your time.