We have production server(windows server 2008) with 8GB Ram and around 15 websites are hosted on the server. What is the optimum memory that needs to be reserved for OS, so that remaining memory can be divided between the websites. We don't have any other processes or applications running on the server other than websites.
You can't limit the amount of memory the OS uses. Assuming you have a separate Application pool for each site, you should set limits for each one of those instead.