Our ISA server administrator is saying this:
How the ISA cache works is as follows: when the ISA server receives a request to access a website, the first step it takes is check through the cache to see if it can load it locally. As the cache is currently 50Gb in size, the server has to check through this entire cache file before a) directly returning a result to the user or b) passing the request out to the internet to retrieve the web page (and in turn place it in cache). Having to parse this much data is likely to be slowing down the internet access.
Having a smaller cache would be beneficial in that the ISA server can handle requests faster by not having to constantly check the local disk for lengthy (in computer terms) periods of time.
He is advising we reduce our cache file from 50GB to 10GB.
While I agree with his sentiment, surely, this could be taken to extremes: if I don't have any cache file, surely I can skip those horrible slow disks, and just go to the nice fast Internet every time I get a request.
The question therefore: what is the optimal size for an ISA server disk cache file? Does ISA server have a convenient index or hash-table of cached requests such that it doesn't matter how large the cache file is, it will be able to find the offset of any cached content you want, inside that file in O(1) time?
It doesn't really sound plausible as the cache is as you suspect properly indexed. It is re-indexed upon service startup where ISA verifies the content of the cache. The sentiment is vague and generalized and doesn't take into account object indexing or part of the cache data also living in memory for instance - but nothing can be denied without actual testing so who knows? Reducing it to 10GB most likely wouldn't hurt either though? ^^
Performance of the cache is dependent on a lot of other factors though. ISA Server 2004 will by default use 10% of the memory for caching recently used cache objects while older objects will obviously only be on disk. Increasing memory and ensuring at least 1GB for the web cache would be advisable - and yes the disk system for the cache will have a strong impact on the general cache performance - but the size really shouldn't matter.
There is some information about ISA 2000 having a database file limit of 10GB and that it creates multiple files when that limit is exceeded - but nothing about that having a negative performance impact. There are several posts recommending no more than 10GB cache size on this version because "more wouldn't really help" but not because it would slow down.
Here're some general tips on optimizing cache performance on 2004.