The following excerpt from this post led me to this question:
Cache contention
On a large site, if you are using MyISAM, contention occurs in the database tables when the cache is forced to clear after a node or a comment is added. With tens of thousands of filter text snippets needing to be deleted, the table will be locked for a long period, and any accesses to it will be queued pending the purge of the data in it. The same is true for the page cache as well.
This often causes a "site hang" for a minute or two. During that time new requests keep piling up, and if you do not have the MaxClients parameter in Apache setup correctly, the system can go into thrashing because of excessive swapping.
I think what you need to ask is "what is thrashing".
"excessive swapping" == "too much swapping", by itself is not so meaningful. I think you already know that.
Usually modern operating system has this "virtual memory" system, which means you can define the accessible memory much more than actual physical (e.g. RAM) memory that your hardware have. This usually achieved by having only portion of all your virtual memory in the primary storage (usually RAM), and move the rest to secondary storage (usually hard disks). The process of loading from secondary to primary storage is called "page in". The process of storing memory from primary to secondary storage is called "page out".
The "page out" operation followed by "page in" operation is called "swapping", usually to push the portion of memory that the virtual memory manager thinks is no longer in use, to secondary storage and pull the portion of memory that is about to be used from secondary storage.
In a way, the RAM is the cache for virtual memory stored in your secondary storage. As in all caching mechanism, it is a matter of hit/miss game. If your hit rate is good, performance will be good (which is usually the case for a good virtual memory manager). If your miss rate is high (hit rate low), too bad, performance is even worse than the speed of secondary storage.
Thrashing is the condition where the miss rate is very high, where this condition occurs a lot: something you just paged out recently need to be "paged in" again, makes it too costly just to read/write from memory. As you know, hard disk read/write operation is very expensive compared to RAM access.
If you are from computer science background, may be you know that quicksort can be slow O(n^2) when you put the input in certain sequence, that is the worst case sequence. Thrashing is something like you are having the worst case or near worst case. You get the performance penalty most of the time.
When your processes require more memory than you have RAM on your server, they "swap" some of their memory space onto disk, in your swap partition. Accessing this virtual memory is very much slower than accessing real RAM, so if it gets to the point where frequently used data has to be swapped in and out, it will cripple the performance of your system.