Optimising busy website with Varnish and lots of ram

What exactly is that 85,000/day visitors number? Unique visitors, total HTTP hits, something else?

Varnish should be able to handle thousands of hits per second, with little CPU and memory, as long as the requests can be served from cache. Especially when Slashdotted, since most people will be looking for the exact same content. It does require finetuning though. It's quite conservative by default, since it doesn't know much about the content that passes through. It makes its descisions based on the headers it sees, and a simple ruleset. For example, by default Varnish caches objects for 2 minutes, but only if no cookies are present in the request, and the object's TTL is >0, and... etc. Check the default VCL (specifically vcl_recv and vcl_fetch) to determine the default logic, and make sure you understand it.

So, a single Google Analytics cookie set on your domain causes all requests to be passed to the backend, even though GA cookies are not process by your backend server but by Google's javascript. A WordPress application sets all kinds of cookies, most of which are only applicable to dynamic content, which get returned by the browser on every single request. If your page contains 49 static assets and 1 dynamic page, that means none of those static assets will be cached because the requests contain a cookie that you don't care about. Only the cookie on the dynamic request should get through. A mistake like that essentially disables Varnish. Also, the various cache-control (and related) HTTP headers that your code returns are important. If your application claims that the object that Varnish retrieves from the backend is already expired, for example with an Expires header in the past, Varnish will not cache that object.

In other words, you'll need to adjust your application to emit the correct headers, so the clients (both Varnish and the browser) are allowed to cache the returned content. Anything you can't correct in your application, you can override in Varnish's VCL.

For example, here is my code to remove various client-side tracking cookies from reaching the server. This belongs in vcl_recv:

# Remove tracking cookies. The server doesn't need to see them.
if (req.http.Cookie) {
    # Remove has_js and Google Analytics __* cookies.
    set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js|sifrFetch)=[^;]*", "");
    # Remove a ";" prefix, if present.
    set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
    # Remove cookie header if empty
    if (req.http.Cookie == "") {
        remove req.http.Cookie;
    }
}

Similarly I remove incoming cookies to certain paths, to make those requests cacheable:

if  ( req.url ~ "^/cms/cms_files/(?:css|img|js)/" ||  #CMS1
      req.url ~ "^/site_files/(?:css|img|imgc|js|swf|uploads|xml/.+\.xml$)" ||  #CMS1
      req.url ~ "^/(?:images|stylesheets|javascripts|swf|site_files/js_libs|site/image|favicon\.ico$|robots\.txt$)" ||  #CMS2
      ( req.http.host ~ "(?:shop\.example\.com|www\.example\.nl)"  &&  #Magento
        req.url ~  "^/(?:404|js|media|skin|favicon\.ico$)" ) || #Magento
) {
    unset req.http.cookie;
}

I use a similar stanza in vcl_fetch, with unset beresp.http.cookie; instead, to prevent the backend from setting any cookies on paths I don't want.

You could add some debug headers that give you info about how varnish processed requests. View those with Firebug, and you'll understand a lot more about your app. Another good source of information is the Varnish Book. For example, see: https://www.varnish-software.com/static/book/VCL_Basics.html

Most of our dynamic content is cached for 60s, which is enough to fend off a stampede. If you require some individual content, but most of the content on your pages is quite static, look into Varnish's ESI (edge-side-includes) which allows you to specify different cache-TTL's for different parts of the page.

Now that you've reduced the backend requests to the bare minimum, optimize those requests. Profile your application, find and fix problems.

You're correct that: MaxClients x (maximum physical memory per Apache process) = (total memory Apache can use)

That's physical memory though, not the virtual memory that you mentioned. In top, the res column shows the physical memory used by each process. Each Apache process grows to the biggest scripts your site runs. Limit MaxClients to the amount your server can handle. It makes no sense to accept requests that you don't have the resources for. As soon as you start swapping, you've lost. Increase the amount of processes (Servers) that Apache preforks, since forking is a heavy operation that you want to do when it's already busy. The ServerLimit line is redundant. Either disable KeepAlive or set it to 1-2 seconds.

If you serve many static assets, consider switching from mod_php to PHP-FPM. This keeps your Apache processes light-weight.