A couple weeks ago, I started experimenting with varnish as a reverse proxy server. My setup has changed a lot since then, I like to think it improved. So here are my experiences so far.
Use -trunk (or version 2.0), not the 1.1.2 release
Initially I was using the 1.1.2 release, but I ran into a couple problems. The worst one was the white screen of death for users behind a proxy server, such as squid. There was a problem with the way HTTP/1.0 requests were handled, resulting in blank pages being sent to the client.
The solution was to upgrade to -trunk, it contains several bug fixes and interesting new features and is pretty stable (at least at the moment). I usually don't like using the development version, but at the moment it seems to be the most reliable, and was also recommended by several people.
Version 2.0 will be released soon, use that one when its available.
One thing to keep in mind though is that there is a problem with subroutines. This bug has been reported, so for now I'd recommend not to use them.
Don't duplicate the default VCL functions
In VCL you can have multiple functions with the same name, they will simply be concatenated. This means that you don't have to duplicate the entire default functions if you just want to add a single line. This can simplify your VCL code a lot, and you still benefit from improvements made in the default functions.
For an example, see my new VCL below. I'm using this in all functions except for vcl_recv.
Again, for future reference, a copy of my new VCL code can be found below. As you can see, it has some new "features" but is simpler than the previous one. Note that the backend syntax has changed a little in trunk.
# This is the vcl.conf file for andromeda.motd.be backend default { .host = "208.68.209.225"; .port = "80"; } # # handling of request that are received from clients. # decide whether or not to lookup data in the cache first. # if we have multiple backends, we could specify them here but for now there is just the default # sub vcl_recv { # If the client sent an X-Forwarded-For header, remove it. It cannot be trusted. unset req.http.X-Forwarded-For; # Note that we don't need to add the client ip to the X-Forwarded-For header, varnish will do that for us if (req.http.Accept-Encoding) { # Handle compression correctly. Varnish treats headers literally, not # semantically. So it is very well possible that there are cache misses # because the headers sent by different browsers aren't the same. # @see: http:// varnish.projects.linpro.no/wiki/FAQ/Compression if (req.http.Accept-Encoding ~ "gzip") { # if the browser supports it, we'll use gzip set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate") { # next, try deflate if it is supported set req.http.Accept-Encoding = "deflate"; } else { # unknown algorithm. Probably junk, remove it unset req.http.Accept-Encoding; } } # If we get a request for a page that has just been requested by another thread and # is still being fetched from the backend, allow serving a cached page so long as it hasn't # expired more than 30 seconds ago (prevents thread pileup on cache refreshes). # Note that this only works if obj.grace is also set: # "If no in-ttl object was found AND we have a graced object AND it is also # graced by req.grace AND it is being fetched: serve the graced object." set req.grace = 30s; # the default vcl_recv will only look up objects in the cache if there is no cookie # present in the request. Therefor, we remove the cookie header from all requests for # files which we know to be static. # # These are drupal-specific: # - everything in most of the standard drupal directories. # - txt and ico files (most important: robots.txt and favicon.ico) if (req.url ~ "^/(files|misc|sites|themes|modules)/" || req.url ~ "\.(txt|ico)$") { unset req.http.Cookie; } # No final action. Continues in the default vcl_recv which will only look up # items from the cache for GET and HEAD requests without cookies. } # # Called when entering pipe mode # sub vcl_pipe { # If we don't set the Connection: close header, any following # requests from the client will also be piped through and # left untouched by varnish. We don't want that. set req.http.connection = "close"; # Note: no "pipe" action here - we'll fall back to the default # pipe method so that when any changes are made there, we # still inherit them. } # # Called when the requested object has been retrieved from the # backend, or the request to the backend has failed # sub vcl_fetch { if (obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~ "(no-cache|no-store|private)") { # varnish by default ignores Pragma and Cache-Control headers. It # only looks at the "max-age=" value in the Cache-Control header to # determine the TTL. So we need this rule so that the cache respects # the wishes of the backend application. pass; } if (obj.ttl < 180s) { # force minimum ttl of 3 minutes for all cached objects. set obj.ttl = 180s; } # set the grace period for this object to a maximum of 30s. This is how # long after its expire time, it is still allowed to be served from cache if # directed to do so by vcl_recv. set obj.grace = 30s; if (req.http.Authorization && !obj.http.Cache-Control ~ "public") { # don't allow caching pages that are protected by basic authentication # unless when they explicitly set the cache-control to public. pass; } # Note: no final action - continue in the default vcl_fetch }





Thank you for your feedback :)
Could we cache pages on per-user basis using varnish?
Also I have'n found disabling cookies
That is probably possible. Have a look at the vcl_hash function. By default it uses hostname + url as a key to store objects in the cache. You can extend this method to include another user-specific key (some authentication or cookie header maybe) so that for each user, a different hash will be used.
Of course, it will become a bit more complex than this. You'll want to change the hash for the dynamic, user-specific pages but not for other static objects such as images/css/js. Otherwise you'll end up with a very low cache hit ratio.
I found out that the Varnish configuration above does not work well with Drupal 6. It seems that the "unset req.http.Cookie" for certain paths will cause user session to be terminated because the cookie is missing. I.e. the user gets one page-view after successful login, and any subsequent attempt to view content restricted to authenticated users only (e.g. admin pages) will result to "access denied".
I had no time to investigate further what was happening there, but removing the "unset req.http.Cookie" cleared the issue.
Can we ask apache not to compress the static contents every time and use the compressed version which is would have prepared for the first time access.
I've implemented something similar to this, but found that it doesn't work if the server side method throws an exception. The Content-Encoding header somehow gets dropped, so the client isn't able to read the response. That causes the failure callback on the client to never get invoked.
Any idea of a workaround for that?
http://www.punenet.com
I really like the way you write code. I used version 2.0 to do the same, and it works. Question, why can't I use version 1.12? Thanks.
Post new comment