I have a server that provides software downloads. Unfortunately this server is on a rather slow internet connection because it needs to access views on some databases that I don't want to expose to the internet. It is more than fast enough to serve a couple of pages, but using this connection to serve large files would just be testing people's patience.

Hosting static files on another server is easy enough, but I needed some access control to make sure that people wouldn't be able to download files to which they don't have access. The slow server performs the authorization and needed to pass this information along to the file server.

The solution turned out to be very simple. I set up a lighttpd webserver to serve the files, and used mod secdownload to restrict access to the files so that only people who passed the authorization check on the other server could download files.
What the secdownload module does is create URLs that are valid for a limited time. These URLs contain a token which is the MD5sum of the current time, filename and a secret. Only applications that known the secret can generate the correct URL to access a file. This is what the slow server does, it checks whether or not some is allowed to download a file and if so, generates a link to the other server containing this token.

The advantage is that no cookies are needed. This means that people can copy the URLs to other browsers (or wget as I often do) and they are still granted access.

Configuring this takes only a few lines in the lighttpd configuration:

  secdownload.secret          = "EveryoneHasSecrets"
  secdownload.document-root   = server.document-root + "/../files"
  secdownload.uri-prefix      = "/secure-download/"
  secdownload.timeout         = 300

And a similar amount of lines in the other application to generate valid download links. The website has examples for PHP and Rails which are pretty easy to adapt. I extended this a little to first check if the file already exists on the remote server because we only rsync newly added files to the remote server at night when there is enough available bandwidth.

Unfortunately my server is running out of diskspace with all these files (data really does expand to fill the space available for storage). I'll probably have to move this to another server in the future. There aren't that many companies providing lighttpd hosting yet, but I think I know one that could help me out there.