Next Previous Contents

4. Caching

4.1 HTTP caching vs local caching

With HTTP caching frox rewrites ftp retrieve requests in HTTP and sends them to a proxy server like squid which actually retrieves the file and does the caching. With local caching frox maintains its own cache of recently downloaded files on the hard disk.

The main advantage of the HTTP method is that the cache can be shared with any web browsers which use the same proxy directly for their ftp requests. Local caching is much faster (especially for small files) and lighter on network resources.

Caching needs to be enabled at compile time by running ./configure with --enable-http-cache or --enable-local-cache. You can have both compiled in if you want.

For HTTP caching your config file needs to say:

        CacheModule HTTP
        Cache HTTPProxy 192.168.2.1:3128 # proxy address
        # and optionally...
        Cache MinCacheSize 64000 # in bytes
        

Files smaller than MinCacheSize will not be cached. Latency on small files can be really bad with this method since the HTTP proxy has to make a separate connection to the ftp server - setting MinCacheSize fairly high will help with this.

For Local caching your config file should say:

        CacheModule Local
        Cache Dir /path/to/cache/dir
        # and optionally...
        Cache CacheSize 100 # in MB
        

The directory specified in Dir needs to be writable by the user frox has dropped its privileges to. If CacheSize is unspecified the cache will grow indefinitely.

4.2 StrictCaching

With strict caching off frox works out the URI for the file it is retrieving by sending a PWD to ask the server what the current working directory is. On a small minority of servers this may be broken, and a server may return the same answer to PWD for more than one directory. If additionally two such directories contain files with the same names, sizes, and times of modification then frox could cache the wrong file by mistake.

StrictCaching tries to get around this by keeping track of all directory changes and making up its own path instead of asking the server for it. This has the disadvantage that ``cd pub; get readme'' will not cache to the same file as ``get pub/readme'' or ``get /pub/readme'' or ``cd /pub; get readme''. As a result your cache hit rate will probably drop considerably, but you will have the extra security. Note that you can set this on a per server basis if there is a particular server about which you are worried.

Thanks to Henrik Nordstrom for pointing this problem out and suggesting a solution.

4.3 CacheOnFQDN

This option involves using FQDNs instead of IP addresses for the squid requests and allowing squid to guess whether to use binary or ascii file transfer rather than using the value supplied by the ftp client. For transparently proxied connections this requires a reverse DNS lookup to get the FQDN, and may cause problems with un-synced mirrored ftp hosts all running off the same FQDN, although most of the time there will be no problems. The advantage is that you should increase the hit rate, in particular with squid based caching for files which have been previously downloaded through squid with a web browser.

If all your clients are being non transparently proxied then this option is fine. Otherwise use with some care, although it may be worth the extra hit rate.

4.4 I get an ``Unable to initialise cache'' error.

Check that frox was compiled with support for the cache module you are attempting to use, and that you have given the correct Cache options for that module. If doing local caching then the user/group that frox is running as (ie. User/Group config options) must have read/write access to the cache directory. If doing http caching check that the http proxy name resolves.

4.5 I retrieved a file I had retrieved before and it doesn't seem to have been cached?

Files are not cached for non-anonymous ftp sessions, or if the ftp server doesn't support the MDTM and SIZE extensions. Additionally if the host's DNS name resolves to more than one IP address then the file will be cached, but frox has no way of knowing that the different IP addresses refer to the same host - unless your ftp client happens to pick the same IP both times you will not get the cached file back.

4.6 Virus scanning

You can configure frox to run a virus scanner on downloaded files between downloading them to the proxy and them being forwarded to the client. You need to ./configure with --enable-virus-scan, and set the config file variable VirusScanner to something like '"/usr/local/bin/yourviruscanner" "--options" "%s"'. You no longer need to be doing local caching for this to work - it will work with caching through squid or with no caching at all.

Note that the whole variable is enclosed in single quotes ('), while the individual arguments within are enclosed with double quotes ("). The "%s" will be replaced by the name of the file to scan. NB. unlike all the other paths in the config file, any pathnames or filenames in VirusScanner will not be stripped in the case that you are running in a chroot jail. If your virus scanner exits with a value other than 0 when a file is scanned and is virus free you will need to set VSOK to that value.

There are currently some problems with this:

If the VSProgressMsgs option is set to something other than 0 then every time that number of seconds passes during the delay a progress message will be sent to the client. On the plus side these may be displayed to users of command line ftp clients, and will probably prevent clients from giving up and timing out. On the down side the progress messages could confuse some ftp clients (I think they do with some Mozilla builds).


Next Previous Contents