[OpenAFS] clarification on client caches

[Dexter 'Kim' Kimball]

Here are some things I remember from messing with this same issue years
ago.

The number of Vfiles IIRC is directly related to cache-purge time.

If you're dealing almost exclusively with files smaller than 10K you'll
run out of Vfiles long before you run out of space -- unless you tell
afsd to create more Vfiles.  The algorithm for creating Vfiles seems to
assume that each Vfile will contain 10K, and afsd creates Vfiles
accordingly.

By limiting the number of Vfiles you're reducing the cache-purge task --
fewer files to handle.  afsd will automatically create 50000 files for a
500MB cache since it assumes 10K per Vfile.  A 500MB cache does not
present a difficult cache-purge task.  Reality check: my 900MB cache --
with default chunksize and # vfiles -- has 90000 vfiles.

If what you need is the ability to handle large files in the AFS cache,
you'll want 1) large chunksize and 2) smallest reasonable number of
Vfiles based on your expectation that most of them will be full.

Haven't looked at the source to see how AFS is handling the cache purge,
but casual empirical evidence is that the number of Vfiles is
significant in cache-purge tasks.  Makes sense since the Vfiles are only
marked "free" and are not deleted/emptied, just overwritten later.

I used afsmonitor -cm <client IP or name> -freq 2 and looked at the
"Cached blocks in use" column while simultaneously writing files from
AFS space to /dev/null.  My observations are that under some loads
purging starts at around 90+ % of "cached blocks in use"  (Vfiles in
use) and that with very intense loads it was possible to push very close
to 100% Vfile use.

This is also how I found out that the purging task corresponded to the
hangs -- lots of Vfiles to process, hang; not so many, no hang.

When messing with this I also added -verbose to the afsd options -- so
that I knew for sure when Vfile creation was finished.  IIRC there was a
correlation between finishing Vfile creation and allowing access to
/afs.

Let me know what you find.

Kim

=================================
Kim (Dexter) Kimball
CCRE, Inc.
kim<dot>kimball<at>jpl.nasa.gov
dhk<at>ccre.com



> -----Original Message-----
> From: openafs-info-admin@openafs.org
> [mailto:openafs-info-admin@openafs.org] On Behalf Of Wes Chow
> Sent: Monday, September 06, 2004 5:17 PM
> To: openafs-info@openafs.org
> Subject: Re: [OpenAFS] clarification on client caches
> 
> 
> 
> > >Things work fine.  If I then run "fs setcachesize
> 30000000", accesses
> > >to /afs then proceed to hang.  I can change the cache size
> to about
> > >25000000 and it still behaves properly.  The partition the
> cache is
> > >on is 60 gigs, so it's not running out of space.  Any
> ideas as to why
> > >it's hanging?
> > 
> > This is basically the same question you asked before,
> right? I still
> > have
> > no idea. Why not start with the cache size you want?
> 
> The symptoms are different.
> 
> Case 1: If I start with the cache size I want and "-chunksize
> 20" as the only afsd option, after afsd finishes creation of 
> the cache, afs_cachetrim then proceeds to use up most of the 
> CPU in what appears to be an infinite loop.  I've let it run 
> for about 4 hours and it never stopped.  Accesses to /afs 
> work, but the computer is understandably slow.
> 
> Case 2: What I'm doing now is limiting the number of V files
> by using the "-files 50000" option.  If I start it out at the 
> cache size I want (30 gigs), accesses to /afs hang without 
> afs_cachetrim running amok. However, if I lower the cache 
> size it seems to work.  Starting at a lower cache size and 
> resizing upwards doesn't seem to help.
> 
> Is it not typical to have cache sizes of this size?
> 
> Thanks for your very prompt responses!
> 
> Wes
> 
> -- 
> http://www.woahnelly.net/~wes/          OpenPGP key = 0xA5CA6644
> fingerprint = FDE5 21D8 9D8B 386F 128F  DF52 3F52 D582 A5CA
> 6644 _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org 
> https://lists.openafs.org/mailman/listinfo/openafs-info
>