Re: Multiple storeio types?

Henrik Nordstrom wrote:

> On Friday 09 November 2001 08.58, Joe Cooper wrote:
>
>
>>>Should work.. the aufs store should also work. The amount of aufs threads
>>>can be tuned to fit your needs.
>>>
>>Per cache_dir?
>>
>
> There is only a total amount in the way async-io is designed.
>
> By default this total amount is based on the number of cache directories you
> have, but you can tune this to whatever you want at compile time.
>
> I also have a patch somewhere allowing tuning via the cache_dir lines..
>
> But using async-io or diskd for a RAM disk is pure waste of resources. "ufs"
> is the correct model there.

I agree. I just don't really know how to make it play well with the
least-load scheduler (as it seems to be more effective than
round-robin--but maybe that's because round-robin didn't use the
max-size, and so filled up the ramdisk with big objects making it less
effective).

>>Yep, I have my reasons. ;-) Because Squid wants to write everything to
>>disk, and with only two IDE disks to write to, I can't afford those
>>write ops. I'm trying to push 150 reqs/sec+ out of two 7200 RPM IDE
>>disks...which just does not work when Squid is writing every object (we
>>max at about 110--no matter what the bdflush/elvtune parameters are).
>>So I'm 'filtering' the smaller objects into a RAM disk...because they're
>>so small and the block size is 1024, we can cram a couple hundred
>>thousand objects into a 512MB RAM disk...about half the number of
>>objects that go into the real disks at 7.5GB each. Works surprisingly
>>well, and with 2GB of RAM costing under $200, it's a performance bargain
>>(two more IDE disks would cost the same, but gain less throughput).
>>
>
> So what you actually want is to tune the policy to stuff small objects onto
> the ramdisk, and larger objects onto the drives. No need to adjust priorities
> for this, only size selections.

Yes. But the selection algorithms behave oddly in most of the
configurations I've tried. The only one so far that is proven working
is aufs for all cache_dirs (including RAM) and least-load scheduler.

I'll see if I have time to do more work on this before shipping out the
box (I have to ship it first thing tomorrow morning for it to arrive at
the event on time--and I have a ton of other preparation to do before I
can leave myself). I've got to get one complete PM-4 run in before
then, or Alex won't let me play when I get there.

>>Maybe if I feel smart one day, I'll try to
>>integrate that functionality on the assumption that the Linux kernel
>>will be fixed eventually. I'm guessing Squid is probably more efficient
>>without the fake disk overhead of the RAM disk. ;-)
>>
>
> Without first fixing the "hot object cache" to actually act like a cache
> using a ram disk is most likely more efficient.

The ram disk also can't bring things in from the disk dirs to make them
'hot' again, which I think is what's really missing from the current
memory model...so it's doubtful their is any benefit for this reason.
They both use a replacement policy to decide what gets dropped--and if
the pool is the same size for each, then the effect should be the same.
  (But if the OS freaks out on a 500+MB process, then obviously the ram
disk wins.) But I think both have the same flaws as a memory caching
model. Correct me if I'm wrong?

-- 
Joe Cooper <joe@swelltech.com>
http://www.swelltech.com
Web Caching Appliances and Support