RE: Storing partial responses

On Wed, 13 Dec 2000, Robert Collins wrote:

> I think perhaps a happy compromise would be to split the discussion
> about meta level information into two levels
> - a request satisfaction level and
> - a cache store level
>
> Give the request satisfaction level the "Ranges" logic and so forth,
> give the cache store level the object assembly smarts, breaking objects
> into chunks and so on as per your recap.

Yes. Just because all non-FS actions were called "meta-level" code in
the previous discussion, does not mean that meta level is a single
monolithic structure. Splitting the meta level into "protocol logic
layer" and "object assembly" layer is a natural choice, and more
layers may be needed to support a variety of tasks that Squid
performs.
 
> Rethinking my blue sky comment above, where would the changes you
> are discussing regarding a news style spooling architecture sit?
> Would it be a new FS in the current API or would it be a clean
> break with the existing FS's?

Good question.

     protocol logic -- blobs assembly layer -- blob storage layer (FS)

The changes would be isolated to FS layer, provided the assembly layer
is added to Squid and used for all FSs. Some FSs will refuse to
operate on partial objects and the assembly layer will propagate the
errors back to the protocol layer.

A similar question would be "where does the replacement policy fit?".
I suspect that may have to be a part of the FS layer.

        [ Side note: a FS may not support storage of partial objects,
          but may still support retrieval of partial objects, just
          like in the current Range support ]

> I was writing with the assumption that you were proposing a clean
> break, necessitating the construction of the object splitting
> logic. If you are talking about a new FS type, then the logic for
> that should sit in that IMO, and therefore we come back to Alex's
> (0, 1 or 2) choice. And again I still favour (2a).

I think it makes sense to factor out object splitting / blob assembly
layer, as many FSs (and eventually most FSs) will need pretty much the
same logic. It will also make this layer a plug-and-play parameter,
just like a FS.

One could argue that FS may benefit from knowing that two blobs belong
to the same object. On the other hand, FS could also benefit from
knowing the relations between HTML container and embedded images, etc.
So we would come back to our "keep it simple/smart" discussion. I
suspect that eventually we might add a third parameter to the store
interface:
        store(key, blob, related_to_keys)
just to give some FSs additional hints for dataplacement while keeping
the interface simple and abstract.

$0.02,

Alex.
Received on Wed Dec 13 2000 - 07:38:07 MST