Re: [squid-users] Cache large file in chunks?

On 30/04/11 06:55, Henry Yuan wrote:
> Hi Amos,
>
> Thanks for the reply. I have a new question in the content.
>
> On Fri, Apr 29, 2011 at 1:01 PM, Amos Jeffries<squid3_at_treenet.co.nz> wrote:
>> On 30/04/11 05:08, Henry Yuan wrote:
>>>
>>> Hi,
>>>
>>> I'm wondering whether the current squid implementation cache large
>>> files in chunks?
>>
>> No. It caches them as whole files.
>>
>>>
>>> Let's say the Squid is downloading a large file A for client 1, when
>>> it just finished transferring say 10% of the file, another client,
>>> Client 2 asked for the same file A. Will Squid start serve Client 2
>>> immediately with the 10% of file A it has or Squid has to wait until
>>> all the parts of file A is downloaded in order to get a cache hit.
>>
>> Depends on your version of Squid.
>> Squid-2.6 or 2.7 will start sending client 2 the same file client 1
>> fetched.
>> Squid 3.x will fetch a new copy in parallel. As soon as one of the two has
>> finished it will start serving client 3,4,5 etc as HITs.
>>
>
> 1. Are you saying Squid 3.x will fetch AT MOST 2 copies of the same
> file, or it will fetch as many as the pending requests before the
> whole file is downloaded (so if 100 clients ask for the same large
> file, 100 fetches will be conducted if the whole file is not
> downloaded yet)?

Sadly no. It will fetch as many in parallel as clients ask for until one
finishes and its cached file becomes available to other future client
requests. At which point those future requests use it as a HIT.

>
> 2. What's the motivation of fetching the same content twice in
> parallel for Squid 3.x? It seems to me that serving the file right
> after parts of it being cached is more efficient.

Nobody has had time to port the collapsed forwarding feature from 2.6 to
the 3.x series yet. That is the only reason.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.12
   Beta testers wanted for 3.2.0.7 and 3.1.12.1