|
Home > Archive > Squid > April 2004 > [squid-users] cache cluster config management
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
[squid-users] cache cluster config management
|
|
| Will Lowe 2004-04-29, 6:55 pm |
| I'm looking at setting up a fairly large (> 20 machines) cluster of
Squid caches in an http-acceleration type setup. In some cases the
content we're caching will come from third parties (think of an
Akamai-type setup), so the ability to peer caches and avoid hits on
the origin server is key for us.
I'm trying to find a configuration scheme that allows us to add more
machines to the cluster without having to edit 20 different squid.conf
files to list a new cache_peer. Anybody got a solution for this
already?
If not, here are some questions:
1) Will squid be unhappy if the local machine has itself listed as a
peer? In this case I could put the SAME squid.conf on every machine
(via cfengine/rdist/whatever) and just list all 20 caches in it.
2) Is there any way to specify cache_peer siblings via an acl? This
is a private network, so I can predict the ip ranges where peers will
be.
3) Is there any way to adapt the multicast code to be used to locate
other peers without them being listed in the config file? Again, on a
private network I'm not too worried about rogue untrusted caches,
although it would be really cool if there was some MD5sum
authentication or something.
--
thanks,
Will
| |
| Henrik Nordstrom 2004-04-29, 6:55 pm |
| On Wed, 28 Apr 2004, Will Lowe wrote:
> If not, here are some questions:
>
> 1) Will squid be unhappy if the local machine has itself listed as a
> peer? In this case I could put the SAME squid.conf on every machine
> (via cfengine/rdist/whatever) and just list all 20 caches in it.
>
> 2) Is there any way to specify cache_peer siblings via an acl? This
> is a private network, so I can predict the ip ranges where peers will
> be.
>
> 3) Is there any way to adapt the multicast code to be used to locate
> other peers without them being listed in the config file? Again, on a
> private network I'm not too worried about rogue untrusted caches,
> although it would be really cool if there was some MD5sum
> authentication or something.
Everything is possible with some coding.
But I think the answer to your questions is no on all three issues if you
are not prepared to do some coding to have the features implemented.
Regards
Henrik
| |
| Will Lowe 2004-04-29, 6:55 pm |
|
> But I think the answer to your questions is no on all three issues if you
> are not prepared to do some coding to have the features implemented.
Well, I need to get (a first draft of this) implemented fairly
quickly, so I might have to live with lots of config files in the
short term. In the longer term I don't mind writing code at all. Of
the three ideas I suggested, is there one that people see as "best"?
r4--
thanks,
Will
| |
| Henrik Nordstrom 2004-04-29, 6:55 pm |
| On Wed, 28 Apr 2004, Will Lowe wrote:
> short term. In the longer term I don't mind writing code at all. Of
> the three ideas I suggested, is there one that people see as "best"?
Can you repeat them? (memory is short.. did not see them as exclusive of
each other)
Regards
Henrik
| |
| Will Lowe 2004-04-29, 6:55 pm |
| > Can you repeat them? (memory is short.. did not see them as exclusive of
> each other)
What I'm looking for is a way to have multiple squid peers (siblings)
autodiscover each other, or at least be able to use the same
squid.conf file. That way if I have a large number of them I don't
have to have a different config file for each one.
I had two different ideas for how to do this. Both could be done at
the same time.
1) Have squid ignore a "cache_peer" config line that lists the local
machine. This way I can put the same squid.conf file on all servers
and list all the servers as peers in it.
2) Have squid be able to "discover" other siblings on the network via
multicast. Modify the existing multicast support squid has to *use*
responses from caches that aren't listed as cache_peers instead of
discarding them. HTCP would be a better protocol here than ICP, since
it allows for authenticating the discovered peers via a preshared key.
My analysis:
#1 is probably easy. #2 is more elegant but probably requires a lot
more code, since so far squid doesn't do HTCP multicast and also
doesn't support authentication in HTCP.
--
thanks,
Will
| |
| Will Lowe 2004-04-29, 6:55 pm |
| > > 1) Have squid ignore a "cache_peer" config line that lists the local
> This is good. I think I have even seen a patch for it.
Great! Any idea where I'd find that patch?
> Multicast needs a bit of work in Squid, currently nobody is maintaining
> this aspect.
It also looks like HTCP needs some work. I might work on this in the
background, but don't hold your breath. 
--
thanks,
Will
| |
| Henrik Nordstrom 2004-04-29, 7:37 pm |
| On Thu, 29 Apr 2004, Will Lowe wrote:
>
>
> Great! Any idea where I'd find that patch?
It should be on either the squid-dev or squid-users list. Probably
squid-dev. I am not 100% sure, but I have a vague memory of seeing such
patch around somewhere..
>
> It also looks like HTCP needs some work. I might work on this in the
> background, but don't hold your breath.
I will wait patiently, and be very happy if you get some result ;-)
Regards
Henrik
|
|
|
|
|