Squid - [squid-users] Limiting the bandwidth of certain fyles

Author

[squid-users] Limiting the bandwidth of certain fyles

Xavier Baez

2004-04-29, 6:54 pm

Dear users

Please take a moment to read the lines I've added/changed to my
squid.conf file. I run squid on port 80 (http accelelator with proxy)
and apache at port 81
I redirect incoming request for port 80 to port 81 using Iptables.
HOwever, I want my site's visitors to download files (such as .exe,
..zip... ) with a limit (say 10 kb/sec during days, and no limit during
the nights)

However, I've read about pools, acls... discussed a lot with no luck. I
think it's a great use of Squid, I'd really like to be able to limit my
user's download rates. Anyway, here's the file:

http_port 80
icp_port 0
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 5 MB
cache_dir ufs /usr/local/squid/var/cache 500 16 256
emulate_httpd_log on
redirect_rewrites_host_header off
cache_replacement_policy GDSF
http_access allow all
cache_mgr news@socceraccess.com
cache_effective_user squid
cache_effective_group squid
httpd_accel_host virtual
httpd_accel_port 81
log_icp_queries off
cachemgr_passwd xhbc@2004 all
buffered_logs on
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
httpd_accel_single_host off

logfile_rotate 10
maximum_object_size 20000 KB
store_avg_object_size 50 KB

#We want to limit bandwidth during the day, and allow
#full bandwidth during the night
#Caution! with the acl below your downloads are likely to break
#at 23:59. Read the FAQ in this bandwidth if you want to avoid it.
acl socceraccess url_regex -i 192.168
acl badinternet url_regex -i ftp \.exe \.zip \.rar \.r01 \.r02 \.r03
\.r04 \.r05
acl day time 09:00-23:59

#We have two different delay_pools
delay_pools 2

#First delay pool
#We don't want to delay our local traffic.
#There are three pool classes; here we will deal only with the second.
#First delay class (1) of second type (2).
delay_class 1 2

#-1/-1 mean that there are no limits.
delay_parameters 1 -1/-1 -1/-1

#socceraccess: 192.168 we have set before
delay_access 1 allow socceraccess

#Second delay pool.
#we want to delay downloading files mentioned in badinternet.
#Second delay class (2) of second type (2).
delay_class 2 1

#The numbers here are values in bytes;
#we must remember that Squid doesn't consider start/stop bits
#5000/150000 are values for the whole network
#5000/120000 are values for the single IP
#after downloaded files exceed about 150000 bytes,
#(or even twice or three times as much)
#they will continue to download at about 5000 bytes/s

delay_parameters 2 1250/1250 1250/1250

#We have set day to 09:00-23:59 before.
delay_access 2 allow day
delay_access 2 deny !day
delay_access 2 allow badintern

Regards

--------------------
S. A. Tech Department

Andreas Pettersson

2004-04-29, 6:55 pm

You seem to have get yourself tangled in some delay hell ;-)
These lines should do what you want:

delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 1250/1250
delay_access 1 allow badinternet time
delay_access 1 deny all

You only need a delay pool to slow down traffic, not to give your users full
speed.
Btw, you _really_ should use a src acl to identify your local network
(socceraccess) instead of a regex..

/Andreas

----- Original Message -----
From: "Xavier Baez" <tech@socceraccess.net>

> acl socceraccess url_regex -i 192.168
> acl badinternet url_regex -i ftp \.exe \.zip \.rar \.r01 \.r02 \.r03
> \.r04 \.r05
> acl day time 09:00-23:59
> delay_pools 2
> delay_class 1 2
> delay_parameters 1 -1/-1 -1/-1
> delay_access 1 allow socceraccess
> delay_class 2 1
> delay_parameters 2 1250/1250 1250/1250
> delay_access 2 allow day
> delay_access 2 deny !day
> delay_access 2 allow badintern

Xavier Baez

2004-04-29, 6:55 pm

Dear Henrik

Could you please tell me an advice?

Should I recompile the linux kernel and patch it with Netfilter?

Or should I try to configure Squid so that it will limit the bandwidth
rate of the users downloading files from my site?

The main problem that I have, is that when we release a popular file
(say an EXE file), there are so many people downloading the file (mostly
with Internet Explorer and Windows XP) than it's broken randomly. And
users have to redownload the file again. Many times, they download an
incomplete file, and when they try to install it, they receive an
"Invalid data" warning.

Regards

--------------------
S. A. Tech Department

Henrik Nordstrom wrote:

>On Tue, 27 Apr 2004, Xavier Baez wrote:
>
>
>
>
>Ok. Please note that the delay pools feature available in Squid is
>designed for proxies and limiting the Internet bandwidth used, not for
>shaping clients. Because of this it only applies on cache misses.
>
>
>
>
>This looks very odd for being an url_regex.. what is it you want this acl
>to match?
>
>
>
>
>This pool is equivalent to not assigning a pool to the request, but is
>wasting a lot of memory only to keep track of that the clients are not
>limited. Why have you defined this pool?
>
>
>
>
>As per the url_regex comment above, I do not think this does what you
>want..
>
>
>
>
>There is no use of defining a higher class pool if the per-user limit is
>identical to the global limit. You would get the same effect using a class
>1 pool here as you have defined the global limit to 1250 and each single
>user is allowed to use up to 1250...
>
>
>
>
>The last lime will never be reached as the first two lines matches all
>requests.
>
>What is your goal with these lines?
>
>Regards
>Henrik
>
>
>
>
>

Henrik Nordstrom

2004-04-29, 6:55 pm

On Tue, 27 Apr 2004, Xavier Baez wrote:

> Should I recompile the linux kernel and patch it with Netfilter?

Won't do what you are looking for as netfilter only knows packets, not
HTTP or URLs.

But in theory it could be possible to combine the string match, connmark
and tc to shape certain types of http requests using netfilter/iptables +
tc.

> Or should I try to configure Squid so that it will limit the bandwidth
> rate of the users downloading files from my site?

More than just configuration is needed for this unless you can accept a
single global limit for all users together.

> The main problem that I have, is that when we release a popular file
> (say an EXE file), there are so many people downloading the file (mostly
> with Internet Explorer and Windows XP) than it's broken randomly. And
> users have to redownload the file again. Many times, they download an
> incomplete file, and when they try to install it, they receive an
> "Invalid data" warning.

This is some other error.

Regards
Henrik

Xavier Baez

2004-04-29, 6:55 pm

Dear Henrik

Thanks for the info. I've subscribed to the dev mailing list.

| Won't do what you are looking for as netfilter only knows packets, not
HTTP or URLs.

But in theory it could be possible to combine the string match, connmark
and tc to shape certain types of http requests using netfilter/iptables +
tc. |

So if use Netfilter the kernel with limit ALL the bandwidth (http requests, ftp, downloads....) to my end users? I really wouldn't like to do that (I only want to limit .zip and .exe files

Has anybody experienced problem when users are downloading files (ex: .zip files, specially with IE and WinXP) and the download is being cut in any point of the transfer? Squid didn't helped me in that problem neither.
When I post a popular programs, it's virutally impossible for my end users (specially the ones using a 56 modem) to download files, since DAP doesn't catches the URL. Therefore they download files with Internet Explorer's download "manager" and then I rec
eive a lot of complians.

Regards

--------------------
S. A. Tech Department

Henrik Nordstrom wrote:

>On Tue, 27 Apr 2004, Xavier Baez wrote:
>
>
>
>
>http://www.squid-cache.org/mailing-lists.html#squid-dev
>
>Regards
>Henrik
>
>
>
>
>