Apache Server configuration support - mod_header: how to prevent caching

Author

mod_header: how to prevent caching

patpro ~ Patrick Proniewski

2007-01-25, 7:21 am

Hello,

I would like to prevent browsers/proxies to cache one item on my server.

I've tried something like this:

----------------first try------------------
<Files my-file>
ExpiresActive On
ExpiresDefault now
Header unset Cache-Control
Header append Cache-Control "no-cache, must-revalidate"
Header add Set-Cookie "foo=bar; path=/"
</Files>

---------------second try-----------------
<Files my-file>
Header unset Cache-Control
Header append Cache-Control "no-cache, must-revalidate"
Header add Set-Cookie "foo=bar; path=/"
Header set Expires "Thu, 01 Dec 1994 16:00:00 GMT"
</Files>

but the file is still cached by browsers. Also, I would be happy to
remove mod_expires from the prerequisites. So, it would be great if I
could do something that actually works without using mod_expires.

I've read that when the Expires header is the same as the Date header,
then the file is not cached. Unfortunately, I don't know how to set the
Expires header (and the Last-modified headers) to the same value as the
Date header.

The Expires header set to a date in the past fails to force the reload
of the file.

Any idea ?

patpro

--
http://www.patpro.net/

patpro ~ Patrick Proniewski

2007-01-25, 7:21 am

In article <patpro-FF8657.11065825012007@localhost>,
patpro ~ Patrick Proniewski <patpro@boleskine.patpro.net> wrote:

> ----------------first try------------------
> <Files my-file>
> ExpiresActive On
> ExpiresDefault now
> Header unset Cache-Control
> Header append Cache-Control "no-cache, must-revalidate"
> Header add Set-Cookie "foo=bar; path=/"
> </Files>
>
> ---------------second try-----------------
> <Files my-file>
> Header unset Cache-Control
> Header append Cache-Control "no-cache, must-revalidate"
> Header add Set-Cookie "foo=bar; path=/"
> Header set Expires "Thu, 01 Dec 1994 16:00:00 GMT"
> </Files>
>
>
> but the file is still cached by browsers. Also, I would be happy to
> remove mod_expires from the prerequisites. So, it would be great if I
> could do something that actually works without using mod_expires.

ok, it looks like this works:

<Files> my-file
ExpiresActive On
ExpiresDefault now
Header unset Cache-Control
Header append Cache-Control "no-cache, no-store"
Header add Set-Cookie "foo=bar; path=/"
</Files>

next assignment, get rid of Expires* directives.

patpro

--
http://www.patpro.net/

shimmyshack

2007-01-26, 1:41 am

On 25 Jan, 10:37, patpro ~ Patrick Proniewski
<pat...@boleskine.patpro.net> wrote:
> In article <patpro-FF8657.11065825012007@localhost>,
> patpro ~ Patrick Proniewski <pat...@boleskine.patpro.net> wrote:
>
>
>
>
>
>
> <Files> my-file
> ExpiresActive On
> ExpiresDefault now
> Header unset Cache-Control
> Header append Cache-Control "no-cache, no-store"
> Header add Set-Cookie "foo=bar; path=/"
> </Files>
>
> next assignment, get rid of Expires* directives.
>
> patpro
>
> --http://www.patpro.net/

user agents are notorious for not following this type of caching
demand, just make sure that you don't think that you have solved the
issue for evey user agent, or rely on it for anything sensitive to
error.

patpro ~ patrick proniewski

2007-01-26, 1:20 pm

In article <1169792706.966895.317100@a75g2000cwd.googlegroups.com>,
"shimmyshack" <matt.farey@gmail.com> wrote:

here it goes :

<Files my-file>
Header unset Cache-Control
Header append Cache-Control "no-cache, no-store"
Header add Set-Cookie "foo=bar; path=/"
Header set Expires "Thu, 01 Dec 1994 16:00:00 GMT"
</Files>

perfect result for IE6, Firefox 2, Safari (latest).
[vbcol=seagreen]
> user agents are notorious for not following this type of caching
> demand, just make sure that you don't think that you have solved the
> issue for evey user agent, or rely on it for anything sensitive to
> error.

I know, but it's not for critical use. And well, if user agent do not
abide by the standard, it's none of my concern. First user agents that
do not follow standards are spambot. I don't see any reason for me to
help them

regards,
patpro

--
http://www.patpro.net/

Rik

2007-01-26, 1:20 pm

On Fri, 26 Jan 2007 15:01:30 +0100, patpro ~ patrick proniewski =

<patpro@boleskine.patpro.net> wrote:
>
> I know, but it's not for critical use. And well, if user agent do not
> abide by the standard, it's none of my concern. First user agents that=

> do not follow standards are spambot. I don't see any reason for me to
> help them

How I usually prevent caching is to give the url a random or time-relate=
d =

GET variable (like example.com/file?foo=3D<timestamp_of_request> ). That =
way, =

browsers can be fooled to think it's en entirely different file all =

together. This would require a server-side language for the creating of =
=

the GET variable though.
-- =

Rik Wasmus

patpro ~ patrick proniewski

2007-01-26, 1:20 pm

In article <op.tmrlk8akqnv3q9@misant.kabel.utwente.nl>,
Rik <luiheidsgoeroe@hotmail.com> wrote:

> How I usually prevent caching is to give the url a random or time-related
> GET variable (like example.com/file?foo=<timestamp of request> ). That way,
> browsers can be fooled to think it's en entirely different file all
> together. This would require a server-side language for the creating of
> the GET variable though.

I've used this trick a lot in the past, but for some reason I don't want
to use it here. By the way, my no-cache+cookie trick will not prevent
access to information, so no real problem for the user, even with a
"bad" user agent.

patpro

--
http://www.patpro.net/

shimmyshack

2007-01-26, 1:20 pm

On Jan 26, 2:26 pm, patpro ~ patrick proniewski
<pat...@boleskine.patpro.net> wrote:
> In article <op.tmrlk8akqnv...@misant.kabel.utwente.nl>,
>
> Rik <luiheidsgoe...@hotmail.com> wrote:
> to use it here. By the way, my no-cache+cookie trick will not prevent
> access to information, so no real problem for the user, even with a
> "bad" user agent.
>
> patpro
>
> --http://www.patpro.net/

I was really thinking of the term "prevent", there are of course ways
to prevent, but using headers and cookies isnt really fit for purpose.

As for your initial thoughts
I've read that when the Expires header is the same as the Date header,
then the file is not cached. Unfortunately, I don't know how to set the
Expires header (and the Last-modified headers) to the same value as the
Date header.
Im not sure if this is actually the case, but you are free to set any
header, provided you are using a later version of apache. Obtaining the
current teimstamp is easy using the following:

SetEnvIf User-Agent good$ caching_on
Header add Caching "%D %t" env=caching_on

I use an if with "good" and "bad" user agents, just to show it can be
done, and the header Caching can be changed for another name, the %D
and %t are fairly self evident.

Remember though that not all bad UAs are spambots, most are just older
versions, or these days the plethora of mobile devices, if you dont
have to worry about mobile UAs for this file, then youre in a better
position, but to truely prevent, I recommend a page which has all of
the above, send loads of no caching headers and etag and timestamps, a
cookie which tried to refresh if stale content, plus tail the URI with
a timestamp - which you can do using rewrites
http://httpd.apache.org/docs/2.0/mo...te.html#mapfunc
system stuff: specials:
TIME_YEAR
TIME_MON
TIME_DAY
TIME_HOUR
TIME_MIN
TIME_SEC
TIME_WDAY
TIME

hope it helps

patpro ~ patrick proniewski

2007-01-26, 1:20 pm

In article <1169828207.458744.166480@k78g2000cwa.googlegroups.com>,
"shimmyshack" <matt.farey@gmail.com> wrote:

>
> I was really thinking of the term "prevent", there are of course ways
> to prevent, but using headers and cookies isnt really fit for purpose.

I don't see your point, may be there is a slight misunderstanding here

> Im not sure if this is actually the case, but you are free to set any
> header, provided you are using a later version of apache. Obtaining the
> current teimstamp is easy using the following:
>
> SetEnvIf User-Agent good$ caching_on
> Header add Caching "%D %t" env=caching_on

according to mod_header documentation, "%D %t" yields to something like
this:

Header add MyHeader "%D %t"
--> MyHeader: D=3775428 t=991424704447256

<http://httpd.apache.org/docs/2.0/mod/mod_headers.html>

but Date (or date related headers) must be sent in RFC 1123 format:

Date: Tue, 15 Nov 1994 08:12:31 GMT

<http://www.w3.org/Protocols/rfc2616...4.html#sec14.18>

So i don't see any way to enforce custom dates in
Last-Modified/Expires/... headers.

But that's ok. The solution I've posted earlier is fine. It supports all
mainstream browsers, and even with a "broken" browser, the user has
about 1 chance on a million to notice something.

thanks,

patpro

--
http://www.patpro.net/

shimmyshack

2007-01-27, 1:28 am

On 26 Jan, 18:50, patpro ~ patrick proniewski
<pat...@boleskine.patpro.net> wrote:
> In article <1169828207.458744.166...@k78g2000cwa.googlegroups.com>,
>
> "shimmyshack" <matt.fa...@gmail.com> wrote:
>
>
>
> this:
>
> Header add MyHeader "%D %t"
> --> MyHeader: D=3775428 t=991424704447256
>
> <http://httpd.apache.org/docs/2.0/mod/mod_headers.html>
>
> but Date (or date related headers) must be sent in RFC 1123 format:
>
> Date: Tue, 15 Nov 1994 08:12:31 GMT
>
> <http://www.w3.org/Protocols/rfc2616...4.html#sec14.18>
>
> So i don't see any way to enforce custom dates in
> Last-Modified/Expires/... headers.
>
> But that's ok. The solution I've posted earlier is fine. It supports all
> mainstream browsers, and even with a "broken" browser, the user has
> about 1 chance on a million to notice something.
>
> thanks,
>
> patpro
>
> --http://www.patpro.net/

Hiya,
well when I said "prevent" I was quoting your intial post where you
said

"I would like to prevent browsers/proxies to cache one item on my
server"

You just can't do that, the behaviour of browsers is just not under
your control, but I think what you really want is as you later said, a
"mainstream" solution - and by mainstream I think you mean the major
modern browsers of desktops. Added to this, we are not necessarily
talking sbout only user agents, we have to include caching proxies
etc.. which may well not honour your cookie rules.

Again you can add any header, you may overright or add any header with
any variable you can get from apache, you focused on my (rather
confusing) example of "%D %t" which does indeed yield exactly what you
were looking for, but I did go on to show that there were more (system)
variables that Apache can use besides %D and %t.
You then quoted RFC2616, but let us see #14.21

HTTP/1.1 clients and caches MUST treat other invalid date formats,
especially including the value "0", as in the past (i.e., "already
expired").

To mark a response as "already expired," an origin server sends an
Expires date that is equal to the Date header value. (See the rules
for expiration calculations in section 13.2.4.)

A value of hamster would also lead to the Expires header to expire to
have the value of the date in the required RFC2616 # 3.3.1 format at
serve time.

Furthermore your example which works perfectly for you in your
mainstream browsers works only through HTTP/1.1 caching servers, the
same RFC2616 (#14.9) states that:
Note that HTTP/1.0 caches might not implement Cache-Control and
might only implement Pragma: no-cache (see section 14.32).
So I would not assume that because it works for you in your latest
browsers that your clients will see the latest version of your content,
although as you say, they will see something.

If I were you I would either use a script to finely control the headers
serving a plethora of them (ala rfc2616 which is my view has
contributed to the confusion and lack of precision in this area), or -
which I know you have ruled out at the start - use mod_expires, with
its tight fine grained control. Mod_cache would also work without fault
in my opinion - if you found that a suitable module to use, you will be
able to make conditional requests based on last modifed and a whole
host of other options. You could set your own Etag header, but why
bother when it has all been done and is no longer considered
experimental, as is in my opinion your approach.

patpro ~ patrick proniewski

2007-01-27, 7:26 am

In article <1169865688.528234.273180@l53g2000cwa.googlegroups.com>,
"shimmyshack" <matt.farey@gmail.com> wrote:

> Hiya,
> well when I said "prevent" I was quoting your intial post where you
> said
>
> "I would like to prevent browsers/proxies to cache one item on my
> server"

OK ! your sentence was just below this quote "my no-cache+cookie trick
will not prevent access to information, so no real problem for the
user", so I thought you were referring to this "prevent".

> You just can't do that, the behaviour of browsers is just not under
> your control, but I think what you really want is as you later said, a
> "mainstream" solution - and by mainstream I think you mean the major
> modern browsers of desktops. Added to this, we are not necessarily
> talking sbout only user agents, we have to include caching proxies
> etc.. which may well not honour your cookie rules.

Sure, I'll have to test this.

> A value of hamster would also lead to the Expires header to expire to
> have the value of the date in the required RFC2616 # 3.3.1 format at
> serve time.

Yep, sure, but a valid date is better, isn't it ?

> Furthermore your example which works perfectly for you in your
> mainstream browsers works only through HTTP/1.1 caching servers, the
> same RFC2616 (#14.9) states that:
> Note that HTTP/1.0 caches might not implement Cache-Control and
> might only implement Pragma: no-cache (see section 14.32).
> So I would not assume that because it works for you in your latest
> browsers that your clients will see the latest version of your content,
> although as you say, they will see something.

As I said before, even the most broken client/proxie will see the exact
same information that any other RFC-compliant browser : no harm done

About HTTP/1.0: I still have to make some tests, but according to my
logs, HTTP/1.0 requests account for 43% of all requests. HTTP/1.0
requests are 99% webcrawlers, 0.8% spambots, 0.2% genuine clients.

So I have about 0.1% clients that *may be* won't handle the no-cache
trick right. And for having tested some of them personally, I know some
do handle no-cache correctly. Chances are way greater that perfectly
"no-cache" capable browser will come with "accept cookie" disabled. So I
don't see any reason to focus on the cache/no-cache capability of old
proxies/clients when the major drawback is the cookie ;)

> If I were you I would either use a script to finely control the headers
> serving a plethora of them (ala rfc2616 which is my view has

I plan to design a php script to reproduce the same behaviour for people
who can't access mod_header (low cost hosting...).

> contributed to the confusion and lack of precision in this area), or -
> which I know you have ruled out at the start - use mod_expires, with
> its tight fine grained control. Mod_cache would also work without fault
> in my opinion - if you found that a suitable module to use, you will be
> able to make conditional requests based on last modifed and a whole
> host of other options.

I want to have the less dependencies as possible, so if I can stick with
mod_header alone it's good.

> You could set your own Etag header, but why
> bother when it has all been done and is no longer considered
> experimental, as is in my opinion your approach.

What I've done is still experimental, I've designed it for my own use,
and it runs on my very own server. I don't plan to release the whole
thing to the public now (I will -so the php equivalent- when I'm done
with the testing).

regards,

patpro

--
http://www.patpro.net/