|
Home > Archive > Web Servers on Unix and Linux > March 2007 > apache log and missing referrer
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
apache log and missing referrer
|
|
| news@celticbear.com 2007-03-09, 1:19 pm |
| We just installed AWStats to parse and display our Web server log. It
seems to be working fine--showing the hits and errors and pages, etc.
Except...it's not showing any of the referrer information. All 0's,
and the browser information (agent?)
No info.
Looking at my httpd.conf, it appears to be set to collect that info.
What might I be missing?
Below is a piece from the conf.
Thanks for any feedback or suggestions of where I should look.
Liam
#
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
#
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i
\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
#
# The location and format of the access logfile (Common Logfile
Format).
# If you do not define any access logfiles within a <VirtualHost>
# container, they will be logged here. Contrariwise, if you *do*
# define per-<VirtualHost> access logfiles, transactions will be
# logged therein and *not* in this file.
#
#CustomLog logs/access_log common
#
# If you would like to have agent and referer logfiles, uncomment the
# following directives.
#
#CustomLog logs/referer_log referer
#CustomLog logs/agent_log agent
#
# For a single logfile with access, agent, and referer information
# (Combined Logfile Format), use the following directive:
#
CustomLog logs/access_log combined
| |
|
| news@celticbear.com <news@celticbear.com> wrote:
> We just installed AWStats to parse and display our Web server log. It
> seems to be working fine--showing the hits and errors and pages, etc.
> Except...it's not showing any of the referrer information. All 0's,
> and the browser information (agent?)
> No info.
> Looking at my httpd.conf, it appears to be set to collect that info.
> What might I be missing?
What is in the actual log, are they logged there? Could be an AWStats
configuration error.
--
Rik Wasmus
Posted on Usenet, not any forum you might see this in.
Ask Smart Questions: http://tinyurl.com/anel
| |
| news@celticbear.com 2007-03-09, 1:19 pm |
| On Mar 9, 10:41 am, Rik <luiheidsgoe...@hotmail.com> wrote:
> n...@celticbear.com <n...@celticbear.com> wrote:
>
> What is in the actual log, are they logged there? Could be an AWStats
> configuration error.
> --
> Rik Wasmus
> Posted on Usenet, not any forum you might see this in.
> Ask Smart Questions:http://tinyurl.com/anel
Thanks for replying!
The log is in an odd place, not the usual (we're using RedHat ES4 and
Apache 2.0.52).
/var/www/html/home/logs/access_log
(Why it's there is a long story involving people who had even less of
an idea about server management than I have.)
Here's the tail of the log:
202.158.106.110 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
thumbnails/car-103bcthumb.jpg HTTP/1.1" 200 2631
207.200.116.197 - - [09/Mar/2007:10:07:38 -0600] "GET /studio/
studiobackground.bkk?
sel_companyid=jdubo79& sel_typeproduct=pricingbusinesscards2x35
HTTP/
1.1" 200 20900
207.200.116.9 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
thumbnails/auto38thumb.jpg HTTP/1.1" 200 3332
207.200.116.130 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
thumbnails/ERGrizzlyBkgrndthumb.jpg HTTP/1.1" 200 9775
207.200.116.10 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
thumbnails/abstract9thumb.jpg HTTP/1.1" 200 3828
207.200.116.7 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
thumbnails/ERCrayolaBkgrndthumb.jpg HTTP/1.1" 200 17805
72.30.226.171 - - [09/Mar/2007:10:07:42 -0600] "GET /robots.txt HTTP/
1.0" 200 146
202.158.106.110 - - [09/Mar/2007:10:07:41 -0600] "GET /studio/themes/
thumbnails/ERBiPlaneBkgrndthumb.jpg HTTP/1.1" 200 14304
Hmm, not sure what referrer and agent info would look like in
there...but I'm guessing it's not there.
But, the httpd.conf appears to be configured to do so... any ideas?
Thanks!
Liam
| |
| Jim Hayter 2007-03-09, 1:19 pm |
| news@celticbear.com wrote:
> The log is in an odd place, not the usual (we're using RedHat ES4 and
> Apache 2.0.52).
> /var/www/html/home/logs/access_log
> (Why it's there is a long story involving people who had even less of
> an idea about server management than I have.)
> Here's the tail of the log:
> 202.158.106.110 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
> thumbnails/car-103bcthumb.jpg HTTP/1.1" 200 2631
> 207.200.116.197 - - [09/Mar/2007:10:07:38 -0600] "GET /studio/
> studiobackground.bkk?
> sel_companyid=jdubo79& sel_typeproduct=pricingbusinesscards2x35
HTTP/
> 1.1" 200 20900
> 207.200.116.9 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
> thumbnails/auto38thumb.jpg HTTP/1.1" 200 3332
> 207.200.116.130 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
> thumbnails/ERGrizzlyBkgrndthumb.jpg HTTP/1.1" 200 9775
> 207.200.116.10 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
> thumbnails/abstract9thumb.jpg HTTP/1.1" 200 3828
> 207.200.116.7 - - [09/Mar/2007:10:07:40 -0600] "GET /studio/themes/
> thumbnails/ERCrayolaBkgrndthumb.jpg HTTP/1.1" 200 17805
> 72.30.226.171 - - [09/Mar/2007:10:07:42 -0600] "GET /robots.txt HTTP/
> 1.0" 200 146
> 202.158.106.110 - - [09/Mar/2007:10:07:41 -0600] "GET /studio/themes/
> thumbnails/ERBiPlaneBkgrndthumb.jpg HTTP/1.1" 200 14304
>
> Hmm, not sure what referrer and agent info would look like in
> there...but I'm guessing it's not there.
> But, the httpd.conf appears to be configured to do so... any ideas?
> Thanks!
> Liam
>
Those entries correspond to your common log format. Are there any other
CustomLog directives in your config?
| |
| shimmyshack 2007-03-09, 1:19 pm |
| On 9 Mar, 17:23, Jim Hayter <see.reply...@nowhere.invalid> wrote:
> n...@celticbear.com wrote:
>
>
> Those entries correspond to your common log format. Are there any other
> CustomLog directives in your config?- Hide quoted text -
>
> - Show quoted text -
There must be further directives perhaps in a vhost section.
Also do heed cyrus' advice about NOT allowing your logs to be public.
Use at least basic authentication or you will be log spammed so badly
that you will have to answer endless questions from your client. (You
will always get a few odd/nasty looking referrers but if the logs are
public you will get many more I'm guessing)
Also with awstats the ability exists to turn off on demand parsing of
the access file, it's a good idea because it puts a lot of load on
your server, and if you have a very large access log there will could
be a large amount of DNS lookups even if you enable caching of DNS, so
be aware of that too, and perhaps use CRON at night. my $0.02.
| |
| news@celticbear.com 2007-03-09, 1:19 pm |
| On Mar 9, 11:23 am, Jim Hayter <see.reply...@nowhere.invalid> wrote:
> n...@celticbear.com wrote:
>
>
> Those entries correspond to your common log format. Are there any other
> CustomLog directives in your config?
OK, it just got weirder.
CustomLog appears nowhere else in the conf than that copied text
above.
However, there is this section which defines where the log goes:
<VirtualHost *:80>
DocumentRoot /var/www/html/home/printingautomation
ServerName printingautomation.com
ServerAlias www.printingautomation.com
TransferLog /var/www/html/home/logs/access_log
ErrorLog /var/www/html/home/logs/error_log
</VirtualHost>
I don't know if that's related, having the log defined like that.
But I just took a closer look at the normally default log:
[root@s75712 httpd]# ls -al /var/log/httpd/access_log
-rw-r--r-- 1 root root 522126 Mar 9 11:37 /var/log/httpd/access_log
and I see it's WAY smaller than log I thought was the only log, but
it's being written to!
It's tail:
[root@s75712 httpd]# tail access_log
71.232.89.244 - - [09/Mar/2007:11:37:35 -0600] "GET /design02/images/
arrow.gif HTTP/1.1" 200 211 "http://site.printingautomation.com/
design02/index.html?sel_companyid=bcsource&sel_affiliateid=" "Mozilla/
4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
1.1.4322; .NET CLR 2.0.50727)"
71.232.89.244 - - [09/Mar/2007:11:37:35 -0600] "GET /design02/images/
visa.gif HTTP/1.1" 200 1511 "http://site.printingautomation.com/
design02/index.html?sel_companyid=bcsource&sel_affiliateid=" "Mozilla/
4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
1.1.4322; .NET CLR 2.0.50727)"
THAT looks like the way it should be!
I can understand then that the log for the main domain is going to
that other location, and is 15GB as it's never been a part of
logrotate which this properly located access log is only showing
today's stats--I don't understand why the "main" one is showing common
config and the other one is combined.
The httpd.conf only has combined defined.
Well, here's a question: If I remove the line " TransferLog /
var/www/html/home/logs/access_log" from the VirtualDir, would that
possibly have the main domain's traffic entered into the /var/log/
httpd/access_log file?
(And thus subject to logrotate?)
I guess I can just try it and see... but I'd love to get experienced
feedback before I give it a try. =)
Thanks!
Liam
| |
| news@celticbear.com 2007-03-09, 1:19 pm |
| On Mar 9, 11:41 am, "shimmyshack" <matt.fa...@gmail.com> wrote:
> On 9 Mar, 17:23, Jim Hayter <see.reply...@nowhere.invalid> wrote:
>
>
>
>
>
>
>
> There must be further directives perhaps in a vhost section.
> Also do heed cyrus' advice about NOT allowing your logs to be public.
> Use at least basic authentication or you will be log spammed so badly
> that you will have to answer endless questions from your client. (You
> will always get a few odd/nasty looking referrers but if the logs are
> public you will get many more I'm guessing)
> Also with awstats the ability exists to turn off on demand parsing of
> the access file, it's a good idea because it puts a lot of load on
> your server, and if you have a very large access log there will could
> be a large amount of DNS lookups even if you enable caching of DNS, so
> be aware of that too, and perhaps use CRON at night. my $0.02.
Excellent advice!! Thank you for the reply! =)
-Liam
| |
| JP. Baker 2007-03-12, 7:17 am |
| In article <1173462697.653960.88890@j27g2000cwj.googlegroups.com>,
news@celticbear.com <news@celticbear.com> wrote:
>CustomLog appears nowhere else in the conf than that copied text
>above.
>However, there is this section which defines where the log goes:
>
><VirtualHost *:80>
> DocumentRoot /var/www/html/home/printingautomation
> ServerName printingautomation.com
> ServerAlias www.printingautomation.com
> TransferLog /var/www/html/home/logs/access_log
Try replacing this with CustomLog /var/www/html/home/logs/access_log combined
> ErrorLog /var/www/html/home/logs/error_log
></VirtualHost>
--
John P Baker
|
|
|
|
|