AOL Webserver - aolserver vs lighthttpd, benchmarks

Author

aolserver vs lighthttpd, benchmarks

John Buckman

2007-09-25, 7:11 am

I did some benchmarks of aolserver vs lighthttpd on plain files of
various sizes, on my 8cpu 64 bit server.

For 3 runs, with a 4k text file, the lighthttpd stats were 15103.87/
s, 14845.20/s and 15307.17/s, vs (as Dossy reports http://dossy.org/
archives/000517.html) aolserver's 15237.00/s.

Result: Aolserver is performing virtually identically to lighthttpd
with a small 4k file.

With a 44k JPG (the BookMooch home page illustration) aolserver is
slightly slower:
aolserver: 8164.54/s, 8283.93/s
lighthttpd: 10281.01/s, 9969.91/s

With 1,414k sized zip, aolserver is about 1/2 as fast:
aolserver: 406.36/s, 421.10/s
lighthttpd: 844.05/s, 807.83/s

with a 128mb file, aolserver is 40% slower:
aolserver: 5.42/s
lighthttpd: 8.88/s

And just FYI "hello world" in tcl (<% ns_adp_puts "hello" %> ):
aolserver: 950.85/s

I haven't done a "hello world" in C: I assume it'd be close to the
16k/s speed

-john

www64:/b# ab -c 5 -n 50000 http://images.bookmooch.com/x.txt
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking images.bookmooch.com (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Finished 50000 requests

Server Software: lighttpd/1.4.18
Server Hostname: images.bookmooch.com
Server Port: 80

Document Path: /x.txt
Document Length: 4070 bytes

Concurrency Level: 5
Time taken for tests: 3.266443 seconds
Complete requests: 50000
Failed requests: 0
Write errors: 0
Total transferred: 218950000 bytes
HTML transferred: 203500000 bytes
Requests per second: 15307.17 [#/sec] (mean)
Time per request: 0.327 [ms] (mean)
Time per request: 0.065 [ms] (mean, across all concurrent
requests)
Transfer rate: 65458.97 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 0 0 0.1 0 12
Waiting: 0 0 0.1 0 12
Total: 0 0 0.1 0 12

Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 0
99% 0
100% 12 (longest request)

Dossy Shiobara

2007-09-25, 1:11 pm

On 2007.09.25, John Buckman <john@MAGNATUNE.COM> wrote:
> I did some benchmarks of aolserver vs lighthttpd on plain files of
> various sizes, on my 8cpu 64 bit server.

Thanks! This is all fantastic information to have!

> www64:/b# ab -c 5 -n 50000 http://images.bookmooch.com/x.txt

Since your machine is a 2-CPU 8-core box, could you try runs with -c 8
and -c 16? It may have no effect, but it'd be nice to know that for
sure.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

John Buckman

2007-09-25, 1:11 pm

>> www64:/b# ab -c 5 -n 50000 http://images.bookmooch.com/x.txt
>
> Since your machine is a 2-CPU 8-core box, could you try runs with -c 8
> and -c 16? It may have no effect, but it'd be nice to know that for
> sure.

I get more-or-less the same numbers at -c 8, -c 16 and -c 32, varying
about 100 requests/second between runs.

I'm not sure why the tcl "hello world" test yields speeds so much
slower, with
ab -c 8 -n 5000 http://bookmooch.com/test.adp

yielding:
Requests per second: 971.45 [#/sec] (mean)

On the positive side, a real world function, namely a registered tcl
proc for returning a member's uploaded photo from the database:
ab -c 8 -n 5000 http://bookmooch.com/photo/johnbuckman.jpg

(I know it looks like an image url, but it's actually tcl) yields:
Requests per second: 892.18 [#/sec] (mean)

which is a very good real-world speed (fyi, the jpg is 6831 bytes long).

-john

Tom Jackson

2007-09-27, 1:11 pm

I was looking at lighttpd performance
(at http://trac.lighttpd.net/trac/wiki/Docs%3APerformance )

Considering how well AOLserver stands up to lighttpd, my question was why does
lighttpd do better?

For sending small files AOLserver is slightly better, but then performance
goes down.

One reason might be that lighttpd uses a syscall that sends the file directly
to the network adapter, bypassing lighttpd. I wonder if AOLserver does this,
or if it is possible to detect errors in the webserver if the whole file
isn't sent.

Another reason might be that lighttpd doesn't write an access.log by default.
I wonder if you can turn this off in AOLserver? Or, if not, can the benchmark
be run for lighttpd with the access.log enabled:

http://trac.lighttpd.net/trac/wiki/Docs%3AModAccessLog

With multi-cpus and lighttpd processes, lighttpd might mess up the access.log
writing, but if these writes are taken out of the performance equation, it
sure gives lighttpd a headstart.

Are there any hints on how to setup the benchmarking? I can run one here on a
1cpu/2core/64bit laptop.

If we can establish a benchmarking suite, it would help test various setting
for ns_limits and ns_pools.

It would also be interesting to see benchmarks under misbehaving clients.

One other factor is number of disks. Lighttpd recommends 2 x number of disks
to determine processes, although it seems like this wouldn't be such an easy
choice.

Ideas?

tom jackson

On Tuesday 25 September 2007 00:50, John Buckman wrote:
> I did some benchmarks of aolserver vs lighthttpd on plain files of
> various sizes, on my 8cpu 64 bit server.
>
> For 3 runs, with a 4k text file, the lighthttpd stats were 15103.87/
> s, 14845.20/s and 15307.17/s, vs (as Dossy reports http://dossy.org/
> archives/000517.html) aolserver's 15237.00/s.
>
> Result: Aolserver is performing virtually identically to lighthttpd
> with a small 4k file.
>
> With a 44k JPG (the BookMooch home page illustration) aolserver is
> slightly slower:
> aolserver: 8164.54/s, 8283.93/s
> lighthttpd: 10281.01/s, 9969.91/s
>
> With 1,414k sized zip, aolserver is about 1/2 as fast:
> aolserver: 406.36/s, 421.10/s
> lighthttpd: 844.05/s, 807.83/s
>
> with a 128mb file, aolserver is 40% slower:
> aolserver: 5.42/s
> lighthttpd: 8.88/s
>
> And just FYI "hello world" in tcl (<% ns_adp_puts "hello" %> ):
> aolserver: 950.85/s
>
> I haven't done a "hello world" in C: I assume it'd be close to the
> 16k/s speed

John Buckman

2007-09-27, 1:11 pm

When comparing lighthttpd vs aolserver, notice that aolserver only
does worse than lighthttpd for large files, and on the same file
system/hardware. Thus, the difference in benchmarks is not likely to
be the access logs or disk.

Lighthttpd is *not* using the system call to send a file to a socket
(I forget the name) as this call was taken out of the Linux kernel, I
believe with 2.4. I remember reading a note about this from Linus,
that the performance for that system call was terrible, so they were
taking it out.

Based on my own experience with sending large files over tcpip, the
difference is that aolserver uses a thread-based approach vs a
lighthttpd's single-thread async approach.

But really, I'm not sure aolserver's performance is really an issue
anyhow, because aolserver only slows down with large files (100mb+)
and at that point, you're completely saturating your network
connection (in my 128mb benchmark below, aolserver is sending 695mb
per second!)

So, I don't see any point in worrying about improving aolserver's
plain-file-sending, at least until we get start getting 10 gigabit
network cards (!)

-john

On Sep 27, 2007, at 4:53 PM, Tom Jackson wrote:

> I was looking at lighttpd performance
> (at http://trac.lighttpd.net/trac/wiki/Docs%3APerformance )
>
> Considering how well AOLserver stands up to lighttpd, my question
> was why does
> lighttpd do better?
>
> For sending small files AOLserver is slightly better, but then
> performance
> goes down.
>
> One reason might be that lighttpd uses a syscall that sends the
> file directly
> to the network adapter, bypassing lighttpd. I wonder if AOLserver
> does this,
> or if it is possible to detect errors in the webserver if the whole
> file
> isn't sent.
>
> Another reason might be that lighttpd doesn't write an access.log
> by default.
> I wonder if you can turn this off in AOLserver? Or, if not, can the
> benchmark
> be run for lighttpd with the access.log enabled:
>
> http://trac.lighttpd.net/trac/wiki/Docs%3AModAccessLog
>
> With multi-cpus and lighttpd processes, lighttpd might mess up the
> access.log
> writing, but if these writes are taken out of the performance
> equation, it
> sure gives lighttpd a headstart.
>
> Are there any hints on how to setup the benchmarking? I can run one
> here on a
> 1cpu/2core/64bit laptop.
>
> If we can establish a benchmarking suite, it would help test
> various setting
> for ns_limits and ns_pools.
>
> It would also be interesting to see benchmarks under misbehaving
> clients.
>
> One other factor is number of disks. Lighttpd recommends 2 x number
> of disks
> to determine processes, although it seems like this wouldn't be
> such an easy
> choice.
>
> Ideas?
>
> tom jackson
>
> On Tuesday 25 September 2007 00:50, John Buckman wrote:
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <listserv@listserv.aol.com> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.

Dossy Shiobara

2007-09-27, 1:11 pm

On 2007.09.27, Tom Jackson <tom@RMADILO.COM> wrote:
> Another reason might be that lighttpd doesn't write an access.log by
> default. I wonder if you can turn this off in AOLserver?

Yes, you can. If you don't load the nslog module, you turn off access
logging.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Jeff Rogers

2007-09-27, 7:11 pm

Tom Jackson wrote:
> I was looking at lighttpd performance
> (at http://trac.lighttpd.net/trac/wiki/Docs%3APerformance )
>
> Considering how well AOLserver stands up to lighttpd, my question was why does
> lighttpd do better?

I was wondering if it had something to do with how aolserver does its
writes, in particular the Ns_ConnFlushDirect|Ns_ConnWrite -> Ns_ConnSend
-> NsConnSend -> DriverSend chain. Each connection thread writes to its
own socket and loops to ensure that all the data is written. It seems
at first glance that it would make more sense to hand the task of
writing to the connection back to the driver thread once the connection
thread is done with it, and the single driver thread could multiplex
sending over multiple sockets with a single poll loop as well as leaving
the connection threads free to do other work. This is the same as the
stated rationale for the Ns_QueueWait api - "I/O events are cheap so do
those upfront instead of having expensive connection threads burdened
with wasteful blocking I/O" but on the writing side instead of the
reading side.

Since this is such an obvious change, would I be correct in guessing
that it was either tried and discarded as an overall performance loss
(there would be alot of data being passed between threads meaning a lot
of copying or extra mutexes if the connection thread and the driver
thread both accessed the buffer concurrently), or rejected as an
incompatible api change (Ns_ConnSend would do something subtly
different, and I guess it could make it harder to detect an incompletely
sent result)?

-J

Dossy Shiobara

2007-09-27, 7:11 pm

On 2007.09.27, John Buckman <john@MAGNATUNE.COM> wrote:
> Lighthttpd is *not* using the system call to send a file to a socket
> (I forget the name) as this call was taken out of the Linux kernel, I
> believe with 2.4. I remember reading a note about this from Linus,
> that the performance for that system call was terrible, so they were
> taking it out.

Hmm--I'm not sure what you're thinking of or referring to, but the
common "optimization" is to use sendfile(2), which seems to be
alive and well in the 2.6 tree.

Sendfile requires that the data to be written come from a file
descriptor. The assumption is that the program opens a file, then wants
to pass it off to have it sent out another file descriptor, typically a
network connection. But, what if you want to avoid disk I/O and cache
that data in memory? sendfile() isn't necessarily workable, there.

Perhaps you could mmap() to a fd and use that--should test the
performance of such an implementation.

> Based on my own experience with sending large files over tcpip, the
> difference is that aolserver uses a thread-based approach vs a
> lighthttpd's single-thread async approach.

It might be possible to push static file processing further up the chain
into the DriverThread and get better performance on larger static
files--or, have one dedicated I/O thread separate from the main driver
thread to handle async I/O of static assets. (I'm in favor of the
latter, separate thread, just to avoid further complication of the
main DriverThread.)

> But really, I'm not sure aolserver's performance is really an issue
> anyhow, because aolserver only slows down with large files (100mb+)
> and at that point, you're completely saturating your network
> connection (in my 128mb benchmark below, aolserver is sending 695mb
> per second!)

Right--the cry of "AOLserver isn't fast enough for static file serving"
is kinda BS. If you need one single box to be able to saturate its GigE
link, I think your technical requirements are questionable. :-)

The bottleneck is very likely to be your upstream bandwidth to your
peer before you run out of juice in AOLserver.

> So, I don't see any point in worrying about improving aolserver's
> plain-file-sending, at least until we get start getting 10 gigabit
> network cards (!)

I am concerned about your "hello world" dynamic request benchmark,
though. I would have expected at least 4k req/sec--not the sub-1k
req/sec you saw. I have a feeling it has to do with the default
ns_pools/ns_limits settings, which naturally are NOT tuned for a 8-core
CPU box.

I bet with just a few minutes of tweaking and tuning, we can get between
4k-8k simple dynamic req/sec out of your hardware.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Tom Jackson

2007-09-27, 7:11 pm

On Thursday 27 September 2007 09:55, John Buckman wrote:
> When comparing lighthttpd vs aolserver, notice that aolserver only
> does worse than lighthttpd for large files, and on the same file
> system/hardware. Thus, the difference in benchmarks is not likely to
> be the access logs or disk.

Yeah, that would be a contradiction. I wonder what causes AOLserver to fall
behind with large files, and I assume a lower number of new requests per sec.

> Lighthttpd is *not* using the system call to send a file to a socket
> (I forget the name) as this call was taken out of the Linux kernel, I
> believe with 2.4. I remember reading a note about this from Linus,
> that the performance for that system call was terrible, so they were
> taking it out.

I guess if a single syscall was used to send a large file, everything would
have to wait for it to finish.

> Based on my own experience with sending large files over tcpip, the
> difference is that aolserver uses a thread-based approach vs a
> lighthttpd's single-thread async approach.
>

Right, but AOLserver also does a lot of reading before passing off to a
connection thread, so the input is a single thread event loop.

> But really, I'm not sure aolserver's performance is really an issue
> anyhow, because aolserver only slows down with large files (100mb+)
> and at that point, you're completely saturating your network
> connection (in my 128mb benchmark below, aolserver is sending 695mb
> per second!)

How is the benchmark run? Do you use any particular config for your AOLserver?

> So, I don't see any point in worrying about improving aolserver's
> plain-file-sending, at least until we get start getting 10 gigabit
> network cards (!)

I agree, I'm not interested in improving AOLserver's peformance by making code
changes. I'm more interested in exploring how AOlserver performs under
different configuration and load situations.

I'm still amazed that AOLserver performs so well for static content, but I
guess everything gets moved in or out of the socket as static bytes, so it is
a great way to measure script performance...that is we can't blame any
slowness on the basic socket/thread management. And of course, the
configuration cound have an impact.

tom jackson

Dossy Shiobara

2007-09-27, 7:11 pm

On 2007.09.27, Jeff Rogers <dvrsn@DIPHI.COM> wrote:
> [...] It seems at first glance that it would make more sense to hand
> the task of writing to the connection back to the driver thread once
> the connection thread is done with it, [...]
>
> Since this is such an obvious change, would I be correct in guessing
> [...]

I don't know if the change was attempted, but it was certainly
considered and acknowledged as ultimately the "right thing to do" but as
I mentioned in a previous message, AOLserver's write speed hasn't been a
significant bottleneck for AOL, so the work was not pursued.

If anyone has real and actual needs to do large file serving out of
AOLserver directly and would like to either (a) make the change and
contribute it, or (b) fund its development, I would think it would be
welcomed.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Tom Jackson

2007-09-27, 7:11 pm

On Thursday 27 September 2007 11:08, Dossy Shiobara wrote:
> It might be possible to push static file processing further up the chain
> into the DriverThread and get better performance on larger static
> files--or, have one dedicated I/O thread separate from the main driver
> thread to handle async I/O of static assets. (I'm in favor of the
> latter, separate thread, just to avoid further complication of the
> main DriverThread.)

I also just noticed that there is a prequeue filter point. This gets run
before the socket is handed off to a connection thread. If you could direct
static file requests to a particular driver, you could use prequeue to
completely eliminate the connection queue (I think, I haven't tested how or
if you can break out of the filter pipeline).

It seems like some kind of specialization of the queue.c code could be used to
create a single thread event loop.

But it seems strange that lower number of requests would make AOLserver
perform worse than lighttpd, when it does so well with more requests. To me
this implies that the event/loop code in driver.c and queue.c is working very
well.

> I am concerned about your "hello world" dynamic request benchmark,
> though. I would have expected at least 4k req/sec--not the sub-1k
> req/sec you saw. I have a feeling it has to do with the default
> ns_pools/ns_limits settings, which naturally are NOT tuned for a 8-core
> CPU box.
>
> I bet with just a few minutes of tweaking and tuning, we can get between
> 4k-8k simple dynamic req/sec out of your hardware.

This is what I was trying to get at. Knowing that the current code is very
fast, it is easier to focus attention on configuration tuning.

tom jackson

John Buckman

2007-09-27, 7:11 pm

> Hmm--I'm not sure what you're thinking of or referring to, but the
> common "optimization" is to use sendfile(2), which seems to be
> alive and well in the 2.6 tree.

Whoops, you're right, I just remembered a sysadmin email about
sendfile not existing on the kernel we're using. Hmm...

> Sendfile requires that the data to be written come from a file
> descriptor. The assumption is that the program opens a file, then
> wants
> to pass it off to have it sent out another file descriptor,
> typically a
> network connection. But, what if you want to avoid disk I/O and cache
> that data in memory? sendfile() isn't necessarily workable, there.
>
> Perhaps you could mmap() to a fd and use that--should test the
> performance of such an implementation.

I've done exactly that in the past, and found virtually no speed
improvement, because Unix has a very aggressive disk cache, so adding
your own cache on top of that just skips a few system calls.

>
> I am concerned about your "hello world" dynamic request benchmark,
> though. I would have expected at least 4k req/sec--not the sub-1k
> req/sec you saw. I have a feeling it has to do with the default
> ns_pools/ns_limits settings, which naturally are NOT tuned for a 8-
> core
> CPU box.
>
> I bet with just a few minutes of tweaking and tuning, we can get
> between
> 4k-8k simple dynamic req/sec out of your hardware.

I have an 8 cpu Mac I, but it's not on the net (it's at my home), so
I can't give you ssh access. If you can suggest a thing or two to
try, I'll give it a whirl.

I benchmarked the "/helloworld" C module, and got about 16k responses
per second vs 900/s with the tcl equivalent.

I've been looking at C-caching of Tcl dynamic content, with "dirty
cache" support. For example, replacing the Tcl code that returns a
user's uploaded photo with C code.

I wrote C code to do this, and got 14k/second vs 240/s for the same
tcl function. So, C really is a good replacement for Tcl when a
particular URL needs to be very fast.

-john

Dave Bauer

2007-09-27, 7:11 pm

On 9/27/07, John Buckman <john@magnatune.com> wrote:
> When comparing lighthttpd vs aolserver, notice that aolserver only
> does worse than lighthttpd for large files, and on the same file
> system/hardware. Thus, the difference in benchmarks is not likely to
> be the access logs or disk.
>
> Lighthttpd is *not* using the system call to send a file to a socket
> (I forget the name) as this call was taken out of the Linux kernel, I
> believe with 2.4. I remember reading a note about this from Linus,
> that the performance for that system call was terrible, so they were
> taking it out.
>
> Based on my own experience with sending large files over tcpip, the
> difference is that aolserver uses a thread-based approach vs a
> lighthttpd's single-thread async approach.
>

>
Maybe try the background delivery using libthread?
http://openacs.org/xowiki/ Boost_yo..._large_files%21
if it is actually an issue.

Dave

Bas Scheffers

2007-09-27, 7:11 pm

On 28/09/2007, at 3:38 AM, Dossy Shiobara wrote:
> I bet with just a few minutes of tweaking and tuning, we can get
> between
> 4k-8k simple dynamic req/sec out of your hardware.
Hear, hear. I just blasted my brand-spanking-new Quad 2.6 Mac Pro
using ab. I was testing my ns_session, so it was doing even more than
just "hello, world". I was getting between 2500 and 4000 reqs/sec,
even though I did not tune anything. base.tcl only with just the ip/
port changed.

I would suspect Linux being faster on a box like this than OS X (BSD)
because of better threading support. Or am I wrong in that assumption?

Cheers,
Bas.

Bas Scheffers

2007-09-27, 7:11 pm

On 28/09/2007, at 5:04 AM, John Buckman wrote:
> I've been looking at C-caching of Tcl dynamic content, with "dirty
> cache" support. For example, replacing the Tcl code that returns a
> user's uploaded photo with C code.
>
> I wrote C code to do this, and got 14k/second vs 240/s for the same
> tcl function. So, C really is a good replacement for Tcl when a
> particular URL needs to be very fast.
My solution to that problem was simply caching in the filesystem and
serving static files. The way this works in a multi-server
environment is that the custom 404 handler figures out the request
was for "/photo/123/axbcgsfdt.jpg" and just grabs it from the
database, caches it and redirects back to it.

That way I get the convenience of Tcl code and the speed of static
files - save for the first request.

Bas.

Rusty Brooks

2007-09-27, 7:11 pm

Yeah, I do this too. To "dirty" the cache you can just delete the
file. I do regular rounds to delete old files.

I don't use the 404 though, that's a neat idea. Instead I register a
proc that ns_returnfiles the cache file if it exists, otherwise it makes
it and then returns it.

Bas Scheffers wrote:
> On 28/09/2007, at 5:04 AM, John Buckman wrote:
> My solution to that problem was simply caching in the filesystem and
> serving static files. The way this works in a multi-server environment
> is that the custom 404 handler figures out the request was for
> "/photo/123/axbcgsfdt.jpg" and just grabs it from the database, caches
> it and redirects back to it.
>
> That way I get the convenience of Tcl code and the speed of static
> files - save for the first request.
>
> Bas.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <listserv@listserv.aol.com> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.

Dossy Shiobara

2007-09-28, 1:11 am

On 2007.09.28, Bas Scheffers <bas@SCHEFFERS.NET> wrote:
> I would suspect Linux being faster on a box like this than OS X (BSD)
> because of better threading support. Or am I wrong in that assumption?

I'm only guessing, but I doubt there would be a significant performance
difference between OS X's Mach/BSD-like OS vs. Linux.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Dossy Shiobara

2007-09-28, 1:11 am

On 2007.09.28, Bas Scheffers <bas@SCHEFFERS.NET> wrote:
> My solution to that problem was simply caching in the filesystem and
> serving static files. The way this works in a multi-server
> environment is that the custom 404 handler figures out the request
> was for "/photo/123/axbcgsfdt.jpg" and just grabs it from the
> database, caches it and redirects back to it.
>
> That way I get the convenience of Tcl code and the speed of static
> files - save for the first request.

Yes! Finally, someone else who uses the 404-handler-as-request-processor
pattern! Indeed, you can't beat static file serving performance. And,
if you want to "clear the cache" you just unlink the file out of the
filesystem, and the next request through "re-caches" it.

The only tricky thing is to handle the "stampeding herd" situation,
where many requests for the same uncached object come in at the same
time. You only want one request to populate the cache--you want all
other requests to block/yield. A simple condvar/broadcast works well
for this--before starting the "expensive" work involved in populating
the cache, you push a request for the object to be generated into a
queue (producers) then wait on the condvar. A thread pool (consumers)
gathers all requests for the same object and generates it once, then
broadcasts to unblock all the waiters. They wake up, check to see if
their request was fulfilled--if it was, they consume from the cache.
Otherwise, they wait on the condvar again, until they time out.

This level of care is only really important when the cost of populating
a cache entry is "expensive" and the cost increases as concurrency
increases (i.e., concurrent database queries).

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Dossy Shiobara

2007-09-28, 1:11 am

On 2007.09.27, Rusty Brooks <me@RUSTYBROOKS.COM> wrote:
> Yeah, I do this too. To "dirty" the cache you can just delete the
> file. I do regular rounds to delete old files.
>
> I don't use the 404 though, that's a neat idea. Instead I register a
> proc that ns_returnfiles the cache file if it exists, otherwise it makes
> it and then returns it.

The reason to use the 404-handler method is that static file serving
should be even faster than a registered proc--as John discovered through
benchmarking.

If you change your approach to use the 404-handler instead of the
registered proc/ns_returnfile, I think we'd all appreciate a quick
benchmark test to see what the difference was.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Bas Scheffers

2007-09-28, 1:11 am

On 28/09/2007, at 9:28 AM, Dossy Shiobara wrote:
> Yes! Finally, someone else who uses the 404-handler-as-request-
> processor
> pattern! Indeed, you can't beat static file serving performance.
> And,
My first inspiration for this came way back in the last century, from
working with Vignette StoryServer. Rather bad technology that HAD to
use caching to be able to serve more than a few pages a second, but
the way they did it was a neat trick.

> The only tricky thing is to handle the "stampeding herd" situation,
> where many requests for the same uncached object come in at the same
> time. You only want one request to populate the cache--you want all
At the moment, there is only one server and when the file is
uploaded, I also pre-cache it. If we ever do need more servers, I am
thinking of simply having the upload process cycle through a list of
servers it then does a http request to. I plan to do a similar thing
for clearing the cache. If the request fails, you can log and queue
it for a re-try.

If the farm would get beyond two servers, it would be worth looking
into serving the images only out of two servers (for fail over); no
need to have all of the app servers keep their own copy!

Bas.

Rusty Brooks

2007-09-28, 1:11 am

I see your point, and I might change it at some point, but it's sort of
"good enough" as it is... I'm replacing stuff that takes, say, 10s with
something that I can fetch, say, 1000/s. It doesn't matter too much if
I can do it 1000, 2000, or 5000/s, as lot as it's not 1/s.

Rusty

Dossy Shiobara wrote:
> On 2007.09.27, Rusty Brooks <me@RUSTYBROOKS.COM> wrote:
>
>
> The reason to use the 404-handler method is that static file serving
> should be even faster than a registered proc--as John discovered through
> benchmarking.
>
> If you change your approach to use the 404-handler instead of the
> registered proc/ns_returnfile, I think we'd all appreciate a quick
> benchmark test to see what the difference was.
>
> -- Dossy
>
>

John Buckman

2007-09-28, 7:11 am

On Sep 28, 2007, at 8:43 AM, John Buckman wrote:
> My solution to that problem was simply caching in the filesystem
> and serving static files. The way this works in a multi-server
> environment is that the custom 404 handler figures out the request
> was for "/photo/123/axbcgsfdt.jpg" and just grabs it from the
> database, caches it and redirects back to it.
> That way I get the convenience of Tcl code and the speed of static
> files - save for the first request.

The 404 handler approach is very clever, that'll do exactly what I
need, thanks!

Dossy: not sure if it's even worth benchmarking this, as this
approach will yield the static-file-speeds, which are amazing, except
in the rare no-static-file-available case, where it's back to 900/s.

-john

Gustaf Neumann

2007-09-28, 7:11 am

Dossy Shiobara schrieb:
> On 2007.09.27, Jeff Rogers <dvrsn@DIPHI.COM> wrote:
>
>
> I don't know if the change was attempted, but it was certainly
> considered and acknowledged as ultimately the "right thing to do" but as
> I mentioned in a previous message, AOLserver's write speed hasn't been a
> significant bottleneck for AOL, so the work was not pursued.
>
> If anyone has real and actual needs to do large file serving out of
> AOLserver directly and would like to either (a) make the change and
> contribute it, or (b) fund its development, I would think it would be
> welcomed.
>
This is very similar what the "background delivery" in openacs does.

Most of this can be done on the tcl layer with little effort.
I have developed a small patch for the aolserver that extends
ns_conn by two additional subcommands "channel"
and "contentsentlength", where the first one returns
the tcl channel name for a connection and the second
one is for passing the content-length to the log file.
The background delivery code passes the channel
to a event aware thread (using tcl libthread), which spools the file
to via tcl fcopy. The single delivery thread can easily handle
a few hundred simultaneous deliveries without sweat (probably
much more).

The motivation of the patch was mostly to avoid blocking of connection
threads, especially, when large files are delivered to users with slow
connections. This is especially important for openacs, where the
threads are rather fat (a few thousand tcl procs are in the blueprint),
such that the number of configured threads is rather small.

There are as well other useful applications for the patch, see e.g.
http://www.openacs.org/xowiki/weblo...ptag=bgdelivery

I have asked dossy for commit permissions to add my patches to
the aolserver on Sept 22, 2006 and sent him the patches as well,
but my impression was that there was very little or no interest.

The patch is as well incoporated in naviserver since about this time.

The most developed install script of openacs adds the patch by default
http://www.cognovis.de/developer/en/aolserver_install

-gustaf neumann

Dossy Shiobara

2007-09-28, 7:11 am

On 2007.09.28, John Buckman <john@MAGNATUNE.COM> wrote:
> The 404 handler approach is very clever, that'll do exactly what I
> need, thanks!
>
> Dossy: not sure if it's even worth benchmarking this, as this
> approach will yield the static-file-speeds, which are amazing, except
> in the rare no-static-file-available case, where it's back to 900/s.

Yeah, if you do go the 404-handler route, there's no need to benchmark
except if you still have requests that are completely dynamic and cannot
be cached in this manner.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Dossy Shiobara

2007-09-28, 1:11 pm

On 2007.09.28, Gustaf Neumann <neumann@WU-WIEN.AC.AT> wrote:
> I have asked dossy for commit permissions to add my patches to
> the aolserver on Sept 22, 2006 and sent him the patches as well,
> but my impression was that there was very little or no interest.

Sorry, I totally forgot to respond to your last email on the matter. I
am so sorry about that.

In the future: If you're waiting for a response from me, please don't
hesitate to send me a follow-up to nudge me. I am constantly juggling
so many things and I get distracted easily.

Gustaf, email me your SourceForge username and I'll add you to the
AOLserver project with CVS access.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Tom Jackson

2007-09-28, 1:11 pm

When I look at the patch, it seems to me that this could be put into a module.
The new C level command doesn't need to be a static command, it uses only
external functions and variables (Ns_*, Tcl_*).

Can we work togeather to get a module instead of a patch?

I'll work up a module file today if someone can test it, or tell me how.

tom jackson

On Friday 28 September 2007 03:04, Gustaf Neumann wrote:
> Dossy Shiobara schrieb:
>
> This is very similar what the "background delivery" in openacs does.
>
> Most of this can be done on the tcl layer with little effort.
> I have developed a small patch for the aolserver that extends
> ns_conn by two additional subcommands "channel"
> and "contentsentlength", where the first one returns
> the tcl channel name for a connection and the second
> one is for passing the content-length to the log file.
> The background delivery code passes the channel
> to a event aware thread (using tcl libthread), which spools the file
> to via tcl fcopy. The single delivery thread can easily handle
> a few hundred simultaneous deliveries without sweat (probably
> much more).
>
> The motivation of the patch was mostly to avoid blocking of connection
> threads, especially, when large files are delivered to users with slow
> connections. This is especially important for openacs, where the
> threads are rather fat (a few thousand tcl procs are in the blueprint),
> such that the number of configured threads is rather small.
>
> There are as well other useful applications for the patch, see e.g.
> http://www.openacs.org/xowiki/weblo...ptag=bgdelivery
>
> I have asked dossy for commit permissions to add my patches to
> the aolserver on Sept 22, 2006 and sent him the patches as well,
> but my impression was that there was very little or no interest.
>
> The patch is as well incoporated in naviserver since about this time.
>
> The most developed install script of openacs adds the patch by default
> http://www.cognovis.de/developer/en/aolserver_install
>
> -gustaf neumann
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <listserv@listserv.aol.com> with the body of "SIGNOFF AOLSERVER" in the
> email message. You can leave the Subject: field of your email blank.

Tom Jackson

2007-09-28, 1:11 pm

Okay, after looking further into this patch, I see that it doesn't actually
add any functionality to AOLserver. It looks like you would have to install a
newer version of OpenACS to use this.

I have the stubs in for an AOLserver C module, but I'm not sure about a few
things.

Two commands are being added. One dups a conn sock and wraps it with a
Tcl_Channel. The second just returns the number of bytes sent:

case CContentSentLenIdx:
if (objc == 2) {
Tcl_SetIntObj(result, connPtr->nContentSent);
} else if (objc == 3) {
if (Tcl_GetIntFromObj(interp, objv[2], &connPtr->nContentSent) != TCL_OK) {
return TCL_ERROR;
}
} else {
Tcl_WrongNumArgs(interp, 2, objv, "?value?");
return TCL_ERROR;
}

where objc == 2, we can use the existing Ns_ConnContentSent:
Tcl_SetIntObj(result, Ns_ConnContentSent(conn));

But what is objc == 3?
Is there some code somewhere for setting the amount of sent content?

tom jackson

On Friday 28 September 2007 07:44, Tom Jackson wrote:
> When I look at the patch, it seems to me that this could be put into a
> module. The new C level command doesn't need to be a static command, it
> uses only external functions and variables (Ns_*, Tcl_*).
>
> Can we work togeather to get a module instead of a patch?
>
> I'll work up a module file today if someone can test it, or tell me how.

Gustaf Neumann

2007-09-29, 7:11 pm

Tom Jackson schrieb:
> Okay, after looking further into this patch, I see that it doesn't actually
> add any functionality to AOLserver. It looks like you would have to install a
> newer version of OpenACS to use this.
>
as i wrote in my earlier mail, the patch is simple and small and adds
just two
subcommands to ns_conn. Thanks to dossy, the patch is commited to cvs. head.

The applications to "ns_conn channel" are on the tcl layer and
are quite simple to use. Look into the xotcl-core package
(xotcl-core/tcl/bgdelivery-procs.tcl) in the openacs cvs repository.
With the patch, xotcl-core and libthread, one can replace e.g.
ns_returnfile 200 $mime_type $filename
by
ad_returnfile_background 200 $mime_type $filename
in cr_write_content in acs-content-repository/tcl/revision-procs.tcl
to use the background delivery for content sent from the
content repository (e.g. file store).

best regards
-gustaf neumann

Tom Jackson

2007-10-01, 1:11 pm

Gustaf,

I'm going to ask that the patch be removed and replaced with a module. I've
already written one which does the same thing.

We are in a bad habit in this community of letting anything happen at the
least possible cost to those who modify core code. This is a perfect example
of the situation. We now have two new subcommands for ns_conn. Some would
phrase it as only two. But, for instance, most commands have like four total
subcommands.

Regardless of the number, we have just added two undocumented commands to the
core code. We have a major problem with documentation here, and we know why:
developers are not required (not even encouraged!) to do any work documenting
new code or features. They don't have to provide any examples which work with
AOLserver. All that they have to do is to commit a patch. This easy five
minute job now requires everyone else to do work.

Second, the patch does nothing for AOLserver without at least figuring out
where to get the ::thread package. Again, there is no documentation on where
to get it, how to install it or use it. But the example code which was
referenced below is part of a much larger project: OpenACS, and a very recent
version of it. In fact, it is part of OpenACS that isn't even necessary for
the core features, and is only there for optimization as you have pointed
out. The examples are written in xotcl, which I have tried to figure out for
a year or more with little success.

Third, some technical issues:

New features which don't require core changes should be done as modules,
unless there is an obvious close relationship to the core functions. In this
case, the new feature is so foreign to the normal workings of AOLserver that
the code should be isolated from the core until it is fully understood. Since
you didn't write the original code, I assume that you can't really explain
all the details of what is going on and why. I spent a lot of time looking at
this new feature and seeing if there was anything similar in AOLserver. What
I discovered was quite the opposite. There is not a single example where two
conns (conns, not sockets) are returning data to clients in the same thread.
It is possible that the AOL team once considered this, but none of the core
API support this in any way. Each is written to work in a single thread (or
block) and contains an internal event loop (in C). It is impossible to
imagine that the core could be rearranged to support a generic multiplexing
of client returns. When you survey how many different types of returns there
are, you will immediately understand why. The easiest way to put it is that
reading client data is essentially a binary read until passed to a connection
thread. Once the connection thread has all the binary data, it can do
conversions as needed.

AOLserver request model is a pipeline which is specialized per request. We can
consider this new feature as creating a branch where the original pipeline
continues (like after ns_conn close), and another branch sends data to the
client. But AOLserver has no control over the new branch. It could fail and
we would never know. There are no logs of failures. Actually, not quite true,
it looks like if the delivery fails for some reason, or if you don't call
[ns_conn contentsentlength $filelength], you get zero bytes in the
access.log.

On the surface, the patch appears (as described) to do something like
ns_returnfile. All the patch does is to make a copy of the connection and
wrap the sock in a Tcl_Channel. It doesn't create a worker thread, doesn't
pass the conn to this thread, it doesn't write headers or open the static
file. There is no way to signal for shutdown and to a thread join. All of
this work has to be done by the developer, including debugging, testing, and
everything else. So what we have is a new way for everyone to start doing
things in different ways, wondering why it doesn't work, complaining about
why it doesn't work the way they expect, etc. Some will probably offer their
own patches to make it work better.

When we accept the patch, we allow you to transfer all this mess to the
AOLserver community. Now we have to answer all these concerns, you don't.

This also short circuits discussion about the best way to do this in
AOLserver, and what to recommend to new users. Why? We just had another good
discussion about the performance of AOLserver for static content. We
discovered that the model for AOLserver works very well in this context, so
much so that we are kind of scrathing our heads over any need to think too
hard about performance improvements. There really are better things to work
on. There seems to be an obsession among new developers that performance
tweaking is the first thing to be considered when writing code. They think
about a few lines of code, one syscall vs another. But when you read the
AOLserver sources, it really starts to dawn on you: it ain't easy.
Fortunately it is already written, so why not use it to best advantage? Part
of that being done for you is the module system. If new functionality can be
added without patching the core it does two important things: it provides
examples of how to do it, and proves (as in tests) the sound design of the
core. If developers are encouraged to provide patches instead of modules it
works against these goals.

Anyway, I fully support the code in module form. I don't completely understand
the performance or security impact, but as a module, users can take it or not
by their own choice. If I had to do a similar task, I would probably create a
unique code and create a symlink to the actual file, then redirect the user
to that file using another instance of AOLserver. The other instance would
check the age of the symlink and serve the file if under some configured
amount. If the entire file gets served, remove the link. This way, you can
fully leverage the REST style, and it will likely release your expensive
connection thread even faster than the current example.

Using the code as a module (minus the ability to transfer the channel), here
is an example of use (the string bgwrite could easily be changed to anything
else, I chose it before I realized that the code doesn't do any bgwriting.):

# Get nContentSent (should be 0)
set contentsentlength [ns_bgwrite contentsentlength]
# Set nContentSent (to zero)
ns_bgwrite contentsentlength $contentsentlength

# Dup sock and wrap in Tcl_Channel
set channel [ns_bgwrite channel]

# File to return:
set dir [file dirname [info script]]
set image sns-thumb.jpg
set file [file join $dir $image]

# Get length of file:
set contentLength [file size $file]

# Get content type:
set contentType [ns_guesstype $file]

# Create last modified header data:
set lastModified [ns_httptime [file mtime $file]]

# Create header:
set header "HTTP/1.1 200 OK
Last-Modified: $lastModified
MIME-Version: 1.0
Date: [ns_httptime [ns_time]]
Server: AOLserver/4.5.0
Content-Type: $contentType
Content-Length: $contentLength
Connection: close
"

# Open file to send:
set inFd [open $file r]

# Configure channels:
fconfigure $inFd -translation binary
fconfigure $channel -translation binary

# put headers (could block forever):
puts $channel $header

# fcopy file to channel.
fcopy $inFd $channel -command [list close $channel]

#^^in a conn thread, this only sends the first 4096 bytes
# while in background mode (caused by -command switch)

# The following still happens regardless of fcopy status:

# Set nContentSent so log file will look normal:
ns_bgwrite contentsentlength $contentLength

# Log something
ns_log Notice "File $file has contentlength = $contentLength"

# No need to return, as [ns_bgwrite channel] has effect of [ns_conn close].

# Traces will still run on this thread
# what happens to keepalive?

### Log File Messages:
# error.log
[-default:0-] Notice: File /web/nsd45/servers/jnm/pages/sns-thumb.jpg has
contentlength = 28672
[-default:0-] Notice: running ::traceFilter, sleeping for '1' seconds

# access.log
192.168.1.102 - - [01/Oct/2007:06:58:23 -0700] "GET /nsbgwrite.tcl HTTP/1.0"
200 28672 "" ""

# access.log without setting nContentSent
192.168.1.102 - - [30/Sep/2007:22:00:25 -0700] "GET /nsbgwrite.tcl HTTP/1.0"
200 0 "" ""

tom jackson

On Saturday 29 September 2007 13:55, Gustaf Neumann wrote:
> Tom Jackson schrieb:
>
> as i wrote in my earlier mail, the patch is simple and small and adds
> just two
> subcommands to ns_conn. Thanks to dossy, the patch is commited to cvs.
> head.
>
> The applications to "ns_conn channel" are on the tcl layer and
> are quite simple to use. Look into the xotcl-core package
> (xotcl-core/tcl/bgdelivery-procs.tcl) in the openacs cvs repository.
> With the patch, xotcl-core and libthread, one can replace e.g.
> ns_returnfile 200 $mime_type $filename
> by
> ad_returnfile_background 200 $mime_type $filename
> in cr_write_content in acs-content-repository/tcl/revision-procs.tcl
> to use the background delivery for content sent from the
> content repository (e.g. file store).

Andrew Piskorski

2007-10-01, 7:11 pm

On Mon, Oct 01, 2007 at 09:40:19AM -0700, Tom Jackson wrote:
> Gustaf,
>
> I'm going to ask that the patch be removed and replaced with a module.

Why? What possible harm does including Gustaf's patch in the stock
server cause, rather than loading it as an additional module?

> Regardless of the number, we have just added two undocumented
> commands to the core code.

So what?

> We have a major problem with documentation here, and we know why:
> developers are not required (not even encouraged!) to do any work documenting
> new code or features.

Uh, so you'd rather have NEITHER useful new features nor
documentation? Because in general, I think that's what you're likely
to get.

And besides, Gustaf has written about his patch at length multiple
times over the last few years. The content is out there - in what way
is it not adequate? And just where is Gustaf SUPPOSED to add docs for
his new feature anyway?

> Second, the patch does nothing for AOLserver without at least
> figuring out where to get the ::thread package.

So what? Again, where's the harm?

And AOLserver should probably be changed somtime to always use and
ship with the Tcl Thread package anyway.

> So what we have is a new way for everyone to start doing
> things in different ways,

Everyone? No one is required to use Gustaf's feature, and indeed, the
only way to use it is by explicitly writing Tcl code to do so, right?

> wondering why it doesn't work, complaining about why it doesn't work
> the way they expect, etc. Some will probably offer their own patches
> to make it work better.

God forbid that anyone would offer potential improvements... Oh yeah,
I'm really worried about that!

> This also short circuits discussion about the best way to do this in
> AOLserver, and what to recommend to new users.

Tom, Gustaf has been both discussing and heavily using this one patch
for several years. Just how much more discussion do you want? It
obviously has been working well for U. Wien and others for years now,
so why not just adopt their proven patch as is, rather than screwing
around turning it into a loadable module?

What's the actual problem with Gustaf's code? You've obviously read
and thought about it, Tom (which I have not), but so far I see a lot
of theoretical hand wavy complaints from you, but little solid
criticism of the actual code.

--
Andrew Piskorski <atp@piskorski.com>
http://www.piskorski.com/

Dossy Shiobara

2007-10-01, 7:11 pm

On 2007.10.01, Andrew Piskorski <atp@PISKORSKI.COM> wrote:
> And just where is Gustaf SUPPOSED to add docs for
> his new feature anyway?

I will try to update doc/ns_conn.n soon and include the two new ns_conn
subcommands that Gustaf's patch adds.

> And AOLserver should probably be changed somtime to always use and
> ship with the Tcl Thread package anyway.

I'm not sure I agree with this. Not right now, anyway. But, this is a
conversation that I want to defer until I've had sufficient time to
think about it carefully.

-- Dossy

--
Dossy Shiobara | dossy@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

Jeff Rogers

2007-10-01, 7:11 pm

Andrew Piskorski wrote:

> What's the actual problem with Gustaf's code? You've obviously read
> and thought about it, Tom (which I have not), but so far I see a lot
> of theoretical hand wavy complaints from you, but little solid
> criticism of the actual code.

I also think the code belongs on a module rather than in the core.

My reason is because the code is the wrong solution. I'm not saying the
code is bad or that it doesn't work, because it obviously works and
solves a problem for alot of people. The problem is that with this
code, fixing a problem (simple network i/o blocking heavyweight
connection threads) requires doing something different at the script
level, meaning it is likely that some pages will use the fix and some
will not. This is a hack allowing a limitation to be worked around, not
a fix for the limitation. When the patch fixes the core so that all
connections (or selectively as specified in the config file) receive the
benefit, then it belongs in the core.

I see this as similar to the inclusion of gzip compression. Several
versions ago a module was contributed that added a new command
nz_zreturn to allow for gzip-compressed output. This was wrong because
someone would need to explicitly choose to use it on every individual
page. Including gzip compression in the core is the right way, and
that's where it is now. ns_return_background is fine to work around the
network writing problem for now, but it should be only a module; the
core patch should be a complete fix, one that applies the solution to
every connection using plain old ns_return.

Yes, we may need to wait a little while for this "real" fix. And when
we do create it, this patch becomes unnecessary except as backward
compatibility, which again will belong in a module, for those who need it.

-J

Tom Jackson

2007-10-01, 7:11 pm

On Monday 01 October 2007 10:48, Andrew Piskorski wrote:
> Tom, Gustaf has been both discussing and heavily using this one patch
> for several years. Just how much more discussion do you want? It
> obviously has been working well for U. Wien and others for years now,
> so why not just adopt their proven patch as is, rather than screwing
> around turning it into a loadable module?

Patching core code with experimental features is screwing around. There is
nothing wrong with screwing around with your own copy, but accepting this
type of code into core would lower the bar for any change. This code has
nothing to do with the normal workings of AOLserver. I just spent a few days
looking for any other example of this type of thing. Instead I found some
nice comments like this:

tclsock.c-766- /*
tclsock.c-767- * Pass a dup of the socket to the callback thread, allowing
tclsock.c-768- * this thread's cleanup to close the current socket. It's
tclsock.c-769- * not possible to simply register the channel again with
tclsock.c-770- * a NULL interp because the Tcl channel code is not entirely
tclsock.c:771: * thread safe.
tclsock.c-772- */
tclsock.c-773-
tclsock.c-774- sock = ns_sockdup(sock);

....

tclsock.c-1077- if (cbPtr->chan == NULL) {
tclsock.c-1078- /*
tclsock.c-1079- * Create and register the channel on first use. Because
tclsock.c:1080: * the Tcl channel code is not entirely thread safe, it's
tclsock.c-1081- * not possible for the scheduling thread to create and
tclsock.c-1082- * register the channel.
tclsock.c-1083- */
tclsock.c-1084-
tclsock.c-1085- cbPtr->chan = Tcl_MakeTcpClientChannel((ClientData) sock);

> What's the actual problem with Gustaf's code? You've obviously read
> and thought about it, Tom (which I have not), but so far I see a lot
> of theoretical hand wavy complaints from you, but little solid
> criticism of the actual code.

Both of these are used in the patch, and that doesn't mean there is a problem,
but these dup/copy are happening once per receiving thread. What happens when
multiple channels get pushed into a single thread? You also have to fake up
the nContentSent so that the access.log is 'maybe' correct.

Anyway, my criticism is limited to the fact that you don't need to patch the
core to add this feature, and the change does not at all match up with
anything else in AOLserver core. Is it okay to add anything? How about remove
anything? If Gustaf can add commands, why can't I remove them? Isn't this the
definition of anarchy? There is no logical basis for making a decision, just
the ability to do so. The module concept completely eliminates this problem
by dividing up responsibilities and giving it to those who are interested in
it. The idea that modules have owners (usually folks negotiate with the
maintainer) but the core is a free pass is pretty strange.

The mere fact that the code has been in use for several years does not mean
that this is the way it should go in the public community code.

tom jackson

Vasiljevic Zoran

2007-10-01, 7:11 pm

On 01.10.2007, at 21:04, Tom Jackson wrote:

> tclsock.c-766- /*
> tclsock.c-767- * Pass a dup of the socket to the callback
> thread, allowing
> tclsock.c-768- * this thread's cleanup to close the current
> socket. It's
> tclsock.c-769- * not possible to simply register the channel
> again with
> tclsock.c-770- * a NULL interp because the Tcl channel code is
> not entirely
> tclsock.c:771: * thread safe.
> tclsock.c-772- */
> tclsock.c-773-
> tclsock.c-774- sock = ns_sockdup(sock);
>
> ...
>
> tclsock.c-1077- if (cbPtr->chan == NULL) {
> tclsock.c-1078- /*
> tclsock.c-1079- * Create and register the channel on first
> use. Because
> tclsock.c:1080: * the Tcl channel code is not entirely thread
> safe, it's
> tclsock.c-1081- * not possible for the scheduling thread to
> create and
> tclsock.c-1082- * register the channel.
> tclsock.c-1083- */
> tclsock.c-1084-
> tclsock.c-1085- cbPtr->chan = Tcl_MakeTcpClientChannel
> ((ClientData) sock);

This is all old stuff as since some time the Tcl channel
handling code is thread-safe and you can create channels
in one, de-register them and register them again in the
other thread.

Beside that, the patch that Gustaf added is neutral in the sense
what usage people will make out of it. It merely allows you to
re-use the connection socket from your Tcl code w/o the need to
add yet-another ns_return-like of command. You can use this for a
variety of purposes, not necessarily only for implementing a
single-thread-static-content server.
It is safe to use with [ns_channel] machinery as well...

Cheers,
Zoran

Gustaf Neumann

2007-10-02, 7:11 am

Hi everybody,

Tom Jackson schrieb:
> Gustaf,
>
> I'm going to ask that the patch be removed and replaced with a module.
i got the - wrong - impression that you (Tom) revised your proposal to
change
the patch into a module, when you realized, that the patch is NOT
implementing background delivery, but something much more simple
and generic. In the xotcl-core OpenACS module, "ns_conn channel"
is as well used for purposes, quite different from background delivery,
namely for providing a COMET-style back-channel from the server to
the client
http://www.ajaxian.com/archives/com...ax-applications

For example, the xotcl-core module uses this channel for implementing
a streaming chat (clients subscribe via HTTP to a named channel,
which is disconnected from the connection thread and transfered
to a spooling thread; messages sent to the channel are broadcasted
to all subscribers of the channels; the subscribers keep the channels
open and read asynchronously from the browser either via ajax or iframe).
This is not something we "could do in some future versions",
but something we have working since one and a half years.
http://openacs.org/forums/message-v...ssage_id=423582

The reason why i have described this is to show that the ability to
obtain from a connection thread a Tcl channel opens a much larger
range of applications than just the background delivery. The
application range is just limited by the creativity of a developer.

The ns_conn subcommands extend the connection "object"
ns_conn to provide a method to obtain the channel the Tcl
channel, and to query or set the sent-content-length (setting
is in my background delivery implementation is somewhat
over-optimistic, since errors during file delivery are not
reflected in access log (only in the error log). But only those
errors are missed in the access log, which are happening
between start of the asynchronous transfer and the close.
The most typical "error" is, when a longer transfer is interrupted
(e.g. user closes browser during transfer)). A maybe better
solution would be to extend the nslog module
to deal with asynchronous deliveries, such it is able
to receive a call, when the full content is delivered, and
to prevent logging in the trace immediately after a request
in a connection thread).

In general, i would prefer to see on the longer range
some convergence between the aolserver way and
"ordinary" tcl. Obtaining the tcl channel is a small
step this way.

Additionally, i would prefer aolserver and naviserver
to be as compatible as possible (all parties will profit
from this). As mentioned earlier, both ns_conn subcommands
are available already in naviserver.

My change sits only in the head version of the cvs tree.
Most probably, nobody is running a production environment
from head. So the potential harm is very limited. The
question is rather, whether or not this functionality should
be included in the next release of aolserver, and when
and how the next release will be made available. I was
quite surprised to see that the aolserver has 10 admins
and 58 developers registered at sourceforge (considered
the low number of changes over the last year). My feeling is
that whoever is and feels responsible for next release
should decide what patches should be included
in the next release or not. From my point of view, dossy
is the premier candidate.
> Regardless of the number, we have just added two undocumented commands to the
> core code.
I was not sure, whether the panoptic wiki intends to document
4.5 or cvs head. Most probably the intention is to document 4.5.
I mentioned this in my mail to dossy immediately after the
commit and wrote the commit message in a style suitable for
documentation (see below). Your rhetoric is somewhat unfair
(my be not willingly unfair due to limited information).

========================================
== BEGIN COMMIT MSG
Extend ns_conn by two commands:

ns_conn channel

Return a Tcl channel from the current connection.
This channel can be used to talk via Tcl I/O to
the remote client afterwards.

ns_conn contentsentlength ?number?

Query or set the number of octets sent to the client. This command
can be used in connection with "ns_conn channel" to adjust the length
entry in a log file in cases, the file was sent via Tcl I/O.

"ns_conn channel" was originally written by Zoran.

========================================
== END COMMIT MSG

> When we accept the patch, we allow you to transfer all this mess to the
> AOLserver community. Now we have to answer all these concerns, you don't.
Come on. The new subcommands empower the developer and
add missing functionality enabling the development of different
kinds of applications the tcl way. Don't patronize the developers by saying
"you might end up with non working code". By the same argument,
many commands the tcl api must be forbitten/removed/whatever.

I would accept arguments like "do it the aolserver way"
instead of the "tcl way", to
- provide ns_* means to detach a stream from a connection thread and
- transfer it to a spooling thread and
- reimplement there something like an ns_fcopy working on streams
without making the visible to the user. But why?

The committed changes aka "this mess" is working since a few
years reliably in production. i just looked it up: yesterday
background delivery was used more than 200.000 times on a
single server here. Are you trying to argue that the change is
reducing the reliability of the aolserver/openacs?
Talk to to nima, who was close to giving up on openacs due to
performance problems, especially on days when many file downloads
happened.

I should also mention that the code in the aolserver module works
together with the request-monitor modules of openacs, where you
see the ongoing foreground and background activities.

And how does "this mess" change, if there is a module just implementing
the two subcommands? you just end up with more maintenance,
more installation effort (updating aolserver startup scripts, updating
build processes, updating FAQs). Maybe it helps the ego of some
developer if one can say "use my module", something, i don't care
about.
> Using the code as a module (minus the ability to transfer the channel), here
> is an example of use (the string bgwrite could easily be changed to anything
> else, I chose it before I realized that the code doesn't do any bgwriting.):
>
How about trying to understand what the code does before making a module?

-gustaf neumann

Tom Jackson

2007-10-02, 7:13 pm

On Tuesday 02 October 2007 02:50, Gustaf Neumann wrote:
>
> i got the - wrong - impression that you (Tom) revised your proposal to
> change
> the patch into a module, when you realized, that the patch is NOT
> implementing background delivery, but something much more simple
> and generic.

I wonder how many times I have to repeat myself, but I'll try again:

This 'simple' and 'generic' feature has nothing to do with ns_conn or how
connections are processed in AOLserver. The patch and the module do the same
thing, so what the code does is not of concern to me. It is an interesting
feature. I wish there were examples of how to use it with plain AOLserver
plus thread::transfer.

What we are really discussing is your resistence to putting the feature in the
correct place. And the reason for your resistence appears to be that it is
too much work for you. Even with the module completely written and
functional, that isn't enough. You want a special arrangement which saves you
from having to undo the bad decision to patch AOLserver for two years,
instead of writing a module.

Maybe we should ask why it took two years to submit such a simple patch?

I'm not interested in fame and glory or in saying 'use my module'. I'm trying
to provide an example of how to contribute to the AOLserver community in a
way which respects the extremely well developed API and the community.
Imagine what the sources would look like if the goal was to just do it as
easily as possible. There would be no ability to extend the server. Future
maintainers would have to carefully consider every change as it might have
difficult to detect consequences.

There are lots of things which are inconvenient in programming. But we do them
because in the long run they save us time and effort. The AOLserver sources
provide a good example of how inconvenient good programming can be.

Of all the talk about design patterns, I haven't ever heard about convenience
being one of these patterns. Why should convenience be the criteria for
adding new features to AOLserver?

> Additionally, i would prefer aolserver and naviserver
> to be as compatible as possible

I prefer to advocate for AOLserver, not naviserver, OpenACS or Tcl. And in
this case it would be nice if new code followed AOLserver coding norms. Maybe
you can get naviserver to take out their code. The module I wrote at least
compiled against their server, but nsd doesn't run on my 64bit laptop:

web/navi $ ./bin/nsd -f -t sample-config.tcl
[02/Oct/2007:10:37:45][6858.690331232][-main-] Notice: nsmain: Tcl version:
8.4.14
[02/Oct/2007:10:37:45][6858.690331232][-main-] Fatal: NsTclInitObjs:
sizeof(int) < sizeof(long)
Aborted

tom jackson

Vasiljevic Zoran

2007-10-02, 7:13 pm

On 02.10.2007, at 19:51, Tom Jackson wrote:

> web/navi $ ./bin/nsd -f -t sample-config.tcl
> [02/Oct/2007:10:37:45][6858.690331232][-main-] Notice: nsmain: Tcl
> version:
> 8.4.14
> [02/Oct/2007:10:37:45][6858.690331232][-main-] Fatal: NsTclInitObjs:
> sizeof(int) < sizeof(long)
> Aborted

This has been corrected since some time already.
CVS contains the correct code. The last released
code 4.99.2 is from 2006-02-04. Please update
your CVS sandbox and recompile.
We wanted to release 5.0 since some time but the
CVS head is so stable that we do not have any
pressing need so far.

Cheers,
Zoran

Stephen Deasey

2007-10-02, 7:13 pm

On 10/2/07, Tom Jackson <tom@rmadilo.com> wrote:

> ... it would be nice if new code followed AOLserver coding norms. Maybe
> you can get naviserver to take out their code. The module I wrote at least
> compiled against their server...

You're using symbols declared in nsd/nsd.h, which is private to the
server and not installed by default. Things in here change regularly.

You're asking for trouble...

> ...but nsd doesn't run on my 64bit laptop:
>
> [02/Oct/2007:10:37:45][6858.690331232][-main-] Fatal: NsTclInitObjs: sizeof(int) < sizeof(long)
> Aborted

That's *really* old code you're running...

....which is our fault for not making a release. Sorry.

You'll have more joy running cvs head and reporting bugs on the mailing list:

http://naviserver.sourceforge.net/w/Mailing_Lists

Tom Jackson

2007-10-03, 1:11 am

Stephen,

Thanks for pointing that out. I was wondering about that.

I have posted new module code which avoids the private symbols.

But this removes the contentsentlength option.

I still wonder what the difference between dup'ing and not dup'ing is for
(spliceout vs. non-spliceout). In the original code, the not dup'ing was hard
coded in as the only option. The current code hard codes this to dup.

Anyway, I don't really know if this will work as the original, I was able to
fcopy in background a short text file, but larger than 4096 bytes just gets
that amount according to wget.

In foreground/blocking mode, fcopy returns larger files.

I've included my test script as well:

http://rmadilo.com/files/nsbgwrite/

So all this does now is to return the sock wrapped in a Tcl_Channel.

[ns_conn close] also has the effect of closing the channel. Maybe if it is
moved to another thread that will not happen?

tom jackson

On Tuesday 02 October 2007 14:07, Stephen Deasey wrote:
> On 10/2/07, Tom Jackson <tom@rmadilo.com> wrote:
>
> You're using symbols declared in nsd/nsd.h, which is private to the
> server and not installed by default. Things in here change regularly.
>
> You're asking for trouble...
>
>
> That's *really* old code you're running...
>
> ...which is our fault for not making a release. Sorry.
>
>
> You'll have more joy running cvs head and reporting bugs on the mailing
> list:
>
> http://naviserver.sourceforge.net/w/Mailing_Lists
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <listserv@listserv.aol.com> with the body of "SIGNOFF AOLSERVER" in the
> email message. You can leave the Subject: field of your email blank.

Stephen Deasey

2007-10-03, 1:11 am

On 10/3/07, Tom Jackson <tom@rmadilo.com> wrote:

> Anyway, I don't really know if this will work as the original, I was able to
> fcopy in background a short text file, but larger than 4096 bytes just gets
> that amount according to wget.
>
> In foreground/blocking mode, fcopy returns larger files.
>
> I've included my test script as well:
>
> http://rmadilo.com/files/nsbgwrite/nsbgwrite.tcl

By using the -command option to fcopy you've enabled background mode,
but I don't see a call to vwait. You're not entering the event loop.

4096 happens to be the default buffer size of a Tcl channel.

Tom Jackson

2007-10-03, 1:11 am

Stephen,

Thanks, that worked. The -command was executed logging 'okay' + number of
bytes written.

But the ns_log statements following vwait are not executed. I assume that is
expected? I posted the updated script at:

http://rmadilo.com/files/nsbgwrite/nsbgwrite.tcl

tom jackson

On Tuesday 02 October 2007 18:23, Stephen Deasey wrote:
> On 10/3/07, Tom Jackson <tom@rmadilo.com> wrote:
>
> By using the -command option to fcopy you've enabled background mode,
> but I don't see a call to vwait. You're not entering the event loop.
>
> 4096 happens to be the default buffer size of a Tcl channel.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <listserv@listserv.aol.com> with the body of "SIGNOFF AOLSERVER" in the
> email message. You can leave the Subject: field of your email blank.

Stephen Deasey

2007-10-03, 1:11 pm

On 10/3/07, Tom Jackson <tom@rmadilo.com> wrote:

> Thanks, that worked. The -command was executed logging 'okay' + number of
> bytes written.
>
> But the ns_log statements following vwait are not executed. I assume that is
> expected? I posted the updated script at:
>
> http://rmadilo.com/files/nsbgwrite/nsbgwrite.tcl

fcopy $inFd $channel -command [list ns_log Notice "okay"]
vwait $channel

vwait is expecting a variable *name*. Here, it's waiting for the
variable 'sock23' (or whatever) to be set, which never happens.

There's a short example on the man page:

http://www.tcl.tk/man/tcl8.4/TclCmd/vwait.htm

Tom Jackson

2007-10-08, 7:11 pm

I have a patch for conn.c and ns.h which enables the functionality of the
previous conn.c patch to be handled in a module, and/or written more clearly.
The patch is an extension of the external Ns_Conn* API, best seen in context
using the patch of ns.h:

Index: ns.h
========================================
===========================
RCS file: /cvsroot/aolserver/aolserver/include/ns.h,v
retrieving revision 1.86
diff -u -r1.86 ns.h
--- ns.h 7 Jul 2006 03:27:22 -0000 1.86
+++ ns.h 8 Oct 2007 14:11:36 -0000
@@ -659,6 +659,7 @@
NS_EXTERN char *Ns_ConnServer(Ns_Conn *conn);
NS_EXTERN int Ns_ConnResponseStatus(Ns_Conn *conn);
NS_EXTERN int Ns_ConnContentSent(Ns_Conn *conn);
+NS_EXTERN int Ns_ConnSetContentSent(Ns_Conn *conn, int nContentSent);
NS_EXTERN int Ns_ConnResponseLength(Ns_Conn *conn);
NS_EXTERN Ns_Time *Ns_ConnStartTime(Ns_Conn *conn);
NS_EXTERN char *Ns_ConnPeer(Ns_Conn *conn);
@@ -666,7 +667,10 @@
NS_EXTERN char *Ns_ConnLocation(Ns_Conn *conn);
NS_EXTERN char *Ns_ConnHost(Ns_Conn *conn);
NS_EXTERN int Ns_ConnPort(Ns_Conn *conn);
-NS_EXTERN int Ns_ConnSock(Ns_Conn *conn);
+NS_EXTERN SOCKET Ns_ConnSock(Ns_Conn *conn);
+NS_EXTERN int NS_ConnSetSock(Ns_Conn *conn, SOCKET sock);
+NS_EXTERN int Ns_ConnSetSockInvalid(Ns_Conn *conn);
NS_EXTERN char *Ns_ConnDriverName(Ns_Conn *conn);
NS_EXTERN void *Ns_ConnDriverContext(Ns_Conn *conn);
NS_EXTERN int Ns_ConnGetKeepAliveFlag(Ns_Conn *conn);

I also updated the module to use the new API so that it functions the same as
ns_conn channel and ns_conn contentsentlength.

Everything is available at http://rmadilo.com/files/nsbgwrite/

There is also an associated Tcl module which creates the worker thread using
thread::create, storing the id in an nsv_array, along with a counter to track
the number of uses. An example tcl page takes requests and passes them off to
the worker thread. The tcl page works with both [ns_conn channel] and
[ns_bgwrite channel].

I used apache-bench to saturate the driver, threadpool and worker queues,
using the default threadpool with a maxthreads of 10.

The summary of requests/sec for concurrencies from 1-100 (28k image):

Requests per second: 163.53 [#/sec] (mean) -n 1 -c 1
Requests per second: 187.54 [#/sec] (mean)
Requests per second: 217.56 [#/sec] (mean)
Requests per second: 309.01 [#/sec] (mean)
Requests per second: 318.85 [#/sec] (mean)
Requests per second: 300.49 [#/sec] (mean)
Requests per second: 284.14 [#/sec] (mean)
Requests per second: 409.18 [#/sec] (mean)
Requests per second: 484.74 [#/sec] (mean)
Requests per second: 400.49 [#/sec] (mean)
Requests per second: 421.17 [#/sec] (mean)
Requests per second: 386.16 [#/sec] (mean)
Requests per second: 387.18 [#/sec] (mean)
Requests per second: 540.77 [#/sec] (mean)
Requests per second: 509.06 [#/sec] (mean)
Requests per second: 497.58 [#/sec] (mean)
Requests per second: 390.64 [#/sec] (mean)
Requests per second: 407.27 [#/sec] (mean)
Requests per second: 398.91 [#/sec] (mean)
Requests per second: 725.58 [#/sec] (mean)
Requests per second: 765.01 [#/sec] (mean) -n 2000 -c 20
Requests per second: 761.57 [#/sec] (mean) -n 2000 -c 40
Requests per second: 685.17 [#/sec] (mean)
Requests per second: 724.04 [#/sec] (mean) -n 2000 -c 75
Requests per second: 588.27 [#/sec] (mean)
Requests per second: 766.57 [#/sec] (mean) -n 4000 -c 100

Full results and summaries are at:
http://rmadilo.com/files/nsbgwrite/logs/

Everything seems to work pretty well. Although the response time goes up with
concurrency, the timing in the server log (log.txt) shows that this is mostly
due to time waiting in a queue, not in processing time, as both time in the
tcl page and the worker thread are about equal or less than the average
response time.

One issue which seems strange is that with concurrency > 1, there is an
additional request showing up in the logs (actually (concurrency -1)
additional requests). These end in error, the sock is not available. Apache
doesn't show that it sends these and then disconnects, but they show up even
at prequeue in the driver thread. It may be a bug in apache bench. Whatever
the cause, it doesn't appear to be anything more than the client
disconnecting at some intermediate step. The exact same behavior is seen with
[ns_conn channel] using the cvs head version. Or I have a bug in my tcl code.

New APIs:

Ns_ConnSetSock can be used for two purposes. One is to change the sock used by
a connection. The other is to signal that the sock is no longer valid, so the
connection pipeline will do something different. (Not sure exactly what is
different, but at least the Conn doesn't try to close the sock, since it
doesn't know about it anymore. But Ns_Sock still exists.)

Ns_ConnSetSockInvalid is an easier to understand version of Ns_ConnSetSock, it
simply calls Ns_ConnSetSock(conn, INVALID_SOCKET).

The other new API is Ns_ConnSetContentSent. I'm not too sure about this API,
but it does allow you to log the expected number of bytes sent to the client
when you handle the return outside of a conn. There are no good other options
at the moment. the access log module looks pretty unfriendly to additional
inputs.

ns_conn:

Instead of being such an XXX on this, I'll remove my objection to the new
ns_conn channel, etc. Tcl API, but I think both should be deprecated for
future use, replaced with one or more modules which use the new Ns_Conn APIs.
The main reason being that ns_conn channel does way too much to be in conn.c.
The real missing feature was the ability to invalidate the sock (or make
Ns_Sock think the socket was invalid) so that it wouldn't be cleaned up
immediately (and restore the conn sock if something goes wrong while creating
a channel).

async without Thread package?:

After looking at too much code, task.c and tclhttp.c (which gives ns_http)
caught my eye. It turns out that tclhttp.c is a very complicated example of
how to use task.c (Ns_Task). But the purpose of ns_http is very different
than handing off client returns to a single worker thread. In fact, ns_http
does a lot of _extra_ work to ensure that each http request is handled by a
single (not the queue) thread. But multiple threads could all work at the
same time using the same Ns_Task queue.

So, the bottom like appears to be that Ns_Task could be used to create a
worker thread. Adding a task to a queue triggers the event loop if it isn't
already running. It should be very easy to write the callback for the queue
to initialize by transfering the sock, if it hasn't already been transfered,
and opening the file to send, creating the headers, etc.

The task event loop works by doing one callback operation for each sock that
is ready, like copy the read buffer to the write buffer. The fact that the
callback proc doesn't have to do anything but very simple operations makes it
easier to program, but this was far from apparent with the ns_http example,
which uses blocking wait for reading. Ns_Task breaks up the 'what to do' from
the 'where to do it' and 'when to do it'.

tom jackson