Apache Directory Project - [mina] Writing to an IoSession

Author

[mina] Writing to an IoSession

Irving, Dave

2005-10-25, 7:45 am

Hi,

Im writing an Http server based on Mina. The tutorial was great and got
me started in the right direction. I'm using a "stateful" parser
approach like that used in ASN1 - and it fitted in very nicely.
Currently, I can ** parse ** around 24,000 HTTP requests per second -
and with multiple (non-HTTP-pipelining) connections, I can service
around 700 requests a second. This will improve massively in a few days
when I implement keep-alive.
So - after that background, I first want to say a big thank-you for
MINA. So far, it has proved very stable, and very fast - and hey, I was
getting bored of writing selector loops anyway :o)

One question I do have: Currently Im holding on to the IoSession
associated with a request so that when a response becomes available
(asyncronously), it knows where to write it to. I cant see any other way
of knowing where a response is destined for.
E.g, (cut down, sort of psudo-code)

/**
* Invoked asyncronously when a response is comitted - and ready to be=20
* written
*/
void handleResponse(Request req, Response resp, Object marker) {
IoSession session =3D req.getContext().getSession();
ByteBuffer buff =3D getBufferFor(resp);
session.write(buff, marker);
}

Should I be holding on to sessions like this - or looking them up by
some other means?
It doesn't seem clear from the tutorial.

Many thanks,

Dave

This e-mail and any attachment is for authorised use by the intended recipi=
ent(s) only. It may contain proprietary material, confidential information =
and/or be subject to legal privilege. It should not be copied, disclosed to=
, retained or used by, any other party. If you are not an intended recipien=
t then please promptly delete this e-mail and any attachment and all copies=
and inform the sender. Thank you.

Trustin Lee

2005-10-25, 7:45 am

Hi Dave,

2005/10/25, Irving, Dave <dave.irving-iKsOTpgdUR76V6G2DxALlg@public.gmane.org>:
>
> Im writing an Http server based on Mina. The tutorial was great and got
> me started in the right direction. I'm using a "stateful" parser
> approach like that used in ASN1 - and it fitted in very nicely.
> Currently, I can ** parse ** around 24,000 HTTP requests per second -
> and with multiple (non-HTTP-pipelining) connections, I can service
> around 700 requests a second. This will improve massively in a few days
> when I implement keep-alive.
> So - after that background, I first want to say a big thank-you for
> MINA. So far, it has proved very stable, and very fast - and hey, I was
> getting bored of writing selector loops anyway :o)

This sounds great. It would be very great if you can provide us some
benchmark results against Tomcat and Apache HTTPD / Apache HTTPD 2 if you
can. Are you able to do that for us? Of course we can run benchmark by
ourselves if we can access your HTTP server implementation.

One question I do have: Currently Im holding on to the IoSession
> associated with a request so that when a response becomes available
> (asyncronously), it knows where to write it to. I cant see any other way
> of knowing where a response is destined for.
> E.g, (cut down, sort of psudo-code)
>
> /**
> * Invoked asyncronously when a response is comitted - and ready to be
> * written
> */
> void handleResponse(Request req, Response resp, Object marker) {
> IoSession session = req.getContext().getSession();
> ByteBuffer buff = getBufferFor(resp);
> session.write(buff, marker);
> }

Yes there's no other way to find out what session is associated with request
right now. But you could maintain an IdentityHashMap<Request, IoSession>
based on sessionOpened() and sessionCreated() event:

void handleResponse(Request req, Response resp, Object marker) {
IoSession session = req2sessionMap.get( req );
byteBuffer buf = getBufferFor( resp );
session.write( buf, marker );
}

BTW which version of MINA are you using? It seems like you're using only I/O
layer. Am I right?

HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/

Irving, Dave

2005-10-25, 7:45 am

Hi Trustin,
=20
> his sounds great.=20=20
> It would be very great if you can provide us some benchmark results
against=20
> Tomcat and Apache HTTPD / Apache HTTPD 2 if you can. Are you able to
do that=20
> for us? Of course we can run benchmark by ourselves if we can access=20
> your HTTP server implementation.=20

Its still early days at the moment :o)
Basically, I work on a product which is used as a gateway in some
scenarios (e.g, receive an multi-media message, send it out to another
(off-board) application, and return a response to the original client
when we receive a response from the application.
In this example latency can be high (up to 20 secs per request).

So, what I needed was high-throughput even at high latency.
In tomcat, this means having a very large number of processor threads -
(which can obviously cause scalability problems).

So what Im doing is coming up with a through-and-through asyncronous API
for web applications, and a (NIO driven) web server to host such
applications.

At the moment, its still early days. But I can give you an example of a
test I ran this morning:

=20
________________________________________
________________________________
___
/ | Client Threadds | Server Threads | Requests |
Requests/second \
|_______________________________________
________________________________
____|
|AsyncWeb | 50 | 21 | 50,000 | 607
|
|Tomcat | 50 | 50 | 50,000 | 471
|
\_______________________________________
________________________________
____/

Tomcat had a simple Servlet configured which just gave a "hello world"
reply to each HTTP request.
Likewise, AsyncWeb had a simple AsyncHttpService configured which gave
the same "hello world" replies.
No connection keep alives were used (that comes later...).

Question:=20

Could you possibly tell me where to look to find out how to control the
number of threads created for processing MINA events? I noticed that I
was receiving call-backs from MINA on 21 threads....=20

> BTW which version of MINA are you using?=20=20
> It seems like you're using only I/O layer. Am I right?

0.7.4. I read about all the protocol layer stuff - but I don't really
have any need for it at the moment. The io layer simply delegates to the
stateful parsers and gets told when to write to and shut the connection.
I've got my own protocol layer at the moment which controls pipelining,
keep-alives, continue expectations etc.=20

Dave
_____________

From: Trustin Lee [mailto:trustin-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]=20
Sent: 25 October 2005 12:00
To: Apache Directory Developers List
Subject: Re: [mina] Writing to an IoSession

Hi Dave,

2005/10/25, Irving, Dave <dave.irving-iKsOTpgdUR76V6G2DxALlg@public.gmane.org>:=20

Im writing an Http server based on Mina. The tutorial was great
and got
me started in the right direction. I'm using a "stateful" parser
approach like that used in ASN1 - and it fitted in very nicely.
Currently, I can ** parse ** around 24,000 HTTP requests per
second -=20
and with multiple (non-HTTP-pipelining) connections, I can
service
around 700 requests a second. This will improve massively in a
few days
when I implement keep-alive.
So - after that background, I first want to say a big thank-you
for=20
MINA. So far, it has proved very stable, and very fast - and
hey, I was
getting bored of writing selector loops anyway :o)

This sounds great. It would be very great if you can provide us some
benchmark results against Tomcat and Apache HTTPD / Apache HTTPD 2 if
you can. Are you able to do that for us? Of course we can run
benchmark by ourselves if we can access your HTTP server implementation.

One question I do have: Currently Im holding on to the IoSession
associated with a request so that when a response becomes
available=20
(asyncronously), it knows where to write it to. I cant see any
other way
of knowing where a response is destined for.
E.g, (cut down, sort of psudo-code)
=09
/**
* Invoked asyncronously when a response is comitted - and ready
to be=20
* written
*/
void handleResponse(Request req, Response resp, Object marker) {
IoSession session =3D req.getContext().getSession();
ByteBuffer buff =3D getBufferFor(resp);
session.write(buff, marker);=20
}

Yes there's no other way to find out what session is associated with
request right now. But you could maintain an IdentityHashMap<Request,
IoSession> based on sessionOpened() and sessionCreated() event:=20

void handleResponse(Request req, Response resp, Object marker) {
IoSession session =3D req2sessionMap.get( req );
byteBuffer buf =3D getBufferFor( resp );
session.write( buf, marker );
}

BTW which version of MINA are you using? It seems like you're using
only I/O layer. Am I right?

HTH,
Trustin

--=20
what we call human nature is actually human habit
--
http://gleamynode.net/=20

This e-mail and any attachment is for authorised use by the intended recipi=
ent(s) only. It may contain proprietary material, confidential information =
and/or be subject to legal privilege. It should not be copied, disclosed to=
, retained or used by, any other party. If you are not an intended recipien=
t then please promptly delete this e-mail and any attachment and all copies=
and inform the sender. Thank you.

Trustin Lee

2005-10-25, 7:45 am

Hi Dave,

2005/10/25, Irving, Dave <dave.irving-iKsOTpgdUR76V6G2DxALlg@public.gmane.org>:
>
> Its still early days at the moment :o)
> Basically, I work on a product which is used as a gateway in some
> scenarios (e.g, receive an multi-media message, send it out to another
> (off-board) application, and return a response to the original client
> when we receive a response from the application.
> In this example latency can be high (up to 20 secs per request).
>
> So, what I needed was high-throughput even at high latency.
> In tomcat, this means having a very large number of processor threads -
> (which can obviously cause scalability problems).

Most web applications nowadays uses external resources such as JDBC
connection, so this means that MINA-based (NIO-based strictly speaking) HTTP
server can outperform in overload situation.

So what Im doing is coming up with a through-and-through asyncronous API
> for web applications, and a (NIO driven) web server to host such
> applications.
>
> At the moment, its still early days. But I can give you an example of a
> test I ran this morning:
>
>
> ________________________________________
________________________________
> ___
> / | Client Threadds | Server Threads | Requests |
> Requests/second \
> |_______________________________________
________________________________
> ____|
> |AsyncWeb | 50 | 21 | 50,000 | 607
> |
> |Tomcat | 50 | 50 | 50,000 | 471
> |
> \_______________________________________
________________________________
> ____/
>
> Tomcat had a simple Servlet configured which just gave a "hello world"
> reply to each HTTP request.
> Likewise, AsyncWeb had a simple AsyncHttpService configured which gave
> the same "hello world" replies.
> No connection keep alives were used (that comes later...).

The result shows 28.8% throughput improvement with less than half number of
threads. Sounds cool.

Question:
>
> Could you possibly tell me where to look to find out how to control the
> number of threads created for processing MINA events? I noticed that I
> was receiving call-backs from MINA on 21 threads....

Are you using SimpleServiceRegistry?

You have to get an instance of IoThreadPoolFilter and call
setMaximumPoolSize() event:

ServiceRegistry registry = new SimpleServiceRegistry();

IoThreadPoolFilter threadPoolFilter = ( IoThreadPoolFilter )
registry.getIoAcceptor( TransportType.SOCKET ).getFilterChain().get(
"threadPool" );
threadPoolFilter.setMaximumPoolSize( 10 );

HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/

Irving, Dave

2005-10-25, 7:45 am

Hi Trustin,

> Most web applications nowadays uses external resources such as JDBC=20
> connection, so this means that MINA-based (NIO-based strictly
speaking)=20
> HTTP server can outperform in overload situation.=20=20

This is true... However, most (java) web applications nowadays also use
the servlet API - which is blocking request / response based.
Most also sit on top of frameworks such a struts, spring, et al which
are also blocking.

Hence Im chuking the servlet Api out of the window and starting from
scratch (its just not worth trying to get async behaviour hooked in to
the servlet API - you end up chasing your tail!!).
I don't expect many people in the "main stream" would want to be using
it (no struts etc).
However, if it turns out to be good - then who knows - maybe one day
it'll be the next tomcat :o)
I think the emergence of technologies like AJAX might also increase the
need for this sort of application - so maybe there is a market emerging
:o)

> HTH

Yes - that definitely helps. Thanks!=20

Regards,

Dave
________________________________

From: Trustin Lee [mailto:trustin-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]=20
Sent: 25 October 2005 12:38
To: Apache Directory Developers List
Subject: Re: [mina] Writing to an IoSession

Hi Dave,

2005/10/25, Irving, Dave <dave.irving-iKsOTpgdUR76V6G2DxALlg@public.gmane.org>:=20

Its still early days at the moment :o)
Basically, I work on a product which is used as a gateway in
some
scenarios (e.g, receive an multi-media message, send it out to
another
(off-board) application, and return a response to the original
client=20
when we receive a response from the application.
In this example latency can be high (up to 20 secs per request).
=09
So, what I needed was high-throughput even at high latency.
In tomcat, this means having a very large number of processor
threads -=20
(which can obviously cause scalability problems).

Most web applications nowadays uses external resources such as JDBC
connection, so this means that MINA-based (NIO-based strictly speaking)
HTTP server can outperform in overload situation.=20=20

So what Im doing is coming up with a through-and-through
asyncronous API
for web applications, and a (NIO driven) web server to host such

applications.
=09
At the moment, its still early days. But I can give you an
example of a
test I ran this morning:
=09
=09
=09
________________________________________
________________________________
___
/ | Client Threadds | Server Threads | Requests |=20
Requests/second \
=09
|_______________________________________
________________________________
____|
|AsyncWeb | 50 | 21 | 50,000 |
607
|
|Tomcat | 50 | 50 | 50,000 |
471=20
|
=09
\_______________________________________
________________________________
____/
=09
Tomcat had a simple Servlet configured which just gave a "hello
world"
reply to each HTTP request.
Likewise, AsyncWeb had a simple AsyncHttpService configured
which gave=20
the same "hello world" replies.
No connection keep alives were used (that comes later...).

The result shows 28.8% throughput improvement with less than half number
of threads. Sounds cool.=20

Question:
=09
Could you possibly tell me where to look to find out how to
control the=20
number of threads created for processing MINA events? I noticed
that I
was receiving call-backs from MINA on 21 threads....

Are you using SimpleServiceRegistry?

You have to get an instance of IoThreadPoolFilter and call
setMaximumPoolSize() event:=20

ServiceRegistry registry =3D new SimpleServiceRegistry();

IoThreadPoolFilter threadPoolFilter =3D ( IoThreadPoolFilter )
registry.getIoAcceptor( TransportType.SOCKET ).getFilterChain().get(
"threadPool" );=20
threadPoolFilter.setMaximumPoolSize( 10 );

HTH,
Trustin

--=20
what we call human nature is actually human habit
--
http://gleamynode.net/=20

This e-mail and any attachment is for authorised use by the intended recipi=
ent(s) only. It may contain proprietary material, confidential information =
and/or be subject to legal privilege. It should not be copied, disclosed to=
, retained or used by, any other party. If you are not an intended recipien=
t then please promptly delete this e-mail and any attachment and all copies=
and inform the sender. Thank you.

Irving, Dave

2005-10-25, 7:45 am

Along sort of related lines....
When I've got something demonstratable (which shouldn't be too long),
I'd really like to get this open-sourced. I think it'd be one of the
first - if not the first - Open Source Java asynchronous implementation
of HTTP. Id would be interesting to see if it got uptake for AJAX etc.
As its based on MINA, I'd really like to keep it @ apache if possible -
and I'd need a sponsor for a sandbox project. Is that something you
might consider doing - or do you know anyone that might?

Many thanks,

Dave
________________________________

From: Trustin Lee [mailto:trustin-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]=20
Sent: 25 October 2005 12:38
To: Apache Directory Developers List
Subject: Re: [mina] Writing to an IoSession

Hi Dave,

2005/10/25, Irving, Dave <dave.irving-iKsOTpgdUR76V6G2DxALlg@public.gmane.org>:=20

Its still early days at the moment :o)
Basically, I work on a product which is used as a gateway in
some
scenarios (e.g, receive an multi-media message, send it out to
another
(off-board) application, and return a response to the original
client=20
when we receive a response from the application.
In this example latency can be high (up to 20 secs per request).
=09
So, what I needed was high-throughput even at high latency.
In tomcat, this means having a very large number of processor
threads -=20
(which can obviously cause scalability problems).

Most web applications nowadays uses external resources such as JDBC
connection, so this means that MINA-based (NIO-based strictly speaking)
HTTP server can outperform in overload situation.=20=20

So what Im doing is coming up with a through-and-through
asyncronous API
for web applications, and a (NIO driven) web server to host such

applications.
=09
At the moment, its still early days. But I can give you an
example of a
test I ran this morning:
=09
=09
=09
________________________________________
________________________________
___
/ | Client Threadds | Server Threads | Requests |=20
Requests/second \
=09
|_______________________________________
________________________________
____|
|AsyncWeb | 50 | 21 | 50,000 |
607
|
|Tomcat | 50 | 50 | 50,000 |
471=20
|
=09
\_______________________________________
________________________________
____/
=09
Tomcat had a simple Servlet configured which just gave a "hello
world"
reply to each HTTP request.
Likewise, AsyncWeb had a simple AsyncHttpService configured
which gave=20
the same "hello world" replies.
No connection keep alives were used (that comes later...).

The result shows 28.8% throughput improvement with less than half number
of threads. Sounds cool.=20

Question:
=09
Could you possibly tell me where to look to find out how to
control the=20
number of threads created for processing MINA events? I noticed
that I
was receiving call-backs from MINA on 21 threads....

Are you using SimpleServiceRegistry?

You have to get an instance of IoThreadPoolFilter and call
setMaximumPoolSize() event:=20

ServiceRegistry registry =3D new SimpleServiceRegistry();

IoThreadPoolFilter threadPoolFilter =3D ( IoThreadPoolFilter )
registry.getIoAcceptor( TransportType.SOCKET ).getFilterChain().get(
"threadPool" );=20
threadPoolFilter.setMaximumPoolSize( 10 );

HTH,
Trustin

--=20
what we call human nature is actually human habit
--
http://gleamynode.net/=20

This e-mail and any attachment is for authorised use by the intended recipi=
ent(s) only. It may contain proprietary material, confidential information =
and/or be subject to legal privilege. It should not be copied, disclosed to=
, retained or used by, any other party. If you are not an intended recipien=
t then please promptly delete this e-mail and any attachment and all copies=
and inform the sender. Thank you.