|
Home > Archive > Apache Directory Project > March 2007 > [Integration tests] How to speed up the tests
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
[Integration tests] How to speed up the tests
|
|
| Emmanuel Lecharny 2007-03-23, 1:11 pm |
| Hi,
with the new schema handling system (the schema is now stored as Ldap
entries in the server, and loaded at startup), the integration tests are
taking forever (17 mins on my 3GhZ desktop, 13 minutes on my shinny Imac
dual-core).
The reason is that all the schema elements have to be read out of the
server, and as the server is able to manage around 500 req/s (more or less),
each test cost around a second, bare minimum.
As we have around 150 tests in server-unit, this is around 3 minutes to run
integration tests, and we have the double in core-unit, so it's 6 more
minutes.
We have a few options :
1) Improve the server so it can handle 5000 req/s, dividing the tests
duration by ten : yeah, that would be cool, but...
2) Improve the tests to limit the number of server startup and shutdown to
one by test (the setup should contain a test to check if the server has been
started or not before, and add code to feed the server and some other code
to clean the server)
3) Write a new tool which will register all the operations to play them in
revert order when finishing a test
4) any other smart idea, guys ?
--
Cordialement,
Emmanuel Lécharny
www.iktek.com
| |
| David Jencks 2007-03-23, 1:11 pm |
|
On Mar 23, 2007, at 1:09 PM, Emmanuel Lecharny wrote:
> Hi,
>
> with the new schema handling system (the schema is now stored as
> Ldap entries in the server, and loaded at startup), the integration
> tests are taking forever (17 mins on my 3GhZ desktop, 13 minutes on
> my shinny Imac dual-core).
>
> The reason is that all the schema elements have to be read out of
> the server, and as the server is able to manage around 500 req/s
> (more or less), each test cost around a second, bare minimum.
>
> As we have around 150 tests in server-unit, this is around 3
> minutes to run integration tests, and we have the double in core-
> unit, so it's 6 more minutes.
>
> We have a few options :
> 1) Improve the server so it can handle 5000 req/s, dividing the
> tests duration by ten : yeah, that would be cool, but...
> 2) Improve the tests to limit the number of server startup and
> shutdown to one by test (the setup should contain a test to check
> if the server has been started or not before, and add code to feed
> the server and some other code to clean the server)
(2) is a junit TestSetup IIRC? I might not remember the name but
there's an easy way to have setUp and tearDown methods called once
per class or if you code a Suite once per suite. I think this is
what you're looking for here. The setUp and tearDown methods in the
regular test class still get called once per test.
thanks
david jencks
> 3) Write a new tool which will register all the operations to play
> them in revert order when finishing a test
> 4) any other smart idea, guys ?
>
> --
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
| |
| Emmanuel Lecharny 2007-03-23, 7:11 pm |
| >
> (2) is a junit TestSetup IIRC? I might not remember the name but there's
> an easy way to have setUp and tearDown methods called once per class or if
> you code a Suite once per suite. I think this is what you're looking for
> here. The setUp and tearDown methods in the regular test class still get
> called once per test.
>
I gonna test this guy, seems pretty cool !
Thanks !
--
Cordialement,
Emmanuel Lécharny
www.iktek.com
| |
| Alex Karasulu 2007-03-27, 1:11 pm |
| Just watch out for tests that may colide. For example some tests will add
stuff into the server then count and assert the number of entries. Some
tests will delete entries. All these tests presume a pristine setup with a
fresh server. So basically we're going to need to cleanup for each test.
Some tests presume custom setups. So this is not such a clean and simple
approach with what we have. Although there the techique will work it will
take some labor to get us there.
Alex
On 3/23/07, Emmanuel Lecharny <elecharny-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> (2) is a junit TestSetup IIRC? I might not remember the name but
>
> I gonna test this guy, seems pretty cool !
>
> Thanks !
>
>
> --
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
| |
| Emmanuel Lecharny 2007-03-27, 7:11 pm |
| Alex Karasulu a écrit :
> Just watch out for tests that may colide. For example some tests will
> add
> stuff into the server then count and assert the number of entries. Some
> tests will delete entries. All these tests presume a pristine setup
> with a
> fresh server. So basically we're going to need to cleanup for each test.
That's true. I have looked at this method, but when I saw the
implications, I just step back.
>
> Some tests presume custom setups. So this is not such a clean and simple
> approach with what we have. Although there the techique will work it
> will
> take some labor to get us there.
We nned something a little bit different. Most of the tests we have
should use this kind of process :
1) initialize data
2) do the test
3) restore the initial state.
For instance, if we want to test a DelRequest, the step (1) would simply
inject the data, then (2) will remove it, and (3) will be empty.
Pierre-Arnaud is worling on such tests that use DSML, because you can
write them in one single file containing the AddRequest, the
RemoveRequest and the SearchRequest to be sure we don't have the removed
entry anymore.
Wait and see 
Emmanuel
| |
| Alex Karasulu 2007-03-29, 1:11 am |
| Emmanuel,
On 3/27/07, Emmanuel Lecharny <elecharny-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Alex Karasulu a écrit :
>
> test.
>
> That's true. I have looked at this method, but when I saw the
> implications, I just step back.
Man I don't blame you .
> simple
>
> We nned something a little bit different. Most of the tests we have
> should use this kind of process :
> 1) initialize data
> 2) do the test
> 3) restore the initial state.
Exactly this would work effectively restoring the initial state without
requiring the stop, cleanup, and restart process.
> For instance, if we want to test a DelRequest, the step (1) would simply
> inject the data, then (2) will remove it, and (3) will be empty.
> Pierre-Arnaud is worling on such tests that use DSML, because you can
> write them in one single file containing the AddRequest, the
> RemoveRequest and the SearchRequest to be sure we don't have the removed
> entry anymore.
>
> Wait and see
Hey this would be a slick replay feature for something like LS btw. I know
that's OT on this thread but it's a cool thought.
Alex
|
|
|
|
|