|
Home > Archive > Apache Mod-Python > February 2006 > Refactoring of the test suite
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Refactoring of the test suite
|
|
| Nicolas Lehuen 2006-02-10, 7:46 am |
| Hi,
There is something I'd like to do for the 3.3 version : it is to
refactor the test suite. It's more a chore than real development, but
the current test suite is slowly becoming big and quite difficult to
maintain.
What I'd like to do is simply split the test runner and the published
tests in various parts, matching the different functionalities that
are tested.
What would be great, then, is to introduce platform-specific tests, of
even MPM-specific tests, so that we could exercise some parts of the
code that are only available with a threaded MPM (for example, the
MemorySession implementation, or the publisher reload test which I
erroneously implemented with a threaded MPM in mind).
One thing that I'd like to settle with you is whether you are OK to
split the big PerRequestTestCase class into multiple classes, to ease
maintenance and configuration switches. This has the downside that
Apache server is likely to be stopped and restarted a few more times
during the tests, so the tests will take more time to run. On the
other hand, I'd like to introduce a way to select the tests to run
when launching the test suite, so that we won't have to wait for the
whole test suite to pass while debugging a given problem.
Ah, and one thing I'd definitely get rid of - usage of backquotes like
in `rsp`. I really don't like this magic quoting thingy, it reminds me
all too much of php  .
Like Graham says : "comments ?"
Regards, Nicolas
| |
| Graham Dumpleton 2006-02-10, 5:47 pm |
| My only comment is that, especially if tests are being split into
separate files, that the access/error log files be distinct for each
test with a name incorporating the name of the test. I always
found it a pain to try and dig through a big log of multiple tests
trying to work out which was the output from the one that was
failing.
Graham
On 11/02/2006, at 2:23 AM, Jim Gallacher wrote:
> Nicolas Lehuen wrote:
>
> Also, I think the size and complexity may be intimidating to new
> contributors. I don't mind so much now that I'm familiar with it but
> when you first dive in it looks like a lot of code. We want the tests
> themselves to be as robust as possible and if refactoring helps I'm
> all
> for it. We've had a few problems where the a unit test was
> incorrect and
> as a result was missing mod_python problems.
>
>
> +1
>
>
> I think this is a good idea, and I've been thinking along the same
> lines
> in light of the changes that are necessary to support apache 2.2 (ie,
> different test configurations required for 2.0 and 2.2). I'm not too
> concerned about taking more time for the tests to complete.
> Anything we
> can do to make the tests more reliable and easier to write will be
> worth
> the extra testing time and the initial refactoring effort.
>
>
> Well, we can't have that now can we?
>
> I don't know if this would be practical, but one idea I have is to put
> each test in it's own file in a test_modules direcotry. This directory
> would be scanned on test startup and the tests added dynamically. That
> way if someone contributes some new code, they can also provide the
> corresponding unit test without worrying about editing the unwieldy
> test.py and htdocs/tests.py files. All they need to do is create
> test_modules/mytest.py and htdocs/mytest.py files which get dropped
> into
> the test framework with no additional configuration required.
>
> Oh and if we are refactoring the tests, I want a "make tests" rule.
> I'm
> tired of doing: ./configure; make; sudo make install; make tests;
> DOH!
> cd test; Python test.py.
>
> Jim
| |
| Jim Gallacher 2006-02-10, 8:45 pm |
| Nicolas Lehuen wrote:
> Hi,
>
> There is something I'd like to do for the 3.3 version : it is to
> refactor the test suite. It's more a chore than real development, but
> the current test suite is slowly becoming big and quite difficult to
> maintain.
Also, I think the size and complexity may be intimidating to new
contributors. I don't mind so much now that I'm familiar with it but
when you first dive in it looks like a lot of code. We want the tests
themselves to be as robust as possible and if refactoring helps I'm all
for it. We've had a few problems where the a unit test was incorrect and
as a result was missing mod_python problems.
> What I'd like to do is simply split the test runner and the published
> tests in various parts, matching the different functionalities that
> are tested.
>
> What would be great, then, is to introduce platform-specific tests, of
> even MPM-specific tests, so that we could exercise some parts of the
> code that are only available with a threaded MPM (for example, the
> MemorySession implementation, or the publisher reload test which I
> erroneously implemented with a threaded MPM in mind).
+1
> One thing that I'd like to settle with you is whether you are OK to
> split the big PerRequestTestCase class into multiple classes, to ease
> maintenance and configuration switches. This has the downside that
> Apache server is likely to be stopped and restarted a few more times
> during the tests, so the tests will take more time to run. On the
> other hand, I'd like to introduce a way to select the tests to run
> when launching the test suite, so that we won't have to wait for the
> whole test suite to pass while debugging a given problem.
I think this is a good idea, and I've been thinking along the same lines
in light of the changes that are necessary to support apache 2.2 (ie,
different test configurations required for 2.0 and 2.2). I'm not too
concerned about taking more time for the tests to complete. Anything we
can do to make the tests more reliable and easier to write will be worth
the extra testing time and the initial refactoring effort.
> Ah, and one thing I'd definitely get rid of - usage of backquotes like
> in `rsp`. I really don't like this magic quoting thingy, it reminds me
> all too much of php .
Well, we can't have that now can we?
I don't know if this would be practical, but one idea I have is to put
each test in it's own file in a test_modules direcotry. This directory
would be scanned on test startup and the tests added dynamically. That
way if someone contributes some new code, they can also provide the
corresponding unit test without worrying about editing the unwieldy
test.py and htdocs/tests.py files. All they need to do is create
test_modules/mytest.py and htdocs/mytest.py files which get dropped into
the test framework with no additional configuration required.
Oh and if we are refactoring the tests, I want a "make tests" rule. I'm
tired of doing: ./configure; make; sudo make install; make tests; DOH!
cd test; Python test.py.
Jim
| |
| Mike Looijmans 2006-02-13, 2:47 am |
| > Oh and if we are refactoring the tests, I want a "make tests" rule. I'm
> tired of doing: ./configure; make; sudo make install; make tests; DOH!
> cd test; Python test.py.
Make that "make check" (like autotools), to not confuse old-skool
autoconfers like myself.
| |
| Jim Gallacher 2006-02-13, 7:46 am |
| Mike Looijmans wrote:
>
>
> Make that "make check" (like autotools), to not confuse old-skool
> autoconfers like myself.
That works for me.
Jim
|
|
|
|
|