| Joachim Worringen 2005-12-16, 5:48 pm |
|
[ Announcement of a GPL'ed software toolkit ]
Anyone who needs to do performance evaluation or similar tasks probably knows
the tedious task of gathering, managing, processing, analysing and visualising
the outcome of the related tests. This applies even more if it is not a one-time
evaluation, or if the results between identical tests vary significantly.
We at NEC C&C Research Labs, St.Augustin, Germany, are in this situation, too.
Recently, we have released 'perfbase' under GPL. 'perfbase' is a toolkit
designed for this sort of experiment management and analysis that we use to
solve the aforementioned problem.
'perfbase' allows to freely define "experiments" consisting of parameter and
result values. It can process arbitrary text files (like benchmark results,
application output etc.) to extract data as input for "runs" of the experiment.
The data of all runs can than be managed, analysed, related and visualized in a
manner which is consistent over time, statistically firm and reproducible. This
makes 'perfbase' a useful tool for anyone who regularly need to assess and
document the performance of arbitrary systems.
A typical usage scenario for 'perfbase' is a benchmark that is run on different
platforms with varying parameters. However, due to its generic design,
'perfbase' can be applied to arbitrary experimental setups which rely on input
from text files. It is also possible to integrate perfbase into an automated
test/evaluation environment, i.e. for nightly performance tests.
'perfbase' consists of a set of Python scripts, uses XML for experiment, input
and query descriptions, and relies on PostgreSQL for a back end. It is stable
software which is used on a daily basis and comes with several examples.
For more information, documents and download, please visit
http://perfbase.tigris.org
--
Joachim - reply to joachim at domain ccrl-nece dot de
Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
--
|