| Barry Pearce 2005-03-30, 7:20 pm |
| Hi all,
OK I have been beavering away and now have a revised util.py tested with
a 698MB file, as well as various binaries, and text files all checked
with md5sum for accuracy.
Note: this works for files which are streamed to disk. It does not solve
the streamed to memory issue.
Ive introduced a new variable to the module readBlockSize currently
defined at 64KB. Ive also made some minor performance improvements - at
least I think they are - depending on the Python internals I figured the
sline = line.strip() would be expensive for 64KB and therefore worked
out a rough boundary size and only call the strip if its close. Anyway
my changes are fairly well confined - well marked and relatively obvious.
Where should I send the revised util.py for consideration??
(although at least I have found the right list this time!!)
Cheers,
Barry Pearce
|