Memory leak with the TCL CSV package

[Richard Hart]

Hi All.

We have an interface that takes a ‘multi-row’ CSV file and loads this into an XML structure. I am using the CSV package to sensibly split the CSV file and keep my code ‘simple’.

In testing (and not caring that the output is different!) with 10 data files, if I change

   set mySplitList [csv::split [msgget $aMsgId]]

to

   set mySplitList [split [msgget $aMsgId] ,]

there is a reduction in memory usage of about 1 meg, so the CSV package ‘internals’ appear to be at fault.

Has anyone else used the CSV package, is there a cleanup operation I should be completing?

Thanks.

Topic

[Charlie Bursell]

I use this package a lot and have not seen a memory leak. I don’t see anything in the package that will cause this behaviour but I normally use it for one record at a time and not for an entire message.  You did not state how large the message is.

One thing you can try is the -alternate flag.  That will cause it to use a different parser algorithm

Like set mySplitList [csv::split -alternate [msgget $aMsgId]]

(No spaces in -alternate)

It is just a Tcl proc located in $HCIROOT/tcl/lib/tcllib1.x/csv/csv.tcl

You could add some Debug statements

Also, if push come to shove, you can send an e-mail to Jeff Hobbs at ActiveState or Andreas Kupries <andreas_kupries@users.sourceforge.net).  Both are usually pretty good about answering

[Richard Hart]

Hi Charlie.

Thanks for the reply. I’ve added further debug and there is NO issue with the CSV package. I’ll create a different post about the new leak after some more investigation.

As an FYI, the large CSV files are around 90k and contain around 300 pathology results.

[Author]

Replies