gdbm

[Yves Guerin]

Hello,

I would like to know the advantages to use gdbm extension (tcl) to store information over a keylist (with keylset): keylset taglist toto123456 {adtOut1 adtOut2}

I need to store about 100000 and over keys.

Thank you in advance for your help.

Yves

Topic

[David Barr]

The biggest advantage of GDBM is probably persistence.  The contents of keyed lists go away when the thread is shut down.  Gdbm is written to disk.

The entire contents of keyed lists are kept in memory, so look-ups are probably faster than GDBM look-ups.

[Yves Guerin]

Hello,

Persistence is not a problem, because I got some ADT_A08 message who is used to tag and untag my patient, so my tagging package will add or remove information directly in the keyed list in memory and write the change to a file (only when tagging and untagging).  So when the thread is restarted I restore the keyed list from the file.

I did some tests to load a 99000 item into a keyed list and from a gdbm file, It took about 10 minutes for the keyed list to be build from file and 12 minutes for the gdbm file.

The keyed list file is about 4.4MB and the gdbm file is about 8.4MB, so the gdbm file is twice the size and it took about 4 minutes more to load

My next step is to do some menory check the use of the keyed list and the gdbm file.  Because I would like to know which solution is friendly with the memory and the fastest (I check very hl7 message passing through my interface).

Thank you for your help

Yves

[David Barr]

For what you’re trying to do, GDBM sounds better to me.

You won’t have to write code to load your data when you start your thread, and saving a value with gdbm_set (or whatever the function is) sounds a lot more efficient than a keylset followed by writing the entire table to disk when only one value changes.

Also, you won’t have to keep the entire contents of your table in memory. With such a small table, this reason probably doesn’t matter as much.

[garry r fisher]

Hi Yves,

I regularly use gdbm to store data I want for use later in a process or for verifying returned data from a third party system. The impact of using gdbm on performance appears to be minimal and works out of the box.

Alternatively you could use sqlite – This would be an ideal application for the sqlite but requires a bit more work to setup and implement.

Regards

Garry

[Levy Lazarre]

Yves,

You may want to take a look at the Metakit for Tcl. It is much more powerful and faster than GDBM.

http://www.equi4.com/metakit/tcl.html

Mk4tcl is great for storing and managing data if you don’t need a full SQL database.

[Yves Guerin]

Dear Levy,

Thank you to remember me the mk4tcl, but I did some testing and it was very slow to retrieve information, that why I stick with the keyed list or to use sqlite3.

Regards and thank you for your help

Yves

[Yves Guerin]

Dear Garry,

The big problem is to compile the extension for aix, I tried many time without any success  😥

Regards and thank you for your help

Yves

[Yves Guerin]

Dear David,

David Barr wrote:

For what you’re trying to do, GDBM sounds better to me.

You won’t have to write code to load your data when you start your thread, and saving a value with gdbm_set (or whatever the function is) sounds a lot more efficient than a keylset followed by writing the entire table to disk when only one value changes.

Also, you won’t have to keep the entire contents of your table in memory. With such a small table, this reason probably doesn’t matter as much.

Right on target, I realized that to save a 100000 keyed list to a file will be a dead end for my interface.  Anyway, at start-up, the interface took 12 minutes to charge the gdbm file vs 9 minutes for the keyed list but I use the gdbm_reorganize to clean up the gdbm file.  I do not care about the start-up time but the save time is more important and may be the memory usage too (tests and tests are everything).

Regards and thank you for your help  😉

Yves

[Author]

Replies