gdbm

  • Creator
    Topic
  • #50997
    Yves Guerin
    Participant

    Hello,

    I would like to know the advantages to use gdbm extension (tcl) to store information over a keylist (with keylset): keylset taglist toto123456 {adtOut1 adtOut2}

    I need to store about 100000 and over keys.

    Thank you in advance for your help.

    Yves

Viewing 7 reply threads
  • Author
    Replies
    • #68344
      David Barr
      Participant

      The biggest advantage of GDBM is probably persistence.  The contents of keyed lists go away when the thread is shut down.  Gdbm is written to disk.

      The entire contents of keyed lists are kept in memory, so look-ups are probably faster than GDBM look-ups.

    • #68345
      Yves Guerin
      Participant

      Hello,

      Persistence is not a problem, because I got some ADT_A08 message who is used to tag and untag my patient, so my tagging package will add or remove information directly in the keyed list in memory and write the change to a file (only when tagging and untagging).  So when the thread is restarted I restore the keyed list from the file.

      I did some tests to load a 99000 item into a keyed list and from a gdbm file, It took about 10 minutes for the keyed list to be build from file and 12 minutes for the gdbm file.

      The keyed list file is about 4.4MB and the gdbm file is about 8.4MB, so the gdbm file is twice the size and it took about 4 minutes more to load

      My next step is to do some menory check the use of the keyed list and the gdbm file.  Because I would like to know which solution is friendly with the memory and the fastest (I check very hl7 message passing through my interface).

      Thank you for your help

      Yves

    • #68346
      David Barr
      Participant

      For what you’re trying to do, GDBM sounds better to me.

      You won’t have to write code to load your data when you start your thread, and saving a value with gdbm_set (or whatever the function is) sounds a lot more efficient than a keylset followed by writing the entire table to disk when only one value changes.

      Also, you won’t have to keep the entire contents of your table in memory. With such a small table, this reason probably doesn’t matter as much.

    • #68347
      garry r fisher
      Participant

      Hi Yves,

      I regularly use gdbm to store data I want for use later in a process or for verifying returned data from a third party system. The impact of using gdbm on performance appears to be minimal and works out of the box.

      Alternatively you could use sqlite – This would be an ideal application for the sqlite but requires a bit more work to setup and implement.

      Regards

      Garry

    • #68348
      Levy Lazarre
      Participant

      Yves,

      You may want to take a look at the Metakit for Tcl. It is much more powerful and faster than GDBM.

      http://www.equi4.com/metakit/tcl.html

      Mk4tcl is great for storing and managing data if you don’t need a full SQL database.

    • #68349
      Yves Guerin
      Participant

      Dear Levy,

      Thank you to remember me the mk4tcl, but I did some testing and it was very slow to retrieve information, that why I stick with the keyed list or to use sqlite3.

      Regards and thank you for your help

      Yves

    • #68350
      Yves Guerin
      Participant

      Dear Garry,

      The big problem is to compile the extension for aix, I tried many time without any success  😥

      Regards and thank you for your help

      Yves

    • #68351
      Yves Guerin
      Participant

      Dear David,

      David Barr wrote:

      For what you’re trying to do, GDBM sounds better to me.

      You won’t have to write code to load your data when you start your thread, and saving a value with gdbm_set (or whatever the function is) sounds a lot more efficient than a keylset followed by writing the entire table to disk when only one value changes.

      Also, you won’t have to keep the entire contents of your table in memory. With such a small table, this reason probably doesn’t matter as much.

      Right on target, I realized that to save a 100000 keyed list to a file will be a dead end for my interface.  Anyway, at start-up, the interface took 12 minutes to charge the gdbm file vs 9 minutes for the keyed list but I use the gdbm_reorganize to clean up the gdbm file.  I do not care about the start-up time but the save time is more important and may be the memory usage too (tests and tests are everything).

      Regards and thank you for your help  😉

      Yves

Viewing 7 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
4,966
Forums
28
Topics
9,106
Replies
33,627
Topic Tags
248