Handling very large SMAT files

Homepage Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Handling very large SMAT files

  • Creator
    Topic
  • #50271
    Steve Drozdowski
    Participant

    Hello, we are running into situtations where the SMAT files for some specific threads can become very large.  Sometimes they are too large to open with the regular SMAT tool.  Sometimes we get lucky, and are able to use the old tk (X-Windows) hcismat tool, but not always.  In some cases we have had to resort to converting these SMAT files to newline terminated file using various scripts/tool/etc.

    We are now considering having some of these SMAT files cycle 2 or more times per day (in addition to our normal schedule midnight cycle).

    However, I think it would be really great if we could develop (or locate) a script that could “split up” a large SMAT file into multiple smaller SMAT files.  Does anyone have such a script, that they would be willing to share?

    Thanks in advance!

Viewing 12 reply threads
  • Author
    Replies
    • #65412
      Steve Drozdowski
      Participant

      Charlie, if you are reading this, I’ll bet you could whip up something in no time!   😀

    • #65413
      John Mercogliano
      Participant

      Here is a tcl script I wrote to do what you ask.  It takes the idx filename and an optional max messages per file.  

      Hope this helps

      John Mercogliano
      Sentara Healthcare
      Hampton Roads, VA

    • #65414
      Steve Drozdowski
      Participant

      Thanks John, I will give it a try!

    • #65415
      Charlie Bursell
      Participant

      There are many ways to split the file but in all cases you must cycle SMAT first.  If you are going to do that why not simply cycle more often?  I don’t see the advantage of splitting a large SMAT file over cycling.

      I would run a CRON job say every hour and check the size of the SMAT msg file and if over a certain size, simply cycle it.

      As was stated I could certainly write something to do that but my guess is so can you.  Besides, I get paid to that  😀

    • #65416
      Bob Schmid
      Participant

      Going back to the gui solution for a moment…..

      I have found the need to increase the option for JVM size in order to load larger smat files:

      on the IDE gui: options, client, advanced  “-Xmx312m” 😀

    • #65417
      Steve Drozdowski
      Participant

      Thanks everyone.  All excellent suggestions.

    • #65418
      Steve Drozdowski
      Participant

      Hi John, I downloaded and tested your “splitmsgfile.tcl” code, however when I try to load one of the split smat files, I receive this error message below…

      Errors were encountered while reading the index and message files:

      Error reading TCL code in index file: Keyed list entry 0 is not a 2 element list

      0 messages were successfully read.

      I am able to load the split smat file using the old tk X-Windows hcismat tool, however I am unable to close them (the tool just locks up).

      Note – We are running CL 5.4.1 on AIX 5.3.

      Any ideas?  Thanks again.

    • #65419
      John Mercogliano
      Participant

      I created this for 5.2 smat files.  I also am running on HP-UX 11.11.  I don’t have access to 5.4 or there idx files so I can’t investigate to much but I’m hoping that it is the way that tcl is interpreting the EOL character in the version shipped after 8.4.  

      Here is the link that discusses the issue.

      http://clovertech.infor.com/viewtopic.php?t=1648” class=”bbcode_url”>http://clovertech.infor.com/viewtopic.php?t=1648

      I’ve added fconfigure commands as suggested and it still works fine on 5.2.  Hopely, this one will port better.  Otherwise, the code should give you some ideas.

      John Mercogliano
      Sentara Healthcare
      Hampton Roads, VA

    • #65420
      Rob Abbott
      Keymaster

      John, this is pretty cool.  Please post a copy to the Tcl library forum — thanks!

      Rob Abbott
      Director, Product Management - Infor Cloverleaf

    • #65421
      Steve Drozdowski
      Participant

      It still doesn’t work with 5.4.1, but thanks allot for trying.

    • #65422
      James Cobane
      Participant

      Steve,

      Attached is a utility written to read the SMAT file and produce a newline terminated output file; you can supply a ‘regular expression’ to extract specific messages (just like in SMAT).  You may need to tweak the first line for your environment.

      The syntax for the tcl program is:

      convert_smat.tcl “SMAT smatfilename” “FILE outputfile” “TABLE tablename” “REGEXP {ADT^A01}”

      Note that the arguments are in a keyed-list format – “KEY value” – where:

      SMAT smatfilename        — designates the desired Saved Message file to read (do not include the .msg or .idx)

      FILE outputfilename        — designates the output file to write the newline terminated records to

      TABLE tablename          — designates the conversion table to use if needed (i.e. hci_e2a for EBCDIC to ASCII conversion)

      REGEXP {ADT^A01}      — designates the string to search for; must be enclosed in curly braces {}

      The TABLE and REGEXP arguments are optional.

      Jim Cobane

      Henry Ford Health

    • #65423
      John Mercogliano
      Participant

      Ok Steve one last time.  I think I figured out the problem.  I was assuming that the idx file was  a true list and writing it out as such.  In 5.2, I guess it is not as fickle.  We just installed 5.6 and I tried it and received the same error.  I changed the program to write out each list element on it’s own line just like it is in the original idx file and now it works fine in 5.6.  So, I’m hoping it will work fine for you also.

      I’m also posting this to the tcl library forum

      John Mercogliano
      Sentara Healthcare
      Hampton Roads, VA

    • #65424
      Steve Drozdowski
      Participant

      Thanks James and John!!!

Viewing 12 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
5,042
Forums
28
Topics
9,200
Replies
34,021
Topic Tags
267