Handling very large SMAT files

[Steve Drozdowski]

Hello, we are running into situtations where the SMAT files for some specific threads can become very large.  Sometimes they are too large to open with the regular SMAT tool.  Sometimes we get lucky, and are able to use the old tk (X-Windows) hcismat tool, but not always.  In some cases we have had to resort to converting these SMAT files to newline terminated file using various scripts/tool/etc.

We are now considering having some of these SMAT files cycle 2 or more times per day (in addition to our normal schedule midnight cycle).

However, I think it would be really great if we could develop (or locate) a script that could “split up” a large SMAT file into multiple smaller SMAT files.  Does anyone have such a script, that they would be willing to share?

Thanks in advance!

Topic

[Steve Drozdowski]

Charlie, if you are reading this, I’ll bet you could whip up something in no time!   😀

[John Mercogliano]

Here is a tcl script I wrote to do what you ask.  It takes the idx filename and an optional max messages per file.  

Hope this helps

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[Steve Drozdowski]

Thanks John, I will give it a try!

[Charlie Bursell]

There are many ways to split the file but in all cases you must cycle SMAT first.  If you are going to do that why not simply cycle more often?  I don’t see the advantage of splitting a large SMAT file over cycling.

I would run a CRON job say every hour and check the size of the SMAT msg file and if over a certain size, simply cycle it.

As was stated I could certainly write something to do that but my guess is so can you.  Besides, I get paid to that  😀

[Bob Schmid]

Going back to the gui solution for a moment…..

I have found the need to increase the option for JVM size in order to load larger smat files:

on the IDE gui: options, client, advanced  “-Xmx312m” 😀

[Steve Drozdowski]

Thanks everyone.  All excellent suggestions.

[Steve Drozdowski]

Hi John, I downloaded and tested your “splitmsgfile.tcl” code, however when I try to load one of the split smat files, I receive this error message below…

Errors were encountered while reading the index and message files:

Error reading TCL code in index file: Keyed list entry 0 is not a 2 element list

0 messages were successfully read.

I am able to load the split smat file using the old tk X-Windows hcismat tool, however I am unable to close them (the tool just locks up).

Note – We are running CL 5.4.1 on AIX 5.3.

Any ideas?  Thanks again.

[John Mercogliano]

I created this for 5.2 smat files.  I also am running on HP-UX 11.11.  I don’t have access to 5.4 or there idx files so I can’t investigate to much but I’m hoping that it is the way that tcl is interpreting the EOL character in the version shipped after 8.4.  

Here is the link that discusses the issue.

http://clovertech.infor.com/viewtopic.php?t=1648” class=”bbcode_url”>http://clovertech.infor.com/viewtopic.php?t=1648

I’ve added fconfigure commands as suggested and it still works fine on 5.2.  Hopely, this one will port better.  Otherwise, the code should give you some ideas.

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[Rob Abbott]

John, this is pretty cool.  Please post a copy to the Tcl library forum — thanks!

Rob Abbott
Cloverleaf Emeritus

[Steve Drozdowski]

It still doesn’t work with 5.4.1, but thanks allot for trying.

[James Cobane]

Steve,

Attached is a utility written to read the SMAT file and produce a newline terminated output file; you can supply a ‘regular expression’ to extract specific messages (just like in SMAT).  You may need to tweak the first line for your environment.

The syntax for the tcl program is:

convert_smat.tcl “SMAT smatfilename” “FILE outputfile” “TABLE tablename” “REGEXP {ADT^A01}”

Note that the arguments are in a keyed-list format – “KEY value” – where:

SMAT smatfilename        — designates the desired Saved Message file to read (do not include the .msg or .idx)

FILE outputfilename        — designates the output file to write the newline terminated records to

TABLE tablename          — designates the conversion table to use if needed (i.e. hci_e2a for EBCDIC to ASCII conversion)

REGEXP {ADT^A01}      — designates the string to search for; must be enclosed in curly braces {}

The TABLE and REGEXP arguments are optional.

Jim Cobane

Henry Ford Health

[John Mercogliano]

Ok Steve one last time.  I think I figured out the problem.  I was assuming that the idx file was  a true list and writing it out as such.  In 5.2, I guess it is not as fickle.  We just installed 5.6 and I tried it and received the same error.  I changed the program to write out each list element on it’s own line just like it is in the original idx file and now it works fine in 5.6.  So, I’m hoping it will work fine for you also.

I’m also posting this to the tcl library forum

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[Steve Drozdowski]

Thanks James and John!!!

[Author]

Replies