CL 6.0: hcismat issue ‘bad vfariable name’

Jim,

We’re not up on 6.0 yet, but I did get some sample SMAT files from the vendor so we can be prepared. The OFFSET and LENGTH values from the idx file will still give you the messages you want without the new stuff that is tacked onto the end of each message.

Cheers.

Jim:

Any chance I can get a copy of the SMAT file.  I can’t duplicate the error

This was written with 6.0 in mind and I have never had a problem with it

I think I found it Jim.  It seems that getopt acts differently in Tcl 8.5 and Tcl 8.4.  Try this one and let me know.

I still need to work on making it work with old style SMAT files

If on Unix just remove the .htc

Was hoping to normalize my date/time range for a more accurate compare between smat files.  Looking for the correct syntax for hcismat.

Using something like this:

hcismat -i to_somewhere_adt.obsv.141109.1.idx -nl -sds 201411091200 -sde 201411091700 -oruf /hci/cis6.1/integrator/somewhereprod/data/61_to_somewhere_adt.nl

Not having much luck with the date/time range.  Any suggestions?

It does write out all the messages and not the specified range.  I have tried changing oruf to orsf which returns nothing.

Kieth:

I rewrote hcismat as the old version depended on using Tk which does not work any more.  I tried to keep it the same as before only adding the -nl flag.

I tested dates with a SMAT file of my own and it seems to work OK for me.  Could it be that you used the -oruf flag that says you want unselected records? If you want the selected records, use -orsf.

I tested it using 5.8 since that is where I wrote it.  I seem to be having a problem with running using my Cloverleaf 6.0.  I get the error message:

hcismat: Invalid argument bad variable name

“::SMAT::argArray(TARGET)”: upvar won’t create a scalar variable that looks like an array element

It looks like the optargs routine I use to parse arguments may have changed with Tcl  8.5

Will someone that has both the hcismat script and 6.0+ give this  a try and see if you get the same error?  I just want to make sure I am not spoofing myself before I chase this too much.

I am using the version downloaded from above jun 17 2013, I believe.  I am on CLoverlaf 6.1.  I was able to run it on 5.7 and 6.1 SMATS, just not for a selected date and time.

I just ran against all my SMATS across sites using this command inside a script. Similar for 57.

hcismat -i ${smatfile} -nl -orsf ${in}/${curDir}/data/61_${filebase}_${dt}.nl -sall

The date still eludes me.  I have used both oruf and orsf.  I have even thrown in -sdn.  It looks like whether sdn is used or not it saves to the SELECTED array, before you use intersect3 for res.  I haven’t spent a lot of time on it yet…

I just figured if I could take a range of time where I knew both SMAT files were relatively equal, I could compare messages more easily…

You know I appreciate all the help you have provided.  I will be looking in the morning but need to keeep moving forward on my compares….deadlines loom.

Strange!  I am able to select a range of dates using mine/  I will download he one I uploaded before to see if differences

Kieth:

Did some more check on the one I downloaded from here.  First I no longer get error under 6.0 – very strange.

Next I found I had forgotten to change the time from SMAT from offset to YYYYMMDDHHMM before compare.  I have changed this one.  I will continue to test here.  I appreciate any further testing you do with this

Most people just use it as a quick and dirty to convert SMAT to NL but it should be more than that

Thanks

Hi Charlie/Keith,

I have been looking at hcismat as well, as we are getting 12 MB messages (=files) in SMAT that we can’t open in the GUI.

I tried this last version, but it still won’t work. I had a look and put in some echo’s:

The SMAT contains 3 messages, 1 message each day.

So the first try should give me 1 message: the message of today. But because the startdate is only a date (YYYYMMDD), it will be always smaller than a complete timestamp (YYYYMMDDHHMMSS).

In tries 2 and 3 the enddate is given and it should work, but the ‘clock format $time’ is missing the %S, so time is always smaller than startdate and no messages are returned. Adding %S will make it work if you

I think the best thing to do is ‘clock scan’ all input-dates into seconds and compare those, but you will need to take into account that only dates can be input instead of complete timestamps. I haven’t got the time to do this myself right now; will keep you updated if I find some time.

Did you try the new one  I just posted?

Hi Charlie,

Yes, this was tested with the new one.

When you give a start and no end date the end date will default to something like the year 2045.  It expects a timestamp not a date.  If you wanted everyting since 20141111 just input 201411110000.  If you want only that date then input 201411110000 – 201411112359

Won’t that work for you?

You will need to change

into

so with %S in the time format.

If you do this then yes, inputting a complete timestamp will work:

However (and don’t shoot the messenger…) the ‘help’ mentions that it is possible to input a date, without a time.

I made this change.  I think this works, but will run a few more tests.

regsub — {(d{8})(.*)$} $START {1 2} s_start

regsub — {(d{8})(.*)$} $END {1 2} s_end

           if {$time >= [clock scan $s_start] && $time <= [clock scan $s_end]} {

So far I got the correct results.

Hi Keith,

If you don’t specify either the start or end, then these will be defaulted to a ‘seconds’ value (0 resp. 2145852000). Then your regsubs will go wrong, right?

And Charlie: I see now that you mentioned the start/end dates in the form ‘YYYMMMDDHHMM’, without the seconds. Used like this,  the last version will work. But this also deviates from what is mentioned in the ‘usage’ of this script.

I may be nitpicking though…

I thought the usage statement listed it as a time stamp.  I will take a look and change as required

Let me know of any needed enhancements and I will consider them

Just trying to make an old command usable in today’s environment  😀

I set START to default to 000000000000 so the criteria is met.  Not ssure how much effort to make old smat files from 5.7 work.  That would be helpful especial using the START and END Date/Times.  It looks like in some of the code lgets is used instead of gets in the while loop to accomodate the stanza structure of the .idx file.  I ran the command manual and it appeared to work, however only pulled ‘{‘ more often than not.  The code would exit where the one of the three keys is blank. This is in the selectDate proc.

# If no OFFSET or LENGTH or TIMEIN bad file

  if {$off eq “” || $len eq “” || $time eq “”} {

   puts stderr “n$myName: Invalid or corruted SMAT file

$argArray(SMATFILE).idx”

puts stderr “Exit ……………n”

exit -64

   }

I stand corrected.  I forgot the time was optional.  I guess I never did test the dste stuff enough – sorry bout that.

While in there I found a couple of other “undocumented enhancements”  😀

Plus it seems the clock command will not scan a date past 20371231. I did not know that.  I will query the Tcl gurus about that.

Here is my latest.  Please feel free to put it through the wringer and make suggestions or comments, even snide comments.

As always for UNIX simply remove the .htc suffix make it executable and owned by hci.  If on pre-6.0 you will have to replace the one in the $HCIROOT/bin directory with this one.   On 6.0+ you could just ut in the contrib directory

I don’t have any pre-5.8 SMAT files to test with or I would see about making the script work with those.  I will look around and see if I can find some.   If you have any that do not contain PHI data send me some

I should have checked.  I was testing using 5.8

The scan limitaton with the clock command only applies to pre-Tcl 8.5 – Cloverleaf 5.8 or less.  You can scan well into the next century with the new clock command

Hi Charlie,

Just tested with dates (full timestamps and only dates), mids, not, sending (un)selected, newline or len10. Everything works!

Only thing I haven’t tested is resending the messages to a thread.

Thanks!