hcismatdb

[Charlie Bursell]

Some of you are probably aware that some time back I rewrote the old hcismat command line utility to work with current SMAT files.

Topic

[James Cobane]

Charlie,

It’s always great to see your posts and always appreciated!  Hope retirement is treating you well 🙂

Jim Cobane

Henry Ford Health

[Charlie Bursell]

Thanks Jim  all is well

[Keith McLeod]

Thanks Charlie.  I will give it a kick around the tires.  I just converted one of my upgrades to use SMATDB vs idx/msg.  I borrowed a few procs from other scripts and wrote hcicyclesmatdb in tcl again using most of the same flags as hcicyclesavemsgs.  I have also included -x to use with -a and exclude some sites.  I have had this running on Linux for over a month now to allow the daily cycling of SMATDB files and purge beyond a certain age.

Of course, I have been exploring the built in controls  and although they may not cycle every day, effectively the search across smatdb’s allows you to accomplish the same search without having to find the specific file….

I will let you know what I find on the hcismatdb…on Linux…

[Keith McLeod]

First pass on hcismatdb was successful on Linux.

Renamed file from hcismatdb.htc to hcismatdb

Changed permissions to 755

Removed r using regsub

Tested one file of ACK messages:

hcismatdb -i $HCISITEDIR/exec/processes/adt/SmatHistory/mil_adt_in.obsv.20150727235906.smatdb -nl -orsf $HCISITEDIR/data/61_mil_adt_in.obsv.20150727.nl -sall

hcismatdb: 20319 messages selected!

Does it do any kind of join by using a wildcard?  I am going to test a date range next.

[Keith McLeod]

I was able to update my scripts that use hcismat by simply changing the arguments of my find command to use the .smatdb extension for 6.1 in the search.

[Charlie Bursell]

Keith

I don’t what you mean by “join by wildcard”

The query returns a list of messages.  If -nl is selected it just does a

puts $fd [join $MSGS n]   Pretty simple stuff

Of course if length encoded is requested it has to loop through each message to prepend the 10 bytes of length

[Keith McLeod]

Let’s say the date range crosses multiple files for a given thread.  I would like to be able to use a wildcard for the file description like

hcismatdb -i $HCISITEDIR/exec/processes/adt/SmatHistory/mil_adt_in.obsv.*.smatdb -nl -orsf $HCISITEDIR/data/61_mil_adt_in1.obsv.nl -sds 20150725070000 -sde 20150727080000

I suppose I can have it cycle through each file and append to the output…No matter what… Thanks I use these tools frequently for Upgrades etc…..

[Charlie Bursell]

That may be doable Keith.  Let me take a look

[Charlie Bursell]

Kieth

it was not as hard to do as I thought

Try this.  It will allow you to specify wild cards for the file names

However, if one DB is bad the script will fail for all

Give it a try, let me know what you think.

[Keith McLeod]

Results using the wildcard.

hcismatdb1 -i mil_adt_in.obsv.*.smatdb -nl -orsf $HCISITEDIR/data/61_mil_adt_in1.obsv.nl -sds 20150725070000 -sde 20150725080000

can not find channel named “DB”

   while executing

“close DB”

   (procedure “SMAT::selectMsgs” line 83)

   invoked from within

“SMAT::selectMsgs”

   (procedure “hcismatdb” line 10)

   invoked from within

“hcismatdb $argv”

   (file “/hci/cis6.1/integrator/contrib/hcismatdb1” line 1224)

[Charlie Bursell]

I knew i should have tested more

Change that to $DB

[Keith McLeod]

At first glance I suspect the wildcard did not get expanded.  Not sure though…

This would come in handy if the GUI would allow for this type of selection as well so we could more easily focus on the files representing the Outbound context for example of a thread….

[Charlie Bursell]

I will do more testing this weekend

[Charlie Bursell]

Keith or anyone else in being a guina guinea  pig:

Sorry about the mess I sent last time.  After I added capibility to do multiple files I should have done more testing.  Since I was in a loop I was stepping on some of my own variables.  Please try again.

It will do multiple files as long as the string meets normal file globbing.  One way to test is to do an ls or dir with same string.  If it works there it will work in the script.

Thanks ahead for anyone willing to test.  I hope you find it useful

Feel free to provide feedback or questions

[Keith McLeod]

Thanks Charlie.  Will try sometime this morning.

[Keith McLeod]

It appears to stop on the first file if no messages found.  Here is my test result.

hcismatdb -i ./mil_adt_in.obsv.*.smatdb -nl -orsf $HCISITEDIR/data/61_mil_adt_in1.obsv.nl -sds 20150725070000 -sde 20150725080000

hcismatdb: No records found in ./mil_adt_in.obsv.20150703235904.smatdb!

hcismatdb: No messages selected!

[Charlie Bursell]

Did you try an ls command on the glob string?  It seems to work well here on my Windows box.  You may have to put the glob string in quotes so the shell does not try to interpret the * character before passing to the script

I don’t have Unix here.  Try and put an echo just before the glob command in the script to see what it is passed

I will look some more at my end

[Charlie Bursell]

Keith:

Just tried it again on Windows, no problem.  Maybe a Unix thing.  I will try to get on a Unix box somewhere

C:cloverleafcis6.1integratorcbtestexecprocessescbtest>hcismatdb  i .testibSmat.*.smatdb -orsf foo  -sds 20150730161654 -sde 20150732161654

hcismatdb: No records found in ./test/ibSmat.20150728.smatdb!

hcismatdb: No records found in ./test/ibSmat.20150729.smatdb!

hcismatdb: No records found in ./test/ibSmat.20150730.smatdb!

hcismatdb: No messages selected!

[Keith McLeod]

Charlie,

I think I found the difference.  UNIX expands the glob on the command line.

So I ran this…

hcismatdb -i ./mil_adt_in.ibsv.*.smatdb -orsf test_output.nl -sds 201507300700 -sde 201507300800

and got:

…

hcismatdb: No records found in ./mil_adt_in.ibsv.20150715235904.smatdb!

hcismatdb: No records found in ./mil_adt_in.ibsv.20150713235903.smatdb!

hcismatdb: No records found in ./mil_adt_in.ibsv.20150719235902.smatdb!

hcismatdb: 834 records found in ./mil_adt_in.ibsv.20150730235904.smatdb

hcismatdb: No records found in ./mil_adt_in.ibsv.20150803235903.smatdb!

hcismatdb: No records found in ./mil_adt_in.ibsv.20150726235902.smatdb!

…

[Charlie Bursell]

I think you could accomplish the same thing by putting the file path in quotes

[Keith McLeod]

That worked.  Thanks…

hcismatdb -i “./mil_adt_in.ibsv.*.smatdb” -orsf test_output.nl -sds 201507300700 -sde 201507300800

hcismatdb: No records found in ./mil_adt_in.ibsv.20150801235903.smatdb!

hcismatdb: No records found in ./mil_adt_in.ibsv.20150708235903.smatdb!

hcismatdb: No records found in ./mil_adt_in.ibsv.20150722235904.smatdb!

hcismatdb: No records found in ./mil_adt_in.ibsv.20150707235904.smatdb!

…

[Charlie Bursell]

Good news Keith.  Anyone else find this useful?

Kieth send me your e-mail address

cbursell@fastem.com

For anyone else that would like to contact me, feel free to use this address

[Charlie Bursell]

Sorry for some premature posts of this script.  I hope it is better tested and enhanced now.

I know that CL 6.1.1.1  (I think) now allows one to search multiple SMAT databases and will generate a regular expression.

For those that sometime prefer to use the command line for

[Darcy Kemp]

Hi, it may be an oldie, but I have am familiar with using these.  May I get a copy of these from someone that Charlie wrote in 2001?  Will it work with 5.7.3?  Thanks.

[Charlie Bursell]

Here is the hcismat script.  It should work just fine on 5.7.3

Let me know if problems

Run hcismat with no arguments for help screen

[Keith McLeod]

hcismatdb syntax

Here are a few syntax snippets for hcismatdb.

# hcismatdb command syntax

# Write file to newline all messages

hcismatdb -i $HCISITEDIR/exec/processes/adt/SmatHistory/mil_adt_in.obsv.20150727235906.smatdb -nl -orsf $HCISITEDIR/data/61_mil_adt_in.obsv.20150727.nl -sall

# Write file to newline using date range

hcismatdb -i $HCISITEDIR/exec/processes/adt/SmatHistory/mil_adt_in.obsv.20150727235906.smatdb -nl -orsf $HCISITEDIR/data/61_mil_adt_in1.obsv.20150727.nl -sds 20150727070000 -sde 20150727080000

# Write file(s) to newline file using glob(Notice ‘”‘ around glob pattern)

hcismatdb -i “$HCISITEDIR/exec/processes/adt/SmatHistory/mil_adt_in.i*.smatdb” -nl -orsf $HCISITEDIR/data/mil_adt_in_smatdb.nl -sall

# Write file(s) to newline file using glob(Notice ‘”‘ around glob pattern, had to remove -sall flag when using regx, also single quotes around regular expression)

hcismatdb -i “$HCISITEDIR/exec/processes/adt/SmatHistory/mil_adt_in.i*.smatdb” -regx ‘TEST^NICOLE’ -nl -orsf $HCISITEDIR/data/mil_adt_in_smatdb.nl

Hope this helps anyone using smatdb files that needs the command line.

[Peter Heggie]

Yes that helps, especially the regx argument.

Peter Heggie

[Charlie Bursell]

Just read the script.  I think it is pretty well documented at the as to usage, etc.

[Author]

Replies