Dynamically Load Tables

[Amit Anand]

Guys

Is there a way to dynamically load the tables without having to bounce the whole process.

I think it was an enhancement request for future versions but I am not sure if there was any way around it other than bouncing the process. We are on Quovadx version 5.5 Rev1.

Thanks

Amit

Topic

[James Cobane]

Amit,

You can perform a ‘Purge Cache’ to force a re-load of the table without bouncing the process.  From the command line, it would be:

hcicmd -p myprocess -c “thread1,thread2,thread3 purgex”

Where myprocess is the process name, and thread1 – thread3 are all of the threadnames associated with that process.

Hope this helps.

Jim Cobane

Henry Ford Health

[Jim Kosloskey]

Amit,

Purge Caches either from the Netmonitor (thread or process Full menu) or via command line will refresh Tables, Xlates, and variants although you cannot control which – they all get refreshed.

I do this all the time.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Charlie Bursell]

I don’t believe Cloverleaf tables are the best option for dynamic tables, especially if the tables change often.  First, it ties up a Cloverleaf Analyst to make the changes then there is the matter of someone having to purge caches.

What I do with Doctor Codes etc which are constanly changing:

Usually there is a admin person that maintains these tables in a spreadsheet like Excel.  When they want to make a change I have them export the spreadsheet as a well known csv file where Cloverleaf can access it.

Then I write a Tcl proc that upon start up reads the csv file and stores it in a lookup form like a keyed list or an array.  It also stats the file and saves last modified time in a global variable.

Then when the message comes through I first stat the file and if the last modified time has changed, I reload the table and save the last modified time.  Otherwise, I use what is in memory. Then I perform the required lookup vis my in-memory table.

It frees up the Cloverleaf Analyst to do all of the highly technical stuff they do  🙂  and requires no intervention to do purge cache, etc.

Piece of cake!

[Tim Pancost]

A couple of caveats on purging caches, at least some things we’ve noticed.  I can’t remember which version it started with (5.3?, we’re on 5.5 now), but at some point it became “unreliable” to purge caches from NetMonitor.  When it builds the command to send to the command thread, it included every thread in the site, not just the threads in the process in question, so the command would error out.

The other thing is the syntax of the command.  We’ve found the easiest form is:

hcicmd -p my_process -c ‘. purgex’

HTH,

TIM

Tim Pancost
Trinity Health

[Amit Anand]

Thanks for your responses.

[Bala Pisupati]

I tried purge on tcl it did work. so does this means we don’t neet to stop and start the proc.

[Jim Kosloskey]

Bala,

Purge Caches does nothing for Tcl – it only loads Xlates, Formats (I think), and Tables.

For Tcl procs if you have a new version, you need to do a ‘reload’. If you have a new proc and mktclindex has been executed, you can do a ‘reindex’ followed by a ‘reload’.

The bottom line is you do not need to stop and start the processes/threads to get certain objects refreshed.

There was a caveat that sometimes (especially with the use of PATHCOPY) in some releases problems were experienced with purge caches. We do not use either BULKCOPY or PATHCOPY here as a rule so that has never caused a problem.

Having said that, there are still some team members here that do not use purge caches, etc.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Bala Pisupati]

Jim,

I did a minor change to my existing tcl and tested and the change took effect. How did the changes took effect?

[Jim Kosloskey]

Bala,

Was this a Tps or an xltp proc?

I am not sure which command you want the cons of but I will assume purge caches, reload, and reindex.

Purge Caches is indisciminate. That is it loads all of the objects it loads – you can’t just choose Tables for example. You can control whether the caches get purged for just one thread or all threads in a process by where you execute (in the NetMonitor) such as thread or process.

That could be an issue if you only want a Table reloaded and someone has a new verion of an Xlate tht they don’t want loaded. That is not a good practice – to rely on the fact that objects don’t get loaded until a stop/start. But it is possible to get something refreshed you do not want refreshed. The caveat – ‘know what you are doing’ comes into play here.

If you purge caches at the thread level, it is possible you will not refresh something needed for the integration (for example purging caches on an inbound thread and a Table is not referenced in the Inbound thread but is in the outbound thread – the Table will not be refreshed).

Many people just purge caches at the Process level – but that might refresh more than you want.

Re-read the caveat above.

Reload is specific in that you need to select the Tcl proc you want reloaded (note if you create a new proc and don’t do a reindex first the proc will not appear in the pull down for the procs that can be reloaded). You can only do one proc at a time. So if you want to do multiple procs, it might be advantageous to stop the process – or not – your choice.

Reindex rebuilds the in memory index of all available Tcl procs. I don’t think it matters at what level in the NetMonitor you execute this one – it is effective for the entire site.

You can specify the purge caches, reload, and reindex activities in the NetMonitor at the thread level, the process level, or a view level. That gives you many potential levels of control.

I rarely use command line executions but if I want to know the command line execution I just do the command in the NetMonitor and look at the ‘Command and Engine Output’ in the NetMonitor where the command will be displayed.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Gary Atkinson]

On 5.5 if you do purgex at the process level (in the gui) the whole site purges, not just the process.  If you do purgex from the command line, this does not happen.

I found that for me I just bounce the process when bringing in new changes, that way I know for sure the changes were brought in.  Oh though, the down side is an interruption in connection.

[Bob Richardson]

Folks,

Just a caveat on using reindex:  if you have implemented Unix style version control (such as SCCS) the Tcl control files (for example, .s)

will be added to the tclIndex file and really create a mess in the engine.

For what it’s worth: I prefer using purgex, reload when possible to avoid dropping connections especially for “sensitive” interfaces such as to our Canopy application.

But then the brute force method of cycling the process does insure that everything (desired or not) gets refreshed.  And some of my team members prefer the hammer.

BobR

[Gary Atkinson]

Would anyone be willing to share their tcl that reads in a .csv file in lue of using the CL tables?

[Scott Folley]

Here is a fun one that I came up with:

######################################################################

# Name:         csvTableLookup

# Purpose:      

# UPoC type:    tps

# Args:         tps keyedlist containing the following keys:

#               MODE    run mode (“start”, “run” or “time”)

#               MSGID   message handle

#               ARGS    user-supplied arguments:

#                      

#

# Returns: tps disposition list:

#          

#

proc csvTableLookup { args } {

   global HciSiteDir

   global HciMasterSiteDir

   if { [llength $args] == 0 } {

       error “wrong # args: should be “[lindex [info level 0] 0]

               ?-site site? ?-table table? ?-side side? string””

   }

   set optionStrings

   for {set i 0} {$i < [llength $args] – 1} {incr i} {

       set arg [lindex $args $i]

       set index [lsearch -glob $optionStrings “${arg}*”]

       if { $index == -1 } {

           error “unknown option “$arg”: must be -site or -table or -side”

       }

       incr i

       if { $i >= [llength $args] – 1 } {

           error “value for “$arg” missing”

       }

       set val [lindex $args $i]

       # The name of the variable to assign the value to is extracted

       # from the list of known options, all of which have an

       # associated variable of the same name as the option without

       # a leading “-“. The [string range] command is used to strip

       # of the leading “-” from the name of the option.

       #

       # FRINK: nocheck

       set [string range [lindex $optionStrings $index] 1 end] $val

   }

   if { ! [ info exists table ] } {

       error “-table argument is required”

   }

   if { ! [ info exists site ] } {

       set site current

   }

   if { $site == “current” } {

       set site $HciSiteDir

   }

   if { $site == “master” } {

       set site $HciMasterSiteDir

   }

   if { ![ info exists side ] } {

       set side “left”

   }

   #puts $site

   set tableValue [ lindex $args end ]

   if { [ file exists “$site/Tables/$table.csv” ] } {

       set tableData [ read_file $site/Tables/$table.csv ]

       set tableList [ split $tableData “n” ]

       set inboundList {}

       set outboundList {}

       foreach entry $tableList {

          set entry [ split $entry | ]

          if { $side == “left” } {

              lappend inboundList [ string trim [ lindex $entry 0 ] ]

              lappend outboundList [ string trim [ lindex $entry 1 ] ]

          } else {

              lappend inboundList [ string trim [ lindex $entry 1 ] ]

              lappend outboundList [ string trim [ lindex $entry 0 ] ]

          }

       }

       set default $tableValue

       set elementNum [ lsearch -exact $inboundList $tableValue ]

       if { $elementNum == -1 } {

           return $default

       } else {

           set retval [ lindex $outboundList $elementNum ]

           if { [ string trim $retval ] == “” } {

               return $default

           } else {

               return $retval

           }

       }

   } else {

       error “table $table does not exist encountered while attempting to lookup $tableValue.”

   }

}

It acts on a “table” file at the master or site level and, though it says csv, actually looks at the file as separated by a pipe.

inboundvalue1|outboundvalue1

.

.

inboundvalueN|outboundvalueN

As you can imagine, creating something to load that table is a cinch, you just grab what you need out of a file or a database and put it in that form.

Use it if you like…modify it if you like…we use it in production wherever we need a dynamically loaded table.

Of course I do not warrant that it is fit for any particular use…yada yada.

[Charlie Bursell]

You are much better off using the csv package that is part of tcllib

package require csv

csv::split $rec       ;# Returns a list of fields

csv::join $list         ;# Create CSV record from a list

It is a pretty simple matter to stat your file at startup and save last mod time in a global or shared variable.  Then for each run, stat the file and if not the same as the saved one, reload the table.

I usually store the table in an array where, usually, the first CSV field is the index and the rest are data.

As with any table, make sure you have unique indexes

[Gary Atkinson]

would anyone have any Tcl code they would be willing share that uses the csv tcl package as Charlie has described above?

[Charlie Bursell]

Don’t know why you would need a proc it is so easy

package require csv

# Split a CSV record into a list of fields

set fldLst [csv::split $rec]

# Make a CSV record from a list of fields

set rec [csv::join $fldLst]

[Gary Atkinson]

Actually an example proc that does the “whole thing” as you described.  Grab the file…check for modification etc etc  😆

[Author]

Replies