Making TPS tcl procs more self-aware

[Jon Blanchard]

I am working on a TPS tcl proc that will share a global variable with other instances of the same tcl proc within the same thread and within the same interpreter instance.  Is there a way for the tcl proc to determine it’s placement within the thread or a message route, without requiring additional args to the tcl proc call?  I don’t see enough information in the message metadata to be useful.

Migration Consultant
Coffee Regional Medical Center
Eastern Time Zone

Topic

[Jim Kosloskey]

Would the Context be useful (that is an argument normally provided in UPoC invocations)?

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Steve Williams]

The context only works at a high level. If the TCL script were to appear in the same TPS context multiple times, nothing from Clvf seems to provide the “position” in the stack. A user arg could provide the positon, but that’s not very elegant and could be (human) error prone.

Having said that, I have a script that uses a generated CRC32 value from the user args to provide a “key” into a shared array within a single interpreter. This allows me to preparse (speed up) the user arguments and prevent the over writing of shared data in the event this script is used multiple times in the same stack.

[Jon Blanchard]

So, within a TPS tcl proc, the processing order using the CRC32 key value could be

Code:

start {
  convert user args into key
  preparse user args
  store preparsed user args: set global_var_array($key) $preparse
  .
  .
  (other code)
  .
  .
}
run {
  convert user args into key
  retrieve preparsed args: set preparse $global_var_array($key)
  .
  .
  (other code)
  .
  .
}
shutdown {
  convert user args into key
  purge preparsed args: unset $global_var_array($key)
  .
  .
  (other code)
  .
  .
}

How much CPU time is involved in the CRC32 conversion?

Migration Consultant
Coffee Regional Medical Center
Eastern Time Zone

[Mark Thompson]

Have you considered adding a counter to the message USERDATA and letting your proc increment the counter each time it is called by that message?  Way more efficient to use a key like procname:counter if that will satisfy your needs.  

It doesn’t provide an absolute TPS stack position, but it does let you know which call of your specific proc you are handling.

- Mark Thompson
HealthPartners

[Steve Williams]

Mark T has a really good idea if your needs are simple. However, if you are trying to do what’s in your pseudo-code: keep data in persistent memory without overwriting it with multiple instances; that’s exactly how I use it.

Regarding the crc32 code: it’s an included package with TCL (package require crc32). I have never done a performance analysis on the crc32 package code, but it is fast.

[Jeff Dinsmore]

Jon,

I’m still a little confused about what you’re trying to do.

Are you looking to share global data between all instances of a given TPS script in a given interpreter?

Or, rather, are you trying to store global data separately for each instance?

Jeff Dinsmore
Chesapeake Regional Healthcare

[Charlie Bursell]

If I wanted to share data between procs in the same interpreter, I would use namespaces.  It allows for more protection of the variables.

A namespace defined in one proc can be accessed by any proc within the same interpreter.

You can tell where in the engine you are by use of CONTEXT and the global variable HciConnName.  As for where you are in the stack, where would that be relevant?

Like Jeff, I would like to know what your end game is.  The good thing about Tcl there is always a way to do what you want to do and as many ways to do a there are people trying to do it  😀

[Jon Blanchard]

In a single instance of the tcl proc, in startup mode, a global array will be created if it does not already exist, then populated with components needed by the tcl proc as part of its run mode processing.  In run mode, each entry in the global array will be accessed during the tcl proc’s handling of the message data; no new rows will be created, so the array won’t eat up memory over time.    In shutdown mode, each element in the global array that was created by that tcl proc instance will be removed and, if no more entries exist in global array, the array will be unset.

The global array is accessed using a subscript in the form , where DataItem is one or more tokens used by all instances of the tcl proc and UniqueTclProcID is the ID assigned to the individual tcl proc instance.

The use of the CRC checksum sounds great; the only question I have left is if two instances of the tcl proc, which are in the same interpreter and use the same args, will end up using the same CRC checksum value.  Using a namespace sounds cool, too, but I will want to benchmark each method to see which consumes fewer CPU clock ticks.  Using the counter method might be great for simpler use cases, but my concern is that the counter value will be the same for two instances of the tcl proc at any chance point in time, based on the message volume that each instance is handling.

So the end goal is to create a tcl proc which can handle messages at the TPS, Pre- or Post-Xlate and store the information that it needs to handle each message in the same manner, but efficiently.  I don’t envision this tcl proc being used within an Xlate.

Hope that answers your questions.

Migration Consultant
Coffee Regional Medical Center
Eastern Time Zone

[Jeff Dinsmore]

Assuming the data you’re passing along in this persistent data is relative to a given message running through a stack of Tcl procedures…

I’m theorizing here – and may be way off the mark – but would it work to just write the args to each message’s USERDATA metadata?

This meta data should persist for the life of the message.  When the message is destroyed, the meta data is destroyed too.

You could include an element in the keyed list that is a counter to indicate relative position.

So, for any execution of your script, check if USERDATA contains your args data.  

Code:

set argsCounter [msgmetaget $msgHandle USERDATA argsCounter]

If so, read it:

Code:

set argsList [msgmetaget $msgHandle USERDATA -all]

If not, write it:

Code:

msgmetaset $msgHandle USERDATA “argsCounter 1 argName1 argValue1 argName2 argValue2 argName3 argValue3”

At the end of the script, increment your counter:

Code:

msgmetaset $msgHandle USERDATA “argsCounter [incr argsCounter]”

Note: This is theoretical (untested) code, so it’s likely that some twiddling would be required to get it to work…  I also have no idea what kind of overhead this might introduce, but you could certainly do timing tests to determine that.

Jeff Dinsmore
Chesapeake Regional Healthcare

[Robert Milfajt]

Love the title of this topic, Making TPS tcl procs more self-aware.  Pretty soon Cloverleaf will be saying, “I’m sorry Dave, I can’t do that”!!   🙂

Robert Milfajt
Northwestern Medicine
Chicago, IL

[Jon Blanchard]

Forgot a couple of details…sorry.

The global array will be accessed by each instance of the tcl proc, but the data stored will be specific to the tcl instance that generated it.  Multiple instances of the tcl proc access the global array within the same interpreter, but there is no sharing of data between them.  The global array exists as a common resource by name only, simply because the same script is executing in different UPOCs.

My understanding is that the message and its metadata exist for the life of its passage through a single thread.  Once the thread sends the message to an external system or another thread, the message and metadata are destroyed.  The next thread then adds the message and new metadata into the database.  Since each instance of the tcl proc will most likely have a unique argument, there is no common set of args that can be carried in metadata.

Thus, each instance of the tcl proc needs to be uniquely ID’d so that its entries in the global array are not accessed by any other instance of the tcl proc using the same global array.

Migration Consultant
Coffee Regional Medical Center
Eastern Time Zone

[Charlie Bursell]

I am still trying to wrap my head around this.  Very confusing!

As I said, I would store the common array in a namespace.  As for having data unique to each instance, pass an argument to each proc which could serve as the ID.

The message and metadata are not necessarily changed as it moves between threads.  The message, which is part of the metadata, is changed only if it were modified by the preceding thread.   Metadata, with the exception a a few flags, will not change unless you purposely change.  The USERDATA field will remain the same.

An example of what the end goal is here would be helpful.

[Steve Williams]

Jon, I’m not sure if you are aware, but there are multiple TCL interpreters running associated with each single Clvf process and they do not have a shared memory space. Basically, each Clvf thread has it’s own TCL interpreter. The xlate thread may have one or multiple TCL interpreters depending on it’s configuration at the process level.

As such, you cannot pass data from one interpreter to another via global memory. This also means that most UPoCs will have no knowledge of other TCL scripts being invoked. If you need to pass information from one script to another, then message metadata is the solution you are looking for.

Message metadata does not get lost from one thread to another or from process to another as long as the message remains in the recovery database. However, if you send a message from one thread to another through an external connection, like a TCP/IP connection or a file based hand off, then yes, the metadata would be lost. In those cases, I invoke a custom Z segment (in HL7 msgs) to carry the data and usually strip it out just before sending the msgs to the actual recipient.

Also, I agree with Charlie, to assist further, we would really need to know exactly what your trying accomplish.

[Author]

Replies