[Jason Russell]

Cloverleaf is really kind of ‘not good’ in this regard. You have to keep two sets of variables, one standard translate variable (@var) where you can do MATH functions, then the iterate variable (%g1 – where g can be i or f depending on context, and a number), which you have to put a $ to store into (see Lisa’s copy line of @idx -> $%g2). Cloverleaf won’t allow you to math on the % variable, so you have to update the @variable then store it into the % variable. It’s a really odd design choice. Otherwise, as the others said, you need to keep your iterate index and copy index separate and keep the counter current.

[Jason Russell]

I’m curious as to why your engine IP changes? We’ve always been careful that we never change the IP (nor the hostname). Any upgrades are tested on our test system first, then migrated to production with a downtime. Unless there are major OS changes (IE: we went from CentOS to RHEL) or engine changes (eGate to cloverleaf), we never changed our IP or host name. Even major version updates use the same information. In the 10 years I’ve been here, we’ve only changed our engine IPs twice. Once was when we migrated to Epic, and the eGate engine then was heavily consolidated from five servers to two. The only reason we didn’t repurpose some of those IPs was because all the interfaces has to be up while we imported/brought in data before go-live to Epic. The second time was our recent migration from eGate to Cloverleaf.

However, the only real solution (and you should be doing this anyways) is use a NAT IP that the vendors connect to, and Network routes that to the actual server. This is not applicable to internal servers that use the engine, only external servers that use the VPN. External systems should not know your internal engine IP. This way the NAT IP stays the same, and your engine IP can be pointed to by your network team.

We really can’t change our IPs even more than before since we are now running a HA set up with a DR failover. We have a floating IP that points to the correct server, and changing any of those would cause massive headaches for everyone.

[Jason Russell]

Looks like changing it back to ASCII actually worked.

[Jason Russell]

Have you tracked it down to a process? The only time I’ve seen cloverleaf leak memory like that is when you have folder being processed with fileset-local and it wasn’t cleaning itself out (or looking too quickly) and eating RAM. Clearing the folder almost immediately cleared the memory issue.

[Jason Russell]

So I’m on TCL 8.6 cloverleaf version 2022.09. tcllib is version 1.19 and TCL has some updates that make the packages pointless, and they don’t seem to work correctly. The following code works the way I’d expect:

set ucode [lindex $xlateInVals 0]
set ucode [string map { \\X0D\\ “” \\X0A\\ \n \\F\\ | \\S\\ ^ \\T\\ & \\R\\ ~ \\E\\ \\ } $ucode]
set ucode [join [lrange [split $ucode \r\n] 1 end] \n]
set ucode [binary decode uuencode $ucode]
set ucode [binary encode base64 $ucode]
set xlateOutVals $ucode

It matches what happens on the command line.

[Jason Russell]

Here’s the updated code, and it’s still adding extra characters:

——

set inVal [lindex $xlateInVals 0]
set outVal “”

package require base64
package require uuencode

set ucode [lindex $xlateInVals 0]
echo $ucode
set ucode [string map { \\X0D\\ \r \\X0A\\ \n \\F\\ | \\S\\ ^ \\T\\ & \\R\\ ~ \\E\\ \\ } $ucode]
echo $ucode
set ucode [join [lrange [split $ucode \r\n] 1 end] \r\n]
echo $ucode

set pdf [uuencode::decode $ucode]

echo $pdf

set b64 [base64::encode -wrapchar “” $pdf]
echo $b64
# set b64 [string range $b64 1 end]
set xlateOutVals $b64

<hr />

Here are some of the initial echos that I have:

Direct data (first ucode): begin 644 WAV.DAT\X0D\\X0A\M)5!$1BTQ+C*)>+CS]/H"B@-2`P(\T\]B:CP\E\+U1Y<\T\4@+

second ucode: begin 644 WAV.DAT
M)5!$1BTQ+C*)>+CS]/H"B@-2`P(&]B:CP\+U1Y<&4@+T-A=&%L;V<@+U!A

Third ucode: M)5!$1BTQ+C*)>+CS]/H"B@-2`P(&]B:CP\+U1Y<&4@+T-A=&%L;V<@+U!A

pdf: Lot of binary data that is unreadable:

b64: tJVBERi0xLjAKJeLjz9PoCiAgNSAwIG9iajw8L1R5cGUgL0NhdGFsb2cgL1BhqtZ2VzIDE

The issue is in the b64 value, there are a lot of ‘t’ and ‘qt’ characters that are in place:

t JVBERi0xLjAKJeLjz9PoCiAgNSAwIG9iajw8L1R5cGUgL0NhdGFsb2cgL1Bh qt Z2VzIDE

If I leave off the -wrapchar “”, the PDF is then broken into newlines, and that’s where the qt seems to come into play, but not sure why they’re being inserted like that. If I leave the \r in place it puts a lot of extra tqt characters.

Not really sure what is going on with this.

[Jason Russell]

The ‘cleanup’ comes when the specific accounts are no longer being sent at all (visit is over!), and there’s nothing to tell the script that this is the last message (it likely just stops). So to keep the DB minimal, you’d have to run a process outside the script itself to remove old accounts over x days old.

[Jason Russell]

Did you restart your process fully? Is the tclproc fully saved and showing up in the list? Just seems like the new code hasn’t been updated in memory.

[Jason Russell]

We do something similar with another vendor who sends a lot of patient data over and over. Just got to make sure you have enough data to make sure you can tell the difference. As for cleaning I would have a scheduled task that would run on a daily/weekly and checks the date/time stamp and removes them if they’re more than x time frame old.

[Jason Russell]

I did, using a script inte parsing directory as you mentioned. It picks up all the results and sends a single message as response, works as expected, thank you!

[Jason Russell]

So each instance of the trigger only happens once, so if it picks up a file based on the directory, it won’t pull the others until the next time trigger?

[Jason Russell]

Jim Kosloskey wrote:

Well, I had a few minutes, and my test shows the results are dependent upon the variant definition. This is not surprising. In the standard ORU^R01 message the OBR is a required segment in the ORC Group (see layout in an above post). So, when PATHCOPYing @null at the group level, as many empty OBR segments as were created before the PATHCOPY will remain as empty OBR segments. Those can be reused but cannot be removed within the Xlate. The Xlate always outputs required segments even as empty segments. However, if one makes the OBR segment optional it will be gone after the PATHCOPY. The relationship between message variants and what wants to be done in the Xlate is essential to accomplish the transformation needs.

This confirms our findings. A little annoying, but it’s what we have to deal with.

[Jason Russell]

I think you’ve noted some of the major pros and cons between the two. I think Jim Kosloskey has done a load and timing comparison doing TCL vs Xlate, and I think the difference was minimal.

I will add this, when we were coming onboard with Cloverleaf, they (Cloverleaf reps/trainers) pushed against filtering in Xlate.

However, the answer comes down to “it depends”. It’s a whole lot easier and cleaner to write generic filters (we have a lot of facility and department filtering) that are stored in master that a LOT of our interfaces use. You call the filter, pass in a table, and it does it’s work based on that. When troubleshooting, while it does add an extra step, you know it’s there, it’s marked and you immediately understand what it does. There is no need to parse the entire message (just pop out the data you need, check to see if it needs to go, and work it based on that).

On the inverse, when you get to specific filtering that is unique to a specific vendor/interface, it’s almost easier to do in the XLate. However, when doing it that way, there has to be an easy way to see if you’re filtering in the Xlate. Comments and whatnot can help, but again, that’s a high level design choice. Filtering in TCL helps quickly see IF it’s happening, and then you dig deeper to see how it’s happening. Keeping the filters in Xlates means you have to dig into the Xlate itself and see if it’s filtering and how it’s filtering it.

And this is my personal opinion, to me it’s easier to read filter rules in TCL, depending on how it’s written (assuming some decent style guides and good variable naming).

One of the things that a senior tech has emphasized is “make it so it’s easy as possible to troubleshoot at 3 AM after being woken up out of a deep sleep, and you didn’t write it”.

[Jason Russell]

That’s what I would look for, but clearing that group doesn’t seem to clear all iterations of it, just the first. There is an interface where we do that, but in this instance we may have two repetitions before needing it cleared.

I’ll try that again just to be sure, but it didn’t seem to work the way we wanted.

[Jason Russell]

I agree with Jim. Your scan interval controls how often (in seconds) it looks, read controls how often it reads a new set of lines in the file, and number of messages controls how many messages it reads at a time. I usually keep ours at 1 scan, 1 read, and 10000 messages, but we rarely load large datasets, and when we did (millions of messages), we had to adjust so we wouldn’t cause data delays due to cloverleaf processing inbounds first.

Beyond that, there is a bit of a delay even picking up the files, usually a few seconds (it also depends on how large the file is).

with the resend function, you’re injecting the messages directly into the engine skipping all of the file reading delays.

But with the defaults, method 1 would take forever (picking up a message, waiting 30 seconds then picking up a new message). Method 2 wouldn’t be much better (picking up a message, reading a line then waiting 5 seconds to read the next message).

If you’re looking to pick up a single file that you control when it picks up, file is another option that should be faster. You would control when it picks up by restarting the thread. Once it reads the file it’s done until another file with the same name is placed and the interface restarts.

[Author]

Replies