[Jason Russell]

I think you’ve noted some of the major pros and cons between the two. I think Jim Kosloskey has done a load and timing comparison doing TCL vs Xlate, and I think the difference was minimal.

I will add this, when we were coming onboard with Cloverleaf, they (Cloverleaf reps/trainers) pushed against filtering in Xlate.

However, the answer comes down to “it depends”. It’s a whole lot easier and cleaner to write generic filters (we have a lot of facility and department filtering) that are stored in master that a LOT of our interfaces use. You call the filter, pass in a table, and it does it’s work based on that. When troubleshooting, while it does add an extra step, you know it’s there, it’s marked and you immediately understand what it does. There is no need to parse the entire message (just pop out the data you need, check to see if it needs to go, and work it based on that).

On the inverse, when you get to specific filtering that is unique to a specific vendor/interface, it’s almost easier to do in the XLate. However, when doing it that way, there has to be an easy way to see if you’re filtering in the Xlate. Comments and whatnot can help, but again, that’s a high level design choice. Filtering in TCL helps quickly see IF it’s happening, and then you dig deeper to see how it’s happening. Keeping the filters in Xlates means you have to dig into the Xlate itself and see if it’s filtering and how it’s filtering it.

And this is my personal opinion, to me it’s easier to read filter rules in TCL, depending on how it’s written (assuming some decent style guides and good variable naming).

One of the things that a senior tech has emphasized is “make it so it’s easy as possible to troubleshoot at 3 AM after being woken up out of a deep sleep, and you didn’t write it”.

[Jason Russell]

That’s what I would look for, but clearing that group doesn’t seem to clear all iterations of it, just the first. There is an interface where we do that, but in this instance we may have two repetitions before needing it cleared.

I’ll try that again just to be sure, but it didn’t seem to work the way we wanted.

[Jason Russell]

I agree with Jim. Your scan interval controls how often (in seconds) it looks, read controls how often it reads a new set of lines in the file, and number of messages controls how many messages it reads at a time. I usually keep ours at 1 scan, 1 read, and 10000 messages, but we rarely load large datasets, and when we did (millions of messages), we had to adjust so we wouldn’t cause data delays due to cloverleaf processing inbounds first.

Beyond that, there is a bit of a delay even picking up the files, usually a few seconds (it also depends on how large the file is).

with the resend function, you’re injecting the messages directly into the engine skipping all of the file reading delays.

But with the defaults, method 1 would take forever (picking up a message, waiting 30 seconds then picking up a new message). Method 2 wouldn’t be much better (picking up a message, reading a line then waiting 5 seconds to read the next message).

If you’re looking to pick up a single file that you control when it picks up, file is another option that should be faster. You would control when it picks up by restarting the thread. Once it reads the file it’s done until another file with the same name is placed and the interface restarts.

[Jason Russell]

Yes, the IB and OB are the same, however, my understanding of chaining may be lacking where we would use SEND vs CONTINUE.

I still find it interesting that Cloverleaf doesn’t have a way to simply clear the outbound message completely, allowing you to start a new message from scratch.

[Jason Russell]

Interestingly enough, we use a mix of TCL and xlate for our interfaces. TCL for filtering, XLates for actual HL7-HL7 translations and functionality.

As far as I am aware, the xlate is built on top of TCL and the only extra overhead is loading the structure and parsing the message out completely, which should be nearly instant. I would be curious as to your throughput (how many messages) and what kind of hardware you’re on (even if you’re in VMs).

How many threads and processes are in your sites? There are some things that we were doing (and still are to some extent) that could help reduce processing on messages that cloverleaf recommends.

As for field lengths, we disabled field length checks completely. Considering data changes and software updates, if we need to limit fields we do that manually on an as-needed basis.

[Jason Russell]

You should also be checking your monitord logs ($sitedir/exec/hcimonitord/hcimonitord.log). We had a site do this, the monitor daemon would crash causing the issues you described. the threads were still processing, but the gui would never load right. Ours was caused by an issue where alerts sending emails were stepping on each other’s toes, and would leave a file that the engine couldn’t do anything with, causing the monitord to fail.

Sometimes restarting the monitord process would work, sometimes we’d have to kill. Sometimes it’d come back up on it’s own, but the pid file in the same directory as the log was not clearing out properly causing the ‘status’ to read incorrectly.

[Jason Russell]

Chaining is an interesting thought, but I may be mistaken on how it works. My understanding is that you have multiple translates, and each step modifies the translate before hitting the end point. I’ve worked with branching where you do work at a high (chain) level, then do some minor work for each individual system on the branch, but it seems that wouldn’t work for chaining.

[Jason Russell]

I’m curious why you’re converting xlates to TCL? They both have their pros and cons. TCL can do some really fancy stuff as scripts and basic bulk actions. Xlates use HL7 formats to make sure data goes from a message to an outgoing message. They really have different functionalities. The entire point of the Xlate is to use the structures to make sure the messages are formatted properly, with the correct segments in the correct places.

There are HL7 libraries for TCL, but they are very basic processors, and don’t use formatted structures. There is a reason you want to use a structure for HL7, they can get messy quickly if you don’t.

[Jason Russell]

What version of Cloverleaf are you using?

[Jason Russell]

We’re in a virtual cluster, we have 8 CPUs allotted for our production box. The CPUs are Intel E5-2697’s.

The clustering system is older and they’ve been talking about upgrading them for some time. Do not know when that is planned.

I think most of your decisions should be based around what kind of load you’re looking at doing, how many million messages you’re processing, etc. We’re a smaller shop and do around 2 million message (all inbound, outbound, site-to-site counted here). RAM,  disk write speed are just as (in some cases more) important than your CPU. Running sar we rarely go below 90% idle, and drops below that are usually us processing a cron job off hours. We’re not fully migrated yet, but I don’t think our CPU usage is going to change that much. Ram, disk speed, etc will change as we’re going to be processing bigger files (getting to migrating the PDFs).

[Jason Russell]

So after a bit of back-and-forth with support, it seems on-premises customers are not given access to the “Release Center” even though all their documentation points to the Release Center without mentioning on-prem’s lack of access. Here are the necessary KB articles you’re expected to know and subscribe to for updates.
KB2287007 – The announcement. No KB articles linked, instructions are for non-on-premises customers (Cloud? Only Infor hosted? They weren’t clear): (https://customerportal.infor.com/csmcore?id=kb_article_view&sysparm_article=KB2287007)</p>
KB2293813 – Release Report. This is the report that has the enhancements and bug fixes listed. https://customerportal.infor.com/csmcore?id=kb_article_view&sys_kb_id=f2471229c35f52900d906cbeb0013189</p>
KB2282305 – Technical Compatibility Overview – The actual server compatibility listing.  https://customerportal.infor.com/csmcore?id=kb_article_view&sysparm_article=KB2282305</p>
If you want updates you need to subscribe to those KB articles or at least have them handy to get any release information.

Naturally, to access this you will need to have an active Infor Concierge account with your facility.

• This reply was modified 2 months ago by Jason Russell.

[Jason Russell]

The reason I can’t find them it seems is because I don’t have ‘release center’ in my concierge portal. I can find the “release record” if I dig around and search, but I can’t ever find the notes, and it seems that is the root cause. The thing is, the release report doesn’t specify supported systems and whatnot, whereas the new “Technical Compatibility Overview” is essentially what I’m looking for.

Makes things a pain when you don’t have the direct access and they change names of their documents so it’s impossible to find via search.

[Jason Russell]

Thanks James. None of the hcidb* commands would touch it, but it it looks like it actually cleared up on it’s own. That’s good to know for future use, my SQLite knowledge is bare-minimum.

[Jason Russell]

** This is given assuming you’re writing in Linux, not windows. If you’re in Windows you’ll have to adjust the slash direction (/ vs \) and file path.

My script renames the file in place. $msg_list contains the full path, and I’m simply appending ‘.processed’ so it doesn’t get picked up by the file-path glob/regex. So I pick up *.txt normally, so having it end in *.txt.processed means it won’t get picked up again.

file rename -force /opt/cloverleaf/cis2022.09/integrator/sd_lab1/indata/file_to_run.txt /opt/cloverleaf/cis2022.09/integrator/sd_lab1/indata/file_to_run.txt.processed

If you want to change the folder completely and put it in an archival folder, you would have to get the file name by stripping off everything but the last ‘/’ ( set fileName [lindex [split [$msg_list] /] end] ), and set the full path in the destination. Paul got his ‘archive’ path by passing it in. You can code it in if you want.

Also, I haven’t had much time to play with the script since I put it in place, but you may want to change file rename to file copy, as I think cloverleaf will automatically clean up the file regardless, and I get an error saying it can’t find the original file.

[Jason Russell]

Your best best is probably to have a test server and install and run some testing. You’ll find there’s probably not a lot of people who are “bleeding edge” in terms of OSes.

I don’t know anything about your environment in general, but we use VMs for everything. If you have a test server (you should), I would install on 2025 and migrate and see how it pans out. If it fails, you can move back to the old server. That is generally the process we would follow if you can’t do an in-place upgrade.

[Author]

Replies