[Timothy O’Donnell]

This 100% seems like what is happening. What I don’t understand is why it happens at all but then resolves itself? Sometimes it takes multiple resends and sometimes it’s just one or two. We’ve seen it stuck up to an hour doing resends.

We’ve tried switching the thread to tcpip with no change in behavior.

I’ve got a handful of logs from various times this has happened. There’s not much in the way of issues in between messages – it just shows the queueing of the OB message, waiting for replies, OB-Data queue has NO WORK or # msgs, depending then processing SOCKET event for the ACK coming in (albeit after the wait reply time has elapsed.)

[Timothy O’Donnell]

I’ve upped the engine noise and the incoming ACKs have the proper wrapper as you mentioned.

 

[Timothy O’Donnell]

Jeff,

I know this is an old thread but wondering if you’ve started your upgrade to 20.1.1.1 and if you’ve noticed any performance differences between that and 6.2?

-Timothy

[Timothy O’Donnell]

We have an incident open with R&D as we are still in the processing of upgrading with Infor and haven’t implemented anything in Production yet.

I have the JVM Arguments as: Xmx2g I think just to see if that would help – nothing. RAM usage on the server is always on the higher side – and we just have test data so the message volume is MUCH lower than it would be in Production – but CPU usage doesn’t seem out of sorts either.

-Timothy

[Timothy O’Donnell]

Right now we have Basic Security on. In 6.1.4, we had Advanced Security. OS is Windows Server 2019 for 20.1.1.1 but Windows Server 2012 for 6.1.4. It just feels like a newer, 64-bit application shouldn’t be noticeably slower than a really old product.

[Timothy O’Donnell]

UPDATE: After being delayed for a few months, we finally went live with this project only to find out that the process I outlined above didn’t work exactly as planned. The overall concept worked but the PDF was blank – correct title and number of pages, but no content. We didn’t run into this when first testing but we also tested with smaller-sized PDFs and I relied on the vendor to confirm the PDFs were valid. Lessons learned.

I decided to reassess how I set this up and came up with a solution that actually worked. The setup is roughly the same as before but on my fileset-local inbound thread, on the Inbound tab > TPS Inbound Data, I changed the tcl to only pull the filename of the PDF (which is a patient identifier in this case) and pass that along to the xlate. The VRL I changed to have one field – the filename – instead of before where it had two – one for filename and one for the base64-encoded PDF string.

I also updated the Archive tcl on the fileset-local inbound thread copying the PDF to a “staging” folder on the CL server.

The xlate builds the HL7, SQL-querying with the filename to build the patient demographics out as required by the vendor. This is the same as it was before. The difference is I now have a tcl snippet taking in the filename value, which lets me open the PDF that I put in the “staging” folder from the Archive tcl and then I do the base64 encoding here, using fconfigure to translate the PDF to binary first. Outbound variable now was the base64 string and then I delete the copied file that I put in the “staging” folder as it’s no longer needed. I’ve put the relevant tcl snippet below.

package require base64
set visitNumber $xlateInVals
set filePath “$HciRoot/data/UTF/Staging/$visitNumber.PDF”
set fileHandle [open $filePath r]
fconfigure $fileHandle -translation binary
set encodedPDF [string map {\n “”} [base64::encode [read $fileHandle [file size $filePath]]]]
close $fileHandle
set xlateOutVals $encodedPDF
file delete -force “$HciRoot/data/UTF/Staging/$visitNumber.PDF”

This may not be the most elegant or efficient way to do this, but given the size and scope of this interface – 20-30 600KB PDFs dropped overnight during non-peak hours for an interface that doesn’t require a huge amount of patient demographics in the HL7 – it works swimmingly. I also found a base64 encode/decode site is a huge help for quickly identifying if my base64-encoded PDF string was correct or not. Definitely recommend for speedy confirmation.

-Timothy

[Timothy O’Donnell]

The PDF filename will always be the Patient Visit Number from our EMR and then I have an Advanced Database Lookup to our EMR SQL server in the XLATE to pull the minimum necessary demographics like Name, DOB, Gender, Department, etc. based on that Visit Number to fill in the gaps in the HL7. The volume for this interface is likely to be relatively small and controlled – files will always be dropped at a specified time and not in large quantities – so the combination of PDF file sizes and SQL queries shouldn’t be too taxing. If the overall volume was larger, I’d probably want the patient demographics in file metadata or in the file name as well.

-Timothy

[Timothy O’Donnell]

For anyone who comes across this thread in the future looking to do something similar, I finally got this to work with the help of everyone on this thread:

Setup Inbound thread as fileset-local with Style: single (Important if each file is a single non-base64 encoded PDF!)

I’m using a dirParse but just to skip any non-PDF file. Doesn’t do anything else. I also have an Archive tcl for Deletion to just copy the PDF to another folder for testing purposes. This won’t be in Production.

I have a tcl on TPS Inbound Data for the base64 encode that returns the file name (without extension) and the base64 encoded PDF with the data separated by commas. That data is then passed to the xlate which has uses an inbound VRL with two fields – file name and embedded PDF – then I build the HL7 from there with the necessary data. Here’s the run from my TCL, your mileage may vary depending on what you need to do with the information. At the very least, this could be a good jumping off point.

run {
# ‘run’ mode always has a MSGID; fetch and process it

keylget args MSGID mh
package require base64
set msg [msgget $mh]
set filepath {}
set drvCtl [msgmetaget $mh DRIVERCTL]
keylget drvCtl FILENAME filepath
set filename [file rootname [file tail $filepath]]
lappend new_msg $filename
set encodedPDF [base64::encode -maxlen 0 $msg]
lappend new_msg $encodedPDF

msgset $mh [join $new_msg “,”]
lappend dispList “CONTINUE $mh”
}

Hope this helps!

-Timothy

[Timothy O’Donnell]

Jim,

I removed that fconfigure as well. I’m still getting just the string “%PDF-1.7” so I’m wondering if you know what would cause that to be passed from the Directory Parse tcl.

-Timothy

[Timothy O’Donnell]

Jim,

I removed that unnecessary read but the value being passed to me, instead of the actual PDF data, is just the string “%PDF-1.7” instead of the actual PDF that I’m feeding it as a test.

This is the relevant code snippet for my dirParse:

run {
# ‘run’ mode always has a MSGID; fetch and process it

keylget args MSGID mh

# add double quotes around filename with spaces
#set files [string map {“\x0d\x0a” “\x01”} [msgget $mh]]

set files [msgget $mh]
set newlist {}
foreach f $files {
if ![regexp — {.*\.pdf} $f] {
echo Skip $f
continue
}
lappend newlist $f
}
msgset $mh $newlist
lappend dispList “CONTINUE $mh”
}

And this is the relevant code snippet for my TPS Inbound Data tcl:

run {
# ‘run’ mode always has a MSGID; fetch and process it

keylget args MSGID mh
package require base64

set msg [msgget $mh]
echo “This is the Message: $msg”
fconfigure $msg -translation binary
set encodedPDF [base64::encode $msg]
echo “The encoded PDF is $encodedPDF”
lappend new_msg $encodedPDF

msgset $mh $new_msg
lappend dispList “CONTINUE $mh”

}

 

[Timothy O’Donnell]

Jim,

I didn’t have the Style set to “Single” so I have changed that and bounced the process.

For the Directory Parse, I’m simply saying “PDF only” and then I have a tps on TPS Inbound Data (code snippet below) to encode the PDF:

run {
# ‘run’ mode always has a MSGID; fetch and process it

keylget args MSGID mh
package require base64

set msg [msgget $mh]
fconfigure $msg -translation binary
set encodedPDF [base64::encode [read -nonewline $msg]]
lappend new_msg $encodedPDF

msgset $mh $new_msg
lappend dispList “continue $mh”

}

Now I’m getting this ERR: can not find channel named “%PDF-1.7 so I’m not sure what I should be writing for this TPS Inbound Data tcl in order to pass the appropriate data along to the xlate.

-Timothy

[Timothy O’Donnell]

Jim,

Release is 6.1.4

I don’t have any control really over how the file is formatted but each of the fields, except the last, is a single, non-repeating field. The last field will have a variable amount of subfields, all delineated by a tilde. I’ve tried the HRL configuration but I don’t have a good way to keep looping through that last field and its subfield because essentially I’d want to keep doing that for as long as the subfield has values.

I think I need v.6.2 or higher to iterate from a list variable, and then I could use that function to easily resolve my problem.

-Timothy

[Timothy O’Donnell]

Thanks to Jim for helping with the solution. Thought I would post it here for anyone searching for this in the future. Below is dependent on your HL7 variant so adjust as needed:

SUPPRESS (kill original message)

PATHCOPY other segments as needed

ITERATE on group %g1

PATHCOPY 1(0).1(%g1).OBR(0) -> 1(0).1(0).OBR(0)

ITERATE on segment %s1

PATHCOPY 1(0).1(%g1).OBX(%s1) -> 1(0).1(0).OBX(%s1)

CONTINUE

ITERATE on segment %s2

PATHCOPY @null -> 1(0)1(0).OBX(%s2)

This will give you one OBR per message and the corresponding OBXs.

Hope this helps and Jim, thank you again!

-Timothy

[Timothy O’Donnell]

David,

It worked actually, I was using a bad example message where each OBR has the exact same OBXs so it looked like it was copying wrong – I used a few different messages and it worked as expected!

-Timothy

[Timothy O’Donnell]

Jim,

I’m reaching out by email now. Thank you!

-Timothy

[Author]

Replies