Automated Cloverleaf Testing-production messages in test env

[Varun Sinha]

Hi,

 We are upgrading from Cloverleaf 6.1 to 6.2 on Linux environment.

Currently, We have Cloverleaf production environment running on 6.1 and the upgraded test environment on Cloverleaf on 6.2.

We want to automate the testing process and want to run the corresponding production inbound messages (at least 10 messages in numbers) from Cloverleaf 6.1 to its corresponding outbound threads in Cloverleaf 6.2.

In order to achieve this:

–> We have taken a copy of the test site (xlate and tcl procs are all same as production site) on Cloverleaf 6.1 and put them onto the upgraded Cloverleaf 6.2.

–>Change all of the threads protocol to file:dev/null.

–> Since, MSH-10 is unique for all inbound threads:

I am planning to retrieve at least 20 messages for inbound thread from production environment (Cloverleaf 6.1) and it’s corresponding outbound messages to outbound thread from smatdb and write it down to a file. [Still unable to figure out as how to achieve it through TCL script or any other ways?]

I can run the same messages picked for inbound thread from production environment (Cloverleaf 6.1) and run it to inbound thread of Cloverleaf 6.2 and as we have already set outbound thread to file: (some location/dev/null), we can compare output through TCL scripts or using diff command to find any differences in the file.

Is there any other better approach to automate the testing?

Regards,

 Varun

Topic

[Steve Williams]

Hi Varun,

  What I do is use cp, boxing and migration tools to make exact, upgraded versions of all production sites. Note that boxing alone may not capture all scripts or tables used in an interface. I capture production data from SMAT files (at least a couple of hundred for slow interfaces and several thousand for high volume interfaces like ADT). I run those messages through the route testing tool in both environments, capturing the output to files. I then compare those files using a program called ExamDiff Pro (runs on Windows). I log all interface files tested with comparison outcomes. The output should be exact in most all cases. Differences can indicate an difference in engine behavior or it could indicate a difference in interface configurations. Some interfaces may use the @now variable (xlates) or some other time sensitive scripts that can change the output accordingly.

I make note of those issues and use ExamDiff Pro to ignore those field components (via RegEx settings). I also use HL7Spy’s (3.x) built in message comparison tool for more difficult cases involving HL7 messages. It takes human testing to determine the difference and take appropriate action.

You can use Linux/Unix diff, which is great when there are no difference between the files. When there are differences, it becomes much more time consuming to isolate the issues.

You could automate the overall process, but I found that it can take much more effort to automate this process than to just do the work. Since major upgrades only come around every few years, it does not make much sense to spend so much time on automating (scripting/debugging) a one-off process that you’ll most likely never use again.

Disclaimer: Your mileage may vary… 🙂

[Varun Sinha]

Thanks Steve for the insight. 🙂

Even, I think that full automation is waste of time and we need manual intervention if we find any difference.

I found it little difficult to stage production data based on MSH-10 through smatDB as output file just gets overwritten instead of appending. Currently, I am trying to test the interfaces based on single message (input/output) by converting interfaces to use file connection.

 Is there any reference/document/post to create file interface [file to file]?

Regards,

 Varun

[Steve Williams]

Duplicating the production environment and modifying all interfaces to be file input to file output adds a layer of complexity that I try to avoid as the tediousness of this method is much more time consuming. Of course, there are some engine deployments that are so chaotic and haphazard in their design that the only way to adequately test it is to go down the path you are taking.

If you’re running a single message through the engine at a time, I suggest you check the box “Append to File” in the OB thread File Protocol properties. That’ll prevent the overwrite issue you’ve encountered.

I’ll also point out that in using the Route Testing Tool to write to file, it automatically appends to the existing output file(s). This does bite me back as I do a quick one off test and sometimes forget to delete the output file(s) before running the entire data set I’m testing.

Hope this helps.

[Varun Sinha]

Thanks Steve!

I am using  “Append to File” in the OB thread File Protocol properties.

I faced issue when I tried to save inbound thread production messages from smatDB through Resend Message to a file. It definitely allows you to save multiple messages at one go.

Steve! Please let me know if my approach is correct or not for testing messages.

Step1: Take the entire inbound thread production message for a single day in a file from Cloverleaf 6.1 smatDB.

Like create a file from test_in.ibsv.20180530235934.smatdb

          Also, take the same day outbound thread production message in a file from Cloverleaf 6.1 smatDB.

Like create a file from test_out.obsv.20180530235934.smatdb

Step2: We’ll go to Cloverleaf 6.2 version where we have same test_in as inbound thread and test_out as outbound thread.

test_in will be having some port not clashing with other environments. Though, it will be in opening state.

test_out thread will be changed to PROTOCOL:file where Output filename will be test_out; file format: Newline-terminated; Append to file is checked.

Step3: Go to Network Monitor. Right Click on test_in thread. Go to Control -> Full. Click on resend. Select Context as Inbound post-TPS [We can’t select Inbound pre-TPS as inbound conn is in opening state. Though, we’ll not be able to execute Pre procs. Right?]. Under File Options, we can select inbound thread prod data (same file created from test_out.obsv.20180530235934.smatdb). We can run this and since the output thread is a file connection, we can compare the output of this file connection with file created from test_out.obsv.20180530235934.smatdb.

This should be exact match.

Please let me know if I am missing anything or any improvement in this testing process.

Regards,

 Varun

[Author]

Replies