Regression Testing

We did this for 3.7 to 5.3, and 5.3 to 5.5.  My process went something like this…

-Upgrade test server with new code.

-use a modified form of siteTar (found on this site) to grab production site(s) from prod server, move it to the test server into a special directory.

-hcirootcopy that special directory

-ensure all threads are set to autostart 0

-Save 24 hours worth of data on each inbound production thread, COPY those save files to the test server.

-On the production server (during NON-peak times), run hciroutetest on each inbound save files, producing a uniquely named outbound file.  Move those outbound files to a “prod” folder in special directory on the test server

-On the test server, run hciroutetest on each of the inbound save files producing a similarly named (to the production file) outbound set of files.  MOVE those to the special directory structure.

-Ensure all outbound files are CRLF instead of LEN10.

-Compare (diff) oldProd/.crlf files to newProd/.crlf files for differences.  If a differences exists, use the appropriate recordFormat test tool to run a report on oldProd files and newProd files, output the report to a file.  compare the files to determine which fields are different (Usually, it will be a date/time stamp field, like date of birth using current time for the time portion of the field).

Doing this took me 3-5 days of half day sessions to complete all of our 100+ routes, all done manually, but using complex command line structures, and in some cases, for-loops entered on the command line.  Having done it manually twice, I might try to develop a script, or series of scripts for the 5.5 to 5.6 upgrade to ensure my oldProd and newProd files are similarly named.

When done, I deleted the newProd site from the test server, performed an hcirootcopy on the real test sites, and started using the new version in test (ensuring everyone knew there could be no production changes from that point until prod was completely upgraded).

Oh, it goes much better if your Cloverleaf is running on a real OS, something that ends in ..IX.

Todd wrote:

“On the production server (during NON-peak times), run hciroutetest on each inbound save files, producing a uniquely named outbound file.  Move those outbound files to a “prod” folder in special directory on the test server “

One thing you may want to consider when using the routetester GUI for testing is the fact that a routetest will not run the message through any outbound TPS proc you may have in place.  We are running 5.2 here (currently testing for upgrade to 5.5) and I know the routetester has this behavior in 5.2.   I’m not sure on later releases if this is still an issue.

What we do here is we take a previous days inbound SMAT file from production and run it through the test interfaces.  We save outbound SMAT files as well in production.   We then compare the outbound SMAT files from prod and test to see if they are equal.  We do convert the length-encoded files to newline before comparison and run a diff on any files that aren’t equal.  It makes it easier to determine what, if any, issues you may encounter.

Hope this helps…

Tom Rioux

Baylor Health Care

Thomas is correct.  Inbound and outbound procs are not processed using the routetest tool.  It simply tests all translates, pre/post translates code, and route determination.  That is exactly why when the routetests have been verified by my “diff” processing, it is all tossed in the bit-bucket, and we convert our test system and start end-to-end testing, which will catch the inbound/outbound procs.

We can’t process production messages through to our test systems.  HIPAA concerns, and test system integrity come into play.

Also… the GUI is your enemy for large batches like this.  I guess I glossed over that point in my initial post.  I stick solely to the command line for activities like this; much faster, and I can get several going at once if I feel like screwing with my co-workers system performance.  😈  😈