Testing guidelines

We are working on upgrade from CL 5.4 to 5.7 (on Windows) and I wonder if others have some kind of guidelines of what should be tested.

So far I made sure that all translations in 5.7 produce exactly the same messages as in 5.4. Here is description how I do it. You can skip to the end if you are not interested in details.

So far I built a pair of sites on 5.7. One is a copy of production site from 5.4 where all outbound threads changed to file type threads. These are saving messages translated by 5.7 into files. All inbound threads are the same as in 5.4 but they are receiving via local ports from the second site. That way I don’t need to change port numbers and any other configuration data on receiving threads. Every receiving thread on this site has a corresponding sending thread on the second site.

Second site (although logically it should be called 1st) is receiving all messages from production 5.4 site, where each message has an extra Z segment with information like source thread and destination thread (I already had this code in place in 5.4 – I use it to store all messages in MS SQL Server database). The receiving thread on 5.7 uses trxId proc to determine where to route corresponding message depending on info in that Z segment.

If received message is a raw message from 5.4 it routed to corresponding sending thread (that sends via local port to the receiving thread on the 1st site). I use a simple Tcl proc in route details to remove Z segment before sending the message.

If message received from 5.4 is translated message I route it to a different thread that is a local fileset thread with TPS OB proc that determines file name. This thread saves xlated messages from 5.4 in separate files – one file per original outbound thread.

That way I have two sets of files for each of 5.4 outbound threads – one xlated by 5.4, another xlated by 5.7. After running this pair of sites for a day or few I stop receiving thread from 5.4 (to keep the files in sync) and compare each pair of files 5.4 vs. 5.7. If I found discrepancies  in translated messages I fix translation on 5.7, then repeat the whole process again until I got desired results (100% match)

So I reached the point when I am happy with my translations (after comparing over 10,000 messages per translation).

Now I wonder what other kinds of tests it will be good to run? Any suggestions and tips will be appreciated

Thanks