Mismatch of message sent to message in SMAT

[Lonnie Wright]

I have an issue where the messages stored in the smat file are different than the messages being sent by the ancillary system.  To verify that the sending system was not at fault, I set up a hcitcptest and it received the message as shown by the sending system.

This is a very old interface from the late 1990s and the developer is no longer with us.  We are adding several new facilities and I need to set the interface to handle them.

The difference between the messages is the MSH Sending Facility Field (see sample below).  The pdl being used is the standard mlp_tcp.pdl, and I see nothing in it that would do this.  I have searched through all of the libraries for something that could be changing the data and found nothing.

Any suggestions on where I should search would be appreciated.

Message MSH sent by vendor:

MSH|^~&|MEDILINKS|ECLH|EMORY|ECLH|20120506180149||DFT^P03|aa00a7cc9d07465f94bede9fef686a77|T|2.4

Message MSH in SMAT:

MSH|^~&|MEDILINKS|C>RESP|EMORY|ECLH|20120506180149||DFT^P03|aa00a7cc9d07465f94bede9fef686a77|T|2.4

Topic

[David Barr]

To narrow the problem down you could try echoing the message in the inbound TPS proc.

Is the protocol pdl-tcpip?

What’s the encoding? Your version of Cloverleaf might not have this.

[Lonnie Wright]

By TPS proc do you mean the SMS/in_data proc?

The protocol is pdl-tcpip.

Because of decisions made by our management, we are still running Cloverleaf 5.1.

[David Barr]

Lonnie Wright wrote:

By TPS proc do you mean the SMS/in_data proc?

Yes.

[Russ Ross]

My first off the cuff thought is to confirm that there isn’t another additional unknown interface communicating on the same port.

You mentioned adding new facilities and they might of duplicated the interface at another facility creating the situation you might be describing.

I’m on AIX and iptrace is a command I’ve used to help rule this sort of thing out in the past.

I think I’ve also used lsof to help with this type of problem.

In the past when I encountered 2 interfaces using the same port it was caused by one of these situations:

– a test interface was also pointed to PROD or vice versa.

– disaster recovery interface was turned on without knowing it.

– use of a port number in the ephemeral range (32K-64K = 32,768 – 65,536)

– use of an unrealsitic port number above 64K

Russ Ross
RussRoss318@gmail.com

[Lonnie Wright]

As you can see below, the change has been made before it reaches the SMS/IN_DATA proc.

This is what was sent:

MSH|^~&|MEDILINKS|ECLH|EMORY|ECLH|20120508070154||DFT^P03|837db85768f04f52901e5fdb0f132798|T|2.4

EVN|P03|20120508070154|20120508070154

PID|||6546546||SUAVE^RICO||19710303|M|||||(   )   -|||||TEST123456

FT1|||050812|20120508100000|20120508070154|C1|17194640||SUAVE, RICO|1||0|CHG_MED_THERA|||||IN|||VRT6ZJP

And this is what is displayed by the in_data proc:

MSH|^~&|MEDILINKS|C>RESP|EMORY|ECLH|20120508070154||DFT^P03|837db85768f04f52901e5fdb0f132798|T|2.4^MEVN|P03|20120508070154|20120508070154^MPID|||6546546||SUAVE^RICO||19710303|M|||||(   )  -||||TEST123456^MFT1|||050812|20120508100000|20120508070154|C1|17194640||SUAVE, RICO|1||0|CHG_MED_THERA|||||IN|||VRT6ZJP

[Lonnie Wright]

Russ,

This is happening in both the PROD and TEST which are run on 2 different AIX servers.  The change is intentional because the routing script is set to use the modified facility.  

There are no other interfaces on the system using the same port, I used netstat to verify this as well as a search of all of the NetConfigs on the server.

The hcitcptest tool receives the message in the unaltered form which indicates that the modification is taking place somewhere in Cloverleaf, but the SMAT file and an echo from a SMS/IN_DATA proc shows the altered transaction.  The only place I can think of that could be changing it before this point is the PDL, but that is the standard mlp_tcp.pdl.

I have coded the routing UPOC to get around this by using the facility as sent by the vendor, but I really would like to know where this is being changed.

I was hoping someone on this forum could tell me where to look, if not the next step will be to contact support.

[Russ Ross]

If you think the problem is in the PDL, which your hcitcptest not matching SMAT certainly supports that possibility, have you confirmed there isn’t a site specific PDL being used in $HCISITEDIR/pdls?

If there isn’t a site specific PDL maybe create one and put some echos in it to try and debug the PDL to determine if it is the culprit.

If you go that way remember you have to complie PDLs for the changes to take effect, for example:

cd $HCISITEDIR/pdls

hcipdc my.pdl

which will create a my.pdo file.

Sometimes when I get stuck and can’t find a tree in the forest, I start with a new site and a simple interface and keep adding to it until I either:

– make the problem go away and I don’t know what I did to fix it

– I find the problem by this incremental approach of replanting each tree one at a time.

Russ Ross
RussRoss318@gmail.com

[John Hamilton]

I would suggest that you try a trace. In the past in solving problems such as this. I have always confirmed with a network trace what was on the wire.   The results have sometimes surprised everyone.

[Lonnie Wright]

Russ,

Good suggestion.  I checked the site pdls directory when I first discovered the problem, but I had not thought of modifying the mlp_tcp.pdl to add an echo.  I’ll change the name to be sure it gets the correct one.

John,

Are you talking about a network trace?  It seems that since the hcitcptest received the unmodified message, that the correct message is making it to the server, but I’m willing to try anything.  I’m going to try the pdl change first and if that doesn’t help, I’ll try your suggestion.

[Lonnie Wright]

John, sorry, you did say network trace in your message, I just missed it the first time.

[Russ Ross]

That makes me realize I could of clarified that when I said iptrace early, that I presumed others understood that to mean network trace.

Russ Ross
RussRoss318@gmail.com

[David Barr]

I’m not sure what’s going on. You might want to do something like “find / -type f | xargs grep -i ‘C>RESP'” to see if you can find some code that might be doing this.

[Russ Ross]

Check to see if you have more than one interface in the same Cloverleaf site configured to write to the same SMAT file.

Sometimes this happens in the Netconfig when copying a thread within the site and forgetting to change the SMAT file name after making the copy.

Russ Ross
RussRoss318@gmail.com

[Lonnie Wright]

Thanks to all of you for your help.  

It turns out that the code that changed the facility field was on the sending system.  The resend that they did the first time I tested with the hcitcptest was not sent through the normal path, so it was not changed.  After running several tests that some of you recommended, I set up another hcitcptest and this time asked them to create a new transaction.  This one came across with the facility changed.  I then showed that to a programmer on their side and he was able to find the code and make the needed modification.

I’m sorry to have run you down a wild goose change, but I truly do appreciate your help.

[Author]

Replies