Philosophical Issues with Data Manipulation

[Bryan Dort]

I

Topic

[Michael Hertel]

I’m fairly flexible with adt and orders changes unless the analysts are on a fishing expedition trying to “see” if “this” or “that” works.

Two things I am not flexible on are:

1) Altering results data.

2) Using translation tables on the engine.

Sometimes though, in the past, I’ve gotten the line, “I can ask you to do it or have you told to do it”. In which case, I feel your pain.

[Jim Kosloskey]

I try to insist we create a detailed specification of the integration.

That is, there is a description of the inbound message down to the component/sub-component level as well as the outbound message down to the component/sub-component level. The document also contains specifications for the protocol, message flow, testing plans, exceptional conditions, etc. When the document is completed, the Integration Engineer knows what needs to be done and the receiving system and user should have a clearer understanding of what data will be received and how it will integrate into and look in their system.

A team constructs the specification indicating in the outbound section exactly how each component/sub-component will be populated. This team includes representative(s) of the sending system (typically a user or analyst and sometimes a vendor); the integration engineer(s) assigned; and representative(s) of the receiving system (again typically a user or analyst and frequently a vendor). Typically the project manager also attends.

During this time, virtually all potential data integrity issues are raised and resolved. Other issues are also raised and resolved.

If the team cannot resolve an issue, the project manager takes the issue to the appropriate person(s) to get a resolution.

If, during the specification building process, unreasonable requests are made, they become issues which traverse through issue resolution.

Having a representative of the sending system on the team fairly assures oversight on valid reconstructions.

We use Translation Tables if appropriate (we do have a resistance to using Cloverleaf(R) translation tables for frequently changing data).

We have no fast rule against reconstructing results or any potential message type. Frankly there are simply situations where the sending system sends results as repeating segments and the receiving system wants those as repeating OBX-5 fields or vice versa, for example.

There are many other such examples and as long as the integrity of the result is preserved we accomodate to get the job done.

So, in our situation (and the situation in virtually everywhere I have done Integrations for the past 35 years), each integration is evaluated by a team on its needs and balanced against what the players (sending system and receiving system and management) agree is proper.

There can be some rather vigorous discussions during the specification generation phase – but hopefully the outcome is a functioning integration that properly represents the information being integrated.

Jim Kosloskey

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Richard Hart]

Like Jim, I request that we received two detailed specs for the sending and receiving applications and progress from there.

In my experience, many vendors don’t provide specs, or they are inaccurate!

For PAS messages, we delete message that aren’t required, create (based on input) messages that are required and manipulate as required.

For LAB messages, we have application specific code tables where we change codes or add an extra OBX as required. In some cases, a LAB application communicates Pending and (later) Final results, so we process messages and delete if all contents are pending.

We also route LAB messages based on content and produce reports for an application support group for messages where we cannot determine exactly if the message is required

We don’t change the textual messages unless requested to cancel a LAB message by the support group.

We store raw messages as they enter Cloverleaf (SMAT) and display ‘old’, ‘new’ or ‘del’ message in the process log file.  We also send email alerts to support groups if something unusual (as defined by the support group) occurs.

My personal view is that an application’s installation should be vanilla and Cloverleaf should be used to perform for as much ‘work’ as required.  In reality there is often vendor application work required.

In some projects, management have attempted to keep Cloverleaf as a ‘delivery’ application and request vendor changes!!  The vendors love these guys!!

[Author]

Replies