Site Structuring Questions

[Brandon Grudt]

I apologize if this gets long, but my questions don’t revolve around any specific problems, but deal more with the philosophy of how to build a site.  Some of the questions won’t have a “right” answer, but I’d appreciate some feedback.

A little background context before I get into my questions:

I’ve been an analyst for about 7 months now.  I’m at a small shop where I am the only analyst here, and when I stepped into the position, I quickly realized our interfaces were pretty disjointed, and not at all intuitive.  Procs were peppered around the site with no real organization, our inbound feeds were being filtered by the sending programs to custom suit what was needed at the time, which really hurt the scalability of the site.  Thread names did not conform to the any convention.

Because of this, I’ve initiated a change in how our hospital is going to route data from our main software (Paragon) to our interface engine.  Instead of multiple feeds, I’ve had McKesson create a main feed that supplies all of the messages we need.  I’ve also changed the organizational aspect of our production site quite a bit to make it more intuitive.  I’ve avoided any major pitfalls so far, but I do have some pretty broad questions about the structure.

– I’ve been to the TCL script section of the board and there seems to be a lot of good code.  Most of it, however, seems to be focused on specific issues.  What TCL scipts do you find yourself using as jack-of-all-trades scripts?  I currently use scripts to pass/kill based on field values, change field values, and remove segments.  I’m working on one that will use arguments to define the field to reference for a translation table, as the scripts we use point to specific fields for a table.  What else is handy that I am missing?

– What’s the preference between Xlate’s and TCL scripts?  I understand that they serve different purposes, but in that gray area, what do you prefer?

– For programs that are capable of handling all of the hospital ADT data, is it still better to filter off messages based on qualifying fields?  If I use TCL procs to filter all of my messages, is it likely I see any performance issues with the engine?  Most of our ADT data is routed using raw feeds that pass more data than is needed.  I want to clean that up if it doesn’t negatively affect performance.

– Is uploading a screenshot of the Network Monitor an accepted practice on here, or do many sites not want to show their hand?

– I’ve reviewed some topics on here and found that many organizations put a soft cap on the number of threads per proc, and procs per site.  What is the harm in having too many procs, relative to number of threads?  I currently structure my procs around inbound threads and loopins.  We have 7 large procs that revolve around inbound threads, as well as 2 large procs that revolve around loops.  I also have 10 procs that contain only an inbound and outbound thread each.  Should I consolidate some of these smaller procs?

– In general, are there any posts or links to information of this kind?  I’m coming into my own with the X’s and O’s but still feel I’m lacking quite a bit by not being able to draw on experience.  

Again, sorry for the novel.  I just wanted to throw some feelers out.  I appreciate any advice.

Thanks in advance.

Topic

[James Cobane]

Brandon,

I’m sure you will get lots of perspectives on your questions, so it will be a matter of determing what makes sense for you/your organization.

[Jim Kosloskey]

Brandon,

Welcome to Cloverleaf and Clovertech.

Is your platform Windows or Unix (AIX, Linux, etc.)?

Is your license thread limited?

Now for my perspective (remember these are my perspectives):

– I’ve been to the TCL script section of the board and there seems to be a lot of good code.  Most of it, however, seems to be focused on specific issues.  What TCL scipts do you find yourself using as jack-of-all-trades scripts?  I currently use scripts to pass/kill based on field values, change field values, and remove segments.  I’m working on one that will use arguments to define the field to reference for a translation table, as the scripts we use point to specific fields for a table.  What else is handy that I am missing?

My Perspective: I prefer to use reusable Tcl code – that is programs that are like tools in themselves which are driven by arguments. I prefer that within the Xlate as well rather than in-line code. This is a long discussion and I would be more than happy to echange views on this an other topics you raise at length via email of telephone (my email is at the bottom of this reply.

– What’s the preference between Xlate’s and TCL scripts?  I understand that they serve different purposes, but in that gray area, what do you prefer?

My preference: Use the Xlate until it does not do what you want for all of the reasons Jim Cobane lists, then use Tcl. Use Tcl as argument driven reusable tools rather than in-line.

– For programs that are capable of handling all of the hospital ADT data, is it still better to filter off messages based on qualifying fields?  If I use TCL procs to filter all of my messages, is it likely I see any performance issues with the engine?  Most of our ADT data is routed using raw feeds that pass more data than is needed.  I want to clean that up if it doesn’t negatively affect performance.

My preference: Filter messages within Cloverleaf using Tcl (because Cloverleaf does not yet have a filtering tool – it has been requested and hopefully one day…). We do most of our filtering using one master proc with individual reusable subroutines (reusable meaning they can be used by other main procs).

Only send the data the receiving system can prove and the sending system agrees the receiving system needs. This should be determined through an exhaustive specification process.

– Is uploading a screenshot of the Network Monitor an accepted practice on here, or do many sites not want to show their hand?

My opinion: It is acceptable I just don’t find it easy on the current forum.

– I’ve reviewed some topics on here and found that many organizations put a soft cap on the number of threads per proc, and procs per site.  What is the harm in having too many procs, relative to number of threads?  I currently structure my procs around inbound threads and loopins.  We have 7 large procs that revolve around inbound threads, as well as 2 large procs that revolve around loops.  I also have 10 procs that contain only an inbound and outbound thread each.  Should I consolidate some of these smaller procs?

My Opinion: As far as I know there is no hard limit imposed by Clovelreaf. However, practicality and ease of operations may dictate otherwise. Not to mention the capacity of your platform (hence the question above about what platform you are using). Again this is an exhaustive topic best lent to conversation. I have been a proponent of multi-site environments since my first year using Cloverleaf (1995) and maintain virtually every customer will eventually be multi-site. There is no one way to decide how to organize sites but rather a process. Again feel free to email me and we can discuss this at length.

– In general, are there any posts or links to information of this kind?  I’m coming into my own with the X’s and O’s but still feel I’m lacking quite a bit by not being able to draw on experience.

There are pros and cons regaring whether to have the vendor send everything over one connection. My preference in general is to have Message Types come over individual connections (ADT on one port, Orders on another (maybe even Lab orders, Rad orders, Dietary orders, etc. on their own ports).

You are embarking on a very exciting hourney and everyone on theis forum will be there to help you the best we can so feel free to ask.

Also feel free to post your ideas, opinions, etc.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Jeff Dinsmore]

The discussion in in the link below has some interesting discussion from when I started with Cloverleaf

<a href="https://usspvlclovertch2.infor.com/viewtopic.php?t=4508&postdays=0&postorder=asc&start=0&#8243; class=”bbcode_url”>https://usspvlclovertch2.infor.com/viewtopic.php?t=4508&postdays=0&postorder=asc&start=0

Of particular note is Bob Richardson’s statement from 6/10/2010:

Quote:

Isolate the original TCP inbound (or other protocol) and raw route the data to your outbounds, that is, distribute only with no additional processing to your multiple sites.

This has an additional – and I think more important – benefit.  If you’re routing and translating/modifying messages in your inbound process to multiple outbound destinations, and you have a failure in one of the translations, it can stop processing of any further translations for the message that fails.

For example, assume that you’re receiving on thread A and translating/routing to threads B, C and D. If the translation for destination B encounters an error – an unhandled Tcl error, for example – it will stop processing that message. The message is not delivered to destination B, but it also skips processing for C and D altogether even though the translation for them may have been just fine.

So, in addition to being more efficient, it limits translation related message drops to a single message for a single outbound interface for any given error.

Jeff Dinsmore
Chesapeake Regional Healthcare

[Brandon Grudt]

Jeff,

That thread was very useful, so much so that I’ll probably peruse it again in a week or two.  

Thanks to everyone for the info.  It’s definitely helpful.  A lot of the ideas I’ve had have been validated, but a few of them may need rethought.  If anyone cares to give some advice, I’ve uploaded screenshots of our production environment.  It is in much better shape than it was, as there is a naming standard, and the procs have been distributed to be centered around a single IB thread (mostly).

The first two numbers correspond to the proc that the thread resides on.  The rest should be pretty easy to figure out – a=adt, o=orders, r=result, c=charges or dft, v=vaccination for MU S2.  I’m currently replacing procs 01, 02, 03, 10, and 11 with what I currently have listed as 00.  Those feeds are incomplete feeds from Paragon, which I’m replacing with a single complete feed.  I’m doing this on production to more easily compare SMAT output.  

For reference, lpo and lpi stand for loopout and loopin, just TCPIP routing to localhost.  My biggest concern with the new configuration is that the ADT loopin proc will be under a lot of strain.  Any thoughts?

[James Cobane]

Brandon,

A couple of suggestions I would make.

[Brandon Grudt]

James,

The standard I came up with was:

Proc Number_Program or Description_Message Type_direction

ie. 01_pgon_ao_ib

What kind of complexity results from starting threads with numbers?  Would the same apply to ending the thread with numbers?  It’s how I sort which proc is on which thread.

Are you using Windows as the OS?  I was told by a McKesson tech that keeping thread names 15 characters or less is good practice, due to some issues with from what I understood as threads not resetting daily.  I saw this in a couple of threads prior to taking over the system, where the current SMAT file went back months.

[Jim Kosloskey]

Brandon,

The issue with the MSI Stats is it is supposed to be agnostic as to the beginning of the thread however it is not. Instead if the thread begins with a number MSI attempts to use the thread name as an index into the TOC instead of a key to the thread. Makes for some curious results.

I agree completely with Jim – don’t start your thread names with numerics (also don’t use capitalization).

As for your sending inter-process thread it is possible it may queue up.

One of the reasons could be related to you only getting one Recovery per site. If your localhost echange were to another site now you have 2 recovery DBs handling the messages and so on.

This could also depend on your volume and the capacity of your system.

I would try the architecture in Test first and apply a load equivalent to your peak demand period. Then observe whether your system is keeping up with the ‘happy numbers’ you have established/

If not you have some choices to make. If it is then it is important to have a periodic evaluation in Production during the peak demand period to discover and analyze if you are creeping up on the outer limit of your ‘happy numbers’. This will give you time to adjust your architecture.

As for associating your threads with the process in which they reside, that should be accomplishable without using a process identifier in the thread name.

So you have decided to use the ‘distribution’ architecture. That is what we use and it has served us well (we just distribute to sites).

You don’t have to fill every process or site either.

Consider these points:

   Sometimes there is a need to bring down a process and every destination is affected. Here and I suspect there as well, we have to negotiate a good time for everyone affected. If you can associate those receiving sites which can tolerate common down times you could put them in the same process.

   We all have integrations (not always many – sometimes too many) which do not ‘play nice with the other kids’. Things like they can’t keep up with the message flow (thus causing deep queues) or they are consantly problematic in some othr way. You could associate them together (especially in a separate site) minimizing the imapct on the nice kids. Thus the ‘bad’ kids can fight it out among themselves.

There are other plausible reasons to organize sites.

The proper and active use of the Master Site makes multiple site architecture more manageable.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Brandon Grudt]

Jim,

As of now, our peak demand hasn’t given me any reason to believe there is any lag, which is nice.  Outside of a couple of “problem” threads that seem to take a while to process, everything passes quickly.  I don’t know how soon I will bottom out though, as I seem to have a new project every day.

I’d love to be more educated on the volume of my messages I pass, but I have never even heard of MSI until this thread.  I’m going to look into it a bit more, as it is something that I feel would be important in order to really know what is going on with the interfaces.

Quick question, is there any reason not to run multiple instances of the Cloverleaf client on my PC, so long as I am on different sites on each?  It would make navigating the sites a lot easier.

[Jim Kosloskey]

Brandon,

You can have as many clients open as your PC can tolerate.

When I say peak demand period I am not talking about utilization I am talking arrival rate.

So counting the number of messages/records during a peak period.

That gives you the demand on the engine. Then separate measurements can be taken to observe the utlization or service levels to make sure they fall within your ‘happy number’ range.

I use archived SMAT files to determine a peak demand period. Once that is determined it only needs to be validated periodically. For a slow growth environment twice a year would probably suffice.

Once that peak demand period is established, measurements to match against ‘happy numbers’ only need to take place during that period.

The peak demand period is usually expressed as either a 15 minute or 60 minute time frame on the peak demand day (shorter time periods can be used for a day but longer time periods don’t make sense). The exact time frame for you will be deteremined by observation.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[James Cobane]

Brandon,

The thread naming convention of 15-characters is simply a display issue for output from the hci commands, (i.e. hciconnstatus, etc.), not a technical limitation, nor should it have any affect on being able to reset stats or cycling of message files.  I would take a look at the scripts you are invoking to cycle the messages/stats to see how they are referencing the threadnames.  Also keep in mind that a thread needs to be running to cycle the SMAT files, so if you have threads that are down during the time the script is trying to cycle, they won’t.

Jim Cobane

Henry Ford Health

[Brandon Grudt]

Jim,

I guess my reply was misleading.  I was talking about peak hours as well, but further qualifying that by saying that in the near future, the peak usage will be higher.

Thanks for the advice, all.  I think I am going to take the new pgon_all thread to another site before launching it.  That will mitigate about 50% of all traffic on my current production site.  Instead of waiting until it is needed, I may as well plan ahead.

Threads like this really help us new guys, so if and when I have questions about the process, I’ll make sure to add it here to bump the entire thread.  Is there any advice on creating a new site, or is that a pretty standard RTM type question?

[Richard Hart]

We use a standard naming convention, indicating the application, ‘level’ message flow, internal or external to Cloverleaf.

This is an example of a PAS application site. The rcv and snd are for half-duplex communication and ‘out’ communicates to another Cloverleaf site.

Process              Connection                          State Proto Status Count  Started            

---

---

---

---

---

---

 

top_test_os          cdc_test_os_adt_out                 up    up           0      19/03/13 17:03:10  

top_test_os          cdd_test_os_adt_out                 up    up           0      19/03/13 17:03:11  

top_test_os          cdr_test_os_adt_out                 up    up           0      19/03/13 17:03:09  

top_test_os          har_test_os_adt_out                 up    up           0      19/03/13 17:03:12  

top_test_os          hih_test_os_adt_out                 up    up           0      19/03/13 17:03:11  

top_test_os          icm_test_os_adt_out                 up    up           0      19/03/13 17:03:11  

top_test_os          pbr_test_os_adt_out                 up    up           0      19/03/13 17:03:10  

top_test_os          pia_test_os_adt_out                 up    up           0      19/03/13 17:03:11  

top_test_os          pma_test_os_adt_out                 up    up           0      19/03/13 17:03:10  

top_test_os          qcr_test_os_adt_out                 up    up           0      19/03/13 17:03:09  

top_test_os          ris_test_os_adt_out                 up    up           0      19/03/13 17:03:10  

top_test_os          snk_tuat_os_adt_out                 up    up           0      19/03/13 17:03:11  

top_test_os          sol_test_os_adt_out                 up    up           0      19/03/13 17:03:11  

top_test_os          top_test_os_adt_rcv                 up    up           0      19/03/13 17:03:10  

top_test_os          top_test_os_adt_snd                 up    up           0      19/03/13 17:03:11  

top_test_os          ult_test_os_adt_out                 up    up           0      19/03/13 17:03:10

This is an example of an application site, the ‘in’ thread communicates with another Cloverleaf sites ‘out’ thread and the ‘snd’ thread communicates to an external application.

Process              Connection                          State Proto Status Count  Started            

---

---

---

---

---

---

 

cdc_test_os          cdc_test_os_adt_snd                 up    up           0      27/03/13 12:41:57  

cdc_test_os          top_test_os_adt_in                  up    up           0      27/03/13 14:48:04

[Darcy Kemp]

Hello,

I am new to Clovertech and new to my hospital, but not new to Cloverleaf.  I am looking to re-organize an engine and am very interested in theories of naming conventions and process organization.

I worked on an engine where we re-organized obsoleted process and thread system names, to generic, like HIS, RIS, LIS… etc., and it seemed to help as systems were replaced.   What are your thoughts?

In my new hospital, we have names for example, like ‘abc_adt_in’.  I always found the direction in the later part of the name to be non-intuitive, so I named them “in_frm_his”, but I don’t want to confuse my co-workers by changing the names, unless they can easily translate them.  

How you group your processes?  Our current engine is set up by system group, but I have used, critical vs. non critical, (like Clinical, ADT, CHGS, MFN/MDM,…so non-criticals didn’t alert me at night), and really would like to know how others have decided to organize their processes.  

Thank You in Advance.

[Brandon Grudt]

Darcy,

I read through this thread and it seems like AGES ago.  After receiving some good advice on here, I settled on a standardized thread naming structure.  

All of my threads look like this –

pa_01_i_pgon_aor

I use the underscore to split the thread name in tcl for various things.  The sub-components are as follows –

1. pa – This stands for PROD_A or my first production site.  This give site information to locate the thread.  for example, tc would be TEST_C.

2. 01 – This is the proc in which the thread is located.

3. i – This stands for inbound.  It’s the direction of the messages in relation to the engine.  O is for outbound, L is for loop, and M is for merge, which I use for my destinations if they merge feeds for OB delivery.

4. pgon – This is the four character code of the system, in this case, Paragon.  I kept it at a fixed length in case I needed it to be, but I’ve never really run into anything that I need a fixed length for.

5. aor – ADT, Order, Results.  There are also m – MDM, d – DFT, b – BAR, etc.

I hope this helps.  It’s worked well for me.  It’s nice to have the entire breakdown of the thread listed in the name.  I’d probably never find anything if I didn’t list it in this way…

[James Cobane]

Here is an excerpt of some of our “guidelines” regarding naming conventions for threads and processes:

[Darcy Kemp]

Thank you.  That is very helpful.

[Darcy Kemp]

I appreciate your input, thank you!

[Author]

Replies