Clovertech

Engine server spiking at 100%

› Clovertech Forums › Cloverleaf › Engine server spiking at 100%

  • Creator
    Topic
  • October 10, 2019 at 10:29 am #113015
    Mark Brown
    Participant

      We are experiencing an issue that cropped up about a month ago and has progressively gotten worse.  We’re running CL6.0.2 on a Windows 2008 server (we’re upgrading soon). What I’m observing is that when an ADT message is sent from our Cerner Millennium system to the interface engine, the CPU spikes to 100%.  We have several other feeds from Cerner and  none of them behave this way.  If we shut down this feed, the engine stops spiking.  This sustained spiking is really bogging down the  engine. Infor is puzzled. Cerner is puzzled and management is unhappy.

      Our server guy say it’s not the anti-virus/anti-malware software and he has excluded the CL folder from scanning. I’m lobbying to have it shutdown for trouble-shooting.

      Any idea what could cause this spiking to happen?  No ADT from Cerner and it all runs smoothly.

      Thanks

    • Creator
      Topic
    Viewing 5 reply threads
    • Author
      Replies
      • October 10, 2019 at 10:59 am #113016
        Keith McLeod
        Participant

          Could you possibly have a loop in your code?  Or something where you control your loop with discrete indexes as opposed to relative indexes?    I once had code where I needed to pull a certain index from a list and found that whenever an & was used it created another split and moved the values associated with the index.  Ultimately processed but was fundamentally wrong when all was said and done….  Just thoughts.  Could be in translate or tcl….

        • October 10, 2019 at 12:51 pm #113030
          Jim Kosloskey
          Participant

            Echoing Keith’s thoughts – I would investigate all Tcl code (UPoC or Xlate extensions) including inherited code or even code provided with Cloverleaf.

            Perhaps turning up the engine Output to ALL will give a clue as to what module is executed prior to the spike.

            Does this also happen in test or is it just production?

            If not happening in production, I would compare all configuration and code to see what is different.

            More investigative work needed I think.

            email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

          • October 16, 2019 at 1:11 pm #113123
            Mark Brown
            Participant

              This is happening in production. About the time this started happening we had gone live with a system called Ignite. it creates a huge alias in PID:3 that exceeds the HL7 spec for length . They provided a script to truncate this field.  We hadn’t installed it. But now we did and it has no impact.  Infor support has no idea what could cause it

            • October 16, 2019 at 2:40 pm #113126
              Jim Kosloskey
              Participant

                So does this integration also exist in a Test environment?

                If so does the spike happen there as well?

                If not – what are the differences between the Test and Production environment?

                I really doubt a too long field in a message would cause this truncated or not (unless the message length itself is excessive).

                email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

              • October 16, 2019 at 3:22 pm #113127
                Jay Clements
                Participant

                  Check the sizes of your databases in /exec/databases for each site. Anything over 500mb is unwieldy – according to Infor. We had an issue where one got to nine gigabytes do to an upstream change we had no knowledge of (accumulating data at the rate of 1 gb a day). First thing we noticed off was CPU spiking. Hope your issue is unrelated because it was a pain to figure out and fix.

                   

                   

                • October 16, 2019 at 4:00 pm #113135
                  Jim Kosloskey
                  Participant

                    Rob – good input very on topic.

                    Mark this is probably a good thing to check.

                    email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

                • Author
                  Replies
                Viewing 5 reply threads
                • You must be logged in to reply to this topic.