Throttling messages inbound to Cloverleaf from a foreign sys

Homepage Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Throttling messages inbound to Cloverleaf from a foreign sys

  • Creator
    Topic
  • #54590
    John Bonnie
    Participant

    Hello all,

    I’m looking for a way to throttle messages inbound to Cloverleaf from a foreign system.  In this case I want an inbound thread to process only one hl7 tcp/ip message a second from a foreign system.  Any ideas for this?

    Thanks in advance.

Viewing 5 reply threads
  • Author
    Replies
    • #82119
      Robert Kersemakers
      Participant

      Hi John,

      I wouldn’t try to throttle messages as they are being received by Cloverleaf. You could put in a ‘sleep 1’ or something similar at the moment you send back an ACK, which would take care of your problem. But this ‘sleep 1’ will stop the entire process! So not a good idea.

      Normally you will do this because the receiving system isn’t capable to process a large amount of messages. So just receive the messages from your sending system and write each message as a file in a certain directory. Then use fileset-local to read this directory and only process 1 file per second.

      You can even do this in 1 thread.

      A -> B -> C

      A the thread that receives the messages from your sending system.

      B is a fileset-local thread that writes the messages to a directory (outbound). But B is also inbound, so it reads the same directory and sends a message to C. You can configure this in such a way that only one file/message per second is taken.

      C is the thread sending the messages to the receiving system.

      Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

    • #82120
      Terry Kellum
      Participant

      I once did this with a sqlite table.  One process would put in the transaction with a time stamp.  The next process inline would search the table for transactions with timestamp less than (now – x).  The tricky part is that you need to also mind the transactions metadata.

      I did this because our LIS would process orders quicker than ADTs, and we were generating ADTs from the order transactions.  The order would hit without a lillypad to sit on.  Slowing down the orders by 5 seconds allowed the ADT to hit first.

    • #82121
      Mark Thompson
      Participant

      Terry,

      Were you able to preserve the metadata – especially Original Message MID?

      We were looking at switching our file based queues into a database, and hoping to preserve the metadata as a bonus.

      - Mark Thompson
      HealthPartners

    • #82122
      Terry Kellum
      Participant

      I think I was dummying up the metadata.  I knew where it was going so there are only about three fields that you NEED if you’re not doing something complex.  I would think that you could save the metadata using msgmetaget.  You could then set the metadata on the newly (re) minted message using msgmetaset.  RTFM has some good data regarding the RO and RW fields.  (TCL Extensions).

      Here’s the code that I used.  Keep in mind that this code may cross the streams, or result in causing our own sol to go supernova.  If you break your engine or messages with it, you own both pieces.

      Code:


      ######################################################################
      # Name: delay_msg.tcl
      # Purpose: Provides delay of messages moving thru the engine.
      #                Places the message into a sqlite database table stamped with the
      #                tcl time (clock seconds).  In read_delay_msg the databse tables
      #                is queried for messages that are old enough to be sent on.
      #                Suggested use is in UPOC protocol.  “write” in outbound and
      #                “read” in inbound.
      #
      # UPoC type: tps
      # Args: tps keyedlist containing the following keys:
      #       MODE    run mode (”start”, “run” or “time”)
      #       MSGID   message handle
      #       ARGS    user-supplied arguments:
      #               TIME : Numeric Number of Seconds to delay
      #
      # Returns: tps disposition list:
      #          
      #
      ###############################################
      # Modification and Notes
      ###############################################
      # 20100428 – Final cleanups for deployment to outreach site
      #
      # 20110805 – Terry Kellum – Added delay message for test.
      #                            write_delay_msg_test
      #                            read_delay_msg_test

      proc write_delay_msg { args } {
         package require sqlite
         keylget args MODE mode               ;# Fetch mode
         keylget args CONTEXT context

         set dispList {} ;# Nothing to return
         
         
         ###################################################################
         # Main Code
         
         # Setup the db
         sqlite DelayDB ./delayed_transaction_database.db
         
         switch -exact — $mode {
             start {
                 # Perform special init functions
         # N.B.: there may or may not be a MSGID key in args
         
         # Setup the Table
         DelayDB eval {CREATE TABLE IF NOT EXISTS dTransList(transnum integer primary key autoincrement,
      insert_secs integer,
      trans text,
      meta text)}
         
         set NumOldRecs [DelayDB eval {select COUNT(*) from dTransList}]
         echo ”  >> tclproc: write_delay_msg -> Database Initialized. $NumOldRecs found.”
         echo ”                           Directory: [pwd]”
         echo ”  >> tclproc: write_delay_msg -> Context: $context n n”
         
             }

             run {
         # ‘run’ mode always has a MSGID; fetch and process it
                 keylget args MSGID mh
         set msg [msgget $mh]
         set meta_fields [msgmetaget -rw $mh]
         set metalist “”
         foreach field $meta_fields {
      lappend metalist [list $field [msgmetaget $mh $field]]
         }
         #echo $metalist
         set currsecs [clock seconds]
         if {[catch {DelayDB eval {INSERT INTO dTransList VALUES(NULL,$currsecs,$msg,$metalist)}} result] } {
      echo ”  >> tclproc: write_delay_msg -> FAILED insert – Continuing message.”
      lappend dispList “CONTINUE $mh”
         } else {
      # If this is in UPOC Write, then we continue the message
      if {[string equal $context pdupoc_write] == 1} {
         lappend dispList “CONTINUE $mh”
      } else {
         lappend dispList “KILL $mh”    
      }
         }
         
             }

             time {
                 # Timer-based processing
         # N.B.: there may or may not be a MSGID key in args
             }
             
             shutdown {
         # Doing some clean-up work
         DelayDB eval {VACUUM}
      }
         }

         return $dispList
      }

      proc read_delay_msg { args } {
         package require sqlite
         keylget args MODE mode               ;# Fetch mode
         keylget args CONTEXT context
         keylget args ARGS arguments
         keylget arguments TIME delay

         set dispList {} ;# Nothing to return
         
         
         ###################################################################
         # Main Code
         
         # Setup the db
         sqlite DelayDB ./delayed_transaction_database.db
         

         switch -exact — $mode {
             start {
                 # Perform special init functions
         # N.B.: there may or may not be a MSGID key in args
         
         # Setup the Table
         DelayDB eval {CREATE TABLE IF NOT EXISTS dTransList(transnum integer primary key autoincrement,
      insert_secs integer,
      trans text,
      meta text)}
         
         set NumOldRecs [DelayDB eval {select COUNT(*) from dTransList}]
         echo ”  >> tclproc: read_delay_msg -> Database Initialized. $NumOldRecs found.”
         echo ”                           Directory: [pwd]”
         echo ”  >> tclproc: read_delay_msg -> Context: $context”
         echo ”  >> tclproc: read_delay_msg -> Delay: $delay n n”
         
             }

             run {
         # ‘run’ mode always has a MSGID; fetch and process it
                   #keylget args MSGID mh
         #Run mode not used here
             }

             time {
                 # Timer-based processing
         # N.B.: there may or may not be a MSGID key in args
         # Set 60 Second delay
         set cutoff [expr [clock seconds] – 60]
         DelayDB eval {SELECT transnum, trans, meta FROM dTransList WHERE insert_secs < $cutoff} row {
      #echo "num = $row(transnum)"
      #echo "trans = $row(trans)"
      #echo "meta = $row(meta)"
      set mh [msgcreate]
      msgset $mh $row(trans)
      msgmetaset $mh PRIORITY 5120
      DelayDB eval {delete from dTransList where transnum = $row(transnum)}
      lappend dispList "CONTINUE $mh"
         }
         
             }
             
             shutdown {
         # Doing some clean-up work
         # vacuum done by creating table in write.
      }
         }

         return $dispList
      }

      Looks like I wasn’t doing much with the metadata that I saved.  Perhaps that is because I did my xlate on the inbound side, and raw-routed on the outbound side.  I used an interval of 5 on the inbound UPOC (read_delay_msg).

    • #82123
      Richard Hart
      Participant

      I’ve previously throttled inbound messaging by delaying the ACK sent back from Cloverleaf using the thread ‘phold_obd’ feature.

      I’ve not attempted what you are trying achieve, but it may be worh investigating.

      Note that this will only work in the ACK message is of type data.

    • #82124
      Rob Lindsey
      Participant

      You could build a “wait” loop into your ACK to slow down the inbound data.  I would not recommend the   sleep   command or the   after    command because both of them block the TCL interrupter.  

      There is however a set of commands that is CPU intensive

      set WaitCnt 0 ; while { $WaitCnt < 10000000 } { incr WaitCnt }

      Rob

Viewing 5 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
5,117
Forums
28
Topics
9,292
Replies
34,435
Topic Tags
286
Empty Topic Tags
10