Delaying Outbound messages.

Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Delaying Outbound messages.

  • Creator
    Topic
  • #50076
    John Hamilton
    Participant

      I have unique problem.

      First I’m running AIX 5.3 cloverleaf 5.5 Rev1.

      This is a one time conversion process. I have 10 years of data about 2 million messages to send.

      I need to limit the number of message I send to about 60,000 per day.

      The outbound system can generate 100,000 plus per day.  

      If you were given this issue how would you limit the outbound message to what the receiving system can handle ?

      Any thoughts suggestion are welcome.

    Viewing 4 reply threads
    • Author
      Replies
      • #64780
        Scott Lee
        Participant

          I had a similar requirement for an outbound thread that the recieving system could not handle the volume of messages the sending system was supplying.  I added a ‘sleep’ command to a tclproc in the SEND OK UPOC.

          A couple caveats – Be careful you are not tying up the process manager if there are other threads in the process.  Also be careful you will not have too many messages queueing up in the engine.

          Here is the proc…

          Code:

          ######################################################################
          # Name: tps_msg_throttle
          # Programmer: Scott R. Lee
          # Purpose: Slow down messages going out to EDM.  Charges from Affinity
          # can be sent too quickly for EDM to handle them and
          # other messages (mainly ADTs) get queued up behind them.
          #
          # Edits: 6/4/07 – Scott R. Lee
          # Changed sleep time from 1 second to .5 seconds.
          # 6/6/07 – Scott R. Lee
          # Changed sleep time back to 1 second.
          #
          # UPoC type: tps
          # Args: tps keyedlist containing the following keys:
          #       MODE    run mode (”start”, “run” or “time”)
          #       MSGID   message handle
          #       ARGS    user-supplied arguments:
          #              
          #
          # Returns: tps disposition list:
          #          

          #

          proc tps_msg_throttle { args } {
          keylget args MODE mode               ;# Fetch mode

          set dispList {} ;# Nothing to return

          switch -exact — $mode {
          start {
          # Perform special init functions
          # N.B.: there may or may not be a MSGID key in args
          }

          run {
          # ‘run’ mode always has a MSGID; fetch and process it
          keylget args MSGID mh
          echo sleeping 1
          sleep 1
          lappend dispList “CONTINUE $mh”
          }

          time {
          # Timer-based processing
          # N.B.: there may or may not be a MSGID key in args
          }
                 
          shutdown {
          # Doing some clean-up work
          }
          }

          return $dispList
          }

          HTH

          Scott

        • #64781
          Keith McLeod
          Participant

            Are the messages in a file and Newline delimited?  If they are in a file and you are on UNIX, you can use the UNIX split command to create smaller files to resend.  I recently did a conversion and found that the receiving system would create files with particular extension for the type of message it was receiving.  I scripted individual messages into multiple zip files that we assigned a person to extract every few minutes to allow production messages to continue to flow with minimal delay.  This removed the overhead off of the engine.  If I had left the processing on the engine, it would have taken days with current production backing up….

            This happened to be GE Centricity so it depends on your destination system for similar ways to address.

          • #64782
            Russ Ross
            Participant

              John:

              Look at my posts at URL:

              https://usspvlclovertch2.infor.com/viewtopic.php?t=1621&” class=”bbcode_url”>https://usspvlclovertch2.infor.com/viewtopic.php?t=1621&

              where I describe how we handle downtime mode and resending and throttle massive amounts of messages when the foreign system comes back up.

              Russ Ross
              RussRoss318@gmail.com

            • #64783
              Richard Hart
              Participant

                Hi John.

                We performed a data load of millions of messages a few years ago.  We were sending to an application that internally queued message before processing, so we didn’t get an application ACK until later.

                This is how our solution worked .

                The messages were in zipped (gzip on Unix) SMAT files that we had cycled daily over the years.

                We wrote some TPS that scanned the data directory and returned a list that was either blank or contained one file to read.

                A blank list was returned if:

                   a ‘stop’ flag was set (a simple file existence check); or

                   the difference between the message sent and ACKs received was greater than a threshold value; or    

                   a SMAT file was the next file and this was unpacked in the background.

                A file was returned if:

                   a normal file was.

                This ensured that we had easy control to stop the message flow – once the code was scanning the directory.

                We ensured that the processing application queue (MSMQ) was never too large as a crash would lose the messages!

                Messages came from our SMAT files, so we could use ‘cron’ or ‘at’ to load more SMAT files in.

                We had this going for about 3 months!

              • #64784
                Russ Ross
                Participant

                  John:

                  Richard’s post reminded me of a point worth mentioning and you may know this already but not everyone will.

                  When I end up with one very large new-line terminated file of messages to resend, I can use the AIX/UNIX split command to chop the file up into smaller files and send those one at a time.

                  For example:

                  split -10000 bigFile

                  will split the bigFile into smaller files with 10,000 messages in each of the smaller files.

                  Here is my generic resend script ( resend_que.ksh ) that I call to automate doing most of the work of resending multiple files:

                  Code:

                  #!/usr/bin/ksh

                  # Begin Module Header ==============================================================================
                  #
                  #
                  #——
                  # Name:
                  #——
                  #
                  #   resend_que.ksh
                  #
                  #———
                  # Purpose:
                  #———
                  #
                  #   will resend the next que file in name sorted order
                  #
                  #   the requirements are that the resend process is already running and the outbound thread is UP
                  #
                  #——-
                  # Notes:
                  #——-
                  #
                  # You need to at least manually start the resend process and confirm the resend_thread is up
                  # before invoking this script
                  #
                  # Have the calling script export/set these arguements before invoking this script.
                  #
                  # Here is an example:
                  #
                  #    export que_dir=”$HCISITEDIR/data/golive_archive”
                  #    export load_dir=”$HCISITEDIR/data/load_rxtfc_adt_9027″
                  #    export sent_dir=”$HCISITEDIR/data/golive_archive_resent”
                  #    export sent_log=$sent_dir/.log
                  #    export kill_file=”$que_dir/.kill”
                  #    export lock_file=”$que_dir/.lock”
                  #    export rs_proc=”resend_rxtfc_adt_9027″
                  #    export load_thread=”load_rxtfc_adt_9027″
                  #    export rs_thread=”rs_rxtfc_adt_9027″
                  #    resend_que.ksh
                  #
                  # Be sure to make an entry in /hci/cron/reminders.ksh as long as this script is scheduled to run via cron
                  #
                  #———
                  # History:
                  #———
                  #
                  #   2004.04.28 Russ Ross
                  #             – wrote intial version
                  #
                  #
                  #   2004.04.30 Russ Ross
                  #             – modified to only recycle the load thread instead of recycling the process
                  #             – modified so this script will not run unless the following conditions are true:
                  #
                  #                  the resend process is running
                  #                  the resend thread is UP
                  #
                  # End of Module Header =============================================================================

                  #—————————————-
                  # make sure the resend process is running
                  #—————————————-

                     if [ ! -f $HCISITEDIR/exec/processes/$rs_proc/pid ]; then
                         echo “”
                         echo ”     STOPPING because the resend process is not running”
                         echo ”     $rs_proc”
                         echo “”
                         exit
                     fi

                  #———————————————–
                  # make sure the resend thread is UP=2 or INEOF=6
                  #———————————————–

                     proto_status=`hcimsiutil -dd $rs_thread | grep “^Proto Status”     | awk -F: ‘{print $2}’ | tr -d ‘ ‘`
                     if [[ “$proto_status” != “2” && “$proto_status” != “6” ]]; then
                         echo “”
                         echo ”     STOPPING because the resend thread is not UP or INEOF”
                         echo ”     $rs_thread”
                         echo “”
                         exit
                     fi

                  #—————————————-
                  # check if the necessary directories exit
                  #—————————————-

                     if [ ! -d $que_dir ]; then
                         echo “”
                         echo ”     FATAL ERROR – directory does not exist”
                         echo ”     $que_dir”
                         echo “”
                         exit
                     fi

                     if [ ! -d $load_dir ]; then
                         echo “”
                         echo ”     FATAL ERROR – directory does not exist”
                         echo ”     $load_dir”
                         echo “”
                         exit
                     fi

                     if [ ! -d $sent_dir ]; then
                         echo “”
                         echo ”     FATAL ERROR – directory does not exist”
                         echo ”     $sent_dir”
                         echo “”
                         exit
                     fi

                  #——————————————
                  # check if this script has been toggled off
                  #——————————————

                     if [ -f $kill_file ]; then
                         echo “”
                         echo ”     STOPPING because this script has been toggled off”
                         echo ”     $kill_file”
                         echo “”
                         exit
                     fi

                  #—————————————-
                  # check if this script is already running
                  #—————————————-

                     if [ -f $lock_file ]; then
                         echo “”
                         echo ”     STOPPING because this script is already running”
                         echo ”     $lock_file”
                         echo “”
                         exit
                     fi

                  #————————————————————
                  # create lock file to prevent more than one occurance running
                  #————————————————————

                     touch $lock_file

                  #————————————-
                  # make sure the last load file is done
                  #————————————-

                     if [ -f $load_dir/* ]; then
                         echo “”
                         echo ”     STOPPING because not done resending last load file in directory”
                         echo ”     $load_dir”
                         echo “”
                         rm -f $lock_file
                         exit
                     fi

                  #—————————————————————————
                  # make sure that no outbound messages are queued up over the next 60 seconds
                  #—————————————————————————

                     let count=1

                     while ((count <= 7)); do        ob_que_depth=`hcimsiutil -dd $rs_thread | grep "^OB Data QD" | awk -F: '{print $2}' | tr -d ' '`        if [[ $ob_que_depth > 0 ]]; then
                             echo “”
                             echo ”     STOPPING because outbound messages are still queued up”
                             echo “”
                             rm -f $lock_file
                             exit
                         fi

                         sleep 10
                         let count=count+1

                     done

                  #————————————-
                  # stop thread if it is UP=2 or INEOF=6
                  #————————————-

                     proto_status=`hcimsiutil -dd $load_thread | grep “^Proto Status”     | awk -F: ‘{print $2}’ | tr -d ‘ ‘`
                     if [[ “$proto_status” = “2” || “$proto_status” = “6” ]]; then
                         hcicmd -p $rs_proc -c “$load_thread pstop”
                         sleep 5
                     fi

                  #——————————————-
                  # zero out thread counts for the load thread
                  #——————————————-

                     hcimsiutil -zt $load_thread >/dev/null
                     sleep 5

                  #—————————————-
                  # get the name of the next file to resend
                  #—————————————-

                     resend_file=””

                     if [ -f $que_dir/* ]; then
                         resend_file=`cd $que_dir; ls -1 * | sort | head -1`
                     fi

                     if [[ “$resend_file” = “” ]]; then
                         echo “resend_que.ksh – no file found”
                         rm -f $lock_file
                         exit
                     fi

                  #——————————–
                  # make entry into resend log file
                  #——————————–

                     dts=`date “+%Y.%m.%d_%H:%M:%S”`
                     echo “$dts   resending   $resend_file” >> $sent_log

                  #—————————————————–
                  # copy the next file to resend into the load directory
                  #—————————————————–

                     cp -p   $que_dir/$resend_file   $load_dir

                  #——————————————————-
                  # move the next file to resend into the resent directory
                  #——————————————————-

                     mv   $que_dir/$resend_file   $sent_dir

                  #———————-
                  # start the load thread
                  #———————-

                     hcicmd -p $rs_proc -c “$load_thread pstart”
                     sleep 5

                  #———————
                  # get rid of lock file
                  #———————

                     rm -f $lock_file

                  Here is an example of a calling script ( rkr_xcelera_resend.ksh ):

                  Code:

                  #!/usr/bin/ksh

                  # Begin Module Header ==============================================================================
                  #
                  #
                  #——
                  # Name:
                  #——
                  #
                  #   rkr_xcelera_resend.ksh
                  #
                  #———
                  # Purpose:
                  #———
                  #
                  #   export/set the variables that get used by the generic script resend_que.ksh
                  #
                  #   will resend the next que file in name sorted order
                  #
                  #   if you schedule this in cron to run every 10 minutes
                  #   it will resend each que file one by one in name sorted order
                  #   by cycling the resend process and thread counts for each file sent before proceeding to the next file
                  #   plus will perpetually restartied every 10 minutes until all files have been resent
                  #   or until the kill_file has been touched
                  #
                  #——-
                  # Notes:
                  #——-
                  #
                  # Example of how to schedule this to run every minute in cron:
                  #
                  #   00-59 * * * * /bin/ksh -c ‘eval . ~/.profile.cron /quovadx/qdx5.2/integrator test_xcelera; rkr_xcelera_resend.ksh > /dev/null’
                  #
                  # Be sure to make an entry in /hci/cron/reminders.ksh as long as this script is scheduled to run via cron
                  #
                  #———
                  # History:
                  #———
                  #
                  #   2006.11.09 Russ Ross
                  #             – wrote intial version
                  #
                  # End of Module Header =============================================================================

                  #—————————————————————————–
                  # directory of the name sorted files to be sent
                  # (hidden file like .kill or .lock will not be a part of the sorted name list)
                  #—————————————————————————–

                     export que_dir=”$HCISITEDIR/data/que_xcelera_adt”

                  #————————————————————————
                  # directory to move the next name sorted file to one by one for resending
                  #————————————————————————

                     export load_dir=”$HCISITEDIR/data/load_xcelera_adt”

                  #—————————————————
                  # directory that will hold the files that got resent
                  #—————————————————

                     export sent_dir=”$HCISITEDIR/data/sent_xcelera_adt”

                  #—————————-
                  # log file of what was resent
                  #—————————-

                     export sent_log=$sent_dir/.log

                  #————————————————————————————————–
                  # toggle file that can be touched to gracefully prevent future invoking of this script from running
                  #————————————————————————————————–

                     export kill_file=”$que_dir/.kill”

                  #—————————————————————————–
                  # file to prevent multiple invokations of this script running at the same time
                  #—————————————————————————–

                     export lock_file=”$que_dir/.lock”

                  #—————————
                  # name of the resend process
                  #—————————

                     export rs_proc=”resend_xcelera_adt”

                  #————————
                  # name of the load thread
                  #————————

                     export load_thread=”load_xcelera_adt”

                  #————————–
                  # name of the resend thread
                  #————————–

                     export rs_thread=”ob_xcelera_adt”

                  #—————————————-
                  # pass these arguements to resend_que.ksh
                  #—————————————-

                     # resend_que.ksh >/dev/null
                       resend_que.ksh

                  Russ Ross
                  RussRoss318@gmail.com

              Viewing 4 reply threads
              • The forum ‘Cloverleaf’ is closed to new topics and replies.