tcl help to count no of messages and append to the top

This topic has 8 replies, 4 voices, and was last updated 18 years, 7 months ago by Kevin Kinnell.

Creator

Topic
May 15, 2006 at 1:54 pm #48519
Ray Mullen
Participant
Hello everyone,

I have a tclproc that creates a seperate file for each message and ftps them to a server.

I need to add to this proc to include a a log file that will be picked up by the vendor every hour , and needs a message

count at the top of the file.

I have the audit file generating ok where i copy the message and create the log.

The thread protocol is ftp and its ftping the file and creating the log file , but my problem is , i need to be able to count

the no of messages in the log file before i write to it and appent a count to the top of the file. Hope this makes sense,

A snip of the code is as follows tha generates the audit log and is a as follows:

set mh2 [msgcopy $mh]

set msg [msgget $mh2]

set MSHSeg [getHL7Segment $msg “MSH”]

set c_createdate [getHL7Field $MSHSeg 6]

msgset $mh2 “

set newfilename “pims/data/audit.txt”

keylset alist OBFILE $newfilename

keylset fslist FILESET $alist

msgmetaset $mh2 DRIVECTL $fslist

lappend dispList “CONTINUE $mh2”

The file generates the following:

What I need is something similar to this

I have worked out the TCL to count the no of messages in any file by using the and its like as follows:

set fh [open y:/quovadx/test.txt r]

set count 0

set srch “UniqueID”

while {[gets $fh line] >=0 } {

if regexp $srch $line]} {

incr count }

}

}

close $fh

puts $count

return $count

What I need to do is just before I write to the audit log I need to count the no of messages and append to the top of the

file, but am not sure how to go about it. I hope all this makes sense.

Any help would be great

Thanks

Ray
Creator

Topic

Viewing 7 reply threads

Author

Replies
- May 15, 2006 at 3:55 pm #58845
  Rentian Huang
  Participant
  Ray,
  
  I guess if you want to constantly counting msgs within a file, you need to put the counting proc you created in the Time mode of the Read TPS within a UPoC protocol.
  
  Sam[/b]
- May 16, 2006 at 7:13 am #58846
  Ray Mullen
  Participant
  Hi Rentian , thanks for the call, I’m not sure if I can do it that way (or how to do it that way) because I am using the ftp protocol and the file is been written to a different server.
  
  The file will be snapped up by an external source every hour , so just before i write the details to the audit file i want to quickly count the no of messages and append it to the top.
  
  The directory structure can’t be changed on the external server can’t me changed so i cant write the file somewhere else and then move it out, so i am tryig to do it all in one go if you know what i mean.
  
  Also in this proc i am writing individual message files for each message to other directories on the external server(that bits working fine). i.e the log is just a file that contains the basic info of the messages written and a count at the top.
  
  ray
- May 16, 2006 at 1:26 pm #58847
  Kevin Kinnell
  Participant
  ~~Ray Mullen wrote:~~
  
  Also in this proc i am writing individual message files for each message to other directories on the external server(that bits working fine). i.e the log is just a file that contains the basic info of the messages written and a count at the top.
  
  ray
  
  How big is your message count likely to get? If it’s not too big, I would
- Read the message file into a list.
  find the list size.
  
  write the size, and loop through the list writing the messages, to some file.
  
  rename the file to { audit.log }

May 18, 2006 at 7:46 am #58848

Ray Mullen

Participant

Hi Kevin ,

The problem is i’m not sure how to open the file (or a file) i am writing to within the program and read from it,as it is on an external server. I am using the ftp protocol on the thread to log onto that server and the program that writes the log is on the tps outbound data side of the thread. I would need to maybe store a temp file on the external server read that and then append to it and copy it within the outbound proc , but i dont want to use a windows schedule to ftp, i want to try to do it using the ftp protocol on the thread. Hope this makes sense.

ray

May 18, 2006 at 12:33 pm #58849

Michael Lacriola

Participant

Another option could be this: Build two separate files, one that has the messages, and another that keeps the counter. When you are ready to send the complete file, concatenate the two together putting the number of messages at the top. Just a thought.

May 18, 2006 at 1:50 pm #58850

Kevin Kinnell

Participant

Ray

I’m still not quite clear on the sequence here, I think. I was presuming that you are counting the messages that are in the separate files which you ftp out, and generating a new log file which is then ftp’d out as well. What I think now is that there is a log file on the ftp server which will be picked up once an hour, but which needs to be kept updated continuously.

Does the log file continuously grow, with insufficient space for it locally? Log files can’t grow forever–how often does the log get ‘zeroed’ and a new log started? How reliable is the retrieval time? Does it happen at the same time every hour?

I’d think retrieving/processing a file via ftp in a tps would be too darned slow. I think what I’d do if there were no way around it would be a slight modification of what Michael suggested: generate your log file locally, and a separate file with the current count (the hci extensions have a set of Ctr* procs for manipulating counter files.) First you’d append the new messages to the message file, updating the count. Open your new log, write the count, then close it and cat the messages onto it and ftp the result to the server.

Now, you could ftp the log file back to your site from the server. You’d slow things down and introduce a bigger window in the implicit race condition–what if the log gets picked up while you’ve got it? But, you’d still have to be doing all the processing locally. You actually just narrow the window on the race condition a bit by keeping the data local if there is a copy on the server. So Michael’s unmodified solution is actually your best bet: generate everything locally, and don’t hand it to the server until it’s needed. This completely avoids the ‘two copies of the “same” data’ problem, which is an huge problem that you want to avoid like the plague.

If you can’t control when the file will be picked up, you can use clock to prevent your proc from ftp’ing until you are ready (and avoid the Windoze scheduler altogether.) Which may, now that I consider it, be what you wanted to know in the first place. You would still have to make sure that the previous file has been retrieved before you sent the new one (assuming that retrieval removes or zeroes the log file. Does it?)

Whatever you do, for this kind of thing, you really need to build in a bunch of tests for not re-transmitting information that’s already been retrieved (I speak from bitter experience.) Never, ever have more than one copy of the data available for different entities to process. There is no way to guarantee things will be in sych if you do.

May 19, 2006 at 9:30 am #58851

Ray Mullen

Participant

Great thanks for all your responses , i really appreciate all the time you spent trying to help me out. There’s a lot of food for thought there. In response to some of your queries Kevin, the way it works is as follows:

there is a from thread that picks up messages and routes them to a ftp thread which i have configured to create individual message files but also write a log just containing the control id of the message e.t.c. The file will be in xml format, but the snag is they want the msg count at the top OF the log .

They dial in at certain intervals and snap up all the files including the log, so when all files are gone the process just carries on creating new files and a new log. The count will only be the number of messages in the log at that time.

The thing is I was trying to do it in one go rather than writing locally first and then ftping. I didnt want to create more threads in order to ftp.

Once Again , I appreciate all the effort everyone spent , helping me out

Kind Regards

Ray

May 19, 2006 at 12:21 pm #58852

Kevin Kinnell

Participant

Well at least I finally get it. You wouldn’t be able to do it without either deleting the log on the server and recreating it, or allowing multiple logs. The whole thing would be darned difficult without using at least one temp file.

One last thing I’m curious about: are you talking about Tcl interpreter threads, or Cloverleaf threads? I’m assuming the former.

Viewing 7 reply threads

The forum ‘Cloverleaf’ is closed to new topics and replies.