Want to get date and time from incoming file in tcl

[Gene Millard]

I would like to get the date and time from the incoming file using  a tcl.

protocol fileset – ftp

• This topic was modified 2 years, 9 months ago by Gene Millard.

The Guthrie Clinic
Sayre, PA

Topic

[Peter Heggie]

I think the only information about an input file that is available in the message is the DRIVERCTL (in the message header), which only has the file name.

But if you are using Fileset – FTP, you can have a Directory Parse TPS designated to receive control as the source folder is scanned, and you can configure it to capture not only the filename, but also the file size and modification datetime (basically whatever info would be gathered if you were at the command line and entered DIR).

So your Directory Parse TPS would get that information in a list and you can split up the list into lines and loop through it, and split each line into the directory elements (name, size, date, time, etc.) and get access to the date and time. There may be variation in these directory elements between a source folder on Windows vs. a source folder on UNIX.

Peter

Peter Heggie

[Charlie Bursell]

The information is available in an IB Directory Parse proc.  From the docs:

Fileset/FTP Local TPS pane

<section id=”mnz1527795503307__section_N10011_N1000E_N10001″ class=”section”>
<p class=”p”>This tables shows TPS parameters:</p>

<colgroup><col /><col /></colgroup>
<thead class=”thead”>

<th id=”d276783e34″ class=”entry nocellnorowborder”>Parameter
<th id=”d276783e37″ class=”entry nocellnorowborder”>Description

<tbody class=”tbody”>

<td class=”entry nocellnorowborder” headers=”d276783e34 “>TPS
<td class=”entry nocellnorowborder” headers=”d276783e37 “>Select this to use directory parse or deletion TPS procs.
<p class=”p”>When this box is cleared, the Directory Parse and Deletion options are unavailable.</p>

<td class=”entry nocellnorowborder” headers=”d276783e34 “>Directory Parse
<td class=”entry nocellnorowborder” headers=”d276783e37 “>Use this TPS to select programmatically which files to process on each Scan Interval, and in what order. Click Edit to open the TPS Editor. Edit TPS proc properties in this dialog box.
<p class=”p”>In the Directory Parse TPS Editor dialog box, List Full Directory is a special flag for the engine to pass a full directory list to TPS. By default, this is cleared. When this is selected, you must modify the TPS code to handle the list and return a name-only list back to the engine.</p>
<p class=”p”>When this is selected, the <code class=”ph codeph”>msg_list</code> that is passed to the name parse TPS contains a full list, not only file names.</p>
<p class=”p”>This list is similar to <code class=”ph codeph”>{{filename1 filesize mtime} {filename2 size mtime}……}</code>.</p>
<p class=”p”>To get a full list from FTP, use list instead of the default nlst . The format of the full list is similar to the result of <code class=”ph codeph”>ls -l</code>, depending on the FTP servers.</p>

Note: When this option is selected, using the default directory parse TPS causes an engine error or panic.

<td class=”entry nocellnorowborder” headers=”d276783e34 “>Deletion
<td class=”entry nocellnorowborder” headers=”d276783e37 “>Click Edit to open the TPS Editor. Use this dialog box to select the TPS through which the driver passes processed file names.

</section>

[Gene Millard]

Having not done a directory parse before.  I still do not know how to do it.

The Guthrie Clinic
Sayre, PA

[Peter Heggie]

Here is a sample to get you going. If you are doing this with an FTP inbound, then there are more options. In the FTP options you can choose between NLST and LIST for the Dir List Command. I believe that the LIST option will return more than just the file names – it will also return the file size and date & time.

This example is for a Fileset Local.

 

######################################################################
# Name: tpsDirParseAssessments
# Purpose: Displays all file names read
# UPoC type: tps
# Args: tps keyedlist containing the following keys:
# MODE run mode (“start”, “run” or “test”)
# MSGID message handle
# ARGS user-supplied arguments:
# <describe user-supplied args here>
#
# Returns: tps disposition list:
# Returns a list of messages for the fileset
# driver to process.
#

proc tpsDirParseAssessments { args } {
keylget args MODE mode ;# Fetch mode

set debug 0
set module “tpsDirParseAssessments”
set path [hcitbllookup “assorted_site_parms” “assessment_builder_path”]

set outList {}

switch -exact — $mode {
start {
# Perform special init functions
return “”
}

run {
keylget args MSGID mh

set msg [msgget $mh]
set fileList [split $msg ” “]
foreach lFile $fileList {
if {$debug} {echo “$module input file: $lFile”}

set lFile2 [lindex [split $lFile “/”] end]
if {$debug} {echo “$module processing file: $lFile2”}

set fileseconds [file mtime “$path$lFile”]
if {$debug} {echo “$module file time $fileseconds”}

set nowseconds [clock scan now]
if {$debug} {echo “$module nowsecs $nowseconds”}

set diffsecs [expr $nowseconds – $fileseconds]
if {$debug} {echo “$module diffsecs $diffsecs”}
set file_age [hcitbllookup “assorted_site_parms” “assessment_file_age”]
if {$diffsecs > $file_age} {
lappend outList $lFile2
if {$debug} {echo “$module adding to output: $lFile”}
if {$debug} {echo “$module outlist content: $outList”}
}

}

msgset $mh $outList
return “{CONTINUE $mh}”

}

time {
# Timer-based processing
# N.B.: there may or may not be a MSGID key in args
}

shutdown {
# Doing some clean-up work
}

default {
error “Unknown mode ‘$mode’ in fileset_dir_parse”
}
}
}

Peter Heggie

[Gene Millard]

Sorry to take so long to write back about this.

I am unable to get this to work.

Does this tcl go in the TPS : Directory Parse section or the TPS inbound data?

set path [hcitbllookup “assorted_site_parms” “assessment_builder_path”]

Where is this table?

 

 

The Guthrie Clinic
Sayre, PA

[Peter Heggie]

Sorry – should have been more detailed –

The tcl is specified in the Protocol Configuration – Within the Inbound portion of the Protocol Configuration, there is a block labelled TPS. In that block, there is a field labelled Directory Parse, having an Edit button to find your TPS. That is where you specify the directory parse tcl proc. Also, check the checkbox called TPS in order to activate it.

The same setup is used for both Fileset-Local and Fileset-FTP. Keep in mind that if this applies to Fileset-FTP there is an additional consideration. On the second tab of the Properties window, the tab called FTP Options, about half way down is a field called Dir List Command. Depending on what is put in there, the format of the data sent to your directory parse TPS will be different – for using “list” vs. “nlst”. One option has a longer record, with more detail, than the other. I can’t remember which is which. But please add some kind of debugging statement near the top of your proc to display the contents of the incoming message to the process log. It will help a lot to understand what kind of data you are getting.

As far as that table referenced in the code – that is something we created for our organization, just a general purpose table that holds configuration values that may change over time, so we put the value in the table instead of hard-coding it in the proc.

This particular table value is the directory path, for example: ../../../data/assessments/ which is the location where the input files are coming from. It allows me to execute the “file” command with the “mtime” parameter for each filename that is given to the TCL proc. I get the age of the file and then I compare it to a static value, like 30 (seconds), and then if it is older than 30 seconds, then I go ahead and add that filename to the list. The list is the output of the proc. It tells Cloverleaf which files I actually want. Then Cloverleaf will read those files into messages and send the messages into the Inbound UPOC for normal message processing.

Peter Heggie

[Gene Millard]

I have set my path to the below.

if {$debug} {echo “$module input file: $lFile”}

for this echo I get.

[tcl :out :INFO/0: ADT:05/04/2022 14:28:18] tpsDirParseAssessments input file: dft_provation.txt

get error on the next line

set lFile2 [lindex [split $lFile “/”] end] if {$debug} {echo “$module processing file: $lFile2″}

[pd :pdtd:ERR /0: ADT:05/04/2022 14:28:18] Tcl error:
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] msgId = message0
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] proc = ‘tpsDirParseAssessments’
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] args = ”
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] result = ‘wrong # args: should be “set varName ?newValue?”‘
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] errorInfo: ‘
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] wrong # args: should be “set varName ?newValue?”
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] while executing
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] “set lFile2 [lindex [split $lFile “/”] end] if {$debug} {echo “$module processing file: $lFile2″}”
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] (procedure “tpsDirParseAssessments” line 25)
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] invoked from within
[pd :pdtd:ERR /0: ADT:–/–/—- –:–:–] “tpsDirParseAssessments {MSGID message0} {CONTEXT fileset_ibdirparse} {ARGS {}} {MODE run} {VERSION 3.0}”‘

Can’t get any futher.

proc tpsDirParseAssessments { args } {
keylget args MODE mode ;# Fetch mode

set debug 1
set module “tpsDirParseAssessments”
#set path [hcitbllookup “assorted_site_parms” “assessment_builder_path”]
set path “/mnt/ghcnt1/interface_dev/genesfolder/test/”

set outList “”

switch -exact — $mode {
#start {
# Perform special init functions
#return “”
#}

run {
keylget args MSGID mh

set msg [msgget $mh]
set fileList [split $msg ” “]
foreach lFile $fileList {
if {$debug} {echo “$module input file: $lFile”}

set lFile2 [lindex [split $lFile “/”] end] if {$debug} {echo “$module processing file: $lFile2”}

The Guthrie Clinic
Sayre, PA

[Peter Heggie]

very sorry – I have a typo in my code!

In that line that is erroring out – set lFile2 – etc., please split that line up into two lines. The “if” statement should begin on a new line.

Peter Heggie

[Gene Millard]

Thank you Peter.  I am able to get the date/time from the file.

Do you know how I would be able to add the date/time to the file name?

The Guthrie Clinic
Sayre, PA

[Peter Heggie]

right – so you can add code in your script to put together a new filename, based on the old filename and the date and time. Then you would call one of the tcl File functions:

file rename “$path$lFile” “$path<newname>”

and the output of the script would be different – instead of appending each filename to the output list, you would append the new, renamed filename to the output list.

Peter Heggie

[Gene Millard]

Thank you that worked great.

Gene

The Guthrie Clinic
Sayre, PA

[Peter Heggie]

Glad to hear it

Peter Heggie

[Author]

Replies