Homepage › Clovertech Forums › Read Only Archives › Cloverleaf › Cloverleaf › Reset and Collect Statistics into a DB – TCL – hcimsiutil
Tagged: statistic database
- This topic has 7 replies, 8 voices, and was last updated 5 years, 3 months ago by
Dustin Sayes.
-
CreatorTopic
-
December 17, 2015 at 11:44 am #54919
Erik Mueller
ParticipantHi, I’m wondering if anybody has written something similar to this already that is willing to share it with me.
I’m wanting to collect the statistics for the threads on a regular basis and then reset them e.g. once a week, I collect the stats and then reset them.
Has anybody written a TCL that will extract the statistics for a thread and store them in a SQLite DB (or a text file)?
I’m not interested in all the statistics, just the Msgs In, Msgs Out, Xlated and Failed.
Anything out there would be appreciated,
Thanks,
Erik
-
CreatorTopic
-
AuthorReplies
-
-
December 17, 2015 at 2:46 pm #83452
Jim Kosloskey
ParticipantI have an old proc that dumps the entire msi stats to a delimited file whiich could then be imported to any tool allowing importation of a delimited file (DB included).
Email me and I will send it – you can do with it what you want.
email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.
-
December 17, 2015 at 11:54 pm #83453
Elisha Gould
ParticipantI’d actually be interested in this as well. I was just thinking about this the other day to help with statistics when users ask about volumes throughout the day, and also to help detect potential issues on an interface.
-
January 6, 2016 at 10:58 pm #83454
Rob Lindsey
ParticipantI have something that writes data out to a file per site. I then have another program that reads each one of those site files and puts them into a SQLite DB. I have another TCL proc that reads the DB and puts out a daily report. I got the code from J.Cobane years ago and it works well.
Code to write it out to a data file:
Code:
HciStartup
set run_time [clock format [clock seconds] -format “%m-%d-%Y-%H:%M:%S”]
set Date [clock format [clock seconds] -format “%Y%m%d”]
set arg_values $argv
if { $argc < 2 } {
puts "Usage is: get_stats_data SYSTEM SITE [EMAIL]"
return
}
set system [lindex $arg_values 0]
set sitename [lindex $arg_values 1]
set email [lindex $arg_values 2]
set outfilename [append outfilename "/info/stats/" $system "_" $sitename "_statsfile_" $Date ".csv"]
set subject "get_stats_data for Sitename: $sitename at $run_time"
set debug 0
set outrec ""if { [file exists $outfilename] } {
set outfileID [open $outfilename a]
} else {
set outfileID [open $outfilename w]
append header "SYSTEM,SITENAME,THREADNAME,MSGSIN,MSGSOUT,BYTESIN,BYTESOUT"#append header "SITENAME,THREADNAME,SAMPLE_DAY,SAMPLE_DATE,SAMPLE_TIME,THREADSTART_DAY,THREADSTART_DATE,"
#append header "THREADSTART_TIME,THREADSTOP_DAY,THREADSTOP_DATE,THREADSTOP_TIME,"
#append header "PSTATUS,LASTREAD_DAY,LASTREAD_DATE,LASTREAD_TIME,"
#append header "LASTWRITE_DAY,LASTWRITE_DATE,LASTWRITE_TIME,"
#append header "LASTERR_DAY,LASTERR_DATE,LASTERR_TIME,LASTERRTEXT,"
#append header "ERRORCNT,MSGSIN,MSGSOUT,BYTESIN,BYTESOUT,OBDATAQD,"
#append header "IBLATENCY,OBLATENCY,TOTLATENCY,IBPRESMSQD,IBPOSTSMSQD,OBPRESMSQD,OBREPLYQD"
puts $outfileID $header
}# Attach to the Shared Memory Region
msiAttach
set threadlist [msiTocEntry]
set threadcount [llength $threadlist]foreach threadname $threadlist {
if { [string is digit [string range $threadname 0 0]] } {
puts "$threadname skipped"
continue ;# This code added to skip threadnames that start with a digit
}
if { [string equal $threadname ""] } {
continue
}
set threadkeys [msiGetStatSample $threadname]
if {![string equal $threadkeys ""]} {
set LASTEXTRACT [clock format [keylget threadkeys LASTEXTRACT] -format "%a,%m-%d-%Y,%H:%M:%S"]
set THREADSTART [clock format [keylget threadkeys START] -format "%a,%m-%d-%Y,%H:%M:%S"]
set THREADSTOP [clock format [keylget threadkeys STOP] -format "%a,%m-%d-%Y,%H:%M:%S"]
set PSTATUS [keylget threadkeys PSTATUS]
set PLASTREAD [clock format [keylget threadkeys PLASTREAD] -format "%a,%m-%d-%Y,%H:%M:%S"]
set PLASTWRITE [clock format [keylget threadkeys PLASTWRITE] -format "%a,%m-%d-%Y,%H:%M:%S"]
set PLASTERROR [clock format [keylget threadkeys PLASTERROR] -format "%a,%m-%d-%Y,%H:%M:%S"]
set PLASTERRTEXT [keylget threadkeys PLASTERRTEXT]
set ERRORCNT [keylget threadkeys ERRORCNT]
set MSGSIN [keylget threadkeys MSGSIN]
set MSGSOUT [keylget threadkeys MSGSOUT]
set BYTESIN [keylget threadkeys BYTESIN]
set BYTESOUT [keylget threadkeys BYTESOUT]
set OBDATAQD [keylget threadkeys OBDATAQD]
set IBLATENCY [keylget threadkeys IBLATENCY]
set OBLATENCY [keylget threadkeys OBLATENCY]
set TOTLATENCY [keylget threadkeys TOTLATENCY]
set IBPRESMSQD [keylget threadkeys IBPRESMSQD]
set IBPOSTSMSQD [keylget threadkeys IBPOSTSMSQD]
set OBPRESMSQD [keylget threadkeys OBPRESMSQD]
set OBREPLYQD [keylget threadkeys OBREPLYQD]
set statsrec ""append statsrec "$system,$sitename,$threadname,$MSGSIN,$MSGSOUT,$BYTESIN,$BYTESOUT"
#append statsrec "$sitename,$threadname,$LASTEXTRACT,$THREADSTART,$THREADSTOP,$PSTATUS,$PLASTREAD,"
#append statsrec "$PLASTWRITE,$PLASTERROR,$PLASTERRTEXT,$ERRORCNT,$MSGSIN,$MSGSOUT,$BYTESIN,$BYTESOUT,$OBDATAQD,"
#append statsrec "$IBLATENCY,$OBLATENCY,$TOTLATENCY,$IBPRESMSQD,$IBPOSTSMSQD,$OBPRESMSQD,$OBREPLYQD"puts $outfileID $statsrec
}
}
close $outfileIDCode to read the csv files and write to SQLite DB:
Code:
package require sqlite
package require csvsqlite DBCMD “/hci/cloverleaf/cis6.1/integrator/databases/engine_stats.db”
DBCMD timeout 6000HciStartup
set run_time [clock format [clock seconds] -format “%m-%d-%Y-%H:%M:%S”]
set Date [clock format [clock seconds] -format “%Y%m%d”]
set arg_values $argv
if { $argc 0 } {foreach file $locallist {
if { [file isdirectory $file] } { continue }
puts “file ==> $file”
set FileId [open $file]
set lines [split [read $FileId] “n”]
close $FileId
foreach line $lines {lassign [::csv::split $line $sepChar] system sitename threadname MSGSIN MSGSOUT BYTESIN BYTESOUT
if { [string equal $system “SYSTEM”] } { continue }
if { [string equal $system “”] } { continue }
set sqlcmd “insert into engine_stats values(’$UserDate’,’$system’,’$sitename’,’$threadname’,$MSGSIN,$MSGSOUT,$BYTESIN,$BYTESOUT)”
puts “sqlcmd ==> $sqlcmd”
DBCMD eval $sqlcmd
}
}
}exit
Let me know if you have questions.
Rob
-
March 15, 2017 at 4:08 pm #83455
Greg Tataryn
ParticipantSince the message and byte counts are a running total, if you need to get an hour over hour count, how are you going about that? Or are you just interested in daily/weekly totals and not hourly?
-
June 15, 2018 at 8:52 am #83456
Varun Sinha
ParticipantHi Rob,
I know it’s very old post, but I can see that you have shared
code to write it out to a data file
Code to read the csv files and write to SQLite DB
but I do not see the TCL proc that reads the DB and puts out a daily report.
Can you please share the TCL proc that reads the DB and puts out a daily report?
I am new to cloverleaf TCL. Appreciate your help in advance.
Regards,
Varun
-
June 19, 2018 at 7:48 pm #83457
Rob Abbott
KeymasterHi Varun, please note that in 6.2.0 the option to store statistics in a database is built into the core application. You may want to leverage this rather than building something custom.
Rob Abbott
Cloverleaf Emeritus -
June 6, 2019 at 3:11 pm #109096
Dustin Sayes
ParticipantI’m very interested in learning about the statistic database.
Where do we configure which DB will be used for the statistic db?
-
-
AuthorReplies
- The forum ‘Cloverleaf’ is closed to new topics and replies.