[Tim Jipson]

Hi Jason, We were running 9.1 with 2022.09.03 and everything was running great. Then the server admin applied ‘patches’ that brought the engine up to RHEL 9.7. It has been an absolute living hell since then.  The server admin says 9.7 can’t be backed out so we trying to avoid a server rebuilt and hoping that 2022.09.04, being one minor version from certified, will be more stable.

The biggest issue we are seeing is hcienginestop does not work. Migrating to hcienginerestart has helped but there is still massive instability.

 

This is a rely from Infor I got a couple weeks ago:

Based on KB2282305, RHEL 9.1 is the latest authorized version for Cloverleaf 2022.09.03. For Cloverleaf 2022.09.04, the authorized version is RHEL 9.6. For more information, please refer to the attached KB article.

[Tim Jipson]

Interconnect doesn’t Need https. We have a couple standard tcp-ip connections interconnect. The big issue we had was a load balancer routing issue. The connection worked when connecting directly to one one the interconnect servers but not the load balancer.

[Tim Jipson]

Thanks, Ramachandran! I’d love an update next week if you have time.

Jason, It is far easier to install a patch than do a major version upgrade. Also, most hospitals I’ve worked at have no desire to be bleeding edge.

[Tim Jipson]

I did double check all the folders, but I’ve never seen an issue with fileset-local folders not automatically cleaning up.

Maybe memory leak is the wrong term, but we are now monitoring the server’s free memory and restarting hciss when memory gets low, about every 3-4 weeks I think.

[Tim Jipson]

Thanks Jim, unfortunetly our smatdb files are encrypted so that won’t work. I was able to use Charlie’s hcismatdb script to get my counts. Aside from the code for looping through the directories, this is all the code I needed then I just parse what is returned in msgCount.

 

msgCount=/hci-scripts/msgCount/hcismatdb.tcl -i $smatFile -orsf tempsmat -site ${site:1}

[Tim Jipson]

We have tracked this issue down to a memory leak in CloverleafHostServer. I’ve seen the host sever use over 5GB over the course of a month, restarting the host server drops memory usage to under 1GB. Since we’ve been doing weekly host server restarts, we have not had any site crashing issues.

[Tim Jipson]

Jim, All that script does is read messages out of smat and then creates a new file. I need to modify the smatdb file. I opened a ticket with Infor, hopefully I’ll get some useage info for smatdbdelete.

[Tim Jipson]

Thanks Jim, I’ll check that out

[Tim Jipson]

I actually just found “smatdbdelete”, I think this is what I need. Would anyone have the proper syntax for this command?

 

[Tim Jipson]

I’m doing something similar using a couple global variables in an xLate. If the new message matches the old GVs then the message is supressed. If the new message does not match the old GVs then the GVs are updated and the message is sent.

[Tim Jipson]

Hi Jason,

That makes a lot of sense, we do run a lot of script based alerts. I have spot checked the pid files but I haven’t done a full audit while the issue was occurring. That will be my next step, Thank you!

[Tim Jipson]

Hi Vince,

Sometimes that has helped. Sometimes I did a whole site db rebuild and there was no change. Sometimes I’d restart everything and have no immediate change but after 5min the gui starts responding correctly. The randomness of the issue has made troubleshooting almost impossible.

[Tim Jipson]

Have you tried manually editing the netconfig file for the site that the thread is on?

[Tim Jipson]

In short, it would be nice to have a place in the thread properties to add the interface owners email address and possibly the vendor contact. It would be great to assign these fields a variable in the alert config.

I’ve also got an alert script that I use for several alerts. The script fixes most issues by restarting the interface but if the issue isn’t resolved then an email is sent to the interface team and we need to figure out which analyst needs to be notified.

I just started using the Groups field in Thread properties to track interface ownership. Now the script also parses the netconfig file to get the interface owners email address and the owner is emailed directly saving a lot of time.

Yes, I could create an alert for each interface but that’s too much maintenance. The script allows me to have one alert for all OB threads and another for all IB threads and that covers 98% of our alerts.

[Tim Jipson]

I was missing the gvsave, I did not know that was needed after a gvsetvar. Thank you!

[Author]

Replies