Cycling SMAT and Log Files

Greg, do a search on the forum for cycling. There are some good, thorough discussions on the topic.

In short, yes, there are very good reasons for using SMAT DB, encryption, and auto-cycling.

It doesn’t cycle on a schedule, it cycles on file size. The search tool can search across live and archived files eliminating the need for scheduled backups.

Greg,

We just upgraded to 6.1.2.   We use cron to cycle the smatdb files.  That option works best for us because we know each file will have messages from basically the same time period.    Cycling on size can produce varied results depending on the amount of data you push through at times.

As for weekly vs daily, with the implementation of the smatdb, we have moved most of our cycling to a weekly schedule.  So far, the only things still have on a daily schedule are those interfaces that handle device data (which pumps out thousands of messages per day) and those interfaces that include an embedded PDF (which are huge messages).

One question I do have for the INFOR staff…is there a size limitation to the smatdb files?  We encountered an issue with one of our weekly files that caused the process to crash.   The engine wasn’t able to write to the smatdb due to the file size.   At that point, cycling the smatdb file was impossible.  We had to move the .smtadb file manually.  Once that was moved out of the process directory,  the -shm and the -wal files returned to normal size.

Also,  when it crashed, I noticed that files were created with a -sec as part of the file extension.  How are those -sec file used?

Thanks…

Tom Rioux

… we know each file will have messages from basically the same time period.

This is no longer necessary with SMAT DB. The searching is much more powerful. It searches not only the live files, but also the historical, archived data. In the search parameters, you can specify HL7 fields, date ranges, and many other criteria, too. The messages are stored as database rows which means there are columns that can be used for searching. The tool is fast and easy to use, and is still being improved by implementing user feedback.

An common example is to search on an MRN. You specify “PID.03=” which will find all records in the entire site for this MRN, including all archived messages! You can narrow the search to specific threads or choose specific files or date ranges or whatever you need. You no loner need to know which file contains the data!

Moving away from cron and into the engine makes migrations and patching much easier.

You’ll have to check on the file size limitations by submitting a ticket on Infor Xtreme as I don’t recall the size limitations.

We still cycle our SMAT DBs on a daily basis, even after converting from the file-based SMAT.  It keeps the file/DB sizes smaller and takes up less disk space in the file system that the engine is running; we move off the cycled files/DBs to a different file system.  I would think performance-wise having large SMAT DBs might have some impact as the engine has to write to a larger file/DB.  Just a theory.

Jim Cobane

Henry Ford Health System

I have to submit a ticket to just get file size limitations?  Shouldn’t that be part of the documentation?

Thanks….

Tom,

It may be the O/S which is providing the file size limit. Check with your Admin to see where that is set.

If SqlLite is being used for the DB then perhaps a check of the SqlLite support page will point out if there are limitations imposed by SqlLite.

We have not moved to SMATDB (we are on 6.0.0)) but I would think since the default archiving is based on size, one could limit the live (engine using) consumption that way (might mean very frequent archiving I guess given a small environment).

However, if the archiving can only be to the engine’s File System then I could see a resource issue arising there.

Is it not possible to archive to a different File System?

We still cycle our SMAT DBs on a daily basis, even after converting from the file-based SMAT.

From SqlLite:

Maximum Database Size

Every database consists of one or more “pages”. Within a single database, every page is the same size, but different database can have page sizes that are powers of two between 512 and 65536, inclusive. The maximum size of a database file is 2147483646 pages. At the maximum page size of 65536 bytes, this translates into a maximum database size of approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or 140,000 gigabytes or 128,000 gibibytes).

This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit. However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

We have a few configured to cycle based on size, but we like having our daily snapshots of the saved messages.  We are “old school” that way 🙂

Jim Cobane

Henry Ford Health System

With the old file based SMAT, you only had the .idx and the .msg.  Now you have 3 files in the process directory.  A .smatdb, a .smatdb-shm, and a .smatdb-wal.   When you have a process that includes imbedded PDF’s, all 3 of these files can be huge.

I love the new smatbd, don’t get me wrong.   It just takes some adjusting.

Where we have automatic cycling based on size configured are specifically those threads that contain the embedded PDFs; because they do get so large.

Jim Cobane

Henry Ford Health System

Not trying to change anyone’s mind, but I do want new users, future readers, etc. to understand the function and best practices.