Instructions on How to for de-encrypting SMAT DB from the command line.

Homepage Clovertech Forums Cloverleaf Instructions on How to for de-encrypting SMAT DB from the command line.

  • Creator
    Topic
  • #118229
    Lawrence Nelson
    Participant

    We are preparing to make a data lake with ORM messaegs that are being sent in our Lab system.

    We run SMAT db encrypted.

    We will be looking to pull ‘yesterdays’ SMAT DB file each day – de-encrypt it and move it so our dba team can work with it to make dashboards etc with PowerBi.

    So to repeat the subject line –

    Looking for the command line instruction for de-encrypting SMAT DB files.

    Thank you.

    Lawrence Nelson

     

    Lawrence Nelson
    System Architect - MaineHealth IT

Viewing 11 reply threads
  • Author
    Replies
    • #118230
      Jim Kosloskey
      Participant

      Do you have Charlie Bursell’s hcismatdb utility?

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

      • #118231
        Lawrence Nelson
        Participant

        Hi Jim –

        I’m going with a No on that. Is there not a single command that can be issued?

        To make sure I’m clear – this is for SMAT db and not SMAT (before db) which I recall having to decrypt once years ago.

         

         

        Lawrence Nelson
        System Architect - MaineHealth IT

      • #118233
        Jim Kosloskey
        Participant

        I sent you my copy via email.

        The date in the comments of the proc is 07/30/2015. I think this is the latest but if Charlie or anyone else has a later version perhaps they can post here and get you a copy.

        email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #118232
      Jeff Dinsmore
      Participant

      Tcl syntax to open an encrypted DB file:

      <sqlite command> <dbHandle> <dbFilePath> -readonly <boolean> -key <encryptKey>

      Once open, you can query the DB using dbHandle as usual

      Jeff Dinsmore
      Chesapeake Regional Healthcare

      • #118238
        Lawrence Nelson
        Participant

        Hi Jeff –

        2 questions (requests)

         

        1

        Could you express this string with a less techical example (or an actual example) –

        I know it will be different for our location but I’m not picking up everything you are stating in this string below.

        <sqlite command> <dbHandle> <dbFilePath> -readonly <boolean> -key <encryptKey>

        2 Where would you actually get the encryption key from?

        Thank you for the response

        Lawrence Nelson
        System Architect - MaineHealth IT

      • #118239
        Jim Kosloskey
        Participant

        Lawrence,

        By default the key for SMAT DB encryption is the site name.

        So if you will execute the code from outside the site where the encryption took place, you will need to have a method to provide the site name.

        Does hcismatdb not accomplish what you need?

        email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

      • #118253
        Lawrence Nelson
        Participant

        Hi Jim –

        It’s been a holiday week for me – and Monday as well.

        I took a very quick look at the script – and it looks to quite lengthy – as opposed to a single command line to decrypt a SMAT db file.

        Lawrence

        Lawrence Nelson
        System Architect - MaineHealth IT

      • #118264
        Jeff Dinsmore
        Participant

        Lawrence,

        If we assume our our process is in the epic_ib site, the Tcl syntax would be:

        sqlite3 dbHandle /opt/cloverleaf/cis19.1/integrator/epic_ib/exec/processes/epic_adt_ib/adt_ib.smatdb -readonly 1 -key epic_ib

        Once the DB is open, you can query it with SQL – again this is Tcl syntax:

        set numMsgs [db eval “select count(*) from smat_msgs”]

        Jeff.

        Jeff Dinsmore
        Chesapeake Regional Healthcare

      • #118265
        Lawrence Nelson
        Participant

        Jeff –

        Excuse my lack of knowledge –

        but is what you’ve stated the command to decrypt and encrypted SMAT file or about pulling it into a external database – or a combination of both?

        Regards,

        Lawrence

        Lawrence Nelson
        System Architect - MaineHealth IT

      • #118266
        Lawrence Nelson
        Participant

        Jeff –

        Sorry – I read backwards in this chain and see what you are stating about decrypting. Thanks.

        The current issue this leaves me with is that I’m not actually looking to decrypt it – into a database at the time (the sqlite3 dbhandle piece)  – I’m looking to decrypt it period – as a file only  – so it can be moved unencrypted by our EFT processes to where our database team can then work with it in their environment (not via ODBC).

        Lawrence Nelson
        System Architect - MaineHealth IT

    • #118254
      Jim Kosloskey
      Participant

      Lawrence,

      Well I would simply place a command line entry for hcismatdb in a O/S script Korn shell or whatever you are using with the appropriate arguments- or – you can exec hcismatdb from inside a Tcl proc with the appropriate arguments. Your choice.

      But since I think you said the goal was to create a file of extracted entries from SMATDB I suspect you just want to place the file in a directory for subsequent processing so a O/S script would probably work well.

      Maybe we should discuss off-line – email me.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #118270
      Charlie Bursell
      Participant

      You want to copy the database as unencrypted to another location?   Remember an SQLite database is simply a file and can easily copied.

      As for the encryption; 2 options.

      Use as unencrypted on Cloverleaf or open the SMAT Database using command line logic provided and copy to another database.  Here, Google is your friend 🙂

      <b>SQLite – How to copy data from one database to another?</b>
      <ol class=”X5LH0c”>
      <li class=”TrT0Xe”>Using SQL query. First you attach <b>database</b> new_db with old_db to associate them in the current <b>database</b> connection. Assume that you are connected to the old_db , now run the ATTACH command to attach the new_db : …
      <li class=”TrT0Xe”>Using <b>SQLite</b> command-line tool. Open the old <b>database</b> then dump the table

       

       

       

    • #118304
      Lawrence Nelson
      Participant

      Charlie –

      I know I’m being obtuse here – but our db team is not being successful with un-encrypting the SMAT db file – remotely aka OFF of the Cloverleaf platform.

      Additionally, we’ve taken your script that Jim K. sent to me – and reviewed the contents in detail. We’ve confirmed the site ID spelling/case – and also that we haven’t set a password on the SMAT db – so it should be the site ID name.

      “Using the SQL Cipher version of DB Browser (free to download), I was able to create my own test DB and set the encryption.

      When I re-open the test DB, it prompts me for the key value.  If I enter the correct value it opens as expected.  If I enter the wrong value I get the message  “Could not open database file.  Reason: Invalid file format.”    

      I get the same message when I use this tool to open the encrypted db with encryption key previously provided, and the un-encrypted one from Cloverleaf (CLOTEST) SMAT opens as well.

       

      Lawrence Nelson
      System Architect - MaineHealth IT

    • #118305
      Charlie Bursell
      Participant

      From the web:

      <b>SQLite</b> doesn’t support <b>encrypting</b> database files by default. Instead, you need to use a modified version of <b>SQLite</b> like SEE, SQLCipher, SQLiteCrypt, or wxSQLite3.

      The Cloverleaf version uses SQLCipher

      Probably what you are using has a different encoding method with the sqlite version.

      From the command line using the version of sqlite installed with Cloverleaf, with or without a root set I can easily do the following with an encrypted SMAT file:

      sqlite myob.smatdb
      SQLCipher version 3.15.2 2016-11-28 19:13:37
      Enter “.help” for instructions
      Enter SQL statements terminated with a “;”
      sqlite> PRAGMAY key = “cbtest”;
      Error: near “PRAGMAY”: syntax error
      sqlite> PRAGMA key = “cbtest”;
      sqlite> .tables
      smat_info smat_msgs

      sqlite> .quit;

      Maybe move the encrypted DB to unencrypted for your use?  See site below:

      I have not personally tested this method but give it a try and let us know

      https://stackoverflow.com/questions/25132477/how-to-decrypt-an-encrypted-sqlcipher-database-file-on-command-line#:~:text=Decrypt%20the%20database%20to%20a%20plaintext%20database&text=db%20sqlite%3E%20PRAGMA%20key%20%3D%20&#8242;,decrypted%20database%20at%20~%2Fplaintext.

       

    • #118306
      Charlie Bursell
      Participant

      More info:
      I just installed DB Browser for Windows from: https://sqlitebrowser.org/dl/

      It installed two executables;  DB Browser (SQLite) and DB Browser (SQLCipher)

      I was able to decrypt and view a SMAT file using SQLCipher.

      I had to change default Encryption Settings from “SQLCyphe 4 defaults” to “SQLCypher 3 defaults”

      But I was then able to enter my site name for password and view and or manipulate the DB.

      I hope this helps.

       

      • #118339
        Lawrence Nelson
        Participant

        Hi Charlie –

        I’ve been forwarding your responses to the dba architect I’m working with. He has attempted all of the items you’ve suggested – but without success.

        When I send him an un-encrypted SMAT db file – it opens for him without issue.

        The encrypted ones are a no go with the use of the site name.

        I know that the sites SMAT setting have not been set up with passwords – that would over-write the use of the site name as the password.

        I’ve gone in to a test site – and turned on encryption to assure that no password was set –  it’s still a no go. I sent in a file pre and post encryption. The pre opened – the post does not.  So we are at an impasse – and I’m not sure where to go with for next steps.

         

        Here is the most recent response from the dba

         

        I’ve already tried below with no success.  I question if the password provided for the encrypted SMAT db you sent to me was in fact the correct password.

        If it was, I should have been able to open the file with DB Browser (SQLCipher) using either “SQLCyphe 4 defaults” or “SQLCypher 3 defaults” as outlined

        below.  The resulting message in either case is the same and implies an incorrect password.  Is there anyone in your group that may have opened an encrypted SMAT file outside of Cloverleaf?

        [inserted picture content he sent reads as follows]

        DB Browser for SQLite

        Could not open database file.

        Reason: Invalid file format.

         

         

         

        Lawrence Nelson
        System Architect - MaineHealth IT

    • #118340
      Charlie Bursell
      Participant

      What version of Cloverleaf were these SMAT DB files created under?

      If they were created pre-19.1 you will have to use hcismatcrypt as I outlined in the post about using hcidbscript to migrate them to SQLCipher.  You can then open them using DB Browser(SQLCipher).  I have tried and know this works.

      If you do not have Cloverleaf 19.1+ you may need to update.  I am not sure what encryption methods were used by previous versions of Cloverleaf.

      • #118343
        Lawrence Nelson
        Participant

        Well there you go – we are not on 19 yet – we are on the last version of 6.24(?)

         

        Lawrence Nelson
        System Architect - MaineHealth IT

      • #118344
        Lawrence Nelson
        Participant

        Charlie – Again sorry for being obtuse but I don’t see any prior references in here when I search for  hcismatcrypt or hcidbscript in this conversation on Clovertech.

        The only script I have from this conversation is hcidbsmat.tcl

        Are you able to pass me these other scripts?

         

         

         

        Lawrence Nelson
        System Architect - MaineHealth IT

    • #118360
      Charlie Bursell
      Participant

      hcismatcrypt  is a utility provided with Cloverleaf.  Maybe some misunderstanding about hcidbscript , probably a typo.  Here I was referring to the script hcismatdb which you have.  Again, please see my post about using hcismatdb for more info on hcismatcrypt.

      The hcismatcrypt command is available in version 6.2 but no migrate option.  You can use this command to validate the password or change the password of your SMAT files. Look for the command in your Cloverleaf documentation.

    • #118361
      Charlie Bursell
      Participant

      Lawrence:

      I was playing around with this on my old version of Cloverleaf 6.2.  Here is what you do

      setroot and setsite to where SMAT files are located

      First, run:  hcismatcrypt validatekey  <Name of SMAT File> <Site Name>

      It should return “Successful for command validatekey” if key is correct else it will return:
      “Validation failed.
      Incorrect password”

      If key is not correct and you cannot open as unencrypted, I am at a loss.  you will have to get your key.  Call Support, they have methods.

      If the key is correct then run:
      hcismatcrypt.htc rekey <sitename> <key>  <SMAT FILE>  “” <NEW FILE>

      Note that sitename and key are probably the same.  SMAT FILE is the file to copy
      “” means no key.  NEW FILE is name of file to create, i.e., xxx.db.  The give the new file to your analyst.

      The new file will be a copy of your SMAT file unencrypted.  I tried it and it works for me.

       

      Good Luck!!

      • #118432
        Lawrence Nelson
        Participant

        Hi Charlie –

        I’m sending the decrypted file to my dba partner – I think we have success!

        Thank you!

        2 follow ups –

        What do the 2 quotes between the file names indicate. I had to change your characters to be my ‘keyboards’ version?

        First attempt example

        clotest::hci> hcismatcrypt rekey t80ndxorders t80ndxorders to_SCC_orders_outQ.20210119234432.smatdb ▒^▒^▒▒^▒^▒ SCC_orders_outQ.20210119234432.db

         

        Also

        My first attempt gave me this until I stopped all processes – seems odd that all the processes have to be stopped to decrypt a file.

        clotest::hci> hcismatcrypt rekey t80ndxorders t80ndxorders to_SCC_orders_outQ.20210119234432.smatdb “” SCC_orders_outQ.20210119234432.db
        Warning: Engines are running in current site. Please stop the process and try again.

        clotest::hci> hcismatcrypt rekey t80ndxorders t80ndxorders to_SCC_orders_outQ.20210119234432.smatdb “” SCC_orders_outQ.20210119234432.db
        Successful for command rekey.

        Lawrence Nelson
        System Architect - MaineHealth IT

    • #118433
      Charlie Bursell
      Participant

      The double quotes represent a null for encryption key.  It should result in a file with no encryption.

      If the SMAT file is in use it would make sense to stop it first.

      Glad you got it working.

      • #118472
        Lawrence Nelson
        Participant

        Hi Charlie –

        One other follow up for my setting up where I’m moving files –

        Is this command able to be made to have the decrypted to a different folder/path location?

        hcismatcrypt.htc rekey <sitename> <key>  <SMAT FILE>  “” <NEW FILE>

        Something like ?

        hcismatcrypt.htc rekey <sitename> <key>  <SMAT FILE>  “” <fullpath>//smatdb_decrypt/<NEW FILE>

        Lawrence Nelson
        System Architect - MaineHealth IT

    • #118479
      Charlie Bursell
      Participant

      Works for me.  Did you try it?

Viewing 11 reply threads
  • You must be logged in to reply to this topic.

Forum Statistics

Registered Users
5,117
Forums
28
Topics
9,292
Replies
34,435
Topic Tags
286
Empty Topic Tags
10