Python Libraries

[Jonathan Souders]

Curious if anyone knows…

I’m currently tinkering with the contrib Python box on CL 6.2.3 and am wondering if it’s possible to install libraries to function with hcipython. I installed jip from pip, but that didn’t seem to help.

Anyone else experiment with this at all?

Topic

[Lonnie Davis]

This is an old question, but I have been playing with Python support in Cloverleaf 19.1.

Here is what I did to get a 3rd party Python library called ‘python-hl7’ to work from hcipython.  I will create a new thread for an issue I am having running Python scripts as a UPoC.  More on that later.

According to the 19.1 documentation, the Python scripts need to be put in the $HCISITEDIR/scripts directory as Cloverleaf searches this directory and subdirectories for Python and Javascript files.  This is different from how the site in the JavascriptAndPython.box file works where Python scripts are put in the $HCISITEDIR/pythonmodules directory.

I downloaded the python-hl7 library from the developer’s GitHub page at https://github.com/johnpaulett/python-hl7 then copied the directory containing the Python scripts, called ‘hl7’, to the $HCISITEDIR/scripts/ directory.

From there, I could load up the hcipython interpreter, import the python-hl7 library, and start using all of the classes according to the developer’s instructions.  I couldn’t get any Python package manager to work such as pip or easy_install.  The generic Jython documentation doesn’t seem to work with the Cloverleaf implementation so it looks like the only way to use a Python library is to download the Python source code and copy it to the $HCISITEDIR/scripts/directory.

It all breaks down for me when I use the Cloverleaf 19.1 documentation to run a Python script as a TPS UPoC though.  At least it works from hcipython so that is the first step.

If there have been any new developments on using Python libraries in Cloverleaf since the original question was asked, please let me know. 🙂

• This reply was modified 4 years, 7 months ago by Lonnie Davis.

[Jonathan Souders]

Clever!

I never followed up on this since there didn’t seem to be any interest, but I eventually did find a way to install libraries from pip.

It required downloading an updated version of the Jython Standalone .jar file that is in $HCIROOT/lib/java – the version that already exists in that directory wouldn’t allow the ensurepip command to work for whatever reason, but installing the libraries in the updated version is backwards compatible for the older version – and running the following commands:

$ java -jar jython-standalone-2.7.1.jar -m ensurepip
$ java -jar jython-standalone-2.7.1.jar -m pip install –upgrade pip
$ java -jar jython-standalone-2.7.1.jar -m pip install hl7
$ java -jar jython-standalone-2.7.1.jar -m pip install zxJDBC

 

After doing this, I could write my scripts with any libraries I had  installed from pip and set the threads to run with the Script TPS module.

• This reply was modified 4 years, 7 months ago by Jonathan Souders.

[Lonnie Davis]

Jonathan,

Thanks for your input. After following your instructions,  I was able to use easy_install but I can’t get pip to work.  I think I just need to read more about pip to get it to work though.  I’m new to the whole Python world and the Jython piece is raising the learning curve since I know nothing about Java.

Using easy_install though, I was able to reinstall the python-hl7 package…

java -jar jython-standalone-2.7.1.jar -m easy_install hl7

Then I deleted that ‘hl7’ directory from my ‘pythonmodules’ directory from the JavaScript/Python BOX and my test script still worked.  I switched back to using the BOXed site for testing because using the ‘scripts’ directory just didn’t work right for me.

I see that Python packages are installed to $HCIROOT/lib/java/Lib/site-packages

It is worth mentioning I’m doing this testing on a Windows machine running Cloverleaf 19.1.  All of my real work is done on AIX boxes running Cloverleaf 6.2.2.

Can you elaborate on how you are getting your script TPS modules to work?  Are you able to use user arguments in the TPS Properties ‘args’ box?

I can’t seem to get the Cloverleaf documentation method of using the ScriptTPS Java class plus the {LANG python}{FILE <file_name>} {FUNC <func_name>} args to work properly.  Why does it ask me for a func name?

But I can use the method used in the BOX example of using the JythonTPSBeta Java class with the args {MODULE <file_name>}

 

[Jonathan Souders]

This is how I have my TPS set up with my .py files in$HCIROOT/scripts – which I’m currently just running against some sample files with success (still need to get the team on board with using Python since no one else is familiar):

Java Class: ScriptTPS

Args: {LANG python} {FILE Tps_py_template.py} {FUNC __MODULE_NAME__}

 

ScriptTPS being the Java class Infor allows these to run through, {LANG python} being the language I want to use (as opposed to JavaScript), {FILE Tps_py_template.py} being the file containing my relevant functions, and {FUNC __MODULE_NAME__} being the Python function I am calling from my .py file. Think of this like your Tcl procs, where you have a file that can contain many procs, and your Tcl_index contains a list of your procs which you can then select from the TPS properties window – with Tcl it’s inherently in the application, but for Python/JavaScript you have to specify the arguments for those parameters since they are not built in like the Tcl components.

 

TL;DR – LANG tells Cloverleaf what language to use, FILE specifies the file in which to look for functions, FUNC specifies which function in the FILE to use.

[Rob Abbott]

Great input guys!

 

Jonathan – will investigate upgrading Jython for the next release.

Lonnie – if you’re having trouble running python UPoC and the documentation is incorrect please open a support case so we can help you and get anything fixed that needs fixing.

 

• This reply was modified 4 years, 7 months ago by Rob Abbott.

Rob Abbott
Cloverleaf Emeritus

[Jonathan Souders]

Appreciate that, Rob!

[Rob Abbott]

We are actually investigating moving to JEP (Java Embedded Python) which supports Python 3 (yay), if there are no major issues we might be able to squeeze that into 20.1.

We will also plan to upgrade Jython in any future patches to 19.1 or 6.2.

Rob Abbott
Cloverleaf Emeritus

[Jonathan Souders]

Rob:

 

Should I open a case on the following issues?

For Javascript, I’m not sure how to debug – I’ve tried console.log(<variable>), but it errors out my scripts in testing so I can’t read any of my variables from the log file.

For testing tool, I try to pass arguments (JS and PY) the same way I do in the NetConfig, but I never get any output to the screen.

I’ll open a case if necessary, but thought I’d ask if that was worth my time first.

 

Thanks.

[Lonnie Davis]

Success!  I got a test Python script to work correctly in a site other than the BOXed JavaScript/Python site using the ‘scripts’ directory and your instructions.  I’m not sure what I was doing before to get the errors.

One more question for you.  How do you supply the userArgs to the Java class ‘ScriptTPS’?

[Jonathan Souders]

Awesome! There are some template outlines in $HCIROOT/scripts/templates that give you some general starting places on the different types of functions you might use. The one for TPS includes some function parameters, one of which is ‘userArgs’ which you can then call print(userArgs), or however you plan to use them, within your function. Below is my general understanding of ways to use (most of) the parameters:

def __MODULE_NAME__(cloverEnv, context, mode, msg, userArgs, dl):

    if mode == ‘run’:

        # Obtain thread name

        hciConnName = cloverEnv.getThreadName()

        # Print user arguments

        uargs = userArgs

        # Obtain content of message handle

        data = msg.getContent())

        # Commit any changes you’ve made to your message content

        msg.setContent(str(data))

        # Add message to disposition list

        dl.add(CONTINUE, msg)

• This reply was modified 4 years, 7 months ago by Jonathan Souders.
• This reply was modified 4 years, 7 months ago by Jonathan Souders.
• This reply was modified 4 years, 7 months ago by Jonathan Souders.

[Lonnie Davis]

Yeah, I don’t know what I was doing before, but it works great now.  What I mean with the userArgs was how to populate userArgs from the Java args in the TPS properties where you set…

Java Class: ScriptTPS

Args: {LANG python} {FILE Tps_py_template.py} {FUNC __MODULE_NAME__}

There has got to be an ARGS parameter or something that tells the ScriptTPS Java class to populate the userArgs object/ parameter so it can be used from within the Python script.

[Jonathan Souders]

You can do that like so (just an example):

 

NetConfig

Java Class: ScriptTPS

Args: {LANG python} {FILE Tps_py_template.py} {FUNC __MODULE_NAME__} {userArgs TESTING}

Python Script

def __MODULE_NAME__(cloverEnv, context, mode, msg, userArgs, dl):

    # mode can be “run”, “start”, “shutdown”, or “time”

    #hciConnName = cloverEnv.getThreadName()

    import hl7

    lang = userArgs.get(‘LANG’)

    file = userArgs.get(‘FILE’)

    func – userArgs.get(‘FUNC’)

    uargs = userArgs.get(‘userArgs’)

    print(lang)

    print(uargs)

Log Output

[prod:prod:INFO/0:    python_in:05/08/2020 13:34:22] Applying EO config: ”

python

Tps_py_template.py

__MODULE_NAME__

TESTING

[cmd :cmd :INFO/0:   python_cmd:05/08/2020 13:34:26] Receiving a command

[cmd :cmd :INFO/0:   python_cmd:05/08/2020 13:34:26] Received command: ‘python_xlate xrel_post’

[cmd :cmd :INFO/0: python_xlate:05/08/2020 13:34:26] Doing ‘xrel_post’ command with args ‘<none>’

[Lonnie Davis]

Awesome!  I was completely over complicating the matter.  I’m all set for now.  Thank you much for all of your help Jonathan!  Now I am off to experiment further in Cloverleaf and  continue learning Python.

[Rob Abbott]

Lonnie, did you find anything in the documentation that was lacking?  If you have any suggestions for improvement I would welcome them.  thanks

Rob Abbott
Cloverleaf Emeritus

[Lonnie Davis]

Rob,

After Jonathan showed that all parameters that are entered in the Java class ‘Args’ box can be accessed with userArgs object, the Cloverleaf 19.1 documentation made more sense.

I can’t say anything specific is missing from the documentation, but my complaint with the documentation in general is its lack of real world examples.  The documentation is written as if the reader should already know how every feature in Cloverleaf works.  This has been a problem for me in every new version of Cloverleaf where new features are added.  How am I supposed to know how to take advantage of new features if the documentation doesn’t elaborate on how they work or how to use them?  Infor doesn’t provide free training on anything so I’m always stuck spending weeks or months trying to figure it out on my own.

For the Python and JavaScript support for example, I had no idea that you could execute the Jython runtime by executing ‘java -jar jython-standalone-2.7.1.jar -m <module>’ or even where the Jython standalone jar file was.  This was not covered in the documentation.  I don’t even know how the hcipython command fits into all of this, other than as an interactive command prompt, since it doesn’t seem to work the same way that executing the Jython stand alone jar does.  As in, executing ‘hcipython -m <module_name>’ does not actually run a module.

My suggestion for the Cloverleaf documentation is to provide more details about how new features actually work in the context of Cloverleaf’s implementation with the inclusion of examples so Cloverleaf customers/ end users can get up to speed and start using them more quickly.

Also, I do have a question for you.  In what version of Cloverleaf is Python and JavaScript fully supported?  It appears to be 19.1 since my 19.1 installation on my dev box has the ‘jython-standalone-2.7.1.jar’ file, along with a few other files, in $HCIROOT/lib/java/ where in my 6.2.2 installation used in test and production does not.  The included BOX’d site for Python and Javascript contains the files, including the Jython stand-alone file, in the site created from it, but that doesn’t look like global support; just something to use for testing.

[Rob Abbott]

Hi Lonnie many thanks for the feedback.  We are doing our best to provide more example-based documentation as we move forward.  We will continue to try to improve this.

 

Regarding support for Python and JavaScript, that is available for production use in 19.1.0 and trial use / contrib in 6.2.x.

 

We plan to have GUI support for JS and Python in 20.1 (meaning you will have Python and JS buttons in the UPoC configuration, and not have to select Java and send arguments as described above).

Rob Abbott
Cloverleaf Emeritus

[Author]

Replies