Automating changes from development to production

[Michael Lacriola]

I’m looking at a way to automate moving objects from development to production. Here’s my thoughts so far and where it gets sticky. I would create a Tcl program that will read a file line by line. The data in the file would look something like:

xlate|my_changed_xlate.xlt

tcl|tclfile.tcl

frl|frl_file.frl

hl7|2.2|hl7_name

bounce|thread|thread_name

bounce|engine|engine_process_name

bounce|engine|other_engine_process_name

The script would read each line and based on the first component (after splitting it of coarse) would do the following:

Make copies of the existing objects in production with the extension of the date, then copy the development objects into production, bounce any engine processes or threads when needed.  This should work fine as long as NetConfig does not change. That’s the sticky point. Our NetConfig in development is nowhere near the same as in production, so that cannot be copied over.

Another thought would be to create a duplicate of production site into a stage site that would never run (no running monitord, lockmgr, or engine processes). Development objects would then be handled manually (like now) into stage, then the Tcl script could move objects from stage to production (could copy NetConfig directly now from stage to production — a bit scary).

Reason for all of this: ITIL. I’m not going to go into it, but our change windows are now between 0200 and 0600 on certain days. It would be great to automate changes so that I do not have to get up in the morning. I enjoy my rest.

Anyone done something like this? Suggestions are certainly welcomed and appreciated.

Topic

[Steve Carter]

I’m currently working on the same thing.  What I decided to tackle first were the changes to the NetConfig (particulary moving of threads).

The script displays prompts for each step.  I first ask for the site, then it will load all of the protocol threads into a list.  The user can either enter a ‘?’ for a list of threads or key in the thread name.  It then determines the protocol of the thread (right now I’m targeting pdl-tcpip threads).  Depending on whether or not it is client or server determines whether or not I prompt for an IP address.

Once the user has entered enough information about the source thread, I then prompt for a production site name.  I pull the NetConfig from the production box back to test as a temp file.  I parse the temp file to make sure the thread they have chosen doesn’t already exist.  I then prompt for the process in which the thread belongs.  Again, the user can enter a ‘?’ for a list of processes.

Once this is complete, I build a file containing the protocol thread information and then push it back to the production box.  I have another script on the prod box that appends it to the NetConfig.

There is a lot of verification going on in the background to prevent typo’s and invalid IP’s, port numbers and overwriting of existing threads.

This script uses the netcfgLoad, etc. to parse the NetConfig’s.

The next piece I’m working on is the DATAXLATE portion for obtaining routes from other threads that route data to the thread that I’m moving.  I haven’t gotten too far into this yet as a few other projects have taken my time.

Shoot me an email if you have any questions.

Hope this helps.

Steve

[Anonymous]

We developed that in tcl/tk some years ago…

The GUI compiled a list of sites, and you had to select the source and the target sites.  After the selection of the two sites, it will give you a list of processes and threads in both sites (two windows side by side)

With the mouse you could click in the processes and threads that you wanted to copy.

Once you selected the processes and/or threads, it scans the NetConfig for the appropriate files: tclprocs, xlates, pdls, formats (variants) etc.

The program would display in two windows the tclprocs in the source and target directories, recommending the ones that needed to be selected because there are missing, or have changed. I believe that it also warns if there are differences but the target file has a more recent timestamp. Once you click on the tclprocs that you want copied (you can ignore the recommendations) it will do the same with the xlates, pdls, etc.

We used the program to copy threads from one site to another site. It worked fine but since we had strict procedures to move things to production (and it was in another box) we only used this in test and development.

With time we stop using it and

[Richard Hart]

We perform almost all of our translation code in TCL.  We also use Unix links as we have many sites that share the same code. When we have a change to implement, we create a second ‘tclprocs’ directory so the link can be changed and the current code is not ‘lost’.

The upgrade script (codeInstall.sh)  uses the file name as parameters to determine the tclprocs link, sites to check and threads within the sites and the RCS revision of the script

eg

cI:ccm1:ccm_prod:ris_:ccm_tables_ris:1.19.sh -> /hci/InfoHEALTH/bin/codeInstall.sh

The script, when executed will process all sites and bounce each thread, checking the log file to ensure that the correct revision has been installed.

Once completed, it reports the resuts.

We have ‘dummy’ prodcution sites that use the same links, so the code installation can be tested on the ‘real’ production setup.

A ‘backout’ version does the same thing,  using the old link.

For NetConfig changes, we generally use links to the NetConfig file, so a quick switch back is easy.

[Lin Yue]

We use subversion for all our sites (each developers development sites, test and production).  We have created a makefile to help with that and also help Cloverleaf out so the GUI does not bomb out due to the .svn folders that are out there.  In Subversion the recommended layout is:

project/trunk – development version

project/branches – all branches that have been made

project/tags – all tasg that have been made

When we are getting closer to deploying a new version (not bug fixes) of our sites we create a new branch (copy of trunk at that time).  In our case we name the production branches “production-ccyymmdd”.  The person responsible for migration will then update the symbolic link to the test site in $HCIROOT (like joe_projectname_test) to point to the new production branch in his local subversion tree.  The site is tested and updated to have the appropriate timeouts and other stuff that may differ between production and development).

When everything is ready and approval has been received for the production move, we shut all processes for the site in question down in production and shut down the site daemons.  We update the production server’s copy of the subversion repository for the new production branch, and update the symbolic link in $HCIROOT to point to the new production branch.  We run our makefile (make newsite – meaning the site does not have an exec folder) and start everything up again.

The benefit: If something goes wrong we change the symbolic link back and start the old production branch up.  No time lost.  The branch stays exacly the way it was before.

[Author]

Replies