For years, the Hewlett Packard Alpha/AXP OpenVMS platform has hosted major M (MUMPS) implementations, including GT.M from Fidelity Information Services, as well as DSM, MSM and Caché from InterSystems. With the sunset of the Alpha/AXP OpenVMS platform looming and no longer over the horizon, GT.M on x86 GNU/Linux provides an economical replacement platform. A low-risk migration approach consisting of reversible small steps eliminates the risk of traditional “big bang” conversions.
This document shows you how.
GT.M is a mature – in production since 1986 – highly robust and scalable implementation of M. It currently runs large production systems on Alpha/AXP OpenVMS & x86 GNU/Linux, as well as on other platforms, including what is, to the best of our knowledge, the largest real time core processing system that is in daily live production at any bank anywhere in the world today. Individual database files of hundreds of GB, with aggregate database sizes in the TB range are in use by multiple customers.
GT.M has unique functionality for deploying mission critical applications that require 24x365 continuity of business, applications that must remain available not only in the face of unplanned events but also in the face of planned events, and even planned events such as application software upgrades that require a change to the organization of the data in the database (i.e., the application schema). This functionality is mature and has been in production since 1999. It is this functionality that facilitates the migration from Alpha/AXP OpenVMS to x86 Linux.
GT.M on several platforms, including both Alpha/AXP OpenVMS and x86 GNU/Linux, is released as free / open source software (FOSS). There are no artificial barriers restricting its use. At the same time, for production use, support with service levels is available on a commercial basis.
There exist many successful ports to GT.M from other M implementations – on multiple platforms, including Alpha/AXP OpenVMS. Many are private (references can be provided – e-mail email@example.com to discuss your use of GT.M). One very visible and public port is that of the VistA healthcare information system, developed by the US Department of Veterans Affairs (VA) and used by the VA on other M implementations on the Alpha/AXP OpenVMS platform – ported to GT.M on x86 GNU/Linux by the community, VistA now runs on GT.M on x86 GNU/Linux at clinics and hospitals in the US and internationally.
At a conceptual level, migrating your M application from Alpha/AXP OpenVMS to x86 GNU/Linux involves two steps:
On the Alpha/AXP OpenVMS platform, migrate from your current M to GT.M. This step can of course be skipped for applications that are already on GT.M on this platform.
Migrate from GT.M on Alpha/AXP OpenVMS to GT.M on x86 GNU/Linux.
Migrating to GT.M from another M implementation requires migrating the routines, migrating the global variables, and bringing up your application in production on GT.M.
With the following exceptions, GT.M supports the M standard (ISO/IEC 11756:1999):
While GT.M does not implement Structured System Variable Names (SSVNs), it provides equivalent functionality.
GT.M short-circuits the evaluation of logical expressions. As a practical matter, unless your application uses functions with side effects, this makes no difference to your application. In the event your application does use functions with side effects, this difference can be handled by minor localized modifications (which are backwards compatible with your current M implementation).
As provided for by the standard, GT.M implements extensions that begin with “Z” – some of these are compatible with other popular M implementations and some are different. To facilitate the migration, depending on the complexity of your application, George James Software has well-regarded tools (RE/parser and RE/data).
Since you will be running your application on GT.M on both Alpha/AXP OpenVMS and on x86 GNU/Linux, you may need some minor refactoring of the application to reflect differences between the platforms. For example, GT.M on Alpha/AXP comes packaged with calls to OpenVMS services that are not necessarily meaningful on Linux; on Linux, you can use the external call mechanism to access standard libraries via POSIX APIs. As another example, your application may have logic that assumes an OpenVMS directory structure; this will have to be modified for a Linux directory structure
Migrating your application to GT.M and testing it can be performed while you continue running your application on your existing M.
For development and testing purposes, it is a straightforward matter to extract a snapshot of the application's global variables from a backup and load them into GT.M databases. A program to extract all of them into a text file that GT.M will load directly is a few lines of standard M code, and a program for Caché is available for download.
Multiple extracts can be run in parallel, depending on available CPU and IO bandwidth, and multiple loads can likewise be performed in parallel.
There are several approaches to achieving this goal. The simplest conceptually, is to bring down the application on your prior M, extract the data, load it into GT.M and bring the application up on GT.M. Depending on the size of your database, and the capacity of your server, this may take from minutes to hours.
A more complex approach, but one that minimizes switching time is:
Take a back up of the database, and use the extract / load approach to move it to GT.M. Let the application continue to run on your prior M, with journaling turned on.
After the load completes, the GT.M database is some hours hours behind the other database. Now, create a new set of journal files and let the application continue to run on your existing M. Extract from the journal files all the updates that were performed, and load this into the GT.M database.
After the above is completed, the GT.M database should lag the other database by a shorter period of time. Repeat the process as needed so that each time the lag between the other M database and GT.M database gets smaller and smaller.
When the time difference between the databases is very small, bring down the application on the other M. Migrate the last set of updates to the GT.M database and bring up the application on GT.M.
Both approaches are low risk in that you will have the routines and database from your prior M implementation. In the event you want to revert temporarily for whatever reason, you can run a mupip journal extract command on GT.M to extract the journal output in a text file that you can easily use to apply the updates on your prior M database with a small program of perhaps a half dozen lines of code. In fact, depending on your needs, you can do this process periodically, say every hour or every day. By managing the gap between your new GT.M application on Alpha/AXP OpenVMS and that on your prior M implementation, you can bound the maximum time that it will take you to switch back, in the event you ever need to.
Many popular M implementations on Alpha/AXP OpenVMS support a protocol called DDP, a bare Ethernet protocol. You may also be able to run a hybrid application during the transition, where some processes run their application logic on your prior M, some run on GT.M, and they all use a common database. This allows the switchover of the database to be decoupled from the switchover of the routines at the cost of a little additional complexity during the transition.
The transition from running your application on your prior M, to running it on GT.M, on the Alpha/AXP OpenVMS platform is a manageable effort, low risk, and with the ability to mitigate even that low risk. Once your application is running to your satisfaction on GT.M on Alpha/AXP OpenVMS, you can migrate to GT.M on x86 GNU/Linux.
Once you are on GT.M on Alpha/AXP OpenVMS, database replication facilitates a straightforward migration to GT.M on x86 GNU/Linux. Skip the Introduction to GT.M Database Replication if you are familiar with GT.M database replication.
In GT.M database replication, an “originating instance” executes business logic and commits updates to its database. As long as an instance exists that can operate in its primary role, the application is available (in this regard, an application on GT.M is no different from an application on any other platform). GT.M database replication adds the ability to stream database updates in real time to a “replicating instance” operating in a secondary role. This replication occurs over a TCP/IP connection. There is no distance restriction between instances – as of this date, we are aware of at least one customer with over one thousand miles between instances. Pictorially, it looks like this:
If either the replicating instance (Site B) or the network fail, the originating instance (Site A) operates unaffected; when the replicating instance and/or network recover, the replicating instance catches up with database updates performed by the originating primary. If Site A fails in an unplanned event, or must come down for a planned event, within seconds Site B can take over as the originating instance. When Site A is recovered, it becomes the new replicating instance:
GT.M provides hooks that an application can use to recover from more complicated failure scenarios. Furthermore, GT.M database replication also has functionality that an application can exploit to provide continuity of business through planned events such as application and system upgrades, and even many application and system upgrades that involve changes to the organization of data in the database.
In migrating from GT.M on Alpha/AXP OpenVMS to GT.M on x86 GNU/Linux, you will use GT.M's database replication across heterogeneous systems.1
The first step is to migrate any routines. With the exception of a few calls to OpenVMS that are prepackaged functions on GT.M on OpenVMS, and which must be replaced with equivalent calls to library functions on GNU/Linux using GT.M's call-out mechanism, the M language functionality on GNU/Linux is a superset of that on OpenVMS.2 IO device parameters can differ. Of course, if you have code that assumes OpenVMS device and file names, that will have to be migrated to GNU/Linux. One operational difference is that GT.M on GNU/Linux does not support DDP.
Before migrating the database, ensure that your OpenVMS instance is operating with a Source Server in passive mode. Migrating the database is a simple matter of using Mupip Extract to export the global variables from a backup on OpenVMS and using Mupip Load on GNU/Linux to populate freshly created database files. One additional step to facilitate replication is to find the Region Seqno from the database files on OpenVMS, and to set the database files on GNU/Linux to the maximum value found on OpenVMS. The Linux instance is now ready. The last part of the migration consists of the following steps:
Bring up the GNU/Linux instance in a secondary role, and switch the Source Server process on the OpenVMS instance from passive to active. The two instances will establish communication. Using the Region Seqno field, the GNU/Linux instance will catch up with updates made on the OpenVMS instance since the backup was taken, and then keep up with the OpenVMS instance as it streams database updates to it in real time. Run in this mode until you are comfortable that the GNU/Linux instance is stable, and that it is keeping up with the database update rates being generated by the OpenVMS instance.
Switchover so that the GNU/Linux instance is in a primary role and the OpenVMS instance is in a secondary role. The GT.M part of this instance can be scripted and accomplished in seconds. Now your application is live on GNU/Linux, but database updates are streamed to the OpenVMS instance to keep it up to date. If you find you have an issue with your application on GNU/Linux, you can again switchover so that the OpenVMS is again your production environment. You can take corrective action for your application on the GNU/Linux instance and try again. Once you are comfortable that your application is stable on GNU/Linux, you can bring down the OpenVMS instance.
Having accomplished the migration, FIS recommends that you bring up a second instance on GNU/Linux so that your application can continue to be available when the production instance goes down in an unplanned or planned event. On GNU/Linux, GT.M permits an unlimited number of additional instances for whatever level of redundancy you need.
While platform migration is not always trivial, the process of migrating from any M implementation on Alpha/AXP OpenVMS to GT.M on x86 GNU/Linux virtually eliminates risk by breaking down the process into a series of straightforward, manageable steps, each of which can be rapidly reversed in the event unforeseen behavior occurs when that step is taken.
The fact that GT.M on x86 GNU/Linux is FOSS is simply an added bonus that makes it a cost effective target platform with 24x7 support available on commercial terms.
The techniques described here also apply to GT.M on Linux on Itanium based servers. While GT.M is not FOSS on that platform, it is competitively priced.
As of this date, GT.M is also available on AIX on IBM pSeries, z/OS on IBM zSeries, Solaris on Sun SPARC, and HP-UX on Itanium (as with GNU/Linux on Itanium, GT.M on these platforms, it is competitively priced even though it is not FOSS). Although an originating instance on OpenVMS on Alpha/AXP can replicate to a replicating instance on these platforms, an originating instance on these platforms cannot replicate to a replicating instance on OpenVMS on Alpha/AXP. To effect a migration from M on OpenVMS Alpha/AXP to GT.M on one of these platforms in a series of small, reversible steps, first migrate to GT.M on x86 GNU/Linux, and then to your preferred final destination platform.