Design Decisions in the Translator

Introduction

This blog entry attempts to explain the philosophy and choices made within the Translator with respect to it’s architecture.

Note: This is article is a page from the Iguana 5 wiki so some of the references are a bit dated – but the philosophy remains the same. Iguana 6 continues to build on the same principles.

Minimalist approach to design [top]

We have worked hard with the Iguana Translator to put very little functionality within the product itself. This might seem a little counter intuitive but it has many advantages. Have a look at what happens without minimalism.

One advantage is that it will make it easier to upgrade to new releases of Iguana without breaking interfaces.

From experience with Chameleon and Iguana I know that nobody likes upgrading interfaces. This is one of the biggest pain areas with interface engines that causes a lot of aggravation for their users. Every library that is shipped as a core part of the engine becomes point of potential incompatibility where the slightest subtle change can result in major breakages of interfaces. For instance I recall one instance when we modified Iguana in 4.1 to use unicode and the python regex library was changed and the major pain this caused one customer.

Part of resolving that pain is to build in regression testing as a base part of the platform. We’ve made the first step with this by having the environment have sample data associated with each Translator instance. That is very easy to accomplish with the open nature of the Iguana Translator platform.

The second way is that Lua itself is very clean, very minimalist language with very few built in libraries. Those that do exist are very well thought out, mature and will not change.

The third way we can ease the pain of upgrading is by the way we distribute utility libraries within the environment. Because we have built in source control bundled with the product it provides a very convenient safe way to distribute helper libraries. We can put some of them into the initial repository (Iguana 5 documentation) but that’s it.
Note: This has improved dramatically in Iguana 6 with the ability to import channels from Git repositories.

After the first install the libraries are not updated. We leave it to the user to pick and choose what they need by using resources like this wiki.

Even when libraries are updated the milestone mechanisms in the Translator will help give a lot of protection.

To explain, imagine you have a Channel Foo which uses a library called hl7util. Even if hl7util is upgraded to a new version that explicitly breaks the functionality of Channel Foo, Channel Foo will still work with the last saved milestone, until the point when you try to alter it. At this point your attention is on the interface and so it should be possible to update it.

Doing things this way means as a middleware vendor we can be a lot more adventurous and creative about what library functionality we provide with Iguana since these mechanisms protect customers from upgrade problems.

The environment itself means that it’s much easier to visually understand and debug libraries should they not work.

So for instance we could build nice libraries to do all sorts of interesting things like automatically pulling height and weight out of ADT messages. For handling date/time formats we provide you with a great library that does fuzzy recognition of adhoc date/time formats (Iguana 5 documentation). These libraries can be improved over time, interfaces polished. New interfaces can continue to benefit from improvements and old interfaces don’t need to broken or changed for the sake of change.

I think this will make people very happy once they understand it.

Subtle changes in libraries can break interfaces [top]

In Iguana 4.1 we did a major change to the product to make it more friendly to internationalization by having it do everything internally with UTF8 and using transcoders at the inputs to translate from different encodings into UTF8.

As part of that we needed to shift Python from using the ascii regex version of the library to the unicode library.

One of our customers got burned by this small change. The problem was they had some code within their python that was relying on some implicit behaviour of the ascii regex library which was not a documented part of the interface. This implicit behaviour changed with the switch to the unicode version of the python regex libraries, which broke their code in a manner they didn’t notice.

Hence the emphasis in the Translator on keeping the core libraries very small and building in regression testing as part of the platform.

Why minimalism is important [top]

When people ask why Iguana is lacking “feature XYZ” I direct them to the page on the minimalist design.

See the screenshots below for the “From File” component in Iguana 5.

Actually that’s a lie, please let me correct that.

The screen is so large it needs two screen shots to show all the options.

This is the organic result of listening faithfully to customer requirements and implementing each valid feature request one after another. Every option. Every flag. Every field was driven in response to meeting a customer need.

The problem is that the end result is overwhelming. Although every option is valuable to someone for the majority of people all they see is a mass of incomprehensible choices.

Do you know what is worse?

There are still an awful lot of problems with integration that cover files that are not solvable using this component and it’s companion, the To File component. We could go on forever and add even more options and still not solve all the problems that customers have with files in integration.

Let’s look at the impacts this complexity has:

  • It makes life more complicated for users. There is a lot to understand in order to solve each specific problem. This results in more support calls.
  • It bloats out our documentation. Our old version 4 online manual is huge. It has over 2000 pages of material. Too much documentation becomes as bad as no documentation. It becomes hard to find what you need. That’s why this wiki (Iguana 5 documentation) has become the new official source of documentation for Iguana 5.
    Note: The Iguana 6 wiki is even more user friendly!
  • It makes maintenance difficult. Working on the file components is hard. The component has so many features that if someone adds functionality the likelihood of breaking something is high.
  • It’s expensive and time consuming to train our own staff. To learn every nook and cranny of the options can take months.

Think of what it takes to QA such a component.

Let’s do some rough math and calculate the number of approximate code paths we have here.

2 x 3 x 2 x 4 x 2 x 3 x 3 x 2 x 3 x 2 x 2 x 3 x 2 x 2 = 248,832 different paths!

Although I employ some real genius talent in my dev team, they are people, not minor deities. The chances that every single one of these combinations works perfectly is slim.

With the Translator we have:

  • The elegant built in Lua IO library which handles files.
  • The Lua string library.
  • Some interfaces to the curl library

These simple APIs together into the environment of the Translator (Iguana 5 documenation) and gives simple tools to solve any integration file problem. Iguana 5 really has no need for the To/From File components. They are only needed for backwards compatibility. See the file operations (Iguana 5 documentation) and ftp interfaces (Iguana 5 documentation) with the Translator.

At some point it would make sense to make a new version of Iguana that could shed the skin of complex past and rise like phoenix, a beautiful simple solution.

From File Component

The number of options in the From File component are so large we need to use two screen shots to see them all.

Orthogonal APIs: Rethinking Present but Null [top]

One of the design mistakes in Chameleon was the way there was bleed over in the table APIs of HL7 concepts which really had no place being there.

In the HL7 standard there is a somewhat ill defined and inconsistently implemented concept of that if you see a field with |””| in it that it means that field is defined in the feed, but it is NULL. This is opposed to || which mean field might either be:

  • Not implemented.
  • Not changed.

At some point in an effort to be helpful a patch was introduced into Chameleon that used the python None value to represent some of these concepts, see: Null, Empty and Invalid Values (Iguana 5 documentation).

The problem is that it means that:

  • The API has become complicated.
  • It makes too many assumptions about the given feed which are likely to be wrong, the interface programmer implementing the feed is in a much better position to know what |””| and || really mean with the context of this feed.
  • There is bleed over of HL7 concepts going into what should be a database API.

Finding simple solutions is a lot harder than slapping together complex ones and in the first prototype of the table API in the Translator some of these bad choices were being ported over. See my joke about the pot roast. This stuff always smelled wrong to me. After much discussion and thought we came up with a much simpler more logical solution.

Basically we added an API to the table object that would allow the interface programmer to explicitly remove a column from a record that was to be inserted into the database, like this:

Patient:remove("PhoneHome")

This makes the choices very explicit and clear and 100% under the control of person who really understands the interface, the interface programmer actually writing the interface.

C Level Perspective on the Translator [top]

If you are the person doing the work with the Translator it’s not hard to see it’s value.

It’s an easy environment to learn and be productive in quickly. It’s pleasant to use. Lua is a powerful (Iguana 5 documentation) language, but simple to learn since it is so minimal (Iguana 5 documentation). The graphical annotations and data aware auto-completion provide a ease of use that no other environment offers.

In fact as a programmer it’s a lot less effort to learn this engine than to try and wrestle with an opaque graphical mapper where you have to dig into a whole bunch of unfamiliar screens which don’t do what you need them to do so you end up coding everything in Java or C# anyways. A lot of traditional integration engines end up being expensive shelf-ware.

But a manager does have a broader perspective in making choices with respect to picking technologies. If you are such a person, then this section is for you. Like you I too face the same challenges and concerns that you have with managing technology in the context of an organization.

This section is about quickly giving you the information you need to know to understand why the Translator is a great solution for your organization.

Eliot Muir, CEO of iNTERFACEWARE

The hood of the anorak

I had a old boss back in the UK in 96′ that has a great expression, “Just give me the hood of the anorak, not the entire coat“.

If you don’t understand the expression see the link, sometimes British humour doesn’t translate well to North America.

If you are busy and you don’t want to read through this entire section here’s the high level goods on how the Translator is going to help your bottom line:

  • It’s an integration engine which really works.
  • It’s not expensive.
  • Your people will actually want to use it.
  • It’s simple and easy to use.
  • You’ll get systems integrated faster with better quality.
  • It makes things transparent and gives your organization security as staff turn over.

Bottom line, it will save your organization money by making integration more cost effective and allowing more work to be done.

I understand what it’s like to manage a technical team

In my experience from running a software company, managing a technology team can be stressful. The analogy of herding cats while being a cliche is quite appropriate.

Technical people often forget that they aren’t necessarily going to be around or present at the time that the technology they have implemented goes wrong. What is more frustrating is that your team always has much more time to look at more technologies that one single manager can ever absorb.

As a technical manager you often have to play the role of gate keeper, keeping out technologies that one of your people might like but that you know would be headache overall for the organization to support.

It’s challenging. I can look at my own decisions, sometimes I have made good decisions in keeping out certain technologies. Other times I have been the one putting the breaks on what would have been good ideas just because at the time I just didn’t have the mental bandwidth to comprehend what my staff were trying to say.

Technical people can often not be skilled at quickly articulating the relevant information you need to know as a manager to make a good decision. I hope this section can give you the information you need to understand how the Translator will help your business.

Eliot Muir, CEO of iNTERFACEWARE

Will it be hard and expensive to employ people with the skills to use this product?

Middleware is usually over sold on the promise that you can take a lower cost non-technical person, put them in front of this high priced graphical drag and drop integration engine and have them productively chugging out interfaces for you.

It’s really not true, the economics do not pan out. Integrating data does require technical people to be involved. No matter what solution you pick some people with technical skills are going to be required. A pure hardware guy is not going to be able to operate an engine. You can only go so cheap on staff before you will start to pay for it.

I used to have a mentality of hiring cheap staff. But then I discovered that cheap people are cheap for reason. I paid for it. I got headaches that I had to pay competent people to clean up.

When it comes to staff, you do get what you pay for.

The Translator is a solution to make a competent person more productive.

The Iguana Translator does require that the person using it is comfortable with simple programming. It’s the most productive environment possible for coding because of it’s unique visual nature. Look at the video (Iguana 5 documentation). The good things are:

  • You don’t need a ‘professional Lua programmer’. There is no such thing. It’s like being a professional tiddlywinks player! Anyone who knows any kind of language whether it’s C#, Java or Delphi is going to find picking up Lua a doddle. That’s why I picked it (Iguana 5 documentation).
  • It’s much easier to find someone with development experience than to find someone familiar with a specific interface engine or who has experience with HL7. The beauty of the Translator is that any programmer can pick it up and get the work done without having ever dealt with EDI or HL7 before.
  • As a manager you will love the productivity you get out of people using Lua within the Translator. I love the productivity I get out of my own people with it.

It’s also going to be a much easier sell for you as a manager to convince your people to use the Translator. It’s an environment which is fun to use and technical people who use it will feel that they can use their development skills in a manner which helps them grow. This is in contrast to a traditional engine which most developers hate since:

  • They find it cumbersome.
  • It feels like it dumbs things down which doesn’t help them keep their skills sharp.

I don’t know about you, but I have certainly bought products which my team has refused to use. The Iguana Translator is not one of those products.

Remember that good quality employees will be sizing you up as an employer just as much as you are trying to assess their fit for your organization. The Translator makes your organization a more attractive place to work for.

So in summary, using the Translator will make it easier to find, recruit and retain the kind of people that have the skills required to handle mapping medical data. It’s not a challenge from a human resource perspective.

Eliot Muir, CEO of iNTERFACEWARE

Aren’t GUIs easier to learn?

The Translator is a GUI. Just not a conventional one, in a good way.

The reality is that if you talk to our existing customers, people that have really had to make Chameleon work vs. doing the equivalent work in the Translator they find the new environment a lot more straightforward and easy to work with. Look at the comments on my blog about the failed promise of graphical mappers.

One comment I really liked was from Danney Cairney from MD Staff:

After receiving a demo regarding the Iguana 5 beta, I decided to create a new channel from scratch. Despite having no experience with Lua, I was able to complete the channel and begin inserting records in the database in just a couple of hours. With previous versions of Iguana, it could take days to create the the mapping and then debug and test the Python snippets that were needed to keep track of the relationships between all the items.

With Iguana 5, not only is it much easier to keep track of the relationships between repeating segments, but testing and (more importantly) debugging is much easier with the real-time preview that includes both the data contained in the message and where it is mapped in the table. At any point in time you can see the values that will be going into the database tables for any sample message you may have.

Although this blog post focuses on how scripting is better than the graphical mapper, scripting in Iguana 5 includes the most useful portions of the graphical mapper allowing you to bring up segments and fields based on their names (or even their values) rather than having to know their position. More importantly viewing and filtering the available columns, segments, etc. as you type makes mapping and transforming data much faster and easier to follow than the various mapping and Python scripting windows found throughout the previous versions.

Another one was from Tony J Julian Sr from the Mayo Clinic and Co-Chair of HL7 Infra-structure:

Scripting is the only answer when you have a large quantity of interfaces to handle.

People that actually know what HL7 interfacing is really about do get it.

Eliot Muir, CEO of iNTERFACEWARE

Let’s compare mapping patient demographics with Chameleon and the Translator

A picture can be worth a thousand words. People often don’t realize how little of the mapping process a traditional middleware GUI actually tells you. These comparative screen shots should make it more apparent. Given a simple interface where we are mapping patient demographics and next of kin this is what it would look like in the Translator:

So the Translator is showing the following:

  • The mappings, in raw numerical form and in expanded text form.
  • Values of the live data that is actually being mapped.
  • Plus you can see there are four invocations of the part of the mapping next of kin.
  • Both mappings can be seen within one screen
  • It’s transparent to see that the patient ID is being passed into the MapNextOfKin

It’s really powerful way to visually see the mapping, it is truly a GUI in it own right. Furthermore as a user you can click on the objects like the patient row and the PID segment to graphically see the data like so:

Now let’s compare the equivalent mappings with Chameleon. First of all you have to dig quite a bit of find the actual mapping screen. That’s normal for any graphical mapper. The information is then fragmented between two windows:

and

As you can see the graphical screens in Chameleon do not have any more information than the Translator screens. They have less. Comparing the screens:

  • The Translator screens show live data which is not shown in Chameleon screens which separate the configuration from the testing environment.
  • We cannot see how how many times the next of kin mapping is invoked using this GUI.
  • The Chameleon GUI is pretty fiddly. There is a lot of busy work required to manipulate the splitter windows and scroll bars.
  • The Translator is much more transparent, everything can seen within a single window.
  • The GUI does not show how we get the patient ID key into the NextOfKin table. This would need to be done elsewhere.

Obviously the Translator is a more modern interface, you’d hope so, iNTERFACEWARE has been in business since 1997 so it’s a reasonable expectation that we have learnt one or two things since then. Modern web browsers provide a much more powerful environment to make great GUIs compared to what was possible in 1997 using Windows, which is when the bulk of Chameleon was written.

Things get worse for Chameleon once you throw in the requirement for doing any kind of logic which cannot be done using straight graphical mapping. This is more the rule than the exception (see my blog again). It’s not transparent at all when python scripting comes into the mix.

Eliot Muir, CEO of iNTERFACEWARE

Managing custom code development is a pain in the proverbial

Many organizations do find it a bit of a nightmare trying to have the infra-structure required to manage software development. It’s not surprising, it does take a lot to properly manage the issues associated with code and software development. Believe me I know, it took my company years to develop the infra-structure we needed.

To manage software development you need at least the following:

  • Source control
  • Regression testing
  • A configuration management and release system

Many organizations have a hard time because they lack the resources to properly set up this infra-structure. It’s also a headache actually getting your team to properly use and follow the processes.

It’s easy for disasters to happen like finding out that the source code to that critical production system got lost. Or that no one knows what magical background application was making the data flow because Bob the programmer has left. Ouch.

Little wonder many managers avoid any kind of software development within their organization like the plague.

The Translator addresses these pain areas:

  • It has all the above infra-structure built in.
  • It’s simple and intuitive to use.
  • It’s impossible to deploy an interface into production without versioning it.

Furthermore unlike traditional programming environments the live annotations of the Translator make it extremely transparent what the code is doing which eliminates the biggest issue that organizations normally have with code. It always tough figuring out someone else’s code. The Translator actually makes it very easy.

Eliot Muir, CEO of iNTERFACEWARE

What happens when I’m stuck without a programmer?

The fear every manager has is that someone is going to leave and you’ll be stuck with supporting something that no body else in the organization understands.

Relax.

Be happy.

One of the biggest value propositions of an interface engine like Iguana is the relationship and support you get with us as the vendor. Our business model is very straight forward. We like to sell lots of lots of licenses of Iguana. We achieve this by:

  1. Making a good quality product.
  2. Documenting it throughly
  3. When documentation and product is not enough by providing good quality friendly support.

Our support quality is excellent, largely because even though we have over 300 direct customers with over 10,000 sites deployed we do not have a lot of day to day support issues. That’s the consequence of having a good quality product and comprehensive documentation.

Because an interface engine forces problems to be solved in a consistent manner it’s easy for any one of our technical staff to jump in and sort out a problem when it occurs. Effectively our organization becomes a safety net that gives you the insurance you need when staff turn over.

If you are stuck and you need help, we are here for you.

Eliot Muir, CEO of iNTERFACEWARE

Conclusion [top]

Iguana will improve the efficiency of your organization:

  • It will be easy to find and keep people to work with it
  • Your interfaces will be more transparent and easy to manage
  • Iguana really does work and your staff will want to use it

If there are other concerns that I have not managed to address please let me know. I will do my best to address them.

Eliot Muir, CEO of iNTERFACEWARE

Leave A Comment?

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.