Eliot’s Tips and Tricks

Introduction

If you are impatient like me and you are already proficient in other programming languages, you should appreciate this section of the wiki. The fastest way to learn what you need to know about Lua is to install Iguana and look at the examples. Then look at these tips.

This is a small collection of tips from my experience of learning Lua, it only describes the surprises that I found different from other languages I program in.

Other languages [top]

I could say my principle language is C++ which I have a love-hate relationship with it. I love how you can write very efficient code with C++. I dislike it’s complexity. It’s an easy language to shoot yourself in foot with. You really have to work hard to stop your team from using every feature in the language. Template meta programming is a great way to transform time and money into nothing. I tend to advocate what I call C+ programming which is cutting things down to a very limited subset of the language. That tends to grate with experienced C++ programmers who tend to love to show their mastery of every part of the language.

Sadly I would say C++ is not suited for most commercial development. It’s good in that it’s an open standard language and supported in many environments.

I’ve done a lot of Delphi development since the entire Chameleon GUI was written in it. It’s a pain though that it’s now stuck in 32 bit form of a declining platform, WIN32. For its time Delphi was impressive. It’s still one of the fastest strongly typed language compilers I know of. Its biggest drawback is it’s owned by one company.

I’ve done a smattering of C# and Java since we needed to implement stub code support for them from Chameleon.

Done a fair bit of Python but it’s not a language I ever had much love for. I picked it back in 1998 for Chameleon. Seems to be popular for many people. I always saw it as a means to an end. Compared to other dynamic languages like Javascript and Lua to me it seems rather hairy with too many unnecessary features.

Javascript (ECMA) is an important language for since it’s the language of the web. There you really have no choice unless you want to go and use that funny thing that google has GWT. I’ve never used it myself but I know people that swear by it. For us though the Translator is built on a lot of Javascript with the data most served up from the backend using JSON.

Lua is obviously another language that I have had a lot of experience with. It’s my favourite dynamic language. Lua had a lot more time to be ‘fully baked’ compared to Javascript. I would agree with Douglas Crockford’s statement about Javascript that : “It never got a trial period in which it could be corrected and polished based on actual use. The language is powerful and flawed.”

Lua on the hand has had a long time to be carefully honed and refined over time. It’s surprisingly widely used in applications where performance and having a small memory footprint is important. Lua is about the most simple a dynamic language could be without losing it’s power. I’ll have more to say about it later.

In general at this point in my career I think you could program ‘good’ code in just about anything. For me good code is about keeping things simple, modular and decoupled. I try to use as few features of the language as I need to. The most challenging part of programming is always to get a good understanding of the domain problem.

I had a programmer called Randy that I think put it very well. (he was complaining about some awful code we had in Chameleon back in 2007). The way he put it:

In the early part of my career I wrote simple code because that was all I knew how to write. When I got more experienced I wrote complex code. Then I went back to writing simple code not because I didn’t know how to write complicated code but for the reason it is much easier to maintain and use.

Although I didn’t appreciate the message at the time (it was the code I earned my living from) he was absolutely right. It makes total sense to avoid getting fancy with code more than you need to and keep things simple and dumb. That way you can debug it more easily, other people can understand it and you yourself can understand it when you come back to it in six months time.

It’s actually A-okay to write a few more lines of simple linear code than to try and write it using more clever code. It’s just a matter of giving yourself permission to do so.

1 based indexes [top]

Lua uses 1 based indexes throughout all its APIs.

i.e. if you have an array of 3 items then they are indexed as 1, 2 and 3 – not 0, 1 and 2.

This is different convention from most other languages like C++, Java and C# so it takes a little getting used to.

Personally I do not find it a big deal when using Iguana – because if I get it wrong the annotations immediately tell me how I screwed up.

The good news is that all the APIs in the Iguana Translator are 1 based. This is much better than the Python APIs in Chameleon which use both 1 and 0 based indexes (ouch). Not much we can do about that without making current users upset – fixing it would break backwards compatibility.

_G: The Translator API [top]

_G gives the ‘global’ table which has all the libraries and functions under it. So you can use it to browse the entire namespace. These screen snaps give a hint of what can be done. Here we can see all the Lua table library functions:

Doing trace(_G) allows you to get a browseable tree of all the functions available, in this screen shot shows the expansion of the net.http namespace:

Deep auto-completion works nicely too if you know roughly what you are looking for:

You may also find the Iguana Translator Modules reference useful too.

Use ‘..’ to join strings [top]

Lua has a slightly unusual operator for concatenating strings, it is ‘..’

For example:

local Count = 10
local S = 'Fred '..' the dog had '..Count..' balls.'

Concatenation will change some types into their number representation.

Local and Global Variables [top]

This is small flaw in the language, but it is hard for the Lua designers to change, because it will break backward compatibility with existing code. If you define a variable in a function always remember to prefix it with the keyword “local” otherwise it will have global scope in the program.

This code fragment illustrates the issue:

Here you can see that because the variable A was not explicitly defined as being local in scope, the fooBar() function can access the value it was set to in the foo() function.

Loops and # [top]

The loop count operation is always only invoked once. That is a really good design decision in my opinion since it is efficient.

The loop variable is a temporary local variable which is only valid within the loop. If you try and access it afterwards you will get ‘nil’.

As you can see the loopCounter() function is only called once when the loop is initialized

Lua defines # as the count operator. This makes it very straightforward to loop through repeating parts of an data-structure, be it an XML document, HL7 message, results for a query, the interface is always the same.

Putting # in front of the object gives it number of children it has. For example #Msg.NK1 might give the number of next of kin NK1 segments. Once you have your head around that concept you can handle any type of repetition. This is a over simplified example :

The for statement can also be used to print or process every element of a table. For more information on how to do this, see the Lua for tutorial.

Not equal to [top]

This was one aspect of Lua’s syntax that confused me when I was learning the language. I even had a ticket in our bug tracking system suggesting we add “!=” to the Lua language.

There is another gotcha with comparison operations in Lua. If you compare two variables that point to different types then they are not considered equal. Notice in this example code that the not equal operation in line 12 is not triggered. This is because a comparison is being made between a string and what we call a “node tree” in the Translator:

Here is the code to copy:

local function trace(a,b,c,d) return end

function main()
   local X = xml.parse{data = "<patient name = 'Fred'></patient>"}
   compare(X.patient.name)
end

function compare(name)
   -- comparison between node tree and string is not equal
   if name ~= 'Fred' then 
      trace(name.." isn't here") 
   else
      trace('Hello '..name) 
   end

   -- convert node value to a string and the name matches
   if name:nodeValue() ~= 'Fred' then 
      trace(name.." isn't here") 
   else
      trace('Hello '..name) 
   end
end

Multiple function returns [top]

Returning multiple values from a function is something that you really learn to enjoy! It makes sense really, functions can take multiple inputs, why not multiple outputs?

Here’s the syntax:

local A,B = Foo('blah')

function Foo(Msg)
   return 1, "Two"
end

The colon operator [top]

Lua has a bit of syntactic sugar that allows one to have object orientated semantics. When you see:

O:blah(A)

It’s equivalent to:

O.blah(O, A)

i.e., call the function that is defined using the property ‘blah’ of object O and call it with the object O as the first parameter. Hopefully this screen snap of the Translator gives more clarity.

The example shows different ways we can invoke the Lua “sub” function which gives a substring. The sub function is defined in a special Lua table called ‘string’. Strings in Lua have a set of library functions which are very handy in manipulating Lua strings.

Functions are first class values in Lua, which means they can be assigned to variables, as you can see in the above code.

Writing extension functions [top]

This is a very useful part of the language. It is easy to extend the built in libraries with custom methods. Each Lua library is effectively a collection of functions within a Lua table. You can easily add more functions into these tables. Two important tables are called “string” which covers the Lua string library and “node” which covers the node tree objects of the Iguana Translator.

This shows an example that we ship in the stringutil module that extends the string library. Here is its definition and you can see the guts of how it works from the annotations. The sub, upper and lower functions are all part of the standard Lua string library.

We have thus extended the string library and this new method can invoked on any string object in Lua. Here is an example of where we call it:

Because we are in effect adding functions to a Lua table there are alternative syntaxes that could be used:

string.capitalize = function(self)  
   -- body goes here
end
-- or
string['capitalize'] = function(self) 
   -- body goes here
end

Take a look at the code in the stringutil module for several more examples of extension functions.

Finding members by name [top]

This childIndex node extension method can be used to verify the existence of a child node with a given name:

Here’s the code in a format which can be copied:

function main(Data)
   local Msg = hl7.parse{vmd='demo.vmd', data=Data}

   checkForPID(Msg)
end

function checkForPID(msg)
   if msg:childIndex('PID') then
      trace('Has PID')
   end
end

function node.childIndex(Node, Name)
   for i=1, #Node do
      if Node[i]:nodeName() == Name then 
         return i 
      end
   end
   return nil
end

Note: We included the node function node.childIndex() in the main module for convenience, best practice is to put it in the “node” module.

Key libraries [top]

There are three key libraries in Lua that you should familiarize yourself with:

  • The string library – for doing string operations. It includes a good regex like pattern matching library.
  • The io library – operations on files.
  • The os library – date/time call and invoking processes.

Like Lua itself these libraries are small and elegant.

Calling functions with {} [top]

Some functions in the Translator API are called using curly braces {}. This is because the function takes a single table as an argument. Lua supplies some syntactic sugar here, allowing us to omit the implicit brackets (). The following code shows several equivalent ways of calling a function that takes a Lua table:

The reason for using this format in many of the Translator APIs is to give the flexibility to introduce new parameters and deprecate older ones, without breaking backward compatibility. It is a technique often used with dynamic languages like Lua and Javascript to future proof APIs.

Use short functions [top]

Keep your functions short! This is good advice for any programming language.

You should always be able to see the entire body of a function within one screen. In general if I am writing a mapping function I will try to only map one ‘node’ to another. This code is poor practice:

The problem is:

  • It is hard to reuse these mappings. If the PID segment was in a different spot in the message grammar say under a group called “PATIENT” then we have to define all the same mappings again.
  • It results in repetitive long code

A better structure is like this:

The MapPID function is modular and succinct. It can be used from any context where a PID segment needs to be mapped into the patient table. I have deliberately chosen a short variable name of “T”. This is because:

  • It makes for more compressed code giving more space for the annotations.
  • The annotations show that T is “patient Row” in this context anyway so there is no benefit in using a descriptive name.

The MapWeight function is being handled using the example code found in the HL7 to Database example.

Handling optional segments [top]

This can be one of the more subtle areas of mapping HL7, how to map optional segments. A few pictures will go a long way to explaining the concept and how to deal with them in the Translator.

Imagine we have a message that has a repeating group of segments. The first segment of each group is always the a patient identification segment (PID) which is followed by an optional NTE segment:

MSH|^~&|MESA_RPT_MGR|EAST_RADIOLOGY|REPOSITORY|XYZ|||ORU^R01|MESA3b781ae8|P|2.3.1||||||||
PID|||4525285^^^ADT1||Smith^Tracy||19980210|F||Martian|86 Yonge St.^^ST. LOUIS^MO^51460|||||||10-346-6|284-517-569|
NTE|1|Patient is silly!|
PID|||5488754^^^ADT1||Smith^Fred||20041012|M||Martian|20 Delphi Cres.^^Chicago^IL^55037|||||||59-693-654|558-171-617|

The life.vmd is used to define the grammar of the message as:

<> means that the PATIENT group is optional and repeating.

[ ] means that the NTE segment is optional.

Now when use this vmd file to parse the message and look at it in the Translator we see something like this:

In this case the optional segments are not showing. I can show them by checking Show Empty Nodes option.

As you can see the non-present segments are shown greyed out:

There is a helpful utility method supported by nodes which helps us to detect non present HL7 segments through the API. It is called isNull().

Here you can see isNull() returning false for the first call to the MapNote function:

Looking at the second annotation you can see isNull() returns true and so you can see than we do not invoke the mapping code in this case.

A mistake people often make it is write code like this:

if NTE ~= nil then
   T.Note = NTE[2]
end
-- or
if NTE then
   T.Note = NTE[2]
end

This will not work as expected because an optional segment which is not present does not result in the member being equal to ‘nil’. It is present in the node tree but just empty.

Hopefully that explains how to use the isNull() method. If it is not clear then let us know and we will try and do a better job of explaining it.

Here is a copy of the code used for this example:

require("node")

function main(Data)
   local Msg,Name = hl7.parse{vmd='life.vmd', data=Data}
   local Out      = db.tables{vmd='life.vmd', name=Name}

   if     Name == 'Lab' then ProcessLab(Out, Msg) end
end

function MapPatient(T, PID) 
   T.Id = PID[3][1][1]
   T.LastName = PID[5][1][1][1]:nodeValue()
   T.GivenName = PID[5][1][2]
   T.Race = PID[10][1][1]
   return T
end  

function MapNote(T, NTE)
   if not NTE:isNull() then
     T.Note = NTE[2]
   end   
end

function ProcessLab(Out, Msg)
   for i=1, #Msg.PATIENT do
      MapPatient(Out.patient[i], Msg.PATIENT[1].PID)
      MapNote(Out.patient[i], Msg.PATIENT[i].NTE)
   end
   return Out
end

Working with errors [top]

Different, simple and elegant.

The error call can be used to explicitly raise an error and will stop a script. If instead you wish to catch the error then use the pcall function to call a Lua function in protected mode. This means you can check the first argument returned to see if the function ran without error, true or false. If an error occurred the second argument returned is the error message.

So it means your scripts can skip past a message that creates an error or perform any other kind of error handling you choose to do. This code demonstrates the use of pcall hello

The function you want to call is passed as the first parameter, followed by the parameters to the function. pcall() will return false if an exception is thrown.

If we comment out the error message:

The format of error strings can be a tricky to break apart, you may need to parse strings for proprietary error numbers/phrases (which will differ depending on the system raising the errors). If you have questions about this please contact support at support@interfaceware.com.

One slight inconvenience with pcall is that it will suppress the Translator’s helpful error catching mechanism. A helpful technique for dealing with that is to do this:

Here’s the code:

function main(Data)
   local Success, Err = pcall(ProtectedMain, "DataIn")
   if not Success then
      print("Skipping error: "..Err)
   end
end

function ProtectedMain(Data)
   error("Bang!")
   return "Some", "Data"
end

Here’s the code with isTest() added:

function main(Data)
   if iguana.isTest() then
      -- We're in test mode so we won't use pcall
      ProtectedMain("DataIn")
   else
      local Success, Err = pcall(ProtectedMain, "DataIn")
      if not Success then
         print("Skipping error: "..Err)
      end
   end
end

function ProtectedMain(Data)
   error("Bang!")
   return "Some", "Data"
end

More information [top]

Tagged: