• Motivations – typical use cases
• Serializing as native HL7
• Serializing as JSON
• Serializing as XML
• Serializing in Lua Table format

Introduction

There are many motivations for the need to serialize HL7 data into a database and then retrieve it to construct outgoing messages. This section discusses some typical use cases and various solutions, with their pros and cons.

Motivations – typical use cases [top]

There are many situations that would lead to wanting to serialize structured data from an HL7 message or other source into a database or file.

One reason is that your database doesn’t have any fields to (temporarily) store the intermediate data needed to construct outgoing messages. For example, imagine that your application takes in ADT (Admit/Discharge/Transfer) messages and you need to store a few specific fields so that you can mirror that information in outbound lab result messages. You could modify your application database and add the extra fields, or you can structured text data to store the data in a more ad hoc manner.

Another use case might arise in a complex integration situation, where you have a lab system that requires batching of lab orders. They do this in order to share specimens between different tests (since you don’t want to draw 10 blood samples for 10 different lab tests). The business logic will need to delay processing until the full set of orders have been placed (either by waiting for specified timeout or checking a flag), then arrange all order data by specimen type, then create the batch orders as required by the lab system.

Another major advantage of this approach is flexibility, if the data you need to create your report changes you don’t need to change the database.

Serializing as native HL7 [top]

Serializing data as native HL7 has a lot of advantages:

• It’s non-lossy. The complete data set is stored
• Advantageous for HIPAA. In the unpleasant event of a lawsuit, there is a complete faithful record of the original transaction that might have been responsible for the problem
• Compact. Version 2.0 HL7 is very efficient, and does not take up too much space within a database
• Efficient, no transformation of the data is required when you store it
• Easy, very little code is required
• Returns a data structure that plays nicely with the Translator’s auto-completion feature

Here’s an example of dehydrating an HL7 message:

In this case we simply parsed the code into an HL7 node tree.

Here’s the code:

function main(Data)
   DehydrateHl7(Data)
end

function DehydrateHl7(H)
   return hl7.parse{vmd="example/demo.vmd", data=H}
end

Serializing as JSON [top]

The pros and cons of using JSON to serialize data are that:

• If the original data is in JSON, it’s non-lossy
• You can serialize the complete data set, or choose a subset
• It’s an open, non-proprietary format that can be easily parsed and read by many different software programs
• It’s very compact

You can get the Dehydrate (de-serialize) as JSON code from our code repository.

Serializing as XML [top]

Serializing data as native XML has these pros and cons:

Serializing in Lua Table format [top]

Another strategy for serializing data is to place the data you wish to keep into a Lua table. Tables are a fundamental cornerstone of the Lua language. The more you learn about the language, the more apparent it is that almost everything in Lua is a table.

Using a Lua table to store specific fields out of a message has the following pros and cons:

• You only store the data you wish to: this is positive and negative, since, by implication, you are losing some data. This might be significant from a HIPAA standpoint or if you need a complete record in the event of a lawsuit.
• Relatively compact: the formats to which you can serialize Lua tables are similar to JSON.
• Relatively readable: for a technical person, not for a layperson.
• Fast: especially for de-serialization. If you serialize in a format that is compatible with the Lua table literal format, as we show here, the parser processing it is the Lua JIT, which is implemented in machine code.
• Fairly easy: a small amount of code is required, which can be put into a reusable shared module.

The following example shows a relatively simple method for serializing a Lua table containing strings, numbers and booleans (or other tables that are similar):

function SaveTable(Table)
   local savedTables = {} -- used to record tables that have been saved, so that we do not go into an infinite recursion
   local outFuncs = {
      ['string']  = function(value) return string.format("%q",value) end;
      ['boolean'] = function(value) if (value) then return 'true' else return 'false' end end;
      ['number']  = function(value) return string.format('%f',value) end;
   }
   local outFuncsMeta = {
      __index = function(t,k) error('Invalid Type For SaveTable: '..k) end      
   }
   setmetatable(outFuncs,outFuncsMeta)
   local tableOut = function(value)
      if (savedTables[value]) then
         error('There is a cyclical reference (table value referencing another table value) in this set.');
      end
      local outValue = function(value) return outFuncs[type(value)](value) end
      local out = '{'
      for i,v in pairs(value) do out = out..'['..outValue(i)..']='..outValue(v)..';' end
      savedTables[value] = true; --record that it has already been saved
      return out..'}'
   end
   outFuncs['table'] = tableOut;
   return tableOut(Table);
end

function LoadTable(Input)
   -- note that this does not enforce anything, for simplicity
   return assert(loadstring('return '..Input))()
end

Usage is very simple. For example:

function main(Data)
   local test = {['int_test'] = 1.23; ['string_test']= 'a'; ['boolean_test'] = true;}
   test['table_test'] = {['int'] = 2.23;}
   -- yields the string
   -- {["int_test"]=1.230000;["boolean_test"]=true;["table_test"]={["int"]=2.230000;};["string_test"]="a";}
   local result = SaveTable(test)

   --we reverse the operation
   local decoded = LoadTable(result)
   print(decoded['table_test']['int']) -- 2.23
   print(decoded['int_test']) -- 1.23
end

This particular method doesn’t handle tables with cycles – it will throw an error in this case. Cyclical tables are tables that have entries that eventually reference one of their elements.

For instance, if we did the following:

local test = {['int_test'] = 1.23; ['string_test']= 'a'; ['boolean_test'] = true;}
test['table_test'] = {['int'] = 2.23;}
test['table_2'] = test; -- create a cycle

This serialization implementation will raise the following error:

There is a cyclical reference (table value referencing another table value) in this set.

For more information on the basics of serialization, see:

• Saving Tables Without Cycles
• Saving Tables With Cycles

To see other complete serialization methods, see:

• http://lua-users.org/wiki/TableSerialization

Some simplifications that could be considered for this code are:

• Only escaping the names and values when they have special characters. This would make the serialized format smaller and more readable for most instances.
• Only supporting string types would simplify the code and still meet the objective, since the Translator treats every thing as a string anyway.