Working with HL7

Introduction

This article contains useful techniques that we have developed for working with HL7. We suggest you use this article as a reference and read the section(s) that are relevant to what you are doing. You may also want to skim the article to get an overview of the various techniques.

For an overview of how Iguana works with databases read HL7 Fundamentals.

Tip: In this article we use local functions in the main module, with the main() function at the end of the module. Why do we do this?

  • It is the simplest way to make a well-structured Lua program in Iguana
  • It allows us to demonstrate all the code in a single screenshot
  • It allows you to copy a single snippet of code to the main module to try it out
  • Using local functions hides them from other channels (which is a very good thing)
  • We need to put main() at the end so it can see the local functions (because Lua uses a single pass compiler)

Note: We recommend using local and shared modules for production code.

Dealing with massive lab reports [top]

This is kind of the opposite problem from dealing with large lab reports or other data broken into smaller parts (see continuation messages). This is handling the performance issues of dealing with really massive amounts of data within one message.

The hl7.parse{} routine is in essence a ‘DOM‘ style parser. A DOM style parse is that we take the entire tree of a message and parses it into memory all at once. It’s very convenient to deal with data in this manner since a DOM style parse gives complete random access to any element of data in the message.

Where it runs into problems is when the amount of data becomes very large. This can often happen when you have very large lab results or binary data. These can consist of reams and reams of OBX segments which all have a very similar structure.

For this type of problem is may be desirable to switch more to a SAX style parse. That is we start at the first OBX segment and then we sequentially move through them processing them one by one and consuming the data as we go.

Now for the other part of the message it’s still quite likely to be useful to process that data using the DOM style parse – it’s a matter of chopping out the large repetitive part of the message first.

This can give you the best of both worlds:

  • DOM style parse for the field level data for convenient clean APIs
  • SAX style parse for the body of the lab report for speed

SAX parsing is one possibility – another cool trick is leverage things like Lua’s fast global pattern matching substitution for which we give an example below.

If you are curious let us know if you are interested in seeing a fully worked example and we can put it into the wiki here.

Example lab message with a large textual component

Say you have a message like this:

MSH|^~\&|RESULT|CHRS:CLA:CFC^CHRS:CLA^CFC|||20100819141949||ORU^R01|ORU-R01-20100819141949830|^PK.E|2.3|4.3.2. MOBILIZER_4_3_2_2-20090417_0946  094605 CLA1
PID|||dsf^^^CHRS:CLA^BACKENDID^^dfsd:201008180900 HPL^true|^^^^MRN^^^true||||||||||||||MF00380390:MF286640:20100817.1015:IN:AF0000676886^^^CHRS:CLA:CFC^BACKENDID^^MF286640:961424734201008181BENROR1JONGAP1^true
PV1
OBR|||R:758481^CHRS:CLA:CFC^^^z7-3961495615SignedRADCFCRADLEMMONS,GARLAND JR:19320417:MF00380390:|Radiology:RADIOLOGY REPORT^RADIOLOGY REPORT^dddf:CLA:CFC^Radiology|||20100817165600||||||||||||||||||Signed^^sdfds:CLA:CFC
OBX|1|TX|||Big honking report...
OBX|2|TX|||Big honking report...
OBX|3|TX|||Big honking report...
OBX|4|TX|||Big honking report...
OBX|5|TX|||Big honking report...
OBX|6|TX|||Big honking report...
:
OBX|93634|TX|||Order: 0817-0137
OBX|93635|TX||| Procedure: CHEST 2 VIEWS
OBX|93636|TX||| Ordered by: dsfd,34sdfs M.D.
OBX|93637|TX||| 
OBX|93638|TX|||Dictated: esd,gdfdf S M.D.   Aug 17, 2010 4:56pm
OBX|93639|TX|||Transcribed: sdf S rdfdf   Aug 17, 2010 5:02pm
OBX|93640|TX|||Signed: erwss,ersds S M.D.   Aug 19, 2010 10:06am
OBX|93641|TX||| 
OBX|93642|TX|||SECTION: 1  Final
OBX|93643|TX|||TWO VIEW CHEST DATED 08-17-2010 
OBX|93644|TX|||CLINICAL HISTORY OF PRE OPERATIVE FOR CORONARY ARTERY BY-PASS GRAFT. 
OBX|93645|TX|||THERE IS NO PRIOR EXAMINATION AVAILABLE FOR COMPARISON. 
OBX|93646|TX|||FINDINGS: 
OBX|93647|TX|||THERE ARE SOME CALCIFIED DENSITIES SEEN OVERLYING BOTH HEMITHORACES.  SOME OF THESE MAY 
OBX|93648|TX|||REPRESENT CALCIFIED GRANULOMAS.  OTHER AREAS MAY REPRESENT SOME CALCIFIED PLAQUES.  NO 
OBX|93649|TX|||INFILTRATE IS SEEN.  HEART IS NORMAL IN APPEARANCE. AORTA IS ECTATIC.  PULMONARY 
OBX|93650|TX|||VASCULARITY IS UNREMARKABLE.  BONES AND JOINTS SHOW NO ACUTE ABNORMALITY.
OBX|93651|TX|||IMPRESSION:  
OBX|93652|TX|||MULTIPLE CALCIFIED DENSITIES SEEN OVERLYING BOTH LUNGS, POSSIBLY REPRESENTING PREVIOUS 
OBX|93653|TX|||GRANULOMATOUS DISEASE EXPOSURE.  SOME AREAS MAY REPRESENT CALCIFIED PLAQUES, AS WELL.
OBR|||R:758481^CHRS:CLA:CFC^^^z7-dfsdfds,dsfd JR:19320417:MF00380390:|Radiology:RADIOLOGY REPORT^RADIOLOGY REPORT^CHRS:CLA:CFC^Radiology|||20100817165600||||||||||||||||||Signed^^CHRS:CLA:CF

Then we can write code which follows this algorithm:

  1. Find the data before the first OBX segment.
  2. Find the data after the last OBX segment.
  3. Concatenate that together to make a very small message which can be parsed using hl7.parse{}
  4. Take all the OBX data and use Lua’s global pattern replacing to convert it into just the text.

This turns out to be a very fast algorithm since Lua’s pattern replacing is extremely fast and we avoid the overhead of creating an entire tree of OBX segments. Here’s the code:

local function ObxChop(Data)
   local First = Data:find('OBX')-1
   local LastObx
   for i=#Data, 1, -1 do
      if Data:sub(i,i+3) == '\rOBX' then
         LastObx = i+1
         break
      end
   end 
   local E = Data:find('\r', LastObx)
   return Data:sub(1, First-1)..Data:sub(E), 
          Data:sub(First+1, E)
end 

function main(Data)
   local Data, LabReport = ObxChop(Data)
   LabReport = LabReport:gsub('OBX|%d+|TX|||', '\n')
   LabReport = LabReport:sub(2, #LabReport)
   local Msg = hl7.parse{vmd='example/demo.vmd', data=Data}
   print(Msg.PATIENT.PID[18][1]:nodeValue())
   print(LabReport)
end

Specify the minimum number of fields in an HL7 segment [top]

There are times when you need to show empty segments in an HL7 message. Perhaps the receiving application expects certain fields in a message, even if they are empty. For example you may require the first 26 fields in the PID segment.

So if you get a message that looks like this:

You need to make it look like this:

This is how to achieve this using Iguana:

  1. Create a node extension function node.setMinFields().
  2. Call it from main.
  3. I used the trace() function to show the message before and after.

You can import the project from the specify_no_of_fields.zip file, the sample data is included in the project.

If you want to know a bit more about how the code works – then here we go…

This is code from the node module in the zip:

  1. All it does is put an empty string ” in the Nth place in the node tree
  2. That makes Iguana recognize all the fields up to at least that point
  3. As the node tree only accepts data in leaf nodes, I recurse the tree to find the leaf

Formatting output HL7 without trailing | characters [top]

If you apply the tostring() function (which is how the :S() short cut method is implemented) to a HL7 node tree is will format the output such that:

  • There are trailing “|” characters
  • The default HL7 delimiters are used

Sometimes you may wish to have more control over the output format. The best way to do this is to make a node extension routine which gives you the precise format you need.

Don’t worry, we don’t expect customers to write these, if you run this issue just let us know and we’re happy to put together examples of different routines to do this for you.

This example uses a custom function node.flatwire() to format a message without any trailing “|” characters:

function node.flatwire(Node) 
   local R = ''
   for i=1, #Node do
      if Node[i]:nodeType() == 'segment' and not Node[i]:isNull() then
         R = R..Node[i]:S()
         R = R:sub(1, #R-1)  -- get rid of trailing delimiters
         R = R..'\r'
      elseif Node[i]:nodeType() == 'segment_repeated' then
         R = R..Node[i]:flatwire()
      elseif Node[i]:nodeType() == 'segment_group' then
         R = R..Node[i]:flatwire()
      end
   end
   return R
end

function main(Data)   
   local Msg = hl7.parse  {vmd = 'example/demo.vmd', data = Data}
   
   -- Out1 contains the message without trailing "|" delimiters
   local Out1 = Msg:flatwire()
   
   -- Out2 contains the message with the default formatting.
   local Out2 = Msg:S()
end

Tip: Best practice is to put node.flatwire() in a module like “mynode” or “flatwire” (use a local module if the function is only used in this channel – not shared).

Comparing the output, Out1 will contain the custom format:

MSH|^~\&|AcmeMed|Lab|Main HIS|St. Micheals|20110213144932||ADT^A03|9B38584D9903051F0D2B52CC0148965775D2D23FE4C51BE060B33B6ED27DA820|P|2.6
EVN||20110213144532||||20110213145902
PID|||4525285^^^ADT1||Smith^Tracy||19980210|F||Martian\E\|86 Yonge St.^^ST. LOUIS^MO^51460|||||||10-346-6|284-517-569
NK1|1|Smith^Gary|Second Cousin
PV1||E||||||5101^Garland^Mary^F^^DR|||||||||||1318095^^^ADT1|||||||||||||||||||||||||20110213144956
OBX|||WT^WEIGHT||102|pounds
OBX|||HT^HEIGHT||32|cm

And Out2 contains a standard message:

MSH|^~\&|AcmeMed|Lab|Main HIS|St. Micheals|20110213144932||ADT^A03|9B38584D9903051F0D2B52CC0148965775D2D23FE4C51BE060B33B6ED27DA820|P|2.6|
EVN||20110213144532||||20110213145902|
PID|||4525285^^^ADT1||Smith^Tracy||19980210|F||Martian\E\|86 Yonge St.^^ST. LOUIS^MO^51460|||||||10-346-6|284-517-569|
NK1|1|Smith^Gary|Second Cousin|
PV1||E||||||5101^Garland^Mary^F^^DR|||||||||||1318095^^^ADT1|||||||||||||||||||||||||20110213144956|
OBX|||WT^WEIGHT||102|pounds|
OBX|||HT^HEIGHT||32|cm|

Translating Code Sets [top]

The codemap module can be used to translate code sets, this can be useful for any channel not just one using HL7, see the Translating code sets page for more information.

There are two versions of the codemap module: one allows for a default value to be returned if there is no matching code, the second returns nil if there is no match. We generally recommend the second more general module.

Simple list

You can define a list like “AmigoSet”:

Code translation table

Looking up an unknown code “W” returns nil:

Now we add a default so “other” is returned:

Tagged: