This topic contains 5 replies, has 3 voices, and was last updated by  Sparks260 6 years, 7 months ago.

How to handle special characters in message field?

  • We are receiving HL7 messages that sometimes have HL7 Separator Characters, i.e. ‘&’ imbedded in a field, i.e. RXE.GiveIndication. Is there a way to convert these special characters prior to the message being parsed into its segments (Which results in the field being broken into 2 separate fields)?

    I’ve looked at the code repository examples on how to prevent escaping of special characters, but it appears that these examples are converting from \T\ back to &.


    Yes, in this situation you’d need to pre-process the message before handing it off to hl7.parse(). The challenge is to know where it’s safe to do this. Since RXE-27 in HL7 v2.3.1 (I didn’t check later versions) doesn’t define subcomponent fields, you should be OK here.

    Here’s an unpolished example; the magic happens in the fixRXE() function:

    function main(Data)
       local msgData = fixRXE(Data)
       local M,mType,mErr = hl7.parse{data=msgData,vmd='PHRM-HL7-V04.vmd'} -- parse message
       -- message transformation code goes here
    function fixRXE(D)
       local msgData = D
       local segList = msgData:split('\r') -- split segments into a segment table
       for i=1,#segList do -- iterate through the segments
          if segList[i]:match('^RXE') then -- look for RXE segment
             local fldList = segList[i]:split('|') -- split RXE segment into a field table
             fldList[28] = fldList[28]:gsub('&','\\T\\') -- replace Give Indication '&' with escaped version
             segList[i] = table.concat(fldList,'|') -- reassemble RXE segment
       return table.concat(segList,'\r') -- reassemble message

    You’ll notice that RXE-27 is referenced as element 28 of the fldList table. That’s because the RXE segment ID ends up in element 1, which creates an offset of 1 as compared to HL7’s numbering scheme vs. the resulting table.

    Jeff Drumm ◊ VP and COO ◊ HICG, LLC. ◊

    Jeff —

    Your ‘unpolished’ example worked without any changes required!

    Not only did you save me the development time, but you gave me great insight into how to approach the problem of where in the flow to do the cleanup.

    Thanks much!


    Hi Jeff and Chuck,

    I took the example code and recrafted it a little to make it a little more general and published it as a channel in the Iguana-Protocols Github repository:

    This shows the usage:


    Thanks, Eliot; I appreciate the attribution too.

    And if you’re wondering who submitted the change to fix my last name, that was me 🙂

    Jeff Drumm ◊ VP and COO ◊ HICG, LLC. ◊

    We needed to be able to cleanup a variety of special characters, so I wrote a method that can do multiple replacements with one call. Rather than write a separate line to do the gsub(), just provide a comma-separated list of characters you want escaped:

             fldList[28] = Escape(fldList[28], '&,~')
    function Escape(segment, escapes)
       local escapeList = escapes:split(',')
       for i=1, #escapeList do
          local escape = escapeList[i]
          if escape == '&' then
             segment = segment:gsub('&','\\T\\')
          elseif escape == '|' then
             segment = segment:gsub('|','\\F\\')
          elseif escape == '^' then
             -- Remove the code that splits the field 
             -- into multiple elements
             segment = segment:gsub('\\^','')
          elseif escape == '~' then
             segment = segment:gsub('~','\\R\\')
          elseif escape == '\\' then
             -- Remove the escape separator
             segment = segment:gsub('\\','')
          elseif escape == '\.br' then
             -- Remove the <br>
             segment = segment:gsub('\.br',' ')      
          elseif escape == 'trim' then
             -- Trims white space on both sides
             segment = string.trimWS(segment)    
          elseif escape == 'compact' then
             -- Replace multiple spaces with single spaces 
             segment = string.compactWS(segment)
             local msg = 'Invalid escape: '..escape
       return segment
    You must be logged in to view attached files.


You must be logged in to reply to this topic.