This topic contains 7 replies, has 2 voices, and was last updated by  Casey Trauer 5 months, 1 week ago.

Batch process including headers/footers

  • Hi folks!
    I have a project in Mirth that I need to convert over to Iguana. We take a flat file of HL7 messages with a header and 1+ footers, save the information to SQL, and then rebuild the file with updated information. The header and footer(s) need to be available for the rebuild.
    I see the built in batch process sample, but it says excluding the headers and footers.
    Anyone have an example of a file reader type that can extract headers, footers, and messages?

    Thanks!
    Steve
    Iguana Newbie

    Steve Ela
    Macro Helix / McKesson
    Software Developer / Integration Lead / Scrum Master

    Hi Steve,

    I can give a shot at helping you out. Are the headers and footers static or do they differ for each batch? Also, are the outbound messages the same message types and structures as the inbound?

    (You are correct that the batch process sample assumes that we are only consuming the batch data, discarding the envelope and processing each message individually).

    Regards,

    Casey Trauer,
    Director, Client Education
    iNTERFACEWARE

    Per channel, the headers should be the same format (different data values).
    The Footers can be 1 to many, depending on how many sites are included in the one file.
    The file itself contains DFT messages.

    The outbound file needs to be exactly like the inbound, with specific data values updated after our process, so the client can consume the updated information after retrieval.

    Hope that answers your questions.

    Thanks!

    Steve Ela
    Macro Helix / McKesson
    Software Developer / Integration Lead / Scrum Master

    Hi Steve,

    I know we discussed this over a phone call, but I’ll just share the gist of the answer here.

    Once you have determined how you are going to bring the files in (From File or From Translator), you can extract headers and footers separately from DFT messages by creating VMD files that model the structure of just those headers. The hl7.parse function then will discard any segments not defined in that VMD. So separate VMD files for headers and DFT messages may be the way to go.

    Casey Trauer,
    Director, Client Education
    iNTERFACEWARE

    I set up a VMD with just the BHS and BTS segments.
    When I try to load in a file with the headers, it says it expects MSH first, and will not get past that.

    Am I missing something?

    Steve Ela
    Macro Helix / McKesson
    Software Developer / Integration Lead / Scrum Master

    Hi Steve,

    First of all, I believe I led you astray a bit on the previous answer. As you have seen, the hl7.parse function expects a MSH segment at the top. So we cannot have a VMD to model headers and footers. I am going to recommend two ways to extract and store those segments.

    1. There is a module for extracting and storing Z segments in a table. (https://help.interfaceware.com/v6/hl7-custom-zsegment). You can pretty easily adapt this module to extract BHS/BTS/etc segments. For example, If you download this module, you will see the following test:

    if Segments[i]:sub(1,1) == 'Z'

    you would likely change that test to something like:

    if Segments[i]:sub(1,3) == 'BHS'

    (and then use elseif statements to capture the other segments.)

    Then you can use the regular batch split module to extract the messages themselves.

    2. You could also adapt the batch split module to capture headers/footers and messages. Here is the following code I used to adapt it.

    -- Public API of module --
    local function split(Data)
    -- takes HL7 Batch data, including Headers
    -- returns List of Headers, and nested List of Batches of HL7 messages
    local Headers = {}
    local Messages = {}
    local StartIndex = -1

    local Segments = Data:split('\r')

    if Segments[#Data] == '' then table.remove(Data,#Data) end

    -- Extract headers and footers
    for i, Segment in ipairs(Segments) do
    local SegmentId = Segment:sub(1,3)
    if isFileHeader(SegmentId) then
    Headers[i] = Segment
    elseif isBatchHeader(SegmentId) then
    Headers[i] = Segment
    end
    end

    -- Extract messages
    for i, Segment in ipairs(Segments) do
    local SegmentId = Segment:sub(1,3)
    if isFileHeader(SegmentId) then
    ExtractMessage(Segments, StartIndex, i, Messages)
    StartIndex = -1
    elseif isBatchHeader(SegmentId) then
    ExtractMessage(Segments, StartIndex, i, Messages)
    StartIndex = -1
    elseif SegmentId == "MSH" then
    ExtractMessage(Segments, StartIndex, i, Messages)
    StartIndex = i
    end
    end

    -- Get last trailing message if it exists.
    ExtractMessage(Segments, StartIndex, #Segments, Messages)
    return Messages, Headers
    end

    Note that the split function now returns Messages and Headers, so you would call the function something like:

    local Msgs, Headers = split(Data)

    The code for this second way just captures the header/footer segment as a string and puts it in a table. It doesn’t parse the segment like the first approach will.

    I hope this helps.

    Casey Trauer,
    Director, Client Education
    iNTERFACEWARE

    So far, this seems to work. Will need to test against a really big file to see how long it takes, since it is scanning the whole thing for text strings.

    Thanks!

    Steve Ela
    Macro Helix / McKesson
    Software Developer / Integration Lead / Scrum Master

    If the headers/footers are in the first X segments of the file and the last X segments of the file, the code could be adapted to not scan all segments. That might save a bit of processing time. Let me know how it turns out.

    Casey Trauer,
    Director, Client Education
    iNTERFACEWARE

You must be logged in to reply to this topic.