Batch process including headers/footers

Hi folks!
I have a project in Mirth that I need to convert over to Iguana. We take a flat file of HL7 messages with a header and 1+ footers, save the information to SQL, and then rebuild the file with updated information. The header and footer(s) need to be available for the rebuild.
I see the built in batch process sample, but it says excluding the headers and footers.
Anyone have an example of a file reader type that can extract headers, footers, and messages?

Thanks!
Steve
Iguana Newbie

Steve Ela
Macro Helix / McKesson
Software Developer / Integration Lead

Hi Steve,

I can give a shot at helping you out. Are the headers and footers static or do they differ for each batch? Also, are the outbound messages the same message types and structures as the inbound?

(You are correct that the batch process sample assumes that we are only consuming the batch data, discarding the envelope and processing each message individually).

Regards,

Casey Trauer,
Director, Client Education
iNTERFACEWARE

Per channel, the headers should be the same format (different data values).
The Footers can be 1 to many, depending on how many sites are included in the one file.
The file itself contains DFT messages.

The outbound file needs to be exactly like the inbound, with specific data values updated after our process, so the client can consume the updated information after retrieval.

Hope that answers your questions.

Thanks!

Steve Ela
Macro Helix / McKesson
Software Developer / Integration Lead

Hi Steve,

I know we discussed this over a phone call, but I’ll just share the gist of the answer here.

Once you have determined how you are going to bring the files in (From File or From Translator), you can extract headers and footers separately from DFT messages by creating VMD files that model the structure of just those headers. The hl7.parse function then will discard any segments not defined in that VMD. So separate VMD files for headers and DFT messages may be the way to go.

Casey Trauer,
Director, Client Education
iNTERFACEWARE

I set up a VMD with just the BHS and BTS segments.
When I try to load in a file with the headers, it says it expects MSH first, and will not get past that.

Am I missing something?

Steve Ela
Macro Helix / McKesson
Software Developer / Integration Lead

Hi Steve,

First of all, I believe I led you astray a bit on the previous answer. As you have seen, the hl7.parse function expects a MSH segment at the top. So we cannot have a VMD to model headers and footers. I am going to recommend two ways to extract and store those segments.

1. There is a module for extracting and storing Z segments in a table. (https://help.interfaceware.com/v6/hl7-custom-zsegment). You can pretty easily adapt this module to extract BHS/BTS/etc segments. For example, If you download this module, you will see the following test:

if Segments[i]:sub(1,1) == 'Z'

you would likely change that test to something like:

if Segments[i]:sub(1,3) == 'BHS'

(and then use elseif statements to capture the other segments.)

Then you can use the regular batch split module to extract the messages themselves.

2. You could also adapt the batch split module to capture headers/footers and messages. Here is the following code I used to adapt it.

-- Public API of module --
local function split(Data)
-- takes HL7 Batch data, including Headers
-- returns List of Headers, and nested List of Batches of HL7 messages
local Headers = {}
local Messages = {}
local StartIndex = -1

local Segments = Data:split('\r')

if Segments[#Data] == '' then table.remove(Data,#Data) end

-- Extract headers and footers
for i, Segment in ipairs(Segments) do
local SegmentId = Segment:sub(1,3)
if isFileHeader(SegmentId) then
Headers[i] = Segment
elseif isBatchHeader(SegmentId) then
Headers[i] = Segment
end
end

-- Extract messages
for i, Segment in ipairs(Segments) do
local SegmentId = Segment:sub(1,3)
if isFileHeader(SegmentId) then
ExtractMessage(Segments, StartIndex, i, Messages)
StartIndex = -1
elseif isBatchHeader(SegmentId) then
ExtractMessage(Segments, StartIndex, i, Messages)
StartIndex = -1
elseif SegmentId == "MSH" then
ExtractMessage(Segments, StartIndex, i, Messages)
StartIndex = i
end
end

-- Get last trailing message if it exists.
ExtractMessage(Segments, StartIndex, #Segments, Messages)
return Messages, Headers
end

Note that the split function now returns Messages and Headers, so you would call the function something like:

local Msgs, Headers = split(Data)

The code for this second way just captures the header/footer segment as a string and puts it in a table. It doesn’t parse the segment like the first approach will.

I hope this helps.

Casey Trauer,
Director, Client Education
iNTERFACEWARE

So far, this seems to work. Will need to test against a really big file to see how long it takes, since it is scanning the whole thing for text strings.

Thanks!

Steve Ela
Macro Helix / McKesson
Software Developer / Integration Lead

If the headers/footers are in the first X segments of the file and the last X segments of the file, the code could be adapted to not scan all segments. That might save a bit of processing time. Let me know how it turns out.

Casey Trauer,
Director, Client Education
iNTERFACEWARE