Introduction
HL7 Continuation Message: In HL7 it’s possible that you may have to deal with a legacy counter party system that has odd limits of the size of the messages they send. They might for instance be implemented using a mainframe system that can only cope with strings that 64k or less.
This page shows you how to use Iguana to stitch these parts together to make a single message.
Note: This solution assumes that messages are broken into two parts, and will need to be adapted for messages with more parts.
Before You Start
Part of the HL7 standard covers the ability to break larger messages containing say lab data, into smaller parts which then have to concatenated together by the receiving system.
How It Works
Here’s an example of the first part of the message where the presence of a DSC segment indicates that more data is to follow:
MSH|^~\&|SOFTMED^DIS|MGH|EGATE||20101110061841||MDM^T04|101110061841784|P|2.3.1|| EVN| PID||1234567^^^MGH|||SMITH^THEODORE^E^^^^L^A||19350212|M|||||||||||| PV1|||W12^W1236^B||||018840|075325|||||||||018840|||||||||||||||||||||||||||20101025|20101103| TXA|1|DIS||201011101818||201011030000|201011031346||018840||DEX|1307506|||||LA|||||^201011101818| OBX|1|FT|&ZCR||DISCHARGE SUMMARY.br.brNAME: SMITH THEODORE E UNIT NUMBER: 123-45-67.br FLOOR: W12 W1236B.brADMISSION DATE: 10/25/2010 DISCHARGE DATE: 11/03/2010.br.br.br.br.brPRINCIPAL DIAGNOSIS.brAVM.br.brASSOCIATED DIAGNOSES.brAltered mental status, Abnormal gait, Seizure disorder, Respiratory.brinsufficiency, Aphasia, Edema, Headache.br.brOPERATIONS AND PROCEDURES.br10/25/10: PROCEDURE: Left temporal craniotomy with microsurgical.brresection of arteriovenous malformation using BrainLab stereotactic.brguidance. Post-operative cerebral angiogram..br.br.brALLERGIES.brHCTZ (Unknown).brHeparin (Severe bleeding).brAspirin (severe bleeding).brALFUZOSIN (Unknown).brAMINOCAPROIC ACID (rhabdomyolysis).brALTEPLASE (severe bleeding).br.br.brHISTORY AND REASON FOR HOSPITALIZATION AND SIGNIFICANT FINDINGS.brThe patient is a 76yoM in for elective craniotomy for clipping of.brarteriovenous malformation with Dr. Ogilvy..br.brHPI: The patient has a complicated medical history including a h/o.brhereditary hemorrhagic telangiectasia (HHT) c/b cerebral, GI and.brpulmonary AVMs, chronic GIB, and progressive dyspnea. He is s/p.brBillroth II, duodenal resection for bleeding AVMs 6/19/2009 and s/p.brcoil embolization of left occipital artery to external jugular vein.brshunt vessel on 9/14/2010. He is transfusion dependent with chronic.brmelena/BRBPR and s/p cerebral bleed with resultant seizure. Pt. now.brpresents for surgical intervention for a temporal arteriovenous.brmalformation..br.brExam on admission:.brBP: 163/78 mm Hg.brAP: 95 bpm.brO2 Sat: 99%.brTemp: 97.8.brResp: 18.brHt: 66 inches.brWt: 153 lbs.brGeneral: Patients is a well appearing, well nourished male in no.brapparent distress..brSkin: Warm and dry to touch. No sores, lesions, rashes, bruising or.brpetecchiae noted..brHEENT: Normocephalic, face symmetricalADD| DSC|MGH-SM-1307506-20101110061841-1
This is the continuation part which indicates that it is a follow on message by having the continuation ID in it’s MSH message header segment:
MSH|^~\&|SOFTMED^DIS|MGH|EGATE||20101110061841||MDM^T04|101110061841784|P|2.3.1||MGH-SM-1307506-20101110061841-1 ADD|until cleared by this MD at follow-up.\.br\\.br\\.br\||||||S|||201011031346| PR1|||||| ZTR|||||SOFTMED|MGH||||DE^DEX Discharge Summary^MGH-RPTL-REPTYPE|This article has not been written.
The example in the following three pages explains how to leverage the Translator to solve this problem.
The code for this example is effectively “plug and play” as it will automatically create its own SQLite database and tables if they do not exist.
Algorithm [top]
The algorithm is easy. We create a database table “continuation” which is used to store the messages that need continuation data.
This will have a simple structure:
| Column Name | Description |
|---|---|
| ContinuationId (key) | Unique code identifying the continuation number. Needs to be large enough to store the keys, say 100 characters |
| Message | The raw HL7 message. Should be about 100k in size depending on the largest message size. |
When we get a message that needs us to wait for a continuation part we insert it into this table using the Continuation Pointer value as the key value for the ContinuationId.
Then when we get the remainder message we can recall that data from the table and parse it and combine the data together with the original message. Fortunately Iguana has no trouble coping with messages larger than 64k in length!
For simplicity we only show how to process the continuation message, the concatenated message will need further processing/saving etc (code for this is not included).
Example Code [top]
This was a fun problem to solve with the Translator, and as if often the case the solution is quite simple
This is what we need to do:
- Identify (partial) messages with continuations and save them to the database
- Identify the continuation messages and retrieve the initial message
- Stitch the messages together to create a complete HL7 message
Note: This example assumes that messages are broken into two parts, and will need to be adapted to handle more parts.
How It Works
The first step is to identify (partial) messages with continuations and save them to the database.
First we create the SQLite database and tables if they do not exist. Then we test for the existence of the DSC continuation segment and use the cont module to push the message into the database using the continuation ID as the primary key.
Save a continued message to the Database
Here is a screen shot:
Note: to ensure that the code works on any system the DB and VMD files do not specify a path and therefore default to using the Iguana install directory. You can customize this for your system by changing SQLITE_DB and VMD_FILE variables to something like
SQLITE_DB='D:tempcontinuation.sqlite' VMD_FILE='D:tempContinuation.vmd'
The PushContinuation() function uses cont.put() to save the message to the database:
The cont.put() function inserts the message associated with that Continuation ID into the database, or replaces it if it already exists. If you are interested in in how this works you can look at the code for the module (it uses the SQL REPLACE keyword).
Retrieve a continued message from the Database
Now when we get the continuation message we simply retrieve the first message and concatenate the data together.
When we get a message with a continuation pointer in the MSH segment we do this:
- Fetch the original HL7 message from the database using the continuation pointer
- Extract the first part of the lab text from the OBX segment
- Strip off the ADD part of the lab text
- Concatenate it together with a space to the end part of the lab data
- Translate .br escape sequences to n
This formats the message in a nice human-readable format. Obviously, you can process the in other ways, whatever is suited to your needs.
Here is a screenshot:
By clicking on the return annotation of the MergeText() routine we can see the full lab result in human-readable format:
And as you can see the continuation string “until cleared by this MD at follow-up.” has been appended.
What’s Next?
We have shown you how the code to concatenate a two part message works, the next step is to add your own code to process the concatenated message. The sample code is available on the next page.
If you need to support messages with more than two parts you will need to modify the code. One way to do this would be to change the FetchContinuation() function to recursively retrieve all the continuations for a message.
Please contact support at support@interfaceware.com if you need more help.
Source Code [top]
The code is specific to SQLite and will need tweaking to work with other databases.
This is the SQL used to create the Continuation table for SQLite. We could have generated the table from the VMD file, but as it will never change it is simpler to hard code it.
Because SQLite uses a “dynamic type system” the the Message field will happily store a string of any length. In fact in SQLite the datatype is only a recommendation and if we had declared Message as an integer we could still store string data in it. For more information see Datatypes In SQLite Version 3.
Other databases use strict static typing, something like this should work with most other database servers:
CREATE TABLE Continuation ( ContinuationId VARCHAR(100) NOT NULL, Message VARCHAR(100000), PRIMARY KEY (ContinuationId) );
Sample Data
Here’s an example of the first part of the message where the presence of a DSC segment indicates that more data is to follow:
MSH|^~&|SOFTMED^DIS|MGH|EGATE||20101110061841||MDM^T04|101110061841784|P|2.3.1|| EVN| PID||1234567^^^MGH|||SMITH^THEODORE^E^^^^L^A||19350212|M|||||||||||| PV1|||W12^W1236^B||||018840|075325|||||||||018840|||||||||||||||||||||||||||20101025|20101103| TXA|1|DIS||201011101818||201011030000|201011031346||018840||DEX|1307506|||||LA|||||^201011101818| OBX|1|FT|&ZCR||DISCHARGE SUMMARY.br.brNAME: SMITH THEODORE E UNIT NUMBER: 123-45-67.br FLOOR: W12 W1236B.brADMISSION DATE: 10/25/2010 DISCHARGE DATE: 11/03/2010.br.br.br.br.brPRINCIPAL DIAGNOSIS.brAVM.br.brASSOCIATED DIAGNOSES.brAltered mental status, Abnormal gait, Seizure disorder, Respiratory.brinsufficiency, Aphasia, Edema, Headache.br.brOPERATIONS AND PROCEDURES.br10/25/10: PROCEDURE: Left temporal craniotomy with microsurgical.brresection of arteriovenous malformation using BrainLab stereotactic.brguidance. Post-operative cerebral angiogram..br.br.brALLERGIES.brHCTZ (Unknown).brHeparin (Severe bleeding).brAspirin (severe bleeding).brALFUZOSIN (Unknown).brAMINOCAPROIC ACID (rhabdomyolysis).brALTEPLASE (severe bleeding).br.br.brHISTORY AND REASON FOR HOSPITALIZATION AND SIGNIFICANT FINDINGS.brThe patient is a 76yoM in for elective craniotomy for clipping of.brarteriovenous malformation with Dr. Ogilvy..br.brHPI: The patient has a complicated medical history including a h/o.brhereditary hemorrhagic telangiectasia (HHT) c/b cerebral, GI and.brpulmonary AVMs, chronic GIB, and progressive dyspnea. He is s/p.brBillroth II, duodenal resection for bleeding AVMs 6/19/2009 and s/p.brcoil embolization of left occipital artery to external jugular vein.brshunt vessel on 9/14/2010. He is transfusion dependent with chronic.brmelena/BRBPR and s/p cerebral bleed with resultant seizure. Pt. now.brpresents for surgical intervention for a temporal arteriovenous.brmalformation..br.brExam on admission:.brBP: 163/78 mm Hg.brAP: 95 bpm.brO2 Sat: 99%.brTemp: 97.8.brResp: 18.brHt: 66 inches.brWt: 153 lbs.brGeneral: Patients is a well appearing, well nourished male in no.brapparent distress..brSkin: Warm and dry to touch. No sores, lesions, rashes, bruising or.brpetecchiae noted..brHEENT: Normocephalic, face symmetricalADD| DSC|MGH-SM-1307506-20101110061841-1
This is the continuation part which indicates that it is a follow on message by having the continuation ID in it’s MSH message header segment:
MSH|^~&|SOFTMED^DIS|MGH|EGATE||20101110061841||MDM^T04|101110061841784|P|2.3.1||MGH-SM-1307506-20101110061841-1 ADD|until cleared by this MD at follow-up..br.br.br||||||S|||201011031346| PR1|||||| ZTR|||||SOFTMED|MGH||||DE^DEX Discharge Summary^MGH-RPTL-REPTYPE|This article has not been written.
Sample Code
This is vmd file used in the example: Continuation.vmd
This is the main routine:
require 'cont'
-- change the locations of the VMD and DB files as required
-- in this case (no path specified) the path defaults to the Iguana install directory
SQLITE_DB='continuation.sqlite'
-- vmd must be in XML format
VMD_FILE='Continuation.vmd'
conn = db.connect{
api=db.SQLITE,
name=SQLITE_DB
}
function main(Data)
-- create the database file and tables if it does not exist
local r=conn:query('SELECT * FROM sqlite_master')
if #r==0 then CreateTable() end
local Msg = hl7.parse{data=Data, vmd=VMD_FILE}
if not Msg.DSC:isNull() then
PushContinuation(Msg, Data)
return
end
if Msg.MSH[14] then
trace('We have a continuation.')
local T = FetchContinuation(Msg)
end
----------------------------------------------------
-- add code here to process the concatenated message
----------------------------------------------------
end
function PushContinuation(Msg,Data)
local Id = Msg.DSC[1]:nodeValue()
trace("Encountered continuation segment, ID = "..Id)
cont.put(Id, Data)
end
function FetchContinuation(Msg)
local OrigMsg = cont.get(Msg.MSH[14]:nodeValue())
if not OrigMsg then
error("Could not find original message for "..Msg.MSH[14]:nodeValue())
end
local Orig = hl7.parse{vmd=VMD_FILE, data=OrigMsg}
local Text = MergeText(Orig, Msg)
return Text
end
function MergeText(Msg1, Msg2)
local T = Msg1.OBX[5][1][1]:nodeValue()
T = T:sub(1, #T-3)
trace(Msg2.ADD[1]:nodeValue())
T = T.." "..Msg2.ADD[1]
T = T:gsub(".br", "\n")
return T
end
-- create continuation table - the SQL used is ***SQLite*specific***
function CreateTable()
local Sql=
'CREATE TABLE IF NOT EXISTS Continuation\n'..
'(\n'..
'ContinuationId TEXT(255) NOT NULL PRIMARY KEY,\n'..
'Message TEXT(255) NULL\n'..
');'
trace(Sql)
conn:execute{sql=Sql,live=true}
end
This is the cont module:
cont = {}
function cont.get(Id, DbName)
local Sql =
'SELECT * FROM Continuation WHERE ContinuationId = '..conn:quote(Id)
local R = conn:query(Sql)
if (#R == 0) then return nil end
return R[1].Message:nodeValue()
end
function cont.put(Id, Data)
trace(Esc,conn)
local Sql =
"REPLACE INTO Continuation(ContinuationId, Message)"..
" VALUES("..conn:quote(Id)..","..conn:quote(Data)..")"
trace(Sql)
conn:execute{sql=Sql,live=true}
end