• Introduction
• Task
• Implementation
• Only about pcall()
• How it works
• More information

Introduction

Using pcall() is all about error handling — so you should really give a little thought to error handling first before you use pcall().

You should only use pcall() for non-critical errors.

If you just want to look at the pcall() by itself then go to Only about pcall().

There are various types of errors, lets break them down into three categories:

1. Non-critical errors: These can be handled by skipping (and logging) the error.

   These are any errors where you can continue processing without any risk corrupting data. This might be attempting to update demographic information for a person that does not exist on your system (yet) — in this case it is safe to log the issue and proceed to the next update. Or this could occur when trying to import a disk file that does not exist (yet) — in which case you can try the import later.

2. Temporary errors: These should be handled by retrying using our retry module.

   These are things like timeouts, database disconnections, web disconnections, network issues and other self-correcting problems — which can often be resolved by retrying after a delay.

3. Fatal errors: These errors should stop the Channel so they can be investigated.

   These are serious errors that could cause damage to data or loss of data. Any new error should be treated as fatal until proven otherwise.

See Thinking through interface error handling for more information.

Task [top]

Using pcall() to handle non-critical errors, we will demonstrate with two scenarios:

1. Scenario one: Parsing HL7 messages:
   1. Issue: Sometimes you get unidentified message types, and you would like to identify and log what they are.

      At present you are using a “catchall” message in the VMD, this is convenient as it allows processing to continue for unknown messages (as they will match “catchall”). However it is inconvenient as the Warning returned from hl7.parse do not identify unknown messages.

   2. Solution: Remove the “catchall” message from the VMD and use pcall() to trap the error which contains the unknown message type.

      When you do not have a “catchall” message in your VMD them hl7.parse{} will throw an exception/error when it gets an unknown message type. The error contains the type of the unknown message,  which you can conveniently trap and process with pcall() — we will log the message (and you could take other action like emailing an administrator).

      Tip: There are other ways than pcall() to handle unknown messages, for example you could check the content of the MSH.9 field — and take action (logging etc.) if it was an unexpected message type.

      However the advantage of using pcall() is that you can also detect (and log) other unexpected error conditions as well — which can be very useful.

2. Scenario two: Inserting patient demographic information into a database:
   1. Issue: Sometimes the person already exists in the database.

      We are receiving new patients to insert into our DB  from an external system. Our system already has a subset of the people from the external system. Unfortunately there is no way for the external system to identify the people on our system — so it sends inserts for everyone.

   2. Solution: Use pcall() to ignore the error caused by attempting to add an existing patient.

      We will use pcall() to trap the exception, log the error message and continue processing for the next patient.

Implementation [top]

We designed the solutions so you can load the code into an Iguana channel and it should “just work”.

Scenario one: Parsing HL7 messages when some are of unknown type.

1. Follow the instructions in this FAQ Send HL7 sample data using LLP Client to create an HL7 LLP simulator channel.
2. Create a channel named Test HL7 parse (or similar), using a LLP Listener source and a To Translator destination.
3. Import the TestHL7parsing-ToTranslator.zip project file into the To Translator.
4. Go to the last sample message — you should get an error like this:
   
5. Save a commit.
6. Run the channel:
   1. Run the channel from the dashboard.
   2. Start and stop the HL7 LLP simulator channel (that you created in step 1).

      We suggest you start and then immediately stop the HL7 LLP simulator channel so you only process the sample data file once. If you leave the channel running it will process the sample file multiple times, which is fine, but you can send a lot of repeated messages.

   3. The channel should run successfully, but with (48) errors.

      Tip:  If your Test HL7 parse channel is not receiving HL7 messages you may need to set it’s LLP Listener port to match the HL7 LLP simulator channel’s LLP Client port.

      

   4. Click on the error link to view the errors in the logs.
      
   5. You should see errors like these:
      

Scenario two: Inserting patient demographic information into a database.

1. Follow the instructions in this FAQ Send HL7 sample data using LLP Client to create an HL7 LLP simulator channel.
   1. Save a commit “original code”.

      We will revert to this commit later.

   2. Download the db_data.txt file.

      This file contains half of the messages from our sample_data.txt. This means that later when we try to insert all of the sample_data.txt messages half of the inserts will fail.

   3. Replace the code in the From Translator so it uses the db_data.txt file.

      -- NOTE: using the "/" forward slash in the path works Windows as well 
      SAMPLE_DATA = iguana.workingDir()..'Demo/sample_data.txt'
      
      -- NOTE: substitute with your own path to where you downloaded the file
      DB_DATA = '/Users/julianmuir/Downloads/db_data.txt'
      
      function main()
         -- open the sample data file
         --local f = io.open(SAMPLE_DATA)
         
         -- use db_data.txt file instead
         local f = io.open(DB_DATA)
         
         -- read & queue each message
         while true do
            -- uncomment to pause between messages
            --util.sleep(1000) -- 1000 = 1 second
            local r = f:read()
            if r == nil then 
               break 
            else
               queue.push{data=r}
            end
         end
      end

   4. Change the DB_DATA path to point to where you saved the db_data.txt file.
   5. Create a commit “now using DB data”.
2. Create the demo.sqlite database:
   1. Create a channel named Create SQLite DB (or similar), using a LLP Listener source and a To Translator destination.
   2. Import the CreateSQLiteDB-ToTranslator.zip project file into the To Translator.
   3. Save a commit.
   4. Run the code in the editor:
      1. Click through the sample messages to check it is working.
      2. You will only see one record in the table:
         
   5. Run the channel:
      1. Run the Create SQLite DB channel from the dashboard.
      2. Start and stop the HL7 LLP simulator channel (that you created in step 1).

         We suggest you start and then immediately stop the HL7 LLP simulator channel so you only process the sample data file once. If you leave the channel running it will process the sample file multiple times, which repeats the same messages — causing a primary key violation which will stop the Create SQLite DB channel. If this occurs don’t worry the DB is fine — so you can simply continue to the next step.

         Tip: If your Create SQLite DB channel is not receiving HL7 messages you may need to set it’s LLP Listener port to match the HL7 LLP simulator channel’s LLP Client port.

         Warning: Do not run the Create SQLite DB channel in the Iguana Editor at this point — it will delete all the patient table records. We will check later to confirm the records exist.

         If you do accidentally delete the records — simply run the Create SQLite DB channel again to recreate them.

   6. Optional: Delete the Create SQLite DB channel.
3. Create a channel named Test DB insert (or similar), using a From Channel source and a To Translator destination.
4. Import the TestDBinsert-ToTranslator.zip project file into the To Translator:
5. Check that patient data was imported:
   1. The simplest way is to create a Test channel, using this code:

      conn = db.connect{
         api=db.SQLITE,
         name='demo.sqlite',
         live=true
      }
      
      function main(Data)
         local Success
         
         -- check that we created patient records correctly
         conn:query('SELECT COUNT(*) FROM patient')
         conn:query('SELECT * FROM patient')
      end

   2. There should be 100 records in the patient table:
      

      Tip: You can also download SLiteStudio tool and run the same queries from there — it is a nice tool.

6. Click through a few sample messages — you should get errors like this:
   
7. Save a commit.
8. Update the HL7 LLP simulator channel so it reverts to using the sample_data.txt file:
   1. Revert to the “original code” commit you created in step 1.1.

      Alternatively if you forgot to create the commit you can always copy/paste the original code from the Send HL7 sample data using LLP Client FAQ — or just edit the code to use sample_data.txt.

   2. This is what the original code looks like:
      
9. Run the channel:
   1. Run the Test DB insert channel from the dashboard.
   2. Start and stop the HL7 LLP simulator channel.

      We suggest you start and then immediately stop the HL7 LLP simulator channel so you only process the sample data file once. If you leave the channel running it will process the sample file multiple times, which repeats the same messages — which will simply cause more errors.

   3. The channel should run successfully, but with (up to 133) errors.

      If you allow the HL7 LLP simulator channel to keep running you will get more errors — which is fine as they will just get caught by the code. The exact number of errors will depend on how many messages you clicked through in step 6 above — the total number of errors is 133 (including the errors in step 6).

   4. Click on the error link to view the (primary key conflict) errors in the logs:
      
   5. You should see errors like these:
      

Only about pcall() [top]

This section focuses on pcall() and how the the pcall() code  for each scenario works.

How pcall() works:

• pcall() calls a function in protected mode — which traps any exception that occurs within the function
• Parameters for pcall():
  • First parameter: This is the function to call
  • Following parameters: These are the parameters to the function
    • The function parameters can be variables or tables
• Returns from pcall():
  • First return: Is a status — true if the function succeeded, false if there was an exception
  • Following returns:
    • If the function succeeds (status = true): The returns (one or more) from the function follow
    • If the function fails (status = false): The second return is the error message from the exception

      There can be other returns corresponding to the function returns — but in this case they will all be nil and should be ignored.

Scenario one: Parsing HL7 messages when some are of unknown type.

• This is the pcall() code used:

   Success, Msg, Name, Warnings = pcall(hl7.parse,{vmd='no_catchall.vmd', data=Data})

• This is how it works:
  • We use a VMD (no_catchall.vmd) that only recognizes HL7 ADT messages — for all other messages hl7.parse will raise an exception

    The no_catchall.vmd is simply the standard demo.vmd included with Iguana — but with the catchall segment removed. Removing the catchall segment leaves the VMD with only an ADT segment defined, which means it will only recognize ADT messages — all other (non ADT) messages will cause hl7.parse to raise an exception (which is caught by pcall).

  • pcall() is used to call hl7.parse in protected mode — to catch any exceptions
  • pcall() parameters:
    • First parameter: This is the function to call, hl7.parse
    • Second parameter: There is a single table parameter containing two elements

      The hl7.parse function has two parameters that are passed to it as a table — therefore the second parameter for pcall() is the exact same table.

  • Returns:
    • First return: Is the status — true if the function succeeded, false if there was an exception
    • Following returns:
      • If the function succeeds (status = true), then:
        • Success = true
        • Msg = the parsed HL7 message
        • Name = the message type
        • Warnings = any warning returned from hl7.parse
      • If the function fails (status = false), then
        • Success = false
        • Msg = the error returned by the exception
          Note: In this case the error (from hl7.parse) is a string
        • Name and Warning = nil (and should be ignored)

Scenario two: Inserting patient demographic information into a database.

• This is the pcall() code used:

  Success, Error = pcall(conn.execute, conn, {sql = sql, live = true})
  
  -- NOTE: using the ":" (colon) sytax does NOT work with pcall() 
  --       you need to pass the conn object as the first parameter
  -- Which means that the code below is incorrect and will fail
  Success, Error = pcall(conn:execute, {sql = sql, live = true})

• This is how it works:
  • pcall() is used to call conn.execute in protected mode — to catch any exceptions
  • pcall() parameters:
    • First parameter: This is the function to call, conn.execute
    • Second parameter: The conn connection object
    • Third parameter: There is a single table parameter containing two elements

      The conn.execute function has two parameters that are passed to it as a table — therefore the second parameter for pcall() is the exact same table.

  • Returns:
    • First return: Is the status — true if the function succeeded, false if there was an exception
    • Following returns:
      • If the function succeeds (status = true), then:
        • Success = true
        • Error = nil (and should be ignored)
      • If the function fails (status = false), then
        • Success = false
        • Error = the error returned by the exception
          Note: In this case the error (from conn.execute) is a table

 

 

How it works [top]

Scenario one: Parsing HL7 messages when some are of unknown type.

This example is based on the article Using the Warnings returned from hl7.parse{}. But we extended the requirements to include identifying the type of HL7 messages that were not being handled.

1. Identify the type of HL7 messages that are not being handled:
   1. We modified the demo.vmd file by removing the “catchall” segment, and named it no_catchall.vmd.

      Removing the catchall segment causes hl7.parse{} to throw an exception when it tries to process an HL7 message that does not match no_catchall.vmd –the error that is generated includes the type of message.

   2. We added pcall() to trap the error.

      Using pcall() allows us to trap the error caused by an unhandled HL7 message type — as well as any other errors that may occur.

      Note: Using pcall() is not the only way to identify unknown messages, you could check the content of the MSH.9 field to see if it was an unexpected message type.

      However the advantage of using pcall() is that you can also detect (and log) other unexpected error conditions as well — which can be useful.

   3. We then log the error to the Iguana logs.

      To keep things simple we just used the iguana.logError() function to add the error to the iguana logs. However we could also have done other processing, like emailing an administrator etc.

   4. Finally we used iguana.isTest() to display the error when using the code interactively in the editor.

      Seeing error dialogs while you are editing the code is useful because it makes them more obvious. To achieve this we used iguana.isTest() to prevent using pcall() (which hides the error) while we are editing the code.

Scenario two: Inserting new patient demographic information in a database.

This scenario is a bit more complex so we will break it down into two parts, conceptual and technical.

1. Conceptual Problem:
   1. We have a database with demographic information for some patients.
   2. We need to import demographic information for new patients.
2. Technical Solution:
   1. First we create HL7 LLP simulator channel to read and send HL7 messages.

      The HL7 LLP simulator channel reads in the db_data.txt file, splits it into HL7 messages, and sends them out using an HL7 Client destination component.

   2. Next we create the Create SQLite DB channel, which will create and load the database:

      We will use the HL7 messages output from the HL7 LLP simulator channel as input to the Create SQLite DB channel. The Create SQLite DB channel will insert a row into the patient table for each of the incoming messages. The db_data.txt file contains half of the messages from the sample_data.txt message file (included with the Iguana install). Later we will try to import the full sample_data.txt message file — which will generate primary key conflicts with the existing message records (in the patient table).

      1. First it creates the patient table (in the sqlite.lua module).
      2. Then it generates an INSERT query based on the incoming HL7 message.
      3. Then it runs the query to insert the new record into the database.
   3. Then we actually create and load the demo.sqlite DB:
      1. Run the Create SQLite DB channel and then run the HL7 LLP simulator channel

         The Create SQLite DB channel creates the patient table, and inserts rows in the table from the HL7 messages generated by the HL7 LLP simulator channel.

   4. Modify the HL7 LLP simulator channel to read the sample_data.txt message file.

      We are using the sample_data.txt message file as the file to update the database with new patients. As there are duplicates they will generate errors which will be caught using pcall() in the Test DB insert channel.

   5. Next we create the Test DB insert channel, which insert the new patients into the database:

      We will use the HL7 messages output from the updated HL7 LLP simulator channel as input to the Test DB insert channel. The Test DB insert channel will try to insert a row into the patient table for each of the incoming messages. Because the sample_data.txt message file contains all the patient data it will generate primary key conflicts with the existing message records (in the patient table) — these DB errors will be trapped using pcal() and then looged to the Iguana logs.

      1. First it generates an INSERT query based on the incoming HL7 message.
      2. Then it runs the query to insert the new record into the database.
      3. Then if an error occurs it is trapped with pcall() and logged.

         Note: Using pcall() is not the only way to trap duplicate messages, you could query the patient table each time to see if a corresponding row exists. If fact you could read all the Ssn values into a table and check against that in memory and only do updates for new Ssn values (this could be more efficient solution).

         However the advantage of using pcall() is that you can also detect (and log) other unexpected error conditions as well — which can be useful.

   6. Next we will actually insert new patients into the database using the Test DB insert channel:
      1.  Run the Test DB insert channel and then run the HL7 LLP simulator channel.

         The Test DB insert channel inserts rows in the table from the HL7 messages generated by the HL7 LLP simulator channel. Because about half of the rows already exist it will generate a lot of errors — which are caught and logged using pcall().

   7. We can view the errors generated in Iguana logs.

      The Test DB insert channel traps duplicate primary key errors and logs them in the Iguana logs — where we can view them using the standard Iguana log viewer. We could also log errors in a file or send email notifications etc.

More information [top]

• Trap errors with pcall()
• API help for pcall()
• Eliot’s tips and tricks → working with errors
• Thinking through interface error handling
• Using the retry.lua module
• Retry periodic failure
• Retrying unreliable external resources