• Introduction
• Task
• Implementation
• Code
• How it works
• More information

Introduction

It is very common for Web Services to be unreliable because internet connections between a client and a web server are never perfect.

• Connections can be reset without warning.
• Web servers can be temporarily overloaded with traffic.
• Many Web Services apply throttling — allowing each client a limited number of requests per second (or minute/hour/day).

In short, using we services can be frustrating — and you cannot expect every request to succeed on the first try. The good news is it’s easy to use retry logic in the Translator to make your interfaces reliable, even when web services are not.

If you need more help using the retry module please contact us at support@interfaceware.com.

Task [top]

Retry a Web Service when an error occurs.

Implementation [top]

We use the retry.lua module with a custom error handler function. The example error handler demonstrates how to work with common HTTP error codes. You will need to customize the error handler function WebServiceError() to meet your own requirements.

Follow these steps to use the code:

1. Create a new channel to try out the code:
   • Create a channel like this:
     • Name: Retry a Web Service (or similar)
     • Source: From Translator
     • Destination: To Channel
   • Alternatively use an existing channel:

     Your channel will need a From/To Translator or a Filter component.

2. Load the code into your channel:
   • If you are using a new channel then load this RetryaWebService-FromTranslator project zip file:

     You can load the project zip file into a From/To Translator or a Filter component. This will overwrite the code in your main module — you may not want to d this with an existing channel.

   • Alternatively for an existing channel you can paste the code from below into the main module:

     This allows you to paste the code without overriding the code in your main module. This is useful if you are using an existing channel and you want to modify the existing code.

     • If you have not used the retry.lua module before then you will need to create it:
       • You can import the Retry periodic failure repository channel.
       • Alternatively you can create the retry.lua module and paste the retry.lua module code from github.
3. Try out the code:
   • Change the Web Service parameters to use your chosen web service:
     
   • Alternatively you can create a channel to run our Hello Web Service example.
   • Play around with stopping and starting the web service to see how the error processing works.
   • You can also manually change the values passed to the WebServiceError() function for testing purposes:

     An easy way to do this is to modify the parameters within the WebServiceError() function itself by setting the values at line 51, before the if statement (be sure to delete this code after testing).

      

4. Modify the code:
   • Move the Web Service (CallWebService) and Error (WebServiceError) functions to a local module:

     These functions below are shown here (in the main module) for simplicity of presentation. However it is best practice to place functions in a module, so in a production system they should be placed in a local module (or perhaps a shared module).

   • Modify the error handling function to match your requirements:
     • Add extra HTTP codes that you need to handle.
     • Remove any codes that are not required.
     • Change the error messages.
     • Add code to email the administrator for errors that cannot be resolved by retries.
     • Add extra processing for more complex error scenarios:

       For example you may want to check “retry-after” field in the HTTP header to determine the delay you use (though not all web services implement this).

   • You may want to rename Web Service (CallWebService) and Error (WebServiceError) functions to match your own coding standards.

Code [top]

This is the example code for the main module:

local retry = require 'retry'

function main(Data)
--   local R, R2, M = retry.call{func=DoInsert, retry=1000, pause=10, funcname='DoInsert', errorfunc=myError}
   
   local R, R2, M = retry.call{func=CallWebService, retry=100, pause=10, funcname='CallWebService', errorfunc=WebServiceError}
end

-- the functions below are shown here (in the main module) for simplicity of presentation
-- in a real/production system they should be placed in a local module 

-- we used our "Get current date and time" example for testing
-- https://help.interfaceware.com/v6/creating-web-services#basic

-- replace this with your own URL and credentials
-- also you may wish to encrypt the password and store it in a file
-- https://help.interfaceware.com/v6/encrypt-password-in-file
URL = 'localhost:6544/current_time'
USER = 'admin'
PASSWORD = 'password'

function CallWebService()
   -- get data from web service
   local R, C, H = net.http.get{url=URL,
      auth={username=USER, password=PASSWORD}, 
      live=true}

   -- on success (2xx) parse and extract data
   -- NOTE: this allows for all (2xx) success codes
   local c = tostring(C):sub(1,1)
   c = tonumber(c)
   if c == 2 then -- C = 2xx
      local X = xml.parse{data=R}

      -- add data processing here
      -- NOTE: You could customize this based on which (2xx) 
      --       code that you receive
      --       (or you can process a code in the error function)

      -- push the result to the queue
      queue.push(X:S())
   end

   -- return the error code and headers
   -- for use in myWebServiceError()
   return R, C, H
end

function WebServiceError(Success, ErrMsgOrReturnCode, Code, Headers)
   local funcSuccess
   if Success then
      -- successfully call the web service
      funcSuccess = true -- don't retry
      -- we still need to check the Code (HTTP return code)
      if Code == 503 then
         iguana.logInfo('WebService connection: 503 - Service Unavailable: unavailable probably temporarily overloaded')
         -- wait longer before retrying (in this case an extra 30 seconds)
         -- this is an example and can be customized to your requirements
         -- NOTE: Some WebServices may include a retry-after field in the Headers
         --       if so you can use this to customize the delay (sleep) period
         if not iguana.isTest() then
            util.sleep(30000)
         end
         funcSuccess = false -- retry
      elseif Code == 404 then
         iguana.logInfo('WebService connection: 404 - Not Found: the resource was not found but may be available in the future')
         -- wait longer before retrying (in this case an extra 120 seconds)
         -- this is an example and can be customized to your requirements
         -- NOTE: Some WebServices may include a retry-after field in the Headers
         --       if so you can use this to customize the delay (sleep) period
         if not iguana.isTest() then
            util.sleep(120000)
         end
      elseif Code == 408 then
         iguana.logInfo('WebService connection: 408 - Request Timeout: the server timed out probably temporarily overloaded')
         -- wait longer before retrying (in this case an extra 30 seconds)
         -- this is an example and can be customized to your requirements
         -- NOTE: Some WebServices may include a retry-after field in the Headers
         --       if so you can use this to customize the delay (sleep) period
         if not iguana.isTest() then
            util.sleep(30000)
         end
      elseif Code == 429 then
         iguana.logInfo('WebService connection: 429 - Too Many Requests: too many requests or concurrent connections')         
         -- wait longer before retrying (in this case an extra 60 seconds)
         -- this is an example and can be customized to your requirements
         -- NOTE: Some WebServices may include a retry-after field in the Headers
         --       if so you can use this to customize the delay (sleep) period
         if not iguana.isTest() then
            util.sleep(60000)
         end
         funcSuccess = false -- retry
      elseif Code == 401 then
         iguana.logError('WebService connection: 401 - Unauthorized: usually indicates Login failure')
         -- you could/should also email the administrator here
         -- https://help.interfaceware.com/code/details/send-an-email-from-lua?v=6.0.0
         
         -- in this case we don't want to retry 
         -- so we don't set funcSuccess = false
      elseif Code == 403 then
         iguana.logError('WebService connection: 403 - Forbidden: you do not have permission to access this resource')
         -- you could/should also email the administrator here
         -- https://help.interfaceware.com/code/details/send-an-email-from-lua?v=6.0.0
         
         -- in this case we don't want to retry 
         -- so we don't set funcSuccess = false
      elseif Code == 413 then
         iguana.logError('WebService connection: 413 - Payload Too Large: the requested result is too large to process')
         -- you could/should also email the administrator here
         -- https://help.interfaceware.com/code/details/send-an-email-from-lua?v=6.0.0
         
         -- in this case we don't want to retry 
         -- so we don't set funcSuccess = false
      end
   else
      -- anything else should be a genuine error that we don't want to retry
      -- you can customize the logic here to retry specific errors if needed
      iguana.logError('WebService connection: ERROR '..tostring(Code)..' - '..tostring(ErrMsgOrReturnCode))
      funcSuccess = true -- we don't want to retry
   end
   return funcSuccess
end

How it works [top]

We use the retry.lua module with a custom error handler function WebServiceError() that handles common HTTP error codes. The code forms a good foundation as it deals with the most common web service retry scenarios, and also with some errors that cannot be resolved by retries. You will need to customize the error handler function WebServiceError() to meet your own business requirements.

Possible WebServiceError() error handler function Customizations:

• Remove error codes that you do not need
• Add other error codes that you require
• Change the wording of the logged errors (though we tried to make them helpful)
• Add code to email the administrator for errors that cannot be resolved by retries
• Add more complex business logic for error handling

Error conditions handled:

Errors that can be resolved by retries:

1. The Web Service is temporarily unavailable because the server:
   1. Too busy indicated by  timeout errors:
      • 503 – Service Unavailable: The web server is temporarily unavailable probably because it is overloaded
      • 408 – Request Timeout: The server timed out probably because it is temporarily overloaded
   2. The web service is down:
      • 404 – Not Found: The resource was not found but may be available in the future
2. Connections or data requests are limited by throttling:
   1. Connections are refused if you exceeded your quota of requests or concurrent connections
      • 429 – Too Many Requests: Indicates too many requests or concurrent connections

Errors that cannot be resolved by retries:

1. Connections or data requests are limited by throttling:
   1. If the requested result is too large the web server will refuse to process the request:
      • 413 – Payload Too Large: The requested result is too large to process
2. Authorization and access issues:
   1. Logon fails because of invalid logon credentials:
      • 401 – Unauthorized: Usually indicates logon failure
   2. Access is denied to a resource because the user does not have sufficient permissions:
      • 403 – Forbidden: The user you connected as does not have permission to access the requested resource

More information [top]

• Source code for the retry.lua module on github
• Iguana Tools repository: Retry periodic failure
• Retrying unreliable external resources
• Using the retry.lua module
• Retry a database connection
• Wikipedia: List of HTTP status codes