Forum Replies Created

Using negative indexes with string:sub()

  • Recently, we were asked if it was possible to retrieve data from a PDF file. During the discussions that followed, someone raised idea of using a third-party utility to convert PDF to XML. I decided to come up with a way to invoke this method from within the Translator itself. The following steps summarize my solution:

    1. Create a channel for the incoming message and its PDF attachment.
    2. Write the PDF to a scratch directory.
    3. Invoke the third party conversion program to convert the PDF file into XML data.
    4. Use Iguana’s xml functions to parse and manipulate the XML data as desired.

    Here is the specific procedure:

    1. Download and install an appropriate third-party conversion utility.

      For the code example below, we chose PDF to XML. (It may require LibXml2 from GNU/XMLSoft.)

    2. Create and configure a new channel with an LLP Listener source component and a To Translator destination component.
    3. Open the script and commit the first milestone.
    4. Copy and past the following code snippet into your script, or require it as an external module.

    Note: The purpose of each section in this script is identified with notes right in the code.

    local pdf = {}
     
    -- Online help
    local pdfhelp = {
           Title="pdf.convert";
           Usage="local Xml, Images = pdf.convert{file='Doctor Report.pdf',}",
           Desc=[[Under the hood this function invokes the PDFtoXML utility. Please note: if the "images" option is set,
                  it is the calling function's responsibility to clean up the generated image files.]];
           ["Returns"] = {
              {Desc="A XML node tree with the converted contents of the PDF document"},
              {Desc=[[Optionally a list of the images found in the document.  This can
                      be suppressed using the '-noImage' flag.]]}
           };
           ParameterTable= true,
           Parameters= {
               {file= {Desc='The source PDF file to convert.'}},
               {images= {Desc='This is an optional flag you can set to true to also extract images.'; Opt=true}},
           };
           Examples={
               "local Xml = pdf.convert{file='Doctor Report.pdf'}",
           };
           SeeAlso={
               {
                   Title="Tips and tricks from John Verne",
                   Link="http://wiki.interfaceware.com/1338.html"
               }
           }
       }
     
    --
    -- Helper functions
    --
     
    LUA_DIRSEP = string.sub(package.config, 1, 1)
     
    -- Validate input. Returns true if we think we can continue.
    local function validateOpts(T)
       file = T.file
       images = T.images
       
       if file == nil then
          return false, 'Missing required parameter "file"'
       end
       
       if file == '' or type(file) ~= 'string' then
          return false, 'Parameter "file" must be of type string and not empty.'
       end
       
       if images ~= nil and type(images) ~= 'boolean' then
          return false, 'Parameter "images" must be of type boolean'
       end
       
       return true
    end
     
    -- Invoke the third-party with the right options to create the
    -- XML data for the given PDF file.
    local function generateXMLoutput(pdffile, cmdOpts)
       -- Tricky regex to get all the parts of a Windows pathname.
       local path, vol, _, basename, _
          = pdffile:match("(([%a]:).-)(([^\\/]-%.?)([^%.\\/]*))$")
       
       if path == nil or vol == nil or basename == nil then
          return nil, nil, 'source file "' .. pdffile .. '" is not valid.'
       end
       
       -- Default locations.
       local xmlfile = path .. basename .. 'xml'
       local imagedir = xmlfile .. '_data'
       
       -- Run the command and get the output.
       local path, exe, _ = pdf.toolPath:match('(.-)([^\\/]-%.?([^%.\\/]*))

    Implementation Notes

    Please note that PDF to XML is not under active development, and there is no source code for the project currently available. However, there are binaries available for Windows and Linux. Currently, there is no good free cross-platform solution for creating XML or XHTML data from PDF files.

    The solution provided here is biased toward running Iguana on Windows. It would not be too difficult to modify the platform-specific functions to work on Linux.

    Let us know if you find this tip useful!

    During a visit to a client hospital that is about to go live with a large eGate migration, one of the questions that came up during our discussions was how to see what channels are using a given shared module.

    That functionality isn’t built into Iguana directly but because it’s easy to access the fossil source code repository (see Fossil repository layout) and the IguanaConfiguration.xml file it’s not a big job to write a script to generate such a report. So while I was down in Detroit that is exactly what I did.

    The algorithm is quite straightforward:

    1. Open up the repository into a temporary directory – the script assumes that the Iguana instance is running on Windows with a C:\temp\ directory.
    2. Iterate through each of the GUID subdirectories and open their project.prg files.
    3. Iterate through the shared dependencies in those project.prg files and build up a list of which shared modules use which channels.
    4. Open the IguanaConfiguration.xml file, parse it and figure out which channels use which translator instances.

    I put the report into a From HTTPS channel to make it easy to generate the report out as an HTML table so you get a header column which each shared module on the left. Then on the right you get a list of channels which use each module – with brackets indicating what component is using it – since there 5 possible translator components:

    • LLP Listeners with custom ACK translator instance. (LLP)
    • From HTTPS (H)
    • From Translator (S)
    • Filter (F)
    • To Translator (D)

    The formatting of the report is quite rough – but the report is useful so I thought I would put it up and see if other customers find it useful. For the source code go and look at the next tip in this series which is when I added search capability to the example script.

    You can see the output of this script there too.

    This is little example how Iguana channel can be utilized to create a load-balancer.

    This can balance between any number of inbound channels not in necessary belonging to same Iguana instance.
    Please note we balance not Iguana instances, but Iguana channels as our atomic operands, and these channels can belong to various Iguana instances.

    Approach is based on example ‘Monitoring Queue Size‘.

    If you aren’t interested in principal discussion about various aspects of load balancing as it is applicable with Iguana 5 Translator then skip down to sample code.
    Discussion:
    Iguana 5 Translator can load balance listening inbound channels.

    Let’s see first what balancing is: there are numerous scenarios, but one basic scenario says – engage next processor when processors in use become saturated, and take it back off when load goes down.

    In other words load balancing is a process of increasing reliability through processors redundancy.

    Outbound Iguana channels balancing cannot be covered by offered example, however it is same approach, to take advantage of, should we wish to build one.

    Let’s establish some jargon. Let’s call Iguana channel that takes initial workload and needs extra help a ‘balanced’ channel.
    Iguana channels which kick in to help a balanced channel we will call ‘balancing’ channels.
    Iguana channel orchestrating operations will be ‘dispatcher’ channel.

    Please note: we balance not Iguana instances but Iguana channels as our atomic operands; these channels can belong to various Iguana instances.

    Below example assumes that first balanced channel is on localhost, while 2nd, 3rd, to Nth channels located on any host. However, it can be adjusted to have 1st channel on any host only principal example becomes too complex in my taste.

    One may oppose location of dispatcher channel on same host where balanced channel resides, but it is only mater of your own implementation – dispatcher channel can be on any machine (think of potential availability scenario).
    Confirmation for above claim can be easily noticed from sample code; just replace localhost reference with some other IP, in function where dispatcher code reads queue levels.

    Additional directions this example can potentially be enhanced into include ’round robin’ , scheduling, errors count for specific balancing channel, disk free, or count of already assigned requests. But again, it will look too complex for proof-of-concept example.

    Few words about logs. This may look like a drawback for brief moment, our daily log records become distributed de facto.

    Daily information may get split among multiple Iguana instances, but not to forget that Global Dashboard makes listing between instances a breeze, and one-click ability to export any Iguana instance logs into CSV formatted file allows for further consolidation with any 3rd party reports analyzing software or Excel spreadsheet.

    Now it is time to mention subsequent requests routed by dispatcher to balancing channels residing with different Iguana instances, sort of analogy to ‘persistence’ required from load balancing system.

    Examples can include query/response scenarios where response served by Translator script, or scenarios of ‘don’t issue outbound message unless specific message already got processed’.

    To satisfy this sort of persistence, involved balancing channels will require to read database tables shared among involved balancing channels. This in order to share complete ‘knowledge’ about current state of enterprise.
    Example:
    This is our main module:

    require ('queuemon')
    require ('llp')
    local function trace(a,b,c,d) return end
     
    function main(Msg)
       
       local secondChannel= 'load balance 2' -- 
       local firstChannel = 'load balance 1' -- 
       local count = 3
       -- Setting the channel name and count explicitly
       local QC = queuemon.checkQueue{channel = firstChannel, count = count}  
       
       if QC then 
          send2Channel(secondChannel,Msg) -- use it to send to channel on remote machine
          push2queue(secondChannel,Msg)   -- use it to push to queue for channel on this machine
       else 
          push2queue(firstChannel,Msg)   -- use it to push to queue for channel on this machine
       end
       
    end
     
     
    function push2queue(channel,msg)
       --[[ use example from Wiki to see how we 
       serialize as JSON object; and in target channels source 
       component 'From Translator' filter messages by value 
       of first field in JSON object, respectively.
       
       if JSON objects 1st field has index/name of *this* channel 
       - then process this message, else ignore this message by 
       immediately executing 'return' in function main() of 
       target channel.
       ]]
       
       local q= toJSON(channel,msg)
       queue.push{data = q}
       iguana.logDebug(channel..' > '..msg)
    end
     
     
    function toJSON(channel,msg)
       local F = {}
       F.a1 = channel
       F.b1 = msg
       return json.serialize{data=F}
    end
     
     
    function send2Channel(channel,msg)
       --[[ use example from Wiki how to use 
       Translator as LLP Client in order 
       to send msg to IP and port of 
       channel 1 or channel 2 respectively.
       
       If you use IP=127.0.0.1 (i.e. localhost)
       then msg will be sent to another channel 
       on same (source) machine. 
       
       Please note that even if you have more 
       than one Iguana instance on a machine, 
       msg will be sent to channel respective 
       to specified port value of listening 
       channel.
       ]]
       iguana.logDebug(channel..' > '..msg)
    end

    and we will use few shared modules:

    Module iguanaconfig from ‘Monitoring Queue Size’ page, taken ‘as is’.

    Module node extension:

    -- Utilities for "node" Values
     
    -- Coerce a node value into a string.
    function node.S(ANode)
       return tostring(ANode)
    end
     
    function node:V()
       return self:nodeValue()
       end

    Module llp from ‘Using Translator as LLP Client’ page, taken ‘as is’. Please note that you will have to use example from this page to complete project to your particular specifications.

    A little modified module queuemon from ‘Monitoring Queue Size‘ page, see modified code below.

    require("node")
    require("iguanaconfig")
     
    queuemon = {}
     
    local function CheckChannel(Chan, Count, Name)
       if Chan.Name:nodeValue() == Name then
          local QC = tonumber(Chan.MessagesQueued:nodeValue())
          if QC > Count then
             iguana.logDebug('ALERT:\n Channel '..Name..' has '..QC..' messages queued.') 
             return true
          end   
       end
    end
     
    function queuemon.checkQueue(Param)
       -- We default to a queue count of 100
       if not Param then Param = {} end
       if not Param.count then Param.count = 100 end
       if not Param.channel then Param.channel= iguana.channelName() end
       local url = 'http://localhost:'..
        iguanaconfig.config().iguana_config.web_config.port..'/status.html'
       iguana.logDebug(tostring(url))
       -- We need a user login here.  Best to use a user with few
       -- permissions.
       local S = net.http.get{url=url, 
          parameters={UserName='admin',Password='password', Format='xml'}, 
          live=true}
       S = xml.parse{data=S}
     
       for i = 1, S.IguanaStatus:childCount('Channel') do
          local Chan = S.IguanaStatus:child("Channel", i)
          local QC = CheckChannel(Chan, Param.count, Param.channel)
          if QC then return QC end
       end
       return
    end

    For impatient a complete project file offered load_balancing.zip.

    Create listening channel ‘From LLP Listener to Translator’ and import suggested project into Filter script. Let’s call it balancing channel.
    Modify channel names and add/remove channels to balance as needed; along with adjusting other adjustable parameters like IP/port/user/password/etc…

    Complete functions send2Channel() and push2queue() as applicable with your requirements.
    If you balance channels on local instance, then configure balanced channels to be ‘From Channel to …’ and to read messages from queue of balancing channel.

    If you balance using LLP over TCP, then queue of balancing channel is expected to remain empty, since in function send2Channel() we don’t push message to queue, for not to be piped to Destination component.

    Last comment: this example was created using Iguana version 5.5.1. May possibly work with earlier versions as well, but it is good practice to run latest available version.

    Iguana Translator users, who require precise to millisecond timestamps, had notice that Lua time module offered with Iguana doesn’t allow for milliseconds to be reported. This has its own reasons beyond focus of this discussion.

    This example code in this article demonstrates how to use a persistent counter to simulate a count of milliseconds.

    The Lua time function os.time() only returns a time value to the nearest second. There are many occasions where milliseconds would be useful to distinguish between timed events. The code in this article solves this problem by simulating milliseconds, and concatenating them to the current time.

    Warning! If real timed milliseconds required then please see suggestion at the bottom of this page. The “millisecond values” in example on this page are not timed milliseconds, they are just a sequential count from 1-999 within a second. But this example demonstrates how one could build a restartable counter to memorize given state.

    How it works

    The code uses a counter as a surrogate for milliseconds. The counter value is appended to the time returned by the os.time()command, and returned as a string. Should processing yield more than 999 newly generated timestamps within less than one second, the code will sleep until current second rolls over to next second and the milliseconds counter resets itself.

    The magic is all done in the msec module, all that is needed to do is to call the msec.timestamp() function, as shown:

    The “milliseconds”, in this case “023”, have been appended to the time.

    Sample Code

    Code for main():

    -- Example how to simulate milliseconds in timestamp
    require 'msec'
     
    function trace(a,b,c,d) return end
     
    function main()
       trace(msec.timestamp())
    end

    Code for the msec module:

    msec ={}
     
    SQLITE_DB ='test.sqlite' 
    conn = db.connect{
       api=db.SQLITE,
       name=SQLITE_DB
    }
     
    local function zfill(s,N)
       if s:len() < 3 then
          repeat
             s = '0' .. s
          until s:len() == 3
       end 
       return s
    end
     
     
    local function Msec() 
          
       -- the only configurable parameter is name of table 't'
       local t = 'milliseconds'
       
       
       local function createTable()  
          local Sql='drop table if exists '..t
          conn:execute{sql=Sql,live=true}
          Sql = "CREATE TABLE IF NOT EXISTS "..t.." \
          ('msec' INT(255) NULL, 'sec' INT(255) NULL);"
          conn:execute{sql=Sql,live=true}
       end
       
       
       local function tableExists()
          
          local rs=conn:query('SELECT name FROM sqlite_master') 
          
          
          local function foo(i)
             if rs[i].name:nodeValue()
                == t  then 
                return true  
             end
          end
          
          for i=1,#rs do
             if foo(i) then return true end         
          end
       end
       
       
       local function currentSec()
          local rs=conn:query('SELECT * FROM '..t) 
          if rs[1].sec:nodeValue() 
             == os.date("%S",os.time()) then         
             return true
          end
       end
       
       
       local function currentMsec()
          local rs=conn:query('SELECT * FROM '..t) 
          return rs[1].msec:nodeValue() 
       end
       
       
       local function writeMsec(msec)
          conn:execute{
             sql="INSERT INTO "..t.." \
             (msec,sec) VALUES ("..msec..",\
             "..os.date("%S",os.time())..");",
             live=true
          }
       end
       
       
       local function updateMsec(msec)
          conn:execute{
             sql="UPDATE "..t.." \
             set msec="..msec..",\
             sec="..os.date("%S",os.time())..";",
             live=true
          }    
       end
       
       
       if tableExists() then
          local ms=currentMsec()
          local msnext=ms+1
          if ms=='999' then
             repeat
                util.sleep(1)
             until os.date("%S",os.time())~=currentSec()
             updateMsec(1)
             return 1
          else
             updateMsec(msnext)
             return msnext
          end
       else
          createTable()
          writeMsec(1)
          return 1
       end
       
    end  
     
     
    function msec.timestamp()
       local v = Msec()
       v=zfill(tostring(v),3)
       local t = os.date("%Y%m%d%H%M%S",os.time())
       return t..v   
    end

    What's Next?

    The above example shows how to use a counter to simulate milliseconds. Just remember these are not timed milliseconds, just a count from 1-999 within a second.

    If timed milliseconds needed, then solution is different. Please use OS independent solution, a call to os.clock(), as explained in this example.

    Recently, a client asked us if it was possible to send attachments via SMTP through Iguana. Even though the net.smtp documentation does not explicitly mention MIME or attachments, sending files as attachments is simply a process of formatting the body of the SMTP message with both MIME headers and encoded attachment contents. With this information, it is reasonably easy to create a MIME/SMTP module that wraps smtp.send{} so that it takes a table of attachments and sends them off. The following steps summarize this solution:

    1. Create a channel for the incoming message(s) that contain the attachment data.
    2. Write the attachment data to a scratch directory as separate files.
    3. Use the mime.send{} functionality to format and encode the files as MIME attachments to a message body.

    Here is the specific procedure:

    1. Create and configure a new channel with an LLP Listener source component and a To Translator destination component.
    2. Open the script and commit the first milestone.
    3. Copy and paste the following code snippet into your script, or require it as an external module.
    4. Capture the the attachment data and save it to a scratch location as separate files.
    5. Collect the filenames that you want to attach to the message into a single table.
    6. Invoke mime.send with the same parameters as you would for using smtp.send, passing it the filenames via the additional attachments parameter.

    Note: The purpose of each section in this script is identified with notes right in the code.

    -- $Revision: 3.5 $
    -- $Date: 2012-12-17 15:55:58 $
     
    --
    -- The mime module
    -- Copyright (c) 2011-2012 iNTERFACEWARE Inc. ALL RIGHTS RESERVED
    -- iNTERFACEWARE permits you to use, modify, and distribute this file in
    -- accordance with the terms of the iNTERFACEWARE license agreement
    -- accompanying the software in which it is used.
    --
     
    -- Basic SMTP/MIME module for sending MIME formatted attachments via
    -- SMTP.
    --
    -- An attempt is made to format the MIME parts with the correct headers,
    -- and pathnames that represent non-plain-text data are Base64 encoded
    -- when constructing the part for that attachment.
    --
    -- SMTP/MIME is a large and complicated standard; only part of those
    -- standards are supported here. The assumption is that most mailers
    -- and mail transfer agents will do their best to handle inconsistencies.
    --
    -- Example usage:
    --
    -- local Results = mime.send{
    --    server='smtp://mysmtp.com:25', username='john', password='password',
    --    from='john@smith.com', to={'john@smith.com', 'jane@smith.com'},
    --    header={['Subject']='Test Subject'}, body='Test Email Body', use_ssl='try',
    --    attachments={'/home/jsmith/pictures/test.jpeg'},
    -- }
     
    mime = {}
     
    if help then
      local mimehelp = {
           Title="mime.send";
           Usage="mime.send{server= [, username=] [, ...]}",
           Desc=[[Sends an email using the SMTP protocol. A wrapper around net.smtp.send.
                  Accepts the same parameters as net.smtp.send, with an additional "attachments"
                  parameter:
                ]];
           ["Returns"] = {
              {Desc="nothing."},
           };
           ParameterTable= true,
           Parameters= {
               {attachments= {Desc='A table of absolute filenames to be attached to the email.'}},
           };
           Examples={
               [[local Results = mime.send{
                 server='smtp://mysmtp.com:25', username='john', password='password',
                 from='john@smith.com', to={'john@smith.com', 'jane@smith.com'},
                 header={['Subject']='Test Subject'}, body='Test Email Body', use_ssl='try',
                 attachments={'/home/jsmith/pictures/test.jpeg'},
               }]],
           };
           SeeAlso={
               {
                   Title="net.smtp - sending mail",
                   Link="http://wiki.interfaceware.com/1039.html#send"
               },
               {
                   Title="Tips and tricks from John Verne",
                   Link="http://wiki.interfaceware.com/1342.html"
               }
           }
       }
    end
     
    -- Common file extensions and the corresponding
    -- MIME sub-type we will probably encounter.
    -- Add more as necessary.
    local MIMEtypes = {
      ['pdf']  = 'application/pdf',
      ['jpeg'] = 'image/jpeg',
      ['jpg']  = 'image/jpeg',
      ['gif']  = 'image/gif',
      ['png']  = 'image/png',
      ['zip']  = 'application/zip',
      ['gzip'] = 'application/gzip',
      ['tiff'] = 'image/tiff',
      ['html'] = 'text/html',
      ['htm']  = 'text/html',
      ['mpeg'] = 'video/mpeg',
      ['mp4']  = 'video/mp4',
      ['txt']  = 'text/plain',
      ['exe']  = 'application/plain',
      ['js']   = 'application/javascript',
    }
     
    -- Most mailers support UTF-8
    local defaultCharset = 'utf8'
     
    --
    -- Local helper functions
    --
     
    -- Given a filespec, open it up and see if it is a
    -- "binary" file or not. This is a best guess.
    -- Tweak the pattern to suit.
    local function isBinary(filename)
      local input = assert(io.open(filename, "rb"))
     
      local isbin = false
      local chunk_size = 2^12 -- 4k bytes
     
      repeat
        local chunk = input.read(input, chunk_size)
        if not chunk then break end
     
        if (string.find(chunk, "[^\f\n\r\t\032-\128]")) then
          isbin = true
          break
        end
      until false
      input:close()
     
      return isbin
    end
     
    -- Read the passed in filespec into a local variable.
    local function readFile(filename)
      local f = assert(io.open(filename, "rb"))
      -- We could read this in chunks, but at the end of the day
      -- we are still streaming it into a local anyway.
      local data = f:read("*a")
      f:close()
     
      return data
    end
        
    -- Based on extension return an appropriate MIME sub-type
    -- for the filename passed in.
    -- Return 'application/unknown' if we can't figure it out.
    local function getContentType(extension)
      local MIMEtype = 'application/unknown'
     
      for ext, subtype in pairs(MIMEtypes) do
        if ext == extension then
          MIMEtype = subtype
        end
      end
     
      return MIMEtype
    end
     
    -- Base64 encode the content passed in. Break the encoded data
    -- into reasonable lengths per RFC2821 and friends.
    local function ASCIIarmor(content)
      local armored = ''
      local encoded = filter.base64.enc(content)
      
      -- SMTP RFCs suggests that 990 or 1000 is valid for most MTAs and
      -- MUAs. For debugging set this to 72 or some other human-readable
      -- break-point.
      local maxl = 990 - 2  -- Less 2 for the trailing CRLF pair
      local len = encoded:len()
      local start = 1
      local lineend = start + maxl
      while lineend < = len do
        local line = encoded:sub(start, lineend)
        armored = string.format("%s\r\n%s", armored, line)
     
        -- We got it all; leave now.
        if lineend == len then break end
     
        -- Move the counters forward
        start = lineend + 1
        lineend = start + maxl
     
        -- Make sure we pick up the last fragment
        if lineend > len then lineend = len end
      end
     
      if armored == '' then
        return encoded
      else
        return armored
      end
    end
     
    -- Similar to net.smtp.send with a single additional required parameter
    -- of an array of local absolute filenames to add to the message
    -- body as attachments.
    --
    -- An attempt is made to add the attachment parts with the right
    -- MIME-related headers.
    function mime.send(args)
      local server = args.server
      local to = args.to
      local from = args.from
      local header = args.header
      local body = args.body
      local attachments = args.attachments
      local username = args.username
      local password = args.password
      local timeout = args.timeout
      local use_ssl = args.use_ssl
      local live = args.live
      local debug = args.debug
      
      -- Blanket non-optional parameter enforcement.
      if server == nil or to == nil or from == nil
          or header == nil or body == nil
             or attachments == nil then
          error("Missing required parameter.", 2)
      end
     
      -- Create a unique ID to use for multi-part boundaries.
      local boundaryID = util.guid(128)
      if debug then
        -- Debug hook
        boundaryID = 'xyzzy_0123456789_xyzzy'
      end
      local partBoundary = '--' .. boundaryID
      local endBoundary = '--' .. boundaryID .. '--'
     
      -- Append our headers, set up the multi-part message.
      header['MIME-Version'] = '1.0'
      header['Content-Type'] = 'multipart/mixed; boundary=' .. boundaryID
     
      -- Preload the body part.
      local msgBody =
        string.format(
          '%s\r\nContent-Type: text/plain; charset="%s"\r\n\r\n%s',
            partBoundary, defaultCharset, body)
     
      -- Iterate over each attachment filespec, building up the 
      -- SMTP body chunks as we go.
      for _, filespec in ipairs(attachments) do
        local path, filename, extension =
              string.match(filespec, "(.-)([^\\/]-%.?([^%.\\/]*))$")
     
        -- Get the (best guess) content-type and file contents.
        -- Cook the contents into Base64 if necessary.
        local contentType = getContentType(extension)
        local isBinary = isBinary(filespec)
        local content = readFile(filespec)
        if isBinary then
          content = ASCIIarmor(content)
        end
        
        -- Existing BodyCRLF
        -- Part-BoundaryCRLF
        -- Content-Type:...CRLF
        -- Content-Disposition:...CRLF
        -- [Content-Transfer-Encoding:...CRLF]
        -- contentCRLF
        local msgContentType =
          string.format('Content-Type: %s; charset="%s"; name="%s"',
            contentType, isBinary and 'B' or defaultCharset, filename)
        local msgContentDisposition =
          string.format('Content-Disposition: inline; filename="%s"',
            filename)
        -- We could use "quoted-printable" to make sure we handle
        -- occasional non-7-bit text data, but then we'd have to break
        -- the passed-in data into max 76 char lines. We don't really
        -- want to munge the original data that much. Defaulting to 
        -- 7bit should work in most cases, and supporting quoted-printable
        -- makes things pretty complicated (and increases the message
        -- size even more.)
        local msgContentTransferEncoding = isBinary and
          'Content-Transfer-Encoding: base64\r\n' or ''
     
        -- Concatenate the current chunk onto the entire body.
        msgBody =
          string.format('%s\r\n\r\n%s\r\n%s\r\n%s\r\n%s%s\r\n', 
            msgBody, partBoundary, msgContentType, msgContentDisposition,
            msgContentTransferEncoding, content)
      end
     
      -- End the message body
      msgBody = string.format('%s\r\n%s', msgBody, endBoundary)
     
      -- Send the message via net.smtp.send()
      net.smtp.send{
        server = server,
        to = to,
        from = from,
        header = header,
        body = msgBody,
        username = username,
        password = password,
        timeout = timeout,
        use_ssl = use_ssl,
        live = live,
        debug = debug
      }
     
      -- Debug hook
      if debug then
        return msgBody, header
      end
     
    end
     
    -- Hook up the help, if present.
    if help then
      help.set{input_function=mime.send, help_data=mimehelp}
    end
     
    return mime

    Implementation Notes

    Since sending attachments requires modifying the original message body and headers, this script will modify these parameters before passing them onto smtp.send.

    Notice that we are making the distinction between binary and non-binary attachment types. This is, primarily, about creating a message that renders nicely in a variety of mailers upon receipt. The notion of what is a binary attachment and what sort of MIME types are supported is implementation-specific. For example, there is nothing that says that you cannot Base64-encode all attachments, or create your own MIME attachment type (usually preceded by “X-“) to suit.

    Most mailers and SMTP servers are quite forgiving when it comes to SMTP formatting errors. Unfortunately, many mailers and servers might be strict enough that they will reject sufficiently complicated MIME messages. Some experimentation may be necessary.

    Let us know if you find this tip useful!

    This was a question asked by Dick on our linked in forum. I thought I would try posting it here and see what this forum is like to use.

    Jeffrey Lewis gave these links:

    A quick example for parsing xml is here:

    http://wiki.interfaceware.com/634.html

    Some more details are here:

    http://wiki.interfaceware.com/418.html

    For inserting into table you can use:

    http://wiki.interfaceware.com/336.html.

    Jeff wrote:

    Also if using 5.5 should be able to try this:

    Channel= Source=LLP, Filter=Yes, Destination=To Channel
    In filter copy code below:
    Create database from demo.vmd and call it test_demo.sqlite
    http://wiki.interfaceware.com/343.html

    function main(Data)
      -- database
      DB = db.connect{api=db.SQLITE, name='test_demo.sqlite'}
    
      -- XML
      local XML = [[12345JeffMcLaughlin]]
    
      local X = xml.parse{data=XML}
    
      -- tables you will have to
      local T = db.tables{vmd='example/demo.vmd', name='ADT'}
    
      -- data
      T.patient[1].Id = X.Patient.Id[1] 
      T.patient[1].LastName = X.Patient.Name.Last[1]
      print(T)
    
      -- the data into the database table 
      DB:merge{data=T, live=true}  
      DB:query{sql='Select * from patient'}
    end

    There are quite a few little helpful tips that you can have with using the translator to take XML data from files and put them into a database. Next couple of days are busy for me but there are a number of tips I think are helpful.

    Starting off with the file piece. You could use the From File component. Personally I’d recommend just using a From translator component and reading the files in using Lua – it’s more flexible. This is a good example of reading in a file:

    http://wiki.interfaceware.com/374.html

    Then another useful tip is:

    http://wiki.interfaceware.com/638.html

    Which is a helpful utility function to grab text enclosed in a tag.

    If you are not familiar with the translator then doing one of the introductory tutorials using it would be handy.

    Thank you for all your help. I was able to get started using Jeff’s example above and have IGUANA write the sample XML data to a SQL Server DB on another server. Next up is to setup the Translator to parse some XML files I have and extract the data and send it over to the DB server.