File and FTP Interfaces

Introduction

Using the APIs supplied by the Translator provides an incredibly powerful and convenient set of tools to handle custom requirements using file and FTP operations.

On the whole we recommend you consider using the Translator for most of your file handling needs because it is so simple to use. See minimalism. You can even show the file settings through the Iguana dashboard using the APIs we supply.

Also see the section on invoking external programs to see how you can find the current working directory and get the list of files in a directory.

Using the content of a message to form the output file name [top]

This example shows how easy it is to use the built Lua ‘io’ library to use the message control ID to form the file name of the message. This is just one example:

Note: The main() function uses print() to log messages. When an Iguana channel is run print() messages are logged as informational messages. For more sophisticated logging you can use the iguana.logInfo(), iguana.logWarning(), iguana.logDebug() functions.

Because of the dynamic nature of the Translator environment it’s easy to:

  • Use data from any source to define the file name
  • Dynamically see the generated file name
  • Actually verify that the resulting file is generated as you would like

It makes for a beautifully productive work flow and it’s easy to see the flow of data. Rather cool. Here’s the code in a format you can copy. Enjoy!

require 'node'

function main(Data)
   local Msg = hl7.parse{data=Data, vmd='example/demo.vmd'}
   local FileName = Msg.MSH[10]..'.hl7'
   local F = io.open('D:temp'..FileName, "w")
   F:write(Data)
   F:close()
   print('Wrote "'..FileName..'"')
end

Tip: The file actions are active in the editor (test mode), which means you can create a large number of “test” files. Once your code is working correctly you may want to prevent this. Also you will need to disable this for a live system, to prevent accidental updates.

The solution is simply to use iguana.isTest() to prevent the code from running.

Moving files around [top]

Lua does not have very many built-in file operations. Here are the few that it does have:

  • os.remove(file) – Remove a file. Returns true if the file was removed. Returns nil, plus a string describing the error, if the file was not removed. Example:
    local Result, ErrorString = os.remove('foo.txt')
    if not Result then
       print(ErrorString)
    end
  • os.rename(source, dest) – Renames and/or moves a file. Returns true if the rename/move was successful. Returns nil, plus a string describing the error, if the rename/move failed. Example:
    local Result, ErrorString = os.rename('foo.txt', 'subdir/bar.txt')
    if not Result then
       print(ErrorString)
    end
  • io.open(name, [mode]) – Opens a file, creating it if necessary, provided the ‘mode’ allows creation of the file. Returns a file handle object if the open was successful. Returns nil, plus a string describing the error, if the open was not successful. ‘mode’ is a string that is in the same format as the ‘mode’ argument to standard C function, fopen. The most common modes are:
    • ‘r’ – read (the default mode). The file must exist.
    • ‘w’ – write. This will create the file if necessary.
    • ‘a’ – append. This will create the file if necessary.
  • The file handle returned from io.open has the following operations:
    • handle:close() – Closes the file. It is always a good idea to explicitly call this whenever you are done with the file.
    • handle:flush() – Save any written data to the file.
    • handle:lines() – Returns an iterator function which can be used to iterate over each line in the file. Example:
      local File = io.open('foo.txt')
      for Line in File:lines() do
         print(Line)
      end
    • handle:read(…) – Read from the file. By default, will read one line from the file. To read all lines, pass specify ‘*a’ as the argument. For a complete list of valid arguments, see the file:read entry in the lua reference manual.
    • handle:write(…) – Writes all arguments to the file.
    • See the lua reference manual for the less common operations, file:seek and file:setvbuf.
  • os.execute(command) – Executes ‘command’, and returns a return code (zero for success, non-zero if an error occurred). We will see in the section below how os.execute can be used for every other file operation.

Using os.execute() for Everything Else

os.execute can be used to do pretty much anything you can imagine. But it does have one limitation – you can’t get the standard output of the command. Here is a basic wrapper of the function you can use to get around this:

-- Execute a command.
-- The first return value is the output from the command
-- (or nil if there was no output).
-- The second is the returned status code from executing the command.
--
function MyExecute(Command)
   -- We use os.tmpname(), a built-in Lua function, to give us a
   -- temporary file name.
   local TempFileName = os.tmpname()
   local Result = os.execute(Command..' > '..TempFileName..' 2>&1')
   local TempFile = io.open(TempFileName)
   if TempFile then
      local CommandOutput = TempFile:read('*a')
      TempFile:close()
      os.remove(TempFileName)
      return CommandOutput, Result
   else
      return nil, Result
   end
end

This writes the output of the command to a temporary file, then reads the file to get the output, and deletes the temporary file. Now the example in Recursive file search can be simplified like so:

-- Does a recursive search of the directory specified
-- by Root, and returns the full path of the file with
-- the name File, if it is found.  Returns nil otherwise.
--
function findFile(Root, File)
   return MyExecute('dir /S /B "'..Root..'" | find "'..File..'"')
end

So you can see, anything you can do from the command-line, you can do with os.execute(). Other uses include, but are not limited to:

  • Copying files (using the MS ‘copy’ command, or the posix ‘cp’ command)
  • Creating a directory (using the MS ‘md’ command, or the posix ‘mkdir’ command)
  • Removing a directory (using the MS ‘rd’ command, or the posix ‘rmdir’ command)

Sending files via FTP, FTPS (SSL/TLS) [top]

Say you have the means of getting the path to an image file in your file system. Here is a sample script for a “To Translator” component, showing how you can use the net api to upload that file to a server via FTP/FTPS/SFTP:

-- The main function is the first function called from Iguana.
-- The Data argument will contain the message to be processed.
function main(FileId)
   local FileName = 'Images/'..FileId..'.tiff'

   local ImageFile = io.open(FileName)
   if not ImageFile then
      -- TODO make an email notification rule that sends emails
      -- whenever a log message starting with
      -- 'Image file does not exist' is found.
      print('Image file does not exist: "'..FileName..'".')
      return
   else
      ImageFile:close()
   end

   -- set up the connection and send the file
   local Ftp = net.ftp.init{server='192.168.0.100',username='fred',password='secret'}
   Ftp:put{local_path=FileName,remote_path='/ScanImages/'..FileId..'.tiff'}
end

In this example, I’ve just constructed the path from the ID that gets passed into the main function. This ID is produced from a “From Translator” component (see Polling a database, and generating CSV files). Note the “TODO” comment – you can set up an Iguana email notification rule to send out an email when a specific problem is encountered.

Note: FTPS/SFTP are also supported, see the api documentation for details.

For more information regarding “Details on Server SSL Certificates”, please refer to: http://curl.haxx.se/docs/sslcerts.html

Recursive file search [top]

Because Lua has an extremely minimal interface, there is no built-in functions for many tasks. However, the os.execute() function can be used to accomplish nearly anything. In this case, we will use it to recursively search through a directory and its subdirectories to find the full path of a specified file, given its name. Here is my function:

-- Does a recursive search of the directory specified
-- by Root, and returns the full path of the file with
-- the name File, if it is found.  Returns nil otherwise.
--
function findFile(Root, File)
   local OutputFileName = File..'.fs'
   os.execute('dir /S /B "'..Root..
      '" | find "'..File..'" > "'..OutputFileName..'"')

   local OutputFile = io.open(OutputFileName)
   local FullPath = OutputFile:read()
   OutputFile:close()
   os.remove(OutputFileName)

   return FullPath
end

This code is written for Windows. The /S flag makes the command search through subdirectories, and the /B flag makes only the plain file paths printed (and no other info on the file). For Posix, the following command will work:

os.execute('find "'..Root..'" | grep "'..File..'" > "'..OutputFileName..'"')

Where is the best place to call net.xxx.init{} [top]

We had this question come up in our Forums: Is it better to call net.sftp.init{} at the beginning of the main.lua file (so that it is only executed on channel start), or inside of main()?

The simple answer is that it makes very little difference. The net.sftp.init{} call simply caches the connection parameters, a separate connection is then opened and closed for each subsequent SFTP command.

There is a minor performance overhead depending on where the call is placed. When the call is outside main() it is only called once when the channel is started, if it is inside it is inside main() then it is called for each message, which is a marginally greater overhead.

Note: The same thing applies for net.ftp.init{} and net.ftps.init{}.

Please contact support at support@interfaceware.com if you need more help.

One Comment

  1. Tracy Harnden

    Looking for how to get a file from a remote server using SFTP and pushing it through a queue to another channel and then deleting the file that was downloaded. I found the references to using net.sftp.get and net.sftp.delete, but how do I know I am deleting the file that I got and not another file in the source folder? Also, how do I push the file to queue once I get it? If a simple example of the code which would accomplish this could be added to this page, that would be helpful.