Streaming File Operations

Introduction

This example shows how to use the Iguana APIs to deal with large files, that need to be processed a little bit at a time. We use use the builtin io.xxx file functions the stream.lua module that contains a range of streaming functions for use with large files, sockets, pipes, etc.

If you have any questions please contact us at support@interfaceware.com.

Using the Code [top]

  • Import the Streaming File Operations channel from the Builtin: Iguana Files repository
  • Experiment with the code to find out how it works
  • Then add the module to your Translator project
  • Copy the require statement from the channel and add it at the top of your script
    Note: This module uses require to return a table
  • Adapt the code to your own requirements
  • Interactive scripting help is included for this module

This is the github code for the main module:

How it works [top]

The code is fairly simple:

  • We use the stream.toFile() function to stream some sample data to a file
  • Then we open the file we create with io.open()
  • Next process it line by line using an iterator returned from io.lines()
  • Finally we process the data as fixed length by using io.read() to repeatedly read 7 characters

More information [top]