AWS S3 Adapter

Introduction

The AWS S3 Adapter module allows seamless integration with increasingly popular AWS S3 service. This module is designed to post files into an S3 bucket and to read the content of previously stored files.

This module helps to centralize S3 integration code in one place, rather than having to edit Translator code in individual channels.

Using the Code [top]

  1. Setup AWS:
    1. Follow AWS documentation to create an IAM user and assign it with AmazonS3FullAccess permission.

      Note: Make a note of IAM user Access Key and of Access Key ID

    2. Follow Amazon Documentation to create AWS S3 bucket:

      For beginners there is no need to change the default options. At minimum, only bucket name and desired region need to be provided.

  2. Setup Translator module:
    1. Import channel AWS_S3_Adapter-To-… from repository.

      Tip: The Source component for Translator channel can be any supported type. To customize a channel, all it takes is to export/import the Destination Component Translator Project into another channel.

    2. Update S3 service parameters in aws/configuration.lua, namely:
      • bucket name
      • region
      • access Key ID
      • access Key

      Note: If you have used AWS buckets before you may have accessed them without using “aws-region” — using a URL naming scheme like this http://bucket.s3.amazonaws.com. However buckets created in Regions launched after January 30, 2019 are not reachable with this scheme — you must include the region, like this: http://bucket.s3.aws-region.amazonaws.com. See https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro for more information.

      This is why the code uses the new scheme as it will work with all AWS buckets.

  3.  Results verification
    1. Navigate to last line of function main(), click in Annotations on returned from AWS S3 value. Read the content of file as it is stored in the S3 bucket.
    2. Alternatively, one can navigate browser to bucket content at AWS S3 site and to review same file content in web browser, for comparison.    

How it works [top]

This AWS S3 adapter supports processing of one file at a time, for upload or reading, using the configured basket. For multiple files processing, please consider repeating subsequent calls.

We recommend using this module to store and read data to/from AWS S3 in development, test, and production environments. This means that when you migrate a channel between environments you simply need to update credentials in configuration.

More information [top]