Generating an API

To do this I needed to use a couple of more advanced features in the Lua language called “closures” and “meta-tables” together with the translator’s own built in help system.

Don’t panic! I will gradually introduce these concepts one by one with some simple examples so it should be easy to understand. Closures and meta-tables are a pair of features in Lua that allow one to do ‘object orientated’ programming in the language.

Closures are not unique to Lua – other languages like Javascript support closures also. In Lua closures are functions which can access variables in the scope that they are created. We can create functions on the fly in Lua and this is technique I used in my sales force adapter to make the methods to access all the salesforce.com objects.

Have a look at this code fragment:

The code in question loops through the array listing ‘tools’ and then makes a method which is <tool name> + “Select”.

Notice how the functions use the “self” object which is the making use of the colon syntax in Lua.

The table “R” returned from the MakeMethods function is in fact a table of functions. If you were to click on it and browse a window of it’s contents this is what you would see:

Like many other dynamic languages like JavaScript and Python functions can be stored as values in Lua. This next screenshot shows the how it all works nicely with auto-completion:

I strongly encourage you to try out the code for yourself within a translator. Here it is in a format you can copy and paste:

-- We are using closures here to generate a set of methods
function MakeMethods(Owner,MethodList)
   local R = {}
   R.owner = Owner
   trace(R)
   for i=1,#MethodList do
      local Tool = MethodList[i];
      trace(Tool)
 
      R[Tool.."Select"] = function(self) return self.owner.." selected a "..Tool end 
    end
 return R
end

function main(Data)
   local MethodList={"rake", "shovel", "pick", "hammer"}
   trace(MethodList)
   local O = MakeMethods("Jake", MethodList)
   O:hammerSelect()
   O:rakeSelect()
   O:shovelSelect()
   O:pickSelect()
end

Now we can take this to next level using what Lua calls meta-tables. You can set a meta-table for a table in Lua. We’re going to use just one feature of meta-tables which is to have it define a set of default properties for a table. Here’s the code:

If you click on the MetaTable which is returned from MakeMetaTable, this is what you would see:

For a meta-table the __index key has a special meaning:

1. If you look up a property on a table and it has a meta table.
2. And that meta table has the entry __index.
3. And that entry is a table of values.

Then these will treated as properties of that table. You can learn more about this property in the Lua documentation.

Notice how we can use the same meta-table for more than one ‘object’ table – in this example the Mary and Jake tables. The advantage of the meta-table is just that’s it’s a more efficient way to define the set of methods since we only need to do it once and then we can reuse them again and again for many Lua tables. In this context you can think of Lua tables as ‘objects’.

If you know JavaScript well you might notice that Lua meta-tables are very similar to JavaScript prototypes.

Go ahead try out the code yourself and play with it. You can copy it into a translator right from here:

-- We are using closures here to generate a set of methods for a method table
function MakeMetaTable(MethodList)
   local R = {}
   for i=1,#MethodList do
      local Tool = MethodList[i];
      trace(Tool)
 
      R[Tool.."Select"] = function(S) return S.owner.." selected a "..Tool end 
    end
    local MetaTable = {}
    MetaTable.__index = R
    return MetaTable
end

function main(Data)
   local MethodList={"toothBrush", "pick", "hammer"}
   local MetaTable = MakeMetaTable(MethodList)
   local Mary = {owner="Mary"}
   local Jake = {owner="Jake"}
   setmetatable(Mary, MetaTable)
   setmetatable(Jake, MetaTable)
   Mary:hammerSelect()
   Jake:hammerSelect()
   Jake:toothBrushSelect()
end

The next layer to add is support for the translator’s help system. This is what gives the nice informative information and auto-completion of arguments for help functions. The beauty of the system is that one can generate help programmatically also rather than coding it up by hand.

If you look at one of the HelpInfo objects generated in the help method it looks like this:

This means that in the auto-completion for the objects we see descriptions of the methods:

And we select one of these functions we get suggestions on the names for the parameters:

Here’s a version of the code you can copy and play with inside your own translator:

-- We are using closures here to generate a set of methods for a method table
function MakeMetaTable(MethodList)
   local R = {}
   for i=1,#MethodList do
      local Tool = MethodList[i];
      trace(Tool)
 
      R[Tool.."Select"] = function(S) return S.owner.." selected a "..Tool end 
   end
   local MetaTable = {}
   MetaTable.__index = R
   return MetaTable
end

function MakeHelp(MethodList, MetaTable)
   local Methods = MetaTable.__index;
   for i=1, #MethodList do
      local HelpInfo ={}
      local Tool = MethodList[i]
      local MethodName = Tool.."Select"
      trace(Tool, MethodName)
      HelpInfo.Desc = "Select a "..MethodList[i]
      HelpInfo.Title = MethodName
      HelpInfo.ParameterTable = true
      HelpInfo.Parameters = {}
      HelpInfo.Parameters[1]={live={Desc="Active live tool"}}
      trace(HelpInfo)
      help.set{input_function=Methods[MethodName], 
      help_data=HelpInfo}
   end
   help.example() 
end

function main(Data)
   local MethodList={"rake", "shovel", "pick", "hammer"}
   local MetaTable = MakeMetaTable(MethodList)
   MakeHelp(MethodList, MetaTable)
   local Mary = {owner="Mary"}
   local Jake = {owner="Jake"}
   setmetatable(Mary, MetaTable)
   setmetatable(Jake, MetaTable)
   Mary:hammerSelect()
   Jake:hammerSelect() 
end

So the salesforce.com adapter that I wrote makes use of all these techniques to code generate methods to handle the CRUD operations on all the saleforce.com objects we are interested in manipulating.

I came up with this format for defining a object:

objectDefs = {}

objectDefs.contact = {object='Contact', fields={
    OtherPhone="Alternative phone number",
    FirstName="First name of contact",
    AssistantPhone="Their secretary's phone",
    MailingState="State mailing address",
    MRN__c="MRN number",
    Salutation="How to greet them",
    Languages__c="Languages they speak",
    Phone="Phone number of the contact",
    Email="Email of contact",
    AccountId="Account ID of the contact",
    Birthdate="Birthdate of the contact",
    Description="Description of the contact",
    HomePhone="Home phone number",
    LastName="Last name of contact",
    LastActivityDate="Last activity date",
    CreatedById="Who created this",
    LastModifiedDate="When last modified",
    LastModifiedById="Who modified this",
    CreatedDate="Date created",
    MailingPostalCode="Postal code for mailing address",
    Department="What department",
    MobilePhone="Their mobile phone",
    MailingCity="City to mail to.",
    MailingStreet="Street to mail to",
    OwnerId="Owner of this contact",
    IsDeleted="Has this contact been deleted",
    Fax="FAX number",
    Title="Title of contact - CEO etc."
 }}

Now the great thing is that’s not necessary to completely define this by hand. salesforce.com has some handy methods in it’s RESTful API that will give you all the properties of a given object. I was able to write a helper function to go retrieve that and print out my formatted meta data definition. The only hassle was that returned meta-data doesn’t include descriptions of the columns – I had to enter that information by hand.

Notice how a couple of the fields have trailing __c at the end? These are custom fields and the great thing about salesforce.com is that it makes it simple to add these custom fields to the GUI and easy access them from the RESTful api.

So putting everything together meant combining:

1. Caching using the SQLite store module.
2. Dynamically querying the meta data definitions.
3. Generating methods with help using closures, meta-tables and making help on the fly.

Quite a few pieces to put together but the results are pretty fantastic.  Let’s look at the actual adapter and how to use it.