Simple Oauth 1.0 Implementation in Iguana (Lua)

The easiest way to understand how to implement Oauth 1.0 with Xero for a private application is to start with looking how a valid request is structured to the Xero API for a private application.

This is a standard HTTP GET request with some fancy stuff in the Authorization header:

GET /api.xro/2.0/Contacts?where=Name+%3D%3D%22Espresso+31%22 HTTP/1.1\r\n 
Host: api.xero.com\r\n 
Accept: application/json\r\n 
Authorization: OAuth oauth_timestamp="1446759095", oauth_nonce="144675909553796837755", oauth_token="C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA", oauth_consumer_key="C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA", oauth_version="1.0", oauth_signature_method="RSA-SHA1", oauth_signature="QxXVIZFB8sJ3pE3StUJ6o8guCUkGwr6I48syF1SKKLfXmGkYx%2B6AXuSk%2Fxd0YNx86YFZXr5LVRFGhf5jyrKeZeJ8ckzb9dtW0YqzHRlPHd3rSjBmnkjrVmBaIdxP9rnzgYLQowO3OQKmZ29H2RZ%2FABkEDcpz%2BPE9dERMmzlvegc%3D"\r\n

This shows the Xero web api call for getting a list of Contacts where the name is equal to “Espresso 31”.

I’ll go through the steps required to generate this message. The tools I am using are:

1. The openssl command line tool.  Just the default one which comes on both Linux and Mac OS X – so easy since in production we use Linux servers.
2. Lua – as the scripting language we use inside of Iguana.
3. A few API calls from Iguana’s Lua implementation but you could replace those calls with other tools.  In fact porting this to another language shouldn’t be a big deal.

The first few steps of doing this involve setting up the Xero instance so that it has the public key corresponding to a private key we have created.

Step 1

Use openssl to generate the public and private key as described by Xero’s documentation:

https://developer.xero.com/documentation/api-guides/create-publicprivate-key

Only the PEM files are needed so one can skip the third step to make a pfx file since this isn’t needed.

Step 2

Add a private application to the Xero instance that the API will access. It’s necessary to upload the public key as described by Xero’s documentation:

https://developer.xero.com/documentation/auth-and-limits/private-applications

Setting up a Xero public app is more complicated and not necessary for our purposes.

Step 3

This is the complete Lua code for generating the web call to query the list of contacts from Xero with a filtering condition to just return those Contacts with the name “Espresso 31”.  It comes in at just under 90 lines of code:

function main(Data)
   local CONSUMER_KEY = 'C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA'
   local Url = 'https://api.xero.com/api.xro/2.0/Contacts'
   local Params = {}
   Params.where = 'Name =="Espresso 31"' 
   local Headers = {}
   Headers.Accept = 'application/json'
   local Auth = {}
   Auth.oauth_nonce = os.ts.time()..math.random(100000000000)
   Auth.oauth_timestamp = os.ts.time()
   Auth.oauth_version = "1.0"
   Auth.oauth_signature_method="RSA-SHA1"
   Auth.oauth_consumer_key = CONSUMER_KEY
   Auth.oauth_token = CONSUMER_KEY
   -- Create table which merges GET parameters and Oauth parameters
   local AllParams = {}
   for K,V in pairs(Params) do AllParams[K] = V end
   for K,V in pairs(Auth) do AllParams[K] = V end
   local SortedParamAuthString = ConcatenateSigParams(AllParams)
   local SignatureText = 'GET&'..filter.uri.enc(Url)..
                        '&'..filter.uri.enc(SortedParamAuthString)
   Headers.Authorization = 'OAuth'..AuthInfo(Auth) .. ', oauth_signature="'..oauth_escape(SignSha1(SignatureText))..'"'
   local Result = net.http.get{url=Url,headers=Headers, parameters=Params, live=true, debug=true}
   Result = json.parse{data=Result}
end

-- Oauth escaping requires ' ' --> '%20' rather than +
function oauth_escape(V)
   V = filter.uri.enc(V)
   V = V:gsub("%+", "%%20");
   return V
end

function WriteFile(Name, Content)
   local F = io.open(Name, "wb")
   F:write(Content)
   F:close() 
end

function ReadFile(Name)
   local F = io.open(Name, "rb")
   local Content = F:read('*a')
   F:close()
   return Content
end

-- This is an iterator function that sorts the table keys with 
function SortedPairs(t, SortFunc)
   local a = {}
   for n in pairs(t) do table.insert(a, n) end
      table.sort(a, SortFunc)
      local i = 0 -- iterator variable
      local iter = function () -- iterator function
      i = i + 1
      if a[i] == nil then return nil
      else return a[i], t[a[i]]
      end
   end
   return iter
end

function ConcatenateSigParams(Params)
   local X = ''
   local Sorted = {}
   for K,V in SortedPairs(Params) do
      X = X..K..'='..oauth_escape(V)..'&'
   end
   X = X:sub(1, #X-1)
   return X
end

-- Writing out signature text to a file sign it with openssl and reading the signature back - works for the purpose 
function SignSha1(Content)
   WriteFile('text.tmp', Content)
   os.execute('cat text.tmp | openssl dgst -sha1 -sign privatekey.pem -binary > signature.bin')
   local Data = ReadFile('signature.bin')
   local Result = filter.base64.enc(Data)
   return Result
end

function AuthInfo(Auth)
   local R = ''
   for K,V in pairs(Auth) do
      R = R..' '..K..'="'..V..'",'
   end
   return R:sub(1, #R-1)
end

Relatively straightforward code.   There are many things this code does not do:

1. No support for 3 legged Oauth authentication as required for a public or partner application.
2. I haven’t implemented support for doing PUT etc. operations as required to write to the Xero API.  I don’t have need of this although from this point it would be easy to do.
3. The SHA1-RSA signing implementation is to write out the text to be signed in a file and then signing it with the openssl command line tool – could be more efficient I guess.  But Xero doesn’t encourage users to call the API a lot anyway – for our purposes I plan on doing some big queries and just pulling down a bunch of data.

There are definitely are some further things I plan on doing with this code.  One thing is implementing caching like I did with the salesforce.com module. That will make manipulating the data coming out of Xero way more convenient and also mean I don’t get offside with the rules on the number of times one is allowed to make calls to their APIs. I will probably end up bundling up the code in a more modular fashion etc.

But it is worthwhile looking at this simple version of the code as a means of understanding Oauth 1.0 without the complexity of a big wrapper library.

Let’s go through the code and understand what it is doing.

 local CONSUMER_KEY = 'C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA'
 local Url = 'https://api.xero.com/api.xro/2.0/Contacts'
 local Params = {}
 Params.where = 'Name =="Espresso 31"' 
 local Headers = {}
 Headers.Accept = 'application/json'

The ‘CONSUMER_KEY’ comes from a screen in Xero with Step 2 in this document. If you were to use similar code you’d need to replace that.

The API we are hitting is to get a list of Contacts in Xero. See https://developer.xero.com/documentation/api/contacts/.

I’ve applied an optional ‘where’ parameter to search for contacts matching the name “Espresso 31”.

By using the “Accept” HTTP field with the value application/json it indicates to Xero to return the data from the API in JSON rather than XML.

 local Auth = {}
 Auth.oauth_nonce = os.ts.time()..math.random(100000000000)
 Auth.oauth_timestamp = os.ts.time()
 Auth.oauth_version = "1.0"
 Auth.oauth_signature_method="RSA-SHA1"
 Auth.oauth_consumer_key = CONSUMER_KEY
 Auth.oauth_token = CONSUMER_KEY

This section sets up the oauth parameters needed to authenticate with Xero.

The oauth_timestamp value is set to unix epoch time – i.e. the number of seconds since 1970.

The “nonce” parameter is meant to be a unique number that you can never use again to stop a query being reused. One could be more rigorous with this – in theory this code has a 1/100000000000 chance of hitting the same number quickly in succession make a query to the Xero API. Not a big deal for my purposes. If you wanted to be more rigorous one could use a counter.

Because we are only setting things up as a private application and not needing to use full 3 legged Oauth authentication the oauth_token and oauth_consumer_key are both set to the same CONSUMER_KEY value.

Now the fun part begins which is taking all these things:

• The HTTP operation – in this case “GET”
• The URL we call
• GET parameters
• OAUTH parameters

And combining them all together using the special rules of Oauth 1.0 into a block of text which we then sign with our private key using the RSA-SHA1 algorithm.

The issue with signing is that the combination of all the above information has to be done in a very specific way so that you always get the same block of text out the end of it so that the server can verify that the request has been validly signed by the private key.

The first step is to combine all the oauth parameters so that they are alphabetically sorted. This is what we are looking at:

Now in a full blown oauth library one might face an interface which has parameters with the same name. In that scenario you’d have to sort by the values as well as their names. I don’t think this is an issue for Xero.

 local AllParams = {}
 for K,V in pairs(Params) do AllParams[K] = V end
 for K,V in pairs(Auth) do AllParams[K] = V end

The code above combines the GET query parameters and the oauth parameters into a single table.

local SortedParamAuthString = ConcatenateSigParams(AllParams)

function ConcatenateSigParams(Params)
   local X = ''
   local Sorted = {}
   for K,V in SortedPairs(Params) do
      X = X..K..'='..oauth_escape(V)..'&'
   end
   X = X:sub(1, #X-1)
   return X
end

Then we produce a single string out of the parameters.  The string requires that we:

1. Alphabetically sort the parameters by their key.  If keys match we would need sort by value – as I mentioned before, not an issue with Xero’s API.
2. We URI escape the values – in theory the keys should also be escaped but in practice the Xero API doesn’t have any parameters that one needs to escape.
3. We put & characters between each pair and ‘=’ between the key and value.

The escaping is close to standard URI encoding, i.e. non ascii characters are escaped into %HH characters.  But the space character is translated into ‘%20’ rather than ‘+’.  I wrote a simple helper function oauth_escape which used the built in function in Iguana for URI encoding in Iguana and tweaked the output for the space character exception.  There some other special cases like ‘~’  etc. which I haven’t bothered to think about – it is not important for my needs.

function oauth_escape(V)
 V = filter.uri.enc(V)
 V = V:gsub("%+", "%%20");
 return V
end

In Lua pattern matching % and + are special characters so the V:gsub(“%+”, “%%20”) just means replace ‘+’ with ‘%20’.

This code would be easy enough to port over to some other implementation of the same type of escaping routine.

This is the an example string which comes out of ConcatenateSigParams is:

oauth_consumer_key=C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA&oauth_nonce=14467583093494297755&oauth_signature_method=RSA-SHA1&oauth_timestamp=1446758309&oauth_token=C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA&oauth_version=1.0&where=Name%20%3D%3D%22Espresso%2031%22

See how the key are alphabetically sorted? The next step is to combine that string with together with the URL and GET delimited with & and with each component URI escaped again:

  SignatureText = 'GET&'..filter.uri.enc(Url)..'&'..filter.uri.enc(SignatureText)

The “..” in Lua is just notation for string concatenation. Notice how each of the components is URI escaped again?  In theory the HTTP verb “GET” should be escaped also except that it doesn’t have any special characters.  Also in theory I should be translating + –> %20 but in practice all the + characters got converted in the previous step so I just call filter.url.enc. Now the combined text is like this:

The double uri escaping means that at this point the space character ‘ ‘ is now represented with %2520.

To generate the RSA-SHA1 signature I made use of the openssl command line tool. Nothing too fancy.  I just write the above text into a file and invoke openssl and then read the resulting signature back in and encode it in base64.  Not super efficient but easy enough to understand and one could easily swap in an alternative choice which does the same thing:

function SignSha1(Content)
   WriteFile('text.tmp', Content)
   os.execute('cat text.tmp | openssl dgst -sha1 -sign privatekey.pem -binary > signature.bin')
   local Data = ReadFile('signature.bin')
   local Result = filter.base64.enc(Data)
   return Result
end

This is the resulting signature in this case:

pc71jDpoS8NGlbmDzN/Qx9ZJwez9A3FBNQ1ZXrkR8gIfj7KbrZCD2E/7DaGSCtN+OVbqXsWzYU3eWqZfeHklBM0TvAFc1s6c89bQGX3PUgjyj/dhsekukH1K/8FZbN1+QQ7qNeZK4p+I8QI0YCEivo+LPk31GRnpOVctbWW01zw=

Then everything gets combined together.  The oauth parameters are used to populate an Authorization header that starts with “OAuth” then has the parameter names with their values in quotes, and divided with commas.  That is done with this code:

 Headers.Authorization = 'OAuth'..AuthInfo(Auth) .. ', oauth_signature="'..oauth_escape(SignSha1(SignatureText))..'"'

function AuthInfo(Auth)
   local R = ''
   for K,V in pairs(Auth) do
      R = R..' '..K..'="'..V..'",'
   end
   return R:sub(1, #R-1)
end

The final step is to dispatch things off across the wire, which is done with this line:

 local Result = net.http.get{url=Url,headers=Headers, parameters=Params, live=true, debug=true}

Notice that it’s not necessary to use the public key we are signing the oauth request with etc. The easiest way to visualize everything is to see the final HTTP GET request:

GET /api.xro/2.0/Contacts?where=Name+%3D%3D%22Espresso+31%22 HTTP/1.1\r\n 
Host: api.xero.com\r\n 
Accept: application/json\r\n 
Authorization: OAuth oauth_timestamp="1446759095", oauth_nonce="144675909553796837755", oauth_token="C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA", oauth_consumer_key="C00WGMXDTS5QSXWVN5WDOAJ1JHBRKA", oauth_version="1.0", oauth_signature_method="RSA-SHA1", oauth_signature="QxXVIZFB8sJ3pE3StUJ6o8guCUkGwr6I48syF1SKKLfXmGkYx%2B6AXuSk%2Fxd0YNx86YFZXr5LVRFGhf5jyrKeZeJ8ckzb9dtW0YqzHRlPHd3rSjBmnkjrVmBaIdxP9rnzgYLQowO3OQKmZ29H2RZ%2FABkEDcpz%2BPE9dERMmzlvegc%3D"\r\n\r\n

There you can see all the parts which make up a valid request.  Anyways – implementing Oauth 1.0 from scratch in a single file really isn’t too bad. From my perspective this a good way to connect to the Xero API in a transparent manner that makes it easy to see all the moving parts without having a lot of dependencies to manage.

To prove my point I ported this script over to python on Friday morning…