This topic contains 19 replies, has 3 voices, and was last updated by  Wade 8 years, 7 months ago.

Calling OpenSSL with popen() for Secure XML

  • I thought I’d share some work I’ve been doing around Signed XML and Encrypted XML in case it will help someone else.

    Secure XML is a general term that encompasses the signed XML and encrypted XML profiles ( to allow secure delivery of confidential information across public networks. Producing signed and encrypted XML in Iguana Translator requires the use of hashing and both symmetric and asymmetric encryption.

    The full project zip covering all that I’m describing here, plus a little extra for placing the result in a SOAP envelope and addressing for delivery, is attached.


    Hashing uses an algorithm to take an input of text or bytes and generate a unique result. Hashing the same input produces the same result while changing a single character or byte with produce a completely different result. There are a number of hashing algorithms. is available in the util module in Translator however SHA1 that I have used for XML signing and encryption is not.

    The hash algorithm returns byte values which need to be converted to a text based code. Base64 is ideal for this. Base64 uses a set of 64 ASCII characters to represent possible values. The input is grouped into sets of 3 bytes (24 bits) which are split into 4, 6 bit values. Each value is then represented by one of the 64 ACSII characters. A 30 byte input generates a 40 character string.

    Encryption can be symmetric or asymmetric. Symmetric encryption uses a key to encrypt text or bytes and the same key is used to decrypt it. The encrypted result is returned as bytes with will need to be converted to Base64. There is no restriction on the size of the encrypted payload. Translator includes encryption in the filter module, AES-128 CBC mode to be precise.

    Asymmetric encryption uses 2 keys called the public key and the private key, and this is generally referred to as Public Key Infrastructure (PKI) or RSA. A payload encrypted by a public key can only be decrypted by the matching private key and a payload encrypted by the private key can only be decrypted by the public key. As the name suggests, the private key is never shared while the public key can be shared freely.

    Encrypting an payload with the public key ensures only the private key owner, the intended recipient, can decrypt it. Encrypting something with the private key means that anyone with the public key can decrypt it but verifies that the identity of the sender is the holder of the private key. This process is called ‘signing’.

    For PKI, the size of the payload is restricted to being smaller than the key where the key is either 1024 or 2048 bits. It is best suited to securely passing small pieces of information such as the key used to encrypt a payload or the hash of the payload.

    SHA1 and PKI are not available natively in Translator but can be implemented using the OpenSSL command line interface through the io.popen() function.

    OpenSSL and popen()

    Of course, since this project requires OpenSSL, you will need to download and install it first:

    Popen() launches an external application and interacts with it through stdin and stdout. It can be in write mode or read mode. In write mode, Translator can write to the stdin for the application and in read mode, the stdout output from the application can be returned to Translator.

    local _sTempFolder = ‘c:\\temp\\’
    local  _sOpenSSL_Path = ‘c:\\PpenSSL-Win64\\bin\\’
    function util:sha1(_sPayload)
       -- call OpenSSL to return sha1 hash
       -- write payload to file
       local _file = _sTempFolder .. util.guid(128) .. '.dat'
       local _f =, 'wb')
       --call OpenSSL with popen and read result from stdout
       local _sCommand = _sOpenSSL_Path .. 'openssl dgst -binary -sha1 ' .. _file
       local _pipe = io.popen(_sCommand)
       local _sha1 = filter.base64.enc(_pipe:read('*a'))
       -- remove temporary file
       if _sha1 == '' then
          error('Error executing command: ' .. _sCommand, 2)
       return _sha1

    This function generates the SHA1 hash of the input value and returns it as a Base64 string. It begins by saving the input to a temporary file then initiating the OpenSSL command line utility using popen(). The command would look like “C:\OpenSSL-Win64\bin\openssl dgst -binary -sha1 c:\temp\9D0FA1531E025EFC9C9FF553FFFE53A8.dat”.

    Popen() returns a file-like reference from which stdout can be read, then converted to Base64. The result is returned to the calling function.

    Signing and encryption functions are almost identical but use a different command line:
    filter.RSA = {}

    function filter.RSA:sign(_sPayload, _sPrivateKeyFile)
       -- call OpenSSL to return signature value
       -- write payload to file
       local _file = _sTempFolder .. util.guid(128) .. '.dat'
       local _f =, 'wb')
       --call OpenSSL with popen and read result from stdout
       local _sCommand = _sOpenSSL_Path .. 'openssl pkeyutl -sign -in ' .. _file .. ' -inkey ' .. _sPrivateKeyFile
       local _pipe = io.popen(_sCommand)
       local _sSign = filter.base64.enc(_pipe:read('*a'))
       -- throw error if no value returned
       if _sSign == '' then
          error('Error executing command: ' .. _sCommand, 2)
       -- remove temporary file
       return _sSign
    function filter.RSA:encrypt(_sPayload, _sPublicKeyFile)
       -- call OpenSSL to return encrypted value
       -- write payload to file
       local _file = _sTempFolder .. util.guid(128) .. '.dat'
       local _f =, 'wb')
       --call OpenSSL with popen and read result from stdout
       local _sCommand = _sOpenSSL_Path .. 'openssl pkeyutl -encrypt -pubin -in ' .. _file .. ' -inkey ' .. _sPublicKeyFile
       local _pipe = io.popen(_sCommand)
       local _sEnc = filter.base64.enc(_pipe:read('*a'))
       -- throw error if no value returned
       if _sEnc == '' then
          error('Error executing command: ' .. _sCommand, 2)
       -- remove temporary file
       return _sEnc

    Signed XML

    Signed XML provides a standard XML structure in which to contain a payload. The payload remains in clear sight. Signing the XML document has 2 main functions – it verifies that the payload has not been altered from its original form by including the hash value and the sender attests to the authenticity of the payload but encrypting the hash with the private key.

    The Signature element contains all the information relating to the signing while the payload can be located within the signature element (enveloped or enveloping) or outside the signature element (detached), whether within the same document or an external resource. In this example, the payload is detached and within the XML document.

    The basic algorithm for signing is to produce a canonical representation of the XML element containing the payload and hash it, using SHA1 in this case. More about canonicalization shortly. The hash value is signed using the private key of the sender. The result of the signing is included in the XML. It is not uncommon to also include the public key within the signature element.

    This function accepts a payload, perhaps a HL7 V2 message and embeds it in the XML document then signs it and returns the XML. = {} = {} = {}
    -- set parameters
    -- key files = '' = '' = '' = '' = [[
    <sp:signedPayload xmlns:sp="">
    		<ds:Signature xmlns:ds="">
    				<ds:CanonicalizationMethod Algorithm="" />
    				<ds:SignatureMethod Algorithm="" />
    				<ds:Reference URI="#">
    						<ds:Transform Algorithm="" />
    					<ds:DigestMethod Algorithm="" />
    	<sp:signedPayloadData id="">
       -- insert a payload into a signed xml structure and sign it
       -- get template
       local _xmlSigned = xml.parse{data =}
       -- add payload
       -- set id
       _sID = util.guid(128)
       _xmlSigned["sp:signedPayload"]["sp:signedPayloadData"].id = _sID
       _xmlSigned["sp:signedPayload"]["sp:signatures"]["ds:Signature"]["ds:SignedInfo"]["ds:Reference"].URI = '#' .. _sID
       -- hash the payload
       -- canonicalize it first
       local _sHash = util:sha1(_xmlSigned["sp:signedPayload"]["sp:signedPayloadData"]:xmlCanonicalize(_xmlSigned):xmlCrRef())
       -- hash the signed info that includes the hash of the payload
       _sHash = util:sha1(_xmlSigned["sp:signedPayload"]["sp:signatures"]["ds:Signature"]["ds:SignedInfo"]:xmlCanonicalize(_xmlSigned))
       -- sign the hash with the private key
       local _sSignature = filter.RSA:sign(_sHash,
       -- include the public key to verify the signature
       -- return the signed xml
       return _xmlSigned:S()    

    Canonical XML

    Since changing a single character within the payload will result in a different hash value, it is important that a standard representation of the XML element(s) is used. This is called Canonicalized XML and includes – but not limited to – the following transformations:
    • The document is encoded in UTF-8
    • Linebreaks are normalized to line feeds (0x0a)
    • Attribute delimiters are set to double-quote (“)
    • Empty and redundant namespace attributes are removed
    • All namespace attributes appear before other attributes
    • Lexographic order is imposed on attributes – sorted alphabetically
    • If the element is part of an XML document and does not have a namespace attribute, the namespace should be explicitly set from the parent or ancestor.
    • Reserved characters within attribute and character content is replated with character references
    • CDATA sections are replaced with character content
    • Elements are converted to start / end pairs – no shorthand representation
    • All whitespace within the document element is preserved
    • All whitespace outside the document element is removed (leading and trailing)

    In general, the important issues when canonicalizing XML in Translator are to sort the attributes, convert any cdata, remove redundant or empty namespace declarations, insert the parent namespace if it is not defined and preserve the internal white space. This last one means that the leading spaces need to account for the indenting from the top of the document, not just the element we are processing. Getting the parent namespace is also interesting. I understand that not including a Parent property in the node was a conscious decision made by Interfaceware for very good reasons, but …. damn.

    Check out the project to see the code for doing this. It takes a general approach to canonicalizing the XML as the payload may itself be an XML document. The code is a bit too extensive for inclusion here.

    Encrypted XML

    The encryption part takes the signed XML and encrypts it using AES. So that it can be decrypted by the receiver, the encryption key that was used is itself encrypted with the public key and included in the EncryptedPayload XML structure. = [[
    <ep:encryptedPayload xmlns:ep="">
    		<xenc:EncryptedKey xmlns:xenc="" Id="">
    			<xenc:EncryptionMethod Algorithm="" />
    			<ds:KeyInfo xmlns:ds="">
    				<xenc:DataReference URI="" />
    		<xenc:EncryptedData xmlns:xenc="" Id="" Type="">
    			<xenc:EncryptionMethod Algorithm="" />
       -- encrypt the payload (signed xml)
       -- insert encrypted data into encrypted xml structure
       -- get emcrypted xml structure
       local _xmlEncrypted = xml.parse{data =}
       -- set random 128 bit aes key
       local _sKey = util.guid(128)
       --  symetrical encrypt with key (aes128-cbc)
       local _sEnc = filter.base64.enc(filter.aes.enc{data=_sPayload,key=_sKey})
       -- set id
       _sID = util.guid(128)
       _xmlEncrypted["ep:encryptedPayload"]["ep:encryptedPayloadData"]["xenc:EncryptedData"].Id = _sID
       _xmlEncrypted["ep:encryptedPayload"]["ep:keys"]["xenc:EncryptedKey"].Id = util.guid(128)
       _xmlEncrypted["ep:encryptedPayload"]["ep:keys"]["xenc:EncryptedKey"]["xenc:ReferenceList"]["xenc:DataReference"].URI = '#' .. _sID
       -- set SKI
       -- encrypt aes key with private key 
       -- return encrypted xml structure
       return _xmlEncrypted:S()

    Sending the Encrypted XML

    The encrypted XML will be most commonly delivered through web services. There may be additional wrappers, as in the project zip where a SOAP envelope is used.

    The receiver will decrypt the payload using their private key. This will reveal the signed XML document. The original payload will be available and the receiver can apply the same hash algorithm to it as compare with the hash in the signed XML. The will also decrypt the signature using the public key and ensure that the hash values match.

    In the end, the payload information can be assured that it has been received only by the intended recipient and they can confirm that it is both unaltered and originated from the correct sender.


    Thanks to Lev Blum and an unfortunately unnamed developer from Interfaceware for getting me started.

    For more information about Secure XML or Excrypted XML, refer to:
    XML Signature Syntax and Processing:
    XML Encryption Syntax and Processing:
    Canonical XML:

    Full code is included in the attached project zip.

    You must be logged in to view attached files.

    Thanks so much for sharing this Garry – it was Peter Antoniw our VP of Development who figured out the right flags to invoke in openssl.

    I had a go at loading up the example on my Mac. I’d like to introduce it into the Iguana Community Repository if I can. One advantage is that we could make sure that it doesn’t have modules that clash with the standard ones.

    I noticed you had some very nice extensions to the stringutils and node which might be handy for other people to explore although I think I will put them into a differently named module just because it may cause some unexpected effects if people load this into their own Iguana instances with existing production code.

    I was very happy to see that you use the Iguana help system a lot for documenting your functions. You might like to have a look at this project:

    (The URL isn’t appropriately named at this point 🙂 )

    One thing was odd is that I got some errors in the project when I imported it. Firstly there was a Trace module which didn’t exist. And a lot of the help documents had titles with \ in them like this:

    h.Title = ‘node\trimNode’

    The translator complained that these were illegal escape sequences and I had to change the \ to a “.” to make it work. I wonder if this is a bug caused from character encoding coming from what I presume is a Windows machine into my Mac OS X machine?

    There are some helpful utilities in the ‘file’ module we have in the repository which can automatically give you the right TEMP directory – we can change that over once we get this working.

    The final problem I ran into is that the openssl command line tool (once I modified the path) I have installed as part of the default environment on my Mac didn’t seem to like the command line arguments:

    openssl x509 -inform PEM -pubkey -noout -in

    I’m on the road in LA so I ran out of time at this point. But I will pass this over to Wade and Peter at home to see if they can reproduce the same issues. Be nice if we can get this scrubbed up and in the repository.

    There are some nice auto-detection tricks one can do with command line tools like openssl – we do that sort of thing in Iguana with detecting database libraries and so on. Be nice if put that in once we get this working.

    G’day Eliot,
    Thanks for the pointer to the file module. There are some functions I can certainly use.

    I’ll admit that I fell into my own versioning trap. I did this work on a different machine to the one I usually use and it had some old modules on it, still using the default module names. As for the backslash in the help title, I have no idea. Seniors moment when I was creating them I think.

    I’m not sure about the issue with the OpenSSL command line, it works OK on my PC but perhaps it’s Mac specific. I wish I could say I knew what I was talking about but I’m still pretty much a novice in this whole PKI / OpenSSL thing. I assume the certificate file name is passed in the function call – the -in parameter is the cert file and it wasn’t in your sample command. The errors don’t seem to be streamed back to Translator so I did a lot of pasting into the Command Box when testing commands so I could see the errors.

    I’d look forward to get some real experts like Wade and Peter to go over it.

    I had a look at the files with unpacking it in StuffIt and got the same slashes.

    Seems like I am running a different version of OpenSSL on my Mac.

    The errors show up in standard error output – running my iguana instance in a command line window I can see it.

    Just tapping on my iPhone at lax so hard to go deeper…

    Nice well written Lua code though – well structured.

    G’day Eliot,
    Yes, the slashes are real. The result of some brain-fade on my part I think. 🙂


    Looks like my openssl version is this:

    OpenSSL 0.9.8y 5 Feb 2013

    If you type:

    openssl version

    Apparently it prints the version.

    Mine is 1.0.1g 7 Apr 2014

    OpenSSL Command Line Documentation
    I don’t know if that will help.

    LAX is fun, isn’t it. 🙂 Happy travelling.

    The script could probably be tweaked to check for the existence of the right version of openssl and raise a nice informative error if one has the wrong version. When I get somewhere with decent internet.

    This is the sort of thing which would be handy to put into a GIT fork repo so we could fix it up before shifting it into core repo. Ever played with GIT?

    LAX isn’t too bad today although our flight is delayed…

    Good idea. I’ve added this code in the CryptoUtil module, just before the first function so it should execute only once.

    -- call OpenSSL to check the version
    local _sCommand = _sOpenSSL_Path .. 'openssl version'
    local _pipe = io.popen(_sCommand)
    local _sVer = _pipe:read('*a')
    -- throw error if no value returned
    if _sVer == '' then
       error('Error executing command: ' .. _sCommand, 2)
    if _sVer:sub(_sVer:find('[%d.]+')) < '1.0.1' then
       error('OpenSSL versions prior to 1.0.1 are not supported', 2)      

    If it does fail, you can’t continue work till it’s fixed. Interesting – Translator throws the error on line 1, regardless of where the ‘require’ is located.

    Looking into GIT is on my to-do list, along with so much other stuff. 🙂

    I believe I’ve got it working now in OSX Mavericks (10.9.3) using OpenSSL “0.9.8y 5 Feb 2013”. As far as I can tell this works with later versions of OpenSSL as well (I tested on an fresh Ubuntu 12.04.4 VM that’s still running 1.0.1 and it worked).

    After importing the project Garry attached with his first post there are only a few changes to make.

    First generate a private key and x509 certificate file using OpenSSL – this can be done with the command below which will walk you through a series of prompts when generating the certificate.

    openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout private.key -out certificate.cert

    Next you’ll want to set the private key, certificate, and public key paths in main. Garry’s project extracts the public key for us automatically so we don’t need to worry about generating that from the command line. As an example this is what my 4 lines in main look like after modification.'/Users/wade.shrewsbury/work/certs/test.key')'/Users/wade.shrewsbury/work/certs/test.cert')

    Following that you’ll want to set your OpenSSL and temp paths at the top of CryptoUtil. In the case of OS X and Ubuntu OpenSSL is located in /usr/bin/ and the temp path would be /tmp/ (Note: the trailing slash is important here)

    Finally you need to change some of the OpenSSL commands in CryptoUtil to use rsautl instead of pkeyutl.

    1. Line 44: change pkeyutl to rsautl
    2. Line 97: change pkeyutl to rsautl
    3. Line 134: change pkey to rsa

    Hopefully I didn’t miss any steps. Also, I know pkeyutl is the successor in some ways to rsautl, but I think they’re doing the same thing. If there’s a reason not to use rsautl please make a comment as pkeyutl isn’t available in the OpenSSL library included in Mavericks.

    Thanks for taking a look at it Wade.

    I’ve attached a new version:

    • Renamed node and stringutil modules to avoid overwriting default modules and removed the slash from the help titles
    • Automatically set the temp folder
    • Detect if Windows or Linux / OS X and change OpenSSL commands eg pkeyutl to rsautl
    • Detect OpenSSL version. The minimum version can be easily changed but I’ve left it at 1.0.1 because I don’t know when pkeyutl was introduced.

    Also, I noticed the URLs for the references didn’t come up in the original post. Here they are:
    Signed XML
    Encrypted XML
    Canonical XML

    You must be logged in to view attached files.

    Alright – guess we can give a little bit more inspection and see if we can merge it into the main Iguana app repository. What we try and do with the channels in the repository is make them easy for people to install and get running without too much manual configuration.

    Mostly people are pretty busy so it’s discouraging if they install something which doesn’t work out of the box…

    OK, last update I hope.

    • uses absolute path for key files
    • fixes bug when there’s a space in the path
    • replaces backslash with forward slash in hard paths, hopefully more platform independent and still works on Windows

    Wade, I’d be interested if this version works without modification other than OpenSSL location.


    You must be logged in to view attached files.

    Hi Garry,

    Thanks for the updated project. I hope you don’t mind but I’ve made a couple very small modifications – you can check them out in the attached project. I’ll explain my thoughts.

    The first change I made was just leaving the OpenSSL path empty. I figure that the majority of people probably have OpenSSL set in their PATH and this then leaves one less piece of configuring for those people.

    My second change was where you determine the temp directory. I modified this a bit for 2 reasons. The first was that I was getting an error because of concatenating a forward slash onto a nil. The second thing I did was default to ‘/tmp/’ for people on POSIX systems if the directory exists with read and write access. I did the check this way first rather than using an environment variable because some systems (Ubuntu for example) don’t come with any sort of temp directory variable. Of course Windows and OSX both include such a variable.

    My third change was just resolving the absolute path on the key locations. The reason why I did this was because relative paths on Unix-like operating systems like “~” (Note that that’s a tilde. For some reason my font seems to make this look like a dash) weren’t resolvable otherwise.

    The final change I made was just adding some instruction comments at the top and pulling out the key paths to their own variables.

    There’s one change I realize needs some work for (hopefully) full OS ambiguity – this is the way the script handles newline characters. It’s currently set to look for a \r but I figure it would be nice to think of a way to handle \n or \r\n as well.

    My first though that was it would be easy to gsub newlines out to an expected type and then gsub again for any double occurrences. i.e. “\r\n” could be gsubbed to “\n\n” and then gsubbed once more to “\n”. Thoughts?

    Thanks Garry – great project by the way. I’ve had fun tinkering with it the last few days.

    You must be logged in to view attached files.

    In regards to the newlines I figure it could look something like this in main where sSMDXML gets set. This way it handles any number of carriage returns and newlines.

    Data = Data:gsub("\r+", "\n"):gsub("\n+", "\n")
    local sSMDXML =, Data:split('\n')[1]:split('|')[12])

    Thanks Wade. I think all your modifications are worthwhile.

    As for the new line characters, there’s only 2 places that it really needs to be considered. The first is in the SMDDeliver function call where the HL7 version is passed. This is assuming that the payload is a HL7 message as is the case with the Secure Message Delivery standard. There should be no OS implications as CR is always the segment separator in the HL7 message.

    The other place the CR needs to be managed is when the payload element is canonicalised. The standard says that all CRs should be replaced with LFs. The result is only used to calculate the hash so it shouldn’t factor anywhere else.

    Thanks for looking over the project and trying it out on different platforms.

    Garry I think you’re right about the line feeds and it’s best to just assume messages coming in will be valid HL7.

    I actually forgot to mention one other change I made in my previous post. I got rid of the version check first so I could run it on Mavericks, and then I left it out simply because it looks like just comparing version numbers doesn’t necessarily tell you which version is newer with OpenSSL. For example 0.9.8y (which ships with Mavericks) was released in 2013, but 1.0.1 was first released in 2012.

You must be logged in to reply to this topic.