“Lua” (pronounced LOO-ah) means “Moon” in Portuguese.
It is designed, implemented, and maintained by a team at PUC-Rio, the Pontifical Catholic University of Rio de Janeiro in Brazil. Lua was born and raised in Tecgraf, the Computer Graphics Technology Group of PUC-Rio, and is now housed at Lablua. Both Tecgraf and Lablua are laboratories of the Department of Computer Science of PUC-Rio.

The official Lua website is at:

http://www.lua.org

Lua has well deserved reputation for being very small and fast. It was specifically designed to be an embeddable language rather like TCL. It is the most widely used embedded scripting language in game development. Mike Pall the author of LuaJIT explains some of its key benefits. It can fit within 256k per instance.

A lot of games in the iPhone and iPad use Lua because it has high performance and a small footprint. Some well known applications of Lua include:

• World of Warcraft scripting
• Adobe Lightroom
• Tap tap revenge on the iPhone
• The Logitech harmony remote
• Wireshark

For more examples of where Lua is used see this list.

Different APIs [top]

Normally in the development of Iguana and Chameleon we go out of our way to try and preserve backward compatibility. No one enjoys upgrading interfaces. The Translator on the other hand is a complete re-think of our mapping technology.

With completely different APIs there was no benefit in staying with the Python scripting language used in Chameleon.

It made more sense to pick the best language available for the Translator.

Will all my Chameleon based vmd mapping files still work in Iguana 5.5, i.e. is vmd backwards compatible?

Absolutely 100% positively yes.

Dynamic vs static languages [top]

Lua is a dynamic language. This means that a variable can point to any type of object at runtime. Other dynamic languages are python, ruby, perl and javascript (ECMA script).

This is different to static languages like C#, Java and C++. These languages define the type of an object at compile time.

Historically static languages compiled into machine code would run a lot faster than dynamic languages. However with advances in Just In Time (JIT) compiler technology this distinction has become less important. In the Iguana Translator for instance we make use of the LuaJIT implementation of Lua.

Dynamic languages are usually faster to develop in than their strongly typed cousins. This is because most static languages:

• Have a compile and linking cycle that is quite slow.
• Require defining classes and objects explicitly before you can use them.

Usually one has to write a lot more code in a static language than a dynamic language to achieve the same tasks.

Dynamic languages require a lot more regression testing when the logic becomes complex – because bugs only become apparent at runtime. Auto-completion is more of challenge since an editor cannot tell the type of an object from the code. To see why consider this C++ function vs an equivalent Lua function:

void HelloWorld(const string& Message){
   // do stuff
}

vs in Lua:

function HelloWorld(Message)
   -- do stuff
end

In C++ we know that the Message variable will always have the type “string”. It is defined at compile time. In Lua the Message variable could contain anything. We can only know what it contains by running the code.

That is exactly how we solve the problem in the Translator. It is a technique which only works because of the nature of middleware – we always know what the input data is. At the same time this resolves the major headache with dynamic languages – regression testing – our built in sample data provides the perfect regression testing framework to maintain code written in a dynamic language.

Because of that the Translator provides an exceptionally productive environment for programming deterministic business logic. i.e. data in–> data out in a fraction of a second.

Lua vs Python [top]

Lua has a simpler and more elegant design than Python.

It starts by having a small type system. There are only eight core types in Lua nil, boolean, number, string, userdata, function, thread and table. See http://www.lua.org/pil/2.html a minimal type system keeps the implementation simple and efficient.

The only complex data structure in Lua is a table. They have a dual nature as a hash table or an array depending on how they are used. Tables are the foundation of everything in Lua. A Lua program is stored in memory as a Lua table. Each program function is an entry in this table. To implement objects in Lua you use tables. It is similar to the way javascript objects are constructed.

Python has a very large built in type system. It has objects that represent dates, arrays, tables, slice objects, objects etc. That type system is not static either. It keeps on growing with successive releases of Python. All that adds up to a lot of surface area and complexity.

Lua was built for portability and is designed to have a small foot print. The standard library is small and has clean interfaces. Lua does not include a Regex library since:

Unlike several other scripting languages, Lua does not use POSIX regular expressions (regexp) for pattern matching. The main reason for this is size: A typical implementation of POSIX regexp takes more than 4,000 lines of code. This is bigger than all Lua standard libraries put together. In comparison, the implementation of pattern matching in Lua has less than 500 lines. Of course, the pattern matching in Lua cannot do all that a full POSIX implementation does. Nevertheless, pattern matching in Lua is a powerful tool and includes some features that are difficult to match with standard POSIX implementations.

In practice the Lua string matching library does everything required for transformation code. It is compact, powerful and unicode compatible. Having this standard pattern matching library built into Lua is a big advantage.

Lua’s C APIs reflect its internal design. The Lua API is very small and well thought compared to Python. One is never given a reference to a Lua object. This is different to most scripting language APIs like JNI and Python that give reference counted pointers to internal objects. Instead all interaction with the Lua virtual machine is done via a virtual “Stack” – all the operations involve pushing and pulling values off this stack. It is safe by design.

Lua has very few external dependencies. The core is very small. Python pulls in a heap of external libraries. It is impossible to run a Python interpreter without linking in a lot of libraries and including many core Python library files. Upgrading Python is a difficult exercise – in my experience it takes 2-3 weeks to apply all the patches, tweak the various flags and switch off undesired behavior to install a new version of Python.

Lua is built for efficiency. Whereas Python, for purity, reference counts the Python “None” object which is equivalent to nil in Python. This is something that has caused headaches for us in the past.

Lua interpreters are completely independent. There are no issues with running independent Lua interpreters on separate threads. This is in contrast to the GIL for Python.

Lua has an extremely clean simple design and a small API. I think this is the reason that it has the world’s fastest JIT implementation for a dynamic scripting language. Lua is extremely popular within the gaming market because of its speed (see also speed compared to python).

One of the challenges moving forward with Python is that Python 3.0 is not backwards compatible with Python 2.x. So we were faced with an unpleasant situation of needing to figure out how to support two versions of Python. This is not something that anyone is likely to thank us for.

Fortunately by going down this route we can freeze our Python support, we will continue supporting and patching – but no big changes will be made. Which is ideal since no one likes to rewrite interfacing code.

Lua vs JavaScript [top]

I like Javascript and we considered it instead of Lua. It is just not as simple or as fast as Lua.

If you know Javascript well then you will not any trouble picking up Lua. Both languages derive a lot of their ideas from the Scheme language. They both support features like closures.

Javascript is just a little more complicated.

I would agree with Douglas Crockford’s statement about Javascript that : “It never got a trial period in which it could be corrected and polished based on actual use. The language is powerful and flawed.”

Lua had a much more gentle development curve and in my opinion is better for it.

Douglas Crockford the guy responsible for popularizing JSON.

Lua is simple and fast [top]

The fact that Lua is simple and fast is not a coincidence.

Being simple has lots of benefits:

• Easy to learn. Every technical person in my company has picked it up within days.
• Great for a team – no complicated language features to remember.

Being simple is a big reason for why the Lua JIT implementation is the fastest just in time compiler for any dynamic language in the world. It gets close to the performance of C (which I would not recommend as a choice to write interfacing code in).

There is interesting comment in a discussion on JIT compilers by Brendan Eich the inventor of Javascript on the fact that Lua’s simplicity is a a big driver in making it possible for LuaJIT to achieve the speeds that it does.

Less features – less code to write – less code to optimize.

Result – one genius – in this case Mike Pall was able to implement it by himself.

He is undoubtedly a very smart guy but it would have been much harder to pull off with a more complex language. Compare that with results of the unladen swallow project.

It is the simplicity of Lua that played a big part in us actually managing to deliver the Iguana Translator platform.

Historical perspective [top]

The Iguana Translator project got its start in 2010 when I was working on an abstraction library to expose C code generically to many different scripting languages.

Originally my intent was just to make a better programming environment for the next mapper. Which would be natively hosted within Iguana.

I spent over four weeks wrestling with the Python APIs and was about to call it a day. I decided it would be enough to just support Python for now – since in theory my interface could be extended to support other languages. The process had been so painful I could not imagine repeating it just to support other languages. It is better to support one language well than multiple languages badly.

Then I begrudgingly agreed I should try supporting Lua with my library just to test my abstraction.

In two days I achieved what had taken me four weeks with the python APIs!

I ditched my abstraction and stopped looking at other APIs after that.

I was sold.

Lua was clearly an order of magnitude better in the quality of its design. Lua with its total focus on minimalist design has been a source of inspiration throughout the development of the Translator project.

Eliot Muir – CEO of iNTERFACEWARE