Prevent the “Chunk has too many syntax levels” error when creating a Very Large string

Introduction

When you generate a long string using the Lua concatenation operator “..”, you can get this error Chunk has too many syntax levels. It seems that the error occurs after the 218th instance of the “..” operator in a single string assignment (but you should not rely on this as a fixed limit).

This error should be quite rare — as you will not often need to concatenate 218 or more items into a single string. However one case where it can occur is if you are creating a very complex SQL query that uses hundreds of fields. In fact this error originally occurred when generating a very complex SQL INSERT statement for a CCDA.

Here you can see the error after the 218th instance of the “..” operator:

concatenation error

The length of the concatenated strings has no effect the error still occurs after 218th instance:

concatenation error

 Task [top]

Demonstrate how to create a very long string without a concatenation error.

Implementation [top]

We place the strings into a table and concatenate them using the table.concat() function.

Follow these steps to try the example code:

  1. Create a new channel to try out the code:
    • Create a channel like this:
      • Name: How to create a large string (or similar)
      • Source: From Translator
      • Destination: To Channel
    • Alternatively use an existing channel:

      Your channel will need a From/To Translator or a Filter component.

  2. Load the code into your channel:
    • If you are using a new channel then load this Howtocreatealargestring-FromTranslator project zip file:

      You can load the project zip file into a From/To Translator or a Filter component. This will overwrite the code in your main module — you may not want to do this with an existing channel.

    • Alternatively for an existing channel you can paste the code from below into the main module:

      This allows you to paste the code without overriding the code in your main module. This is useful if you are using an existing channel and you want to modify the existing code.

  3. Try out the code to see how it works — it is very simple it just presents the table.concat() function to prevent the error.
  4. Adapt it to your requirements.

Code [top]

This is the example code for the main module:

function main()
   
   local myString = 
   -- uncomment the next line to produce the error
   --'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'.. 
   'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..'x'..
   'x'
   
   
   trace(#myString, myString)
   
   -- SOLUTION: place the strings in a table and use the table.concat() function
   -- as you can see the string is much longer than the one that causes the error
   myTable = 
   {
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x','x','x','x','x','x','x','x','x','x','x','x','x','x','x','x',
   'x'
   }
   trace(myTable)
   myString = table.concat(myTable)
   trace(#myString, myString)
   
end

How it works [top]

The code is very simple it just uses the table.concat() function for preventing the error.

  1. You can uncomment line 8 to produce the error
  2. The solution is placing the strings to be concatenated into a table and using the table.concat() function:

    Placing the strings into a table avoids the error — because there is no limit on the number of fields in a table. You then use the table.concat() API function to combine all the table fields into a string.

     

More information [top]

 

Tagged: