Occasionally, we receive Iguana memory leak support inquiries. Here are some of the strategies that can help you to isolate and troubleshoot.
To determine whether there is a memory leak issue, try to observe:
- Iguana Instance’s Memory (found on the dashboard)
- Iguana Crash due to memory leak (found in the crash dump files)
- Slow performance on Iguana (observe slow performance)
And ask the following questions:
- Has this memory usage resulted in Iguana crashing?
- How many channels are on this Iguana instance, and how many are running concurrently?
- What kind of workflows are these channels implementing?
- Has the Iguana service been restarted recently, and if so, has the problem still persisted?
- What operating system is Iguana running on, and what is the exact version of Iguana
- Memory usage should not be increasing:
- Do you see in your code any global variables that should be declared local variables?
- Also, was there any code changes made immediately before the issue was noticed?
- Are there any sheduled or frequent tasks that may not close connections correctly?
Most of time memory leak issues happen on a single or a few channels and sometimes is difficult or impossible to recreate memory leak issues on DEV or TEST due to environment specific variables or message volume.
Here are the suggested troubleshooting techniques that you can use:
- Create backup Iguana instances on the same production server (PROD for example)
- Identify the leaking channel: Turn on and off individual channels on the Iguana Backup instance which channel is causing the memory leak issue:
- Check the global variable in order to make sure that there is no calls that store information in Iguana
- Observe the setmetatable lua function to make sure that you absolutely need that piece of code
- Third party lua module or functions that save and store information on Iguana
- Non persistent database connections can cause a memory leak:
Note: A persistent database connection is when there is a single connection that is re-used for all database queries. This is implemented by opening a single DB connection in a translator channel. The connection is initialized outside the
main()function — so it only runs once when the channel is started, and is closed when the channel is stopped. All queries in the channel will use the same connection. This is very efficient as it reduces database connections/disconnections to the absolute minimum.
A non-persistent connection is opened and closed inside
main()or another function and runs every time a message is processed (or potentially several times per message if several queries are required to process each message). This is much less efficient as every query requires a database connection/disconnection.
- Check the code to see if the database connections are persistent or not.
- If DB connections are not persistent then you can modify the code to use a persistent connection: