Sometimes our clients will get a “too many open files” error on busy servers. This error occurs when a large number of files, databases or web service connections are open – this usually occurs because connections are not closed promptly.
The maximum number of files you can open for a Windows Iguana Server is 512 and 1024 on Linux or Mac – if you attempt to exceed these numbers you will this error.
Note: This page is about the mindset of limiting the number of connections that you use – rather than specific techniques. The specific techniques may vary depending on what you need to do in each case – but the principle of keeping connections to a minimum remains the same.
Too many file, database or web service connections are open at one time.
- Inspect the code to ensure that it closes connections immediately it has finished using them.
- Check that the code opens the minimum number of connections required:
- If you are processing a large number of files you can open one or a group of several at a time rather than all of them. Then you can close the first group before processing the next group.
- You might set an upper limit on connections and not open more when the limit is reached. Then when the number of connections drops you can open more.
- In particular you need to ensure that persistent connections are kept to a minimum:
- Persistent connections to resources that you are re-using all the time make good sense – they are particularly useful for databases
- This page explains about persistent database connections: https://help.interfaceware.com/v6/database-connection
- Make sure that you are not continuing to open extra or duplicate persistent connections unintentionally (i.e., logic error that opens multiple database connections).
- Check that the code does not allow the number of connections to continually increase:
- Make sure connections are closed as soon as possible.
- Set limits on the maximum number of connections.
- In the unlikely situation where you actually need more files open than the limits allow you can split the processing between several Iguana servers.
Warning: Experienced Linux admins will know that it is possible to change the number of files (sockets) available for a process. Do not increase this number above 1024 as this will cause problems for Iguana.
Iguana uses the select library on Linux and Mac – select has a maximum of 1024 files (sockets).
There are various ways to prevent this issue:
- Open the minimum number of connections.
- Close connections immediately when you are finished using them.
- Set limits on the maximum number of connections that you allow.
- Ensure that your code does not allow a continually increasing number of connections.
- If you need to open an extremely large number of files you can split the processing across multiple Iguana instances.