Evaluating approaches

We considered two approaches for improving the effective message throughput of Iguana when acting as a web client. Approach #1 was an attempt to close sockets quickly and avoid reaching operating-system-imposed limits on the number of open sockets possible. Approach #2 concentrated on making sure socket reuse was in effect. The Iguana HTTP client ran on Mac OSX. The HTTP server was Apache 2.2.25, running on Windows. Five variations of web client scenarios are described below:

1. Close sockets quickly using ‘Connection=close’

The HTTP client header ‘Connection = close’ was set in the Iguana Translator script.

When the test was run, Iguana queued up 30,000 messages to send to the web server. Iguana started to report errors at ~ 16,000 messages. Netstat output showed 16,000 sockets in a TIME_WAIT state. After about 2 minutes, netstat output showed that all sockets had been closed.

The results show that Iguana works within the operating-system-imposed handle limit, but it can’t continue once that limit is reached. The operating system cleans up closed sockets after a 2-minute time delay.

2. Close sockets quickly using ‘Connection=close’ and ‘TcpTimedWaitDelay=30’

The HTTP client header ‘Connection = close’ was set in the Iguana Translator script. We also set the registry entry ‘TcpTimedWaitDelay’ to 30 seconds, and the machine was re-booted.

When the test was run, Iguana queued up 1,000 messages to send to the web server. The messages were delivered to the web server in under 3 seconds. The Iguana process and the web server process were both exited. Netstat reported approximately 1000 sockets in a TIMED_WAIT. After 30 seconds, netstat reported 2 sockets open. No errors were reported. The results show that cleaning up closed sockets is the responsibility of the operating system, and that the timing can be configured.

Note that a second registry entry (‘MaxUserPort’) can also be increased in support of having a large number of connections to a server.

3. Reuse sockets by using ‘Connection=Keep-Alive’

The HTTP client header ‘Connection = Keep-Alive’ was set in the Iguana Translator script. 100,000 messages were sent and results recorded. The sending of 100,000 messages was iterated until there were 10 sets of results. The time to completely send one batch and note the socket count was observed on the Windows machine by watching the Apache access.log, and by using netstat.

Iteration #	Time to Send 100,000 Messages (in Seconds)	Total Number of Sockets at End of Iteration (netsat report)
1	69	330
2	79	357
3	80	399
4	85	355
5	94	318
6	101	291
7	110	300
8	114	241
9	123	220
10	129	244
Average	98	306

• Average “Messages per Second” were calculated as 100,000 messages over 98 seconds = 1020 messages per second
• 100,000 messages over 306 sockets = 327 messages per socket
• Netstat reported that on average, 306 sockets were open. No errors were reported.

These results show that Iguana could send 100,000 messages without interruption at very high message throughput, and that more than one message could be sent on a single socket.

4. Reuse sockets by using ‘Connection=Keep-Alive’ and running two channels

The HTTP client header ‘Connection = Keep-Alive’ was set in the Iguana Translator script. Two identical channels were run concurrently, sending 250,000 messages through each channel.

Interval #	Elapsed Time Recorded At Every 100,000 messages sent Interval	Total Number of Sockets at End of Interval (nestat report)
1	50	616
2	109	536
3	150	793
4	194	744
5	240	609
Average	n/a	660

• 500,000 messages were sent in 240 seconds
• “Messages per Second” were calculated as 500,000 messages over 240 seconds = 2,083 messages per second
• Netstat reported that on average, 660 sockets were open

These results show the following:

• Iguana could send 500,000 messages without interruption at very high message throughput
• More than one message could be sent on a single socket
• Channels could operate concurrently for improved message throughput

5. Reuse Sockets by using ‘Connection=Keep-Alive’ and by using POST instead of GET

The HTTP client header ‘Connection = Keep-Alive’ was set in the Iguana Translator script.

The HTTP POST verb was used instead of GET, and 100,000 POST Requests were sent.

Iteration #	Time Interval for Iteration in Seconds	Netstat Reported Total Number of Sockets at End of Interation
1	70	448

• “Messages per second” calculated as 100,000 messages over 70 seconds = 1,428 messages per second
• Netstat reported that the number of sockets in use peaked at 448

These results show that GET and POST had a similar timing.

Conclusion

Both the Iguana web client and the third party web server participating in an HTTP 1.1 conversation have to be configured to reuse sockets with ‘Connection=Keep-Alive’ to achieve high performance under heavy message load.