feature: added backlog queueing to cosocket connection pools.

Now we can pass 'pool_size' and 'backlog' options in connect method to
specify the pool size and the queue size of the connection pool.

When there is more connect operations than the pool_size, new connect
operation will be queued until former connect operation fails or
connection finalizes.

When there is more queueing connect operations than the backlog, new
connect operation will be dropped.

This pr also migrated connection pool creation to connect method.  Note
that the meaning of `active_connections` is changed from the previous
setkeepalive mechanism. The connection pool is destroyed only if all
related connections closed now.

Signed-off-by: Thibault Charbonnier <thibaultcha@me.com>

Author: spacewander

README.markdown

@@ -7091,6 +7091,43 @@ An optional Lua table can be specified as the last argument to this method to sp
 * `pool`
 	specify a custom name for the connection pool being used. If omitted, then the connection pool name will be generated from the string template `"<host>:<port>"` or `"<unix-socket-path>"`.
 
+* `pool_size`
+	specify the size of the connection pool. If omitted and no
+	`backlog` option was provided, no pool will be created. If omitted
+	but `backlog` was provided, the pool will be created with a default
+	size equal to the value of the [lua_socket_pool_size](#lua_socket_pool_size)
+	directive.
+	The connection pool holds up to `pool_size` alive connections
+	ready to be reused by subsequent calls to [connect](#tcpsockconnect), but
+	note that there is no upper limit to the total number of opened connections
+	outside of the pool. If you need to restrict the total number of opened
+	connections, specify the `backlog` option.
+	When the connection pool would exceed its size limit, the least recently used
+	(kept-alive) connection already in the pool will be closed to make room for
+	the current connection.
+	Note that the cosocket connection pool is per Nginx worker process rather
+	than per Nginx server instance, so the size limit specified here also applies
+	to every single Nginx worker process. Also note that the size of the connection
+	pool cannot be changed once it has been created.
+	This option was first introduced in the `v0.10.14` release.
+
+* `backlog`
+	if specified, this module will limit the total number of opened connections
+	for this pool. No more connections than `pool_size` can be opened
+	for this pool at any time. If the connection pool is full, subsequent
+	connect operations will be queued into a queue equal to this option's
+	value (the "backlog" queue).
+	If the number of queued connect operations is equal to `backlog`,
+	subsequent connect operations will fail and return `nil` plus the
+	error string `"too many waiting connect operations"`.
+	The queued connect operations will be resumed once the number of connections
+	in the pool is less than `pool_size`.
+	The queued connect operation will abort once they have been queued for more
+	than `connect_timeout`, controlled by
+	[settimeouts](#tcpsocksettimeouts), and will return `nil` plus
+	the error string `"timeout"`.
+	This option was first introduced in the `v0.10.14` release.
+
 The support for the options table argument was first introduced in the `v0.5.7` release.
 
 This method was first introduced in the `v0.5.0rc1` release.
@@ -7422,13 +7459,31 @@ Puts the current socket's connection immediately into the cosocket built-in conn
 
 The first optional argument, `timeout`, can be used to specify the maximal idle timeout (in milliseconds) for the current connection. If omitted, the default setting in the [lua_socket_keepalive_timeout](#lua_socket_keepalive_timeout) config directive will be used. If the `0` value is given, then the timeout interval is unlimited.
 
-The second optional argument, `size`, can be used to specify the maximal number of connections allowed in the connection pool for the current server (i.e., the current host-port pair or the unix domain socket file path). Note that the size of the connection pool cannot be changed once the pool is created. When this argument is omitted, the default setting in the [lua_socket_pool_size](#lua_socket_pool_size) config directive will be used.
-
-When the connection pool exceeds the available size limit, the least recently used (idle) connection already in the pool will be closed to make room for the current connection.
-
-Note that the cosocket connection pool is per Nginx worker process rather than per Nginx server instance, so the size limit specified here also applies to every single Nginx worker process.
-
-Idle connections in the pool will be monitored for any exceptional events like connection abortion or unexpected incoming data on the line, in which cases the connection in question will be closed and removed from the pool.
+The second optional argument `size` is considered deprecated since
+the `v0.10.14` release of this module, in favor of the
+`pool_size` option of the [connect](#tcpsockconnect) method.
+Since the `v0.10.14` release, this option will only take effect if
+the call to [connect](#tcpsockconnect) did not already create a connection
+pool.
+When this option takes effect (no connection pool was previously created by
+[connect](#tcpsockconnect)), it will specify the size of the connection pool,
+and create it.
+If omitted (and no pool was previously created), the default size is the value
+of the [lua_socket_pool_size](#lua_socket_pool_size) directive.
+The connection pool holds up to `size` alive connections ready to be
+reused by subsequent calls to [connect](#tcpsockconnect), but note that there
+is no upper limit to the total number of opened connections outside of the
+pool.
+When the connection pool would exceed its size limit, the least recently used
+(kept-alive) connection already in the pool will be closed to make room for
+the current connection.
+Note that the cosocket connection pool is per Nginx worker process rather
+than per Nginx server instance, so the size limit specified here also applies
+to every single Nginx worker process. Also note that the size of the connection
+pool cannot be changed once it has been created.
+If you need to restrict the total number of opened connections, specify both
+the `pool_size` and `backlog` option in the call to
+[connect](#tcpsockconnect).
 
 In case of success, this method returns `1`; otherwise, it returns `nil` and a string describing the error.
 

doc/HttpLuaModule.wiki

@@ -5979,6 +5979,43 @@ An optional Lua table can be specified as the last argument to this method to sp
 * <code>pool</code>
 : specify a custom name for the connection pool being used. If omitted, then the connection pool name will be generated from the string template <code>"<host>:<port>"</code> or <code>"<unix-socket-path>"</code>.
 
+* <code>pool_size</code>
+: specify the size of the connection pool. If omitted and no
+: <code>backlog</code> option was provided, no pool will be created. If omitted
+: but <code>backlog</code> was provided, the pool will be created with a default
+: size equal to the value of the [[#lua_socket_pool_size|lua_socket_pool_size]]
+: directive.
+: The connection pool holds up to <code>pool_size</code> alive connections
+: ready to be reused by subsequent calls to [[#tcpsock:connect|connect]], but
+: note that there is no upper limit to the total number of opened connections
+: outside of the pool. If you need to restrict the total number of opened
+: connections, specify the <code>backlog</code> option.
+: When the connection pool would exceed its size limit, the least recently used
+: (kept-alive) connection already in the pool will be closed to make room for
+: the current connection.
+: Note that the cosocket connection pool is per Nginx worker process rather
+: than per Nginx server instance, so the size limit specified here also applies
+: to every single Nginx worker process. Also note that the size of the connection
+: pool cannot be changed once it has been created.
+: This option was first introduced in the <code>v0.10.14</code> release.
+
+* <code>backlog</code>
+: if specified, this module will limit the total number of opened connections
+: for this pool. No more connections than <code>pool_size</code> can be opened
+: for this pool at any time. If the connection pool is full, subsequent
+: connect operations will be queued into a queue equal to this option's
+: value (the "backlog" queue).
+: If the number of queued connect operations is equal to <code>backlog</code>,
+: subsequent connect operations will fail and return <code>nil</code> plus the
+: error string <code>"too many waiting connect operations"</code>.
+: The queued connect operations will be resumed once the number of connections
+: in the pool is less than <code>pool_size</code>.
+: The queued connect operation will abort once they have been queued for more
+: than <code>connect_timeout</code>, controlled by
+: [[#tcpsock:settimeouts|settimeouts]], and will return <code>nil</code> plus
+: the error string <code>"timeout"</code>.
+: This option was first introduced in the <code>v0.10.14</code> release.
+
 The support for the options table argument was first introduced in the <code>v0.5.7</code> release.
 
 This method was first introduced in the <code>v0.5.0rc1</code> release.
@@ -6273,13 +6310,31 @@ Puts the current socket's connection immediately into the cosocket built-in conn
 
 The first optional argument, <code>timeout</code>, can be used to specify the maximal idle timeout (in milliseconds) for the current connection. If omitted, the default setting in the [[#lua_socket_keepalive_timeout|lua_socket_keepalive_timeout]] config directive will be used. If the <code>0</code> value is given, then the timeout interval is unlimited.
 
-The second optional argument, <code>size</code>, can be used to specify the maximal number of connections allowed in the connection pool for the current server (i.e., the current host-port pair or the unix domain socket file path). Note that the size of the connection pool cannot be changed once the pool is created. When this argument is omitted, the default setting in the [[#lua_socket_pool_size|lua_socket_pool_size]] config directive will be used.
-
-When the connection pool exceeds the available size limit, the least recently used (idle) connection already in the pool will be closed to make room for the current connection.
-
-Note that the cosocket connection pool is per Nginx worker process rather than per Nginx server instance, so the size limit specified here also applies to every single Nginx worker process.
-
-Idle connections in the pool will be monitored for any exceptional events like connection abortion or unexpected incoming data on the line, in which cases the connection in question will be closed and removed from the pool.
+The second optional argument <code>size</code> is considered deprecated since
+the <code>v0.10.14</code> release of this module, in favor of the
+<code>pool_size</code> option of the [[#tcpsock:connect|connect]] method.
+Since the <code>v0.10.14</code> release, this option will only take effect if
+the call to [[#tcpsock:connect|connect]] did not already create a connection
+pool.
+When this option takes effect (no connection pool was previously created by
+[[#tcpsock:connect|connect]]), it will specify the size of the connection pool,
+and create it.
+If omitted (and no pool was previously created), the default size is the value
+of the [[#lua_socket_pool_size|lua_socket_pool_size]] directive.
+The connection pool holds up to <code>size</code> alive connections ready to be
+reused by subsequent calls to [[#tcpsock:connect|connect]], but note that there
+is no upper limit to the total number of opened connections outside of the
+pool.
+When the connection pool would exceed its size limit, the least recently used
+(kept-alive) connection already in the pool will be closed to make room for
+the current connection.
+Note that the cosocket connection pool is per Nginx worker process rather
+than per Nginx server instance, so the size limit specified here also applies
+to every single Nginx worker process. Also note that the size of the connection
+pool cannot be changed once it has been created.
+If you need to restrict the total number of opened connections, specify both
+the <code>pool_size</code> and <code>backlog</code> option in the call to
+[[#tcpsock:connect|connect]].
 
 In case of success, this method returns <code>1</code>; otherwise, it returns <code>nil</code> and a string describing the error.