From 91fc078c44cd76ab173ad4e538ff8fd630d71740 Mon Sep 17 00:00:00 2001 From: "pj.vegan" Date: Fri, 7 Oct 2016 10:58:50 +0900 Subject: [PATCH 001/848] doc: use *_by_lua_block {} in the examples. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 222 ++++++++++++++++++++--------------------- doc/HttpLuaModule.wiki | 222 ++++++++++++++++++++--------------------- 2 files changed, 222 insertions(+), 222 deletions(-) diff --git a/README.markdown b/README.markdown index 99b5275329..25c1ded549 100644 --- a/README.markdown +++ b/README.markdown @@ -585,10 +585,10 @@ and then accessing it from `nginx.conf`: ```nginx location /lua { - content_by_lua ' + content_by_lua_block { local mydata = require "mydata" ngx.say(mydata.get_age("dog")) - '; + } } ``` @@ -685,9 +685,9 @@ The [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi] ```nginx location /foo { - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/bar") - '; + } } location /bar { echo_location /blah; @@ -859,9 +859,9 @@ servers in Lua. For example, datagram { server { listen 1953; - handler_by_lua ' + handler_by_lua_block { -- custom Lua code implementing the special UDP server... - '; + } } } ``` @@ -1223,9 +1223,9 @@ Usually you can register (true) Lua global variables or pre-load Lua modules at server { location = /api { - content_by_lua ' + content_by_lua_block { ngx.say(cjson.encode({dog = 5, cat = 6})) - '; + } } } ``` @@ -1243,10 +1243,10 @@ You can also initialize the [lua_shared_dict](#lua_shared_dict) shm storage at t server { location = /api { - content_by_lua ' + content_by_lua_block { local dogs = ngx.shared.dogs; ngx.say(dogs:get("Tom")) - '; + } } } ``` @@ -2062,7 +2062,7 @@ When the Lua code may change the length of the response body, then it is require location /foo { # fastcgi_pass/proxy_pass/... - header_filter_by_lua 'ngx.header.content_length = nil'; + header_filter_by_lua_block { ngx.header.content_length = nil } body_filter_by_lua 'ngx.arg[1] = string.len(ngx.arg[1]) .. "\\n"'; } ``` @@ -2172,7 +2172,7 @@ Here is an example of gathering average data for [$upstream_response_time](http: } location = /status { - content_by_lua ' + content_by_lua_block { local log_dict = ngx.shared.log_dict local sum = log_dict:get("upstream_time-sum") local nb = log_dict:get("upstream_time-nb") @@ -2183,7 +2183,7 @@ Here is an example of gathering average data for [$upstream_response_time](http: else ngx.say("no data yet") end - '; + } } } ``` @@ -2329,15 +2329,15 @@ Determines whether to force the request body data to be read before running rewr To read the request body data within the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable, [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) must have the same value as [client_max_body_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size). Because when the content length exceeds [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) but less than [client_max_body_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size), Nginx will buffer the data into a temporary file on the disk, which will lead to empty value in the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable. -If the current location includes [rewrite_by_lua](#rewrite_by_lua) or [rewrite_by_lua_file](#rewrite_by_lua_file) directives, -then the request body will be read just before the [rewrite_by_lua](#rewrite_by_lua) or [rewrite_by_lua_file](#rewrite_by_lua_file) code is run (and also at the +If the current location includes [rewrite_by_lua*](#rewrite_by_lua) directives, +then the request body will be read just before the [rewrite_by_lua*](#rewrite_by_lua) code is run (and also at the `rewrite` phase). Similarly, if only [content_by_lua](#content_by_lua) is specified, the request body will not be read until the content handler's Lua code is about to run (i.e., the request body will be read during the content phase). It is recommended however, to use the [ngx.req.read_body](#ngxreqread_body) and [ngx.req.discard_body](#ngxreqdiscard_body) functions for finer control over the request body reading process instead. -This also applies to [access_by_lua](#access_by_lua) and [access_by_lua_file](#access_by_lua_file). +This also applies to [access_by_lua*](#access_by_lua). [Back to TOC](#directives) @@ -2859,7 +2859,7 @@ rewrite_by_lua_no_postpone **context:** *http* -Controls whether or not to disable postponing [rewrite_by_lua](#rewrite_by_lua)* directives to run at the end of the `rewrite` request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the `rewrite` phase. +Controls whether or not to disable postponing [rewrite_by_lua*](#rewrite_by_lua) directives to run at the end of the `rewrite` request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the `rewrite` phase. This directive was first introduced in the `v0.5.0rc29` release. @@ -2874,7 +2874,7 @@ access_by_lua_no_postpone **context:** *http* -Controls whether or not to disable postponing [access_by_lua](#access_by_lua)* directives to run at the end of the `access` request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the `access` phase. +Controls whether or not to disable postponing [access_by_lua*](#access_by_lua) directives to run at the end of the `access` request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the `access` phase. This directive was first introduced in the `v0.9.20` release. @@ -3119,7 +3119,7 @@ Nginx API for Lua Introduction ------------ -The various `*_by_lua` and `*_by_lua_file` configuration directives serve as gateways to the Lua API within the `nginx.conf` file. The Nginx Lua API described below can only be called within the user Lua code run in the context of these configuration directives. +The various `*_by_lua`, `*_by_lua_block` and `*_by_lua_file` configuration directives serve as gateways to the Lua API within the `nginx.conf` file. The Nginx Lua API described below can only be called within the user Lua code run in the context of these configuration directives. The API is exposed to Lua in the form of two standard packages `ngx` and `ndk`. These packages are in the default global scope within ngx_lua and are always available within ngx_lua directives. @@ -3160,7 +3160,7 @@ ngx.arg **context:** *set_by_lua*, body_filter_by_lua** -When this is used in the context of the [set_by_lua](#set_by_lua) or [set_by_lua_file](#set_by_lua_file) directives, this table is read-only and holds the input arguments to the config directives: +When this is used in the context of the [set_by_lua*](#set_by_lua) directives, this table is read-only and holds the input arguments to the config directives: ```lua @@ -3185,7 +3185,7 @@ Here is an example that writes out `88`, the sum of `32` and `56`. -When this table is used in the context of [body_filter_by_lua](#body_filter_by_lua) or [body_filter_by_lua_file](#body_filter_by_lua_file), the first element holds the input data chunk to the output filter code and the second element holds the boolean flag for the "eof" flag indicating the end of the whole output data stream. +When this table is used in the context of [body_filter_by_lua*](#body_filter_by_lua), the first element holds the input data chunk to the output filter code and the second element holds the boolean flag for the "eof" flag indicating the end of the whole output data stream. The data chunk and "eof" flag passed to the downstream Nginx output filters can also be overridden by assigning values directly to the corresponding table elements. When setting `nil` or an empty Lua string value to `ngx.arg[1]`, no data chunk will be passed to the downstream Nginx output filters at all. @@ -3212,10 +3212,10 @@ For example: location /foo { set $my_var ''; # this line is required to create $my_var at config time - content_by_lua ' + content_by_lua_block { ngx.var.my_var = 123; ... - '; + } } ``` @@ -3252,7 +3252,7 @@ This API requires a relatively expensive metamethod call and it is recommended t Core constants -------------- -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```lua @@ -3278,7 +3278,7 @@ The `ngx.DECLINED` constant was first introduced in the `v0.5.0rc19` release. HTTP method constants --------------------- -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ngx.HTTP_GET @@ -3304,7 +3304,7 @@ These constants are usually used in [ngx.location.capture](#ngxlocationcapture) HTTP status constants --------------------- -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```nginx @@ -3348,7 +3348,7 @@ HTTP status constants Nginx log level constants ------------------------- -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```lua @@ -3371,7 +3371,7 @@ print ----- **syntax:** *print(...)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Writes argument values into the nginx `error.log` file with the `ngx.NOTICE` log level. @@ -3390,7 +3390,7 @@ There is a hard coded `2048` byte limitation on error message lengths in the Ngi ngx.ctx ------- -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua** This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). @@ -3399,15 +3399,15 @@ Consider the following example, ```nginx location /test { - rewrite_by_lua ' + rewrite_by_lua_block { ngx.ctx.foo = 76 - '; - access_by_lua ' + } + access_by_lua_block { ngx.ctx.foo = ngx.ctx.foo + 3 - '; - content_by_lua ' + } + content_by_lua_block { ngx.say(ngx.ctx.foo) - '; + } } ``` @@ -3425,21 +3425,21 @@ Every request, including subrequests, has its own copy of the table. For example ```nginx location /sub { - content_by_lua ' + content_by_lua_block { ngx.say("sub pre: ", ngx.ctx.blah) ngx.ctx.blah = 32 ngx.say("sub post: ", ngx.ctx.blah) - '; + } } location /main { - content_by_lua ' + content_by_lua_block { ngx.ctx.blah = 73 ngx.say("main pre: ", ngx.ctx.blah) local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("main post: ", ngx.ctx.blah) - '; + } } ``` @@ -3460,16 +3460,16 @@ Internal redirection will destroy the original request `ngx.ctx` data (if any) a ```nginx location /new { - content_by_lua ' + content_by_lua_block { ngx.say(ngx.ctx.foo) - '; + } } location /orig { - content_by_lua ' + content_by_lua_block { ngx.ctx.foo = "hello" ngx.exec("/new") - '; + } } ``` @@ -3671,13 +3671,13 @@ This option is set to `false` by default location /lua { set $dog 'hello'; - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/other", { share_all_vars = true }); ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) - '; + } } ``` @@ -3699,13 +3699,13 @@ The `copy_all_vars` option provides a copy of the parent request's Nginx variabl location /lua { set $dog 'hello'; - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/other", { copy_all_vars = true }); ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) - '; + } } ``` @@ -3728,21 +3728,21 @@ unescaping them in the Nginx config file. ```nginx location /other { - content_by_lua ' + content_by_lua_block { ngx.say("dog = ", ngx.var.dog) ngx.say("cat = ", ngx.var.cat) - '; + } } location /lua { set $dog ''; set $cat ''; - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/other", { vars = { dog = "hello", cat = 32 }}); ngx.print(res.body) - '; + } } ``` @@ -3758,18 +3758,18 @@ The `ctx` option can be used to specify a custom Lua table to serve as the [ngx. ```nginx location /sub { - content_by_lua ' + content_by_lua_block { ngx.ctx.foo = "bar"; - '; + } } location /lua { - content_by_lua ' + content_by_lua_block { local ctx = {} res = ngx.location.capture("/sub", { ctx = ctx }) ngx.say(ctx.foo); ngx.say(ngx.ctx.foo); - '; + } } ``` @@ -3785,15 +3785,15 @@ It is also possible to use this `ctx` option to share the same [ngx.ctx](#ngxctx ```nginx location /sub { - content_by_lua ' + content_by_lua_block { ngx.ctx.foo = "bar"; - '; + } } location /lua { - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/sub", { ctx = ngx.ctx }) ngx.say(ngx.ctx.foo); - '; + } } ``` @@ -3891,7 +3891,7 @@ Please also refer to restrictions on capturing locations configured by [subreque ngx.status ---------- -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** Read and write the current request's response status. This should be called before sending out the response headers. @@ -3916,7 +3916,7 @@ ngx.header.HEADER **syntax:** *value = ngx.header.HEADER* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** Set, add to, or clear the current request's `HEADER` response header that is to be sent. @@ -3983,7 +3983,7 @@ Reading `ngx.header.HEADER` will return the value of the response header named ` Underscores (`_`) in the header names will also be replaced by dashes (`-`) and the header names will be matched case-insensitively. If the response header is not present at all, `nil` will be returned. -This is particularly useful in the context of [header_filter_by_lua](#header_filter_by_lua) and [header_filter_by_lua_file](#header_filter_by_lua_file), for example, +This is particularly useful in the context of [header_filter_by_lua*](#header_filter_by_lua), for example, ```nginx @@ -3992,11 +3992,11 @@ This is particularly useful in the context of [header_filter_by_lua](#header_fil proxy_pass http://some-backend; - header_filter_by_lua ' + header_filter_by_lua_block { if ngx.header["X-My-Header"] == "blah" then ngx.var.footer = "some value" end - '; + } echo_after_body $footer; } @@ -4028,7 +4028,7 @@ ngx.resp.get_headers -------------------- **syntax:** *headers = ngx.resp.get_headers(max_headers?, raw?)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, balancer_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** Returns a Lua table holding all the current response headers for the current request. @@ -4223,17 +4223,17 @@ or equivalently, ngx.req.set_uri("/foo") ``` -The `jump` argument can only be set to `true` in [rewrite_by_lua](#rewrite_by_lua) and [rewrite_by_lua_file](#rewrite_by_lua_file). Use of jump in other contexts is prohibited and will throw out a Lua exception. +The `jump` argument can only be set to `true` in [rewrite_by_lua*](#rewrite_by_lua). Use of jump in other contexts is prohibited and will throw out a Lua exception. A more sophisticated example involving regex substitutions is as follows ```nginx location /test { - rewrite_by_lua ' + rewrite_by_lua_block { local uri = ngx.re.sub(ngx.var.uri, "^/test/(.*)", "/$1", "o") ngx.req.set_uri(uri) - '; + } proxy_pass http://my_backend; } ``` @@ -4316,14 +4316,14 @@ ngx.req.get_uri_args -------------------- **syntax:** *args = ngx.req.get_uri_args(max_args?)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, balancer_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** Returns a Lua table holding all the current request URL query arguments. ```nginx location = /test { - content_by_lua ' + content_by_lua_block { local args = ngx.req.get_uri_args() for key, val in pairs(args) do if type(val) == "table" then @@ -4332,7 +4332,7 @@ Returns a Lua table holding all the current request URL query arguments. ngx.say(key, ": ", val) end end - '; + } } ``` @@ -4412,14 +4412,14 @@ ngx.req.get_post_args --------------------- **syntax:** *args, err = ngx.req.get_post_args(max_args?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** Returns a Lua table holding all the current request POST query arguments (of the MIME type `application/x-www-form-urlencoded`). Call [ngx.req.read_body](#ngxreqread_body) to read the request body first or turn on the [lua_need_request_body](#lua_need_request_body) directive to avoid errors. ```nginx location = /test { - content_by_lua ' + content_by_lua_block { ngx.req.read_body() local args, err = ngx.req.get_post_args() if not args then @@ -4433,7 +4433,7 @@ Returns a Lua table holding all the current request POST query arguments (of the ngx.say(key, ": ", val) end end - '; + } } ``` @@ -4514,7 +4514,7 @@ ngx.req.get_headers ------------------- **syntax:** *headers = ngx.req.get_headers(max_headers?, raw?)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** Returns a Lua table holding all the current request headers. @@ -4588,7 +4588,7 @@ ngx.req.set_header ------------------ **syntax:** *ngx.req.set_header(header_name, header_value)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua* +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua** Set the current request's request header named `header_name` to value `header_value`, overriding any existing ones. @@ -4805,7 +4805,7 @@ The usage of this function is often like this: ngx.req.finish_body() ``` -This function can be used with [ngx.req.append_body](#ngxreqappend_body), [ngx.req.finish_body](#ngxreqfinish_body), and [ngx.req.socket](#ngxreqsocket) to implement efficient input filters in pure Lua (in the context of [rewrite_by_lua](#rewrite_by_lua)* or [access_by_lua](#access_by_lua)*), which can be used with other Nginx content handler or upstream modules like [ngx_http_proxy_module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [ngx_http_fastcgi_module](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). +This function can be used with [ngx.req.append_body](#ngxreqappend_body), [ngx.req.finish_body](#ngxreqfinish_body), and [ngx.req.socket](#ngxreqsocket) to implement efficient input filters in pure Lua (in the context of [rewrite_by_lua*](#rewrite_by_lua) or [access_by_lua*](#access_by_lua)), which can be used with other Nginx content handler or upstream modules like [ngx_http_proxy_module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [ngx_http_fastcgi_module](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). This function was first introduced in the `v0.5.11` release. @@ -4823,7 +4823,7 @@ When the data can no longer be hold in the memory buffer for the request body, t It is important to always call the [ngx.req.finish_body](#ngxreqfinish_body) after all the data has been appended onto the current request body. -This function can be used with [ngx.req.init_body](#ngxreqinit_body), [ngx.req.finish_body](#ngxreqfinish_body), and [ngx.req.socket](#ngxreqsocket) to implement efficient input filters in pure Lua (in the context of [rewrite_by_lua](#rewrite_by_lua)* or [access_by_lua](#access_by_lua)*), which can be used with other Nginx content handler or upstream modules like [ngx_http_proxy_module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [ngx_http_fastcgi_module](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). +This function can be used with [ngx.req.init_body](#ngxreqinit_body), [ngx.req.finish_body](#ngxreqfinish_body), and [ngx.req.socket](#ngxreqsocket) to implement efficient input filters in pure Lua (in the context of [rewrite_by_lua*](#rewrite_by_lua) or [access_by_lua*](#access_by_lua)), which can be used with other Nginx content handler or upstream modules like [ngx_http_proxy_module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [ngx_http_fastcgi_module](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). This function was first introduced in the `v0.5.11` release. @@ -4839,7 +4839,7 @@ ngx.req.finish_body Completes the construction process of the new request body created by the [ngx.req.init_body](#ngxreqinit_body) and [ngx.req.append_body](#ngxreqappend_body) calls. -This function can be used with [ngx.req.init_body](#ngxreqinit_body), [ngx.req.append_body](#ngxreqappend_body), and [ngx.req.socket](#ngxreqsocket) to implement efficient input filters in pure Lua (in the context of [rewrite_by_lua](#rewrite_by_lua)* or [access_by_lua](#access_by_lua)*), which can be used with other Nginx content handler or upstream modules like [ngx_http_proxy_module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [ngx_http_fastcgi_module](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). +This function can be used with [ngx.req.init_body](#ngxreqinit_body), [ngx.req.append_body](#ngxreqappend_body), and [ngx.req.socket](#ngxreqsocket) to implement efficient input filters in pure Lua (in the context of [rewrite_by_lua*](#rewrite_by_lua) or [access_by_lua*](#access_by_lua)), which can be used with other Nginx content handler or upstream modules like [ngx_http_proxy_module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [ngx_http_fastcgi_module](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). This function was first introduced in the `v0.5.11` release. @@ -4914,20 +4914,20 @@ Named locations are also supported but the second `args` argument will be ignore ```nginx location /foo { - content_by_lua ' + content_by_lua_block { ngx.exec("@bar", "a=goodbye"); - '; + } } location @bar { - content_by_lua ' + content_by_lua_block { local args = ngx.req.get_uri_args() for key, val in pairs(args) do if key == "a" then ngx.say(val) end end - '; + } } ``` @@ -4937,7 +4937,7 @@ it is purely an internal redirect and that no new external HTTP traffic is invol Also note that this method call terminates the processing of the current request and that it *must* be called before [ngx.send_headers](#ngxsend_headers) or explicit response body outputs by either [ngx.print](#ngxprint) or [ngx.say](#ngxsay). -It is recommended that a coding style that combines this method call with the `return` statement, i.e., `return ngx.exec(...)` be adopted when this method call is used in contexts other than [header_filter_by_lua](#header_filter_by_lua) to reinforce the fact that the request processing is being terminated. +It is recommended that a coding style that combines this method call with the `return` statement, i.e., `return ngx.exec(...)` be adopted when this method call is used in contexts other than [header_filter_by_lua*](#header_filter_by_lua) to reinforce the fact that the request processing is being terminated. [Back to TOC](#nginx-api-for-lua) @@ -5019,7 +5019,7 @@ URI arguments can be specified as well, for example: Note that this method call terminates the processing of the current request and that it *must* be called before [ngx.send_headers](#ngxsend_headers) or explicit response body outputs by either [ngx.print](#ngxprint) or [ngx.say](#ngxsay). -It is recommended that a coding style that combines this method call with the `return` statement, i.e., `return ngx.redirect(...)` be adopted when this method call is used in contexts other than [header_filter_by_lua](#header_filter_by_lua) to reinforce the fact that the request processing is being terminated. +It is recommended that a coding style that combines this method call with the `return` statement, i.e., `return ngx.redirect(...)` be adopted when this method call is used in contexts other than [header_filter_by_lua*](#header_filter_by_lua) to reinforce the fact that the request processing is being terminated. [Back to TOC](#nginx-api-for-lua) @@ -5034,7 +5034,7 @@ Explicitly send out the response headers. Since `v0.8.3` this function returns `1` on success, or returns `nil` and a string describing the error otherwise. Note that there is normally no need to manually send out response headers as ngx_lua will automatically send headers out -before content is output with [ngx.say](#ngxsay) or [ngx.print](#ngxprint) or when [content_by_lua](#content_by_lua) exits normally. +before content is output with [ngx.say](#ngxsay) or [ngx.print](#ngxprint) or when [content_by_lua*](#content_by_lua) exits normally. [Back to TOC](#nginx-api-for-lua) @@ -5145,7 +5145,7 @@ ngx.exit When `status >= 200` (i.e., `ngx.HTTP_OK` and above), it will interrupt the execution of the current request and return status code to nginx. -When `status == 0` (i.e., `ngx.OK`), it will only quit the current phase handler (or the content handler if the [content_by_lua](#content_by_lua) directive is used) and continue to run later phases (if any) for the current request. +When `status == 0` (i.e., `ngx.OK`), it will only quit the current phase handler (or the content handler if the [content_by_lua*](#content_by_lua) directive is used) and continue to run later phases (if any) for the current request. The `status` argument can be `ngx.OK`, `ngx.ERROR`, `ngx.HTTP_NOT_FOUND`, `ngx.HTTP_MOVED_TEMPORARILY`, or other [HTTP status constants](#http-status-constants). @@ -5206,11 +5206,11 @@ When you disable the HTTP 1.1 keep-alive feature for your downstream connections location = /async { keepalive_timeout 0; - content_by_lua ' + content_by_lua_block { ngx.say("got the task!") ngx.eof() -- a descent HTTP client will close the connection at this point -- access MySQL, PostgreSQL, Redis, Memcached, and etc here... - '; + } } ``` @@ -5455,7 +5455,7 @@ For example, ```nginx location = /md5 { - content_by_lua 'ngx.say(ngx.md5("hello"))'; + content_by_lua_block { ngx.say(ngx.md5("hello")) } } ``` @@ -6073,17 +6073,17 @@ Here is an example: lua_shared_dict dogs 10m; server { location /set { - content_by_lua ' + content_by_lua_block { local dogs = ngx.shared.dogs dogs:set("Jim", 8) ngx.say("STORED") - '; + } } location /get { - content_by_lua ' + content_by_lua_block { local dogs = ngx.shared.dogs ngx.say(dogs:get("Jim")) - '; + } } } } @@ -6500,7 +6500,7 @@ Here is an example for connecting to a UDP (memcached) server: location /test { resolver 8.8.8.8; - content_by_lua ' + content_by_lua_block { local sock = ngx.socket.udp() local ok, err = sock:setpeername("my.memcached.server.domain", 11211) if not ok then @@ -6509,7 +6509,7 @@ Here is an example for connecting to a UDP (memcached) server: end ngx.say("successfully connected to memcached!") sock:close() - '; + } } ``` @@ -6696,7 +6696,7 @@ Here is an example for connecting to a TCP server: location /test { resolver 8.8.8.8; - content_by_lua ' + content_by_lua_block { local sock = ngx.socket.tcp() local ok, err = sock:connect("www.google.com", 80) if not ok then @@ -6705,7 +6705,7 @@ Here is an example for connecting to a TCP server: end ngx.say("successfully connected to google!") sock:close() - '; + } } ``` @@ -7086,31 +7086,31 @@ ngx.get_phase Retrieves the current running phase name. Possible return values are * `init` - for the context of [init_by_lua](#init_by_lua) or [init_by_lua_file](#init_by_lua_file). + for the context of [init_by_lua*](#init_by_lua). * `init_worker` - for the context of [init_worker_by_lua](#init_worker_by_lua) or [init_worker_by_lua_file](#init_worker_by_lua_file). + for the context of [init_worker_by_lua*](#init_worker_by_lua). * `ssl_cert` - for the context of [ssl_certificate_by_lua_block](#ssl_certificate_by_lua_block) or [ssl_certificate_by_lua_file](#ssl_certificate_by_lua_file). + for the context of [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block). * `ssl_session_fetch` for the context of [ssl_session_fetch_by_lua*](#ssl_session_fetch_by_lua_block). * `ssl_session_store` for the context of [ssl_session_store_by_lua*](#ssl_session_store_by_lua_block). * `set` - for the context of [set_by_lua](#set_by_lua) or [set_by_lua_file](#set_by_lua_file). + for the context of [set_by_lua*](#set_by_lua). * `rewrite` - for the context of [rewrite_by_lua](#rewrite_by_lua) or [rewrite_by_lua_file](#rewrite_by_lua_file). + for the context of [rewrite_by_lua*](#rewrite_by_lua). * `balancer` - for the context of [balancer_by_lua_block](#balancer_by_lua_block) or [balancer_by_lua_file](#balancer_by_lua_file). + for the context of [balancer_by_lua*](#balancer_by_lua_block). * `access` - for the context of [access_by_lua](#access_by_lua) or [access_by_lua_file](#access_by_lua_file). + for the context of [access_by_lua*](#access_by_lua). * `content` - for the context of [content_by_lua](#content_by_lua) or [content_by_lua_file](#content_by_lua_file). + for the context of [content_by_lua*](#content_by_lua). * `header_filter` - for the context of [header_filter_by_lua](#header_filter_by_lua) or [header_filter_by_lua_file](#header_filter_by_lua_file). + for the context of [header_filter_by_lua*](#header_filter_by_lua). * `body_filter` - for the context of [body_filter_by_lua](#body_filter_by_lua) or [body_filter_by_lua_file](#body_filter_by_lua_file). + for the context of [body_filter_by_lua*](#body_filter_by_lua). * `log` - for the context of [log_by_lua](#log_by_lua) or [log_by_lua_file](#log_by_lua_file). + for the context of [log_by_lua*](#log_by_lua). * `timer` for the context of user callback functions for [ngx.timer.*](#ngxtimerat). @@ -7458,7 +7458,7 @@ Here is a simple example: location / { ... - log_by_lua ' + log_by_lua_block { local function push_data(premature, uri, args, status) -- push the data uri, args, and status to the remote -- via ngx.socket.tcp or ngx.socket.udp @@ -7471,7 +7471,7 @@ Here is a simple example: ngx.log(ngx.ERR, "failed to create timer: ", err) return end - '; + } } ``` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 028a98d96f..dffecc853d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -463,10 +463,10 @@ and then accessing it from nginx.conf: location /lua { - content_by_lua ' + content_by_lua_block { local mydata = require "mydata" ngx.say(mydata.get_age("dog")) - '; + } } @@ -545,9 +545,9 @@ The [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_m location /foo { - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/bar") - '; + } } location /bar { echo_location /blah; @@ -693,9 +693,9 @@ servers in Lua. For example, datagram { server { listen 1953; - handler_by_lua ' + handler_by_lua_block { -- custom Lua code implementing the special UDP server... - '; + } } } @@ -957,9 +957,9 @@ Usually you can register (true) Lua global variables or pre-load Lua modules at server { location = /api { - content_by_lua ' + content_by_lua_block { ngx.say(cjson.encode({dog = 5, cat = 6})) - '; + } } } @@ -976,10 +976,10 @@ You can also initialize the [[#lua_shared_dict|lua_shared_dict]] shm storage at server { location = /api { - content_by_lua ' + content_by_lua_block { local dogs = ngx.shared.dogs; ngx.say(dogs:get("Tom")) - '; + } } } @@ -1707,7 +1707,7 @@ When the Lua code may change the length of the response body, then it is require location /foo { # fastcgi_pass/proxy_pass/... - header_filter_by_lua 'ngx.header.content_length = nil'; + header_filter_by_lua_block { ngx.header.content_length = nil } body_filter_by_lua 'ngx.arg[1] = string.len(ngx.arg[1]) .. "\\n"'; } @@ -1806,7 +1806,7 @@ Here is an example of gathering average data for [[HttpUpstreamModule#$upstream_ } location = /status { - content_by_lua ' + content_by_lua_block { local log_dict = ngx.shared.log_dict local sum = log_dict:get("upstream_time-sum") local nb = log_dict:get("upstream_time-nb") @@ -1817,7 +1817,7 @@ Here is an example of gathering average data for [[HttpUpstreamModule#$upstream_ else ngx.say("no data yet") end - '; + } } } @@ -1946,15 +1946,15 @@ Determines whether to force the request body data to be read before running rewr To read the request body data within the [[HttpCoreModule#$request_body|$request_body]] variable, [[HttpCoreModule#client_body_buffer_size|client_body_buffer_size]] must have the same value as [[HttpCoreModule#client_max_body_size|client_max_body_size]]. Because when the content length exceeds [[HttpCoreModule#client_body_buffer_size|client_body_buffer_size]] but less than [[HttpCoreModule#client_max_body_size|client_max_body_size]], Nginx will buffer the data into a temporary file on the disk, which will lead to empty value in the [[HttpCoreModule#$request_body|$request_body]] variable. -If the current location includes [[#rewrite_by_lua|rewrite_by_lua]] or [[#rewrite_by_lua_file|rewrite_by_lua_file]] directives, -then the request body will be read just before the [[#rewrite_by_lua|rewrite_by_lua]] or [[#rewrite_by_lua_file|rewrite_by_lua_file]] code is run (and also at the +If the current location includes [[#rewrite_by_lua|rewrite_by_lua*]] directives, +then the request body will be read just before the [[#rewrite_by_lua|rewrite_by_lua*]] code is run (and also at the rewrite phase). Similarly, if only [[#content_by_lua|content_by_lua]] is specified, the request body will not be read until the content handler's Lua code is about to run (i.e., the request body will be read during the content phase). It is recommended however, to use the [[#ngx.req.read_body|ngx.req.read_body]] and [[#ngx.req.discard_body|ngx.req.discard_body]] functions for finer control over the request body reading process instead. -This also applies to [[#access_by_lua|access_by_lua]] and [[#access_by_lua_file|access_by_lua_file]]. +This also applies to [[#access_by_lua|access_by_lua*]]. == ssl_certificate_by_lua_block == @@ -2408,7 +2408,7 @@ This directive was first introduced in the v0.5.0rc19 release. '''context:''' ''http'' -Controls whether or not to disable postponing [[#rewrite_by_lua|rewrite_by_lua]]* directives to run at the end of the rewrite request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the rewrite phase. +Controls whether or not to disable postponing [[#rewrite_by_lua|rewrite_by_lua*]] directives to run at the end of the rewrite request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the rewrite phase. This directive was first introduced in the v0.5.0rc29 release. @@ -2420,7 +2420,7 @@ This directive was first introduced in the v0.5.0rc29 release. '''context:''' ''http'' -Controls whether or not to disable postponing [[#access_by_lua|access_by_lua]]* directives to run at the end of the access request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the access phase. +Controls whether or not to disable postponing [[#access_by_lua|access_by_lua*]] directives to run at the end of the access request-processing phase. By default, this directive is turned off and the Lua code is postponed to run at the end of the access phase. This directive was first introduced in the v0.9.20 release. @@ -2504,7 +2504,7 @@ This directive was first introduced in the v0.8.0 release. == Introduction == -The various *_by_lua and *_by_lua_file configuration directives serve as gateways to the Lua API within the nginx.conf file. The Nginx Lua API described below can only be called within the user Lua code run in the context of these configuration directives. +The various *_by_lua, *_by_lua_block and *_by_lua_file configuration directives serve as gateways to the Lua API within the nginx.conf file. The Nginx Lua API described below can only be called within the user Lua code run in the context of these configuration directives. The API is exposed to Lua in the form of two standard packages ngx and ndk. These packages are in the default global scope within ngx_lua and are always available within ngx_lua directives. @@ -2540,7 +2540,7 @@ Network I/O operations in user code should only be done through the Nginx Lua AP '''context:''' ''set_by_lua*, body_filter_by_lua*'' -When this is used in the context of the [[#set_by_lua|set_by_lua]] or [[#set_by_lua_file|set_by_lua_file]] directives, this table is read-only and holds the input arguments to the config directives: +When this is used in the context of the [[#set_by_lua|set_by_lua*]] directives, this table is read-only and holds the input arguments to the config directives: value = ngx.arg[n] @@ -2563,7 +2563,7 @@ Here is an example that writes out 88, the sum of 32 and 56. -When this table is used in the context of [[#body_filter_by_lua|body_filter_by_lua]] or [[#body_filter_by_lua_file|body_filter_by_lua_file]], the first element holds the input data chunk to the output filter code and the second element holds the boolean flag for the "eof" flag indicating the end of the whole output data stream. +When this table is used in the context of [[#body_filter_by_lua|body_filter_by_lua*]], the first element holds the input data chunk to the output filter code and the second element holds the boolean flag for the "eof" flag indicating the end of the whole output data stream. The data chunk and "eof" flag passed to the downstream Nginx output filters can also be overridden by assigning values directly to the corresponding table elements. When setting nil or an empty Lua string value to ngx.arg[1], no data chunk will be passed to the downstream Nginx output filters at all. @@ -2585,10 +2585,10 @@ For example: location /foo { set $my_var ''; # this line is required to create $my_var at config time - content_by_lua ' + content_by_lua_block { ngx.var.my_var = 123; ... - '; + } } @@ -2620,7 +2620,7 @@ Undefined NGINX variables are evaluated to `nil` while uninitialized (but define This API requires a relatively expensive metamethod call and it is recommended to avoid using it on hot code paths. == Core constants == -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' ngx.OK (0) @@ -2641,7 +2641,7 @@ The ngx.null constant is a NULL light userdata usually The ngx.DECLINED constant was first introduced in the v0.5.0rc19 release. == HTTP method constants == -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' ngx.HTTP_GET @@ -2664,7 +2664,7 @@ The ngx.DECLINED constant was first introduced in the v0.5.0r These constants are usually used in [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] method calls. == HTTP status constants == -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' value = ngx.HTTP_CONTINUE (100) (first added in the v0.9.20 release) @@ -2704,7 +2704,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture == Nginx log level constants == -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' ngx.STDERR @@ -2723,7 +2723,7 @@ These constants are usually used by the [[#ngx.log|ngx.log]] method. == print == '''syntax:''' ''print(...)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Writes argument values into the nginx error.log file with the ngx.NOTICE log level. @@ -2738,7 +2738,7 @@ Lua nil arguments are accepted and result in literal "nil"2048 byte limitation on error message lengths in the Nginx core. This limit includes trailing newlines and leading time stamps. If the message size exceeds this limit, Nginx will truncate the message text accordingly. This limit can be manually modified by editing the NGX_MAX_ERROR_STR macro definition in the src/core/ngx_log.h file in the Nginx source tree. == ngx.ctx == -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, ngx.timer.*, balancer_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*'' This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). @@ -2746,15 +2746,15 @@ Consider the following example, location /test { - rewrite_by_lua ' + rewrite_by_lua_block { ngx.ctx.foo = 76 - '; - access_by_lua ' + } + access_by_lua_block { ngx.ctx.foo = ngx.ctx.foo + 3 - '; - content_by_lua ' + } + content_by_lua_block { ngx.say(ngx.ctx.foo) - '; + } } @@ -2770,21 +2770,21 @@ Every request, including subrequests, has its own copy of the table. For example location /sub { - content_by_lua ' + content_by_lua_block { ngx.say("sub pre: ", ngx.ctx.blah) ngx.ctx.blah = 32 ngx.say("sub post: ", ngx.ctx.blah) - '; + } } location /main { - content_by_lua ' + content_by_lua_block { ngx.ctx.blah = 73 ngx.say("main pre: ", ngx.ctx.blah) local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("main post: ", ngx.ctx.blah) - '; + } } @@ -2803,16 +2803,16 @@ Internal redirection will destroy the original request ngx.ctx data location /new { - content_by_lua ' + content_by_lua_block { ngx.say(ngx.ctx.foo) - '; + } } location /orig { - content_by_lua ' + content_by_lua_block { ngx.ctx.foo = "hello" ngx.exec("/new") - '; + } } @@ -2999,13 +2999,13 @@ This option is set to false by default location /lua { set $dog 'hello'; - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/other", { share_all_vars = true }); ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) - '; + } } @@ -3026,13 +3026,13 @@ The copy_all_vars option provides a copy of the parent request's Ng location /lua { set $dog 'hello'; - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/other", { copy_all_vars = true }); ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) - '; + } } @@ -3054,21 +3054,21 @@ unescaping them in the Nginx config file. location /other { - content_by_lua ' + content_by_lua_block { ngx.say("dog = ", ngx.var.dog) ngx.say("cat = ", ngx.var.cat) - '; + } } location /lua { set $dog ''; set $cat ''; - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/other", { vars = { dog = "hello", cat = 32 }}); ngx.print(res.body) - '; + } } @@ -3083,18 +3083,18 @@ The ctx option can be used to specify a custom Lua table to serve a location /sub { - content_by_lua ' + content_by_lua_block { ngx.ctx.foo = "bar"; - '; + } } location /lua { - content_by_lua ' + content_by_lua_block { local ctx = {} res = ngx.location.capture("/sub", { ctx = ctx }) ngx.say(ctx.foo); ngx.say(ngx.ctx.foo); - '; + } } @@ -3109,15 +3109,15 @@ It is also possible to use this ctx option to share the same [[#ngx location /sub { - content_by_lua ' + content_by_lua_block { ngx.ctx.foo = "bar"; - '; + } } location /lua { - content_by_lua ' + content_by_lua_block { res = ngx.location.capture("/sub", { ctx = ngx.ctx }) ngx.say(ngx.ctx.foo); - '; + } } @@ -3206,7 +3206,7 @@ of this function. Logically speaking, the [[#ngx.location.capture|ngx.location.c Please also refer to restrictions on capturing locations configured by [[#Locations_Configured_by_Subrequest_Directives_of_Other_Modules|subrequest directives of other modules]]. == ngx.status == -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Read and write the current request's response status. This should be called before sending out the response headers. @@ -3227,7 +3227,7 @@ Setting ngx.status after the response header is sent out has no eff '''syntax:''' ''value = ngx.header.HEADER'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Set, add to, or clear the current request's HEADER response header that is to be sent. @@ -3287,7 +3287,7 @@ Reading ngx.header.HEADER will return the value of the response hea Underscores (_) in the header names will also be replaced by dashes (-) and the header names will be matched case-insensitively. If the response header is not present at all, nil will be returned. -This is particularly useful in the context of [[#header_filter_by_lua|header_filter_by_lua]] and [[#header_filter_by_lua_file|header_filter_by_lua_file]], for example, +This is particularly useful in the context of [[#header_filter_by_lua|header_filter_by_lua*]], for example, location /test { @@ -3295,11 +3295,11 @@ This is particularly useful in the context of [[#header_filter_by_lua|header_fil proxy_pass http://some-backend; - header_filter_by_lua ' + header_filter_by_lua_block { if ngx.header["X-My-Header"] == "blah" then ngx.var.footer = "some value" end - '; + } echo_after_body $footer; } @@ -3327,7 +3327,7 @@ For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_head == ngx.resp.get_headers == '''syntax:''' ''headers = ngx.resp.get_headers(max_headers?, raw?)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, balancer_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' Returns a Lua table holding all the current response headers for the current request. @@ -3492,16 +3492,16 @@ or equivalently, ngx.req.set_uri("/foo") -The jump argument can only be set to true in [[#rewrite_by_lua|rewrite_by_lua]] and [[#rewrite_by_lua_file|rewrite_by_lua_file]]. Use of jump in other contexts is prohibited and will throw out a Lua exception. +The jump argument can only be set to true in [[#rewrite_by_lua|rewrite_by_lua*]]. Use of jump in other contexts is prohibited and will throw out a Lua exception. A more sophisticated example involving regex substitutions is as follows location /test { - rewrite_by_lua ' + rewrite_by_lua_block { local uri = ngx.re.sub(ngx.var.uri, "^/test/(.*)", "/$1", "o") ngx.req.set_uri(uri) - '; + } proxy_pass http://my_backend; } @@ -3571,13 +3571,13 @@ See also [[#ngx.req.set_uri|ngx.req.set_uri]]. == ngx.req.get_uri_args == '''syntax:''' ''args = ngx.req.get_uri_args(max_args?)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*, balancer_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' Returns a Lua table holding all the current request URL query arguments. location = /test { - content_by_lua ' + content_by_lua_block { local args = ngx.req.get_uri_args() for key, val in pairs(args) do if type(val) == "table" then @@ -3586,7 +3586,7 @@ Returns a Lua table holding all the current request URL query arguments. ngx.say(key, ": ", val) end end - '; + } } @@ -3655,13 +3655,13 @@ Removing the max_args cap is strongly discouraged. == ngx.req.get_post_args == '''syntax:''' ''args, err = ngx.req.get_post_args(max_args?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Returns a Lua table holding all the current request POST query arguments (of the MIME type application/x-www-form-urlencoded). Call [[#ngx.req.read_body|ngx.req.read_body]] to read the request body first or turn on the [[#lua_need_request_body|lua_need_request_body]] directive to avoid errors. location = /test { - content_by_lua ' + content_by_lua_block { ngx.req.read_body() local args, err = ngx.req.get_post_args() if not args then @@ -3675,7 +3675,7 @@ Returns a Lua table holding all the current request POST query arguments (of the ngx.say(key, ": ", val) end end - '; + } } @@ -3745,7 +3745,7 @@ Removing the max_args cap is strongly discouraged. == ngx.req.get_headers == '''syntax:''' ''headers = ngx.req.get_headers(max_headers?, raw?)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua, log_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Returns a Lua table holding all the current request headers. @@ -3809,7 +3809,7 @@ The __index metamethod will not be added when the raw == ngx.req.set_header == '''syntax:''' ''ngx.req.set_header(header_name, header_value)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*'' Set the current request's request header named header_name to value header_value, overriding any existing ones. @@ -3995,7 +3995,7 @@ The usage of this function is often like this: ngx.req.finish_body() -This function can be used with [[#ngx.req.append_body|ngx.req.append_body]], [[#ngx.req.finish_body|ngx.req.finish_body]], and [[#ngx.req.socket|ngx.req.socket]] to implement efficient input filters in pure Lua (in the context of [[#rewrite_by_lua|rewrite_by_lua]]* or [[#access_by_lua|access_by_lua]]*), which can be used with other Nginx content handler or upstream modules like [[HttpProxyModule]] and [[HttpFastcgiModule]]. +This function can be used with [[#ngx.req.append_body|ngx.req.append_body]], [[#ngx.req.finish_body|ngx.req.finish_body]], and [[#ngx.req.socket|ngx.req.socket]] to implement efficient input filters in pure Lua (in the context of [[#rewrite_by_lua|rewrite_by_lua*]] or [[#access_by_lua|access_by_lua*]]), which can be used with other Nginx content handler or upstream modules like [[HttpProxyModule]] and [[HttpFastcgiModule]]. This function was first introduced in the v0.5.11 release. @@ -4010,7 +4010,7 @@ When the data can no longer be hold in the memory buffer for the request body, t It is important to always call the [[#ngx.req.finish_body|ngx.req.finish_body]] after all the data has been appended onto the current request body. -This function can be used with [[#ngx.req.init_body|ngx.req.init_body]], [[#ngx.req.finish_body|ngx.req.finish_body]], and [[#ngx.req.socket|ngx.req.socket]] to implement efficient input filters in pure Lua (in the context of [[#rewrite_by_lua|rewrite_by_lua]]* or [[#access_by_lua|access_by_lua]]*), which can be used with other Nginx content handler or upstream modules like [[HttpProxyModule]] and [[HttpFastcgiModule]]. +This function can be used with [[#ngx.req.init_body|ngx.req.init_body]], [[#ngx.req.finish_body|ngx.req.finish_body]], and [[#ngx.req.socket|ngx.req.socket]] to implement efficient input filters in pure Lua (in the context of [[#rewrite_by_lua|rewrite_by_lua*]] or [[#access_by_lua|access_by_lua*]]), which can be used with other Nginx content handler or upstream modules like [[HttpProxyModule]] and [[HttpFastcgiModule]]. This function was first introduced in the v0.5.11 release. @@ -4023,7 +4023,7 @@ See also [[#ngx.req.init_body|ngx.req.init_body]]. Completes the construction process of the new request body created by the [[#ngx.req.init_body|ngx.req.init_body]] and [[#ngx.req.append_body|ngx.req.append_body]] calls. -This function can be used with [[#ngx.req.init_body|ngx.req.init_body]], [[#ngx.req.append_body|ngx.req.append_body]], and [[#ngx.req.socket|ngx.req.socket]] to implement efficient input filters in pure Lua (in the context of [[#rewrite_by_lua|rewrite_by_lua]]* or [[#access_by_lua|access_by_lua]]*), which can be used with other Nginx content handler or upstream modules like [[HttpProxyModule]] and [[HttpFastcgiModule]]. +This function can be used with [[#ngx.req.init_body|ngx.req.init_body]], [[#ngx.req.append_body|ngx.req.append_body]], and [[#ngx.req.socket|ngx.req.socket]] to implement efficient input filters in pure Lua (in the context of [[#rewrite_by_lua|rewrite_by_lua*]] or [[#access_by_lua|access_by_lua*]]), which can be used with other Nginx content handler or upstream modules like [[HttpProxyModule]] and [[HttpFastcgiModule]]. This function was first introduced in the v0.5.11 release. @@ -4088,20 +4088,20 @@ Named locations are also supported but the second args argument wil location /foo { - content_by_lua ' + content_by_lua_block { ngx.exec("@bar", "a=goodbye"); - '; + } } location @bar { - content_by_lua ' + content_by_lua_block { local args = ngx.req.get_uri_args() for key, val in pairs(args) do if key == "a" then ngx.say(val) end end - '; + } } @@ -4111,7 +4111,7 @@ it is purely an internal redirect and that no new external HTTP traffic is invol Also note that this method call terminates the processing of the current request and that it ''must'' be called before [[#ngx.send_headers|ngx.send_headers]] or explicit response body outputs by either [[#ngx.print|ngx.print]] or [[#ngx.say|ngx.say]]. -It is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.exec(...) be adopted when this method call is used in contexts other than [[#header_filter_by_lua|header_filter_by_lua]] to reinforce the fact that the request processing is being terminated. +It is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.exec(...) be adopted when this method call is used in contexts other than [[#header_filter_by_lua|header_filter_by_lua*]] to reinforce the fact that the request processing is being terminated. == ngx.redirect == '''syntax:''' ''ngx.redirect(uri, status?)'' @@ -4181,7 +4181,7 @@ URI arguments can be specified as well, for example: Note that this method call terminates the processing of the current request and that it ''must'' be called before [[#ngx.send_headers|ngx.send_headers]] or explicit response body outputs by either [[#ngx.print|ngx.print]] or [[#ngx.say|ngx.say]]. -It is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.redirect(...) be adopted when this method call is used in contexts other than [[#header_filter_by_lua|header_filter_by_lua]] to reinforce the fact that the request processing is being terminated. +It is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.redirect(...) be adopted when this method call is used in contexts other than [[#header_filter_by_lua|header_filter_by_lua*]] to reinforce the fact that the request processing is being terminated. == ngx.send_headers == '''syntax:''' ''ok, err = ngx.send_headers()'' @@ -4193,7 +4193,7 @@ Explicitly send out the response headers. Since v0.8.3 this function returns 1 on success, or returns nil and a string describing the error otherwise. Note that there is normally no need to manually send out response headers as ngx_lua will automatically send headers out -before content is output with [[#ngx.say|ngx.say]] or [[#ngx.print|ngx.print]] or when [[#content_by_lua|content_by_lua]] exits normally. +before content is output with [[#ngx.say|ngx.say]] or [[#ngx.print|ngx.print]] or when [[#content_by_lua|content_by_lua*]] exits normally. == ngx.headers_sent == '''syntax:''' ''value = ngx.headers_sent'' @@ -4284,7 +4284,7 @@ Since v0.8.3 this function returns 1 on success, or re When status >= 200 (i.e., ngx.HTTP_OK and above), it will interrupt the execution of the current request and return status code to nginx. -When status == 0 (i.e., ngx.OK), it will only quit the current phase handler (or the content handler if the [[#content_by_lua|content_by_lua]] directive is used) and continue to run later phases (if any) for the current request. +When status == 0 (i.e., ngx.OK), it will only quit the current phase handler (or the content handler if the [[#content_by_lua|content_by_lua*]] directive is used) and continue to run later phases (if any) for the current request. The status argument can be ngx.OK, ngx.ERROR, ngx.HTTP_NOT_FOUND, ngx.HTTP_MOVED_TEMPORARILY, or other [[#HTTP status constants|HTTP status constants]]. @@ -4338,11 +4338,11 @@ When you disable the HTTP 1.1 keep-alive feature for your downstream connections location = /async { keepalive_timeout 0; - content_by_lua ' + content_by_lua_block { ngx.say("got the task!") ngx.eof() -- a descent HTTP client will close the connection at this point -- access MySQL, PostgreSQL, Redis, Memcached, and etc here... - '; + } } @@ -4546,7 +4546,7 @@ For example, location = /md5 { - content_by_lua 'ngx.say(ngx.md5("hello"))'; + content_by_lua_block { ngx.say(ngx.md5("hello")) } } @@ -5085,17 +5085,17 @@ Here is an example: lua_shared_dict dogs 10m; server { location /set { - content_by_lua ' + content_by_lua_block { local dogs = ngx.shared.dogs dogs:set("Jim", 8) ngx.say("STORED") - '; + } } location /get { - content_by_lua ' + content_by_lua_block { local dogs = ngx.shared.dogs ngx.say(dogs:get("Jim")) - '; + } } } } @@ -5448,7 +5448,7 @@ Here is an example for connecting to a UDP (memcached) server: location /test { resolver 8.8.8.8; - content_by_lua ' + content_by_lua_block { local sock = ngx.socket.udp() local ok, err = sock:setpeername("my.memcached.server.domain", 11211) if not ok then @@ -5457,7 +5457,7 @@ Here is an example for connecting to a UDP (memcached) server: end ngx.say("successfully connected to memcached!") sock:close() - '; + } } @@ -5619,7 +5619,7 @@ Here is an example for connecting to a TCP server: location /test { resolver 8.8.8.8; - content_by_lua ' + content_by_lua_block { local sock = ngx.socket.tcp() local ok, err = sock:connect("www.google.com", 80) if not ok then @@ -5628,7 +5628,7 @@ Here is an example for connecting to a TCP server: end ngx.say("successfully connected to google!") sock:close() - '; + } } @@ -5967,31 +5967,31 @@ This feature was first introduced in the v0.5.0rc1 release. Retrieves the current running phase name. Possible return values are * init -: for the context of [[#init_by_lua|init_by_lua]] or [[#init_by_lua_file|init_by_lua_file]]. +: for the context of [[#init_by_lua|init_by_lua*]]. * init_worker -: for the context of [[#init_worker_by_lua|init_worker_by_lua]] or [[#init_worker_by_lua_file|init_worker_by_lua_file]]. +: for the context of [[#init_worker_by_lua|init_worker_by_lua*]]. * ssl_cert -: for the context of [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua_block]] or [[#ssl_certificate_by_lua_file|ssl_certificate_by_lua_file]]. +: for the context of [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]]. * ssl_session_fetch : for the context of [[#ssl_session_fetch_by_lua_block|ssl_session_fetch_by_lua*]]. * ssl_session_store : for the context of [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]]. * set -: for the context of [[#set_by_lua|set_by_lua]] or [[#set_by_lua_file|set_by_lua_file]]. +: for the context of [[#set_by_lua|set_by_lua*]]. * rewrite -: for the context of [[#rewrite_by_lua|rewrite_by_lua]] or [[#rewrite_by_lua_file|rewrite_by_lua_file]]. +: for the context of [[#rewrite_by_lua|rewrite_by_lua*]]. * balancer -: for the context of [[#balancer_by_lua_block|balancer_by_lua_block]] or [[#balancer_by_lua_file|balancer_by_lua_file]]. +: for the context of [[#balancer_by_lua_block|balancer_by_lua*]]. * access -: for the context of [[#access_by_lua|access_by_lua]] or [[#access_by_lua_file|access_by_lua_file]]. +: for the context of [[#access_by_lua|access_by_lua*]]. * content -: for the context of [[#content_by_lua|content_by_lua]] or [[#content_by_lua_file|content_by_lua_file]]. +: for the context of [[#content_by_lua|content_by_lua*]]. * header_filter -: for the context of [[#header_filter_by_lua|header_filter_by_lua]] or [[#header_filter_by_lua_file|header_filter_by_lua_file]]. +: for the context of [[#header_filter_by_lua|header_filter_by_lua*]]. * body_filter -: for the context of [[#body_filter_by_lua|body_filter_by_lua]] or [[#body_filter_by_lua_file|body_filter_by_lua_file]]. +: for the context of [[#body_filter_by_lua|body_filter_by_lua*]]. * log -: for the context of [[#log_by_lua|log_by_lua]] or [[#log_by_lua_file|log_by_lua_file]]. +: for the context of [[#log_by_lua|log_by_lua*]]. * timer : for the context of user callback functions for [[#ngx.timer.at|ngx.timer.*]]. @@ -6318,7 +6318,7 @@ Here is a simple example: location / { ... - log_by_lua ' + log_by_lua_block { local function push_data(premature, uri, args, status) -- push the data uri, args, and status to the remote -- via ngx.socket.tcp or ngx.socket.udp @@ -6331,7 +6331,7 @@ Here is a simple example: ngx.log(ngx.ERR, "failed to create timer: ", err) return end - '; + } } From 1173cc7c936b797782ad03bc3cb8b3b6761c6fce Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 2 Oct 2016 12:00:49 -0700 Subject: [PATCH 002/848] feature: added new config directive "lua_malloc_trim N" to periodically call malloc_trim(1) every N requests when malloc_trim() is available. By default, "lua_malloc_trim 1000" is configured. This should fix the glibc oddity of holding too much freed memory when it fails to use brk() to allocate memory in the data segment. --- README.markdown | 26 +++ config | 18 ++ doc/HttpLuaModule.wiki | 26 +++ src/ngx_http_lua_common.h | 4 + src/ngx_http_lua_logby.c | 37 +++++ src/ngx_http_lua_module.c | 54 ++++++ t/146-malloc-trim.t | 337 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 502 insertions(+) create mode 100644 t/146-malloc-trim.t diff --git a/README.markdown b/README.markdown index 25c1ded549..58996fc3de 100644 --- a/README.markdown +++ b/README.markdown @@ -1002,6 +1002,7 @@ Directives ========== * [lua_use_default_type](#lua_use_default_type) +* [lua_malloc_trim](#lua_malloc_trim) * [lua_code_cache](#lua_code_cache) * [lua_regex_cache_max_entries](#lua_regex_cache_max_entries) * [lua_regex_match_limit](#lua_regex_match_limit) @@ -1089,6 +1090,31 @@ This directive was first introduced in the `v0.9.1` release. [Back to TOC](#directives) +lua_malloc_trim +--------------- +**syntax:** *lua_malloc_trim <request-count>* + +**default:** *lua_malloc_trim 1000* + +**context:** *http* + +Asks the underlying `libc` runtime library to release its cached free memory back to the operating system every +`N` requests processed by the NGINX core. By default, `N` is 1000. You can configure the request count +by using your own numbers. Smaller numbers mean more frequent releases, which may introduce higher CPU time consumption and +smaller memory footprint while larger numbers usually lead to less CPU time overhead and relatively larger memory footprint. +Just tune the number for your own use cases. + +Configuring the argument to `0` essentially turns off the periodical memory trimming altogether. + +The current implementation uses an NGINX log phase handler to do the request counting. So the appearance of the +[log_subrequest on](http://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest) directives in `nginx.conf` +may make the counting faster when subrequests are involved. By default, only "main requests" count. + +Note that this directive does *not* affect the memory allocated by LuaJIT's own allocator based on the `mmap` +system call. + +[Back to TOC](#directives) + lua_code_cache -------------- **syntax:** *lua_code_cache on | off* diff --git a/config b/config index 0f2749d32f..01a6b3c3cc 100644 --- a/config +++ b/config @@ -519,6 +519,24 @@ CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" +# ---------------------------------------- + +ngx_feature="malloc_trim" +ngx_feature_libs= +ngx_feature_name="NGX_HTTP_LUA_HAVE_MALLOC_TRIM" +ngx_feature_run=yes +ngx_feature_incs="#include +#include " +ngx_feature_test="int rc = malloc_trim((size_t) 0); printf(\"%d\", rc);" +SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" +CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" + +. auto/feature + +CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" + +# ---------------------------------------- + if test -n "$ngx_module_link"; then ngx_module_type=HTTP_AUX_FILTER ngx_module_name=$ngx_addon_name diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index dffecc853d..a4fb2c4932 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -840,6 +840,32 @@ This directive is turned on by default. This directive was first introduced in the v0.9.1 release. +== lua_malloc_trim == +'''syntax:''' ''lua_malloc_trim '' + +'''default:''' ''lua_malloc_trim 1000'' + +'''context:''' ''http'' + +Asks the underlying libc runtime library to release its cached free memory back to the operating system every +N requests processed by the NGINX core. By default, N is 1000. You can configure the request count +by using your own numbers. Smaller numbers mean more frequent releases, which may introduce higher CPU time consumption and +smaller memory footprint while larger numbers usually lead to less CPU time overhead and relatively larger memory footprint. +Just tune the number for your own use cases. + +Configuring the argument to 0 essentially turns off the periodical memory trimming altogether. + + + lua_malloc_trim 0; # turn off trimming completely + + +The current implementation uses an NGINX log phase handler to do the request counting. So the appearance of the +[http://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest log_subrequest on] directives in nginx.conf +may make the counting faster when subrequests are involved. By default, only "main requests" count. + +Note that this directive does *not* affect the memory allocated by LuaJIT's own allocator based on the mmap +system call. + == lua_code_cache == '''syntax:''' ''lua_code_cache on | off'' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index d5e0dc8052..6aa0f171c1 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -193,6 +193,10 @@ struct ngx_http_lua_main_conf_s { ngx_http_lua_sema_mm_t *sema_mm; + ngx_uint_t malloc_trim_cycle; /* a cycle is defined as the number + of reqeusts */ + ngx_uint_t malloc_trim_req_count; + unsigned requires_header_filter:1; unsigned requires_body_filter:1; unsigned requires_capture_filter:1; diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index 49fcd08365..0f1d2f35b3 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -27,6 +27,9 @@ #include "ngx_http_lua_shdict.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_exception.h" +#if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) +#include +#endif static ngx_int_t ngx_http_lua_log_by_chunk(lua_State *L, ngx_http_request_t *r); @@ -67,9 +70,43 @@ ngx_http_lua_log_by_lua_env(lua_State *L, ngx_http_request_t *r) ngx_int_t ngx_http_lua_log_handler(ngx_http_request_t *r) { +#if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) + ngx_uint_t trim_cycle, trim_nreq; + ngx_http_lua_main_conf_t *lmcf; +#endif ngx_http_lua_loc_conf_t *llcf; ngx_http_lua_ctx_t *ctx; +#if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + + trim_cycle = lmcf->malloc_trim_cycle; + + if (trim_cycle > 0) { + + dd("cycle: %d", (int) trim_cycle); + + trim_nreq = ++lmcf->malloc_trim_req_count; + + if (trim_nreq >= trim_cycle) { + lmcf->malloc_trim_req_count = 0; + +#if (NGX_DEBUG) + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "malloc_trim(1) returned %d", malloc_trim(1)); +#else + (void) malloc_trim(1); +#endif + } + } +# if (NGX_DEBUG) + else { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "malloc_trim() disabled"); + } +# endif +#endif + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua log handler, uri:\"%V\" c:%ud", &r->uri, r->main->count); diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index c5e10f9159..fe39391901 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -53,6 +53,8 @@ void ngx_http_lua_limit_data_segment(void); static ngx_int_t ngx_http_lua_pre_config(ngx_conf_t *cf); # endif #endif +static char *ngx_http_lua_malloc_trim(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); static ngx_conf_post_t ngx_http_lua_lowat_post = @@ -578,6 +580,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { #endif /* NGX_HTTP_SSL */ + { ngx_string("lua_malloc_trim"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, + ngx_http_lua_malloc_trim, + NGX_HTTP_MAIN_CONF_OFFSET, + 0, + NULL }, + ngx_null_command }; @@ -832,6 +841,10 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) lmcf->postponed_to_rewrite_phase_end = NGX_CONF_UNSET; lmcf->postponed_to_access_phase_end = NGX_CONF_UNSET; +#if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) + lmcf->malloc_trim_cycle = NGX_CONF_UNSET_UINT; +#endif + #ifndef NGX_LUA_NO_FFI_API rc = ngx_http_lua_sema_mm_init(cf, lmcf); if (rc != NGX_OK) { @@ -868,6 +881,12 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) lmcf->max_running_timers = 256; } +#if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) + if (lmcf->malloc_trim_cycle == NGX_CONF_UNSET_UINT) { + lmcf->malloc_trim_cycle = 1000; /* number of reqs */ + } +#endif + lmcf->cycle = cf->cycle; return NGX_CONF_OK; @@ -1283,4 +1302,39 @@ ngx_http_lua_limit_data_segment(void) } #endif + +static char * +ngx_http_lua_malloc_trim(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ +#if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) + + ngx_int_t nreqs; + ngx_str_t *value; + + ngx_http_lua_main_conf_t *lmcf = conf; + + value = cf->args->elts; + + nreqs = ngx_atoi(value[1].data, value[1].len); + if (nreqs == NGX_ERROR) { + return "invalid number in the 1st argument"; + } + + lmcf->malloc_trim_cycle = (ngx_uint_t) nreqs; + + if (nreqs == 0) { + return NGX_CONF_OK; + } + + lmcf->requires_log = 1; + +#else + + ngx_conf_log_error(NGX_LOG_WARN, cf, 0, "lua_malloc_trim is not supported " + "on this platform, ignored"); + +#endif + return NGX_CONF_OK; +} + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/146-malloc-trim.t b/t/146-malloc-trim.t new file mode 100644 index 0000000000..45fb9f2463 --- /dev/null +++ b/t/146-malloc-trim.t @@ -0,0 +1,337 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_process_enabled(1); +#log_level('warn'); + +#repeat_each(2); + +plan tests => repeat_each() * (blocks() * 4 + 3); + +#no_diff(); +no_long_string(); +#master_on(); +#workers(2); + +run_tests(); + +__DATA__ + +=== TEST 1: malloc_trim() every 1 req, in subreq +--- http_config + lua_malloc_trim 1; +--- config + location = /t { + return 200 "ok\n"; + } + + location = /main { + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + } +--- request +GET /main +--- response_body +ok +ok +ok +ok +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +malloc_trim(1) returned 0 +--- wait: 0.2 +--- no_error_log +[error] + + + +=== TEST 2: malloc_trim() every 1 req, in subreq +--- http_config + lua_malloc_trim 1; +--- config + location = /t { + log_subrequest on; + return 200 "ok\n"; + } + + location = /main { + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + } +--- request +GET /main +--- response_body +ok +ok +ok +ok +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +--- wait: 0.2 +--- no_error_log +[error] + + + +=== TEST 3: malloc_trim() every 2 req, in subreq +--- http_config + lua_malloc_trim 2; +--- config + location = /t { + log_subrequest on; + return 200 "ok\n"; + } + + location = /main { + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + } +--- request +GET /main +--- response_body +ok +ok +ok +ok +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +--- wait: 0.2 +--- no_error_log +[error] + + + +=== TEST 4: malloc_trim() every 3 req, in subreq +--- http_config + lua_malloc_trim 3; +--- config + location = /t { + log_subrequest on; + return 200 "ok\n"; + } + + location = /main { + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + } +--- request +GET /main +--- response_body +ok +ok +ok +ok +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +malloc_trim(1) returned 0 +malloc_trim(1) returned 0 +--- wait: 0.2 +--- no_error_log +[error] + + + +=== TEST 5: malloc_trim() every 2 req, in subreq, big memory usage +--- http_config + lua_malloc_trim 2; + lua_package_path "$prefix/html/?.lua;;"; +--- config + location = /t { + log_subrequest on; + content_by_lua_block { + require("foo")() + } + } + + location = /main { + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + } +--- user_files +>>> foo.lua +local ffi = require "ffi" + +ffi.cdef[[ + void *malloc(size_t sz); + void free(void *p); +]] + +return function () + local t = {} + for i = 1, 10 do + t[i] = ffi.C.malloc(1024 * 128) + end + for i = 1, 10 do + ffi.C.free(t[i]) + end + ngx.say("ok") +end +--- request +GET /main +--- response_body +ok +ok +ok +ok +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +malloc_trim(1) returned 1 +malloc_trim(1) returned 1 +malloc_trim(1) returned 1 +--- wait: 0.2 +--- no_error_log +[error] + + + +=== TEST 6: zero count means off +--- http_config + lua_malloc_trim 0; + lua_package_path "$prefix/html/?.lua;;"; +--- config + location = /t { + content_by_lua_block { + require("foo")() + } + } +--- user_files +>>> foo.lua +local ffi = require "ffi" + +ffi.cdef[[ + void *malloc(size_t sz); + void free(void *p); +]] + +return function () + local t = {} + for i = 1, 10 do + t[i] = ffi.C.malloc(1024 * 128) + end + for i = 1, 10 do + ffi.C.free(t[i]) + end + ngx.say("ok") +end + +--- request +GET /t +--- response_body +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +--- wait: 0.2 +--- no_error_log +malloc_trim() disabled +[error] + + + +=== TEST 7: zero count means off, log_by_lua +--- http_config + lua_malloc_trim 0; + lua_package_path "$prefix/html/?.lua;;"; +--- config + location = /t { + content_by_lua_block { + require("foo")() + } + log_by_lua_block { + print("Hello from log") + } + } +--- user_files +>>> foo.lua +local ffi = require "ffi" + +ffi.cdef[[ + void *malloc(size_t sz); + void free(void *p); +]] + +return function () + local t = {} + for i = 1, 10 do + t[i] = ffi.C.malloc(1024 * 128) + end + for i = 1, 10 do + ffi.C.free(t[i]) + end + ngx.say("ok") +end + +--- request +GET /t +--- response_body +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +--- wait: 0.2 +--- error_log +Hello from log +malloc_trim() disabled +--- no_error_log +[error] + + + +=== TEST 8: malloc_trim() every 1 req +--- http_config + lua_malloc_trim 1; +--- config + location = /t { + return 200 "ok\n"; + } + + location = /main { + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + echo_location /t; + } +--- request +GET /main +--- response_body +ok +ok +ok +ok +ok +--- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ +--- grep_error_log_out +malloc_trim(1) returned 0 +--- wait: 0.2 +--- no_error_log +[error] From 66405877d6686ff87a2d1cf2653229ef6606bec5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 22 Oct 2016 20:03:09 -0700 Subject: [PATCH 003/848] bumped the API version to 0.10.7; also documented that lua_malloc_trim was first introduced in ngx_lua 0.10.7. --- README.markdown | 7 +++++++ doc/HttpLuaModule.wiki | 2 ++ src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 58996fc3de..5f5f008f87 100644 --- a/README.markdown +++ b/README.markdown @@ -1106,6 +1106,11 @@ Just tune the number for your own use cases. Configuring the argument to `0` essentially turns off the periodical memory trimming altogether. +```nginx + + lua_malloc_trim 0; # turn off trimming completely +``` + The current implementation uses an NGINX log phase handler to do the request counting. So the appearance of the [log_subrequest on](http://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest) directives in `nginx.conf` may make the counting faster when subrequests are involved. By default, only "main requests" count. @@ -1113,6 +1118,8 @@ may make the counting faster when subrequests are involved. By default, only "ma Note that this directive does *not* affect the memory allocated by LuaJIT's own allocator based on the `mmap` system call. +This directive was first introduced in the `v0.10.7` release. + [Back to TOC](#directives) lua_code_cache diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a4fb2c4932..d2fb07f229 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -866,6 +866,8 @@ may make the counting faster when subrequests are involved. By default, only "ma Note that this directive does *not* affect the memory allocated by LuaJIT's own allocator based on the mmap system call. +This directive was first introduced in the v0.10.7 release. + == lua_code_cache == '''syntax:''' ''lua_code_cache on | off'' diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 88b41ad17a..f6c197855b 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10006 +#define ngx_http_lua_version 10007 typedef struct { From 28df10b1661039c8dd70683badf68ad729d250c3 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Thu, 25 Aug 2016 16:44:28 +0800 Subject: [PATCH 004/848] feature: added a new api function tcpsock:settimeouts(connect_timeout, send_timeout, read_timeout). Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 27 +- doc/HttpLuaModule.wiki | 23 +- src/ngx_http_lua_socket_tcp.c | 124 ++++++-- t/062-count.t | 4 +- t/147-tcp-socket-timeouts.t | 516 ++++++++++++++++++++++++++++++++++ 5 files changed, 666 insertions(+), 28 deletions(-) create mode 100644 t/147-tcp-socket-timeouts.t diff --git a/README.markdown b/README.markdown index 5f5f008f87..323b8a55f3 100644 --- a/README.markdown +++ b/README.markdown @@ -2654,7 +2654,7 @@ lua_socket_connect_timeout **context:** *http, server, location* -This directive controls the default timeout value used in TCP/unix-domain socket object's [connect](#tcpsockconnect) method and can be overridden by the [settimeout](#tcpsocksettimeout) method. +This directive controls the default timeout value used in TCP/unix-domain socket object's [connect](#tcpsockconnect) method and can be overridden by the [settimeout](#tcpsocksettimeout) or [settimeouts](#tcpsocksettimeouts) methods. The ` or 302 redirection to uri. -The optional status parameter specifies whether -301 or 302 to be used. It is 302 (ngx.HTTP_MOVED_TEMPORARILY) by default. +The optional status parameter specifies the HTTP status code to be used. The following status codes are supported right now: + +* 301 +* 302 (default) +* 307 + +It is 302 (ngx.HTTP_MOVED_TEMPORARILY) by default. Here is an example assuming the current server name is localhost and that it is listening on port 1984: From 7bd68906b7a7503e6ee77fda0f19f743cdee8c62 Mon Sep 17 00:00:00 2001 From: detailyang Date: Wed, 2 Nov 2016 20:01:39 +0800 Subject: [PATCH 014/848] doc: log level constants are also available in init_by_lua* and init_worker_by_lua* contexts. thanks kraml for the report in #876. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index a9ce4bf641..4689bf5753 100644 --- a/README.markdown +++ b/README.markdown @@ -3394,7 +3394,7 @@ HTTP status constants Nginx log level constants ------------------------- -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```lua diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 80ebbf54e2..f4f14800cf 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2744,7 +2744,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture == Nginx log level constants == -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' ngx.STDERR From 3cbe5be4a66794af7cd8bc7f4aadaece5bce5edb Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 2 Nov 2016 19:58:17 -0700 Subject: [PATCH 015/848] tests: skipped the newly added test case that cannot run in check leak test mode. --- t/028-req-header.t | 1 + 1 file changed, 1 insertion(+) diff --git a/t/028-req-header.t b/t/028-req-header.t index c9a0fb0fc5..1ebcdff4e2 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -1643,3 +1643,4 @@ GeT / HTTP/1.1 ok --- no_error_log [error] +--- no_check_leak From 00643248825f61614541bb409a241e87ad87a554 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 4 Nov 2016 12:09:18 -0700 Subject: [PATCH 016/848] doc: bumped version to 0.10.7. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 4689bf5753..3699a72de4 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.6](https://github.com/openresty/lua-nginx-module/tags) released on 15 August 2016. +This document describes ngx_lua [v0.10.7](https://github.com/openresty/lua-nginx-module/tags) released on 4 November 2016. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f4f14800cf..be454af326 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.6] released on 15 August 2016. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.7] released on 4 November 2016. = Synopsis = From 75b03949c1898a4ff688eb390b4549519b6991bf Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 13 Nov 2016 12:30:56 -0800 Subject: [PATCH 017/848] doc: added a note about the LRU regex cache used in the ngx.re.* implementation of lua-resty-core. --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index 3699a72de4..eaac1456c9 100644 --- a/README.markdown +++ b/README.markdown @@ -1176,6 +1176,8 @@ The default number of entries allowed is 1024 and when this limit is reached, ne 2011/08/27 23:18:26 [warn] 31997#0: *1 lua exceeding regex cache max entries (1024), ... +If you are using the `ngx.re.*` implementation of [lua-resty-core](https://github.com/openresty/lua-resty-core) by loading the `resty.core.regex` module (or just the `resty.core` module), then an LRU cache is used for the regex cache being used here. + Do not activate the `o` option for regular expressions (and/or `replace` string arguments for [ngx.re.sub](#ngxresub) and [ngx.re.gsub](#ngxregsub)) that are generated *on the fly* and give rise to infinite variations to avoid hitting the specified limit. [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index be454af326..aa357a1c8d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -918,6 +918,8 @@ The default number of entries allowed is 1024 and when this limit is reached, ne 2011/08/27 23:18:26 [warn] 31997#0: *1 lua exceeding regex cache max entries (1024), ... +If you are using the `ngx.re.*` implementation of [lua-resty-core](https://github.com/openresty/lua-resty-core) by loading the `resty.core.regex` module (or just the `resty.core` module), then an LRU cache is used for the regex cache being used here. + Do not activate the o option for regular expressions (and/or replace string arguments for [[#ngx.re.sub|ngx.re.sub]] and [[#ngx.re.gsub|ngx.re.gsub]]) that are generated ''on the fly'' and give rise to infinite variations to avoid hitting the specified limit. == lua_regex_match_limit == From ce525fb5060a75941aa5b21c6f0290ca84ddee67 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 14 Nov 2016 14:43:59 -0800 Subject: [PATCH 018/848] tests: made tests less likely to fail due to bad network. --- t/087-udp-socket.t | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index 79ba452edf..c618fd7ad0 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -667,7 +667,8 @@ lua udp socket receive buffer size: 8192 === TEST 12: github issue #215: Handle the posted requests in lua cosocket api (failed to resolve) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; + resolver_timeout 5s; location = /sub { content_by_lua ' @@ -704,12 +705,13 @@ resolve name done --- no_error_log [error] +--- timeout: 10 === TEST 13: github issue #215: Handle the posted requests in lua cosocket api (successfully resolved) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 5s; location = /sub { From 0a1de31ff65803929a0e144d1e1c8fd2b3335375 Mon Sep 17 00:00:00 2001 From: jimtan <312841925@qq.com> Date: Mon, 14 Nov 2016 16:56:32 +0800 Subject: [PATCH 019/848] bugfix: C API: ngx_http_lua_add_package_preload() might not take effect when lua_code_cache is off. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_api.c | 5 +- t/150-fake-delayed-load.t | 56 +++++++++++++ t/data/fake-delayed-load-module/config | 3 + .../ngx_http_lua_fake_delayed_load_module.c | 78 +++++++++++++++++++ util/build.sh | 1 + 5 files changed, 140 insertions(+), 3 deletions(-) create mode 100644 t/150-fake-delayed-load.t create mode 100644 t/data/fake-delayed-load-module/config create mode 100644 t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index b72c7077f9..6c3d83de50 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -56,11 +56,10 @@ ngx_http_lua_add_package_preload(ngx_conf_t *cf, const char *package, lua_pushcfunction(L, func); lua_setfield(L, -2, package); lua_pop(L, 2); - - return NGX_OK; } - /* L == NULL */ + /* we always register preload_hooks since we always create new Lua VMs + * when lua code cache is off. */ if (lmcf->preload_hooks == NULL) { lmcf->preload_hooks = diff --git a/t/150-fake-delayed-load.t b/t/150-fake-delayed-load.t new file mode 100644 index 0000000000..232bbf3a83 --- /dev/null +++ b/t/150-fake-delayed-load.t @@ -0,0 +1,56 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: +use lib 'lib'; +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_process_enabled(1); +#log_level('warn'); + +#repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3); + +#no_diff(); +no_long_string(); +#master_on(); +#workers(2); + +run_tests(); + +__DATA__ + +=== TEST 1: lua code cache on +--- http_config + lua_code_cache on; +--- config + location = /cache_on { + content_by_lua_block { + local delayed_load = require("ngx.delayed_load") + ngx.say(type(delayed_load.get_function)) + } + } +--- request +GET /cache_on +--- response_body +function +--- no_error_log +[error] + + + +=== TEST 2: lua code cache off +--- http_config + lua_code_cache off; +--- config + location = /cache_off { + content_by_lua_block { + local delayed_load = require("ngx.delayed_load") + ngx.say(type(delayed_load.get_function)) + } + } +--- request +GET /cache_off +--- response_body +function +--- no_error_log +[error] diff --git a/t/data/fake-delayed-load-module/config b/t/data/fake-delayed-load-module/config new file mode 100644 index 0000000000..a5fa6fb48f --- /dev/null +++ b/t/data/fake-delayed-load-module/config @@ -0,0 +1,3 @@ +ngx_addon_name="ngx_http_lua_fake_delayed_load_module" +HTTP_AUX_FILTER_MODULES="$HTTP_AUX_FILTER_MODULES ngx_http_lua_fake_delayed_load_module" +NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_lua_fake_delayed_load_module.c" diff --git a/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c b/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c new file mode 100644 index 0000000000..8548d878e7 --- /dev/null +++ b/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c @@ -0,0 +1,78 @@ +/* Copyright (C) Jim Tan + * + * This fake_delayed_load delayed load module was used to reproduce + * a bug in ngx_lua's function ngx_http_lua_add_package_preload. + */ + + +#include +#include +#include +#include + + +#include "ngx_http_lua_api.h" + + +static ngx_int_t ngx_http_lua_fake_delayed_load_init(ngx_conf_t *cf); +static int ngx_http_lua_fake_delayed_load_preload(lua_State *L); +static int ngx_http_lua_fake_delayed_load_function(lua_State * L); + + +static ngx_http_module_t ngx_http_lua_fake_delayed_load_module_ctx = { + NULL, /* preconfiguration */ + ngx_http_lua_fake_delayed_load_init, /* postconfiguration */ + + NULL, /* create main configuration */ + NULL, /* init main configuration */ + + NULL, /* create server configuration */ + NULL, /* merge server configuration */ + + NULL, /* create location configuration */ + NULL, /* merge location configuration */ +}; + +/* flow identify module struct */ +ngx_module_t ngx_http_lua_fake_delayed_load_module = { + NGX_MODULE_V1, + &ngx_http_lua_fake_delayed_load_module_ctx, /* module context */ + NULL, /* module directives */ + NGX_HTTP_MODULE, /* module type */ + NULL, /* init master */ + NULL, /* init module */ + NULL, /* init process */ + NULL, /* init thread */ + NULL, /* exit thread */ + NULL, /* exit process */ + NULL, /* exit master */ + NGX_MODULE_V1_PADDING +}; + + +static ngx_int_t +ngx_http_lua_fake_delayed_load_init(ngx_conf_t *cf) +{ + ngx_http_lua_add_package_preload(cf, "ngx.delayed_load", + ngx_http_lua_fake_delayed_load_preload); + return NGX_OK; +} + + +static int +ngx_http_lua_fake_delayed_load_preload(lua_State *L) +{ + lua_createtable(L, 0, 1); + + lua_pushcfunction(L, ngx_http_lua_fake_delayed_load_function); + lua_setfield(L, -2, "get_function"); + + return 1; +} + + +static int +ngx_http_lua_fake_delayed_load_function(lua_State * L) +{ + return 1; +} diff --git a/util/build.sh b/util/build.sh index 7887fe93ac..e45c00a73f 100755 --- a/util/build.sh +++ b/util/build.sh @@ -52,6 +52,7 @@ time ngx-build $force $version \ --add-module=$root/../redis2-nginx-module \ --add-module=$root/t/data/fake-module \ --add-module=$root/t/data/fake-shm-module \ + --add-module=$root/t/data/fake-delayed-load-module \ --with-http_gunzip_module \ --with-http_dav_module \ --with-select_module \ From 3b42b8bb8c2c2f4f6baf3c566fe0dc059ec9f5c3 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 14 Nov 2016 15:19:43 -0800 Subject: [PATCH 020/848] tests: avoided solving IPv6 addresses. --- t/023-rewrite/tcp-socket.t | 8 ++++---- t/058-tcp-socket.t | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index cf9d80af2c..e84734cca6 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -204,7 +204,7 @@ attempt to send data on a closed socket: --- timeout: 10 --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { rewrite_by_lua ' @@ -296,7 +296,7 @@ qr/connect\(\) failed \(\d+: Connection refused\)/ === TEST 6: connection timeout (tcp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_socket_connect_timeout 100ms; lua_socket_send_timeout 100ms; lua_socket_read_timeout 100ms; @@ -372,7 +372,7 @@ connected: 1 === TEST 8: resolver error (host not found) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { rewrite_by_lua ' @@ -415,7 +415,7 @@ attempt to send data on a closed socket === TEST 9: resolver error (timeout) --- config server_tokens off; - resolver 8.8.8.8; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 1ms; location /t { rewrite_by_lua ' diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index acf69f0408..5b0d79005a 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -2038,7 +2038,7 @@ resolve name done === TEST 34: github issue #215: Handle the posted requests in lua cosocket api (successfully resolved) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 5s; location = /sub { From 0a83639f1985e0add654369b6a514fc8310bc8e5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 14 Nov 2016 16:08:51 -0800 Subject: [PATCH 021/848] tests: avoided solving IPv6 addresses in more tests. --- t/129-ssl-socket.t | 58 +++++++++++++++++++++++----------------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 9da9a5c245..cc14594cb3 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -125,7 +125,7 @@ SSL reused session === TEST 2: no SNI, no verify --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -202,7 +202,7 @@ SSL reused session === TEST 3: SNI, no verify --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -280,7 +280,7 @@ SSL reused session === TEST 4: ssl session reuse --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -374,7 +374,7 @@ lua ssl free session The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.org". --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 5; location /t { @@ -453,7 +453,7 @@ SSL reused session The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.org". --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_socket_log_errors off; lua_ssl_verify_depth 2; @@ -532,7 +532,7 @@ SSL reused session === TEST 7: certificate does not match host name (no verify) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -611,7 +611,7 @@ SSL reused session === TEST 8: iscribblet.org: passing SSL verify --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 2; location /t { @@ -696,7 +696,7 @@ SSL reused session === TEST 9: ssl verify depth not enough (with automatic error logging) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 1; location /t { @@ -774,7 +774,7 @@ SSL reused session === TEST 10: ssl verify depth not enough (without automatic error logging) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 1; lua_socket_log_errors off; @@ -1033,7 +1033,7 @@ SSL reused session === TEST 13: iscribblet.org: passing SSL verify with multiple certificates --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 2; location /t { @@ -1119,7 +1119,7 @@ SSL reused session === TEST 14: default cipher --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1198,7 +1198,7 @@ SSL reused session === TEST 15: explicit cipher configuration --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_ciphers RC4-SHA; location /t { #set $port 5000; @@ -1278,7 +1278,7 @@ SSL reused session === TEST 16: explicit ssl protocol configuration --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_protocols TLSv1; location /t { #set $port 5000; @@ -1358,7 +1358,7 @@ SSL reused session === TEST 17: unsupported ssl protocol --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_protocols SSLv2; lua_socket_log_errors off; location /t { @@ -1435,7 +1435,7 @@ SSL reused session === TEST 18: iscribblet.org: passing SSL verify: keepalive (reuse the ssl session) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 2; location /t { @@ -1512,7 +1512,7 @@ SSL reused session === TEST 19: iscribblet.org: passing SSL verify: keepalive (no reusing the ssl session) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 2; location /t { @@ -1592,7 +1592,7 @@ SSL reused session === TEST 20: downstream cosockets do not support ssl handshake --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; lua_ssl_verify_depth 2; location /t { @@ -1647,7 +1647,7 @@ attempt to call method 'sslhandshake' (a nil value) } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1750,7 +1750,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/test.crt; location /t { @@ -1854,7 +1854,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1946,7 +1946,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_crl ../html/test.crl; lua_ssl_trusted_certificate ../html/test.crt; lua_socket_log_errors off; @@ -2031,7 +2031,7 @@ SSL reused session === TEST 25: multiple handshake calls --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2115,7 +2115,7 @@ SSL reused session === TEST 26: handshake timed out --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2183,7 +2183,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2254,7 +2254,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2328,7 +2328,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2405,7 +2405,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/test.crt; location /t { @@ -2510,7 +2510,7 @@ SSL reused session } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; #lua_ssl_trusted_certificate ../html/test.crt; location /t { @@ -2589,7 +2589,7 @@ SSL reused session === TEST 32: handshake, too many arguments --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; From 38dfb174a219754e8275ca7b41bf3d8ef7840fcf Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 14 Nov 2016 20:33:17 -0800 Subject: [PATCH 022/848] tests: fixed a bug in the example ngx_fake_delayed module. thanks Dejiang Zhu. also removed the copyright notice in this module since it is already granted. --- .../ngx_http_lua_fake_delayed_load_module.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c b/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c index 8548d878e7..2898255c21 100644 --- a/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c +++ b/t/data/fake-delayed-load-module/ngx_http_lua_fake_delayed_load_module.c @@ -1,5 +1,4 @@ -/* Copyright (C) Jim Tan - * +/* * This fake_delayed_load delayed load module was used to reproduce * a bug in ngx_lua's function ngx_http_lua_add_package_preload. */ @@ -74,5 +73,5 @@ ngx_http_lua_fake_delayed_load_preload(lua_State *L) static int ngx_http_lua_fake_delayed_load_function(lua_State * L) { - return 1; + return 0; } From a24db2cb992494d6e30fb3b96812b677f6b0e63a Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 14 Nov 2016 20:34:36 -0800 Subject: [PATCH 023/848] tests: ngx_http_fake_module: removed the coyyright notice since the copyright is already granted. --- t/data/fake-module/ngx_http_fake_module.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/t/data/fake-module/ngx_http_fake_module.c b/t/data/fake-module/ngx_http_fake_module.c index 42cde555bd..650f4f7ccc 100644 --- a/t/data/fake-module/ngx_http_fake_module.c +++ b/t/data/fake-module/ngx_http_fake_module.c @@ -1,5 +1,4 @@ -/* Copyright (C) ZHANG Heng (chiyouhen) - * +/* * This fake module was used to reproduce a bug in ngx_lua's * init_worker_by_lua implementation. */ From 4bc90422224c7274795754f23df84be513e7282f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 14 Nov 2016 20:45:29 -0800 Subject: [PATCH 024/848] tests: fixed a perl warning "Unescaped left brace in regex is deprecated, passed through in regex". --- t/026-mysql.t | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/t/026-mysql.t b/t/026-mysql.t index e14ffb6f14..569f96fbc9 100644 --- a/t/026-mysql.t +++ b/t/026-mysql.t @@ -66,7 +66,7 @@ __DATA__ ^status = 504 thread id = \d+ kill status = 200 -kill body = {"errcode":0}$ +kill body = \{"errcode":0\}$ --- error_log eval qr{upstream timed out \(\d+: Connection timed out\) while sending query to drizzle upstream} @@ -126,6 +126,5 @@ qr{upstream timed out \(\d+: Connection timed out\) while sending query to drizz ^status = 504 thread id = \d+ kill status = 200 -kill body = {"errcode":0}$ +kill body = \{"errcode":0\}$ --- SKIP - From 5e59e8c897c4256dc14bdd7f1b2f0b8d090a9434 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 16 Nov 2016 08:41:47 +0000 Subject: [PATCH 025/848] bugfix: compilation failures with nginx cores older than 1.9.0. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_api.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index 6c3d83de50..7b590e7a9f 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -175,7 +175,9 @@ ngx_http_lua_shared_memory_init(ngx_shm_zone_t *shm_zone, void *data) } zone->shm = shm_zone->shm; +#if defined(nginx_version) && nginx_version >= 1009000 zone->noreuse = shm_zone->noreuse; +#endif if (zone->init(zone, odata) != NGX_OK) { return NGX_ERROR; From 793531df5451e6cc45cdce0319302add72c79e90 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 18 Nov 2016 17:34:27 -0800 Subject: [PATCH 026/848] doc: removed ngx.re.split() from TODO section since it is already implemented in ngx.re. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 1 - doc/HttpLuaModule.wiki | 1 - 2 files changed, 2 deletions(-) diff --git a/README.markdown b/README.markdown index eaac1456c9..ea57ef1bce 100644 --- a/README.markdown +++ b/README.markdown @@ -871,7 +871,6 @@ servers in Lua. For example, * cosocket: pool-based backend concurrency level control: implement automatic `connect` queueing when the backend concurrency exceeds its connection pool limit. * cosocket: review and merge aviramc's [patch](https://github.com/openresty/lua-nginx-module/pull/290) for adding the `bsdrecv` method. * add new API function `ngx.resp.add_header` to emulate the standard `add_header` config directive. -* review and apply Jader H. Silva's patch for `ngx.re.split()`. * review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option * use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), [ngx.header.HEADER](#ngxheaderheader), and etc. * add configure options for different strategies of handling the cosocket connection exceeding in the pools. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index aa357a1c8d..876e7d9970 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -705,7 +705,6 @@ servers in Lua. For example, * cosocket: pool-based backend concurrency level control: implement automatic connect queueing when the backend concurrency exceeds its connection pool limit. * cosocket: review and merge aviramc's [https://github.com/openresty/lua-nginx-module/pull/290 patch] for adding the bsdrecv method. * add new API function ngx.resp.add_header to emulate the standard add_header config directive. -* review and apply Jader H. Silva's patch for ngx.re.split(). * review and apply vadim-pavlov's patch for [[#ngx.location.capture|ngx.location.capture]]'s extra_headers option * use ngx_hash_t to optimize the built-in header look-up process for [[#ngx.req.set_header|ngx.req.set_header]], [[#ngx.header.HEADER|ngx.header.HEADER]], and etc. * add configure options for different strategies of handling the cosocket connection exceeding in the pools. From aa43ca0639255af9decb2f2affada6087cd69c4d Mon Sep 17 00:00:00 2001 From: Andreas Lubbe Date: Mon, 14 Nov 2016 09:13:52 +0100 Subject: [PATCH 027/848] doc: stripped line trailing whitespaces. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 50 ++++++------- doc/HttpLuaModule.wiki | 156 ++++++++++++++++++++--------------------- 2 files changed, 103 insertions(+), 103 deletions(-) diff --git a/README.markdown b/README.markdown index ea57ef1bce..21f0816d4e 100644 --- a/README.markdown +++ b/README.markdown @@ -623,9 +623,9 @@ Known Issues TCP socket connect operation issues ----------------------------------- -The [tcpsock:connect](#tcpsockconnect) method may indicate `success` despite connection failures such as with `Connection Refused` errors. +The [tcpsock:connect](#tcpsockconnect) method may indicate `success` despite connection failures such as with `Connection Refused` errors. -However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation. +However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation. This issue is due to limitations in the Nginx event model and only appears to affect Mac OS X. @@ -755,7 +755,7 @@ To avoid this, *double* escape the backslash: Here, `\\\\d+` is stripped down to `\\d+` by the Nginx config file parser and this is further stripped down to `\d+` by the Lua language parser before running. -Alternatively, the regex pattern can be presented as a long-bracketed Lua string literal by encasing it in "long brackets", `[[...]]`, in which case backslashes have to only be escaped once for the Nginx config file parser. +Alternatively, the regex pattern can be presented as a long-bracketed Lua string literal by encasing it in "long brackets", `[[...]]`, in which case backslashes have to only be escaped once for the Nginx config file parser. ```nginx @@ -772,7 +772,7 @@ Alternatively, the regex pattern can be presented as a long-bracketed Lua string Here, `[[\\d+]]` is stripped down to `[[\d+]]` by the Nginx config file parser and this is processed correctly. -Note that a longer from of the long bracket, `[=[...]=]`, may be required if the regex pattern contains `[...]` sequences. +Note that a longer from of the long bracket, `[=[...]=]`, may be required if the regex pattern contains `[...]` sequences. The `[=[...]=]` form may be used as the default form if desired. ```nginx @@ -788,7 +788,7 @@ The `[=[...]=]` form may be used as the default form if desired. # evaluates to "1234" ``` -An alternative approach to escaping PCRE sequences is to ensure that Lua code is placed in external script files and executed using the various `*_by_lua_file` directives. +An alternative approach to escaping PCRE sequences is to ensure that Lua code is placed in external script files and executed using the various `*_by_lua_file` directives. With this approach, the backslashes are only stripped by the Lua language parser and therefore only need to be escaped once each. ```lua @@ -800,8 +800,8 @@ With this approach, the backslashes are only stripped by the Lua language parser -- evaluates to "1234" ``` -Within external script files, PCRE sequences presented as long-bracketed Lua string literals do not require modification. - +Within external script files, PCRE sequences presented as long-bracketed Lua string literals do not require modification. + ```lua -- test.lua @@ -1152,7 +1152,7 @@ The ngx_lua module does not support the `stat` mode available with the Apache `mod_lua` module (yet). Disabling the Lua code cache is strongly -discouraged for production use and should only be used during +discouraged for production use and should only be used during development as it has a significant negative impact on overall performance. For example, the performance a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. [Back to TOC](#directives) @@ -1538,15 +1538,15 @@ set_by_lua_file **phase:** *rewrite* -Equivalent to [set_by_lua](#set_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [set_by_lua](#set_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. Nginx variable interpolation is supported in the `` argument string of this directive. But special care must be taken for injection attacks. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. -When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. -The Lua code cache can be temporarily disabled during development by +The Lua code cache can be temporarily disabled during development by switching [lua_code_cache](#lua_code_cache) `off` in `nginx.conf` to avoid reloading Nginx. This directive requires the [ngx_devel_kit](https://github.com/simpl/ngx_devel_kit) module. @@ -1565,7 +1565,7 @@ content_by_lua **WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; use the new [content_by_lua_block](#content_by_lua_block) directive instead. -Acts as a "content handler" and executes Lua code string specified in `` for every request. +Acts as a "content handler" and executes Lua code string specified in `` for every request. The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). Do not use this directive and other content handler directives in the same location. For example, this directive and the [proxy_pass](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive should not be used in the same location. @@ -1614,9 +1614,9 @@ Nginx variables can be used in the `` string to provide When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. -When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. -The Lua code cache can be temporarily disabled during development by +The Lua code cache can be temporarily disabled during development by switching [lua_code_cache](#lua_code_cache) `off` in `nginx.conf` to avoid reloading Nginx. Nginx variables are supported in the file path for dynamic dispatch, for example: @@ -1934,7 +1934,7 @@ Nginx variables can be used in the `` string to provide When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. -When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching [lua_code_cache](#lua_code_cache) `off` in `nginx.conf` to avoid repeatedly reloading Nginx. @@ -2175,7 +2175,7 @@ Runs the Lua source code inlined as the `` at the `log` request Note that the following API functions are currently disabled within this context: * Output API functions (e.g., [ngx.say](#ngxsay) and [ngx.send_headers](#ngxsend_headers)) -* Control API functions (e.g., [ngx.exit](#ngxexit)) +* Control API functions (e.g., [ngx.exit](#ngxexit)) * Subrequest API functions (e.g., [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi)) * Cosocket API functions (e.g., [ngx.socket.tcp](#ngxsockettcp) and [ngx.req.socket](#ngxreqsocket)). @@ -3439,7 +3439,7 @@ ngx.ctx ------- **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua** -This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). +This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). Consider the following example, @@ -3702,7 +3702,7 @@ The `args` option can also take plain query strings: This is functionally identical to the previous examples. -The `share_all_vars` option controls whether to share nginx variables among the current request and its subrequests. +The `share_all_vars` option controls whether to share nginx variables among the current request and its subrequests. If this option is set to `true`, then the current request and associated subrequests will share the same Nginx variable scope. Hence, changes to Nginx variables made by a subrequest will affect the current request. Care should be taken in using this option as variable scope sharing can have unexpected side effects. The `args`, `vars`, or `copy_all_vars` options are generally preferable instead. @@ -3769,7 +3769,7 @@ In addition to the two settings above, it is possible to specify values for variables in the subrequest using the `vars` option. These variables are set after the sharing or copying of variables has been evaluated, and provides a more efficient method of passing specific -values to a subrequest over encoding them as URL arguments and +values to a subrequest over encoding them as URL arguments and unescaping them in the Nginx config file. ```nginx @@ -3854,7 +3854,7 @@ Note that subrequests issued by [ngx.location.capture](#ngxlocationcapture) inhe request headers of the current request by default and that this may have unexpected side effects on the subrequest responses. For example, when using the standard `ngx_proxy` module to serve subrequests, an "Accept-Encoding: gzip" header in the main request may result -in gzipped responses that cannot be handled properly in Lua code. Original request headers should be ignored by setting +in gzipped responses that cannot be handled properly in Lua code. Original request headers should be ignored by setting [proxy_pass_request_headers](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers) to `off` in subrequest locations. When the `body` option is not specified and the `always_forward_body` option is false (the default value), the `POST` and `PUT` subrequests will inherit the request bodies of the parent request (if any). @@ -3994,7 +3994,7 @@ will yield Set-Cookie: b=4; path=/ ``` -in the response headers. +in the response headers. Only Lua tables are accepted (Only the last element in the table will take effect for standard headers such as `Content-Type` that only accept a single value). @@ -4026,7 +4026,7 @@ The same applies to assigning an empty table: Setting `ngx.header.HEADER` after sending out response headers (either explicitly with [ngx.send_headers](#ngxsend_headers) or implicitly with [ngx.print](#ngxprint) and similar) will throw out a Lua exception. -Reading `ngx.header.HEADER` will return the value of the response header named `HEADER`. +Reading `ngx.header.HEADER` will return the value of the response header named `HEADER`. Underscores (`_`) in the header names will also be replaced by dashes (`-`) and the header names will be matched case-insensitively. If the response header is not present at all, `nil` will be returned. @@ -4537,7 +4537,7 @@ That is, they will take Lua boolean values `true`. However, they are different f Empty key arguments are discarded. `POST /test` with body `=hello&=world` will yield empty outputs for instance. -Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. However, the optional `max_args` function argument can be used to override this limit: @@ -4598,7 +4598,7 @@ the value of `ngx.req.get_headers()["Foo"]` will be a Lua (array) table such as: {"foo", "bar", "baz"} ``` -Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. However, the optional `max_headers` function argument can be used to override this limit: @@ -6007,7 +6007,7 @@ When the `replace` is a string, then it is treated as a special template for str where `$0` referring to the whole substring matched by the pattern and `$1` referring to the first parenthesized capturing substring. -Curly braces can also be used to disambiguate variable names from the background string literals: +Curly braces can also be used to disambiguate variable names from the background string literals: ```lua diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 876e7d9970..5b1d8bcc2b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -16,34 +16,34 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t # set search paths for pure Lua external libraries (';;' is the default path): lua_package_path '/foo/bar/?.lua;/blah/?.lua;;'; - + # set search paths for Lua external libraries written in C (can also use ';;'): lua_package_cpath '/bar/baz/?.so;/blah/blah/?.so;;'; - + server { location /lua_content { # MIME type determined by default_type: default_type 'text/plain'; - + content_by_lua_block { ngx.say('Hello,world!') } } - + location /nginx_var { # MIME type determined by default_type: default_type 'text/plain'; - + # try access /nginx_var?a=hello,world content_by_lua_block { ngx.say(ngx.var.arg_a) } } - + location = /request_body { client_max_body_size 50k; client_body_buffer_size 50k; - + content_by_lua_block { ngx.req.read_body() -- explicitly read the req body local data = ngx.req.get_body_data() @@ -62,13 +62,13 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t end } } - + # transparent non-blocking I/O in Lua via subrequests # (well, a better way is to use cosockets) location = /lua { # MIME type determined by default_type: default_type 'text/plain'; - + content_by_lua_block { local res = ngx.location.capture("/some_other_location") if res then @@ -78,23 +78,23 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t end } } - + location = /foo { rewrite_by_lua_block { res = ngx.location.capture("/memc", { args = { cmd = "incr", key = ngx.var.uri } } ) } - + proxy_pass http://blah.blah.com; } - + location = /mixed { rewrite_by_lua_file /path/to/rewrite.lua; access_by_lua_file /path/to/access.lua; content_by_lua_file /path/to/content.lua; } - + # use nginx var in code path # WARNING: contents in nginx var must be carefully filtered, # otherwise there'll be great security risk! @@ -102,27 +102,27 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t set $path $1; content_by_lua_file /path/to/lua/app/root/$path.lua; } - + location / { client_max_body_size 100k; client_body_buffer_size 100k; - + access_by_lua_block { -- check the client IP address is in our black list if ngx.var.remote_addr == "132.5.72.3" then ngx.exit(ngx.HTTP_FORBIDDEN) end - + -- check if the URI contains bad words if ngx.var.uri and string.match(ngx.var.request_body, "evil") then return ngx.redirect("/terms_of_use.html") end - + -- tests passed } - + # proxy_pass/fastcgi_pass/etc settings } } @@ -220,17 +220,17 @@ Build the source with this module: # tell nginx's build system where to find LuaJIT 2.1: export LUAJIT_LIB=/path/to/luajit/lib export LUAJIT_INC=/path/to/luajit/include/luajit-2.1 - + # or tell where to find Lua if using Lua instead: #export LUA_LIB=/path/to/lua/lib #export LUA_INC=/path/to/lua/include - + # Here we assume Nginx is to be installed under /opt/nginx/. ./configure --prefix=/opt/nginx \ --with-ld-opt="-Wl,-rpath,/path/to/luajit-or-lua/lib" \ --add-module=/path/to/ngx_devel_kit \ --add-module=/path/to/lua-nginx-module - + make -j2 make install @@ -451,7 +451,7 @@ Here is a complete small example: cat = 4, pig = 5, } - + function _M.get_age(name) return data[name] end @@ -495,9 +495,9 @@ If server-wide data sharing is required, then use one or more of the following a = Known Issues = == TCP socket connect operation issues == -The [[#tcpsock:connect|tcpsock:connect]] method may indicate success despite connection failures such as with Connection Refused errors. +The [[#tcpsock:connect|tcpsock:connect]] method may indicate success despite connection failures such as with Connection Refused errors. -However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation. +However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation. This issue is due to limitations in the Nginx event model and only appears to affect Mac OS X. @@ -606,7 +606,7 @@ To avoid this, ''double'' escape the backslash: Here, \\\\d+ is stripped down to \\d+ by the Nginx config file parser and this is further stripped down to \d+ by the Lua language parser before running. -Alternatively, the regex pattern can be presented as a long-bracketed Lua string literal by encasing it in "long brackets", [[...]], in which case backslashes have to only be escaped once for the Nginx config file parser. +Alternatively, the regex pattern can be presented as a long-bracketed Lua string literal by encasing it in "long brackets", [[...]], in which case backslashes have to only be escaped once for the Nginx config file parser. # nginx.conf @@ -622,7 +622,7 @@ Alternatively, the regex pattern can be presented as a long-bracketed Lua string Here, [[\\d+]] is stripped down to [[\d+]] by the Nginx config file parser and this is processed correctly. -Note that a longer from of the long bracket, [=[...]=], may be required if the regex pattern contains [...] sequences. +Note that a longer from of the long bracket, [=[...]=], may be required if the regex pattern contains [...] sequences. The [=[...]=] form may be used as the default form if desired. @@ -637,7 +637,7 @@ The [=[...]=] form may be used as the default form if desired. # evaluates to "1234" -An alternative approach to escaping PCRE sequences is to ensure that Lua code is placed in external script files and executed using the various *_by_lua_file directives. +An alternative approach to escaping PCRE sequences is to ensure that Lua code is placed in external script files and executed using the various *_by_lua_file directives. With this approach, the backslashes are only stripped by the Lua language parser and therefore only need to be escaped once each. @@ -648,8 +648,8 @@ With this approach, the backslashes are only stripped by the Lua language parser -- evaluates to "1234" -Within external script files, PCRE sequences presented as long-bracketed Lua string literals do not require modification. - +Within external script files, PCRE sequences presented as long-bracketed Lua string literals do not require modification. + -- test.lua local regex = [[\d+]] @@ -897,7 +897,7 @@ The ngx_lua module does not support the stat mode available with th Apache mod_lua module (yet). Disabling the Lua code cache is strongly -discouraged for production use and should only be used during +discouraged for production use and should only be used during development as it has a significant negative impact on overall performance. For example, the performance a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. == lua_regex_cache_max_entries == @@ -1178,15 +1178,15 @@ a time. However, a workaround is possible using the [[#ngx.var.VARIABLE|ngx.var. location /foo { set $diff ''; # we have to predefine the $diff variable here - + set_by_lua $sum ' local a = 32 local b = 56 - + ngx.var.diff = a - b; -- write to $diff directly return a + b; -- return the $sum value normally '; - + echo "sum = $sum, diff = $diff"; } @@ -1236,15 +1236,15 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''rewrite'' -Equivalent to [[#set_by_lua|set_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#set_by_lua|set_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. Nginx variable interpolation is supported in the argument string of this directive. But special care must be taken for injection attacks. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. -When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. -The Lua code cache can be temporarily disabled during development by +The Lua code cache can be temporarily disabled during development by switching [[#lua_code_cache|lua_code_cache]] off in nginx.conf to avoid reloading Nginx. This directive requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_kit] module. @@ -1260,7 +1260,7 @@ This directive requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_ki '''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; use the new [[#content_by_lua_block|content_by_lua_block]] directive instead. -Acts as a "content handler" and executes Lua code string specified in for every request. +Acts as a "content handler" and executes Lua code string specified in for every request. The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). Do not use this directive and other content handler directives in the same location. For example, this directive and the [[HttpProxyModule#proxy_pass|proxy_pass]] directive should not be used in the same location. @@ -1302,9 +1302,9 @@ Nginx variables can be used in the string When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. -When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. -The Lua code cache can be temporarily disabled during development by +The Lua code cache can be temporarily disabled during development by switching [[#lua_code_cache|lua_code_cache]] off in nginx.conf to avoid reloading Nginx. Nginx variables are supported in the file path for dynamic dispatch, for example: @@ -1377,7 +1377,7 @@ The right way of doing this is as follows: return ngx.redirect("/bar"); end '; - + echo "res = $b"; } @@ -1389,11 +1389,11 @@ Note that the [http://www.grid.net.ru/nginx/eval.en.html ngx_eval] module can be eval $res { proxy_pass http://foo.com/check-spam; } - + if ($res = 'spam') { rewrite ^ /terms-of-use.html redirect; } - + fastcgi_pass ...; } @@ -1405,7 +1405,7 @@ can be implemented in ngx_lua as: internal; proxy_pass http://foo.com/check-spam; } - + location / { rewrite_by_lua ' local res = ngx.location.capture("/check-spam") @@ -1413,7 +1413,7 @@ can be implemented in ngx_lua as: return ngx.redirect("/terms-of-use.html") end '; - + fastcgi_pass ...; } @@ -1503,12 +1503,12 @@ Note that this handler always runs ''after'' the standard [[HttpAccessModule]]. allow 192.168.1.0/24; allow 10.1.1.0/16; deny all; - + access_by_lua ' local res = ngx.location.capture("/mysql", { ... }) ... '; - + # proxy_pass/fastcgi_pass/... } @@ -1520,7 +1520,7 @@ Note that the [http://mdounin.ru/hg/ngx_http_auth_request_module/ ngx_auth_reque location / { auth_request /auth; - + # proxy_pass/fastcgi_pass/postgres_pass/... } @@ -1531,18 +1531,18 @@ can be implemented in ngx_lua as: location / { access_by_lua ' local res = ngx.location.capture("/auth") - + if res.status == ngx.HTTP_OK then return end - + if res.status == ngx.HTTP_FORBIDDEN then ngx.exit(res.status) end - + ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) '; - + # proxy_pass/fastcgi_pass/postgres_pass/... } @@ -1592,7 +1592,7 @@ Nginx variables can be used in the string When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. -When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching [[#lua_code_cache|lua_code_cache]] off in nginx.conf to avoid repeatedly reloading Nginx. @@ -1805,7 +1805,7 @@ Runs the Lua source code inlined as the at the @@ -2781,7 +2781,7 @@ There is a hard coded 2048 byte limitation on error message lengths == ngx.ctx == '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*'' -This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). +This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). Consider the following example, @@ -2817,7 +2817,7 @@ Every request, including subrequests, has its own copy of the table. For example ngx.say("sub post: ", ngx.ctx.blah) } } - + location /main { content_by_lua_block { ngx.ctx.blah = 73 @@ -2848,7 +2848,7 @@ Internal redirection will destroy the original request ngx.ctx data ngx.say(ngx.ctx.foo) } } - + location /orig { content_by_lua_block { ngx.ctx.foo = "hello" @@ -3025,7 +3025,7 @@ The args option can also take plain query strings: This is functionally identical to the previous examples. -The share_all_vars option controls whether to share nginx variables among the current request and its subrequests. +The share_all_vars option controls whether to share nginx variables among the current request and its subrequests. If this option is set to true, then the current request and associated subrequests will share the same Nginx variable scope. Hence, changes to Nginx variables made by a subrequest will affect the current request. Care should be taken in using this option as variable scope sharing can have unexpected side effects. The args, vars, or copy_all_vars options are generally preferable instead. @@ -3090,7 +3090,7 @@ In addition to the two settings above, it is possible to specify values for variables in the subrequest using the vars option. These variables are set after the sharing or copying of variables has been evaluated, and provides a more efficient method of passing specific -values to a subrequest over encoding them as URL arguments and +values to a subrequest over encoding them as URL arguments and unescaping them in the Nginx config file. @@ -3172,7 +3172,7 @@ Note that subrequests issued by [[#ngx.location.capture|ngx.location.capture]] i request headers of the current request by default and that this may have unexpected side effects on the subrequest responses. For example, when using the standard ngx_proxy module to serve subrequests, an "Accept-Encoding: gzip" header in the main request may result -in gzipped responses that cannot be handled properly in Lua code. Original request headers should be ignored by setting +in gzipped responses that cannot be handled properly in Lua code. Original request headers should be ignored by setting [[HttpProxyModule#proxy_pass_request_headers|proxy_pass_request_headers]] to off in subrequest locations. When the body option is not specified and the always_forward_body option is false (the default value), the POST and PUT subrequests will inherit the request bodies of the parent request (if any). @@ -3202,11 +3202,11 @@ This function issues several parallel subrequests specified by the input table a { "/bar" }, { "/baz", { method = ngx.HTTP_POST, body = "hello" } }, } - + if res1.status == ngx.HTTP_OK then ... end - + if res2.body == "BLAH" then ... end @@ -3224,10 +3224,10 @@ Lua tables can be used for both requests and responses when the number of subreq table.insert(reqs, { "/postgres" }) table.insert(reqs, { "/redis" }) table.insert(reqs, { "/memcached" }) - + -- issue all the requests at once and wait until they all return local resps = { ngx.location.capture_multi(reqs) } - + -- loop over the responses table for i, resp in ipairs(resps) do -- process the response table "resp" @@ -3279,7 +3279,7 @@ The header names are matched case-insensitively. -- equivalent to ngx.header["Content-Type"] = 'text/plain' ngx.header.content_type = 'text/plain'; - + ngx.header["X-My-Header"] = 'blah blah'; @@ -3296,7 +3296,7 @@ will yield Set-Cookie: b=4; path=/ -in the response headers. +in the response headers. Only Lua tables are accepted (Only the last element in the table will take effect for standard headers such as Content-Type that only accept a single value). @@ -3324,7 +3324,7 @@ The same applies to assigning an empty table: Setting ngx.header.HEADER after sending out response headers (either explicitly with [[#ngx.send_headers|ngx.send_headers]] or implicitly with [[#ngx.print|ngx.print]] and similar) will throw out a Lua exception. -Reading ngx.header.HEADER will return the value of the response header named HEADER. +Reading ngx.header.HEADER will return the value of the response header named HEADER. Underscores (_) in the header names will also be replaced by dashes (-) and the header names will be matched case-insensitively. If the response header is not present at all, nil will be returned. @@ -3656,8 +3656,8 @@ Arguments without the = parts are treated as boolean argumen That is, they will take Lua boolean values true. However, they are different from arguments taking empty string values. GET /test?foo=&bar= will give something like - foo: - bar: + foo: + bar: Empty key arguments are discarded. GET /test?=hello&=world will yield an empty output for instance. @@ -3761,13 +3761,13 @@ Arguments without the = parts are treated as boolean argumen That is, they will take Lua boolean values true. However, they are different from arguments taking empty string values. POST /test with request body foo=&bar= will return something like - foo: - bar: + foo: + bar: Empty key arguments are discarded. POST /test with body =hello&=world will yield empty outputs for instance. -Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. However, the optional max_args function argument can be used to override this limit: @@ -3819,7 +3819,7 @@ the value of ngx.req.get_headers()["Foo"] will be a Lua (array) tab {"foo", "bar", "baz"} -Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. However, the optional max_headers function argument can be used to override this limit: @@ -5025,7 +5025,7 @@ When the replace is a string, then it is treated as a special templ where $0 referring to the whole substring matched by the pattern and $1 referring to the first parenthesized capturing substring. -Curly braces can also be used to disambiguate variable names from the background string literals: +Curly braces can also be used to disambiguate variable names from the background string literals: local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "${0}00") @@ -6188,7 +6188,7 @@ Then it will generate the output ngx.thread.spawn(query_mysql) -- create thread 1 ngx.thread.spawn(query_memcached) -- create thread 2 - ngx.thread.spawn(query_http) -- create thread 3 + ngx.thread.spawn(query_http) -- create thread 3 This API was first enabled in the v0.7.0 release. @@ -6414,7 +6414,7 @@ One can also create infinite re-occurring timers, for instance, a timer getting return end end - + local ok, err = ngx.timer.at(delay, handler) if not ok then ngx.log(ngx.ERR, "failed to create the timer: ", err) From db28ece8e4c3c7956e07305654f750c472be0393 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 1 Dec 2016 10:24:53 -0800 Subject: [PATCH 028/848] doc: a typo fix from Andrei Belov. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 21f0816d4e..bc16200531 100644 --- a/README.markdown +++ b/README.markdown @@ -2884,7 +2884,7 @@ lua_http10_buffering **context:** *http, server, location, location-if* -Enables or disables automatic response buffering for HTTP 1.0 (or older) requests. This buffering mechanism is mainly used for HTTP 1.0 keep-alive which replies on a proper `Content-Length` response header. +Enables or disables automatic response buffering for HTTP 1.0 (or older) requests. This buffering mechanism is mainly used for HTTP 1.0 keep-alive which relies on a proper `Content-Length` response header. If the Lua code explicitly sets a `Content-Length` response header before sending the headers (either explicitly via [ngx.send_headers](#ngxsend_headers) or implicitly via the first [ngx.say](#ngxsay) or [ngx.print](#ngxprint) call), then the HTTP 1.0 response buffering will be disabled even when this directive is turned on. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 5b1d8bcc2b..693275ca44 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2431,7 +2431,7 @@ See also [[#lua_ssl_trusted_certificate|lua_ssl_trusted_certificate]]. '''context:''' ''http, server, location, location-if'' -Enables or disables automatic response buffering for HTTP 1.0 (or older) requests. This buffering mechanism is mainly used for HTTP 1.0 keep-alive which replies on a proper Content-Length response header. +Enables or disables automatic response buffering for HTTP 1.0 (or older) requests. This buffering mechanism is mainly used for HTTP 1.0 keep-alive which relies on a proper Content-Length response header. If the Lua code explicitly sets a Content-Length response header before sending the headers (either explicitly via [[#ngx.send_headers|ngx.send_headers]] or implicitly via the first [[#ngx.say|ngx.say]] or [[#ngx.print|ngx.print]] call), then the HTTP 1.0 response buffering will be disabled even when this directive is turned on. From 7c9503493501573a8383c6a59cf13b4131ae7826 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 12 Dec 2016 18:28:24 -0800 Subject: [PATCH 029/848] tests: added a passing test for ngx.re.find + \G regexes. --- t/120-re-find.t | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/t/120-re-find.t b/t/120-re-find.t index 34c0207ea5..92e33bdab0 100644 --- a/t/120-re-find.t +++ b/t/120-re-find.t @@ -891,3 +891,29 @@ not matched! --- no_error_log [error] + + +=== TEST 31: match with ctx and a pos (anchored by \G) +--- config + location /re { + content_by_lua ' + local ctx = { pos = 3 } + local from, to, err = ngx.re.find("1234, hello", [[(\G[0-9]+)]], "", ctx) + if from then + ngx.say("from: ", from) + ngx.say("to: ", to) + ngx.say("pos: ", ctx.pos) + else + ngx.say("not matched!") + ngx.say("pos: ", ctx.pos) + end + '; + } +--- request + GET /re +--- response_body +from: 3 +to: 4 +pos: 5 +--- no_error_log +[error] From 26be99deffc369608b9b10e558ad6fa5dfc3c03c Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 14 Dec 2016 12:34:33 -0800 Subject: [PATCH 030/848] tests: t/090-log-socket-errors.t: avoided connecting to 8.8.8.8:80. --- t/090-log-socket-errors.t | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/t/090-log-socket-errors.t b/t/090-log-socket-errors.t index a5943c27c6..1c3099d6e2 100644 --- a/t/090-log-socket-errors.t +++ b/t/090-log-socket-errors.t @@ -11,6 +11,8 @@ repeat_each(2); plan tests => repeat_each() * (blocks() * 3); +$ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; + #no_diff(); #no_long_string(); run_tests(); @@ -19,12 +21,14 @@ __DATA__ === TEST 1: log socket errors off (tcp) --- config + resolver $TEST_NGINX_RESOLVER; + location /t { lua_socket_connect_timeout 1ms; lua_socket_log_errors off; content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("8.8.8.8", 80) + local ok, err = sock:connect("agentzh.org", 12345) ngx.say(err) '; } @@ -39,12 +43,14 @@ timeout === TEST 2: log socket errors on (tcp) --- config + resolver $TEST_NGINX_RESOLVER; + location /t { lua_socket_connect_timeout 1ms; lua_socket_log_errors on; content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("8.8.8.8", 80) + local ok, err = sock:connect("agentzh.org", 12345) ngx.say(err) '; } @@ -59,12 +65,14 @@ lua tcp socket connect timed out === TEST 3: log socket errors on (udp) --- config + resolver $TEST_NGINX_RESOLVER; + location /t { lua_socket_log_errors on; lua_socket_read_timeout 1ms; content_by_lua ' local sock = ngx.socket.udp() - local ok, err = sock:setpeername("8.8.8.8", 80) + local ok, err = sock:setpeername("agentzh.org", 12345) ok, err = sock:receive() ngx.say(err) '; @@ -80,12 +88,14 @@ lua udp socket read timed out === TEST 4: log socket errors off (udp) --- config + resolver $TEST_NGINX_RESOLVER; + location /t { lua_socket_log_errors off; lua_socket_read_timeout 1ms; content_by_lua ' local sock = ngx.socket.udp() - local ok, err = sock:setpeername("8.8.8.8", 80) + local ok, err = sock:setpeername("agentzh.org", 12345) ok, err = sock:receive() ngx.say(err) '; @@ -96,4 +106,3 @@ GET /t timeout --- no_error_log [error] - From 9ed76a7328aa65e265b9140b319bf18e070ae317 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Wed, 14 Dec 2016 23:07:51 +0500 Subject: [PATCH 031/848] travis-ci: several improvements. * moved mysql to the right place * simplified wget calls * no, we were not going to cache drizzle (cache only works on successful build) * apt actually does not cache (I've looked to the travis-ci sources) Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/.travis.yml b/.travis.yml index dc1185e106..28d0f087ec 100644 --- a/.travis.yml +++ b/.travis.yml @@ -12,7 +12,6 @@ compiler: addons: apt: packages: - - mysql-server - axel - cpanminus - libtest-base-perl @@ -24,7 +23,6 @@ addons: - libgd-dev cache: - apt: true directories: - download-cache @@ -57,13 +55,12 @@ env: services: - memcache - redis-server + - mysql install: - - if [ ! -d download-cache ]; then mkdir download-cache; fi - - rm -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz # alas, we may end up with corrupted drizzle tarball stuck in the download-cache/ forever. - - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -O download-cache/drizzle7-$DRIZZLE_VER.tar.gz http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -O download-cache/pcre-$PCRE_VER.tar.gz http://ftp.cs.stanford.edu/pub/exim/pcre/pcre-$PCRE_VER.tar.gz; fi - - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -O download-cache/openssl-$OPENSSL_VER.tar.gz https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz; fi + - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi + - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache http://ftp.cs.stanford.edu/pub/exim/pcre/pcre-$PCRE_VER.tar.gz; fi + - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz; fi - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx From 6e38c11eb401b9e8afa88ed51d78e1e2a989f1ae Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 14 Dec 2016 15:23:00 -0800 Subject: [PATCH 032/848] tests: disable IPv6 in resolver. --- t/014-bugs.t | 2 +- t/023-rewrite/tcp-socket-timeout.t | 28 ++++++++++----------- t/058-tcp-socket.t | 26 +++++++++---------- t/065-tcp-socket-timeout.t | 40 +++++++++++++++--------------- t/090-log-socket-errors.t | 8 +++--- t/091-coroutine.t | 8 +++--- t/124-init-worker.t | 2 +- t/127-uthread-kill.t | 2 +- t/142-ssl-session-store.t | 24 +++++++++--------- t/143-ssl-session-fetch.t | 24 +++++++++--------- t/147-tcp-socket-timeouts.t | 2 +- 11 files changed, 83 insertions(+), 83 deletions(-) diff --git a/t/014-bugs.t b/t/014-bugs.t index 44337e325b..8e5ee3f7f2 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -849,7 +849,7 @@ ok "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config location /t { - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; set $myhost 'agentzh.org.'; proxy_pass http://$myhost/misc/.vimrc; } diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index 79cd0b9495..68a27059a7 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -41,7 +41,7 @@ __DATA__ --- config server_tokens off; lua_socket_connect_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t1 { rewrite_by_lua ' @@ -73,7 +73,7 @@ lua tcp socket connect timed out server_tokens off; lua_socket_connect_timeout 60s; lua_socket_log_errors off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t2 { rewrite_by_lua ' @@ -108,7 +108,7 @@ lua tcp socket connect timeout: 150 server_tokens off; lua_socket_log_errors off; lua_socket_connect_timeout 102ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; #resolver_timeout 3s; location /t3 { rewrite_by_lua ' @@ -143,7 +143,7 @@ lua tcp socket connect timeout: 102 server_tokens off; lua_socket_connect_timeout 102ms; lua_socket_log_errors off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t4 { rewrite_by_lua ' @@ -179,7 +179,7 @@ lua tcp socket connect timeout: 102 server_tokens off; lua_socket_connect_timeout 102ms; lua_socket_log_errors off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t5 { rewrite_by_lua ' @@ -251,7 +251,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 60s; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -292,7 +292,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -333,7 +333,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -375,7 +375,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -416,7 +416,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_send_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -455,7 +455,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 60s; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -496,7 +496,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -537,7 +537,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() @@ -578,7 +578,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { rewrite_by_lua ' local sock = ngx.socket.tcp() diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 5b0d79005a..4734926a16 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -200,7 +200,7 @@ attempt to send data on a closed socket: --- timeout: 10 --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' @@ -290,7 +290,7 @@ qr/connect\(\) failed \(\d+: Connection refused\)/ === TEST 6: connection timeout (tcp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_socket_connect_timeout 100ms; lua_socket_send_timeout 100ms; lua_socket_read_timeout 100ms; @@ -362,7 +362,7 @@ connected: 1 === TEST 8: resolver error (host not found) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' @@ -403,7 +403,7 @@ attempt to send data on a closed socket === TEST 9: resolver error (timeout) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 1ms; location /t { content_by_lua ' @@ -1995,7 +1995,7 @@ close: 1 nil === TEST 33: github issue #215: Handle the posted requests in lua cosocket api (failed to resolve) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location = /sub { content_by_lua ' @@ -2136,7 +2136,7 @@ close: nil closed --- config server_tokens off; lua_socket_read_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -2178,7 +2178,7 @@ lua tcp socket read timed out === TEST 37: successful reread after a read time out happen (receive -> receive) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -2255,7 +2255,7 @@ lua tcp socket read timed out === TEST 38: successful reread after a read time out happen (receive -> receiveuntil) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -2335,7 +2335,7 @@ lua tcp socket read timed out === TEST 39: successful reread after a read time out happen (receiveuntil -> receiveuntil) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -2417,7 +2417,7 @@ lua tcp socket read timed out === TEST 40: successful reread after a read time out happen (receiveuntil -> receive) --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -3015,7 +3015,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):16: bad request/ === TEST 50: cosocket resolving aborted by coroutine yielding failures (require) --- http_config lua_package_path "$prefix/html/?.lua;;"; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; --- config location = /t { @@ -3049,7 +3049,7 @@ runtime error: attempt to yield across C-call boundary === TEST 51: cosocket resolving aborted by coroutine yielding failures (xpcall err) --- http_config lua_package_path "$prefix/html/?.lua;;"; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; --- config location = /t { @@ -3307,7 +3307,7 @@ close: 1 nil --- config server_tokens off; lua_socket_connect_timeout 1s; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index ec7989128b..6f903e89f1 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -46,7 +46,7 @@ __DATA__ --- config server_tokens off; lua_socket_connect_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' @@ -75,7 +75,7 @@ lua tcp socket connect timed out --- config server_tokens off; lua_socket_connect_timeout 60s; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' @@ -105,7 +105,7 @@ lua tcp socket connect timed out --- config server_tokens off; lua_socket_connect_timeout 102ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -133,7 +133,7 @@ lua tcp socket connect timed out --- config server_tokens off; lua_socket_connect_timeout 102ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' @@ -163,7 +163,7 @@ lua tcp socket connect timed out --- config server_tokens off; lua_socket_connect_timeout 102ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -191,7 +191,7 @@ lua tcp socket connect timed out --- config server_tokens off; lua_socket_read_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -228,7 +228,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 60s; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -267,7 +267,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -306,7 +306,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -346,7 +346,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_read_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -385,7 +385,7 @@ lua tcp socket read timed out --- config server_tokens off; lua_socket_send_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -436,7 +436,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 60s; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -475,7 +475,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -514,7 +514,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -553,7 +553,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_send_timeout 102ms; - #resolver $TEST_NGINX_RESOLVER; + #resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -686,7 +686,7 @@ after --- config server_tokens off; lua_socket_connect_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /t { content_by_lua ' @@ -724,7 +724,7 @@ lua tcp socket connect timed out --- config server_tokens off; lua_socket_send_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -779,7 +779,7 @@ lua tcp socket write timed out === TEST 19: abort when upstream sockets pending on writes --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() @@ -832,7 +832,7 @@ lua tcp socket write timed out === TEST 20: abort when downstream socket pending on writes --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' ngx.send_headers() @@ -888,7 +888,7 @@ lua tcp socket write timed out --- config server_tokens off; lua_socket_read_timeout 100ms; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua ' local sock = ngx.socket.tcp() diff --git a/t/090-log-socket-errors.t b/t/090-log-socket-errors.t index 1c3099d6e2..8e498cdeba 100644 --- a/t/090-log-socket-errors.t +++ b/t/090-log-socket-errors.t @@ -21,7 +21,7 @@ __DATA__ === TEST 1: log socket errors off (tcp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { lua_socket_connect_timeout 1ms; @@ -43,7 +43,7 @@ timeout === TEST 2: log socket errors on (tcp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { lua_socket_connect_timeout 1ms; @@ -65,7 +65,7 @@ lua tcp socket connect timed out === TEST 3: log socket errors on (udp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { lua_socket_log_errors on; @@ -88,7 +88,7 @@ lua udp socket read timed out === TEST 4: log socket errors off (udp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { lua_socket_log_errors off; diff --git a/t/091-coroutine.t b/t/091-coroutine.t index b047ccb1f9..5f348c2950 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -160,7 +160,7 @@ cc3: 2 === TEST 3: basic coroutine and cosocket --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' function worker(url) @@ -359,7 +359,7 @@ GET /lua === TEST 7: coroutine wrap and cosocket --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' function worker(url) @@ -987,7 +987,7 @@ test10 --- http_config init_by_lua 'return'; --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' function worker(url) @@ -1041,7 +1041,7 @@ successfully connected to: agentzh.org init_by_lua_file html/init.lua; --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' function worker(url) diff --git a/t/124-init-worker.t b/t/124-init-worker.t index d6ea6754fa..22a943e39e 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -447,7 +447,7 @@ warn(): Thu, 01 Jan 1970 01:34:38 GMT --- timeout: 10 --- http_config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; init_worker_by_lua ' -- global diff --git a/t/127-uthread-kill.t b/t/127-uthread-kill.t index 2ab8abee15..7e7d3dd6af 100644 --- a/t/127-uthread-kill.t +++ b/t/127-uthread-kill.t @@ -190,7 +190,7 @@ resolve name done: -2 === TEST 4: kill pending connect --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' local ready = false diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 5c9fad35f6..73b6e1979b 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -38,7 +38,7 @@ __DATA__ } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -108,7 +108,7 @@ ssl_session_store_by_lua_block:1: ssl session store by lua is running! } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -183,7 +183,7 @@ API disabled in the context of ssl_session_store_by_lua* } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -275,7 +275,7 @@ my timer run! } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -343,7 +343,7 @@ API disabled in the context of ssl_session_store_by_lua* } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -415,7 +415,7 @@ ngx.exit does not yield and the error code is eaten. } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -488,7 +488,7 @@ ssl_session_store_by_lua*: handler return value: 0, sess new cb exit code: 0 } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -556,7 +556,7 @@ should never reached here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -627,7 +627,7 @@ get_phase: ssl_session_store } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -696,7 +696,7 @@ qr/elapsed in ssl cert by lua: 0.(?:09|1[01])\d+,/, } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -778,7 +778,7 @@ a.lua:1: ssl store session by lua is running! } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -849,7 +849,7 @@ qr/\[emerg\] .*? "ssl_session_store_by_lua_block" directive is not allowed here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index bd800ff891..701ead7296 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -39,7 +39,7 @@ __DATA__ } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -120,7 +120,7 @@ qr/ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!/s } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -204,7 +204,7 @@ qr/elapsed in ssl fetch session by lua: 0.(?:09|1[01])\d+,/, } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -305,7 +305,7 @@ qr/my timer run!/s } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -385,7 +385,7 @@ qr/received memc reply: OK/s } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -466,7 +466,7 @@ should never reached here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -548,7 +548,7 @@ should never reached here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -629,7 +629,7 @@ should never reached here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -712,7 +712,7 @@ should never reached here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; @@ -793,7 +793,7 @@ should never reached here } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -879,7 +879,7 @@ qr/get_phase: ssl_session_fetch/s --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { @@ -1049,7 +1049,7 @@ qr/\S+:\d+: ssl fetch sess by lua is running!/s } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; location /t { diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 52f015aee2..0689a9b29d 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -315,7 +315,7 @@ lua tcp socket write timed out === TEST 5: connection timeout (tcp) --- config - resolver $TEST_NGINX_RESOLVER; + resolver $TEST_NGINX_RESOLVER ipv6=off; resolver_timeout 3s; location /test { content_by_lua_block { From e7031127e33bf84d5e6a241707c06c497f381033 Mon Sep 17 00:00:00 2001 From: Tom Thorogood Date: Thu, 25 Aug 2016 13:26:06 +0930 Subject: [PATCH 033/848] feature: added support for the 303 status code in ngx.redirect(). Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 1 + doc/HttpLuaModule.wiki | 1 + src/ngx_http_lua_control.c | 4 ++- t/022-redirect.t | 53 +++++++++++++++++++++++++++++++++++++- 4 files changed, 57 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index bc16200531..162343ff0f 100644 --- a/README.markdown +++ b/README.markdown @@ -5000,6 +5000,7 @@ The optional `status` parameter specifies the HTTP status code to be used. The f * `301` * `302` (default) +* `303` * `307` It is `302` (`ngx.HTTP_MOVED_TEMPORARILY`) by default. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 693275ca44..dce94dcccf 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4165,6 +4165,7 @@ The optional status parameter specifies the HTTP status code to be * 301 * 302 (default) +* 303 * 307 It is 302 (ngx.HTTP_MOVED_TEMPORARILY) by default. diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 249d7632c2..ae36505f9d 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -212,10 +212,12 @@ ngx_http_lua_ngx_redirect(lua_State *L) if (rc != NGX_HTTP_MOVED_TEMPORARILY && rc != NGX_HTTP_MOVED_PERMANENTLY + && rc != NGX_HTTP_SEE_OTHER && rc != NGX_HTTP_TEMPORARY_REDIRECT) { return luaL_error(L, "only ngx.HTTP_MOVED_TEMPORARILY, " - "ngx.HTTP_MOVED_PERMANENTLY, and " + "ngx.HTTP_MOVED_PERMANENTLY, " + "ngx.HTTP_SEE_OTHER, and " "ngx.HTTP_TEMPORARY_REDIRECT are allowed"); } diff --git a/t/022-redirect.t b/t/022-redirect.t index 57c7add987..fae39e3adf 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -84,7 +84,7 @@ GET /read --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log -only ngx.HTTP_MOVED_TEMPORARILY, ngx.HTTP_MOVED_PERMANENTLY, and ngx.HTTP_TEMPORARY_REDIRECT are allowed +only ngx.HTTP_MOVED_TEMPORARILY, ngx.HTTP_MOVED_PERMANENTLY, ngx.HTTP_SEE_OTHER, and ngx.HTTP_TEMPORARY_REDIRECT are allowed @@ -218,3 +218,54 @@ GET /read Location: http://agentzh.org/foo?a=b&c=d --- response_body_like: 307 Temporary Redirect --- error_code: 307 + + + +=== TEST 12: explicit 303 +--- config + location /read { + content_by_lua_block { + ngx.redirect("http://agentzh.org/foo", ngx.HTTP_SEE_OTHER); + ngx.say("hi") + } + } +--- request +GET /read +--- response_headers +Location: http://agentzh.org/foo +--- response_body_like: 303 See Other +--- error_code: 303 + + + +=== TEST 13: explicit 303 with args +--- config + location /read { + content_by_lua_block { + ngx.redirect("http://agentzh.org/foo?a=b&c=d", ngx.HTTP_SEE_OTHER); + ngx.say("hi") + } + } +--- request +GET /read +--- response_headers +Location: http://agentzh.org/foo?a=b&c=d +--- response_body_like: 303 See Other +--- error_code: 303 + + + +=== TEST 14: explicit 303 +--- config + location /read { + content_by_lua_block { + ngx.redirect("http://agentzh.org/foo?a=b&c=d", 303); + ngx.say("hi") + } + } +--- request +GET /read +--- response_headers +Location: http://agentzh.org/foo?a=b&c=d +--- response_body_like: 303 See Other +--- error_code: 303 From 3cf7062366af4ec79e4047140b96d6a184ed5070 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Sat, 17 Dec 2016 21:15:58 +0700 Subject: [PATCH 034/848] travis-ci: cosmetic "wget" improvement. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 28d0f087ec..e85f77d926 100644 --- a/.travis.yml +++ b/.travis.yml @@ -104,7 +104,7 @@ script: - cd .. - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - - if [ ! -f ../download-cache/openssl-1.0.2h-sess_set_get_cb_yield.patch ]; then wget -O ../download-cache/openssl-1.0.2h-sess_set_get_cb_yield.patch https://raw.githubusercontent.com/openresty/openresty/master/patches/openssl-1.0.2h-sess_set_get_cb_yield.patch; fi + - if [ ! -f ../download-cache/openssl-1.0.2h-sess_set_get_cb_yield.patch ]; then wget -P ../download-cache https://raw.githubusercontent.com/openresty/openresty/master/patches/openssl-1.0.2h-sess_set_get_cb_yield.patch; fi - patch -p1 < ../download-cache/openssl-1.0.2h-sess_set_get_cb_yield.patch - ./config shared --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) From 5028d63536ac795dc56a996416f79279f2f13d5d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 18 Dec 2016 14:05:18 -0800 Subject: [PATCH 035/848] doc: updated copyright notice. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 162343ff0f..f7044606f5 100644 --- a/README.markdown +++ b/README.markdown @@ -956,7 +956,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2016, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2016, by Yichun "agentzh" Zhang (章亦春) , CloudFlare Inc. +Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index dce94dcccf..f9b98a4ee5 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -781,7 +781,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2016, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2016, by Yichun "agentzh" Zhang (章亦春) , CloudFlare Inc. +Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved. From da11870db65eb827ae00c0b24255ba749aefd060 Mon Sep 17 00:00:00 2001 From: qleex001 Date: Mon, 14 Nov 2016 09:10:07 +0000 Subject: [PATCH 036/848] bugfix: balancer_by_lua*: the number of retres might exceed the limit of proxy_next_upstream_tries or alike. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 1 + src/ngx_http_lua_balancer.c | 7 +++-- t/138-balancer.t | 57 +++++++++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index e85f77d926..07d92b5747 100644 --- a/.travis.yml +++ b/.travis.yml @@ -79,6 +79,7 @@ install: - git clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module + - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git before_script: diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 0adf787350..03d9eac082 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -640,7 +640,7 @@ ngx_http_lua_ffi_balancer_set_more_tries(ngx_http_request_t *r, int count, char **err) { #if (nginx_version >= 1007005) - ngx_uint_t max_tries; + ngx_uint_t max_tries, total; #endif ngx_http_lua_ctx_t *ctx; ngx_http_upstream_t *u; @@ -681,9 +681,10 @@ ngx_http_lua_ffi_balancer_set_more_tries(ngx_http_request_t *r, #if (nginx_version >= 1007005) max_tries = r->upstream->conf->next_upstream_tries; + total = bp->total_tries + r->upstream->peer.tries - 1; - if (bp->total_tries + count > max_tries) { - count = max_tries - bp->total_tries; + if (max_tries && total + count > max_tries) { + count = max_tries - total; *err = "reduced tries due to limit"; } else { diff --git a/t/138-balancer.t b/t/138-balancer.t index 8d45c24563..b300c2eb27 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -431,3 +431,60 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ ] --- no_error_log [warn] + + + +=== TEST 15: test if execeed proxy_next_upstream_limit +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + proxy_next_upstream_tries 5; + upstream backend { + server 0.0.0.1; + balancer_by_lua_block { + local b = require "ngx.balancer" + + if not ngx.ctx.tries then + ngx.ctx.tries = 0 + end + + if ngx.ctx.tries >= 6 then + ngx.log(ngx.ERR, "retry count exceed limit") + ngx.exit(500) + end + + ngx.ctx.tries = ngx.ctx.tries + 1 + print("retry counter: ", ngx.ctx.tries) + + local ok, err = b.set_more_tries(2) + if not ok then + return error("failed to set more tries: ", err) + elseif err then + ngx.log(ngx.WARN, "set more tries: ", err) + end + + assert(b.set_current_peer("127.0.0.1", 81)) + } + } +--- config + location = /t { + proxy_pass http://backend/back; + } + + location = /back { + return 404; + } +--- request + GET /t +--- response_body_like: 502 Bad Gateway +--- error_code: 502 +--- grep_error_log eval: qr/\bretry counter: \w+/ +--- grep_error_log_out +retry counter: 1 +retry counter: 2 +retry counter: 3 +retry counter: 4 +retry counter: 5 + +--- error_log +set more tries: reduced tries due to limit From b3c872e124a44d1d76524430fac7e7b03f455e8f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 26 Dec 2016 21:45:06 -0800 Subject: [PATCH 037/848] added a passing test for #853. thanks jinhua luo for the original test case in openresty/lua-resty-core#58. --- t/138-balancer.t | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/t/138-balancer.t b/t/138-balancer.t index b300c2eb27..fc2617350a 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -488,3 +488,41 @@ retry counter: 5 --- error_log set more tries: reduced tries due to limit + + + +=== TEST 16: set_more_tries bugfix +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + proxy_next_upstream_tries 0; + upstream backend { + server 0.0.0.1; + balancer_by_lua_block { + local balancer = require "ngx.balancer" + local ctx = ngx.ctx + if not ctx.has_run then + ctx.has_run = true + local _, err = balancer.set_more_tries(3) + if err then + ngx.log(ngx.ERR, "failed to set more tries: ", err) + end + end + balancer.set_current_peer("127.0.0.1", 81) + } + } +--- config + location = /t { + proxy_pass http://backend; + } +--- request + GET /t +--- error_code: 502 +--- grep_error_log eval: qr/http next upstream, \d+/ +--- grep_error_log_out +http next upstream, 2 +http next upstream, 2 +http next upstream, 2 +http next upstream, 2 +--- no_error_log +failed to set more tries: reduced tries due to limit +[alert] From 5fdbb56ad53b6be94f80857e6e625e0969ea9e31 Mon Sep 17 00:00:00 2001 From: WenMing Date: Wed, 21 Dec 2016 09:15:45 +0800 Subject: [PATCH 038/848] bugfix: setting response headers would change the Content-Type response header. thanks leafo for the report in openresty/openresty#92. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_headers.c | 36 ++++++++++++++++++++ t/016-resp-header.t | 70 +++++++++++++++++++++++++++++++++++++- 2 files changed, 105 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 0af56f66ad..a7cfc3c74e 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -444,6 +444,8 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) ngx_list_part_t *part; ngx_table_elt_t *header; ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + ngx_int_t rc; u_char *lowcase_key = NULL; size_t lowcase_key_sz = 0; ngx_uint_t i; @@ -475,6 +477,22 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) return luaL_error(L, "no request object found"); } + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return luaL_error(L, "no ctx found"); + } + + if (!ctx->headers_set) { + rc = ngx_http_lua_set_content_type(r); + if (rc != NGX_OK) { + return luaL_error(L, + "failed to set default content type: %d", + (int) rc); + } + + ctx->headers_set = 1; + } + ngx_http_lua_check_fake_request(L, r); part = &r->headers_out.headers.part; @@ -603,12 +621,19 @@ ngx_http_lua_ngx_header_get(lua_State *L) ngx_uint_t i; size_t len; ngx_http_lua_loc_conf_t *llcf; + ngx_http_lua_ctx_t *ctx; + ngx_int_t rc; r = ngx_http_lua_get_req(L); if (r == NULL) { return luaL_error(L, "no request object found"); } + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return luaL_error(L, "no ctx found"); + } + ngx_http_lua_check_fake_request(L, r); /* we skip the first argument that is the table */ @@ -640,6 +665,17 @@ ngx_http_lua_ngx_header_get(lua_State *L) key.len = len; + if (!ctx->headers_set) { + rc = ngx_http_lua_set_content_type(r); + if (rc != NGX_OK) { + return luaL_error(L, + "failed to set default content type: %d", + (int) rc); + } + + ctx->headers_set = 1; + } + return ngx_http_lua_get_output_header(L, r, &key); } diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 179b411b8c..1fae2922a5 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 38); +plan tests => repeat_each() * (blocks() * 3 + 41); #no_diff(); no_long_string(); @@ -1441,3 +1441,71 @@ Content-Type: ; blah test --- no_error_log [error] + + + +=== TEST 69: return the matched content-type instead of default_type +--- http_config +types { + image/png png; +} +--- config +location /set/ { + default_type text/html; + content_by_lua_block { + ngx.say(ngx.header["content-type"]) + } +} +--- request +GET /set/hello.png +--- response_headers +Content-Type: image/png +--- response_body +image/png +--- no_error_log +[error] + + + +=== TEST 70: always return the matched content-type +--- config + location /set/ { + default_type "image/png"; + content_by_lua_block { + ngx.say(ngx.header["content-type"]) + ngx.say(ngx.header["content-type"]) + } + } +--- request +GET /set/hello.png +--- response_headers +Content-Type: image/png +--- response_body +image/png +image/png +--- no_error_log +[error] + + + +=== TEST 71: return the matched content-type after ngx.resp.get_headers() +--- http_config +types { + image/png png; +} +--- config + location /set/ { + default_type text/html; + content_by_lua_block { + local h = ngx.resp.get_headers() + ngx.say(h["content-type"]) + } + } +--- request +GET /set/hello.png +--- response_headers +Content-Type: image/png +--- response_body +image/png +--- no_error_log +[error] From 39783100f5b7e821dd929a9426c459e72dcf9361 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Sun, 4 Sep 2016 01:09:00 +0800 Subject: [PATCH 039/848] feature: tcpsock:connect(): allows the options_table argument being nil. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 6 ++++ t/058-tcp-socket.t | 52 ++++++++++++++++++++++++++++++++++- 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 6db6e2da92..c3a266da03 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -501,6 +501,12 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) n--; } + /* the fourth argument is not a table */ + if (n == 4) { + lua_pop(L, 1); + n--; + } + if (n == 3) { port = luaL_checkinteger(L, 3); diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 4734926a16..1ee113b9ce 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 187; +plan tests => repeat_each() * 190; our $HtmlDir = html_dir; @@ -3640,3 +3640,53 @@ failed to receive a line: closed [] close: 1 nil --- error_log lua http cleanup reuse + + + +=== TEST 60: options_table is nil +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port, nil) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "flush_all\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local line, err, part = sock:receive() + if line then + ngx.say("received: ", line) + + else + ngx.say("failed to receive a line: ", err, " [", part, "]") + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } +--- request +GET /t +--- response_body +connected: 1 +request sent: 11 +received: OK +close: 1 nil +--- no_error_log +[error] From 1ada6d0e59d28f958227c6eedc6ae1a4867830e1 Mon Sep 17 00:00:00 2001 From: Andreas Lubbe Date: Tue, 10 Jan 2017 10:54:40 +0100 Subject: [PATCH 040/848] tests: require PCRE 8.39 instead of 8.33 to run our test suite. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 2 +- t/034-match.t | 4 ++-- t/035-gmatch.t | 4 ++-- t/036-sub.t | 4 ++-- t/037-gsub.t | 4 ++-- t/120-re-find.t | 4 ++-- 6 files changed, 11 insertions(+), 11 deletions(-) diff --git a/.travis.yml b/.travis.yml index 07d92b5747..b1cbddc498 100644 --- a/.travis.yml +++ b/.travis.yml @@ -34,7 +34,7 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_VER=8.33 + - PCRE_VER=8.39 - PCRE_PREFIX=/opt/pcre - PCRE_LIB=$PCRE_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include diff --git a/t/034-match.t b/t/034-match.t index 35149f1b7b..ebe5762be4 100644 --- a/t/034-match.t +++ b/t/034-match.t @@ -1017,7 +1017,7 @@ exec opts: 0 === TEST 45: just hit match limit --- http_config - lua_regex_match_limit 5600; + lua_regex_match_limit 5000; --- config location /re { content_by_lua_file html/a.lua; @@ -1057,7 +1057,7 @@ error: pcre_exec() failed: -8 === TEST 46: just not hit match limit --- http_config - lua_regex_match_limit 5700; + lua_regex_match_limit 5100; --- config location /re { content_by_lua_file html/a.lua; diff --git a/t/035-gmatch.t b/t/035-gmatch.t index 69b9512b52..cbc673385e 100644 --- a/t/035-gmatch.t +++ b/t/035-gmatch.t @@ -814,7 +814,7 @@ exec opts: 0 === TEST 30: just hit match limit --- http_config - lua_regex_match_limit 5600; + lua_regex_match_limit 5000; --- config location /re { content_by_lua_file html/a.lua; @@ -860,7 +860,7 @@ error: pcre_exec() failed: -8 === TEST 31: just not hit match limit --- http_config - lua_regex_match_limit 5700; + lua_regex_match_limit 5100; --- config location /re { content_by_lua_file html/a.lua; diff --git a/t/036-sub.t b/t/036-sub.t index f08c50f4bc..2b4b07510a 100644 --- a/t/036-sub.t +++ b/t/036-sub.t @@ -580,7 +580,7 @@ s: a好 === TEST 28: just hit match limit --- http_config - lua_regex_match_limit 5600; + lua_regex_match_limit 5000; --- config location /re { content_by_lua_file html/a.lua; @@ -617,7 +617,7 @@ error: pcre_exec() failed: -8 === TEST 29: just not hit match limit --- http_config - lua_regex_match_limit 5700; + lua_regex_match_limit 5100; --- config location /re { content_by_lua_file html/a.lua; diff --git a/t/037-gsub.t b/t/037-gsub.t index 2a1e00f37a..4c5810d527 100644 --- a/t/037-gsub.t +++ b/t/037-gsub.t @@ -501,7 +501,7 @@ s: aa === TEST 23: just hit match limit --- http_config - lua_regex_match_limit 5600; + lua_regex_match_limit 5000; --- config location /re { content_by_lua_file html/a.lua; @@ -538,7 +538,7 @@ error: pcre_exec() failed: -8 === TEST 24: just not hit match limit --- http_config - lua_regex_match_limit 5700; + lua_regex_match_limit 5100; --- config location /re { content_by_lua_file html/a.lua; diff --git a/t/120-re-find.t b/t/120-re-find.t index 92e33bdab0..73e6134fd7 100644 --- a/t/120-re-find.t +++ b/t/120-re-find.t @@ -612,7 +612,7 @@ matched: 你 === TEST 22: just hit match limit --- http_config - lua_regex_match_limit 5600; + lua_regex_match_limit 5000; --- config location /re { content_by_lua_file html/a.lua; @@ -654,7 +654,7 @@ error: pcre_exec() failed: -8 === TEST 23: just not hit match limit --- http_config - lua_regex_match_limit 5700; + lua_regex_match_limit 5100; --- config location /re { content_by_lua_file html/a.lua; From 5c54198a369961cca351c98ad447e802124521e0 Mon Sep 17 00:00:00 2001 From: detailyang Date: Sat, 14 Jan 2017 22:47:24 +0800 Subject: [PATCH 041/848] bugfix: tcp cosockets: sslhandshake(): typo in the error message. also fixed a code comment for the last argument recently added. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 4 ++-- t/129-ssl-socket.t | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index c3a266da03..de8762f8ce 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1214,11 +1214,11 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) ngx_http_lua_socket_tcp_upstream_t *u; - /* Lua function arguments: self [,session] [,host] [,verify] */ + /* Lua function arguments: self [,session] [,host] [,verify] [,send_status_req] */ n = lua_gettop(L); if (n < 1 || n > 5) { - return luaL_error(L, "ngx.socket connect: expecting 1 ~ 5 " + return luaL_error(L, "ngx.socket sslhandshake: expecting 1 ~ 5 " "arguments (including the object), but seen %d", n); } diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index cc14594cb3..726b4423c9 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -2614,7 +2614,7 @@ SSL reused session GET /t --- ignore_response --- error_log eval -qr/\[error\] .* ngx.socket connect: expecting 1 ~ 5 arguments \(including the object\), but seen 0/ +qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including the object\), but seen 0/ --- no_error_log [alert] --- timeout: 5 From 0c3706298899370049d866d133aefc61ca746a7f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 18 Jan 2017 11:45:54 -0800 Subject: [PATCH 042/848] fixed a coding style issue in the previous commit. --- src/ngx_http_lua_socket_tcp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index de8762f8ce..06a0f6fcfd 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1214,7 +1214,8 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) ngx_http_lua_socket_tcp_upstream_t *u; - /* Lua function arguments: self [,session] [,host] [,verify] [,send_status_req] */ + /* Lua function arguments: self [,session] [,host] [,verify] + [,send_status_req] */ n = lua_gettop(L); if (n < 1 || n > 5) { From 1028333e48c3397113b4dac60565e45a1325e083 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 18 Jan 2017 11:56:19 -0800 Subject: [PATCH 043/848] travis-ci: added checks for use of tabs and source lines exceeding 80 cols. --- .travis.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.travis.yml b/.travis.yml index b1cbddc498..72a61a572c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -57,6 +57,10 @@ services: - redis-server - mysql +before_install: + - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' + - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' + install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache http://ftp.cs.stanford.edu/pub/exim/pcre/pcre-$PCRE_VER.tar.gz; fi From 5ae4aba8385c60443df328698f2126c73739f37a Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 26 Jan 2017 11:40:25 -0800 Subject: [PATCH 044/848] travis-ci: upgraded openssl to 1.0.2k. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 72a61a572c..c935543265 100644 --- a/.travis.yml +++ b/.travis.yml @@ -41,7 +41,7 @@ env: - OPENSSL_PREFIX=/opt/ssl - OPENSSL_LIB=$OPENSSL_PREFIX/lib - OPENSSL_INC=$OPENSSL_PREFIX/include - - OPENSSL_VER=1.0.2j + - OPENSSL_VER=1.0.2k - LIBDRIZZLE_PREFIX=/opt/drizzle - LIBDRIZZLE_INC=$LIBDRIZZLE_PREFIX/include/libdrizzle-1.0 - LIBDRIZZLE_LIB=$LIBDRIZZLE_PREFIX/lib From 827a5d6f83de4aed10fd1182e10656945d5bcf2c Mon Sep 17 00:00:00 2001 From: Tom Thorogood Date: Fri, 27 Jan 2017 12:59:35 +1030 Subject: [PATCH 045/848] feature: fixed build compatibility with BoringSSL. There are two issues that prevent lua-nginx-module from building when compiled against BoringSSL. They are: |SSL_set_tlsext_host_name| changing from a macro that internally casts name to a char* to a function that expects a char*, and |sk_X509_num| returning a size_t and no longer an int. The first issue is fixed by changing the caller of |SSL_set_tlsext_host_name|. This is identical to the fix applied in nginx/nginx@9b8b33b. The second issue is fixed by changing variable `i` to be a size_t rather than an int when built against BoringSSL, i.e. when OPENSSL_IS_BORINGSSL is defined. This change only addresses build failures and does not attempt to address missing features or correctness. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 3 ++- src/ngx_http_lua_ssl_certby.c | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 06a0f6fcfd..49811168d1 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1334,7 +1334,8 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME - if (SSL_set_tlsext_host_name(c->ssl->connection, name.data) + if (SSL_set_tlsext_host_name(c->ssl->connection, + (char *) name.data) == 0) { lua_pushnil(L); diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index aca473557c..84e309316a 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1129,7 +1129,11 @@ ngx_http_lua_ffi_set_cert(ngx_http_request_t *r, # else +#ifdef OPENSSL_IS_BORINGSSL + size_t i; +#else int i; +#endif X509 *x509 = NULL; ngx_ssl_conn_t *ssl_conn; STACK_OF(X509) *chain = cdata; From e958cb2caf9a3e17e511a5b3ce8ceab0783c41f4 Mon Sep 17 00:00:00 2001 From: detailyang Date: Thu, 2 Feb 2017 20:04:36 +0800 Subject: [PATCH 046/848] tests: fixed a bug in testing the nullness of cdata pointers. Signed-off-by: Yichun Zhang (agentzh) --- t/140-ssl-c-api.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 44ad93d1c0..854ff30aad 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -561,7 +561,7 @@ failed to parse PEM priv key: PEM_read_bio_PrivateKey() failed f:close() local pkey = ffi.C.ngx_http_lua_ffi_parse_pem_priv_key(pkey_data, #pkey_data, errmsg) - if not pkey then + if pkey == nil then ngx.log(ngx.ERR, "failed to parse PEM priv key: ", ffi.string(errmsg[0])) return @@ -711,7 +711,7 @@ lua ssl server name: "test.com" f:close() local pkey = ffi.C.ngx_http_lua_ffi_parse_pem_priv_key(pkey_data, #pkey_data, errmsg) - if not pkey then + if pkey == nil then ngx.log(ngx.ERR, "failed to parse PEM priv key: ", ffi.string(errmsg[0])) return From 4fb510f5fb1cb66042b6037aee701dfc053c0d76 Mon Sep 17 00:00:00 2001 From: tan <312841925@qq.com> Date: Mon, 6 Feb 2017 23:09:28 +0800 Subject: [PATCH 047/848] fixed a function parameter name. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 267952be21..079a4dc3b8 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -138,7 +138,7 @@ typedef struct ngx_http_lua_sema_mm_s ngx_http_lua_sema_mm_t; typedef ngx_int_t (*ngx_http_lua_main_conf_handler_pt)(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, lua_State *L); typedef ngx_int_t (*ngx_http_lua_srv_conf_handler_pt)(ngx_http_request_t *r, - ngx_http_lua_srv_conf_t *lmcf, lua_State *L); + ngx_http_lua_srv_conf_t *lscf, lua_State *L); typedef struct { From e73cd4b90013b23cf6af60432b1d367b2985708e Mon Sep 17 00:00:00 2001 From: detailyang Date: Fri, 10 Feb 2017 16:43:59 +0800 Subject: [PATCH 048/848] tweak: guarded SSL related function declarations with the macro NGX_HTTP_SSL. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_ssl.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl.h b/src/ngx_http_lua_ssl.h index 7a245ffda7..acb8c4b16c 100644 --- a/src/ngx_http_lua_ssl.h +++ b/src/ngx_http_lua_ssl.h @@ -12,6 +12,8 @@ #if (NGX_HTTP_SSL) + + typedef struct { ngx_connection_t *connection; /* original true connection */ ngx_http_request_t *request; /* fake request */ @@ -31,7 +33,6 @@ typedef struct { unsigned entered_cert_handler:1; unsigned entered_sess_fetch_handler:1; } ngx_http_lua_ssl_ctx_t; -#endif ngx_int_t ngx_http_lua_ssl_init(ngx_log_t *log); @@ -40,4 +41,7 @@ ngx_int_t ngx_http_lua_ssl_init(ngx_log_t *log); extern int ngx_http_lua_ssl_ctx_index; +#endif + + #endif /* _NGX_HTTP_LUA_SSL_H_INCLUDED_ */ From a14b9cd374904e02dd46b30e925d106285188cc4 Mon Sep 17 00:00:00 2001 From: Dayo Akanji Date: Sat, 11 Feb 2017 21:11:15 +0300 Subject: [PATCH 049/848] doc: various wording tweaks and more code examples. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * instances of “Warning” changed to more appropriate terms such as “Note” or “Caution” * removed informal terms such as “kinda” Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 107 +++++++++++++++++------------ doc/HttpLuaModule.wiki | 152 +++++++++++++++++++++++------------------ 2 files changed, 151 insertions(+), 108 deletions(-) diff --git a/README.markdown b/README.markdown index f7044606f5..c13f2dacc2 100644 --- a/README.markdown +++ b/README.markdown @@ -150,7 +150,7 @@ Synopsis } # use nginx var in code path - # WARNING: contents in nginx var must be carefully filtered, + # CAUTION: contents in nginx var must be carefully filtered, # otherwise there'll be great security risk! location ~ ^/app/([-_a-zA-Z0-9/]+) { set $path $1; @@ -312,8 +312,9 @@ Starting from NGINX 1.9.11, you can also compile this module as a dynamic module directive, for example, ```nginx -load_module /path/to/modules/ndk_http_module.so; # assuming NDK is built as a dynamic module too -load_module /path/to/modules/ngx_http_lua_module.so; + + load_module /path/to/modules/ndk_http_module.so; # assuming NDK is built as a dynamic module too + load_module /path/to/modules/ngx_http_lua_module.so; ``` [Back to TOC](#table-of-contents) @@ -656,15 +657,29 @@ instead of the old deprecated form: Here is the reason: by design, the global environment has exactly the same lifetime as the Nginx request handler associated with it. Each request handler has its own set of Lua global variables and that is the idea of request isolation. The Lua module is actually loaded by the first Nginx request handler and is cached by the `require()` built-in in the `package.loaded` table for later reference, and the `module()` builtin used by some Lua modules has the side effect of setting a global variable to the loaded module table. But this global variable will be cleared at the end of the request handler, and every subsequent request handler all has its own (clean) global environment. So one will get Lua exception for accessing the `nil` value. -Generally, use of Lua global variables is a really really bad idea in the context of ngx_lua because +The use of Lua global variables is a generally inadvisable in the ngx_lua context as: + +1. the misuse of Lua globals has detrimental side effects on concurrent requests when such variables should instead be local in scope, +1. Lua global variables require Lua table look-ups in the global environment which is computationally expensive, and +1. some Lua global variable references may include typing errors which make such difficult to debug. + +It is therefore *highly* recommended to always declare such within an appropriate local scope instead. -1. misuse of Lua globals has very bad side effects for concurrent requests when these variables are actually supposed to be local only, -1. Lua global variables require Lua table look-up in the global environment (which is just a Lua table), which is kinda expensive, and -1. some Lua global variable references are just typos, which are hard to debug. +```lua + + -- Avoid + foo = 123 + -- Recomended + local foo = 123 -It's *highly* recommended to always declare them via "local" in the scope that is reasonable. + -- Avoid + function foo() return 123 end + -- Recomended + local function foo() return 123 end +``` -To find out all the uses of Lua global variables in your Lua code, you can run the [lua-releng tool](https://github.com/openresty/nginx-devel-utils/blob/master/lua-releng) across all your .lua source files: + +To find all instances of Lua global variables in your Lua code, run the [lua-releng tool](https://github.com/openresty/nginx-devel-utils/blob/master/lua-releng) across all `.lua` source files: $ lua-releng Checking use of Lua global variables in file lib/foo/bar.lua ... @@ -709,28 +724,27 @@ will not work as expected. Cosockets Not Available Everywhere ---------------------------------- -Due the internal limitations in the nginx core, the cosocket API are disabled in the following contexts: [set_by_lua*](#set_by_lua), [log_by_lua*](#log_by_lua), [header_filter_by_lua*](#header_filter_by_lua), and [body_filter_by_lua](#body_filter_by_lua). +Due to internal limitations in the nginx core, the cosocket API is disabled in the following contexts: [set_by_lua*](#set_by_lua), [log_by_lua*](#log_by_lua), [header_filter_by_lua*](#header_filter_by_lua), and [body_filter_by_lua](#body_filter_by_lua). The cosockets are currently also disabled in the [init_by_lua*](#init_by_lua) and [init_worker_by_lua*](#init_worker_by_lua) directive contexts but we may add support for these contexts in the future because there is no limitation in the nginx core (or the limitation might be worked around). -There exists a work-around, however, when the original context does *not* need to wait for the cosocket results. That is, creating a 0-delay timer via the [ngx.timer.at](#ngxtimerat) API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. +There exists a work-around, however, when the original context does *not* need to wait for the cosocket results. That is, creating a zero-delay timer via the [ngx.timer.at](#ngxtimerat) API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. [Back to TOC](#table-of-contents) Special Escaping Sequences -------------------------- -**WARNING** We no longer suffer from this pitfall since the introduction of the -`*_by_lua_block {}` configuration directives. +**NOTE** Following the `v0.9.17` release, this pitfall can be avoided by using the `*_by_lua_block {}` configuration directives. -PCRE sequences such as `\d`, `\s`, or `\w`, require special attention because in string literals, the backslash character, `\`, is stripped out by both the Lua language parser and by the Nginx config file parser before processing. So the following snippet will not work as expected: +PCRE sequences such as `\d`, `\s`, or `\w`, require special attention because in string literals, the backslash character, `\`, is stripped out by both the Lua language parser and by the nginx config file parser before processing if not within a `*_by_lua_block {}` directive. So the following snippet will not work as expected: ```nginx # nginx.conf ? location /test { ? content_by_lua ' - ? local regex = "\d+" -- THIS IS WRONG!! + ? local regex = "\d+" -- THIS IS WRONG OUTSIDE OF A *_by_lua_block DIRECTIVE ? local m = ngx.re.match("hello, 1234", regex) ? if m then ngx.say(m[0]) else ngx.say("not matched!") end ? '; @@ -811,6 +825,22 @@ Within external script files, PCRE sequences presented as long-bracketed Lua str -- evaluates to "1234" ``` +As noted earlier, PCRE sequences presented within `*_by_lua_block {}` directives (available following the `v0.9.17` release) do not require modification. + +```nginx + + # nginx.conf + location /test { + content_by_lua_block { + local regex = "\d+" + local m = ngx.re.match("hello, 1234", regex) + if m then ngx.say(m[0]) else ngx.say("not matched!") end + } + } + # evaluates to "1234" +``` + + [Back to TOC](#table-of-contents) Mixing with SSI Not Supported @@ -885,7 +915,7 @@ servers in Lua. For example, Changes ======= -The changes of every release of this module can be obtained from the OpenResty bundle's change logs: +The changes made in every release of this module are listed in the change logs of the OpenResty bundle: @@ -1081,7 +1111,7 @@ lua_use_default_type **context:** *http, server, location, location if* -Specifies whether to use the MIME type specified by the [default_type](http://nginx.org/en/docs/http/ngx_http_core_module.html#default_type) directive for the default value of the `Content-Type` response header. If you do not want a default `Content-Type` response header for your Lua request handlers, then turn this directive off. +Specifies whether to use the MIME type specified by the [default_type](http://nginx.org/en/docs/http/ngx_http_core_module.html#default_type) directive for the default value of the `Content-Type` response header. Deactivate this directive if a default `Content-Type` response header for Lua request handlers is not desired. This directive is turned on by default. @@ -1153,7 +1183,7 @@ Apache `mod_lua` module (yet). Disabling the Lua code cache is strongly discouraged for production use and should only be used during -development as it has a significant negative impact on overall performance. For example, the performance a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. +development as it has a significant negative impact on overall performance. For example, the performance of a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. [Back to TOC](#directives) @@ -1242,8 +1272,7 @@ init_by_lua **phase:** *loading-config* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [init_by_lua_block](#init_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [init_by_lua_block](#init_by_lua_block) directive instead. Runs the Lua code specified by the argument `` on the global Lua VM level when the Nginx master process (if any) is loading the Nginx config file. @@ -1359,7 +1388,7 @@ init_worker_by_lua **phase:** *starting-worker* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; use the new [init_worker_by_lua_block](#init_worker_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [init_worker_by_lua_block](#init_worker_by_lua_block) directive instead. Runs the specified Lua code upon every Nginx worker process's startup when the master process is enabled. When the master process is disabled, this hook will just run after [init_by_lua*](#init_by_lua). @@ -1448,7 +1477,7 @@ set_by_lua **phase:** *rewrite* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; use the new [set_by_lua_block](#set_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [set_by_lua_block](#set_by_lua_block) directive instead. Executes code specified in `` with optional input arguments `$arg1 $arg2 ...`, and returns string output to `$res`. The code in `` can make [API calls](#nginx-api-for-lua) and can retrieve input arguments from the `ngx.arg` table (index starts from `1` and increases sequentially). @@ -1562,8 +1591,7 @@ content_by_lua **phase:** *content* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [content_by_lua_block](#content_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [content_by_lua_block](#content_by_lua_block) directive instead. Acts as a "content handler" and executes Lua code string specified in `` for every request. The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). @@ -1623,7 +1651,7 @@ Nginx variables are supported in the file path for dynamic dispatch, for example ```nginx - # WARNING: contents in nginx var must be carefully filtered, + # CAUTION: contents in nginx var must be carefully filtered, # otherwise there'll be great security risk! location ~ ^/app/([-_a-zA-Z0-9/]+) { set $path $1; @@ -1644,8 +1672,7 @@ rewrite_by_lua **phase:** *rewrite tail* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [rewrite_by_lua_block](#rewrite_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [rewrite_by_lua_block](#rewrite_by_lua_block) directive instead. Acts as a rewrite phase handler and executes Lua code string specified in `` for every request. The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). @@ -1821,8 +1848,7 @@ access_by_lua **phase:** *access tail* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [access_by_lua_block](#access_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [access_by_lua_block](#access_by_lua_block) directive instead. Acts as an access phase handler and executes Lua code string specified in `` for every request. The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). @@ -1951,8 +1977,7 @@ header_filter_by_lua **phase:** *output-header-filter* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [header_filter_by_lua_block](#header_filter_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [header_filter_by_lua_block](#header_filter_by_lua_block) directive instead. Uses Lua code specified in `` to define an output header filter. @@ -2030,8 +2055,7 @@ body_filter_by_lua **phase:** *output-body-filter* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [body_filter_by_lua_block](#body_filter_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [body_filter_by_lua_block](#body_filter_by_lua_block) directive instead. Uses Lua code specified in `` to define an output body filter. @@ -2167,8 +2191,7 @@ log_by_lua **phase:** *log* -**WARNING** Since the `v0.9.17` release, use of this directive is *discouraged*; -use the new [log_by_lua_block](#log_by_lua_block) directive instead. +**NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [log_by_lua_block](#log_by_lua_block) directive instead. Runs the Lua source code inlined as the `` at the `log` request processing phase. This does not replace the current access logs, but runs before. @@ -2952,7 +2975,7 @@ lua_check_client_abort This directive controls whether to check for premature client connection abortion. -When this directive is turned on, the ngx_lua module will monitor the premature connection close event on the downstream connections. And when there is such an event, it will call the user Lua function callback (registered by [ngx.on_abort](#ngxon_abort)) or just stop and clean up all the Lua "light threads" running in the current request's request handler when there is no user callback function registered. +When this directive is on, the ngx_lua module will monitor the premature connection close event on the downstream connections and when there is such an event, it will call the user Lua function callback (registered by [ngx.on_abort](#ngxon_abort)) or just stop and clean up all the Lua "light threads" running in the current request's request handler when there is no user callback function registered. According to the current implementation, however, if the client closes the connection before the Lua code finishes reading the request body data via [ngx.req.socket](#ngxreqsocket), then ngx_lua will neither stop all the running "light threads" nor call the user callback (if [ngx.on_abort](#ngxon_abort) has been called). Instead, the reading operation on [ngx.req.socket](#ngxreqsocket) will just return the error message "client aborted" as the second return value (the first return value is surely `nil`). @@ -3281,7 +3304,7 @@ Setting `ngx.var.Foo` to a `nil` value will unset the `$Foo` Nginx variable. ngx.var.args = nil ``` -**WARNING** When reading from an Nginx variable, Nginx will allocate memory in the per-request memory pool which is freed only at request termination. So when you need to read from an Nginx variable repeatedly in your Lua code, cache the Nginx variable value to your own Lua variable, for example, +**CAUTION** When reading from an Nginx variable, Nginx will allocate memory in the per-request memory pool which is freed only at request termination. So when you need to read from an Nginx variable repeatedly in your Lua code, cache the Nginx variable value to your own Lua variable, for example, ```lua @@ -3550,7 +3573,7 @@ Because of the metamethod magic, never "local" the `ngx.ctx` table outside your local _M = {} -- the following line is bad since ngx.ctx is a per-request - -- data while this `ctx` variable is on the Lua module level + -- data while this ctx variable is on the Lua module level -- and thus is per-nginx-worker. local ctx = ngx.ctx @@ -5253,7 +5276,7 @@ ngx.eof Explicitly specify the end of the response output stream. In the case of HTTP 1.1 chunked encoded output, it will just trigger the Nginx core to send out the "last chunk". -When you disable the HTTP 1.1 keep-alive feature for your downstream connections, you can rely on descent HTTP clients to close the connection actively for you when you call this method. This trick can be used do back-ground jobs without letting the HTTP clients to wait on the connection, as in the following example: +When you disable the HTTP 1.1 keep-alive feature for your downstream connections, you can rely on well written HTTP clients to close the connection actively for you when you call this method. This trick can be used do back-ground jobs without letting the HTTP clients to wait on the connection, as in the following example: ```nginx @@ -5261,7 +5284,7 @@ When you disable the HTTP 1.1 keep-alive feature for your downstream connections keepalive_timeout 0; content_by_lua_block { ngx.say("got the task!") - ngx.eof() -- a descent HTTP client will close the connection at this point + ngx.eof() -- well written HTTP clients will close the connection at this point -- access MySQL, PostgreSQL, Redis, Memcached, and etc here... } } @@ -6497,7 +6520,7 @@ Fetch a list of the keys from the dictionary, up to ``. By default, only the first 1024 keys (if any) are returned. When the `` argument is given the value `0`, then all the keys will be returned even there is more than 1024 keys in the dictionary. -**WARNING** Be careful when calling this method on dictionaries with a really huge number of keys. This method may lock the dictionary for quite a while and block all the nginx worker processes that are trying to access the dictionary. +**CAUTION** Avoid calling this method on dictionaries with a very large number of keys as it may lock the dictionary for significant amount of time and block Nginx worker processes trying to access the dictionary. This feature was first introduced in the `v0.7.3` release. @@ -7282,7 +7305,7 @@ Then it will generate the output 4 -"Light threads" are mostly useful for doing concurrent upstream requests in a single Nginx request handler, kinda like a generalized version of [ngx.location.capture_multi](#ngxlocationcapture_multi) that can work with all the [Nginx API for Lua](#nginx-api-for-lua). The following example demonstrates parallel requests to MySQL, Memcached, and upstream HTTP services in a single Lua handler, and outputting the results in the order that they actually return (very much like the Facebook BigPipe model): +"Light threads" are mostly useful for making concurrent upstream requests in a single Nginx request handler, much like a generalized version of [ngx.location.capture_multi](#ngxlocationcapture_multi) that can work with all the [Nginx API for Lua](#nginx-api-for-lua). The following example demonstrates parallel requests to MySQL, Memcached, and upstream HTTP services in a single Lua handler, and outputting the results in the order that they actually return (similar to Facebook's BigPipe model): ```lua diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f9b98a4ee5..71912d8796 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -96,7 +96,7 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t } # use nginx var in code path - # WARNING: contents in nginx var must be carefully filtered, + # CAUTION: contents in nginx var must be carefully filtered, # otherwise there'll be great security risk! location ~ ^/app/([-_a-zA-Z0-9/]+) { set $path $1; @@ -237,14 +237,14 @@ Build the source with this module: == Building as a dynamic module == -Starting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the `--add-dynamic-module=PATH` option instead of `--add-module=PATH` on the -`./configure` command line above. And then you can explicitly load the module in your `nginx.conf` via the [load_module](http://nginx.org/en/docs/ngx_core_module.html#load_module) +Starting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the --add-dynamic-module=PATH option instead of --add-module=PATH on the +./configure command line above. And then you can explicitly load the module in your nginx.conf via the [load_module](http://nginx.org/en/docs/ngx_core_module.html#load_module) directive, for example, -```nginx + load_module /path/to/modules/ndk_http_module.so; # assuming NDK is built as a dynamic module too load_module /path/to/modules/ngx_http_lua_module.so; -``` + == C Macro Configurations == @@ -520,15 +520,28 @@ instead of the old deprecated form: Here is the reason: by design, the global environment has exactly the same lifetime as the Nginx request handler associated with it. Each request handler has its own set of Lua global variables and that is the idea of request isolation. The Lua module is actually loaded by the first Nginx request handler and is cached by the require() built-in in the package.loaded table for later reference, and the module() builtin used by some Lua modules has the side effect of setting a global variable to the loaded module table. But this global variable will be cleared at the end of the request handler, and every subsequent request handler all has its own (clean) global environment. So one will get Lua exception for accessing the nil value. -Generally, use of Lua global variables is a really really bad idea in the context of ngx_lua because +The use of Lua global variables is a generally inadvisable in the ngx_lua context as: + +# the misuse of Lua globals has detrimental side effects on concurrent requests when such variables should instead be local in scope, +# Lua global variables require Lua table look-ups in the global environment which is computationally expensive, and +# some Lua global variable references may include typing errors which make such difficult to debug. + +It is therefore *highly* recommended to always declare such within an appropriate local scope instead. + + + -- Avoid + foo = 123 + -- Recomended + local foo = 123 -# misuse of Lua globals has very bad side effects for concurrent requests when these variables are actually supposed to be local only, -# Lua global variables require Lua table look-up in the global environment (which is just a Lua table), which is kinda expensive, and -# some Lua global variable references are just typos, which are hard to debug. + -- Avoid + function foo() return 123 end + -- Recomended + local function foo() return 123 end + -It's *highly* recommended to always declare them via "local" in the scope that is reasonable. -To find out all the uses of Lua global variables in your Lua code, you can run the [https://github.com/openresty/nginx-devel-utils/blob/master/lua-releng lua-releng tool] across all your .lua source files: +To find all instances of Lua global variables in your Lua code, run the [https://github.com/openresty/nginx-devel-utils/blob/master/lua-releng lua-releng tool] across all .lua source files: $ lua-releng Checking use of Lua global variables in file lib/foo/bar.lua ... @@ -565,24 +578,23 @@ will not work as expected. == Cosockets Not Available Everywhere == -Due the internal limitations in the nginx core, the cosocket API are disabled in the following contexts: [[#set_by_lua|set_by_lua*]], [[#log_by_lua|log_by_lua*]], [[#header_filter_by_lua|header_filter_by_lua*]], and [[#body_filter_by_lua|body_filter_by_lua]]. +Due to internal limitations in the nginx core, the cosocket API is disabled in the following contexts: [[#set_by_lua|set_by_lua*]], [[#log_by_lua|log_by_lua*]], [[#header_filter_by_lua|header_filter_by_lua*]], and [[#body_filter_by_lua|body_filter_by_lua]]. The cosockets are currently also disabled in the [[#init_by_lua|init_by_lua*]] and [[#init_worker_by_lua|init_worker_by_lua*]] directive contexts but we may add support for these contexts in the future because there is no limitation in the nginx core (or the limitation might be worked around). -There exists a work-around, however, when the original context does *not* need to wait for the cosocket results. That is, creating a 0-delay timer via the [[#ngx.timer.at|ngx.timer.at]] API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. +There exists a work-around, however, when the original context does *not* need to wait for the cosocket results. That is, creating a zero-delay timer via the [[#ngx.timer.at|ngx.timer.at]] API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. == Special Escaping Sequences == -'''WARNING''' We no longer suffer from this pitfall since the introduction of the -*_by_lua_block {} configuration directives. +'''NOTE''' Following the v0.9.17 release, this pitfall can be avoided by using the *_by_lua_block {} configuration directives. -PCRE sequences such as \d, \s, or \w, require special attention because in string literals, the backslash character, \, is stripped out by both the Lua language parser and by the Nginx config file parser before processing. So the following snippet will not work as expected: +PCRE sequences such as \d, \s, or \w, require special attention because in string literals, the backslash character, \, is stripped out by both the Lua language parser and by the nginx config file parser before processing if not within a *_by_lua_block {} directive. So the following snippet will not work as expected: # nginx.conf ? location /test { ? content_by_lua ' - ? local regex = "\d+" -- THIS IS WRONG!! + ? local regex = "\d+" -- THIS IS WRONG OUTSIDE OF A *_by_lua_block DIRECTIVE ? local m = ngx.re.match("hello, 1234", regex) ? if m then ngx.say(m[0]) else ngx.say("not matched!") end ? '; @@ -658,6 +670,21 @@ Within external script files, PCRE sequences presented as long-bracketed Lua str -- evaluates to "1234" +As noted earlier, PCRE sequences presented within *_by_lua_block {} directives (available following the v0.9.17 release) do not require modification. + + + # nginx.conf + location /test { + content_by_lua_block { + local regex = "\d+" + local m = ngx.re.match("hello, 1234", regex) + if m then ngx.say(m[0]) else ngx.say("not matched!") end + } + } + # evaluates to "1234" + + + == Mixing with SSI Not Supported == Mixing SSI with ngx_lua in the same Nginx request is not supported at all. Just use ngx_lua exclusively. Everything you can do with SSI can be done atop ngx_lua anyway and it can be more efficient when using ngx_lua. @@ -716,7 +743,7 @@ servers in Lua. For example, = Changes = -The changes of every release of this module can be obtained from the OpenResty bundle's change logs: +The changes made in every release of this module are listed in the change logs of the OpenResty bundle: http://openresty.org/#Changes @@ -771,7 +798,7 @@ To run specific test files: prove -I/path/to/test-nginx/lib t/002-content.t t/003-errors.t -To run a specific test block in a particular test file, add the line --- ONLY to the test block you want to run, and then use the `prove` utility to run that .t file. +To run a specific test block in a particular test file, add the line --- ONLY to the test block you want to run, and then use the prove utility to run that .t file. There are also various testing modes based on mockeagain, valgrind, and etc. Refer to the [http://search.cpan.org/perldoc?Test::Nginx Test::Nginx documentation] for more details for various advanced testing modes. See also the test reports for the Nginx test cluster running on Amazon EC2: http://qa.openresty.org. @@ -833,7 +860,7 @@ how the result will be used. Below is a diagram showing the order in which direc '''context:''' ''http, server, location, location if'' -Specifies whether to use the MIME type specified by the [http://nginx.org/en/docs/http/ngx_http_core_module.html#default_type default_type] directive for the default value of the Content-Type response header. If you do not want a default Content-Type response header for your Lua request handlers, then turn this directive off. +Specifies whether to use the MIME type specified by the [http://nginx.org/en/docs/http/ngx_http_core_module.html#default_type default_type] directive for the default value of the Content-Type response header. Deactivate this directive if a default Content-Type response header for Lua request handlers is not desired. This directive is turned on by default. @@ -898,7 +925,7 @@ Apache mod_lua module (yet). Disabling the Lua code cache is strongly discouraged for production use and should only be used during -development as it has a significant negative impact on overall performance. For example, the performance a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. +development as it has a significant negative impact on overall performance. For example, the performance of a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. == lua_regex_cache_max_entries == '''syntax:''' ''lua_regex_cache_max_entries '' @@ -917,7 +944,7 @@ The default number of entries allowed is 1024 and when this limit is reached, ne 2011/08/27 23:18:26 [warn] 31997#0: *1 lua exceeding regex cache max entries (1024), ... -If you are using the `ngx.re.*` implementation of [lua-resty-core](https://github.com/openresty/lua-resty-core) by loading the `resty.core.regex` module (or just the `resty.core` module), then an LRU cache is used for the regex cache being used here. +If you are using the ngx.re.* implementation of [lua-resty-core](https://github.com/openresty/lua-resty-core) by loading the resty.core.regex module (or just the resty.core module), then an LRU cache is used for the regex cache being used here. Do not activate the o option for regular expressions (and/or replace string arguments for [[#ngx.re.sub|ngx.re.sub]] and [[#ngx.re.gsub|ngx.re.gsub]]) that are generated ''on the fly'' and give rise to infinite variations to avoid hitting the specified limit. @@ -972,8 +999,7 @@ As from the v0.5.0rc29 release, the special notation $prefix< '''phase:''' ''loading-config'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#init_by_lua_block|init_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#init_by_lua_block|init_by_lua_block]] directive instead. Runs the Lua code specified by the argument on the global Lua VM level when the Nginx master process (if any) is loading the Nginx config file. @@ -1042,7 +1068,7 @@ This directive was first introduced in the v0.5.5 release. Similar to the [[#init_by_lua|init_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1077,7 +1103,7 @@ This directive was first introduced in the v0.5.5 release. '''phase:''' ''starting-worker'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; use the new [[#init_worker_by_lua_block|init_worker_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#init_worker_by_lua_block|init_worker_by_lua_block]] directive instead. Runs the specified Lua code upon every Nginx worker process's startup when the master process is enabled. When the master process is disabled, this hook will just run after [[#init_by_lua|init_by_lua*]]. @@ -1122,7 +1148,7 @@ This directive was first introduced in the v0.9.5 release. Similar to the [[#init_worker_by_lua|init_worker_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1155,7 +1181,7 @@ This directive was first introduced in the v0.9.5 release. '''phase:''' ''rewrite'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; use the new [[#set_by_lua_block|set_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#set_by_lua_block|set_by_lua_block]] directive instead. Executes code specified in with optional input arguments $arg1 $arg2 ..., and returns string output to $res. The code in can make [[#Nginx API for Lua|API calls]] and can retrieve input arguments from the ngx.arg table (index starts from 1 and increases sequentially). @@ -1214,7 +1240,7 @@ This directive requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_ki Similar to the [[#set_by_lua|set_by_lua]] directive except that # this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping), and # this directive does not support extra arguments after the Lua script as in [[#set_by_lua|set_by_lua]]. @@ -1257,8 +1283,7 @@ This directive requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_ki '''phase:''' ''content'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#content_by_lua_block|content_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#content_by_lua_block|content_by_lua_block]] directive instead. Acts as a "content handler" and executes Lua code string specified in for every request. The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). @@ -1275,7 +1300,7 @@ Do not use this directive and other content handler directives in the same locat Similar to the [[#content_by_lua|content_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1310,7 +1335,7 @@ switching [[#lua_code_cache|lua_code_cache]] off in nginx.con Nginx variables are supported in the file path for dynamic dispatch, for example: - # WARNING: contents in nginx var must be carefully filtered, + # CAUTION: contents in nginx var must be carefully filtered, # otherwise there'll be great security risk! location ~ ^/app/([-_a-zA-Z0-9/]+) { set $path $1; @@ -1328,8 +1353,7 @@ But be very careful about malicious user inputs and always carefully validate or '''phase:''' ''rewrite tail'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#rewrite_by_lua_block|rewrite_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#rewrite_by_lua_block|rewrite_by_lua_block]] directive instead. Acts as a rewrite phase handler and executes Lua code string specified in for every request. The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). @@ -1448,7 +1472,7 @@ The rewrite_by_lua code will always run at the end of the rew Similar to the [[#rewrite_by_lua|rewrite_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1489,8 +1513,7 @@ Nginx variables are supported in the file path for dynamic dispatch just as in [ '''phase:''' ''access tail'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#access_by_lua_block|access_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#access_by_lua_block|access_by_lua_block]] directive instead. Acts as an access phase handler and executes Lua code string specified in for every request. The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). @@ -1565,7 +1588,7 @@ of NGINX. Similar to the [[#access_by_lua|access_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1606,8 +1629,7 @@ Nginx variables are supported in the file path for dynamic dispatch just as in [ '''phase:''' ''output-header-filter'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#header_filter_by_lua_block|header_filter_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#header_filter_by_lua_block|header_filter_by_lua_block]] directive instead. Uses Lua code specified in to define an output header filter. @@ -1639,7 +1661,7 @@ This directive was first introduced in the v0.2.1rc20 release. Similar to the [[#header_filter_by_lua|header_filter_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1674,8 +1696,7 @@ This directive was first introduced in the v0.2.1rc20 release. '''phase:''' ''output-body-filter'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#body_filter_by_lua_block|body_filter_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#body_filter_by_lua_block|body_filter_by_lua_block]] directive instead. Uses Lua code specified in to define an output body filter. @@ -1762,7 +1783,7 @@ This directive was first introduced in the v0.5.0rc32 release. Similar to the [[#body_filter_by_lua|body_filter_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -1797,8 +1818,7 @@ This directive was first introduced in the v0.5.0rc32 release. '''phase:''' ''log'' -'''WARNING''' Since the v0.9.17 release, use of this directive is ''discouraged''; -use the new [[#log_by_lua_block|log_by_lua_block]] directive instead. +'''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#log_by_lua_block|log_by_lua_block]] directive instead. Runs the Lua source code inlined as the at the log request processing phase. This does not replace the current access logs, but runs before. @@ -1863,7 +1883,7 @@ This directive was first introduced in the v0.5.0rc31 release. Similar to the [[#log_by_lua|log_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping). For instance, @@ -2487,7 +2507,7 @@ This directive was first introduced in the v0.5.0rc32 release. This directive controls whether to check for premature client connection abortion. -When this directive is turned on, the ngx_lua module will monitor the premature connection close event on the downstream connections. And when there is such an event, it will call the user Lua function callback (registered by [[#ngx.on_abort|ngx.on_abort]]) or just stop and clean up all the Lua "light threads" running in the current request's request handler when there is no user callback function registered. +When this directive is on, the ngx_lua module will monitor the premature connection close event on the downstream connections and when there is such an event, it will call the user Lua function callback (registered by [[#ngx.on_abort|ngx.on_abort]]) or just stop and clean up all the Lua "light threads" running in the current request's request handler when there is no user callback function registered. According to the current implementation, however, if the client closes the connection before the Lua code finishes reading the request body data via [[#ngx.req.socket|ngx.req.socket]], then ngx_lua will neither stop all the running "light threads" nor call the user callback (if [[#ngx.on_abort|ngx.on_abort]] has been called). Instead, the reading operation on [[#ngx.req.socket|ngx.req.socket]] will just return the error message "client aborted" as the second return value (the first return value is surely nil). @@ -2647,7 +2667,7 @@ Setting ngx.var.Foo to a nil value will unset the -'''WARNING''' When reading from an Nginx variable, Nginx will allocate memory in the per-request memory pool which is freed only at request termination. So when you need to read from an Nginx variable repeatedly in your Lua code, cache the Nginx variable value to your own Lua variable, for example, +'''CAUTION''' When reading from an Nginx variable, Nginx will allocate memory in the per-request memory pool which is freed only at request termination. So when you need to read from an Nginx variable repeatedly in your Lua code, cache the Nginx variable value to your own Lua variable, for example, local val = ngx.var.some_var @@ -2656,7 +2676,7 @@ Setting ngx.var.Foo to a nil value will unset the nil while uninitialized (but defined) NGINX variables are evaluated to an empty Lua string. This API requires a relatively expensive metamethod call and it is recommended to avoid using it on hot code paths. @@ -2884,7 +2904,7 @@ Because of the metamethod magic, never "local" the ngx.ctx table ou local _M = {} -- the following line is bad since ngx.ctx is a per-request --- data while this `ctx` variable is on the Lua module level +-- data while this ctx variable is on the Lua module level -- and thus is per-nginx-worker. local ctx = ngx.ctx @@ -2908,7 +2928,7 @@ end return _M -That is, let the caller pass the `ctx` table explicitly via a function argument. +That is, let the caller pass the ctx table explicitly via a function argument. == ngx.location.capture == '''syntax:''' ''res = ngx.location.capture(uri, options?)'' @@ -4380,14 +4400,14 @@ an asynchronous operation and will return immediately. This behavior may change Explicitly specify the end of the response output stream. In the case of HTTP 1.1 chunked encoded output, it will just trigger the Nginx core to send out the "last chunk". -When you disable the HTTP 1.1 keep-alive feature for your downstream connections, you can rely on descent HTTP clients to close the connection actively for you when you call this method. This trick can be used do back-ground jobs without letting the HTTP clients to wait on the connection, as in the following example: +When you disable the HTTP 1.1 keep-alive feature for your downstream connections, you can rely on well written HTTP clients to close the connection actively for you when you call this method. This trick can be used do back-ground jobs without letting the HTTP clients to wait on the connection, as in the following example: location = /async { keepalive_timeout 0; content_by_lua_block { ngx.say("got the task!") - ngx.eof() -- a descent HTTP client will close the connection at this point + ngx.eof() -- well written HTTP clients will close the connection at this point -- access MySQL, PostgreSQL, Redis, Memcached, and etc here... } } @@ -5345,7 +5365,7 @@ The value argument and init argument can be any valid This method was first introduced in the v0.3.1rc22 release. -The optional `init` parameter was first added in the v0.10.6 release. +The optional init parameter was first added in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. @@ -5447,7 +5467,7 @@ Fetch a list of the keys from the dictionary, up to . By default, only the first 1024 keys (if any) are returned. When the argument is given the value 0, then all the keys will be returned even there is more than 1024 keys in the dictionary. -'''WARNING''' Be careful when calling this method on dictionaries with a really huge number of keys. This method may lock the dictionary for quite a while and block all the nginx worker processes that are trying to access the dictionary. +'''CAUTION''' Avoid calling this method on dictionaries with a very large number of keys as it may lock the dictionary for significant amount of time and block Nginx worker processes trying to access the dictionary. This feature was first introduced in the v0.7.3 release. @@ -5726,7 +5746,7 @@ session userdata returned by a previous sslhandshake call for exactly the same target. For short-lived connections, reusing SSL sessions can usually speed up the handshake by one order by magnitude but it is not so useful if the connection pool is enabled. This argument defaults to -`nil`. If this argument takes the boolean `false` value, no SSL session +nil. If this argument takes the boolean false value, no SSL session userdata would return by this call and only a Lua boolean will be returned as the first return value; otherwise the current SSL session will always be returned as the first argument in case of successes. @@ -5739,7 +5759,7 @@ also used to validate the server name specified in the server certificate sent f the remote. The optional ssl_verify argument takes a Lua boolean value to -control whether to perform SSL verification. When set to `true`, the server +control whether to perform SSL verification. When set to true, the server certificate will be verified according to the CA certificates specified by the [[#lua_ssl_trusted_certificate|lua_ssl_trusted_certificate]] directive. You may also need to adjust the [[#lua_ssl_verify_depth|lua_ssl_verify_depth]] @@ -6150,7 +6170,7 @@ Then it will generate the output 4 -"Light threads" are mostly useful for doing concurrent upstream requests in a single Nginx request handler, kinda like a generalized version of [[#ngx.location.capture_multi|ngx.location.capture_multi]] that can work with all the [[#Nginx API for Lua|Nginx API for Lua]]. The following example demonstrates parallel requests to MySQL, Memcached, and upstream HTTP services in a single Lua handler, and outputting the results in the order that they actually return (very much like the Facebook BigPipe model): +"Light threads" are mostly useful for making concurrent upstream requests in a single Nginx request handler, much like a generalized version of [[#ngx.location.capture_multi|ngx.location.capture_multi]] that can work with all the [[#Nginx API for Lua|Nginx API for Lua]]. The following example demonstrates parallel requests to MySQL, Memcached, and upstream HTTP services in a single Lua handler, and outputting the results in the order that they actually return (similar to Facebook's BigPipe model): -- query mysql, memcached, and a remote http service at the same time, @@ -6483,8 +6503,8 @@ This directive was first introduced in the v0.9.20 release. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' -This string field indicates the current NGINX subsystem the current Lua environment is based on. For this module, this field always takes the string value `"http"`. For -[https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module], however, this field takes the value `"stream"`. +This string field indicates the current NGINX subsystem the current Lua environment is based on. For this module, this field always takes the string value "http". For +[https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module], however, this field takes the value "stream". This field was first introduced in the 0.10.1. @@ -6565,7 +6585,7 @@ This API was first introduced in the 0.9.5 release. Returns the total number of the Nginx worker processes (i.e., the value configured by the [http://nginx.org/en/docs/ngx_core_module.html#worker_processes worker_processes] -directive in `nginx.conf`). +directive in nginx.conf). This API was first introduced in the 0.9.20 release. @@ -6577,11 +6597,11 @@ This API was first introduced in the 0.9.20 release. Returns the ordinal number of the current Nginx worker processes (starting from number 0). -So if the total number of workers is `N`, then this method may return a number between 0 -and `N - 1` (inclusive). +So if the total number of workers is N, then this method may return a number between 0 +and N - 1 (inclusive). This function returns meaningful values only for NGINX 1.9.1+. With earlier versions of NGINX, it -always returns `nil`. +always returns nil. See also [[#ngx.worker.count|ngx.worker.count]]. From 7f9387cedc5161e14c3b7fe01559b94c0a33e678 Mon Sep 17 00:00:00 2001 From: soul11201 Date: Wed, 15 Feb 2017 17:35:38 +0800 Subject: [PATCH 050/848] doc: fixed a typo. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index c13f2dacc2..349dbed7f8 100644 --- a/README.markdown +++ b/README.markdown @@ -3565,7 +3565,7 @@ When being used in the context of [init_worker_by_lua*](#init_worker_by_lua), th The `ngx.ctx` lookup requires relatively expensive metamethod calls and it is much slower than explicitly passing per-request data along by your own function arguments. So do not abuse this API for saving your own function arguments because it usually has quite some performance impact. -Because of the metamethod magic, never "local" the `ngx.ctx` table outside your Lua function scope on the Lua module level level due to [worker-level data sharing](#data-sharing-within-an-nginx-worker). For example, the following is bad: +Because of the metamethod magic, never "local" the `ngx.ctx` table outside your Lua function scope on the Lua module level due to [worker-level data sharing](#data-sharing-within-an-nginx-worker). For example, the following is bad: ```lua diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 71912d8796..8bc2acb6ee 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2897,7 +2897,7 @@ When being used in the context of [[#init_worker_by_lua|init_worker_by_lua*]], t The ngx.ctx lookup requires relatively expensive metamethod calls and it is much slower than explicitly passing per-request data along by your own function arguments. So do not abuse this API for saving your own function arguments because it usually has quite some performance impact. -Because of the metamethod magic, never "local" the ngx.ctx table outside your Lua function scope on the Lua module level level due to [[#Data_Sharing_within_an_Nginx_Worker|worker-level data sharing]]. For example, the following is bad: +Because of the metamethod magic, never "local" the ngx.ctx table outside your Lua function scope on the Lua module level due to [[#Data_Sharing_within_an_Nginx_Worker|worker-level data sharing]]. For example, the following is bad: -- mymodule.lua From 32fea9e665e5acd14d8d0348bfb0d53c666ea076 Mon Sep 17 00:00:00 2001 From: YuanSheng Date: Thu, 16 Feb 2017 10:35:37 +0800 Subject: [PATCH 051/848] util/reindex: trims the extra newline characters at the end of the test file. Signed-off-by: Yichun Zhang (agentzh) --- util/reindex | 2 ++ 1 file changed, 2 insertions(+) diff --git a/util/reindex b/util/reindex index bd54c9dfe1..f094a43dd4 100755 --- a/util/reindex +++ b/util/reindex @@ -44,7 +44,9 @@ sub reindex { push @lines, $_; } close $in; + my $text = join '', @lines; + $text =~ s/\n+\z/\n/sm; $text =~ s/(?x) \n+ === \s+ TEST/\n\n\n\n=== TEST/ixsg; $text =~ s/__(DATA|END)__\n+=== TEST/__${1}__\n\n=== TEST/; #$text =~ s/\n+$/\n\n/s; From 772e6fdaa99baa9a0dd2dd1a20dbdd80b35c7896 Mon Sep 17 00:00:00 2001 From: YuanSheng Date: Thu, 16 Feb 2017 15:20:33 +0800 Subject: [PATCH 052/848] tests: removed superfluous empty lines at the end of the test files. Signed-off-by: Yichun Zhang (agentzh) --- t/000--init.t | 1 - t/000-sanity.t | 1 - t/001-set.t | 1 - t/002-content.t | 1 - t/003-errors.t | 1 - t/004-require.t | 1 - t/005-exit.t | 1 - t/006-escape.t | 1 - t/007-md5.t | 1 - t/008-today.t | 1 - t/009-log.t | 1 - t/010-request_body.t | 1 - t/012-now.t | 1 - t/014-bugs.t | 1 - t/017-exec.t | 1 - t/018-ndk.t | 1 - t/019-const.t | 1 - t/021-cookie-time.t | 1 - t/023-rewrite/client-abort.t | 1 - t/023-rewrite/exec.t | 1 - t/023-rewrite/exit.t | 1 - t/023-rewrite/mixed.t | 1 - t/023-rewrite/multi-capture.t | 1 - t/023-rewrite/on-abort.t | 1 - t/023-rewrite/redirect.t | 1 - t/023-rewrite/req-body.t | 1 - t/023-rewrite/req-socket.t | 1 - t/023-rewrite/request_body.t | 1 - t/023-rewrite/sanity.t | 1 - t/023-rewrite/sleep.t | 1 - t/023-rewrite/socket-keepalive.t | 1 - t/023-rewrite/subrequest.t | 1 - t/023-rewrite/tcp-socket-timeout.t | 1 - t/023-rewrite/tcp-socket.t | 1 - t/023-rewrite/unix-socket.t | 1 - t/023-rewrite/uthread-redirect.t | 1 - t/023-rewrite/uthread-spawn.t | 1 - t/024-access/auth.t | 1 - t/024-access/client-abort.t | 1 - t/024-access/exit.t | 1 - t/024-access/multi-capture.t | 1 - t/024-access/on-abort.t | 1 - t/024-access/redirect.t | 1 - t/024-access/req-body.t | 1 - t/024-access/request_body.t | 1 - t/024-access/sanity.t | 1 - t/024-access/satisfy.t | 1 - t/024-access/sleep.t | 1 - t/024-access/subrequest.t | 1 - t/024-access/uthread-exec.t | 1 - t/024-access/uthread-exit.t | 1 - t/024-access/uthread-redirect.t | 1 - t/024-access/uthread-spawn.t | 1 - t/025-codecache.t | 1 - t/027-multi-capture.t | 1 - t/029-http-time.t | 1 - t/030-uri-args.t | 1 - t/032-iolist.t | 1 - t/033-ctx.t | 1 - t/035-gmatch.t | 1 - t/038-match-o.t | 1 - t/039-sub-o.t | 1 - t/040-gsub-o.t | 1 - t/041-header-filter.t | 1 - t/042-crc32.t | 1 - t/045-ngx-var.t | 1 - t/046-hmac.t | 1 - t/047-match-jit.t | 1 - t/048-match-dfa.t | 1 - t/049-gmatch-jit.t | 1 - t/051-sub-jit.t | 1 - t/053-gsub-jit.t | 1 - t/055-subreq-vars.t | 1 - t/056-flush.t | 1 - t/057-flush-timeout.t | 1 - t/060-lua-memcached.t | 1 - t/061-lua-redis.t | 1 - t/063-abort.t | 1 - t/064-pcall.t | 1 - t/065-tcp-socket-timeout.t | 1 - t/066-socket-receiveuntil.t | 1 - t/067-req-socket.t | 1 - t/069-null.t | 1 - t/071-idle-socket.t | 1 - t/072-conditional-get.t | 1 - t/073-backtrace.t | 1 - t/074-prefix-var.t | 1 - t/075-logby.t | 1 - t/077-sleep.t | 1 - t/078-hup-vars.t | 1 - t/079-unused-directives.t | 1 - t/080-hup-shdict.t | 1 - t/083-bad-sock-self.t | 1 - t/084-inclusive-receiveuntil.t | 1 - t/085-if.t | 1 - t/087-udp-socket.t | 1 - t/088-req-method.t | 1 - t/089-phase.t | 1 - t/091-coroutine.t | 1 - t/092-eof.t | 1 - t/093-uthread-spawn.t | 1 - t/094-uthread-exit.t | 1 - t/095-uthread-exec.t | 1 - t/096-uthread-redirect.t | 1 - t/097-uthread-rewrite.t | 1 - t/098-uthread-wait.t | 1 - t/100-client-abort.t | 1 - t/101-on-abort.t | 1 - t/102-req-start-time.t | 1 - t/103-req-http-ver.t | 1 - t/104-req-raw-header.t | 1 - t/105-pressure.t | 1 - t/106-timer.t | 1 - t/107-timer-errors.t | 1 - t/108-timer-safe.t | 1 - t/109-timer-hup.t | 1 - t/110-etag.t | 1 - t/111-req-header-ua.t | 1 - t/112-req-header-conn.t | 1 - t/113-req-header-cookie.t | 1 - t/115-quote-sql-str.t | 1 - t/116-raw-req-socket.t | 1 - t/117-raw-req-socket-timeout.t | 1 - t/119-config-prefix.t | 1 - t/121-version.t | 1 - t/122-worker.t | 1 - t/123-lua-path.t | 1 - t/125-configure-args.t | 1 - t/126-shdict-frag.t | 1 - t/127-uthread-kill.t | 1 - t/128-duplex-tcp-socket.t | 1 - t/130-internal-api.t | 1 - t/137-req-misc.t | 1 - 133 files changed, 133 deletions(-) diff --git a/t/000--init.t b/t/000--init.t index 364334f5f0..ad2d70e76d 100644 --- a/t/000--init.t +++ b/t/000--init.t @@ -84,4 +84,3 @@ GET /flush --- timeout: 10 --- no_error_log [error] - diff --git a/t/000-sanity.t b/t/000-sanity.t index 3f667522c1..87854ef99e 100644 --- a/t/000-sanity.t +++ b/t/000-sanity.t @@ -31,4 +31,3 @@ GET /lua GET /lua --- response_body helloworld - diff --git a/t/001-set.t b/t/001-set.t index 2295a2d7d9..ba8f22cb68 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -795,4 +795,3 @@ GET /lua?a=1&b=2 --- error_code: 500 --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ - diff --git a/t/002-content.t b/t/002-content.t index e6cb62d1d0..3f2460e676 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -836,4 +836,3 @@ GET /lua --- error_code: 500 --- error_log eval qr/failed to load inlined Lua code: / - diff --git a/t/003-errors.t b/t/003-errors.t index 764300a8ca..ad3a506da0 100644 --- a/t/003-errors.t +++ b/t/003-errors.t @@ -126,4 +126,3 @@ GET /main GET /main --- response_body 500 - diff --git a/t/004-require.t b/t/004-require.t index 3250b2f012..ec74116647 100644 --- a/t/004-require.t +++ b/t/004-require.t @@ -208,4 +208,3 @@ GET /ndk GET /ndk --- response_body %20 - diff --git a/t/005-exit.t b/t/005-exit.t index 781531f38a..a5a28b3e28 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -723,4 +723,3 @@ GET /t --- response_body --- no_error_log [error] - diff --git a/t/006-escape.t b/t/006-escape.t index 7b26bba7a1..0fb67a30f1 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -180,4 +180,3 @@ GET /t GET /t --- response_body [32] - diff --git a/t/007-md5.t b/t/007-md5.t index 501e3afab8..2ae9efb257 100644 --- a/t/007-md5.t +++ b/t/007-md5.t @@ -100,4 +100,3 @@ d41d8cd98f00b204e9800998ecf8427e GET /md5 --- response_body 6c8349cc7260ae62e3b1396831a8398f - diff --git a/t/008-today.t b/t/008-today.t index 54bd949e45..ec2f4338e4 100644 --- a/t/008-today.t +++ b/t/008-today.t @@ -36,4 +36,3 @@ GET /today --- request GET /today --- response_body_like: ^\d{4}-\d{2}-\d{2}$ - diff --git a/t/009-log.t b/t/009-log.t index 0c6a9a5e19..68c057f82d 100644 --- a/t/009-log.t +++ b/t/009-log.t @@ -542,4 +542,3 @@ ok [error] --- error_log eval "2: hello\0world, client: " - diff --git a/t/010-request_body.t b/t/010-request_body.t index 2640a544bf..e669d9480e 100644 --- a/t/010-request_body.t +++ b/t/010-request_body.t @@ -270,4 +270,3 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug - diff --git a/t/012-now.t b/t/012-now.t index abcb735431..58851879db 100644 --- a/t/012-now.t +++ b/t/012-now.t @@ -116,4 +116,3 @@ GET /time --- request GET /time --- response_body_like: ^\d{10,}(\.\d{1,3})?$ - diff --git a/t/014-bugs.t b/t/014-bugs.t index 8e5ee3f7f2..9aadff0e25 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -1018,4 +1018,3 @@ write timer set: 1 --- no_error_log [error] [alert] - diff --git a/t/017-exec.t b/t/017-exec.t index 4c7a918fd7..535c4ab8ee 100644 --- a/t/017-exec.t +++ b/t/017-exec.t @@ -572,4 +572,3 @@ hello, bah ["dummy", "dummy"] --- no_error_log [error] - diff --git a/t/018-ndk.t b/t/018-ndk.t index d68306bcd8..14293772d6 100644 --- a/t/018-ndk.t +++ b/t/018-ndk.t @@ -171,4 +171,3 @@ ok foo = a b --- no_error_log [error] - diff --git a/t/019-const.t b/t/019-const.t index fe79bfb651..4f9c7640c7 100644 --- a/t/019-const.t +++ b/t/019-const.t @@ -44,4 +44,3 @@ GET /read GET /read --- response_body 504 - diff --git a/t/021-cookie-time.t b/t/021-cookie-time.t index c00bbeac05..b05e401884 100644 --- a/t/021-cookie-time.t +++ b/t/021-cookie-time.t @@ -43,4 +43,3 @@ Thu, 18-Nov-10 11:27:35 GMT GET /lua --- response_body Thu, 18-Nov-10 11:27:35 GMT - diff --git a/t/023-rewrite/client-abort.t b/t/023-rewrite/client-abort.t index e9708023e3..117d17e5ef 100644 --- a/t/023-rewrite/client-abort.t +++ b/t/023-rewrite/client-abort.t @@ -848,4 +848,3 @@ delete thread 1 --- no_error_log [error] [alert] - diff --git a/t/023-rewrite/exec.t b/t/023-rewrite/exec.t index a063b5b22f..bd97968bc7 100644 --- a/t/023-rewrite/exec.t +++ b/t/023-rewrite/exec.t @@ -376,4 +376,3 @@ ngx.exec("@proxy") GET /main --- response_body hello, bah - diff --git a/t/023-rewrite/exit.t b/t/023-rewrite/exit.t index 9d292f7546..39ea5cbd62 100644 --- a/t/023-rewrite/exit.t +++ b/t/023-rewrite/exit.t @@ -595,4 +595,3 @@ F(ngx_http_send_header) { --- error_code: 204 --- no_error_log [error] - diff --git a/t/023-rewrite/mixed.t b/t/023-rewrite/mixed.t index 1156567605..0f742b255a 100644 --- a/t/023-rewrite/mixed.t +++ b/t/023-rewrite/mixed.t @@ -167,4 +167,3 @@ world\x03\x04\xff hello\x00\x01\x02 world\x03\x04\xff " - diff --git a/t/023-rewrite/multi-capture.t b/t/023-rewrite/multi-capture.t index 44629b079f..083ec78c90 100644 --- a/t/023-rewrite/multi-capture.t +++ b/t/023-rewrite/multi-capture.t @@ -392,4 +392,3 @@ res4.status = 201 res4.body = STORED\r " - diff --git a/t/023-rewrite/on-abort.t b/t/023-rewrite/on-abort.t index aca2ab6b13..336b7ce4b5 100644 --- a/t/023-rewrite/on-abort.t +++ b/t/023-rewrite/on-abort.t @@ -654,4 +654,3 @@ delete thread 2 --- no_error_log [error] - diff --git a/t/023-rewrite/redirect.t b/t/023-rewrite/redirect.t index 8843f1a7ee..99f3fd4f70 100644 --- a/t/023-rewrite/redirect.t +++ b/t/023-rewrite/redirect.t @@ -122,4 +122,3 @@ GET /read --- raw_response_headers_like: Location: /foo\r\n --- response_body_like: 302 Found --- error_code: 302 - diff --git a/t/023-rewrite/req-body.t b/t/023-rewrite/req-body.t index 2f42e0aeaf..13bdcb2511 100644 --- a/t/023-rewrite/req-body.t +++ b/t/023-rewrite/req-body.t @@ -221,4 +221,3 @@ hiya, world"] --- no_error_log [error] [alert] - diff --git a/t/023-rewrite/req-socket.t b/t/023-rewrite/req-socket.t index 34aedaab8c..87cbbbef26 100644 --- a/t/023-rewrite/req-socket.t +++ b/t/023-rewrite/req-socket.t @@ -532,4 +532,3 @@ Expect: 100-Continue \breceived: hello\b.*?\breceived: worl\b --- no_error_log [error] - diff --git a/t/023-rewrite/request_body.t b/t/023-rewrite/request_body.t index 0594001f48..b867d3a82c 100644 --- a/t/023-rewrite/request_body.t +++ b/t/023-rewrite/request_body.t @@ -170,4 +170,3 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug - diff --git a/t/023-rewrite/sanity.t b/t/023-rewrite/sanity.t index 20b00e29ec..b90aa0e167 100644 --- a/t/023-rewrite/sanity.t +++ b/t/023-rewrite/sanity.t @@ -799,4 +799,3 @@ test test --- no_error_log [alert] - diff --git a/t/023-rewrite/sleep.t b/t/023-rewrite/sleep.t index 1719784bd9..8d4c2da2ea 100644 --- a/t/023-rewrite/sleep.t +++ b/t/023-rewrite/sleep.t @@ -219,4 +219,3 @@ hello world hello world --- no_error_log [error] - diff --git a/t/023-rewrite/socket-keepalive.t b/t/023-rewrite/socket-keepalive.t index 50de0b39cf..489a70fc93 100644 --- a/t/023-rewrite/socket-keepalive.t +++ b/t/023-rewrite/socket-keepalive.t @@ -1007,4 +1007,3 @@ Not found, dear... --- error_code: 404 --- no_error_log [error] - diff --git a/t/023-rewrite/subrequest.t b/t/023-rewrite/subrequest.t index a307388ece..5d1e8f0839 100644 --- a/t/023-rewrite/subrequest.t +++ b/t/023-rewrite/subrequest.t @@ -639,4 +639,3 @@ the nginx core requires the patch https://github.com/agentzh/ngx_openresty/blob/ GET /t --- response_body done - diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index 68a27059a7..15bec7fad7 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -612,4 +612,3 @@ failed to send: timeout lua tcp socket send timeout: 102 lua tcp socket connect timeout: 60000 lua tcp socket write timed out - diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index e84734cca6..bff69a5876 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -2390,4 +2390,3 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):16: bad request/ --- no_error_log [alert] - diff --git a/t/023-rewrite/unix-socket.t b/t/023-rewrite/unix-socket.t index 098dd673aa..8a5f00019b 100644 --- a/t/023-rewrite/unix-socket.t +++ b/t/023-rewrite/unix-socket.t @@ -150,4 +150,3 @@ received: received: foo failed to receive a line: closed close: 1 nil - diff --git a/t/023-rewrite/uthread-redirect.t b/t/023-rewrite/uthread-redirect.t index 0f125e0913..83de1a36ed 100644 --- a/t/023-rewrite/uthread-redirect.t +++ b/t/023-rewrite/uthread-redirect.t @@ -186,4 +186,3 @@ free request --- error_code: 302 --- no_error_log [error] - diff --git a/t/023-rewrite/uthread-spawn.t b/t/023-rewrite/uthread-spawn.t index 58af7d064b..5552107e39 100644 --- a/t/023-rewrite/uthread-spawn.t +++ b/t/023-rewrite/uthread-spawn.t @@ -1449,4 +1449,3 @@ status: 204 --- no_error_log [error] --- timeout: 3 - diff --git a/t/024-access/auth.t b/t/024-access/auth.t index 5da09cbf0c..56e9862621 100644 --- a/t/024-access/auth.t +++ b/t/024-access/auth.t @@ -107,4 +107,3 @@ Location: /terms_of_use\.html GET /lua --- response_body_like: 403 Forbidden --- error_code: 403 - diff --git a/t/024-access/client-abort.t b/t/024-access/client-abort.t index a94f822d09..c16f4eaab7 100644 --- a/t/024-access/client-abort.t +++ b/t/024-access/client-abort.t @@ -850,4 +850,3 @@ delete thread 1 --- no_error_log [error] [alert] - diff --git a/t/024-access/exit.t b/t/024-access/exit.t index 8470ab9cef..d6d66b8cfe 100644 --- a/t/024-access/exit.t +++ b/t/024-access/exit.t @@ -545,4 +545,3 @@ Not found, dear... --- response_body Not found, dear... --- error_code: 404 - diff --git a/t/024-access/multi-capture.t b/t/024-access/multi-capture.t index 368d40154e..930b74dad9 100644 --- a/t/024-access/multi-capture.t +++ b/t/024-access/multi-capture.t @@ -392,4 +392,3 @@ res4.status = 201 res4.body = STORED\r " - diff --git a/t/024-access/on-abort.t b/t/024-access/on-abort.t index 0c17b55fee..5bb948bb68 100644 --- a/t/024-access/on-abort.t +++ b/t/024-access/on-abort.t @@ -649,4 +649,3 @@ delete thread 2 --- no_error_log [error] - diff --git a/t/024-access/redirect.t b/t/024-access/redirect.t index c7ce512eb0..b45fac718b 100644 --- a/t/024-access/redirect.t +++ b/t/024-access/redirect.t @@ -122,4 +122,3 @@ GET /read --- raw_response_headers_like: Location: /foo\r\n --- response_body_like: 302 Found --- error_code: 302 - diff --git a/t/024-access/req-body.t b/t/024-access/req-body.t index fd33aff37e..70db85c1c2 100644 --- a/t/024-access/req-body.t +++ b/t/024-access/req-body.t @@ -218,4 +218,3 @@ hiya, world"] --- no_error_log [error] [alert] - diff --git a/t/024-access/request_body.t b/t/024-access/request_body.t index a6fead207d..fa03195272 100644 --- a/t/024-access/request_body.t +++ b/t/024-access/request_body.t @@ -170,4 +170,3 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug - diff --git a/t/024-access/sanity.t b/t/024-access/sanity.t index de63a68ef7..7ff177fcf7 100644 --- a/t/024-access/sanity.t +++ b/t/024-access/sanity.t @@ -741,4 +741,3 @@ test test --- no_error_log [alert] - diff --git a/t/024-access/satisfy.t b/t/024-access/satisfy.t index 10d3ecef39..7902f4938e 100644 --- a/t/024-access/satisfy.t +++ b/t/024-access/satisfy.t @@ -209,4 +209,3 @@ something important --- error_code: 200 --- no_error_log [error] - diff --git a/t/024-access/sleep.t b/t/024-access/sleep.t index 2eb08227af..fc1fc024b5 100644 --- a/t/024-access/sleep.t +++ b/t/024-access/sleep.t @@ -219,4 +219,3 @@ hello world hello world --- no_error_log [error] - diff --git a/t/024-access/subrequest.t b/t/024-access/subrequest.t index bfe96d6a91..b6ccf11990 100644 --- a/t/024-access/subrequest.t +++ b/t/024-access/subrequest.t @@ -599,4 +599,3 @@ the nginx core requires the patch https://github.com/agentzh/ngx_openresty/blob/ GET /t --- response_body done - diff --git a/t/024-access/uthread-exec.t b/t/024-access/uthread-exec.t index d7c23213eb..7add3d4218 100644 --- a/t/024-access/uthread-exec.t +++ b/t/024-access/uthread-exec.t @@ -344,4 +344,3 @@ free request end --- error_log attempt to abort with pending subrequests - diff --git a/t/024-access/uthread-exit.t b/t/024-access/uthread-exit.t index da4a9dbfe4..7c146ae7ec 100644 --- a/t/024-access/uthread-exit.t +++ b/t/024-access/uthread-exit.t @@ -1311,4 +1311,3 @@ free request end --- error_log attempt to abort with pending subrequests - diff --git a/t/024-access/uthread-redirect.t b/t/024-access/uthread-redirect.t index 8b030ac317..4eb4759585 100644 --- a/t/024-access/uthread-redirect.t +++ b/t/024-access/uthread-redirect.t @@ -187,4 +187,3 @@ free request --- error_code: 302 --- no_error_log [error] - diff --git a/t/024-access/uthread-spawn.t b/t/024-access/uthread-spawn.t index 415e4c0c4c..7c7ba3b931 100644 --- a/t/024-access/uthread-spawn.t +++ b/t/024-access/uthread-spawn.t @@ -1116,4 +1116,3 @@ body: hello world)$ --- no_error_log [error] - diff --git a/t/025-codecache.t b/t/025-codecache.t index f33452a780..0b02a2722f 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1244,4 +1244,3 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, "decrementing the reference count for Lua VM: 1", "lua close the global Lua VM", ] - diff --git a/t/027-multi-capture.t b/t/027-multi-capture.t index 3588c69f45..9227fe5329 100644 --- a/t/027-multi-capture.t +++ b/t/027-multi-capture.t @@ -752,4 +752,3 @@ proxy_cache_path conf/cache levels=1:2 keys_zone=STATIC:10m inactive=10m max_siz GET /foo --- response_body ok - diff --git a/t/029-http-time.t b/t/029-http-time.t index 71a7758cad..ecef4921a7 100644 --- a/t/029-http-time.t +++ b/t/029-http-time.t @@ -85,4 +85,3 @@ GET /lua GET /lua --- response_body nil - diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 8ee8401688..30d66d0401 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1389,4 +1389,3 @@ GET /lua GET /foo?world --- response_body_like ^HTTP/1.0 (a=3&b|b&a=3)$ - diff --git a/t/032-iolist.t b/t/032-iolist.t index ddf3d7f073..3c56032892 100644 --- a/t/032-iolist.t +++ b/t/032-iolist.t @@ -76,4 +76,3 @@ GET /lua GET /lua --- response_body_like: 500 Internal Server Error --- error_code: 500 - diff --git a/t/033-ctx.t b/t/033-ctx.t index eefb21661b..8fc50aa7a0 100644 --- a/t/033-ctx.t +++ b/t/033-ctx.t @@ -440,4 +440,3 @@ lua release ngx.ctx at ref --- response_body --- no_error_log [error] - diff --git a/t/035-gmatch.t b/t/035-gmatch.t index cbc673385e..5b63ae45a2 100644 --- a/t/035-gmatch.t +++ b/t/035-gmatch.t @@ -901,4 +901,3 @@ end GET /re --- response_body failed to match - diff --git a/t/038-match-o.t b/t/038-match-o.t index 628e27f558..d61ff1fdaa 100644 --- a/t/038-match-o.t +++ b/t/038-match-o.t @@ -740,4 +740,3 @@ false hello false false - diff --git a/t/039-sub-o.t b/t/039-sub-o.t index a861b6cce8..580a67161e 100644 --- a/t/039-sub-o.t +++ b/t/039-sub-o.t @@ -578,4 +578,3 @@ a [b c] [b] [c] [] [] d --- response_body a [b c] [b] [c] d 1 - diff --git a/t/040-gsub-o.t b/t/040-gsub-o.t index 90619b06ee..534726611d 100644 --- a/t/040-gsub-o.t +++ b/t/040-gsub-o.t @@ -198,4 +198,3 @@ hello, world --- response_body [hello,h], [world,w] 2 - diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 553ee432e2..9cca3b749f 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -790,4 +790,3 @@ GET /t --- error_code: 302 --- no_error_log [error] - diff --git a/t/042-crc32.t b/t/042-crc32.t index 86d9201212..73aa1f433d 100644 --- a/t/042-crc32.t +++ b/t/042-crc32.t @@ -54,4 +54,3 @@ GET /test GET /test --- response_body 0 - diff --git a/t/045-ngx-var.t b/t/045-ngx-var.t index 8f2dcb38ac..b5fcc8107b 100644 --- a/t/045-ngx-var.t +++ b/t/045-ngx-var.t @@ -226,4 +226,3 @@ GET /test?hello --- error_log variable "query_string" not changeable --- error_code: 500 - diff --git a/t/046-hmac.t b/t/046-hmac.t index 686b479e11..32e222ba72 100644 --- a/t/046-hmac.t +++ b/t/046-hmac.t @@ -29,4 +29,3 @@ __DATA__ GET /lua --- response_body R/pvxzHC4NLtj7S+kXFg/NePTmk= - diff --git a/t/047-match-jit.t b/t/047-match-jit.t index 077ebb6684..2417a63ad3 100644 --- a/t/047-match-jit.t +++ b/t/047-match-jit.t @@ -212,4 +212,3 @@ end GET /re --- response_body failed to match - diff --git a/t/048-match-dfa.t b/t/048-match-dfa.t index 8a0a328f43..28b5a604c3 100644 --- a/t/048-match-dfa.t +++ b/t/048-match-dfa.t @@ -207,4 +207,3 @@ exec opts: 0 你 --- no_error_log [error] - diff --git a/t/049-gmatch-jit.t b/t/049-gmatch-jit.t index 614c440388..5134d526b5 100644 --- a/t/049-gmatch-jit.t +++ b/t/049-gmatch-jit.t @@ -226,4 +226,3 @@ qr/pcre JIT compiling result: \d+/ error: pcre_compile() failed: missing ) in "(abc" --- no_error_log [error] - diff --git a/t/051-sub-jit.t b/t/051-sub-jit.t index 789e897aa2..c8a49c0516 100644 --- a/t/051-sub-jit.t +++ b/t/051-sub-jit.t @@ -147,4 +147,3 @@ error: pcre_compile() failed: missing ) in "(abc" error: pcre_compile() failed: missing ) in "(abc" --- no_error_log [error] - diff --git a/t/053-gsub-jit.t b/t/053-gsub-jit.t index 3a2a1aa4e5..3efc0a24a5 100644 --- a/t/053-gsub-jit.t +++ b/t/053-gsub-jit.t @@ -147,4 +147,3 @@ error: pcre_compile() failed: missing ) in "(abc" error: pcre_compile() failed: missing ) in "(abc" --- no_error_log [error] - diff --git a/t/055-subreq-vars.t b/t/055-subreq-vars.t index 2fc696033b..eb5e24d447 100644 --- a/t/055-subreq-vars.t +++ b/t/055-subreq-vars.t @@ -336,4 +336,3 @@ dog = hiya cat = 56 parent dog: blah parent cat: foo - diff --git a/t/056-flush.t b/t/056-flush.t index d189cd5837..6b697a4233 100644 --- a/t/056-flush.t +++ b/t/056-flush.t @@ -520,4 +520,3 @@ qr/lua flush requires waiting: buffered 0x[0-9a-f]+, delayed:1/, --- no_error_log [error] --- timeout: 4 - diff --git a/t/057-flush-timeout.t b/t/057-flush-timeout.t index d6e0f868ef..a04653991e 100644 --- a/t/057-flush-timeout.t +++ b/t/057-flush-timeout.t @@ -318,4 +318,3 @@ qr/failed to flush: client aborted/, --- timeout: 0.2 --- abort --- wait: 1 - diff --git a/t/060-lua-memcached.t b/t/060-lua-memcached.t index e1ebb92b84..0751238627 100644 --- a/t/060-lua-memcached.t +++ b/t/060-lua-memcached.t @@ -166,4 +166,3 @@ some_key: hello 1234 [error] --- error_log lua reuse free buf memory - diff --git a/t/061-lua-redis.t b/t/061-lua-redis.t index d7b1876849..ebfa6d1997 100644 --- a/t/061-lua-redis.t +++ b/t/061-lua-redis.t @@ -182,4 +182,3 @@ abort: function msg type: nil --- no_error_log [error] - diff --git a/t/063-abort.t b/t/063-abort.t index c112ee10a2..411a07eee9 100644 --- a/t/063-abort.t +++ b/t/063-abort.t @@ -1017,4 +1017,3 @@ foo --- no_error_log [error] --- timeout: 2 - diff --git a/t/064-pcall.t b/t/064-pcall.t index cf90a07ae3..3011f3e46a 100644 --- a/t/064-pcall.t +++ b/t/064-pcall.t @@ -104,4 +104,3 @@ $/ --- no_error_log [error] - diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 6f903e89f1..212766ecf6 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -994,4 +994,3 @@ close: 1 nil --- no_error_log [error] - diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 3bf5229694..ffe74aa354 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -1329,4 +1329,3 @@ this exposed a memory leak in receiveuntil ok --- no_error_log [error] - diff --git a/t/067-req-socket.t b/t/067-req-socket.t index 30a653ad21..229d5ccf41 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -1096,4 +1096,3 @@ done --- grep_error_log_out lua finalize socket GC cycle done - diff --git a/t/069-null.t b/t/069-null.t index 7761c917cf..e4c26afb90 100644 --- a/t/069-null.t +++ b/t/069-null.t @@ -93,4 +93,3 @@ done ngx.null: null --- no_error_log [error] - diff --git a/t/071-idle-socket.t b/t/071-idle-socket.t index 55d25c650e..c9002be8ea 100644 --- a/t/071-idle-socket.t +++ b/t/071-idle-socket.t @@ -431,4 +431,3 @@ failed to set keepalive: unread data in buffer } --- no_error_log [error] - diff --git a/t/072-conditional-get.t b/t/072-conditional-get.t index 4bb567a9e7..7cf2dcd22d 100644 --- a/t/072-conditional-get.t +++ b/t/072-conditional-get.t @@ -88,4 +88,3 @@ delete thread 1 say failed: nginx output filter error --- no_error_log [error] - diff --git a/t/073-backtrace.t b/t/073-backtrace.t index aae4bae8a5..6c5469218d 100644 --- a/t/073-backtrace.t +++ b/t/073-backtrace.t @@ -187,4 +187,3 @@ probe process("$LIBLUA_PATH").function("lua_concat") { :79: in function 'func20' :83: in function 'func21' ... - diff --git a/t/074-prefix-var.t b/t/074-prefix-var.t index 3cc4587d05..c116d8465a 100644 --- a/t/074-prefix-var.t +++ b/t/074-prefix-var.t @@ -64,4 +64,3 @@ GET /t Greetings from module foo. --- no_error_log [error] - diff --git a/t/075-logby.t b/t/075-logby.t index f987c8ee58..520c62e314 100644 --- a/t/075-logby.t +++ b/t/075-logby.t @@ -581,4 +581,3 @@ qr{log_by_lua\(nginx\.conf:\d+\):1: content-type: text/plain} --- no_error_log [error] - diff --git a/t/077-sleep.t b/t/077-sleep.t index a7c251a500..7d295c2b5d 100644 --- a/t/077-sleep.t +++ b/t/077-sleep.t @@ -404,4 +404,3 @@ ok --- no_error_log [error] [alert] - diff --git a/t/078-hup-vars.t b/t/078-hup-vars.t index 8acc346d21..5072c4d292 100644 --- a/t/078-hup-vars.t +++ b/t/078-hup-vars.t @@ -62,4 +62,3 @@ GET /t localhost --- no_error_log [error] - diff --git a/t/079-unused-directives.t b/t/079-unused-directives.t index cba0e414c6..aacd0d31d5 100644 --- a/t/079-unused-directives.t +++ b/t/079-unused-directives.t @@ -340,4 +340,3 @@ GET /t sub: sub --- no_error_log [error] - diff --git a/t/080-hup-shdict.t b/t/080-hup-shdict.t index f9a0278f45..c7625568ce 100644 --- a/t/080-hup-shdict.t +++ b/t/080-hup-shdict.t @@ -82,4 +82,3 @@ GET /test 10502 number --- no_error_log [error] - diff --git a/t/083-bad-sock-self.t b/t/083-bad-sock-self.t index b93849f055..7a76752d9f 100644 --- a/t/083-bad-sock-self.t +++ b/t/083-bad-sock-self.t @@ -136,4 +136,3 @@ bad argument #1 to 'close' (table expected, got number) --- error_code: 500 --- error_log bad argument #1 to 'setkeepalive' (table expected, got number) - diff --git a/t/084-inclusive-receiveuntil.t b/t/084-inclusive-receiveuntil.t index c96601582b..f7d5d3d189 100644 --- a/t/084-inclusive-receiveuntil.t +++ b/t/084-inclusive-receiveuntil.t @@ -743,4 +743,3 @@ close: 1 nil } --- no_error_log [error] - diff --git a/t/085-if.t b/t/085-if.t index e08b43caec..33f57caf7e 100644 --- a/t/085-if.t +++ b/t/085-if.t @@ -198,4 +198,3 @@ GET /proxy-pass-uri --- response_body_like: It works! --- no_error_log [error] - diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index c618fd7ad0..a58709b5a7 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -1102,4 +1102,3 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ --- no_error_log [alert] - diff --git a/t/088-req-method.t b/t/088-req-method.t index 9ced40ccd6..198b3b3492 100644 --- a/t/088-req-method.t +++ b/t/088-req-method.t @@ -262,4 +262,3 @@ method: PATCH method: TRACE --- no_error_log [error] - diff --git a/t/089-phase.t b/t/089-phase.t index 57921fc55e..94b7619ec9 100644 --- a/t/089-phase.t +++ b/t/089-phase.t @@ -176,4 +176,3 @@ GET /lua init_worker --- no_error_log [error] - diff --git a/t/091-coroutine.t b/t/091-coroutine.t index 5f348c2950..bceb512460 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -1315,4 +1315,3 @@ co yield: 2 --- no_error_log [error] - diff --git a/t/092-eof.t b/t/092-eof.t index a75711f36d..86778dedbf 100644 --- a/t/092-eof.t +++ b/t/092-eof.t @@ -80,4 +80,3 @@ GET /t --- error_log hello, tom hello, jim - diff --git a/t/093-uthread-spawn.t b/t/093-uthread-spawn.t index 772e866c00..b622d30123 100644 --- a/t/093-uthread-spawn.t +++ b/t/093-uthread-spawn.t @@ -1672,4 +1672,3 @@ delete thread 2 f --- no_error_log [error] - diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index cd678c0727..a62377392f 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -1648,4 +1648,3 @@ free request [alert] [error] [warn] - diff --git a/t/095-uthread-exec.t b/t/095-uthread-exec.t index 4459360ca6..56156c48a8 100644 --- a/t/095-uthread-exec.t +++ b/t/095-uthread-exec.t @@ -423,4 +423,3 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] - diff --git a/t/096-uthread-redirect.t b/t/096-uthread-redirect.t index b0258fb75e..003c6423de 100644 --- a/t/096-uthread-redirect.t +++ b/t/096-uthread-redirect.t @@ -277,4 +277,3 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] - diff --git a/t/097-uthread-rewrite.t b/t/097-uthread-rewrite.t index 178a3749ce..a93adc8604 100644 --- a/t/097-uthread-rewrite.t +++ b/t/097-uthread-rewrite.t @@ -344,4 +344,3 @@ free request end --- error_log attempt to abort with pending subrequests - diff --git a/t/098-uthread-wait.t b/t/098-uthread-wait.t index aaf020a0ef..4948596f91 100644 --- a/t/098-uthread-wait.t +++ b/t/098-uthread-wait.t @@ -1321,4 +1321,3 @@ $s; --- no_error_log [error] [alert] - diff --git a/t/100-client-abort.t b/t/100-client-abort.t index cd4685933e..89c1f6ac08 100644 --- a/t/100-client-abort.t +++ b/t/100-client-abort.t @@ -1064,4 +1064,3 @@ GET /t eof succeeded --- error_log eof failed: nginx output filter error - diff --git a/t/101-on-abort.t b/t/101-on-abort.t index 81cec78eb0..182d12c546 100644 --- a/t/101-on-abort.t +++ b/t/101-on-abort.t @@ -846,4 +846,3 @@ done client prematurely closed connection on abort called main handler done - diff --git a/t/102-req-start-time.t b/t/102-req-start-time.t index 1d5fe288aa..3b041afda7 100644 --- a/t/102-req-start-time.t +++ b/t/102-req-start-time.t @@ -113,4 +113,3 @@ true$ --- no_error_log [error] --- SKIP - diff --git a/t/103-req-http-ver.t b/t/103-req-http-ver.t index 6ded75a68a..e6de2388c5 100644 --- a/t/103-req-http-ver.t +++ b/t/103-req-http-ver.t @@ -46,4 +46,3 @@ GET /t HTTP/1.0 1 --- no_error_log [error] - diff --git a/t/104-req-raw-header.t b/t/104-req-raw-header.t index 439b9def3e..efea03cb27 100644 --- a/t/104-req-raw-header.t +++ b/t/104-req-raw-header.t @@ -876,4 +876,3 @@ Foo: bar\r --- no_error_log [error] --- timeout: 5 - diff --git a/t/105-pressure.t b/t/105-pressure.t index 10f495e79a..9d1ba1ff1a 100644 --- a/t/105-pressure.t +++ b/t/105-pressure.t @@ -51,4 +51,3 @@ $::Id = int rand 10000; --- no_error_log [error] - diff --git a/t/106-timer.t b/t/106-timer.t index d0f6f36bdf..04a532e91c 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -2193,4 +2193,3 @@ ok --- error_log Bad bad bad --- skip_nginx: 4: < 1.7.1 - diff --git a/t/107-timer-errors.t b/t/107-timer-errors.t index 9ed1334d54..32016125cf 100644 --- a/t/107-timer-errors.t +++ b/t/107-timer-errors.t @@ -1420,4 +1420,3 @@ qr/\[error\] .*? runtime error: content_by_lua\(nginx\.conf:\d+\):3: API disable "lua ngx.timer expired", "http lua close fake http connection" ] - diff --git a/t/108-timer-safe.t b/t/108-timer-safe.t index 19c9f6d438..2795998a51 100644 --- a/t/108-timer-safe.t +++ b/t/108-timer-safe.t @@ -1395,4 +1395,3 @@ registered timer lua ngx.timer expired http lua close fake http connection trace: [m][f][g] - diff --git a/t/109-timer-hup.t b/t/109-timer-hup.t index 15aaa0ea5d..0864046361 100644 --- a/t/109-timer-hup.t +++ b/t/109-timer-hup.t @@ -500,4 +500,3 @@ ok --- grep_error_log_out lua found 8191 pending timers --- timeout: 20 - diff --git a/t/110-etag.t b/t/110-etag.t index 351fec44f2..0a94d3d4f0 100644 --- a/t/110-etag.t +++ b/t/110-etag.t @@ -81,4 +81,3 @@ If-Modified-Since: Thu, 10 May 2012 07:50:59 GMT --- no_error_log [error] --- skip_nginx: 3: < 1.3.3 - diff --git a/t/111-req-header-ua.t b/t/111-req-header-ua.t index 7c980d00fa..9e501e3c08 100644 --- a/t/111-req-header-ua.t +++ b/t/111-req-header-ua.t @@ -673,4 +673,3 @@ content: konqueror: 1 User-Agent: Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.10 (like Gecko) (Kubuntu) --- no_error_log [error] - diff --git a/t/112-req-header-conn.t b/t/112-req-header-conn.t index 51df730dc2..51bc8a4319 100644 --- a/t/112-req-header-conn.t +++ b/t/112-req-header-conn.t @@ -146,4 +146,3 @@ content: conn type: 0 connection: bad --- no_error_log [error] - diff --git a/t/113-req-header-cookie.t b/t/113-req-header-cookie.t index b26b7095f9..4a9305342b 100644 --- a/t/113-req-header-cookie.t +++ b/t/113-req-header-cookie.t @@ -247,4 +247,3 @@ Cookie: boo=123; foo=bar --- no_error_log [error] - diff --git a/t/115-quote-sql-str.t b/t/115-quote-sql-str.t index 0c20dfb9b9..66553b1341 100644 --- a/t/115-quote-sql-str.t +++ b/t/115-quote-sql-str.t @@ -74,4 +74,3 @@ GET /set 'a\Zb\Z' --- no_error_log [error] - diff --git a/t/116-raw-req-socket.t b/t/116-raw-req-socket.t index 6518f0e5e7..ab06ee4a31 100644 --- a/t/116-raw-req-socket.t +++ b/t/116-raw-req-socket.t @@ -876,4 +876,3 @@ request body: hey, hello world --- no_error_log [error] [alert] - diff --git a/t/117-raw-req-socket-timeout.t b/t/117-raw-req-socket-timeout.t index 4e3929b260..07abadcc6a 100644 --- a/t/117-raw-req-socket-timeout.t +++ b/t/117-raw-req-socket-timeout.t @@ -114,4 +114,3 @@ lua tcp socket write timed out server: failed to send: timeout --- no_error_log [alert] - diff --git a/t/119-config-prefix.t b/t/119-config-prefix.t index 4581aa1209..3f793206e4 100644 --- a/t/119-config-prefix.t +++ b/t/119-config-prefix.t @@ -30,4 +30,3 @@ GET /lua ^prefix: \/\S+$ --- no_error_log [error] - diff --git a/t/121-version.t b/t/121-version.t index 2b7a306e50..9000eb3708 100644 --- a/t/121-version.t +++ b/t/121-version.t @@ -46,4 +46,3 @@ GET /lua ^version: \d+$ --- no_error_log [error] - diff --git a/t/122-worker.t b/t/122-worker.t index fa42b1d9ad..b74c81fbcd 100644 --- a/t/122-worker.t +++ b/t/122-worker.t @@ -79,4 +79,3 @@ worker pid: \d+ worker pid is correct\. --- no_error_log [error] - diff --git a/t/123-lua-path.t b/t/123-lua-path.t index da9790968d..23681a9dbb 100644 --- a/t/123-lua-path.t +++ b/t/123-lua-path.t @@ -68,4 +68,3 @@ GET /lua [error] --- error_log eval qr/\[alert\] .*? lua_code_cache is off/ - diff --git a/t/125-configure-args.t b/t/125-configure-args.t index adec12923c..4160d4e3a9 100644 --- a/t/125-configure-args.t +++ b/t/125-configure-args.t @@ -29,4 +29,3 @@ GET /configure_args ^\s*\-\-[^-]+ --- no_error_log [error] - diff --git a/t/126-shdict-frag.t b/t/126-shdict-frag.t index 94422fb8c8..5a2aff8202 100644 --- a/t/126-shdict-frag.t +++ b/t/126-shdict-frag.t @@ -1264,4 +1264,3 @@ ok --- no_error_log [error] --- timeout: 60 - diff --git a/t/127-uthread-kill.t b/t/127-uthread-kill.t index 7e7d3dd6af..cc43c62764 100644 --- a/t/127-uthread-kill.t +++ b/t/127-uthread-kill.t @@ -505,4 +505,3 @@ thread created: zombie [alert] lua tcp socket abort resolver --- error_log - diff --git a/t/128-duplex-tcp-socket.t b/t/128-duplex-tcp-socket.t index d3ef3f5926..d64ae6a106 100644 --- a/t/128-duplex-tcp-socket.t +++ b/t/128-duplex-tcp-socket.t @@ -623,4 +623,3 @@ close: 1 nil --- no_error_log [error] - diff --git a/t/130-internal-api.t b/t/130-internal-api.t index d641ab5083..eba0980f8f 100644 --- a/t/130-internal-api.t +++ b/t/130-internal-api.t @@ -47,4 +47,3 @@ content req=0x[a-f0-9]{4,} $ --- no_error_log [error] - diff --git a/t/137-req-misc.t b/t/137-req-misc.t index 20ada3c138..d56f80e6fd 100644 --- a/t/137-req-misc.t +++ b/t/137-req-misc.t @@ -59,4 +59,3 @@ not internal GET /test --- response_body internal - From 2e7031f01dd5bffa09a98773ad649c9e4930706d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 16 Feb 2017 09:51:32 -0800 Subject: [PATCH 053/848] util/reindex: fixed the author and copyright notice. --- util/reindex | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/util/reindex b/util/reindex index f094a43dd4..3e9aebc9b9 100755 --- a/util/reindex +++ b/util/reindex @@ -1,8 +1,7 @@ #!/usr/bin/perl #: reindex.pl #: reindex .t files for Test::Base based test files -#: Copyright (c) 2006 Agent Zhang -#: 2006-04-27 2006-05-09 +#: Copyright (c) Yichun Zhang use strict; use warnings; From 3217a078164fe4af806150bc7fdc6af58f92e50b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 16 Feb 2017 09:53:08 -0800 Subject: [PATCH 054/848] util/reindex: fixed the shebang line. --- util/reindex | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/util/reindex b/util/reindex index 3e9aebc9b9..ffc053e7be 100755 --- a/util/reindex +++ b/util/reindex @@ -1,6 +1,6 @@ -#!/usr/bin/perl -#: reindex.pl -#: reindex .t files for Test::Base based test files +#!/usr/bin/env perl + +#: reindexes .t files for Test::Base based test files #: Copyright (c) Yichun Zhang use strict; From 11495c28a045e3f5b4b3a574f71579db1c497125 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 16 Feb 2017 10:22:03 -0800 Subject: [PATCH 055/848] util/reindex: tweaked the regex to avoid useless substitutions. this is a followup fix for commit 772e6fd. --- util/reindex | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/reindex b/util/reindex index ffc053e7be..d0a0927603 100755 --- a/util/reindex +++ b/util/reindex @@ -45,7 +45,7 @@ sub reindex { close $in; my $text = join '', @lines; - $text =~ s/\n+\z/\n/sm; + $text =~ s/\n\n+\z/\n/sm; $text =~ s/(?x) \n+ === \s+ TEST/\n\n\n\n=== TEST/ixsg; $text =~ s/__(DATA|END)__\n+=== TEST/__${1}__\n\n=== TEST/; #$text =~ s/\n+$/\n\n/s; From e86dd9662841cdf0e5cc5b800dd9506eb7a785dd Mon Sep 17 00:00:00 2001 From: SmirnovW Date: Thu, 16 Feb 2017 23:04:31 +0400 Subject: [PATCH 056/848] doc: added more hints as comments to installation commands. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 6 ++++++ doc/HttpLuaModule.wiki | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/README.markdown b/README.markdown index 349dbed7f8..826b1e9510 100644 --- a/README.markdown +++ b/README.markdown @@ -298,6 +298,12 @@ Build the source with this module: --add-module=/path/to/ngx_devel_kit \ --add-module=/path/to/lua-nginx-module + # Note that you may also want to add `./configure` options which are used in your + # current nginx build. + # You can get usually those options using command nginx -V + + # you can change the parallism number 2 below to fit the number of spare CPU cores in your + # machine. make -j2 make install ``` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 8bc2acb6ee..fec84cd00d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -231,6 +231,12 @@ Build the source with this module: --add-module=/path/to/ngx_devel_kit \ --add-module=/path/to/lua-nginx-module + # Note that you may also want to add `./configure` options which are used in your + # current nginx build. + # You can get usually those options using command nginx -V + + # you can change the parallism number 2 below to fit the number of spare CPU cores in your + # machine. make -j2 make install From fdbfdaecf8b38d6d498c28e01b03975b23706d2e Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 24 Feb 2017 22:27:06 +0800 Subject: [PATCH 057/848] semaphore: fixed a typo in a code comment. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_semaphore.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 8a3f832806..604d943471 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -387,8 +387,8 @@ ngx_http_lua_ffi_sema_wait(ngx_http_request_t *r, return NGX_ERROR; } - /* we keep the order, will resume the older waited firtly - * in ngx_http_lua_sema_handler + /* we keep the order, will first resume the thread waiting for the + * longest time in ngx_http_lua_sema_handler */ if (ngx_queue_empty(&sem->wait_queue) && sem->resource_count > 0) { From 2fd7daa69816be7ac2298a789cb3b819e0d12052 Mon Sep 17 00:00:00 2001 From: Robert Paprocki Date: Thu, 16 Feb 2017 09:39:31 -0800 Subject: [PATCH 058/848] bugfix: typo fix in C POST args handler debug log. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_args.c | 2 +- t/031-post-args.t | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index 0c307d6ab7..b43697c7a1 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -184,7 +184,7 @@ ngx_http_lua_ngx_req_get_post_args(lua_State *L) if (r->request_body->temp_file) { lua_pushnil(L); - lua_pushliteral(L, "requesty body in temp file not supported"); + lua_pushliteral(L, "request body in temp file not supported"); return 2; } diff --git a/t/031-post-args.t b/t/031-post-args.t index c2b6b8f372..62c88c1ec7 100644 --- a/t/031-post-args.t +++ b/t/031-post-args.t @@ -351,6 +351,6 @@ CORE::join("", @k); POST /lua a=3&b=4&c --- response_body -requesty body in temp file not supported +request body in temp file not supported --- no_error_log [error] From 38dd154a78a4291d2cffba4f23ace56f3b59f2c4 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 11 Mar 2017 17:59:18 -0800 Subject: [PATCH 059/848] doc: ngx.exit() also returns immediately in the balancer_by_lua* context. thanks Jinhua Tan for the patch in #1012. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 826b1e9510..dd47e9cb0a 100644 --- a/README.markdown +++ b/README.markdown @@ -5268,7 +5268,7 @@ Note that while this method accepts all [HTTP status constants](#http-status-con Also note that this method call terminates the processing of the current request and that it is recommended that a coding style that combines this method call with the `return` statement, i.e., `return ngx.exit(...)` be used to reinforce the fact that the request processing is being terminated. -When being used in the contexts of [header_filter_by_lua](#header_filter_by_lua) and +When being used in the contexts of [header_filter_by_lua*](#header_filter_by_lua), [balancer_by_lua*](#balancer_by_lua_block), and [ssl_session_store_by_lua*](#ssl_session_store_by_lua_block), `ngx.exit()` is an asynchronous operation and will return immediately. This behavior may change in future and it is recommended that users always use `return` in combination as suggested above. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index fec84cd00d..0598d5787a 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4395,7 +4395,7 @@ Note that while this method accepts all [[#HTTP status constants|HTTP status con Also note that this method call terminates the processing of the current request and that it is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.exit(...) be used to reinforce the fact that the request processing is being terminated. -When being used in the contexts of [[#header_filter_by_lua|header_filter_by_lua]] and +When being used in the contexts of [[#header_filter_by_lua|header_filter_by_lua*]], [[#balancer_by_lua_block|balancer_by_lua*]], and [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]], ngx.exit() is an asynchronous operation and will return immediately. This behavior may change in future and it is recommended that users always use return in combination as suggested above. From 8a6448a3906af265083848306bd49b1cc87a2c8d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 19 Mar 2017 19:20:59 -0700 Subject: [PATCH 060/848] travis-ci: upgraded pcre to 8.40. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index c935543265..eb0748c138 100644 --- a/.travis.yml +++ b/.travis.yml @@ -34,7 +34,7 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_VER=8.39 + - PCRE_VER=8.40 - PCRE_PREFIX=/opt/pcre - PCRE_LIB=$PCRE_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include From 312bf3ed18be3031a5ac1e423f5ca47d118737b6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 25 Mar 2017 14:56:28 -0700 Subject: [PATCH 061/848] tests: avoided flooding google dns servers in the check leak test mode. --- t/087-udp-socket.t | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index a58709b5a7..a847031aec 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -596,6 +596,7 @@ received a good response. --- log_level: debug --- error_log lua udp socket receive buffer size: 8192 +--- no_check_leak @@ -662,6 +663,7 @@ received a good response. --- log_level: debug --- error_log lua udp socket receive buffer size: 8192 +--- no_check_leak From 743a10ab0712dda5cdf6b4f1a7c6f6aed4da7b7b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 25 Mar 2017 14:57:22 -0700 Subject: [PATCH 062/848] travis-ci: avoided installing the Test::Nginx perl module; just use it right away. --- .travis.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index eb0748c138..7534c3c658 100644 --- a/.travis.yml +++ b/.travis.yml @@ -99,7 +99,6 @@ script: - make libdrizzle-1.0 -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make install-libdrizzle-1.0 > build.log 2>&1 || (cat build.log && exit 1) - cd ../mockeagain/ && make CC=$CC -j$JOBS && cd .. - - cd test-nginx/ && sudo cpanm . && cd .. - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. - tar zxf download-cache/pcre-$PCRE_VER.tar.gz - cd pcre-$PCRE_VER/ @@ -125,4 +124,4 @@ script: - export TEST_NGINX_RESOLVER=8.8.4.4 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 - dig +short @$TEST_NGINX_RESOLVER agentzh.org || exit 0 - - prove -r t + - prove -Itest-nginx/lib -r t From 6b01840dc97887cecbb15853bf1702a4dbe986fb Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 30 Mar 2017 10:32:02 -0700 Subject: [PATCH 063/848] change: disabled the mmap(sbrk(0)) memory trick since glibc leaks memory when it is forced to use mmap to fulfill malloc(). --- config | 27 ----------------------- src/ngx_http_lua_module.c | 45 --------------------------------------- t/141-luajit.t | 3 ++- t/146-malloc-trim.t | 41 +++++++++++++++++++---------------- 4 files changed, 25 insertions(+), 91 deletions(-) diff --git a/config b/config index 01a6b3c3cc..5d2ee659d9 100644 --- a/config +++ b/config @@ -472,33 +472,6 @@ ngx_feature_test='setsockopt(1, SOL_SOCKET, SO_PASSCRED, NULL, 0);' . auto/feature -ngx_feature="mmap(sbrk(0))" -ngx_feature_libs= -ngx_feature_name="NGX_HTTP_LUA_HAVE_MMAP_SBRK" -ngx_feature_run=yes -ngx_feature_incs="#include -#include -#include -#include -#define align_ptr(p, a) \ - (u_char *) (((uintptr_t) (p) + ((uintptr_t) a - 1)) & ~((uintptr_t) a - 1)) -" -ngx_feature_test=" -#if defined(__x86_64__) -exit(mmap(align_ptr(sbrk(0), getpagesize()), 1, PROT_READ, - MAP_FIXED|MAP_PRIVATE|MAP_ANON, -1, 0) < (void *) 0x40000000LL - ? 0 : 1); -#else -exit(1); -#endif -" -SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" -CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" - -. auto/feature - -CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" - ngx_feature="__attribute__(constructor)" ngx_feature_libs= ngx_feature_name="NGX_HTTP_LUA_HAVE_CONSTRUCTOR" diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 6e93c8eed7..3dc2817bb0 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -45,14 +45,6 @@ static char *ngx_http_lua_lowat_check(ngx_conf_t *cf, void *post, void *data); static ngx_int_t ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf); #endif -#if (NGX_HTTP_LUA_HAVE_MMAP_SBRK) && (NGX_LINUX) -/* we cannot use "static" for this function since it may lead to compiler - * warnings */ -void ngx_http_lua_limit_data_segment(void); -# if !(NGX_HTTP_LUA_HAVE_CONSTRUCTOR) -static ngx_int_t ngx_http_lua_pre_config(ngx_conf_t *cf); -# endif -#endif static char *ngx_http_lua_malloc_trim(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); @@ -592,13 +584,7 @@ static ngx_command_t ngx_http_lua_cmds[] = { ngx_http_module_t ngx_http_lua_module_ctx = { -#if (NGX_HTTP_LUA_HAVE_MMAP_SBRK) \ - && (NGX_LINUX) \ - && !(NGX_HTTP_LUA_HAVE_CONSTRUCTOR) - ngx_http_lua_pre_config, /* preconfiguration */ -#else NULL, /* preconfiguration */ -#endif ngx_http_lua_init, /* postconfiguration */ ngx_http_lua_create_main_conf, /* create main configuration */ @@ -1267,37 +1253,6 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) #endif /* NGX_HTTP_SSL */ -#if (NGX_HTTP_LUA_HAVE_MMAP_SBRK) \ - && (NGX_LINUX) \ - && !(NGX_HTTP_LUA_HAVE_CONSTRUCTOR) -static ngx_int_t -ngx_http_lua_pre_config(ngx_conf_t *cf) -{ - ngx_http_lua_limit_data_segment(); - return NGX_OK; -} -#endif - - -/* - * we simply assume that LuaJIT is used. it does little harm when the - * standard Lua 5.1 interpreter is used instead. - */ -#if (NGX_HTTP_LUA_HAVE_MMAP_SBRK) && (NGX_LINUX) -# if (NGX_HTTP_LUA_HAVE_CONSTRUCTOR) -__attribute__((constructor)) -# endif -void -ngx_http_lua_limit_data_segment(void) -{ - if (sbrk(0) < (void *) 0x40000000LL) { - mmap(ngx_align_ptr(sbrk(0), getpagesize()), 1, PROT_READ, - MAP_FIXED|MAP_PRIVATE|MAP_ANON, -1, 0); - } -} -#endif - - static char * ngx_http_lua_malloc_trim(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { diff --git a/t/141-luajit.t b/t/141-luajit.t index be03fe3927..36418d102c 100644 --- a/t/141-luajit.t +++ b/t/141-luajit.t @@ -1,6 +1,7 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +use Test::Nginx::Socket::Lua + skip_all => 'no mmap(sbrk(0)) trick since glibc leaks memory in this case'; #worker_connections(1014); #master_on(); diff --git a/t/146-malloc-trim.t b/t/146-malloc-trim.t index 45fb9f2463..fc425ce72a 100644 --- a/t/146-malloc-trim.t +++ b/t/146-malloc-trim.t @@ -43,8 +43,9 @@ ok ok ok --- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ ---- grep_error_log_out -malloc_trim(1) returned 0 +--- grep_error_log_out eval +qr/\Amalloc_trim\(1\) returned [01] +\z/ --- wait: 0.2 --- no_error_log [error] @@ -76,13 +77,14 @@ ok ok ok --- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ ---- grep_error_log_out -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 +--- grep_error_log_out eval +qr/\Amalloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +\z/ --- wait: 0.2 --- no_error_log [error] @@ -114,10 +116,11 @@ ok ok ok --- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ ---- grep_error_log_out -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 +--- grep_error_log_out eval +qr/\Amalloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +\z/ --- wait: 0.2 --- no_error_log [error] @@ -149,9 +152,10 @@ ok ok ok --- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ ---- grep_error_log_out -malloc_trim(1) returned 0 -malloc_trim(1) returned 0 +--- grep_error_log_out eval +qr/\Amalloc_trim\(1\) returned [01] +malloc_trim\(1\) returned [01] +\z/ --- wait: 0.2 --- no_error_log [error] @@ -330,8 +334,9 @@ ok ok ok --- grep_error_log eval: qr/malloc_trim\(\d+\) returned \d+/ ---- grep_error_log_out -malloc_trim(1) returned 0 +--- grep_error_log_out eval +qr/\Amalloc_trim\(1\) returned [01] +\z/ --- wait: 0.2 --- no_error_log [error] From 113d138096095a5afa7aba59acfca7843138fcd5 Mon Sep 17 00:00:00 2001 From: WenMing Date: Wed, 5 Apr 2017 10:59:47 +0800 Subject: [PATCH 064/848] bugfix: lacking the fix from #936 in the C API for lua-resty-core. #1031 Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_headers.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index a7cfc3c74e..7c6e935b2c 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -1298,6 +1298,7 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, ngx_uint_t i; ngx_table_elt_t *h; ngx_list_part_t *part; + ngx_http_lua_ctx_t *ctx; ngx_http_lua_loc_conf_t *llcf; @@ -1305,6 +1306,21 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, return NGX_HTTP_LUA_FFI_BAD_CONTEXT; } + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + /* *errmsg = "no ctx found"; */ + return NGX_ERROR; + } + + if (!ctx->headers_set) { + if (ngx_http_lua_set_content_type(r) != NGX_OK) { + /* *errmsg = "failed to set default content type"; */ + return NGX_ERROR; + } + + ctx->headers_set = 1; + } + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (llcf->transform_underscores_in_resp_headers && memchr(key, '_', key_len) != NULL) From 844e842cb3d03c3b9b796809bb86102b538fab9e Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Mon, 3 Apr 2017 21:02:50 -0700 Subject: [PATCH 065/848] change: removed the use of luaL_getn() macro as it is no longer available in the latest LuaJIT v2.1. fixes #1029. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_headers.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 7c6e935b2c..2392598495 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -754,7 +754,7 @@ ngx_http_lua_ngx_header_set(lua_State *L) ngx_str_null(&value); } else if (lua_type(L, 3) == LUA_TTABLE) { - n = luaL_getn(L, 3); + n = lua_objlen(L, 3); if (n == 0) { ngx_str_null(&value); @@ -888,7 +888,7 @@ ngx_http_lua_ngx_req_header_set_helper(lua_State *L) ngx_str_null(&value); } else if (lua_type(L, 2) == LUA_TTABLE) { - n = luaL_getn(L, 2); + n = lua_objlen(L, 2); if (n == 0) { ngx_str_null(&value); From 94248d64e41d9f3d0864f721c830c8af30568fa6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 6 Apr 2017 16:28:23 -0700 Subject: [PATCH 066/848] tests: added valgrind suppression rules for false positives in nginx 1.11.11+ and pcre 3.40+'s JIT compiler. --- valgrind.suppress | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/valgrind.suppress b/valgrind.suppress index fe161e8c77..d0bcc56d27 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -149,3 +149,18 @@ fun:main fun:__libc_res_nquerydomain fun:__libc_res_nsearch } +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_set_environment + fun:ngx_single_process_cycle + fun:main +} +{ + + Memcheck:Cond + obj:* +} From 2ba183d3c2bfe2a370242713ecb94e01244e1a6d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 7 Apr 2017 15:46:27 -0700 Subject: [PATCH 067/848] tests: made a test for duplex cosockets less sensitive to timing error. --- t/128-duplex-tcp-socket.t | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/t/128-duplex-tcp-socket.t b/t/128-duplex-tcp-socket.t index d64ae6a106..c0b8fd9c36 100644 --- a/t/128-duplex-tcp-socket.t +++ b/t/128-duplex-tcp-socket.t @@ -315,19 +315,24 @@ failed to send request: closed)$ local data = "" local ntm = 0 - local done = false + local aborted = false for i = 1, 3 do - local res, err, part = sock:receive(1) - if not res then - ngx.say("failed to receive: ", err) - return - else - data = data .. res + if not aborted then + local res, err, part = sock:receive(1) + if not res then + ngx.say("failed to receive: ", err) + aborted = true + else + data = data .. res + end end + ngx.sleep(0.001) end - ngx.say("received: ", data) + if not aborted then + ngx.say("received: ", data) + end '; } From 385ae4cd15ec57c043372f291b4493b631b552f5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 8 Apr 2017 00:24:52 -0700 Subject: [PATCH 068/848] tests: a followup fix for the previous commit (2ba183d3). --- t/128-duplex-tcp-socket.t | 1 + 1 file changed, 1 insertion(+) diff --git a/t/128-duplex-tcp-socket.t b/t/128-duplex-tcp-socket.t index c0b8fd9c36..e8434a3f84 100644 --- a/t/128-duplex-tcp-socket.t +++ b/t/128-duplex-tcp-socket.t @@ -355,6 +355,7 @@ F(ngx_http_lua_socket_tcp_finalize_write_part) { --- tcp_query_len: 11 --- no_error_log [error] +--- wait: 0.05 From 975c3f83889f125a2c669af39c55d447e25bcd99 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 8 Apr 2017 14:11:54 -0700 Subject: [PATCH 069/848] bumped version to 0.10.8. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index dd47e9cb0a..946f002c61 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.7](https://github.com/openresty/lua-nginx-module/tags) released on 4 November 2016. +This document describes ngx_lua [v0.10.8](https://github.com/openresty/lua-nginx-module/tags) released on 8 April 2017. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 0598d5787a..bbf2a2ecd7 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.7] released on 4 November 2016. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.8] released on 8 April 2017. = Synopsis = diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 3737149b2c..b0df5ae031 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10007 +#define ngx_http_lua_version 10008 typedef struct { From 4b724934231dd67aea07a93390aae72d10760601 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Thu, 2 Mar 2017 11:30:48 +0800 Subject: [PATCH 070/848] tests: added passing test cases for testing Lua errors thrown in init_by_lua*. Signed-off-by: Yichun Zhang (agentzh) --- t/086-init-by.t | 19 ++++- t/151-initby-hup.t | 168 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 t/151-initby-hup.t diff --git a/t/086-init-by.t b/t/086-init-by.t index 3a474fe82f..bea34a461a 100644 --- a/t/086-init-by.t +++ b/t/086-init-by.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 3); +plan tests => repeat_each() * (blocks() * 3 + 2); #no_diff(); #no_long_string(); @@ -304,3 +304,20 @@ INIT 2: foo = 3 ", "", ] + + + +=== TEST 12: error in init +--- http_config + init_by_lua_block { + error("failed to init") + } +--- config + location /t { + echo ok; + } +--- must_die +--- error_log +failed to init +--- error_log +[error] diff --git a/t/151-initby-hup.t b/t/151-initby-hup.t new file mode 100644 index 0000000000..f2788670db --- /dev/null +++ b/t/151-initby-hup.t @@ -0,0 +1,168 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_CHECK_LEAK}) { + $SkipReason = "unavailable for the hup tests"; + + } else { + $ENV{TEST_NGINX_USE_HUP} = 1; + undef $ENV{TEST_NGINX_USE_STAP}; + } +} + +use Test::Nginx::Socket::Lua 'no_plan'; + +#worker_connections(1014); +#master_on(); +#workers(2); +#log_level('warn'); + +repeat_each(1); + +#plan tests => repeat_each() * (blocks() * 3 + 3); + +#no_diff(); +#no_long_string(); +no_shuffle(); + +run_tests(); + +__DATA__ + +=== TEST 1: no error in init before HUP +--- http_config + init_by_lua_block { + foo = "hello, FOO" + } +--- config + location /lua { + content_by_lua_block { + ngx.say(foo) + } + } +--- request +GET /lua +--- response_body +hello, FOO +--- no_error_log +[error] + + + +=== TEST 2: error in init after HUP (master still alive, worker process still the same as before) +--- http_config + init_by_lua_block { + error("failed to init") + } +--- config + location /lua { + content_by_lua_block { + ngx.say(foo) + } + } +--- request +GET /lua +--- response_body +hello, FOO +--- error_log +failed to init +--- reload_fails + + + +=== TEST 3: no error in init again +--- http_config + init_by_lua_block { + foo = "hello, foo" + } +--- config + location /lua { + content_by_lua_block { + ngx.say(foo) + } + } +--- request +GET /lua +--- response_body +hello, foo +--- no_error_log +[error] + + + +=== TEST 4: no error in init before HUP, used ngx.shared.DICT +--- http_config + lua_shared_dict dogs 1m; + + init_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("foo", "hello, FOO") + } +--- config + location /lua { + content_by_lua_block { + local dogs = ngx.shared.dogs + local foo = dogs:get("foo") + ngx.say(foo) + } + } +--- request +GET /lua +--- response_body +hello, FOO +--- no_error_log +[error] + + + +=== TEST 5: error in init after HUP, not reloaded but foo have changed. +--- http_config + lua_shared_dict dogs 1m; + + init_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("foo", "foo have changed") + + error("failed to init") + } +--- config + location /lua { + content_by_lua_block { + ngx.say("HUP reload failed") + } + } +--- request +GET /lua +--- response_body +foo have changed +--- error_log +failed to init +--- reload_fails + + + +=== TEST 6: no error in init again, reload success and foo still have changed. +--- http_config + lua_shared_dict dogs 1m; + + init_by_lua_block { + -- do nothing + } +--- config + location /lua { + content_by_lua_block { + local dogs = ngx.shared.dogs + local foo = dogs:get("foo") + ngx.say(foo) + ngx.say("reload success") + } + } +--- request +GET /lua +--- response_body +foo have changed +reload success +--- no_error_log +[error] From 0459a285ca0159d45e73da8bd1164edb5c57cde3 Mon Sep 17 00:00:00 2001 From: Andrei Belov Date: Wed, 22 Mar 2017 07:50:57 +0300 Subject: [PATCH 071/848] feature: nginx 1.11.11+ can now build with this module. Note: nginx 1.11.11+ are still not an officially supported target yet. More work needed. Closes openresty/lua-nginx-module#1016 See also: http://hg.nginx.org/nginx/rev/e662cbf1b932 --- src/ngx_http_lua_common.h | 6 ++++ src/ngx_http_lua_headers.c | 68 ++++++++++++++++++++++++++++++++++++++ src/ngx_http_lua_headers.h | 3 ++ src/ngx_http_lua_module.c | 13 +++++++- 4 files changed, 89 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 079a4dc3b8..f37d776a9e 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -199,6 +199,12 @@ struct ngx_http_lua_main_conf_s { of reqeusts */ ngx_uint_t malloc_trim_req_count; +#if nginx_version >= 1011011 + /* the following 2 fields are only used by ngx.req.raw_headers() for now */ + ngx_buf_t **busy_buf_ptrs; + ngx_int_t busy_buf_ptr_count; +#endif + unsigned requires_header_filter:1; unsigned requires_body_filter:1; unsigned requires_capture_filter:1; diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 2392598495..6700ce80db 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -26,6 +26,9 @@ static int ngx_http_lua_ngx_req_get_headers(lua_State *L); static int ngx_http_lua_ngx_req_header_clear(lua_State *L); static int ngx_http_lua_ngx_req_header_set(lua_State *L); static int ngx_http_lua_ngx_resp_get_headers(lua_State *L); +#if nginx_version >= 1011011 +void ngx_http_lua_ngx_raw_header_cleanup(void *data); +#endif static int @@ -77,6 +80,11 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) size_t size; ngx_buf_t *b, *first = NULL; ngx_int_t i, j; +#if nginx_version >= 1011011 + ngx_buf_t **bb; + ngx_chain_t *cl; + ngx_http_lua_main_conf_t *lmcf; +#endif ngx_connection_t *c; ngx_http_request_t *r, *mr; ngx_http_connection_t *hc; @@ -93,6 +101,10 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) return luaL_error(L, "no request object found"); } +#if nginx_version >= 1011011 + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); +#endif + ngx_http_lua_check_fake_request(L, r); mr = r->main; @@ -109,8 +121,13 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) dd("hc->nbusy: %d", (int) hc->nbusy); if (hc->nbusy) { +#if nginx_version >= 1011011 + dd("hc->busy: %p %p %p %p", hc->busy->buf->start, hc->busy->buf->pos, + hc->busy->buf->last, hc->busy->buf->end); +#else dd("hc->busy: %p %p %p %p", hc->busy[0]->start, hc->busy[0]->pos, hc->busy[0]->last, hc->busy[0]->end); +#endif } dd("request line: %p %p", mr->request_line.data, @@ -146,9 +163,37 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) dd("size: %d", (int) size); if (hc->nbusy) { +#if nginx_version >= 1011011 + if (hc->nbusy > lmcf->busy_buf_ptr_count) { + if (lmcf->busy_buf_ptrs) { + ngx_free(lmcf->busy_buf_ptrs); + } + + lmcf->busy_buf_ptrs = ngx_alloc(hc->nbusy * sizeof(ngx_buf_t *), + r->connection->log); + + if (lmcf->busy_buf_ptrs == NULL) { + return luaL_error(L, "no memory"); + } + + lmcf->busy_buf_ptr_count = hc->nbusy; + } + + bb = lmcf->busy_buf_ptrs; + for (cl = hc->busy; cl; cl = cl->next) { + *bb++ = cl->buf; + } +#endif b = NULL; + +#if nginx_version >= 1011011 + bb = lmcf->busy_buf_ptrs; + for (i = hc->nbusy; i > 0; i--) { + b = bb[i - 1]; +#else for (i = 0; i < hc->nbusy; i++) { b = hc->busy[i]; +#endif dd("busy buf: %d: [%.*s]", (int) i, (int) (b->pos - b->start), b->start); @@ -223,8 +268,15 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) } if (hc->nbusy) { + +#if nginx_version >= 1011011 + bb = lmcf->busy_buf_ptrs; + for (i = hc->nbusy - 1; i >= 0; i--) { + b = bb[i]; +#else for (i = 0; i < hc->nbusy; i++) { b = hc->busy[i]; +#endif if (!found) { if (b != first) { @@ -1431,4 +1483,20 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, #endif /* NGX_LUA_NO_FFI_API */ +#if nginx_version >= 1011011 +void +ngx_http_lua_ngx_raw_header_cleanup(void *data) +{ + ngx_http_lua_main_conf_t *lmcf; + + lmcf = (ngx_http_lua_main_conf_t *) data; + + if (lmcf->busy_buf_ptrs) { + ngx_free(lmcf->busy_buf_ptrs); + lmcf->busy_buf_ptrs = NULL; + } +} +#endif + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_headers.h b/src/ngx_http_lua_headers.h index 39f1114cdf..ee4d21c1ae 100644 --- a/src/ngx_http_lua_headers.h +++ b/src/ngx_http_lua_headers.h @@ -15,6 +15,9 @@ void ngx_http_lua_inject_resp_header_api(lua_State *L); void ngx_http_lua_inject_req_header_api(lua_State *L); void ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L); +#if nginx_version >= 1011011 +void ngx_http_lua_ngx_raw_header_cleanup(void *data); +#endif #endif /* _NGX_HTTP_LUA_HEADERS_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 3dc2817bb0..875f9334f7 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -28,6 +28,7 @@ #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_ssl_session_storeby.h" #include "ngx_http_lua_ssl_session_fetchby.h" +#include "ngx_http_lua_headers.h" static void *ngx_http_lua_create_main_conf(ngx_conf_t *cf); @@ -624,7 +625,7 @@ ngx_http_lua_init(ngx_conf_t *cf) volatile ngx_cycle_t *saved_cycle; ngx_http_core_main_conf_t *cmcf; ngx_http_lua_main_conf_t *lmcf; -#ifndef NGX_LUA_NO_FFI_API +#if !defined(NGX_LUA_NO_FFI_API) || nginx_version >= 1011011 ngx_pool_cleanup_t *cln; #endif @@ -716,6 +717,16 @@ ngx_http_lua_init(ngx_conf_t *cf) cln->handler = ngx_http_lua_sema_mm_cleanup; #endif +#if nginx_version >= 1011011 + cln = ngx_pool_cleanup_add(cf->pool, 0); + if (cln == NULL) { + return NGX_ERROR; + } + + cln->data = lmcf; + cln->handler = ngx_http_lua_ngx_raw_header_cleanup; +#endif + if (lmcf->lua == NULL) { dd("initializing lua vm"); From e39e4429113737a512ba271314cd10edc20ff8b3 Mon Sep 17 00:00:00 2001 From: spacewander Date: Sun, 16 Apr 2017 00:15:11 +0800 Subject: [PATCH 072/848] bugfix: fixed typos in error messages. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_module.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 875f9334f7..72f934d015 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -952,7 +952,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) #ifdef LIBRESSL_VERSION_NUMBER ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "LibreSSL does not support ssl_ceritificate_by_lua*"); + "LibreSSL does not support ssl_certificate_by_lua*"); return NGX_CONF_ERROR; #else @@ -964,7 +964,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) # else ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "OpenSSL too old to support ssl_ceritificate_by_lua*"); + "OpenSSL too old to support ssl_certificate_by_lua*"); return NGX_CONF_ERROR; # endif From 55f91d782b3bbee646a55c41adbb99b7101afa68 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 19 Apr 2017 20:34:51 -0700 Subject: [PATCH 073/848] doc: updated the Installation section to reflect recent changes. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 946f002c61..4e5aca07e8 100644 --- a/README.markdown +++ b/README.markdown @@ -263,7 +263,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is highly recommended to use the [OpenResty bundle](http://openresty.org) that bundles Nginx, ngx_lua, LuaJIT 2.0/2.1 (or the optional standard Lua 5.1 interpreter), as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: `./configure --with-luajit && make && make install`. +It is *highly* recommended to use the [OpenResty bundle](http://openresty.org) that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: `./configure --with-luajit && make && make install`. Do not build this module with nginx yourself since it is tricky to set up exactly right. Alternatively, ngx_lua can be manually compiled into Nginx: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index bbf2a2ecd7..760087a100 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -197,7 +197,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is highly recommended to use the [http://openresty.org OpenResty bundle] that bundles Nginx, ngx_lua, LuaJIT 2.0/2.1 (or the optional standard Lua 5.1 interpreter), as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: ./configure --with-luajit && make && make install. +It is *highly* recommended to use the [http://openresty.org OpenResty bundle] that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: ./configure --with-luajit && make && make install. Do not build this module with nginx yourself since it is tricky to set up exactly right. Alternatively, ngx_lua can be manually compiled into Nginx: From 398db9517c2bd5d577315a3d105fb6c5033434d9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 19 Apr 2017 20:35:36 -0700 Subject: [PATCH 074/848] doc: more fixes in the Installation section for OpenResty. --- doc/HttpLuaModule.wiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 760087a100..9e20a1ce9a 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -197,7 +197,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use the [http://openresty.org OpenResty bundle] that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: ./configure --with-luajit && make && make install. Do not build this module with nginx yourself since it is tricky to set up exactly right. +It is *highly* recommended to use the [http://openresty.org OpenResty bundle] that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: ./configure -j2 && make -j2 && make install. Do not build this module with nginx yourself since it is tricky to set up exactly right. Alternatively, ngx_lua can be manually compiled into Nginx: From c7d9691e5bb1e17527a071e02fa3100d517b0f95 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 19 Apr 2017 20:37:38 -0700 Subject: [PATCH 075/848] doc: more tweaks in the Installation section. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 4e5aca07e8..a9e44600ac 100644 --- a/README.markdown +++ b/README.markdown @@ -263,7 +263,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use the [OpenResty bundle](http://openresty.org) that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: `./configure --with-luajit && make && make install`. Do not build this module with nginx yourself since it is tricky to set up exactly right. +It is *highly* recommended to use the [OpenResty bundle](http://openresty.org) that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. Do not build this module with nginx yourself since it is tricky to set up exactly right. Alternatively, ngx_lua can be manually compiled into Nginx: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 9e20a1ce9a..c9a8f64c15 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -197,7 +197,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use the [http://openresty.org OpenResty bundle] that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. The basic installation step is a simple command: ./configure -j2 && make -j2 && make install. Do not build this module with nginx yourself since it is tricky to set up exactly right. +It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Alternatively, ngx_lua can be manually compiled into Nginx: From 6c7ac361f3a41b811a001fae35b8e30a3e47be81 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 19 Apr 2017 20:39:28 -0700 Subject: [PATCH 076/848] doc: more tweaks in the Installation section. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index a9e44600ac..36140c1626 100644 --- a/README.markdown +++ b/README.markdown @@ -263,7 +263,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use the [OpenResty bundle](http://openresty.org) that bundles Nginx, ngx_lua, LuaJIT 2.1, as well as a package of powerful companion Nginx modules. Do not build this module with nginx yourself since it is tricky to set up exactly right. +It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or slower. Alternatively, ngx_lua can be manually compiled into Nginx: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index c9a8f64c15..a108644ef1 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -197,7 +197,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. +It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or slower. Alternatively, ngx_lua can be manually compiled into Nginx: From 39bec7f40a928ed02cc2acef8eee63210d20fba5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 19 Apr 2017 20:41:42 -0700 Subject: [PATCH 077/848] doc: minor tweaks. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 36140c1626..e2ee0f2a12 100644 --- a/README.markdown +++ b/README.markdown @@ -263,7 +263,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or slower. +It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or run slower. Alternatively, ngx_lua can be manually compiled into Nginx: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a108644ef1..31b8db2be4 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -197,7 +197,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or slower. +It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or run slower. Alternatively, ngx_lua can be manually compiled into Nginx: From ca8ed0e8cd746c41450b14abff5e40d8f713ccc9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 19 Apr 2017 20:43:04 -0700 Subject: [PATCH 078/848] doc: minor tweaks. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index e2ee0f2a12..16d56bfe4e 100644 --- a/README.markdown +++ b/README.markdown @@ -263,7 +263,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or run slower. +It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. Alternatively, ngx_lua can be manually compiled into Nginx: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 31b8db2be4..e496672bbb 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -197,7 +197,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features disabled, not work properly, or run slower. +It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. Alternatively, ngx_lua can be manually compiled into Nginx: From c03462637bc1317da225441dc80dfe2203758622 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 28 Apr 2017 13:51:21 -0700 Subject: [PATCH 079/848] change: removed util/reindex. put it into the openresty/openresty-devel-utils repo instead. --- util/reindex | 65 ---------------------------------------------------- 1 file changed, 65 deletions(-) delete mode 100755 util/reindex diff --git a/util/reindex b/util/reindex deleted file mode 100755 index d0a0927603..0000000000 --- a/util/reindex +++ /dev/null @@ -1,65 +0,0 @@ -#!/usr/bin/env perl - -#: reindexes .t files for Test::Base based test files -#: Copyright (c) Yichun Zhang - -use strict; -use warnings; - -#use File::Copy; -use Getopt::Std; - -my %opts; -getopts('hb:', \%opts); -if ($opts{h} or ! @ARGV) { - die "Usage: reindex [-b 0] t/*.t\n"; -} - -my $init = $opts{b}; -$init = 1 if not defined $init; - -my @files = map glob, @ARGV; -for my $file (@files) { - next if -d $file or $file !~ /\.t_?$/; - reindex($file); -} - -sub reindex { - my $file = $_[0]; - open my $in, $file or - die "Can't open $file for reading: $!"; - my @lines; - my $counter = $init; - my $changed; - while (<$in>) { - s/\r$//; - my $num; - s/ ^ === \s+ TEST \s+ (\d+)/$num=$1; "=== TEST " . $counter++/xie; - next if !defined $num; - if ($num != $counter-1) { - $changed++; - } - } continue { - push @lines, $_; - } - close $in; - - my $text = join '', @lines; - $text =~ s/\n\n+\z/\n/sm; - $text =~ s/(?x) \n+ === \s+ TEST/\n\n\n\n=== TEST/ixsg; - $text =~ s/__(DATA|END)__\n+=== TEST/__${1}__\n\n=== TEST/; - #$text =~ s/\n+$/\n\n/s; - if (! $changed and $text eq join '', @lines) { - warn "reindex: $file:\tskipped.\n"; - return; - } - #File::Copy::copy( $file, "$file.bak" ); - open my $out, "> $file" or - die "Can't open $file for writing: $!"; - binmode $out; - print $out $text; - close $out; - - warn "reindex: $file:\tdone.\n"; -} - From f5633cacc0a7fdd781f18a9864f972af95ba5e8f Mon Sep 17 00:00:00 2001 From: Andreas Lubbe Date: Tue, 31 May 2016 10:23:03 +0200 Subject: [PATCH 080/848] feature: added pure C API for tuning the jit_stack_size option in PCRE. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_common.h | 18 +++++++++ src/ngx_http_lua_module.c | 1 + src/ngx_http_lua_regex.c | 82 +++++++++++++++++++++++++++++++++------ src/ngx_http_lua_regex.h | 2 + 4 files changed, 92 insertions(+), 11 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index f37d776a9e..3b0a417963 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -22,6 +22,19 @@ #include +#if (NGX_PCRE) + +#include + +#if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) +# define LUA_HAVE_PCRE_JIT 1 +#else +# define LUA_HAVE_PCRE_JIT 0 +#endif + +#endif + + #if !defined(nginx_version) || (nginx_version < 1006000) #error at least nginx 1.6.0 is required but found an older version #endif @@ -168,6 +181,11 @@ struct ngx_http_lua_main_conf_s { ngx_int_t regex_cache_entries; ngx_int_t regex_cache_max_entries; ngx_int_t regex_match_limit; + +#if (LUA_HAVE_PCRE_JIT) + pcre_jit_stack *jit_stack; +#endif + #endif ngx_array_t *shm_zones; /* of ngx_shm_zone_t* */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 72f934d015..7b17ac41ce 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -814,6 +814,7 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) * lmcf->running_timers = 0; * lmcf->watcher = NULL; * lmcf->regex_cache_entries = 0; + * lmcf->jit_stack = NULL; * lmcf->shm_zones = NULL; * lmcf->init_handler = NULL; * lmcf->init_src = { 0, NULL }; diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index d882061033..80519ecd15 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -17,14 +17,6 @@ #include "ngx_http_lua_script.h" #include "ngx_http_lua_pcrefix.h" #include "ngx_http_lua_util.h" -#include - - -#if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) -# define LUA_HAVE_PCRE_JIT 1 -#else -# define LUA_HAVE_PCRE_JIT 0 -#endif #if (PCRE_MAJOR >= 6) @@ -42,6 +34,8 @@ #define NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT (100) +#define NGX_LUA_RE_MIN_JIT_STACK_SIZE 32 * 1024 + typedef struct { #ifndef NGX_LUA_NO_FFI_API @@ -364,6 +358,10 @@ ngx_http_lua_ngx_re_match_helper(lua_State *L, int wantcaps) sd = pcre_study(re_comp.regex, PCRE_STUDY_JIT_COMPILE, &msg); + if (sd && lmcf->jit_stack) { + pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); + } + ngx_http_lua_pcre_malloc_done(old_pool); # if (NGX_DEBUG) @@ -826,6 +824,10 @@ ngx_http_lua_ngx_re_gmatch(lua_State *L) sd = pcre_study(re_comp.regex, PCRE_STUDY_JIT_COMPILE, &msg); + if (sd && lmcf->jit_stack) { + pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); + } + ngx_http_lua_pcre_malloc_done(old_pool); # if (NGX_DEBUG) @@ -1922,6 +1924,60 @@ ngx_http_lua_ngx_re_sub_helper(lua_State *L, unsigned global) } +ngx_int_t +ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, + size_t *errstr_size) +{ +#if LUA_HAVE_PCRE_JIT + + ngx_http_lua_main_conf_t *lmcf; + ngx_pool_t *pool, *old_pool; + + lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, + ngx_http_lua_module); + + if (size < NGX_LUA_RE_MIN_JIT_STACK_SIZE) { + size = NGX_LUA_RE_MIN_JIT_STACK_SIZE; + } + + pool = lmcf->pool; + + dd("server pool %p", lmcf->pool); + + if (lmcf->jit_stack) { + old_pool = ngx_http_lua_pcre_malloc_init(pool); + + pcre_jit_stack_free(lmcf->jit_stack); + + ngx_http_lua_pcre_malloc_done(old_pool); + } + + old_pool = ngx_http_lua_pcre_malloc_init(pool); + + lmcf->jit_stack = pcre_jit_stack_alloc(NGX_LUA_RE_MIN_JIT_STACK_SIZE, + size); + + ngx_http_lua_pcre_malloc_done(old_pool); + + if (lmcf->jit_stack == NULL) { + *errstr_size = ngx_snprintf(errstr, *errstr_size, + "pcre jit stack allocation failed") + - errstr; + return NGX_ERROR; + } + + return NGX_OK; + +#else /* LUA_HAVE_PCRE_JIT */ + + *errstr_size = ngx_snprintf(errstr, *errstr_size, + "no pcre jit support found") - errstr; + return NGX_ERROR; + +#endif /* LUA_HAVE_PCRE_JIT */ +} + + void ngx_http_lua_inject_regex_api(lua_State *L) { @@ -2170,6 +2226,9 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, goto error; } + lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, + ngx_http_lua_module); + #if (LUA_HAVE_PCRE_JIT) if (flags & NGX_LUA_RE_MODE_JIT) { @@ -2205,10 +2264,11 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, ngx_http_lua_pcre_malloc_done(old_pool); } -#endif /* LUA_HAVE_PCRE_JIT */ + if (sd && lmcf->jit_stack) { + pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); + } - lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, - ngx_http_lua_module); +#endif /* LUA_HAVE_PCRE_JIT */ if (sd && lmcf && lmcf->regex_match_limit > 0) { sd->flags |= PCRE_EXTRA_MATCH_LIMIT; diff --git a/src/ngx_http_lua_regex.h b/src/ngx_http_lua_regex.h index f5f8e2f875..03dffb80b9 100644 --- a/src/ngx_http_lua_regex.h +++ b/src/ngx_http_lua_regex.h @@ -14,6 +14,8 @@ #if (NGX_PCRE) void ngx_http_lua_inject_regex_api(lua_State *L); +ngx_int_t ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, + size_t *errstr_size); #endif From b98da9a1914444e86f88ba6f7f2719ba28b16215 Mon Sep 17 00:00:00 2001 From: Yuansheng Date: Sun, 16 Apr 2017 21:41:56 +0800 Subject: [PATCH 081/848] feature: added pure C functions ngx_http_lua_ffi_worker_type() & ngx_http_lua_ffi_worker_privileged(). Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_worker.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index ff09b5b2e4..4af311bc2e 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -13,6 +13,9 @@ #include "ngx_http_lua_worker.h" +#define NGX_PROCESS_PRIVILEGED_AGENT 99 + + static int ngx_http_lua_ngx_worker_exiting(lua_State *L); static int ngx_http_lua_ngx_worker_pid(lua_State *L); static int ngx_http_lua_ngx_worker_id(lua_State *L); @@ -130,4 +133,39 @@ ngx_http_lua_ffi_worker_count(void) return (int) ccf->worker_processes; } + + +int +ngx_http_lua_ffi_get_process_type(void) +{ +#ifdef HAVE_PRIVILEGED_PROCESS_PATCH + if (ngx_process == NGX_PROCESS_HELPER) { + if (ngx_is_privileged_agent) { + return NGX_PROCESS_PRIVILEGED_AGENT; + } + } +#endif + + return ngx_process; +} + + +int +ngx_http_lua_ffi_enable_privileged_agent(char **err) +{ +#ifdef HAVE_PRIVILEGED_PROCESS_PATCH + ngx_core_conf_t *ccf; + + ccf = (ngx_core_conf_t *) ngx_get_conf(ngx_cycle->conf_ctx, + ngx_core_module); + + ccf->privileged_agent = 1; + + return NGX_OK; + +#else + *err = "missing privileged agent process patch in the nginx core"; + return NGX_ERROR; +#endif +} #endif From 23cb8e83296f4e586912d32fc9735c72575108d0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 5 May 2017 23:07:54 -0700 Subject: [PATCH 082/848] travis-ci: enable -msse4.2 when building luajit2. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 7534c3c658..a0dcec40cb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -91,7 +91,7 @@ before_script: script: - cd luajit2/ - - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT' > build.log 2>&1 || (cat build.log && exit 1) + - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1) - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1) - cd .. - tar xzf download-cache/drizzle7-$DRIZZLE_VER.tar.gz && cd drizzle7-$DRIZZLE_VER From edbded4056dae51e2b1e23ebd76e59e22fee2705 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 6 May 2017 18:23:57 -0700 Subject: [PATCH 083/848] change: bumped API version to 0.10.9. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index b0df5ae031..5083799a04 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10008 +#define ngx_http_lua_version 10009 typedef struct { From 6a2b4ff5565d21bf9fd759c8886d470320e35fc9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 7 May 2017 10:49:46 -0700 Subject: [PATCH 084/848] doc: a typo fix from Oleg Abrosimov. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 16d56bfe4e..99ac286c35 100644 --- a/README.markdown +++ b/README.markdown @@ -5607,7 +5607,7 @@ ngx.time Returns the elapsed seconds from the epoch for the current time stamp from the nginx cached time (no syscall involved unlike Lua's date library). -Updates of the Nginx time cache an be forced by calling [ngx.update_time](#ngxupdate_time) first. +Updates of the Nginx time cache can be forced by calling [ngx.update_time](#ngxupdate_time) first. [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index e496672bbb..f075fa95c5 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4674,7 +4674,7 @@ This is the local time. Returns the elapsed seconds from the epoch for the current time stamp from the nginx cached time (no syscall involved unlike Lua's date library). -Updates of the Nginx time cache an be forced by calling [[#ngx.update_time|ngx.update_time]] first. +Updates of the Nginx time cache can be forced by calling [[#ngx.update_time|ngx.update_time]] first. == ngx.now == '''syntax:''' ''secs = ngx.now()'' From 1a5a33e61f4d747efa8460555407357ca166eca1 Mon Sep 17 00:00:00 2001 From: Yuansheng Date: Thu, 6 Apr 2017 18:31:56 +0800 Subject: [PATCH 085/848] feature: added new config directive lua_intercept_error_log for capturing nginx error logs on Lua land. The corresponding Lua API is provided by the ngx.errlog module in lua-resty-core. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 4 +- config | 2 + src/ngx_http_lua_common.h | 1 + src/ngx_http_lua_directive.c | 67 +++++++++++ src/ngx_http_lua_directive.h | 3 +- src/ngx_http_lua_log.c | 96 ++++++++++++++++ src/ngx_http_lua_log.h | 4 + src/ngx_http_lua_log_ringbuf.c | 201 +++++++++++++++++++++++++++++++++ src/ngx_http_lua_log_ringbuf.h | 31 +++++ src/ngx_http_lua_module.c | 7 ++ src/ngx_http_lua_util.c | 1 + 11 files changed, 414 insertions(+), 3 deletions(-) create mode 100644 src/ngx_http_lua_log_ringbuf.c create mode 100644 src/ngx_http_lua_log_ringbuf.h diff --git a/.travis.yml b/.travis.yml index a0dcec40cb..359c8cc654 100644 --- a/.travis.yml +++ b/.travis.yml @@ -68,7 +68,7 @@ install: - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - - git clone https://github.com/openresty/nginx-devel-utils.git + - git clone https://github.com/openresty/openresty-devel-utils.git - git clone https://github.com/openresty/mockeagain.git - git clone https://github.com/openresty/lua-cjson.git - git clone https://github.com/openresty/lua-upstream-nginx-module.git ../lua-upstream-nginx-module @@ -114,7 +114,7 @@ script: - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. - - export PATH=$PWD/work/nginx/sbin:$PWD/nginx-devel-utils:$PATH + - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH - export NGX_BUILD_CC=$CC - sh util/build.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - nginx -V diff --git a/config b/config index 5d2ee659d9..044deb974e 100644 --- a/config +++ b/config @@ -360,6 +360,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_storeby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl.c \ + $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.c \ " HTTP_LUA_DEPS=" \ @@ -420,6 +421,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_storeby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl.h \ + $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.h \ " CFLAGS="$CFLAGS -DNDK_SET_VAR" diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 3b0a417963..a923aedf02 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -230,6 +230,7 @@ struct ngx_http_lua_main_conf_s { unsigned requires_access:1; unsigned requires_log:1; unsigned requires_shm:1; + unsigned requires_intercept_log:1; }; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 862eddaa8a..a92f056d71 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -27,6 +27,8 @@ #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_lex.h" #include "api/ngx_http_lua_api.h" +#include "ngx_http_lua_log_ringbuf.h" +#include "ngx_http_lua_log.h" typedef struct ngx_http_lua_block_parser_ctx_s @@ -1700,6 +1702,71 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, } +char * +ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ +#ifndef HAVE_INTERCEPT_ERROR_LOG_PATCH + return "not found: missing the intercept error log patch for nginx"; +#else + ngx_str_t *value; + ssize_t size; + u_char *data; + ngx_cycle_t *cycle; + ngx_http_lua_main_conf_t *lmcf = conf; + ngx_http_lua_log_ringbuf_t *ringbuf; + + value = cf->args->elts; + cycle = cf->cycle; + + if (lmcf->requires_intercept_log) { + return "is duplicate"; + } + + if (value[1].len == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid intercept error log size \"%V\"", + &value[1]); + return NGX_CONF_ERROR; + } + + size = ngx_parse_size(&value[1]); + + if (size < NGX_MAX_ERROR_STR) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid intercept error log size \"%V\", " + "minimum size is %d", &value[1], + NGX_MAX_ERROR_STR); + return NGX_CONF_ERROR; + } + + if (cycle->intercept_error_log_handler) { + return "intercept error log handler has been hooked"; + } + + ringbuf = (ngx_http_lua_log_ringbuf_t *) + ngx_palloc(cf->pool, sizeof(ngx_http_lua_log_ringbuf_t)); + if (ringbuf == NULL) { + return NGX_CONF_ERROR; + } + + data = ngx_palloc(cf->pool, size); + if (data == NULL) { + return NGX_CONF_ERROR; + } + + ngx_http_lua_log_ringbuf_init(ringbuf, data, size); + + lmcf->requires_intercept_log = 1; + cycle->intercept_error_log_handler = (ngx_log_intercept_pt) + ngx_http_lua_intercept_log_handler; + cycle->intercept_error_log_data = ringbuf; + + return NGX_CONF_OK; +#endif +} + + /* * ngx_http_lua_strlstrn() is intended to search for static substring * with known length in string until the argument last. The argument n diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index be591f34ee..aabb118994 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -67,9 +67,10 @@ ngx_int_t ngx_http_lua_filter_set_by_lua_file(ngx_http_request_t *r, char *ngx_http_lua_rewrite_no_postpone(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); - char *ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd); +char *ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); #endif /* _NGX_HTTP_LUA_DIRECTIVE_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index c2b226914b..9e87a590a7 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -13,6 +13,7 @@ #include "ngx_http_lua_log.h" #include "ngx_http_lua_util.h" +#include "ngx_http_lua_log_ringbuf.h" static int ngx_http_lua_print(lua_State *L); @@ -313,4 +314,99 @@ ngx_http_lua_inject_log_consts(lua_State *L) /* }}} */ } + +#ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH +ngx_int_t +ngx_http_lua_intercept_log_handler(ngx_log_t *log, + ngx_uint_t level, void *buf, size_t n) +{ + ngx_http_lua_log_ringbuf_t *ringbuf; + + dd("enter"); + + ringbuf = (ngx_http_lua_log_ringbuf_t *) + ngx_cycle->intercept_error_log_data; + + if (level > ringbuf->filter_level) { + return NGX_OK; + } + + ngx_http_lua_log_ringbuf_write(ringbuf, level, buf, n); + + dd("intercept log: %s\n", buf); + + return NGX_OK; +} +#endif + + +#ifndef NGX_LUA_NO_FFI_API +int +ngx_http_lua_ffi_set_errlog_filter(int level, u_char *err, size_t *errlen) +{ +#ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH + ngx_http_lua_log_ringbuf_t *ringbuf; + + ringbuf = ngx_cycle->intercept_error_log_data; + + if (ringbuf == NULL) { + *errlen = ngx_snprintf(err, *errlen, + "API \"set_errlog_filter\" depends on " + "directive \"lua_intercept_error_log\"") + - err; + return NGX_ERROR; + } + + if (level > NGX_LOG_DEBUG || level < NGX_LOG_STDERR) { + *errlen = ngx_snprintf(err, *errlen, "bad log level: %d", level) + - err; + return NGX_ERROR; + } + + ringbuf->filter_level = level; + return NGX_OK; +#else + *errlen = ngx_snprintf(err, *errlen, + "missing the intercept error log patch for nginx") + - err; + return NGX_ERROR; +#endif +} + + +int +ngx_http_lua_ffi_get_errlog_data(char **log, int *loglevel, u_char *err, + size_t *errlen) +{ +#ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH + ngx_uint_t loglen; + + ngx_http_lua_log_ringbuf_t *ringbuf; + + ringbuf = ngx_cycle->intercept_error_log_data; + + if (ringbuf == NULL) { + *errlen = ngx_snprintf(err, *errlen, + "API \"get_errlog_data\" depends on directive " + "\"lua_intercept_error_log\"") + - err; + return NGX_ERROR; + } + + if (ringbuf->count == 0) { + return NGX_DONE; + } + + ngx_http_lua_log_ringbuf_read(ringbuf, loglevel, (void **)log, &loglen); + return loglen; +#else + *errlen = ngx_snprintf(err, *errlen, + "missing the intercept error log patch for nginx") + - err; + return NGX_ERROR; +#endif +} + +#endif + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_log.h b/src/ngx_http_lua_log.h index 42f1839473..115679af3f 100644 --- a/src/ngx_http_lua_log.h +++ b/src/ngx_http_lua_log.h @@ -13,6 +13,10 @@ void ngx_http_lua_inject_log_api(lua_State *L); +#ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH +ngx_int_t ngx_http_lua_intercept_log_handler(ngx_log_t *log, + ngx_uint_t level, void *buf, size_t n); +#endif #endif /* _NGX_HTTP_LUA_LOG_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_log_ringbuf.c b/src/ngx_http_lua_log_ringbuf.c new file mode 100644 index 0000000000..dca610df56 --- /dev/null +++ b/src/ngx_http_lua_log_ringbuf.c @@ -0,0 +1,201 @@ + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#include "ngx_http_lua_common.h" +#include "ngx_http_lua_log_ringbuf.h" + + +typedef struct { + unsigned len:28; /* :24 is big enough if the max log size is 4k */ + unsigned log_level:4; +} ngx_http_lua_log_ringbuf_header_t; + + +enum { + HEADER_LEN = sizeof(ngx_http_lua_log_ringbuf_header_t) +}; + + +static void *ngx_http_lua_log_ringbuf_next_header( + ngx_http_lua_log_ringbuf_t *rb); +static void ngx_http_lua_log_ringbuf_append( + ngx_http_lua_log_ringbuf_t *rb, int sentinel, int log_level, void *buf, + int n); +static size_t ngx_http_lua_log_ringbuf_free_spaces( + ngx_http_lua_log_ringbuf_t *rb); + + +void +ngx_http_lua_log_ringbuf_init(ngx_http_lua_log_ringbuf_t *rb, void *buf, + size_t len) +{ + rb->data = buf; + rb->size = len; + + rb->tail = rb->data; + rb->head = rb->data; + rb->sentinel = rb->data + rb->size; + rb->count = 0; + rb->filter_level = NGX_LOG_DEBUG; + + return; +} + + +void +ngx_http_lua_log_ringbuf_reset(ngx_http_lua_log_ringbuf_t *rb) +{ + rb->tail = rb->data; + rb->head = rb->data; + rb->sentinel = rb->data + rb->size; + rb->count = 0; + + return; +} + + +/* + * get the next data header, it'll skip the useless data space or + * placehold data + */ +static void * +ngx_http_lua_log_ringbuf_next_header(ngx_http_lua_log_ringbuf_t *rb) +{ + /* useless data */ + if (rb->size - (rb->head - rb->data) < HEADER_LEN) + { + rb->head = rb->data; + return rb->head; + } + + /* placehold data */ + if (rb->head >= rb->sentinel) { + rb->head = rb->data; + return rb->head; + } + + return rb->head; +} + + +/* append data to ring buffer directly */ +static void +ngx_http_lua_log_ringbuf_append(ngx_http_lua_log_ringbuf_t *rb, + int sentinel, int log_level, void *buf, int n) +{ + ngx_http_lua_log_ringbuf_header_t *head; + + if (sentinel) { + rb->sentinel = rb->tail; + rb->tail = rb->data; + return; + } + + head = (ngx_http_lua_log_ringbuf_header_t *) rb->tail; + head->len = n; + head->log_level = log_level; + + rb->tail += HEADER_LEN; + ngx_memcpy(rb->tail, buf, n); + rb->tail += n; + rb->count++; + + return; +} + + +/* size of free spaces */ +static size_t +ngx_http_lua_log_ringbuf_free_spaces(ngx_http_lua_log_ringbuf_t *rb) +{ + if (rb->tail == rb->head && rb->tail == rb->data) { + return rb->size; + } + + if (rb->tail > rb->head) { + return rb->data + rb->size - rb->tail; + } + + return rb->head - rb->tail; +} + + +/* + * try to write log data to ring buffer, throw away old data + * if there was not enough free spaces. + */ +ngx_int_t +ngx_http_lua_log_ringbuf_write(ngx_http_lua_log_ringbuf_t *rb, int log_level, + void *buf, size_t n) +{ + size_t free_spaces; + + ngx_http_lua_log_ringbuf_header_t *head; + + if (n + HEADER_LEN > rb->size) { + return NGX_ERROR; + } + + free_spaces = ngx_http_lua_log_ringbuf_free_spaces(rb); + + if (free_spaces < n + HEADER_LEN) { + /* if the right space is not enough, mark it as placehold data */ + if ((size_t)(rb->data + rb->size - rb->tail) < n + HEADER_LEN) { + ngx_http_lua_log_ringbuf_append(rb, 1, 0, NULL, 0); + } + + do { /* throw away old data */ + if (rb->head != ngx_http_lua_log_ringbuf_next_header(rb)) { + break; + } + + head = (ngx_http_lua_log_ringbuf_header_t *) rb->head; + rb->head += HEADER_LEN + head->len; + rb->count--; + + ngx_http_lua_log_ringbuf_next_header(rb); + free_spaces = ngx_http_lua_log_ringbuf_free_spaces(rb); + } while (free_spaces < n + HEADER_LEN); + } + + ngx_http_lua_log_ringbuf_append(rb, 0, log_level, buf, n); + + return NGX_OK; +} + + +/* read log from ring buffer, do reset if all of the logs were readed. */ +ngx_int_t +ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, int *log_level, + void **buf, size_t *n) +{ + ngx_http_lua_log_ringbuf_header_t *head; + + if (rb->count == 0) { + return NGX_ERROR; + } + + head = (ngx_http_lua_log_ringbuf_header_t *) + ngx_http_lua_log_ringbuf_next_header(rb); + + if (rb->head >= rb->sentinel) { + return NGX_ERROR; + } + + *log_level = head->log_level; + *n = head->len; + rb->head += HEADER_LEN; + *buf = rb->head; + rb->head += head->len; + rb->count--; + + if (rb->count == 0) { + ngx_http_lua_log_ringbuf_reset(rb); + } + + return NGX_OK; +} diff --git a/src/ngx_http_lua_log_ringbuf.h b/src/ngx_http_lua_log_ringbuf.h new file mode 100644 index 0000000000..e855e2514a --- /dev/null +++ b/src/ngx_http_lua_log_ringbuf.h @@ -0,0 +1,31 @@ + +#ifndef _NGX_HTTP_LUA_RINGBUF_H_INCLUDED_ +#define _NGX_HTTP_LUA_RINGBUF_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +typedef struct { + ngx_uint_t filter_level; + char *tail; /* writed point */ + char *head; /* readed point */ + char *data; /* buffer */ + char *sentinel; + size_t size; /* buffer total size */ + size_t count; /* count of logs */ +} ngx_http_lua_log_ringbuf_t; + + +void ngx_http_lua_log_ringbuf_init(ngx_http_lua_log_ringbuf_t *rb, + void *buf, size_t len); +void ngx_http_lua_log_ringbuf_reset(ngx_http_lua_log_ringbuf_t *rb); +ngx_int_t ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, + int *log_level, void **buf, size_t *n); +ngx_int_t ngx_http_lua_log_ringbuf_write(ngx_http_lua_log_ringbuf_t *rb, + int log_level, void *buf, size_t n); + + +#endif /* _NGX_HTTP_LUA_RINGBUF_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 7b17ac41ce..0ad2206911 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -94,6 +94,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, NULL }, + { ngx_string("lua_intercept_error_log"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, + ngx_http_lua_intercept_error_log, + 0, + 0, + NULL }, + #if (NGX_PCRE) { ngx_string("lua_regex_cache_max_entries"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 7f59833371..ba9c9579e8 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -51,6 +51,7 @@ #include "ngx_http_lua_socket_tcp.h" #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_ssl.h" +#include "ngx_http_lua_log_ringbuf.h" #if 1 From 8d8f2c24a19e0ea2ad60fab04f0f1c3b70f926d6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 12 May 2017 17:56:47 -0700 Subject: [PATCH 086/848] doc: documented the lua_intercept_error_log directive. --- README.markdown | 37 +++++++++++++++++++++++++++++++++++++ doc/HttpLuaModule.wiki | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 71 insertions(+) diff --git a/README.markdown b/README.markdown index 99ac286c35..7c040c7ea6 100644 --- a/README.markdown +++ b/README.markdown @@ -1036,6 +1036,7 @@ See Also Directives ========== +* [lua_intercept_error_log](#lua_intercept_error_log) * [lua_use_default_type](#lua_use_default_type) * [lua_malloc_trim](#lua_malloc_trim) * [lua_code_cache](#lua_code_cache) @@ -1109,6 +1110,42 @@ how the result will be used. Below is a diagram showing the order in which direc [Back to TOC](#table-of-contents) +lua_intercept_error_log +----------------------- +**syntax:** *lua_intercept_error_log size* + +**default:** *none* + +**context:** *http* + +Enables a buffer of the specified `size` for capturing all the nginx error log message data (not just those produced +by this module or the nginx http subsystem, but everything) without touching files or disks. + +You can use units like `k` and `m` in the `size` value, as in + +```nginx + + lua_intercept_error_log 100k; +``` + +The buffer never grows. If it is full, new error log messages will replace the oldest ones in the buffer. + +The size of the buffer must be bigger than the maximum length of a single error log message (which is 4K in OpenResty and 2K in stock NGINX). + +You can read the messages in the buffer on the Lua land via the +[get_error_log()](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#get_error_logs) +Lua function of the +[ngx.errlog](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme) +Lua module of the [lua-resty-core](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme) +library. This Lua API function will return the captured error log messages and +also remove these already read from the global capturing buffer, making room +for any new error log data. For this reason, the user should not configure this +buffer to be too big if the user read the buffered error log data fast enough. + +This directive was first introduced in the `v0.10.9` release. + +[Back to TOC](#directives) + lua_use_default_type -------------------- **syntax:** *lua_use_default_type on | off* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f075fa95c5..38ad6ca607 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -859,6 +859,40 @@ how the result will be used. Below is a diagram showing the order in which direc ![Lua Nginx Modules Directives](https://cloud.githubusercontent.com/assets/2137369/15272097/77d1c09e-1a37-11e6-97ef-d9767035fc3e.png) +== lua_intercept_error_log == +'''syntax:''' ''lua_intercept_error_log size'' + +'''default:''' ''none'' + +'''context:''' ''http'' + +Enables a buffer of the specified size for capturing all the nginx error log message data (not just those produced +by this module or the nginx http subsystem, but everything) without touching files or disks. + +You can use units like `k` and `m` in the size value, as in + + + lua_intercept_error_log 100k; + + +As a rule of thumb, a 4KB buffer can usually hold about 20 typical error log messages. So do the maths! + +This buffer never grows. If it is full, new error log messages will replace the oldest ones in the buffer. + +The size of the buffer must be bigger than the maximum length of a single error log message (which is 4K in OpenResty and 2K in stock NGINX). + +You can read the messages in the buffer on the Lua land via the +[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#get_error_logs get_error_log()] +Lua function of the +[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme ngx.errlog] +Lua module of the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme lua-resty-core] +library. This Lua API function will return the captured error log messages and +also remove these already read from the global capturing buffer, making room +for any new error log data. For this reason, the user should not configure this +buffer to be too big if the user read the buffered error log data fast enough. + +This directive was first introduced in the v0.10.9 release. + == lua_use_default_type == '''syntax:''' ''lua_use_default_type on | off'' From 4ed1f96051c06606ad617c4939c5b39ee125b3f6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 12 May 2017 18:03:29 -0700 Subject: [PATCH 087/848] doc: added more material on capturing error log filtering. --- README.markdown | 16 +++++++++++++++- doc/HttpLuaModule.wiki | 12 ++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 7c040c7ea6..59ce1caa1b 100644 --- a/README.markdown +++ b/README.markdown @@ -1128,7 +1128,9 @@ You can use units like `k` and `m` in the `size` value, as in lua_intercept_error_log 100k; ``` -The buffer never grows. If it is full, new error log messages will replace the oldest ones in the buffer. +As a rule of thumb, a 4KB buffer can usually hold about 20 typical error log messages. So do the maths! + +This buffer never grows. If it is full, new error log messages will replace the oldest ones in the buffer. The size of the buffer must be bigger than the maximum length of a single error log message (which is 4K in OpenResty and 2K in stock NGINX). @@ -1142,6 +1144,18 @@ also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough. +Note that the log level specified in the standard [error_log](http://nginx.org/r/error_log) directive +has no effect on this capturing facility. It always captures *everything* including those with a log +level below the specified log level in the [error_log](http://nginx.org/r/error_log) directive. +The user can still choose to set a dynamic filtering log level via the Lua API function +[errlog.set_errlog_filter](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_errlog_filter). +It is much more flexible than the static [error_log](http://nginx.org/r/error_log) directive. + +It is worth noting that there is no way to capture the debugging logs +without building OpenResty or NGINX with the `./configure` +option `--with-debug`. And enabling debugging logs is +strongly discouraged in production builds due to high overhead. + This directive was first introduced in the `v0.10.9` release. [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 38ad6ca607..debba06dec 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -891,6 +891,18 @@ also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough. +Note that the log level specified in the standard [http://nginx.org/r/error_log error_log] directive +has no effect on this capturing facility. It always captures ''everything'' including those with a log +level below the specified log level in the [http://nginx.org/r/error_log error_log] directive. +The user can still choose to set a dynamic filtering log level via the Lua API function +[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_errlog_filter errlog.set_errlog_filter]. +It is much more flexible than the static [http://nginx.org/r/error_log error_log] directive. + +It is worth noting that there is no way to capture the debugging logs +without building OpenResty or NGINX with the ./configure +option --with-debug. And enabling debugging logs is +strongly discouraged in production builds due to high overhead. + This directive was first introduced in the v0.10.9 release. == lua_use_default_type == From ab63ba8df2ad0450fe533cf369ab0f6573b3ea00 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 12 May 2017 18:23:02 -0700 Subject: [PATCH 088/848] change: renamed lua_intercept_error_log directive to lua_capture_error_log. --- README.markdown | 10 +++++----- doc/HttpLuaModule.wiki | 6 +++--- src/ngx_http_lua_common.h | 2 +- src/ngx_http_lua_directive.c | 16 ++++++++-------- src/ngx_http_lua_directive.h | 2 +- src/ngx_http_lua_log.c | 14 +++++++------- src/ngx_http_lua_log.h | 4 ++-- src/ngx_http_lua_module.c | 4 ++-- 8 files changed, 29 insertions(+), 29 deletions(-) diff --git a/README.markdown b/README.markdown index 59ce1caa1b..7cf7bf6366 100644 --- a/README.markdown +++ b/README.markdown @@ -1036,7 +1036,7 @@ See Also Directives ========== -* [lua_intercept_error_log](#lua_intercept_error_log) +* [lua_capture_error_log](#lua_capture_error_log) * [lua_use_default_type](#lua_use_default_type) * [lua_malloc_trim](#lua_malloc_trim) * [lua_code_cache](#lua_code_cache) @@ -1110,9 +1110,9 @@ how the result will be used. Below is a diagram showing the order in which direc [Back to TOC](#table-of-contents) -lua_intercept_error_log ------------------------ -**syntax:** *lua_intercept_error_log size* +lua_capture_error_log +--------------------- +**syntax:** *lua_capture_error_log size* **default:** *none* @@ -1125,7 +1125,7 @@ You can use units like `k` and `m` in the `size` value, as in ```nginx - lua_intercept_error_log 100k; + lua_capture_error_log 100k; ``` As a rule of thumb, a 4KB buffer can usually hold about 20 typical error log messages. So do the maths! diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index debba06dec..50881381ac 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -859,8 +859,8 @@ how the result will be used. Below is a diagram showing the order in which direc ![Lua Nginx Modules Directives](https://cloud.githubusercontent.com/assets/2137369/15272097/77d1c09e-1a37-11e6-97ef-d9767035fc3e.png) -== lua_intercept_error_log == -'''syntax:''' ''lua_intercept_error_log size'' +== lua_capture_error_log == +'''syntax:''' ''lua_capture_error_log size'' '''default:''' ''none'' @@ -872,7 +872,7 @@ by this module or the nginx http subsystem, but everything) without touching fil You can use units like `k` and `m` in the size value, as in - lua_intercept_error_log 100k; + lua_capture_error_log 100k; As a rule of thumb, a 4KB buffer can usually hold about 20 typical error log messages. So do the maths! diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index a923aedf02..e38978389c 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -230,7 +230,7 @@ struct ngx_http_lua_main_conf_s { unsigned requires_access:1; unsigned requires_log:1; unsigned requires_shm:1; - unsigned requires_intercept_log:1; + unsigned requires_capture_log:1; }; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index a92f056d71..6a562f47b4 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1703,11 +1703,11 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, char * -ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, +ngx_http_lua_capture_error_log(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { #ifndef HAVE_INTERCEPT_ERROR_LOG_PATCH - return "not found: missing the intercept error log patch for nginx"; + return "not found: missing the capture error log patch for nginx"; #else ngx_str_t *value; ssize_t size; @@ -1719,13 +1719,13 @@ ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, value = cf->args->elts; cycle = cf->cycle; - if (lmcf->requires_intercept_log) { + if (lmcf->requires_capture_log) { return "is duplicate"; } if (value[1].len == 0) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "invalid intercept error log size \"%V\"", + "invalid capture error log size \"%V\"", &value[1]); return NGX_CONF_ERROR; } @@ -1734,14 +1734,14 @@ ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, if (size < NGX_MAX_ERROR_STR) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "invalid intercept error log size \"%V\", " + "invalid capture error log size \"%V\", " "minimum size is %d", &value[1], NGX_MAX_ERROR_STR); return NGX_CONF_ERROR; } if (cycle->intercept_error_log_handler) { - return "intercept error log handler has been hooked"; + return "capture error log handler has been hooked"; } ringbuf = (ngx_http_lua_log_ringbuf_t *) @@ -1757,9 +1757,9 @@ ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, ngx_http_lua_log_ringbuf_init(ringbuf, data, size); - lmcf->requires_intercept_log = 1; + lmcf->requires_capture_log = 1; cycle->intercept_error_log_handler = (ngx_log_intercept_pt) - ngx_http_lua_intercept_log_handler; + ngx_http_lua_capture_log_handler; cycle->intercept_error_log_data = ringbuf; return NGX_CONF_OK; diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index aabb118994..5abfe4dde2 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -69,7 +69,7 @@ char *ngx_http_lua_rewrite_no_postpone(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd); -char *ngx_http_lua_intercept_error_log(ngx_conf_t *cf, ngx_command_t *cmd, +char *ngx_http_lua_capture_error_log(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index 9e87a590a7..e0ac5a654e 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -317,8 +317,8 @@ ngx_http_lua_inject_log_consts(lua_State *L) #ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH ngx_int_t -ngx_http_lua_intercept_log_handler(ngx_log_t *log, - ngx_uint_t level, void *buf, size_t n) +ngx_http_lua_capture_log_handler(ngx_log_t *log, + ngx_uint_t level, u_char *buf, size_t n) { ngx_http_lua_log_ringbuf_t *ringbuf; @@ -333,7 +333,7 @@ ngx_http_lua_intercept_log_handler(ngx_log_t *log, ngx_http_lua_log_ringbuf_write(ringbuf, level, buf, n); - dd("intercept log: %s\n", buf); + dd("capture log: %s\n", buf); return NGX_OK; } @@ -352,7 +352,7 @@ ngx_http_lua_ffi_set_errlog_filter(int level, u_char *err, size_t *errlen) if (ringbuf == NULL) { *errlen = ngx_snprintf(err, *errlen, "API \"set_errlog_filter\" depends on " - "directive \"lua_intercept_error_log\"") + "directive \"lua_capture_error_log\"") - err; return NGX_ERROR; } @@ -367,7 +367,7 @@ ngx_http_lua_ffi_set_errlog_filter(int level, u_char *err, size_t *errlen) return NGX_OK; #else *errlen = ngx_snprintf(err, *errlen, - "missing the intercept error log patch for nginx") + "missing the capture error log patch for nginx") - err; return NGX_ERROR; #endif @@ -388,7 +388,7 @@ ngx_http_lua_ffi_get_errlog_data(char **log, int *loglevel, u_char *err, if (ringbuf == NULL) { *errlen = ngx_snprintf(err, *errlen, "API \"get_errlog_data\" depends on directive " - "\"lua_intercept_error_log\"") + "\"lua_capture_error_log\"") - err; return NGX_ERROR; } @@ -401,7 +401,7 @@ ngx_http_lua_ffi_get_errlog_data(char **log, int *loglevel, u_char *err, return loglen; #else *errlen = ngx_snprintf(err, *errlen, - "missing the intercept error log patch for nginx") + "missing the capture error log patch for nginx") - err; return NGX_ERROR; #endif diff --git a/src/ngx_http_lua_log.h b/src/ngx_http_lua_log.h index 115679af3f..56cd7719a0 100644 --- a/src/ngx_http_lua_log.h +++ b/src/ngx_http_lua_log.h @@ -14,8 +14,8 @@ void ngx_http_lua_inject_log_api(lua_State *L); #ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH -ngx_int_t ngx_http_lua_intercept_log_handler(ngx_log_t *log, - ngx_uint_t level, void *buf, size_t n); +ngx_int_t ngx_http_lua_capture_log_handler(ngx_log_t *log, + ngx_uint_t level, u_char *buf, size_t n); #endif diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 0ad2206911..9d914e86e8 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -94,9 +94,9 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, NULL }, - { ngx_string("lua_intercept_error_log"), + { ngx_string("lua_capture_error_log"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, - ngx_http_lua_intercept_error_log, + ngx_http_lua_capture_error_log, 0, 0, NULL }, From eb0bb285378f88790630e6723bda6c65ccb04791 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 13 May 2017 19:30:12 -0700 Subject: [PATCH 089/848] lua_capture_error_log: various documentation fixes and API name changes. --- README.markdown | 16 ++++++++-------- doc/HttpLuaModule.wiki | 16 ++++++++-------- src/ngx_http_lua_log.c | 10 ++++------ 3 files changed, 20 insertions(+), 22 deletions(-) diff --git a/README.markdown b/README.markdown index 7cf7bf6366..cb2057bded 100644 --- a/README.markdown +++ b/README.markdown @@ -1135,21 +1135,21 @@ This buffer never grows. If it is full, new error log messages will replace the The size of the buffer must be bigger than the maximum length of a single error log message (which is 4K in OpenResty and 2K in stock NGINX). You can read the messages in the buffer on the Lua land via the -[get_error_log()](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#get_error_logs) -Lua function of the +[get_logs()](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#get_logs) +function of the [ngx.errlog](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme) -Lua module of the [lua-resty-core](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme) +module of the [lua-resty-core](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme) library. This Lua API function will return the captured error log messages and also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough. Note that the log level specified in the standard [error_log](http://nginx.org/r/error_log) directive -has no effect on this capturing facility. It always captures *everything* including those with a log -level below the specified log level in the [error_log](http://nginx.org/r/error_log) directive. -The user can still choose to set a dynamic filtering log level via the Lua API function -[errlog.set_errlog_filter](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_errlog_filter). -It is much more flexible than the static [error_log](http://nginx.org/r/error_log) directive. +*does* have effect on this capturing facility. It only captures log +messages of a level no lower than the specified log level in the [error_log](http://nginx.org/r/error_log) directive. +The user can still choose to set an even higher filtering log level on the fly via the Lua API function +[errlog.set_filter_level](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_filter_level). +So it is more flexible than the static [error_log](http://nginx.org/r/error_log) directive. It is worth noting that there is no way to capture the debugging logs without building OpenResty or NGINX with the `./configure` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 50881381ac..c0c73624fe 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -882,21 +882,21 @@ This buffer never grows. If it is full, new error log messages will replace the The size of the buffer must be bigger than the maximum length of a single error log message (which is 4K in OpenResty and 2K in stock NGINX). You can read the messages in the buffer on the Lua land via the -[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#get_error_logs get_error_log()] -Lua function of the +[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#get_logs get_logs()] +function of the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme ngx.errlog] -Lua module of the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme lua-resty-core] +module of the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#readme lua-resty-core] library. This Lua API function will return the captured error log messages and also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough. Note that the log level specified in the standard [http://nginx.org/r/error_log error_log] directive -has no effect on this capturing facility. It always captures ''everything'' including those with a log -level below the specified log level in the [http://nginx.org/r/error_log error_log] directive. -The user can still choose to set a dynamic filtering log level via the Lua API function -[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_errlog_filter errlog.set_errlog_filter]. -It is much more flexible than the static [http://nginx.org/r/error_log error_log] directive. +''does'' have effect on this capturing facility. It only captures log +messages of a level no lower than the specified log level in the [http://nginx.org/r/error_log error_log] directive. +The user can still choose to set an even higher filtering log level on the fly via the Lua API function +[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_filter_level errlog.set_filter_level]. +So it is more flexible than the static [http://nginx.org/r/error_log error_log] directive. It is worth noting that there is no way to capture the debugging logs without building OpenResty or NGINX with the ./configure diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index e0ac5a654e..097c544ed3 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -342,7 +342,7 @@ ngx_http_lua_capture_log_handler(ngx_log_t *log, #ifndef NGX_LUA_NO_FFI_API int -ngx_http_lua_ffi_set_errlog_filter(int level, u_char *err, size_t *errlen) +ngx_http_lua_ffi_errlog_set_filter_level(int level, u_char *err, size_t *errlen) { #ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH ngx_http_lua_log_ringbuf_t *ringbuf; @@ -351,8 +351,7 @@ ngx_http_lua_ffi_set_errlog_filter(int level, u_char *err, size_t *errlen) if (ringbuf == NULL) { *errlen = ngx_snprintf(err, *errlen, - "API \"set_errlog_filter\" depends on " - "directive \"lua_capture_error_log\"") + "directive \"lua_capture_error_log\" is not set") - err; return NGX_ERROR; } @@ -375,7 +374,7 @@ ngx_http_lua_ffi_set_errlog_filter(int level, u_char *err, size_t *errlen) int -ngx_http_lua_ffi_get_errlog_data(char **log, int *loglevel, u_char *err, +ngx_http_lua_ffi_errlog_get_logs(char **log, int *loglevel, u_char *err, size_t *errlen) { #ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH @@ -387,8 +386,7 @@ ngx_http_lua_ffi_get_errlog_data(char **log, int *loglevel, u_char *err, if (ringbuf == NULL) { *errlen = ngx_snprintf(err, *errlen, - "API \"get_errlog_data\" depends on directive " - "\"lua_capture_error_log\"") + "directive \"lua_capture_error_log\" is not set") - err; return NGX_ERROR; } From 76e2138975463aa9162f482579b688f0e9e06d54 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 13 May 2017 19:35:25 -0700 Subject: [PATCH 090/848] change: renamed the C API function ngx_http_lua_ffi_errlog_get_logs to ngx_http_lua_ffi_errlog_get_msg. --- src/ngx_http_lua_log.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index 097c544ed3..5fc79dd232 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -374,7 +374,7 @@ ngx_http_lua_ffi_errlog_set_filter_level(int level, u_char *err, size_t *errlen) int -ngx_http_lua_ffi_errlog_get_logs(char **log, int *loglevel, u_char *err, +ngx_http_lua_ffi_errlog_get_msg(char **log, int *loglevel, u_char *err, size_t *errlen) { #ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH From 2a87704933546b4239ab0ebfa37b9be231d0dc04 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 14 May 2017 11:25:46 -0700 Subject: [PATCH 091/848] feature: added new pure C API ngx_http_lua_ffi_process_signal_graceful_exit() for the signal_graceful_exit() function of the ngx.process module in lua-resty-core. --- src/ngx_http_lua_worker.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 4af311bc2e..f4425b914a 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -168,4 +168,11 @@ ngx_http_lua_ffi_enable_privileged_agent(char **err) return NGX_ERROR; #endif } + + +void +ngx_http_lua_ffi_process_signal_graceful_exit(void) +{ + ngx_quit = 1; +} #endif From 9b16e254e84956cf344750caf29dc7cb66d8e1c9 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Thu, 1 Sep 2016 10:52:32 +0800 Subject: [PATCH 092/848] feature: implemented the ngx.timer.every() API function for creating recurring timers. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 31 +++- doc/HttpLuaModule.wiki | 27 ++- src/ngx_http_lua_timer.c | 209 ++++++++++++++++++++- t/062-count.t | 2 +- t/152-timer-every.t | 381 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 634 insertions(+), 16 deletions(-) create mode 100644 t/152-timer-every.t diff --git a/README.markdown b/README.markdown index cb2057bded..24f10731da 100644 --- a/README.markdown +++ b/README.markdown @@ -1471,8 +1471,8 @@ This hook is often used to create per-worker reoccurring timers (via the [ngx.ti end end - local ok, err = new_timer(delay, check) - if not ok then + local hdl, err = new_timer(delay, check) + if not hdl then log(ERR, "failed to create timer: ", err) return end @@ -3217,6 +3217,7 @@ Nginx API for Lua * [ngx.thread.kill](#ngxthreadkill) * [ngx.on_abort](#ngxon_abort) * [ngx.timer.at](#ngxtimerat) +* [ngx.timer.every](#ngxtimerevery) * [ngx.timer.running_count](#ngxtimerrunning_count) * [ngx.timer.pending_count](#ngxtimerpending_count) * [ngx.config.subsystem](#ngxconfigsubsystem) @@ -7568,7 +7569,7 @@ See also [lua_check_client_abort](#lua_check_client_abort). ngx.timer.at ------------ -**syntax:** *ok, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)* +**syntax:** *hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -7594,7 +7595,7 @@ Premature timer expiration happens when the Nginx worker process is trying to shut down, as in an Nginx configuration reload triggered by the `HUP` signal or in an Nginx server shutdown. When the Nginx worker is trying to shut down, one can no longer call `ngx.timer.at` to -create new timers with nonzero delays and in that case `ngx.timer.at` will return `nil` and +create new timers with nonzero delays and in that case `ngx.timer.at` will return a "conditional false" value and a string describing the error, that is, "process exiting". Starting from the `v0.9.3` release, it is allowed to create zero-delay timers even when the Nginx worker process starts shutting down. @@ -7653,6 +7654,9 @@ One can also create infinite re-occurring timers, for instance, a timer getting end ``` +It is recommended, however, to use the [ngx.timer.every](#ngxtimerevery) API function +instead for creating recurring timers since it is more robust. + Because timer callbacks run in the background and their running time will not add to any client request's response time, they can easily accumulate in the server and exhaust system resources due to either @@ -7692,6 +7696,25 @@ This API was first introduced in the `v0.8.0` release. [Back to TOC](#nginx-api-for-lua) +ngx.timer.every +--------------- +**syntax:** *hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)* + +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** + +Similar to the [ngx.timer.at](#ngxtimerat) API function, but + +1. `delay` *cannot* be zero, +1. timer will be created every `delay` seconds until the current Nginx worker process starts exiting. + +When success, returns a "conditional true" value (but not a `true`). Otherwise, returns a "conditional false" value and a string describing the error. + +This API also respect the [lua_max_pending_timers](#lua_max_pending_timers) and [lua_max_running_timers](#lua_max_running_timers). + +This API was first introduced in the `v0.10.9` release. + +[Back to TOC](#nginx-api-for-lua) + ngx.timer.running_count ----------------------- **syntax:** *count = ngx.timer.running_count()* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index c0c73624fe..1e8d0295b8 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1180,8 +1180,8 @@ This hook is often used to create per-worker reoccurring timers (via the [[#ngx. end end - local ok, err = new_timer(delay, check) - if not ok then + local hdl, err = new_timer(delay, check) + if not hdl then log(ERR, "failed to create timer: ", err) return end @@ -6412,7 +6412,7 @@ This API was first introduced in the v0.7.4 release. See also [[#lua_check_client_abort|lua_check_client_abort]]. == ngx.timer.at == -'''syntax:''' ''ok, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)'' +'''syntax:''' ''hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -6438,7 +6438,7 @@ Premature timer expiration happens when the Nginx worker process is trying to shut down, as in an Nginx configuration reload triggered by the HUP signal or in an Nginx server shutdown. When the Nginx worker is trying to shut down, one can no longer call ngx.timer.at to -create new timers with nonzero delays and in that case ngx.timer.at will return nil and +create new timers with nonzero delays and in that case ngx.timer.at will return a "conditional false" value and a string describing the error, that is, "process exiting". Starting from the v0.9.3 release, it is allowed to create zero-delay timers even when the Nginx worker process starts shutting down. @@ -6495,6 +6495,9 @@ One can also create infinite re-occurring timers, for instance, a timer getting end +It is recommended, however, to use the [[#ngx.timer.every|ngx.timer.every]] API function +instead for creating recurring timers since it is more robust. + Because timer callbacks run in the background and their running time will not add to any client request's response time, they can easily accumulate in the server and exhaust system resources due to either @@ -6532,6 +6535,22 @@ You can pass most of the standard Lua values (nils, booleans, numbers, strings, This API was first introduced in the v0.8.0 release. +== ngx.timer.every == +'''syntax:''' ''hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)'' + +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' + +Similar to the [[#ngx.timer.at|ngx.timer.at]] API function, but + +# delay ''cannot'' be zero, +# timer will be created every delay seconds until the current Nginx worker process starts exiting. + +When success, returns a "conditional true" value (but not a true). Otherwise, returns a "conditional false" value and a string describing the error. + +This API also respect the [[#lua_max_pending_timers|lua_max_pending_timers]] and [[#lua_max_running_timers|lua_max_running_timers]]. + +This API was first introduced in the v0.10.9 release. + == ngx.timer.running_count == '''syntax:''' ''count = ngx.timer.running_count()'' diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 01b4777896..596b2f77e9 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -21,11 +21,6 @@ typedef struct { void **srv_conf; void **loc_conf; - /* event ident must be after 3 words (i.e. 3 pointers' size) as in - * ngx_connection_t. and we use the Lua coroutine reference number as - * the event ident */ - int co_ref; - unsigned premature; /* :1 */ lua_State *co; ngx_pool_t *pool; @@ -36,12 +31,18 @@ typedef struct { ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_vm_state_t *vm_state; + int co_ref; + unsigned delay:31; + unsigned premature:1; } ngx_http_lua_timer_ctx_t; static int ngx_http_lua_ngx_timer_at(lua_State *L); +static int ngx_http_lua_ngx_timer_every(lua_State *L); +static int ngx_http_lua_ngx_timer_helper(lua_State *L, int every); static int ngx_http_lua_ngx_timer_running_count(lua_State *L); static int ngx_http_lua_ngx_timer_pending_count(lua_State *L); +static ngx_int_t ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx); static void ngx_http_lua_timer_handler(ngx_event_t *ev); static u_char *ngx_http_lua_log_timer_error(ngx_log_t *log, u_char *buf, size_t len); @@ -51,11 +52,14 @@ static void ngx_http_lua_abort_pending_timers(ngx_event_t *ev); void ngx_http_lua_inject_timer_api(lua_State *L) { - lua_createtable(L, 0 /* narr */, 3 /* nrec */); /* ngx.timer. */ + lua_createtable(L, 0 /* narr */, 4 /* nrec */); /* ngx.timer. */ lua_pushcfunction(L, ngx_http_lua_ngx_timer_at); lua_setfield(L, -2, "at"); + lua_pushcfunction(L, ngx_http_lua_ngx_timer_every); + lua_setfield(L, -2, "every"); + lua_pushcfunction(L, ngx_http_lua_ngx_timer_running_count); lua_setfield(L, -2, "running_count"); @@ -106,6 +110,24 @@ ngx_http_lua_ngx_timer_pending_count(lua_State *L) static int ngx_http_lua_ngx_timer_at(lua_State *L) +{ + return ngx_http_lua_ngx_timer_helper(L, 0); +} + + +/* + * TODO: return a timer handler instead which can be passed to + * the ngx.timer.cancel method to cancel the timer. + */ +static int +ngx_http_lua_ngx_timer_every(lua_State *L) +{ + return ngx_http_lua_ngx_timer_helper(L, 1); +} + + +static int +ngx_http_lua_ngx_timer_helper(lua_State *L, int every) { int nargs, co_ref; u_char *p; @@ -134,6 +156,10 @@ ngx_http_lua_ngx_timer_at(lua_State *L) delay = (ngx_msec_t) (luaL_checknumber(L, 1) * 1000); + if (every && delay == 0) { + return luaL_error(L, "delay cannot be zero"); + } + luaL_argcheck(L, lua_isfunction(L, 2) && !lua_iscfunction(L, 2), 2, "Lua function expected"); @@ -233,7 +259,7 @@ ngx_http_lua_ngx_timer_at(lua_State *L) lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); lua_rawget(L, LUA_REGISTRYINDEX); - /* L stack: time func [args] thread corountines */ + /* L stack: time func [args] thread coroutines */ lua_pushvalue(L, -2); @@ -265,6 +291,8 @@ ngx_http_lua_ngx_timer_at(lua_State *L) tctx = (ngx_http_lua_timer_ctx_t *) p; + tctx->delay = every ? delay : 0; + tctx->premature = 0; tctx->co_ref = co_ref; tctx->co = co; @@ -338,6 +366,164 @@ ngx_http_lua_ngx_timer_at(lua_State *L) } +static ngx_int_t +ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) +{ + int nargs, co_ref, i; + u_char *p; + lua_State *vm; /* the main thread */ + lua_State *co; + lua_State *L; + ngx_event_t *ev = NULL; + ngx_http_lua_timer_ctx_t *tctx = NULL; + ngx_http_lua_main_conf_t *lmcf; + + /* L stack: func [args] */ + L = old_tctx->co; + + lmcf = old_tctx->lmcf; + + vm = old_tctx->vm_state ? old_tctx->vm_state->vm : lmcf->lua; + + co = lua_newthread(vm); + + lua_createtable(co, 0, 0); /* the new globals table */ + + /* co stack: global_tb */ + + lua_createtable(co, 0, 1); /* the metatable */ + ngx_http_lua_get_globals_table(co); + lua_setfield(co, -2, "__index"); + lua_setmetatable(co, -2); + + /* co stack: global_tb */ + + ngx_http_lua_set_globals_table(co); + + /* co stack: */ + + dd("stack top: %d", lua_gettop(L)); + + lua_xmove(vm, L, 1); /* move coroutine from main thread to L */ + + /* L stack: func [args] thread */ + /* vm stack: empty */ + + lua_pushvalue(L, 1); /* copy entry function to top of L*/ + + /* L stack: func [args] thread func */ + + lua_xmove(L, co, 1); /* move entry function from L to co */ + + /* L stack: func [args] thread */ + /* co stack: func */ + + ngx_http_lua_get_globals_table(co); + lua_setfenv(co, -2); + + /* co stack: func */ + + lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_rawget(L, LUA_REGISTRYINDEX); + + /* L stack: func [args] thread coroutines */ + + lua_pushvalue(L, -2); + + /* L stack: func [args] thread coroutines thread */ + + co_ref = luaL_ref(L, -2); + lua_pop(L, 2); + + /* L stack: func [args] */ + + nargs = lua_gettop(L); + if (nargs > 1) { + for (i = 2; i <= nargs; i++) { + lua_pushvalue(L, i); + } + + /* L stack: func [args] [args] */ + + lua_xmove(L, co, nargs - 1); + + /* L stack: func [args] */ + /* co stack: func [args] */ + } + + p = ngx_alloc(sizeof(ngx_event_t) + sizeof(ngx_http_lua_timer_ctx_t), + ngx_cycle->log); + if (p == NULL) { + goto nomem; + } + + ev = (ngx_event_t *) p; + + ngx_memzero(ev, sizeof(ngx_event_t)); + + p += sizeof(ngx_event_t); + + tctx = (ngx_http_lua_timer_ctx_t *) p; + + ngx_memcpy(tctx, old_tctx, sizeof(ngx_http_lua_timer_ctx_t)); + + tctx->co_ref = co_ref; + tctx->co = co; + + tctx->pool = ngx_create_pool(128, ngx_cycle->log); + if (tctx->pool == NULL) { + goto nomem; + } + + if (tctx->client_addr_text.len) { + tctx->client_addr_text.data = ngx_palloc(tctx->pool, + tctx->client_addr_text.len); + if (tctx->client_addr_text.data == NULL) { + goto nomem; + } + + ngx_memcpy(tctx->client_addr_text.data, old_tctx->client_addr_text.data, + tctx->client_addr_text.len); + } + + if (tctx->vm_state) { + tctx->vm_state->count++; + } + + ev->handler = ngx_http_lua_timer_handler; + ev->data = tctx; + ev->log = ngx_cycle->log; + + lmcf->pending_timers++; + + ngx_add_timer(ev, tctx->delay); + + return NGX_OK; + +nomem: + + if (tctx && tctx->pool) { + ngx_destroy_pool(tctx->pool); + } + + if (ev) { + ngx_free(ev); + } + + /* L stack: func [args] */ + + lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_rawget(L, LUA_REGISTRYINDEX); + luaL_unref(L, -1, co_ref); + + /* L stack: func [args] coroutines */ + + lua_pop(L, 1); + + return NGX_ERROR; +} + + static void ngx_http_lua_timer_handler(ngx_event_t *ev) { @@ -364,6 +550,15 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) lmcf->pending_timers--; + if (!ngx_exiting && tctx.delay > 0) { + rc = ngx_http_lua_timer_copy(&tctx); + if (rc != NGX_OK) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "failed to create the next timer of delay %ud ms", + (unsigned) tctx.delay); + } + } + if (lmcf->running_timers >= lmcf->max_running_timers) { ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, "%i lua_max_running_timers are not enough", diff --git a/t/062-count.t b/t/062-count.t index c3973adc2f..a69c33edb9 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -323,7 +323,7 @@ ngx. entry count: 116 --- request GET /test --- response_body -n = 3 +n = 4 --- no_error_log [error] diff --git a/t/152-timer-every.t b/t/152-timer-every.t new file mode 100644 index 0000000000..5026ed086c --- /dev/null +++ b/t/152-timer-every.t @@ -0,0 +1,381 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_on(); +#workers(2); +#log_level('warn'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 7 + 2); + +#no_diff(); +no_long_string(); + +our $HtmlDir = html_dir; + +$ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; +$ENV{TEST_NGINX_HTML_DIR} = $HtmlDir; + +worker_connections(1024); +run_tests(); + +__DATA__ + +=== TEST 1: simple very +--- config + location /t { + content_by_lua_block { + local begin = ngx.now() + local function f(premature) + print("elapsed: ", ngx.now() - begin) + print("timer prematurely expired: ", premature) + end + + local ok, err = ngx.timer.every(0.05, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + ngx.say("registered timer") + } + } +--- request +GET /t +--- response_body +registered timer +--- wait: 0.11 +--- no_error_log +[error] +[alert] +[crit] +timer prematurely expired: true +--- error_log eval +[ +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:4[4-9]|5[0-6])\d*, context: ngx\.timer, client: \d+\.\d+\.\d+\.\d+, server: 0\.0\.0\.0:\d+/, +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:09|10)\d*, context: ngx\.timer, client: \d+\.\d+\.\d+\.\d+, server: 0\.0\.0\.0:\d+/, +"lua ngx.timer expired", +"http lua close fake http connection", +"timer prematurely expired: false", +] + + + +=== TEST 2: separated global env +--- config + location /t { + content_by_lua_block { + local begin = ngx.now() + local function f() + foo = 3 + print("foo in timer: ", foo) + end + local ok, err = ngx.timer.every(0.05, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + ngx.sleep(0.11) + ngx.say("foo = ", foo) + } + } +--- request +GET /t +--- response_body +foo = nil +--- wait: 0.12 +--- no_error_log +[error] +[alert] +[crit] +--- error_log eval +[ +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: foo in timer: 3/, +"lua ngx.timer expired", +"http lua close fake http connection" +] + + + +=== TEST 3: lua variable sharing via upvalue +--- config + location /t { + content_by_lua_block { + local begin = ngx.now() + local foo = 0 + local function f() + foo = foo + 3 + print("foo in timer: ", foo) + end + local ok, err = ngx.timer.every(0.05, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + ngx.say("registered timer") + ngx.sleep(0.11) + ngx.say("foo = ", foo) + } + } +--- request +GET /t +--- response_body +registered timer +foo = 6 +--- wait: 0.12 +--- no_error_log +[error] +[alert] +[crit] +--- error_log eval +[ +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: foo in timer: 3/, +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: foo in timer: 6/, +"lua ngx.timer expired", +"http lua close fake http connection" +] + + + +=== TEST 4: create the next timer immediately when timer start running +--- config + location /t { + content_by_lua_block { + local begin = ngx.now() + local foo = 0 + local function f() + foo = foo + 3 + print("foo in timer: ", foo) + + ngx.sleep(0.1) + end + local ok, err = ngx.timer.every(0.05, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + ngx.say("registered timer") + ngx.sleep(0.11) + ngx.say("foo = ", foo) + } + } +--- request +GET /t +--- response_body +registered timer +foo = 6 +--- wait: 0.12 +--- no_error_log +[error] +[alert] +[crit] +--- error_log eval +[ +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: foo in timer: 3/, +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: foo in timer: 6/, +"lua ngx.timer expired", +"http lua close fake http connection" +] + + + +=== TEST 5: callback args +--- config + location /t { + content_by_lua_block { + local n = 0 + + local function f(premature, a, b, c) + n = n + 1 + print("the ", n, " time, args: ", a, ", ", b, ", ", c) + + a, b, c = 0, 0, 0 + end + + local ok, err = ngx.timer.every(0.05, f, 1, 2) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + ngx.say("registered timer") + ngx.sleep(0.11) + } + } +--- request +GET /t +--- response_body +registered timer +--- wait: 0.12 +--- no_error_log +[error] +[alert] +[crit] +--- error_log eval +[ +"the 1 time, args: 1, 2, nil", +"the 2 time, args: 1, 2, nil", +"lua ngx.timer expired", +"http lua close fake http connection" +] + + + +=== TEST 6: memory leak check +--- config + location /t { + content_by_lua_block { + local function f() + local a = 1 + -- do nothing + end + + for i = 1, 100 do + local ok, err = ngx.timer.every(0.1, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + end + + ngx.say("registered timer") + + collectgarbage("collect") + local start = collectgarbage("count") + + ngx.sleep(0.21) + + collectgarbage("collect") + local growth1 = collectgarbage("count") - start + + ngx.sleep(0.51) + + collectgarbage("collect") + local growth2 = collectgarbage("count") - start + + ngx.say("growth1 == growth2: ", growth1 == growth2) + } + } +--- request +GET /t +--- response_body +registered timer +growth1 == growth2: true +--- no_error_log +[error] +[alert] +[crit] + + + +=== TEST 7: respect lua_max_pending_timers +--- http_config + lua_max_pending_timers 10; +--- config + location /t { + content_by_lua_block { + local function f() + local a = 1 + -- do nothing + end + + for i = 1, 11 do + local ok, err = ngx.timer.every(0.1, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + end + + ngx.say("registered 10 timers") + } + } +--- request +GET /t +--- response_body +failed to set timer: too many pending timers +--- no_error_log +[error] +[alert] +[crit] + + + +=== TEST 8: respect lua_max_running_timers +--- http_config + lua_max_pending_timers 100; + lua_max_running_timers 9; +--- config + location /t { + content_by_lua_block { + local function f() + local a = 1 + ngx.sleep(0.02) + -- do nothing + end + + for i = 1, 10 do + local ok, err = ngx.timer.every(0.01, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + end + + ngx.say("registered 10 timers") + + ngx.sleep(0.03) + } + } +--- request +GET /t +--- response_body +registered 10 timers +--- no_error_log +[error] +[crit] +--- error_log +lua_max_running_timers are not enough + + + +=== TEST 9: lua_code_cache off +--- http_config + lua_code_cache off; +--- config + location /t { + content_by_lua_block { + local function f() + local a = 1 + -- do nothing + end + + local ok, err = ngx.timer.every(0.01, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + collectgarbage("collect") + ngx.say("registered timer") + + ngx.sleep(0.03) + + collectgarbage("collect") + + ngx.sleep(0.03) + + collectgarbage("collect") + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +registered timer +ok +--- no_error_log +[error] +[crit] From 2dd44623d7823c89ee081e10cf5346f5a8972ef4 Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Mon, 24 Apr 2017 15:15:08 -0700 Subject: [PATCH 093/848] feature: ngx.sleep(0) now always yield the control to the nginx event loop. This can be used to do voluntary CPU time slicing when running CPU intensive computations on the Lua land and to avoid such computations from blocking the nginx event loop for too long. This feature requires OpenResty's delayed-posted-event patch for the nginx core. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_sleep.c | 30 ++++++++++-- t/077-sleep.t | 100 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 125 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_sleep.c b/src/ngx_http_lua_sleep.c index 4c0d016672..ffee97f84b 100644 --- a/src/ngx_http_lua_sleep.c +++ b/src/ngx_http_lua_sleep.c @@ -72,10 +72,25 @@ ngx_http_lua_ngx_sleep(lua_State *L) coctx->sleep.data = coctx; coctx->sleep.log = r->connection->log; - dd("adding timer with delay %lu ms, r:%.*s", (unsigned long) delay, - (int) r->uri.len, r->uri.data); + if (delay == 0) { +#ifdef HAVE_POSTED_DELAYED_EVENTS_PATCH + dd("posting 0 sec sleep event to head of delayed queue"); + + coctx->sleep.delayed = 1; + ngx_post_event(&coctx->sleep, &ngx_posted_delayed_events); +#else + ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, "ngx.sleep(0)" + " called without delayed events patch, this will" + " hurt performance"); + ngx_add_timer(&coctx->sleep, (ngx_msec_t) delay); +#endif - ngx_add_timer(&coctx->sleep, (ngx_msec_t) delay); + } else { + dd("adding timer with delay %lu ms, r:%.*s", (unsigned long) delay, + (int) r->uri.len, r->uri.data); + + ngx_add_timer(&coctx->sleep, (ngx_msec_t) delay); + } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua ready to sleep for %d ms", delay); @@ -147,6 +162,15 @@ ngx_http_lua_sleep_cleanup(void *data) ngx_del_timer(&coctx->sleep); } + +#ifdef HAVE_POSTED_DELAYED_EVENTS_PATCH + if (coctx->sleep.posted) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua clean up the posted event for pending ngx.sleep"); + + ngx_delete_posted_event(&coctx->sleep); + } +#endif } diff --git a/t/077-sleep.t b/t/077-sleep.t index 7d295c2b5d..96f04bd98e 100644 --- a/t/077-sleep.t +++ b/t/077-sleep.t @@ -9,10 +9,10 @@ log_level('debug'); repeat_each(2); -plan tests => repeat_each() * 63; +plan tests => repeat_each() * 71; #no_diff(); -#no_long_string(); +no_long_string(); run_tests(); __DATA__ @@ -404,3 +404,99 @@ ok --- no_error_log [error] [alert] + + + +=== TEST 16: sleep 0 +--- config + location /t { + content_by_lua_block { + local function f (n) + print("f begin ", n) + ngx.sleep(0) + print("f middle ", n) + ngx.sleep(0) + print("f end ", n) + ngx.sleep(0) + end + + for i = 1, 3 do + assert(ngx.thread.spawn(f, i)) + end + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- no_error_log +[error] +--- grep_error_log eval: qr/\bf (?:begin|middle|end)\b|\bworker cycle$|\be?poll timer: \d+$/ +--- grep_error_log_out eval +qr/f begin +f begin +f begin +worker cycle +e?poll timer: 0 +f middle +f middle +f middle +worker cycle +e?poll timer: 0 +f end +f end +f end +worker cycle +e?poll timer: 0 +/ + + + +=== TEST 17: sleep short times less than 1ms +--- config + location /t { + content_by_lua_block { + local delay = 0.0005 + + local function f (n) + print("f begin ", n) + ngx.sleep(delay) + print("f middle ", n) + ngx.sleep(delay) + print("f end ", n) + ngx.sleep(delay) + end + + for i = 1, 3 do + assert(ngx.thread.spawn(f, i)) + end + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- no_error_log +[error] +--- grep_error_log eval: qr/\bf (?:begin|middle|end)\b|\bworker cycle$|\be?poll timer: \d+$/ +--- grep_error_log_out eval +qr/f begin +f begin +f begin +worker cycle +e?poll timer: 0 +f middle +f middle +f middle +worker cycle +e?poll timer: 0 +f end +f end +f end +worker cycle +e?poll timer: 0 +/ From c1317e412f98c3de6502f3f4be6f5b883971065d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 15 May 2017 15:52:32 -0700 Subject: [PATCH 094/848] travis-ci: we no longer want to cover nginx 1.9.15. --- .travis.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 359c8cc654..14ee63603c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,6 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.9.15 - NGINX_VERSION=1.11.2 services: From 0c297939930b2f391738a28eca30b1bca015fffa Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 18 May 2017 10:00:04 -0700 Subject: [PATCH 095/848] doc: made it explicit that shdict methods are all atomic. --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index 24f10731da..e7808cd131 100644 --- a/README.markdown +++ b/README.markdown @@ -6199,6 +6199,8 @@ The resulting object `dict` has the following methods: * [flush_expired](#ngxshareddictflush_expired) * [get_keys](#ngxshareddictget_keys) +All these methods are *atomic* operations, that is, safe from concurrent accesses from multiple nginx worker processes for the same `lua_shared_dict` zone. + Here is an example: ```nginx diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 1e8d0295b8..84fd00045f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5197,6 +5197,8 @@ The resulting object dict has the following methods: * [[#ngx.shared.DICT.flush_expired|flush_expired]] * [[#ngx.shared.DICT.get_keys|get_keys]] +All these methods are ''atomic'' operations, that is, safe from concurrent accesses from multiple nginx worker processes for the same lua_shared_dict zone. + Here is an example: From 993aec8eb4e60320c47b5bcfac722339f481a301 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 22 May 2017 11:29:42 -0700 Subject: [PATCH 096/848] refactor: removed the obsolete util/gdbinit script script. we have much better tools in openresty-gdb-utils already. --- util/gdbinit | 415 --------------------------------------------------- 1 file changed, 415 deletions(-) delete mode 100644 util/gdbinit diff --git a/util/gdbinit b/util/gdbinit deleted file mode 100644 index 1508c64684..0000000000 --- a/util/gdbinit +++ /dev/null @@ -1,415 +0,0 @@ -# This gdb script provides several useful routines for debugging ngx_lua or -# standalone Lua/LuaJIT. -# -# You need gdb >= v7.3 to make this script working correctly. -# -# Installation: place it at $HOME/.gdbinit -# -# -- chaoslawful gmail com - -#### Lua type defines #### - -set $__LUA_TNONE = -1 -set $__LUA_TNIL = 0 -set $__LUA_TBOOLEAN = 1 -set $__LUA_TLIGHTUSERDATA = 2 -set $__LUA_TNUMBER = 3 -set $__LUA_TSTRING = 4 -set $__LUA_TTABLE = 5 -set $__LUA_TFUNCTION = 6 -set $__LUA_TUSERDATA = 7 -set $__LUA_TTHREAD = 8 - -#### Lua constants #### - -set $__LUA_GLOBALSINDEX = -10002 -set $__LUA_ENVIRONINDEX = -10001 -set $__LUA_REGISTRYINDEX = -10000 - -#### Auxiliary methods #### - -define __lua_debug_instance - if !$__lua_debug_instance - set $__lua_debug_instance = (lua_Debug*)malloc(sizeof(lua_Debug)) - end -end - -define __free_lua_debug_instance - if $__lua_debug_instance - set $rc = free($__lua_debug_instance) - set $__lua_debug_instance = 0 - end -end - -set $__BUCKET_SIZE = 16 -define __set_instance - if !$__set_instance - set $__set_instance = (void*(*(*))[2])malloc($__BUCKET_SIZE*sizeof(void*(*)[2])) - set $rc = memset($__set_instance, 0, $__BUCKET_SIZE*sizeof(void*(*)[2])) - end -end - -define __free_set_instance - if $__set_instance - __set_clean - set $rc = free($__set_instance) - set $__set_instance = 0 - end -end - -define __set_add - set $p = (void*)$arg0 - set $__bkt_idx = (int)$p%$__BUCKET_SIZE - - __set_instance - set $__elem = (void*(*)[2])$__set_instance[$__bkt_idx] - set $__found = 0 - while $__elem - if (*$__elem)[0] == $p - set $__found = 1 - loop_break - end - set $__elem = (void*(*)[2])(*$__elem)[1] - end - if $__found - set $existed_in_set = 1 - else - set $existed_in_set = 0 - - set $rc = (void*(*)[2])calloc(1, sizeof(void*)*2) - set (*$rc)[0] = $p - set (*$rc)[1] = $__set_instance[$__bkt_idx] - set $__set_instance[$__bkt_idx] = $rc - end -end - -define __set_is_exist - set $p = (void*)$arg0 - set $__bkt_idx = (int)$p%$__BUCKET_SIZE - - __set_instance - set $__elem = (void*(*)[2])$__set_instance[$__bkt_idx] - set $__found = 0 - while $__elem - if (*$__elem)[0] == $p - set $__found = 1 - loop_break - end - set $__elem = (void*(*)[2])(*$__elem)[1] - end - if $__found - set $existed_in_set = 1 - else - set $existed_in_set = 0 - end -end - -define __set_clean - __set_instance - - set $__bkt_idx = 0 - while $__bkt_idx < $__BUCKET_SIZE - set $__elem = (void*(*)[2])$__set_instance[$__bkt_idx] - while $__elem - set $__next = (void*(*)[2])(*$__elem)[1] - set $rc = free($__elem) - set $__elem = $__next - end - set $__set_instance[$__bkt_idx] = 0 - set $__bkt_idx = $__bkt_idx+1 - end -end - -define hook-quit - __free_lua_debug_instance - __free_set_instance -end - -define hook-detach - __free_lua_debug_instance - __free_set_instance -end - -define hook-disconnect - __free_lua_debug_instance - __free_set_instance -end - -define _lua_pop - set $l = (lua_State*)$arg0 - set $_n = (int)$arg1 - set $_rc = lua_settop($l, -$_n-1) -end - -define _lua_dump_locals - set $l = (lua_State*)$arg0 - set $dbg = (lua_Debug*)$arg1 - set $idx = 1 - - set $rc = lua_getlocal($l, $dbg, $idx) - if $rc - printf "\t----[[ Locals ]]----\n" - while $rc - printf "\t%d:\t'%s' = ", $idx, $rc - __lua_dump_stack $l -1 - printf "\n" - - _lua_pop $l 1 - set $idx = $idx + 1 - set $rc = lua_getlocal($l, $dbg, $idx) - end - else - printf "\tNo locals!\n" - end - printf "\n" -end - -define _lua_dump_upvalues - set $l = (lua_State*)$arg0 - set $dbg = (lua_Debug*)$arg1 - set $idx = 1 - - set $rc = lua_getinfo($l, "f", $dbg) - if $rc - set $rc = lua_getupvalue($l, -1, $idx) - if $rc - printf "\t----[[ Upvalues ]]----\n" - while $rc - printf "\t%d:\t'%s' = ", $idx, $rc - __lua_dump_stack $l -1 - printf "\n" - - _lua_pop $l 1 - set $idx = $idx + 1 - set $rc = lua_getupvalue($l, -1, $idx) - end - else - printf "\tNo upvalues!\n" - end - _lua_pop $l 1 - else - printf "\tFailed to get function closure!\n" - end - printf "\n" -end - -define __lua_dump_stack - __set_clean - __lua_dump_stack_aux $arg0 $arg1 0 -end - -define __lua_dump_stack_aux - set $l = (lua_State*)$arg0 - set $nidx_$arg2 = (int)$arg1 - set $cidx_$arg2 = (int)$arg2+1 - - # relative stack index to absolute index - if $nidx_$arg2 < 0 && $nidx_$arg2 > $__LUA_REGISTRYINDEX - set $nidx_$arg2 = $nidx_$arg2 + (int)lua_gettop($l) + 1 - end - - set $vt_$arg2 = (int)lua_type($l, $nidx_$arg2) - - if $vt_$arg2 == $__LUA_TNONE - echo - end - if $vt_$arg2 == $__LUA_TNIL - echo (nil) - end - if $vt_$arg2 == $__LUA_TBOOLEAN - printf "(bool) %d", lua_toboolean($l, $nidx_$arg2) - end - if $vt_$arg2 == $__LUA_TLIGHTUSERDATA - printf "(ludata) %p", lua_touserdata($l, $nidx_$arg2) - end - if $vt_$arg2 == $__LUA_TNUMBER - printf "%g", lua_tonumber($l, $nidx_$arg2) - end - if $vt_$arg2 == $__LUA_TSTRING - set $tmplen = (size_t*)malloc(sizeof(size_t)) - set $tmp = lua_pushvalue($l, $nidx_$arg2) - set $tmp = lua_tolstring($l, -1, $tmplen) -#printf "(string:%d) ", *$tmplen - eval "output/r *(const char (*)[%d])$tmp", *$tmplen - _lua_pop $l 1 - set $tmp = free($tmplen) - end - if $vt_$arg2 == $__LUA_TTABLE - set $rc = lua_topointer($l, $nidx_$arg2) -#printf "(table) %p { ", $rc - printf "{ " - __set_add $rc - if $existed_in_set - printf "... " - else - set $rc = lua_pushnil($l) - set $rc = lua_next($l, $nidx_$arg2) - while $rc != 0 - printf "[" - __lua_dump_stack_aux $l -2 $cidx_$arg2 - printf "]" - printf " = " - __lua_dump_stack_aux $l -1 $cidx_$arg2 - printf ", " - _lua_pop $l 1 - set $rc = lua_next($l, $nidx_$arg2) - end - end - printf "}" - end - if $vt_$arg2 == $__LUA_TFUNCTION - printf "(func) %p", lua_topointer($l, $nidx_$arg2) - end - if $vt_$arg2 == $__LUA_TUSERDATA - printf "(udata) %p", lua_topointer($l, $nidx_$arg2) - end - if $vt_$arg2 == $__LUA_TTHREAD - printf "(thread) %p", lua_topointer($l, $nidx_$arg2) - else - if $vt_$arg2 > $__LUA_TTHREAD || $vt_$arg2 < 0 - echo - end - end -end - -#### Command methods #### - -define lbt - if $argc < 1 - echo Please specify Lua state and/or dump flag!\n - else - set $l = (lua_State*)$arg0 - if $argc > 1 - set $dump_local = ($arg1&1)==1 - set $dump_upvalue = ($arg1&2)==2 - else - set $dump_local = 0 - set $dump_upvalue = 0 - end - - __lua_debug_instance - set $dbg = $__lua_debug_instance - - set $level = 0 - set $rc = lua_getstack($l, $level, $dbg) - while $rc > 0 - set $rc = lua_getinfo($l, "Sln", $dbg) - set $name = $dbg->name - if !$name - set $name = "???" - end - - printf "#%d\t%s\t[%s]\tat %s:%d\n", $level, $name, $dbg->what, $dbg->source, $dbg->currentline - - if $dump_local - _lua_dump_locals $l $dbg - end - if $dump_upvalue - _lua_dump_upvalues $l $dbg - end - - set $level = $level+1 - set $rc = lua_getstack($l, $level, $dbg) - end - end -end - -document lbt -lbt []: Dump the backtrace of the specified Lua state. is a mask value, whose bit 1/2 controls the dump of locals/upvalues at each stack frame correspondingly. So set to 1 dumps only locals; set to 2 dumps only upvalues; and set to 3 dumps both locals and upvalues. -end - -define ll - if $argc != 2 - echo Please specify Lua state and stack frame number (0-based)!\n - else - set $l = (lua_State*)$arg0 - set $level = (int)$arg1 - - __lua_debug_instance - set $dbg = $__lua_debug_instance - - set $rc = lua_getstack($l, $level, $dbg) - if $rc > 0 - _lua_dump_locals $l $dbg - else - echo Failed to get Lua stack frame!\n - end - end -end - -document ll -ll : Dump all local vars in the specified Lua stack frame (0-based). -end - -define lu - if $argc != 2 - echo Please specify Lua state and stack frame number (0-based)!\n - else - set $l = (lua_State*)$arg0 - set $level = (int)$arg1 - - __lua_debug_instance - set $dbg = $__lua_debug_instance - - set $rc = lua_getstack($l, $level, $dbg) - if $rc > 0 - _lua_dump_upvalues $l $dbg - else - echo Failed to get Lua stack frame!\n - end - end -end - -document lu -lu : Dump all upvalues in the specified Lua stack frame (0-based). -end - -define lg - if $argc != 1 - echo Please specify Lua state!\n - else - set $l = (lua_State*)$arg0 - __lua_dump_stack $l $__LUA_GLOBALSINDEX - printf "\n" - end -end - -document lg -lg : Dump all entries in Lua global table. -end - -define lr - if $argc != 1 - echo Please specify Lua state!\n - else - set $l = (lua_State*)$arg0 - __lua_dump_stack $l $__LUA_REGISTRYINDEX - printf "\n" - end -end - -document lr -lr : Dump all entries in Lua registry table. -end - -define ls - if $argc != 1 - echo Please specify Lua state!\n - else - set $l = (lua_State*)$arg0 - set $idx = lua_gettop($l) - while $idx >= 1 - printf "#%d ", $idx - __lua_dump_stack $l $idx - printf "\n" - set $idx = $idx - 1 - end - end -end - -document ls -ls : Dump all entries in call stack. -end - -# vi:ft=gdb ts=4 sw=4 - From e80d2925717d57dffadff3b5ec76519467e7494b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 23 May 2017 22:23:28 -0700 Subject: [PATCH 097/848] tests: ssl: avoided those domains using startcom ssl certificates. --- t/129-ssl-socket.t | 219 ++++++++++++++++++++++--------------------- t/cert/comodo-ca.crt | 29 ++++++ t/cert/startcom.crt | 87 ----------------- 3 files changed, 139 insertions(+), 196 deletions(-) create mode 100644 t/cert/comodo-ca.crt delete mode 100644 t/cert/startcom.crt diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 726b4423c9..928701f521 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -26,7 +26,7 @@ sub read_file { $cert; } -our $StartComRootCertificate = read_file("t/cert/startcom.crt"); +our $ComodoRootCertificate = read_file("t/cert/comodo-ca.crt"); our $EquifaxRootCertificate = read_file("t/cert/equifax.crt"); our $TestCertificate = read_file("t/cert/test.crt"); our $TestCertificateKey = read_file("t/cert/test.key"); @@ -135,7 +135,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("g.sregex.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -151,7 +151,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: g.sregex.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -180,8 +180,8 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 57 bytes. -received: HTTP/1.1 401 Unauthorized +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -207,12 +207,12 @@ SSL reused session #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; - content_by_lua ' + content_by_lua_block { local sock = ngx.socket.tcp() sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -220,7 +220,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -228,7 +228,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\r\nHost: openresty.org\r\nConnection: close\r\n\r\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -249,7 +249,7 @@ SSL reused session ngx.say("close: ", ok, " ", err) end -- do collectgarbage() - '; + } } --- request @@ -257,8 +257,8 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -268,7 +268,7 @@ qr/^lua ssl save session: ([0-9A-F]+):2 lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] @@ -281,6 +281,7 @@ SSL reused session --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; + lua_ssl_protocols TLSv1.2; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -293,7 +294,7 @@ SSL reused session local session for i = 1, 2 do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("agentzh.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -301,7 +302,7 @@ SSL reused session ngx.say("connected: ", ok) - session, err = sock:sslhandshake(session, "iscribblet.org") + session, err = sock:sslhandshake(session, "agentzh.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -309,7 +310,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: agentzh.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -340,12 +341,12 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. +sent http request: 56 bytes. received: HTTP/1.1 200 OK close: 1 nil connected: 1 ssl handshake: userdata -sent http request: 59 bytes. +sent http request: 56 bytes. received: HTTP/1.1 200 OK close: 1 nil @@ -371,7 +372,7 @@ lua ssl free session === TEST 5: certificate does not match host name (verify) -The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.org". +The certificate of "openresty.org" does not contain the name "blah.openresty.org". --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -386,7 +387,7 @@ The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.o sock:settimeout(2000) do - local ok, err = sock:connect("agentzh.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -394,14 +395,14 @@ The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.o ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "blah.agentzh.org", true) + local session, err = sock:sslhandshake(nil, "blah.openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) else ngx.say("ssl handshake: ", type(session)) end - local req = "GET / HTTP/1.1\\r\\nHost: agentzh.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -427,7 +428,7 @@ The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.o --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -440,8 +441,8 @@ failed to send http request: closed --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ --- grep_error_log_out --- error_log -lua ssl server name: "blah.agentzh.org" -lua ssl certificate does not match host "blah.agentzh.org" +lua ssl server name: "blah.openresty.org" +lua ssl certificate does not match host "blah.openresty.org" --- no_error_log SSL reused session [alert] @@ -450,7 +451,7 @@ SSL reused session === TEST 6: certificate does not match host name (verify, no log socket errors) -The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.org". +The certificate for "openresty.org" does not contain the name "blah.openresty.org". --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -466,7 +467,7 @@ The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.o sock:settimeout(2000) do - local ok, err = sock:connect("agentzh.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -474,14 +475,14 @@ The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.o ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "blah.agentzh.org", true) + local session, err = sock:sslhandshake(nil, "blah.openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) else ngx.say("ssl handshake: ", type(session)) end - local req = "GET / HTTP/1.1\\r\\nHost: blah.agentzh.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: blah.openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -507,7 +508,7 @@ The certificate for "blah.agentzh.org" does not contain the name "blah.agentzh.o --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -520,7 +521,7 @@ failed to send http request: closed --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ --- grep_error_log_out --- error_log -lua ssl server name: "blah.agentzh.org" +lua ssl server name: "blah.openresty.org" --- no_error_log lua ssl certificate does not match host SSL reused session @@ -542,7 +543,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("agentzh.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -550,7 +551,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "agentzh.org", false) + local session, err = sock:sslhandshake(nil, "openresty.org", false) if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -558,7 +559,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: agentzh.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -587,8 +588,8 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 56 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -599,7 +600,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "agentzh.org" +lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] @@ -608,7 +609,7 @@ SSL reused session -=== TEST 8: iscribblet.org: passing SSL verify +=== TEST 8: openresty.org: passing SSL verify --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -623,7 +624,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -631,7 +632,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org", true) + local session, err = sock:sslhandshake(nil, "openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -639,7 +640,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -665,15 +666,15 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -684,7 +685,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] @@ -708,7 +709,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -716,14 +717,14 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org", true) + local session, err = sock:sslhandshake(nil, "openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) else ngx.say("ssl handshake: ", type(session)) end - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -749,7 +750,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -762,7 +763,7 @@ failed to send http request: closed --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ --- grep_error_log_out --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" lua ssl certificate verify error: (20: unable to get local issuer certificate) --- no_error_log SSL reused session @@ -787,7 +788,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -795,14 +796,14 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org", true) + local session, err = sock:sslhandshake(nil, "openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) else ngx.say("ssl handshake: ", type(session)) end - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -828,7 +829,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -841,7 +842,7 @@ failed to send http request: closed --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ --- grep_error_log_out --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" --- no_error_log lua ssl certificate verify error SSL reused session @@ -1010,7 +1011,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -1030,7 +1031,7 @@ SSL reused session -=== TEST 13: iscribblet.org: passing SSL verify with multiple certificates +=== TEST 13: openresty.org: passing SSL verify with multiple certificates --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1045,7 +1046,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1053,7 +1054,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org", true) + local session, err = sock:sslhandshake(nil, "openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1061,7 +1062,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1088,15 +1089,15 @@ SSL reused session --- user_files eval ">>> trusted.crt $::EquifaxRootCertificate -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -1107,7 +1108,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] @@ -1129,7 +1130,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1137,7 +1138,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1145,7 +1146,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1174,8 +1175,8 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -1185,8 +1186,8 @@ qr/^lua ssl save session: ([0-9A-F]+):2 lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" -SSL: TLSv1.2, cipher: "ECDHE-RSA-RC4-SHA SSLv3 +lua ssl server name: "openresty.org" +SSL: TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 --- no_error_log SSL reused session [error] @@ -1199,7 +1200,7 @@ SSL reused session --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; - lua_ssl_ciphers RC4-SHA; + lua_ssl_ciphers ECDHE-RSA-AES256-SHA; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1209,7 +1210,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1217,7 +1218,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1225,7 +1226,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1254,8 +1255,8 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -1265,8 +1266,8 @@ qr/^lua ssl save session: ([0-9A-F]+):2 lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" -SSL: TLSv1.2, cipher: "RC4-SHA SSLv3 +lua ssl server name: "openresty.org" +SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-SHA --- no_error_log SSL reused session [error] @@ -1289,7 +1290,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1297,7 +1298,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1305,7 +1306,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1334,8 +1335,8 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -1345,8 +1346,8 @@ qr/^lua ssl save session: ([0-9A-F]+):2 lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" -SSL: TLSv1, cipher: "ECDHE-RSA-RC4-SHA SSLv3 +lua ssl server name: "openresty.org" +SSL: TLSv1, cipher: "ECDHE-RSA-AES128-SHA --- no_error_log SSL reused session [error] @@ -1370,7 +1371,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1378,14 +1379,14 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) else ngx.say("ssl handshake: ", type(session)) end - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1422,7 +1423,7 @@ failed to send http request: closed --- error_log eval [ qr/\[crit\] .*?SSL_do_handshake\(\) failed .*?unsupported protocol/, -'lua ssl server name: "iscribblet.org"', +'lua ssl server name: "openresty.org"', ] --- no_error_log SSL reused session @@ -1432,7 +1433,7 @@ SSL reused session -=== TEST 18: iscribblet.org: passing SSL verify: keepalive (reuse the ssl session) +=== TEST 18: openresty.org: passing SSL verify: keepalive (reuse the ssl session) --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1450,7 +1451,7 @@ SSL reused session local session for i = 1, 3 do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1458,7 +1459,7 @@ SSL reused session ngx.say("connected: ", ok) - session, err = sock:sslhandshake(session, "iscribblet.org", true) + session, err = sock:sslhandshake(session, "openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1477,7 +1478,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -1509,7 +1510,7 @@ SSL reused session -=== TEST 19: iscribblet.org: passing SSL verify: keepalive (no reusing the ssl session) +=== TEST 19: openresty.org: passing SSL verify: keepalive (no reusing the ssl session) --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1526,7 +1527,7 @@ SSL reused session do for i = 1, 3 do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1534,7 +1535,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "iscribblet.org", true) + local session, err = sock:sslhandshake(nil, "openresty.org", true) if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1553,7 +1554,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request GET /t @@ -1612,7 +1613,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::StartComRootCertificate" +$::ComodoRootCertificate" --- request POST /t @@ -2042,7 +2043,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -2051,7 +2052,7 @@ SSL reused session ngx.say("connected: ", ok) for i = 1, 2 do - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -2060,7 +2061,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) end - local req = "GET / HTTP/1.1\\r\\nHost: iscribblet.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -2090,8 +2091,8 @@ GET /t connected: 1 ssl handshake: userdata ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 200 OK +sent http request: 58 bytes. +received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug @@ -2103,7 +2104,7 @@ lua ssl free session: ([0-9A-F]+):2 lua ssl free session: ([0-9A-F]+):1 $/ --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] @@ -2126,7 +2127,7 @@ SSL reused session sock:settimeout(2000) do - local ok, err = sock:connect("iscribblet.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -2135,7 +2136,7 @@ SSL reused session ngx.say("connected: ", ok) sock:settimeout(1); -- should timeout immediately - local session, err = sock:sslhandshake(nil, "iscribblet.org") + local session, err = sock:sslhandshake(nil, "openresty.org") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -2157,7 +2158,7 @@ failed to do SSL handshake: timeout --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ --- grep_error_log_out --- error_log -lua ssl server name: "iscribblet.org" +lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] @@ -2598,7 +2599,7 @@ SSL reused session local sock = ngx.socket.tcp() sock:settimeout(2000) - local ok, err = sock:connect("g.sregex.org", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/cert/comodo-ca.crt b/t/cert/comodo-ca.crt new file mode 100644 index 0000000000..444461b95a --- /dev/null +++ b/t/cert/comodo-ca.crt @@ -0,0 +1,29 @@ +-----BEGIN CERTIFICATE----- +MIIF2DCCA8CgAwIBAgIQTKr5yttjb+Af907YWwOGnTANBgkqhkiG9w0BAQwFADCBhTELMAkGA1UE +BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgG +A1UEChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNVBAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNhdGlv +biBBdXRob3JpdHkwHhcNMTAwMTE5MDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBhTELMAkGA1UEBhMC +R0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UE +ChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNVBAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNhdGlvbiBB +dXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCR6FSS0gpWsawNJN3Fz0Rn +dJkrN6N9I3AAcbxT38T6KhKPS38QVr2fcHK3YX/JSw8Xpz3jsARh7v8Rl8f0hj4K+j5c+ZPmNHrZ +FGvnnLOFoIJ6dq9xkNfs/Q36nGz637CC9BR++b7Epi9Pf5l/tfxnQ3K9DADWietrLNPtj5gcFKt+ +5eNu/Nio5JIk2kNrYrhV/erBvGy2i/MOjZrkm2xpmfh4SDBF1a3hDTxFYPwyllEnvGfDyi62a+pG +x8cgoLEfZd5ICLqkTqnyg0Y3hOvozIFIQ2dOciqbXL1MGyiKXCJ7tKuY2e7gUYPDCUZObT6Z+pUX +2nwzV0E8jVHtC7ZcryxjGt9XyD+86V3Em69FmeKjWiS0uqlWPc9vqv9JWL7wqP/0uK3pN/u6uPQL +OvnoQ0IeidiEyxPx2bvhiWC4jChWrBQdnArncevPDt09qZahSL0896+1DSJMwBGB7FY79tOi4lu3 +sgQiUpWAk2nojkxl8ZEDLXB0AuqLZxUpaVICu9ffUGpVRr+goyhhf3DQw6KqLCGqR84onAZFdr+C +GCe01a60y1Dma/RMhnEw6abfFobg2P9A3fvQQoh/ozM6LlweQRGBY84YcWsr7KaKtzFcOmpH4MN5 +WdYgGq/yapiqcrxXStJLnbsQ/LBMQeXtHT1eKJ2czL+zUdqnR+WEUwIDAQABo0IwQDAdBgNVHQ4E +FgQUu69+Aj36pvE8hI6t7jiY7NkyMtQwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8w +DQYJKoZIhvcNAQEMBQADggIBAArx1UaEt65Ru2yyTUEUAJNMnMvlwFTPoCWOAvn9sKIN9SCYPBMt +rFaisNZ+EZLpLrqeLppysb0ZRGxhNaKatBYSaVqM4dc+pBroLwP0rmEdEBsqpIt6xf4FpuHA1sj+ +nq6PK7o9mfjYcwlYRm6mnPTXJ9OV2jeDchzTc+CiR5kDOF3VSXkAKRzH7JsgHAckaVd4sjn8OoSg +tZx8jb8uk2IntznaFxiuvTwJaP+EmzzV1gsD41eeFPfR60/IvYcjt7ZJQ3mFXLrrkguhxuhoqEwW +sRqZCuhTLJK7oQkYdQxlqHvLI7cawiiFwxv/0Cti76R7CZGYZ4wUAc1oBmpjIXUDgIiKboHGhfKp +pC3n9KUkEEeDys30jXlYsQab5xoq2Z0B15R97QNKyvDb6KkBPvVWmckejkk9u+UJueBPSZI9FoJA +zMxZxuY67RIuaTxslbH9qh17f4a+Hg4yRvv7E491f0yLS0Zj/gA0QHDBw7mh3aZw4gSzQbzpgJHq +ZJx64SIDqZxubw5lT2yHh17zbqD5daWbQOhTsiedSrnAdyGN/4fy3ryM7xfft0kL0fJuMAsaDk52 +7RH89elWsn2/x20Kk4yl0MC2Hb46TpSi125sC8KKfPog88Tk5c0NqMuRkrF8hey1FGlmDoLnzc7I +LaZRfyHBNVOFBkpdn627G190 +-----END CERTIFICATE----- diff --git a/t/cert/startcom.crt b/t/cert/startcom.crt deleted file mode 100644 index a5185ca100..0000000000 --- a/t/cert/startcom.crt +++ /dev/null @@ -1,87 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHhzCCBW+gAwIBAgIBLTANBgkqhkiG9w0BAQsFADB9MQswCQYDVQQGEwJJTDEW -MBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwg -Q2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3RhcnRDb20gQ2VydGlmaWNh -dGlvbiBBdXRob3JpdHkwHhcNMDYwOTE3MTk0NjM3WhcNMzYwOTE3MTk0NjM2WjB9 -MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi -U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh -cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUA -A4ICDwAwggIKAoICAQDBiNsJvGxGfHiflXu1M5DycmLWwTYgIiRezul38kMKogZk -pMyONvg45iPwbm2xPN1yo4UcodM9tDMr0y+v/uqwQVlntsQGfQqedIXWeUyAN3rf -OQVSWff0G0ZDpNKFhdLDcfN1YjS6LIp/Ho/u7TTQEceWzVI9ujPW3U3eCztKS5/C -Ji/6tRYccjV3yjxd5srhJosaNnZcAdt0FCX+7bWgiA/deMotHweXMAEtcnn6RtYT -Kqi5pquDSR3l8u/d5AGOGAqPY1MWhWKpDhk6zLVmpsJrdAfkK+F2PrRt2PZE4XNi -HzvEvqBTViVsUQn3qqvKv3b9bZvzndu/PWa8DFaqr5hIlTpL36dYUNk4dalb6kMM -Av+Z6+hsTXBbKWWc3apdzK8BMewM69KN6Oqce+Zu9ydmDBpI125C4z/eIT574Q1w -+2OqqGwaVLRcJXrJosmLFqa7LH4XXgVNWG4SHQHuEhANxjJ/GP/89PrNbpHoNkm+ -Gkhpi8KWTRoSsmkXwQqQ1vp5Iki/untp+HDH+no32NgN0nZPV/+Qt+OR0t3vwmC3 -Zzrd/qqc8NSLf3Iizsafl7b4r4qgEKjZ+xjGtrVcUjyJthkqcwEKDwOzEmDyei+B -26Nu/yYwl/WL3YlXtq09s68rxbd2AvCl1iuahhQqcvbjM4xdCUsT37uMdBNSSwID -AQABo4ICEDCCAgwwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwHQYD -VR0OBBYEFE4L7xqkQFulF2mHMMo0aEPQQa7yMB8GA1UdIwQYMBaAFE4L7xqkQFul -F2mHMMo0aEPQQa7yMIIBWgYDVR0gBIIBUTCCAU0wggFJBgsrBgEEAYG1NwEBATCC -ATgwLgYIKwYBBQUHAgEWImh0dHA6Ly93d3cuc3RhcnRzc2wuY29tL3BvbGljeS5w -ZGYwNAYIKwYBBQUHAgEWKGh0dHA6Ly93d3cuc3RhcnRzc2wuY29tL2ludGVybWVk -aWF0ZS5wZGYwgc8GCCsGAQUFBwICMIHCMCcWIFN0YXJ0IENvbW1lcmNpYWwgKFN0 -YXJ0Q29tKSBMdGQuMAMCAQEagZZMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUg -c2VjdGlvbiAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0 -aWZpY2F0aW9uIEF1dGhvcml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly93 -d3cuc3RhcnRzc2wuY29tL3BvbGljeS5wZGYwEQYJYIZIAYb4QgEBBAQDAgAHMDgG -CWCGSAGG+EIBDQQrFilTdGFydENvbSBGcmVlIFNTTCBDZXJ0aWZpY2F0aW9uIEF1 -dGhvcml0eTANBgkqhkiG9w0BAQsFAAOCAgEAjo/n3JR5fPGFf59Jb2vKXfuM/gTF -wWLRfUKKvFO3lANmMD+x5wqnUCBVJX92ehQN6wQOQOY+2IirByeDqXWmN3PH/UvS -Ta0XQMhGvjt/UfzDtgUx3M2FIk5xt/JxXrAaxrqTi3iSSoX4eA+D/i+tLPfkpLst -0OcNOrg+zvZ49q5HJMqjNTbOx8aHmNrs++myziebiMMEofYLWWivydsQD032ZGNc -pRJvkrKTlMeIFw6Ttn5ii5B/q06f/ON1FE8qMt9bDeD1e5MNq6HPh+GlBEXoPBKl -CcWw0bdT82AUuoVpaiF8H3VhFyAXe2w7QSlc4axa0c2Mm+tgHRns9+Ww2vl5GKVF -P0lDV9LdJNUso/2RjSe15esUBppMeyG7Oq0wBhjA2MFrLH9ZXF2RsXAiV+uKa0hK -1Q8p7MZAwC+ITGgBF3f0JBlPvfrhsiAhS90a2Cl9qrjeVOwhVYBsHvUwyKMQ5bLm -KhQxw4UtjJixhlpPiVktucf3HMiKf8CdBUrmQk9io20ppB+Fq9vlgcitKj1MXVuE -JnHEhV5xJMqlG2zYYdMa4FTbzrqpMrUi9nNBCV24F10OD5mQ1kfabwo6YigUZ4LZ -8dCAWZvLMdibD4x3TrVoivJs9iQOLWxwxXPR3hTQcY+203sC9uO41Alua551hDnm -fyWl8kgAwKQB2j8= ------END CERTIFICATE----- ------BEGIN CERTIFICATE----- -MIIHyTCCBbGgAwIBAgIBATANBgkqhkiG9w0BAQUFADB9MQswCQYDVQQGEwJJTDEW -MBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwg -Q2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3RhcnRDb20gQ2VydGlmaWNh -dGlvbiBBdXRob3JpdHkwHhcNMDYwOTE3MTk0NjM2WhcNMzYwOTE3MTk0NjM2WjB9 -MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi -U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh -cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUA -A4ICDwAwggIKAoICAQDBiNsJvGxGfHiflXu1M5DycmLWwTYgIiRezul38kMKogZk -pMyONvg45iPwbm2xPN1yo4UcodM9tDMr0y+v/uqwQVlntsQGfQqedIXWeUyAN3rf -OQVSWff0G0ZDpNKFhdLDcfN1YjS6LIp/Ho/u7TTQEceWzVI9ujPW3U3eCztKS5/C -Ji/6tRYccjV3yjxd5srhJosaNnZcAdt0FCX+7bWgiA/deMotHweXMAEtcnn6RtYT -Kqi5pquDSR3l8u/d5AGOGAqPY1MWhWKpDhk6zLVmpsJrdAfkK+F2PrRt2PZE4XNi -HzvEvqBTViVsUQn3qqvKv3b9bZvzndu/PWa8DFaqr5hIlTpL36dYUNk4dalb6kMM -Av+Z6+hsTXBbKWWc3apdzK8BMewM69KN6Oqce+Zu9ydmDBpI125C4z/eIT574Q1w -+2OqqGwaVLRcJXrJosmLFqa7LH4XXgVNWG4SHQHuEhANxjJ/GP/89PrNbpHoNkm+ -Gkhpi8KWTRoSsmkXwQqQ1vp5Iki/untp+HDH+no32NgN0nZPV/+Qt+OR0t3vwmC3 -Zzrd/qqc8NSLf3Iizsafl7b4r4qgEKjZ+xjGtrVcUjyJthkqcwEKDwOzEmDyei+B -26Nu/yYwl/WL3YlXtq09s68rxbd2AvCl1iuahhQqcvbjM4xdCUsT37uMdBNSSwID -AQABo4ICUjCCAk4wDAYDVR0TBAUwAwEB/zALBgNVHQ8EBAMCAa4wHQYDVR0OBBYE -FE4L7xqkQFulF2mHMMo0aEPQQa7yMGQGA1UdHwRdMFswLKAqoCiGJmh0dHA6Ly9j -ZXJ0LnN0YXJ0Y29tLm9yZy9zZnNjYS1jcmwuY3JsMCugKaAnhiVodHRwOi8vY3Js -LnN0YXJ0Y29tLm9yZy9zZnNjYS1jcmwuY3JsMIIBXQYDVR0gBIIBVDCCAVAwggFM -BgsrBgEEAYG1NwEBATCCATswLwYIKwYBBQUHAgEWI2h0dHA6Ly9jZXJ0LnN0YXJ0 -Y29tLm9yZy9wb2xpY3kucGRmMDUGCCsGAQUFBwIBFilodHRwOi8vY2VydC5zdGFy -dGNvbS5vcmcvaW50ZXJtZWRpYXRlLnBkZjCB0AYIKwYBBQUHAgIwgcMwJxYgU3Rh -cnQgQ29tbWVyY2lhbCAoU3RhcnRDb20pIEx0ZC4wAwIBARqBl0xpbWl0ZWQgTGlh -YmlsaXR5LCByZWFkIHRoZSBzZWN0aW9uICpMZWdhbCBMaW1pdGF0aW9ucyogb2Yg -dGhlIFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5IFBvbGljeSBhdmFp -bGFibGUgYXQgaHR0cDovL2NlcnQuc3RhcnRjb20ub3JnL3BvbGljeS5wZGYwEQYJ -YIZIAYb4QgEBBAQDAgAHMDgGCWCGSAGG+EIBDQQrFilTdGFydENvbSBGcmVlIFNT -TCBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTANBgkqhkiG9w0BAQUFAAOCAgEAFmyZ -9GYMNPXQhV59CuzaEE44HF7fpiUFS5Eyweg78T3dRAlbB0mKKctmArexmvclmAk8 -jhvh3TaHK0u7aNM5Zj2gJsfyOZEdUauCe37Vzlrk4gNXcGmXCPleWKYK34wGmkUW -FjgKXlf2Ysd6AgXmvB618p70qSmD+LIU424oh0TDkBreOKk8rENNZEXO3SipXPJz -ewT4F+irsfMuXGRuczE6Eri8sxHkfY+BUZo7jYn0TZNmezwD7dOaHZrzZVD1oNB1 -ny+v8OqCQ5j4aZyJecRDjkZy42Q2Eq/3JR44iZB3fsNrarnDy0RLrHiQi+fHLB5L -EUTINFInzQpdn4XBidUaePKVEFMy3YCEZnXZtWgo+2EuvoSoOMCZEoalHmdkrQYu -L6lwhceWD3yJZfWOQ1QOq92lgDmUYMA0yZZwLKMS9R9Ie70cfmu3nZD0Ijuu+Pwq -yvqCUqDvr0tVk+vBtfAii6w0TiYiBKGHLHVKt+V9E9e4DGTANtLJL4YSjCMJwRuC -O3NJo2pXh5Tl1njFmUNj403gdy3hZZlyaQQaRwnmDwFWJPsfvw55qVguucQJAX6V -um0ABj6y6koQOdjQK/W/7HW/lwLFCRsI3FU34oH7N4RDYiDK51ZLZer+bMEkkySh -NOsF/5oirpt9P/FlUQqmMGqz9IgcgA38corog14= ------END CERTIFICATE----- From 7a8798437abf3b36a67c280828354e9b759ad7e8 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 31 May 2017 09:54:03 +0800 Subject: [PATCH 098/848] feature: added pure C API function +ngx_http_lua_ffi_errlog_get_sys_filter_level for ngx.errlog module's get_sys_filter_level() function in the lua-resty-core library. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_log.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index 5fc79dd232..ab5fd9dd13 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -405,6 +405,28 @@ ngx_http_lua_ffi_errlog_get_msg(char **log, int *loglevel, u_char *err, #endif } + +int +ngx_http_lua_ffi_errlog_get_sys_filter_level(ngx_http_request_t *r) +{ + ngx_log_t *log; + int log_level; + + if (r && r->connection && r->connection->log) { + log = r->connection->log; + + } else { + log = ngx_cycle->log; + } + + log_level = log->log_level; + if (log_level == NGX_LOG_DEBUG_ALL) { + log_level = NGX_LOG_DEBUG; + } + + return log_level; +} + #endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From 69d995513b17df9ebb09dd4ea6656e1f78b28a34 Mon Sep 17 00:00:00 2001 From: guanglinlv Date: Sun, 4 Jun 2017 22:59:45 +0800 Subject: [PATCH 099/848] bugfix: the running timer counter might go out of sync when non-timer handlers using fake requests are involved (like ssl_certficate_by_lua* and ssl_session_fetch_by_lua*). Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_util.c | 6 ++- t/139-ssl-cert-by.t | 102 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 107 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index ba9c9579e8..d20549e73e 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -919,7 +919,11 @@ ngx_http_lua_request_cleanup(ngx_http_lua_ctx_t *ctx, int forcible) #if 1 if (r->connection->fd == (ngx_socket_t) -1) { /* being a fake request */ - lmcf->running_timers--; + + if (ctx->context == NGX_HTTP_LUA_CONTEXT_TIMER) { + /* being a timer handler */ + lmcf->running_timers--; + } } #endif diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index b9fd60d431..595e383c9c 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -1574,3 +1574,105 @@ qr/\[error\] .*? send\(\) failed/, --- no_error_log [alert] ssl_certificate_by_lua:1: ssl cert by lua is running! + + + +=== TEST 19: check the count of running timers +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { print("ssl cert by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /timers { + default_type 'text/plain'; + content_by_lua_block { + ngx.timer.at(0.1, function() ngx.sleep(0.3) end) + ngx.timer.at(0.11, function() ngx.sleep(0.3) end) + ngx.timer.at(0.09, function() ngx.sleep(0.3) end) + ngx.sleep(0.2) + ngx.say(ngx.timer.running_count()) + } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /timers HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 59 bytes. +received: HTTP/1.1 200 OK +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 2 +received: Connection: close +received: +received: 3 +close: 1 nil + +--- error_log eval +[ +'ssl_certificate_by_lua:1: ssl cert by lua is running!', +'lua ssl server name: "test.com"', +] +--- no_error_log +[error] +[alert] From 2360565f5fbc9c73ae6c02b8eca4f60a9af05191 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 14 Jun 2017 15:19:38 +0800 Subject: [PATCH 100/848] doc: fixed some typos detected by misspell. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 6 +++--- doc/HttpLuaModule.wiki | 6 +++--- src/ngx_http_lua_ssl_session_fetchby.c | 2 +- src/ngx_http_lua_ssl_session_storeby.c | 4 ++-- t/139-ssl-cert-by.t | 2 +- t/140-ssl-c-api.t | 4 ++-- 6 files changed, 12 insertions(+), 12 deletions(-) diff --git a/README.markdown b/README.markdown index e7808cd131..992aa28558 100644 --- a/README.markdown +++ b/README.markdown @@ -675,12 +675,12 @@ It is therefore *highly* recommended to always declare such within an appropriat -- Avoid foo = 123 - -- Recomended + -- Recommended local foo = 123 -- Avoid function foo() return 123 end - -- Recomended + -- Recommended local function foo() return 123 end ``` @@ -914,7 +914,7 @@ servers in Lua. For example, * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add `stat` mode similar to [mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html). -* cosocket: add client SSL certificiate support. +* cosocket: add client SSL certificate support. [Back to TOC](#table-of-contents) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 84fd00045f..259874bfa7 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -537,12 +537,12 @@ It is therefore *highly* recommended to always declare such within an appropriat -- Avoid foo = 123 - -- Recomended + -- Recommended local foo = 123 -- Avoid function foo() return 123 end - -- Recomended + -- Recommended local function foo() return 123 end @@ -745,7 +745,7 @@ servers in Lua. For example, * add ignore_resp_headers, ignore_resp_body, and ignore_resp options to [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add stat mode similar to [https://httpd.apache.org/docs/trunk/mod/mod_lua.html mod_lua]. -* cosocket: add client SSL certificiate support. +* cosocket: add client SSL certificate support. = Changes = diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 4c450b56ba..60728f29a4 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -107,7 +107,7 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, dd("enter"); - /* must specifiy a content handler */ + /* must specify a content handler */ if (cmd->post == NULL) { return NGX_CONF_ERROR; } diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index b5596bc71c..50adf2040c 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -105,7 +105,7 @@ ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, dd("enter"); - /* must specifiy a content handler */ + /* must specify a content handler */ if (cmd->post == NULL) { return NGX_CONF_ERROR; } @@ -386,7 +386,7 @@ ngx_http_lua_ssl_sess_store_by_chunk(lua_State *L, ngx_http_request_t *r) dd("rc == %d", (int) rc); if (rc != 0) { - /* error occured when running loaded code */ + /* error occurred when running loaded code */ err_msg = (u_char *) lua_tolstring(L, -1, &len); if (err_msg == NULL) { diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 595e383c9c..e6d0da8a66 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -1383,7 +1383,7 @@ uthread: done -=== TEST 17: simple logging - use ssl_certificiate_by_lua* on the http {} level +=== TEST 17: simple logging - use ssl_certificate_by_lua* on the http {} level GitHub openresty/lua-resty-core#42 --- http_config ssl_certificate_by_lua_block { print("ssl cert by lua is running!") } diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 854ff30aad..8734d1477d 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -626,7 +626,7 @@ failed to parse PEM priv key: PEM_read_bio_PrivateKey() failed while true do local line, err = sock:receive() if not line then - -- ngx.say("failed to recieve response status line: ", err) + -- ngx.say("failed to receive response status line: ", err) break end @@ -776,7 +776,7 @@ lua ssl server name: "test.com" while true do local line, err = sock:receive() if not line then - -- ngx.say("failed to recieve response status line: ", err) + -- ngx.say("failed to receive response status line: ", err) break end From 6f49242a8feeccecfed33b1225a9b2dc0ff16177 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 20 Jun 2017 12:43:12 -0700 Subject: [PATCH 101/848] fixed tests to reflect recent changes. --- t/129-ssl-socket.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 928701f521..a8871e5473 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1187,7 +1187,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log lua ssl server name: "openresty.org" -SSL: TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 +SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 --- no_error_log SSL reused session [error] @@ -1347,7 +1347,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log lua ssl server name: "openresty.org" -SSL: TLSv1, cipher: "ECDHE-RSA-AES128-SHA +SSL: TLSv1, cipher: "ECDHE-RSA-AES256-SHA --- no_error_log SSL reused session [error] From 9cb970978cd9a023606404987075ce671dd28d84 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 26 Jun 2017 12:12:52 -0700 Subject: [PATCH 102/848] tests: updated tests on ssl cosockets to reflect recent changes in our CDN network. --- t/129-ssl-socket.t | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index a8871e5473..530406d6ed 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 219; +plan tests => repeat_each() * 217; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -179,21 +179,14 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata -sent http request: 58 bytes. -received: HTTP/1.1 302 Moved Temporarily -close: 1 nil +failed to do SSL handshake: handshake failed --- log_level: debug --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ ---- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 -$/ +--- grep_error_log_out --- no_error_log lua ssl server name: SSL reused session -[error] [alert] --- timeout: 5 @@ -434,7 +427,7 @@ $::ComodoRootCertificate" GET /t --- response_body connected: 1 -failed to do SSL handshake: certificate host mismatch +failed to do SSL handshake: handshake failed failed to send http request: closed --- log_level: debug @@ -442,7 +435,6 @@ failed to send http request: closed --- grep_error_log_out --- error_log lua ssl server name: "blah.openresty.org" -lua ssl certificate does not match host "blah.openresty.org" --- no_error_log SSL reused session [alert] @@ -514,7 +506,7 @@ $::ComodoRootCertificate" GET /t --- response_body connected: 1 -failed to do SSL handshake: certificate host mismatch +failed to do SSL handshake: handshake failed failed to send http request: closed --- log_level: debug @@ -589,7 +581,7 @@ GET /t connected: 1 ssl handshake: userdata sent http request: 56 bytes. -received: HTTP/1.1 302 Moved Temporarily +received: HTTP/1.1 404 Not Found close: 1 nil --- log_level: debug From 031e0000429e18d3220c0d530fa03b9ba20d9489 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 26 Jun 2017 14:46:20 -0700 Subject: [PATCH 103/848] doc: use the .ljbc file extension instead of .luac for LuaJIT bytecode files. --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 992aa28558..c30944d0ce 100644 --- a/README.markdown +++ b/README.markdown @@ -413,14 +413,14 @@ Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible wi ```bash - /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.luac + /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc ``` The `-bg` option can be used to include debug information in the LuaJIT bytecode file: ```bash - /path/to/luajit/bin/luajit -bg /path/to/input_file.lua /path/to/output_file.luac + /path/to/luajit/bin/luajit -bg /path/to/input_file.lua /path/to/output_file.ljbc ``` Please refer to the official LuaJIT documentation on the `-b` option for more details: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 259874bfa7..ef8ecc3b81 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -315,13 +315,13 @@ As from the v0.5.0rc32 release, all *_by_lua_file conf Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: - /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.luac + /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc The -bg option can be used to include debug information in the LuaJIT bytecode file: - /path/to/luajit/bin/luajit -bg /path/to/input_file.lua /path/to/output_file.luac + /path/to/luajit/bin/luajit -bg /path/to/input_file.lua /path/to/output_file.ljbc Please refer to the official LuaJIT documentation on the -b option for more details: From df708216cf71082e94a96792306c8cc9d0105e68 Mon Sep 17 00:00:00 2001 From: Yuansheng Date: Tue, 27 Jun 2017 05:57:53 +0800 Subject: [PATCH 104/848] feature: errlog capture: now we also record the UNIX timestamp (in sec with msec precision through the decimal part) for each error log entry. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_log.c | 5 +++-- src/ngx_http_lua_log_ringbuf.c | 12 +++++++++++- src/ngx_http_lua_log_ringbuf.h | 2 +- 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index ab5fd9dd13..c18e6b03ee 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -375,7 +375,7 @@ ngx_http_lua_ffi_errlog_set_filter_level(int level, u_char *err, size_t *errlen) int ngx_http_lua_ffi_errlog_get_msg(char **log, int *loglevel, u_char *err, - size_t *errlen) + size_t *errlen, double *log_time) { #ifdef HAVE_INTERCEPT_ERROR_LOG_PATCH ngx_uint_t loglen; @@ -395,7 +395,8 @@ ngx_http_lua_ffi_errlog_get_msg(char **log, int *loglevel, u_char *err, return NGX_DONE; } - ngx_http_lua_log_ringbuf_read(ringbuf, loglevel, (void **)log, &loglen); + ngx_http_lua_log_ringbuf_read(ringbuf, loglevel, (void **) log, &loglen, + log_time); return loglen; #else *errlen = ngx_snprintf(err, *errlen, diff --git a/src/ngx_http_lua_log_ringbuf.c b/src/ngx_http_lua_log_ringbuf.c index dca610df56..ab1a23dee6 100644 --- a/src/ngx_http_lua_log_ringbuf.c +++ b/src/ngx_http_lua_log_ringbuf.c @@ -10,6 +10,7 @@ typedef struct { + double time; unsigned len:28; /* :24 is big enough if the max log size is 4k */ unsigned log_level:4; } ngx_http_lua_log_ringbuf_header_t; @@ -88,6 +89,7 @@ ngx_http_lua_log_ringbuf_append(ngx_http_lua_log_ringbuf_t *rb, int sentinel, int log_level, void *buf, int n) { ngx_http_lua_log_ringbuf_header_t *head; + ngx_time_t *tp; if (sentinel) { rb->sentinel = rb->tail; @@ -99,6 +101,9 @@ ngx_http_lua_log_ringbuf_append(ngx_http_lua_log_ringbuf_t *rb, head->len = n; head->log_level = log_level; + tp = ngx_timeofday(); + head->time = tp->sec + tp->msec / 1000.0L; + rb->tail += HEADER_LEN; ngx_memcpy(rb->tail, buf, n); rb->tail += n; @@ -171,7 +176,7 @@ ngx_http_lua_log_ringbuf_write(ngx_http_lua_log_ringbuf_t *rb, int log_level, /* read log from ring buffer, do reset if all of the logs were readed. */ ngx_int_t ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, int *log_level, - void **buf, size_t *n) + void **buf, size_t *n, double *log_time) { ngx_http_lua_log_ringbuf_header_t *head; @@ -191,6 +196,11 @@ ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, int *log_level, rb->head += HEADER_LEN; *buf = rb->head; rb->head += head->len; + + if (log_time) { + *log_time = head->time; + } + rb->count--; if (rb->count == 0) { diff --git a/src/ngx_http_lua_log_ringbuf.h b/src/ngx_http_lua_log_ringbuf.h index e855e2514a..4c065393ca 100644 --- a/src/ngx_http_lua_log_ringbuf.h +++ b/src/ngx_http_lua_log_ringbuf.h @@ -21,7 +21,7 @@ void ngx_http_lua_log_ringbuf_init(ngx_http_lua_log_ringbuf_t *rb, void *buf, size_t len); void ngx_http_lua_log_ringbuf_reset(ngx_http_lua_log_ringbuf_t *rb); ngx_int_t ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, - int *log_level, void **buf, size_t *n); + int *log_level, void **buf, size_t *n, double *log_time); ngx_int_t ngx_http_lua_log_ringbuf_write(ngx_http_lua_log_ringbuf_t *rb, int log_level, void *buf, size_t n); From 20d6558b074f1bda3584827412f2e364aeb00b0f Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Fri, 23 Jun 2017 19:07:06 -0700 Subject: [PATCH 105/848] feature: balancer_by_lua*: now the user Lua code can terminate the current request with arbitrary HTTP response status codes via ngx.exit(). Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_balancer.c | 8 +++++++- t/138-balancer.t | 4 ++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 03d9eac082..2fa634eb4d 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -314,7 +314,13 @@ ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) if (ctx->exited && ctx->exit_code != NGX_OK) { rc = ctx->exit_code; - if (rc == NGX_ERROR || rc == NGX_BUSY || rc == NGX_DECLINED) { + if (rc == NGX_ERROR + || rc == NGX_BUSY + || rc == NGX_DECLINED +#ifdef HAVE_BALANCER_STATUS_CODE_PATCH + || rc >= NGX_HTTP_SPECIAL_RESPONSE +#endif + ) { return rc; } diff --git a/t/138-balancer.t b/t/138-balancer.t index fc2617350a..ddd0da577a 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -58,8 +58,8 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ } --- request GET /t ---- response_body_like: 500 Internal Server Error ---- error_code: 500 +--- response_body_like: 403 Forbidden +--- error_code: 403 --- error_log [lua] balancer_by_lua:2: hello from balancer by lua! while connecting to upstream, --- no_error_log eval From 66f073128119ba1f1dae1132d72d2d613f94bd84 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20Pracha=C5=99?= Date: Tue, 27 Jun 2017 19:42:26 +0200 Subject: [PATCH 106/848] bugfix: fixed several bugs in ngx_http_lua_log_ringbuf.c. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_log_ringbuf.c | 84 ++++++++++++++++++++-------------- 1 file changed, 49 insertions(+), 35 deletions(-) diff --git a/src/ngx_http_lua_log_ringbuf.c b/src/ngx_http_lua_log_ringbuf.c index ab1a23dee6..7ff4ae8414 100644 --- a/src/ngx_http_lua_log_ringbuf.c +++ b/src/ngx_http_lua_log_ringbuf.c @@ -24,8 +24,7 @@ enum { static void *ngx_http_lua_log_ringbuf_next_header( ngx_http_lua_log_ringbuf_t *rb); static void ngx_http_lua_log_ringbuf_append( - ngx_http_lua_log_ringbuf_t *rb, int sentinel, int log_level, void *buf, - int n); + ngx_http_lua_log_ringbuf_t *rb, int log_level, void *buf, int n); static size_t ngx_http_lua_log_ringbuf_free_spaces( ngx_http_lua_log_ringbuf_t *rb); @@ -69,14 +68,12 @@ ngx_http_lua_log_ringbuf_next_header(ngx_http_lua_log_ringbuf_t *rb) /* useless data */ if (rb->size - (rb->head - rb->data) < HEADER_LEN) { - rb->head = rb->data; - return rb->head; + return rb->data; } /* placehold data */ if (rb->head >= rb->sentinel) { - rb->head = rb->data; - return rb->head; + return rb->data; } return rb->head; @@ -86,17 +83,11 @@ ngx_http_lua_log_ringbuf_next_header(ngx_http_lua_log_ringbuf_t *rb) /* append data to ring buffer directly */ static void ngx_http_lua_log_ringbuf_append(ngx_http_lua_log_ringbuf_t *rb, - int sentinel, int log_level, void *buf, int n) + int log_level, void *buf, int n) { ngx_http_lua_log_ringbuf_header_t *head; ngx_time_t *tp; - if (sentinel) { - rb->sentinel = rb->tail; - rb->tail = rb->data; - return; - } - head = (ngx_http_lua_log_ringbuf_header_t *) rb->tail; head->len = n; head->log_level = log_level; @@ -109,6 +100,35 @@ ngx_http_lua_log_ringbuf_append(ngx_http_lua_log_ringbuf_t *rb, rb->tail += n; rb->count++; + if (rb->tail > rb->sentinel) { + rb->sentinel = rb->tail; + } + + return; +} + + +/* throw away data at head */ +static void +ngx_http_lua_log_ringbuf_throw_away(ngx_http_lua_log_ringbuf_t *rb) +{ + ngx_http_lua_log_ringbuf_header_t *head; + + if (rb->count == 0) { + return; + } + + head = (ngx_http_lua_log_ringbuf_header_t *) rb->head; + + rb->head += HEADER_LEN + head->len; + rb->count--; + + if (rb->count == 0) { + ngx_http_lua_log_ringbuf_reset(rb); + } + + rb->head = ngx_http_lua_log_ringbuf_next_header(rb); + return; } @@ -117,7 +137,7 @@ ngx_http_lua_log_ringbuf_append(ngx_http_lua_log_ringbuf_t *rb, static size_t ngx_http_lua_log_ringbuf_free_spaces(ngx_http_lua_log_ringbuf_t *rb) { - if (rb->tail == rb->head && rb->tail == rb->data) { + if (rb->count == 0) { return rb->size; } @@ -137,37 +157,30 @@ ngx_int_t ngx_http_lua_log_ringbuf_write(ngx_http_lua_log_ringbuf_t *rb, int log_level, void *buf, size_t n) { - size_t free_spaces; - - ngx_http_lua_log_ringbuf_header_t *head; - if (n + HEADER_LEN > rb->size) { return NGX_ERROR; } - free_spaces = ngx_http_lua_log_ringbuf_free_spaces(rb); - - if (free_spaces < n + HEADER_LEN) { + if (ngx_http_lua_log_ringbuf_free_spaces(rb) < n + HEADER_LEN) { /* if the right space is not enough, mark it as placehold data */ if ((size_t)(rb->data + rb->size - rb->tail) < n + HEADER_LEN) { - ngx_http_lua_log_ringbuf_append(rb, 1, 0, NULL, 0); - } - do { /* throw away old data */ - if (rb->head != ngx_http_lua_log_ringbuf_next_header(rb)) { - break; + while (rb->head >= rb->tail && rb->count) { + /* head is after tail, so we will throw away all data between + * head and sentinel */ + ngx_http_lua_log_ringbuf_throw_away(rb); } - head = (ngx_http_lua_log_ringbuf_header_t *) rb->head; - rb->head += HEADER_LEN + head->len; - rb->count--; + rb->sentinel = rb->tail; + rb->tail = rb->data; + } - ngx_http_lua_log_ringbuf_next_header(rb); - free_spaces = ngx_http_lua_log_ringbuf_free_spaces(rb); - } while (free_spaces < n + HEADER_LEN); + while (ngx_http_lua_log_ringbuf_free_spaces(rb) < n + HEADER_LEN) { + ngx_http_lua_log_ringbuf_throw_away(rb); + } } - ngx_http_lua_log_ringbuf_append(rb, 0, log_level, buf, n); + ngx_http_lua_log_ringbuf_append(rb, log_level, buf, n); return NGX_OK; } @@ -184,8 +197,7 @@ ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, int *log_level, return NGX_ERROR; } - head = (ngx_http_lua_log_ringbuf_header_t *) - ngx_http_lua_log_ringbuf_next_header(rb); + head = (ngx_http_lua_log_ringbuf_header_t *) rb->head; if (rb->head >= rb->sentinel) { return NGX_ERROR; @@ -207,5 +219,7 @@ ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, int *log_level, ngx_http_lua_log_ringbuf_reset(rb); } + rb->head = ngx_http_lua_log_ringbuf_next_header(rb); + return NGX_OK; } From 3294b1dfd0e67a3f8194c36225d109114bc11763 Mon Sep 17 00:00:00 2001 From: goecho Date: Wed, 5 Aug 2015 15:44:06 +0800 Subject: [PATCH 107/848] bugfix: ngx.encode_args() did not escape "|", ",", "$", "@", and "`". now it is now consistent with what Google Chrome's JavaScript API function encodeURIComponent() does. See: https://tools.ietf.org/html/rfc2396. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_util.c | 45 +++++++++++++++++++++++++++++++---------- t/030-uri-args.t | 19 ++++++++++++++++- 2 files changed, 52 insertions(+), 12 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index d20549e73e..c7bee3e89a 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1840,6 +1840,26 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) /* ~}| {zyx wvut srqp onml kjih gfed cba` */ 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + }; + + /* not ALPHA, DIGIT, "-", ".", "_", "~" */ + + static uint32_t uri_component[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0xfc00987d, /* 1111 1100 0000 0000 1001 1000 0111 1101 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x78000001, /* 0111 1000 0000 0000 0000 0000 0000 0001 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0xb8000001, /* 1011 1000 0000 0000 0000 0000 0000 0001 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ @@ -1909,8 +1929,7 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) /* mail_auth is the same as memcached */ static uint32_t *map[] = - { uri, args, html, refresh, memcached, memcached }; - + { uri, args, uri_component, html, refresh, memcached, memcached }; escape = map[type]; @@ -2321,7 +2340,7 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, key = (u_char *) lua_tolstring(L, -2, &key_len); key_escape = 2 * ngx_http_lua_escape_uri(NULL, key, key_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT); total_escape += key_escape; switch (lua_type(L, -1)) { @@ -2330,7 +2349,7 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, value = (u_char *) lua_tolstring(L, -1, &value_len); total_escape += 2 * ngx_http_lua_escape_uri(NULL, value, value_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT); len += key_len + value_len + (sizeof("=") - 1); n++; @@ -2371,7 +2390,7 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, total_escape += 2 * ngx_http_lua_escape_uri(NULL, value, value_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT); len += key_len + value_len + (sizeof("=") - 1); } @@ -2428,7 +2447,8 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, if (total_escape) { p = (u_char *) ngx_http_lua_escape_uri(p, key, key_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT + ); } else { dd("shortcut: no escape required"); @@ -2442,7 +2462,8 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, if (total_escape) { p = (u_char *) ngx_http_lua_escape_uri(p, value, value_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT + ); } else { p = ngx_copy(p, value, value_len); @@ -2461,7 +2482,7 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, if (lua_toboolean(L, -1)) { if (total_escape) { p = (u_char *) ngx_http_lua_escape_uri(p, key, key_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT); } else { dd("shortcut: no escape required"); @@ -2489,7 +2510,7 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, if (total_escape) { p = (u_char *) ngx_http_lua_escape_uri(p, key, key_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT); } else { dd("shortcut: no escape required"); @@ -2508,7 +2529,8 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, p = (u_char *) ngx_http_lua_escape_uri(p, key, key_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT + ); } else { dd("shortcut: no escape required"); @@ -2524,7 +2546,8 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, p = (u_char *) ngx_http_lua_escape_uri(p, value, value_len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT + ); } else { p = ngx_copy(p, value, value_len); diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 30d66d0401..96e216c15b 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,7 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 17); +plan tests => repeat_each() * (blocks() * 2 + 18); no_root_location(); @@ -1389,3 +1389,20 @@ GET /lua GET /foo?world --- response_body_like ^HTTP/1.0 (a=3&b|b&a=3)$ + + + +=== TEST 57: ngx.encode_args (escaping) +--- config + location /lua { + content_by_lua_block { + local t = {bar = "-_.!~*'()", foo = ",$@|`"} + ngx.say("args: ", ngx.encode_args(t)) + } + } +--- request +GET /lua +--- response_body +args: foo=%2C%24%40%7C%60&bar=-_.!~*'() +--- no_error_log +[error] From bb30f6d8b6cd022f7acb55c806067025dd1f65d5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 3 Jul 2017 22:47:00 -0700 Subject: [PATCH 108/848] bugfix: ngx.escape_uri() did not escape "|", ",", "$", "@", and "`". --- src/ngx_http_lua_string.c | 9 +++++---- t/006-escape.t | 21 +++++++++++++++++++-- 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index 22b4c00303..239b2322ab 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -125,12 +125,13 @@ ngx_http_lua_ngx_escape_uri(lua_State *L) return 1; } - escape = 2 * ngx_http_lua_escape_uri(NULL, src, len, NGX_ESCAPE_URI); + escape = 2 * ngx_http_lua_escape_uri(NULL, src, len, + NGX_ESCAPE_URI_COMPONENT); if (escape) { dlen = escape + len; dst = lua_newuserdata(L, dlen); - ngx_http_lua_escape_uri(dst, src, len, NGX_ESCAPE_URI); + ngx_http_lua_escape_uri(dst, src, len, NGX_ESCAPE_URI_COMPONENT); lua_pushlstring(L, (char *) dst, dlen); } @@ -751,14 +752,14 @@ size_t ngx_http_lua_ffi_uri_escaped_length(const u_char *src, size_t len) { return len + 2 * ngx_http_lua_escape_uri(NULL, (u_char *) src, len, - NGX_ESCAPE_URI); + NGX_ESCAPE_URI_COMPONENT); } void ngx_http_lua_ffi_escape_uri(const u_char *src, size_t len, u_char *dst) { - ngx_http_lua_escape_uri(dst, (u_char *) src, len, NGX_ESCAPE_URI); + ngx_http_lua_escape_uri(dst, (u_char *) src, len, NGX_ESCAPE_URI_COMPONENT); } #endif diff --git a/t/006-escape.t b/t/006-escape.t index 0fb67a30f1..a21d3cb509 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -3,9 +3,8 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -#repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 1); +plan tests => repeat_each() * (blocks() * 2 + 2); no_long_string(); @@ -180,3 +179,21 @@ GET /t GET /t --- response_body [32] + + + +=== TEST 14: reserved chars +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.escape_uri("-_.!~*'()")) + ngx.say(ngx.escape_uri(",$@|`")) + } + } +--- request +GET /lua +--- response_body +-_.!~*'() +%2C%24%40%7C%60 +--- no_error_log +[error] From 8827a7f8332b9556c04fe91153ba55d677489907 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 3 Jul 2017 23:39:26 -0700 Subject: [PATCH 109/848] bugfix: segmentation fault would occur when several server {} blocks listen on the same port or unix domain socket file path *and* some of them are using ssl_certificate_by_lua* configurations while some are not. thanks petrovich-ua for the report and original patch in #1055. --- src/ngx_http_lua_ssl_certby.c | 11 ++ t/139-ssl-cert-by.t | 192 +++++++++++++++++++++++++++++++++- 2 files changed, 202 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 84e309316a..5abbe7598e 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -193,6 +193,7 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) ngx_http_lua_srv_conf_t *lscf; ngx_http_core_loc_conf_t *clcf; ngx_http_lua_ssl_ctx_t *cctx; + ngx_http_core_srv_conf_t *cscf; c = ngx_ssl_get_connection(ssl_conn); @@ -298,6 +299,16 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) c->log->action = "loading SSL certificate by lua"; + if (lscf->srv.ssl_cert_handler == NULL) { + cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module); + + ngx_log_error(NGX_LOG_ALERT, c->log, 0, + "no ssl_certificate_by_lua* defined in " + "server %V", &cscf->server_name); + + goto failed; + } + rc = lscf->srv.ssl_cert_handler(r, lscf, L); if (rc >= NGX_OK || rc == NGX_ERROR) { diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index e6d0da8a66..c13044fc28 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -11,7 +11,7 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); } else { - plan tests => repeat_each() * (blocks() * 6 + 10); + plan tests => repeat_each() * (blocks() * 6 + 6); } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -1676,3 +1676,193 @@ close: 1 nil --- no_error_log [error] [alert] + + + +=== TEST 20: some server {} block missing ssl_certificate_by_lua* handlers (literal server name) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { print("ssl cert by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /timers { + default_type 'text/plain'; + content_by_lua_block { + ngx.timer.at(0.1, function() ngx.sleep(0.3) end) + ngx.timer.at(0.11, function() ngx.sleep(0.3) end) + ngx.timer.at(0.09, function() ngx.sleep(0.3) end) + ngx.sleep(0.2) + ngx.say(ngx.timer.running_count()) + } + more_clear_headers Date; + } + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test2.com; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test2.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /timers HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +qr/\[alert\] .*? no ssl_certificate_by_lua\* defined in server test2\.com\b/, +qr/\[crit\] .*? SSL_do_handshake\(\) failed\b/, +] + + + +=== TEST 21: some server {} block missing ssl_certificate_by_lua* handlers (regex server name) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { print("ssl cert by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /timers { + default_type 'text/plain'; + content_by_lua_block { + ngx.timer.at(0.1, function() ngx.sleep(0.3) end) + ngx.timer.at(0.11, function() ngx.sleep(0.3) end) + ngx.timer.at(0.09, function() ngx.sleep(0.3) end) + ngx.sleep(0.2) + ngx.say(ngx.timer.running_count()) + } + more_clear_headers Date; + } + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name ~test2\.com; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test2.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /timers HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +qr/\[alert\] .*? no ssl_certificate_by_lua\* defined in server ~test2\\\.com\b/, +qr/\[crit\] .*? SSL_do_handshake\(\) failed\b/, +] From 4527ec0bb8a24f48e297601396f2834ce91782bf Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 4 Jul 2017 10:50:15 -0700 Subject: [PATCH 110/848] tests: relaxed the expected ouptut constraint in tests for testing ssl host name mismatch. --- t/129-ssl-socket.t | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 530406d6ed..1c3f7cd016 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -425,10 +425,11 @@ $::ComodoRootCertificate" --- request GET /t ---- response_body -connected: 1 -failed to do SSL handshake: handshake failed +--- response_body_like chomp +\Aconnected: 1 +failed to do SSL handshake: (?:handshake failed|certificate host mismatch) failed to send http request: closed +\z --- log_level: debug --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ @@ -504,10 +505,11 @@ $::ComodoRootCertificate" --- request GET /t ---- response_body -connected: 1 -failed to do SSL handshake: handshake failed +--- response_body_like chomp +\Aconnected: 1 +failed to do SSL handshake: (?:handshake failed|certificate host mismatch) failed to send http request: closed +\z --- log_level: debug --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ From 81b8e4823d22686810355977e6ab70264fc08648 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 4 Jul 2017 10:56:57 -0700 Subject: [PATCH 111/848] tests: made a test less possible to fail due to timing error. --- t/135-worker-id.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/135-worker-id.t b/t/135-worker-id.t index 3c1f24d462..984f44650a 100644 --- a/t/135-worker-id.t +++ b/t/135-worker-id.t @@ -54,7 +54,7 @@ GET /lua content_by_lua_block { local counters = ngx.shared.counters local ok, c - for i = 1, 45 do + for i = 1, 100 do c = counters:get("c") if c >= 4 then ok = true From e84d69b269f4645dbe17b1780ef10c752ea4c51d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 4 Jul 2017 11:12:28 -0700 Subject: [PATCH 112/848] tests: made a slow test less possible to time out on slow machines. --- t/152-timer-every.t | 1 + 1 file changed, 1 insertion(+) diff --git a/t/152-timer-every.t b/t/152-timer-every.t index 5026ed086c..a60f7d7613 100644 --- a/t/152-timer-every.t +++ b/t/152-timer-every.t @@ -267,6 +267,7 @@ growth1 == growth2: true [error] [alert] [crit] +--- timeout: 8 From 67664fef0937268e509a7bfc542bb0912064aa31 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 4 Jul 2017 11:33:59 -0700 Subject: [PATCH 113/848] minor coding style fixes. --- src/ngx_http_lua_log_ringbuf.h | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/ngx_http_lua_log_ringbuf.h b/src/ngx_http_lua_log_ringbuf.h index 4c065393ca..c9c2c2d148 100644 --- a/src/ngx_http_lua_log_ringbuf.h +++ b/src/ngx_http_lua_log_ringbuf.h @@ -7,13 +7,13 @@ typedef struct { - ngx_uint_t filter_level; - char *tail; /* writed point */ - char *head; /* readed point */ - char *data; /* buffer */ - char *sentinel; - size_t size; /* buffer total size */ - size_t count; /* count of logs */ + ngx_uint_t filter_level; + char *tail; /* writed point */ + char *head; /* readed point */ + char *data; /* buffer */ + char *sentinel; + size_t size; /* buffer total size */ + size_t count; /* count of logs */ } ngx_http_lua_log_ringbuf_t; From c372e1cc0ae2ed497d72d907b7c5f185afcf32e9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 4 Jul 2017 11:35:00 -0700 Subject: [PATCH 114/848] optimize: made ngx_http_lua_log_ringbuf_header_t have the same size on linux i386 and x64. --- src/ngx_http_lua_log_ringbuf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_log_ringbuf.c b/src/ngx_http_lua_log_ringbuf.c index 7ff4ae8414..04640697ff 100644 --- a/src/ngx_http_lua_log_ringbuf.c +++ b/src/ngx_http_lua_log_ringbuf.c @@ -11,8 +11,8 @@ typedef struct { double time; - unsigned len:28; /* :24 is big enough if the max log size is 4k */ - unsigned log_level:4; + unsigned len; + unsigned log_level; } ngx_http_lua_log_ringbuf_header_t; From 720bd5dbb9b20cef4f5f2e183d6c50698f87feef Mon Sep 17 00:00:00 2001 From: skyever Date: Thu, 29 Jun 2017 15:39:32 +0800 Subject: [PATCH 115/848] bugfix: the fake requests/connections might leak when memory allocations fail. as discussed in #1098. Signed-off-by: skyever Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_ssl_certby.c | 12 +++++++++--- src/ngx_http_lua_ssl_session_fetchby.c | 12 +++++++++--- src/ngx_http_lua_ssl_session_storeby.c | 4 +++- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 5abbe7598e..c3591d1ea3 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -464,7 +464,9 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) if (ctx == NULL) { ctx = ngx_http_lua_create_ctx(r); if (ctx == NULL) { - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } } else { @@ -481,7 +483,9 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "lua: failed to create new coroutine to handle request"); - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } /* move code closure to new coroutine */ @@ -505,7 +509,9 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) if (ctx->cleanup == NULL) { cln = ngx_http_cleanup_add(r, 0); if (cln == NULL) { - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } cln->handler = ngx_http_lua_request_cleanup_handler; diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 60728f29a4..556b73204c 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -468,7 +468,9 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) if (ctx == NULL) { ctx = ngx_http_lua_create_ctx(r); if (ctx == NULL) { - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } } else { @@ -485,7 +487,9 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "lua: failed to create new coroutine to handle request"); - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } /* move code closure to new coroutine */ @@ -509,7 +513,9 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) if (ctx->cleanup == NULL) { cln = ngx_http_cleanup_add(r, 0); if (cln == NULL) { - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } cln->handler = ngx_http_lua_request_cleanup_handler; diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 50adf2040c..bae8273d69 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -351,7 +351,9 @@ ngx_http_lua_ssl_sess_store_by_chunk(lua_State *L, ngx_http_request_t *r) if (ctx == NULL) { ctx = ngx_http_lua_create_ctx(r); if (ctx == NULL) { - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; } } else { From ef4a0b1db5b32f7ebf3004ca353b06825c2995ab Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 5 Jul 2017 18:42:14 -0700 Subject: [PATCH 116/848] tests: skipped the test case for lua_code_cache off + ngx.timer.every in the "check leak" test mode since there is a known memory leak in this case. --- t/152-timer-every.t | 3 +++ 1 file changed, 3 insertions(+) diff --git a/t/152-timer-every.t b/t/152-timer-every.t index a60f7d7613..c8d62e704d 100644 --- a/t/152-timer-every.t +++ b/t/152-timer-every.t @@ -343,6 +343,8 @@ lua_max_running_timers are not enough === TEST 9: lua_code_cache off +FIXME: it is know that this test case leaks memory. +so we skip it in the "check leak" testing mode. --- http_config lua_code_cache off; --- config @@ -380,3 +382,4 @@ ok --- no_error_log [error] [crit] +--- no_check_leak From cc0a793a27af48a364b951a374716b8cd5221487 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 10 Jul 2017 10:22:40 -0700 Subject: [PATCH 117/848] ocsp: removed a useless line of code, which unbreak the libressl build. thanks Kyra Zimmer for the original patch. --- src/ngx_http_lua_ssl_ocsp.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_ocsp.c b/src/ngx_http_lua_ssl_ocsp.c index 3904aa8ea4..31b4f24382 100644 --- a/src/ngx_http_lua_ssl_ocsp.c +++ b/src/ngx_http_lua_ssl_ocsp.c @@ -490,7 +490,6 @@ ngx_http_lua_ffi_ssl_set_ocsp_status_resp(ngx_http_request_t *r, dd("set ocsp resp: resp_len=%d", (int) resp_len); (void) SSL_set_tlsext_status_ocsp_resp(ssl_conn, p, resp_len); - ssl_conn->tlsext_status_expected = 1; return NGX_OK; From a5094766702d97ab43c742f26d6d55098aa1b1f0 Mon Sep 17 00:00:00 2001 From: Yuansheng Date: Mon, 10 Jul 2017 12:01:24 +0800 Subject: [PATCH 118/848] bugfix: ngx.semaphore: when nginx workers exit, the harmless error message "semaphore gc wait queue is not empty" might be logged. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 1 + src/ngx_http_lua_semaphore.c | 7 +- t/153-semaphore-hup.t | 152 +++++++++++++++++++++++++++++++++++ t/154-semaphore.t | 118 +++++++++++++++++++++++++++ 4 files changed, 276 insertions(+), 2 deletions(-) create mode 100644 t/153-semaphore-hup.t create mode 100644 t/154-semaphore.t diff --git a/.travis.yml b/.travis.yml index 14ee63603c..39f9278b80 100644 --- a/.travis.yml +++ b/.travis.yml @@ -83,6 +83,7 @@ install: - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core + - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git before_script: diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 604d943471..eda0141f71 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -557,8 +557,11 @@ ngx_http_lua_ffi_sema_gc(ngx_http_lua_sema_t *sem) return; } - if (!ngx_queue_empty(&sem->wait_queue)) { - ngx_log_error(NGX_LOG_CRIT, ngx_cycle->log, 0, + if (!ngx_terminate + && !ngx_quit + && !ngx_queue_empty(&sem->wait_queue)) + { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, "in lua semaphore gc wait queue is" " not empty while the semaphore %p is being " "destroyed", sem); diff --git a/t/153-semaphore-hup.t b/t/153-semaphore-hup.t new file mode 100644 index 0000000000..f47074c629 --- /dev/null +++ b/t/153-semaphore-hup.t @@ -0,0 +1,152 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: +use lib 'lib'; +use Test::Nginx::Socket::Lua; + +#worker_connections(10140); +#workers(1); +log_level('warn'); +master_process_enabled(1); +repeat_each(1); + +plan tests => repeat_each() * (blocks() * 3); + +no_long_string(); +#no_diff(); + +add_block_preprocessor(sub { + my $block = shift; + + my $http_config = $block->http_config || ''; + $http_config .= <<'_EOC_'; + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + lua_shared_dict shdict 4m; + + init_by_lua_block { + require "resty.core" + local process = require "ngx.process" + local ok, err = process.enable_privileged_agent() + if not ok then + ngx.log(ngx.ERR, "failed to enable_privileged_agent: ", err) + end + } + + init_worker_by_lua_block { + local function test(pre) + if pre then + return + end + + local semaphore = require "ngx.semaphore" + local sem = semaphore.new() + + ngx.log(ngx.ERR, "created semaphore object") + + local function sem_wait() + + local ok, err = sem:wait(100) + if not ok then + ngx.log(ngx.ERR, "err: ", err) + else + ngx.log(ngx.ERR, "wait success") + end + end + + while not ngx.worker.exiting() do + local co = ngx.thread.spawn(sem_wait) + ngx.thread.wait(co) + end + end + + local ok, err = ngx.timer.at(0, test) + if not ok then + ngx.log(ngx.ERR, "failed to create semaphore timer err: ", err) + end + + local function reload(pre) + if pre then + return + end + + shdict = ngx.shared.shdict + local success = shdict:add("reloaded", 1) + if not success then + return + end + + ngx.log(ngx.ERR, "try to reload nginx") + + local f, err = io.open(ngx.config.prefix() .. "/logs/nginx.pid", "r") + if not f then + ngx.say("failed to open nginx.pid: ", err) + return + end + + local pid = f:read() + + f:close() + os.execute("kill -HUP " .. pid) + end + + local typ = require "ngx.process".type + if typ() == "privileged agent" then + local ok, err = ngx.timer.at(0.1, reload) + if not ok then + ngx.log(ngx.ERR, "failed to create semaphore timer err: ", err) + end + end + } +_EOC_ + $block->set_value("http_config", $http_config); +}); + +run_tests(); + +__DATA__ + +=== TEST 1: timer + reload +--- config + location /test { + content_by_lua_block { + ngx.sleep(1) + ngx.say("hello") + } + } +--- request +GET /test +--- response_body +hello +--- grep_error_log eval: qr/created semaphore object|try to reload nginx|semaphore gc wait queue is not empty/ +--- grep_error_log_out +created semaphore object +created semaphore object +try to reload nginx +created semaphore object +created semaphore object +--- skip_nginx: 3: < 1.11.2 +--- no_check_leak + + + +=== TEST 2: timer + reload (lua code cache off) +--- http_config + lua_code_cache off; +--- config + location /test { + content_by_lua_block { + ngx.sleep(1) + ngx.say("hello") + } + } +--- request +GET /test +--- response_body +hello +--- grep_error_log eval: qr/created semaphore object|try to reload nginx|semaphore gc wait queue is not empty/ +--- grep_error_log_out +created semaphore object +created semaphore object +try to reload nginx +created semaphore object +created semaphore object +--- skip_nginx: 3: < 1.11.2 +--- no_check_leak diff --git a/t/154-semaphore.t b/t/154-semaphore.t new file mode 100644 index 0000000000..b45de63bfa --- /dev/null +++ b/t/154-semaphore.t @@ -0,0 +1,118 @@ +# vim:set ft=ts=4 sw=4 et fdm=marker: +use lib 'lib'; +use Test::Nginx::Socket::Lua; + +#worker_connections(10140); +#workers(1); +#log_level('warn'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3) + blocks(); + +no_long_string(); +#no_diff(); + +add_block_preprocessor(sub { + my $block = shift; + + my $http_config = $block->http_config || ''; + $http_config .= <<'_EOC_'; + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + init_by_lua_block { + require "resty.core" + } +_EOC_ + $block->set_value("http_config", $http_config); +}); + +run_tests(); + +__DATA__ + +=== TEST 1: timer + shutdown error log +--- config + location /test { + content_by_lua_block { + local function test(pre) + + local semaphore = require "ngx.semaphore" + local sem = semaphore.new() + + local function sem_wait() + + local ok, err = sem:wait(10) + if not ok then + ngx.log(ngx.ERR, "err: ", err) + else + ngx.log(ngx.ERR, "wait success") + end + end + + while not ngx.worker.exiting() do + local co = ngx.thread.spawn(sem_wait) + ngx.thread.wait(co) + end + end + + local ok, err = ngx.timer.at(0, test) + ngx.log(ngx.ERR, "hello, world") + ngx.say("time: ", ok) + } + } +--- request +GET /test +--- response_body +time: 1 +--- grep_error_log eval: qr/hello, world|semaphore gc wait queue is not empty/ +--- grep_error_log_out +hello, world +--- shutdown_error_log +--- no_shutdown_error_log +semaphore gc wait queue is not empty + + + +=== TEST 2: timer + shutdown error log (lua code cache off) +--- http_config + lua_code_cache off; +--- config + location /test { + content_by_lua_block { + local function test(pre) + + local semaphore = require "ngx.semaphore" + local sem = semaphore.new() + + local function sem_wait() + + local ok, err = sem:wait(10) + if not ok then + ngx.log(ngx.ERR, "err: ", err) + else + ngx.log(ngx.ERR, "wait success") + end + end + + while not ngx.worker.exiting() do + local co = ngx.thread.spawn(sem_wait) + ngx.thread.wait(co) + end + end + + local ok, err = ngx.timer.at(0, test) + ngx.log(ngx.ERR, "hello, world") + ngx.say("time: ", ok) + } + } +--- request +GET /test +--- response_body +time: 1 +--- grep_error_log eval: qr/hello, world|semaphore gc wait queue is not empty/ +--- grep_error_log_out +hello, world +--- shutdown_error_log +--- no_shutdown_error_log +semaphore gc wait queue is not empty From 1e71ff2cbac1fe7115fdee6634fbdcfb35341216 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 16 Jul 2017 20:51:47 -0700 Subject: [PATCH 119/848] tests: fixed the vim mode line in a test file. --- t/154-semaphore.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/154-semaphore.t b/t/154-semaphore.t index b45de63bfa..3c1f004431 100644 --- a/t/154-semaphore.t +++ b/t/154-semaphore.t @@ -1,4 +1,4 @@ -# vim:set ft=ts=4 sw=4 et fdm=marker: +# vim:set ft= ts=4 sw=4 et fdm=marker: use lib 'lib'; use Test::Nginx::Socket::Lua; From c26854856a84e2a8da8178687f2b572f54dc9bdf Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 17 Jul 2017 10:34:53 +0800 Subject: [PATCH 120/848] tests: added a test case to make sure $proxy_add_x_forwarded_for is accessible on the Lua land. Signed-off-by: Yichun Zhang (agentzh) --- t/045-ngx-var.t | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/t/045-ngx-var.t b/t/045-ngx-var.t index b5fcc8107b..6475f1e809 100644 --- a/t/045-ngx-var.t +++ b/t/045-ngx-var.t @@ -154,14 +154,15 @@ invalid referer: 1 -=== TEST 8: $proxy_host & $proxy_port +=== TEST 8: $proxy_host & $proxy_port & $proxy_add_x_forwarded_for --- config location = /t { proxy_pass http://127.0.0.1:$server_port/back; - header_filter_by_lua ' + header_filter_by_lua_block { ngx.header["Proxy-Host"] = ngx.var.proxy_host ngx.header["Proxy-Port"] = ngx.var.proxy_port - '; + ngx.header["Proxy-Add-X-Forwarded-For"] = ngx.var.proxy_add_x_forwarded_for + } } location = /back { @@ -172,6 +173,7 @@ GET /t --- raw_response_headers_like Proxy-Host: 127.0.0.1\:\d+\r Proxy-Port: \d+\r +Proxy-Add-X-Forwarded-For: 127.0.0.1\r --- response_body hello --- no_error_log From 2fd469d468c6b1cdb1561557a0967f6c9b3e8f0c Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 17 Jul 2017 16:08:46 -0700 Subject: [PATCH 121/848] refactor: removed the duplicate C function decelaration for ngx_http_lua_socket_read_handler. --- src/ngx_http_lua_socket_tcp.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 49811168d1..fe089638dc 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -69,8 +69,6 @@ static void ngx_http_lua_socket_dummy_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); static ngx_int_t ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); -static void ngx_http_lua_socket_read_handler(ngx_http_request_t *r, - ngx_http_lua_socket_tcp_upstream_t *u); static int ngx_http_lua_socket_tcp_receive_retval_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); static ngx_int_t ngx_http_lua_socket_read_line(void *data, ssize_t bytes); From 51c14a898138b9d734e7a908b5ca2937cc2cccf1 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 17 Jul 2017 16:10:57 -0700 Subject: [PATCH 122/848] bugfix: segmentation fault might happen when a stale read event happens after the downstream cosocket object is closed. thanks Dejiang Zhu for the report. --- src/ngx_http_lua_socket_tcp.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index fe089638dc..382a94de7c 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4418,15 +4418,18 @@ ngx_http_lua_req_socket_rev_handler(ngx_http_request_t *r) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { + r->read_event_handler = ngx_http_block_reading; return; } u = ctx->downstream; - if (u) { - u->read_event_handler(r, u); + if (u == NULL || u->peer.connection == NULL) { + r->read_event_handler = ngx_http_block_reading; + return; } -} + u->read_event_handler(r, u); +} static int ngx_http_lua_socket_tcp_getreusedtimes(lua_State *L) From 0f0af5094880f94ff4d2185576c1385981a49750 Mon Sep 17 00:00:00 2001 From: WenMing Date: Wed, 19 Jul 2017 11:28:37 +0800 Subject: [PATCH 123/848] doc: fixed a typo in a code example for `ngx.re.match`. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index c30944d0ce..5a6d878148 100644 --- a/README.markdown +++ b/README.markdown @@ -5909,7 +5909,7 @@ The optional fourth argument, `ctx`, can be a Lua table holding an optional `pos local ctx = { pos = 2 } local m, err = ngx.re.match("1234, hello", "[0-9]+", "", ctx) - -- m[0] = "34" + -- m[0] = "234" -- ctx.pos == 5 ``` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ef8ecc3b81..1f188b260d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4932,7 +4932,7 @@ The optional fourth argument, ctx, can be a Lua table holding an op local ctx = { pos = 2 } local m, err = ngx.re.match("1234, hello", "[0-9]+", "", ctx) - -- m[0] = "34" + -- m[0] = "234" -- ctx.pos == 5 From f170505186ff61af36b3e126772b671793af9428 Mon Sep 17 00:00:00 2001 From: Henry Lee Date: Thu, 27 Jul 2017 14:47:53 +0800 Subject: [PATCH 124/848] bugfix: ngx.escape_uri: we did not escape URI reserved chars. #1124 ngx.escape_uri should also escape RFC 3986 section 2.2 Reserved Characters. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_util.c | 2 +- t/006-escape.t | 2 +- t/030-uri-args.t | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c7bee3e89a..b6dfae9530 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1852,7 +1852,7 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ - 0xfc00987d, /* 1111 1100 0000 0000 1001 1000 0111 1101 */ + 0xfc009fff, /* 1111 1100 0000 0000 1001 1111 1111 1111 */ /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ 0x78000001, /* 0111 1000 0000 0000 0000 0000 0000 0001 */ diff --git a/t/006-escape.t b/t/006-escape.t index a21d3cb509..695aaf7060 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -193,7 +193,7 @@ GET /t --- request GET /lua --- response_body --_.!~*'() +-_.%21~%2A%27%28%29 %2C%24%40%7C%60 --- no_error_log [error] diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 96e216c15b..752e0f5cf8 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1403,6 +1403,6 @@ GET /lua --- request GET /lua --- response_body -args: foo=%2C%24%40%7C%60&bar=-_.!~*'() +args: foo=%2C%24%40%7C%60&bar=-_.%21~%2A%27%28%29 --- no_error_log [error] From 691370ec274f5e94a095eae0298096a66b04d983 Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 31 Jul 2017 09:57:47 +0800 Subject: [PATCH 125/848] Revert "bugfix: ngx.escape_uri: we did not escape URI reserved chars. #1124" Commit f170505186ff61af36b3e126772b671793af9428 breaks the compatibility with RFC 3986. Here is two reasons: 1. Quote from RFC 3986 Section 2.2: > A subset of the reserved characters (gen-delims) is used as delimiters of the generic URI components described in Section 3 Note that RFC 3986 says 'a subset of the reserved characters (gen-delims)', not all the reserved characters. The characters escaped in that commit are 'sub-delims'. They are not required to be escaped according to Section 2.2. 2. Refer to RFC 3986 "Appendix A. Collected ABNF for URI", sub-delims could be used as part of query and other components. This use case shows that sub-delims are valid in some component of URI. Therefore, it would be better if we don't escape them for URI component. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_util.c | 2 +- t/006-escape.t | 2 +- t/030-uri-args.t | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index b6dfae9530..c7bee3e89a 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1852,7 +1852,7 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ - 0xfc009fff, /* 1111 1100 0000 0000 1001 1111 1111 1111 */ + 0xfc00987d, /* 1111 1100 0000 0000 1001 1000 0111 1101 */ /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ 0x78000001, /* 0111 1000 0000 0000 0000 0000 0000 0001 */ diff --git a/t/006-escape.t b/t/006-escape.t index 695aaf7060..a21d3cb509 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -193,7 +193,7 @@ GET /t --- request GET /lua --- response_body --_.%21~%2A%27%28%29 +-_.!~*'() %2C%24%40%7C%60 --- no_error_log [error] diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 752e0f5cf8..96e216c15b 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1403,6 +1403,6 @@ GET /lua --- request GET /lua --- response_body -args: foo=%2C%24%40%7C%60&bar=-_.%21~%2A%27%28%29 +args: foo=%2C%24%40%7C%60&bar=-_.!~*'() --- no_error_log [error] From 82b21887127765b43991f0356cccb12882cfbf73 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 6 Aug 2017 19:48:20 -0700 Subject: [PATCH 126/848] tests: made 2 test cases less possible to fail on slower machines. --- t/153-semaphore-hup.t | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/153-semaphore-hup.t b/t/153-semaphore-hup.t index f47074c629..c85a21dae4 100644 --- a/t/153-semaphore-hup.t +++ b/t/153-semaphore-hup.t @@ -124,6 +124,7 @@ created semaphore object created semaphore object --- skip_nginx: 3: < 1.11.2 --- no_check_leak +--- wait: 0.2 @@ -150,3 +151,4 @@ created semaphore object created semaphore object --- skip_nginx: 3: < 1.11.2 --- no_check_leak +--- wait: 0.2 From 89de7802227fd97005cb03b5d1fc38b809093b70 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 8 Aug 2017 12:25:45 -0700 Subject: [PATCH 127/848] doc: bumped version to 0.10.9. --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 5a6d878148..0b71c2cc0a 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.8](https://github.com/openresty/lua-nginx-module/tags) released on 8 April 2017. +This document describes ngx_lua [v0.10.9](https://github.com/openresty/lua-nginx-module/tags) released on 8 August 2017. Synopsis ======== @@ -990,7 +990,7 @@ Copyright and License This module is licensed under the BSD license. -Copyright (C) 2009-2016, by Xiaozhe Wang (chaoslawful) . +Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 1f188b260d..13b3d894b3 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.8] released on 8 April 2017. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.9] released on 8 August 2017. = Synopsis = @@ -812,7 +812,7 @@ There are also various testing modes based on mockeagain, valgrind, and etc. Ref This module is licensed under the BSD license. -Copyright (C) 2009-2016, by Xiaozhe Wang (chaoslawful) . +Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. From 7b7178a958f33eb5086a4e66c18f13e68a39c859 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 8 Aug 2017 19:44:12 -0700 Subject: [PATCH 128/848] bugfix: fixed a compilation error on Windows. this regression had appeaered in 0.10.9. --- src/ngx_http_lua_worker.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index f4425b914a..e1cfec4102 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -138,7 +138,7 @@ ngx_http_lua_ffi_worker_count(void) int ngx_http_lua_ffi_get_process_type(void) { -#ifdef HAVE_PRIVILEGED_PROCESS_PATCH +#if defined(HAVE_PRIVILEGED_PROCESS_PATCH) && !NGX_WIN32 if (ngx_process == NGX_PROCESS_HELPER) { if (ngx_is_privileged_agent) { return NGX_PROCESS_PRIVILEGED_AGENT; From cdd2ae921f67bf396c743406493127be496e57ce Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 8 Aug 2017 19:51:56 -0700 Subject: [PATCH 129/848] bumped version to 0.10.10. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index 0b71c2cc0a..fac3a0d79b 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.9](https://github.com/openresty/lua-nginx-module/tags) released on 8 August 2017. +This document describes ngx_lua [v0.10.10](https://github.com/openresty/lua-nginx-module/tags) released on 8 August 2017. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 13b3d894b3..83930567b6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.9] released on 8 August 2017. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.10] released on 8 August 2017. = Synopsis = diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 5083799a04..cd64fc8c35 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10009 +#define ngx_http_lua_version 10010 typedef struct { From e9609d25cb72eda55398d5552d1cef2084138f97 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 10 Aug 2017 12:11:28 -0700 Subject: [PATCH 130/848] valgrind: suppressed a known false positive inside the nginx core. --- valgrind.suppress | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/valgrind.suppress b/valgrind.suppress index d0bcc56d27..fb539f2ae0 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -164,3 +164,13 @@ fun:main Memcheck:Cond obj:* } +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_set_environment + fun:ngx_worker_process_init + fun:ngx_worker_process_cycle +} From b87ad242dc9e50439393c5aa2d219433be9be421 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 10 Aug 2017 12:13:26 -0700 Subject: [PATCH 131/848] tests: skipped a test case which is known to leak memory when lua_code_cache is off. --- t/154-semaphore.t | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/154-semaphore.t b/t/154-semaphore.t index 3c1f004431..875f181ce7 100644 --- a/t/154-semaphore.t +++ b/t/154-semaphore.t @@ -75,6 +75,7 @@ semaphore gc wait queue is not empty === TEST 2: timer + shutdown error log (lua code cache off) +FIXME: this test case leaks memory. --- http_config lua_code_cache off; --- config @@ -116,3 +117,4 @@ hello, world --- shutdown_error_log --- no_shutdown_error_log semaphore gc wait queue is not empty +--- SKIP From e96aec767d32a691fa91cdcf993fe5998a85714e Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 18 Aug 2017 12:19:39 -0700 Subject: [PATCH 132/848] doc: fixed the stale Lua module preloading docs as reminded by Denis. --- README.markdown | 13 ++++++++----- doc/HttpLuaModule.wiki | 13 ++++++++----- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/README.markdown b/README.markdown index fac3a0d79b..3ef8d24be0 100644 --- a/README.markdown +++ b/README.markdown @@ -1335,16 +1335,19 @@ Runs the Lua code specified by the argument `` on the global Lua When Nginx receives the `HUP` signal and starts reloading the config file, the Lua VM will also be re-created and `init_by_lua` will run again on the new Lua VM. In case that the [lua_code_cache](#lua_code_cache) directive is turned off (default on), the `init_by_lua` handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. -Usually you can register (true) Lua global variables or pre-load Lua modules at server start-up by means of this hook. Here is an example for pre-loading Lua modules: +Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating system's copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: ```nginx - init_by_lua 'cjson = require "cjson"'; + # this runs before forking out nginx worker processes: + init_by_lua_block { require "cjson" } server { location = /api { content_by_lua_block { - ngx.say(cjson.encode({dog = 5, cat = 6})) + -- the following require() will just return + -- the alrady loaded module from package.loaded: + ngx.say(require "cjson".encode({dog = 5, cat = 6})) } } } @@ -1356,10 +1359,10 @@ You can also initialize the [lua_shared_dict](#lua_shared_dict) shm storage at t lua_shared_dict dogs 1m; - init_by_lua ' + init_by_lua_block { local dogs = ngx.shared.dogs; dogs:set("Tom", 56) - '; + } server { location = /api { diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 83930567b6..73b93a5fd1 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1057,15 +1057,18 @@ Runs the Lua code specified by the argument on the When Nginx receives the HUP signal and starts reloading the config file, the Lua VM will also be re-created and init_by_lua will run again on the new Lua VM. In case that the [[#lua_code_cache|lua_code_cache]] directive is turned off (default on), the init_by_lua handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. -Usually you can register (true) Lua global variables or pre-load Lua modules at server start-up by means of this hook. Here is an example for pre-loading Lua modules: +Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating system's copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: - init_by_lua 'cjson = require "cjson"'; + # this runs before forking out nginx worker processes: + init_by_lua_block { require "cjson" } server { location = /api { content_by_lua_block { - ngx.say(cjson.encode({dog = 5, cat = 6})) + -- the following require() will just return + -- the alrady loaded module from package.loaded: + ngx.say(require "cjson".encode({dog = 5, cat = 6})) } } } @@ -1076,10 +1079,10 @@ You can also initialize the [[#lua_shared_dict|lua_shared_dict]] shm storage at lua_shared_dict dogs 1m; - init_by_lua ' + init_by_lua_block { local dogs = ngx.shared.dogs; dogs:set("Tom", 56) - '; + } server { location = /api { From aee499ac21f68d449c2eebbe78204d8fff419fe0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 18 Aug 2017 12:21:04 -0700 Subject: [PATCH 133/848] doc: minor fixes. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 3ef8d24be0..17d22cd559 100644 --- a/README.markdown +++ b/README.markdown @@ -1335,7 +1335,7 @@ Runs the Lua code specified by the argument `` on the global Lua When Nginx receives the `HUP` signal and starts reloading the config file, the Lua VM will also be re-created and `init_by_lua` will run again on the new Lua VM. In case that the [lua_code_cache](#lua_code_cache) directive is turned off (default on), the `init_by_lua` handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. -Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating system's copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: +Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating systems' copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: ```nginx diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 73b93a5fd1..735e3d5f4f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1057,7 +1057,7 @@ Runs the Lua code specified by the argument on the When Nginx receives the HUP signal and starts reloading the config file, the Lua VM will also be re-created and init_by_lua will run again on the new Lua VM. In case that the [[#lua_code_cache|lua_code_cache]] directive is turned off (default on), the init_by_lua handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. -Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating system's copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: +Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating systems' copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: # this runs before forking out nginx worker processes: From 3fc5aea08f1ed6c7af2f7b9bc9f4058855a32af6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 18 Aug 2017 12:23:09 -0700 Subject: [PATCH 134/848] doc: simplified a Lua function call a bit. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 17d22cd559..9dec49fe5d 100644 --- a/README.markdown +++ b/README.markdown @@ -1347,7 +1347,7 @@ Usually you can pre-load Lua modules at server start-up by means of this hook an content_by_lua_block { -- the following require() will just return -- the alrady loaded module from package.loaded: - ngx.say(require "cjson".encode({dog = 5, cat = 6})) + ngx.say(require "cjson".encode{dog = 5, cat = 6}) } } } diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 735e3d5f4f..2518196fc0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1068,7 +1068,7 @@ Usually you can pre-load Lua modules at server start-up by means of this hook an content_by_lua_block { -- the following require() will just return -- the alrady loaded module from package.loaded: - ngx.say(require "cjson".encode({dog = 5, cat = 6})) + ngx.say(require "cjson".encode{dog = 5, cat = 6}) } } } From 77a285163689835d6a6680e345af1a4ada2d12c8 Mon Sep 17 00:00:00 2001 From: spacewander Date: Tue, 22 Aug 2017 11:40:19 +0800 Subject: [PATCH 135/848] bugfix: tcpsock: settimeout/settimeouts: throw an error when the timeout argument values overflow. Here we only support timeout not greater than max value of a 32 bits integer to avoid overflow. Supporting too large timeout values doesn't make sense anyway. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 19 +++++++ t/065-tcp-socket-timeout.t | 32 +++++++++++- t/147-tcp-socket-timeouts.t | 93 +++++++++++++++++++++++++++++++++++ 3 files changed, 143 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 382a94de7c..9c790619f8 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2708,6 +2708,11 @@ ngx_http_lua_socket_tcp_settimeout(lua_State *L) } timeout = (ngx_int_t) lua_tonumber(L, 2); + if (timeout > NGX_MAX_INT32_VALUE) { + return luaL_error(L, "lua tcp socket timeout %f will overflow", + (lua_Number) timeout); + } + lua_pushinteger(L, timeout); lua_pushinteger(L, timeout); @@ -2751,8 +2756,22 @@ ngx_http_lua_socket_tcp_settimeouts(lua_State *L) } connect_timeout = (ngx_int_t) lua_tonumber(L, 2); + if (connect_timeout > NGX_MAX_INT32_VALUE) { + return luaL_error(L, "lua tcp socket connect timeout %f will overflow", + (lua_Number) connect_timeout); + } + send_timeout = (ngx_int_t) lua_tonumber(L, 3); + if (send_timeout > NGX_MAX_INT32_VALUE) { + return luaL_error(L, "lua tcp socket send timeout %f will overflow", + (lua_Number) send_timeout); + } + read_timeout = (ngx_int_t) lua_tonumber(L, 4); + if (read_timeout > NGX_MAX_INT32_VALUE) { + return luaL_error(L, "lua tcp socket read timeout %f will overflow", + (lua_Number) read_timeout); + } lua_rawseti(L, 1, SOCKET_READ_TIMEOUT_INDEX); lua_rawseti(L, 1, SOCKET_SEND_TIMEOUT_INDEX); diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 212766ecf6..721c326a6c 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -28,7 +28,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 12); +plan tests => repeat_each() * (blocks() * 4 + 11); our $HtmlDir = html_dir; @@ -994,3 +994,33 @@ close: 1 nil --- no_error_log [error] + + + +=== TEST 23: timeout overflow detection +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = pcall(sock.settimeout, sock, (2 ^ 31) - 1) + if not ok then + ngx.say("failed to set timeout: ", err) + else + ngx.say("settimeout: ok") + end + + ok, err = pcall(sock.settimeout, sock, 2 ^ 31) + if not ok then + ngx.say("failed to set timeout: ", err) + else + ngx.say("settimeout: ok") + end + } + } +--- request +GET /t +--- response_body_like +settimeout: ok +failed to set timeout: lua tcp socket timeout 2147483648(?:\.\d+)? will overflow +--- no_error_log +[error] diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 0689a9b29d..b0a4fa0415 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -532,3 +532,96 @@ received: ok failed to receive a line: closed [] --- no_error_log [error] + + + +=== TEST 8: connection timeout overflow detection +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = pcall(sock.settimeouts, sock, + (2 ^ 31) - 1, 500, 500) + if not ok then + ngx.say("failed to set timeouts: ", err) + else + ngx.say("settimeouts: ok") + end + + ok, err = pcall(sock.settimeouts, sock, 2 ^ 31, 500, 500) + if not ok then + ngx.say("failed to set timeouts: ", err) + else + ngx.say("settimeouts: ok") + end + } + } +--- request +GET /t +--- response_body_like +settimeouts: ok +failed to set timeouts: lua tcp socket connect timeout 2147483648(?:\.\d+)? will overflow +--- no_error_log +[error] + + + +=== TEST 9: send timeout overflow detection +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = pcall(sock.settimeouts, sock, + 500, (2 ^ 31) - 1, 500) + if not ok then + ngx.say("failed to set timeouts: ", err) + else + ngx.say("settimeouts: ok") + end + + ok, err = pcall(sock.settimeouts, sock, 500, 2 ^ 31, 500) + if not ok then + ngx.say("failed to set timeouts: ", err) + else + ngx.say("settimeouts: ok") + end + } + } +--- request +GET /t +--- response_body_like +settimeouts: ok +failed to set timeouts: lua tcp socket send timeout 2147483648(?:\.\d+)? will overflow +--- no_error_log +[error] + + + +=== TEST 10: read timeout overflow detection +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = pcall(sock.settimeouts, sock, + 500, 500, (2 ^ 31) - 1) + if not ok then + ngx.say("failed to set timeouts: ", err) + else + ngx.say("settimeouts: ok") + end + + ok, err = pcall(sock.settimeouts, sock, 500, 500, 2 ^ 31) + if not ok then + ngx.say("failed to set timeouts: ", err) + else + ngx.say("settimeouts: ok") + end + } + } +--- request +GET /t +--- response_body_like +settimeouts: ok +failed to set timeouts: lua tcp socket read timeout 2147483648(?:\.\d+)? will overflow +--- no_error_log +[error] From 36d6ef406b98c63f2f47d1deb790fc8f32615e0f Mon Sep 17 00:00:00 2001 From: root Date: Sat, 19 Aug 2017 19:38:03 +0800 Subject: [PATCH 136/848] bugfix: tcpsock:connect(): when the nginx resolver's send() immediately fails without yielding, we didn't clean up the coroutine ctx state properly. This might lead to segmentation faults. thanks xiaocang for the discovery. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 1 + src/ngx_http_lua_socket_tcp.c | 3 +++ t/058-tcp-socket.t | 39 ++++++++++++++++++++++++++++++++++- 3 files changed, 42 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 39f9278b80..c30fb55f25 100644 --- a/.travis.yml +++ b/.travis.yml @@ -90,6 +90,7 @@ before_script: - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;' script: + - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT - cd luajit2/ - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1) - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 9c790619f8..9eda286bb7 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -743,6 +743,9 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; + coctx->cleanup = NULL; + coctx->data = NULL; + u->resolved->ctx = NULL; lua_pushnil(L); lua_pushfstring(L, "%s could not be resolved", host.data); diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 1ee113b9ce..0743dd73d0 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 190; +plan tests => repeat_each() * 193; our $HtmlDir = html_dir; @@ -3690,3 +3690,40 @@ received: OK close: 1 nil --- no_error_log [error] + + + +=== TEST 61: resolver send query failing immediately in connect() +this case did not clear coctx->cleanup properly and would lead to memory invalid accesses. + +this test case requires the following iptables rule to work properly: + +sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT + +--- config + location /t { + resolver 127.0.0.1:10086 ipv6=off; + resolver_timeout 10ms; + + content_by_lua_block { + local sock = ngx.socket.tcp() + + for i = 1, 3 do -- retry + local ok, err = sock:connect("www.google.com", 80) + if not ok then + ngx.say("failed to connect: ", err) + end + end + + ngx.say("hello!") + } + } +--- request +GET /t +--- response_body +failed to connect: www.google.com could not be resolved +failed to connect: www.google.com could not be resolved +failed to connect: www.google.com could not be resolved +hello! +--- error_log eval +qr{\[alert\] .*? send\(\) failed \(\d+: Operation not permitted\) while resolving} From c97473b5e66fdfa118b91e428dd8a0417ce81f8c Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 13 Sep 2017 11:48:46 -0700 Subject: [PATCH 137/848] bugfix: segmentation faults might happen when pipelined http requests are used in the downsteram connection. thanks Gao Yan for the report. --- src/ngx_http_lua_accessby.c | 10 ++++++---- src/ngx_http_lua_output.c | 7 +++++-- src/ngx_http_lua_req_body.c | 6 ++++-- src/ngx_http_lua_rewriteby.c | 10 ++++++---- src/ngx_http_lua_semaphore.c | 6 ++++-- src/ngx_http_lua_sleep.c | 6 ++++-- src/ngx_http_lua_socket_tcp.c | 6 ++++-- src/ngx_http_lua_socket_udp.c | 6 ++++-- src/ngx_http_lua_subrequest.c | 6 ++++-- src/ngx_http_lua_util.c | 10 ++++++---- src/ngx_http_lua_util.h | 2 +- t/017-exec.t | 24 +++++++++++++++++++++++- t/023-rewrite/exec.t | 24 +++++++++++++++++++++++- t/024-access/exec.t | 24 +++++++++++++++++++++++- 14 files changed, 117 insertions(+), 30 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 1a4ba6dab7..56bf0fa0ad 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -238,6 +238,7 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) { int co_ref; ngx_int_t rc; + ngx_uint_t nreqs; lua_State *co; ngx_event_t *rev; ngx_connection_t *c; @@ -329,6 +330,9 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) r->read_event_handler = ngx_http_block_reading; } + c = r->connection; + nreqs = c->requests; + rc = ngx_http_lua_run_thread(L, r, ctx, 0); dd("returned %d", (int) rc); @@ -337,10 +341,8 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) return rc; } - c = r->connection; - if (rc == NGX_AGAIN) { - rc = ngx_http_lua_run_posted_threads(c, L, r, ctx); + rc = ngx_http_lua_run_posted_threads(c, L, r, ctx, nreqs); if (rc == NGX_ERROR || rc == NGX_DONE || rc > NGX_OK) { return rc; @@ -353,7 +355,7 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) } else if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - rc = ngx_http_lua_run_posted_threads(c, L, r, ctx); + rc = ngx_http_lua_run_posted_threads(c, L, r, ctx, nreqs); if (rc == NGX_ERROR || rc == NGX_DONE || rc > NGX_OK) { return rc; diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index b410ba4511..0fe8840f95 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -724,6 +724,7 @@ ngx_http_lua_flush_resume_helper(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) int n; lua_State *vm; ngx_int_t rc; + ngx_uint_t nreqs; ngx_connection_t *c; c = r->connection; @@ -748,18 +749,20 @@ ngx_http_lua_flush_resume_helper(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) } vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; + rc = ngx_http_lua_run_thread(vm, r, ctx, n); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } /* rc == NGX_ERROR || rc >= NGX_OK */ diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 6f2ae38108..e6bf3c192e 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -1125,6 +1125,7 @@ ngx_http_lua_read_body_resume(ngx_http_request_t *r) { lua_State *vm; ngx_int_t rc; + ngx_uint_t nreqs; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; @@ -1134,6 +1135,7 @@ ngx_http_lua_read_body_resume(ngx_http_request_t *r) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; rc = ngx_http_lua_run_thread(vm, r, ctx, 0); @@ -1141,12 +1143,12 @@ ngx_http_lua_read_body_resume(ngx_http_request_t *r) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (ctx->entered_content_phase) { diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 44d8941623..077c2d3076 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -235,6 +235,7 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) int co_ref; lua_State *co; ngx_int_t rc; + ngx_uint_t nreqs; ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; @@ -324,20 +325,21 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) r->read_event_handler = ngx_http_block_reading; } + c = r->connection; + nreqs = c->requests; + rc = ngx_http_lua_run_thread(L, r, ctx, 0); if (rc == NGX_ERROR || rc > NGX_OK) { return rc; } - c = r->connection; - if (rc == NGX_AGAIN) { - rc = ngx_http_lua_run_posted_threads(c, L, r, ctx); + rc = ngx_http_lua_run_posted_threads(c, L, r, ctx, nreqs); } else if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - rc = ngx_http_lua_run_posted_threads(c, L, r, ctx); + rc = ngx_http_lua_run_posted_threads(c, L, r, ctx, nreqs); } if (rc == NGX_OK || rc == NGX_DECLINED) { diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index eda0141f71..0b70aea722 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -258,6 +258,7 @@ ngx_http_lua_sema_resume(ngx_http_request_t *r) lua_State *vm; ngx_connection_t *c; ngx_int_t rc; + ngx_uint_t nreqs; ngx_http_lua_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -269,6 +270,7 @@ ngx_http_lua_sema_resume(ngx_http_request_t *r) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; if (ctx->cur_co_ctx->sem_resume_status == SEMAPHORE_WAIT_SUCC) { lua_pushboolean(ctx->cur_co_ctx->co, 1); @@ -285,12 +287,12 @@ ngx_http_lua_sema_resume(ngx_http_request_t *r) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } /* rc == NGX_ERROR || rc >= NGX_OK */ diff --git a/src/ngx_http_lua_sleep.c b/src/ngx_http_lua_sleep.c index ffee97f84b..09ea0f6d40 100644 --- a/src/ngx_http_lua_sleep.c +++ b/src/ngx_http_lua_sleep.c @@ -180,6 +180,7 @@ ngx_http_lua_sleep_resume(ngx_http_request_t *r) lua_State *vm; ngx_connection_t *c; ngx_int_t rc; + ngx_uint_t nreqs; ngx_http_lua_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -191,6 +192,7 @@ ngx_http_lua_sleep_resume(ngx_http_request_t *r) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; rc = ngx_http_lua_run_thread(vm, r, ctx, 0); @@ -198,12 +200,12 @@ ngx_http_lua_sleep_resume(ngx_http_request_t *r) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (ctx->entered_content_phase) { diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 9eda286bb7..6ec37a7c52 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5256,6 +5256,7 @@ ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) int nret; lua_State *vm; ngx_int_t rc; + ngx_uint_t nreqs; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx; @@ -5306,6 +5307,7 @@ ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; rc = ngx_http_lua_run_thread(vm, r, ctx, nret); @@ -5313,12 +5315,12 @@ ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (ctx->entered_content_phase) { diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 1ec0c00a0f..08ba1cc4d7 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -1514,6 +1514,7 @@ ngx_http_lua_socket_udp_resume(ngx_http_request_t *r) int nret; lua_State *vm; ngx_int_t rc; + ngx_uint_t nreqs; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx; @@ -1549,6 +1550,7 @@ ngx_http_lua_socket_udp_resume(ngx_http_request_t *r) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; rc = ngx_http_lua_run_thread(vm, r, ctx, nret); @@ -1556,12 +1558,12 @@ ngx_http_lua_socket_udp_resume(ngx_http_request_t *r) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (ctx->entered_content_phase) { diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 6f7aa970a8..47096e9511 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1587,6 +1587,7 @@ ngx_http_lua_subrequest_resume(ngx_http_request_t *r) { lua_State *vm; ngx_int_t rc; + ngx_uint_t nreqs; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx; @@ -1620,6 +1621,7 @@ ngx_http_lua_subrequest_resume(ngx_http_request_t *r) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; rc = ngx_http_lua_run_thread(vm, r, ctx, coctx->nsubreqs); @@ -1627,12 +1629,12 @@ ngx_http_lua_subrequest_resume(ngx_http_request_t *r) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } /* rc == NGX_ERROR || rc >= NGX_OK */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c7bee3e89a..df0aae8137 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -3055,13 +3055,13 @@ ngx_http_lua_create_co_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) /* this is for callers other than the content handler */ ngx_int_t ngx_http_lua_run_posted_threads(ngx_connection_t *c, lua_State *L, - ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) + ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_uint_t nreqs) { ngx_int_t rc; ngx_http_lua_posted_thread_t *pt; for ( ;; ) { - if (c->destroyed) { + if (c->destroyed || c->requests != nreqs) { return NGX_DONE; } @@ -3461,6 +3461,7 @@ ngx_http_lua_on_abort_resume(ngx_http_request_t *r) { lua_State *vm; ngx_int_t rc; + ngx_uint_t nreqs; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; @@ -3480,6 +3481,7 @@ ngx_http_lua_on_abort_resume(ngx_http_request_t *r) c = r->connection; vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; rc = ngx_http_lua_run_thread(vm, r, ctx, 0); @@ -3487,12 +3489,12 @@ ngx_http_lua_on_abort_resume(ngx_http_request_t *r) "lua run thread returned %d", rc); if (rc == NGX_AGAIN) { - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (rc == NGX_DONE) { ngx_http_lua_finalize_request(r, NGX_DONE); - return ngx_http_lua_run_posted_threads(c, vm, r, ctx); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); } if (ctx->entered_content_phase) { diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index a37852f27e..2f995e0455 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -205,7 +205,7 @@ ngx_http_lua_co_ctx_t *ngx_http_lua_create_co_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx); ngx_int_t ngx_http_lua_run_posted_threads(ngx_connection_t *c, lua_State *L, - ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx); + ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_uint_t nreqs); ngx_int_t ngx_http_lua_post_thread(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t *coctx); diff --git a/t/017-exec.t b/t/017-exec.t index 535c4ab8ee..544b8bb898 100644 --- a/t/017-exec.t +++ b/t/017-exec.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 4); +plan tests => repeat_each() * (blocks() * 2 + 8); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; @@ -572,3 +572,25 @@ hello, bah ["dummy", "dummy"] --- no_error_log [error] + + + +=== TEST 25: pipelined requests +--- config + location /t { + content_by_lua_block { + ngx.exec("@foo") + } + } + + location @foo { + return 200; + } +--- pipelined_requests eval +["GET /t", "GET /t"] +--- error_code eval +[200, 200] +--- response_body eval +["", ""] +--- no_error_log +[error] diff --git a/t/023-rewrite/exec.t b/t/023-rewrite/exec.t index bd97968bc7..edd4607355 100644 --- a/t/023-rewrite/exec.t +++ b/t/023-rewrite/exec.t @@ -5,7 +5,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => blocks() * repeat_each() * 2; +plan tests => repeat_each() * (blocks() * 2 + 4); #no_diff(); #no_long_string(); @@ -376,3 +376,25 @@ ngx.exec("@proxy") GET /main --- response_body hello, bah + + + +=== TEST 17: pipelined requests +--- config + location /t { + rewrite_by_lua_block { + ngx.exec("@foo") + } + } + + location @foo { + return 200; + } +--- pipelined_requests eval +["GET /t", "GET /t"] +--- error_code eval +[200, 200] +--- response_body eval +["", ""] +--- no_error_log +[error] diff --git a/t/024-access/exec.t b/t/024-access/exec.t index 3fb87be9a8..43c1a7752a 100644 --- a/t/024-access/exec.t +++ b/t/024-access/exec.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 2); +plan tests => repeat_each() * (blocks() * 2 + 6); #no_diff(); #no_long_string(); @@ -369,3 +369,25 @@ GET /read 'unsafe URI "/hi/../" was detected', qr/runtime error: access_by_lua\(nginx.conf:\d+\):2: unsafe uri/, ] + + + +=== TEST 17: pipelined requests +--- config + location /t { + access_by_lua_block { + ngx.exec("@foo") + } + } + + location @foo { + return 200; + } +--- pipelined_requests eval +["GET /t", "GET /t"] +--- error_code eval +[200, 200] +--- response_body eval +["", ""] +--- no_error_log +[error] From 1626aaa782d43314c0be5d1909ce109de265bfbb Mon Sep 17 00:00:00 2001 From: "Oleg A. Mamontov" Date: Tue, 19 Sep 2017 18:00:10 +0300 Subject: [PATCH 138/848] bugfix: *_by_lua_block might break nginx config dump (-T switch). Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_directive.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 6a562f47b4..014a47237c 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1542,7 +1542,7 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, #if nginx_version >= 1009002 if (dump) { - dump->last = ngx_cpymem(dump->last, b->pos, size); + dump->last = ngx_cpymem(dump->last, b->start + len, size); } #endif } From 837cecb647a6fd2e2e09599a4b18861845eeff29 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 19 Sep 2017 12:05:28 -0700 Subject: [PATCH 139/848] tests: fixed ssl ciphers to reflect recent changes in openresty.org's mini CDN network. --- t/129-ssl-socket.t | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 1c3f7cd016..e7447a7ced 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1181,7 +1181,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log lua ssl server name: "openresty.org" -SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 +SSL: TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 --- no_error_log SSL reused session [error] @@ -1261,7 +1261,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log lua ssl server name: "openresty.org" -SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-SHA +SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-SHA SSLv3 --- no_error_log SSL reused session [error] @@ -1341,7 +1341,7 @@ lua ssl free session: ([0-9A-F]+):1 $/ --- error_log lua ssl server name: "openresty.org" -SSL: TLSv1, cipher: "ECDHE-RSA-AES256-SHA +SSL: TLSv1, cipher: "ECDHE-RSA-AES128-SHA SSLv3 --- no_error_log SSL reused session [error] From ac7615a2863f19244d483501421cfbac40eaa55a Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 19 Sep 2017 18:37:06 -0700 Subject: [PATCH 140/848] dev: util/build.sh: specify --with-pcre-jit for the nginx build. --- util/build.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/util/build.sh b/util/build.sh index e45c00a73f..528fd189af 100755 --- a/util/build.sh +++ b/util/build.sh @@ -23,6 +23,7 @@ force=$2 #--with-http_spdy_module \ time ngx-build $force $version \ + --with-pcre-jit \ --with-ipv6 \ --with-cc-opt="-I$PCRE_INC -I$OPENSSL_INC" \ --with-http_v2_module \ From c338c7b3fd34dd20ce9f5a27856bf68c471db4b1 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 19 Sep 2017 18:39:48 -0700 Subject: [PATCH 141/848] api: bumped version to 0.10.11. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index cd64fc8c35..1e07b7185a 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10010 +#define ngx_http_lua_version 10011 typedef struct { From 9a81a68641514f4ca12412ba145fa908ae9b2973 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 30 Aug 2017 17:39:00 -0700 Subject: [PATCH 142/848] feature: added pure C functions for shdict:ttl() and shdict:expire() API functions. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_shdict.c | 140 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index d6cad7e5e2..46c2a0af6b 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2843,6 +2843,146 @@ ngx_http_lua_ffi_shdict_flush_all(ngx_shm_zone_t *zone) return NGX_OK; } + + +static ngx_int_t +ngx_http_lua_shdict_peek(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, + u_char *kdata, size_t klen, ngx_http_lua_shdict_node_t **sdp) +{ + ngx_int_t rc; + ngx_rbtree_node_t *node, *sentinel; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + + ctx = shm_zone->data; + + node = ctx->sh->rbtree.root; + sentinel = ctx->sh->rbtree.sentinel; + + while (node != sentinel) { + + if (hash < node->key) { + node = node->left; + continue; + } + + if (hash > node->key) { + node = node->right; + continue; + } + + /* hash == node->key */ + + sd = (ngx_http_lua_shdict_node_t *) &node->color; + + rc = ngx_memn2cmp(kdata, sd->data, klen, (size_t) sd->key_len); + + if (rc == 0) { + *sdp = sd; + + return NGX_OK; + } + + node = (rc < 0) ? node->left : node->right; + } + + *sdp = NULL; + + return NGX_DECLINED; +} + + +int +ngx_http_lua_ffi_shdict_get_ttl(ngx_shm_zone_t *zone, u_char *key, + size_t key_len) +{ + uint32_t hash; + uint64_t now; + uint64_t expires; + ngx_int_t rc; + ngx_time_t *tp; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + + if (zone == NULL) { + return NGX_ERROR; + } + + ctx = zone->data; + hash = ngx_crc32_short(key, key_len); + + ngx_shmtx_lock(&ctx->shpool->mutex); + + rc = ngx_http_lua_shdict_peek(zone, hash, key, key_len, &sd); + + if (rc == NGX_DECLINED) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return NGX_DECLINED; + } + + /* rc == NGX_OK */ + + expires = sd->expires; + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + if (expires == 0) { + return 0; + } + + tp = ngx_timeofday(); + now = (uint64_t) tp->sec * 1000 + tp->msec; + + return expires - now; +} + + +int +ngx_http_lua_ffi_shdict_set_expire(ngx_shm_zone_t *zone, u_char *key, + size_t key_len, int exptime) +{ + uint32_t hash; + ngx_int_t rc; + ngx_time_t *tp = NULL; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + + if (zone == NULL) { + return NGX_ERROR; + } + + if (exptime > 0) { + tp = ngx_timeofday(); + } + + ctx = zone->data; + hash = ngx_crc32_short(key, key_len); + + ngx_shmtx_lock(&ctx->shpool->mutex); + + rc = ngx_http_lua_shdict_peek(zone, hash, key, key_len, &sd); + + if (rc == NGX_DECLINED) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return NGX_DECLINED; + } + + /* rc == NGX_OK */ + + if (exptime > 0) { + sd->expires = (uint64_t) tp->sec * 1000 + tp->msec + + (uint64_t) exptime; + + } else { + sd->expires = 0; + } + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return NGX_OK; +} #endif /* NGX_LUA_NO_FFI_API */ From 37cbafe4ebdbe91611bfda4452d3af3323b8d787 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 21 Sep 2017 15:54:37 -0700 Subject: [PATCH 143/848] tests: minor tweaks for the http2 test mode. --- .travis.yml | 1 + t/000--init.t | 73 ++++++++++++++++++++++++------------------------- t/002-content.t | 8 +++++- 3 files changed, 44 insertions(+), 38 deletions(-) diff --git a/.travis.yml b/.travis.yml index c30fb55f25..d2e55906c4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -84,6 +84,7 @@ install: - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache + - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git before_script: diff --git a/t/000--init.t b/t/000--init.t index ad2d70e76d..dbe2c33099 100644 --- a/t/000--init.t +++ b/t/000--init.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 1); +plan tests => repeat_each() * (blocks() * 3); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_MYSQL_PORT} ||= 3306; @@ -14,9 +14,13 @@ our $http_config = <<'_EOC_'; drizzle_server 127.0.0.1:$TEST_NGINX_MYSQL_PORT protocol=mysql dbname=ngx_test user=ngx_test password=ngx_test; } + + lua_package_path "../lua-resty-mysql/lib/?.lua;;"; _EOC_ no_shuffle(); +no_long_string(); + run_tests(); __DATA__ @@ -25,51 +29,46 @@ __DATA__ --- http_config eval: $::http_config --- config location = /init { - drizzle_pass database; - drizzle_query "DROP TABLE IF EXISTS conv_uid"; - } ---- request -GET /init ---- error_code: 200 ---- timeout: 10 ---- no_error_log -[error] - - - -=== TEST 2: conv_uid - create table ---- http_config eval: $::http_config ---- config - location = /init { - drizzle_pass database; - drizzle_query "CREATE TABLE conv_uid(id serial primary key, new_uid integer, old_uid integer)"; + content_by_lua_block { + local mysql = require "resty.mysql" + local db = assert(mysql:new()) + local ok, err, errcode, sqlstate = db:connect{ + host = "127.0.0.1", + port = $TEST_NGINX_MYSQL_PORT, + database = "ngx_test", + user = "ngx_test", + password = "ngx_test", + charset = "utf8", + } + + local queries = { + "DROP TABLE IF EXISTS conv_uid", + "CREATE TABLE conv_uid(id serial primary key, new_uid integer, old_uid integer)", + "INSERT INTO conv_uid(old_uid,new_uid) VALUES(32,56),(35,78)", + } + + for _, query in ipairs(queries) do + local ok, err = db:query(query) + if not ok then + ngx.say("failed to run mysql query \"", query, "\": ", err) + return + end + end + + ngx.say("done!") + } } --- request GET /init ---- error_code: 200 ---- timeout: 10 ---- no_error_log -[error] - - - -=== TEST 3: conv_uid - insert value ---- http_config eval: $::http_config ---- config - location = /init { - drizzle_pass database; - drizzle_query "INSERT INTO conv_uid(old_uid,new_uid) VALUES(32,56),(35,78)"; - } ---- request -GET /init ---- error_code: 200 +--- response_body +done! --- timeout: 10 --- no_error_log [error] -=== TEST 4: flush data from memcached +=== TEST 2: flush data from memcached --- config location /flush { set $memc_cmd flush_all; diff --git a/t/002-content.t b/t/002-content.t index 3f2460e676..cc92d6f756 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -10,7 +10,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 19); +plan tests => repeat_each() * (blocks() * 2 + 23); #no_diff(); #no_long_string(); @@ -539,6 +539,9 @@ GET /main Content-Length: 12 --- response_body chop hello, world +--- no_error_log +[error] +[alert] @@ -761,6 +764,9 @@ Content-Length: 13 hello, world --- timeout: 5 +--- no_error_log +[error] +[alert] From 1d01a6d662b1eafbb2df247c0e842f66f1b1be59 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 20 Sep 2017 19:58:11 -0700 Subject: [PATCH 144/848] doc: documented the shdict:ttl() and shdict:expire() API functions. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 80 ++++++++++++++++++++++++++++++++++++++++++ doc/HttpLuaModule.wiki | 70 ++++++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+) diff --git a/README.markdown b/README.markdown index 9dec49fe5d..1c92932346 100644 --- a/README.markdown +++ b/README.markdown @@ -3191,6 +3191,8 @@ Nginx API for Lua * [ngx.shared.DICT.lpop](#ngxshareddictlpop) * [ngx.shared.DICT.rpop](#ngxshareddictrpop) * [ngx.shared.DICT.llen](#ngxshareddictllen) +* [ngx.shared.DICT.ttl](#ngxshareddictttl) +* [ngx.shared.DICT.expire](#ngxshareddictexpire) * [ngx.shared.DICT.flush_all](#ngxshareddictflush_all) * [ngx.shared.DICT.flush_expired](#ngxshareddictflush_expired) * [ngx.shared.DICT.get_keys](#ngxshareddictget_keys) @@ -6198,6 +6200,8 @@ The resulting object `dict` has the following methods: * [lpop](#ngxshareddictlpop) * [rpop](#ngxshareddictrpop) * [llen](#ngxshareddictllen) +* [ttl](#ngxshareddictttl) +* [expire](#ngxshareddictexpire) * [flush_all](#ngxshareddictflush_all) * [flush_expired](#ngxshareddictflush_expired) * [get_keys](#ngxshareddictget_keys) @@ -6543,6 +6547,82 @@ See also [ngx.shared.DICT](#ngxshareddict). [Back to TOC](#nginx-api-for-lua) +ngx.shared.DICT.ttl +------------------- +**syntax:** *ttl, err = ngx.shared.DICT:ttl(key)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** + +**requires:** `resty.core.shdict` or `resty.core` + +Retrieves the remaining TTL (time-to-live in seconds) of a key-value pair in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the TTL as a number if the operation is successfully completed or `nil` and an error message otherwise. + +If the key does not exist (or has already expired), this method will return `nil` and the error string `"not found"`. + +The TTL is originally determined by the `exptime` argument of the [set](#ngxshareddictset), [add](#ngxshareddictadd), [replace](#ngxshareddictreplace) (and the likes) methods. It has a time resolution of `0.001` seconds. A value of `0` means that the item will never expire. + +Example: + +```lua + + require "resty.core" + + local cats = ngx.shared.cats + local succ, err = cats:set("Marry", "a nice cat", 0.5) + + ngx.sleep(0.2) + + local ttl, err = cats:ttl("Marry") + ngx.say(ttl) -- 0.3 +``` + +This feature was first introduced in the `v0.10.11` release. + +**Note:** This method requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.expire +---------------------- +**syntax:** *success, err = ngx.shared.DICT:expire(key, exptime)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** + +**requires:** `resty.core.shdict` or `resty.core` + +Updates the `exptime` (in second) of a key-value pair in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns a boolean indicating success if the operation completes or `nil` and an error message otherwise. + +If the key does not exist, this method will return `nil` and the error string `"not found"`. + +The `exptime` argument has a resolution of `0.001` seconds. If `exptime` is `0`, then the item will never expire. + +Example: + +```lua + + require "resty.core" + + local cats = ngx.shared.cats + local succ, err = cats:set("Marry", "a nice cat", 0.1) + + succ, err = cats:expire("Marry", 0.5) + + ngx.sleep(0.2) + + local val, err = cats:get("Marry") + ngx.say(val) -- "a nice cat" +``` + +This feature was first introduced in the `v0.10.11` release. + +**Note:** This method requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + ngx.shared.DICT.flush_all ------------------------- **syntax:** *ngx.shared.DICT:flush_all()* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 2518196fc0..45a0176b27 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5196,6 +5196,8 @@ The resulting object dict has the following methods: * [[#ngx.shared.DICT.lpop|lpop]] * [[#ngx.shared.DICT.rpop|rpop]] * [[#ngx.shared.DICT.llen|llen]] +* [[#ngx.shared.DICT.ttl|ttl]] +* [[#ngx.shared.DICT.expire|expire]] * [[#ngx.shared.DICT.flush_all|flush_all]] * [[#ngx.shared.DICT.flush_expired|flush_expired]] * [[#ngx.shared.DICT.get_keys|get_keys]] @@ -5491,6 +5493,74 @@ This feature was first introduced in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. +== ngx.shared.DICT.ttl == +'''syntax:''' ''ttl, err = ngx.shared.DICT:ttl(key)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' + +'''requires:''' resty.core.shdict or resty.core + +Retrieves the remaining TTL (time-to-live in seconds) of a key-value pair in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the TTL as a number if the operation is successfully completed or nil and an error message otherwise. + +If the key does not exist (or has already expired), this method will return nil and the error string "not found". + +The TTL is originally determined by the exptime argument of the [[#ngx.shared.DICT.set|set]], [[#ngx.shared.DICT.add|add]], [[#ngx.shared.DICT.replace|replace]] (and the likes) methods. It has a time resolution of 0.001 seconds. A value of 0 means that the item will never expire. + +Example: + + + require "resty.core" + + local cats = ngx.shared.cats + local succ, err = cats:set("Marry", "a nice cat", 0.5) + + ngx.sleep(0.2) + + local ttl, err = cats:ttl("Marry") + ngx.say(ttl) -- 0.3 + + +This feature was first introduced in the v0.10.11 release. + +'''Note:''' This method requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.expire == +'''syntax:''' ''success, err = ngx.shared.DICT:expire(key, exptime)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' + +'''requires:''' resty.core.shdict or resty.core + +Updates the exptime (in second) of a key-value pair in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns a boolean indicating success if the operation completes or nil and an error message otherwise. + +If the key does not exist, this method will return nil and the error string "not found". + +The exptime argument has a resolution of 0.001 seconds. If exptime is 0, then the item will never expire. + +Example: + + + require "resty.core" + + local cats = ngx.shared.cats + local succ, err = cats:set("Marry", "a nice cat", 0.1) + + succ, err = cats:expire("Marry", 0.5) + + ngx.sleep(0.2) + + local val, err = cats:get("Marry") + ngx.say(val) -- "a nice cat" + + +This feature was first introduced in the v0.10.11 release. + +'''Note:''' This method requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + == ngx.shared.DICT.flush_all == '''syntax:''' ''ngx.shared.DICT:flush_all()'' From 3266b40c7c7a20ff81f8f2d0a2e0e514d0c7e949 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 21 Sep 2017 16:04:04 -0700 Subject: [PATCH 145/848] doc: typo fix from dongbeiouba in #1158. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 1c92932346..b9cbd313c4 100644 --- a/README.markdown +++ b/README.markdown @@ -2694,7 +2694,7 @@ ssl_session_store_by_lua_file **context:** *http* -**phase:** *right-before-SSL-handshake* +**phase:** *right-after-SSL-handshake* Equivalent to [ssl_session_store_by_lua_block](#ssl_session_store_by_lua_block), except that the file specified by `` contains the Lua code, or rather, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 45a0176b27..c4be2c4775 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2279,7 +2279,7 @@ Note that: this directive is only allowed to used in '''http context''' from the '''context:''' ''http'' -'''phase:''' ''right-before-SSL-handshake'' +'''phase:''' ''right-after-SSL-handshake'' Equivalent to [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua_block]], except that the file specified by contains the Lua code, or rather, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. From 4bbf39928cd070d81f349f78a718b0f5b3893711 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 22 Sep 2017 18:40:15 -0700 Subject: [PATCH 146/848] travis-ci: fixed the IPC::Run dep for Test::Nginx and also added nginx 1.13.5 to the matrix. --- .travis.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.travis.yml b/.travis.yml index d2e55906c4..7448f65ac2 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,6 +50,7 @@ env: - TEST_NGINX_SLEEP=0.006 matrix: - NGINX_VERSION=1.11.2 + - NGINX_VERSION=1.13.5 services: - memcache @@ -59,6 +60,7 @@ services: before_install: - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' + - sudo cpanm --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi From aee660bdf90ef7d86b03d1cbe44c4fa8aa57c2c0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 23 Sep 2017 12:58:06 -0700 Subject: [PATCH 147/848] bugfix: suppressed the gcc warning "comparison between signed and unsigned integer expressions". --- src/ngx_http_lua_socket_tcp.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 6ec37a7c52..578a173722 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2711,7 +2711,7 @@ ngx_http_lua_socket_tcp_settimeout(lua_State *L) } timeout = (ngx_int_t) lua_tonumber(L, 2); - if (timeout > NGX_MAX_INT32_VALUE) { + if (timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { return luaL_error(L, "lua tcp socket timeout %f will overflow", (lua_Number) timeout); } @@ -2759,19 +2759,19 @@ ngx_http_lua_socket_tcp_settimeouts(lua_State *L) } connect_timeout = (ngx_int_t) lua_tonumber(L, 2); - if (connect_timeout > NGX_MAX_INT32_VALUE) { + if (connect_timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { return luaL_error(L, "lua tcp socket connect timeout %f will overflow", (lua_Number) connect_timeout); } send_timeout = (ngx_int_t) lua_tonumber(L, 3); - if (send_timeout > NGX_MAX_INT32_VALUE) { + if (send_timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { return luaL_error(L, "lua tcp socket send timeout %f will overflow", (lua_Number) send_timeout); } read_timeout = (ngx_int_t) lua_tonumber(L, 4); - if (read_timeout > NGX_MAX_INT32_VALUE) { + if (read_timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { return luaL_error(L, "lua tcp socket read timeout %f will overflow", (lua_Number) read_timeout); } From afbfe4cf747d850a17b839242d353f043113cf2f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 23 Sep 2017 16:14:18 -0700 Subject: [PATCH 148/848] bugfix: the timeout overflow checks did not work on 32-bit systems. --- src/ngx_http_lua_socket_tcp.c | 20 ++++++++------------ t/147-tcp-socket-timeouts.t | 6 +++--- 2 files changed, 11 insertions(+), 15 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 578a173722..22b190464e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2711,9 +2711,8 @@ ngx_http_lua_socket_tcp_settimeout(lua_State *L) } timeout = (ngx_int_t) lua_tonumber(L, 2); - if (timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { - return luaL_error(L, "lua tcp socket timeout %f will overflow", - (lua_Number) timeout); + if (timeout >> 31) { + return luaL_error(L, "timeout value too large"); } lua_pushinteger(L, timeout); @@ -2759,21 +2758,18 @@ ngx_http_lua_socket_tcp_settimeouts(lua_State *L) } connect_timeout = (ngx_int_t) lua_tonumber(L, 2); - if (connect_timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { - return luaL_error(L, "lua tcp socket connect timeout %f will overflow", - (lua_Number) connect_timeout); + if (connect_timeout >> 31) { + return luaL_error(L, "timeout value too large"); } send_timeout = (ngx_int_t) lua_tonumber(L, 3); - if (send_timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { - return luaL_error(L, "lua tcp socket send timeout %f will overflow", - (lua_Number) send_timeout); + if (send_timeout >> 31) { + return luaL_error(L, "timeout value too large"); } read_timeout = (ngx_int_t) lua_tonumber(L, 4); - if (read_timeout > (ngx_int_t) NGX_MAX_INT32_VALUE) { - return luaL_error(L, "lua tcp socket read timeout %f will overflow", - (lua_Number) read_timeout); + if (read_timeout >> 31) { + return luaL_error(L, "timeout value too large"); } lua_rawseti(L, 1, SOCKET_READ_TIMEOUT_INDEX); diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index b0a4fa0415..d2a466f3cc 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -560,7 +560,7 @@ failed to receive a line: closed [] GET /t --- response_body_like settimeouts: ok -failed to set timeouts: lua tcp socket connect timeout 2147483648(?:\.\d+)? will overflow +failed to set timeouts: timeout value too large --- no_error_log [error] @@ -591,7 +591,7 @@ failed to set timeouts: lua tcp socket connect timeout 2147483648(?:\.\d+)? will GET /t --- response_body_like settimeouts: ok -failed to set timeouts: lua tcp socket send timeout 2147483648(?:\.\d+)? will overflow +failed to set timeouts: timeout value too large --- no_error_log [error] @@ -622,6 +622,6 @@ failed to set timeouts: lua tcp socket send timeout 2147483648(?:\.\d+)? will ov GET /t --- response_body_like settimeouts: ok -failed to set timeouts: lua tcp socket read timeout 2147483648(?:\.\d+)? will overflow +failed to set timeouts: timeout value too large --- no_error_log [error] From 93d1f30c3f04edf7d84fad22a76be900841915d4 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 23 Sep 2017 16:41:15 -0700 Subject: [PATCH 149/848] bugfix: a followup fix for the previous commit. --- src/ngx_http_lua_socket_tcp.c | 8 +++--- t/023-rewrite/tcp-socket-timeout.t | 44 ++++++++++++------------------ t/065-tcp-socket-timeout.t | 40 +++++++++++---------------- t/147-tcp-socket-timeouts.t | 6 ++-- 4 files changed, 41 insertions(+), 57 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 22b190464e..85f9f6a04a 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2712,7 +2712,7 @@ ngx_http_lua_socket_tcp_settimeout(lua_State *L) timeout = (ngx_int_t) lua_tonumber(L, 2); if (timeout >> 31) { - return luaL_error(L, "timeout value too large"); + return luaL_error(L, "bad timeout value"); } lua_pushinteger(L, timeout); @@ -2759,17 +2759,17 @@ ngx_http_lua_socket_tcp_settimeouts(lua_State *L) connect_timeout = (ngx_int_t) lua_tonumber(L, 2); if (connect_timeout >> 31) { - return luaL_error(L, "timeout value too large"); + return luaL_error(L, "bad timeout value"); } send_timeout = (ngx_int_t) lua_tonumber(L, 3); if (send_timeout >> 31) { - return luaL_error(L, "timeout value too large"); + return luaL_error(L, "bad timeout value"); } read_timeout = (ngx_int_t) lua_tonumber(L, 4); if (read_timeout >> 31) { - return luaL_error(L, "timeout value too large"); + return luaL_error(L, "bad timeout value"); } lua_rawseti(L, 1, SOCKET_READ_TIMEOUT_INDEX); diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index 15bec7fad7..a32ed7bed2 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -24,7 +24,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 14); +plan tests => repeat_each() * (blocks() * 4 + 8); our $HtmlDir = html_dir; @@ -174,7 +174,7 @@ lua tcp socket connect timeout: 102 -=== TEST 5: sock:settimeout(-1) does not override lua_socket_connect_timeout +=== TEST 5: -1 is bad timeout value --- config server_tokens off; lua_socket_connect_timeout 102ms; @@ -198,14 +198,12 @@ lua tcp socket connect timeout: 102 } --- request GET /t5 ---- response_body -failed to connect: timeout +--- response_body_like chomp +500 Internal Server Error --- error_log -lua tcp socket connect timeout: 102 ---- no_error_log -[error] -[alert] +bad timeout value --- timeout: 10 +--- error_code: 500 @@ -371,7 +369,7 @@ lua tcp socket read timed out -=== TEST 10: sock:settimeout(-1) does not override lua_socket_read_timeout +=== TEST 10: -1 is bad timeout value --- config server_tokens off; lua_socket_read_timeout 102ms; @@ -385,8 +383,6 @@ lua tcp socket read timed out return end - ngx.say("connected: ", ok) - sock:settimeout(-1) local line @@ -402,13 +398,12 @@ lua tcp socket read timed out } --- request GET /t ---- response_body -connected: 1 -failed to receive: timeout +--- response_body_like chomp +500 Internal Server Error --- error_log -lua tcp socket read timeout: 102 -lua tcp socket connect timeout: 60000 -lua tcp socket read timed out +bad timeout value +--- timeout: 10 +--- error_code: 500 @@ -574,7 +569,7 @@ lua tcp socket write timed out -=== TEST 15: sock:settimeout(-1) does not override lua_socket_send_timeout +=== TEST 15: -1 is bad timeout value --- config server_tokens off; lua_socket_send_timeout 102ms; @@ -588,8 +583,6 @@ lua tcp socket write timed out return end - ngx.say("connected: ", ok) - sock:settimeout(-1) local bytes @@ -605,10 +598,9 @@ lua tcp socket write timed out } --- request GET /t ---- response_body -connected: 1 -failed to send: timeout +--- response_body_like chomp +500 Internal Server Error --- error_log -lua tcp socket send timeout: 102 -lua tcp socket connect timeout: 60000 -lua tcp socket write timed out +bad timeout value +--- timeout: 10 +--- error_code: 500 diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 721c326a6c..9e5460e2cf 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -28,7 +28,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 11); +plan tests => repeat_each() * (blocks() * 4 + 6); our $HtmlDir = html_dir; @@ -159,7 +159,7 @@ lua tcp socket connect timed out -=== TEST 5: sock:settimeout(-1) does not override lua_socket_connect_timeout +=== TEST 5: -1 is bad timeout value --- config server_tokens off; lua_socket_connect_timeout 102ms; @@ -179,11 +179,11 @@ lua tcp socket connect timed out } --- request GET /t ---- response_body -failed to connect: timeout +--- response_body_like chomp +500 Internal Server Error --- error_log -lua tcp socket connect timeout: 102 -lua tcp socket connect timed out +bad timeout value +--- error_code: 500 @@ -342,7 +342,7 @@ lua tcp socket read timed out -=== TEST 10: sock:settimeout(-1) does not override lua_socket_read_timeout +=== TEST 10: -1 is bad timeout value --- config server_tokens off; lua_socket_read_timeout 102ms; @@ -356,8 +356,6 @@ lua tcp socket read timed out return end - ngx.say("connected: ", ok) - sock:settimeout(-1) local line @@ -371,13 +369,11 @@ lua tcp socket read timed out } --- request GET /t ---- response_body -connected: 1 -failed to receive: timeout +--- response_body_like chomp +500 Internal Server Error +--- error_code: 500 --- error_log -lua tcp socket read timeout: 102 -lua tcp socket connect timeout: 60000 -lua tcp socket read timed out +bad timeout value @@ -563,8 +559,6 @@ lua tcp socket write timed out return end - ngx.say("connected: ", ok) - sock:settimeout(-1) local bytes @@ -578,13 +572,11 @@ lua tcp socket write timed out } --- request GET /t ---- response_body -connected: 1 -failed to send: timeout +--- response_body_like chomp +500 Internal Server Error --- error_log -lua tcp socket send timeout: 102 -lua tcp socket connect timeout: 60000 -lua tcp socket write timed out +bad timeout value +--- error_code: 500 @@ -1021,6 +1013,6 @@ close: 1 nil GET /t --- response_body_like settimeout: ok -failed to set timeout: lua tcp socket timeout 2147483648(?:\.\d+)? will overflow +failed to set timeout: bad timeout value --- no_error_log [error] diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index d2a466f3cc..8199e846a7 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -560,7 +560,7 @@ failed to receive a line: closed [] GET /t --- response_body_like settimeouts: ok -failed to set timeouts: timeout value too large +failed to set timeouts: bad timeout value --- no_error_log [error] @@ -591,7 +591,7 @@ failed to set timeouts: timeout value too large GET /t --- response_body_like settimeouts: ok -failed to set timeouts: timeout value too large +failed to set timeouts: bad timeout value --- no_error_log [error] @@ -622,6 +622,6 @@ failed to set timeouts: timeout value too large GET /t --- response_body_like settimeouts: ok -failed to set timeouts: timeout value too large +failed to set timeouts: bad timeout value --- no_error_log [error] From e5d7fc9fd52a9655831c7c41879f89fd38e1b203 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 23 Sep 2017 16:41:49 -0700 Subject: [PATCH 150/848] tests: try to make a test case less possible to fail on slow machines. --- t/135-worker-id.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/135-worker-id.t b/t/135-worker-id.t index 984f44650a..55df6ebc2e 100644 --- a/t/135-worker-id.t +++ b/t/135-worker-id.t @@ -54,7 +54,7 @@ GET /lua content_by_lua_block { local counters = ngx.shared.counters local ok, c - for i = 1, 100 do + for i = 1, 200 do c = counters:get("c") if c >= 4 then ok = true From 0c8df2fa694e772538bb9c9123446d9aa8acaf59 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 24 Sep 2017 13:31:02 -0700 Subject: [PATCH 151/848] Revert "tests: try to make a test case less possible to fail on slow machines." This reverts commit e5d7fc9fd52a9655831c7c41879f89fd38e1b203. --- t/135-worker-id.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/135-worker-id.t b/t/135-worker-id.t index 55df6ebc2e..984f44650a 100644 --- a/t/135-worker-id.t +++ b/t/135-worker-id.t @@ -54,7 +54,7 @@ GET /lua content_by_lua_block { local counters = ngx.shared.counters local ok, c - for i = 1, 200 do + for i = 1, 100 do c = counters:get("c") if c >= 4 then ok = true From bf14723e4e7749c989134c029742185db1c78255 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 24 Sep 2017 22:57:15 -0700 Subject: [PATCH 152/848] tests: fixed a test case that might fail due to timing errors. --- t/106-timer.t | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/106-timer.t b/t/106-timer.t index 04a532e91c..3e4741e50c 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -174,7 +174,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:4[4-9]|5[0-6]) local begin = ngx.now() local function f() print("my lua timer handler") - ngx.sleep(0.02) + ngx.sleep(0.2) print("elapsed: ", ngx.now() - begin) end local ok, err = ngx.timer.at(0.05, f) @@ -199,7 +199,7 @@ delete thread 2 --- response_body registered timer ---- wait: 0.12 +--- wait: 0.3 --- no_error_log [error] [alert] @@ -208,7 +208,7 @@ registered timer --- error_log eval [ qr/\[lua\] .*? my lua timer handler/, -qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-6])/, +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:1[4-9]|2[0-6]?)/, "lua ngx.timer expired", "http lua close fake http connection" ] From 97fbeb0bef1aa85e758210d58063376de8eaed31 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 26 Sep 2017 11:08:23 -0700 Subject: [PATCH 153/848] bugfix: the ssl connections might be drained and reused prematurely when ssl_certificate_by_lua* or ssl_session_fetch_by_lua* were used. this might lead to segmentation faults under load. thanks guanglinlv for the report and the original patch in #1162. --- src/ngx_http_lua_ssl_certby.c | 5 ++++- src/ngx_http_lua_ssl_session_fetchby.c | 5 ++++- src/ngx_http_lua_ssl_session_storeby.c | 3 ++- t/139-ssl-cert-by.t | 8 +++++++- t/142-ssl-session-store.t | 7 ++++++- t/143-ssl-session-fetch.t | 19 ++++++++++++------- 6 files changed, 35 insertions(+), 12 deletions(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index c3591d1ea3..95be47f6b5 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -197,7 +197,8 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) c = ngx_ssl_get_connection(ssl_conn); - dd("c = %p", c); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "ssl cert: connection reusable: %ud", c->reusable); cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); @@ -220,6 +221,8 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) dd("first time"); + ngx_reusable_connection(c, 0); + hc = c->data; fc = ngx_http_lua_create_fake_connection(NULL); diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 556b73204c..3a3c1f54d8 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -191,7 +191,8 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, u_char *id, c = ngx_ssl_get_connection(ssl_conn); - dd("c = %p", c); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "ssl session fetch: connection reusable: %ud", c->reusable); cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); @@ -224,6 +225,8 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, u_char *id, dd("first time"); + ngx_reusable_connection(c, 0); + hc = c->data; fc = ngx_http_lua_create_fake_connection(NULL); diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index bae8273d69..f83e85d91b 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -183,7 +183,8 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, c = ngx_ssl_get_connection(ssl_conn); - dd("c = %p", c); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "ssl session store: connection reusable: %ud", c->reusable); cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index c13044fc28..b521e9e88b 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -113,11 +113,17 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_certificate_by_lua:1: ssl cert by lua is running! --- no_error_log [error] [alert] +--- grep_error_log eval: qr/ssl_certificate_by_lua:.*?,|\bssl cert: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log_out eval +qr/reusable connection: 1 +ssl cert: connection reusable: 1 +reusable connection: 0 +ssl_certificate_by_lua:1: ssl cert by lua is running!, +/ diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 73b6e1979b..825d016302 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -82,11 +82,16 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_session_store_by_lua_block:1: ssl session store by lua is running! --- no_error_log [error] [alert] +--- grep_error_log eval: qr/ssl_session_store_by_lua_block:.*?,|\bssl session store: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log_out eval +qr/^reusable connection: 0 +ssl session store: connection reusable: 0 +ssl_session_store_by_lua_block:1: ssl session store by lua is running!, +/m, diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 701ead7296..7be180f84f 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -83,16 +83,21 @@ connected: 1 ssl handshake: userdata close: 1 nil ---- grep_error_log eval -qr/ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!/s +--- grep_error_log eval: qr/ssl_session_fetch_by_lua_block:.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval [ -'', -'ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running! -', -'ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running! -', +qr/\A(?:reusable connection: [01]\n)+\z/s, +qr/^reusable connection: 1 +ssl session fetch: connection reusable: 1 +reusable connection: 0 +ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +/m, +qr/^reusable connection: 1 +ssl session fetch: connection reusable: 1 +reusable connection: 0 +ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +/m, ] --- no_error_log From 59dff5eec3afd87f02d426a74545bfa35b6e242f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 2 Oct 2017 19:27:19 -0700 Subject: [PATCH 154/848] tests: suppressed one more false positive. --- valgrind.suppress | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/valgrind.suppress b/valgrind.suppress index fb539f2ae0..fd9ee60963 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -174,3 +174,12 @@ fun:main fun:ngx_worker_process_init fun:ngx_worker_process_cycle } +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_create_pool + fun:main +} From f829065b794025c856c9f86d469395e464e782ed Mon Sep 17 00:00:00 2001 From: Hiroaki Nakamura Date: Wed, 30 Aug 2017 23:56:17 +0000 Subject: [PATCH 155/848] feature: shdict: added pure C API for getting free page size and total capacity for lua-resty-core. Signed-off-by: Hiroaki Nakamura Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 77 +++++++++++++++++++++++++++++++++++++++ doc/HttpLuaModule.wiki | 67 ++++++++++++++++++++++++++++++++++ src/ngx_http_lua_shdict.c | 27 ++++++++++++++ 3 files changed, 171 insertions(+) diff --git a/README.markdown b/README.markdown index b9cbd313c4..21bf2d0891 100644 --- a/README.markdown +++ b/README.markdown @@ -3196,6 +3196,8 @@ Nginx API for Lua * [ngx.shared.DICT.flush_all](#ngxshareddictflush_all) * [ngx.shared.DICT.flush_expired](#ngxshareddictflush_expired) * [ngx.shared.DICT.get_keys](#ngxshareddictget_keys) +* [ngx.shared.DICT.capacity](#ngxshareddictcapacity) +* [ngx.shared.DICT.free_space](#ngxshareddictfree_space) * [ngx.socket.udp](#ngxsocketudp) * [udpsock:setpeername](#udpsocksetpeername) * [udpsock:send](#udpsocksend) @@ -6205,6 +6207,8 @@ The resulting object `dict` has the following methods: * [flush_all](#ngxshareddictflush_all) * [flush_expired](#ngxshareddictflush_expired) * [get_keys](#ngxshareddictget_keys) +* [capacity](#ngxshareddictcapacity) +* [free_space](#ngxshareddictfree_space) All these methods are *atomic* operations, that is, safe from concurrent accesses from multiple nginx worker processes for the same `lua_shared_dict` zone. @@ -6669,6 +6673,79 @@ This feature was first introduced in the `v0.7.3` release. [Back to TOC](#nginx-api-for-lua) +ngx.shared.DICT.capacity +------------------------ +**syntax:** *capacity_bytes = ngx.shared.DICT:capacity()* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** + +**requires:** `resty.core.shdict` or `resty.core` + +Retrieves the capacity in bytes for the shm-based dictionary [ngx.shared.DICT](#ngxshareddict) declared with +the [lua_shared_dict](#lua_shared_dict) directive. + +Example: + +```lua + + require "resty.core.shdict" + + local cats = ngx.shared.cats + local capacity_bytes = cats:capacity() +``` + +This feature was first introduced in the `v0.10.11` release. + +**Note:** This method requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. + +This feature requires at least nginx core version `0.7.3`. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.free_space +-------------------------- +**syntax:** *free_page_bytes = ngx.shared.DICT:free_space()* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** + +**requires:** `resty.core.shdict` or `resty.core` + +Retrieves the free page size in bytes for the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). + +**Note:** The memory for ngx.shared.DICT is allocated via the nginx slab allocator which has each slot for +data size ranges like ~8, 9~16, 17~32, ..., 1025~2048, 2048~ bytes. And pages are assigned to a slot if there +is no room in already assigned pages for the slot. + +So even if the return value of the `free_space` method is zero, there may be room in already assigned pages, so +you may successfully set a new key value pair to the shared dict without getting `true` for `forcible` or +non nil `err` from the `ngx.shared.DICT.set`. + +On the other hand, if already assigned pages for a slot are full and a new key value pair is added to the +slot and there is no free page, you may get `true` for `forcible` or non nil `err` from the +`ngx.shared.DICT.set` method. + +Example: + +```lua + + require "resty.core.shdict" + + local cats = ngx.shared.cats + local free_page_bytes = cats:free_space() +``` + +This feature was first introduced in the `v0.10.11` release. + +**Note:** This method requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. + +This feature requires at least nginx core version `1.11.7`. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + ngx.socket.udp -------------- **syntax:** *udpsock = ngx.socket.udp()* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index c4be2c4775..4127365221 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5201,6 +5201,8 @@ The resulting object dict has the following methods: * [[#ngx.shared.DICT.flush_all|flush_all]] * [[#ngx.shared.DICT.flush_expired|flush_expired]] * [[#ngx.shared.DICT.get_keys|get_keys]] +* [[#ngx.shared.DICT.capacity|capacity]] +* [[#ngx.shared.DICT.free_space|free_space]] All these methods are ''atomic'' operations, that is, safe from concurrent accesses from multiple nginx worker processes for the same lua_shared_dict zone. @@ -5598,6 +5600,71 @@ By default, only the first 1024 keys (if any) are returned. When the v0.7.3 release. +== ngx.shared.DICT.capacity == +'''syntax:''' ''capacity_bytes = ngx.shared.DICT:capacity()'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' + +'''requires:''' resty.core.shdict or resty.core + +Retrieves the capacity in bytes for the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] declared with +the [[#lua_shared_dict|lua_shared_dict]] directive. + +Example: + + + require "resty.core.shdict" + + local cats = ngx.shared.cats + local capacity_bytes = cats:capacity() + + +This feature was first introduced in the v0.10.11 release. + +'''Note:''' This method requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. + +This feature requires at least nginx core version 0.7.3. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.free_space == +'''syntax:''' ''free_page_bytes = ngx.shared.DICT:free_space()'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' + +'''requires:''' resty.core.shdict or resty.core + +Retrieves the free page size in bytes for the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. + +'''Note:''' The memory for ngx.shared.DICT is allocated via the nginx slab allocator which has each slot for +data size ranges like ~8, 9~16, 17~32, ..., 1025~2048, 2048~ bytes. And pages are assigned to a slot if there +is no room in already assigned pages for the slot. + +So even if the return value of the free_space method is zero, there may be room in already assigned pages, so +you may successfully set a new key value pair to the shared dict without getting true for forcible or +non nil err from the ngx.shared.DICT.set. + +On the other hand, if already assigned pages for a slot are full and a new key value pair is added to the +slot and there is no free page, you may get true for forcible or non nil err from the +ngx.shared.DICT.set method. + +Example: + + + require "resty.core.shdict" + + local cats = ngx.shared.cats + local free_page_bytes = cats:free_space() + + +This feature was first introduced in the v0.10.11 release. + +'''Note:''' This method requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. + +This feature requires at least nginx core version 1.11.7. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + == ngx.socket.udp == '''syntax:''' ''udpsock = ngx.socket.udp()'' diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 46c2a0af6b..5b48eb4b0d 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2983,6 +2983,33 @@ ngx_http_lua_ffi_shdict_set_expire(ngx_shm_zone_t *zone, u_char *key, return NGX_OK; } + + +size_t +ngx_http_lua_ffi_shdict_capacity(ngx_shm_zone_t *zone) +{ + return zone->shm.size; +} + + +# if nginx_version >= 1011007 +size_t +ngx_http_lua_ffi_shdict_free_space(ngx_shm_zone_t *zone) +{ + size_t bytes; + ngx_http_lua_shdict_ctx_t *ctx; + + ctx = zone->data; + + ngx_shmtx_lock(&ctx->shpool->mutex); + bytes = ctx->shpool->pfree * ngx_pagesize; + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return bytes; +} +# endif /* nginx_version >= 1011007 */ + + #endif /* NGX_LUA_NO_FFI_API */ From 8dd4bdef6f78495bdc1a7cce92eed494a70cc0af Mon Sep 17 00:00:00 2001 From: Hiroaki Nakamura Date: Sat, 7 Oct 2017 23:28:22 +0000 Subject: [PATCH 156/848] doc: fixed strike-through in shdict.free_space. Signed-off-by: Hiroaki Nakamura Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 21bf2d0891..28ba6e68c0 100644 --- a/README.markdown +++ b/README.markdown @@ -6715,7 +6715,7 @@ ngx.shared.DICT.free_space Retrieves the free page size in bytes for the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). **Note:** The memory for ngx.shared.DICT is allocated via the nginx slab allocator which has each slot for -data size ranges like ~8, 9~16, 17~32, ..., 1025~2048, 2048~ bytes. And pages are assigned to a slot if there +data size ranges like \~8, 9\~16, 17\~32, ..., 1025\~2048, 2048\~ bytes. And pages are assigned to a slot if there is no room in already assigned pages for the slot. So even if the return value of the `free_space` method is zero, there may be room in already assigned pages, so diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4127365221..8ad2b2d4a9 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5637,7 +5637,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. Retrieves the free page size in bytes for the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. '''Note:''' The memory for ngx.shared.DICT is allocated via the nginx slab allocator which has each slot for -data size ranges like ~8, 9~16, 17~32, ..., 1025~2048, 2048~ bytes. And pages are assigned to a slot if there +data size ranges like \~8, 9\~16, 17\~32, ..., 1025\~2048, 2048\~ bytes. And pages are assigned to a slot if there is no room in already assigned pages for the slot. So even if the return value of the free_space method is zero, there may be room in already assigned pages, so From 5900df4a6d212cdfd5962286e9018996d697aa7d Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Tue, 24 Oct 2017 01:49:19 -0700 Subject: [PATCH 157/848] doc: added "413 Request Entity Too Large" to the possible short circuit response list. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 1 + doc/HttpLuaModule.wiki | 1 + 2 files changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index 28ba6e68c0..9581f7eb16 100644 --- a/README.markdown +++ b/README.markdown @@ -871,6 +871,7 @@ Nginx may terminate a request early with (at least): * 400 (Bad Request) * 405 (Not Allowed) * 408 (Request Timeout) +* 413 (Request Entity Too Large) * 414 (Request URI Too Large) * 494 (Request Headers Too Large) * 499 (Client Closed Request) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 8ad2b2d4a9..f216c43578 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -706,6 +706,7 @@ Nginx may terminate a request early with (at least): * 400 (Bad Request) * 405 (Not Allowed) * 408 (Request Timeout) +* 413 (Request Entity Too Large) * 414 (Request URI Too Large) * 494 (Request Headers Too Large) * 499 (Client Closed Request) From 56d3098816fedce98c2a22cf3e83792c1b151d7a Mon Sep 17 00:00:00 2001 From: spacewander Date: Sun, 22 Oct 2017 00:20:45 +0800 Subject: [PATCH 158/848] bugfix: add fallthrough comment to silence GCC 7's -Wimplicit-fallthrough. thanks Andriy Kornatskyy for the report in #1171. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 85f9f6a04a..f0988bc51a 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -474,6 +474,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) switch (lua_type(L, -1)) { case LUA_TNUMBER: lua_tostring(L, -1); + /* FALLTHROUGH */ case LUA_TSTRING: custom_pool = 1; From 7730490c1e98a5867da77dc72814fc896000a769 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 25 Oct 2017 21:57:14 -0700 Subject: [PATCH 159/848] dev: util/build.sh: now we require ngx_stream_lua_module to build (for testing and development). --- .travis.yml | 2 +- util/build.sh | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 7448f65ac2..8bfa93e4cc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,6 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.11.2 - NGINX_VERSION=1.13.5 services: @@ -87,6 +86,7 @@ install: - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql + - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git before_script: diff --git a/util/build.sh b/util/build.sh index 528fd189af..164bf9f9a2 100755 --- a/util/build.sh +++ b/util/build.sh @@ -36,6 +36,8 @@ time ngx-build $force $version \ --without-mail_imap_module \ --with-http_image_filter_module \ --without-mail_smtp_module \ + --with-stream \ + --with-stream_ssl_module \ --without-http_upstream_ip_hash_module \ --without-http_memcached_module \ --without-http_auth_basic_module \ @@ -51,6 +53,7 @@ time ngx-build $force $version \ --add-module=$root/../rds-json-nginx-module \ --add-module=$root/../coolkit-nginx-module \ --add-module=$root/../redis2-nginx-module \ + --add-module=$root/../stream-lua-nginx-module \ --add-module=$root/t/data/fake-module \ --add-module=$root/t/data/fake-shm-module \ --add-module=$root/t/data/fake-delayed-load-module \ From 201f7c051c54396095efbf8969cbce03e79588fc Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 26 Oct 2017 10:50:40 -0700 Subject: [PATCH 160/848] travis: bumped nginx core version to 1.13.6. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 8bfa93e4cc..ca12a05f5f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,7 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.13.5 + - NGINX_VERSION=1.13.6 services: - memcache From fd5201c5713f4c219268235b917261c78087ef3d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 27 Oct 2017 13:13:35 -0700 Subject: [PATCH 161/848] tests: valgrind.suppress: removed too aggressive suppressions in nginx mem pools and luajit lj_str_new. --- valgrind.suppress | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/valgrind.suppress b/valgrind.suppress index fd9ee60963..924a15cc7c 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -1,16 +1,16 @@ { - -Memcheck:Addr1 -fun:ngx_init_cycle -fun:ngx_master_process_cycle -fun:main + + Memcheck:Addr1 + fun:ngx_init_cycle + fun:ngx_master_process_cycle + fun:main } { - -Memcheck:Addr4 -fun:ngx_init_cycle -fun:ngx_master_process_cycle -fun:main + + Memcheck:Addr4 + fun:ngx_init_cycle + fun:ngx_master_process_cycle + fun:main } { @@ -104,11 +104,11 @@ fun:main fun:ngx_event_process_init } { - - Memcheck:Param - sendmsg(mmsg[0].msg_hdr) - fun:sendmmsg - fun:__libc_res_nsend + + Memcheck:Param + sendmsg(mmsg[0].msg_hdr) + fun:sendmmsg + fun:__libc_res_nsend } { @@ -120,11 +120,11 @@ fun:main fun:ngx_start_cache_manager_processes } { - - Memcheck:Cond - fun:ngx_init_cycle - fun:ngx_master_process_cycle - fun:main + + Memcheck:Cond + fun:ngx_init_cycle + fun:ngx_master_process_cycle + fun:main } { From 5f766faf09fc67914f4896663cea80d994c64be9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 27 Oct 2017 13:53:25 -0700 Subject: [PATCH 162/848] tests: added new valgrind false positives in the latest nginx core. --- valgrind.suppress | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/valgrind.suppress b/valgrind.suppress index 924a15cc7c..6f688ed81d 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -183,3 +183,12 @@ fun:ngx_create_pool fun:main } +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_set_environment + fun:ngx_single_process_cycle +} From 38780280c0b0194b8fb822ec666de1f0432b5797 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 27 Oct 2017 21:37:10 -0700 Subject: [PATCH 163/848] updated valgrind suppressions for the hv test mode. --- valgrind.suppress | 2 -- 1 file changed, 2 deletions(-) diff --git a/valgrind.suppress b/valgrind.suppress index 6f688ed81d..727398637e 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -157,7 +157,6 @@ fun:ngx_alloc fun:ngx_set_environment fun:ngx_single_process_cycle - fun:main } { @@ -172,7 +171,6 @@ fun:ngx_alloc fun:ngx_set_environment fun:ngx_worker_process_init - fun:ngx_worker_process_cycle } { From 4b871b45e8267a266585575a42b2b2977ff272ca Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 27 Oct 2017 21:49:37 -0700 Subject: [PATCH 164/848] tests: made one stap-based test case less possible to fail due to timing error. --- t/097-uthread-rewrite.t | 1 + 1 file changed, 1 insertion(+) diff --git a/t/097-uthread-rewrite.t b/t/097-uthread-rewrite.t index a93adc8604..2f0c06fb43 100644 --- a/t/097-uthread-rewrite.t +++ b/t/097-uthread-rewrite.t @@ -258,6 +258,7 @@ free request hello foo --- no_error_log [error] +--- wait: 0.1 From c6089171dea9bd86c313fdb727d5b5be383b0b9d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 29 Oct 2017 11:20:15 -0700 Subject: [PATCH 165/848] tests: added full GC cycle operations to avoid false positives in the "check leak" test mode. --- t/004-require.t | 1 + t/035-gmatch.t | 1 + 2 files changed, 2 insertions(+) diff --git a/t/004-require.t b/t/004-require.t index ec74116647..35e04db814 100644 --- a/t/004-require.t +++ b/t/004-require.t @@ -35,6 +35,7 @@ __DATA__ location /load { content_by_lua ' package.loaded.foo = nil; + collectgarbage() local foo = require "foo"; foo.hi() '; diff --git a/t/035-gmatch.t b/t/035-gmatch.t index 5b63ae45a2..042699767a 100644 --- a/t/035-gmatch.t +++ b/t/035-gmatch.t @@ -440,6 +440,7 @@ done location /main { content_by_lua ' package.loaded.foo = nil + collectgarbage() local res = ngx.location.capture("/t") if res.status == 200 then From ed2c9f8e4d87f14de48adcfbd2c85b478363df08 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 3 Nov 2017 15:41:38 -0700 Subject: [PATCH 166/848] release 0.10.11. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 9581f7eb16..fc0940cfc4 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.10](https://github.com/openresty/lua-nginx-module/tags) released on 8 August 2017. +This document describes ngx_lua [v0.10.11](https://github.com/openresty/lua-nginx-module/tags) released on 3 November 2017. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f216c43578..d87373241d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.10] released on 8 August 2017. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.11] released on 3 November 2017. = Synopsis = From ba2b60593010ad5d552a84f3c3ad3352f232d63d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 3 Nov 2017 15:43:09 -0700 Subject: [PATCH 167/848] doc: we now work with nginx cores as new as 1.13.6. --- README.markdown | 7 ++++--- doc/HttpLuaModule.wiki | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index fc0940cfc4..c42bd3ee8d 100644 --- a/README.markdown +++ b/README.markdown @@ -249,6 +249,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.13.x (last tested: 1.13.6) * 1.11.x (last tested: 1.11.2) * 1.10.x * 1.9.x (last tested: 1.9.15) @@ -276,9 +277,9 @@ Build the source with this module: ```bash - wget 'http://nginx.org/download/nginx-1.11.2.tar.gz' - tar -xzvf nginx-1.11.2.tar.gz - cd nginx-1.11.2/ + wget 'http://nginx.org/download/nginx-1.13.6.tar.gz' + tar -xzvf nginx-1.13.6.tar.gz + cd nginx-1.13.6/ # tell nginx's build system where to find LuaJIT 2.0: export LUAJIT_LIB=/path/to/luajit/lib diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index d87373241d..2424d391c2 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -186,6 +186,7 @@ The Lua state (Lua VM instance) is shared across all the requests handled by a s The latest version of this module is compatible with the following versions of Nginx: +* 1.13.x (last tested: 1.13.6) * 1.11.x (last tested: 1.11.2) * 1.10.x * 1.9.x (last tested: 1.9.15) @@ -209,9 +210,9 @@ Alternatively, ngx_lua can be manually compiled into Nginx: Build the source with this module: - wget 'http://nginx.org/download/nginx-1.11.2.tar.gz' - tar -xzvf nginx-1.11.2.tar.gz - cd nginx-1.11.2/ + wget 'http://nginx.org/download/nginx-1.13.6.tar.gz' + tar -xzvf nginx-1.13.6.tar.gz + cd nginx-1.13.6/ # tell nginx's build system where to find LuaJIT 2.0: export LUAJIT_LIB=/path/to/luajit/lib From 8af77d3ca7c307562afdd57cb1f3e366cce434e4 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 3 Nov 2017 16:04:18 -0700 Subject: [PATCH 168/848] doc: we should also work with nginx 1.12.x. --- README.markdown | 1 + doc/HttpLuaModule.wiki | 1 + 2 files changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index c42bd3ee8d..5f6c44a721 100644 --- a/README.markdown +++ b/README.markdown @@ -250,6 +250,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: * 1.13.x (last tested: 1.13.6) +* 1.12.x * 1.11.x (last tested: 1.11.2) * 1.10.x * 1.9.x (last tested: 1.9.15) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 2424d391c2..9f07239cf5 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -187,6 +187,7 @@ The Lua state (Lua VM instance) is shared across all the requests handled by a s The latest version of this module is compatible with the following versions of Nginx: * 1.13.x (last tested: 1.13.6) +* 1.12.x * 1.11.x (last tested: 1.11.2) * 1.10.x * 1.9.x (last tested: 1.9.15) From 3eb1d2f64fc4858b7571679ebd2b21219375242f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 7 Nov 2017 11:59:48 -0800 Subject: [PATCH 169/848] tests: use longer timeout threshold. --- t/129-ssl-socket.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index e7447a7ced..1cb7de0391 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -534,7 +534,7 @@ SSL reused session content_by_lua ' local sock = ngx.socket.tcp() - sock:settimeout(2000) + sock:settimeout(4000) do local ok, err = sock:connect("openresty.org", 443) From b2aae31a85be4b51469ae8f7a128743e6e633b05 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 7 Nov 2017 15:16:44 -0800 Subject: [PATCH 170/848] tests: use larger timeout threshold values for more openresty.org ssl tests. --- t/129-ssl-socket.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 1cb7de0391..470b70e474 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -615,7 +615,7 @@ SSL reused session content_by_lua ' local sock = ngx.socket.tcp() - sock:settimeout(2000) + sock:settimeout(4000) do local ok, err = sock:connect("openresty.org", 443) @@ -1037,7 +1037,7 @@ SSL reused session content_by_lua ' local sock = ngx.socket.tcp() - sock:settimeout(2000) + sock:settimeout(4000) do local ok, err = sock:connect("openresty.org", 443) From 7dc97df41241068907c2384298b3528d297e6390 Mon Sep 17 00:00:00 2001 From: Alex Zhang Date: Wed, 15 Nov 2017 20:20:03 +0800 Subject: [PATCH 171/848] change: fixed a typo in a comment in ngx_http_lua_rewriteby.c. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_rewriteby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 077c2d3076..2fcf16f846 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -346,7 +346,7 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) if (r->header_sent) { dd("header already sent"); - /* response header was already generated in access_by_lua*, + /* response header was already generated in rewrite_by_lua*, * so it is no longer safe to proceed to later phases * which may generate responses again */ From 54135a03ef6eff618e7227dc53138f49e75a408f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 15 Nov 2017 11:23:04 -0800 Subject: [PATCH 172/848] travis: always find out the external IP address of the current travis ci node to help debugging network issues. --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index ca12a05f5f..5e2359724c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -126,6 +126,7 @@ script: - export LD_PRELOAD=$PWD/mockeagain/mockeagain.so - export LD_LIBRARY_PATH=$PWD/mockeagain:$LD_LIBRARY_PATH - export TEST_NGINX_RESOLVER=8.8.4.4 + - dig +short myip.opendns.com @resolver1.opendns.com || exit 0 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 - dig +short @$TEST_NGINX_RESOLVER agentzh.org || exit 0 - prove -Itest-nginx/lib -r t From 4f76a377ed277832068bf2da78f184c977b35ac1 Mon Sep 17 00:00:00 2001 From: spacewander Date: Tue, 14 Nov 2017 14:34:00 +0800 Subject: [PATCH 173/848] feature: logged the tcp cosocket's remote end address when connect() times out and lua_socket_log_errors is on. This feature makes debug connect timeout errors easier, since domain name may map to different ip address in different time. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 51 +++++++++++++++++++++++++++++- t/023-rewrite/tcp-socket-timeout.t | 2 +- t/023-rewrite/tcp-socket.t | 2 +- t/058-tcp-socket.t | 2 +- t/065-tcp-socket-timeout.t | 10 +++--- t/090-log-socket-errors.t | 2 +- t/147-tcp-socket-timeouts.t | 2 +- 7 files changed, 60 insertions(+), 11 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index f0988bc51a..e400f45bed 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -32,6 +32,8 @@ static int ngx_http_lua_socket_tcp_settimeouts(lua_State *L); static void ngx_http_lua_socket_tcp_handler(ngx_event_t *ev); static ngx_int_t ngx_http_lua_socket_tcp_get_peer(ngx_peer_connection_t *pc, void *data); +static void ngx_http_lua_socket_init_peer_connection_addr_text( + ngx_peer_connection_t *pc); static void ngx_http_lua_socket_read_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); static void ngx_http_lua_socket_send_handler(ngx_http_request_t *r, @@ -1002,6 +1004,50 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) } +static void +ngx_http_lua_socket_init_peer_connection_addr_text(ngx_peer_connection_t *pc) +{ + ngx_connection_t *c; + size_t addr_text_max_len; + + c = pc->connection; + + switch (pc->sockaddr->sa_family) { + +#if (NGX_HAVE_INET6) + case AF_INET6: + addr_text_max_len = NGX_INET6_ADDRSTRLEN; + break; +#endif + +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + addr_text_max_len = NGX_UNIX_ADDRSTRLEN; + break; +#endif + + case AF_INET: + addr_text_max_len = NGX_INET_ADDRSTRLEN; + break; + + default: + addr_text_max_len = NGX_SOCKADDR_STRLEN; + break; + } + + c->addr_text.data = ngx_pnalloc(c->pool, addr_text_max_len); + if (c->addr_text.data == NULL) { + ngx_log_error(NGX_LOG_ERR, pc->log, 0, + "init peer connection addr_text failed: no memory"); + return; + } + + c->addr_text.len = ngx_sock_ntop(pc->sockaddr, pc->socklen, + c->addr_text.data, + addr_text_max_len, 0); +} + + static int ngx_http_lua_socket_resolve_retval_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) @@ -3268,8 +3314,11 @@ ngx_http_lua_socket_connected_handler(ngx_http_request_t *r, llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (llcf->log_socket_errors) { + ngx_http_lua_socket_init_peer_connection_addr_text(&u->peer); ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "lua tcp socket connect timed out"); + "lua tcp socket connect timed out," + " when connecting to %V:%ud", + &c->addr_text, ngx_inet_get_port(u->peer.sockaddr)); } ngx_http_lua_socket_handle_conn_error(r, u, diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index a32ed7bed2..7ac09e243b 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -63,7 +63,7 @@ GET /t1 failed to connect: timeout --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index bff69a5876..a951014b74 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -329,7 +329,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 0743dd73d0..1c2f88967b 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -321,7 +321,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 9e5460e2cf..94ade07b0c 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -66,7 +66,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 @@ -96,7 +96,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 150 -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 @@ -125,7 +125,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 102 -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 @@ -154,7 +154,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 102 -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 @@ -707,7 +707,7 @@ GET /t 2: connected: 1 --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 diff --git a/t/090-log-socket-errors.t b/t/090-log-socket-errors.t index 8e498cdeba..8215ec386f 100644 --- a/t/090-log-socket-errors.t +++ b/t/090-log-socket-errors.t @@ -59,7 +59,7 @@ GET /t --- response_body timeout --- error_log -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 8199e846a7..1872d29fb5 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -346,7 +346,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out +lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 --- timeout: 10 From 91d2cd61a7ae1187bf2b1508f8d169585ce8acac Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 16 Nov 2017 12:05:17 -0800 Subject: [PATCH 174/848] bugfix: fixed a compilation error with nginx cores older than 1.11.2 due the lack of the ngx_inet_get_port() C API function (which was first added to nginx 1.11.2). thanks sumantdas for the report. this regression first appeared in the previous commit. --- src/ngx_http_lua_util.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 2f995e0455..69203254cc 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -435,6 +435,36 @@ ngx_http_lua_get_flush_chain(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) } +#if (nginx_version < 1011002) +static ngx_inline in_port_t +ngx_inet_get_port(struct sockaddr *sa) +{ + struct sockaddr_in *sin; +#if (NGX_HAVE_INET6) + struct sockaddr_in6 *sin6; +#endif + + switch (sa->sa_family) { + +#if (NGX_HAVE_INET6) + case AF_INET6: + sin6 = (struct sockaddr_in6 *) sa; + return ntohs(sin6->sin6_port); +#endif + +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + return 0; +#endif + + default: /* AF_INET */ + sin = (struct sockaddr_in *) sa; + return ntohs(sin->sin_port); + } +} +#endif + + extern ngx_uint_t ngx_http_lua_location_hash; extern ngx_uint_t ngx_http_lua_content_length_hash; From 7c7b2e0b1100cd96877c2d1612b572a651b91ae9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 16 Nov 2017 13:01:50 -0800 Subject: [PATCH 175/848] api: bumped version to 0.10.12. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 1e07b7185a..a725a51413 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10011 +#define ngx_http_lua_version 10012 typedef struct { From 012f3163eb67bad2f664b459285913ea5afd6449 Mon Sep 17 00:00:00 2001 From: spacewander Date: Sun, 12 Nov 2017 20:19:33 +0800 Subject: [PATCH 176/848] optimize: destroy the Lua VM and avoid running any init_worker_by_lua* code inside cache helper processes. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 6 ++ doc/HttpLuaModule.wiki | 6 ++ src/ngx_http_lua_common.h | 1 + src/ngx_http_lua_initworkerby.c | 26 ++++- src/ngx_http_lua_util.c | 6 ++ t/124-init-worker.t | 184 +++++++++++++++++++++++++++++++- t/135-worker-id.t | 62 +---------- 7 files changed, 225 insertions(+), 66 deletions(-) diff --git a/README.markdown b/README.markdown index 5f6c44a721..8b3744b7ba 100644 --- a/README.markdown +++ b/README.markdown @@ -1487,6 +1487,8 @@ This hook is often used to create per-worker reoccurring timers (via the [ngx.ti This directive was first introduced in the `v0.9.5` release. +This hook no longer runs in the cache manager and cache loader processes since the `v0.10.12` release. + [Back to TOC](#directives) init_worker_by_lua_block @@ -1514,6 +1516,8 @@ For instance, This directive was first introduced in the `v0.9.17` release. +This hook no longer runs in the cache manager and cache loader processes since the `v0.10.12` release. + [Back to TOC](#directives) init_worker_by_lua_file @@ -1529,6 +1533,8 @@ Similar to [init_worker_by_lua](#init_worker_by_lua), but accepts the file path This directive was first introduced in the `v0.9.5` release. +This hook no longer runs in the cache manager and cache loader processes since the `v0.10.12` release. + [Back to TOC](#directives) set_by_lua diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 9f07239cf5..86825ba4a1 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1196,6 +1196,8 @@ This hook is often used to create per-worker reoccurring timers (via the [[#ngx. This directive was first introduced in the v0.9.5 release. +This hook no longer runs in the cache manager and cache loader processes since the v0.10.12 release. + == init_worker_by_lua_block == '''syntax:''' ''init_worker_by_lua_block { lua-script }'' @@ -1219,6 +1221,8 @@ For instance, This directive was first introduced in the v0.9.17 release. +This hook no longer runs in the cache manager and cache loader processes since the v0.10.12 release. + == init_worker_by_lua_file == '''syntax:''' ''init_worker_by_lua_file '' @@ -1231,6 +1235,8 @@ Similar to [[#init_worker_by_lua|init_worker_by_lua]], but accepts the file path This directive was first introduced in the v0.9.5 release. +This hook no longer runs in the cache manager and cache loader processes since the v0.10.12 release. + == set_by_lua == '''syntax:''' ''set_by_lua $res [$arg1 $arg2 ...]'' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index e38978389c..1ad2f8bcbf 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -162,6 +162,7 @@ typedef struct { struct ngx_http_lua_main_conf_s { lua_State *lua; + ngx_pool_cleanup_t *vm_cleanup; ngx_str_t lua_path; ngx_str_t lua_cpath; diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 6754b4bf7b..91cfef5e47 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -40,10 +40,30 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) lmcf = ngx_http_cycle_get_module_main_conf(cycle, ngx_http_lua_module); - if (lmcf == NULL - || lmcf->init_worker_handler == NULL - || lmcf->lua == NULL) + if (lmcf == NULL || lmcf->lua == NULL) { + return NGX_OK; + } + + /* lmcf != NULL && lmcf->lua != NULL */ + + /* disable init_worker_by_lua* and destroy lua VM in cache processes */ + if (ngx_process == NGX_PROCESS_HELPER +#if defined(HAVE_PRIVILEGED_PROCESS_PATCH) && !NGX_WIN32 + && !ngx_is_privileged_agent +#endif + ) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua close the global Lua VM %p in the " + "cache helper process %P", lmcf->lua, ngx_pid); + + lmcf->vm_cleanup->handler(lmcf->vm_cleanup->data); + lmcf->vm_cleanup->handler = NULL; + + return NGX_OK; + } + + if (lmcf->init_worker_handler == NULL) { return NGX_OK; } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index df0aae8137..34c807a44e 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -3768,6 +3768,12 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, cln->data = state; + if (lmcf->vm_cleanup == NULL) { + /* this assignment will happen only once, + * and also only for the main Lua VM */ + lmcf->vm_cleanup = cln; + } + if (pcln) { *pcln = cln; } diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 22a943e39e..55d89bb6b6 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -3,13 +3,13 @@ use Test::Nginx::Socket::Lua; #worker_connections(1014); -#master_on(); +master_on(); #workers(2); #log_level('warn'); repeat_each(1); -plan tests => repeat_each() * (blocks() * 4); +plan tests => repeat_each() * (blocks() * 4 + 4); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; @@ -756,3 +756,183 @@ GET /t ok --- no_error_log [error] + + + +=== TEST 20: destory Lua VM in cache processes (without privileged agent or shdict) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + #lua_shared_dict dummy 500k; + + init_by_lua_block { + -- require "resty.core.regex" + -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + } + +--- config + location = /t { + return 200; + } +--- request + GET /t +--- grep_error_log eval: qr/lua close the global Lua VM \S+ in the cache helper process \d+|lua close the global Lua VM \S+$/ +--- grep_error_log_out eval +qr/\Alua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +lua close the global Lua VM \1 +lua close the global Lua VM \1 in the cache helper process \d+ +lua close the global Lua VM \1 +(?:lua close the global Lua VM [0-9A-F]+ +)*\z/ +--- no_error_log +[error] +start privileged agent process + + + +=== TEST 21: destory Lua VM in cache processes (without privileged agent but with shdict) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + lua_shared_dict dummy 500k; + + init_by_lua_block { + -- require "resty.core.regex" + -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + } + +--- config + location = /t { + return 200; + } +--- request + GET /t +--- grep_error_log eval: qr/lua close the global Lua VM \S+ in the cache helper process \d+|lua close the global Lua VM \S+$/ +--- grep_error_log_out eval +qr/\Alua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +lua close the global Lua VM \1 +lua close the global Lua VM \1 in the cache helper process \d+ +lua close the global Lua VM \1 +(?:lua close the global Lua VM [0-9A-F]+ +)*\z/ +--- no_error_log +[error] +start privileged agent process + + + +=== TEST 22: destory Lua VM in cache processes (with privileged agent) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + #lua_shared_dict dogs 1m; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + init_by_lua_block { + assert(require "ngx.process".enable_privileged_agent()) + -- require "resty.core.regex" + -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + } + +--- config + location = /t { + return 200; + } +--- request + GET /t +--- grep_error_log eval: qr/lua close the global Lua VM \S+ in the cache helper process \d+|lua close the global Lua VM \S+$/ +--- grep_error_log_out eval +qr/\Alua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +lua close the global Lua VM \1 +lua close the global Lua VM \1 in the cache helper process \d+ +lua close the global Lua VM \1 +(?:lua close the global Lua VM [0-9A-F]+ +)*\z/ +--- error_log eval +qr/start privileged agent process \d+/ +--- no_error_log +[error] + + + +=== TEST 23: destory Lua VM in cache processes (with init worker and privileged agent) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + #lua_shared_dict dogs 1m; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + init_by_lua_block { + assert(require "ngx.process".enable_privileged_agent()) + -- require "resty.core.regex" + -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + } + + init_worker_by_lua_block { + ngx.log(ngx.WARN, "hello from init worker by lua") + } + +--- config + location = /t { + return 200; + } +--- request + GET /t +--- grep_error_log eval: qr/hello from init worker by lua/ +--- grep_error_log_out +hello from init worker by lua +hello from init worker by lua + +--- error_log eval +[ +qr/start privileged agent process \d+$/, +qr/lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+$/, +qr/lua close the global Lua VM ([0-9A-F]+)$/, +] +--- no_error_log +[error] + + + +=== TEST 24: destory Lua VM in cache processes (with init worker but without privileged agent) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + #lua_shared_dict dogs 1m; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + init_by_lua_block { + -- require "resty.core.regex" + -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + } + + init_worker_by_lua_block { + ngx.log(ngx.WARN, "hello from init worker by lua") + } + +--- config + location = /t { + return 200; + } +--- request + GET /t + +--- grep_error_log eval: qr/hello from init worker by lua/ +--- grep_error_log_out +hello from init worker by lua + +--- error_log eval +[ +qr/lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+$/, +qr/lua close the global Lua VM ([0-9A-F]+)$/, +] +--- no_error_log +[error] +start privileged agent process diff --git a/t/135-worker-id.t b/t/135-worker-id.t index 984f44650a..8241157cbe 100644 --- a/t/135-worker-id.t +++ b/t/135-worker-id.t @@ -9,7 +9,7 @@ workers(2); #repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 1); +plan tests => repeat_each() * (blocks() * 3); #no_diff(); #no_long_string(); @@ -31,63 +31,3 @@ GET /lua --- no_error_log [error] --- skip_nginx: 3: <=1.9.0 - - - -=== TEST 2: worker id should be nil for non-worker processes ---- http_config - proxy_cache_path conf/cache levels=1:2 keys_zone=my-cache:8m max_size=10m inactive=60m; - proxy_temp_path conf/temp; - - lua_shared_dict counters 1m; - - init_by_lua_block { - ngx.shared.counters:set("c", 0) - } - - init_worker_by_lua_block { - ngx.shared.counters:incr("c", 1) - ngx.log(ngx.INFO, ngx.worker.pid(), ": worker id ", ngx.worker.id()); - } ---- config - location = /t { - content_by_lua_block { - local counters = ngx.shared.counters - local ok, c - for i = 1, 100 do - c = counters:get("c") - if c >= 4 then - ok = true - break - end - local delay = 0.001 * i - if delay > 0.1 then - delay = 0.1 - end - ngx.sleep(delay) - end - if ok then - ngx.say("ok") - else - ngx.say("not ok: c=", c) - end - } - } - location /cache { - proxy_pass http://127.0.0.1:$server_port; - proxy_cache my-cache; - } ---- request -GET /t ---- response_body -ok ---- grep_error_log eval: qr/worker id nil/ ---- grep_error_log_out -worker id nil -worker id nil ---- no_error_log -[error] ---- wait: 0.1 ---- skip_nginx: 3: <=1.9.0 ---- log_level: info ---- timeout: 6 From 9279612507be0e45565283edc9f6157420a2e492 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 23 Nov 2017 11:13:40 -0800 Subject: [PATCH 177/848] bugfix: use of lua-resty-core's ngx.re API in init_by_lua* might lead to memory issues during nginx HUP reload when no lua_shared_dict directives are used and the regex cache is enabled. --- src/ngx_http_lua_regex.c | 2 ++ t/124-init-worker.t | 25 +++++++++++++++---------- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 80519ecd15..843d1e4e7a 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -2199,6 +2199,8 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, goto error; } + pool->log = (ngx_log_t *) &ngx_cycle->new_log; + re = ngx_palloc(pool, sizeof(ngx_http_lua_regex_t)); if (re == NULL) { ngx_destroy_pool(pool); diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 55d89bb6b6..fab86a8c32 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -768,8 +768,9 @@ ok #lua_shared_dict dummy 500k; init_by_lua_block { - -- require "resty.core.regex" - -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + require "resty.core.regex" + assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + assert(ngx.re.match("hi, world", [[hi, \w+]], "ji")) } --- config @@ -801,8 +802,9 @@ start privileged agent process lua_shared_dict dummy 500k; init_by_lua_block { - -- require "resty.core.regex" - -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + require "resty.core.regex" + assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + assert(ngx.re.match("hi, world", [[hi, \w+]], "ji")) } --- config @@ -835,8 +837,9 @@ start privileged agent process init_by_lua_block { assert(require "ngx.process".enable_privileged_agent()) - -- require "resty.core.regex" - -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + require "resty.core.regex" + assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + assert(ngx.re.match("hi, world", [[hi, \w+]], "ji")) } --- config @@ -870,8 +873,9 @@ qr/start privileged agent process \d+/ init_by_lua_block { assert(require "ngx.process".enable_privileged_agent()) - -- require "resty.core.regex" - -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + require "resty.core.regex" + assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + assert(ngx.re.match("hi, world", [[hi, \w+]], "ji")) } init_worker_by_lua_block { @@ -909,8 +913,9 @@ qr/lua close the global Lua VM ([0-9A-F]+)$/, proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; init_by_lua_block { - -- require "resty.core.regex" - -- assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + require "resty.core.regex" + assert(ngx.re.match("hello, world", [[hello, \w+]], "joi")) + assert(ngx.re.match("hi, world", [[hi, \w+]], "ji")) } init_worker_by_lua_block { From 4e1a9e1d4f8ed1227f28b9dc24e291a9a149ca17 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 23 Nov 2017 11:55:31 -0800 Subject: [PATCH 178/848] tests: relaxed the checks to account for timing races in error logs from multiple cache helper processes. --- t/124-init-worker.t | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/t/124-init-worker.t b/t/124-init-worker.t index fab86a8c32..e2ebd24d03 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -781,11 +781,15 @@ ok GET /t --- grep_error_log eval: qr/lua close the global Lua VM \S+ in the cache helper process \d+|lua close the global Lua VM \S+$/ --- grep_error_log_out eval -qr/\Alua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +qr/\A(?:lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ lua close the global Lua VM \1 lua close the global Lua VM \1 in the cache helper process \d+ lua close the global Lua VM \1 -(?:lua close the global Lua VM [0-9A-F]+ +|lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +lua close the global Lua VM \2 in the cache helper process \d+ +lua close the global Lua VM \2 +lua close the global Lua VM \2 +)(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- no_error_log [error] @@ -815,11 +819,15 @@ start privileged agent process GET /t --- grep_error_log eval: qr/lua close the global Lua VM \S+ in the cache helper process \d+|lua close the global Lua VM \S+$/ --- grep_error_log_out eval -qr/\Alua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +qr/\A(?:lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ lua close the global Lua VM \1 lua close the global Lua VM \1 in the cache helper process \d+ lua close the global Lua VM \1 -(?:lua close the global Lua VM [0-9A-F]+ +|lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +lua close the global Lua VM \2 in the cache helper process \d+ +lua close the global Lua VM \2 +lua close the global Lua VM \2 +)(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- no_error_log [error] @@ -850,11 +858,15 @@ start privileged agent process GET /t --- grep_error_log eval: qr/lua close the global Lua VM \S+ in the cache helper process \d+|lua close the global Lua VM \S+$/ --- grep_error_log_out eval -qr/\Alua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +qr/\A(?:lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ lua close the global Lua VM \1 lua close the global Lua VM \1 in the cache helper process \d+ lua close the global Lua VM \1 -(?:lua close the global Lua VM [0-9A-F]+ +|lua close the global Lua VM ([0-9A-F]+) in the cache helper process \d+ +lua close the global Lua VM \2 in the cache helper process \d+ +lua close the global Lua VM \2 +lua close the global Lua VM \2 +)(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- error_log eval qr/start privileged agent process \d+/ From 4577a6fbc87f905017dd18fd9c5e7b7cf87b78ef Mon Sep 17 00:00:00 2001 From: spacewander Date: Sat, 25 Nov 2017 10:58:02 +0800 Subject: [PATCH 179/848] bugfix: freed all connections when cleaning up socket pools. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 48 +++++++++++++++++------------------ 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index e400f45bed..6869112222 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -101,8 +101,6 @@ static ngx_int_t ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, static void ngx_http_lua_socket_keepalive_dummy_handler(ngx_event_t *ev); static ngx_int_t ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev); static void ngx_http_lua_socket_keepalive_rev_handler(ngx_event_t *ev); -static void ngx_http_lua_socket_free_pool(ngx_log_t *log, - ngx_http_lua_socket_pool_t *spool); static int ngx_http_lua_socket_tcp_upstream_destroy(lua_State *L); static int ngx_http_lua_socket_downstream_destroy(lua_State *L); static ngx_int_t ngx_http_lua_socket_push_input_data(ngx_http_request_t *r, @@ -122,7 +120,11 @@ static ngx_int_t ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op); static void ngx_http_lua_tcp_resolve_cleanup(void *data); static void ngx_http_lua_coctx_cleanup(void *data); +static void ngx_http_lua_socket_free_pool(ngx_log_t *log, + ngx_http_lua_socket_pool_t *spool); static int ngx_http_lua_socket_shutdown_pool(lua_State *L); +static void ngx_http_lua_socket_shutdown_pool_helper( + ngx_http_lua_socket_pool_t *spool); static void ngx_http_lua_socket_empty_resolve_handler(ngx_resolver_ctx_t *ctx); static int ngx_http_lua_socket_prepare_error_retvals(ngx_http_request_t *r, @@ -5009,19 +5011,13 @@ ngx_http_lua_socket_free_pool(ngx_log_t *log, ngx_http_lua_socket_pool_t *spool) } -static int -ngx_http_lua_socket_shutdown_pool(lua_State *L) +static void +ngx_http_lua_socket_shutdown_pool_helper(ngx_http_lua_socket_pool_t *spool) { ngx_queue_t *q; ngx_connection_t *c; - ngx_http_lua_socket_pool_t *spool; ngx_http_lua_socket_pool_item_t *item; - spool = lua_touserdata(L, 1); - if (spool == NULL) { - return 0; - } - while (!ngx_queue_empty(&spool->cache)) { q = ngx_queue_head(&spool->cache); @@ -5035,6 +5031,19 @@ ngx_http_lua_socket_shutdown_pool(lua_State *L) } spool->active_connections = 0; +} + + +static int +ngx_http_lua_socket_shutdown_pool(lua_State *L) +{ + ngx_http_lua_socket_pool_t *spool; + + spool = lua_touserdata(L, 1); + + if (spool != NULL) { + ngx_http_lua_socket_shutdown_pool_helper(spool); + } return 0; } @@ -5451,10 +5460,7 @@ ngx_http_lua_ssl_free_session(lua_State *L) void ngx_http_lua_cleanup_conn_pools(lua_State *L) { - ngx_queue_t *q; - ngx_connection_t *c; ngx_http_lua_socket_pool_t *spool; - ngx_http_lua_socket_pool_item_t *item; lua_pushlightuserdata(L, &ngx_http_lua_socket_pool_key); lua_rawget(L, LUA_REGISTRYINDEX); /* table */ @@ -5464,18 +5470,12 @@ ngx_http_lua_cleanup_conn_pools(lua_State *L) /* tb key val */ spool = lua_touserdata(L, -1); - if (!ngx_queue_empty(&spool->cache)) { - q = ngx_queue_head(&spool->cache); - item = ngx_queue_data(q, ngx_http_lua_socket_pool_item_t, queue); - c = item->connection; - - ngx_http_lua_socket_tcp_close_connection(c); - - ngx_queue_remove(q); + if (spool != NULL) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua tcp socket keepalive: free connection pool %p " + "for \"%s\"", spool, spool->key); - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua tcp socket keepalive: free connection pool " - "for \"%s\"", spool->key); + ngx_http_lua_socket_shutdown_pool_helper(spool); } lua_pop(L, 1); From de3b7def1b3a7575cee3497855478d97c39574da Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E5=86=9B=E4=BC=9F?= Date: Wed, 23 Aug 2017 14:14:52 +0800 Subject: [PATCH 180/848] feature: added pure C API for fetching client address in the SSL handshake phase. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_ssl_certby.c | 68 +++++++++++ t/139-ssl-cert-by.t | 205 +++++++++++++++++++++++++++++++++- 2 files changed, 272 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 95be47f6b5..020b341873 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -884,6 +884,74 @@ ngx_http_lua_ffi_ssl_server_name(ngx_http_request_t *r, char **name, } +int +ngx_http_lua_ffi_ssl_raw_client_addr(ngx_http_request_t *r, char **addr, + size_t *addrlen, int *addrtype, char **err) +{ +#if (NGX_HAVE_UNIX_DOMAIN) + struct sockaddr_un *saun; +#endif + ngx_ssl_conn_t *ssl_conn; + ngx_connection_t *c; + struct sockaddr_in *sin; +#if (NGX_HAVE_INET6) + struct sockaddr_in6 *sin6; +#endif + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + c = ngx_ssl_get_connection(ssl_conn); + + switch (c->sockaddr->sa_family) { + +#if (NGX_HAVE_INET6) + case AF_INET6: + sin6 = (struct sockaddr_in6 *) c->sockaddr; + *addrlen = 16; + *addr = (char *) &sin6->sin6_addr.s6_addr; + *addrtype = NGX_HTTP_LUA_ADDR_TYPE_INET6; + + break; +#endif + +# if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + saun = (struct sockaddr_un *)c->sockaddr; + /* on Linux sockaddr might not include sun_path at all */ + if (c->socklen <= (socklen_t) offsetof(struct sockaddr_un, sun_path)) { + *addr = ""; + *addrlen = 0; + + } else { + *addr = saun->sun_path; + *addrlen = ngx_strlen(saun->sun_path); + } + + *addrtype = NGX_HTTP_LUA_ADDR_TYPE_UNIX; + break; +#endif + + default: /* AF_INET */ + sin = (struct sockaddr_in *) c->sockaddr; + *addr = (char *) &sin->sin_addr.s_addr; + *addrlen = 4; + *addrtype = NGX_HTTP_LUA_ADDR_TYPE_INET; + break; + } + + return NGX_OK; +} + + int ngx_http_lua_ffi_cert_pem_to_der(const u_char *pem, size_t pem_len, u_char *der, char **err) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index b521e9e88b..a65d7038e3 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -11,7 +11,7 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); } else { - plan tests => repeat_each() * (blocks() * 6 + 6); + plan tests => repeat_each() * (blocks() * 6 + 4); } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -1872,3 +1872,206 @@ failed to do SSL handshake: handshake failed qr/\[alert\] .*? no ssl_certificate_by_lua\* defined in server ~test2\\\.com\b/, qr/\[crit\] .*? SSL_do_handshake\(\) failed\b/, ] + + + +=== TEST 22: get raw_client_addr - IPv4 +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + server { + listen 127.0.0.1:12345 ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + local ssl = require "ngx.ssl" + local byte = string.byte + local addr, addrtype, err = ssl.raw_client_addr() + local ip = string.format("%d.%d.%d.%d", byte(addr, 1), byte(addr, 2), + byte(addr, 3), byte(addr, 4)) + print("client ip: ", ip) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", 12345) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +client ip: 127.0.0.1 + +--- no_error_log +[error] +[alert] + + + +=== TEST 23: get raw_client_addr - unix domain socket +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + local ssl = require "ngx.ssl" + local addr, addrtyp, err = ssl.raw_client_addr() + print("client socket file: ", addr) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +client socket file: + +--- no_error_log +[error] +[alert] From 9de3dc7ef21d646638164769972eca2fb01cb066 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 28 Nov 2017 16:23:34 -0800 Subject: [PATCH 181/848] tests: updated the expected debugging log message for freeing cosocket connection pools. --- t/025-codecache.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/025-codecache.t b/t/025-codecache.t index 0b02a2722f..20791d7c70 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1111,7 +1111,7 @@ lua tcp socket keepalive max idle timeout [ qq{lua tcp socket keepalive create connection pool for key "127.0.0.1:$ENV{TEST_NGINX_MEMCACHED_PORT}"}, qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, -"lua tcp socket keepalive: free connection pool for ", +qr/\blua tcp socket keepalive: free connection pool [0-9A-F]+ for "127.0.0.1:/, ] From 18b5de576c13ed19f88ad3844c36776f23fbb61f Mon Sep 17 00:00:00 2001 From: spacewander Date: Sat, 2 Dec 2017 00:05:12 +0800 Subject: [PATCH 182/848] bugfix: the upper bound of port ranges should be 65535 instead of 65536. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 4 ++-- src/ngx_http_lua_socket_udp.c | 4 ++-- t/058-tcp-socket.t | 21 ++++++++++++++++++++- t/087-udp-socket.t | 20 ++++++++++++++++++++ 4 files changed, 44 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 6869112222..ea158d19be 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -513,7 +513,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) if (n == 3) { port = luaL_checkinteger(L, 3); - if (port < 0 || port > 65536) { + if (port < 0 || port > 65535) { lua_pushnil(L); lua_pushfstring(L, "bad port number: %d", port); return 2; @@ -942,7 +942,7 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) #else /* for nginx older than 1.5.8 */ - len = NGX_INET_ADDRSTRLEN + sizeof(":65536") - 1; + len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1; p = ngx_pnalloc(r->pool, len + sizeof(struct sockaddr_in)); if (p == NULL) { diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 08ba1cc4d7..15cadf3100 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -221,7 +221,7 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) if (n == 3) { port = luaL_checkinteger(L, 3); - if (port < 0 || port > 65536) { + if (port < 0 || port > 65535) { lua_pushnil(L); lua_pushfstring(L, "bad port number: %d", port); return 2; @@ -574,7 +574,7 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) #else /* for nginx older than 1.5.8 */ - len = NGX_INET_ADDRSTRLEN + sizeof(":65536") - 1; + len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1; p = ngx_pnalloc(r->pool, len + sizeof(struct sockaddr_in)); if (p == NULL) { diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 1c2f88967b..f0c2b481cd 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 193; +plan tests => repeat_each() * 196; our $HtmlDir = html_dir; @@ -3727,3 +3727,22 @@ failed to connect: www.google.com could not be resolved hello! --- error_log eval qr{\[alert\] .*? send\(\) failed \(\d+: Operation not permitted\) while resolving} + + + +=== TEST 62: the upper bound of port range should be 2^16 - 1 +--- config + location /t { + content_by_lua_block { + local sock, err = ngx.socket.connect("127.0.0.1", 65536) + if not sock then + ngx.say("failed to connect: ", err) + end + } + } +--- request +GET /t +--- response_body +failed to connect: bad port number: 65536 +--- no_error_log +[error] diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index a847031aec..ab41622eee 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -1104,3 +1104,23 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ --- no_error_log [alert] + + + +=== TEST 20: the upper bound of port range should be 2^16 - 1 +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.udp() + local ok, err = sock:setpeername("127.0.0.1", 65536) + if not ok then + ngx.say("failed to connect: ", err) + end + } + } +--- request +GET /t +--- response_body +failed to connect: bad port number: 65536 +--- no_error_log +[error] From 81e140c59b627b1ce8b0741e4372eacf13338d84 Mon Sep 17 00:00:00 2001 From: Ka-Hing Cheung Date: Tue, 26 Sep 2017 15:54:12 -0700 Subject: [PATCH 183/848] bugfix: pure C API for ngx.var assignment: we failed to output the error message length. this might lead to error buffer overreads. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_variable.c | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/src/ngx_http_lua_variable.c b/src/ngx_http_lua_variable.c index b26c8621e9..e489d9e57e 100644 --- a/src/ngx_http_lua_variable.c +++ b/src/ngx_http_lua_variable.c @@ -364,7 +364,7 @@ ngx_http_lua_ffi_var_get(ngx_http_request_t *r, u_char *name_data, int ngx_http_lua_ffi_var_set(ngx_http_request_t *r, u_char *name_data, size_t name_len, u_char *lowcase_buf, u_char *value, size_t value_len, - u_char *errbuf, size_t errlen) + u_char *errbuf, size_t *errlen) { u_char *p; ngx_uint_t hash; @@ -373,12 +373,15 @@ ngx_http_lua_ffi_var_set(ngx_http_request_t *r, u_char *name_data, ngx_http_core_main_conf_t *cmcf; if (r == NULL) { - ngx_snprintf(errbuf, errlen, "no request object found"); + *errlen = ngx_snprintf(errbuf, *errlen, "no request object found") + - errbuf; return NGX_ERROR; } if ((r)->connection->fd == (ngx_socket_t) -1) { - ngx_snprintf(errbuf, errlen, "API disabled in the current context"); + *errlen = ngx_snprintf(errbuf, *errlen, + "API disabled in the current context") + - errbuf; return NGX_ERROR; } @@ -395,8 +398,10 @@ ngx_http_lua_ffi_var_set(ngx_http_request_t *r, u_char *name_data, if (v) { if (!(v->flags & NGX_HTTP_VAR_CHANGEABLE)) { dd("variable not changeable"); - ngx_snprintf(errbuf, errlen, "variable \"%*s\" not changeable", - name_len, lowcase_buf); + *errlen = ngx_snprintf(errbuf, *errlen, + "variable \"%*s\" not changeable", + name_len, lowcase_buf) + - errbuf; return NGX_ERROR; } @@ -475,23 +480,27 @@ ngx_http_lua_ffi_var_set(ngx_http_request_t *r, u_char *name_data, return NGX_OK; } - ngx_snprintf(errbuf, errlen, "variable \"%*s\" cannot be assigned " - "a value", name_len, lowcase_buf); + *errlen = ngx_snprintf(errbuf, *errlen, + "variable \"%*s\" cannot be assigned " + "a value", name_len, lowcase_buf) + - errbuf; return NGX_ERROR; } /* variable not found */ - ngx_snprintf(errbuf, errlen, "variable \"%*s\" not found for writing; " - "maybe it is a built-in variable that is not changeable " - "or you forgot to use \"set $%*s '';\" " - "in the config file to define it first", - name_len, lowcase_buf, name_len, lowcase_buf); + *errlen = ngx_snprintf(errbuf, *errlen, + "variable \"%*s\" not found for writing; " + "maybe it is a built-in variable that is not " + "changeable or you forgot to use \"set $%*s '';\" " + "in the config file to define it first", + name_len, lowcase_buf, name_len, lowcase_buf) + - errbuf; return NGX_ERROR; nomem: - ngx_snprintf(errbuf, errlen, "no memory"); + *errlen = ngx_snprintf(errbuf, *errlen, "no memory") - errbuf; return NGX_ERROR; } #endif /* NGX_LUA_NO_FFI_API */ From d6aa6ea1c776108d1d1af9eb2c9a1871c5994288 Mon Sep 17 00:00:00 2001 From: spacewander Date: Sat, 16 Dec 2017 13:40:31 +0800 Subject: [PATCH 184/848] tests: fixed typos in test tiles in t/124-init-worker.t. Signed-off-by: Yichun Zhang (agentzh) --- t/124-init-worker.t | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/t/124-init-worker.t b/t/124-init-worker.t index e2ebd24d03..9b2a91fcfa 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -759,7 +759,7 @@ ok -=== TEST 20: destory Lua VM in cache processes (without privileged agent or shdict) +=== TEST 20: destroy Lua VM in cache processes (without privileged agent or shdict) --- http_config lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; @@ -797,7 +797,7 @@ start privileged agent process -=== TEST 21: destory Lua VM in cache processes (without privileged agent but with shdict) +=== TEST 21: destroy Lua VM in cache processes (without privileged agent but with shdict) --- http_config lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; @@ -835,7 +835,7 @@ start privileged agent process -=== TEST 22: destory Lua VM in cache processes (with privileged agent) +=== TEST 22: destroy Lua VM in cache processes (with privileged agent) --- http_config lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; @@ -875,7 +875,7 @@ qr/start privileged agent process \d+/ -=== TEST 23: destory Lua VM in cache processes (with init worker and privileged agent) +=== TEST 23: destroy Lua VM in cache processes (with init worker and privileged agent) --- http_config lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; @@ -916,7 +916,7 @@ qr/lua close the global Lua VM ([0-9A-F]+)$/, -=== TEST 24: destory Lua VM in cache processes (with init worker but without privileged agent) +=== TEST 24: destroy Lua VM in cache processes (with init worker but without privileged agent) --- http_config lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; From b1b1504ac449255386c853a9ba5679f807a3fa2c Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Fri, 15 Dec 2017 14:49:55 -0800 Subject: [PATCH 185/848] tests: fixed broken SSL tests due to switch in CA for openresty.org. Signed-off-by: Yichun Zhang (agentzh) --- t/129-ssl-socket.t | 22 ++++++++-------- t/cert/comodo-ca.crt | 29 -------------------- t/cert/dst-ca.crt | 63 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+), 40 deletions(-) delete mode 100644 t/cert/comodo-ca.crt create mode 100644 t/cert/dst-ca.crt diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 470b70e474..dd468bd048 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -26,7 +26,7 @@ sub read_file { $cert; } -our $ComodoRootCertificate = read_file("t/cert/comodo-ca.crt"); +our $DSTRootCertificate = read_file("t/cert/dst-ca.crt"); our $EquifaxRootCertificate = read_file("t/cert/equifax.crt"); our $TestCertificate = read_file("t/cert/test.crt"); our $TestCertificateKey = read_file("t/cert/test.key"); @@ -421,7 +421,7 @@ The certificate of "openresty.org" does not contain the name "blah.openresty.org --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -501,7 +501,7 @@ The certificate for "openresty.org" does not contain the name "blah.openresty.or --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -660,7 +660,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -744,7 +744,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -823,7 +823,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -1005,7 +1005,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -1083,7 +1083,7 @@ SSL reused session --- user_files eval ">>> trusted.crt $::EquifaxRootCertificate -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -1472,7 +1472,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -1548,7 +1548,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request GET /t @@ -1607,7 +1607,7 @@ SSL reused session --- user_files eval ">>> trusted.crt -$::ComodoRootCertificate" +$::DSTRootCertificate" --- request POST /t diff --git a/t/cert/comodo-ca.crt b/t/cert/comodo-ca.crt deleted file mode 100644 index 444461b95a..0000000000 --- a/t/cert/comodo-ca.crt +++ /dev/null @@ -1,29 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIF2DCCA8CgAwIBAgIQTKr5yttjb+Af907YWwOGnTANBgkqhkiG9w0BAQwFADCBhTELMAkGA1UE -BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgG -A1UEChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNVBAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNhdGlv -biBBdXRob3JpdHkwHhcNMTAwMTE5MDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBhTELMAkGA1UEBhMC -R0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UE -ChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNVBAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNhdGlvbiBB -dXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCR6FSS0gpWsawNJN3Fz0Rn -dJkrN6N9I3AAcbxT38T6KhKPS38QVr2fcHK3YX/JSw8Xpz3jsARh7v8Rl8f0hj4K+j5c+ZPmNHrZ -FGvnnLOFoIJ6dq9xkNfs/Q36nGz637CC9BR++b7Epi9Pf5l/tfxnQ3K9DADWietrLNPtj5gcFKt+ -5eNu/Nio5JIk2kNrYrhV/erBvGy2i/MOjZrkm2xpmfh4SDBF1a3hDTxFYPwyllEnvGfDyi62a+pG -x8cgoLEfZd5ICLqkTqnyg0Y3hOvozIFIQ2dOciqbXL1MGyiKXCJ7tKuY2e7gUYPDCUZObT6Z+pUX -2nwzV0E8jVHtC7ZcryxjGt9XyD+86V3Em69FmeKjWiS0uqlWPc9vqv9JWL7wqP/0uK3pN/u6uPQL -OvnoQ0IeidiEyxPx2bvhiWC4jChWrBQdnArncevPDt09qZahSL0896+1DSJMwBGB7FY79tOi4lu3 -sgQiUpWAk2nojkxl8ZEDLXB0AuqLZxUpaVICu9ffUGpVRr+goyhhf3DQw6KqLCGqR84onAZFdr+C -GCe01a60y1Dma/RMhnEw6abfFobg2P9A3fvQQoh/ozM6LlweQRGBY84YcWsr7KaKtzFcOmpH4MN5 -WdYgGq/yapiqcrxXStJLnbsQ/LBMQeXtHT1eKJ2czL+zUdqnR+WEUwIDAQABo0IwQDAdBgNVHQ4E -FgQUu69+Aj36pvE8hI6t7jiY7NkyMtQwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8w -DQYJKoZIhvcNAQEMBQADggIBAArx1UaEt65Ru2yyTUEUAJNMnMvlwFTPoCWOAvn9sKIN9SCYPBMt -rFaisNZ+EZLpLrqeLppysb0ZRGxhNaKatBYSaVqM4dc+pBroLwP0rmEdEBsqpIt6xf4FpuHA1sj+ -nq6PK7o9mfjYcwlYRm6mnPTXJ9OV2jeDchzTc+CiR5kDOF3VSXkAKRzH7JsgHAckaVd4sjn8OoSg -tZx8jb8uk2IntznaFxiuvTwJaP+EmzzV1gsD41eeFPfR60/IvYcjt7ZJQ3mFXLrrkguhxuhoqEwW -sRqZCuhTLJK7oQkYdQxlqHvLI7cawiiFwxv/0Cti76R7CZGYZ4wUAc1oBmpjIXUDgIiKboHGhfKp -pC3n9KUkEEeDys30jXlYsQab5xoq2Z0B15R97QNKyvDb6KkBPvVWmckejkk9u+UJueBPSZI9FoJA -zMxZxuY67RIuaTxslbH9qh17f4a+Hg4yRvv7E491f0yLS0Zj/gA0QHDBw7mh3aZw4gSzQbzpgJHq -ZJx64SIDqZxubw5lT2yHh17zbqD5daWbQOhTsiedSrnAdyGN/4fy3ryM7xfft0kL0fJuMAsaDk52 -7RH89elWsn2/x20Kk4yl0MC2Hb46TpSi125sC8KKfPog88Tk5c0NqMuRkrF8hey1FGlmDoLnzc7I -LaZRfyHBNVOFBkpdn627G190 ------END CERTIFICATE----- diff --git a/t/cert/dst-ca.crt b/t/cert/dst-ca.crt new file mode 100644 index 0000000000..738121f63a --- /dev/null +++ b/t/cert/dst-ca.crt @@ -0,0 +1,63 @@ +subject=/C=US/O=Digital Signature Trust Co./OU=TrustID Server/CN=TrustID Server CA A5 +issuer=/O=Digital Signature Trust Co./CN=DST Root CA X3 +-----BEGIN CERTIFICATE----- +MIIGiTCCBXGgAwIBAgIRALCqqJD4YLUEJLy76IZcvf8wDQYJKoZIhvcNAQEFBQAw +PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD +Ew5EU1QgUm9vdCBDQSBYMzAeFw0wNzA4MjQxNzU0MTlaFw0xNDA4MjQxNDEzMzVa +MGsxCzAJBgNVBAYTAlVTMSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVz +dCBDby4xFzAVBgNVBAsTDlRydXN0SUQgU2VydmVyMR0wGwYDVQQDExRUcnVzdElE +IFNlcnZlciBDQSBBNTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOae +jyufM+mrKgTCpPm9ha5UBAORNLhAv3PFlmzA8n6lnEvBTJfeG5vUZYQrlZMMKmZl +F1D3mYRFAhT/qFbBsTMlF9XsDNF40NB21ABTquuEB3Ji8mmC1Z2cZjwa4VeKxGIq +kTdBypQgWekopNVvoOGLPbRXgYKW4UCc6pt7Q+We+eUuonp2XIRIOvmJQtXb/rNz +4B1ahu3QB4HnystmQ9dW8BySSGDhZ/kzIcSU8GuHcL58SHsei07gjFzXRhMkkbJH +MT8zS/hrjNsB8Z5kamI4QL7nLVNuCF1Sg9zgB1POdMPLEnm4uDx+BRLwVMGZMwza +MNfEATnWTvLazzIDarMCAwEAAaOCA1IwggNOMA8GA1UdEwEB/wQFMAMBAf8wDgYD +VR0PAQH/BAQDAgHGMEcGA1UdJQRAMD4GCCsGAQUFBwMBBggrBgEFBQcDAwYIKwYB +BQUHAwUGCCsGAQUFBwMGBggrBgEFBQcDBwYKKwYBBAGCNwoDATCCATIGA1UdIASC +ASkwggElMIIBIQYKYIZIAYb5LwAGAzCCAREwSgYIKwYBBQUHAgEWPmh0dHBzOi8v +c2VjdXJlLmlkZW50cnVzdC5jb20vY2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRl +eC5odG1sMIHCBggrBgEFBQcCAjCBtRqBslRoaXMgVHJ1c3RJRCBTZXJ2ZXIgQ2Vy +dGlmaWNhdGUgaGFzIGJlZW4gaXNzdWVkIGluIGFjY29yZGFuY2Ugd2l0aCBJZGVu +VHJ1c3QncyBUcnVzdElEIENlcnRpZmljYXRlIFBvbGljeSBmb3VuZCBhdCBodHRw +czovL3NlY3VyZS5pZGVudHJ1c3QuY29tLyBjZXJ0aWZpY2F0ZXMvcG9saWN5L3Rz +L2luZGV4Lmh0bWwwgbkGA1UdHwSBsTCBrjAuoCygKoYoaHR0cDovL2NybC5pZGVu +dHJ1c3QuY29tL0RTVFJPT1RDQVgzLmNybDB8oHqgeIZ2bGRhcDovL2xkYXAuaWRl +bnRydXN0LmNvbS9jbj1EU1QlMjBSb290JTIwQ0ElMjBYMyxvPURpZ2l0YWwlMjBT +aWduYXR1cmUlMjBUcnVzdCUyMENvLj9jZXJ0aWZpY2F0ZVJldm9jYXRpb25MaXN0 +O2JpbmFyeTCBrwYIKwYBBQUHAQEEgaIwgZ8wJQYIKwYBBQUHMAGGGWh0dHA6Ly9v +Y3NwLmlkZW50cnVzdC5jb20wdgYIKwYBBQUHMAKGamxkYXA6Ly9sZGFwLmlkZW50 +cnVzdC5jb20vY249RFNUJTIwUm9vdCUyMENBJTIwWDMsbz1EaWdpdGFsJTIwU2ln +bmF0dXJlJTIwVHJ1c3QlMjBDby4/Y0FDZXJ0aWZpY2F0ZTtiaW5hcnkwHwYDVR0j +BBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAwHQYDVR0OBBYEFLeoy2iu3EVD0tyJ +EamBZhn5+3XpMA0GCSqGSIb3DQEBBQUAA4IBAQAtreoVqLQx+G35L8byM/IvGqNa +MzbWYzGIidH3+hT8B2e0duWfZCDazgu272ErWL/T4HL5psUEOIHfzNxEWHA24XhY +zBFsYR64vaTRV7T+iYKv5Nxptidk26zOZ2/F+TKZvOfhPg5IqnH3FyXHEcRDWm1/ +QJsZ4mqgUFx38pc50i1ed6XUdvBObMnGJc1fGdfVqcNqbvHACNrw+WJE9PIeMKJQ +HC/NkE87xFP/Q1OW7hPmqQvmIIbm6BLZ7U0kxqNL3HmegxkJ2yJQntsX+5BooJqj +/uVpwzAEei7IjKsdtomxkHCDPFGEJYBldDLBH0ScSlngQZ6V5HNM473l8bWs +-----END CERTIFICATE----- + +subject=/O=Digital Signature Trust Co./CN=DST Root CA X3 +issuer=/O=Digital Signature Trust Co./CN=DST Root CA X3 +-----BEGIN CERTIFICATE----- +MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/ +MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT +DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow +PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD +Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB +AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O +rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq +OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b +xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw +7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD +aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV +HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG +SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69 +ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr +AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz +R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5 +JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo +Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ +-----END CERTIFICATE----- + From 20ffd853ebd008b7d0396f3474c1856692f8beda Mon Sep 17 00:00:00 2001 From: Robert Paprocki Date: Wed, 4 Jan 2017 19:16:04 -0800 Subject: [PATCH 186/848] feature: implemented pure C function for the pure FFI version of the ngx.get_phase() API function in lua-resty-core. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_phase.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/ngx_http_lua_phase.c b/src/ngx_http_lua_phase.c index 50c53110b8..304c88ae13 100644 --- a/src/ngx_http_lua_phase.c +++ b/src/ngx_http_lua_phase.c @@ -107,4 +107,22 @@ ngx_http_lua_inject_phase_api(lua_State *L) lua_setfield(L, -2, "get_phase"); } + +#ifndef NGX_LUA_NO_FFI_API +int +ngx_http_lua_ffi_get_phase(ngx_http_request_t *r, char **err) +{ + ngx_http_lua_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + *err = "no request context"; + return NGX_ERROR; + } + + return ctx->context; +} +#endif + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From 51b7d649aaad3a8d2f08f2818f2679e47e3c87f9 Mon Sep 17 00:00:00 2001 From: chronolaw Date: Wed, 27 Dec 2017 16:04:21 +0800 Subject: [PATCH 187/848] feature: added pure C API function ngx_http_lua_ffi_master_pid() for the get_master_pid() Lua FFI API in ngx.process of lua-resty-core. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_worker.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index e1cfec4102..461509b436 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -135,6 +135,21 @@ ngx_http_lua_ffi_worker_count(void) } +int +ngx_http_lua_ffi_master_pid(void) +{ +#if (nginx_version >= 1013008) + if (ngx_process == NGX_PROCESS_SINGLE) { + return (int) ngx_pid; + } + + return (int) ngx_parent; +#else + return NGX_ERROR; +#endif +} + + int ngx_http_lua_ffi_get_process_type(void) { From 99a5ba2126879d2b7f98029f220212bbc8319e8e Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sat, 8 Jul 2017 17:24:18 -0700 Subject: [PATCH 188/848] feature: added a pure C API function ngx_http_lua_ffi_raw_log for ngx.errlog module's rawlog() function in the lua-resty-core library. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_log.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index c18e6b03ee..0042a5e2b4 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -428,6 +428,29 @@ ngx_http_lua_ffi_errlog_get_sys_filter_level(ngx_http_request_t *r) return log_level; } + +int +ngx_http_lua_ffi_raw_log(ngx_http_request_t *r, int level, u_char *s, + size_t s_len) +{ + ngx_log_t *log; + + if (level > NGX_LOG_DEBUG || level < NGX_LOG_STDERR) { + return NGX_ERROR; + } + + if (r && r->connection && r->connection->log) { + log = r->connection->log; + + } else { + log = ngx_cycle->log; + } + + ngx_log_error((unsigned) level, log, 0, "%*s", s_len, s); + + return NGX_OK; +} + #endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From c2f68a48ee0f07bbb1e7589aeba592d84a23916f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 2 Jan 2018 15:11:34 -0800 Subject: [PATCH 189/848] style: minor coding style fixes suggested by our ngx-releng tool. --- src/ngx_http_lua_util.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 34c807a44e..c2d5c41147 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1055,7 +1055,8 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, * check if coroutine.resume or coroutine.yield called * lua_yield() */ - switch(ctx->co_op) { + switch (ctx->co_op) { + case NGX_HTTP_LUA_USER_CORO_NOP: dd("hit! it is the API yield"); @@ -2169,8 +2170,7 @@ ngx_http_lua_handle_exec(lua_State *L, ngx_http_request_t *r, #endif rc = ngx_http_named_location(r, &ctx->exec_uri); - if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) - { + if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) { return rc; } From 34d9298577d706543560f432bb203e482f7f176c Mon Sep 17 00:00:00 2001 From: tokers Date: Mon, 4 Dec 2017 11:52:59 +0800 Subject: [PATCH 190/848] bugfix: ngx.req.raw_header(): the first part of the header would be discarded when using single LF as delimiter and the number of headers is large enough. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_headers.c | 7 +-- t/104-req-raw-header.t | 112 +++++++++++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 6700ce80db..2456fd15b5 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -245,7 +245,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) if (b != mr->header_in) { /* skip truncated header entries (if any) */ - while (last > data && last[-1] != LF) { + while (last > data && last[-1] != LF && last[-1] != '\0') { last--; } } @@ -315,7 +315,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) #if 1 /* skip truncated header entries (if any) */ - while (last > p && last[-1] != LF) { + while (last > p && last[-1] != LF && last[-1] != '\0') { last--; } #endif @@ -325,8 +325,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) if (*p == '\0') { j++; if (p + 1 == last) { - /* XXX this should not happen */ - dd("found string end!!"); + *p = LF; } else if (*(p + 1) == LF) { *p = CR; diff --git a/t/104-req-raw-header.t b/t/104-req-raw-header.t index efea03cb27..ac1f89e1b4 100644 --- a/t/104-req-raw-header.t +++ b/t/104-req-raw-header.t @@ -876,3 +876,115 @@ Foo: bar\r --- no_error_log [error] --- timeout: 5 + + + +=== TEST 30: large headers (using single LF as line break) +--- config + location /t { + content_by_lua_block { + ngx.print(ngx.req.raw_header()) + } + } + +--- raw_request eval +"GET /t HTTP/1.1 +Host: localhost +Connection: close +". +(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- response_body eval +qq{GET /t HTTP/1.1 +Host: localhost +Connection: close +} +.(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- no_error_log +[error] +--- timeout: 5 + + + +=== TEST 31: large headers without request line (using single LF as line break) +--- config + location /t { + content_by_lua_block { + ngx.print(ngx.req.raw_header(true)) + } + } + +--- raw_request eval +"GET /t HTTP/1.1 +Host: localhost +Connection: close +". +(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- response_body eval +qq{Host: localhost +Connection: close +} +.(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- no_error_log +[error] +--- timeout: 5 + + + +=== TEST 32: large headers with leading CRLF (using single LF as line break) +--- config + location /t { + content_by_lua_block { + ngx.print(ngx.req.raw_header()) + } + } + +--- raw_request eval +"\r +GET /t HTTP/1.1 +Host: localhost +Connection: close +". +(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- response_body eval +qq{GET /t HTTP/1.1 +Host: localhost +Connection: close +} +.(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- no_error_log +[error] +--- timeout: 5 + + + +=== TEST 33: large headers without request line but contains leading CRLF (using single LF as line break) +--- config + location /t { + content_by_lua_block { + ngx.print(ngx.req.raw_header(true)) + } + } + +--- raw_request eval +"\r +GET /t HTTP/1.1 +Host: localhost +Connection: close +". +(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- response_body eval +qq{Host: localhost +Connection: close +} +.(CORE::join "\n", map { "Header$_: value-$_" } 1..512) . "\n\n" + +--- no_error_log +[error] +--- timeout: 5 From e51a4739a62efa35c7e136f83fbc426a0f0d37cf Mon Sep 17 00:00:00 2001 From: tan jinhua <312841925@qq.com> Date: Thu, 4 Jan 2018 16:33:06 +0800 Subject: [PATCH 191/848] doc: typo fix: fix constrolled to controlled. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 8b3744b7ba..4a07ba01a4 100644 --- a/README.markdown +++ b/README.markdown @@ -7839,7 +7839,7 @@ timers" here mean timers that have not yet been expired and "running timers" are those whose user callbacks are currently running. The maximal number of pending timers allowed in an Nginx -worker is constrolled by the [lua_max_pending_timers](#lua_max_pending_timers) +worker is controlled by the [lua_max_pending_timers](#lua_max_pending_timers) directive. The maximal number of running timers is controlled by the [lua_max_running_timers](#lua_max_running_timers) directive. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 86825ba4a1..f2b477752b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -6660,7 +6660,7 @@ timers" here mean timers that have not yet been expired and "running timers" are those whose user callbacks are currently running. The maximal number of pending timers allowed in an Nginx -worker is constrolled by the [[#lua_max_pending_timers|lua_max_pending_timers]] +worker is controlled by the [[#lua_max_pending_timers|lua_max_pending_timers]] directive. The maximal number of running timers is controlled by the [[#lua_max_running_timers|lua_max_running_timers]] directive. From 7f108923b01e32718b98f37ef824f1383de5c49c Mon Sep 17 00:00:00 2001 From: Alessandro Ghedini Date: Tue, 13 Sep 2016 22:31:32 +0100 Subject: [PATCH 192/848] feature: added support for OpenSSL 1.1.0. thanks polishment work from Dejiang Zhu and Zexuan Luo. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 9 +- src/ngx_http_lua_socket_tcp.c | 15 +- src/ngx_http_lua_ssl_ocsp.c | 4 + src/ngx_http_lua_ssl_session_fetchby.c | 9 +- src/ngx_http_lua_ssl_session_fetchby.h | 6 +- src/ngx_http_lua_ssl_session_storeby.c | 8 +- t/129-ssl-socket.t | 188 +++++++++++++------------ t/142-ssl-session-store.t | 24 ++-- t/143-ssl-session-fetch.t | 26 ++-- 9 files changed, 150 insertions(+), 139 deletions(-) diff --git a/.travis.yml b/.travis.yml index 5e2359724c..36b2e84649 100644 --- a/.travis.yml +++ b/.travis.yml @@ -41,7 +41,6 @@ env: - OPENSSL_PREFIX=/opt/ssl - OPENSSL_LIB=$OPENSSL_PREFIX/lib - OPENSSL_INC=$OPENSSL_PREFIX/include - - OPENSSL_VER=1.0.2k - LIBDRIZZLE_PREFIX=/opt/drizzle - LIBDRIZZLE_INC=$LIBDRIZZLE_PREFIX/include/libdrizzle-1.0 - LIBDRIZZLE_LIB=$LIBDRIZZLE_PREFIX/lib @@ -49,7 +48,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.13.6 + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2k OPENSSL_PATCH_VER=1.0.2h + #- NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0c OPENSSL_PATCH_VER=1.1.0c services: - memcache @@ -112,9 +112,8 @@ script: - cd .. - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - - if [ ! -f ../download-cache/openssl-1.0.2h-sess_set_get_cb_yield.patch ]; then wget -P ../download-cache https://raw.githubusercontent.com/openresty/openresty/master/patches/openssl-1.0.2h-sess_set_get_cb_yield.patch; fi - - patch -p1 < ../download-cache/openssl-1.0.2h-sess_set_get_cb_yield.patch - - ./config shared --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) + - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch + - ./config no-threads shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index ea158d19be..c34a24e5c3 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1368,9 +1368,8 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) return 2; } - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua ssl set session: %p:%d", - *psession, (*psession)->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua ssl set session: %p", *psession); } } @@ -1635,9 +1634,8 @@ ngx_http_lua_ssl_handshake_retval_handler(ngx_http_request_t *r, } else { *ud = ssl_session; - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua ssl save session: %p:%d", ssl_session, - ssl_session->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua ssl save session: %p", ssl_session); /* set up the __gc metamethod */ lua_pushlightuserdata(L, &ngx_http_lua_ssl_session_metatable_key); @@ -5444,9 +5442,8 @@ ngx_http_lua_ssl_free_session(lua_State *L) psession = lua_touserdata(L, 1); if (psession && *psession != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua ssl free session: %p:%d", *psession, - (*psession)->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua ssl free session: %p", *psession); ngx_ssl_free_session(*psession); } diff --git a/src/ngx_http_lua_ssl_ocsp.c b/src/ngx_http_lua_ssl_ocsp.c index 31b4f24382..9ec8b5098c 100644 --- a/src/ngx_http_lua_ssl_ocsp.c +++ b/src/ngx_http_lua_ssl_ocsp.c @@ -468,7 +468,11 @@ ngx_http_lua_ffi_ssl_set_ocsp_status_resp(ngx_http_request_t *r, return NGX_ERROR; } +#ifdef SSL_CTRL_GET_TLSEXT_STATUS_REQ_TYPE + if (SSL_get_tlsext_status_type(ssl_conn) == -1) { +#else if (ssl_conn->tlsext_status_type == -1) { +#endif dd("no ocsp status req from client"); return NGX_DECLINED; } diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 3a3c1f54d8..2c75f6a912 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -171,8 +171,11 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, /* cached session fetching callback to be set with SSL_CTX_sess_set_get_cb */ ngx_ssl_session_t * -ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, u_char *id, - int len, int *copy) +ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, +#if OPENSSL_VERSION_NUMBER >= 0x10100003L + const +#endif + u_char *id, int len, int *copy) { lua_State *L; ngx_int_t rc; @@ -287,7 +290,7 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, u_char *id, cctx->exit_code = 1; /* successful by default */ cctx->connection = c; cctx->request = r; - cctx->session_id.data = id; + cctx->session_id.data = (u_char *) id; cctx->session_id.len = len; cctx->entered_sess_fetch_handler = 1; cctx->done = 0; diff --git a/src/ngx_http_lua_ssl_session_fetchby.h b/src/ngx_http_lua_ssl_session_fetchby.h index 5a6f96f5bf..50c6616dd2 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.h +++ b/src/ngx_http_lua_ssl_session_fetchby.h @@ -25,7 +25,11 @@ char *ngx_http_lua_ssl_sess_fetch_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_ssl_session_t *ngx_http_lua_ssl_sess_fetch_handler( - ngx_ssl_conn_t *ssl_conn, u_char *id, int len, int *copy); + ngx_ssl_conn_t *ssl_conn, +#if OPENSSL_VERSION_NUMBER >= 0x10100003L + const +#endif + u_char *id, int len, int *copy); #endif diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index f83e85d91b..ce832ea13b 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -172,6 +172,8 @@ int ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, ngx_ssl_session_t *sess) { + const u_char *sess_id; + unsigned int sess_id_len; lua_State *L; ngx_int_t rc; ngx_connection_t *c, *fc = NULL; @@ -247,11 +249,13 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, } } + sess_id = SSL_SESSION_get_id(sess, &sess_id_len); + cctx->connection = c; cctx->request = r; cctx->session = sess; - cctx->session_id.data = sess->session_id; - cctx->session_id.len = sess->session_id_length; + cctx->session_id.data = (u_char *) sess_id; + cctx->session_id.len = sess_id_len; cctx->done = 0; dd("setting cctx"); diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index dd468bd048..7289fe81e8 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -108,10 +108,10 @@ sent http request: 59 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil \z ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- no_error_log lua ssl server name: @@ -182,7 +182,7 @@ connected: 1 failed to do SSL handshake: handshake failed --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- no_error_log lua ssl server name: @@ -255,10 +255,10 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log lua ssl server name: "openresty.org" @@ -343,13 +343,13 @@ sent http request: 56 bytes. received: HTTP/1.1 200 OK close: 1 nil ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl set session: \1:2 -lua ssl save session: \1:3 -lua ssl free session: \1:2 -lua ssl free session: \1:1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl set session: \1 +lua ssl save session: \1 +lua ssl free session: \1 +lua ssl free session: \1 $/ --- error_log @@ -432,7 +432,7 @@ failed to send http request: closed \z --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl server name: "blah.openresty.org" @@ -512,7 +512,7 @@ failed to send http request: closed \z --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl server name: "blah.openresty.org" @@ -587,10 +587,10 @@ received: HTTP/1.1 404 Not Found close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log @@ -672,10 +672,10 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]++/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log @@ -693,7 +693,7 @@ SSL reused session server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; - lua_ssl_verify_depth 1; + lua_ssl_verify_depth 0; location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; @@ -748,17 +748,18 @@ $::DSTRootCertificate" --- request GET /t ---- response_body -connected: 1 -failed to do SSL handshake: 20: unable to get local issuer certificate +--- response_body eval +qr{connected: 1 +failed to do SSL handshake: (22: certificate chain too long|20: unable to get local issuer certificate) failed to send http request: closed +} --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out ---- error_log -lua ssl server name: "openresty.org" -lua ssl certificate verify error: (20: unable to get local issuer certificate) +--- error_log eval +['lua ssl server name: "openresty.org"', +qr/lua ssl certificate verify error: \((22: certificate chain too long|20: unable to get local issuer certificate)\)/] --- no_error_log SSL reused session [alert] @@ -771,7 +772,7 @@ SSL reused session server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; - lua_ssl_verify_depth 1; + lua_ssl_verify_depth 0; lua_socket_log_errors off; location /t { #set $port 5000; @@ -827,13 +828,14 @@ $::DSTRootCertificate" --- request GET /t ---- response_body -connected: 1 -failed to do SSL handshake: 20: unable to get local issuer certificate +--- response_body eval +qr/connected: 1 +failed to do SSL handshake: (22: certificate chain too long|20: unable to get local issuer certificate) failed to send http request: closed +/ --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl server name: "openresty.org" @@ -923,10 +925,10 @@ sent http request: 59 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil \z ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log lua ssl server name: "www.google.com" @@ -1013,7 +1015,7 @@ GET /t connected: 1 failed to do SSL handshake: 20: unable to get local issuer certificate ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl server name: "www.google.com" @@ -1095,10 +1097,10 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log @@ -1174,10 +1176,10 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log lua ssl server name: "openresty.org" @@ -1254,14 +1256,14 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ ---- error_log -lua ssl server name: "openresty.org" -SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-SHA SSLv3 +--- error_log eval +['lua ssl server name: "openresty.org"', +qr/SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-SHA (SSLv3|TLSv1)/] --- no_error_log SSL reused session [error] @@ -1334,14 +1336,14 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ ---- error_log -lua ssl server name: "openresty.org" -SSL: TLSv1, cipher: "ECDHE-RSA-AES128-SHA SSLv3 +--- error_log eval +['lua ssl server name: "openresty.org"', +qr/SSL: TLSv1, cipher: "ECDHE-RSA-AES128-SHA (SSLv3|TLSv1)/] --- no_error_log SSL reused session [error] @@ -1412,11 +1414,11 @@ failed to do SSL handshake: handshake failed failed to send http request: closed --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log eval [ -qr/\[crit\] .*?SSL_do_handshake\(\) failed .*?unsupported protocol/, +qr/\[crit\] .*?SSL_do_handshake\(\) failed .*?(unsupported protocol|no protocols available)/, 'lua ssl server name: "openresty.org"', ] --- no_error_log @@ -1488,10 +1490,10 @@ ssl handshake: userdata set keepalive: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: \1:2 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: \1 $/ --- error_log @@ -1564,14 +1566,14 @@ ssl handshake: userdata set keepalive: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl save session: \1:3 -lua ssl save session: \1:4 -lua ssl free session: \1:4 -lua ssl free session: \1:3 -lua ssl free session: \1:2 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl save session: \1 +lua ssl save session: \1 +lua ssl free session: \1 +lua ssl free session: \1 +lua ssl free session: \1 $/ --- error_log @@ -1615,7 +1617,7 @@ hello world --- response_body_like: 500 Internal Server Error --- error_code: 500 --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log attempt to call method 'sslhandshake' (a nil value) @@ -1714,10 +1716,10 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- no_error_log lua ssl server name: @@ -1819,10 +1821,10 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log lua ssl server name: "test.com" @@ -1912,10 +1914,10 @@ failed to do SSL handshake: handshake failed ">>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log eval -qr/SSL_do_handshake\(\) failed .*?unknown protocol/ +qr/SSL_do_handshake\(\) failed .*?(unknown protocol|wrong version number)/ --- no_error_log lua ssl server name: SSL reused session @@ -2011,7 +2013,7 @@ $::TestCertificate >>> test.crl $::TestCRL" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl server name: "test.com" @@ -2090,12 +2092,12 @@ received: HTTP/1.1 302 Moved Temporarily close: 1 nil --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl save session: ([0-9A-F]+):3 -lua ssl free session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log lua ssl server name: "openresty.org" @@ -2149,7 +2151,7 @@ connected: 1 failed to do SSL handshake: timeout --- log_level: debug ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl server name: "openresty.org" @@ -2221,7 +2223,7 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- no_error_log lua ssl server name: @@ -2292,10 +2294,10 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- no_error_log lua ssl server name: @@ -2372,7 +2374,7 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- no_error_log lua ssl server name: @@ -2474,10 +2476,10 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+):2 -lua ssl free session: ([0-9A-F]+):1 +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log --- no_error_log @@ -2570,7 +2572,7 @@ $::TestCertificateKey >>> test.crt $::TestCertificate" ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+:\d+/ +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log lua ssl certificate verify error: (18: self signed certificate) diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 825d016302..cc3a664cfb 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -32,7 +32,7 @@ __DATA__ server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -107,7 +107,7 @@ ssl_session_store_by_lua_block:1: ssl session store by lua is running!, server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -182,7 +182,7 @@ API disabled in the context of ssl_session_store_by_lua* server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -272,9 +272,9 @@ my timer run! listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -340,9 +340,9 @@ API disabled in the context of ssl_session_store_by_lua* server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -412,9 +412,9 @@ ngx.exit does not yield and the error code is eaten. server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -485,9 +485,9 @@ ssl_session_store_by_lua*: handler return value: 0, sess new cb exit code: 0 server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -553,9 +553,9 @@ should never reached here server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -626,7 +626,7 @@ get_phase: ssl_session_store } ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -695,7 +695,7 @@ qr/elapsed in ssl cert by lua: 0.(?:09|1[01])\d+,/, server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -765,7 +765,6 @@ a.lua:1: ssl store session by lua is running! ssl_session_store_by_lua_block { print("handler in test.com") } - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; @@ -775,7 +774,6 @@ a.lua:1: ssl store session by lua is running! server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; @@ -841,7 +839,7 @@ qr/\[emerg\] .*? "ssl_session_store_by_lua_block" directive is not allowed here server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 7be180f84f..4dc992d370 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -33,7 +33,7 @@ __DATA__ server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -119,7 +119,7 @@ ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -203,7 +203,7 @@ qr/elapsed in ssl fetch session by lua: 0.(?:09|1[01])\d+,/, server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -302,9 +302,9 @@ qr/my timer run!/s server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -382,9 +382,9 @@ qr/received memc reply: OK/s server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -463,9 +463,9 @@ should never reached here server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -545,9 +545,9 @@ should never reached here server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -626,9 +626,9 @@ should never reached here server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -709,9 +709,9 @@ should never reached here server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; - ssl_protocols SSLv3; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; server_tokens off; } @@ -792,7 +792,7 @@ should never reached here server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -877,7 +877,7 @@ qr/get_phase: ssl_session_fetch/s } ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -961,7 +961,7 @@ ssl store session by lua is running! server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } @@ -1041,7 +1041,7 @@ qr/\S+:\d+: ssl fetch sess by lua is running!/s server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; + ssl_session_tickets off; server_tokens off; } From 858753bb0dfc191a3a200b41ed2a4f14cc7c829e Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 5 Jan 2018 17:28:43 -0800 Subject: [PATCH 193/848] travis: upgraded pcre to 8.41 and openssl to 1.0.2n. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 36b2e84649..fe0dcea11e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -34,7 +34,7 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_VER=8.40 + - PCRE_VER=8.41 - PCRE_PREFIX=/opt/pcre - PCRE_LIB=$PCRE_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include @@ -48,7 +48,7 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2k OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2n OPENSSL_PATCH_VER=1.0.2h #- NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0c OPENSSL_PATCH_VER=1.1.0c services: From 7c8c503001585a3014858957356d5f7d620ee243 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 7 Jan 2018 11:05:40 -0800 Subject: [PATCH 194/848] bugfix: fixed compatibility with older OpenSSL versions like 0.9.8 and 0.9.7. this regression had appeared in commit 7f108923b. --- src/ngx_http_lua_ssl_session_storeby.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index ce832ea13b..1512eb1ba2 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -249,7 +249,12 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, } } +#if OPENSSL_VERSION_NUMBER >= 0x1000200fL sess_id = SSL_SESSION_get_id(sess, &sess_id_len); +#else + sess_id = sess->session_id; + sess_id_len = sess->session_id_length; +#endif cctx->connection = c; cctx->request = r; From 01b3145227cb8f0c6239cdcb11fe0a365740acb2 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Sun, 7 Jan 2018 22:18:11 +0500 Subject: [PATCH 195/848] travis-ci: enabled openssl-1.1.0g builds. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index fe0dcea11e..9834c8a933 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,7 @@ env: - TEST_NGINX_SLEEP=0.006 matrix: - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2n OPENSSL_PATCH_VER=1.0.2h - #- NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0c OPENSSL_PATCH_VER=1.1.0c + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0g OPENSSL_PATCH_VER=1.1.0d services: - memcache From cf5fb98239ffe87f41c0a046f8e19173569c5e3e Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 8 Jan 2018 10:40:49 +0800 Subject: [PATCH 196/848] tests: corrected the No SNI, No verify case in 129-ssl-socket.t. Signed-off-by: Yichun Zhang (agentzh) --- t/129-ssl-socket.t | 50 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 15 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 7289fe81e8..7f36947d38 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -4,12 +4,13 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 217; +plan tests => repeat_each() * 218; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; +$ENV{TEST_NGINX_SERVER_SSL_PORT} ||= 12345; #log_level 'warn'; log_level 'debug'; @@ -123,19 +124,27 @@ SSL reused session === TEST 2: no SNI, no verify +--- http_config + server { + listen $TEST_NGINX_SERVER_SSL_PORT ssl; + server_name test.com; + ssl_certificate ../html/test.crt; + ssl_certificate_key ../html/test.key; + + location / { + content_by_lua_block { + ngx.exit(201) + } + } + } --- config - server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { - #set $port 5000; - set $port $TEST_NGINX_MEMCACHED_PORT; - - content_by_lua ' - local sock = ngx.socket.tcp() - sock:settimeout(2000) - + content_by_lua_block { do - local ok, err = sock:connect("openresty.org", 443) + local sock = ngx.socket.tcp() + sock:settimeout(2000) + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_SSL_PORT) if not ok then ngx.say("failed to connect: ", err) return @@ -151,7 +160,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\r\nHost: test.com\r\nConnection: close\r\n\r\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -172,21 +181,32 @@ SSL reused session ngx.say("close: ", ok, " ", err) end -- do collectgarbage() - '; + } } --- request GET /t --- response_body connected: 1 -failed to do SSL handshake: handshake failed +ssl handshake: userdata +sent http request: 53 bytes. +received: HTTP/1.1 201 Created +close: 1 nil +--- user_files eval +">>> test.key +$::TestCertificateKey +>>> test.crt +$::TestCertificate" ---- log_level: debug --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ ---- grep_error_log_out +--- grep_error_log_out eval +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +$/ --- no_error_log lua ssl server name: SSL reused session +[error] [alert] --- timeout: 5 From 6d0ecd617a13f7b801f1a95cc53939c8ad78c06a Mon Sep 17 00:00:00 2001 From: yandongxiao Date: Tue, 9 Jan 2018 17:52:39 +0800 Subject: [PATCH 197/848] doc: typo fix: fix "\d+" to [[\d+]]. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 4a07ba01a4..8ddc78af01 100644 --- a/README.markdown +++ b/README.markdown @@ -840,7 +840,7 @@ As noted earlier, PCRE sequences presented within `*_by_lua_block {}` directives # nginx.conf location /test { content_by_lua_block { - local regex = "\d+" + local regex = [[\d+]] local m = ngx.re.match("hello, 1234", regex) if m then ngx.say(m[0]) else ngx.say("not matched!") end } diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f2b477752b..62a0d1ec1e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -684,7 +684,7 @@ As noted earlier, PCRE sequences presented within *_by_lua_block {} # nginx.conf location /test { content_by_lua_block { - local regex = "\d+" + local regex = [[\d+]] local m = ngx.re.match("hello, 1234", regex) if m then ngx.say(m[0]) else ngx.say("not matched!") end } From 5f6e74061126ba0cfcb53d1ae1366c07e99275cf Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 8 Jan 2018 22:57:31 +0800 Subject: [PATCH 198/848] feature: added pure C APIs for the remaining time related Lua APIs (for lua-resyt-core). Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_time.c | 71 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/src/ngx_http_lua_time.c b/src/ngx_http_lua_time.c index 3272a75331..bbb6f2b704 100644 --- a/src/ngx_http_lua_time.c +++ b/src/ngx_http_lua_time.c @@ -272,6 +272,77 @@ ngx_http_lua_ffi_time(void) { return (long) ngx_time(); } + + +void +ngx_http_lua_ffi_update_time(void) +{ + ngx_time_update(); +} + + +void +ngx_http_lua_ffi_today(u_char *buf) +{ + ngx_tm_t tm; + + ngx_gmtime(ngx_time() + ngx_cached_time->gmtoff * 60, &tm); + + ngx_sprintf(buf, "%04d-%02d-%02d", tm.ngx_tm_year, tm.ngx_tm_mon, + tm.ngx_tm_mday); +} + + +void +ngx_http_lua_ffi_localtime(u_char *buf) +{ + ngx_tm_t tm; + + ngx_gmtime(ngx_time() + ngx_cached_time->gmtoff * 60, &tm); + + ngx_sprintf(buf, "%04d-%02d-%02d %02d:%02d:%02d", tm.ngx_tm_year, + tm.ngx_tm_mon, tm.ngx_tm_mday, tm.ngx_tm_hour, tm.ngx_tm_min, + tm.ngx_tm_sec); +} + + +void +ngx_http_lua_ffi_utctime(u_char *buf) +{ + ngx_tm_t tm; + + ngx_gmtime(ngx_time(), &tm); + + ngx_sprintf(buf, "%04d-%02d-%02d %02d:%02d:%02d", tm.ngx_tm_year, + tm.ngx_tm_mon, tm.ngx_tm_mday, tm.ngx_tm_hour, tm.ngx_tm_min, + tm.ngx_tm_sec); +} + + +int +ngx_http_lua_ffi_cookie_time(u_char *buf, long t) +{ + u_char *p; + + p = ngx_http_cookie_time(buf, t); + return p - buf; +} + + +void +ngx_http_lua_ffi_http_time(u_char *buf, long t) +{ + ngx_http_time(buf, t); +} + + +void +ngx_http_lua_ffi_parse_http_time(const u_char *str, size_t len, + long *time) +{ + /* ngx_http_parse_time doesn't modify 'str' actually */ + *time = ngx_http_parse_time((u_char *) str, len); +} #endif /* NGX_LUA_NO_FFI_API */ From 468403e37129632c50bdf7560f09930f593cbb00 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 8 Jan 2018 15:11:32 -0800 Subject: [PATCH 199/848] feature: added the "init_ttl" argument to the pure C function for the shdict:incr() API. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_shdict.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 5b48eb4b0d..b300673e82 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2623,11 +2623,12 @@ ngx_http_lua_ffi_shdict_get(ngx_shm_zone_t *zone, u_char *key, int ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, size_t key_len, double *value, char **err, int has_init, double init, - int *forcible) + long init_ttl, int *forcible) { int i, n; uint32_t hash; ngx_int_t rc; + ngx_time_t *tp = NULL; ngx_http_lua_shdict_ctx_t *ctx; ngx_http_lua_shdict_node_t *sd; double num; @@ -2639,6 +2640,10 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, return NGX_ERROR; } + if (init_ttl > 0) { + tp = ngx_timeofday(); + } + ctx = zone->data; *forcible = 0; @@ -2802,7 +2807,13 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, sd->user_flags = 0; - sd->expires = 0; + if (init_ttl > 0) { + sd->expires = (uint64_t) tp->sec * 1000 + tp->msec + + (uint64_t) init_ttl; + + } else { + sd->expires = 0; + } dd("setting value type to %d", LUA_TNUMBER); From 071bb0ff641c495c69bbbe9ba8e17becf6f03d12 Mon Sep 17 00:00:00 2001 From: yandongxiao Date: Thu, 11 Jan 2018 12:08:53 +0800 Subject: [PATCH 200/848] doc: typo fix: fix core constants "NGX_XX" to "ngx.XX". --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 8ddc78af01..31f3e0f4f4 100644 --- a/README.markdown +++ b/README.markdown @@ -3407,7 +3407,7 @@ Core constants ngx.DECLINED (-5) ``` -Note that only three of these constants are utilized by the [Nginx API for Lua](#nginx-api-for-lua) (i.e., [ngx.exit](#ngxexit) accepts `NGX_OK`, `NGX_ERROR`, and `NGX_DECLINED` as input). +Note that only three of these constants are utilized by the [Nginx API for Lua](#nginx-api-for-lua) (i.e., [ngx.exit](#ngxexit) accepts `ngx.OK`, `ngx.ERROR`, and `ngx.DECLINED` as input). ```lua @@ -5332,7 +5332,7 @@ Number literals can be used directly as the argument, for instance, ngx.exit(501) ``` -Note that while this method accepts all [HTTP status constants](#http-status-constants) as input, it only accepts `NGX_OK` and `NGX_ERROR` of the [core constants](#core-constants). +Note that while this method accepts all [HTTP status constants](#http-status-constants) as input, it only accepts `ngx.OK` and `ngx.ERROR` of the [core constants](#core-constants). Also note that this method call terminates the processing of the current request and that it is recommended that a coding style that combines this method call with the `return` statement, i.e., `return ngx.exit(...)` be used to reinforce the fact that the request processing is being terminated. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 62a0d1ec1e..22adf1910e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2755,7 +2755,7 @@ This API requires a relatively expensive metamethod call and it is recommended t ngx.DECLINED (-5) -Note that only three of these constants are utilized by the [[#Nginx API for Lua|Nginx API for Lua]] (i.e., [[#ngx.exit|ngx.exit]] accepts NGX_OK, NGX_ERROR, and NGX_DECLINED as input). +Note that only three of these constants are utilized by the [[#Nginx API for Lua|Nginx API for Lua]] (i.e., [[#ngx.exit|ngx.exit]] accepts ngx.OK, ngx.ERROR, and ngx.DECLINED as input). ngx.null @@ -4449,7 +4449,7 @@ Number literals can be used directly as the argument, for instance, ngx.exit(501) -Note that while this method accepts all [[#HTTP status constants|HTTP status constants]] as input, it only accepts NGX_OK and NGX_ERROR of the [[#core constants|core constants]]. +Note that while this method accepts all [[#HTTP status constants|HTTP status constants]] as input, it only accepts ngx.OK and ngx.ERROR of the [[#core constants|core constants]]. Also note that this method call terminates the processing of the current request and that it is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.exit(...) be used to reinforce the fact that the request processing is being terminated. From 9a3bbb6f2d00a7ffce4bfc74b5103d8aac715f79 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 12 Jan 2018 13:04:50 -0800 Subject: [PATCH 201/848] tests: updated the server IP for agentzh.org to reflect recent changes. --- t/058-tcp-socket.t | 2 +- t/065-tcp-socket-timeout.t | 10 +++++----- t/090-log-socket-errors.t | 2 +- t/094-uthread-exit.t | 2 +- t/128-duplex-tcp-socket.t | 2 +- t/147-tcp-socket-timeouts.t | 2 +- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index f0c2b481cd..96ea9072cd 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -321,7 +321,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 94ade07b0c..78b8cff7be 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -66,7 +66,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 @@ -96,7 +96,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 150 -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 @@ -125,7 +125,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 102 -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 @@ -154,7 +154,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 102 -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 @@ -707,7 +707,7 @@ GET /t 2: connected: 1 --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 diff --git a/t/090-log-socket-errors.t b/t/090-log-socket-errors.t index 8215ec386f..16576b6fae 100644 --- a/t/090-log-socket-errors.t +++ b/t/090-log-socket-errors.t @@ -59,7 +59,7 @@ GET /t --- response_body timeout --- error_log -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index a62377392f..2f7d9ba731 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -501,7 +501,7 @@ after ngx.say("after") local sock = ngx.socket.tcp() sock:settimeout(12000) - local ok, err = sock:connect("106.184.1.99", 12345) + local ok, err = sock:connect("172.105.207.225", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/128-duplex-tcp-socket.t b/t/128-duplex-tcp-socket.t index e8434a3f84..aed560c18b 100644 --- a/t/128-duplex-tcp-socket.t +++ b/t/128-duplex-tcp-socket.t @@ -396,7 +396,7 @@ F(ngx_http_lua_socket_tcp_finalize_write_part) { end sock:settimeout(300) - local ok, err = sock:connect("106.184.1.99", 12345) + local ok, err = sock:connect("172.105.207.225", 12345) ngx.say("connect: ", ok, " ", err) local ok, err = sock:close() diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 1872d29fb5..4ebc8f370e 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -346,7 +346,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 From 051f35240c2c05bcc434700b6f2c206986918439 Mon Sep 17 00:00:00 2001 From: yandongxiao Date: Thu, 11 Jan 2018 18:24:40 +0800 Subject: [PATCH 202/848] doc: setting ngx.header.HEADER no longer throws out an exception when the header is already sent out; it now just logs an error message. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 31f3e0f4f4..960f7293c5 100644 --- a/README.markdown +++ b/README.markdown @@ -4121,7 +4121,7 @@ The same applies to assigning an empty table: ngx.header["X-My-Header"] = {}; ``` -Setting `ngx.header.HEADER` after sending out response headers (either explicitly with [ngx.send_headers](#ngxsend_headers) or implicitly with [ngx.print](#ngxprint) and similar) will throw out a Lua exception. +Setting `ngx.header.HEADER` after sending out response headers (either explicitly with [ngx.send_headers](#ngxsend_headers) or implicitly with [ngx.print](#ngxprint) and similar) will log an error message. Reading `ngx.header.HEADER` will return the value of the response header named `HEADER`. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 22adf1910e..2d57934787 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3406,7 +3406,7 @@ The same applies to assigning an empty table: ngx.header["X-My-Header"] = {}; -Setting ngx.header.HEADER after sending out response headers (either explicitly with [[#ngx.send_headers|ngx.send_headers]] or implicitly with [[#ngx.print|ngx.print]] and similar) will throw out a Lua exception. +Setting ngx.header.HEADER after sending out response headers (either explicitly with [[#ngx.send_headers|ngx.send_headers]] or implicitly with [[#ngx.print|ngx.print]] and similar) will log an error message. Reading ngx.header.HEADER will return the value of the response header named HEADER. From d125123f8be3db55ab57bf0206ad086c9ff43934 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 12 Jan 2018 14:16:13 -0800 Subject: [PATCH 203/848] tests: updated more uses of the server IP addr for agentzh.org to reflect recent changes. --- t/023-rewrite/tcp-socket-timeout.t | 2 +- t/023-rewrite/tcp-socket.t | 2 +- t/023-rewrite/uthread-exit.t | 2 +- t/024-access/uthread-exit.t | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index 7ac09e243b..329c04513c 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -63,7 +63,7 @@ GET /t1 failed to connect: timeout --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index a951014b74..41aeab74af 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -329,7 +329,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out, when connecting to 106.184.1.99:12345 +lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 --- timeout: 10 diff --git a/t/023-rewrite/uthread-exit.t b/t/023-rewrite/uthread-exit.t index 3df46835b6..6ebbb678bf 100644 --- a/t/023-rewrite/uthread-exit.t +++ b/t/023-rewrite/uthread-exit.t @@ -521,7 +521,7 @@ after ngx.say("after") local sock = ngx.socket.tcp() sock:settimeout(12000) - local ok, err = sock:connect("106.184.1.99", 12345) + local ok, err = sock:connect("172.105.207.225", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/024-access/uthread-exit.t b/t/024-access/uthread-exit.t index 7c146ae7ec..02c2a1f96f 100644 --- a/t/024-access/uthread-exit.t +++ b/t/024-access/uthread-exit.t @@ -502,7 +502,7 @@ after ngx.say("after") local sock = ngx.socket.tcp() sock:settimeout(12000) - local ok, err = sock:connect("106.184.1.99", 12345) + local ok, err = sock:connect("172.105.207.225", 12345) if not ok then ngx.say("failed to connect: ", err) return From c2b6ae2ae3e0ce5b74bf0f659d1ae62635b79935 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 15 Jan 2018 11:44:45 -0800 Subject: [PATCH 204/848] doc: updated the copyright years. --- doc/HttpLuaModule.wiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 2d57934787..f589129b7e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -817,7 +817,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. +Copyright (C) 2009-2018, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved. From e343b3a14e77efa9bc92288d02511c78450e1143 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 15 Jan 2018 17:44:16 -0800 Subject: [PATCH 205/848] tests: updated valgrind.suppress for latest glibc. --- valgrind.suppress | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/valgrind.suppress b/valgrind.suppress index 727398637e..56f081076b 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -190,3 +190,39 @@ fun:ngx_set_environment fun:ngx_single_process_cycle } +{ + + Memcheck:Param + epoll_pwait(sigmask) + fun:epoll_pwait + fun:epoll_wait + fun:ngx_epoll_process_events + fun:ngx_process_events_and_timers +} +{ + + Memcheck:Param + epoll_pwait(sigmask) + fun:epoll_pwait + fun:epoll_wait + fun:ngx_epoll_test_rdhup + fun:ngx_epoll_init + fun:ngx_event_process_init +} +{ + + Memcheck:Param + epoll_pwait(sigmask) + fun:epoll_pwait + fun:ngx_epoll_process_events + fun:ngx_process_events_and_timers +} +{ + + Memcheck:Param + epoll_pwait(sigmask) + fun:epoll_pwait + fun:ngx_epoll_test_rdhup + fun:ngx_epoll_init + fun:ngx_event_process_init +} From 11aa4636bad3c32158d1f330a0d011f59fe14e15 Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 11 Jan 2018 19:35:14 +0800 Subject: [PATCH 206/848] bugfix: avoided sharing the same code object for identical Lua inlined code chunks in different phases due to chunk name conflicts. thanks yandongxiao for the report in #1232. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_balancer.c | 4 +- src/ngx_http_lua_directive.c | 53 ++++++++--- src/ngx_http_lua_ssl_certby.c | 6 +- src/ngx_http_lua_ssl_session_fetchby.c | 6 +- src/ngx_http_lua_ssl_session_storeby.c | 6 +- t/132-lua-blocks.t | 125 ++++++++++++++++++++++++- 6 files changed, 182 insertions(+), 18 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 2fa634eb4d..fdf2af31a7 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -173,13 +173,15 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->balancer.src = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("balancer_by_lua") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } lscf->balancer.src_key = p; + p = ngx_copy(p, "balancer_by_lua", sizeof("balancer_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 014a47237c..fb8c6dc5bf 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -43,7 +43,7 @@ static ngx_int_t ngx_http_lua_set_by_lua_init(ngx_http_request_t *r); #endif static u_char *ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, - size_t tag_len); + size_t tag_len, size_t *chunkname_len); static ngx_int_t ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, ngx_http_lua_block_parser_ctx_t *ctx); static u_char *ngx_http_lua_strlstrn(u_char *s1, u_char *last, u_char *s2, @@ -271,13 +271,15 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) filter_data->size = filter.size; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("set_by_lua") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } filter_data->key = p; + p = ngx_copy(p, "set_by_lua", sizeof("set_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[2].data, value[2].len); *p = '\0'; @@ -448,6 +450,7 @@ ngx_http_lua_rewrite_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, char * ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; u_char *p, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; @@ -482,7 +485,8 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (cmd->post == ngx_http_lua_rewrite_handler_inline) { chunkname = ngx_http_lua_gen_chunk_name(cf, "rewrite_by_lua", - sizeof("rewrite_by_lua") - 1); + sizeof("rewrite_by_lua") - 1, + &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; } @@ -493,13 +497,15 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->rewrite_src.value = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; } llcf->rewrite_src_key = p; + p = ngx_copy(p, chunkname, chunkname_len); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -562,6 +568,7 @@ ngx_http_lua_access_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, char * ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; u_char *p, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; @@ -592,7 +599,8 @@ ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (cmd->post == ngx_http_lua_access_handler_inline) { chunkname = ngx_http_lua_gen_chunk_name(cf, "access_by_lua", - sizeof("access_by_lua") - 1); + sizeof("access_by_lua") - 1, + &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; } @@ -603,13 +611,15 @@ ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->access_src.value = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; } llcf->access_src_key = p; + p = ngx_copy(p, chunkname, chunkname_len); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -672,6 +682,7 @@ ngx_http_lua_content_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, char * ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; u_char *p; u_char *chunkname; ngx_str_t *value; @@ -706,7 +717,8 @@ ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (cmd->post == ngx_http_lua_content_handler_inline) { chunkname = ngx_http_lua_gen_chunk_name(cf, "content_by_lua", - sizeof("content_by_lua") - 1); + sizeof("content_by_lua") - 1, + &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; } @@ -719,13 +731,15 @@ ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->content_src.value = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; } llcf->content_src_key = p; + p = ngx_copy(p, chunkname, chunkname_len); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -795,6 +809,7 @@ ngx_http_lua_log_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, char * ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; u_char *p, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; @@ -825,7 +840,8 @@ ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (cmd->post == ngx_http_lua_log_handler_inline) { chunkname = ngx_http_lua_gen_chunk_name(cf, "log_by_lua", - sizeof("log_by_lua") - 1); + sizeof("log_by_lua") - 1, + &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; } @@ -836,13 +852,15 @@ ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->log_src.value = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; } llcf->log_src_key = p; + p = ngx_copy(p, chunkname, chunkname_len); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -936,13 +954,17 @@ ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, /* Don't eval nginx variables for inline lua code */ llcf->header_filter_src.value = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("header_filter_by_lua") + + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; } llcf->header_filter_src_key = p; + p = ngx_copy(p, "header_filter_by_lua", + sizeof("header_filter_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -1036,13 +1058,16 @@ ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, /* Don't eval nginx variables for inline lua code */ llcf->body_filter_src.value = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("body_filter_by_lua") + + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; } llcf->body_filter_src_key = p; + p = ngx_copy(p, "body_filter_by_lua", sizeof("body_filter_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -1248,7 +1273,8 @@ ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) static u_char * -ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len) +ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, + size_t *chunkname_len) { u_char *p, *out; size_t len; @@ -1282,6 +1308,7 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len) tag_len, tag, cf->conf_file->file.name.data + cf->conf_file->file.name.len - p, p, cf->conf_file->line); + *chunkname_len = len; return out; } diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 020b341873..59ea8ae7ee 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -163,13 +163,17 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_cert_src = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("ssl_certificate_by_lua") + + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } lscf->srv.ssl_cert_src_key = p; + p = ngx_copy(p, "ssl_certificate_by_lua", + sizeof("ssl_certificate_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 2c75f6a912..bd3129d2e5 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -153,13 +153,17 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_sess_fetch_src = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("ssl_session_fetch_by_lua") + + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } lscf->srv.ssl_sess_fetch_src_key = p; + p = ngx_copy(p, "ssl_session_fetch_by_lua", + sizeof("ssl_session_fetch_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 1512eb1ba2..7609c39b6a 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -151,13 +151,17 @@ ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_sess_store_src = value[1]; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + p = ngx_palloc(cf->pool, + sizeof("ssl_session_store_by_lua") + + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } lscf->srv.ssl_sess_store_src_key = p; + p = ngx_copy(p, "ssl_session_store_by_lua", + sizeof("ssl_session_store_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/t/132-lua-blocks.t b/t/132-lua-blocks.t index a57d701735..292af58c37 100644 --- a/t/132-lua-blocks.t +++ b/t/132-lua-blocks.t @@ -10,7 +10,9 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 3 + 3); +plan tests => repeat_each() * (blocks() * 3 + 13); + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); #no_diff(); no_long_string(); @@ -604,3 +606,124 @@ GET /t --- response_body_like: } --- no_error_log [error] + + + +=== TEST 24: inline Lua code cache should not mess up across different phases +--- http_config + # The indent is enforced to generate the same hash tags + ssl_session_fetch_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + ssl_session_store_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + + upstream backend { + server 0.0.0.1; + balancer_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_session_tickets off; + + location /foo { + set_by_lua_block $x { + ngx.log(ngx.INFO, "hello") + } + rewrite_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + access_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + content_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + header_filter_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + body_filter_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + log_by_lua_block { + ngx.log(ngx.INFO, "hello") + } + } + } +--- config + lua_ssl_trusted_certificate ../../cert/test.crt; + + location = /proxy { + proxy_pass http://backend; + } + + location /t { + content_by_lua_block { + ngx.location.capture("/proxy") + + local sock = ngx.socket.tcp() + sock:settimeout(2000) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + package.loaded.session = sess + sock:close() + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + } + } +--- request +GET /t +--- no_error_log +[error] +--- error_log eval +[ +qr/balancer_by_lua:\d+: hello/, +qr/ssl_session_fetch_by_lua_block:\d+: hello/, +qr/ssl_certificate_by_lua:\d+: hello/, +qr/ssl_session_store_by_lua_block:\d+: hello/, +qr/set_by_lua:\d+: hello/, +qr/rewrite_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/access_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/content_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/header_filter_by_lua:\d+: hello/, +qr/body_filter_by_lua:\d+: hello/, +qr/log_by_lua\(nginx\.conf:\d+\):\d+: hello/, +] From a46862afec27eed971570732613bd4db1f86e8a9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 15 Jan 2018 18:34:09 -0800 Subject: [PATCH 207/848] doc: updated the copyright years in markdown as well. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 960f7293c5..f0c3008a8c 100644 --- a/README.markdown +++ b/README.markdown @@ -995,7 +995,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. +Copyright (C) 2009-2018, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved. From 90d7522342e78ecb1f1ea2b5d066ade404a48b24 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 10 Jan 2018 20:12:30 -0800 Subject: [PATCH 208/848] doc: added the new 'init_ttl' argument to the shdict:incr() documentation. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 25 ++++++++++++++++++++++++- doc/HttpLuaModule.wiki | 24 +++++++++++++++++++++++- 2 files changed, 47 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index f0c3008a8c..25bf949d65 100644 --- a/README.markdown +++ b/README.markdown @@ -6451,10 +6451,12 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.incr -------------------- -**syntax:** *newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?)* +**syntax:** *newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?, init_ttl?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**optional requirement:** `resty.core.shdict` or `resty.core` + Increments the (numerical) value for `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict) by the step value `value`. Returns the new resulting number if the operation is successfully completed or `nil` and an error message otherwise. When the key does not exist or has already expired in the shared dictionary, @@ -6464,6 +6466,25 @@ When the key does not exist or has already expired in the shared dictionary, Like the [add](#ngxshareddictadd) method, it also overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. +The optional `init_ttl` argument specifies expiration time (in seconds) of the value when it is initialized via the `init` argument. The time resolution is `0.001` seconds. If `init_ttl` takes the value `0` (which is the default), then the item will never expire. This argument cannot be provided without providing the `init` argument as well, and has no effect if the value already exists (e.g., if it was previously inserted via [set](#ngxshareddictset) or the likes). + +**Note:** Usage of the `init_ttl` argument requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. Example: + +```lua + + require "resty.core" + + local cats = ngx.shared.cats + local newval, err = cats:incr("black_cats", 1, 0, 0.1) + + print(newval) -- 1 + + ngx.sleep(0.2) + + local val, err = cats:get("black_cats") + print(val) -- nil +``` + The `forcible` return value will always be `nil` when the `init` argument is not specified. If this method succeeds in storing the current item by forcibly removing other not-yet-expired items in the dictionary via LRU, the `forcible` return value will be `true`. If it stores the item without forcibly removing other valid items, then the return value `forcible` will be `false`. @@ -6476,6 +6497,8 @@ This method was first introduced in the `v0.3.1rc22` release. The optional `init` parameter was first added in the `v0.10.6` release. +The optional `init_ttl` parameter was introduced in the `v0.10.12rc2` release. + See also [ngx.shared.DICT](#ngxshareddict). [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f589129b7e..919fffebe8 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5412,10 +5412,12 @@ This feature was first introduced in the v0.3.1rc22 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.incr == -'''syntax:''' ''newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?)'' +'''syntax:''' ''newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?, init_ttl?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''optional requirement:''' resty.core.shdict or resty.core + Increments the (numerical) value for key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] by the step value value. Returns the new resulting number if the operation is successfully completed or nil and an error message otherwise. When the key does not exist or has already expired in the shared dictionary, @@ -5425,6 +5427,24 @@ When the key does not exist or has already expired in the shared dictionary, Like the [[#ngx.shared.DICT.add|add]] method, it also overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. +The optional init_ttl argument specifies expiration time (in seconds) of the value when it is initialized via the init argument. The time resolution is 0.001 seconds. If init_ttl takes the value 0 (which is the default), then the item will never expire. This argument cannot be provided without providing the init argument as well, and has no effect if the value already exists (e.g., if it was previously inserted via [[#ngx.shared.DICT.set|set]] or the likes). + +'''Note:''' Usage of the init_ttl argument requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. Example: + + + require "resty.core" + + local cats = ngx.shared.cats + local newval, err = cats:incr("black_cats", 1, 0, 0.1) + + print(newval) -- 1 + + ngx.sleep(0.2) + + local val, err = cats:get("black_cats") + print(val) -- nil + + The forcible return value will always be nil when the init argument is not specified. If this method succeeds in storing the current item by forcibly removing other not-yet-expired items in the dictionary via LRU, the forcible return value will be true. If it stores the item without forcibly removing other valid items, then the return value forcible will be false. @@ -5437,6 +5457,8 @@ This method was first introduced in the v0.3.1rc22 release. The optional init parameter was first added in the v0.10.6 release. +The optional init_ttl parameter was introduced in the v0.10.12rc2 release. + See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.lpush == From 9e5cc6e239bacfbc5c65541bf176323f4f84afbb Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 19 Jan 2018 11:46:58 -0800 Subject: [PATCH 209/848] bugfix: nginx rewrite directive may initiate internal redirects without clearing any module ctx and rewrite_by_lua* handlers might think it was re-entered and thus it might lead to request hang. thanks twistedfall for the report in #1235. --- src/ngx_http_lua_rewriteby.c | 8 ++++++++ t/023-rewrite/redirect.t | 40 ++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 2fcf16f846..6a6b992b31 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -126,6 +126,11 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) return NGX_HTTP_OK; } + + r->write_event_handler = ngx_http_core_run_phases; + ctx->entered_rewrite_phase = 0; + + return NGX_DECLINED; } return rc; @@ -363,6 +368,9 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) return NGX_HTTP_OK; } + r->write_event_handler = ngx_http_core_run_phases; + ctx->entered_rewrite_phase = 0; + return NGX_DECLINED; } diff --git a/t/023-rewrite/redirect.t b/t/023-rewrite/redirect.t index 99f3fd4f70..8fa9aa2e79 100644 --- a/t/023-rewrite/redirect.t +++ b/t/023-rewrite/redirect.t @@ -122,3 +122,43 @@ GET /read --- raw_response_headers_like: Location: /foo\r\n --- response_body_like: 302 Found --- error_code: 302 + + + +=== TEST 7: internal redirects that do not clear module ctx +--- http_config + rewrite_by_lua_no_postpone on; +--- config + rewrite_by_lua_block { + -- this is empty by intention + } + + location /url1 { + rewrite ^ /url2; + } +--- request +GET /url1 +--- response_body_like: 404 Not Found +--- error_log eval +qr{\[error\] .*?/html/url2".*?No such file or directory} +--- error_code: 404 + + + +=== TEST 8: internal redirects that do not clear module ctx (yield in rewrite handler) +--- http_config + rewrite_by_lua_no_postpone on; +--- config + rewrite_by_lua_block { + ngx.sleep(0.001) + } + + location /url1 { + rewrite ^ /url2; + } +--- request +GET /url1 +--- response_body_like: 404 Not Found +--- error_log eval +qr{\[error\] .*?/html/url2".*?No such file or directory} +--- error_code: 404 From fcd0deafff34ebae1fbe16b5e653d0fe6ec57851 Mon Sep 17 00:00:00 2001 From: Alessandro Ghedini Date: Mon, 8 Jan 2018 17:08:02 +0000 Subject: [PATCH 210/848] ssl: support enabling TLSv1.3 via lua_ssl_protocols. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 8 +-- README.markdown | 4 +- doc/HttpLuaModule.wiki | 4 +- src/ngx_http_lua_module.c | 3 ++ t/155-tls13.t | 106 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 120 insertions(+), 5 deletions(-) create mode 100644 t/155-tls13.t diff --git a/.travis.yml b/.travis.yml index 9834c8a933..fcf369e984 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,8 +48,10 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2n OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0g OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2n OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0g OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d + # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" + # to $OPENSSL_OPT. services: - memcache @@ -113,7 +115,7 @@ script: - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch - - ./config no-threads shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) + - ./config no-threads shared enable-ssl3 enable-ssl3-method $OPENSSL_OPT -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. diff --git a/README.markdown b/README.markdown index 25bf949d65..f4be57c867 100644 --- a/README.markdown +++ b/README.markdown @@ -2921,7 +2921,7 @@ This directive was first introduced in the `v0.9.11` release. lua_ssl_protocols ----------------- -**syntax:** *lua_ssl_protocols \[SSLv2\] \[SSLv3\] \[TLSv1\] [TLSv1.1] [TLSv1.2]* +**syntax:** *lua_ssl_protocols \[SSLv2\] \[SSLv3\] \[TLSv1\] [TLSv1.1] [TLSv1.2] [TLSv1.3]* **default:** *lua_ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2* @@ -2929,6 +2929,8 @@ lua_ssl_protocols Enables the specified protocols for requests to a SSL/TLS server in the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. +The support for the `TLSv1.3` parameter requires version `v0.10.12` *and* OpenSSL 1.1.1. + This directive was first introduced in the `v0.9.11` release. [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 919fffebe8..4270be4cf3 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2469,7 +2469,7 @@ This directive was first introduced in the v0.9.11 release. == lua_ssl_protocols == -'''syntax:''' ''lua_ssl_protocols [SSLv2] [SSLv3] [TLSv1] [TLSv1.1] [TLSv1.2]'' +'''syntax:''' ''lua_ssl_protocols [SSLv2] [SSLv3] [TLSv1] [TLSv1.1] [TLSv1.2] [TLSv1.3]'' '''default:''' ''lua_ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2'' @@ -2477,6 +2477,8 @@ This directive was first introduced in the v0.9.11 release. Enables the specified protocols for requests to a SSL/TLS server in the [[#tcpsock:sslhandshake|tcpsock:sslhandshake]] method. +The support for the TLSv1.3 parameter requires version v0.10.12 *and* OpenSSL 1.1.1. + This directive was first introduced in the v0.9.11 release. == lua_ssl_trusted_certificate == diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 9d914e86e8..ae8bc0e36f 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -65,6 +65,9 @@ static ngx_conf_bitmask_t ngx_http_lua_ssl_protocols[] = { { ngx_string("TLSv1"), NGX_SSL_TLSv1 }, { ngx_string("TLSv1.1"), NGX_SSL_TLSv1_1 }, { ngx_string("TLSv1.2"), NGX_SSL_TLSv1_2 }, +#ifdef NGX_SSL_TLSv1_3 + { ngx_string("TLSv1.3"), NGX_SSL_TLSv1_3 }, +#endif { ngx_null_string, 0 } }; diff --git a/t/155-tls13.t b/t/155-tls13.t new file mode 100644 index 0000000000..9f7cf86d2e --- /dev/null +++ b/t/155-tls13.t @@ -0,0 +1,106 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +repeat_each(3); + +# All these tests need to have new openssl +my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; +my $openssl_version = eval { `$NginxBinary -V 2>&1` }; + +if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { + plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +} else { + plan tests => repeat_each() * (blocks() * 5); +} + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); +$ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; + +sub read_file { + my $infile = shift; + open my $in, $infile + or die "cannot open $infile for reading: $!"; + my $cert = do { local $/; <$in> }; + close $in; + $cert; +} + +our $TestCertificate = read_file("t/cert/test.crt"); +our $TestCertificateKey = read_file("t/cert/test.key"); + +#log_level 'warn'; +log_level 'debug'; + +no_long_string(); +#no_diff(); + +run_tests(); + +__DATA__ + +=== TEST 1: handshake, TLSv1.3 +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../html/test.crt; + ssl_certificate_key ../html/test.key; + ssl_protocols TLSv1.2 TLSv1.3; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../html/test.crt; + lua_ssl_protocols TLSv1.2 TLSv1.3; + location /t { + #set $port 5000; + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(3000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + else + ngx.say("ssl handshake: ", type(sess)) + end + end -- do + collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata + +--- user_files eval +">>> test.key +$::TestCertificateKey +>>> test.crt +$::TestCertificate" +--- error_log +SSL: TLSv1.3, +--- no_error_log +[error] +[alert] +--- timeout: 5 From 00b92b3d0052797ee8c3e20f11dbd748e794b294 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 20 Jan 2018 16:25:15 -0800 Subject: [PATCH 211/848] style: fixed minor coding style issues found by ngx-releng. --- src/ngx_http_lua_clfactory.c | 40 ++++++++++++++++++------------------ src/ngx_http_lua_common.h | 8 ++++---- src/ngx_http_lua_exception.h | 6 +++--- src/ngx_http_lua_probe.h | 3 ++- src/ngx_http_lua_util.h | 8 ++++---- 5 files changed, 33 insertions(+), 32 deletions(-) diff --git a/src/ngx_http_lua_clfactory.c b/src/ngx_http_lua_clfactory.c index 89698c798d..041f0466e6 100644 --- a/src/ngx_http_lua_clfactory.c +++ b/src/ngx_http_lua_clfactory.c @@ -59,15 +59,15 @@ * length(Instruction) = 4 or 8 * little endian or big endian */ -#define LUA_LITTLE_ENDIAN_4BYTES_CODE \ +#define LUA_LITTLE_ENDIAN_4BYTES_CODE \ "\x24\x00\x00\x00\x1e\x00\x00\x01\x1e\x00\x80\x00" -#define LUA_LITTLE_ENDIAN_8BYTES_CODE \ - "\x24\x00\x00\x00\x00\x00\x00\x00\x1e\x00\x00\x01" \ +#define LUA_LITTLE_ENDIAN_8BYTES_CODE \ + "\x24\x00\x00\x00\x00\x00\x00\x00\x1e\x00\x00\x01" \ "\x00\x00\x00\x00\x1e\x00\x80\x00\x00\x00\x00\x00" -#define LUA_BIG_ENDIAN_4BYTES_CODE \ +#define LUA_BIG_ENDIAN_4BYTES_CODE \ "\x00\x00\x00\x24\x01\x00\x00\x1e\x00\x08\x00\x1e" -#define LUA_BIG_ENDIAN_8BYTES_CODE \ - "\x00\x00\x00\x00\x00\x00\x00\x24\x00\x00\x00\x00" \ +#define LUA_BIG_ENDIAN_8BYTES_CODE \ + "\x00\x00\x00\x00\x00\x00\x00\x24\x00\x00\x00\x00" \ "\x01\x00\x00\x1e\x00\x00\x00\x00\x00\x08\x00\x1e" #define LUA_LITTLE_ENDIAN_4BYTES_CODE_LEN (4 + 4 + 4) #define LUA_LITTLE_ENDIAN_8BYTES_CODE_LEN (8 + 8 + 8) @@ -156,8 +156,8 @@ #define POS_MAX_STACK_SIZE (POS_IS_VAR_ARG + sizeof(char)) #define POS_NUM_OF_INST (POS_MAX_STACK_SIZE +sizeof(char)) #define POS_BYTECODE (POS_NUM_OF_INST + sizeof(int)) -#define MAX_BEGIN_CODE_SIZE \ - (POS_BYTECODE + LUA_LITTLE_ENDIAN_8BYTES_CODE_LEN \ +#define MAX_BEGIN_CODE_SIZE \ + (POS_BYTECODE + LUA_LITTLE_ENDIAN_8BYTES_CODE_LEN \ + sizeof(int) + sizeof(int)) #define MAX_END_CODE_SIZE (sizeof(int) + sizeof(int) + sizeof(int)) @@ -225,24 +225,24 @@ /* bytecode for luajit 2.0 */ -#define LJ20_LITTLE_ENDIAN_CODE_STRIPPED \ - "\x14\x03\x00\x01\x00\x01\x00\x03" \ - "\x31\x00\x00\x00\x30\x00\x00\x80\x48\x00\x02\x00" \ +#define LJ20_LITTLE_ENDIAN_CODE_STRIPPED \ + "\x14\x03\x00\x01\x00\x01\x00\x03" \ + "\x31\x00\x00\x00\x30\x00\x00\x80\x48\x00\x02\x00" \ "\x00\x00" -#define LJ20_BIG_ENDIAN_CODE_STRIPPED \ - "\x14\x03\x00\x01\x00\x01\x00\x03" \ - "\x00\x00\x00\x31\x80\x00\x00\x30\x00\x02\x00\x48" \ +#define LJ20_BIG_ENDIAN_CODE_STRIPPED \ + "\x14\x03\x00\x01\x00\x01\x00\x03" \ + "\x00\x00\x00\x31\x80\x00\x00\x30\x00\x02\x00\x48" \ "\x00\x00" -#define LJ20_LITTLE_ENDIAN_CODE \ - "\x15\x03\x00\x01\x00\x01\x00\x03\x00" \ - "\x31\x00\x00\x00\x30\x00\x00\x80\x48\x00\x02\x00" \ +#define LJ20_LITTLE_ENDIAN_CODE \ + "\x15\x03\x00\x01\x00\x01\x00\x03\x00" \ + "\x31\x00\x00\x00\x30\x00\x00\x80\x48\x00\x02\x00" \ "\x00\x00" -#define LJ20_BIG_ENDIAN_CODE \ - "\x15\x03\x00\x01\x00\x01\x00\x03\x00" \ - "\x00\x00\x00\x31\x80\x00\x00\x30\x00\x02\x00\x48" \ +#define LJ20_BIG_ENDIAN_CODE \ + "\x15\x03\x00\x01\x00\x01\x00\x03\x00" \ + "\x00\x00\x00\x31\x80\x00\x00\x30\x00\x02\x00\x48" \ "\x00\x00" /* bytecode for luajit 2.1 */ diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 1ad2f8bcbf..3bfaa09054 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -79,20 +79,20 @@ #define NGX_HTTP_LUA_INLINE_TAG "nhli_" -#define NGX_HTTP_LUA_INLINE_TAG_LEN \ +#define NGX_HTTP_LUA_INLINE_TAG_LEN \ (sizeof(NGX_HTTP_LUA_INLINE_TAG) - 1) -#define NGX_HTTP_LUA_INLINE_KEY_LEN \ +#define NGX_HTTP_LUA_INLINE_KEY_LEN \ (NGX_HTTP_LUA_INLINE_TAG_LEN + 2 * MD5_DIGEST_LENGTH) /* Nginx HTTP Lua File tag prefix */ #define NGX_HTTP_LUA_FILE_TAG "nhlf_" -#define NGX_HTTP_LUA_FILE_TAG_LEN \ +#define NGX_HTTP_LUA_FILE_TAG_LEN \ (sizeof(NGX_HTTP_LUA_FILE_TAG) - 1) -#define NGX_HTTP_LUA_FILE_KEY_LEN \ +#define NGX_HTTP_LUA_FILE_KEY_LEN \ (NGX_HTTP_LUA_FILE_TAG_LEN + 2 * MD5_DIGEST_LENGTH) diff --git a/src/ngx_http_lua_exception.h b/src/ngx_http_lua_exception.h index a70708a94c..93dfb8f57a 100644 --- a/src/ngx_http_lua_exception.h +++ b/src/ngx_http_lua_exception.h @@ -12,13 +12,13 @@ #include "ngx_http_lua_common.h" -#define NGX_LUA_EXCEPTION_TRY \ +#define NGX_LUA_EXCEPTION_TRY \ if (setjmp(ngx_http_lua_exception) == 0) -#define NGX_LUA_EXCEPTION_CATCH \ +#define NGX_LUA_EXCEPTION_CATCH \ else -#define NGX_LUA_EXCEPTION_THROW(x) \ +#define NGX_LUA_EXCEPTION_THROW(x) \ longjmp(ngx_http_lua_exception, (x)) diff --git a/src/ngx_http_lua_probe.h b/src/ngx_http_lua_probe.h index 37b25002cc..749fa95e65 100644 --- a/src/ngx_http_lua_probe.h +++ b/src/ngx_http_lua_probe.h @@ -44,7 +44,8 @@ #define ngx_http_lua_probe_socket_tcp_receive_done(r, u, data, len) \ NGINX_LUA_HTTP_LUA_SOCKET_TCP_RECEIVE_DONE(r, u, data, len) -#define ngx_http_lua_probe_socket_tcp_setkeepalive_buf_unread(r, u, data, len)\ +#define ngx_http_lua_probe_socket_tcp_setkeepalive_buf_unread(r, u, data, \ + len) \ NGINX_LUA_HTTP_LUA_SOCKET_TCP_SETKEEPALIVE_BUF_UNREAD(r, u, data, len) #define ngx_http_lua_probe_user_thread_spawn(r, creator, newthread) \ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 69203254cc..e9924e41eb 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -250,9 +250,9 @@ void ngx_http_lua_cleanup_free(ngx_http_request_t *r, ngx_http_cleanup_pt *cleanup); -#define ngx_http_lua_check_if_abortable(L, ctx) \ - if ((ctx)->no_abort) { \ - return luaL_error(L, "attempt to abort with pending subrequests"); \ +#define ngx_http_lua_check_if_abortable(L, ctx) \ + if ((ctx)->no_abort) { \ + return luaL_error(L, "attempt to abort with pending subrequests"); \ } @@ -372,7 +372,7 @@ ngx_http_lua_set_globals_table(lua_State *L) } -#define ngx_http_lua_hash_literal(s) \ +#define ngx_http_lua_hash_literal(s) \ ngx_http_lua_hash_str((u_char *) s, sizeof(s) - 1) From 988756924be5b2e013831d9f9d6affdec924f8f5 Mon Sep 17 00:00:00 2001 From: spacewander Date: Sun, 21 Jan 2018 20:28:44 +0800 Subject: [PATCH 212/848] tests: removed duplicate entries in valgrind.supress. Signed-off-by: Yichun Zhang (agentzh) --- valgrind.suppress | 19 ------------------- 1 file changed, 19 deletions(-) diff --git a/valgrind.suppress b/valgrind.suppress index 56f081076b..7fff542ffe 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -22,16 +22,6 @@ fun:ngx_epoll_process_events fun:ngx_process_events_and_timers } -{ - - Memcheck:Cond - fun:ngx_vslprintf - fun:ngx_snprintf - fun:ngx_sock_ntop - fun:ngx_event_accept - fun:ngx_epoll_process_events - fun:ngx_process_events_and_timers -} { Memcheck:Addr1 @@ -181,15 +171,6 @@ fun:ngx_create_pool fun:main } -{ - - Memcheck:Leak - match-leak-kinds: definite - fun:malloc - fun:ngx_alloc - fun:ngx_set_environment - fun:ngx_single_process_cycle -} { Memcheck:Param From df1b0198ff0262c755029738058f5917a11a398a Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 22 Jan 2018 14:50:40 +0800 Subject: [PATCH 213/848] optimize: corrected the initial table size of req socket objects. The req socket uses two slots in array and three slots in hash at most. At first it uses one array slot and one hash slot: array part: upstream udata hash part: __index After calling settimeouts, it uses two array slots and three hash slots: array part: [1] upstream udata [2] connect timeout hash part: __index [4] send timeout [5] read timeout Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 2 +- t/062-count.t | 74 +++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index c34a24e5c3..72ce97cdb4 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4390,7 +4390,7 @@ ngx_http_lua_req_socket(lua_State *L) r->request_body = rb; } - lua_createtable(L, 3 /* narr */, 1 /* nrec */); /* the object */ + lua_createtable(L, 2 /* narr */, 3 /* nrec */); /* the object */ if (raw) { lua_pushlightuserdata(L, &ngx_http_lua_raw_req_socket_metatable_key); diff --git a/t/062-count.t b/t/062-count.t index a69c33edb9..fcfbfff02d 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -494,3 +494,77 @@ GET /test n = 6 --- no_error_log [error] + + + +=== TEST 22: entries under the req raw sockets +--- config + location = /test { + content_by_lua_block { + local narr = 0 + local nrec = 0 + ngx.req.read_body() + local sock, err = ngx.req.socket(true) + if not sock then + ngx.log(ngx.ERR, "server: failed to get raw req socket: ", err) + return + end + sock:settimeouts(1, 2, 3) + for k, v in ipairs(sock) do + narr = narr + 1 + end + for k, v in pairs(sock) do + nrec = nrec + 1 + end + -- include '__index' + nrec = nrec - narr + 1 + + local ok, err = sock:send("HTTP/1.1 200 OK\r\n\r\nnarr = "..narr.."\nnrec = "..nrec.."\n") + if not ok then + ngx.log(ngx.ERR, "failed to send: ", err) + return + end + } + } +--- request +GET /test +--- response_body +narr = 2 +nrec = 3 +--- no_error_log +[error] + + + +=== TEST 23: entries under the req sockets +--- config + location = /test { + content_by_lua_block { + local narr = 0 + local nrec = 0 + local sock, err = ngx.req.socket() + if not sock then + ngx.log(ngx.ERR, "server: failed to get req socket: ", err) + return + end + sock:settimeouts(1, 2, 3) + for k, v in ipairs(sock) do + narr = narr + 1 + end + for k, v in pairs(sock) do + nrec = nrec + 1 + end + -- include '__index' + nrec = nrec - narr + 1 + + ngx.say("narr = "..narr.."\nnrec = "..nrec) + } + } +--- request +POST /test +hello world +--- response_body +narr = 2 +nrec = 3 +--- no_error_log +[error] From 3078ca6138121a98e22104b9f7015b31d312861b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 25 Jan 2018 16:23:19 -0800 Subject: [PATCH 214/848] bugfix: we now throw a Lua exception when ngx.location.capture* Lua API is used inside an HTTP2 request since it is known to lead to hanging. --- src/ngx_http_lua_headers.c | 2 +- src/ngx_http_lua_subrequest.c | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 2456fd15b5..b2b30016b0 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -113,7 +113,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) #if (NGX_HTTP_V2) if (mr->stream) { - return luaL_error(L, "http v2 not supported yet"); + return luaL_error(L, "http2 requests not supported yet"); } #endif diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 47096e9511..826a43c3a2 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -166,6 +166,12 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) return luaL_error(L, "no request object found"); } +#if (NGX_HTTP_V2) + if (r->main->stream) { + return luaL_error(L, "http2 requests not supported yet"); + } +#endif + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { return luaL_error(L, "no ctx found"); From a698cb0989b34c1e05cd6af5c0d6d9dbe56edd6b Mon Sep 17 00:00:00 2001 From: Marcus Clyne Date: Sat, 27 Jan 2018 23:13:43 +0000 Subject: [PATCH 215/848] doc: updated the NDK repo URL to github.com/simplresty. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 15 +++++++-------- doc/HttpLuaModule.wiki | 15 +++++++-------- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/README.markdown b/README.markdown index f4be57c867..35c04fed6f 100644 --- a/README.markdown +++ b/README.markdown @@ -270,7 +270,7 @@ It is *highly* recommended to use [OpenResty releases](http://openresty.org) whi Alternatively, ngx_lua can be manually compiled into Nginx: 1. Install LuaJIT 2.0 or 2.1 (recommended) or Lua 5.1 (Lua 5.2 is *not* supported yet). LuaJIT can be downloaded from the [LuaJIT project website](http://luajit.org/download.html) and Lua 5.1, from the [Lua project website](http://www.lua.org/). Some distribution package managers also distribute LuaJIT and/or Lua. -1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simpl/ngx_devel_kit/tags). +1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags). 1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags). 1. Download the latest version of Nginx [HERE](http://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) @@ -941,7 +941,7 @@ The following dependencies are required to run the test suite: * Test::Nginx: * Nginx modules: - * [ngx_devel_kit](https://github.com/simpl/ngx_devel_kit) + * [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) * [ngx_set_misc](https://github.com/openresty/set-misc-nginx-module) * [ngx_auth_request](http://mdounin.ru/files/ngx_http_auth_request_module-0.2.tar.gz) (this is not needed if you're using Nginx 1.5.4+. * [ngx_echo](https://github.com/openresty/echo-nginx-module) @@ -1026,7 +1026,7 @@ See Also * [Dynamic Routing Based on Redis and Lua](http://openresty.org/#DynamicRoutingBasedOnRedis) * [Using LuaRocks with ngx_lua](http://openresty.org/#UsingLuaRocks) * [Introduction to ngx_lua](https://github.com/openresty/lua-nginx-module/wiki/Introduction) -* [ngx_devel_kit](https://github.com/simpl/ngx_devel_kit) +* [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) * [echo-nginx-module](http://github.com/openresty/echo-nginx-module) * [drizzle-nginx-module](http://github.com/openresty/drizzle-nginx-module) * [postgres-nginx-module](https://github.com/FRiCKLE/ngx_postgres) @@ -1594,7 +1594,7 @@ This directive can be freely mixed with all directives of the [ngx_http_rewrite_ As from the `v0.5.0rc29` release, Nginx variable interpolation is disabled in the `` argument of this directive and therefore, the dollar sign character (`$`) can be used directly. -This directive requires the [ngx_devel_kit](https://github.com/simpl/ngx_devel_kit) module. +This directive requires the [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) module. [Back to TOC](#directives) @@ -1647,7 +1647,7 @@ and the Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching [lua_code_cache](#lua_code_cache) `off` in `nginx.conf` to avoid reloading Nginx. -This directive requires the [ngx_devel_kit](https://github.com/simpl/ngx_devel_kit) module. +This directive requires the [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) module. [Back to TOC](#directives) @@ -8160,7 +8160,7 @@ ndk.set_var.DIRECTIVE **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -This mechanism allows calling other nginx C modules' directives that are implemented by [Nginx Devel Kit](https://github.com/simpl/ngx_devel_kit) (NDK)'s set_var submodule's `ndk_set_var_value`. +This mechanism allows calling other nginx C modules' directives that are implemented by [Nginx Devel Kit](https://github.com/simplresty/ngx_devel_kit) (NDK)'s set_var submodule's `ndk_set_var_value`. For example, the following [set-misc-nginx-module](http://github.com/openresty/set-misc-nginx-module) directives can be invoked this way: @@ -8191,7 +8191,7 @@ Similarly, the following directives provided by [encrypted-session-nginx-module] * [set_encrypt_session](http://github.com/openresty/encrypted-session-nginx-module#set_encrypt_session) * [set_decrypt_session](http://github.com/openresty/encrypted-session-nginx-module#set_decrypt_session) -This feature requires the [ngx_devel_kit](https://github.com/simpl/ngx_devel_kit) module. +This feature requires the [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) module. [Back to TOC](#nginx-api-for-lua) @@ -8296,4 +8296,3 @@ Special PCRE Sequences ---------------------- This section has been renamed to [Special Escaping Sequences](#special-escaping-sequences). - diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4270be4cf3..f995e71437 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -204,7 +204,7 @@ It is *highly* recommended to use [http://openresty.org OpenResty releases] whic Alternatively, ngx_lua can be manually compiled into Nginx: # Install LuaJIT 2.0 or 2.1 (recommended) or Lua 5.1 (Lua 5.2 is ''not'' supported yet). LuaJIT can be downloaded from the [http://luajit.org/download.html LuaJIT project website] and Lua 5.1, from the [http://www.lua.org/ Lua project website]. Some distribution package managers also distribute LuaJIT and/or Lua. -# Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simpl/ngx_devel_kit/tags HERE]. +# Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE]. # Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE]. # Download the latest version of Nginx [http://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) @@ -766,7 +766,7 @@ The following dependencies are required to run the test suite: ** Test::Nginx: https://github.com/openresty/test-nginx * Nginx modules: -** [https://github.com/simpl/ngx_devel_kit ngx_devel_kit] +** [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] ** [https://github.com/openresty/set-misc-nginx-module ngx_set_misc] ** [http://mdounin.ru/files/ngx_http_auth_request_module-0.2.tar.gz ngx_auth_request] (this is not needed if you're using Nginx 1.5.4+. ** [https://github.com/openresty/echo-nginx-module ngx_echo] @@ -845,7 +845,7 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND * [http://openresty.org/#DynamicRoutingBasedOnRedis Dynamic Routing Based on Redis and Lua] * [http://openresty.org/#UsingLuaRocks Using LuaRocks with ngx_lua] * [https://github.com/openresty/lua-nginx-module/wiki/Introduction Introduction to ngx_lua] -* [https://github.com/simpl/ngx_devel_kit ngx_devel_kit] +* [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] * [[HttpEchoModule]] * [[HttpDrizzleModule]] * [https://github.com/FRiCKLE/ngx_postgres postgres-nginx-module] @@ -1291,7 +1291,7 @@ This directive can be freely mixed with all directives of the [[HttpRewriteModul As from the v0.5.0rc29 release, Nginx variable interpolation is disabled in the argument of this directive and therefore, the dollar sign character ($) can be used directly. -This directive requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_kit] module. +This directive requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] module. == set_by_lua_block == @@ -1337,7 +1337,7 @@ and the Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching [[#lua_code_cache|lua_code_cache]] off in nginx.conf to avoid reloading Nginx. -This directive requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_kit] module. +This directive requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] module. == content_by_lua == @@ -6926,7 +6926,7 @@ This feature requires at least ngx_lua v0.10.0. '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -This mechanism allows calling other nginx C modules' directives that are implemented by [https://github.com/simpl/ngx_devel_kit Nginx Devel Kit] (NDK)'s set_var submodule's ndk_set_var_value. +This mechanism allows calling other nginx C modules' directives that are implemented by [https://github.com/simplresty/ngx_devel_kit Nginx Devel Kit] (NDK)'s set_var submodule's ndk_set_var_value. For example, the following [[HttpSetMiscModule]] directives can be invoked this way: @@ -6956,7 +6956,7 @@ Similarly, the following directives provided by [[HttpEncryptedSessionModule]] c * [[HttpEncryptedSessionModule#set_encrypt_session|set_encrypt_session]] * [[HttpEncryptedSessionModule#set_decrypt_session|set_decrypt_session]] -This feature requires the [https://github.com/simpl/ngx_devel_kit ngx_devel_kit] module. +This feature requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] module. == coroutine.create == '''syntax:''' ''co = coroutine.create(f)'' @@ -7037,4 +7037,3 @@ This section is just holding obsolete documentation sections that have been eith == Special PCRE Sequences == This section has been renamed to [[#Special Escaping Sequences|Special Escaping Sequences]]. - From 3cde1d8aa1887e8b76356013a24a7602440d283e Mon Sep 17 00:00:00 2001 From: hongliang <513918845@qq.com> Date: Fri, 16 Feb 2018 14:41:54 +0800 Subject: [PATCH 216/848] doc: fixed some code comment typo. Signed-off-by: hongliang <513918845@qq.com> Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_rewriteby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 6a6b992b31..1bbe87c9b0 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -110,7 +110,7 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) if (r->header_sent) { dd("header already sent"); - /* response header was already generated in access_by_lua*, + /* response header was already generated in rewrite_by_lua*, * so it is no longer safe to proceed to later phases * which may generate responses again */ From feae6e775339c427d665cd60483b0d4526482015 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 25 Feb 2018 13:30:34 -0800 Subject: [PATCH 217/848] doc: mentioned that OpenResty includes its own version of LuaJIT which is specifically optmized and enhanced for OpenResty. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 35c04fed6f..19eaee696d 100644 --- a/README.markdown +++ b/README.markdown @@ -265,7 +265,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. +It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. Alternatively, ngx_lua can be manually compiled into Nginx: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f995e71437..7389613132 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -199,7 +199,7 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. +It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. Alternatively, ngx_lua can be manually compiled into Nginx: From bb76eb123b4a80324e527062d71fe9d3f4c86d25 Mon Sep 17 00:00:00 2001 From: Tom Thorogood Date: Wed, 21 Feb 2018 13:15:08 +1030 Subject: [PATCH 218/848] change: switched to SSL_version() calls from TLS1_get_version(). TLS1_get_version is a simple wrapper for SSL_version that returns 0 when used with DTLS. However, it was removed from BoringSSL in 2015 so instead use SSL_version directly. Note: BoringSSL is never an officially supported target for this module. ngx_http_lua_ffi_ssl_get_tls1_version can never be reached with DTLS so the behaviour is the same. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_ssl_certby.c | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 59ea8ae7ee..2250eea233 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -553,13 +553,6 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) int ngx_http_lua_ffi_ssl_get_tls1_version(ngx_http_request_t *r, char **err) { -#ifndef TLS1_get_version - - *err = "no TLS1 support"; - return NGX_ERROR; - -#else - ngx_ssl_conn_t *ssl_conn; if (r->connection == NULL || r->connection->ssl == NULL) { @@ -573,11 +566,9 @@ ngx_http_lua_ffi_ssl_get_tls1_version(ngx_http_request_t *r, char **err) return NGX_ERROR; } - dd("tls1 ver: %d", (int) TLS1_get_version(ssl_conn)); - - return (int) TLS1_get_version(ssl_conn); + dd("tls1 ver: %d", SSL_version(ssl_conn)); -#endif + return SSL_version(ssl_conn); } From 439a02ab999bfa369465cd1bb02d14c573a8e4c1 Mon Sep 17 00:00:00 2001 From: Tom Thorogood Date: Tue, 20 Feb 2018 14:07:41 +1030 Subject: [PATCH 219/848] typo: fixed an error message typo in set_der_priv_key(). The error string set in ngx_http_lua_ffi_ssl_set_der_private_key for SSL_use_PrivateKey failure has a typo. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_ssl_certby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 2250eea233..453a5c78e4 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -742,7 +742,7 @@ ngx_http_lua_ffi_ssl_set_der_private_key(ngx_http_request_t *r, } if (SSL_use_PrivateKey(ssl_conn, pkey) == 0) { - *err = "SSL_CTX_use_PrivateKey() failed"; + *err = "SSL_use_PrivateKey() failed"; goto failed; } From 80fa32c8d437eb055e8dbd56fc630d6cb832ade4 Mon Sep 17 00:00:00 2001 From: Mikhail Senin Date: Tue, 23 May 2017 14:19:10 +0300 Subject: [PATCH 220/848] feature: added support for the 308 status code in ngx.redirect(). Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ src/ngx_http_lua_common.h | 3 +++ src/ngx_http_lua_consts.c | 3 +++ src/ngx_http_lua_control.c | 2 ++ src/ngx_http_lua_util.c | 2 +- t/022-redirect.t | 53 +++++++++++++++++++++++++++++++++++++- t/062-count.t | 8 +++--- 8 files changed, 69 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index 19eaee696d..60ed1a54a7 100644 --- a/README.markdown +++ b/README.markdown @@ -3467,6 +3467,7 @@ HTTP status constants value = ngx.HTTP_SEE_OTHER (303) value = ngx.HTTP_NOT_MODIFIED (304) value = ngx.HTTP_TEMPORARY_REDIRECT (307) (first added in the v0.9.20 release) + value = ngx.HTTP_PERMANENT_REDIRECT (308) value = ngx.HTTP_BAD_REQUEST (400) value = ngx.HTTP_UNAUTHORIZED (401) value = ngx.HTTP_PAYMENT_REQUIRED (402) (first added in the v0.9.20 release) @@ -5101,6 +5102,7 @@ The optional `status` parameter specifies the HTTP status code to be used. The f * `302` (default) * `303` * `307` +* `308` It is `302` (`ngx.HTTP_MOVED_TEMPORARILY`) by default. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 7389613132..a89c3963dd 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2807,6 +2807,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture value = ngx.HTTP_SEE_OTHER (303) value = ngx.HTTP_NOT_MODIFIED (304) value = ngx.HTTP_TEMPORARY_REDIRECT (307) (first added in the v0.9.20 release) + value = ngx.HTTP_PERMANENT_REDIRECT (308) value = ngx.HTTP_BAD_REQUEST (400) value = ngx.HTTP_UNAUTHORIZED (401) value = ngx.HTTP_PAYMENT_REQUIRED (402) (first added in the v0.9.20 release) @@ -4253,6 +4254,7 @@ The optional status parameter specifies the HTTP status code to be * 302 (default) * 303 * 307 +* 308 It is 302 (ngx.HTTP_MOVED_TEMPORARILY) by default. diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 3bfaa09054..01ef2bec8b 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -54,6 +54,9 @@ # define NGX_HTTP_LUA_USE_OCSP 1 #endif +#ifndef NGX_HTTP_PERMANENT_REDIRECT +# define NGX_HTTP_PERMANENT_REDIRECT 308 +#endif #ifndef NGX_HAVE_SHA1 # if (nginx_version >= 1011002) diff --git a/src/ngx_http_lua_consts.c b/src/ngx_http_lua_consts.c index 30a86f106e..809e712e7a 100644 --- a/src/ngx_http_lua_consts.c +++ b/src/ngx_http_lua_consts.c @@ -124,6 +124,9 @@ ngx_http_lua_inject_http_consts(lua_State *L) lua_setfield(L, -2, "HTTP_SEE_OTHER"); #endif + lua_pushinteger(L, NGX_HTTP_PERMANENT_REDIRECT); + lua_setfield(L, -2, "HTTP_PERMANENT_REDIRECT"); + lua_pushinteger(L, NGX_HTTP_NOT_MODIFIED); lua_setfield(L, -2, "HTTP_NOT_MODIFIED"); diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index ae36505f9d..6ac2cbf88c 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -213,10 +213,12 @@ ngx_http_lua_ngx_redirect(lua_State *L) if (rc != NGX_HTTP_MOVED_TEMPORARILY && rc != NGX_HTTP_MOVED_PERMANENTLY && rc != NGX_HTTP_SEE_OTHER + && rc != NGX_HTTP_PERMANENT_REDIRECT && rc != NGX_HTTP_TEMPORARY_REDIRECT) { return luaL_error(L, "only ngx.HTTP_MOVED_TEMPORARILY, " "ngx.HTTP_MOVED_PERMANENTLY, " + "ngx.HTTP_PERMANENT_REDIRECT, " "ngx.HTTP_SEE_OTHER, and " "ngx.HTTP_TEMPORARY_REDIRECT are allowed"); } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c2d5c41147..f7a537ee96 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -710,7 +710,7 @@ static void ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log) { - lua_createtable(L, 0 /* narr */, 116 /* nrec */); /* ngx.* */ + lua_createtable(L, 0 /* narr */, 117 /* nrec */); /* ngx.* */ lua_pushcfunction(L, ngx_http_lua_get_raw_phase_context); lua_setfield(L, -2, "_phase_ctx"); diff --git a/t/022-redirect.t b/t/022-redirect.t index fae39e3adf..09374d3e3b 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -84,7 +84,7 @@ GET /read --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log -only ngx.HTTP_MOVED_TEMPORARILY, ngx.HTTP_MOVED_PERMANENTLY, ngx.HTTP_SEE_OTHER, and ngx.HTTP_TEMPORARY_REDIRECT are allowed +only ngx.HTTP_MOVED_TEMPORARILY, ngx.HTTP_MOVED_PERMANENTLY, ngx.HTTP_PERMANENT_REDIRECT, ngx.HTTP_SEE_OTHER, and ngx.HTTP_TEMPORARY_REDIRECT are allowed @@ -269,3 +269,54 @@ GET /read Location: http://agentzh.org/foo?a=b&c=d --- response_body_like: 303 See Other --- error_code: 303 + + + +=== TEST 15: explicit 308 with args +--- config + location /read { + content_by_lua ' + ngx.redirect("http://agentzh.org/foo?a=b&c=d", ngx.HTTP_PERMANENT_REDIRECT); + ngx.say("hi") + '; + } +--- request +GET /read +--- response_body_like: 308 Permanent Redirect +--- response_headers +Location: http://agentzh.org/foo?a=b&c=d +--- error_code: 308 + + + +=== TEST 16: explicit 308 +--- config + location /read { + content_by_lua ' + ngx.redirect("http://agentzh.org/foo?a=b&c=d", 308); + ngx.say("hi") + '; + } +--- request +GET /read +--- response_body_like: 308 Permanent Redirect +--- response_headers +Location: http://agentzh.org/foo?a=b&c=d +--- error_code: 308 + + + +=== TEST 17: explicit 308 with args +--- config + location /read { + content_by_lua ' + ngx.redirect("http://agentzh.org/foo?a=b&c=d", 308); + ngx.say("hi") + '; + } +--- request +GET /read +--- response_body_like: 308 Permanent Redirect +--- response_headers +Location: http://agentzh.org/foo?a=b&c=d +--- error_code: 308 diff --git a/t/062-count.t b/t/062-count.t index fcfbfff02d..b4d9d1c945 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -34,7 +34,7 @@ __DATA__ --- request GET /test --- response_body -ngx: 116 +ngx: 117 --- no_error_log [error] @@ -55,7 +55,7 @@ ngx: 116 --- request GET /test --- response_body -116 +117 --- no_error_log [error] @@ -83,7 +83,7 @@ GET /test --- request GET /test --- response_body -n = 116 +n = 117 --- no_error_log [error] @@ -305,7 +305,7 @@ GET /t --- response_body_like: 404 Not Found --- error_code: 404 --- error_log -ngx. entry count: 116 +ngx. entry count: 117 From 5cd103b1f6a71c90a72f76013d52dfb51a27ef74 Mon Sep 17 00:00:00 2001 From: spacewander Date: Sun, 4 Feb 2018 19:23:26 +0800 Subject: [PATCH 221/848] optimize: avoided the string copy in ngx_http_lua_ffi_req_get_method_name(). Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_req_method.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_req_method.c b/src/ngx_http_lua_req_method.c index 0a2a154ebe..a4969b99f6 100644 --- a/src/ngx_http_lua_req_method.c +++ b/src/ngx_http_lua_req_method.c @@ -158,15 +158,16 @@ ngx_http_lua_ffi_req_get_method(ngx_http_request_t *r) int -ngx_http_lua_ffi_req_get_method_name(ngx_http_request_t *r, char *buf, +ngx_http_lua_ffi_req_get_method_name(ngx_http_request_t *r, u_char **name, size_t *len) { if (r->connection->fd == (ngx_socket_t) -1) { return NGX_HTTP_LUA_FFI_BAD_CONTEXT; } - *len = ngx_min(r->method_name.len, *len); - ngx_memcpy(buf, r->method_name.data, *len); + *name = r->method_name.data; + *len = r->method_name.len; + return NGX_OK; } From 23e69fb200149b92f9baec7773fa2980df86d40c Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 28 Feb 2018 12:03:58 +0800 Subject: [PATCH 222/848] bugfix: ngx_http_lua_ffi_ssl_set_serialized_session(): avoided memory leak when calling it repeatly. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_ssl_session_fetchby.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index bd3129d2e5..0b88a87ee8 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -567,6 +567,7 @@ ngx_http_lua_ffi_ssl_set_serialized_session(ngx_http_request_t *r, ngx_ssl_conn_t *ssl_conn; ngx_connection_t *c; ngx_ssl_session_t *session = NULL; + ngx_ssl_session_t *old_session; ngx_http_lua_ssl_ctx_t *cctx; c = r->connection; @@ -597,8 +598,13 @@ ngx_http_lua_ffi_ssl_set_serialized_session(ngx_http_request_t *r, return NGX_ERROR; } + old_session = cctx->session; cctx->session = session; + if (old_session != NULL) { + ngx_ssl_free_session(old_session); + } + return NGX_OK; } From 809192c96cce68023ffd365363d064ee13f68600 Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 9 Mar 2018 16:00:41 +0800 Subject: [PATCH 223/848] feature: api.h: exposed the ngx_http_lua_ffi_str_t type for other Nginx C modules. Signed-off-by: Yichun Zhang (agentzh) --- src/api/ngx_http_lua_api.h | 7 +++++++ src/ngx_http_lua_util.h | 8 +------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index a725a51413..bf4245dada 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -34,6 +34,13 @@ typedef struct { } ngx_http_lua_value_t; +typedef struct { + int len; + /* this padding hole on 64-bit systems is expected */ + u_char *data; +} ngx_http_lua_ffi_str_t; + + lua_State *ngx_http_lua_get_global_state(ngx_conf_t *cf); ngx_http_request_t *ngx_http_lua_get_request(lua_State *L); diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index e9924e41eb..7dcc6f7585 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -10,6 +10,7 @@ #include "ngx_http_lua_common.h" +#include "ngx_http_lua_api.h" #ifndef NGX_UNESCAPE_URI_COMPONENT @@ -18,13 +19,6 @@ #ifndef NGX_LUA_NO_FFI_API -typedef struct { - int len; - /* this padding hole on 64-bit systems is expected */ - u_char *data; -} ngx_http_lua_ffi_str_t; - - typedef struct { ngx_http_lua_ffi_str_t key; ngx_http_lua_ffi_str_t value; From 6d4475ae3e088cc7bff76c6492b90b7bc39af173 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 21 Mar 2018 13:56:04 +0800 Subject: [PATCH 224/848] tests: increased timeouts to avoid failing in mockagain writing test mode. Signed-off-by: Yichun Zhang (agentzh) --- t/062-count.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/062-count.t b/t/062-count.t index b4d9d1c945..2ad2f9f0ab 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -509,7 +509,7 @@ n = 6 ngx.log(ngx.ERR, "server: failed to get raw req socket: ", err) return end - sock:settimeouts(1, 2, 3) + sock:settimeouts(1000, 2000, 3000) for k, v in ipairs(sock) do narr = narr + 1 end @@ -547,7 +547,7 @@ nrec = 3 ngx.log(ngx.ERR, "server: failed to get req socket: ", err) return end - sock:settimeouts(1, 2, 3) + sock:settimeouts(1000, 2000, 3000) for k, v in ipairs(sock) do narr = narr + 1 end From 53519cd7acc57793562bfb4527d347000a305756 Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 9 Mar 2018 15:59:12 +0800 Subject: [PATCH 225/848] feature: allowed sending boolean and nil values in cosockets. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 37 ++++++++++++++++++++ src/ngx_http_lua_socket_udp.c | 40 ++++++++++++++++++++++ t/058-tcp-socket.t | 64 ++++++++++++++++++++++++++++++++++- t/087-udp-socket.t | 52 +++++++++++++++++++++++++++- 4 files changed, 191 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 72ce97cdb4..efdf427540 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2462,6 +2462,20 @@ ngx_http_lua_socket_tcp_send(lua_State *L) len = ngx_http_lua_calc_strlen_in_table(L, 2, 2, 1 /* strict */); break; + case LUA_TNIL: + len = sizeof("nil") - 1; + break; + + case LUA_TBOOLEAN: + if (lua_toboolean(L, 2)) { + len = sizeof("true") - 1; + + } else { + len = sizeof("false") - 1; + } + + break; + default: msg = lua_pushfstring(L, "string, number, boolean, nil, " "or array table expected, got %s", @@ -2497,6 +2511,29 @@ ngx_http_lua_socket_tcp_send(lua_State *L) b->last = ngx_http_lua_copy_str_in_table(L, -1, b->last); break; + case LUA_TNIL: + *b->last++ = 'n'; + *b->last++ = 'i'; + *b->last++ = 'l'; + break; + + case LUA_TBOOLEAN: + if (lua_toboolean(L, 2)) { + *b->last++ = 't'; + *b->last++ = 'r'; + *b->last++ = 'u'; + *b->last++ = 'e'; + + } else { + *b->last++ = 'f'; + *b->last++ = 'a'; + *b->last++ = 'l'; + *b->last++ = 's'; + *b->last++ = 'e'; + } + + break; + default: return luaL_error(L, "impossible to reach here"); } diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 15cadf3100..8927f4118d 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -845,6 +845,20 @@ ngx_http_lua_socket_udp_send(lua_State *L) len = ngx_http_lua_calc_strlen_in_table(L, 2, 2, 1 /* strict */); break; + case LUA_TNIL: + len = sizeof("nil") - 1; + break; + + case LUA_TBOOLEAN: + if (lua_toboolean(L, 2)) { + len = sizeof("true") - 1; + + } else { + len = sizeof("false") - 1; + } + + break; + default: msg = lua_pushfstring(L, "string, number, boolean, nil, " "or array table expected, got %s", @@ -867,6 +881,32 @@ ngx_http_lua_socket_udp_send(lua_State *L) (void) ngx_http_lua_copy_str_in_table(L, 2, query.data); break; + case LUA_TNIL: + p = query.data; + *p++ = 'n'; + *p++ = 'i'; + *p++ = 'l'; + break; + + case LUA_TBOOLEAN: + p = query.data; + + if (lua_toboolean(L, 2)) { + *p++ = 't'; + *p++ = 'r'; + *p++ = 'u'; + *p++ = 'e'; + + } else { + *p++ = 'f'; + *p++ = 'a'; + *p++ = 'l'; + *p++ = 's'; + *p++ = 'e'; + } + + break; + default: return luaL_error(L, "impossible to reach here"); } diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 96ea9072cd..354a8768c5 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 196; +plan tests => repeat_each() * 199; our $HtmlDir = html_dir; @@ -3746,3 +3746,65 @@ GET /t failed to connect: bad port number: 65536 --- no_error_log [error] + + + +=== TEST 63: send boolean and nil +--- config + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local function send(data) + local bytes, err = sock:send(data) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + end + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\nTest: " + send(req) + send(true) + send(false) + send(nil) + send("\r\n\r\n") + + while true do + local line, err, part = sock:receive() + if line then + ngx.say("received: ", line) + else + break + end + end + + ok, err = sock:close() + } + } + + location /foo { + server_tokens off; + more_clear_headers Date; + echo $http_test; + } + +--- request +GET /t +--- response_body +received: HTTP/1.1 200 OK +received: Server: nginx +received: Content-Type: text/plain +received: Connection: close +received: +received: truefalsenil +--- no_error_log +[error] diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index ab41622eee..8bedc320a9 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (3 * blocks() + 13); +plan tests => repeat_each() * (3 * blocks() + 14); our $HtmlDir = html_dir; @@ -1124,3 +1124,53 @@ GET /t failed to connect: bad port number: 65536 --- no_error_log [error] + + + +=== TEST 21: send boolean and nil +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local socket = ngx.socket + local udp = socket.udp() + local port = ngx.var.port + udp:settimeout(1000) -- 1 sec + + local ok, err = udp:setpeername("127.0.0.1", ngx.var.port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local function send(data) + local bytes, err = udp:send(data) + if not bytes then + ngx.say("failed to send: ", err) + return + end + ngx.say("sent ok") + end + + send(true) + send(false) + send(nil) + } + } +--- request +GET /t +--- response_body +sent ok +sent ok +sent ok +--- no_error_log +[error] +--- grep_error_log eval +qr/send: fd:\d+ \d+ of \d+/ +--- grep_error_log_out eval +qr/send: fd:\d+ 4 of 4 +send: fd:\d+ 5 of 5 +send: fd:\d+ 3 of 3/ +--- log_level: debug From 9ea1c06a64b2b447c609c0b4473977685792f53e Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 26 Mar 2018 20:04:43 -0700 Subject: [PATCH 226/848] feature: ngx_http_lua_ffi_set_resp_header(): now add an override flag argument to control whether to override existing resp headers. this feature is required by the new ngx.resp module's add_header() Lua API (in lua-resty-core). thanks spacewander for the patch. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_headers.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index b2b30016b0..f7f53443c1 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -1166,7 +1166,8 @@ ngx_http_lua_ffi_req_get_headers(ngx_http_request_t *r, int ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, size_t key_len, int is_nil, const u_char *sval, size_t sval_len, - ngx_http_lua_ffi_str_t *mvals, size_t mvals_len, char **errmsg) + ngx_http_lua_ffi_str_t *mvals, size_t mvals_len, int override, + char **errmsg) { u_char *p; ngx_str_t value, key; @@ -1249,7 +1250,7 @@ ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, value.len = len; rc = ngx_http_lua_set_output_header(r, key, value, - i == 0 /* override */); + override && i == 0); if (rc == NGX_ERROR) { *errmsg = "failed to set header"; @@ -1274,7 +1275,7 @@ ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, dd("key: %.*s, value: %.*s", (int) key.len, key.data, (int) value.len, value.data); - rc = ngx_http_lua_set_output_header(r, key, value, 1 /* override */); + rc = ngx_http_lua_set_output_header(r, key, value, override); if (rc == NGX_ERROR) { *errmsg = "failed to set header"; From 5196e65519b1c353f81f116e197ea94bd71c3ca1 Mon Sep 17 00:00:00 2001 From: spacewander Date: Tue, 27 Mar 2018 13:13:14 +0800 Subject: [PATCH 227/848] tests: remove google.com relevant ssl certificate test cases. We could not control Google's certificate, and Google don't promise their root certificate won't change, see https://pki.google.com/faq.html. > Q: What roots should we trust for connecting to Google? > A: Google may decide to have its intermediate signed by another root at any point in time, ... Because in TEST 8 we already tested passing verify case with openresty.org, here I remove TEST 11 and change TEST 12 to test with openresty.org. Signed-off-by: Yichun Zhang (agentzh) --- t/129-ssl-socket.t | 171 +++++++++------------------------------------ 1 file changed, 33 insertions(+), 138 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 7f36947d38..3f056404d6 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 218; +plan tests => repeat_each() * 211; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -867,32 +867,21 @@ SSL reused session -=== TEST 11: www.google.com (SSL verify passes) +=== TEST 11: openresty.org: SSL verify enabled and no corresponding trusted certificates --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate ../html/trusted.crt; - lua_ssl_verify_depth 3; + lua_ssl_verify_depth 2; location /t { - #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; - content_by_lua ' - -- avoid flushing google in "check leak" testing mode: - local counter = package.loaded.counter - if not counter then - counter = 1 - elseif counter >= 2 then - return ngx.exit(503) - else - counter = counter + 1 - end - package.loaded.counter = counter + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(4000) do - local sock = ngx.socket.tcp() - sock:settimeout(2000) - local ok, err = sock:connect("www.google.com", 443) + local ok, err = sock:connect("openresty.org", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -900,21 +889,20 @@ SSL reused session ngx.say("connected: ", ok) - local sess, err = sock:sslhandshake(nil, "www.google.com", true) - if not sess then + local session, err = sock:sslhandshake(nil, "openresty.org", true) + if not session then ngx.say("failed to do SSL handshake: ", err) return end - ngx.say("ssl handshake: ", type(sess)) + ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: www.google.com\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\r\nHost: openresty.org\r\nConnection: close\r\n\r\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) return end - ngx.say("sent http request: ", bytes, " bytes.") local line, err = sock:receive() @@ -929,106 +917,13 @@ SSL reused session ngx.say("close: ", ok, " ", err) end -- do collectgarbage() - '; + } } --- user_files eval ">>> trusted.crt $::EquifaxRootCertificate" ---- request -GET /t ---- response_body_like chop -\Aconnected: 1 -ssl handshake: userdata -sent http request: 59 bytes. -received: HTTP/1.1 (?:200 OK|302 Found) -close: 1 nil -\z ---- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ ---- grep_error_log_out eval -qr/^lua ssl save session: ([0-9A-F]+) -lua ssl free session: ([0-9A-F]+) -$/ ---- error_log -lua ssl server name: "www.google.com" ---- no_error_log -SSL reused session -[error] -[alert] ---- timeout: 5 - - - -=== TEST 12: www.google.com (SSL verify enabled and no corresponding trusted certificates) ---- config - server_tokens off; - resolver $TEST_NGINX_RESOLVER ipv6=off; - lua_ssl_trusted_certificate ../html/trusted.crt; - lua_ssl_verify_depth 3; - location /t { - #set $port 5000; - set $port $TEST_NGINX_MEMCACHED_PORT; - - content_by_lua ' - -- avoid flushing google in "check leak" testing mode: - local counter = package.loaded.counter - if not counter then - counter = 1 - elseif counter >= 2 then - return ngx.exit(503) - else - counter = counter + 1 - end - package.loaded.counter = counter - - do - local sock = ngx.socket.tcp() - sock:settimeout(2000) - local ok, err = sock:connect("www.google.com", 443) - if not ok then - ngx.say("failed to connect: ", err) - return - end - - ngx.say("connected: ", ok) - - local sess, err = sock:sslhandshake(nil, "www.google.com", true) - if not sess then - ngx.say("failed to do SSL handshake: ", err) - return - end - - ngx.say("ssl handshake: ", type(sess)) - - local req = "GET / HTTP/1.1\\r\\nHost: www.google.com\\r\\nConnection: close\\r\\n\\r\\n" - local bytes, err = sock:send(req) - if not bytes then - ngx.say("failed to send http request: ", err) - return - end - - ngx.say("sent http request: ", bytes, " bytes.") - - local line, err = sock:receive() - if not line then - ngx.say("failed to receive response status line: ", err) - return - end - - ngx.say("received: ", line) - - local ok, err = sock:close() - ngx.say("close: ", ok, " ", err) - end -- do - collectgarbage() - '; - } - ---- user_files eval -">>> trusted.crt -$::DSTRootCertificate" - --- request GET /t --- response_body @@ -1038,7 +933,7 @@ failed to do SSL handshake: 20: unable to get local issuer certificate --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log -lua ssl server name: "www.google.com" +lua ssl server name: "openresty.org" lua ssl certificate verify error: (20: unable to get local issuer certificate) --- no_error_log SSL reused session @@ -1047,7 +942,7 @@ SSL reused session -=== TEST 13: openresty.org: passing SSL verify with multiple certificates +=== TEST 12: openresty.org: passing SSL verify with multiple certificates --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1133,7 +1028,7 @@ SSL reused session -=== TEST 14: default cipher +=== TEST 13: default cipher --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1212,7 +1107,7 @@ SSL reused session -=== TEST 15: explicit cipher configuration +=== TEST 14: explicit cipher configuration --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1292,7 +1187,7 @@ SSL reused session -=== TEST 16: explicit ssl protocol configuration +=== TEST 15: explicit ssl protocol configuration --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1372,7 +1267,7 @@ SSL reused session -=== TEST 17: unsupported ssl protocol +=== TEST 16: unsupported ssl protocol --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1449,7 +1344,7 @@ SSL reused session -=== TEST 18: openresty.org: passing SSL verify: keepalive (reuse the ssl session) +=== TEST 17: openresty.org: passing SSL verify: keepalive (reuse the ssl session) --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1526,7 +1421,7 @@ SSL reused session -=== TEST 19: openresty.org: passing SSL verify: keepalive (no reusing the ssl session) +=== TEST 18: openresty.org: passing SSL verify: keepalive (no reusing the ssl session) --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1606,7 +1501,7 @@ SSL reused session -=== TEST 20: downstream cosockets do not support ssl handshake +=== TEST 19: downstream cosockets do not support ssl handshake --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1647,7 +1542,7 @@ attempt to call method 'sslhandshake' (a nil value) -=== TEST 21: unix domain ssl cosocket (no verify) +=== TEST 20: unix domain ssl cosocket (no verify) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -1750,7 +1645,7 @@ SSL reused session -=== TEST 22: unix domain ssl cosocket (verify) +=== TEST 21: unix domain ssl cosocket (verify) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -1856,7 +1751,7 @@ SSL reused session -=== TEST 23: unix domain ssl cosocket (no ssl on server) +=== TEST 22: unix domain ssl cosocket (no ssl on server) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; @@ -1946,7 +1841,7 @@ SSL reused session -=== TEST 24: lua_ssl_crl +=== TEST 23: lua_ssl_crl --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2045,7 +1940,7 @@ SSL reused session -=== TEST 25: multiple handshake calls +=== TEST 24: multiple handshake calls --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2129,7 +2024,7 @@ SSL reused session -=== TEST 26: handshake timed out +=== TEST 25: handshake timed out --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2183,7 +2078,7 @@ SSL reused session -=== TEST 27: unix domain ssl cosocket (no gen session) +=== TEST 26: unix domain ssl cosocket (no gen session) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2254,7 +2149,7 @@ SSL reused session -=== TEST 28: unix domain ssl cosocket (gen session, true) +=== TEST 27: unix domain ssl cosocket (gen session, true) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2328,7 +2223,7 @@ SSL reused session -=== TEST 29: unix domain ssl cosocket (keepalive) +=== TEST 28: unix domain ssl cosocket (keepalive) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2405,7 +2300,7 @@ SSL reused session -=== TEST 30: unix domain ssl cosocket (verify cert but no host name check, passed) +=== TEST 29: unix domain ssl cosocket (verify cert but no host name check, passed) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2510,7 +2405,7 @@ SSL reused session -=== TEST 31: unix domain ssl cosocket (verify cert but no host name check, NOT passed) +=== TEST 30: unix domain ssl cosocket (verify cert but no host name check, NOT passed) --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2603,7 +2498,7 @@ SSL reused session -=== TEST 32: handshake, too many arguments +=== TEST 31: handshake, too many arguments --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; From 91a0ad236c9661f38b78cdc99e05025f7ce5cccb Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 10 Jan 2018 19:45:07 -0800 Subject: [PATCH 228/848] optimize: switched exptime argument type to 'long' in the shdict FFI API to mitigate potential overflows. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_shdict.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index b300673e82..5fb7930a87 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2212,7 +2212,7 @@ ngx_http_lua_find_zone(u_char *name_data, size_t name_len) int ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, size_t key_len, int value_type, u_char *str_value_buf, - size_t str_value_len, double num_value, int exptime, int user_flags, + size_t str_value_len, double num_value, long exptime, int user_flags, char **errmsg, int *forcible) { int i, n; @@ -2229,7 +2229,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, return NGX_ERROR; } - dd("exptime: %d", exptime); + dd("exptime: %ld", exptime); ctx = zone->data; @@ -2903,7 +2903,7 @@ ngx_http_lua_shdict_peek(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, } -int +long ngx_http_lua_ffi_shdict_get_ttl(ngx_shm_zone_t *zone, u_char *key, size_t key_len) { @@ -2951,7 +2951,7 @@ ngx_http_lua_ffi_shdict_get_ttl(ngx_shm_zone_t *zone, u_char *key, int ngx_http_lua_ffi_shdict_set_expire(ngx_shm_zone_t *zone, u_char *key, - size_t key_len, int exptime) + size_t key_len, long exptime) { uint32_t hash; ngx_int_t rc; From 288ebdb93f149129f589e7cafdc0b5326f4a1b1b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 19 Apr 2018 12:33:07 -0700 Subject: [PATCH 229/848] doc: bumped version to 0.10.12. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 60ed1a54a7..df3dc08974 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.11](https://github.com/openresty/lua-nginx-module/tags) released on 3 November 2017. +This document describes ngx_lua [v0.10.12](https://github.com/openresty/lua-nginx-module/tags) released on 19 April 2018. Synopsis ======== From 4ff3dd14ed31a030668f2c8df5d5525fab6ccfb1 Mon Sep 17 00:00:00 2001 From: chronolaw Date: Thu, 19 Apr 2018 17:52:17 +0800 Subject: [PATCH 230/848] doc: fix the link to ngx.ocsp's docs. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index df3dc08974..bc6e3f5951 100644 --- a/README.markdown +++ b/README.markdown @@ -8149,7 +8149,7 @@ This Lua module does not ship with this ngx_lua module itself rather it is shipp the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. -Please refer to the [documentation](https://github.com/openresty/lua-resty-core/blob/ocsp-cert-by-lua-2/lib/ngx/ocsp.md) +Please refer to the [documentation](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ocsp.md) for this `ngx.ocsp` Lua module for more details. This feature requires at least ngx_lua `v0.10.0`. From 19cc6bf8f5ac93c8a1215312a4fbbc0362af8950 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 20 Apr 2018 16:47:49 -0700 Subject: [PATCH 231/848] tests: fixed a test case that cannot be run in parallel test jobs. --- t/138-balancer.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/138-balancer.t b/t/138-balancer.t index ddd0da577a..3745577751 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -307,7 +307,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ === TEST 12: code cache off --- http_config - lua_package_path "t/servroot/html/?.lua;;"; + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; lua_code_cache off; @@ -327,7 +327,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ location = /update { content_by_lua_block { -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("print('me: ', 101)") f:close() ngx.say("updated") From 52af63a5b949d6da2289e2de3fb839e2aba4cbfd Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 20 Apr 2018 14:07:13 -0700 Subject: [PATCH 232/848] feature: ngx.req.get_post_args(), ngx.req.get_uri_args(), ngx.req.get_headers(), ngx.resp.get_headers(), and ngx.decode_args() now would return an error string, "truncated", when the input exceeds the max_args/max_headers limits. bugfix: ngx.resp.get_headers(): the max_headers limit did not cover builtin headers. --- README.markdown | 70 +++-- doc/HttpLuaModule.wiki | 70 +++-- src/ngx_http_lua_args.c | 11 +- src/ngx_http_lua_headers.c | 65 +++-- t/016-resp-header.t | 222 ++++++++++++++-- t/020-subrequest.t | 6 +- t/028-req-header.t | 512 +++++++++++++++++++++++++++++++------ t/030-uri-args.t | 201 ++++++++++++--- t/031-post-args.t | 82 ++++-- t/041-header-filter.t | 6 +- t/044-req-body.t | 8 +- t/138-balancer.t | 2 +- 12 files changed, 1055 insertions(+), 200 deletions(-) diff --git a/README.markdown b/README.markdown index bc6e3f5951..f805505689 100644 --- a/README.markdown +++ b/README.markdown @@ -4173,7 +4173,7 @@ For reading *request* headers, use the [ngx.req.get_headers](#ngxreqget_headers) ngx.resp.get_headers -------------------- -**syntax:** *headers = ngx.resp.get_headers(max_headers?, raw?)* +**syntax:** *headers, err = ngx.resp.get_headers(max_headers?, raw?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** @@ -4181,7 +4181,12 @@ Returns a Lua table holding all the current response headers for the current req ```lua - local h = ngx.resp.get_headers() + local h, err = ngx.resp.get_headers() + + if err == "truncated" then + -- one can choose to ignore or reject the current response here + end + for k, v in pairs(h) do ... end @@ -4189,6 +4194,8 @@ Returns a Lua table holding all the current response headers for the current req This function has the same signature as [ngx.req.get_headers](#ngxreqget_headers) except getting response headers instead of request headers. +Note that a maximum of 100 response headers are parsed by default (including those with the same name) and that additional response headers are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. + This API was first introduced in the `v0.9.5` release. [Back to TOC](#nginx-api-for-lua) @@ -4461,7 +4468,7 @@ See also [ngx.req.set_uri](#ngxreqset_uri). ngx.req.get_uri_args -------------------- -**syntax:** *args = ngx.req.get_uri_args(max_args?)* +**syntax:** *args, err = ngx.req.get_uri_args(max_args?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** @@ -4471,7 +4478,12 @@ Returns a Lua table holding all the current request URL query arguments. location = /test { content_by_lua_block { - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end + for key, val in pairs(args) do if type(val) == "table" then ngx.say(key, ": ", table.concat(val, ", ")) @@ -4523,7 +4535,7 @@ Updating query arguments via the nginx variable `$args` (or `ngx.var.args` in Lu ```lua ngx.var.args = "a=3&b=42" - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() ``` Here the `args` table will always look like @@ -4535,20 +4547,23 @@ Here the `args` table will always look like regardless of the actual request query string. -Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. However, the optional `max_args` function argument can be used to override this limit: ```lua - local args = ngx.req.get_uri_args(10) + local args, err = ngx.req.get_uri_args(10) + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end ``` This argument can be set to zero to remove the limit and to process all request arguments received: ```lua - local args = ngx.req.get_uri_args(0) + local args, err = ngx.req.get_uri_args(0) ``` Removing the `max_args` cap is strongly discouraged. @@ -4569,6 +4584,11 @@ Returns a Lua table holding all the current request POST query arguments (of the content_by_lua_block { ngx.req.read_body() local args, err = ngx.req.get_post_args() + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end + if not args then ngx.say("failed to get post args: ", err) return @@ -4637,20 +4657,23 @@ That is, they will take Lua boolean values `true`. However, they are different f Empty key arguments are discarded. `POST /test` with body `=hello&=world` will yield empty outputs for instance. -Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. However, the optional `max_args` function argument can be used to override this limit: ```lua - local args = ngx.req.get_post_args(10) + local args, err = ngx.req.get_post_args(10) + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end ``` This argument can be set to zero to remove the limit and to process all request arguments received: ```lua - local args = ngx.req.get_post_args(0) + local args, err = ngx.req.get_post_args(0) ``` Removing the `max_args` cap is strongly discouraged. @@ -4659,7 +4682,7 @@ Removing the `max_args` cap is strongly discouraged. ngx.req.get_headers ------------------- -**syntax:** *headers = ngx.req.get_headers(max_headers?, raw?)* +**syntax:** *headers, err = ngx.req.get_headers(max_headers?, raw?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** @@ -4667,7 +4690,12 @@ Returns a Lua table holding all the current request headers. ```lua - local h = ngx.req.get_headers() + local h, err = ngx.req.get_headers() + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end + for k, v in pairs(h) do ... end @@ -4698,20 +4726,24 @@ the value of `ngx.req.get_headers()["Foo"]` will be a Lua (array) table such as: {"foo", "bar", "baz"} ``` -Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. However, the optional `max_headers` function argument can be used to override this limit: ```lua - local headers = ngx.req.get_headers(10) + local headers, err = ngx.req.get_headers(10) + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end ``` This argument can be set to zero to remove the limit and to process all request headers received: ```lua - local headers = ngx.req.get_headers(0) + local headers, err = ngx.req.get_headers(0) ``` Removing the `max_headers` cap is strongly discouraged. @@ -5489,13 +5521,13 @@ This method was first introduced in the `v0.3.1rc27` release. ngx.decode_args --------------- -**syntax:** *table = ngx.decode_args(str, max_args?)* +**syntax:** *table, err = ngx.decode_args(str, max_args?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Decodes a URI encoded query-string into a Lua table. This is the inverse function of [ngx.encode_args](#ngxencode_args). -The optional `max_args` argument can be used to specify the maximum number of arguments parsed from the `str` argument. By default, a maximum of 100 request arguments are parsed (including those with the same name) and that additional URI arguments are silently discarded to guard against potential denial of service attacks. +The optional `max_args` argument can be used to specify the maximum number of arguments parsed from the `str` argument. By default, a maximum of 100 request arguments are parsed (including those with the same name) and that additional URI arguments are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. This argument can be set to zero to remove the limit and to process all request arguments received: @@ -8149,7 +8181,7 @@ This Lua module does not ship with this ngx_lua module itself rather it is shipp the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. -Please refer to the [documentation](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ocsp.md) +Please refer to the [documentation](https://github.com/openresty/lua-resty-core/blob/ocsp-cert-by-lua-2/lib/ngx/ocsp.md) for this `ngx.ocsp` Lua module for more details. This feature requires at least ngx_lua `v0.10.0`. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a89c3963dd..5fcdbc5c58 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.11] released on 3 November 2017. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.12] released on 19 April 2018. = Synopsis = @@ -3453,14 +3453,19 @@ Note that ngx.header is not a normal Lua table and as such, it is n For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_headers]] function instead. == ngx.resp.get_headers == -'''syntax:''' ''headers = ngx.resp.get_headers(max_headers?, raw?)'' +'''syntax:''' ''headers, err = ngx.resp.get_headers(max_headers?, raw?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' Returns a Lua table holding all the current response headers for the current request. -local h = ngx.resp.get_headers() +local h, err = ngx.resp.get_headers() + +if err == "truncated" then + -- one can choose to ignore or reject the current response here +end + for k, v in pairs(h) do ... end @@ -3468,6 +3473,8 @@ end This function has the same signature as [[#ngx.req.get_headers|ngx.req.get_headers]] except getting response headers instead of request headers. +Note that a maximum of 100 response headers are parsed by default (including those with the same name) and that additional response headers are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. + This API was first introduced in the v0.9.5 release. == ngx.req.is_internal == @@ -3697,7 +3704,7 @@ This interface was first introduced in the v0.3.1rc13 release. See also [[#ngx.req.set_uri|ngx.req.set_uri]]. == ngx.req.get_uri_args == -'''syntax:''' ''args = ngx.req.get_uri_args(max_args?)'' +'''syntax:''' ''args, err = ngx.req.get_uri_args(max_args?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' @@ -3706,7 +3713,12 @@ Returns a Lua table holding all the current request URL query arguments. location = /test { content_by_lua_block { - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end + for key, val in pairs(args) do if type(val) == "table" then ngx.say(key, ": ", table.concat(val, ", ")) @@ -3753,7 +3765,7 @@ Updating query arguments via the nginx variable $args (or ngx ngx.var.args = "a=3&b=42" - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() Here the args table will always look like @@ -3764,18 +3776,21 @@ Here the args table will always look like regardless of the actual request query string. -Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. However, the optional max_args function argument can be used to override this limit: - local args = ngx.req.get_uri_args(10) + local args, err = ngx.req.get_uri_args(10) + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end This argument can be set to zero to remove the limit and to process all request arguments received: - local args = ngx.req.get_uri_args(0) + local args, err = ngx.req.get_uri_args(0) Removing the max_args cap is strongly discouraged. @@ -3792,6 +3807,11 @@ Returns a Lua table holding all the current request POST query arguments (of the content_by_lua_block { ngx.req.read_body() local args, err = ngx.req.get_post_args() + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end + if not args then ngx.say("failed to get post args: ", err) return @@ -3854,31 +3874,39 @@ That is, they will take Lua boolean values true. However, they are Empty key arguments are discarded. POST /test with body =hello&=world will yield empty outputs for instance. -Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request arguments are parsed by default (including those with the same name) and that additional request arguments are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. However, the optional max_args function argument can be used to override this limit: - local args = ngx.req.get_post_args(10) + local args, err = ngx.req.get_post_args(10) + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end This argument can be set to zero to remove the limit and to process all request arguments received: - local args = ngx.req.get_post_args(0) + local args, err = ngx.req.get_post_args(0) Removing the max_args cap is strongly discouraged. == ngx.req.get_headers == -'''syntax:''' ''headers = ngx.req.get_headers(max_headers?, raw?)'' +'''syntax:''' ''headers, err = ngx.req.get_headers(max_headers?, raw?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Returns a Lua table holding all the current request headers. - local h = ngx.req.get_headers() + local h, err = ngx.req.get_headers() + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end + for k, v in pairs(h) do ... end @@ -3906,18 +3934,22 @@ the value of ngx.req.get_headers()["Foo"] will be a Lua (array) tab {"foo", "bar", "baz"} -Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. +Note that a maximum of 100 request headers are parsed by default (including those with the same name) and that additional request headers are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. However, the optional max_headers function argument can be used to override this limit: - local headers = ngx.req.get_headers(10) + local headers, err = ngx.req.get_headers(10) + + if err == "truncated" then + -- one can choose to ignore or reject the current request here + end This argument can be set to zero to remove the limit and to process all request headers received: - local headers = ngx.req.get_headers(0) + local headers, err = ngx.req.get_headers(0) Removing the max_headers cap is strongly discouraged. @@ -4582,13 +4614,13 @@ If the argument value is false, then the effect is equivalent to th This method was first introduced in the v0.3.1rc27 release. == ngx.decode_args == -'''syntax:''' ''table = ngx.decode_args(str, max_args?)'' +'''syntax:''' ''table, err = ngx.decode_args(str, max_args?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Decodes a URI encoded query-string into a Lua table. This is the inverse function of [[#ngx.encode_args|ngx.encode_args]]. -The optional max_args argument can be used to specify the maximum number of arguments parsed from the str argument. By default, a maximum of 100 request arguments are parsed (including those with the same name) and that additional URI arguments are silently discarded to guard against potential denial of service attacks. +The optional max_args argument can be used to specify the maximum number of arguments parsed from the str argument. By default, a maximum of 100 request arguments are parsed (including those with the same name) and that additional URI arguments are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. This argument can be set to zero to remove the limit and to process all request arguments received: diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index b43697c7a1..2712bcd7c2 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -311,10 +311,11 @@ ngx_http_lua_parse_args(lua_State *L, u_char *buf, u_char *last, int max) } if (max > 0 && ++count == max) { + lua_pushliteral(L, "truncated"); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua hit query args limit %d", max); - - return 1; + return 2; } } else { @@ -387,7 +388,8 @@ ngx_http_lua_ffi_req_get_querystring_len(ngx_http_request_t *r) int -ngx_http_lua_ffi_req_get_uri_args_count(ngx_http_request_t *r, int max) +ngx_http_lua_ffi_req_get_uri_args_count(ngx_http_request_t *r, int max, + int *truncated) { int count; u_char *p, *last; @@ -396,6 +398,8 @@ ngx_http_lua_ffi_req_get_uri_args_count(ngx_http_request_t *r, int max) return NGX_HTTP_LUA_FFI_BAD_CONTEXT; } + *truncated = 0; + if (max < 0) { max = NGX_HTTP_LUA_MAX_ARGS; } @@ -417,6 +421,7 @@ ngx_http_lua_ffi_req_get_uri_args_count(ngx_http_request_t *r, int max) if (count) { if (max > 0 && count > max) { count = max; + *truncated = 1; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua hit query args limit %d", max); } diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index f7f53443c1..b833577d14 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -397,6 +397,7 @@ ngx_http_lua_ngx_req_get_headers(lua_State *L) int max; int raw = 0; int count = 0; + int truncated = 0; n = lua_gettop(L); @@ -425,15 +426,17 @@ ngx_http_lua_ngx_req_get_headers(lua_State *L) part = &r->headers_in.headers.part; count = part->nelts; - while (part->next) { + while (part->next != NULL) { part = part->next; count += part->nelts; } if (max > 0 && count > max) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua exceeding request header limit %d > %d", count, + max); count = max; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua exceeding request header limit %d", max); + truncated = 1; } lua_createtable(L, 0, count); @@ -480,9 +483,14 @@ ngx_http_lua_ngx_req_get_headers(lua_State *L) "lua request header: \"%V: %V\"", &header[i].key, &header[i].value); - if (--count == 0) { - return 1; + if (--count <= 0) { + break; } + } /* for */ + + if (truncated) { + lua_pushliteral(L, "truncated"); + return 2; } return 1; @@ -504,6 +512,8 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) int max; int raw = 0; int count = 0; + int truncated = 0; + int extra = 0; n = lua_gettop(L); @@ -548,18 +558,12 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) part = &r->headers_out.headers.part; count = part->nelts; - while (part->next) { + while (part->next != NULL) { part = part->next; count += part->nelts; } - if (max > 0 && count > max) { - count = max; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua exceeding request header limit %d", max); - } - - lua_createtable(L, 0, count); + lua_createtable(L, 0, count + 2); if (!raw) { lua_pushlightuserdata(L, &ngx_http_lua_headers_metatable_key); @@ -569,6 +573,7 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) #if 1 if (r->headers_out.content_type.len) { + extra++; lua_pushliteral(L, "content-type"); lua_pushlstring(L, (char *) r->headers_out.content_type.data, r->headers_out.content_type.len); @@ -578,11 +583,13 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) if (r->headers_out.content_length == NULL && r->headers_out.content_length_n >= 0) { + extra++; lua_pushliteral(L, "content-length"); lua_pushfstring(L, "%d", (int) r->headers_out.content_length_n); lua_rawset(L, -3); } + extra++; lua_pushliteral(L, "connection"); if (r->headers_out.status == NGX_HTTP_SWITCHING_PROTOCOLS) { lua_pushliteral(L, "upgrade"); @@ -596,12 +603,21 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) lua_rawset(L, -3); if (r->chunked) { + extra++; lua_pushliteral(L, "transfer-encoding"); lua_pushliteral(L, "chunked"); lua_rawset(L, -3); } #endif + if (max > 0 && count + extra > max) { + truncated = 1; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua exceeding response header limit %d > %d", + count + extra, max); + count = max - extra; + } + part = &r->headers_out.headers.part; header = part->elts; @@ -654,9 +670,14 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) "lua response header: \"%V: %V\"", &header[i].key, &header[i].value); - if (--count == 0) { - return 1; + if (--count <= 0) { + break; } + } /* for */ + + if (truncated) { + lua_pushliteral(L, "truncated"); + return 2; } return 1; @@ -1083,7 +1104,8 @@ ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L) #ifndef NGX_LUA_NO_FFI_API int -ngx_http_lua_ffi_req_get_headers_count(ngx_http_request_t *r, int max) +ngx_http_lua_ffi_req_get_headers_count(ngx_http_request_t *r, int max, + int *truncated) { int count; ngx_list_part_t *part; @@ -1092,21 +1114,26 @@ ngx_http_lua_ffi_req_get_headers_count(ngx_http_request_t *r, int max) return NGX_HTTP_LUA_FFI_BAD_CONTEXT; } + *truncated = 0; + if (max < 0) { max = NGX_HTTP_LUA_MAX_HEADERS; } part = &r->headers_in.headers.part; count = part->nelts; - while (part->next) { + while (part->next != NULL) { part = part->next; count += part->nelts; } if (max > 0 && count > max) { + *truncated = 1; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua exceeding request header limit %d > %d", count, + max); count = max; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua exceeding request header limit %d", max); } return count; diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 1fae2922a5..72c1e05bcc 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 41); +plan tests => repeat_each() * (blocks() * 3 + 46); #no_diff(); no_long_string(); @@ -1151,10 +1151,23 @@ foo: 32 content_by_lua ' ngx.header["Foo"] = "bar" ngx.header["Bar"] = "baz" - ngx.say("Foo: ", ngx.resp.get_headers()["Foo"] or "nil") - ngx.say("foo: ", ngx.resp.get_headers()["foo"] or "nil") - ngx.say("Bar: ", ngx.resp.get_headers()["Bar"] or "nil") - ngx.say("bar: ", ngx.resp.get_headers()["bar"] or "nil") + local headers, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo: ", headers["Foo"] or "nil") + ngx.say("foo: ", headers["foo"] or "nil") + ngx.say("Bar: ", headers["Bar"] or "nil") + + headers, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("bar: ", headers["bar"] or "nil") '; } --- request @@ -1167,6 +1180,8 @@ Foo: bar foo: bar Bar: baz bar: baz +--- no_error_log +[error] @@ -1176,8 +1191,15 @@ bar: baz content_by_lua ' ngx.header["Foo"] = "bar" ngx.header["Bar"] = "baz" + + local headers, err = ngx.resp.get_headers(nil, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + local h = {} - for k, v in pairs(ngx.resp.get_headers(nil, true)) do + for k, v in pairs(headers) do h[k] = v end ngx.say("Foo: ", h["Foo"] or "nil") @@ -1206,10 +1228,17 @@ bar: nil ngx.header["Foo"] = "bar" ngx.header["Foo"] = nil ngx.header["Bar"] = "baz" - ngx.say("Foo: ", ngx.resp.get_headers()["Foo"] or "nil") - ngx.say("foo: ", ngx.resp.get_headers()["foo"] or "nil") - ngx.say("Bar: ", ngx.resp.get_headers()["Bar"] or "nil") - ngx.say("bar: ", ngx.resp.get_headers()["bar"] or "nil") + + local headers, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo: ", headers["Foo"] or "nil") + ngx.say("foo: ", headers["foo"] or "nil") + ngx.say("Bar: ", headers["Bar"] or "nil") + ngx.say("bar: ", headers["bar"] or "nil") '; } --- request @@ -1233,7 +1262,12 @@ bar: baz '; header_filter_by_lua ' - local hs = ngx.resp.get_headers() + local hs, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + print("my Content-Type: ", hs["Content-Type"]) print("my content-type: ", hs["content-type"]) print("my content_type: ", hs["content_type"]) @@ -1261,7 +1295,12 @@ my content_type: text/plain '; header_filter_by_lua ' - local hs = ngx.resp.get_headers() + local hs, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + print("my Content-Length: ", hs["Content-Length"]) print("my content-length: ", hs["content-length"]) print("my content_length: ", hs.content_length) @@ -1289,7 +1328,12 @@ my content_length: 3 '; header_filter_by_lua ' - local hs = ngx.resp.get_headers() + local hs, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + print("my Connection: ", hs["Connection"]) print("my connection: ", hs["connection"]) '; @@ -1315,7 +1359,12 @@ my connection: close '; body_filter_by_lua ' - local hs = ngx.resp.get_headers() + local hs, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + print("my Transfer-Encoding: ", hs["Transfer-Encoding"]) print("my transfer-encoding: ", hs["transfer-encoding"]) print("my transfer_encoding: ", hs.transfer_encoding) @@ -1342,7 +1391,12 @@ my transfer-encoding: chunked '; body_filter_by_lua ' - local hs = ngx.resp.get_headers() + local hs, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + print("my Transfer-Encoding: ", hs["Transfer-Encoding"]) print("my transfer-encoding: ", hs["transfer-encoding"]) print("my transfer_encoding: ", hs.transfer_encoding) @@ -1497,7 +1551,12 @@ types { location /set/ { default_type text/html; content_by_lua_block { - local h = ngx.resp.get_headers() + local h, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + ngx.say(h["content-type"]) } } @@ -1509,3 +1568,134 @@ Content-Type: image/png image/png --- no_error_log [error] + + + +=== TEST 72: exceeding max header limit (default 100) +--- config + location /resp-header { + content_by_lua_block { + for i = 1, 99 do + ngx.header["Foo" .. i] = "Foo" + end + + local headers, err = ngx.resp.get_headers() + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for k, v in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " resp headers"); + } + } +--- request +GET /resp-header +--- response_body +err: truncated +found 100 resp headers +--- no_error_log +[error] +--- log_level: debug +--- error_log +lua exceeding response header limit 101 > 100 + + + +=== TEST 73: NOT exceeding max header limit (default 100) +--- config + location /resp-header { + content_by_lua_block { + for i = 1, 98 do + ngx.header["Foo" .. i] = "Foo" + end + + local headers, err = ngx.resp.get_headers() + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for k, v in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " resp headers"); + } + } +--- request +GET /resp-header +--- response_body +found 100 resp headers +--- no_error_log +[error] +lua exceeding response header limit +--- log_level: debug + + + +=== TEST 74: exceeding max header limit (custom limit, 3) +--- config + location /resp-header { + content_by_lua_block { + for i = 1, 2 do + ngx.header["Foo" .. i] = "Foo" + end + + local headers, err = ngx.resp.get_headers(3) + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for k, v in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " resp headers"); + } + } +--- request +GET /resp-header +--- response_body +err: truncated +found 3 resp headers +--- no_error_log +[error] +--- error_log +lua exceeding response header limit 4 > 3 +--- log_level: debug + + + +=== TEST 75: NOT exceeding max header limit (custom limit, 3) +--- config + location /resp-header { + content_by_lua_block { + for i = 1, 1 do + ngx.header["Foo" .. i] = "Foo" + end + + local headers, err = ngx.resp.get_headers(3) + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for k, v in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " resp headers"); + } + } +--- request +GET /resp-header +--- response_body +found 3 resp headers +--- no_error_log +[error] +lua exceeding response header limit diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 3390d6480b..658a1d232b 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1119,7 +1119,11 @@ hello world --- config location /t { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + if err then + ngx.say("err: ", err) + end + local res = ngx.location.capture("/sub", { args = args }) ngx.print(res.body) '; diff --git a/t/028-req-header.t b/t/028-req-header.t index 1ebcdff4e2..ca92daab55 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -8,10 +8,10 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (2 * blocks() + 31); +plan tests => repeat_each() * (2 * blocks() + 43); #no_diff(); -#no_long_string(); +no_long_string(); run_tests(); @@ -21,8 +21,14 @@ __DATA__ --- config location /req-header { content_by_lua ' - ngx.say("Foo: ", ngx.req.get_headers()["Foo"] or "nil") - ngx.say("Bar: ", ngx.req.get_headers()["Bar"] or "nil") + local headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo: ", headers["Foo"] or "nil") + ngx.say("Bar: ", headers["Bar"] or "nil") '; } --- request @@ -43,8 +49,14 @@ lua exceeding request header limit --- config location /req-header { content_by_lua ' + local headers, err = ngx.req.get_headers(nil, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + local h = {} - for k, v in pairs(ngx.req.get_headers(nil, true)) do + for k, v in pairs(headers) do h[k] = v end ngx.say("Foo: ", h["Foo"] or "nil") @@ -289,16 +301,43 @@ GET /bar --- config location /foo { content_by_lua ' - ngx.say("Foo: ", ngx.req.get_headers()["Foo"] or "nil") + local headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo: ", headers["Foo"] or "nil") ngx.req.set_header("Foo", 32) - ngx.say("Foo 1: ", ngx.req.get_headers()["Foo"] or "nil") + + headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo 1: ", headers["Foo"] or "nil") ngx.req.set_header("Foo", "abc") - ngx.say("Foo 2: ", ngx.req.get_headers()["Foo"] or "nil") + + headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo 2: ", headers["Foo"] or "nil") ngx.req.clear_header("Foo") - ngx.say("Foo 3: ", ngx.req.get_headers()["Foo"] or "nil") + + headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("Foo 3: ", headers["Foo"] or "nil") '; } --- more_headers @@ -319,7 +358,13 @@ Foo 3: nil location /foo { content_by_lua ' collectgarbage() - local vals = ngx.req.get_headers()["Foo"] + local headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + local vals = headers["Foo"] ngx.say("value is of type ", type(vals), ".") if type(vals) == "table" then ngx.say("Foo takes ", #vals or "nil", " values.") @@ -386,11 +431,15 @@ Content-Encoding: gzip -=== TEST 19: default max 100 headers +=== TEST 19: exceeding default max 100 header limit --- config location /lua { content_by_lua ' - local headers = ngx.req.get_headers() + local headers, err = ngx.req.get_headers() + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(headers) do table.insert(keys, key) @@ -407,7 +456,7 @@ GET /lua --- more_headers eval my $i = 1; my $s; -while ($i <= 102) { +while ($i <= 99) { $s .= "X-$i:$i\n"; $i++; } @@ -427,27 +476,34 @@ for my $k (@k) { $k .= ": $&\n"; } } -CORE::join("", @k); +"err: truncated\n" . CORE::join("", @k); --- timeout: 4 --- error_log -lua exceeding request header limit 100 +lua exceeding request header limit 101 > 100 -=== TEST 20: custom max 102 headers +=== TEST 20: NOT exceeding default max 100 header limit --- config location /lua { content_by_lua ' - local headers = ngx.req.get_headers(102) + local headers, err = ngx.req.get_headers() + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(headers) do table.insert(keys, key) end table.sort(keys) + local cnt = 0 for i, key in ipairs(keys) do ngx.say(key, ": ", headers[key]) + cnt = cnt + 1 end + ngx.say("found ", cnt, " headers") '; } --- request @@ -455,7 +511,60 @@ GET /lua --- more_headers eval my $i = 1; my $s; -while ($i <= 103) { +while ($i <= 98) { + $s .= "X-$i:$i\n"; + $i++; +} +$s +--- response_body eval +my @k; +my $i = 1; +while ($i <= 98) { + push @k, "x-$i"; + $i++; +} +push @k, "connection: close\n"; +push @k, "host: localhost\n"; +@k = sort @k; +for my $k (@k) { + if ($k =~ /\d+/) { + $k .= ": $&\n"; + } +} +CORE::join("", @k) . "found 100 headers\n"; +--- timeout: 4 +--- no_error_log +[error] +lua exceeding request header limit + + + +=== TEST 21: execeeding custom max 102 header limit +--- config + location /lua { + content_by_lua ' + local headers, err = ngx.req.get_headers(102) + if err then + ngx.say("err: ", err) + end + + local keys = {} + for key, val in pairs(headers) do + table.insert(keys, key) + end + + table.sort(keys) + for i, key in ipairs(keys) do + ngx.say(key, ": ", headers[key]) + end + '; + } +--- request +GET /lua +--- more_headers eval +my $i = 1; +my $s; +while ($i <= 101) { $s .= "X-$i:$i\n"; $i++; } @@ -475,18 +584,75 @@ for my $k (@k) { $k .= ": $&\n"; } } -CORE::join("", @k); +"err: truncated\n" . CORE::join("", @k); --- timeout: 4 --- error_log -lua exceeding request header limit 102 +lua exceeding request header limit 103 > 102 + + + +=== TEST 22: NOT execeeding custom max 102 header limit +--- config + location /lua { + content_by_lua ' + local headers, err = ngx.req.get_headers(102) + if err then + ngx.say("err: ", err) + end + + local keys = {} + for key, val in pairs(headers) do + table.insert(keys, key) + end + + table.sort(keys) + for i, key in ipairs(keys) do + ngx.say(key, ": ", headers[key]) + end + '; + } +--- request +GET /lua +--- more_headers eval +my $i = 1; +my $s; +while ($i <= 100) { + $s .= "X-$i:$i\n"; + $i++; +} +$s +--- response_body eval +my @k; +my $i = 1; +while ($i <= 100) { + push @k, "x-$i"; + $i++; +} +push @k, "connection: close\n"; +push @k, "host: localhost\n"; +@k = sort @k; +for my $k (@k) { + if ($k =~ /\d+/) { + $k .= ": $&\n"; + } +} +CORE::join("", @k); +--- timeout: 4 +--- no_error_log +[error] +lua exceeding request header limit -=== TEST 21: custom unlimited headers +=== TEST 23: custom unlimited headers --- config location /lua { content_by_lua ' - local headers = ngx.req.get_headers(0) + local headers, err = ngx.req.get_headers(0) + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(headers) do table.insert(keys, key) @@ -528,7 +694,7 @@ CORE::join("", @k); -=== TEST 22: modify subrequest req headers should not affect the parent +=== TEST 24: modify subrequest req headers should not affect the parent --- config location = /main { rewrite_by_lua ' @@ -584,7 +750,7 @@ Bar: [] -=== TEST 23: clear_header should clear all the instances of the user custom header +=== TEST 25: clear_header should clear all the instances of the user custom header --- config location = /t { rewrite_by_lua ' @@ -610,7 +776,7 @@ Test-Header: [1] -=== TEST 24: clear_header should clear all the instances of the builtin header +=== TEST 26: clear_header should clear all the instances of the builtin header --- config location = /t { rewrite_by_lua ' @@ -637,7 +803,7 @@ Test-Header: [1] -=== TEST 25: Converting POST to GET - clearing headers (bug found by Matthieu Tourne, 411 error page) +=== TEST 27: Converting POST to GET - clearing headers (bug found by Matthieu Tourne, 411 error page) --- config location /t { rewrite_by_lua ' @@ -668,7 +834,7 @@ $/ -=== TEST 26: clear_header() does not duplicate subsequent headers (old bug) +=== TEST 28: clear_header() does not duplicate subsequent headers (old bug) --- config location = /t { rewrite_by_lua ' @@ -738,13 +904,19 @@ Foo22: foo22\r -=== TEST 27: iterating through headers (raw form) +=== TEST 29: iterating through headers (raw form) --- config location /t { content_by_lua ' local h = {} local arr = {} - for k, v in pairs(ngx.req.get_headers(nil, true)) do + local headers, err = ngx.req.get_headers(nil, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + for k, v in pairs(headers) do h[k] = v table.insert(arr, k) end @@ -767,11 +939,16 @@ My-Foo: bar -=== TEST 28: __index metamethod not working for "raw" mode +=== TEST 30: __index metamethod not working for "raw" mode --- config location /t { content_by_lua ' - local h = ngx.req.get_headers(nil, true) + local h, err = ngx.req.get_headers(nil, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + ngx.say("My-Foo-Header: ", h.my_foo_header) '; } @@ -784,11 +961,16 @@ My-Foo-Header: nil -=== TEST 29: __index metamethod not working for the default mode +=== TEST 31: __index metamethod not working for the default mode --- config location /t { content_by_lua ' - local h = ngx.req.get_headers() + local h, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + ngx.say("My-Foo-Header: ", h.my_foo_header) '; } @@ -801,7 +983,7 @@ My-Foo-Header: Hello World -=== TEST 30: clear input header (just more than 20 headers) +=== TEST 32: clear input header (just more than 20 headers) --- config location = /t { rewrite_by_lua 'ngx.req.clear_header("R")'; @@ -849,7 +1031,7 @@ Q: q\r -=== TEST 31: clear input header (just more than 20 headers, and add more) +=== TEST 33: clear input header (just more than 20 headers, and add more) --- config location = /t { rewrite_by_lua ' @@ -923,7 +1105,7 @@ foo-21: 21\r -=== TEST 32: clear input header (just more than 21 headers) +=== TEST 34: clear input header (just more than 21 headers) --- config location = /t { rewrite_by_lua ' @@ -974,7 +1156,7 @@ P: p\r -=== TEST 33: clear input header (just more than 21 headers) +=== TEST 35: clear input header (just more than 21 headers) --- config location = /t { rewrite_by_lua ' @@ -1049,21 +1231,27 @@ foo-21: 21\r -=== TEST 34: raw form +=== TEST 36: raw form --- config location /t { content_by_lua ' - -- get ALL the raw headers (0 == no limit, not recommended) - local h = {} - local arr = {} - for k, v in pairs(ngx.req.get_headers(0, true)) do - h[k] = v - table.insert(arr, k) - end - table.sort(arr) - for i, k in ipairs(arr) do - ngx.say(k, ": ", h[k]) - end + local headers, err = ngx.req.get_headers(0, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + -- get ALL the raw headers (0 == no limit, not recommended) + local h = {} + local arr = {} + for k, v in pairs(headers) do + h[k] = v + table.insert(arr, k) + end + table.sort(arr) + for i, k in ipairs(arr) do + ngx.say(k, ": ", h[k]) + end '; } --- request @@ -1081,7 +1269,7 @@ My-Foo: bar -=== TEST 35: clear X-Real-IP +=== TEST 37: clear X-Real-IP --- config location /t { rewrite_by_lua ' @@ -1127,7 +1315,7 @@ X-Real-IP: -=== TEST 36: set custom X-Real-IP +=== TEST 38: set custom X-Real-IP --- config location /t { rewrite_by_lua ' @@ -1172,7 +1360,7 @@ X-Real-IP: 8.8.4.4 -=== TEST 37: clear Via +=== TEST 39: clear Via --- config location /t { rewrite_by_lua ' @@ -1218,7 +1406,7 @@ Via: -=== TEST 38: set custom Via +=== TEST 40: set custom Via --- config location /t { rewrite_by_lua ' @@ -1263,7 +1451,7 @@ Via: 1.0 fred, 1.1 nowhere.com (Apache/1.1) -=== TEST 39: set input header (with underscores in the header name) +=== TEST 41: set input header (with underscores in the header name) --- config location /req-header { rewrite_by_lua ' @@ -1286,12 +1474,18 @@ $} -=== TEST 40: HTTP 0.9 (set & get) +=== TEST 42: HTTP 0.9 (set & get) --- config location /foo { content_by_lua ' ngx.req.set_header("X-Foo", "howdy"); - ngx.say("X-Foo: ", ngx.req.get_headers()["X-Foo"]) + local headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("X-Foo: ", headers["X-Foo"]) '; } --- raw_request eval @@ -1306,12 +1500,18 @@ X-Foo: nil -=== TEST 41: HTTP 0.9 (clear) +=== TEST 43: HTTP 0.9 (clear) --- config location /foo { content_by_lua ' ngx.req.set_header("X-Foo", "howdy"); - ngx.say("X-Foo: ", ngx.req.get_headers()["X-Foo"]) + local headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + ngx.say("X-Foo: ", headers["X-Foo"]) '; } --- raw_request eval @@ -1326,7 +1526,7 @@ X-Foo: nil -=== TEST 42: Host header with port and $host (github issue #292) +=== TEST 44: Host header with port and $host (github issue #292) --- config location /bar { rewrite_by_lua ' @@ -1343,7 +1543,7 @@ http_host var: agentzh.org:1984 -=== TEST 43: Host header with upper case letters and $host (github issue #292) +=== TEST 45: Host header with upper case letters and $host (github issue #292) --- config location /bar { rewrite_by_lua ' @@ -1360,11 +1560,16 @@ http_host var: agentZH.org:1984 -=== TEST 44: clear all and re-insert +=== TEST 46: clear all and re-insert --- config location = /t { content_by_lua ' - local headers = ngx.req.get_headers(100, true) + local headers, err = ngx.req.get_headers(100, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + local n = 0 for header, _ in pairs(headers) do n = n + 1 @@ -1377,7 +1582,13 @@ http_host var: agentZH.org:1984 print("1: reinsert header ", header, ": ", i) ngx.req.set_header(header, value) end - local headers = ngx.req.get_headers(100, true) + + headers, err = ngx.req.get_headers(100, true) + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + n = 0 for header, _ in pairs(headers) do n = n + 1 @@ -1417,13 +1628,18 @@ got 8 headers -=== TEST 45: github issue #314: ngx.req.set_header does not override request headers with multiple values +=== TEST 47: github issue #314: ngx.req.set_header does not override request headers with multiple values --- config #lua_code_cache off; location = /t { content_by_lua ' ngx.req.set_header("AAA", "111") - local headers = ngx.req.get_headers() + local headers, err = ngx.req.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + ngx.say(headers["AAA"]) '; } @@ -1441,7 +1657,7 @@ AAA: 678 -=== TEST 46: clear If-Match req header +=== TEST 48: clear If-Match req header --- config location /t { content_by_lua ' @@ -1463,7 +1679,7 @@ test -=== TEST 47: clear If-Unmodified-Since req header +=== TEST 49: clear If-Unmodified-Since req header --- config location /t { content_by_lua ' @@ -1486,7 +1702,7 @@ test -=== TEST 48: clear If-None-Match req header +=== TEST 50: clear If-None-Match req header --- config location /t { content_by_lua ' @@ -1509,7 +1725,7 @@ test -=== TEST 49: set the Destination request header for WebDav +=== TEST 51: set the Destination request header for WebDav --- config location = /a.txt { rewrite_by_lua_block { @@ -1535,7 +1751,7 @@ client sent no "Destination" header -=== TEST 50: X-Forwarded-For +=== TEST 52: X-Forwarded-For --- config location = /t { access_by_lua_block { @@ -1559,7 +1775,7 @@ Foo: 8.8.8.8, 127.0.0.1 -=== TEST 51: X-Forwarded-For +=== TEST 53: X-Forwarded-For --- config location = /t { access_by_lua_block { @@ -1585,7 +1801,7 @@ Foo: 127.0.0.1 -=== TEST 52: for bad requests (bad request method letter case) +=== TEST 54: for bad requests (bad request method letter case) --- config error_page 400 = /err; @@ -1605,7 +1821,7 @@ ok -=== TEST 53: for bad requests (bad request method names) +=== TEST 55: for bad requests (bad request method names) --- config error_page 400 = /err; @@ -1625,7 +1841,7 @@ ok -=== TEST 54: for bad requests causing segfaults when setting & getting multi-value headers +=== TEST 56: for bad requests causing segfaults when setting & getting multi-value headers --- config error_page 400 = /err; @@ -1644,3 +1860,151 @@ ok --- no_error_log [error] --- no_check_leak + + + +=== TEST 57: execeeding custom 3 header limit +--- config + location /lua { + content_by_lua ' + local headers, err = ngx.req.get_headers(3) + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for key, val in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " headers."); + '; + } +--- request +GET /lua +--- more_headers eval +my $i = 1; +my $s; +while ($i <= 2) { + $s .= "X-$i:$i\n"; + $i++; +} +$s +--- response_body +err: truncated +found 3 headers. +--- timeout: 4 +--- error_log +lua exceeding request header limit 4 > 3 +--- no_error_log +[error] + + + +=== TEST 58: NOT execeeding custom 3 header limit +--- config + location /lua { + content_by_lua ' + local headers, err = ngx.req.get_headers(3) + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for key, val in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " headers."); + '; + } +--- request +GET /lua +--- more_headers eval +my $i = 1; +my $s; +while ($i <= 1) { + $s .= "X-$i:$i\n"; + $i++; +} +$s +--- response_body +found 3 headers. +--- timeout: 4 +--- no_error_log +lua exceeding request header limit +[error] + + + +=== TEST 59: execeeding custom 3 header limit (raw) +--- config + location /lua { + content_by_lua ' + local headers, err = ngx.req.get_headers(3, true) + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for key, val in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " headers."); + '; + } +--- request +GET /lua +--- more_headers eval +my $i = 1; +my $s; +while ($i <= 2) { + $s .= "X-$i:$i\n"; + $i++; +} +$s +--- response_body +err: truncated +found 3 headers. +--- timeout: 4 +--- error_log +lua exceeding request header limit 4 > 3 +--- no_error_log +[error] + + + +=== TEST 60: NOT execeeding custom 3 header limit (raw) +--- config + location /lua { + content_by_lua ' + local headers, err = ngx.req.get_headers(3, true) + if err then + ngx.say("err: ", err) + end + + local cnt = 0 + for key, val in pairs(headers) do + cnt = cnt + 1 + end + + ngx.say("found ", cnt, " headers."); + '; + } +--- request +GET /lua +--- more_headers eval +my $i = 1; +my $s; +while ($i <= 1) { + $s .= "X-$i:$i\n"; + $i++; +} +$s +--- response_body +found 3 headers. +--- timeout: 4 +--- no_error_log +lua exceeding request header limit +[error] diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 96e216c15b..a9d46ff97b 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -24,7 +24,12 @@ __DATA__ --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -49,7 +54,12 @@ c = true --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -74,7 +84,12 @@ foo = true --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -87,7 +102,12 @@ foo = true ngx.say("again...") - args = ngx.req.get_uri_args() + args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -114,7 +134,12 @@ b r = 4a 2 --- config location /t { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -139,7 +164,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -164,7 +194,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -195,7 +230,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -227,7 +267,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} -- ngx.say(args) for key, val in pairs(args) do @@ -253,7 +298,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} -- ngx.say(args) for key, val in pairs(args) do @@ -280,7 +330,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -305,7 +360,12 @@ done --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -320,7 +380,12 @@ done ngx.var.args = "a=3&b=4" - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -768,7 +833,12 @@ rc: false, err: bad argument #1 to '?' (table expected, got boolean) --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args(2) + local args, err = ngx.req.get_uri_args(2) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -783,6 +853,7 @@ rc: false, err: bad argument #1 to '?' (table expected, got boolean) --- request GET /lua?foo=3&bar=4&baz=2 --- response_body +err: truncated bar = 4 foo = 3 --- error_log @@ -795,7 +866,12 @@ lua hit query args limit 2 --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args(2) + local args, err = ngx.req.get_uri_args(2) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -810,6 +886,7 @@ lua hit query args limit 2 --- request GET /lua?foo=3&bar&baz=2 --- response_body +err: truncated bar = true foo = 3 --- error_log @@ -822,7 +899,12 @@ lua hit query args limit 2 --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args(2) + local args, err = ngx.req.get_uri_args(2) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -839,6 +921,7 @@ lua hit query args limit 2 --- request GET /lua?foo=3&=hello&=world --- response_body +err: truncated foo = 3 done --- error_log @@ -851,7 +934,12 @@ lua hit query args limit 2 --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -887,7 +975,8 @@ for my $k (@k) { $k .= " = $&\n"; } } -CORE::join("", @k); + +"err: truncated\n" . CORE::join("", @k); --- timeout: 4 --- error_log lua hit query args limit 100 @@ -899,7 +988,12 @@ lua hit query args limit 100 --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args(102) + local args, err = ngx.req.get_uri_args(102) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -935,7 +1029,8 @@ for my $k (@k) { $k .= " = $&\n"; } } -CORE::join("", @k); + +"err: truncated\n" . CORE::join("", @k); --- timeout: 4 --- error_log lua hit query args limit 102 @@ -947,7 +1042,12 @@ lua hit query args limit 102 --- config location /lua { content_by_lua ' - local args = ngx.req.get_uri_args(0) + local args, err = ngx.req.get_uri_args(0) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -1013,7 +1113,12 @@ HTTP/1.0 a=3&b=5&b=6 location /lua { content_by_lua ' local args = "a=bar&b=foo" - args = ngx.decode_args(args) + args, err = ngx.decode_args(args) + + if err then + ngx.say("err: ", err) + end + ngx.say("a = ", args.a) ngx.say("b = ", args.b) '; @@ -1031,7 +1136,12 @@ b = foo location /lua { content_by_lua ' local args = "a=bar&b=foo&a=baz" - args = ngx.decode_args(args) + args, err = ngx.decode_args(args) + + if err then + ngx.say("err: ", err) + end + ngx.say("a = ", table.concat(args.a, ", ")) ngx.say("b = ", args.b) '; @@ -1049,7 +1159,11 @@ b = foo location /lua { content_by_lua ' local args = "" - args = ngx.decode_args(args) + args, err = ngx.decode_args(args) + if err then + ngx.say("err: ", err) + end + ngx.say("n = ", #args) '; } @@ -1065,7 +1179,11 @@ n = 0 location /lua { content_by_lua ' local args = "a&b" - args = ngx.decode_args(args) + args, err = ngx.decode_args(args) + if err then + ngx.say("err: ", err) + end + ngx.say("a = ", args.a) ngx.say("b = ", args.b) '; @@ -1083,7 +1201,12 @@ b = true location /lua { content_by_lua ' local args = "a=&b=" - args = ngx.decode_args(args) + args, err = ngx.decode_args(args) + + if err then + ngx.say("err: ", err) + end + ngx.say("a = ", args.a) ngx.say("b = ", args.b) '; @@ -1101,7 +1224,11 @@ b = location /lua { content_by_lua ' local args = "a=bar&b=foo" - args = ngx.decode_args(args, 1) + args, err = ngx.decode_args(args, 1) + if err then + ngx.say("err: ", err) + end + ngx.say("a = ", args.a) ngx.say("b = ", args.b) '; @@ -1109,6 +1236,7 @@ b = --- request GET /lua --- response_body +err: truncated a = bar b = nil @@ -1119,7 +1247,12 @@ b = nil location /lua { content_by_lua ' local args = "a=bar&b=foo" - args = ngx.decode_args(args, -1) + args, err = ngx.decode_args(args, -1) + + if err then + ngx.say("err: ", err) + end + ngx.say("a = ", args.a) ngx.say("b = ", args.b) '; @@ -1137,7 +1270,11 @@ b = foo location /lua { content_by_lua ' local s = "f+f=bar&B=foo" - args = ngx.decode_args(s) + args, err = ngx.decode_args(s) + if err then + ngx.say("err: ", err) + end + local arr = {} for k, v in pairs(args) do table.insert(arr, k) @@ -1183,7 +1320,11 @@ s = f+f=bar&B=foo local commands = split(post_data, "||") for _, command in pairs(commands) do --command = ngx.unescape_uri(command) - local request_args = ngx.decode_args(command, 0) + local request_args, err = ngx.decode_args(command, 0) + if err then + ngx.say("err: ", err) + end + local arr = {} for k, v in pairs(request_args) do table.insert(arr, k) diff --git a/t/031-post-args.t b/t/031-post-args.t index 62c88c1ec7..4659f059fc 100644 --- a/t/031-post-args.t +++ b/t/031-post-args.t @@ -11,7 +11,7 @@ repeat_each(2); plan tests => repeat_each() * (blocks() * 2 + 6); #no_diff(); -#no_long_string(); +no_long_string(); run_tests(); __DATA__ @@ -21,7 +21,12 @@ __DATA__ location /lua { lua_need_request_body on; content_by_lua ' - local args = ngx.req.get_post_args() + local args, err = ngx.req.get_post_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -48,7 +53,12 @@ c = true location /lua { lua_need_request_body off; content_by_lua ' - local args = ngx.req.get_post_args() + local args, err = ngx.req.get_post_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -73,7 +83,12 @@ a=3&b=4&c location /lua { lua_need_request_body on; content_by_lua ' - local args = ngx.req.get_post_args() + local args, err = ngx.req.get_post_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -101,7 +116,12 @@ POST /lua location /lua { content_by_lua ' ngx.req.read_body(); - local args = ngx.req.get_post_args(2) + local args, err = ngx.req.get_post_args(2) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -117,6 +137,7 @@ POST /lua POST /lua foo=3&bar=4&baz=2 --- response_body +err: truncated bar = 4 foo = 3 --- error_log @@ -129,7 +150,12 @@ lua hit query args limit 2 location /lua { content_by_lua ' ngx.req.read_body(); - local args = ngx.req.get_post_args(2) + local args, err = ngx.req.get_post_args(2) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -145,6 +171,7 @@ lua hit query args limit 2 POST /lua foo=3&bar&baz=2 --- response_body +err: truncated bar = true foo = 3 --- error_log @@ -157,7 +184,12 @@ lua hit query args limit 2 location /lua { content_by_lua ' ngx.req.read_body(); - local args = ngx.req.get_post_args(2) + local args, err = ngx.req.get_post_args(2) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -175,6 +207,7 @@ lua hit query args limit 2 POST /lua foo=3&=hello&=world --- response_body +err: truncated foo = 3 done --- error_log @@ -187,7 +220,12 @@ lua hit query args limit 2 location /lua { content_by_lua ' ngx.req.read_body(); - local args = ngx.req.get_post_args() + local args, err = ngx.req.get_post_args() + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -223,7 +261,7 @@ for my $k (@k) { $k .= " = $&\n"; } } -CORE::join("", @k); +"err: truncated\n" . CORE::join("", @k); --- timeout: 4 --- error_log lua hit query args limit 100 @@ -235,7 +273,12 @@ lua hit query args limit 100 location /lua { content_by_lua ' ngx.req.read_body() - local args = ngx.req.get_post_args(102) + local args, err = ngx.req.get_post_args(102) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -271,7 +314,7 @@ for my $k (@k) { $k .= " = $&\n"; } } -CORE::join("", @k); +"err: truncated\n" . CORE::join("", @k); --- timeout: 4 --- error_log lua hit query args limit 102 @@ -283,7 +326,12 @@ lua hit query args limit 102 location /lua { content_by_lua ' ngx.req.read_body() - local args = ngx.req.get_post_args(0) + local args, err = ngx.req.get_post_args(0) + + if err then + ngx.say("err: ", err) + end + local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -332,9 +380,11 @@ CORE::join("", @k); content_by_lua_block { local args, err = ngx.req.get_post_args() - if not args then - ngx.say(err) - else + if err then + ngx.say("err: ", err) + end + + if args then local keys = {} for key, val in pairs(args) do table.insert(keys, key) @@ -351,6 +401,6 @@ CORE::join("", @k); POST /lua a=3&b=4&c --- response_body -request body in temp file not supported +err: request body in temp file not supported --- no_error_log [error] diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 9cca3b749f..7d8ee74673 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -254,7 +254,11 @@ uid: 33 header_filter_by_lua ' local str = ""; - local args = ngx.req.get_uri_args() + local args, err = ngx.req.get_uri_args() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end local keys = {} for key, val in pairs(args) do table.insert(keys, key) diff --git a/t/044-req-body.t b/t/044-req-body.t index 28fa818657..e645eeefe5 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -727,7 +727,13 @@ hello, world"] } --- user_files >>> myscript.lua - local data, data2 = ngx.req.get_post_args(), {} + local data, err = ngx.req.get_post_args() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + local data2 = {} for k, v in pairs(data) do if type(v) == "table" then for i, val in ipairs(v) do diff --git a/t/138-balancer.t b/t/138-balancer.t index 3745577751..0aba66d62b 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -160,7 +160,7 @@ qr/\[crit\] .* connect\(\) .*? failed/, upstream backend { server 0.0.0.1; balancer_by_lua_block { - print("arg foo: ", ngx.req.get_uri_args()["foo"]) + print("arg foo: ", (ngx.req.get_uri_args())["foo"]) } } --- config From f341daa2696d84a2267c62f05bf30b90aa7a3da6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 20 Apr 2018 17:17:03 -0700 Subject: [PATCH 233/848] api: bumped API version to 1.10.13. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index bf4245dada..c7c0e6b533 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10012 +#define ngx_http_lua_version 10013 typedef struct { From 0605709c32ab61a277700ef6717b916626912a35 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 19 Apr 2018 23:59:26 -0700 Subject: [PATCH 234/848] bugfix: fixed compatibility regressions on Win32. this regression had appeared in commit 012f3163e. --- src/ngx_http_lua_initworkerby.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 91cfef5e47..4a722a062d 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -46,13 +46,15 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) /* lmcf != NULL && lmcf->lua != NULL */ - /* disable init_worker_by_lua* and destroy lua VM in cache processes */ +#if !(NGX_WIN32) if (ngx_process == NGX_PROCESS_HELPER -#if defined(HAVE_PRIVILEGED_PROCESS_PATCH) && !NGX_WIN32 +# ifdef HAVE_PRIVILEGED_PROCESS_PATCH && !ngx_is_privileged_agent -#endif +# endif ) { + /* disable init_worker_by_lua* and destroy lua VM in cache processes */ + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua close the global Lua VM %p in the " "cache helper process %P", lmcf->lua, ngx_pid); @@ -62,6 +64,7 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) return NGX_OK; } +#endif /* NGX_WIN32 */ if (lmcf->init_worker_handler == NULL) { return NGX_OK; From 612931c0f16b02fd3b1179267cffa6bd5be6d9cf Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 22 Apr 2018 11:41:56 -0700 Subject: [PATCH 235/848] doc: bumped version to 0.10.13. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index f805505689..15ad00ecfb 100644 --- a/README.markdown +++ b/README.markdown @@ -62,7 +62,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.12](https://github.com/openresty/lua-nginx-module/tags) released on 19 April 2018. +This document describes ngx_lua [v0.10.13](https://github.com/openresty/lua-nginx-module/tags) released on 22 April 2018. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 5fcdbc5c58..e3c61e20cb 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -10,7 +10,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.12] released on 19 April 2018. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.13] released on 22 April 2018. = Synopsis = From 15e93a11ed7c6531d9e6fd838a917485dda24486 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 25 Apr 2018 13:08:24 -0700 Subject: [PATCH 236/848] restored the change in commit 4ff3dd14ed since it only changed the auto-generated README.markdown. alas. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 15ad00ecfb..a60d9a70ec 100644 --- a/README.markdown +++ b/README.markdown @@ -8181,7 +8181,7 @@ This Lua module does not ship with this ngx_lua module itself rather it is shipp the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. -Please refer to the [documentation](https://github.com/openresty/lua-resty-core/blob/ocsp-cert-by-lua-2/lib/ngx/ocsp.md) +Please refer to the [documentation](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ocsp.md) for this `ngx.ocsp` Lua module for more details. This feature requires at least ngx_lua `v0.10.0`. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index e3c61e20cb..767670e47a 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -6950,7 +6950,7 @@ This Lua module does not ship with this ngx_lua module itself rather it is shipp the [https://github.com/openresty/lua-resty-core lua-resty-core] library. -Please refer to the [https://github.com/openresty/lua-resty-core/blob/ocsp-cert-by-lua-2/lib/ngx/ocsp.md documentation] +Please refer to the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ocsp.md documentation] for this ngx.ocsp Lua module for more details. This feature requires at least ngx_lua v0.10.0. From 5a3c48d23c99a01088654bf32892de23594a35f9 Mon Sep 17 00:00:00 2001 From: chronolaw Date: Fri, 20 Apr 2018 11:08:26 +0800 Subject: [PATCH 237/848] doc: fixed the links to ssl_session_store_by_lua_block. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index a60d9a70ec..1af9e581d1 100644 --- a/README.markdown +++ b/README.markdown @@ -2602,14 +2602,14 @@ SSL session resumption can then get immediately initiated and bypass the full SS Please note that TLS session tickets are very different and it is the clients' responsibility to cache the SSL session state when session tickets are used. SSL session resumptions based on TLS session tickets would happen automatically without going through this hook (nor the -[ssl_session_store_by_lua_block](#ssl_session_store_by_lua) hook). This hook is mainly +[ssl_session_store_by_lua*](#ssl_session_store_by_lua_block) hook). This hook is mainly for older or less capable SSL clients that can only do SSL sessions by session IDs. When [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block) is specified at the same time, this hook usually runs before [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block). When the SSL session is found and successfully loaded for the current SSL connection, SSL session resumption will happen and thus bypass the [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block) -hook completely. In this case, NGINX also bypasses the [ssl_session_store_by_lua_block](#ssl_session_store_by_lua) +hook completely. In this case, NGINX also bypasses the [ssl_session_store_by_lua*](#ssl_session_store_by_lua_block) hook, for obvious reasons. To easily test this hook locally with a modern web browser, you can temporarily put the following line diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 767670e47a..cce2cb8a70 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2196,14 +2196,14 @@ SSL session resumption can then get immediately initiated and bypass the full SS Please note that TLS session tickets are very different and it is the clients' responsibility to cache the SSL session state when session tickets are used. SSL session resumptions based on TLS session tickets would happen automatically without going through this hook (nor the -[[#ssl_session_store_by_lua*|ssl_session_store_by_lua_block]] hook). This hook is mainly +[[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]] hook). This hook is mainly for older or less capable SSL clients that can only do SSL sessions by session IDs. When [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]] is specified at the same time, this hook usually runs before [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]]. When the SSL session is found and successfully loaded for the current SSL connection, SSL session resumption will happen and thus bypass the [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]] -hook completely. In this case, NGINX also bypasses the [[#ssl_session_store_by_lua*|ssl_session_store_by_lua_block]] +hook completely. In this case, NGINX also bypasses the [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]] hook, for obvious reasons. To easily test this hook locally with a modern web browser, you can temporarily put the following line From b5ffb11fe1cdedd9923cb4fe0211217fbc08b28a Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 17 May 2018 21:12:10 +0800 Subject: [PATCH 238/848] feature: implemented the receiveany() upstream cosocket method. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 36 +++ config | 2 + doc/HttpLuaModule.wiki | 31 +++ src/ngx_http_lua_input_filters.c | 137 ++++++++++++ src/ngx_http_lua_input_filters.h | 29 +++ src/ngx_http_lua_socket_tcp.c | 365 ++++++++++++++++--------------- t/058-tcp-socket.t | 314 +++++++++++++++++++++++++- t/062-count.t | 29 ++- t/156-slow-network.t | 138 ++++++++++++ 9 files changed, 904 insertions(+), 177 deletions(-) create mode 100644 src/ngx_http_lua_input_filters.c create mode 100644 src/ngx_http_lua_input_filters.h create mode 100644 t/156-slow-network.t diff --git a/README.markdown b/README.markdown index 1af9e581d1..b04a763d5f 100644 --- a/README.markdown +++ b/README.markdown @@ -3221,6 +3221,7 @@ Nginx API for Lua * [tcpsock:sslhandshake](#tcpsocksslhandshake) * [tcpsock:send](#tcpsocksend) * [tcpsock:receive](#tcpsockreceive) +* [tcpsock:receiveany](#tcpsockreceiveany) * [tcpsock:receiveuntil](#tcpsockreceiveuntil) * [tcpsock:close](#tcpsockclose) * [tcpsock:settimeout](#tcpsocksettimeout) @@ -7002,6 +7003,7 @@ Creates and returns a TCP or stream-oriented unix domain socket object (also kno * [settimeout](#tcpsocksettimeout) * [settimeouts](#tcpsocksettimeouts) * [setoption](#tcpsocksetoption) +* [receiveany](#tcpsockreceiveany) * [receiveuntil](#tcpsockreceiveuntil) * [setkeepalive](#tcpsocksetkeepalive) * [getreusedtimes](#tcpsockgetreusedtimes) @@ -7231,6 +7233,40 @@ This feature was first introduced in the `v0.5.0rc1` release. [Back to TOC](#nginx-api-for-lua) +tcpsock:receiveany +------------------ +**syntax:** *data, err = tcpsock:receiveany(max)* + +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** + +Returns any data received by the connected socket, at most `max` bytes. + +This method is a synchronous operation just like the [send](#tcpsocksend) method and is 100% nonblocking. + +In case of success, it returns the data received; in case of error, it returns `nil` with a string describing the error. + +If the received data is more than this size, this method will return with exactly this size of data. +The remaining data in the underlying receive buffer could be returned in the next reading operation. + +Timeout for the reading operation is controlled by the [lua_socket_read_timeout](#lua_socket_read_timeout) config directive and the [settimeouts](#tcpsocksettimeouts) method. And the latter takes priority. For example: + +```lua + + sock:settimeouts(1000, 1000, 1000) -- one second timeout for connect/read/write + local data, err = sock:receiveany(10 * 1024 * 1024) -- read any data, at most 10K + if not data then + ngx.say("failed to read any data: ", err) + return + end + ngx.say("successfully read: ", data) +``` + +This method doesn't automatically close the current connection when the read timeout error occurs. For other connection errors, this method always automatically closes the connection. + +This feature was first introduced in the `v0.10.14` release. + +[Back to TOC](#nginx-api-for-lua) + tcpsock:receiveuntil -------------------- **syntax:** *iterator = tcpsock:receiveuntil(pattern, options?)* diff --git a/config b/config index 044deb974e..2005535466 100644 --- a/config +++ b/config @@ -361,6 +361,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl.c \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.c \ + $ngx_addon_dir/src/ngx_http_lua_input_filters.c \ " HTTP_LUA_DEPS=" \ @@ -422,6 +423,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl.h \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.h \ + $ngx_addon_dir/src/ngx_http_lua_input_filters.h \ " CFLAGS="$CFLAGS -DNDK_SET_VAR" diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index cce2cb8a70..4b847c20a6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5894,6 +5894,7 @@ Creates and returns a TCP or stream-oriented unix domain socket object (also kno * [[#tcpsock:settimeout|settimeout]] * [[#tcpsock:settimeouts|settimeouts]] * [[#tcpsock:setoption|setoption]] +* [[#tcpsock:receiveany|receiveany]] * [[#tcpsock:receiveuntil|receiveuntil]] * [[#tcpsock:setkeepalive|setkeepalive]] * [[#tcpsock:getreusedtimes|getreusedtimes]] @@ -6103,6 +6104,36 @@ Since the v0.8.8 release, this method no longer automatically close This feature was first introduced in the v0.5.0rc1 release. +== tcpsock:receiveany == +'''syntax:''' ''data, err = tcpsock:receiveany(max)'' + +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' + +Returns any data received by the connected socket, at most max bytes. + +This method is a synchronous operation just like the [[#tcpsock:send|send]] method and is 100% nonblocking. + +In case of success, it returns the data received; in case of error, it returns nil with a string describing the error. + +If the received data is more than this size, this method will return with exactly this size of data. +The remaining data in the underlying receive buffer could be returned in the next reading operation. + +Timeout for the reading operation is controlled by the [[#lua_socket_read_timeout|lua_socket_read_timeout]] config directive and the [[#tcpsock:settimeouts|settimeouts]] method. And the latter takes priority. For example: + + + sock:settimeouts(1000, 1000, 1000) -- one second timeout for connect/read/write + local data, err = sock:receiveany(10 * 1024 * 1024) -- read any data, at most 10K + if not data then + ngx.say("failed to read any data: ", err) + return + end + ngx.say("successfully read: ", data) + + +This method doesn't automatically close the current connection when the read timeout error occurs. For other connection errors, this method always automatically closes the connection. + +This feature was first introduced in the v0.10.14 release. + == tcpsock:receiveuntil == '''syntax:''' ''iterator = tcpsock:receiveuntil(pattern, options?)'' diff --git a/src/ngx_http_lua_input_filters.c b/src/ngx_http_lua_input_filters.c new file mode 100644 index 0000000000..87a9d8c1bc --- /dev/null +++ b/src/ngx_http_lua_input_filters.c @@ -0,0 +1,137 @@ + +/* + * Copyright (C) by OpenResty Inc. + */ + + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#include "ngx_http_lua_common.h" + + +ngx_int_t +ngx_http_lua_read_bytes(ngx_buf_t *src, ngx_chain_t *buf_in, size_t *rest, + ssize_t bytes, ngx_log_t *log) +{ + if (bytes == 0) { + return NGX_ERROR; + } + + if ((size_t) bytes >= *rest) { + + buf_in->buf->last += *rest; + src->pos += *rest; + *rest = 0; + + return NGX_OK; + } + + /* bytes < *rest */ + + buf_in->buf->last += bytes; + src->pos += bytes; + *rest -= bytes; + + return NGX_AGAIN; +} + + +ngx_int_t +ngx_http_lua_read_all(ngx_buf_t *src, ngx_chain_t *buf_in, ssize_t bytes, + ngx_log_t *log) +{ + if (bytes == 0) { + return NGX_OK; + } + + buf_in->buf->last += bytes; + src->pos += bytes; + + return NGX_AGAIN; +} + + +ngx_int_t +ngx_http_lua_read_any(ngx_buf_t *src, ngx_chain_t *buf_in, size_t *max, + ssize_t bytes, ngx_log_t *log) +{ + if (bytes == 0) { + return NGX_ERROR; + } + + if (bytes >= (ssize_t) *max) { + bytes = (ssize_t) *max; + } + + buf_in->buf->last += bytes; + src->pos += bytes; + + return NGX_OK; +} + + +ngx_int_t +ngx_http_lua_read_line(ngx_buf_t *src, ngx_chain_t *buf_in, ssize_t bytes, + ngx_log_t *log) +{ + u_char *dst; + u_char c; +#if (NGX_DEBUG) + u_char *begin; +#endif + +#if (NGX_DEBUG) + begin = src->pos; +#endif + + if (bytes == 0) { + return NGX_ERROR; + } + + dd("already read: %p: %.*s", buf_in, + (int) (buf_in->buf->last - buf_in->buf->pos), buf_in->buf->pos); + + dd("data read: %.*s", (int) bytes, src->pos); + + dst = buf_in->buf->last; + + while (bytes--) { + + c = *src->pos++; + + switch (c) { + case '\n': + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, + "lua read the final line part: \"%*s\"", + src->pos - 1 - begin, begin); + + buf_in->buf->last = dst; + + dd("read a line: %p: %.*s", buf_in, + (int) (buf_in->buf->last - buf_in->buf->pos), buf_in->buf->pos); + + return NGX_OK; + + case '\r': + /* ignore it */ + break; + + default: + *dst++ = c; + break; + } + } + +#if (NGX_DEBUG) + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, + "lua read partial line data: %*s", dst - begin, begin); +#endif + + buf_in->buf->last = dst; + + return NGX_AGAIN; +} diff --git a/src/ngx_http_lua_input_filters.h b/src/ngx_http_lua_input_filters.h new file mode 100644 index 0000000000..046d40fda1 --- /dev/null +++ b/src/ngx_http_lua_input_filters.h @@ -0,0 +1,29 @@ + +/* + * Copyright (C) by OpenResty Inc. + */ + + +#ifndef _NGX_HTTP_LUA_INPUT_FILTERS_H_INCLUDED_ +#define _NGX_HTTP_LUA_INPUT_FILTERS_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +ngx_int_t ngx_http_lua_read_bytes(ngx_buf_t *src, ngx_chain_t *buf_in, + size_t *rest, ssize_t bytes, ngx_log_t *log); + +ngx_int_t ngx_http_lua_read_all(ngx_buf_t *src, ngx_chain_t *buf_in, + ssize_t bytes, ngx_log_t *log); + +ngx_int_t ngx_http_lua_read_any(ngx_buf_t *src, ngx_chain_t *buf_in, + size_t *max, ssize_t bytes, ngx_log_t *log); + +ngx_int_t ngx_http_lua_read_line(ngx_buf_t *src, ngx_chain_t *buf_in, + ssize_t bytes, ngx_log_t *log); + + +#endif /* _NGX_HTTP_LUA_INPUT_FILTERS_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index efdf427540..0717c46d50 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -11,6 +11,7 @@ #include "ngx_http_lua_socket_tcp.h" +#include "ngx_http_lua_input_filters.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_uthread.h" #include "ngx_http_lua_output.h" @@ -24,6 +25,7 @@ static int ngx_http_lua_socket_tcp_connect(lua_State *L); static int ngx_http_lua_socket_tcp_sslhandshake(lua_State *L); #endif static int ngx_http_lua_socket_tcp_receive(lua_State *L); +static int ngx_http_lua_socket_tcp_receiveany(lua_State *L); static int ngx_http_lua_socket_tcp_send(lua_State *L); static int ngx_http_lua_socket_tcp_close(lua_State *L); static int ngx_http_lua_socket_tcp_setoption(lua_State *L); @@ -69,6 +71,8 @@ static int ngx_http_lua_socket_tcp_conn_retval_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); static void ngx_http_lua_socket_dummy_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); +static int ngx_http_lua_socket_tcp_receive_helper(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); static ngx_int_t ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); static int ngx_http_lua_socket_tcp_receive_retval_handler(ngx_http_request_t *r, @@ -86,6 +90,7 @@ static int ngx_http_lua_socket_write_error_retval_handler(ngx_http_request_t *r, static ngx_int_t ngx_http_lua_socket_read_all(void *data, ssize_t bytes); static ngx_int_t ngx_http_lua_socket_read_until(void *data, ssize_t bytes); static ngx_int_t ngx_http_lua_socket_read_chunk(void *data, ssize_t bytes); +static ngx_int_t ngx_http_lua_socket_read_any(void *data, ssize_t bytes); static int ngx_http_lua_socket_tcp_receiveuntil(lua_State *L); static int ngx_http_lua_socket_receiveuntil_iterator(lua_State *L); static ngx_int_t ngx_http_lua_socket_compile_pattern(u_char *data, size_t len, @@ -292,6 +297,9 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_pushcfunction(L, ngx_http_lua_socket_tcp_receive); lua_setfield(L, -2, "receive"); + lua_pushcfunction(L, ngx_http_lua_socket_tcp_receiveany); + lua_setfield(L, -2, "receiveany"); + lua_pushcfunction(L, ngx_http_lua_socket_tcp_receiveuntil); lua_setfield(L, -2, "receiveuntil"); @@ -1755,20 +1763,173 @@ ngx_http_lua_socket_tcp_conn_retval_handler(ngx_http_request_t *r, } +static int +ngx_http_lua_socket_tcp_receive_helper(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) +{ + ngx_int_t rc; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_co_ctx_t *coctx; + + u->input_filter_ctx = u; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + if (u->bufs_in == NULL) { + u->bufs_in = + ngx_http_lua_chain_get_free_buf(r->connection->log, r->pool, + &ctx->free_recv_bufs, + u->conf->buffer_size); + + if (u->bufs_in == NULL) { + return luaL_error(L, "no memory"); + } + + u->buf_in = u->bufs_in; + u->buffer = *u->buf_in->buf; + } + + dd("tcp receive: buf_in: %p, bufs_in: %p", u->buf_in, u->bufs_in); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket read timeout: %M", u->read_timeout); + + if (u->raw_downstream || u->body_downstream) { + r->read_event_handler = ngx_http_lua_req_socket_rev_handler; + } + + u->read_waiting = 0; + u->read_co_ctx = NULL; + + rc = ngx_http_lua_socket_tcp_read(r, u); + + if (rc == NGX_ERROR) { + dd("read failed: %d", (int) u->ft_type); + rc = ngx_http_lua_socket_tcp_receive_retval_handler(r, u, L); + dd("tcp receive retval returned: %d", (int) rc); + return rc; + } + + if (rc == NGX_OK) { + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket receive done in a single run"); + + return ngx_http_lua_socket_tcp_receive_retval_handler(r, u, L); + } + + /* rc == NGX_AGAIN */ + + u->read_event_handler = ngx_http_lua_socket_read_handler; + + coctx = ctx->cur_co_ctx; + + ngx_http_lua_cleanup_pending_operation(coctx); + coctx->cleanup = ngx_http_lua_coctx_cleanup; + coctx->data = u; + + if (ctx->entered_content_phase) { + r->write_event_handler = ngx_http_lua_content_wev_handler; + + } else { + r->write_event_handler = ngx_http_core_run_phases; + } + + u->read_co_ctx = coctx; + u->read_waiting = 1; + u->read_prepare_retvals = ngx_http_lua_socket_tcp_receive_retval_handler; + + dd("setting data to %p, coctx:%p", u, coctx); + + if (u->raw_downstream || u->body_downstream) { + ctx->downstream = u; + } + + return lua_yield(L, 0); +} + + +static int +ngx_http_lua_socket_tcp_receiveany(lua_State *L) +{ + int n; + lua_Integer bytes; + ngx_http_request_t *r; + ngx_http_lua_loc_conf_t *llcf; + ngx_http_lua_socket_tcp_upstream_t *u; + + n = lua_gettop(L); + if (n != 2) { + return luaL_error(L, "expecting 2 arguments " + "(including the object), but got %d", n); + } + + r = ngx_http_lua_get_req(L); + if (r == NULL) { + return luaL_error(L, "no request found"); + } + + luaL_checktype(L, 1, LUA_TTABLE); + + lua_rawgeti(L, 1, SOCKET_CTX_INDEX); + u = lua_touserdata(L, -1); + + if (u == NULL || u->peer.connection == NULL || u->read_closed) { + + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + + if (llcf->log_socket_errors) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "attempt to receive data on a closed socket: u:%p, " + "c:%p, ft:%d eof:%d", + u, u ? u->peer.connection : NULL, + u ? (int) u->ft_type : 0, u ? (int) u->eof : 0); + } + + lua_pushnil(L); + lua_pushliteral(L, "closed"); + return 2; + } + + if (u->request != r) { + return luaL_error(L, "bad request"); + } + + ngx_http_lua_socket_check_busy_connecting(r, u, L); + ngx_http_lua_socket_check_busy_reading(r, u, L); + + if (!lua_isnumber(L, 2)) { + return luaL_argerror(L, 2, "bad max argument"); + } + + bytes = lua_tointeger(L, 2); + if (bytes <= 0) { + return luaL_argerror(L, 2, "bad max argument"); + } + + u->input_filter = ngx_http_lua_socket_read_any; + u->rest = (size_t) bytes; + u->length = u->rest; + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket calling receiveany() method to read at " + "most %uz bytes", u->rest); + + return ngx_http_lua_socket_tcp_receive_helper(r, u, L); +} + + static int ngx_http_lua_socket_tcp_receive(lua_State *L) { ngx_http_request_t *r; ngx_http_lua_socket_tcp_upstream_t *u; - ngx_int_t rc; - ngx_http_lua_ctx_t *ctx; int n; ngx_str_t pat; lua_Integer bytes; char *p; int typ; ngx_http_lua_loc_conf_t *llcf; - ngx_http_lua_co_ctx_t *coctx; n = lua_gettop(L); if (n != 1 && n != 2) { @@ -1883,119 +2044,27 @@ ngx_http_lua_socket_tcp_receive(lua_State *L) u->rest = 0; } - u->input_filter_ctx = u; - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - - if (u->bufs_in == NULL) { - u->bufs_in = - ngx_http_lua_chain_get_free_buf(r->connection->log, r->pool, - &ctx->free_recv_bufs, - u->conf->buffer_size); - - if (u->bufs_in == NULL) { - return luaL_error(L, "no memory"); - } - - u->buf_in = u->bufs_in; - u->buffer = *u->buf_in->buf; - } - - dd("tcp receive: buf_in: %p, bufs_in: %p", u->buf_in, u->bufs_in); - - if (u->raw_downstream || u->body_downstream) { - r->read_event_handler = ngx_http_lua_req_socket_rev_handler; - } - - u->read_waiting = 0; - u->read_co_ctx = NULL; - - rc = ngx_http_lua_socket_tcp_read(r, u); - - if (rc == NGX_ERROR) { - dd("read failed: %d", (int) u->ft_type); - rc = ngx_http_lua_socket_tcp_receive_retval_handler(r, u, L); - dd("tcp receive retval returned: %d", (int) rc); - return rc; - } - - if (rc == NGX_OK) { - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket receive done in a single run"); - - return ngx_http_lua_socket_tcp_receive_retval_handler(r, u, L); - } - - /* rc == NGX_AGAIN */ - - u->read_event_handler = ngx_http_lua_socket_read_handler; - - coctx = ctx->cur_co_ctx; - - ngx_http_lua_cleanup_pending_operation(coctx); - coctx->cleanup = ngx_http_lua_coctx_cleanup; - coctx->data = u; - - if (ctx->entered_content_phase) { - r->write_event_handler = ngx_http_lua_content_wev_handler; - - } else { - r->write_event_handler = ngx_http_core_run_phases; - } - - u->read_co_ctx = coctx; - u->read_waiting = 1; - u->read_prepare_retvals = ngx_http_lua_socket_tcp_receive_retval_handler; - - dd("setting data to %p, coctx:%p", u, coctx); - - if (u->raw_downstream || u->body_downstream) { - ctx->downstream = u; - } - - return lua_yield(L, 0); + return ngx_http_lua_socket_tcp_receive_helper(r, u, L); } static ngx_int_t ngx_http_lua_socket_read_chunk(void *data, ssize_t bytes) { + ngx_int_t rc; ngx_http_lua_socket_tcp_upstream_t *u = data; - ngx_buf_t *b; -#if (NGX_DEBUG) - ngx_http_request_t *r; - - r = u->request; -#endif - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, "lua tcp socket read chunk %z", bytes); - if (bytes == 0) { + rc = ngx_http_lua_read_bytes(&u->buffer, u->buf_in, &u->rest, + bytes, u->request->connection->log); + if (rc == NGX_ERROR) { u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_CLOSED; return NGX_ERROR; } - b = &u->buffer; - - if (bytes >= (ssize_t) u->rest) { - - u->buf_in->buf->last += u->rest; - b->pos += u->rest; - u->rest = 0; - - return NGX_OK; - } - - /* bytes < u->rest */ - - u->buf_in->buf->last += bytes; - b->pos += bytes; - u->rest -= bytes; - - return NGX_AGAIN; + return rc; } @@ -2004,26 +2073,10 @@ ngx_http_lua_socket_read_all(void *data, ssize_t bytes) { ngx_http_lua_socket_tcp_upstream_t *u = data; - ngx_buf_t *b; -#if (NGX_DEBUG) - ngx_http_request_t *r; - - r = u->request; -#endif - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, "lua tcp socket read all"); - - if (bytes == 0) { - return NGX_OK; - } - - b = &u->buffer; - - u->buf_in->buf->last += bytes; - b->pos += bytes; - - return NGX_AGAIN; + return ngx_http_lua_read_all(&u->buffer, u->buf_in, bytes, + u->request->connection->log); } @@ -2032,72 +2085,40 @@ ngx_http_lua_socket_read_line(void *data, ssize_t bytes) { ngx_http_lua_socket_tcp_upstream_t *u = data; - ngx_buf_t *b; - u_char *dst; - u_char c; -#if (NGX_DEBUG) - u_char *begin; -#endif + ngx_int_t rc; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, "lua tcp socket read line"); - if (bytes == 0) { + rc = ngx_http_lua_read_line(&u->buffer, u->buf_in, bytes, + u->request->connection->log); + if (rc == NGX_ERROR) { u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_CLOSED; return NGX_ERROR; } - b = &u->buffer; - -#if (NGX_DEBUG) - begin = b->pos; -#endif - - dd("already read: %p: %.*s", u->buf_in, - (int) (u->buf_in->buf->last - u->buf_in->buf->pos), - u->buf_in->buf->pos); - - dd("data read: %.*s", (int) bytes, b->pos); - - dst = u->buf_in->buf->last; - - while (bytes--) { - - c = *b->pos++; - - switch (c) { - case '\n': - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, - "lua tcp socket read the final line part: \"%*s\"", - b->pos - 1 - begin, begin); + return rc; +} - u->buf_in->buf->last = dst; - dd("read a line: %p: %.*s", u->buf_in, - (int) (u->buf_in->buf->last - u->buf_in->buf->pos), - u->buf_in->buf->pos); +static ngx_int_t +ngx_http_lua_socket_read_any(void *data, ssize_t bytes) +{ + ngx_http_lua_socket_tcp_upstream_t *u = data; - return NGX_OK; + ngx_int_t rc; - case '\r': - /* ignore it */ - break; + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "lua tcp socket read any"); - default: - *dst++ = c; - break; - } + rc = ngx_http_lua_read_any(&u->buffer, u->buf_in, &u->rest, bytes, + u->request->connection->log); + if (rc == NGX_ERROR) { + u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_CLOSED; + return NGX_ERROR; } -#if (NGX_DEBUG) - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, - "lua tcp socket read partial line data: %*s", - dst - begin, begin); -#endif - - u->buf_in->buf->last = dst; - - return NGX_AGAIN; + return rc; } diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 354a8768c5..05542eab1d 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 199; +plan tests => repeat_each() * 219; our $HtmlDir = html_dir; @@ -3808,3 +3808,315 @@ received: received: truefalsenil --- no_error_log [error] + + + +=== TEST 64: receiveany method in cosocket +--- config + server_tokens off; + location = /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + assert(sock:connect("127.0.0.1", ngx.var.port)) + local req = { + 'GET /foo HTTP/1.0\r\n', + 'Host: localhost\r\n', + 'Connection: close\r\n\r\n', + } + local ok, err = sock:send(req) + if not ok then + ngx.say("send request failed: ", err) + return + end + + -- skip http header + while true do + local data, err, _ = sock:receive('*l') + if err then + ngx.say('unexpected error occurs when receiving http head: ', err) + return + end + + if #data == 0 then -- read last line of head + break + end + end + + -- receive http body + while true do + local data, err = sock:receiveany(1024) + if err then + if err ~= 'closed' then + ngx.say('unexpected err: ', err) + end + break + end + ngx.say(data) + end + + sock:close() + } + } + + location = /foo { + content_by_lua_block { + local resp = { + '1', + '22', + 'hello world', + } + + local length = 0 + for _, v in ipairs(resp) do + length = length + #v + end + + -- flush http header + ngx.header['Content-Length'] = length + ngx.flush(true) + ngx.sleep(0.01) + + -- send http body + for _, v in ipairs(resp) do + ngx.print(v) + ngx.flush(true) + ngx.sleep(0.01) + end + } + } + +--- request +GET /t +--- response_body +1 +22 +hello world +--- no_error_log +[error] +--- error_log +lua tcp socket read any + + + +=== TEST 65: receiveany send data after read side closed +--- config + server_tokens off; + location = /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + assert(sock:connect("127.0.0.1", 7658)) + + while true do + local data, err = sock:receiveany(1024) + if err then + if err ~= 'closed' then + ngx.say('unexpected err: ', err) + break + end + + local data = "send data after read side closed" + local bytes, err = sock:send(data) + if not bytes then + ngx.say(err) + end + + break + end + ngx.say(data) + end + + sock:close() + } + } + +--- request +GET /t +--- tcp_listen: 7658 +--- tcp_shutdown: 1 +--- tcp_query eval: "send data after read side closed" +--- tcp_query_len: 32 +--- response_body +--- no_error_log +[error] + + + +=== TEST 66: receiveany with limited, max <= 0 +--- config + location = /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + assert(sock:connect("127.0.0.1", ngx.var.port)) + + local function receiveany_say_err(...) + local ok, err = pcall(sock.receiveany, sock, ...) + if not ok then + ngx.say(err) + end + end + + + receiveany_say_err(0) + receiveany_say_err(-1) + receiveany_say_err() + receiveany_say_err(nil) + } + } + +--- response_body +bad argument #2 to '?' (bad max argument) +bad argument #2 to '?' (bad max argument) +expecting 2 arguments (including the object), but got 1 +bad argument #2 to '?' (bad max argument) +--- request +GET /t +--- no_error_log +[error] + + + +=== TEST 67: receiveany with limited, max is larger than data +--- config + server_tokens off; + location = /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + assert(sock:connect("127.0.0.1", ngx.var.port)) + local req = { + 'GET /foo HTTP/1.0\r\n', + 'Host: localhost\r\n', + 'Connection: close\r\n\r\n', + } + local ok, err = sock:send(req) + if not ok then + ngx.say("send request failed: ", err) + return + end + + while true do + local data, err, _ = sock:receive('*l') + if err then + ngx.say('unexpected error occurs when receiving http head: ', err) + return + end + + if #data == 0 then -- read last line of head + break + end + end + + local data, err = sock:receiveany(128) + if err then + if err ~= 'closed' then + ngx.say('unexpected err: ', err) + end + else + ngx.say(data) + end + + sock:close() + } + } + + location = /foo { + content_by_lua_block { + local resp = 'hello world' + local length = #resp + + ngx.header['Content-Length'] = length + ngx.flush(true) + ngx.sleep(0.01) + + ngx.print(resp) + } + } + +--- request +GET /t +--- response_body +hello world +--- no_error_log +[error] +--- error_log +lua tcp socket calling receiveany() method to read at most 128 bytes + + + +=== TEST 68: receiveany with limited, max is smaller than data +--- config + server_tokens off; + location = /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + assert(sock:connect("127.0.0.1", ngx.var.port)) + local req = { + 'GET /foo HTTP/1.0\r\n', + 'Host: localhost\r\n', + 'Connection: close\r\n\r\n', + } + local ok, err = sock:send(req) + if not ok then + ngx.say("send request failed: ", err) + return + end + + while true do + local data, err, _ = sock:receive('*l') + if err then + ngx.say('unexpected error occurs when receiving http head: ', err) + return + end + + if #data == 0 then -- read last line of head + break + end + end + + while true do + local data, err = sock:receiveany(7) + if err then + if err ~= 'closed' then + ngx.say('unexpected err: ', err) + end + break + + else + ngx.say(data) + end + end + + sock:close() + } + } + + location = /foo { + content_by_lua_block { + local resp = 'hello world' + local length = #resp + + ngx.header['Content-Length'] = length + ngx.flush(true) + ngx.sleep(0.01) + + ngx.print(resp) + } + } + +--- request +GET /t +--- response_body +hello w +orld +--- no_error_log +[error] +--- error_log +lua tcp socket calling receiveany() method to read at most 7 bytes diff --git a/t/062-count.t b/t/062-count.t index 2ad2f9f0ab..2e6c5419e0 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -444,7 +444,28 @@ worker: 4 -=== TEST 20: entries under the metatable of udp sockets +=== TEST 20: entries under the metatable of tcp sockets +--- config + location = /test { + content_by_lua_block { + local n = 0 + local sock = ngx.socket.tcp() + for k, v in pairs(getmetatable(sock)) do + n = n + 1 + end + ngx.say("n = ", n) + } + } +--- request +GET /test +--- response_body +n = 13 +--- no_error_log +[error] + + + +=== TEST 21: entries under the metatable of udp sockets --- config location = /test { content_by_lua ' @@ -465,7 +486,7 @@ n = 6 -=== TEST 21: entries under the metatable of req raw sockets +=== TEST 22: entries under the metatable of req raw sockets --- config location = /test { content_by_lua ' @@ -497,7 +518,7 @@ n = 6 -=== TEST 22: entries under the req raw sockets +=== TEST 23: entries under the req raw sockets --- config location = /test { content_by_lua_block { @@ -536,7 +557,7 @@ nrec = 3 -=== TEST 23: entries under the req sockets +=== TEST 24: entries under the req sockets --- config location = /test { content_by_lua_block { diff --git a/t/156-slow-network.t b/t/156-slow-network.t new file mode 100644 index 0000000000..2d80506085 --- /dev/null +++ b/t/156-slow-network.t @@ -0,0 +1,138 @@ +BEGIN { + if (!defined $ENV{LD_PRELOAD}) { + $ENV{LD_PRELOAD} = ''; + } + + if ($ENV{LD_PRELOAD} !~ /\bmockeagain\.so\b/) { + $ENV{LD_PRELOAD} = "mockeagain.so $ENV{LD_PRELOAD}"; + } + + if ($ENV{MOCKEAGAIN} eq 'r') { + $ENV{MOCKEAGAIN} = 'rw'; + + } else { + $ENV{MOCKEAGAIN} = 'w'; + } + + $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; +} + +use Test::Nginx::Socket::Lua; + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 4); + +add_block_preprocessor(sub { + my $block = shift; + + if (!defined $block->error_log) { + $block->set_value("no_error_log", "[error]"); + } + + if (!defined $block->request) { + $block->set_value("request", "GET /t"); + } + +}); + + +log_level("debug"); +no_long_string(); +#no_diff(); +run_tests(); + +__DATA__ + +=== TEST 1: receiveany returns anything once socket receives +--- config + server_tokens off; + location = /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + assert(sock:connect("127.0.0.1", ngx.var.port)) + local req = { + 'GET /foo HTTP/1.0\r\n', + 'Host: localhost\r\n', + 'Connection: close\r\n\r\n', + } + local ok, err = sock:send(req) + if not ok then + ngx.say("send request failed: ", err) + return + end + + + -- skip http header + while true do + local data, err, _ = sock:receive('*l') + if err then + ngx.say('unexpected error occurs when receiving http head: ' .. err) + return + end + if #data == 0 then -- read last line of head + break + end + end + + -- receive http body + while true do + local data, err = sock:receiveany(1024) + if err then + if err ~= 'closed' then + ngx.say('unexpected err: ', err) + end + break + end + ngx.say(data) + end + + sock:close() + } + } + + location = /foo { + content_by_lua_block { + local resp = { + '1', + 'hello', + } + + local length = 0 + for _, v in ipairs(resp) do + length = length + #v + end + + -- flush http header + ngx.header['Content-Length'] = length + ngx.flush(true) + ngx.sleep(0.01) + + -- send http body bytes by bytes + for _, v in ipairs(resp) do + ngx.print(v) + ngx.flush(true) + ngx.sleep(0.01) + end + } + } + +--- response_body +1 +h +e +l +l +o +--- grep_error_log eval +qr/lua tcp socket read any/ +--- grep_error_log_out +lua tcp socket read any +lua tcp socket read any +lua tcp socket read any +lua tcp socket read any +lua tcp socket read any +lua tcp socket read any +lua tcp socket read any From 9112536b153707c71da234d8d8f1d78156318337 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 20 May 2018 14:12:22 -0700 Subject: [PATCH 239/848] api: bumped API version to 0.10.14. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index c7c0e6b533..3f75a4df7a 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10013 +#define ngx_http_lua_version 10014 typedef struct { From 6bbfffdece622a4969ad1323c7cb53f023f2a12f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 20 May 2018 14:57:50 -0700 Subject: [PATCH 240/848] tests: we failed to enable master process when configuring multiple workers. --- t/134-worker-count-5.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/134-worker-count-5.t b/t/134-worker-count-5.t index b257c12e61..fbacad3fb7 100644 --- a/t/134-worker-count-5.t +++ b/t/134-worker-count-5.t @@ -3,7 +3,7 @@ use Test::Nginx::Socket::Lua; #worker_connections(1014); -#master_on(); +master_on(); workers(5); #log_level('warn'); From 70b843f82581271c87535394d247033676a84270 Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 21 May 2018 12:12:02 +0800 Subject: [PATCH 241/848] tests: upgraded openssl version to 1.1.0h. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index fcf369e984..f51ad0ce27 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,7 @@ env: - TEST_NGINX_SLEEP=0.006 matrix: - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2n OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0g OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0h OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" # to $OPENSSL_OPT. From 55743aeba3075b34a250380b32bad6366eae6c30 Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Thu, 24 May 2018 13:15:46 -0700 Subject: [PATCH 242/848] bugfix: when variable is cacheable and indexed, setting the request "Host" header should clear any existing cached $host variable value so that subsequent reads return the expected result. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_common.h | 2 ++ src/ngx_http_lua_headers_in.c | 10 +++++++++- src/ngx_http_lua_module.c | 6 ++++++ t/028-req-header.t | 24 +++++++++++++++++++++++- 4 files changed, 40 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 01ef2bec8b..4fcbc9ee26 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -227,6 +227,8 @@ struct ngx_http_lua_main_conf_s { ngx_int_t busy_buf_ptr_count; #endif + ngx_int_t host_var_index; + unsigned requires_header_filter:1; unsigned requires_body_filter:1; unsigned requires_capture_filter:1; diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 4852b2f692..c52cd13db1 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -432,10 +432,14 @@ static ngx_int_t ngx_http_set_host_header(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, ngx_str_t *value) { - ngx_str_t host; + ngx_str_t host; + ngx_http_lua_main_conf_t *lmcf; + ngx_http_variable_value_t *var; dd("server new value len: %d", (int) value->len); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + if (value->len) { host= *value; @@ -449,6 +453,10 @@ ngx_http_set_host_header(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, r->headers_in.server = *value; } + var = &r->variables[lmcf->host_var_index]; + var->valid = 0; + var->not_found = 0; + return ngx_http_set_builtin_header(r, hv, value); } diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index ae8bc0e36f..51e4676774 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -638,9 +638,15 @@ ngx_http_lua_init(ngx_conf_t *cf) #if !defined(NGX_LUA_NO_FFI_API) || nginx_version >= 1011011 ngx_pool_cleanup_t *cln; #endif + ngx_str_t name = ngx_string("host"); lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); + lmcf->host_var_index = ngx_http_get_variable_index(cf, &name); + if (lmcf->host_var_index == NGX_ERROR) { + return NGX_ERROR; + } + if (ngx_http_lua_prev_cycle != ngx_cycle) { ngx_http_lua_prev_cycle = ngx_cycle; multi_http_blocks = 0; diff --git a/t/028-req-header.t b/t/028-req-header.t index ca92daab55..8ddbb29796 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (2 * blocks() + 43); +plan tests => repeat_each() * (2 * blocks() + 44); #no_diff(); no_long_string(); @@ -2008,3 +2008,25 @@ found 3 headers. --- no_error_log lua exceeding request header limit [error] + + + +=== TEST 61: setting Host header clears cached $host variable +--- config + location /req-header { + # this makes $host indexed and cacheable + set $foo $host; + + content_by_lua_block { + ngx.say(ngx.var.host) + ngx.req.set_header("Host", "new"); + ngx.say(ngx.var.host) + } + } +--- request +GET /req-header +--- response_body +localhost +new +--- no_error_log +[error] From 8ba33dac05584ca320df69459c2e9aabc8b28b47 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 23 Apr 2018 16:08:42 -0700 Subject: [PATCH 243/848] optimize: removed a duplicated include directive in ngx_http_lua_regex.c. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_regex.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 843d1e4e7a..5f74843400 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -15,7 +15,6 @@ #include "ngx_http_lua_regex.h" #include "ngx_http_lua_pcrefix.h" #include "ngx_http_lua_script.h" -#include "ngx_http_lua_pcrefix.h" #include "ngx_http_lua_util.h" From 576a10d246daf81c0ce1b959c50ee807769c01a8 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 1 Jun 2018 15:12:14 -0700 Subject: [PATCH 244/848] doc: mention that ngx.req.set_body_data() and ngx.req.set_body_file() must read the request body. This behavior was introduced in e37b58e372467ea5ca3161c5785f1c2c0d30d27f but the documentation was left unchanged. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 12 ++++++++---- doc/HttpLuaModule.wiki | 12 ++++++++---- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/README.markdown b/README.markdown index b04a763d5f..a6ecd677bd 100644 --- a/README.markdown +++ b/README.markdown @@ -4888,7 +4888,7 @@ This function returns `nil` if 1. the request body has been read into disk temporary files, 1. or the request body has zero size. -If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_body) first (or turned on [lua_need_request_body](#lua_need_request_body) to force this module to read the request body. This is not recommended however). +If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_body) first (or turn on [lua_need_request_body](#lua_need_request_body) to force this module to read the request body. This is not recommended however). If the request body has been read into disk files, try calling the [ngx.req.get_body_file](#ngxreqget_body_file) function instead. @@ -4912,7 +4912,7 @@ Retrieves the file name for the in-file request body data. Returns `nil` if the The returned file is read only and is usually cleaned up by Nginx's memory pool. It should not be manually modified, renamed, or removed in Lua code. -If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_body) first (or turned on [lua_need_request_body](#lua_need_request_body) to force this module to read the request body. This is not recommended however). +If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_body) first (or turn on [lua_need_request_body](#lua_need_request_body) to force this module to read the request body. This is not recommended however). If the request body has been read into memory, try calling the [ngx.req.get_body_data](#ngxreqget_body_data) function instead. @@ -4932,7 +4932,9 @@ ngx.req.set_body_data Set the current request's request body using the in-memory data specified by the `data` argument. -If the current request's request body has not been read, then it will be properly discarded. When the current request's request body has been read into memory or buffered into a disk file, then the old request body's memory will be freed or the disk file will be cleaned up immediately, respectively. +If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_body) first (or turn on [lua_need_request_body](#lua_need_request_body) to force this module to read the request body. This is not recommended however). Additionally, the request body must not have been previously discarded by [ngx.req.discard_body](#ngxreqdiscard_body). + +Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. This function was first introduced in the `v0.3.1rc18` release. @@ -4948,11 +4950,13 @@ ngx.req.set_body_file Set the current request's request body using the in-file data specified by the `file_name` argument. +If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_body) first (or turn on [lua_need_request_body](#lua_need_request_body) to force this module to read the request body. This is not recommended however). Additionally, the request body must not have been previously discarded by [ngx.req.discard_body](#ngxreqdiscard_body). + If the optional `auto_clean` argument is given a `true` value, then this file will be removed at request completion or the next time this function or [ngx.req.set_body_data](#ngxreqset_body_data) are called in the same request. The `auto_clean` is default to `false`. Please ensure that the file specified by the `file_name` argument exists and is readable by an Nginx worker process by setting its permission properly to avoid Lua exception errors. -If the current request's request body has not been read, then it will be properly discarded. When the current request's request body has been read into memory or buffered into a disk file, then the old request body's memory will be freed or the disk file will be cleaned up immediately, respectively. +Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. This function was first introduced in the `v0.3.1rc18` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4b847c20a6..8bdc9a1cdc 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4071,7 +4071,7 @@ This function returns nil if # the request body has been read into disk temporary files, # or the request body has zero size. -If the request body has not been read yet, call [[#ngx.req.read_body|ngx.req.read_body]] first (or turned on [[#lua_need_request_body|lua_need_request_body]] to force this module to read the request body. This is not recommended however). +If the request body has not been read yet, call [[#ngx.req.read_body|ngx.req.read_body]] first (or turn on [[#lua_need_request_body|lua_need_request_body]] to force this module to read the request body. This is not recommended however). If the request body has been read into disk files, try calling the [[#ngx.req.get_body_file|ngx.req.get_body_file]] function instead. @@ -4092,7 +4092,7 @@ Retrieves the file name for the in-file request body data. Returns nildata argument. -If the current request's request body has not been read, then it will be properly discarded. When the current request's request body has been read into memory or buffered into a disk file, then the old request body's memory will be freed or the disk file will be cleaned up immediately, respectively. +If the request body has not been read yet, call [[#ngx.req.read_body|ngx.req.read_body]] first (or turn on [[#lua_need_request_body|lua_need_request_body]] to force this module to read the request body. This is not recommended however). Additionally, the request body must not have been previously discarded by [[#ngx.req.discard_body|ngx.req.discard_body]]. + +Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. This function was first introduced in the v0.3.1rc18 release. @@ -4122,11 +4124,13 @@ See also [[#ngx.req.set_body_file|ngx.req.set_body_file]]. Set the current request's request body using the in-file data specified by the file_name argument. +If the request body has not been read yet, call [[#ngx.req.read_body|ngx.req.read_body]] first (or turn on [[#lua_need_request_body|lua_need_request_body]] to force this module to read the request body. This is not recommended however). Additionally, the request body must not have been previously discarded by [[#ngx.req.discard_body|ngx.req.discard_body]]. + If the optional auto_clean argument is given a true value, then this file will be removed at request completion or the next time this function or [[#ngx.req.set_body_data|ngx.req.set_body_data]] are called in the same request. The auto_clean is default to false. Please ensure that the file specified by the file_name argument exists and is readable by an Nginx worker process by setting its permission properly to avoid Lua exception errors. -If the current request's request body has not been read, then it will be properly discarded. When the current request's request body has been read into memory or buffered into a disk file, then the old request body's memory will be freed or the disk file will be cleaned up immediately, respectively. +Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. This function was first introduced in the v0.3.1rc18 release. From e94f2e5d64daa45ff396e262d8dab8e56f5f10e0 Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Wed, 18 Jul 2018 16:21:09 -0700 Subject: [PATCH 245/848] bugfix: fixed segfault in NGINX core >= 1.15.0 when init_worker_by_lua* is used. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_initworkerby.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 4a722a062d..2a82fcb9f8 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -25,6 +25,7 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) void *cur, *prev; ngx_uint_t i; ngx_conf_t conf; + ngx_conf_file_t cf_file; ngx_cycle_t *fake_cycle; ngx_module_t **modules; ngx_open_file_t *file, *ofile; @@ -166,6 +167,10 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) conf.pool = fake_cycle->pool; conf.log = cycle->log; + ngx_memzero(&cf_file, sizeof(cf_file)); + cf_file.file.name = cycle->conf_file; + conf.conf_file = &cf_file; + http_ctx.loc_conf = ngx_pcalloc(conf.pool, sizeof(void *) * ngx_http_max_module); if (http_ctx.loc_conf == NULL) { From f64ec8c9056437bb1b822fc4e99f05f784d36e32 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Thu, 20 Sep 2018 23:46:24 +0800 Subject: [PATCH 246/848] bugfix: tcpsock:setkeepalive: worker processes might take too long to gracefully shut down when the keep alive connections take a long max idle time. Now we avoid putting any new connections into the pool when nginx is already shutting down. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_socket_tcp.c | 10 ++++ t/157-socket-keepalive-hup.t | 91 +++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 t/157-socket-keepalive-hup.t diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 0717c46d50..54b749326e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4690,6 +4690,14 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) return 2; } + if (ngx_terminate || ngx_exiting) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "lua tcp socket set keepalive while process exiting, " + "closing connection %p", c); + + goto finalize; + } + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, "lua tcp socket set keepalive: saving connection %p", c); @@ -4854,6 +4862,8 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) } } +finalize: + #if 1 ngx_http_lua_socket_tcp_finalize(r, u); #endif diff --git a/t/157-socket-keepalive-hup.t b/t/157-socket-keepalive-hup.t new file mode 100644 index 0000000000..5f59ec1c45 --- /dev/null +++ b/t/157-socket-keepalive-hup.t @@ -0,0 +1,91 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_CHECK_LEAK}) { + $SkipReason = "unavailable for the hup tests"; + + } else { + $ENV{TEST_NGINX_USE_HUP} = 1; + undef $ENV{TEST_NGINX_USE_STAP}; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 8); + +#no_diff(); +no_long_string(); + +worker_connections(1024); +run_tests(); + +__DATA__ + +=== TEST 1: exiting +--- config + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local f, err = io.open("t/servroot/logs/nginx.pid", "r") + if not f then + ngx.say("failed to open nginx.pid: ", err) + return + end + + local pid = f:read() + -- ngx.say("master pid: [", pid, "]") + + f:close() + + local i = 0 + local port = ngx.var.port + + local function f(premature) + print("timer prematurely expired: ", premature) + + local sock = ngx.socket.tcp() + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + print("failed to connect: ", err) + return + end + + local ok, err = sock:setkeepalive() + if not ok then + print("failed to setkeepalive: ", err) + return + end + + print("setkeepalive successfully") + end + local ok, err = ngx.timer.at(3, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + ngx.say("registered timer") + os.execute("kill -HUP " .. pid) + } + } +--- request +GET /t + +--- response_body +registered timer + +--- wait: 0.3 +--- no_error_log +[error] +[alert] +[crit] +--- error_log +timer prematurely expired: true +setkeepalive successfully +lua tcp socket set keepalive while process exiting, closing connection From 7286812116940216344ade33722c49ae47037605 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Wed, 26 Jul 2017 15:51:38 +0800 Subject: [PATCH 247/848] feature: added support for ARM64 (Aarch64). now we require OpenResty's LuaJIT branch to work properly on ARM64 since we make use of the thread.exdata Lua API. On architectures other than ARM64, the thread.exdata API also saves the per-request global env table and closure objects for each light thread, which gives a nice ~10% speedup for the simplest Lua handler location loaded by wrk over HTTP 1.1. We now use proper userdata instead of lightuserdata for the ngx.shared.DICT Lua objects. we also mask off the bits higher than 47-bit of lightuserdata keys based on C static variable addresses. We also implemented the pure C API for the ndk.* API (to be used in lua-resty-core). Thanks Dejiang Zhu and Zexuan Luo for the development work of this patch. Thanks Cloudflare for sponsoring this work. Signed-off-by: Yichun Zhang (agentzh) --- .travis.yml | 4 +- src/ngx_http_lua_accessby.c | 2 + src/ngx_http_lua_balancer.c | 3 + src/ngx_http_lua_bodyfilterby.c | 50 +- src/ngx_http_lua_bodyfilterby.h | 2 +- src/ngx_http_lua_cache.c | 17 +- src/ngx_http_lua_clfactory.c | 45 +- src/ngx_http_lua_common.h | 34 +- src/ngx_http_lua_contentby.c | 2 + src/ngx_http_lua_control.c | 3 +- src/ngx_http_lua_coroutine.c | 18 +- src/ngx_http_lua_headerfilterby.c | 3 +- src/ngx_http_lua_headers.c | 9 +- src/ngx_http_lua_logby.c | 3 +- src/ngx_http_lua_ndk.c | 41 ++ src/ngx_http_lua_regex.c | 9 +- src/ngx_http_lua_rewriteby.c | 2 + src/ngx_http_lua_setby.c | 34 +- src/ngx_http_lua_setby.h | 2 +- src/ngx_http_lua_shdict.c | 44 +- src/ngx_http_lua_socket_tcp.c | 60 ++- src/ngx_http_lua_socket_udp.c | 12 +- src/ngx_http_lua_ssl_certby.c | 2 + src/ngx_http_lua_ssl_session_fetchby.c | 2 + src/ngx_http_lua_ssl_session_storeby.c | 3 + src/ngx_http_lua_timer.c | 23 +- src/ngx_http_lua_uthread.c | 3 +- src/ngx_http_lua_util.c | 92 +++- src/ngx_http_lua_util.h | 10 + t/001-set.t | 17 +- t/014-bugs.t | 33 +- t/041-header-filter.t | 12 +- t/073-backtrace.t | 6 +- t/075-logby.t | 8 +- t/081-bytecode.t | 8 +- t/094-uthread-exit.t | 10 +- t/095-uthread-exec.t | 2 +- t/096-uthread-redirect.t | 2 +- t/106-timer.t | 4 +- t/130-internal-api.t | 17 +- t/140-ssl-c-api.t | 23 +- t/143-ssl-session-fetch.t | 105 ++++ t/152-timer-every.t | 4 +- t/157-global-var.t | 510 ++++++++++++++++++ .../ngx_http_lua_fake_shm_module.c | 22 +- util/build.sh | 4 +- 46 files changed, 1106 insertions(+), 215 deletions(-) create mode 100644 t/157-global-var.t diff --git a/.travis.yml b/.travis.yml index f51ad0ce27..8c8e3a757f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -72,7 +72,7 @@ install: - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - git clone https://github.com/openresty/openresty-devel-utils.git - git clone https://github.com/openresty/mockeagain.git - - git clone https://github.com/openresty/lua-cjson.git + - git clone https://github.com/openresty/lua-cjson.git lua-cjson - git clone https://github.com/openresty/lua-upstream-nginx-module.git ../lua-upstream-nginx-module - git clone https://github.com/openresty/echo-nginx-module.git ../echo-nginx-module - git clone https://github.com/openresty/nginx-eval-module.git ../nginx-eval-module @@ -89,7 +89,7 @@ install: - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git + - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 before_script: - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;' diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 56bf0fa0ad..f587806336 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -261,9 +261,11 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) /* move code closure to new coroutine */ lua_xmove(L, co, 1); +#ifndef OPENRESTY_LUAJIT /* set closure's env table to new coroutine's globals table */ ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* save nginx request in coroutine globals table */ ngx_http_lua_set_req(co, r); diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index fdf2af31a7..3ecd592917 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -362,6 +362,8 @@ ngx_http_lua_balancer_by_chunk(lua_State *L, ngx_http_request_t *r) /* init nginx context in Lua VM */ ngx_http_lua_set_req(L, r); + +#ifndef OPENRESTY_LUAJIT ngx_http_lua_create_new_globals_table(L, 0 /* narr */, 1 /* nrec */); /* {{{ make new env inheriting main thread's globals table */ @@ -372,6 +374,7 @@ ngx_http_lua_balancer_by_chunk(lua_State *L, ngx_http_request_t *r) /* }}} */ lua_setfenv(L, -2); /* set new running env for the code closure */ +#endif /* OPENRESTY_LUAJIT */ lua_pushcfunction(L, ngx_http_lua_traceback); lua_insert(L, 1); /* put it under chunk and args */ diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 2b3c38f7ce..f3af14c164 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -32,10 +32,6 @@ static void ngx_http_lua_body_filter_by_lua_env(lua_State *L, static ngx_http_output_body_filter_pt ngx_http_next_body_filter; -/* key for the ngx_chain_t *in pointer in the Lua thread */ -#define ngx_http_lua_chain_key "__ngx_cl" - - /** * Set environment table for the given code closure. * @@ -51,12 +47,14 @@ static void ngx_http_lua_body_filter_by_lua_env(lua_State *L, ngx_http_request_t *r, ngx_chain_t *in) { - /* set nginx request pointer to current lua thread's globals table */ + ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_set_req(L, r); - lua_pushlightuserdata(L, in); - lua_setglobal(L, ngx_http_lua_chain_key); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + lmcf->body_filter_chain = in; +#ifndef OPENRESTY_LUAJIT /** * we want to create empty environment for current script * @@ -79,6 +77,7 @@ ngx_http_lua_body_filter_by_lua_env(lua_State *L, ngx_http_request_t *r, /* }}} */ lua_setfenv(L, -2); /* set new running env for the code closure */ +#endif /* OPENRESTY_LUAJIT */ } @@ -236,8 +235,8 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) ngx_int_t rc; uint16_t old_context; ngx_http_cleanup_t *cln; - lua_State *L; ngx_chain_t *out; + ngx_http_lua_main_conf_t *lmcf; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua body filter for user lua code, uri \"%V\"", &r->uri); @@ -299,11 +298,8 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) return NGX_ERROR; } - L = ngx_http_lua_get_lua_vm(r, ctx); - - lua_getglobal(L, ngx_http_lua_chain_key); - out = lua_touserdata(L, -1); - lua_pop(L, 1); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + out = lmcf->body_filter_chain; if (in == out) { return ngx_http_next_body_filter(r, in); @@ -345,7 +341,7 @@ ngx_http_lua_body_filter_init(void) int -ngx_http_lua_body_filter_param_get(lua_State *L) +ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r) { u_char *data, *p; size_t size; @@ -354,6 +350,8 @@ ngx_http_lua_body_filter_param_get(lua_State *L) int idx; ngx_chain_t *in; + ngx_http_lua_main_conf_t *lmcf; + idx = luaL_checkint(L, 2); dd("index: %d", idx); @@ -363,8 +361,8 @@ ngx_http_lua_body_filter_param_get(lua_State *L) return 1; } - lua_getglobal(L, ngx_http_lua_chain_key); - in = lua_touserdata(L, -1); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + in = lmcf->body_filter_chain; if (idx == 2) { /* asking for the eof argument */ @@ -442,6 +440,8 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, ngx_chain_t *cl; ngx_chain_t *in; + ngx_http_lua_main_conf_t *lmcf; + idx = luaL_checkint(L, 2); dd("index: %d", idx); @@ -450,13 +450,13 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, return luaL_error(L, "bad index: %d", idx); } + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + if (idx == 2) { /* overwriting the eof flag */ last = lua_toboolean(L, 3); - lua_getglobal(L, ngx_http_lua_chain_key); - in = lua_touserdata(L, -1); - lua_pop(L, 1); + in = lmcf->body_filter_chain; if (last) { ctx->seen_last_in_filter = 1; @@ -521,9 +521,7 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, case LUA_TNIL: /* discard the buffers */ - lua_getglobal(L, ngx_http_lua_chain_key); /* key val */ - in = lua_touserdata(L, -1); - lua_pop(L, 1); + in = lmcf->body_filter_chain; last = 0; @@ -557,9 +555,7 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, lua_typename(L, type)); } - lua_getglobal(L, ngx_http_lua_chain_key); - in = lua_touserdata(L, -1); - lua_pop(L, 1); + in = lmcf->body_filter_chain; last = 0; @@ -625,8 +621,8 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, } } - lua_pushlightuserdata(L, cl); - lua_setglobal(L, ngx_http_lua_chain_key); + lmcf->body_filter_chain = cl; + return 0; } diff --git a/src/ngx_http_lua_bodyfilterby.h b/src/ngx_http_lua_bodyfilterby.h index 6a4b306d64..b108202faa 100644 --- a/src/ngx_http_lua_bodyfilterby.h +++ b/src/ngx_http_lua_bodyfilterby.h @@ -21,7 +21,7 @@ ngx_int_t ngx_http_lua_body_filter_inline(ngx_http_request_t *r, ngx_chain_t *in); ngx_int_t ngx_http_lua_body_filter_file(ngx_http_request_t *r, ngx_chain_t *in); -int ngx_http_lua_body_filter_param_get(lua_State *L); +int ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r); int ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx); diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index 5ea3069246..3e0f209f65 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -35,11 +35,14 @@ static ngx_int_t ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, const char *key) { +#ifndef OPENRESTY_LUAJIT int rc; u_char *err; +#endif /* get code cache table */ - lua_pushlightuserdata(L, &ngx_http_lua_code_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + code_cache_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* sp++ */ dd("Code cache table to load: %p", lua_topointer(L, -1)); @@ -52,6 +55,10 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, lua_getfield(L, -1, key); /* sp++ */ if (lua_isfunction(L, -1)) { +#ifdef OPENRESTY_LUAJIT + lua_remove(L, -2); /* sp-- */ + return NGX_OK; +#else /* call closure factory to gen new closure */ rc = lua_pcall(L, 0, 1, 0); if (rc == 0) { @@ -73,6 +80,7 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, key, err); lua_pop(L, 2); return NGX_ERROR; +#endif /* OPENRESTY_LUAJIT */ } dd("Value associated with given key in code cache table is not code " @@ -102,10 +110,13 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, static ngx_int_t ngx_http_lua_cache_store_code(lua_State *L, const char *key) { +#ifndef OPENRESTY_LUAJIT int rc; +#endif /* get code cache table */ - lua_pushlightuserdata(L, &ngx_http_lua_code_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + code_cache_key)); lua_rawget(L, LUA_REGISTRYINDEX); dd("Code cache table to store: %p", lua_topointer(L, -1)); @@ -121,12 +132,14 @@ ngx_http_lua_cache_store_code(lua_State *L, const char *key) /* remove cache table, leave closure factory at top of stack */ lua_pop(L, 1); /* closure */ +#ifndef OPENRESTY_LUAJIT /* call closure factory to generate new closure */ rc = lua_pcall(L, 0, 1, 0); if (rc != 0) { dd("Error: failed to call closure factory!!"); return NGX_ERROR; } +#endif return NGX_OK; } diff --git a/src/ngx_http_lua_clfactory.c b/src/ngx_http_lua_clfactory.c index 041f0466e6..754ed8d2f1 100644 --- a/src/ngx_http_lua_clfactory.c +++ b/src/ngx_http_lua_clfactory.c @@ -15,11 +15,13 @@ #include "ngx_http_lua_clfactory.h" +#ifndef OPENRESTY_LUAJIT #define CLFACTORY_BEGIN_CODE "return function() " #define CLFACTORY_BEGIN_SIZE (sizeof(CLFACTORY_BEGIN_CODE) - 1) #define CLFACTORY_END_CODE "\nend" #define CLFACTORY_END_SIZE (sizeof(CLFACTORY_END_CODE) - 1) +#endif /* @@ -59,6 +61,7 @@ * length(Instruction) = 4 or 8 * little endian or big endian */ +#ifndef OPENRESTY_LUAJIT #define LUA_LITTLE_ENDIAN_4BYTES_CODE \ "\x24\x00\x00\x00\x1e\x00\x00\x01\x1e\x00\x80\x00" #define LUA_LITTLE_ENDIAN_8BYTES_CODE \ @@ -75,6 +78,7 @@ #define LUA_BIG_ENDIAN_8BYTES_CODE_LEN (8 + 8 + 8) #define LUAC_HEADERSIZE 12 #define LUAC_VERSION 0x51 +#endif /* OPENRESTY_LUAJIT */ /* @@ -147,6 +151,7 @@ * --------------------- */ +#ifndef OPENRESTY_LUAJIT #define POS_SOURCE_STR_LEN LUAC_HEADERSIZE #define POS_START_LINE (POS_SOURCE_STR_LEN + sizeof(size_t)) #define POS_LAST_LINE (POS_START_LINE + sizeof(int)) @@ -160,6 +165,7 @@ (POS_BYTECODE + LUA_LITTLE_ENDIAN_8BYTES_CODE_LEN \ + sizeof(int) + sizeof(int)) #define MAX_END_CODE_SIZE (sizeof(int) + sizeof(int) + sizeof(int)) +#endif /* OPENRESTY_LUAJIT */ /* * taken from chaoslawful: @@ -225,6 +231,7 @@ /* bytecode for luajit 2.0 */ +#ifndef OPENRESTY_LUAJIT #define LJ20_LITTLE_ENDIAN_CODE_STRIPPED \ "\x14\x03\x00\x01\x00\x01\x00\x03" \ "\x31\x00\x00\x00\x30\x00\x00\x80\x48\x00\x02\x00" \ @@ -275,6 +282,7 @@ #define LJ21_BCDUMP_VERSION 2 #define LJ20_BCDUMP_VERSION 1 #define LJ_SIGNATURE "\x1b\x4c\x4a" +#endif /* OPENRESTY_LUAJIT */ typedef enum { @@ -292,10 +300,11 @@ enum { typedef struct { ngx_http_lua_clfactory_file_type_e file_type; - int sent_begin; - int sent_end; int extraline; FILE *f; +#ifndef OPENRESTY_LUAJIT + int sent_begin; + int sent_end; size_t begin_code_len; size_t end_code_len; size_t rest_len; @@ -307,13 +316,16 @@ typedef struct { char *ptr; char str[MAX_END_CODE_SIZE]; } end_code; +#endif /* OPENRESTY_LUAJIT */ char buff[NGX_LUA_READER_BUFSIZE]; } ngx_http_lua_clfactory_file_ctx_t; typedef struct { +#ifndef OPENRESTY_LUAJIT int sent_begin; int sent_end; +#endif const char *s; size_t size; } ngx_http_lua_clfactory_buffer_ctx_t; @@ -325,9 +337,12 @@ static int ngx_http_lua_clfactory_errfile(lua_State *L, const char *what, int fname_index); static const char *ngx_http_lua_clfactory_getS(lua_State *L, void *ud, size_t *size); +#ifndef OPENRESTY_LUAJIT static long ngx_http_lua_clfactory_file_size(FILE *f); +#endif +#ifndef OPENRESTY_LUAJIT int ngx_http_lua_clfactory_bytecode_prepare(lua_State *L, ngx_http_lua_clfactory_file_ctx_t *lf, int fname_index) @@ -593,6 +608,7 @@ ngx_http_lua_clfactory_bytecode_prepare(lua_State *L, return LUA_ERRFILE; } +#endif /* OPENRESTY_LUAJIT */ ngx_int_t @@ -612,10 +628,12 @@ ngx_http_lua_clfactory_loadfile(lua_State *L, const char *filename) lf.extraline = 0; lf.file_type = NGX_LUA_TEXT_FILE; +#ifndef OPENRESTY_LUAJIT lf.begin_code.ptr = CLFACTORY_BEGIN_CODE; lf.begin_code_len = CLFACTORY_BEGIN_SIZE; lf.end_code.ptr = CLFACTORY_END_CODE; lf.end_code_len = CLFACTORY_END_SIZE; +#endif lua_pushfstring(L, "@%s", filename); @@ -683,20 +701,27 @@ ngx_http_lua_clfactory_loadfile(lua_State *L, const char *filename) /* skip eventual `#!...' */ } +#ifndef OPENRESTY_LUAJIT status = ngx_http_lua_clfactory_bytecode_prepare(L, &lf, fname_index); if (status != 0) { return status; } +#endif lf.extraline = 0; } +#ifndef OPENRESTY_LUAJIT if (lf.file_type == NGX_LUA_TEXT_FILE) { ungetc(c, lf.f); } lf.sent_begin = lf.sent_end = 0; + +#else + ungetc(c, lf.f); +#endif status = lua_load(L, ngx_http_lua_clfactory_getF, &lf, lua_tostring(L, -1)); @@ -725,8 +750,10 @@ ngx_http_lua_clfactory_loadbuffer(lua_State *L, const char *buff, ls.s = buff; ls.size = size; +#ifndef OPENRESTY_LUAJIT ls.sent_begin = 0; ls.sent_end = 0; +#endif return lua_load(L, ngx_http_lua_clfactory_getS, &ls, name); } @@ -735,7 +762,9 @@ ngx_http_lua_clfactory_loadbuffer(lua_State *L, const char *buff, static const char * ngx_http_lua_clfactory_getF(lua_State *L, void *ud, size_t *size) { +#ifndef OPENRESTY_LUAJIT char *buf; +#endif size_t num; ngx_http_lua_clfactory_file_ctx_t *lf; @@ -748,6 +777,7 @@ ngx_http_lua_clfactory_getF(lua_State *L, void *ud, size_t *size) return "\n"; } +#ifndef OPENRESTY_LUAJIT if (lf->sent_begin == 0) { lf->sent_begin = 1; *size = lf->begin_code_len; @@ -761,12 +791,14 @@ ngx_http_lua_clfactory_getF(lua_State *L, void *ud, size_t *size) return buf; } +#endif /* OPENRESTY_LUAJIT */ num = fread(lf->buff, 1, sizeof(lf->buff), lf->f); dd("fread returned %d", (int) num); if (num == 0) { +#ifndef OPENRESTY_LUAJIT if (lf->sent_end == 0) { lf->sent_end = 1; *size = lf->end_code_len; @@ -780,11 +812,13 @@ ngx_http_lua_clfactory_getF(lua_State *L, void *ud, size_t *size) return buf; } +#endif /* OPENRESTY_LUAJIT */ *size = 0; return NULL; } +#ifndef OPENRESTY_LUAJIT if (lf->file_type == NGX_LUA_BT_LJ) { /* skip the footer(\x00) in luajit */ @@ -800,6 +834,7 @@ ngx_http_lua_clfactory_getF(lua_State *L, void *ud, size_t *size) } } } +#endif /* OPENRESTY_LUAJIT */ *size = num; return lf->buff; @@ -833,19 +868,23 @@ ngx_http_lua_clfactory_getS(lua_State *L, void *ud, size_t *size) { ngx_http_lua_clfactory_buffer_ctx_t *ls = ud; +#ifndef OPENRESTY_LUAJIT if (ls->sent_begin == 0) { ls->sent_begin = 1; *size = CLFACTORY_BEGIN_SIZE; return CLFACTORY_BEGIN_CODE; } +#endif if (ls->size == 0) { +#ifndef OPENRESTY_LUAJIT if (ls->sent_end == 0) { ls->sent_end = 1; *size = CLFACTORY_END_SIZE; return CLFACTORY_END_CODE; } +#endif return NULL; } @@ -857,6 +896,7 @@ ngx_http_lua_clfactory_getS(lua_State *L, void *ud, size_t *size) } +#ifndef OPENRESTY_LUAJIT static long ngx_http_lua_clfactory_file_size(FILE *f) { @@ -882,6 +922,7 @@ ngx_http_lua_clfactory_file_size(FILE *f) return len; } +#endif /* OPENRESTY_LUAJIT */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 4fcbc9ee26..5eb57c69ea 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -17,7 +17,7 @@ #include #include -#include +#include #include #include @@ -140,6 +140,10 @@ typedef struct { #endif +#define ngx_http_lua_lightudata_mask(ludata) \ + ((void *) ((uintptr_t) (&ngx_http_lua_##ludata) & ((1UL << 47) - 1))) + + typedef struct ngx_http_lua_main_conf_s ngx_http_lua_main_conf_t; typedef union ngx_http_lua_srv_conf_u ngx_http_lua_srv_conf_t; @@ -208,9 +212,31 @@ struct ngx_http_lua_main_conf_s { ngx_str_t init_worker_src; ngx_http_lua_balancer_peer_data_t *balancer_peer_data; - /* balancer_by_lua does not support yielding and - * there cannot be any conflicts among concurrent requests, - * thus it is safe to store the peer data in the main conf. + /* neither yielding nor recursion is possible in + * balancer_by_lua*, so there cannot be any races among + * concurrent requests and it is safe to store the peer + * data pointer in the main conf. + */ + + ngx_chain_t *body_filter_chain; + /* neither yielding nor recursion is possible in + * body_filter_by_lua*, so there cannot be any races among + * concurrent requests when storing the chain + * data pointer in the main conf. + */ + + ngx_http_variable_value_t *setby_args; + /* neither yielding nor recursion is possible in + * set_by_lua*, so there cannot be any races among + * concurrent requests when storing the args pointer + * in the main conf. + */ + + size_t setby_nargs; + /* neither yielding nor recursion is possible in + * set_by_lua*, so there cannot be any races among + * concurrent requests when storing the nargs in the + * main conf. */ ngx_uint_t shm_zones_inited; diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index ecd6c0e591..274c9ad33f 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -63,9 +63,11 @@ ngx_http_lua_content_by_chunk(lua_State *L, ngx_http_request_t *r) /* move code closure to new coroutine */ lua_xmove(L, co, 1); +#ifndef OPENRESTY_LUAJIT /* set closure's env table to new coroutine's globals table */ ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* save nginx request in coroutine globals table */ ngx_http_lua_set_req(co, r); diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 6ac2cbf88c..5cd1d64c8e 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -432,7 +432,8 @@ ngx_http_lua_on_abort(lua_State *L) ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx); - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_pushvalue(L, -2); diff --git a/src/ngx_http_lua_coroutine.c b/src/ngx_http_lua_coroutine.c index b790814a2b..99a24235a3 100644 --- a/src/ngx_http_lua_coroutine.c +++ b/src/ngx_http_lua_coroutine.c @@ -104,11 +104,15 @@ ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, coctx->co = co; coctx->co_status = NGX_HTTP_LUA_CO_SUSPENDED; +#ifdef OPENRESTY_LUAJIT + ngx_http_lua_set_req(co, r); +#else /* make new coroutine share globals of the parent coroutine. * NOTE: globals don't have to be separated! */ ngx_http_lua_get_globals_table(L); lua_xmove(L, co, 1); ngx_http_lua_set_globals_table(co); +#endif lua_xmove(vm, L, 1); /* move coroutine from main thread to L */ @@ -288,15 +292,27 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) { const char buf[] = "local keys = {'create', 'yield', 'resume', 'status'}\n" +#ifdef OPENRESTY_LUAJIT + "local get_req = require 'thread.exdata'\n" +#else "local getfenv = getfenv\n" +#endif "for _, key in ipairs(keys) do\n" "local std = coroutine['_' .. key]\n" "local ours = coroutine['__' .. key]\n" "local raw_ctx = ngx._phase_ctx\n" "coroutine[key] = function (...)\n" +#ifdef OPENRESTY_LUAJIT + "local r = get_req()\n" +#else "local r = getfenv(0).__ngx_req\n" - "if r then\n" +#endif + "if r ~= nil then\n" +#ifdef OPENRESTY_LUAJIT + "local ctx = raw_ctx()\n" +#else "local ctx = raw_ctx(r)\n" +#endif /* ignore header and body filters */ "if ctx ~= 0x020 and ctx ~= 0x040 then\n" "return ours(...)\n" diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index b504530b90..a0acd4a114 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -42,9 +42,9 @@ static ngx_http_output_header_filter_pt ngx_http_next_header_filter; static void ngx_http_lua_header_filter_by_lua_env(lua_State *L, ngx_http_request_t *r) { - /* set nginx request pointer to current lua thread's globals table */ ngx_http_lua_set_req(L, r); +#ifndef OPENRESTY_LUAJIT /** * we want to create empty environment for current script * @@ -68,6 +68,7 @@ ngx_http_lua_header_filter_by_lua_env(lua_State *L, ngx_http_request_t *r) /* }}} */ lua_setfenv(L, -2); /* set new running env for the code closure */ +#endif /* OPENRESTY_LUAJIT */ } diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index b833577d14..0ff905b15d 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -442,7 +442,8 @@ ngx_http_lua_ngx_req_get_headers(lua_State *L) lua_createtable(L, 0, count); if (!raw) { - lua_pushlightuserdata(L, &ngx_http_lua_headers_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + headers_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); } @@ -566,7 +567,8 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) lua_createtable(L, 0, count + 2); if (!raw) { - lua_pushlightuserdata(L, &ngx_http_lua_headers_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + headers_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); } @@ -1081,7 +1083,8 @@ ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L) "local new_key = string.gsub(string.lower(key), '_', '-')\n" "if new_key ~= key then return tb[new_key] else return nil end"; - lua_pushlightuserdata(L, &ngx_http_lua_headers_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + headers_metatable_key)); /* metatable for ngx.req.get_headers(_, true) and * ngx.resp.get_headers(_, true) */ diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index 0f1d2f35b3..32d1cba727 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -38,9 +38,9 @@ static ngx_int_t ngx_http_lua_log_by_chunk(lua_State *L, ngx_http_request_t *r); static void ngx_http_lua_log_by_lua_env(lua_State *L, ngx_http_request_t *r) { - /* set nginx request pointer to current lua thread's globals table */ ngx_http_lua_set_req(L, r); +#ifndef OPENRESTY_LUAJIT /** * we want to create empty environment for current script * @@ -64,6 +64,7 @@ ngx_http_lua_log_by_lua_env(lua_State *L, ngx_http_request_t *r) /* }}} */ lua_setfenv(L, -2); /* set new running env for the code closure */ +#endif /* OPENRESTY_LUAJIT */ } diff --git a/src/ngx_http_lua_ndk.c b/src/ngx_http_lua_ndk.c index 24b80b4d36..6344183723 100644 --- a/src/ngx_http_lua_ndk.c +++ b/src/ngx_http_lua_ndk.c @@ -186,6 +186,47 @@ ngx_http_lua_inject_ndk_api(lua_State *L) } +int +ngx_http_lua_ffi_ndk_lookup_directive(const u_char *var_data, + size_t var_len, ndk_set_var_value_pt *func) +{ + *func = ngx_http_lookup_ndk_set_var_directive((u_char *) var_data, var_len); + + if (*func == NULL) { + return NGX_ERROR; + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_ndk_set_var_get(ngx_http_request_t *r, + ndk_set_var_value_pt func, const u_char *arg_data, size_t arg_len, + ngx_http_lua_ffi_str_t *value) +{ + ngx_int_t rc; + ngx_str_t res; + ngx_http_variable_value_t arg; + + ngx_memzero(&arg, sizeof(ngx_http_variable_value_t)); + arg.valid = 1; + + arg.data = (u_char *) arg_data; + arg.len = arg_len; + + rc = func(r, &res, &arg); + + if (rc != NGX_OK) { + return rc; + } + + value->data = res.data; + value->len = res.len; + return NGX_OK; +} + + #endif /* defined(NDK) && NDK */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 5f74843400..8f3373b4d5 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -248,7 +248,8 @@ ngx_http_lua_ngx_re_match_helper(lua_State *L, int wantcaps) dd("server pool %p", lmcf->pool); - lua_pushlightuserdata(L, &ngx_http_lua_regex_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + regex_cache_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* table */ lua_pushliteral(L, "m"); @@ -719,7 +720,8 @@ ngx_http_lua_ngx_re_gmatch(lua_State *L) dd("server pool %p", lmcf->pool); - lua_pushlightuserdata(L, &ngx_http_lua_regex_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + regex_cache_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* table */ lua_pushliteral(L, "m"); @@ -1387,7 +1389,8 @@ ngx_http_lua_ngx_re_sub_helper(lua_State *L, unsigned global) dd("server pool %p", lmcf->pool); - lua_pushlightuserdata(L, &ngx_http_lua_regex_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + regex_cache_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* table */ lua_pushliteral(L, "s"); diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 1bbe87c9b0..08f7566904 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -261,9 +261,11 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) /* move code closure to new coroutine */ lua_xmove(L, co, 1); +#ifndef OPENRESTY_LUAJIT /* set closure's env table to new coroutine's globals table */ ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* save nginx request in coroutine globals table */ ngx_http_lua_set_req(co, r); diff --git a/src/ngx_http_lua_setby.c b/src/ngx_http_lua_setby.c index 2e8762a29d..d4288b50c2 100644 --- a/src/ngx_http_lua_setby.c +++ b/src/ngx_http_lua_setby.c @@ -30,13 +30,6 @@ static void ngx_http_lua_set_by_lua_env(lua_State *L, ngx_http_request_t *r, size_t nargs, ngx_http_variable_value_t *args); -/* keys in Lua thread for fetching args and nargs in set_by_lua* */ - -#define ngx_http_lua_nargs_key "__ngx_nargs" - -#define ngx_http_lua_args_key "__ngx_args" - - ngx_int_t ngx_http_lua_set_by_chunk(lua_State *L, ngx_http_request_t *r, ngx_str_t *val, ngx_http_variable_value_t *args, size_t nargs, ngx_str_t *script) @@ -134,23 +127,24 @@ ngx_http_lua_set_by_chunk(lua_State *L, ngx_http_request_t *r, ngx_str_t *val, int -ngx_http_lua_setby_param_get(lua_State *L) +ngx_http_lua_setby_param_get(lua_State *L, ngx_http_request_t *r) { int idx; int n; ngx_http_variable_value_t *v; + ngx_http_lua_main_conf_t *lmcf; idx = luaL_checkint(L, 2); idx--; - /* get number of args from globals */ - lua_getglobal(L, ngx_http_lua_nargs_key); - n = (int) lua_tointeger(L, -1); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - /* get args from globals */ - lua_getglobal(L, ngx_http_lua_args_key); - v = lua_touserdata(L, -1); + /* get number of args from lmcf */ + n = lmcf->setby_nargs; + + /* get args from lmcf */ + v = lmcf->setby_args; if (idx < 0 || idx > n - 1) { lua_pushnil(L); @@ -178,15 +172,16 @@ static void ngx_http_lua_set_by_lua_env(lua_State *L, ngx_http_request_t *r, size_t nargs, ngx_http_variable_value_t *args) { - /* set nginx request pointer to current lua thread's globals table */ + ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_set_req(L, r); - lua_pushinteger(L, nargs); - lua_setglobal(L, ngx_http_lua_nargs_key); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - lua_pushlightuserdata(L, args); - lua_setglobal(L, ngx_http_lua_args_key); + lmcf->setby_nargs = nargs; + lmcf->setby_args = args; +#ifndef OPENRESTY_LUAJIT /** * we want to create empty environment for current script * @@ -211,6 +206,7 @@ ngx_http_lua_set_by_lua_env(lua_State *L, ngx_http_request_t *r, size_t nargs, /* }}} */ lua_setfenv(L, -2); /* set new running env for the code closure */ +#endif } /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_setby.h b/src/ngx_http_lua_setby.h index da71753eaa..f43eef753e 100644 --- a/src/ngx_http_lua_setby.h +++ b/src/ngx_http_lua_setby.h @@ -7,7 +7,7 @@ ngx_int_t ngx_http_lua_set_by_chunk(lua_State *L, ngx_http_request_t *r, ngx_str_t *val, ngx_http_variable_value_t *args, size_t nargs, ngx_str_t *script); -int ngx_http_lua_setby_param_get(lua_State *L); +int ngx_http_lua_setby_param_get(lua_State *L, ngx_http_request_t *r); #endif /* _NGX_HTTP_LUA_SET_BY_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 5fb7930a87..b017bea658 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -325,6 +325,7 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) ngx_http_lua_shdict_ctx_t *ctx; ngx_uint_t i; ngx_shm_zone_t **zone; + ngx_shm_zone_t **zone_udata; if (lmcf->shdict_zones != NULL) { lua_createtable(L, 0, lmcf->shdict_zones->nelts /* nrec */); @@ -396,7 +397,9 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_createtable(L, 1 /* narr */, 0 /* nrec */); /* table of zone[i] */ - lua_pushlightuserdata(L, zone[i]); /* shared mt key ud */ + zone_udata = lua_newuserdata(L, sizeof(ngx_shm_zone_t *)); + /* shared mt key ud */ + *zone_udata = zone[i]; lua_rawseti(L, -2, SHDICT_USERDATA_INDEX); /* {zone[i]} */ lua_pushvalue(L, -3); /* shared mt key ud mt */ lua_setmetatable(L, -2); /* shared mt key ud */ @@ -431,11 +434,17 @@ static ngx_inline ngx_shm_zone_t * ngx_http_lua_shdict_get_zone(lua_State *L, int index) { ngx_shm_zone_t *zone; + ngx_shm_zone_t **zone_udata; lua_rawgeti(L, index, SHDICT_USERDATA_INDEX); - zone = lua_touserdata(L, -1); + zone_udata = lua_touserdata(L, -1); lua_pop(L, 1); + if (zone_udata == NULL) { + return NULL; + } + + zone = *zone_udata; return zone; } @@ -2209,6 +2218,17 @@ ngx_http_lua_find_zone(u_char *name_data, size_t name_len) #ifndef NGX_LUA_NO_FFI_API +ngx_shm_zone_t * +ngx_http_lua_ffi_shdict_udata_to_zone(void *zone_udata) +{ + if (zone_udata == NULL) { + return NULL; + } + + return *(ngx_shm_zone_t **) zone_udata; +} + + int ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, size_t key_len, int value_type, u_char *str_value_buf, @@ -2225,10 +2245,6 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, ngx_http_lua_shdict_ctx_t *ctx; ngx_http_lua_shdict_node_t *sd; - if (zone == NULL) { - return NGX_ERROR; - } - dd("exptime: %ld", exptime); ctx = zone->data; @@ -2490,10 +2506,6 @@ ngx_http_lua_ffi_shdict_get(ngx_shm_zone_t *zone, u_char *key, ngx_http_lua_shdict_node_t *sd; ngx_str_t value; - if (zone == NULL) { - return NGX_ERROR; - } - *err = NULL; ctx = zone->data; @@ -2636,10 +2648,6 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, u_char *p; ngx_queue_t *queue, *q; - if (zone == NULL) { - return NGX_ERROR; - } - if (init_ttl > 0) { tp = ngx_timeofday(); } @@ -2915,10 +2923,6 @@ ngx_http_lua_ffi_shdict_get_ttl(ngx_shm_zone_t *zone, u_char *key, ngx_http_lua_shdict_ctx_t *ctx; ngx_http_lua_shdict_node_t *sd; - if (zone == NULL) { - return NGX_ERROR; - } - ctx = zone->data; hash = ngx_crc32_short(key, key_len); @@ -2959,10 +2963,6 @@ ngx_http_lua_ffi_shdict_set_expire(ngx_shm_zone_t *zone, u_char *key, ngx_http_lua_shdict_ctx_t *ctx; ngx_http_lua_shdict_node_t *sd; - if (zone == NULL) { - return NGX_ERROR; - } - if (exptime > 0) { tp = ngx_timeofday(); } diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 54b749326e..af7b28921e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -234,7 +234,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_setfield(L, -2, "socket"); /* {{{req socket object metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_req_socket_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + req_socket_metatable_key)); lua_createtable(L, 0 /* narr */, 5 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_receive); @@ -256,7 +257,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* {{{raw req socket object metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_raw_req_socket_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + raw_req_socket_metatable_key)); lua_createtable(L, 0 /* narr */, 6 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_receive); @@ -281,7 +283,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* {{{tcp object metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_tcp_socket_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + tcp_socket_metatable_key)); lua_createtable(L, 0 /* narr */, 12 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_connect); @@ -330,7 +333,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* {{{upstream userdata metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_upstream_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + upstream_udata_metatable_key)); lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ lua_pushcfunction(L, ngx_http_lua_socket_tcp_upstream_destroy); lua_setfield(L, -2, "__gc"); @@ -338,7 +342,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* {{{downstream userdata metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_downstream_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + downstream_udata_metatable_key)); lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ lua_pushcfunction(L, ngx_http_lua_socket_downstream_destroy); lua_setfield(L, -2, "__gc"); @@ -346,7 +351,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* {{{socket pool userdata metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_pool_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + pool_udata_metatable_key)); lua_createtable(L, 0, 1); /* metatable */ lua_pushcfunction(L, ngx_http_lua_socket_shutdown_pool); lua_setfield(L, -2, "__gc"); @@ -354,7 +360,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* {{{socket compiled pattern userdata metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_pattern_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + pattern_udata_metatable_key)); lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ lua_pushcfunction(L, ngx_http_lua_socket_cleanup_compiled_pattern); lua_setfield(L, -2, "__gc"); @@ -364,7 +371,8 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) #if (NGX_HTTP_SSL) /* {{{ssl session userdata metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_ssl_session_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + ssl_session_metatable_key)); lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ lua_pushcfunction(L, ngx_http_lua_ssl_free_session); lua_setfield(L, -2, "__gc"); @@ -412,7 +420,8 @@ ngx_http_lua_socket_tcp(lua_State *L) | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); lua_createtable(L, 5 /* narr */, 1 /* nrec */); - lua_pushlightuserdata(L, &ngx_http_lua_tcp_socket_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + tcp_socket_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); @@ -580,7 +589,8 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) } #if 1 - lua_pushlightuserdata(L, &ngx_http_lua_upstream_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + upstream_udata_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); #endif @@ -1646,7 +1656,8 @@ ngx_http_lua_ssl_handshake_retval_handler(ngx_http_request_t *r, "lua ssl save session: %p", ssl_session); /* set up the __gc metamethod */ - lua_pushlightuserdata(L, &ngx_http_lua_ssl_session_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + ssl_session_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); } @@ -3796,7 +3807,8 @@ ngx_http_lua_socket_tcp_receiveuntil(lua_State *L) return luaL_error(L, "no memory"); } - lua_pushlightuserdata(L, &ngx_http_lua_pattern_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + pattern_udata_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); @@ -4451,10 +4463,12 @@ ngx_http_lua_req_socket(lua_State *L) lua_createtable(L, 2 /* narr */, 3 /* nrec */); /* the object */ if (raw) { - lua_pushlightuserdata(L, &ngx_http_lua_raw_req_socket_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + raw_req_socket_metatable_key)); } else { - lua_pushlightuserdata(L, &ngx_http_lua_req_socket_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + req_socket_metatable_key)); } lua_rawget(L, LUA_REGISTRYINDEX); @@ -4466,7 +4480,8 @@ ngx_http_lua_req_socket(lua_State *L) } #if 1 - lua_pushlightuserdata(L, &ngx_http_lua_downstream_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + downstream_udata_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); #endif @@ -4617,7 +4632,8 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) luaL_checktype(L, 1, LUA_TTABLE); - lua_pushlightuserdata(L, &ngx_http_lua_socket_pool_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_pool_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_rawgeti(L, 1, SOCKET_KEY_INDEX); @@ -4742,7 +4758,8 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) return luaL_error(L, "no memory"); } - lua_pushlightuserdata(L, &ngx_http_lua_pool_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + pool_udata_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); @@ -4896,7 +4913,8 @@ ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, lua_State *L, pc = &u->peer; - lua_pushlightuserdata(L, &ngx_http_lua_socket_pool_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_pool_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* table */ lua_pushvalue(L, key_index); /* key */ lua_rawget(L, -2); @@ -5068,7 +5086,8 @@ ngx_http_lua_socket_free_pool(ngx_log_t *log, ngx_http_lua_socket_pool_t *spool) L = spool->lua_vm; - lua_pushlightuserdata(L, &ngx_http_lua_socket_pool_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_pool_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_pushstring(L, (char *) spool->key); lua_pushnil(L); @@ -5527,7 +5546,8 @@ ngx_http_lua_cleanup_conn_pools(lua_State *L) { ngx_http_lua_socket_pool_t *spool; - lua_pushlightuserdata(L, &ngx_http_lua_socket_pool_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_pool_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* table */ lua_pushnil(L); /* first key */ diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 8927f4118d..cbc32d63ec 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -81,7 +81,8 @@ ngx_http_lua_inject_socket_udp_api(ngx_log_t *log, lua_State *L) lua_setfield(L, -2, "udp"); /* ngx socket */ /* udp socket object metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_socket_udp_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_udp_metatable_key)); lua_createtable(L, 0 /* narr */, 6 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_udp_setpeername); @@ -105,7 +106,8 @@ ngx_http_lua_inject_socket_udp_api(ngx_log_t *log, lua_State *L) /* }}} */ /* udp socket object metatable */ - lua_pushlightuserdata(L, &ngx_http_lua_udp_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + udp_udata_metatable_key)); lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ lua_pushcfunction(L, ngx_http_lua_socket_udp_upstream_destroy); lua_setfield(L, -2, "__gc"); @@ -144,7 +146,8 @@ ngx_http_lua_socket_udp(lua_State *L) | NGX_HTTP_LUA_CONTEXT_SSL_CERT); lua_createtable(L, 3 /* narr */, 1 /* nrec */); - lua_pushlightuserdata(L, &ngx_http_lua_socket_udp_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_udp_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); @@ -263,7 +266,8 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) } #if 1 - lua_pushlightuserdata(L, &ngx_http_lua_udp_udata_metatable_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + udp_udata_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_setmetatable(L, -2); #endif diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 453a5c78e4..622d0a5c1a 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -498,9 +498,11 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) /* move code closure to new coroutine */ lua_xmove(L, co, 1); +#ifndef OPENRESTY_LUAJIT /* set closure's env table to new coroutine's globals table */ ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* save nginx request in coroutine globals table */ ngx_http_lua_set_req(co, r); diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 0b88a87ee8..303efb47aa 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -505,9 +505,11 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) /* move code closure to new coroutine */ lua_xmove(L, co, 1); +#ifndef OPENRESTY_LUAJIT /* set closure's env table to new coroutine's globals table */ ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* save nginx request in coroutine globals table */ ngx_http_lua_set_req(co, r); diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 7609c39b6a..a02f729ced 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -380,6 +380,8 @@ ngx_http_lua_ssl_sess_store_by_chunk(lua_State *L, ngx_http_request_t *r) /* init nginx context in Lua VM */ ngx_http_lua_set_req(L, r); + +#ifndef OPENRESTY_LUAJIT ngx_http_lua_create_new_globals_table(L, 0 /* narr */, 1 /* nrec */); /* {{{ make new env inheriting main thread's globals table */ @@ -390,6 +392,7 @@ ngx_http_lua_ssl_sess_store_by_chunk(lua_State *L, ngx_http_request_t *r) /* }}} */ lua_setfenv(L, -2); /* set new running env for the code closure */ +#endif lua_pushcfunction(L, ngx_http_lua_traceback); lua_insert(L, 1); /* put it under chunk and args */ diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 596b2f77e9..eb10a8fc36 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -220,6 +220,7 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) ngx_http_lua_probe_user_coroutine_create(r, L, co); +#ifndef OPENRESTY_LUAJIT lua_createtable(co, 0, 0); /* the new globals table */ /* co stack: global_tb */ @@ -232,6 +233,7 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) /* co stack: global_tb */ ngx_http_lua_set_globals_table(co); +#endif /* co stack: */ @@ -251,12 +253,15 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) /* L stack: time func [args] thread */ /* co stack: func */ +#ifndef OPENRESTY_LUAJIT ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* co stack: func */ - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* L stack: time func [args] thread coroutines */ @@ -358,7 +363,8 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) ngx_free(ev); } - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); luaL_unref(L, -1, co_ref); @@ -387,6 +393,7 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) co = lua_newthread(vm); +#ifndef OPENRESTY_LUAJIT lua_createtable(co, 0, 0); /* the new globals table */ /* co stack: global_tb */ @@ -399,6 +406,7 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) /* co stack: global_tb */ ngx_http_lua_set_globals_table(co); +#endif /* co stack: */ @@ -418,12 +426,15 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) /* L stack: func [args] thread */ /* co stack: func */ +#ifndef OPENRESTY_LUAJIT ngx_http_lua_get_globals_table(co); lua_setfenv(co, -2); +#endif /* co stack: func */ - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* L stack: func [args] thread coroutines */ @@ -512,7 +523,8 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) /* L stack: func [args] */ - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); luaL_unref(L, -1, co_ref); @@ -692,7 +704,8 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) failed: if (tctx.co_ref && tctx.co) { - lua_pushlightuserdata(tctx.co, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(tctx.co, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(tctx.co, LUA_REGISTRYINDEX); luaL_unref(tctx.co, -1, tctx.co_ref); lua_settop(tctx.co, 0); diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index 8195ec0f82..43517a049d 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -70,7 +70,8 @@ ngx_http_lua_uthread_spawn(lua_State *L) /* anchor the newly created coroutine into the Lua registry */ - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_pushvalue(L, -2); coctx->co_ref = luaL_ref(L, -2); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index f7a537ee96..c6e21f62db 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -171,6 +171,7 @@ ngx_http_lua_set_path(ngx_cycle_t *cycle, lua_State *L, int tab_idx, } +#ifndef OPENRESTY_LUAJIT /** * Create new table and set _G field to itself. * @@ -185,6 +186,7 @@ ngx_http_lua_create_new_globals_table(lua_State *L, int narr, int nrec) lua_pushvalue(L, -1); lua_setfield(L, -2, "_G"); } +#endif /* OPENRESTY_LUAJIT */ static lua_State * @@ -313,11 +315,13 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) base = lua_gettop(L); - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); co = lua_newthread(L); +#ifndef OPENRESTY_LUAJIT /* {{{ inherit coroutine's globals to main thread's globals table * for print() function will try to find tostring() in current * globals table. @@ -332,6 +336,7 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) ngx_http_lua_set_globals_table(co); /* }}} */ +#endif /* OPENRESTY_LUAJIT */ *ref = luaL_ref(L, -2); @@ -356,7 +361,8 @@ ngx_http_lua_del_thread(ngx_http_request_t *r, lua_State *L, ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua deleting light thread"); - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); @@ -657,7 +663,8 @@ ngx_http_lua_init_registry(lua_State *L, ngx_log_t *log) /* {{{ register a table to anchor lua coroutines reliably: * {([int]ref) = [cort]} */ - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_createtable(L, 0, 32 /* nrec */); lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ @@ -668,20 +675,23 @@ ngx_http_lua_init_registry(lua_State *L, ngx_log_t *log) lua_rawset(L, LUA_REGISTRYINDEX); /* create the registry entry for the Lua socket connection pool table */ - lua_pushlightuserdata(L, &ngx_http_lua_socket_pool_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + socket_pool_key)); lua_createtable(L, 0, 8 /* nrec */); lua_rawset(L, LUA_REGISTRYINDEX); #if (NGX_PCRE) /* create the registry entry for the Lua precompiled regex object cache */ - lua_pushlightuserdata(L, &ngx_http_lua_regex_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + regex_cache_key)); lua_createtable(L, 0, 16 /* nrec */); lua_rawset(L, LUA_REGISTRYINDEX); #endif /* {{{ register table to cache user code: * { [(string)cache_key] = } */ - lua_pushlightuserdata(L, &ngx_http_lua_code_cache_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + code_cache_key)); lua_createtable(L, 0, 8 /* nrec */); lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ @@ -695,9 +705,6 @@ ngx_http_lua_init_globals(lua_State *L, ngx_cycle_t *cycle, ngx_log_debug0(NGX_LOG_DEBUG_HTTP, log, 0, "lua initializing lua globals"); - lua_pushlightuserdata(L, cycle); - lua_setglobal(L, "__ngx_cycle"); - #if defined(NDK) && NDK ngx_http_lua_inject_ndk_api(L); #endif /* defined(NDK) && NDK */ @@ -756,6 +763,50 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, lua_setglobal(L, "ngx"); ngx_http_lua_inject_coroutine_api(log, L); + +#ifdef OPENRESTY_LUAJIT + { + int rc; + + const char buf[] = + "local ngx_log = ngx.log\n" + "local ngx_WARN = ngx.WARN\n" + "local type = type\n" + "local ngx_get_phase = ngx.get_phase\n" + "local traceback = require 'debug'.traceback\n" + "local function newindex(table, key, value)\n" + "rawset(table, key, value)\n" + "local phase = ngx_get_phase()\n" + "if phase == 'init_worker' or phase == 'init' then\n" + "return\n" + "end\n" + "ngx_log(ngx_WARN, 'http lua attempting to write to the lua " + "global variable \\'', tostring(key), " + "'\\'\\n', traceback())\n" + "end\n" + "setmetatable(_G, { __newindex = newindex })\n" + ; + + rc = luaL_loadbuffer(L, buf, sizeof(buf) - 1, "=_G write guard"); + + if (rc != 0) { + ngx_log_error(NGX_LOG_ERR, log, 0, + "failed to load Lua code for the _G write guard: ", + "%i: %s", rc, lua_tostring(L, -1)); + + lua_pop(L, 1); + return; + } + + rc = lua_pcall(L, 0, 0, 0); + if (rc != 0) { + ngx_log_error(NGX_LOG_ERR, log, 0, + "failed to run Lua code for the _G write guard: %s", + rc, lua_tostring(L, -1)); + lua_pop(L, 1); + } + } +#endif } @@ -2954,12 +3005,12 @@ ngx_http_lua_param_get(lua_State *L) | NGX_HTTP_LUA_CONTEXT_BODY_FILTER); if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SET)) { - return ngx_http_lua_setby_param_get(L); + return ngx_http_lua_setby_param_get(L, r); } /* ctx->context & (NGX_HTTP_LUA_CONTEXT_BODY_FILTER) */ - return ngx_http_lua_body_filter_param_get(L); + return ngx_http_lua_body_filter_param_get(L, r); } @@ -3156,7 +3207,8 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); inited = 1; @@ -3194,7 +3246,8 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); if (!inited) { - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); inited = 1; } @@ -3223,7 +3276,8 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); if (!inited) { - lua_pushlightuserdata(L, &ngx_http_lua_coroutines_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); inited = 1; } @@ -3778,6 +3832,11 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, *pcln = cln; } +#ifdef OPENRESTY_LUAJIT + /* load FFI library first since cdata needs it */ + luaopen_ffi(L); +#endif + if (lmcf->preload_hooks) { /* register the 3rd-party module's preload hooks */ @@ -4041,7 +4100,12 @@ ngx_http_lua_get_raw_phase_context(lua_State *L) ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; +#ifdef OPENRESTY_LUAJIT + r = lua_getexdata(L); +#else r = lua_touserdata(L, 1); +#endif + if (r == NULL) { return 0; } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 7dcc6f7585..052aa1737e 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -188,7 +188,9 @@ ngx_int_t ngx_http_lua_open_and_stat_file(u_char *name, ngx_chain_t *ngx_http_lua_chain_get_free_buf(ngx_log_t *log, ngx_pool_t *p, ngx_chain_t **free, size_t len); +#ifndef OPENRESTY_LUAJIT void ngx_http_lua_create_new_globals_table(lua_State *L, int narr, int nrec); +#endif int ngx_http_lua_traceback(lua_State *L); @@ -334,6 +336,9 @@ ngx_http_lua_get_lua_vm(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) static ngx_inline ngx_http_request_t * ngx_http_lua_get_req(lua_State *L) { +#ifdef OPENRESTY_LUAJIT + return lua_getexdata(L); +#else ngx_http_request_t *r; lua_getglobal(L, ngx_http_lua_req_key); @@ -341,14 +346,19 @@ ngx_http_lua_get_req(lua_State *L) lua_pop(L, 1); return r; +#endif } static ngx_inline void ngx_http_lua_set_req(lua_State *L, ngx_http_request_t *r) { +#ifdef OPENRESTY_LUAJIT + lua_setexdata(L, (void *) r); +#else lua_pushlightuserdata(L, r); lua_setglobal(L, ngx_http_lua_req_key); +#endif } diff --git a/t/001-set.t b/t/001-set.t index ba8f22cb68..60ab2ff471 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 4); +plan tests => repeat_each() * (blocks() * 3 + 5); #log_level("warn"); no_long_string(); @@ -592,25 +592,30 @@ failed to run set_by_lua*: unknown reason -=== TEST 37: globals get cleared for every single request +=== TEST 37: globals are shared in all requests. --- config location /lua { - set_by_lua $res ' + set_by_lua_block $res { if not foo then foo = 1 else + ngx.log(ngx.INFO, "old foo: ", foo) foo = foo + 1 end return foo - '; + } echo $res; } --- request GET /lua ---- response_body -1 +--- response_body_like chomp +\A[12] +\z --- no_error_log [error] +--- grep_error_log eval: qr/(old foo: \d+|write to the lua global variable '\w+')/ +--- grep_error_log_out eval +["write to the lua global variable 'foo'\n", "old foo: 1\n"] diff --git a/t/014-bugs.t b/t/014-bugs.t index 9aadff0e25..545978ccb6 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -579,7 +579,7 @@ $s -=== TEST 26: unexpected globals sharing by using _G +=== TEST 26: globals sharing by using _G --- config location /test { content_by_lua ' @@ -593,12 +593,12 @@ $s } --- pipelined_requests eval ["GET /test", "GET /test", "GET /test"] ---- response_body eval -["0", "0", "0"] +--- response_body_like eval +[qr/\A[036]\z/, qr/\A[147]\z/, qr/\A[258]\z/] -=== TEST 27: unexpected globals sharing by using _G (set_by_lua*) +=== TEST 27: globals sharing by using _G (set_by_lua*) --- config location /test { set_by_lua $a ' @@ -613,12 +613,12 @@ $s } --- pipelined_requests eval ["GET /test", "GET /test", "GET /test"] ---- response_body eval -["0", "0", "0"] +--- response_body_like eval +[qr/\A[036]\z/, qr/\A[147]\z/, qr/\A[258]\z/] -=== TEST 28: unexpected globals sharing by using _G (log_by_lua*) +=== TEST 28: globals sharing by using _G (log_by_lua*) --- http_config lua_shared_dict log_dict 100k; --- config @@ -633,19 +633,19 @@ $s if _G.t then _G.t = _G.t + 1 else - _G.t = 0 + _G.t = 1 end log_dict:set("cnt", t) '; } --- pipelined_requests eval ["GET /test", "GET /test", "GET /test"] ---- response_body eval -["0", "0", "0"] +--- response_body_like eval +[qr/\A[036]\z/, qr/\A[147]\z/, qr/\A[258]\z/] -=== TEST 29: unexpected globals sharing by using _G (header_filter_by_lua*) +=== TEST 29: globals sharing by using _G (header_filter_by_lua*) --- config location /test { header_filter_by_lua ' @@ -663,12 +663,12 @@ $s } --- pipelined_requests eval ["GET /test", "GET /test", "GET /test"] ---- response_body eval -["0", "0", "0"] +--- response_body_like eval +[qr/\A[036]\z/, qr/\A[147]\z/, qr/\A[258]\z/] -=== TEST 30: unexpected globals sharing by using _G (body_filter_by_lua*) +=== TEST 30: globals sharing by using _G (body_filter_by_lua*) --- config location /test { body_filter_by_lua ' @@ -686,8 +686,9 @@ $s } --- request GET /test ---- response_body -a0 +--- response_body_like eval +qr/\Aa[036] +\z/ --- no_error_log [error] diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 7d8ee74673..5e66e184d7 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -10,7 +10,7 @@ log_level('debug'); repeat_each(2); -plan tests => repeat_each() * 94; +plan tests => repeat_each() * (blocks() * 2 + 13); #no_diff(); #no_long_string(); @@ -416,7 +416,7 @@ lua release ngx.ctx -=== TEST 20: global got cleared for each single request +=== TEST 20: globals are shared by all requests --- config location /lua { set $foo ''; @@ -428,6 +428,7 @@ lua release ngx.ctx if not foo then foo = 1 else + ngx.log(ngx.INFO, "old foo: ", foo) foo = foo + 1 end ngx.var.foo = foo @@ -435,10 +436,13 @@ lua release ngx.ctx } --- request GET /lua ---- response_body -1 +--- response_body_like +^[12]$ --- no_error_log [error] +--- grep_error_log eval: qr/old foo: \d+/ +--- grep_error_log_out eval +["", "old foo: 1\n"] diff --git a/t/073-backtrace.t b/t/073-backtrace.t index 6c5469218d..94dc984d43 100644 --- a/t/073-backtrace.t +++ b/t/073-backtrace.t @@ -37,7 +37,7 @@ GET /lua attempt to call global 'lua_concat' : in function 'bar' :5: in function 'foo' -:7: in function +:7: in main chunk @@ -64,7 +64,7 @@ GET /lua " in function 'error'", ": in function 'bar'", ":5: in function 'foo'", -qr/:7: in function /, +qr/:7: in main chunk/, ] @@ -125,7 +125,7 @@ probe process("$LIBLUA_PATH").function("lua_concat") { :63: in function 'func16' :67: in function 'func17' :71: in function 'func18' -:74: in function +:74: in main chunk diff --git a/t/075-logby.t b/t/075-logby.t index 520c62e314..84c7c466ef 100644 --- a/t/075-logby.t +++ b/t/075-logby.t @@ -220,7 +220,7 @@ failed to run log_by_lua*: unknown reason -=== TEST 11: globals get cleared for every single request +=== TEST 11: globals sharing --- config location /lua { echo ok; @@ -228,6 +228,7 @@ failed to run log_by_lua*: unknown reason if not foo then foo = 1 else + ngx.log(ngx.INFO, "old foo: ", foo) foo = foo + 1 end ngx.log(ngx.WARN, "foo = ", foo) @@ -237,8 +238,9 @@ failed to run log_by_lua*: unknown reason GET /lua --- response_body ok ---- error_log -foo = 1 +--- grep_error_log eval: qr/old foo: \d+/ +--- grep_error_log_out eval +["", "old foo: 1\n"] diff --git a/t/081-bytecode.t b/t/081-bytecode.t index cb50e945d3..29b2f758ab 100644 --- a/t/081-bytecode.t +++ b/t/081-bytecode.t @@ -85,7 +85,7 @@ __DATA__ --- response_body error --- error_log eval -qr/failed to load external Lua file ".*?test\.lua": bad byte-code header/ +qr/failed to load external Lua file ".*?test\.lua": .* cannot load incompatible bytecode/ @@ -121,7 +121,7 @@ qr/failed to load external Lua file ".*?test\.lua": bad byte-code header/ --- response_body error --- error_log -bytecode format version unsupported +cannot load incompatible bytecode @@ -320,7 +320,7 @@ ngx.status = 201 ngx.say("hello from Lua!") --- config location = /t { content_by_lua_block { - local f = assert(loadstring("a = a and a + 1 or 1 ngx.say('a = ', a)", "=code")) + local f = assert(loadstring("local a = 1 ngx.say('a = ', a)", "=code")) local bc = string.dump(f) local f = assert(io.open("t/servroot/html/a.luac", "w")) f:write(bc) @@ -349,7 +349,7 @@ a = 1 --- config location = /t { content_by_lua_block { - local f = assert(loadstring("a = a and a + 1 or 1 ngx.say('a = ', a)", "=code")) + local f = assert(loadstring("local a = 1 ngx.say('a = ', a)", "=code")) local bc = string.dump(f, true) local f = assert(io.open("t/servroot/html/a.luac", "w")) f:write(bc) diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index 2f7d9ba731..0b31a800b8 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -1059,7 +1059,7 @@ after location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -1315,7 +1315,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.location.capture_multi{ {"/echo"}, {"/sleep"} @@ -1407,7 +1407,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(444) @@ -1490,7 +1490,7 @@ free request location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(408) @@ -1573,7 +1573,7 @@ free request location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(499) diff --git a/t/095-uthread-exec.t b/t/095-uthread-exec.t index 56156c48a8..ad0c1888a0 100644 --- a/t/095-uthread-exec.t +++ b/t/095-uthread-exec.t @@ -348,7 +348,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.location.capture("/sleep") ngx.say("end") end diff --git a/t/096-uthread-redirect.t b/t/096-uthread-redirect.t index 003c6423de..73b2bf1eec 100644 --- a/t/096-uthread-redirect.t +++ b/t/096-uthread-redirect.t @@ -194,7 +194,7 @@ free request location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.location.capture_multi{ {"/echo"}, {"/sleep"} diff --git a/t/106-timer.t b/t/106-timer.t index 3e4741e50c..fa25e18795 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -76,7 +76,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:4[4-9]|5[0-6]) -=== TEST 2: separated global env +=== TEST 2: globals are shared --- config location /t { content_by_lua ' @@ -104,7 +104,7 @@ F(ngx_http_lua_timer_handler) { --- response_body registered timer -foo = nil +foo = 3 --- wait: 0.1 --- no_error_log diff --git a/t/130-internal-api.t b/t/130-internal-api.t index eba0980f8f..2158e878be 100644 --- a/t/130-internal-api.t +++ b/t/130-internal-api.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 3; +plan tests => repeat_each() * blocks() * 3; #no_diff(); no_long_string(); @@ -19,12 +19,7 @@ run_tests(); __DATA__ -=== TEST 1: __ngx_req and __ngx_cycle ---- http_config - init_by_lua ' - my_cycle = __ngx_cycle - '; - +=== TEST 1: req --- config location = /t { content_by_lua ' @@ -32,18 +27,14 @@ __DATA__ local function tonum(ud) return tonumber(ffi.cast("uintptr_t", ud)) end - ngx.say(string.format("init: cycle=%#x", tonum(my_cycle))) - ngx.say(string.format("content cycle=%#x", tonum(__ngx_cycle))) - ngx.say(string.format("content req=%#x", tonum(__ngx_req))) + ngx.say(string.format("content req=%#x", tonum(exdata()))) '; } --- request GET /t --- response_body_like chop -^init: cycle=(0x[a-f0-9]{4,}) -content cycle=\1 -content req=0x[a-f0-9]{4,} +^content req=0x[a-f0-9]{4,} $ --- no_error_log [error] diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 8734d1477d..a77f5d4c05 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -10,6 +10,7 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); + } else { plan tests => repeat_each() * (blocks() * 5 + 1); } @@ -68,7 +69,7 @@ _EOC_ my $http_config = $block->http_config || ''; $http_config .= <<'_EOC_'; -lua_package_path "$prefix/html/?.lua;;"; +lua_package_path "$prefix/html/?.lua;../lua-resty-core/lib/?.lua;;"; _EOC_ $block->set_value("http_config", $http_config); }); @@ -91,8 +92,8 @@ __DATA__ local errmsg = ffi.new("char *[1]") - local r = getfenv(0).__ngx_req - if not r then + local r = require "resty.core.base" .get_request() + if r == nil then ngx.log(ngx.ERR, "no request found") return end @@ -245,8 +246,8 @@ lua ssl server name: "test.com" local errmsg = ffi.new("char *[1]") - local r = getfenv(0).__ngx_req - if not r then + local r = require "resty.core.base" .get_request() + if r == nil then ngx.log(ngx.ERR, "no request found") return end @@ -399,8 +400,8 @@ lua ssl server name: "test.com" local errmsg = ffi.new("char *[1]") - local r = getfenv(0).__ngx_req - if not r then + local r = require "resty.core.base" .get_request() + if r == nil then ngx.log(ngx.ERR, "no request found") return end @@ -528,8 +529,8 @@ failed to parse PEM priv key: PEM_read_bio_PrivateKey() failed local errmsg = ffi.new("char *[1]") - local r = getfenv(0).__ngx_req - if not r then + local r = require "resty.core.base" .get_request() + if r == nil then ngx.log(ngx.ERR, "no request found") return end @@ -678,8 +679,8 @@ lua ssl server name: "test.com" local errmsg = ffi.new("char *[1]") - local r = getfenv(0).__ngx_req - if not r then + local r = require "resty.core.base" .get_request() + if r == nil then ngx.log(ngx.ERR, "no request found") return end diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 4dc992d370..78a66cf9b1 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -1114,3 +1114,108 @@ qr/ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!/s [error] [alert] [emerg] + + + +=== TEST 14: keep global variable in ssl_session_(store|fetch)_by_lua when OpenResty LuaJIT is used +--- http_config + ssl_session_store_by_lua_block { + ngx.log(ngx.WARN, "new foo: ", foo) + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + ssl_session_fetch_by_lua_block { + ngx.log(ngx.WARN, "new bar: ", foo) + if not bar then + bar = 1 + else + ngx.log(ngx.WARN, "old bar: ", bar) + bar = bar + 1 + end + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_session_tickets off; + + server_tokens off; + location /foo { + content_by_lua_block { + ngx.say("foo: ", foo) + ngx.say("bar: ", bar) + } + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + package.loaded.session = sess + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + local m, err = ngx.re.match(line, "^foo: (.*)$", "jo") + if err then + ngx.say("failed to match line: ", err) + end + + if m and m[1] then + ngx.print(m[1]) + end + end + + local ok, err = sock:close() + ngx.say("done") + end -- do + } + } + +--- request +GET /t +--- response_body_like chomp +\A[123]done\n\z +--- grep_error_log eval: qr/old (foo|bar): \d+/ +--- grep_error_log_out eval +["", "old foo: 1\n", "old bar: 1\nold foo: 2\n"] +--- no_error_log +[error] +[alert] +[emerg] diff --git a/t/152-timer-every.t b/t/152-timer-every.t index c8d62e704d..7b42d2d724 100644 --- a/t/152-timer-every.t +++ b/t/152-timer-every.t @@ -63,7 +63,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:09|10)\d*, cont -=== TEST 2: separated global env +=== TEST 2: shared global env --- config location /t { content_by_lua_block { @@ -84,7 +84,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:09|10)\d*, cont --- request GET /t --- response_body -foo = nil +foo = 3 --- wait: 0.12 --- no_error_log [error] diff --git a/t/157-global-var.t b/t/157-global-var.t new file mode 100644 index 0000000000..3c4f843fc5 --- /dev/null +++ b/t/157-global-var.t @@ -0,0 +1,510 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +log_level('debug'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3 + 14); + +our $HtmlDir = html_dir; + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); + +no_long_string(); + +sub read_file { + my $infile = shift; + open my $in, $infile + or die "cannot open $infile for reading: $!"; + my $cert = do { local $/; <$in> }; + close $in; + $cert; +} + +our $TestCertificate = read_file("t/cert/test.crt"); +our $TestCertificateKey = read_file("t/cert/test.key"); + +add_block_preprocessor(sub { + my $block = shift; + + if (!defined $block->error_log) { + $block->set_value("no_error_log", "[error]"); + } + + if (!defined $block->request) { + $block->set_value("request", "GET /t"); + } + +}); + +run_tests(); + +__DATA__ + +=== TEST 1: set_by_lua +--- config + location /t { + set_by_lua_block $res { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + return foo + } + echo $res; + } +--- response_body_like chomp +\A[12]\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|set_by_lua:\d+: in main chunk, )/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] + + + +=== TEST 2: rewrite_by_lua +--- config + location /t { + rewrite_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + ngx.say(foo) + } + } +--- response_body_like chomp +\A[12]\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] + + + +=== TEST 3: access_by_lua +--- config + location /t { + access_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + ngx.say(foo) + } + } +--- response_body_like chomp +\A[12]\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] + + + +=== TEST 4: content_by_lua +--- config + location /t { + content_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + ngx.say(foo) + } + } +--- response_body_like chomp +\A[12]\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +content_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] + + + +=== TEST 5: header_filter_by_lua +--- config + location /t { + content_by_lua_block { + ngx.say(foo) + } + header_filter_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + } +--- response_body_like chomp +\A(?:nil|1)\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua:\d+: in main chunk, )/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] + + + +=== TEST 6: body_filter_by_lua +--- config + location /t { + content_by_lua_block { + ngx.say(foo) + } + body_filter_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + } +--- response_body_like chomp +\A(?:nil|2)\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua:\d+: in main chunk, )/ +--- grep_error_log_out eval +[qr/\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +body_filter_by_lua:3: in main chunk, +old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] + + +=== TEST 7: log_by_lua +--- config + location /t { + content_by_lua_block { + ngx.say(foo) + } + log_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + } +--- response_body_like chomp +\A(?:nil|1)\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk)/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] + + + + +=== TEST 8: ssl_certificate_by_lua +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + content_by_lua_block { + ngx.say("foo: ", foo) + } + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + -- ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + -- ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + -- ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + local m, err = ngx.re.match(line, "^foo: (.*)$", "jo") + if err then + ngx.say("failed to match line: ", err) + end + + if m and m[1] then + ngx.print(m[1]) + end + end + + local ok, err = sock:close() + ngx.say("done") + end -- do + } + } + +--- response_body_like chomp +\A[12]done\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua:\d+: in main chunk)/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] + + + + +=== TEST 9: timer +--- config + location /t { + content_by_lua_block { + local function f() + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + end + local ok, err = ngx.timer.at(0, f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + ngx.sleep(0.01) + ngx.say(foo) + } + } +--- response_body_like chomp +\A[12]\n\z +--- grep_error_log eval +qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in\b)/ +--- grep_error_log_out eval +[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +content_by_lua\(nginx\.conf:56\):4: in\n\z/, "old foo: 1\n"] + + + + +=== TEST 10: init_by_lua +--- http_config + init_by_lua_block { + foo = 1 + } +--- config + location /t { + content_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + ngx.say(foo) + } + } +--- response_body_like chomp +\A[23]\n\z +--- grep_error_log eval: qr/old foo: \d+/ +--- grep_error_log_out eval +["old foo: 1\n", "old foo: 2\n"] + + + +=== TEST 11: init_worker_by_lua +--- http_config + init_worker_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } +--- config + location /t { + content_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + ngx.say(foo) + } + } +--- response_body_like chomp +\A[23]\n\z +--- grep_error_log eval: qr/old foo: \d+/ +--- grep_error_log_out eval +["old foo: 1\n", "old foo: 2\n"] + + + +=== TEST 12: init_by_lua + init_worker_by_lua +--- http_config + init_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + init_worker_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } +--- config + location /t { + content_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + ngx.say(foo) + } + } +--- response_body_like chomp +\A[34]\n\z +--- grep_error_log eval: qr/old foo: \d+/ +--- grep_error_log_out eval +["old foo: 1\nold foo: 2\n", "old foo: 3\n"] + + + +=== TEST 13: don't show warn messages in init/init_worker +--- http_config + init_by_lua_block { + foo = 1 + } + + init_worker_by_lua_block { + bar = 2 + } +--- config + location /t { + content_by_lua_block { + ngx.say(foo) + ngx.say(bar) + } + } +--- response_body +1 +2 +--- no_error_log +setting global variable + + + +=== TEST 14: uthread +--- config + location /t { + content_by_lua_block { + local function f() + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + end + local ok, err = ngx.thread.spawn(f) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + ngx.sleep(0.01) + ngx.say(foo) + } + } +--- response_body_like chomp +\A[12]\n\z +--- grep_error_log eval +qr/(old foo: \d+|write to the lua global variable '\w+')/ +--- grep_error_log_out eval +["write to the lua global variable 'foo'\n", "old foo: 1\n"] + + + +=== TEST 15: balancer_by_lua +--- http_config + upstream backend { + server 0.0.0.1; + balancer_by_lua_block { + if not foo then + foo = 1 + else + ngx.log(ngx.WARN, "old foo: ", foo) + foo = foo + 1 + end + } + } +--- config + location = /t { + proxy_pass http://backend; + } +--- response_body_like: 502 Bad Gateway +--- error_code: 502 +--- error_log eval +qr/\[crit\].*?\Qconnect() to 0.0.0.1:80 failed\E/ +--- grep_error_log eval: qr/(old foo: \d+|write to the lua global variable '\w+')/ +--- grep_error_log_out eval +["write to the lua global variable 'foo'\n", "old foo: 1\n"] diff --git a/t/data/fake-shm-module/ngx_http_lua_fake_shm_module.c b/t/data/fake-shm-module/ngx_http_lua_fake_shm_module.c index 0286853e54..ffc32f5795 100644 --- a/t/data/fake-shm-module/ngx_http_lua_fake_shm_module.c +++ b/t/data/fake-shm-module/ngx_http_lua_fake_shm_module.c @@ -213,10 +213,9 @@ ngx_http_lua_fake_shm_preload(lua_State *L) ngx_uint_t i; ngx_shm_zone_t **zone; + ngx_shm_zone_t **zone_udata; - lua_getglobal(L, "__ngx_cycle"); - cycle = lua_touserdata(L, -1); - lua_pop(L, 1); + cycle = (ngx_cycle_t *) ngx_cycle; hmcf_ctx = (ngx_http_conf_ctx_t *) cycle->conf_ctx[ngx_http_module.index]; lfsmcf = hmcf_ctx->main_conf[ngx_http_lua_fake_shm_module.ctx_index]; @@ -242,7 +241,9 @@ ngx_http_lua_fake_shm_preload(lua_State *L) lua_createtable(L, 1 /* narr */, 0 /* nrec */); /* table of zone[i] */ - lua_pushlightuserdata(L, zone[i]); /* shared mt key ud */ + zone_udata = lua_newuserdata(L, sizeof(ngx_shm_zone_t *)); + /* shared mt key ud */ + *zone_udata = zone[i]; lua_rawseti(L, -2, 1); /* {zone[i]} */ lua_pushvalue(L, -3); /* shared mt key ud mt */ lua_setmetatable(L, -2); /* shared mt key ud */ @@ -262,9 +263,10 @@ ngx_http_lua_fake_shm_preload(lua_State *L) static int ngx_http_lua_fake_shm_get_info(lua_State *L) { - ngx_int_t n; - ngx_shm_zone_t *zone; - ngx_http_lua_fake_shm_ctx_t *ctx; + ngx_int_t n; + ngx_shm_zone_t *zone; + ngx_shm_zone_t **zone_udata; + ngx_http_lua_fake_shm_ctx_t *ctx; n = lua_gettop(L); @@ -276,13 +278,15 @@ ngx_http_lua_fake_shm_get_info(lua_State *L) luaL_checktype(L, 1, LUA_TTABLE); lua_rawgeti(L, 1, 1); - zone = lua_touserdata(L, -1); + zone_udata = lua_touserdata(L, -1); lua_pop(L, 1); - if (zone == NULL) { + if (zone_udata == NULL) { return luaL_error(L, "bad \"zone\" argument"); } + zone = *zone_udata; + ctx = (ngx_http_lua_fake_shm_ctx_t *) zone->data; lua_pushlstring(L, (char *) zone->shm.name.data, zone->shm.name.len); diff --git a/util/build.sh b/util/build.sh index 164bf9f9a2..14e9fd46d9 100755 --- a/util/build.sh +++ b/util/build.sh @@ -22,6 +22,8 @@ force=$2 #--without-http_referer_module \ #--with-http_spdy_module \ +add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" + time ngx-build $force $version \ --with-pcre-jit \ --with-ipv6 \ @@ -55,7 +57,7 @@ time ngx-build $force $version \ --add-module=$root/../redis2-nginx-module \ --add-module=$root/../stream-lua-nginx-module \ --add-module=$root/t/data/fake-module \ - --add-module=$root/t/data/fake-shm-module \ + $add_fake_shm_module \ --add-module=$root/t/data/fake-delayed-load-module \ --with-http_gunzip_module \ --with-http_dav_module \ From d067594ca299ec00fcbfe98288844d3bf10e8064 Mon Sep 17 00:00:00 2001 From: Alessandro Ghedini Date: Fri, 31 Aug 2018 14:42:55 +0100 Subject: [PATCH 248/848] bugfix: silenced -Wcast-function-type warnings. Cast to intermediate "void *" to lose compiler knowledge about the original type and pass a gcc8 warning. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_script.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_script.c b/src/ngx_http_lua_script.c index 95ebadf64d..c675a162d4 100644 --- a/src/ngx_http_lua_script.c +++ b/src/ngx_http_lua_script.c @@ -329,7 +329,7 @@ ngx_http_lua_script_add_copy_code(ngx_http_lua_script_compile_t *sc, return NGX_ERROR; } - code->code = (ngx_http_lua_script_code_pt) + code->code = (ngx_http_lua_script_code_pt) (void *) ngx_http_lua_script_copy_len_code; code->len = len; @@ -399,7 +399,7 @@ ngx_http_lua_script_add_capture_code(ngx_http_lua_script_compile_t *sc, return NGX_ERROR; } - code->code = (ngx_http_lua_script_code_pt) + code->code = (ngx_http_lua_script_code_pt) (void *) ngx_http_lua_script_copy_capture_len_code; code->n = 2 * n; From ebfa66ceb7d872a10e5d2a0e0efdd58f94e27a46 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sun, 21 Oct 2018 11:38:40 -0700 Subject: [PATCH 249/848] misc: rename a test file to be properly ordered. Two commits introduced a test file numbered `157-*.t`. This commit renames the most recent test file (from 7286812116940216344ade33722c49ae47037605) to `158-*.t`. --- t/{157-global-var.t => 158-global-var.t} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename t/{157-global-var.t => 158-global-var.t} (100%) diff --git a/t/157-global-var.t b/t/158-global-var.t similarity index 100% rename from t/157-global-var.t rename to t/158-global-var.t From 84338ab85ef0b7de44cfd1fd6e013b81147a34fe Mon Sep 17 00:00:00 2001 From: sbhr Date: Thu, 9 Aug 2018 00:50:38 +0900 Subject: [PATCH 250/848] typo: fixed a debug log in access and rewrite handlers. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_accessby.c | 2 +- src/ngx_http_lua_rewriteby.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index f587806336..d3fe294917 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -60,7 +60,7 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) #endif if (cur_ph < last_ph) { - dd("swaping the contents of cur_ph and last_ph..."); + dd("swapping the contents of cur_ph and last_ph..."); tmp = *cur_ph; diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 08f7566904..1de9968e5e 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -62,7 +62,7 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) #endif if (cur_ph < last_ph) { - dd("swaping the contents of cur_ph and last_ph..."); + dd("swapping the contents of cur_ph and last_ph..."); tmp = *cur_ph; From c9eca03ebb4a77bc0211b404c87c9ce3b01b3d3b Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 29 Oct 2018 15:48:53 -0700 Subject: [PATCH 251/848] misc: updated _G write guard message to be more accurate. * more appropriate message ("writing" instead of "attempting to") * remove http/lua prefix (`ngx.log` will inject `[lua]` and `stream [lua]` already) * remove unused cached global * cache `tostring` global locally * fix the error messages not printing rc and error string * update the stacktrace level to '2' * updated tests --- src/ngx_http_lua_util.c | 16 ++++++++------- t/001-set.t | 4 ++-- t/158-global-var.t | 44 ++++++++++++++++++++--------------------- 3 files changed, 33 insertions(+), 31 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c6e21f62db..6b4b783726 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -771,7 +771,7 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, const char buf[] = "local ngx_log = ngx.log\n" "local ngx_WARN = ngx.WARN\n" - "local type = type\n" + "local tostring = tostring\n" "local ngx_get_phase = ngx.get_phase\n" "local traceback = require 'debug'.traceback\n" "local function newindex(table, key, value)\n" @@ -780,9 +780,11 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, "if phase == 'init_worker' or phase == 'init' then\n" "return\n" "end\n" - "ngx_log(ngx_WARN, 'http lua attempting to write to the lua " - "global variable \\'', tostring(key), " - "'\\'\\n', traceback())\n" + "ngx_log(ngx_WARN, 'writing a global lua variable " + "(\\'', tostring(key), '\\') which may lead to " + "race conditions between concurrent requests, so " + "prefer the use of \\'local\\' variables', " + "traceback('', 2))\n" "end\n" "setmetatable(_G, { __newindex = newindex })\n" ; @@ -791,8 +793,8 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, if (rc != 0) { ngx_log_error(NGX_LOG_ERR, log, 0, - "failed to load Lua code for the _G write guard: ", - "%i: %s", rc, lua_tostring(L, -1)); + "failed to load Lua code (%i): %s", + rc, lua_tostring(L, -1)); lua_pop(L, 1); return; @@ -801,7 +803,7 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, rc = lua_pcall(L, 0, 0, 0); if (rc != 0) { ngx_log_error(NGX_LOG_ERR, log, 0, - "failed to run Lua code for the _G write guard: %s", + "failed to run Lua code (%i): %s", rc, lua_tostring(L, -1)); lua_pop(L, 1); } diff --git a/t/001-set.t b/t/001-set.t index 60ab2ff471..70c7b903cf 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -613,9 +613,9 @@ GET /lua \z --- no_error_log [error] ---- grep_error_log eval: qr/(old foo: \d+|write to the lua global variable '\w+')/ +--- grep_error_log eval: qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ --- grep_error_log_out eval -["write to the lua global variable 'foo'\n", "old foo: 1\n"] +["writing a global lua variable \('foo'\)\n", "old foo: 1\n"] diff --git a/t/158-global-var.t b/t/158-global-var.t index 3c4f843fc5..745e7eaebb 100644 --- a/t/158-global-var.t +++ b/t/158-global-var.t @@ -60,9 +60,9 @@ __DATA__ --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|set_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|set_by_lua:\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] @@ -83,9 +83,9 @@ set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -106,9 +106,9 @@ rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -129,9 +129,9 @@ access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) content_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -154,9 +154,9 @@ content_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|1)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] @@ -179,9 +179,9 @@ header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|2)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\[warn\] .*?writing a global lua variable \('foo'\) body_filter_by_lua:3: in main chunk, old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] @@ -204,9 +204,9 @@ old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] --- response_body_like chomp \A(?:nil|1)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in main chunk)/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk)/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] @@ -297,9 +297,9 @@ log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]done\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua:\d+: in main chunk)/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk)/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] @@ -329,9 +329,9 @@ ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?http lua attempting to write to the lua global variable '[^'\s]+'|\w+_by_lua\(.*?\):\d+: in\b)/ +qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in\b)/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?http lua attempting to write to the lua global variable 'foo' +[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) content_by_lua\(nginx\.conf:56\):4: in\n\z/, "old foo: 1\n"] @@ -478,9 +478,9 @@ setting global variable --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|write to the lua global variable '\w+')/ +qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ --- grep_error_log_out eval -["write to the lua global variable 'foo'\n", "old foo: 1\n"] +["writing a global lua variable \('foo'\)\n", "old foo: 1\n"] @@ -505,6 +505,6 @@ qr/(old foo: \d+|write to the lua global variable '\w+')/ --- error_code: 502 --- error_log eval qr/\[crit\].*?\Qconnect() to 0.0.0.1:80 failed\E/ ---- grep_error_log eval: qr/(old foo: \d+|write to the lua global variable '\w+')/ +--- grep_error_log eval: qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ --- grep_error_log_out eval -["write to the lua global variable 'foo'\n", "old foo: 1\n"] +["writing a global lua variable \('foo'\)\n", "old foo: 1\n"] From 5efd95d793bad0587f75a7d0f701b76693c1311d Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 1 Nov 2018 16:01:55 -0700 Subject: [PATCH 252/848] doc: updated the docs to reflect the change that we now no longer support the standard Lua 5.1 interpreter in this module. also recommended OpenResty's LuaJIT branch version instead of the stock LuaJIT. --- README.markdown | 48 +++++------------------------------------------- 1 file changed, 5 insertions(+), 43 deletions(-) diff --git a/README.markdown b/README.markdown index a6ecd677bd..29d7df88be 100644 --- a/README.markdown +++ b/README.markdown @@ -187,7 +187,7 @@ Synopsis Description =========== -This module embeds Lua, via the standard Lua 5.1 interpreter or [LuaJIT 2.0/2.1](http://luajit.org/luajit.html), into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. +This module embeds Lua, via [LuaJIT 2.0/2.1](http://luajit.org/luajit.html), into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. Unlike [Apache's mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html) and [Lighttpd's mod_magnet](http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet), Lua code executed using this module can be *100% non-blocking* on network traffic as long as the [Nginx API for Lua](#nginx-api-for-lua) provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services. @@ -265,11 +265,11 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. +It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, OpenResty's LuaJIT 2.1 branch version, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. Alternatively, ngx_lua can be manually compiled into Nginx: -1. Install LuaJIT 2.0 or 2.1 (recommended) or Lua 5.1 (Lua 5.2 is *not* supported yet). LuaJIT can be downloaded from the [LuaJIT project website](http://luajit.org/download.html) and Lua 5.1, from the [Lua project website](http://www.lua.org/). Some distribution package managers also distribute LuaJIT and/or Lua. +1. LuaJIT can be downloaded the [latest release of OpenResty's LuaJIT branch version](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.0 and 2.1 releases are also supported though the performance will be signficantly lower in many cases. 1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags). 1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags). 1. Download the latest version of Nginx [HERE](http://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) @@ -345,29 +345,6 @@ To enable one or more of these macros, just pass extra C compiler options to the ./configure --with-cc-opt="-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC" -[Back to TOC](#table-of-contents) - -Installation on Ubuntu 11.10 ----------------------------- - -Note that it is recommended to use LuaJIT 2.0 or LuaJIT 2.1 instead of the standard Lua 5.1 interpreter wherever possible. - -If the standard Lua 5.1 interpreter is required however, run the following command to install it from the Ubuntu repository: - -```bash - - apt-get install -y lua5.1 liblua5.1-0 liblua5.1-0-dev -``` - -Everything should be installed correctly, except for one small tweak. - -Library name `liblua.so` has been changed in liblua5.1 package, it only comes with `liblua5.1.so`, which needs to be symlinked to `/usr/lib` so it could be found during the configuration process. - -```bash - - ln -s /usr/lib/x86_64-linux-gnu/liblua5.1.so /usr/lib/liblua.so -``` - [Back to TOC](#table-of-contents) Community @@ -411,7 +388,7 @@ Lua/LuaJIT bytecode support As from the `v0.5.0rc32` release, all `*_by_lua_file` configure directives (such as [content_by_lua_file](#content_by_lua_file)) support loading Lua 5.1 and LuaJIT 2.0/2.1 raw bytecode files directly. -Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: +Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if you are using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: ```bash @@ -431,20 +408,6 @@ Please refer to the official LuaJIT documentation on the `-b` option for more de Also, the bytecode files generated by LuaJIT 2.1 is *not* compatible with LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added in ngx_lua v0.9.3. -Similarly, if using the standard Lua 5.1 interpreter with ngx_lua, Lua compatible bytecode files must be generated using the `luac` commandline utility as shown: - -```bash - - luac -o /path/to/output_file.luac /path/to/input_file.lua -``` - -Unlike as with LuaJIT, debug information is included in standard Lua 5.1 bytecode files by default. This can be striped out by specifying the `-s` option as shown: - -```bash - - luac -s -o /path/to/output_file.luac /path/to/input_file.lua -``` - Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked to LuaJIT 2.0/2.1 or vice versa, will result in an error message, such as that below, being logged into the Nginx `error.log` file: @@ -642,8 +605,7 @@ This issue is due to limitations in the Nginx event model and only appears to af Lua Coroutine Yielding/Resuming ------------------------------- -* Because Lua's `dofile` and `require` builtins are currently implemented as C functions in both Lua 5.1 and LuaJIT 2.0/2.1, if the Lua file being loaded by `dofile` or `require` invokes [ngx.location.capture*](#ngxlocationcapture), [ngx.exec](#ngxexec), [ngx.exit](#ngxexit), or other API functions requiring yielding in the *top-level* scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file. -* As the standard Lua 5.1 interpreter's VM is not fully resumable, the methods [ngx.location.capture](#ngxlocationcapture), [ngx.location.capture_multi](#ngxlocationcapture_multi), [ngx.redirect](#ngxredirect), [ngx.exec](#ngxexec), and [ngx.exit](#ngxexit) cannot be used within the context of a Lua [pcall()](http://www.lua.org/manual/5.1/manual.html#pdf-pcall) or [xpcall()](http://www.lua.org/manual/5.1/manual.html#pdf-xpcall) or even the first line of the `for ... in ...` statement when the standard Lua 5.1 interpreter is used and the `attempt to yield across metamethod/C-call boundary` error will be produced. Please use LuaJIT 2.x, which supports a fully resumable VM, to avoid this. +* Because Lua's `dofile` and `require` builtins are currently implemented as C functions in LuaJIT 2.0/2.1, if the Lua file being loaded by `dofile` or `require` invokes [ngx.location.capture*](#ngxlocationcapture), [ngx.exec](#ngxexec), [ngx.exit](#ngxexit), or other API functions requiring yielding in the *top-level* scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file. [Back to TOC](#table-of-contents) From 09a365460684ea144a8abf52527431e0ec98763c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sun, 4 Nov 2018 11:53:47 -0800 Subject: [PATCH 253/848] tests: declared many variables as local to avoid triggering the new _G write guard. --- t/001-set.t | 6 +- t/002-content.t | 38 ++++----- t/005-exit.t | 2 +- t/009-log.t | 12 ++- t/011-md5_bin.t | 2 +- t/014-bugs.t | 6 +- t/017-exec.t | 4 +- t/020-subrequest.t | 134 +++++++++++++++---------------- t/023-rewrite/client-abort.t | 2 +- t/023-rewrite/exec.t | 4 +- t/023-rewrite/mixed.t | 4 +- t/023-rewrite/multi-capture.t | 2 +- t/023-rewrite/sanity.t | 28 +++---- t/023-rewrite/subrequest.t | 42 +++++----- t/023-rewrite/tcp-socket.t | 2 +- t/023-rewrite/uthread-exec.t | 12 +-- t/023-rewrite/uthread-exit.t | 32 ++++---- t/023-rewrite/uthread-redirect.t | 4 +- t/023-rewrite/uthread-spawn.t | 62 +++++++------- t/024-access/client-abort.t | 2 +- t/024-access/exec.t | 2 +- t/024-access/mixed.t | 10 +-- t/024-access/multi-capture.t | 2 +- t/024-access/sanity.t | 30 +++---- t/024-access/subrequest.t | 42 +++++----- t/024-access/uthread-exec.t | 12 +-- t/024-access/uthread-exit.t | 32 ++++---- t/024-access/uthread-redirect.t | 4 +- t/024-access/uthread-spawn.t | 62 +++++++------- t/027-multi-capture.t | 2 +- t/030-uri-args.t | 19 +++-- t/034-match.t | 65 +++++++-------- t/035-gmatch.t | 16 ++-- t/036-sub.t | 4 +- t/037-gsub.t | 2 +- t/038-match-o.t | 44 +++++----- t/041-header-filter.t | 3 +- t/043-shdict.t | 4 +- t/047-match-jit.t | 12 +-- t/048-match-dfa.t | 12 +-- t/055-subreq-vars.t | 18 ++--- t/056-flush.t | 2 +- t/057-flush-timeout.t | 2 +- t/058-tcp-socket.t | 10 +-- t/064-pcall.t | 10 +-- t/065-tcp-socket-timeout.t | 2 +- t/066-socket-receiveuntil.t | 30 +++---- t/073-backtrace.t | 8 +- t/075-logby.t | 3 +- t/081-bytecode.t | 21 ++--- t/082-body-filter.t | 3 +- t/084-inclusive-receiveuntil.t | 20 ++--- t/091-coroutine.t | 73 +++++++++-------- t/093-uthread-spawn.t | 66 ++++++++------- t/094-uthread-exit.t | 30 +++---- t/095-uthread-exec.t | 12 +-- t/096-uthread-redirect.t | 4 +- t/097-uthread-rewrite.t | 12 +-- t/098-uthread-wait.t | 74 ++++++++--------- t/099-c-api.t | 10 +-- t/100-client-abort.t | 2 +- t/106-timer.t | 4 +- t/108-timer-safe.t | 2 +- t/109-timer-hup.t | 4 +- t/120-re-find.t | 4 +- t/126-shdict-frag.t | 4 +- t/127-uthread-kill.t | 20 ++--- t/139-ssl-cert-by.t | 2 +- t/147-tcp-socket-timeouts.t | 4 +- t/153-semaphore-hup.t | 2 +- 70 files changed, 637 insertions(+), 600 deletions(-) diff --git a/t/001-set.t b/t/001-set.t index 70c7b903cf..9e8ee7b382 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -46,7 +46,7 @@ helloworld === TEST 3: internal only --- config location /lua { - set_by_lua $res "function fib(n) if n > 2 then return fib(n-1)+fib(n-2) else return 1 end end return fib(10)"; + set_by_lua $res "local function fib(n) if n > 2 then return fib(n-1)+fib(n-2) else return 1 end end return fib(10)"; echo $res; } --- request @@ -76,7 +76,7 @@ GET /lua?a=1&b=2 === TEST 5: fib by arg --- config location /fib { - set_by_lua $res "function fib(n) if n > 2 then return fib(n-1)+fib(n-2) else return 1 end end return fib(tonumber(ngx.arg[1]))" $arg_n; + set_by_lua $res "local function fib(n) if n > 2 then return fib(n-1)+fib(n-2) else return 1 end end return fib(tonumber(ngx.arg[1]))" $arg_n; echo $res; } --- request @@ -760,6 +760,8 @@ GET /lua?name=jim --- config location /t { set_by_lua $a ' + local bar + local foo function foo() bar() end diff --git a/t/002-content.t b/t/002-content.t index cc92d6f756..bb569f51e8 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -86,7 +86,7 @@ qr/content_by_lua\(nginx\.conf:\d+\):1: attempt to call field 'echo' \(a nil val location /lua { # NOTE: the newline escape sequence must be double-escaped, as nginx config # parser will unescape first! - content_by_lua 'v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\\n")'; + content_by_lua 'local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\\n")'; } --- request GET /lua?a=1&b=2 @@ -102,7 +102,7 @@ request_uri: /lua?a=1&b=2 } --- user_files >>> test.lua -v = ngx.var["request_uri"] +local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\n") --- request GET /lua?a=1&b=2 @@ -154,7 +154,7 @@ result: -0.4090441561579 === TEST 7: read $arg_xxx --- config location = /lua { - content_by_lua 'who = ngx.var.arg_who + content_by_lua 'local who = ngx.var.arg_who ngx.print("Hello, ", who, "!")'; } --- request @@ -171,7 +171,7 @@ Hello, agentzh! } location /lua { - content_by_lua 'res = ngx.location.capture("/other"); ngx.print("status=", res.status, " "); ngx.print("body=", res.body)'; + content_by_lua 'local res = ngx.location.capture("/other"); ngx.print("status=", res.status, " "); ngx.print("body=", res.body)'; } --- request GET /lua @@ -183,7 +183,7 @@ status=200 body=hello, world ei= TEST 9: capture non-existed location --- config location /lua { - content_by_lua 'res = ngx.location.capture("/other"); ngx.print("status=", res.status)'; + content_by_lua 'local res = ngx.location.capture("/other"); ngx.print("status=", res.status)'; } --- request GET /lua @@ -194,7 +194,7 @@ GET /lua === TEST 9: invalid capture location (not as expected...) --- config location /lua { - content_by_lua 'res = ngx.location.capture("*(#*"); ngx.say("res=", res.status)'; + content_by_lua 'local res = ngx.location.capture("*(#*"); ngx.say("res=", res.status)'; } --- request GET /lua @@ -247,7 +247,7 @@ GET /lua ngx.print("num is: ", num, "\\n"); if (num > 0) then - res = ngx.location.capture("/recur?num="..tostring(num - 1)); + local res = ngx.location.capture("/recur?num="..tostring(num - 1)); ngx.print("status=", res.status, " "); ngx.print("body=", res.body, "\\n"); else @@ -271,7 +271,7 @@ end ngx.print("num is: ", num, "\\n"); if (num > 0) then - res = ngx.location.capture("/recur?num="..tostring(num - 1)); + local res = ngx.location.capture("/recur?num="..tostring(num - 1)); ngx.print("status=", res.status, " "); ngx.print("body=", res.body); else @@ -353,7 +353,7 @@ location /sub { } location /parent { set $a 12; - content_by_lua 'res = ngx.location.capture("/sub"); ngx.print(res.body)'; + content_by_lua 'local res = ngx.location.capture("/sub"); ngx.print(res.body)'; } --- request GET /parent @@ -369,7 +369,7 @@ location /sub { location /parent { set $a 12; content_by_lua ' - res = ngx.location.capture( + local res = ngx.location.capture( "/sub", { share_all_vars = true } ); @@ -390,7 +390,7 @@ location /sub { } location /parent { content_by_lua ' - res = ngx.location.capture("/sub", { share_all_vars = true }); + local res = ngx.location.capture("/sub", { share_all_vars = true }); ngx.say(ngx.var.a) '; } @@ -408,7 +408,7 @@ location /sub { } location /parent { content_by_lua ' - res = ngx.location.capture("/sub", { share_all_vars = false }); + local res = ngx.location.capture("/sub", { share_all_vars = false }); ngx.say(ngx.var.a) '; } @@ -427,7 +427,7 @@ GET /parent location /lua { content_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); '; } @@ -454,7 +454,7 @@ type: foo/bar location /lua { content_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", type(res.header["Set-Cookie"])); ngx.say("len: ", #res.header["Set-Cookie"]); ngx.say("value: ", table.concat(res.header["Set-Cookie"], "|")) @@ -482,7 +482,7 @@ value: a|hello, world|foo location /lua { content_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); ngx.say("Bar: ", res.header["Bar"]); '; @@ -507,7 +507,7 @@ Bar: Bah location /lua { content_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); ngx.say("Bar: ", res.header["Bar"] or "nil"); '; @@ -524,7 +524,7 @@ Bar: nil --- config location /lua { content_by_lua ' - data = "hello, world" + local data = "hello, world" -- ngx.header["Content-Length"] = #data -- ngx.header.content_length = #data ngx.print(data) @@ -742,7 +742,7 @@ true --- config location /lua { content_by_lua ' - data = "hello,\\nworld\\n" + local data = "hello,\\nworld\\n" ngx.header["Content-Length"] = #data ngx.say("hello,") ngx.flush() @@ -801,7 +801,7 @@ world } --- user_files >>> test.lua -v = ngx.var["request_uri"] +local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\n") --- request GET /lua?a=1&b=2 diff --git a/t/005-exit.t b/t/005-exit.t index a5a28b3e28..a416a753bc 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -452,7 +452,7 @@ Hi --- config location /lua { content_by_lua ' - function f () + local function f () ngx.say("hello") ngx.exit(200) end diff --git a/t/009-log.t b/t/009-log.t index 68c057f82d..cb3895b8b2 100644 --- a/t/009-log.t +++ b/t/009-log.t @@ -414,6 +414,8 @@ GET /log --- config location /log { content_by_lua ' + local foo + local bar function foo() bar() end @@ -431,7 +433,7 @@ GET /log --- response_body done --- error_log eval -qr/\[error\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):7: bar\(\): hello, log12343.14159/ +qr/\[error\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):9: bar\(\): hello, log12343.14159/ @@ -439,6 +441,8 @@ qr/\[error\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):7: bar\(\): hell --- config location /log { content_by_lua ' + local foo + local bar function foo() return bar(5) end @@ -461,7 +465,7 @@ GET /log --- response_body done --- error_log eval -qr/\[error\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):8:(?: foo\(\):)? hello, log12343.14159/ +qr/\[error\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):10:(?: foo\(\):)? hello, log12343.14159/ @@ -472,6 +476,8 @@ qr/\[error\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):8:(?: foo\(\):)? } --- user_files >>> test.lua +local foo +local bar function foo() bar() end @@ -488,7 +494,7 @@ GET /log --- response_body done --- error_log eval -qr/\[error\] \S+: \S+ \[lua\] test.lua:6: bar\(\): hello, log12343.14159/ +qr/\[error\] \S+: \S+ \[lua\] test.lua:8: bar\(\): hello, log12343.14159/ diff --git a/t/011-md5_bin.t b/t/011-md5_bin.t index ae7c974f49..dc0fd9c9b6 100644 --- a/t/011-md5_bin.t +++ b/t/011-md5_bin.t @@ -156,7 +156,7 @@ d41d8cd98f00b204e9800998ecf8427e --- config location = /t { content_by_lua ' - s = ngx.md5_bin(45) + local s = ngx.md5_bin(45) s = string.gsub(s, ".", function (c) return string.format("%02x", string.byte(c)) end) diff --git a/t/014-bugs.t b/t/014-bugs.t index 545978ccb6..c0babf9ad2 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -112,7 +112,7 @@ GET /report/listBidwordPrices4lzExtra.htm?words=123,156,2532 } location = /main { content_by_lua ' - res = ngx.location.capture("/memc?c=get&k=foo&v=") + local res = ngx.location.capture("/memc?c=get&k=foo&v=") ngx.say("1: ", res.body) res = ngx.location.capture("/memc?c=set&k=foo&v=bar"); @@ -204,7 +204,7 @@ https://github.com/chaoslawful/lua-nginx-module/issues/37 content_by_lua ' -- local yajl = require "yajl" ngx.header["Set-Cookie"] = {} - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") for i,j in pairs(res.header) do ngx.header[i] = j @@ -231,7 +231,7 @@ Set-Cookie: TestCookie2=bar.*" } --- user_files >>> foo.lua -res = {} +local res = {} res = {'good 1', 'good 2', 'good 3'} return ngx.redirect("/somedir/" .. ngx.escape_uri(res[math.random(1,#res)])) --- request diff --git a/t/017-exec.t b/t/017-exec.t index 544b8bb898..a73e93ee7d 100644 --- a/t/017-exec.t +++ b/t/017-exec.t @@ -382,7 +382,7 @@ hello --- config location /lua { content_by_lua ' - function f () + local function f () ngx.exec("/hi") end @@ -524,7 +524,7 @@ hello --- config location /main { rewrite_by_lua ' - res = ngx.location.capture("/test_loc"); + local res = ngx.location.capture("/test_loc"); ngx.print("hello, ", res.body) '; content_by_lua return; diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 658a1d232b..5a386dbe1a 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -32,7 +32,7 @@ __DATA__ location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -62,7 +62,7 @@ lua http subrequest "/other?" location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -90,7 +90,7 @@ DELETE location /t { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST }); ngx.print(res.body) @@ -114,7 +114,7 @@ POST location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_HEAD }); ngx.print(res.body) @@ -141,7 +141,7 @@ GET /lua location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_GET }); ngx.print(res.body) @@ -169,7 +169,7 @@ GET location /lua { content_by_lua ' - res = ngx.location.capture("/foo") + local res = ngx.location.capture("/foo") ngx.print(res.body) '; @@ -196,7 +196,7 @@ GET location /lua { content_by_lua ' - res = ngx.location.capture("/foo", {}) + local res = ngx.location.capture("/foo", {}) ngx.print(res.body) '; @@ -226,7 +226,7 @@ GET location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -255,7 +255,7 @@ hello location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -291,7 +291,7 @@ hello location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -329,7 +329,7 @@ GET location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST, body = "hello" }); ngx.print(res.body) @@ -362,7 +362,7 @@ hello content_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); ngx.say("GET: " .. res.status); res = ngx.location.capture("/memc", @@ -404,7 +404,7 @@ cached: hello ngx.location.capture("/flush", { share_all_vars = true }); - res = ngx.location.capture("/memc", + local res = ngx.location.capture("/memc", { share_all_vars = true }); ngx.say("GET: " .. res.status); @@ -435,7 +435,7 @@ cached: hello location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = {} }) ngx.print(res.body) '; @@ -456,7 +456,7 @@ GET /lua location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { ["fo="] = "=>" } }) ngx.print(res.body) '; @@ -478,7 +478,7 @@ fo%3D=%3D%3E location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { ["fo="] = "=>", ["="] = ":" } }) ngx.print(res.body) @@ -503,7 +503,7 @@ GET /lua location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { foo = 3, bar = "hello" } }) ngx.print(res.body) @@ -526,7 +526,7 @@ GET /lua location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { [57] = "hi" } }) ngx.print(res.body) '; @@ -548,7 +548,7 @@ attempt to use a non-string key in the "args" option table location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { "hi" } }) ngx.print(res.body) '; @@ -570,7 +570,7 @@ attempt to use a non-string key in the "args" option table location /lua { content_by_lua ' - res = ngx.location.capture("/foo?a=3", + local res = ngx.location.capture("/foo?a=3", { args = { b = 4 } }) ngx.print(res.body) '; @@ -592,7 +592,7 @@ a=3&b=4 location /lua { content_by_lua ' - res = ngx.location.capture("/foo?a=3", + local res = ngx.location.capture("/foo?a=3", { args = "b=4" }) ngx.print(res.body) '; @@ -685,7 +685,7 @@ https://github.com/chaoslawful/lua-nginx-module/issues/38 location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_GET }); ngx.say("header foo: [", res.body, "]") '; @@ -711,7 +711,7 @@ https://github.com/chaoslawful/lua-nginx-module/issues/38 location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { body = "abc" }); ngx.say("header foo: [", res.body, "]") '; @@ -746,8 +746,8 @@ header foo: [bar] } location /main { content_by_lua ' - res1, res2 = ngx.location.capture_multi({{"/a"}, {"/b"}}) - res3 = ngx.location.capture("/c") + local res1, res2 = ngx.location.capture_multi({{"/a"}, {"/b"}}) + local res3 = ngx.location.capture("/c") ngx.print(res1.body, res2.body, res3.body) '; } @@ -780,7 +780,7 @@ lua reuse free buf memory location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST, body = "hello" }); ngx.print(res.body) @@ -801,7 +801,7 @@ hello --- config location /lua { content_by_lua ' - res = ngx.location.capture("/foo.html") + local res = ngx.location.capture("/foo.html") ngx.say(res.status) ngx.say(res.header["Last-Modified"]) @@ -832,7 +832,7 @@ hello, static file$ location /lua { content_by_lua ' local ctx = {} - res = ngx.location.capture("/sub", { ctx = ctx }) + local res = ngx.location.capture("/sub", { ctx = ctx }) ngx.say(ctx.foo); ngx.say(ngx.ctx.foo); @@ -857,7 +857,7 @@ nil } location /lua { content_by_lua ' - res = ngx.location.capture("/sub", { ctx = ngx.ctx }) + local res = ngx.location.capture("/sub", { ctx = ngx.ctx }) ngx.say(ngx.ctx.foo); '; } @@ -885,7 +885,7 @@ bar location /t { content_by_lua ' - res = ngx.location.capture("/memc", + local res = ngx.location.capture("/memc", { method = ngx.HTTP_PUT, body = "hello 1234" }); -- ngx.say("PUT: " .. res.status); @@ -922,7 +922,7 @@ lua reuse free buf chain, but reallocate memory because location /lua { content_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_GET }); ngx.print(res.body) @@ -959,7 +959,7 @@ nil content_by_lua ' ngx.req.read_body() - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_GET }); ngx.print(res.body) @@ -996,7 +996,7 @@ nil content_by_lua ' ngx.req.read_body() - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST }); ngx.print(res.body) @@ -1033,7 +1033,7 @@ hello, world content_by_lua ' ngx.req.read_body() - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_PUT }); ngx.print(res.body) @@ -1074,7 +1074,7 @@ lua subrequests cycle while processing "/t" location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_OPTIONS }); ngx.print(res.body) @@ -1099,7 +1099,7 @@ OPTIONS location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_OPTIONS, body = "hello world" }); ngx.print(res.body) @@ -1151,7 +1151,7 @@ r%5B%5D=http%3A%2F%2Fajax.googleapis.com%3A80%2Fajax%2Flibs%2Fjquery%2F1.7.2%2Fj location /main { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.say("status: ", res.status) ngx.say("body: ", res.body) '; @@ -1172,7 +1172,7 @@ body: location /main { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.say("status: ", res.status) ngx.say("body: ", res.body) '; @@ -1196,7 +1196,7 @@ body: location /main { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.say("status: ", res.status) ngx.say("body: ", res.body) '; @@ -1242,7 +1242,7 @@ F(ngx_http_finalize_request) { location /main { content_by_lua ' - res = ngx.location.capture("/memc") + local res = ngx.location.capture("/memc") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1312,7 +1312,7 @@ upstream prematurely closed connection location /main { content_by_lua ' - res = ngx.location.capture("/memc") + local res = ngx.location.capture("/memc") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1385,7 +1385,7 @@ upstream timed out location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1445,7 +1445,7 @@ upstream prematurely closed connection location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1507,7 +1507,7 @@ upstream timed out location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1567,7 +1567,7 @@ truncated: false location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1630,7 +1630,7 @@ upstream timed out location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1690,7 +1690,7 @@ truncated: false location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1756,7 +1756,7 @@ upstream timed out ngx.req.read_body() for i = 1, 2 do - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_POST }); ngx.say(res.body) @@ -1796,7 +1796,7 @@ hello world ngx.req.read_body() for i = 1, 2 do - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_POST }); ngx.say(res.body) @@ -1836,7 +1836,7 @@ hello world ngx.req.read_body() for i = 1, 2 do - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_POST }); ngx.say(res.body) @@ -1916,7 +1916,7 @@ a client request body is buffered to a temporary file location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -1979,7 +1979,7 @@ upstream prematurely closed connection location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -2040,7 +2040,7 @@ upstream prematurely closed connection location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -2102,7 +2102,7 @@ upstream timed out location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -2160,7 +2160,7 @@ truncated: false location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -2219,7 +2219,7 @@ truncated: false location /main { content_by_lua ' - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("status: ", res.status) ngx.say("body: ", res.body) ngx.say("truncated: ", res.truncated) @@ -2288,7 +2288,7 @@ upstream prematurely closed connection } for i, method in ipairs(methods) do - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = method }) ngx.print(res.body) end @@ -2324,7 +2324,7 @@ method: TRACE location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -2353,7 +2353,7 @@ nil location /lua { content_by_lua ' ngx.req.read_body() - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_DELETE, always_forward_body = true }); ngx.print(res.body) @@ -2382,7 +2382,7 @@ hello world location /lua { content_by_lua ' ngx.req.read_body() - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_DELETE, always_forward_body = true }); ngx.print(res.body) @@ -2410,7 +2410,7 @@ hello world --- config location = /t { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.print(res.body) '; } @@ -2603,7 +2603,7 @@ qr/Assertion .*? failed/ location = /t { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("pr: User-Agent: ", ngx.var.http_user_agent) ngx.say("pr: Host: ", ngx.var.http_host) @@ -2636,7 +2636,7 @@ pr: Host: localhost location = /t { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("pr: User-Agent: ", ngx.var.http_user_agent) ngx.say("pr: Host: ", ngx.var.http_host) @@ -2669,7 +2669,7 @@ pr: Host: localhost location = /t { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("pr: User-Agent: ", ngx.var.http_user_agent) ngx.say("pr: Host: ", ngx.var.http_host) @@ -2699,7 +2699,7 @@ pr: Host: localhost location = /t { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("pr: Cookie: ", ngx.var.http_cookie) '; @@ -2727,7 +2727,7 @@ pr: Cookie: foo; bar location = /t { content_by_lua ' - res = ngx.location.capture("/sub") + local res = ngx.location.capture("/sub") ngx.print(res.body) ngx.say("pr: Cookie: ", ngx.var.http_cookie) '; @@ -2750,7 +2750,7 @@ pr: Cookie: foo; bar --- config location /lua { content_by_lua ' - res = ngx.location.capture("/index.html", + local res = ngx.location.capture("/index.html", { method = ngx.HTTP_HEAD }); ngx.say("content-length: ", res.header["Content-Length"]) ngx.say("body: [", res.body, "]") diff --git a/t/023-rewrite/client-abort.t b/t/023-rewrite/client-abort.t index 117d17e5ef..19c47872eb 100644 --- a/t/023-rewrite/client-abort.t +++ b/t/023-rewrite/client-abort.t @@ -545,7 +545,7 @@ client prematurely closed connection return end - ok, err = sock:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT) + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err) return diff --git a/t/023-rewrite/exec.t b/t/023-rewrite/exec.t index edd4607355..59691cb599 100644 --- a/t/023-rewrite/exec.t +++ b/t/023-rewrite/exec.t @@ -326,7 +326,7 @@ hello --- config location /main { rewrite_by_lua ' - res = ngx.location.capture("/test_loc"); + local res = ngx.location.capture("/test_loc"); ngx.print("hello, ", res.body) '; content_by_lua return; @@ -354,7 +354,7 @@ hello, bah --- config location /main { rewrite_by_lua ' - res = ngx.location.capture("/test_loc"); + local res = ngx.location.capture("/test_loc"); ngx.print("hello, ", res.body) '; content_by_lua return; diff --git a/t/023-rewrite/mixed.t b/t/023-rewrite/mixed.t index 0f742b255a..5b38d3f45f 100644 --- a/t/023-rewrite/mixed.t +++ b/t/023-rewrite/mixed.t @@ -35,7 +35,7 @@ __DATA__ rewrite_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); print("rewrite GET: " .. res.status); res = ngx.location.capture("/memc", @@ -49,7 +49,7 @@ __DATA__ content_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); ngx.say("content GET: " .. res.status); res = ngx.location.capture("/memc", diff --git a/t/023-rewrite/multi-capture.t b/t/023-rewrite/multi-capture.t index 083ec78c90..a0a02b7a2d 100644 --- a/t/023-rewrite/multi-capture.t +++ b/t/023-rewrite/multi-capture.t @@ -154,7 +154,7 @@ res2.body = b location /main { rewrite_by_lua ' - res = ngx.location.capture("/foo?n=1") + local res = ngx.location.capture("/foo?n=1") ngx.say("top res.status = " .. res.status) ngx.say("top res.body = [" .. res.body .. "]") '; diff --git a/t/023-rewrite/sanity.t b/t/023-rewrite/sanity.t index b90aa0e167..73a85dc562 100644 --- a/t/023-rewrite/sanity.t +++ b/t/023-rewrite/sanity.t @@ -69,7 +69,7 @@ GET /lua location /lua { # NOTE: the newline escape sequence must be double-escaped, as nginx config # parser will unescape first! - rewrite_by_lua 'v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\\n")'; + rewrite_by_lua 'local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\\n")'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -87,7 +87,7 @@ request_uri: /lua?a=1&b=2 } --- user_files >>> test.lua -v = ngx.var["request_uri"] +local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\n") --- request GET /lua?a=1&b=2 @@ -140,7 +140,7 @@ result: -0.4090441561579 === TEST 7: read $arg_xxx --- config location = /lua { - rewrite_by_lua 'who = ngx.var.arg_who + rewrite_by_lua 'local who = ngx.var.arg_who ngx.print("Hello, ", who, "!")'; content_by_lua 'ngx.exit(ngx.OK)'; } @@ -159,7 +159,7 @@ Hello, agentzh! location /lua { rewrite_by_lua ' -res = ngx.location.capture("/other") +local res = ngx.location.capture("/other") ngx.print("status=", res.status, " ") ngx.print("body=", res.body) '; @@ -175,7 +175,7 @@ status=200 body=hello, world === TEST 9: capture non-existed location --- config location /lua { - rewrite_by_lua 'res = ngx.location.capture("/other"); ngx.print("status=", res.status)'; + rewrite_by_lua 'local res = ngx.location.capture("/other"); ngx.print("status=", res.status)'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -187,7 +187,7 @@ GET /lua === TEST 10: invalid capture location (not as expected...) --- config location /lua { - rewrite_by_lua 'res = ngx.location.capture("*(#*"); ngx.say("res=", res.status)'; + rewrite_by_lua 'local res = ngx.location.capture("*(#*"); ngx.say("res=", res.status)'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -270,7 +270,7 @@ end ngx.print("num is: ", num, "\\n"); if (num > 0) then - res = ngx.location.capture("/recur?num="..tostring(num - 1)); + local res = ngx.location.capture("/recur?num="..tostring(num - 1)); ngx.print("status=", res.status, " "); ngx.print("body=", res.body); else @@ -359,7 +359,7 @@ location /sub { } location /parent { set $a 12; - rewrite_by_lua 'res = ngx.location.capture("/sub"); ngx.print(res.body)'; + rewrite_by_lua 'local res = ngx.location.capture("/sub"); ngx.print(res.body)'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -377,7 +377,7 @@ location /parent { set $a ''; rewrite_by_lua ' ngx.var.a = 12; - res = ngx.location.capture( + local res = ngx.location.capture( "/sub", { share_all_vars = true } ); @@ -399,7 +399,7 @@ location /sub { } location /parent { rewrite_by_lua ' - res = ngx.location.capture("/sub", { share_all_vars = true }); + local res = ngx.location.capture("/sub", { share_all_vars = true }); ngx.say(ngx.var.a) '; @@ -421,7 +421,7 @@ location /sub { location /parent { rewrite_by_lua ' - res = ngx.location.capture("/sub", { share_all_vars = false }); + local res = ngx.location.capture("/sub", { share_all_vars = false }); ngx.say(ngx.var.a) '; content_by_lua return; @@ -441,7 +441,7 @@ GET /parent location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); '; @@ -466,7 +466,7 @@ type: foo/bar location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); ngx.say("Bar: ", res.header["Bar"]); '; @@ -494,7 +494,7 @@ Bar: Bah location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); ngx.say("Bar: ", res.header["Bar"] or "nil"); '; diff --git a/t/023-rewrite/subrequest.t b/t/023-rewrite/subrequest.t index 5d1e8f0839..cb8523c46b 100644 --- a/t/023-rewrite/subrequest.t +++ b/t/023-rewrite/subrequest.t @@ -27,7 +27,7 @@ __DATA__ location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -54,7 +54,7 @@ DELETE location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -82,7 +82,7 @@ DELETE location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST }); ngx.print(res.body) @@ -105,7 +105,7 @@ POST location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_HEAD }); ngx.print(res.body) @@ -131,7 +131,7 @@ GET /lua location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_GET }); ngx.print(res.body) @@ -158,7 +158,7 @@ GET location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo") + local res = ngx.location.capture("/foo") ngx.print(res.body) '; @@ -184,7 +184,7 @@ GET location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", {}) + local res = ngx.location.capture("/foo", {}) ngx.print(res.body) '; @@ -213,7 +213,7 @@ GET location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -241,7 +241,7 @@ hello location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -276,7 +276,7 @@ hello location /lua { rewrite_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -313,7 +313,7 @@ GET location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST, body = "hello" }); ngx.print(res.body) @@ -345,7 +345,7 @@ hello rewrite_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); ngx.say("GET: " .. res.status); res = ngx.location.capture("/memc", @@ -386,7 +386,7 @@ cached: hello ngx.location.capture("/flush", { share_all_vars = true }); - res = ngx.location.capture("/memc", + local res = ngx.location.capture("/memc", { share_all_vars = true }); ngx.say("GET: " .. res.status); @@ -417,7 +417,7 @@ cached: hello location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = {} }) ngx.print(res.body) '; @@ -437,7 +437,7 @@ GET /lua location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { ["fo="] = "=>" } }) ngx.print(res.body) '; @@ -458,7 +458,7 @@ fo%3D=%3D%3E location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { ["fo="] = "=>", ["="] = ":" } }) ngx.print(res.body) @@ -480,7 +480,7 @@ GET /lua location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { foo = 3, bar = "hello" } }) ngx.print(res.body) @@ -502,7 +502,7 @@ GET /lua location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { [57] = "hi" } }) ngx.print(res.body) '; @@ -523,7 +523,7 @@ GET /lua location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { "hi" } }) ngx.print(res.body) '; @@ -544,7 +544,7 @@ GET /lua location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo?a=3", + local res = ngx.location.capture("/foo?a=3", { args = { b = 4 } }) ngx.print(res.body) '; @@ -565,7 +565,7 @@ a=3&b=4 location /lua { rewrite_by_lua ' - res = ngx.location.capture("/foo?a=3", + local res = ngx.location.capture("/foo?a=3", { args = "b=4" }) ngx.print(res.body) '; diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index 41aeab74af..d05d80851d 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -940,7 +940,7 @@ close: 1 nil end end - ok, err = sock:close() + local ok, err = sock:close() ngx.say("close: ", ok, " ", err) '; diff --git a/t/023-rewrite/uthread-exec.t b/t/023-rewrite/uthread-exec.t index 2bea7e76e7..9428bd67b2 100644 --- a/t/023-rewrite/uthread-exec.t +++ b/t/023-rewrite/uthread-exec.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.exec("/foo") end @@ -59,7 +59,7 @@ i am foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end @@ -95,7 +95,7 @@ i am foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end @@ -178,12 +178,12 @@ hello foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end - function g() + local function g() ngx.sleep(1) end @@ -270,7 +270,7 @@ hello foo location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end diff --git a/t/023-rewrite/uthread-exit.t b/t/023-rewrite/uthread-exit.t index 6ebbb678bf..54db17d44c 100644 --- a/t/023-rewrite/uthread-exit.t +++ b/t/023-rewrite/uthread-exit.t @@ -24,7 +24,7 @@ __DATA__ --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.exit(0) end @@ -87,7 +87,7 @@ hello in thread --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -172,13 +172,13 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f") ngx.exit(0) end - function g() + local function g() ngx.sleep(1) ngx.say("g") end @@ -262,7 +262,7 @@ f --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("exiting the user thread") ngx.exit(0) @@ -301,7 +301,7 @@ exiting the user thread resolver agentzh.org; resolver_timeout 12s; rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.001) ngx.exit(0) @@ -407,7 +407,7 @@ after #resolver 127.0.0.1; resolver_timeout 12s; rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.001) ngx.exit(0) @@ -510,7 +510,7 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -600,7 +600,7 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -700,7 +700,7 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -806,7 +806,7 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -901,7 +901,7 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -995,7 +995,7 @@ after location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -1080,7 +1080,7 @@ after location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -1164,7 +1164,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exit(0) end @@ -1247,7 +1247,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exit(0) end diff --git a/t/023-rewrite/uthread-redirect.t b/t/023-rewrite/uthread-redirect.t index 83de1a36ed..0e636f7bba 100644 --- a/t/023-rewrite/uthread-redirect.t +++ b/t/023-rewrite/uthread-redirect.t @@ -25,7 +25,7 @@ __DATA__ location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.redirect(301) end @@ -113,7 +113,7 @@ attempt to abort with pending subrequests --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.redirect(301) end diff --git a/t/023-rewrite/uthread-spawn.t b/t/023-rewrite/uthread-spawn.t index 5552107e39..dccef87dac 100644 --- a/t/023-rewrite/uthread-spawn.t +++ b/t/023-rewrite/uthread-spawn.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") end @@ -68,11 +68,11 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("in thread 1") end - function g() + local function g() ngx.say("in thread 2") end @@ -117,7 +117,7 @@ after 2 --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("before sleep") ngx.sleep(0.1) ngx.say("after sleep") @@ -154,13 +154,13 @@ after sleep --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("1: before sleep") ngx.sleep(0.2) ngx.say("1: after sleep") end - function g() + local function g() ngx.say("2: before sleep") ngx.sleep(0.1) ngx.say("2: after sleep") @@ -210,7 +210,7 @@ delete thread 2 --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.blah() end @@ -241,9 +241,9 @@ qr/lua user thread aborted: runtime error: rewrite_by_lua\(nginx\.conf:\d+\):3: --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("before capture") - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("after capture: ", res.body) end @@ -287,7 +287,7 @@ after capture: hello world --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("after capture: ", res.body) @@ -340,7 +340,7 @@ after capture: hello foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("after capture: ", res.body) @@ -394,13 +394,13 @@ capture: hello bar --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("f: before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("f: after capture: ", res.body) end - function g() + local function g() ngx.say("g: before capture") local res = ngx.location.capture("/proxy?bah") ngx.say("g: after capture: ", res.body) @@ -472,7 +472,8 @@ g: after capture: hello bah --- config location /lua { rewrite_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -518,7 +519,8 @@ after g --- config location /lua { rewrite_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -566,7 +568,7 @@ hello in g() location /lua { rewrite_by_lua ' local co - function f() + local function f() co = coroutine.running() ngx.sleep(0.1) end @@ -599,7 +601,7 @@ status: running location /lua { rewrite_by_lua ' local co - function f() + local function f() co = coroutine.running() end @@ -631,7 +633,8 @@ status: zombie location /lua { rewrite_by_lua ' local co - function f() + local g + local function f() co = coroutine.running() local co2 = coroutine.create(g) coroutine.resume(co2) @@ -670,7 +673,8 @@ status: normal --- config location /lua { rewrite_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -717,7 +721,7 @@ after f rewrite_by_lua ' local yield = coroutine.yield - function f() + local function f() local self = coroutine.running() ngx.say("f 1") yield(self) @@ -770,7 +774,7 @@ f 3 rewrite_by_lua ' local yield = coroutine.yield - function f() + local function f() local self = coroutine.running() ngx.say("f 1") yield(self) @@ -779,7 +783,7 @@ f 3 ngx.say("f 3") end - function g() + local function g() local self = coroutine.running() ngx.say("g 1") yield(self) @@ -826,7 +830,7 @@ g 3 --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello in thread") coroutine.yield(coroutine.running) ngx.flush(true) @@ -863,12 +867,12 @@ after --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.say("hello from f") ngx.flush(true) end - function g() + local function g() ngx.say("hello from g") ngx.flush(true) end @@ -914,7 +918,7 @@ hello from g --- config location /lua { rewrite_by_lua ' - function f() + local function f() local sock = ngx.socket.tcp() local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) if not ok then @@ -966,7 +970,7 @@ received: OK --- config location /lua { rewrite_by_lua ' - function f() + local function f() local sock = ngx.socket.udp() local ok, err = sock:setpeername("127.0.0.1", 12345) local bytes, err = sock:send("blah") @@ -1027,7 +1031,7 @@ after)$ --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.req.read_body() local body = ngx.req.get_body_data() ngx.say("body: ", body) @@ -1073,7 +1077,7 @@ body: hello world)$ --- config location /lua { rewrite_by_lua ' - function f() + local function f() local sock = ngx.req.socket() local body, err = sock:receive(11) if not body then diff --git a/t/024-access/client-abort.t b/t/024-access/client-abort.t index c16f4eaab7..299d410311 100644 --- a/t/024-access/client-abort.t +++ b/t/024-access/client-abort.t @@ -546,7 +546,7 @@ client prematurely closed connection return end - ok, err = sock:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT) + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err) return diff --git a/t/024-access/exec.t b/t/024-access/exec.t index 43c1a7752a..d168a47d87 100644 --- a/t/024-access/exec.t +++ b/t/024-access/exec.t @@ -326,7 +326,7 @@ hello --- config location /main { access_by_lua ' - res = ngx.location.capture("/test_loc"); + local res = ngx.location.capture("/test_loc"); ngx.print("hello, ", res.body) '; content_by_lua return; diff --git a/t/024-access/mixed.t b/t/024-access/mixed.t index a9f8039b42..22f0037919 100644 --- a/t/024-access/mixed.t +++ b/t/024-access/mixed.t @@ -35,7 +35,7 @@ __DATA__ access_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); print("access GET: ", res.status); res = ngx.location.capture("/memc", @@ -49,7 +49,7 @@ __DATA__ content_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); ngx.say("content GET: " .. res.status); res = ngx.location.capture("/memc", @@ -187,7 +187,7 @@ world\x03\x04\xff rewrite_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); print("rewrite GET: " .. res.status); res = ngx.location.capture("/memc", @@ -201,7 +201,7 @@ world\x03\x04\xff access_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); print("access GET: " .. res.status); res = ngx.location.capture("/memc", @@ -215,7 +215,7 @@ world\x03\x04\xff content_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); ngx.say("content GET: " .. res.status); res = ngx.location.capture("/memc", diff --git a/t/024-access/multi-capture.t b/t/024-access/multi-capture.t index 930b74dad9..b1757dd120 100644 --- a/t/024-access/multi-capture.t +++ b/t/024-access/multi-capture.t @@ -154,7 +154,7 @@ res2.body = b location /main { access_by_lua ' - res = ngx.location.capture("/foo?n=1") + local res = ngx.location.capture("/foo?n=1") ngx.say("top res.status = " .. res.status) ngx.say("top res.body = [" .. res.body .. "]") '; diff --git a/t/024-access/sanity.t b/t/024-access/sanity.t index 7ff177fcf7..e5612a8b8a 100644 --- a/t/024-access/sanity.t +++ b/t/024-access/sanity.t @@ -69,7 +69,7 @@ GET /lua location /lua { # NOTE: the newline escape sequence must be double-escaped, as nginx config # parser will unescape first! - access_by_lua 'v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\\n")'; + access_by_lua 'local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\\n")'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -87,7 +87,7 @@ request_uri: /lua?a=1&b=2 } --- user_files >>> test.lua -v = ngx.var["request_uri"] +local v = ngx.var["request_uri"] ngx.print("request_uri: ", v, "\n") --- request GET /lua?a=1&b=2 @@ -140,7 +140,7 @@ result: -0.4090441561579 === TEST 7: read $arg_xxx --- config location = /lua { - access_by_lua 'who = ngx.var.arg_who + access_by_lua 'local who = ngx.var.arg_who ngx.print("Hello, ", who, "!")'; content_by_lua 'ngx.exit(ngx.OK)'; } @@ -159,7 +159,7 @@ Hello, agentzh! location /lua { access_by_lua ' -res = ngx.location.capture("/other") +local res = ngx.location.capture("/other") ngx.print("status=", res.status, " ") ngx.print("body=", res.body) '; @@ -175,7 +175,7 @@ status=200 body=hello, world === TEST 9: capture non-existed location --- config location /lua { - access_by_lua 'res = ngx.location.capture("/other"); ngx.print("status=", res.status)'; + access_by_lua 'local res = ngx.location.capture("/other"); ngx.print("status=", res.status)'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -187,7 +187,7 @@ GET /lua === TEST 10: invalid capture location (not as expected...) --- config location /lua { - access_by_lua 'res = ngx.location.capture("*(#*"); ngx.say("res=", res.status)'; + access_by_lua 'local res = ngx.location.capture("*(#*"); ngx.say("res=", res.status)'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -244,7 +244,7 @@ GET /lua ngx.print("num is: ", num, "\\n"); if (num > 0) then - res = ngx.location.capture("/recur?num="..tostring(num - 1)); + local res = ngx.location.capture("/recur?num="..tostring(num - 1)); ngx.print("status=", res.status, " "); ngx.print("body=", res.body, "\\n"); else @@ -271,7 +271,7 @@ access phase not running in subrequests ngx.print("num is: ", num, "\\n"); if (num > 0) then - res = ngx.location.capture("/recur?num="..tostring(num - 1)); + local res = ngx.location.capture("/recur?num="..tostring(num - 1)); ngx.print("status=", res.status, " "); ngx.print("body=", res.body); else @@ -357,7 +357,7 @@ location /sub { } location /parent { set $a 12; - access_by_lua 'res = ngx.location.capture("/sub"); ngx.print(res.body)'; + access_by_lua 'local res = ngx.location.capture("/sub"); ngx.print(res.body)'; content_by_lua 'ngx.exit(ngx.OK)'; } --- request @@ -375,7 +375,7 @@ location /parent { set $a ''; access_by_lua ' ngx.var.a = 12; - res = ngx.location.capture( + local res = ngx.location.capture( "/sub", { share_all_vars = true } ); @@ -397,7 +397,7 @@ location /sub { } location /parent { access_by_lua ' - res = ngx.location.capture("/sub", { share_all_vars = true }); + local res = ngx.location.capture("/sub", { share_all_vars = true }); ngx.say(ngx.var.a) '; @@ -419,7 +419,7 @@ location /sub { location /parent { access_by_lua ' - res = ngx.location.capture("/sub", { share_all_vars = false }); + local res = ngx.location.capture("/sub", { share_all_vars = false }); ngx.say(ngx.var.a) '; content_by_lua return; @@ -439,7 +439,7 @@ GET /parent location /lua { access_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); '; @@ -464,7 +464,7 @@ type: foo/bar location /lua { access_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); ngx.say("Bar: ", res.header["Bar"]); '; @@ -492,7 +492,7 @@ Bar: Bah location /lua { access_by_lua ' - res = ngx.location.capture("/other"); + local res = ngx.location.capture("/other"); ngx.say("type: ", res.header["Content-Type"]); ngx.say("Bar: ", res.header["Bar"] or "nil"); '; diff --git a/t/024-access/subrequest.t b/t/024-access/subrequest.t index b6ccf11990..665780a6dd 100644 --- a/t/024-access/subrequest.t +++ b/t/024-access/subrequest.t @@ -27,7 +27,7 @@ __DATA__ location /lua { access_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -54,7 +54,7 @@ DELETE location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_DELETE }); ngx.print(res.body) @@ -82,7 +82,7 @@ DELETE location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST }); ngx.print(res.body) @@ -105,7 +105,7 @@ POST location /lua { access_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_HEAD }); ngx.print(res.body) @@ -131,7 +131,7 @@ GET /lua location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_GET }); ngx.print(res.body) @@ -158,7 +158,7 @@ GET location /lua { access_by_lua ' - res = ngx.location.capture("/foo") + local res = ngx.location.capture("/foo") ngx.print(res.body) '; @@ -184,7 +184,7 @@ GET location /lua { access_by_lua ' - res = ngx.location.capture("/foo", {}) + local res = ngx.location.capture("/foo", {}) ngx.print(res.body) '; @@ -213,7 +213,7 @@ GET location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -241,7 +241,7 @@ hello location /lua { access_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -276,7 +276,7 @@ hello location /lua { access_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { method = ngx.HTTP_PUT, body = "hello" }); ngx.print(res.body) @@ -313,7 +313,7 @@ GET location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { method = ngx.HTTP_POST, body = "hello" }); ngx.print(res.body) @@ -345,7 +345,7 @@ hello access_by_lua ' ngx.location.capture("/flush"); - res = ngx.location.capture("/memc"); + local res = ngx.location.capture("/memc"); ngx.say("GET: " .. res.status); res = ngx.location.capture("/memc", @@ -386,7 +386,7 @@ cached: hello ngx.location.capture("/flush", { share_all_vars = true }); - res = ngx.location.capture("/memc", + local res = ngx.location.capture("/memc", { share_all_vars = true }); ngx.say("GET: " .. res.status); @@ -417,7 +417,7 @@ cached: hello location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = {} }) ngx.print(res.body) '; @@ -437,7 +437,7 @@ GET /lua location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { ["fo="] = "=>" } }) ngx.print(res.body) '; @@ -458,7 +458,7 @@ fo%3D=%3D%3E location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { ["fo="] = "=>", ["="] = ":" } }) ngx.print(res.body) @@ -480,7 +480,7 @@ GET /lua location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { foo = 3, bar = "hello" } }) ngx.print(res.body) @@ -502,7 +502,7 @@ GET /lua location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { [57] = "hi" } }) ngx.print(res.body) '; @@ -523,7 +523,7 @@ GET /lua location /lua { access_by_lua ' - res = ngx.location.capture("/foo", + local res = ngx.location.capture("/foo", { args = { "hi" } }) ngx.print(res.body) '; @@ -544,7 +544,7 @@ GET /lua location /lua { access_by_lua ' - res = ngx.location.capture("/foo?a=3", + local res = ngx.location.capture("/foo?a=3", { args = { b = 4 } }) ngx.print(res.body) '; @@ -565,7 +565,7 @@ a=3&b=4 location /lua { access_by_lua ' - res = ngx.location.capture("/foo?a=3", + local res = ngx.location.capture("/foo?a=3", { args = "b=4" }) ngx.print(res.body) '; diff --git a/t/024-access/uthread-exec.t b/t/024-access/uthread-exec.t index 7add3d4218..9c88eb33e3 100644 --- a/t/024-access/uthread-exec.t +++ b/t/024-access/uthread-exec.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { access_by_lua ' - function f() + local function f() ngx.exec("/foo") end @@ -59,7 +59,7 @@ i am foo --- config location /lua { access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end @@ -95,7 +95,7 @@ i am foo --- config location /lua { access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end @@ -179,12 +179,12 @@ hello foo --- config location /lua { access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end - function g() + local function g() ngx.sleep(1) end @@ -271,7 +271,7 @@ hello foo location /lua { client_body_timeout 12000ms; access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end diff --git a/t/024-access/uthread-exit.t b/t/024-access/uthread-exit.t index 02c2a1f96f..b884e29b5a 100644 --- a/t/024-access/uthread-exit.t +++ b/t/024-access/uthread-exit.t @@ -24,7 +24,7 @@ __DATA__ --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.exit(0) end @@ -87,7 +87,7 @@ hello in thread --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -172,13 +172,13 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f") ngx.exit(0) end - function g() + local function g() ngx.sleep(1) ngx.say("g") end @@ -262,7 +262,7 @@ f --- config location /lua { access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("exiting the user thread") ngx.exit(0) @@ -301,7 +301,7 @@ exiting the user thread resolver agentzh.org; resolver_timeout 12s; access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.001) ngx.exit(0) @@ -397,7 +397,7 @@ after resolver agentzh.org; resolver_timeout 12s; access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.001) ngx.exit(0) @@ -491,7 +491,7 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -582,7 +582,7 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -682,7 +682,7 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -788,7 +788,7 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -883,7 +883,7 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -977,7 +977,7 @@ after location /lua { client_body_timeout 12000ms; access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -1062,7 +1062,7 @@ after location /lua { client_body_timeout 12000ms; access_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -1146,7 +1146,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exit(0) end @@ -1229,7 +1229,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exit(0) end diff --git a/t/024-access/uthread-redirect.t b/t/024-access/uthread-redirect.t index 4eb4759585..cb99a35125 100644 --- a/t/024-access/uthread-redirect.t +++ b/t/024-access/uthread-redirect.t @@ -25,7 +25,7 @@ __DATA__ location /lua { client_body_timeout 12000ms; access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.redirect(301) end @@ -113,7 +113,7 @@ attempt to abort with pending subrequests --- config location /lua { access_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.redirect(301) end diff --git a/t/024-access/uthread-spawn.t b/t/024-access/uthread-spawn.t index 7c7ba3b931..bc92ccd32c 100644 --- a/t/024-access/uthread-spawn.t +++ b/t/024-access/uthread-spawn.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") end @@ -68,11 +68,11 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("in thread 1") end - function g() + local function g() ngx.say("in thread 2") end @@ -117,7 +117,7 @@ after 2 --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("before sleep") ngx.sleep(0.1) ngx.say("after sleep") @@ -154,13 +154,13 @@ after sleep --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("1: before sleep") ngx.sleep(0.2) ngx.say("1: after sleep") end - function g() + local function g() ngx.say("2: before sleep") ngx.sleep(0.1) ngx.say("2: after sleep") @@ -210,7 +210,7 @@ delete thread 2 --- config location /lua { access_by_lua ' - function f() + local function f() ngx.blah() end @@ -241,9 +241,9 @@ qr/lua user thread aborted: runtime error: access_by_lua\(nginx\.conf:\d+\):3: a --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("before capture") - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("after capture: ", res.body) end @@ -287,7 +287,7 @@ after capture: hello world --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("after capture: ", res.body) @@ -340,7 +340,7 @@ after capture: hello foo --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("after capture: ", res.body) @@ -395,13 +395,13 @@ capture: hello bar --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("f: before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("f: after capture: ", res.body) end - function g() + local function g() ngx.say("g: before capture") local res = ngx.location.capture("/proxy?bah") ngx.say("g: after capture: ", res.body) @@ -473,7 +473,8 @@ g: after capture: hello bah --- config location /lua { access_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -519,7 +520,8 @@ after g --- config location /lua { access_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -567,7 +569,7 @@ hello in g() location /lua { access_by_lua ' local co - function f() + local function f() co = coroutine.running() ngx.sleep(0.1) end @@ -600,7 +602,7 @@ status: running location /lua { access_by_lua ' local co - function f() + local function f() co = coroutine.running() end @@ -632,7 +634,8 @@ status: zombie location /lua { access_by_lua ' local co - function f() + local g + local function f() co = coroutine.running() local co2 = coroutine.create(g) coroutine.resume(co2) @@ -671,7 +674,8 @@ status: normal --- config location /lua { access_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -718,7 +722,7 @@ after f access_by_lua ' local yield = coroutine.yield - function f() + local function f() local self = coroutine.running() ngx.say("f 1") yield(self) @@ -771,7 +775,7 @@ f 3 access_by_lua ' local yield = coroutine.yield - function f() + local function f() local self = coroutine.running() ngx.say("f 1") yield(self) @@ -780,7 +784,7 @@ f 3 ngx.say("f 3") end - function g() + local function g() local self = coroutine.running() ngx.say("g 1") yield(self) @@ -827,7 +831,7 @@ g 3 --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello in thread") coroutine.yield(coroutine.running) ngx.flush(true) @@ -864,12 +868,12 @@ after --- config location /lua { access_by_lua ' - function f() + local function f() ngx.say("hello from f") ngx.flush(true) end - function g() + local function g() ngx.say("hello from g") ngx.flush(true) end @@ -915,7 +919,7 @@ hello from g --- config location /lua { access_by_lua ' - function f() + local function f() local sock = ngx.socket.tcp() local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) if not ok then @@ -967,7 +971,7 @@ received: OK --- config location /lua { access_by_lua ' - function f() + local function f() local sock = ngx.socket.udp() local ok, err = sock:setpeername("127.0.0.1", 12345) local bytes, err = sock:send("blah") @@ -1028,7 +1032,7 @@ after)$ --- config location /lua { access_by_lua ' - function f() + local function f() ngx.req.read_body() local body = ngx.req.get_body_data() ngx.say("body: ", body) @@ -1073,7 +1077,7 @@ body: hello world)$ --- config location /lua { access_by_lua ' - function f() + local function f() local sock = ngx.req.socket() local body, err = sock:receive(11) if not body then diff --git a/t/027-multi-capture.t b/t/027-multi-capture.t index 9227fe5329..2087940b5c 100644 --- a/t/027-multi-capture.t +++ b/t/027-multi-capture.t @@ -151,7 +151,7 @@ res2.body = b location /main { content_by_lua ' - res = ngx.location.capture("/foo?n=1") + local res = ngx.location.capture("/foo?n=1") ngx.say("top res.status = " .. res.status) ngx.say("top res.body = [" .. res.body .. "]") '; diff --git a/t/030-uri-args.t b/t/030-uri-args.t index a9d46ff97b..7d63daf8d2 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -586,7 +586,7 @@ hello #set $args 'hello'; set $err ''; access_by_lua ' - res, err = pcall(ngx.req.set_uri, "/bar", true); + local res, err = pcall(ngx.req.set_uri, "/bar", true); ngx.var.err = err '; echo "err: $err"; @@ -626,7 +626,7 @@ uri: /bar location /foo { #set $args 'hello'; content_by_lua ' - res, err = pcall(ngx.req.set_uri, "/bar", true); + local res, err = pcall(ngx.req.set_uri, "/bar", true); ngx.say("err: ", err) '; } @@ -665,7 +665,7 @@ uri: /bar location /foo { #set $args 'hello'; set_by_lua $err ' - res, err = pcall(ngx.req.set_uri, "/bar", true); + local res, err = pcall(ngx.req.set_uri, "/bar", true); return err '; echo "err: $err"; @@ -788,7 +788,7 @@ GET /lua location /lua { content_by_lua ' local t = {bar = ngx.shared.dogs, foo = 3} - rc, err = pcall(ngx.encode_args, t) + local rc, err = pcall(ngx.encode_args, t) ngx.say("rc: ", rc, ", err: ", err) '; } @@ -1112,6 +1112,7 @@ HTTP/1.0 a=3&b=5&b=6 --- config location /lua { content_by_lua ' + local err local args = "a=bar&b=foo" args, err = ngx.decode_args(args) @@ -1135,6 +1136,7 @@ b = foo --- config location /lua { content_by_lua ' + local err local args = "a=bar&b=foo&a=baz" args, err = ngx.decode_args(args) @@ -1158,6 +1160,7 @@ b = foo --- config location /lua { content_by_lua ' + local err local args = "" args, err = ngx.decode_args(args) if err then @@ -1178,6 +1181,7 @@ n = 0 --- config location /lua { content_by_lua ' + local err local args = "a&b" args, err = ngx.decode_args(args) if err then @@ -1200,6 +1204,7 @@ b = true --- config location /lua { content_by_lua ' + local err local args = "a=&b=" args, err = ngx.decode_args(args) @@ -1223,6 +1228,7 @@ b = --- config location /lua { content_by_lua ' + local err local args = "a=bar&b=foo" args, err = ngx.decode_args(args, 1) if err then @@ -1246,6 +1252,7 @@ b = nil --- config location /lua { content_by_lua ' + local err local args = "a=bar&b=foo" args, err = ngx.decode_args(args, -1) @@ -1270,7 +1277,7 @@ b = foo location /lua { content_by_lua ' local s = "f+f=bar&B=foo" - args, err = ngx.decode_args(s) + local args, err = ngx.decode_args(s) if err then ngx.say("err: ", err) end @@ -1302,7 +1309,7 @@ s = f+f=bar&B=foo lua_need_request_body on; location /t { content_by_lua ' - function split(s, delimiter) + local function split(s, delimiter) local result = {} local from = 1 local delim_from, delim_to = string.find(s, delimiter, from) diff --git a/t/034-match.t b/t/034-match.t index ebe5762be4..473a11fff7 100644 --- a/t/034-match.t +++ b/t/034-match.t @@ -21,7 +21,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)") + local m = ngx.re.match("hello, 1234", "([0-9]+)") if m then ngx.say(m[0]) else @@ -40,7 +40,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "(\\\\d+)") + local m = ngx.re.match("hello, 1234", "(\\\\d+)") if m then ngx.say(m[0]) else @@ -59,7 +59,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "(\\d+)") + local m = ngx.re.match("hello, 1234", "(\\d+)") if m then ngx.say(m[0]) else @@ -80,7 +80,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "[[\\d+]]") + local m = ngx.re.match("hello, 1234", "[[\\d+]]") if m then ngx.say(m[0]) else @@ -101,7 +101,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]{2})[0-9]+") + local m = ngx.re.match("hello, 1234", "([0-9]{2})[0-9]+") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -122,7 +122,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([a-z]+).*?([0-9]{2})[0-9]+", "") + local m = ngx.re.match("hello, 1234", "([a-z]+).*?([0-9]{2})[0-9]+", "") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -145,7 +145,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([a-z]+).*?([0-9]{2})[0-9]+", "o") + local m = ngx.re.match("hello, 1234", "([a-z]+).*?([0-9]{2})[0-9]+", "o") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -168,7 +168,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "foo") + local m = ngx.re.match("hello, 1234", "foo") if m then ngx.say(m[0]) else @@ -187,7 +187,7 @@ not matched: nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "HELLO") + local m = ngx.re.match("hello, 1234", "HELLO") if m then ngx.say(m[0]) else @@ -206,7 +206,7 @@ not matched: nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "HELLO", "i") + local m = ngx.re.match("hello, 1234", "HELLO", "i") if m then ngx.say(m[0]) else @@ -225,7 +225,7 @@ hello --- config location /re { content_by_lua ' - rc, err = pcall(ngx.re.match, "hello章亦春", "HELLO.{2}", "iu") + local rc, err = pcall(ngx.re.match, "hello章亦春", "HELLO.{2}", "iu") if not rc then ngx.say("FAIL: ", err) return @@ -249,7 +249,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", "^world", "m") + local m = ngx.re.match("hello\\nworld", "^world", "m") if m then ngx.say(m[0]) else @@ -268,7 +268,7 @@ world --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", ".*", "m") + local m = ngx.re.match("hello\\nworld", ".*", "m") if m then ngx.say(m[0]) else @@ -287,7 +287,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", "^world", "s") + local m = ngx.re.match("hello\\nworld", "^world", "s") if m then ngx.say(m[0]) else @@ -306,7 +306,7 @@ not matched: nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", ".*", "s") + local m = ngx.re.match("hello\\nworld", ".*", "s") if m then ngx.say(m[0]) else @@ -326,7 +326,7 @@ world --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", "\\\\w \\\\w", "x") + local m = ngx.re.match("hello\\nworld", "\\\\w \\\\w", "x") if m then ngx.say(m[0]) else @@ -372,7 +372,7 @@ error: pcre_compile() failed: missing ) in "(abc" --- config location /re { content_by_lua ' - rc, m = pcall(ngx.re.match, "hello\\nworld", ".*", "Hm") + local rc, m = pcall(ngx.re.match, "hello\\nworld", ".*", "Hm") if rc then if m then ngx.say(m[0]) @@ -395,7 +395,7 @@ error: .*?unknown flag "H" \(flags "Hm"\) --- config location /re { content_by_lua ' - m = ngx.re.match("hello, world", "(world)|(hello)", "x") + local m = ngx.re.match("hello, world", "(world)|(hello)", "x") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -418,7 +418,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)(h?)") + local m = ngx.re.match("hello, 1234", "([0-9]+)(h?)") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -442,7 +442,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)", "a") + local m = ngx.re.match("hello, 1234", "([0-9]+)", "a") if m then ngx.say(m[0]) else @@ -461,7 +461,7 @@ not matched! --- config location /re { content_by_lua ' - m = ngx.re.match("1234, hello", "([0-9]+)", "a") + local m = ngx.re.match("1234, hello", "([0-9]+)", "a") if m then ngx.say(m[0]) else @@ -481,7 +481,7 @@ not matched! location /re { content_by_lua ' local ctx = {} - m = ngx.re.match("1234, hello", "([0-9]+)", "", ctx) + local m = ngx.re.match("1234, hello", "([0-9]+)", "", ctx) if m then ngx.say(m[0]) ngx.say(ctx.pos) @@ -504,7 +504,7 @@ not matched! location /re { content_by_lua ' local ctx = { pos = 3 } - m = ngx.re.match("1234, hello", "([0-9]+)", "", ctx) + local m = ngx.re.match("1234, hello", "([0-9]+)", "", ctx) if m then ngx.say(m[0]) ngx.say(ctx.pos) @@ -526,7 +526,7 @@ not matched! --- config location /re { set_by_lua $res ' - m = ngx.re.match("hello, 1234", "([0-9]+)") + local m = ngx.re.match("hello, 1234", "([0-9]+)") if m then return m[0] else @@ -569,7 +569,7 @@ baz } --- user_files >>> a.lua -m = ngx.re.match("hello, 1234", "(\\\s+)") +local m = ngx.re.match("hello, 1234", "(\\\s+)") if m then ngx.say("[", m[0], "]") else @@ -595,7 +595,7 @@ end local uri = "2" local regex = '(?:>[\\w\\s]*)'; ngx.say("regex: ", regex) -m = ngx.re.match(uri, regex, "oi") +local m = ngx.re.match(uri, regex, "oi") if m then ngx.say("[", m[0], "]") else @@ -613,7 +613,7 @@ regex: (?:>[\w\s]*) --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", [[\\d+]]) + local m = ngx.re.match("hello, 1234", [[\\d+]]) if m then ngx.say(m[0]) else @@ -660,7 +660,7 @@ error: pcre_compile() failed: missing ) in "([0-9]+" --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", [[([0-9]+)]]) + local m = ngx.re.match("hello, 1234", [[([0-9]+)]]) if m then ngx.say(m[0]) else @@ -1027,7 +1027,7 @@ exec opts: 0 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() @@ -1067,7 +1067,7 @@ error: pcre_exec() failed: -8 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() @@ -1101,7 +1101,7 @@ failed to match content_by_lua ' local res = {} local s = "hello, 1234" - m = ngx.re.match(s, [[(\\d)(\\d)]], "o", nil, res) + local m = ngx.re.match(s, [[(\\d)(\\d)]], "o", nil, res) if m then ngx.say("1: m size: ", #m) ngx.say("1: res size: ", #res) @@ -1132,11 +1132,12 @@ failed to match === TEST 48: init_by_lua --- http_config init_by_lua ' - m = ngx.re.match("hello, 1234", "(\\\\d+)") + package.loaded.m = ngx.re.match("hello, 1234", "(\\\\d+)") '; --- config location /re { content_by_lua ' + local m = package.loaded.m if m then ngx.say(m[0]) else diff --git a/t/035-gmatch.t b/t/035-gmatch.t index 042699767a..c526778db1 100644 --- a/t/035-gmatch.t +++ b/t/035-gmatch.t @@ -123,7 +123,7 @@ nil --- config location /re { content_by_lua ' - it = ngx.re.gmatch("hello, 1234", "([0-9]+)", "a") + local it = ngx.re.gmatch("hello, 1234", "([0-9]+)", "a") ngx.say(it()) '; } @@ -418,7 +418,7 @@ hello content_by_lua ' local a = {} for i = 1, 3 do - it = ngx.re.gmatch("hello, world", "[a-z]+") + local it = ngx.re.gmatch("hello, world", "[a-z]+") it() collectgarbage() table.insert(a, {"hello", "world"}) @@ -518,7 +518,7 @@ matched: [] location /re { content_by_lua ' local it = ngx.re.gmatch("1234, 1234", "(?[0-9]+)") - m = it() + local m = it() if m then ngx.say(m[0]) ngx.say(m[1]) @@ -555,7 +555,7 @@ matched: [] content_by_lua ' local it = ngx.re.gmatch("1234, abcd, 1234", "(?[0-9]+)|(?[a-z]+)") - m = it() + local m = it() if m then ngx.say(m[0]) ngx.say(m[1]) @@ -599,7 +599,7 @@ abcd location /re { content_by_lua ' local it = ngx.re.gmatch("hello, 1234", "(?[a-z]+), (?[0-9]+)", "D") - m = it() + local m = it() if m then ngx.say(m[0]) ngx.say(m[1]) @@ -825,7 +825,7 @@ exec opts: 0 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() @@ -871,7 +871,7 @@ error: pcre_exec() failed: -8 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() @@ -881,7 +881,7 @@ if not it then return end -res, err = it() +local res, err = it() --[[ ngx.update_time() diff --git a/t/036-sub.t b/t/036-sub.t index 2b4b07510a..3e88eba5a0 100644 --- a/t/036-sub.t +++ b/t/036-sub.t @@ -590,7 +590,7 @@ s: a好 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() @@ -693,7 +693,7 @@ ab.cd location = /t { content_by_lua ' - function test() + local function test() local data = [[ OUTER {FIRST} ]] diff --git a/t/037-gsub.t b/t/037-gsub.t index 4c5810d527..41f86ac013 100644 --- a/t/037-gsub.t +++ b/t/037-gsub.t @@ -511,7 +511,7 @@ s: aa >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() diff --git a/t/038-match-o.t b/t/038-match-o.t index d61ff1fdaa..f6ae0103fe 100644 --- a/t/038-match-o.t +++ b/t/038-match-o.t @@ -20,7 +20,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)", "o") + local m = ngx.re.match("hello, 1234", "([0-9]+)", "o") if m then ngx.say(m[0]) else @@ -39,7 +39,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "(\\\\d+)", "o") + local m = ngx.re.match("hello, 1234", "(\\\\d+)", "o") if m then ngx.say(m[0]) else @@ -58,7 +58,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "(\\d+)", "o") + local m = ngx.re.match("hello, 1234", "(\\d+)", "o") if m then ngx.say(m[0]) else @@ -79,7 +79,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "[[\\d+]]", "o") + local m = ngx.re.match("hello, 1234", "[[\\d+]]", "o") if m then ngx.say(m[0]) else @@ -100,7 +100,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]{2})[0-9]+", "o") + local m = ngx.re.match("hello, 1234", "([0-9]{2})[0-9]+", "o") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -121,7 +121,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([a-z]+).*?([0-9]{2})[0-9]+", "o") + local m = ngx.re.match("hello, 1234", "([a-z]+).*?([0-9]{2})[0-9]+", "o") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -144,7 +144,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "foo", "o") + local m = ngx.re.match("hello, 1234", "foo", "o") if m then ngx.say(m[0]) else @@ -163,7 +163,7 @@ not matched: nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "HELLO", "o") + local m = ngx.re.match("hello, 1234", "HELLO", "o") if m then ngx.say(m[0]) else @@ -182,7 +182,7 @@ not matched: nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "HELLO", "oi") + local m = ngx.re.match("hello, 1234", "HELLO", "oi") if m then ngx.say(m[0]) else @@ -224,7 +224,7 @@ this version of PCRE is not compiled with PCRE_UTF8 support|^hello章亦$ --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", "^world", "mo") + local m = ngx.re.match("hello\\nworld", "^world", "mo") if m then ngx.say(m[0]) else @@ -243,7 +243,7 @@ world --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", ".*", "om") + local m = ngx.re.match("hello\\nworld", ".*", "om") if m then ngx.say(m[0]) else @@ -262,7 +262,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", "^world", "so") + local m = ngx.re.match("hello\\nworld", "^world", "so") if m then ngx.say(m[0]) else @@ -281,7 +281,7 @@ not matched: nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", ".*", "os") + local m = ngx.re.match("hello\\nworld", ".*", "os") if m then ngx.say(m[0]) else @@ -301,7 +301,7 @@ world --- config location /re { content_by_lua ' - m = ngx.re.match("hello\\nworld", "\\\\w \\\\w", "xo") + local m = ngx.re.match("hello\\nworld", "\\\\w \\\\w", "xo") if m then ngx.say(m[0]) else @@ -347,7 +347,7 @@ error: pcre_compile() failed: missing ) in "(abc" --- config location /re { content_by_lua ' - rc, m = pcall(ngx.re.match, "hello\\nworld", ".*", "Ho") + local rc, m = pcall(ngx.re.match, "hello\\nworld", ".*", "Ho") if rc then if m then ngx.say(m[0]) @@ -370,7 +370,7 @@ error: pcre_compile() failed: missing ) in "(abc" --- config location /re { content_by_lua ' - m = ngx.re.match("hello, world", "(world)|(hello)", "xo") + local m = ngx.re.match("hello, world", "(world)|(hello)", "xo") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -393,7 +393,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)(h?)", "o") + local m = ngx.re.match("hello, 1234", "([0-9]+)(h?)", "o") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -417,7 +417,7 @@ hello --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)", "oa") + local m = ngx.re.match("hello, 1234", "([0-9]+)", "oa") if m then ngx.say(m[0]) else @@ -436,7 +436,7 @@ not matched! --- config location /re { content_by_lua ' - m = ngx.re.match("1234, hello", "([0-9]+)", "ao") + local m = ngx.re.match("1234, hello", "([0-9]+)", "ao") if m then ngx.say(m[0]) else @@ -456,7 +456,7 @@ not matched! location /re { content_by_lua ' local ctx = {} - m = ngx.re.match("1234, hello", "([0-9]+)", "o", ctx) + local m = ngx.re.match("1234, hello", "([0-9]+)", "o", ctx) if m then ngx.say(m[0]) ngx.say(ctx.pos) @@ -479,7 +479,7 @@ not matched! location /re { content_by_lua ' local ctx = { pos = 3 } - m = ngx.re.match("1234, hello", "([0-9]+)", "o", ctx) + local m = ngx.re.match("1234, hello", "([0-9]+)", "o", ctx) if m then ngx.say(m[0]) ngx.say(ctx.pos) @@ -501,7 +501,7 @@ not matched! --- config location /re { set_by_lua $res ' - m = ngx.re.match("hello, 1234", "([0-9]+)", "o") + local m = ngx.re.match("hello, 1234", "([0-9]+)", "o") if m then return m[0] else diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 5e66e184d7..0adc699c6c 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -734,7 +734,8 @@ hello world --- config location /t { header_filter_by_lua ' - function foo() + local bar + local function foo() bar() end diff --git a/t/043-shdict.t b/t/043-shdict.t index 9183bf5e2e..b0528e5cc7 100644 --- a/t/043-shdict.t +++ b/t/043-shdict.t @@ -420,7 +420,7 @@ ngx_slab_alloc() failed: no memory in lua_shared_dict zone ngx.say(dogs:get(key)) key = string.rep("a", 65536) - ok, err = dogs:set(key, "world") + local ok, err = dogs:set(key, "world") if not ok then ngx.say("not ok: ", err) return @@ -2048,7 +2048,7 @@ get_stale ok: false, stale: false ngx.say("get not ok: ", err) return end - flags = err + local flags = err ngx.say("get_stale ok: ", data, ", flags: ", flags, ", stale: ", stale) diff --git a/t/047-match-jit.t b/t/047-match-jit.t index 2417a63ad3..f53a4083a2 100644 --- a/t/047-match-jit.t +++ b/t/047-match-jit.t @@ -20,7 +20,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)", "j") + local m = ngx.re.match("hello, 1234", "([0-9]+)", "j") if m then ngx.say(m[0]) else @@ -41,7 +41,7 @@ pcre JIT compiling result: 1 --- config location /re { content_by_lua ' - m = ngx.re.match("hello, world", "([0-9]+)", "j") + local m = ngx.re.match("hello, world", "([0-9]+)", "j") if m then ngx.say(m[0]) else @@ -62,7 +62,7 @@ pcre JIT compiling result: 1 --- config location /re { content_by_lua ' - m = ngx.re.match("hello, 1234", "([0-9]+)", "jo") + local m = ngx.re.match("hello, 1234", "([0-9]+)", "jo") if m then ngx.say(m[0]) else @@ -87,7 +87,7 @@ qr/pcre JIT compiling result: \d+/ --- config location /re { content_by_lua ' - m = ngx.re.match("hello, world", "([0-9]+)", "jo") + local m = ngx.re.match("hello, world", "([0-9]+)", "jo") if m then ngx.say(m[0]) else @@ -147,7 +147,7 @@ error: pcre_compile() failed: missing ) in "(abc" >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 21) +local s = string.rep([[ABCDEFG]], 21) local start = ngx.now() @@ -187,7 +187,7 @@ error: pcre_exec() failed: -8 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 21) +local s = string.rep([[ABCDEFG]], 21) local start = ngx.now() diff --git a/t/048-match-dfa.t b/t/048-match-dfa.t index 28b5a604c3..edf3662d4f 100644 --- a/t/048-match-dfa.t +++ b/t/048-match-dfa.t @@ -20,7 +20,7 @@ __DATA__ --- config location /re { content_by_lua ' - m = ngx.re.match("hello", "(he|hell)", "d") + local m = ngx.re.match("hello", "(he|hell)", "d") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -43,7 +43,7 @@ nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello", "(he|hell)", "do") + local m = ngx.re.match("hello", "(he|hell)", "do") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -66,7 +66,7 @@ nil --- config location /re { content_by_lua ' - m = ngx.re.match("hello", "(he|hell)", "jd") + local m = ngx.re.match("hello", "(he|hell)", "jd") if m then ngx.say(m[0]) else @@ -85,7 +85,7 @@ hell --- config location /re { content_by_lua ' - m = ngx.re.match("world", "(he|hell)", "d") + local m = ngx.re.match("world", "(he|hell)", "d") if m then ngx.say(m[0]) else @@ -104,7 +104,7 @@ not matched! --- config location /re { content_by_lua ' - m = ngx.re.match("hello", "he|hell", "do") + local m = ngx.re.match("hello", "he|hell", "do") if m then ngx.say(m[0]) ngx.say(m[1]) @@ -127,7 +127,7 @@ nil --- config location /re { content_by_lua ' - m = ngx.re.match("world", "([0-9]+)", "do") + local m = ngx.re.match("world", "([0-9]+)", "do") if m then ngx.say(m[0]) else diff --git a/t/055-subreq-vars.t b/t/055-subreq-vars.t index eb5e24d447..1369992f2b 100644 --- a/t/055-subreq-vars.t +++ b/t/055-subreq-vars.t @@ -28,7 +28,7 @@ __DATA__ location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = { dog = "hello", cat = 32 }}); ngx.print(res.body) @@ -82,7 +82,7 @@ qr/variable "(dog|cat)" cannot be assigned a value \(maybe you forgot to define location /lua { set $dog ''; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = { dog = "hello", cat = 32 }}); ngx.print(res.body) @@ -110,7 +110,7 @@ variable "cat" cannot be assigned a value (maybe you forgot to define it first?) set $dog ''; set $cat ''; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = { dog = "hello", cat = 32 }}); ngx.print(res.body) @@ -137,7 +137,7 @@ cat = 32 set $dog ''; set $cat ''; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = "hello" }); ngx.print(res.body) @@ -165,7 +165,7 @@ Bad vars option value set $dog ''; set $cat ''; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = { cat = true } }); ngx.print(res.body) @@ -188,7 +188,7 @@ attempt to use bad variable value type boolean location /lua { content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = { args = "a=hello&b=32" }}); ngx.print(res.body) @@ -234,7 +234,7 @@ variable "query_string" not changeable location /lua { set $dog 'hello'; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { copy_all_vars = true }); ngx.print(res.body) @@ -259,7 +259,7 @@ GET /lua location /lua { set $dog 'hello'; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { share_all_vars = true }); ngx.print(res.body) @@ -284,7 +284,7 @@ GET /lua location /lua { set $dog 'hello'; content_by_lua ' - res = ngx.location.capture("/other", + local res = ngx.location.capture("/other", { vars = { dog = "hiya" }, copy_all_vars = true }); ngx.print(res.body) diff --git a/t/056-flush.t b/t/056-flush.t index 6b697a4233..bdd33d4f9a 100644 --- a/t/056-flush.t +++ b/t/056-flush.t @@ -317,7 +317,7 @@ lua http 1.0 buffering makes ngx.flush() a no-op --- config location /test { content_by_lua ' - function f() + local function f() ngx.say("hello, world") ngx.flush(true) coroutine.yield() diff --git a/t/057-flush-timeout.t b/t/057-flush-timeout.t index a04653991e..1f8cfaadf9 100644 --- a/t/057-flush-timeout.t +++ b/t/057-flush-timeout.t @@ -127,7 +127,7 @@ del timer 1234 send_timeout 200ms; location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 05542eab1d..298503c2cc 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -914,7 +914,7 @@ close: 1 nil end end - ok, err = sock:close() + local ok, err = sock:close() ngx.say("close: ", ok, " ", err) '; } @@ -961,7 +961,7 @@ close: 1 nil line, err = sock:receive() ngx.say("receive: ", line, " ", err) - ok, err = sock:close() + local ok, err = sock:close() ngx.say("close: ", ok, " ", err) '; } @@ -2611,7 +2611,7 @@ close: 1 nil local ready = false local fatal = false - function f() + local function f() local line, err, part = sock:receive() if not line then ngx.say("failed to receive the 1st line: ", err, " [", part, "]") @@ -3464,7 +3464,7 @@ lua http cleanup reuse ngx.say("failed to create timer: ", err) end - i = 1 + local i = 1 while not done do local time = 0.005 * i if time > 0.1 then @@ -3545,7 +3545,7 @@ lua http cleanup reuse ngx.say("failed to create timer: ", err) end - i = 1 + local i = 1 while not done do local time = 0.005 * i if time > 0.1 then diff --git a/t/064-pcall.t b/t/064-pcall.t index 3011f3e46a..4bf447e43a 100644 --- a/t/064-pcall.t +++ b/t/064-pcall.t @@ -26,7 +26,7 @@ __DATA__ --- config location = /test { content_by_lua ' - function f(a, b) + local function f(a, b) if a == 0 and b == 0 then error("zero error") end @@ -62,7 +62,7 @@ $/s --- config location = /test { content_by_lua ' - function f(a, b) + local function f(a, b) if a == 0 and b == 0 then error("zero error") end @@ -70,15 +70,15 @@ $/s return 23, "hello", true end - function g() + local function g() return f(0, 0) end - function h() + local function h() return f(0) end - function err(...) + local function err(...) ngx.say("error handler called: ", ...) return "this is the new err" end diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 78b8cff7be..66ff6459b3 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -584,7 +584,7 @@ bad timeout value --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index ffe74aa354..89d2abf373 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -245,7 +245,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabd") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -316,7 +316,7 @@ close: 1 nil local reader = sock:receiveuntil("aa") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -387,7 +387,7 @@ close: 1 nil local reader = sock:receiveuntil("aaa") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -458,7 +458,7 @@ close: 1 nil local reader = sock:receiveuntil("aaaaad") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -530,7 +530,7 @@ close: 1 nil local reader = sock:receiveuntil("aaaaad") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -602,7 +602,7 @@ close: 1 nil local reader = sock:receiveuntil("aaaaad") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -673,7 +673,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabdabcabe") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -744,7 +744,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabdabcabe") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -815,7 +815,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabdabcabe") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -886,7 +886,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabdabcabe") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -957,7 +957,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -1028,7 +1028,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc") for i = 1, 7 do - line, err, part = reader(4) + local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -1105,7 +1105,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc") for i = 1, 7 do - line, err, part = reader(4) + local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -1182,7 +1182,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc") for i = 1, 7 do - line, err, part = reader(4) + local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -1270,7 +1270,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabd") for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) diff --git a/t/073-backtrace.t b/t/073-backtrace.t index 94dc984d43..663c3afece 100644 --- a/t/073-backtrace.t +++ b/t/073-backtrace.t @@ -21,10 +21,10 @@ __DATA__ --- config location /lua { content_by_lua - ' function bar() + ' local function bar() return lua_concat(3) end - function foo() + local function foo() bar() end foo() @@ -45,10 +45,10 @@ attempt to call global 'lua_concat' --- config location /lua { content_by_lua - ' function bar() + ' local function bar() error(nil) end - function foo() + local function foo() bar() end foo() diff --git a/t/075-logby.t b/t/075-logby.t index 84c7c466ef..8eece698ec 100644 --- a/t/075-logby.t +++ b/t/075-logby.t @@ -497,7 +497,8 @@ API disabled in the context of log_by_lua* location /t { echo ok; log_by_lua ' - function foo() + local bar + local function foo() bar() end diff --git a/t/081-bytecode.t b/t/081-bytecode.t index 29b2f758ab..4b05438823 100644 --- a/t/081-bytecode.t +++ b/t/081-bytecode.t @@ -24,7 +24,7 @@ __DATA__ content_by_lua ' ngx.req.read_body(); local b = ngx.req.get_body_data(); - f = io.open(ngx.var.realpath_root.."/test.lua", "w"); + local f = io.open(ngx.var.realpath_root.."/test.lua", "w"); -- luajit bytecode: sub(149,-1), lua bytecode: sub(1,147) if jit then if not string.find(jit.version, "LuaJIT 2.0") then @@ -35,7 +35,8 @@ __DATA__ else f:write(string.sub(b, 1, 147)); end - f:close(); res = ngx.location.capture("/call"); + f:close(); + local res = ngx.location.capture("/call"); ngx.print(res.body) '; } @@ -60,14 +61,15 @@ __DATA__ content_by_lua ' ngx.req.read_body(); local b = ngx.req.get_body_data(); - f = io.open(ngx.var.realpath_root.."/test.lua", "w"); + local f = io.open(ngx.var.realpath_root.."/test.lua", "w"); -- luajit bytecode: sub(149,-1), lua bytecode: sub(1,147) if not package.loaded["jit"] then f:write(string.sub(b, 149)); else f:write(string.sub(b, 1, 147)); end - f:close(); res = ngx.location.capture("/call"); + f:close(); + local res = ngx.location.capture("/call"); if res.status == 200 then ngx.print(res.body) else @@ -96,14 +98,15 @@ qr/failed to load external Lua file ".*?test\.lua": .* cannot load incompatible content_by_lua ' ngx.req.read_body(); local b = ngx.req.get_body_data(); - f = io.open(ngx.var.realpath_root.."/test.lua", "w"); + local f = io.open(ngx.var.realpath_root.."/test.lua", "w"); -- luajit bytecode: sub(149,-1), lua bytecode: sub(1,147) if package.loaded["jit"] then f:write(string.sub(b, 149)); else f:write(string.sub(b, 1, 147)); end - f:close(); res = ngx.location.capture("/call"); + f:close(); + local res = ngx.location.capture("/call"); if res.status == 200 then ngx.print(res.body) else @@ -132,7 +135,7 @@ cannot load incompatible bytecode content_by_lua ' ngx.req.read_body(); local b = ngx.req.get_body_data(); - f = io.open(ngx.var.realpath_root.."/test.lua", "w"); + local f = io.open(ngx.var.realpath_root.."/test.lua", "w"); -- luajit bytecode: sub(149,-1), lua bytecode: sub(1,147) local do_jit if jit then @@ -176,7 +179,7 @@ cannot load incompatible bytecode content_by_lua ' ngx.req.read_body(); local b = ngx.req.get_body_data(); - f = io.open(ngx.var.realpath_root.."/test.lua", "w"); + local f = io.open(ngx.var.realpath_root.."/test.lua", "w"); -- luajit bytecode: sub(149,-1), lua bytecode: sub(1,147) local jit; if package.loaded["jit"] then @@ -220,7 +223,7 @@ error content_by_lua ' ngx.req.read_body(); local b = ngx.req.get_body_data(); - f = io.open(ngx.var.realpath_root.."/test.lua", "w"); + local f = io.open(ngx.var.realpath_root.."/test.lua", "w"); -- luajit bytecode: sub(149,-1), lua bytecode: sub(1,147) if jit then if not string.find(jit.version, "LuaJIT 2.0") then diff --git a/t/082-body-filter.t b/t/082-body-filter.t index 98efa84847..5f765fa439 100644 --- a/t/082-body-filter.t +++ b/t/082-body-filter.t @@ -460,7 +460,8 @@ GET /t --- config location /t { body_filter_by_lua ' - function foo() + local bar + local function foo() bar() end diff --git a/t/084-inclusive-receiveuntil.t b/t/084-inclusive-receiveuntil.t index f7d5d3d189..66be893d5c 100644 --- a/t/084-inclusive-receiveuntil.t +++ b/t/084-inclusive-receiveuntil.t @@ -56,7 +56,7 @@ __DATA__ local reader = sock:receiveuntil("abcabd", { inclusive = true }) for i = 1, 3 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -128,7 +128,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabdabcabe", { inclusive = true }) for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -200,7 +200,7 @@ close: 1 nil local reader = sock:receiveuntil("abcabd", { inclusive = true }) for i = 1, 3 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -272,7 +272,7 @@ close: 1 nil local reader = sock:receiveuntil("aa", { inclusive = nil }) for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -343,7 +343,7 @@ close: 1 nil local reader = sock:receiveuntil("aa", { inclusive = false }) for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -414,7 +414,7 @@ close: 1 nil local reader = sock:receiveuntil("aa", { inclusive = true }) for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -485,7 +485,7 @@ close: 1 nil local reader = sock:receiveuntil("aa", { inclusive = "true" }) for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -552,7 +552,7 @@ bad "inclusive" option value type: string local reader = sock:receiveuntil("aa", { inclusive = "true" }) for i = 1, 2 do - line, err, part = reader() + local line, err, part = reader() if line then ngx.say("read: ", line) @@ -620,7 +620,7 @@ bad "inclusive" option value type: string local reader = sock:receiveuntil("--abc", { inclusive = true }) for i = 1, 7 do - line, err, part = reader(4) + local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -697,7 +697,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc", { inclusive = true }) for i = 1, 7 do - line, err, part = reader(4) + local line, err, part = reader(4) if line then ngx.say("read: ", line) diff --git a/t/091-coroutine.t b/t/091-coroutine.t index bceb512460..7e6918d116 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -85,7 +85,7 @@ __DATA__ content_by_lua ' local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield - function f() + local function f() local cnt = 0 for i = 1, 20 do ngx.say("Hello, ", cnt) @@ -120,7 +120,7 @@ Hello, 2 --- config location /lua { content_by_lua ' - function f(fid) + local function f(fid) local cnt = 0 while true do ngx.say("cc", fid, ": ", cnt) @@ -163,7 +163,7 @@ cc3: 2 resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' - function worker(url) + local function worker(url) local sock = ngx.socket.tcp() local ok, err = sock:connect(url, 80) coroutine.yield() @@ -218,14 +218,14 @@ successfully connected to: openresty.org location /lua { content_by_lua ' -- generate all the numbers from 2 to n - function gen (n) + local function gen (n) return coroutine.wrap(function () for i=2,n do coroutine.yield(i) end end) end -- filter the numbers generated by g, removing multiples of p - function filter (p, g) + local function filter (p, g) return coroutine.wrap(function () while 1 do local n = g() @@ -235,8 +235,8 @@ successfully connected to: openresty.org end) end - N = 10 - x = gen(N) -- generate primes up to N + local N = 10 + local x = gen(N) -- generate primes up to N while 1 do local n = x() -- pick a number until done if n == nil then break end @@ -264,14 +264,14 @@ GET /lua coroutine.create = nil coroutine.resume = nil -- generate all the numbers from 2 to n - function gen (n) + local function gen (n) return coroutine.wrap(function () for i=2,n do coroutine.yield(i) end end) end -- filter the numbers generated by g, removing multiples of p - function filter (p, g) + local function filter (p, g) return coroutine.wrap(function () while 1 do local n = g() @@ -281,8 +281,8 @@ GET /lua end) end - N = 10 - x = gen(N) -- generate primes up to N + local N = 10 + local x = gen(N) -- generate primes up to N while 1 do local n = x() -- pick a number until done if n == nil then break end @@ -307,7 +307,7 @@ GET /lua --- config location /lua { content_by_lua ' - function generatefib (n) + local function generatefib (n) return coroutine.wrap(function () local a,b = 1, 1 while a <= n do @@ -362,7 +362,7 @@ GET /lua resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' - function worker(url) + local function worker(url) local sock = ngx.socket.tcp() local ok, err = sock:connect(url, 80) coroutine.yield() @@ -414,7 +414,7 @@ successfully connected to: openresty.org local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield local st, rn = coroutine.status, coroutine.running - function f(self) + local function f(self) local cnt = 0 if rn() ~= self then ngx.say("error"); return end ngx.say("running: ", st(self)) --running @@ -509,7 +509,7 @@ GET /lua --- config location /lua { content_by_lua ' - function print(...) + local function print(...) local args = {...} local is_first = true for i,v in ipairs(args) do @@ -523,12 +523,12 @@ GET /lua ngx.print("\\\n") end - function foo (a) + local function foo (a) print("foo", a) return coroutine.yield(2*a) end - co = coroutine.create(function (a,b) + local co = coroutine.create(function (a,b) print("co-body", a, b) local r = foo(a+1) print("co-body", r) @@ -566,7 +566,8 @@ main false cannot resume dead coroutine local create = coroutine.create local resume = coroutine.resume local yield = coroutine.yield - function f() + local g + local function f() ngx.say("f begin") yield() local c2 = create(g) @@ -627,8 +628,9 @@ main done local yield = coroutine.yield local code = 400 + local g - function f() + local function f() local c2 = create(g) yield() code = code + 1 @@ -683,8 +685,9 @@ exit local yield = coroutine.yield local code = 0 + local g - function f() + local function f() local c2 = create(g) yield() code = code + 1 @@ -740,7 +743,7 @@ num: 3 location /lua { echo hello; header_filter_by_lua ' - function f() + local function f() yield() end @@ -764,13 +767,13 @@ API disabled in the context of header_filter_by_lua* local c1, c2 - function f() + local function f() print("f 1") print(coroutine.resume(c2)) print("f 2") end - function g() + local function g() print("g 1") -- print(coroutine.resume(c1)) print("g 2") @@ -803,13 +806,13 @@ f 2 local c1, c2 - function f() + local function f() print("f 1") print(coroutine.resume(c2)) print("f 2") end - function g() + local function g() print("g 1") print(coroutine.resume(c1)) print("g 2") @@ -859,7 +862,7 @@ falsecannot resume running coroutine location /t { content_by_lua ' local co - function f() + local function f() ngx.say("f: ", coroutine.status(co)) ngx.say("f: ", coroutine.resume(co)) end @@ -885,7 +888,7 @@ chunk: true location /t { content_by_lua ' local co - function f() + local function f() error("bad") end co = coroutine.create(f) @@ -990,7 +993,7 @@ test10 resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' - function worker(url) + local function worker(url) local sock = ngx.socket.tcp() local ok, err = sock:connect(url, 80) coroutine.yield() @@ -1044,7 +1047,7 @@ successfully connected to: agentzh.org resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { content_by_lua ' - function worker(url) + local function worker(url) local sock = ngx.socket.tcp() local ok, err = sock:connect(url, 80) coroutine.yield() @@ -1104,7 +1107,7 @@ successfully connected to: agentzh.org --- config location /cotest { content_by_lua ' - function generator() + local function generator() return co_wrap(function() co_yield("data") end) @@ -1133,7 +1136,7 @@ data --- config location /cotest { content_by_lua ' - function generator() + local function generator() return co_wrap(function() co_yield("data") end) @@ -1166,7 +1169,7 @@ data content_by_lua ' local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield - function f() + local function f() return 3 end @@ -1199,7 +1202,7 @@ ok local coroutine = require "coroutine" local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield - function f() + local function f() local cnt = 0 for i = 1, 20 do ngx.say("Hello, ", cnt) @@ -1238,7 +1241,7 @@ Hello, 2 header_filter_by_lua ' local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield - function f() + local function f() local cnt = 0 for i = 1, 20 do print("co yield: ", cnt) @@ -1278,7 +1281,7 @@ co yield: 2 body_filter_by_lua ' local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield - function f() + local function f() local cnt = 0 for i = 1, 20 do print("co yield: ", cnt) diff --git a/t/093-uthread-spawn.t b/t/093-uthread-spawn.t index b622d30123..ab6db2ff26 100644 --- a/t/093-uthread-spawn.t +++ b/t/093-uthread-spawn.t @@ -24,7 +24,7 @@ __DATA__ --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") end @@ -58,11 +58,11 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("in thread 1") end - function g() + local function g() ngx.say("in thread 2") end @@ -107,7 +107,7 @@ after 2 --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("before sleep") ngx.sleep(0.1) ngx.say("after sleep") @@ -144,13 +144,13 @@ after sleep --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("1: before sleep") ngx.sleep(0.2) ngx.say("1: after sleep") end - function g() + local function g() ngx.say("2: before sleep") ngx.sleep(0.1) ngx.say("2: after sleep") @@ -200,7 +200,7 @@ delete thread 2 --- config location /lua { content_by_lua ' - function f() + local function f() ngx.blah() end @@ -231,9 +231,9 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):3: --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("before capture") - res = ngx.location.capture("/proxy") + local res = ngx.location.capture("/proxy") ngx.say("after capture: ", res.body) end @@ -277,7 +277,7 @@ after capture: hello world --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("after capture: ", res.body) @@ -330,7 +330,7 @@ after capture: hello foo --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("after capture: ", res.body) @@ -384,13 +384,13 @@ capture: hello bar --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("f: before capture") local res = ngx.location.capture("/proxy?foo") ngx.say("f: after capture: ", res.body) end - function g() + local function g() ngx.say("g: before capture") local res = ngx.location.capture("/proxy?bah") ngx.say("g: after capture: ", res.body) @@ -462,7 +462,8 @@ g: after capture: hello bah --- config location /lua { content_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -508,7 +509,8 @@ after g --- config location /lua { content_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -556,7 +558,7 @@ hello in g() location /lua { content_by_lua ' local co - function f() + local function f() co = coroutine.running() ngx.sleep(0.1) end @@ -589,7 +591,7 @@ status: running location /lua { content_by_lua ' local co - function f() + local function f() co = coroutine.running() end @@ -621,7 +623,8 @@ status: zombie location /lua { content_by_lua ' local co - function f() + local g + local function f() co = coroutine.running() local co2 = coroutine.create(g) coroutine.resume(co2) @@ -660,7 +663,8 @@ status: normal --- config location /lua { content_by_lua ' - function f() + local g + local function f() ngx.say("before g") ngx.thread.spawn(g) ngx.say("after g") @@ -707,7 +711,7 @@ after f content_by_lua ' local yield = coroutine.yield - function f() + local function f() local self = coroutine.running() ngx.say("f 1") yield(self) @@ -760,7 +764,7 @@ f 3 content_by_lua ' local yield = coroutine.yield - function f() + local function f() local self = coroutine.running() ngx.say("f 1") yield(self) @@ -769,7 +773,7 @@ f 3 ngx.say("f 3") end - function g() + local function g() local self = coroutine.running() ngx.say("g 1") yield(self) @@ -816,7 +820,7 @@ g 3 --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") coroutine.yield(coroutine.running) ngx.flush(true) @@ -853,12 +857,12 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello from f") ngx.flush(true) end - function g() + local function g() ngx.say("hello from g") ngx.flush(true) end @@ -904,7 +908,7 @@ hello from g --- config location /lua { content_by_lua ' - function f() + local function f() local sock = ngx.socket.tcp() local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) if not ok then @@ -956,7 +960,7 @@ received: OK --- config location /lua { content_by_lua ' - function f() + local function f() local sock = ngx.socket.udp() local ok, err = sock:setpeername("127.0.0.1", 12345) local bytes, err = sock:send("blah") @@ -1018,7 +1022,7 @@ after)$ --- config location /lua { content_by_lua ' - function f() + local function f() ngx.req.read_body() local body = ngx.req.get_body_data() ngx.say("body: ", body) @@ -1063,7 +1067,7 @@ body: hello world)$ --- config location /lua { content_by_lua ' - function f() + local function f() local sock = ngx.req.socket() local body, err = sock:receive(11) if not body then @@ -1113,7 +1117,7 @@ body: hello world)$ --- config location /lua { content_by_lua ' - function f(a, b) + local function f(a, b) ngx.say("hello ", a, " and ", b) end @@ -1647,7 +1651,7 @@ ok --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f") end diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index 0b31a800b8..d4e4a7c7b1 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -24,7 +24,7 @@ __DATA__ --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.exit(0) end @@ -87,7 +87,7 @@ hello in thread --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -171,13 +171,13 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f") ngx.exit(0) end - function g() + local function g() ngx.sleep(1) ngx.say("g") end @@ -261,7 +261,7 @@ f --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("exiting the user thread") ngx.exit(0) @@ -300,7 +300,7 @@ exiting the user thread resolver agentzh.org; resolver_timeout 12s; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.001) ngx.exit(0) @@ -396,7 +396,7 @@ after resolver agentzh.org; resolver_timeout 12s; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.001) ngx.exit(0) @@ -490,7 +490,7 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -580,7 +580,7 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -680,7 +680,7 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -786,7 +786,7 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -881,7 +881,7 @@ after --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -974,7 +974,7 @@ after location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.say("hello in thread") ngx.sleep(0.1) ngx.exit(0) @@ -1142,7 +1142,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exit(0) end @@ -1225,7 +1225,7 @@ attempt to abort with pending subrequests location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exit(0) end diff --git a/t/095-uthread-exec.t b/t/095-uthread-exec.t index ad0c1888a0..7bf37c0e8d 100644 --- a/t/095-uthread-exec.t +++ b/t/095-uthread-exec.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { content_by_lua ' - function f() + local function f() ngx.exec("/foo") end @@ -58,7 +58,7 @@ i am foo --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end @@ -93,7 +93,7 @@ i am foo --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end @@ -175,12 +175,12 @@ hello foo --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end - function g() + local function g() ngx.sleep(1) end @@ -267,7 +267,7 @@ hello foo location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.exec("/foo") end diff --git a/t/096-uthread-redirect.t b/t/096-uthread-redirect.t index 73b2bf1eec..95bff01f3a 100644 --- a/t/096-uthread-redirect.t +++ b/t/096-uthread-redirect.t @@ -25,7 +25,7 @@ __DATA__ location /lua { client_body_timeout 12000ms; content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.redirect(301) end @@ -114,7 +114,7 @@ attempt to abort with pending subrequests --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.redirect(301) end diff --git a/t/097-uthread-rewrite.t b/t/097-uthread-rewrite.t index 2f0c06fb43..998e256221 100644 --- a/t/097-uthread-rewrite.t +++ b/t/097-uthread-rewrite.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.req.set_uri("/foo", true) end @@ -58,7 +58,7 @@ i am foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.req.set_uri("/foo", true) end @@ -93,7 +93,7 @@ i am foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.req.set_uri("/foo", true) end @@ -175,12 +175,12 @@ hello foo --- config location /lua { rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.req.set_uri("/foo", true) end - function g() + local function g() ngx.sleep(1) end @@ -267,7 +267,7 @@ hello foo location /lua { client_body_timeout 12000ms; rewrite_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.req.set_uri("/foo", true) end diff --git a/t/098-uthread-wait.t b/t/098-uthread-wait.t index 4948596f91..c2819ae886 100644 --- a/t/098-uthread-wait.t +++ b/t/098-uthread-wait.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") return "done" end @@ -72,7 +72,7 @@ done --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("hello in thread") return "done" @@ -120,13 +120,13 @@ done --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f: hello") return "done" end - function g() + local function g() ngx.sleep(0.2) ngx.say("g: hello") return "done" @@ -200,13 +200,13 @@ f: done --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f: hello") return "done" end - function g() + local function g() ngx.sleep(0.2) ngx.say("g: hello") return "done" @@ -228,7 +228,7 @@ f: done ngx.say("g thread created: ", coroutine.status(tg)) - ok, res = ngx.thread.wait(tf) + local ok, res = ngx.thread.wait(tf) if not ok then ngx.say("failed to wait f: ", res) return @@ -281,7 +281,7 @@ g: done --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") return "done", 3.14 end @@ -330,7 +330,7 @@ res: done 3.14 --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("hello in thread") return "done", 3.14 @@ -378,7 +378,7 @@ res: done 3.14 --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") error("bad bad!") end @@ -427,7 +427,7 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):4: --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("hello in thread") error("bad bad!") @@ -477,12 +477,12 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):5: --- config location /lua { content_by_lua ' - function g() + local function g() ngx.say("hello in thread") return "done" end - function f() + local function f() local t, err = ngx.thread.spawn(g) if not t then ngx.say("failed to spawn thread: ", err) @@ -533,13 +533,13 @@ done --- config location /lua { content_by_lua ' - function g() + local function g() ngx.sleep(0.1) ngx.say("hello in thread") return "done" end - function f() + local function f() local t, err = ngx.thread.spawn(g) if not t then ngx.say("failed to spawn thread: ", err) @@ -593,12 +593,12 @@ done -- local out = function (...) ngx.log(ngx.ERR, ...) end local out = ngx.say - function f() + local function f() out("f: hello") return "f done" end - function g() + local function g() out("g: hello") return "g done" end @@ -668,13 +668,13 @@ g status: zombie -- local out = function (...) ngx.log(ngx.ERR, ...) end local out = ngx.say - function f() + local function f() ngx.sleep(0.1) out("f: hello") return "f done" end - function g() + local function g() ngx.sleep(0.2) out("g: hello") return "g done" @@ -745,13 +745,13 @@ g: hello -- local out = function (...) ngx.log(ngx.ERR, ...) end local out = ngx.say - function f() + local function f() ngx.sleep(0.2) out("f: hello") return "f done" end - function g() + local function g() ngx.sleep(0.1) out("g: hello") return "g done" @@ -822,12 +822,12 @@ f: hello -- local out = function (...) ngx.log(ngx.ERR, ...) end local out = ngx.say - function f() + local function f() out("f: hello") error("f done") end - function g() + local function g() out("g: hello") error("g done") end @@ -896,13 +896,13 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):7: -- local out = function (...) ngx.log(ngx.ERR, ...) end local out = ngx.say - function f() + local function f() ngx.sleep(0.1) out("f: hello") error("f done") end - function g() + local function g() ngx.sleep(0.2) out("g: hello") error("g done") @@ -969,13 +969,13 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):8: --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) ngx.say("f: hello") return "done" end - function g() + local function g() ngx.sleep(0.2) ngx.say("g: hello") return "done" @@ -997,7 +997,7 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):8: ngx.say("g thread created: ", coroutine.status(tg)) - ok, res = ngx.thread.wait(tf, tg) + local ok, res = ngx.thread.wait(tf, tg) if not ok then ngx.say("failed to wait: ", res) return @@ -1038,13 +1038,13 @@ res: done --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.2) ngx.say("f: hello") return "f done" end - function g() + local function g() ngx.sleep(0.1) ngx.say("g: hello") return "g done" @@ -1066,7 +1066,7 @@ res: done ngx.say("g thread created: ", coroutine.status(tg)) - ok, res = ngx.thread.wait(tf, tg) + local ok, res = ngx.thread.wait(tf, tg) if not ok then ngx.say("failed to wait: ", res) return @@ -1109,12 +1109,12 @@ res: g done content_by_lua ' local t - function f() + local function f() ngx.sleep(0.1) return "done" end - function g() + local function g() t = ngx.thread.spawn(f) end @@ -1154,7 +1154,7 @@ only the parent coroutine can wait on the thread --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) coroutine.yield() return "done" @@ -1192,7 +1192,7 @@ qr/lua entry thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):11 --- config location /lua { content_by_lua ' - function f() + local function f() ngx.sleep(0.1) collectgarbage() error("f done") @@ -1228,7 +1228,7 @@ qr/lua user thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):5: --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello in thread") return "done" end @@ -1283,7 +1283,7 @@ failed to run thread: already waited or killed --- config location /lua { content_by_lua ' - function f() + local function f() -- ngx.say("hello in thread") return "done" end diff --git a/t/099-c-api.t b/t/099-c-api.t index a0b375c810..50334626bd 100644 --- a/t/099-c-api.t +++ b/t/099-c-api.t @@ -86,7 +86,7 @@ dogs zone: defined local buf = ffi.new("char[?]", 4) ffi.copy(buf, "dogs", 4) - zone = ffi.C.ngx_http_lua_find_zone(buf, 4) + local zone = ffi.C.ngx_http_lua_find_zone(buf, 4) local val = ffi.new("ngx_http_lua_value_t[?]", 1) @@ -149,7 +149,7 @@ bar: rc=0, type=3, val=3.14159 local buf = ffi.new("char[?]", 4) ffi.copy(buf, "dogs", 4) - zone = ffi.C.ngx_http_lua_find_zone(buf, 4) + local zone = ffi.C.ngx_http_lua_find_zone(buf, 4) local val = ffi.new("ngx_http_lua_value_t[?]", 1) @@ -212,7 +212,7 @@ bar: rc=0, type=1, val=0 local buf = ffi.new("char[?]", 4) ffi.copy(buf, "dogs", 4) - zone = ffi.C.ngx_http_lua_find_zone(buf, 4) + local zone = ffi.C.ngx_http_lua_find_zone(buf, 4) local val = ffi.new("ngx_http_lua_value_t[?]", 1) @@ -272,7 +272,7 @@ bar: rc=-5 local buf = ffi.new("char[?]", 4) ffi.copy(buf, "dogs", 4) - zone = ffi.C.ngx_http_lua_find_zone(buf, 4) + local zone = ffi.C.ngx_http_lua_find_zone(buf, 4) local s = ffi.new("char[?]", 20) @@ -344,7 +344,7 @@ bar: rc=0, type=4, val=, len=0 local buf = ffi.new("char[?]", 4) ffi.copy(buf, "dogs", 4) - zone = ffi.C.ngx_http_lua_find_zone(buf, 4) + local zone = ffi.C.ngx_http_lua_find_zone(buf, 4) local val = ffi.new("ngx_http_lua_value_t[?]", 1) diff --git a/t/100-client-abort.t b/t/100-client-abort.t index 89c1f6ac08..a5cba319e0 100644 --- a/t/100-client-abort.t +++ b/t/100-client-abort.t @@ -540,7 +540,7 @@ client prematurely closed connection return end - ok, err = sock:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT) + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err) return diff --git a/t/106-timer.t b/t/106-timer.t index fa25e18795..bb5b73a8db 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -1023,7 +1023,7 @@ qr/go\(\): connected: 1, reused: \d+/, content_by_lua ' local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield local function f() - function f() + local function f() local cnt = 0 for i = 1, 20 do print("cnt = ", cnt) @@ -1090,7 +1090,7 @@ registered timer ngx.log(ngx.ERR, ...) end local function handle() - function f() + local function f() print("hello in thread") return "done" end diff --git a/t/108-timer-safe.t b/t/108-timer-safe.t index 2795998a51..30589c7cfc 100644 --- a/t/108-timer-safe.t +++ b/t/108-timer-safe.t @@ -1007,7 +1007,7 @@ registered timer ngx.log(ngx.ERR, ...) end local function handle() - function f() + local function f() print("hello in thread") return "done" end diff --git a/t/109-timer-hup.t b/t/109-timer-hup.t index 0864046361..f0ef1cf938 100644 --- a/t/109-timer-hup.t +++ b/t/109-timer-hup.t @@ -363,7 +363,7 @@ lua found 100 pending timers local line, err = sock:receive("*l") until not line or string.find(line, "^%s*$") - function foo() + local function foo() repeat -- Get and read chunk local line, err = sock:receive("*l") @@ -379,7 +379,7 @@ lua found 100 pending timers until len == 0 end - co = coroutine.create(foo) + local co = coroutine.create(foo) repeat local chunk = select(2,coroutine.resume(co)) until chunk == nil diff --git a/t/120-re-find.t b/t/120-re-find.t index 73e6134fd7..f80ca4f57d 100644 --- a/t/120-re-find.t +++ b/t/120-re-find.t @@ -622,7 +622,7 @@ matched: 你 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() @@ -664,7 +664,7 @@ error: pcre_exec() failed: -8 >>> a.lua local re = [==[(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))]==] -s = string.rep([[ABCDEFG]], 10) +local s = string.rep([[ABCDEFG]], 10) local start = ngx.now() diff --git a/t/126-shdict-frag.t b/t/126-shdict-frag.t index 5a2aff8202..33646d17c9 100644 --- a/t/126-shdict-frag.t +++ b/t/126-shdict-frag.t @@ -1240,8 +1240,8 @@ failed to safe set baz: no memory local key = "mylittlekey" .. rand(maxkeyidx) local ok, err = dogs:get(key) if not ok or rand() > 0.6 then - sz = rand(maxsz) - val = rep("a", sz) + local sz = rand(maxsz) + local val = rep("a", sz) local ok, err, forcible = dogs:set(key, val) if err then ngx.log(ngx.ERR, "failed to set key: ", err) diff --git a/t/127-uthread-kill.t b/t/127-uthread-kill.t index cc43c62764..a919b3d1de 100644 --- a/t/127-uthread-kill.t +++ b/t/127-uthread-kill.t @@ -23,7 +23,7 @@ __DATA__ --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello from f()") ngx.sleep(1) end @@ -81,7 +81,7 @@ lua clean up the timer for pending ngx.sleep --- config location /lua { content_by_lua ' - function f() + local function f() ngx.say("hello from f()") ngx.sleep(0.001) return 32 @@ -141,7 +141,7 @@ lua clean up the timer for pending ngx.sleep resolver agentzh.org:12345; location /lua { content_by_lua ' - function f() + local function f() local sock = ngx.socket.tcp() sock:connect("some.agentzh.org", 12345) end @@ -194,7 +194,7 @@ resolve name done: -2 location /lua { content_by_lua ' local ready = false - function f() + local function f() local sock = ngx.socket.tcp() sock:connect("agentzh.org", 80) sock:close() @@ -262,7 +262,7 @@ lua finalize socket location = /t { content_by_lua ' - function f() + local function f() ngx.location.capture("/sub") end @@ -309,11 +309,11 @@ lua tcp socket abort resolver location = /t { content_by_lua ' - function f() + local function f() ngx.location.capture("/sub") end - function g() + local function g() ngx.sleep(0.3) end @@ -376,7 +376,7 @@ lua tcp socket abort resolver location = /t { content_by_lua ' local ready = false - function f() + local function f() ngx.location.capture("/sub") ready = true ngx.sleep(0.5) @@ -424,7 +424,7 @@ lua tcp socket abort resolver --- config location = /t { content_by_lua ' - function f() + local function f() return end @@ -473,7 +473,7 @@ lua tcp socket abort resolver ngx.say("killed main thread.") end - function f() + local function f() local ok, err = ngx.thread.kill(coroutine.running()) if not ok then ngx.say("failed to kill user thread: ", err) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index a65d7038e3..bebbdb27ae 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -1278,7 +1278,7 @@ lua ssl server name: "test.com" listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; ssl_certificate_by_lua_block { - function f() + local function f() ngx.sleep(0.01) print("uthread: hello in thread") return "done" diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 4ebc8f370e..2fa3f4e3c0 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -411,7 +411,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 local chunk = 4 - function read() + local function read() sock:settimeout(200) -- read: 200 ms local data, err = sock:receive(content_length) @@ -506,7 +506,7 @@ failed to receive data: timeout local chunk = 4 - function read() + local function read() local data, err = sock:receive(content_length) if not data then ngx.log(ngx.ERR, "failed to receive data: ", err) diff --git a/t/153-semaphore-hup.t b/t/153-semaphore-hup.t index c85a21dae4..5c271a48e7 100644 --- a/t/153-semaphore-hup.t +++ b/t/153-semaphore-hup.t @@ -67,7 +67,7 @@ add_block_preprocessor(sub { return end - shdict = ngx.shared.shdict + local shdict = ngx.shared.shdict local success = shdict:add("reloaded", 1) if not success then return From 08a9baa92b64ac06d2c82d1af25298fc36f46264 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Wed, 7 Nov 2018 03:49:39 +0800 Subject: [PATCH 254/848] bugfix: ngx.process: process.type() didn't return 'master' in master process. --- src/ngx_http_lua_worker.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 461509b436..33cfa11968 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -153,6 +153,8 @@ ngx_http_lua_ffi_master_pid(void) int ngx_http_lua_ffi_get_process_type(void) { + ngx_core_conf_t *ccf; + #if defined(HAVE_PRIVILEGED_PROCESS_PATCH) && !NGX_WIN32 if (ngx_process == NGX_PROCESS_HELPER) { if (ngx_is_privileged_agent) { @@ -161,6 +163,15 @@ ngx_http_lua_ffi_get_process_type(void) } #endif + if (ngx_process == NGX_PROCESS_SINGLE) { + ccf = (ngx_core_conf_t *) ngx_get_conf(ngx_cycle->conf_ctx, + ngx_core_module); + + if (ccf->master) { + return NGX_PROCESS_MASTER; + } + } + return ngx_process; } From 1a73e0ab79b11f297b8e9193eb0a91f90049ce64 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 8 Nov 2018 14:06:03 -0800 Subject: [PATCH 255/848] doc: ported the Lua 5.1 interpreter removal to the wiki document. This commit ports changes from 5efd95d793bad0587f75a7d0f701b76693c1311d to the HttpLuaModule.wiki file, with a few minor edits. The wiki2markdown.pl script was also run, and caught a few leftovers in the README.markdown file. --- README.markdown | 3 +-- doc/HttpLuaModule.wiki | 41 +++++------------------------------------ 2 files changed, 6 insertions(+), 38 deletions(-) diff --git a/README.markdown b/README.markdown index 29d7df88be..d756351a1b 100644 --- a/README.markdown +++ b/README.markdown @@ -23,7 +23,6 @@ Table of Contents * [Installation](#installation) * [Building as a dynamic module](#building-as-a-dynamic-module) * [C Macro Configurations](#c-macro-configurations) - * [Installation on Ubuntu 11.10](#installation-on-ubuntu-1110) * [Community](#community) * [English Mailing List](#english-mailing-list) * [Chinese Mailing List](#chinese-mailing-list) @@ -269,7 +268,7 @@ It is *highly* recommended to use [OpenResty releases](http://openresty.org) whi Alternatively, ngx_lua can be manually compiled into Nginx: -1. LuaJIT can be downloaded the [latest release of OpenResty's LuaJIT branch version](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.0 and 2.1 releases are also supported though the performance will be signficantly lower in many cases. +1. LuaJIT can be downloaded from the [latest release of OpenResty's LuaJIT branch version](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.0 and 2.1 releases are also supported, although the performance will be significantly lower in many cases. 1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags). 1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags). 1. Download the latest version of Nginx [HERE](http://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 8bdc9a1cdc..f964274217 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -130,7 +130,7 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t = Description = -This module embeds Lua, via the standard Lua 5.1 interpreter or [http://luajit.org/luajit.html LuaJIT 2.0/2.1], into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. +This module embeds Lua, via [http://luajit.org/luajit.html LuaJIT 2.0/2.1], into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. Unlike [https://httpd.apache.org/docs/trunk/mod/mod_lua.html Apache's mod_lua] and [http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet Lighttpd's mod_magnet], Lua code executed using this module can be ''100% non-blocking'' on network traffic as long as the [[#Nginx API for Lua|Nginx API for Lua]] provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services. @@ -199,11 +199,11 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. +It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, OpenResty's LuaJIT 2.1 branch version, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. Alternatively, ngx_lua can be manually compiled into Nginx: -# Install LuaJIT 2.0 or 2.1 (recommended) or Lua 5.1 (Lua 5.2 is ''not'' supported yet). LuaJIT can be downloaded from the [http://luajit.org/download.html LuaJIT project website] and Lua 5.1, from the [http://www.lua.org/ Lua project website]. Some distribution package managers also distribute LuaJIT and/or Lua. +# LuaJIT can be downloaded from the [https://github.com/openresty/luajit2/releases latest release of OpenResty's LuaJIT branch version]. The official LuaJIT 2.0 and 2.1 releases are also supported, although the performance will be significantly lower in many cases. # Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE]. # Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE]. # Download the latest version of Nginx [http://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) @@ -271,24 +271,6 @@ To enable one or more of these macros, just pass extra C compiler options to the ./configure --with-cc-opt="-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC" -== Installation on Ubuntu 11.10 == - -Note that it is recommended to use LuaJIT 2.0 or LuaJIT 2.1 instead of the standard Lua 5.1 interpreter wherever possible. - -If the standard Lua 5.1 interpreter is required however, run the following command to install it from the Ubuntu repository: - - -apt-get install -y lua5.1 liblua5.1-0 liblua5.1-0-dev - - -Everything should be installed correctly, except for one small tweak. - -Library name liblua.so has been changed in liblua5.1 package, it only comes with liblua5.1.so, which needs to be symlinked to /usr/lib so it could be found during the configuration process. - - -ln -s /usr/lib/x86_64-linux-gnu/liblua5.1.so /usr/lib/liblua.so - - = Community = == English Mailing List == @@ -314,7 +296,7 @@ Please submit bug reports, wishlists, or patches by As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as [[#content_by_lua_file|content_by_lua_file]]) support loading Lua 5.1 and LuaJIT 2.0/2.1 raw bytecode files directly. -Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: +Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if you are using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc @@ -332,18 +314,6 @@ http://luajit.org/running.html#opt_b Also, the bytecode files generated by LuaJIT 2.1 is ''not'' compatible with LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added in ngx_lua v0.9.3. -Similarly, if using the standard Lua 5.1 interpreter with ngx_lua, Lua compatible bytecode files must be generated using the luac commandline utility as shown: - - - luac -o /path/to/output_file.luac /path/to/input_file.lua - - -Unlike as with LuaJIT, debug information is included in standard Lua 5.1 bytecode files by default. This can be striped out by specifying the -s option as shown: - - - luac -s -o /path/to/output_file.luac /path/to/input_file.lua - - Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked to LuaJIT 2.0/2.1 or vice versa, will result in an error message, such as that below, being logged into the Nginx error.log file: @@ -510,8 +480,7 @@ However, later attempts to manipulate the cosocket object will fail and return t This issue is due to limitations in the Nginx event model and only appears to affect Mac OS X. == Lua Coroutine Yielding/Resuming == -* Because Lua's dofile and require builtins are currently implemented as C functions in both Lua 5.1 and LuaJIT 2.0/2.1, if the Lua file being loaded by dofile or require invokes [[#ngx.location.capture|ngx.location.capture*]], [[#ngx.exec|ngx.exec]], [[#ngx.exit|ngx.exit]], or other API functions requiring yielding in the *top-level* scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file. -* As the standard Lua 5.1 interpreter's VM is not fully resumable, the methods [[#ngx.location.capture|ngx.location.capture]], [[#ngx.location.capture_multi|ngx.location.capture_multi]], [[#ngx.redirect|ngx.redirect]], [[#ngx.exec|ngx.exec]], and [[#ngx.exit|ngx.exit]] cannot be used within the context of a Lua [http://www.lua.org/manual/5.1/manual.html#pdf-pcall pcall()] or [http://www.lua.org/manual/5.1/manual.html#pdf-xpcall xpcall()] or even the first line of the for ... in ... statement when the standard Lua 5.1 interpreter is used and the attempt to yield across metamethod/C-call boundary error will be produced. Please use LuaJIT 2.x, which supports a fully resumable VM, to avoid this. +* Because Lua's dofile and require builtins are currently implemented as C functions in LuaJIT 2.0/2.1, if the Lua file being loaded by dofile or require invokes [[#ngx.location.capture|ngx.location.capture*]], [[#ngx.exec|ngx.exec]], [[#ngx.exit|ngx.exit]], or other API functions requiring yielding in the *top-level* scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file. == Lua Variable Scope == Care must be taken when importing modules and this form should be used: From 3754757be7acd4d8118bbfa0d10a334e4f45875a Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 8 Nov 2018 17:12:31 -0800 Subject: [PATCH 256/848] change: we now print an alert when a non openresty-specific version of LuaJIT is detected since many optimizations would be missing. --- src/ngx_http_lua_module.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 51e4676774..c954fb6676 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -746,6 +746,19 @@ ngx_http_lua_init(ngx_conf_t *cf) if (lmcf->lua == NULL) { dd("initializing lua vm"); +#ifndef OPENRESTY_LUAJIT + if (ngx_process != NGX_PROCESS_SIGNALLER && !ngx_test_config) { + ngx_log_error(NGX_LOG_ALERT, cf->log, 0, + "detected a LuaJIT version which is not OpenResty's" + "; many optimizations will be disabled and " + "performance will be compromised (see " + "https://github.com/openresty/luajit2 for " + "OpenResty's LuaJIT or, even better, consider using " + "the OpenResty releases from https://openresty.org/" + "en/download.html)"); + } +#endif + ngx_http_lua_content_length_hash = ngx_http_lua_hash_literal("content-length"); ngx_http_lua_location_hash = ngx_http_lua_hash_literal("location"); From c65f5738a21b57678dee18d2fcfed482e6a4681b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sat, 10 Nov 2018 05:56:01 +0800 Subject: [PATCH 257/848] bugfix: the 'dd' calls in ngx_http_lua_util.h are now guarded by '#ifdef DDEBUG' . --- src/ngx_http_lua_util.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 052aa1737e..eb50f3a743 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -9,6 +9,11 @@ #define _NGX_HTTP_LUA_UTIL_H_INCLUDED_ +#ifdef DDEBUG +#include "ddebug.h" +#endif + + #include "ngx_http_lua_common.h" #include "ngx_http_lua_api.h" @@ -284,7 +289,9 @@ ngx_http_lua_create_ctx(ngx_http_request_t *r) if (!llcf->enable_code_cache && r->connection->fd != (ngx_socket_t) -1) { lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); +#ifdef DDEBUG dd("lmcf: %p", lmcf); +#endif L = ngx_http_lua_init_vm(lmcf->lua, lmcf->cycle, r->pool, lmcf, r->connection->log, &cln); @@ -325,7 +332,11 @@ ngx_http_lua_get_lua_vm(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) } lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + +#ifdef DDEBUG dd("lmcf->lua: %p", lmcf->lua); +#endif + return lmcf->lua; } From 2f7c65009d95ca4a82e53688506c96a03f06bf27 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 12 Nov 2018 18:49:18 -0800 Subject: [PATCH 258/848] feature: implemented a new 'lua_sa_restart' directive, which sets the SA_RESTART flag on nginx workers signal dispositions. --- README.markdown | 18 ++++ config | 11 ++ doc/HttpLuaModule.wiki | 14 +++ src/ngx_http_lua_common.h | 2 + src/ngx_http_lua_initworkerby.c | 6 ++ src/ngx_http_lua_module.c | 15 +++ src/ngx_http_lua_util.c | 47 +++++++++ src/ngx_http_lua_util.h | 3 + t/159-sa-restart.t | 180 ++++++++++++++++++++++++++++++++ 9 files changed, 296 insertions(+) create mode 100644 t/159-sa-restart.t diff --git a/README.markdown b/README.markdown index d756351a1b..2e79622d1b 100644 --- a/README.markdown +++ b/README.markdown @@ -1065,6 +1065,7 @@ Directives * [lua_check_client_abort](#lua_check_client_abort) * [lua_max_pending_timers](#lua_max_pending_timers) * [lua_max_running_timers](#lua_max_running_timers) +* [lua_sa_restart](#lua_sa_restart) The basic building blocks of scripting Nginx with Lua are directives. Directives are used to specify when the user Lua code is run and @@ -3069,6 +3070,23 @@ This directive was first introduced in the `v0.8.0` release. [Back to TOC](#directives) +lua_sa_restart +-------------- + +**syntax:** *lua_sa_restart on|off* + +**default:** *lua_sa_restart on* + +**context:** *http* + +When enabled, this module will set the `SA_RESTART` flag on nginx workers signal dispositions. + +This allows Lua I/O primitives to not be interrupted by nginx's handling of various signals. + +This directive was first introduced in the `v0.10.14` release. + +[Back to TOC](#directives) + Nginx API for Lua ================= diff --git a/config b/config index 2005535466..cf887abf63 100644 --- a/config +++ b/config @@ -476,6 +476,17 @@ ngx_feature_test='setsockopt(1, SOL_SOCKET, SO_PASSCRED, NULL, 0);' . auto/feature +ngx_feature="SA_RESTART" +ngx_feature_libs= +ngx_feature_name="NGX_HTTP_LUA_HAVE_SA_RESTART" +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_path= +ngx_feature_test='struct sigaction act; + act.sa_flags |= SA_RESTART;' + +. auto/feature + ngx_feature="__attribute__(constructor)" ngx_feature_libs= ngx_feature_name="NGX_HTTP_LUA_HAVE_CONSTRUCTOR" diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index f964274217..50603e00a8 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2595,6 +2595,20 @@ When exceeding this limit, Nginx will stop running the callbacks of newly expire This directive was first introduced in the v0.8.0 release. +== lua_sa_restart == + +'''syntax:''' ''lua_sa_restart on|off'' + +'''default:''' ''lua_sa_restart on'' + +'''context:''' ''http'' + +When enabled, this module will set the `SA_RESTART` flag on nginx workers signal dispositions. + +This allows Lua I/O primitives to not be interrupted by nginx's handling of various signals. + +This directive was first introduced in the v0.10.14 release. + = Nginx API for Lua = diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 5eb57c69ea..cf0ab142c7 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -255,6 +255,8 @@ struct ngx_http_lua_main_conf_s { ngx_int_t host_var_index; + ngx_flag_t set_sa_restart; + unsigned requires_header_filter:1; unsigned requires_body_filter:1; unsigned requires_capture_filter:1; diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 2a82fcb9f8..e15390ed59 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -67,6 +67,12 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) } #endif /* NGX_WIN32 */ +#if (NGX_HTTP_LUA_HAVE_SA_RESTART) + if (lmcf->set_sa_restart) { + ngx_http_lua_set_sa_restart(ngx_cycle->log); + } +#endif + if (lmcf->init_worker_handler == NULL) { return NGX_OK; } diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index c954fb6676..bd54547e54 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -104,6 +104,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, NULL }, + { ngx_string("lua_sa_restart"), + NGX_HTTP_MAIN_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_MAIN_CONF_OFFSET, + offsetof(ngx_http_lua_main_conf_t, set_sa_restart), + NULL }, + #if (NGX_PCRE) { ngx_string("lua_regex_cache_max_entries"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, @@ -869,6 +876,8 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) lmcf->postponed_to_rewrite_phase_end = NGX_CONF_UNSET; lmcf->postponed_to_access_phase_end = NGX_CONF_UNSET; + lmcf->set_sa_restart = NGX_CONF_UNSET; + #if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) lmcf->malloc_trim_cycle = NGX_CONF_UNSET_UINT; #endif @@ -909,6 +918,12 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) lmcf->max_running_timers = 256; } +#if (NGX_HTTP_LUA_HAVE_SA_RESTART) + if (lmcf->set_sa_restart == NGX_CONF_UNSET) { + lmcf->set_sa_restart = 1; + } +#endif + #if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) if (lmcf->malloc_trim_cycle == NGX_CONF_UNSET_UINT) { lmcf->malloc_trim_cycle = 1000; /* number of reqs */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 6b4b783726..1b815e0503 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -70,6 +70,25 @@ #endif +#if (NGX_HTTP_LUA_HAVE_SA_RESTART) +#define NGX_HTTP_LUA_SA_RESTART_SIGS { \ + ngx_signal_value(NGX_RECONFIGURE_SIGNAL), \ + ngx_signal_value(NGX_REOPEN_SIGNAL), \ + ngx_signal_value(NGX_NOACCEPT_SIGNAL), \ + ngx_signal_value(NGX_TERMINATE_SIGNAL), \ + ngx_signal_value(NGX_SHUTDOWN_SIGNAL), \ + ngx_signal_value(NGX_CHANGEBIN_SIGNAL), \ + SIGALRM, \ + SIGINT, \ + SIGIO, \ + SIGCHLD, \ + SIGSYS, \ + SIGPIPE, \ + 0 \ +}; +#endif + + char ngx_http_lua_code_cache_key; char ngx_http_lua_regex_cache_key; char ngx_http_lua_socket_pool_key; @@ -4195,4 +4214,32 @@ ngx_http_lua_cleanup_free(ngx_http_request_t *r, ngx_http_cleanup_pt *cleanup) } +#if (NGX_HTTP_LUA_HAVE_SA_RESTART) +void +ngx_http_lua_set_sa_restart(ngx_log_t *log) +{ + int *signo; + int sigs[] = NGX_HTTP_LUA_SA_RESTART_SIGS; + struct sigaction act; + + for (signo = sigs; *signo != 0; signo++) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, + "setting SA_RESTART for signal %d", *signo); + + if (sigaction(*signo, NULL, &act) != 0) { + ngx_log_error(NGX_LOG_WARN, log, ngx_errno, "failed to get " + "sigaction for signal %d", *signo); + } + + act.sa_flags |= SA_RESTART; + + if (sigaction(*signo, &act, NULL) != 0) { + ngx_log_error(NGX_LOG_WARN, log, ngx_errno, "failed to set " + "sigaction for signal %d", *signo); + } + } +} +#endif + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index eb50f3a743..e10bec5948 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -250,6 +250,9 @@ ngx_http_cleanup_t *ngx_http_lua_cleanup_add(ngx_http_request_t *r, void ngx_http_lua_cleanup_free(ngx_http_request_t *r, ngx_http_cleanup_pt *cleanup); +#if (NGX_HTTP_LUA_HAVE_SA_RESTART) +void ngx_http_lua_set_sa_restart(ngx_log_t *log); +#endif #define ngx_http_lua_check_if_abortable(L, ctx) \ if ((ctx)->no_abort) { \ diff --git a/t/159-sa-restart.t b/t/159-sa-restart.t new file mode 100644 index 0000000000..0fc63145d5 --- /dev/null +++ b/t/159-sa-restart.t @@ -0,0 +1,180 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +add_block_preprocessor(sub { + my $block = shift; + + my $http_config = $block->http_config || ''; + + $http_config .= <<_EOC_; + init_by_lua_block { + function test_sa_restart() + local signals = { + "HUP", + --"INFO", + --"XCPU", + --"USR1", + --"USR2", + "ALRM", + --"INT", + "IO", + "CHLD", + "WINCH", + } + + for _, signame in ipairs(signals) do + local cmd = string.format("kill -s %s %d && sleep 0.01", + signame, ngx.worker.pid()) + local err = select(2, io.popen(cmd):read("*a")) + if err then + error("SIG" .. signame .. " caused: " .. err) + end + end + end + } +_EOC_ + + $block->set_value("http_config", $http_config); + + if (!defined $block->config) { + my $config = <<_EOC_; + location /t { + echo ok; + } +_EOC_ + + $block->set_value("config", $config); + } + + if (!defined $block->request) { + $block->set_value("request", "GET /t"); + } + + if (!defined $block->response_body) { + $block->set_value("ignore_response_body"); + } + + if (!defined $block->no_error_log) { + $block->set_value("no_error_log", "[error]"); + } +}); + +plan tests => repeat_each() * (blocks() * 2 + 1); + +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: lua_sa_restart default - sets SA_RESTART in init_worker_by_lua* +--- http_config + init_worker_by_lua_block { + test_sa_restart() + } + + + +=== TEST 2: lua_sa_restart off - does not set SA_RESTART +--- http_config + lua_sa_restart off; + + init_worker_by_lua_block { + test_sa_restart() + } +--- no_error_log +[crit] +--- error_log +Interrupted system call + + + +=== TEST 3: lua_sa_restart on (default) - sets SA_RESTART if no init_worker_by_lua* phase is defined +--- config + location /t { + content_by_lua_block { + test_sa_restart() + } + } + + + +=== TEST 4: lua_sa_restart on (default) - SA_RESTART is effective in rewrite_by_lua* +--- config + location /t { + rewrite_by_lua_block { + test_sa_restart() + } + + echo ok; + } + + + +=== TEST 5: lua_sa_restart on (default) - SA_RESTART is effective in access_by_lua* +--- config + location /t { + access_by_lua_block { + test_sa_restart() + } + + echo ok; + } + + + +=== TEST 6: lua_sa_restart on (default) - SA_RESTART is effective in content_by_lua* +--- config + location /t { + content_by_lua_block { + test_sa_restart() + } + } + + + +=== TEST 7: lua_sa_restart on (default) - SA_RESTART is effective in header_filter_by_lua* +--- config + location /t { + echo ok; + + header_filter_by_lua_block { + test_sa_restart() + } + } + + + +=== TEST 8: lua_sa_restart on (default) - SA_RESTART is effective in body_filter_by_lua* +--- config + location /t { + echo ok; + + body_filter_by_lua_block { + test_sa_restart() + } + } + + + +=== TEST 9: lua_sa_restart on (default) - SA_RESTART is effective in log_by_lua* +--- config + location /t { + echo ok; + + log_by_lua_block { + test_sa_restart() + } + } + + + +=== TEST 10: lua_sa_restart on (default) - SA_RESTART is effective in timer phase +--- config + location /t { + echo ok; + + log_by_lua_block { + ngx.timer.at(0, test_sa_restart) + } + } From 1c72f57ce87d4355d546a97c2bd8f5123a70db5c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 13 Nov 2018 16:29:41 -0800 Subject: [PATCH 259/848] feature: added an FFI API for retrieving env 'directives' values. --- src/ngx_http_lua_misc.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/src/ngx_http_lua_misc.c b/src/ngx_http_lua_misc.c index f96e2f273e..145ca9b5e3 100644 --- a/src/ngx_http_lua_misc.c +++ b/src/ngx_http_lua_misc.c @@ -286,6 +286,33 @@ ngx_http_lua_ffi_headers_sent(ngx_http_request_t *r) return r->header_sent ? 1 : 0; } + + +int +ngx_http_lua_ffi_get_conf_env(u_char *name, u_char **env_buf, size_t *name_len) +{ + ngx_uint_t i; + ngx_str_t *var; + ngx_core_conf_t *ccf; + + ccf = (ngx_core_conf_t *) ngx_get_conf(ngx_cycle->conf_ctx, + ngx_core_module); + + var = ccf->env.elts; + + for (i = 0; i < ccf->env.nelts; i++) { + if (var[i].data[var[i].len] == '=' + && ngx_strncmp(name, var[i].data, var[i].len) == 0) + { + *env_buf = var[i].data; + *name_len = var[i].len; + + return NGX_OK; + } + } + + return NGX_DECLINED; +} #endif /* NGX_LUA_NO_FFI_API */ From b4ade78874437bc4bf0bb569c7e13ee4add05a46 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sun, 25 Nov 2018 07:48:22 +0800 Subject: [PATCH 260/848] bugfix: we now only apply the lightuserdata mask on platforms that are at least 64bits. Thanks Alec Muffett for the report. --- src/ngx_http_lua_common.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index cf0ab142c7..1585d0cddd 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -140,9 +140,14 @@ typedef struct { #endif +#if (NGX_PTR_SIZE >= 8) #define ngx_http_lua_lightudata_mask(ludata) \ ((void *) ((uintptr_t) (&ngx_http_lua_##ludata) & ((1UL << 47) - 1))) +#else +#define ngx_http_lua_lightudata_mask(ludata) (&ngx_http_lua_##ludata) +#endif + typedef struct ngx_http_lua_main_conf_s ngx_http_lua_main_conf_t; typedef union ngx_http_lua_srv_conf_u ngx_http_lua_srv_conf_t; From 60225ad19f00d0259fe82981212ef87ee02d72f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Thu, 29 Nov 2018 01:39:41 +0800 Subject: [PATCH 261/848] bugfix: the lightuserdata mask was broken in WIN64. In WIN64, the pointer type is 64 bits but the long type is 32 bits. Since WIN64 only supports x64 and IA64, we could confirm that it doesn't need the mask. If we need the mask in the future, we might need to use a C compiler which supports C99 (like MSVC 2013+ or any other modern C compiler) to build the WIN64 version. --- src/ngx_http_lua_common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 1585d0cddd..89473469b7 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -140,7 +140,7 @@ typedef struct { #endif -#if (NGX_PTR_SIZE >= 8) +#if (NGX_PTR_SIZE >= 8 && !defined(_WIN64)) #define ngx_http_lua_lightudata_mask(ludata) \ ((void *) ((uintptr_t) (&ngx_http_lua_##ludata) & ((1UL << 47) - 1))) From ec0d5768b25044cda99e35ce3413f82c21d4aac1 Mon Sep 17 00:00:00 2001 From: Alex Zhang Date: Thu, 29 Nov 2018 03:02:48 +0800 Subject: [PATCH 262/848] doc: noted that ngx.req.get_method can be used in the body_filter_by_lua* and log_by_lua* phases. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 2e79622d1b..ef2b4af5e8 100644 --- a/README.markdown +++ b/README.markdown @@ -4282,7 +4282,7 @@ ngx.req.get_method ------------------ **syntax:** *method_name = ngx.req.get_method()* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, balancer_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, balancer_by_lua*, log_by_lua** Retrieves the current request's request method name. Strings like `"GET"` and `"POST"` are returned instead of numerical [method constants](#http-method-constants). diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 50603e00a8..7d5b42fd0b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3546,7 +3546,7 @@ This method does not work in HTTP/2 requests yet. == ngx.req.get_method == '''syntax:''' ''method_name = ngx.req.get_method()'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, balancer_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, balancer_by_lua*, log_by_lua*'' Retrieves the current request's request method name. Strings like "GET" and "POST" are returned instead of numerical [[#HTTP method constants|method constants]]. From 11a31b5409079b0f4e3f81384f105e20722bc4dd Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 3 Dec 2018 14:56:07 -0800 Subject: [PATCH 263/848] tests: increased the timeouts of some ssl cosocket tests. --- t/129-ssl-socket.t | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 3f056404d6..d5068ce3df 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -301,7 +301,7 @@ SSL reused session content_by_lua ' local sock = ngx.socket.tcp() - sock:settimeout(2000) + sock:settimeout(7000) do @@ -380,7 +380,7 @@ lua ssl free session --- no_error_log [error] [alert] ---- timeout: 5 +--- timeout: 10 @@ -2508,7 +2508,7 @@ SSL reused session content_by_lua_block { local sock = ngx.socket.tcp() - sock:settimeout(2000) + sock:settimeout(7000) local ok, err = sock:connect("openresty.org", 443) if not ok then @@ -2529,4 +2529,4 @@ GET /t qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including the object\), but seen 0/ --- no_error_log [alert] ---- timeout: 5 +--- timeout: 10 From f63af5dc0e1d6f152673f73c778c4cb19a032a57 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 3 Dec 2018 14:57:41 -0800 Subject: [PATCH 264/848] travis-ci: upgraded openssl to 1.0.2q and 1.1.0j. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 8c8e3a757f..4d0823d67c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,8 +48,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2n OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0h OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2q OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0j OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" # to $OPENSSL_OPT. From 5d924cc637e4fe54336741511903c1f635b4e3ba Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 4 Dec 2018 14:35:51 -0800 Subject: [PATCH 265/848] tests: require: hardened lua_path and lua_cpath regexes. Those regexes used to not play nice with existing LUA_PATH and LUA_CPATH environment variables, especially empty path segments ;; (which can happen anywhere in those paths) or in cases where those variables are empty (in which case ;; is left unchanged from the specified lua_package_*path). We also made the regexes slightly more robust in other subtle ways. --- t/004-require.t | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/t/004-require.t b/t/004-require.t index 35e04db814..6aa9b952fc 100644 --- a/t/004-require.t +++ b/t/004-require.t @@ -85,7 +85,7 @@ hello, foo --- request GET /main --- user_files ---- response_body_like: ^[^;]+/servroot/html/\?.so$ +--- response_body_like: ^[^;]+/servroot/html/\?\.so$ @@ -100,7 +100,7 @@ GET /main } --- request GET /main ---- response_body_like: ^[^;]+/servroot/html/\?.lua;.+\.lua;$ +--- response_body_like: ^[^;]+/servroot/html/\?\.lua;(.+\.lua)?;*$ @@ -115,7 +115,7 @@ GET /main } --- request GET /main ---- response_body_like: ^[^;]+/servroot/html/\?.so;.+\.so;$ +--- response_body_like: ^[^;]+/servroot/html/\?\.so;(.+\.so)?;*$ @@ -130,7 +130,7 @@ GET /main } --- request GET /main ---- response_body_like: ^.+\.lua;[^;]+/servroot/html/\?.lua$ +--- response_body_like: ^(.+\.lua)?;*?[^;]+/servroot/html/\?\.lua$ @@ -145,7 +145,7 @@ GET /main } --- request GET /main ---- response_body_like: ^.+\.so;[^;]+/servroot/html/\?.so$ +--- response_body_like: ^(.+\.so)?;*?[^;]+/servroot/html/\?\.so$ From ab2e2718bf5126bfcac87a6b086539e311967e6e Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 4 Dec 2018 15:10:43 -0800 Subject: [PATCH 266/848] tests: updated the test suite so that it can be run in parallel mode. --- .gitignore | 1 + t/004-require.t | 10 +++++----- t/025-codecache.t | 22 +++++++++++----------- t/081-bytecode.t | 16 ++++++++-------- t/109-timer-hup.t | 14 +++++++------- t/134-worker-count-5.t | 2 +- t/138-balancer.t | 2 -- t/157-socket-keepalive-hup.t | 2 +- 8 files changed, 34 insertions(+), 35 deletions(-) diff --git a/.gitignore b/.gitignore index eff337246b..e20be8dff2 100644 --- a/.gitignore +++ b/.gitignore @@ -26,6 +26,7 @@ test/blib test.sh t.sh t/t.sh +t/servroot* test/t/servroot/ releng reset diff --git a/t/004-require.t b/t/004-require.t index 6aa9b952fc..9fd28606cb 100644 --- a/t/004-require.t +++ b/t/004-require.t @@ -85,7 +85,7 @@ hello, foo --- request GET /main --- user_files ---- response_body_like: ^[^;]+/servroot/html/\?\.so$ +--- response_body_like: ^[^;]+/servroot(_\d+)?/html/\?\.so$ @@ -100,7 +100,7 @@ GET /main } --- request GET /main ---- response_body_like: ^[^;]+/servroot/html/\?\.lua;(.+\.lua)?;*$ +--- response_body_like: ^[^;]+/servroot(_\d+)?/html/\?\.lua;(.+\.lua)?;*$ @@ -115,7 +115,7 @@ GET /main } --- request GET /main ---- response_body_like: ^[^;]+/servroot/html/\?\.so;(.+\.so)?;*$ +--- response_body_like: ^[^;]+/servroot(_\d+)?/html/\?\.so;(.+\.so)?;*$ @@ -130,7 +130,7 @@ GET /main } --- request GET /main ---- response_body_like: ^(.+\.lua)?;*?[^;]+/servroot/html/\?\.lua$ +--- response_body_like: ^(.+\.lua)?;*?[^;]+/servroot(_\d+)?/html/\?\.lua$ @@ -145,7 +145,7 @@ GET /main } --- request GET /main ---- response_body_like: ^(.+\.so)?;*?[^;]+/servroot/html/\?\.so$ +--- response_body_like: ^(.+\.so)?;*?[^;]+/servroot(_\d+)?/html/\?\.so$ diff --git a/t/025-codecache.t b/t/025-codecache.t index 20791d7c70..0f6a08fc82 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -27,7 +27,7 @@ __DATA__ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("ngx.say(101)") f:close() ngx.say("updated") @@ -61,7 +61,7 @@ updated location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("ngx.say(101)") f:close() ngx.say("updated") @@ -95,7 +95,7 @@ updated location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("ngx.say(101)") f:close() ngx.say("updated") @@ -130,7 +130,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("ngx.say(101)") f:close() ngx.say("updated") @@ -166,7 +166,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("ngx.say(101)") f:close() ngx.say("updated") @@ -204,7 +204,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/foo.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/foo.lua", "w")) f:write("module(..., package.seeall); ngx.say(102);") f:close() ngx.say("updated") @@ -240,7 +240,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/foo.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/foo.lua", "w")) f:write("module(..., package.seeall); ngx.say(102);") f:close() ngx.say("updated") @@ -279,7 +279,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/foo.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/foo.lua", "w")) f:write("module(..., package.seeall); return 102;") f:close() ngx.say("updated") @@ -318,7 +318,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/foo.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/foo.lua", "w")) f:write("module(..., package.seeall); return 102;") f:close() ngx.say("updated") @@ -355,7 +355,7 @@ updated location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("return 101") f:close() ngx.say("updated") @@ -390,7 +390,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ location /update { content_by_lua ' -- os.execute("(echo HERE; pwd) > /dev/stderr") - local f = assert(io.open("t/servroot/html/test.lua", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/test.lua", "w")) f:write("return 101") f:close() ngx.say("updated") diff --git a/t/081-bytecode.t b/t/081-bytecode.t index 4b05438823..92383bcb92 100644 --- a/t/081-bytecode.t +++ b/t/081-bytecode.t @@ -259,9 +259,9 @@ error content_by_lua ' local bcsave = require "jit.bcsave" if jit then - local prefix = ngx.config.prefix() - local infile = prefix .. "html/a.lua" - local outfile = prefix .. "html/a.luac" + local prefix = "$TEST_NGINX_SERVER_ROOT" + local infile = prefix .. "/html/a.lua" + local outfile = prefix .. "/html/a.luac" bcsave.start("-s", infile, outfile) return ngx.exec("/call") end @@ -292,9 +292,9 @@ ngx.status = 201 ngx.say("hello from Lua!") content_by_lua ' local bcsave = require "jit.bcsave" if jit then - local prefix = ngx.config.prefix() - local infile = prefix .. "html/a.lua" - local outfile = prefix .. "html/a.luac" + local prefix = "$TEST_NGINX_SERVER_ROOT" + local infile = prefix .. "/html/a.lua" + local outfile = prefix .. "/html/a.luac" bcsave.start("-g", infile, outfile) return ngx.exec("/call") end @@ -325,7 +325,7 @@ ngx.status = 201 ngx.say("hello from Lua!") content_by_lua_block { local f = assert(loadstring("local a = 1 ngx.say('a = ', a)", "=code")) local bc = string.dump(f) - local f = assert(io.open("t/servroot/html/a.luac", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/a.luac", "w")) f:write(bc) f:close() } @@ -354,7 +354,7 @@ a = 1 content_by_lua_block { local f = assert(loadstring("local a = 1 ngx.say('a = ', a)", "=code")) local bc = string.dump(f, true) - local f = assert(io.open("t/servroot/html/a.luac", "w")) + local f = assert(io.open("$TEST_NGINX_SERVER_ROOT/html/a.luac", "w")) f:write(bc) f:close() } diff --git a/t/109-timer-hup.t b/t/109-timer-hup.t index f0ef1cf938..2c197c2826 100644 --- a/t/109-timer-hup.t +++ b/t/109-timer-hup.t @@ -46,7 +46,7 @@ __DATA__ --- config location /t { content_by_lua ' - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return @@ -99,7 +99,7 @@ timer prematurely expired: true --- config location /t { content_by_lua ' - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return @@ -163,7 +163,7 @@ timer prematurely expired: true --- config location /t { content_by_lua ' - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return @@ -219,7 +219,7 @@ failed to register a new timer after reload: process exiting, context: ngx.timer --- config location /t { content_by_lua ' - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return @@ -284,7 +284,7 @@ g: exiting=true --- config location /t { content_by_lua ' - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return @@ -399,7 +399,7 @@ lua found 100 pending timers end local ok, err = ngx.timer.at(1, background_thread) - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return @@ -453,7 +453,7 @@ lua found 1 pending timers end if kill then - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.log(ngx.ERR, "failed to open nginx.pid: ", err) return diff --git a/t/134-worker-count-5.t b/t/134-worker-count-5.t index fbacad3fb7..ec6d0e7e35 100644 --- a/t/134-worker-count-5.t +++ b/t/134-worker-count-5.t @@ -55,7 +55,7 @@ workers: 5 === TEST 3: init_by_lua + module (github #681) --- http_config - lua_package_path "t/servroot/html/?.lua;;"; + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; init_by_lua_block { local blah = require "file" diff --git a/t/138-balancer.t b/t/138-balancer.t index 0aba66d62b..151db7be44 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -362,8 +362,6 @@ me: 101 === TEST 13: lua subrequests --- http_config - lua_package_path "t/servroot/html/?.lua;;"; - lua_code_cache off; upstream backend { diff --git a/t/157-socket-keepalive-hup.t b/t/157-socket-keepalive-hup.t index 5f59ec1c45..357bb59682 100644 --- a/t/157-socket-keepalive-hup.t +++ b/t/157-socket-keepalive-hup.t @@ -32,7 +32,7 @@ __DATA__ set $port $TEST_NGINX_SERVER_PORT; content_by_lua_block { - local f, err = io.open("t/servroot/logs/nginx.pid", "r") + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") if not f then ngx.say("failed to open nginx.pid: ", err) return From 017e00462ae1bea96f508365726f7cec3cbcc5ca Mon Sep 17 00:00:00 2001 From: spacewander Date: Tue, 13 Nov 2018 19:16:44 +0800 Subject: [PATCH 267/848] bugfix: we now avoid generating a Content-Type header when getting/setting other response headers. Signed-off-by: Thibault Charbonnier --- README.markdown | 37 ++++++++++++++ doc/HttpLuaModule.wiki | 35 +++++++++++++ src/ngx_http_lua_headers.c | 93 ++++++++++++---------------------- src/ngx_http_lua_headers_out.c | 26 +++++++--- src/ngx_http_lua_headers_out.h | 2 +- src/ngx_http_lua_subrequest.c | 8 +++ t/016-resp-header.t | 66 +++++++++++++++++++++++- 7 files changed, 197 insertions(+), 70 deletions(-) diff --git a/README.markdown b/README.markdown index ef2b4af5e8..7e4baa2b24 100644 --- a/README.markdown +++ b/README.markdown @@ -4147,6 +4147,25 @@ to be returned when reading `ngx.header.Foo`. Note that `ngx.header` is not a normal Lua table and as such, it is not possible to iterate through it using the Lua `ipairs` function. +Note: if the current response does not have a Content-Type header, but you use +`ngx.header` to attempt retrieving it, a side effect will be that a +Content-Type header will be created and returned according to your nginx +configuration. + +For example: + +```nginx + + location = /t { + default_type text/html; + content_by_lua_block { + -- even if the response had no Content-Type header, + -- it will now have: Content-Type: text/html + ngx.say(ngx.header['Content-Type']) + } + } +``` + For reading *request* headers, use the [ngx.req.get_headers](#ngxreqget_headers) function instead. [Back to TOC](#nginx-api-for-lua) @@ -4172,6 +4191,24 @@ Returns a Lua table holding all the current response headers for the current req end ``` +Note: if the current response does not have a Content-Type header, but you use +`ngx.resp.get_headers` to attempt retrieving it, a side effect will +be that a Content-Type header will be created and returned according to your +nginx configuration. + +For example: + +```nginx + + location = /t { + default_type text/html; + content_by_lua_block { + -- the Content-Type header will be text/html + ngx.say(ngx.resp.get_headers()['Content-Type']) + } + } +``` + This function has the same signature as [ngx.req.get_headers](#ngxreqget_headers) except getting response headers instead of request headers. Note that a maximum of 100 response headers are parsed by default (including those with the same name) and that additional response headers are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 7d5b42fd0b..091973157b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3433,6 +3433,24 @@ to be returned when reading ngx.header.Foo. Note that ngx.header is not a normal Lua table and as such, it is not possible to iterate through it using the Lua ipairs function. +Note: if the current response does not have a Content-Type header, but you use +ngx.header to attempt retrieving it, a side effect will be that a +Content-Type header will be created and returned according to your nginx +configuration. + +For example: + + + location = /t { + default_type text/html; + content_by_lua_block { + -- even if the response had no Content-Type header, + -- it will now have: Content-Type: text/html + ngx.say(ngx.header['Content-Type']) + } + } + + For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_headers]] function instead. == ngx.resp.get_headers == @@ -3454,6 +3472,23 @@ for k, v in pairs(h) do end +Note: if the current response does not have a Content-Type header, but you use +ngx.resp.get_headers, a side effect will be that a Content-Type +header will be created and returned according to your nginx configuration. + +For example: + + + location = /t { + default_type text/html; + content_by_lua_block { + -- even if the response had no Content-Type header, + -- it will now have: Content-Type: text/html + ngx.say(ngx.resp.get_headers()['Content-Type']) + } + } + + This function has the same signature as [[#ngx.req.get_headers|ngx.req.get_headers]] except getting response headers instead of request headers. Note that a maximum of 100 response headers are parsed by default (including those with the same name) and that additional response headers are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 0ff905b15d..f861676415 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -544,17 +544,6 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) return luaL_error(L, "no ctx found"); } - if (!ctx->headers_set) { - rc = ngx_http_lua_set_content_type(r); - if (rc != NGX_OK) { - return luaL_error(L, - "failed to set default content type: %d", - (int) rc); - } - - ctx->headers_set = 1; - } - ngx_http_lua_check_fake_request(L, r); part = &r->headers_out.headers.part; @@ -573,6 +562,17 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) lua_setmetatable(L, -2); } + if (!r->headers_out.content_type.len) { + rc = ngx_http_lua_set_content_type(r); + if (rc != NGX_OK) { + return luaL_error(L, + "failed to set default content type: %d", + (int) rc); + } + + ctx->headers_set = 1; + } + #if 1 if (r->headers_out.content_type.len) { extra++; @@ -696,7 +696,6 @@ ngx_http_lua_ngx_header_get(lua_State *L) size_t len; ngx_http_lua_loc_conf_t *llcf; ngx_http_lua_ctx_t *ctx; - ngx_int_t rc; r = ngx_http_lua_get_req(L); if (r == NULL) { @@ -739,18 +738,7 @@ ngx_http_lua_ngx_header_get(lua_State *L) key.len = len; - if (!ctx->headers_set) { - rc = ngx_http_lua_set_content_type(r); - if (rc != NGX_OK) { - return luaL_error(L, - "failed to set default content type: %d", - (int) rc); - } - - ctx->headers_set = 1; - } - - return ngx_http_lua_get_output_header(L, r, &key); + return ngx_http_lua_get_output_header(L, r, ctx, &key); } @@ -813,16 +801,7 @@ ngx_http_lua_ngx_header_set(lua_State *L) } } - if (!ctx->headers_set) { - rc = ngx_http_lua_set_content_type(r); - if (rc != NGX_OK) { - return luaL_error(L, - "failed to set default content type: %d", - (int) rc); - } - - ctx->headers_set = 1; - } + ctx->headers_set = 1; if (lua_type(L, 3) == LUA_TNIL) { ngx_str_null(&value); @@ -1244,15 +1223,7 @@ ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, } } - if (!ctx->headers_set) { - rc = ngx_http_lua_set_content_type(r); - if (rc != NGX_OK) { - *errmsg = "failed to set default content type"; - return NGX_ERROR; - } - - ctx->headers_set = 1; - } + ctx->headers_set = 1; if (is_nil) { value.data = NULL; @@ -1373,7 +1344,8 @@ ngx_http_lua_ffi_req_header_set_single_value(ngx_http_request_t *r, int ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, const u_char *key, size_t key_len, - u_char *key_buf, ngx_http_lua_ffi_str_t *values, int max_nvalues) + u_char *key_buf, ngx_http_lua_ffi_str_t *values, int max_nvalues, + char **errmsg) { int found; u_char c, *p; @@ -1390,19 +1362,10 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { - /* *errmsg = "no ctx found"; */ + *errmsg = "no ctx found"; return NGX_ERROR; } - if (!ctx->headers_set) { - if (ngx_http_lua_set_content_type(r) != NGX_OK) { - /* *errmsg = "failed to set default content type"; */ - return NGX_ERROR; - } - - ctx->headers_set = 1; - } - llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (llcf->transform_underscores_in_resp_headers && memchr(key, '_', key_len) != NULL) @@ -1428,6 +1391,7 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, { p = ngx_palloc(r->pool, NGX_OFF_T_LEN); if (p == NULL) { + *errmsg = "no memory"; return NGX_ERROR; } @@ -1441,12 +1405,21 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, break; case 12: - if (r->headers_out.content_type.len - && ngx_strncasecmp(key_buf, (u_char *) "Content-Type", 12) == 0) - { - values[0].data = r->headers_out.content_type.data; - values[0].len = r->headers_out.content_type.len; - return 1; + if (ngx_strncasecmp(key_buf, (u_char *) "Content-Type", 12) == 0) { + if (!r->headers_out.content_type.len) { + if (ngx_http_lua_set_content_type(r) != NGX_OK) { + *errmsg = "failed to set default content type"; + return NGX_ERROR; + } + + ctx->headers_set = 1; + } + + if (r->headers_out.content_type.len) { + values[0].data = r->headers_out.content_type.data; + values[0].len = r->headers_out.content_type.len; + return 1; + } } break; diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index b908eae05a..494a2da728 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -528,10 +528,11 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_str_t key, int ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, - ngx_str_t *key) + ngx_http_lua_ctx_t *ctx, ngx_str_t *key) { ngx_table_elt_t *h; ngx_list_part_t *part; + ngx_int_t rc; ngx_uint_t i; unsigned found; @@ -550,12 +551,23 @@ ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, break; case 12: - if (r->headers_out.content_type.len - && ngx_strncasecmp(key->data, (u_char *) "Content-Type", 12) == 0) - { - lua_pushlstring(L, (char *) r->headers_out.content_type.data, - r->headers_out.content_type.len); - return 1; + if (ngx_strncasecmp(key->data, (u_char *) "Content-Type", 12) == 0) { + if (!r->headers_out.content_type.len) { + rc = ngx_http_lua_set_content_type(r); + if (rc != NGX_OK) { + return luaL_error(L, + "failed to set default content type: %d", + (int) rc); + } + + ctx->headers_set = 1; + } + + if (r->headers_out.content_type.len) { + lua_pushlstring(L, (char *) r->headers_out.content_type.data, + r->headers_out.content_type.len); + return 1; + } } break; diff --git a/src/ngx_http_lua_headers_out.h b/src/ngx_http_lua_headers_out.h index ef5e6d421c..456dca1bd0 100644 --- a/src/ngx_http_lua_headers_out.h +++ b/src/ngx_http_lua_headers_out.h @@ -15,7 +15,7 @@ ngx_int_t ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_str_t key, ngx_str_t value, unsigned override); int ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, - ngx_str_t *key); + ngx_http_lua_ctx_t *ctx, ngx_str_t *key); #endif /* _NGX_HTTP_LUA_HEADERS_OUT_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 826a43c3a2..63969c04d8 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1031,6 +1031,14 @@ ngx_http_lua_post_subrequest(ngx_http_request_t *r, void *data, ngx_int_t rc) dd("pr_coctx status: %d", (int) pr_coctx->sr_statuses[ctx->index]); /* copy subrequest response headers */ + if (ctx->headers_set) { + rc = ngx_http_lua_set_content_type(r); + if (rc != NGX_OK) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "failed to set default content type: %i", rc); + return NGX_ERROR; + } + } pr_coctx->sr_headers[ctx->index] = &r->headers_out; diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 72c1e05bcc..44bfa30276 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 46); +plan tests => repeat_each() * (blocks() * 3 + 51); #no_diff(); no_long_string(); @@ -781,7 +781,10 @@ GET /read --- more_headers Accept-Encoding: gzip --- response_headers -Content-Type: text/plain +Content-Encoding: gzip +--- no_error_log +[error] +http gzip filter --- response_body Hello, world, my dear friend! @@ -1699,3 +1702,62 @@ found 3 resp headers --- no_error_log [error] lua exceeding response header limit + + + +=== TEST 76: don't generate Content-Type header when setting other response headers +--- config + location = /t { + default_type text/html; + rewrite_by_lua_block { + ngx.header.blah = "foo" + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; + } + + location = /backend { + content_by_lua_block { + ngx.say("foo") + } + header_filter_by_lua_block { + ngx.header.content_type = nil + } + } +--- request +GET /t +--- response_body +foo +--- response_headers +blah: foo +Content-Type: +--- no_error_log +[error] + + + +=== TEST 77: don't generate Content-Type header when getting other response headers +--- config + location = /t { + default_type text/html; + rewrite_by_lua_block { + local h = ngx.header.content_length + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; + } + + location = /backend { + content_by_lua_block { + ngx.say("foo") + } + header_filter_by_lua_block { + ngx.header.content_type = nil + } + } +--- request +GET /t +--- response_body +foo +--- response_headers +Content-Type: +--- no_error_log +[error] From 637a4c645e931907d4537c4f323ba514613dccf4 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 3 Jan 2019 17:54:09 -0800 Subject: [PATCH 268/848] doc: updated the copyright year to 2019. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 7e4baa2b24..4f31cb5243 100644 --- a/README.markdown +++ b/README.markdown @@ -956,7 +956,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2018, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. +Copyright (C) 2009-2019, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 091973157b..82a7f6a64f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -786,7 +786,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2018, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. +Copyright (C) 2009-2019, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved. From cec2a09defd155c81d9a90a0efc57cfb8d9f7743 Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 4 Jan 2019 13:24:51 +0800 Subject: [PATCH 269/848] bugfix: ensured Content-Type header gets generated when other headers get set. Fixed a regression introduced in 017e004 and previously described in https://github.com/openresty/lua-nginx-module/issues/92. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_common.h | 2 + src/ngx_http_lua_headers.c | 12 ++--- src/ngx_http_lua_headers_out.c | 12 +++-- src/ngx_http_lua_headers_out.h | 4 +- src/ngx_http_lua_subrequest.c | 2 +- src/ngx_http_lua_util.c | 4 +- src/ngx_http_lua_util.h | 4 +- t/016-resp-header.t | 85 +++++++++++++++++++++++++++++----- 8 files changed, 98 insertions(+), 27 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 89473469b7..0428e2d3fa 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -566,6 +566,8 @@ typedef struct ngx_http_lua_ctx_s { unsigned headers_set:1; /* whether the user has set custom response headers */ + unsigned mime_set:1; /* whether the user has set Content-Type + response header */ unsigned entered_rewrite_phase:1; unsigned entered_access_phase:1; diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index f861676415..fb743dd159 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -563,7 +563,7 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) } if (!r->headers_out.content_type.len) { - rc = ngx_http_lua_set_content_type(r); + rc = ngx_http_lua_set_content_type(r, ctx); if (rc != NGX_OK) { return luaL_error(L, "failed to set default content type: %d", @@ -826,7 +826,7 @@ ngx_http_lua_ngx_header_set(lua_State *L) ngx_memcpy(value.data, p, len); value.len = len; - rc = ngx_http_lua_set_output_header(r, key, value, + rc = ngx_http_lua_set_output_header(r, ctx, key, value, i == 1 /* override */); if (rc == NGX_ERROR) { @@ -853,7 +853,7 @@ ngx_http_lua_ngx_header_set(lua_State *L) dd("key: %.*s, value: %.*s", (int) key.len, key.data, (int) value.len, value.data); - rc = ngx_http_lua_set_output_header(r, key, value, 1 /* override */); + rc = ngx_http_lua_set_output_header(r, ctx, key, value, 1 /* override */); if (rc == NGX_ERROR) { return luaL_error(L, "failed to set header %s (error: %d)", @@ -1250,7 +1250,7 @@ ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, ngx_memcpy(value.data, p, len); value.len = len; - rc = ngx_http_lua_set_output_header(r, key, value, + rc = ngx_http_lua_set_output_header(r, ctx, key, value, override && i == 0); if (rc == NGX_ERROR) { @@ -1276,7 +1276,7 @@ ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, dd("key: %.*s, value: %.*s", (int) key.len, key.data, (int) value.len, value.data); - rc = ngx_http_lua_set_output_header(r, key, value, override); + rc = ngx_http_lua_set_output_header(r, ctx, key, value, override); if (rc == NGX_ERROR) { *errmsg = "failed to set header"; @@ -1407,7 +1407,7 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, case 12: if (ngx_strncasecmp(key_buf, (u_char *) "Content-Type", 12) == 0) { if (!r->headers_out.content_type.len) { - if (ngx_http_lua_set_content_type(r) != NGX_OK) { + if (ngx_http_lua_set_content_type(r, ctx) != NGX_OK) { *errmsg = "failed to set default content type"; return NGX_ERROR; } diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 494a2da728..ee542ee125 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -476,8 +476,8 @@ ngx_http_clear_builtin_header(ngx_http_request_t *r, ngx_int_t -ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_str_t key, - ngx_str_t value, unsigned override) +ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, + ngx_str_t key, ngx_str_t value, unsigned override) { ngx_http_lua_header_val_t hv; ngx_http_lua_set_header_t *handlers = ngx_http_lua_set_handlers; @@ -508,6 +508,10 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_str_t key, hv.offset = handlers[i].offset; hv.handler = handlers[i].handler; + if (hv.handler == ngx_http_set_content_type_header) { + ctx->mime_set = 1; + } + break; } @@ -552,8 +556,8 @@ ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, case 12: if (ngx_strncasecmp(key->data, (u_char *) "Content-Type", 12) == 0) { - if (!r->headers_out.content_type.len) { - rc = ngx_http_lua_set_content_type(r); + if (r->headers_out.content_type.len == 0) { + rc = ngx_http_lua_set_content_type(r, ctx); if (rc != NGX_OK) { return luaL_error(L, "failed to set default content type: %d", diff --git a/src/ngx_http_lua_headers_out.h b/src/ngx_http_lua_headers_out.h index 456dca1bd0..6ec1fe391d 100644 --- a/src/ngx_http_lua_headers_out.h +++ b/src/ngx_http_lua_headers_out.h @@ -12,8 +12,8 @@ #include "ngx_http_lua_common.h" -ngx_int_t ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_str_t key, - ngx_str_t value, unsigned override); +ngx_int_t ngx_http_lua_set_output_header(ngx_http_request_t *r, + ngx_http_lua_ctx_t *ctx, ngx_str_t key, ngx_str_t value, unsigned override); int ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_str_t *key); diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 63969c04d8..f312b65e0a 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1032,7 +1032,7 @@ ngx_http_lua_post_subrequest(ngx_http_request_t *r, void *data, ngx_int_t rc) /* copy subrequest response headers */ if (ctx->headers_set) { - rc = ngx_http_lua_set_content_type(r); + rc = ngx_http_lua_set_content_type(r, ctx); if (rc != NGX_OK) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "failed to set default content type: %i", rc); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 1b815e0503..72558624fb 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -433,7 +433,9 @@ ngx_http_lua_send_header_if_needed(ngx_http_request_t *r, r->headers_out.status = NGX_HTTP_OK; } - if (!ctx->headers_set && ngx_http_lua_set_content_type(r) != NGX_OK) { + if (!ctx->mime_set + && ngx_http_lua_set_content_type(r, ctx) != NGX_OK) + { return NGX_ERROR; } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index e10bec5948..f08f4815db 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -411,10 +411,12 @@ ngx_http_lua_hash_str(u_char *src, size_t n) static ngx_inline ngx_int_t -ngx_http_lua_set_content_type(ngx_http_request_t *r) +ngx_http_lua_set_content_type(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) { ngx_http_lua_loc_conf_t *llcf; + ctx->mime_set = 1; + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (llcf->use_default_type && r->headers_out.status != NGX_HTTP_NOT_MODIFIED) diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 44bfa30276..6a41672b46 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 51); +plan tests => repeat_each() * (blocks() * 3 + 55); #no_diff(); no_long_string(); @@ -1705,8 +1705,17 @@ lua exceeding response header limit -=== TEST 76: don't generate Content-Type header when setting other response headers +=== TEST 76: don't generate Content-Type when setting other response header --- config + location = /backend { + content_by_lua_block { + ngx.say("foo") + } + header_filter_by_lua_block { + ngx.header.content_type = nil + } + } + location = /t { default_type text/html; rewrite_by_lua_block { @@ -1714,7 +1723,20 @@ lua exceeding response header limit } proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; } +--- request +GET /t +--- response_body +foo +--- response_headers +blah: foo +!Content-Type +--- no_error_log +[error] + + +=== TEST 77: don't generate Content-Type when getting other response header +--- config location = /backend { content_by_lua_block { ngx.say("foo") @@ -1723,34 +1745,73 @@ lua exceeding response header limit ngx.header.content_type = nil } } + + location = /t { + default_type text/html; + rewrite_by_lua_block { + local h = ngx.header.content_length + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; + } --- request GET /t --- response_body foo --- response_headers -blah: foo -Content-Type: +!Content-Type --- no_error_log [error] -=== TEST 77: don't generate Content-Type header when getting other response headers +=== TEST 78: generate default Content-Type when setting other response header --- config location = /t { default_type text/html; - rewrite_by_lua_block { - local h = ngx.header.content_length + content_by_lua_block { + ngx.header.blah = "foo" + ngx.say("foo") } - proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; } +--- request +GET /t +--- response_body +foo +--- response_headers +blah: foo +Content-Type: text/html +--- no_error_log +[error] - location = /backend { + + +=== TEST 79: don't generate default Content-Type when Content-Type is cleared +--- config + location = /t { + default_type text/html; content_by_lua_block { + ngx.header["Content-Type"] = nil ngx.say("foo") } - header_filter_by_lua_block { - ngx.header.content_type = nil + } +--- request +GET /t +--- response_body +foo +--- response_headers +!Content-Type +--- no_error_log +[error] + + + +=== TEST 80: don't generate default Content-Type when Content-Type is set +--- config + location = /t { + default_type text/html; + content_by_lua_block { + ngx.header["Content-Type"] = "application/json" + ngx.say("foo") } } --- request @@ -1758,6 +1819,6 @@ GET /t --- response_body foo --- response_headers -Content-Type: +Content-Type: application/json --- no_error_log [error] From fbb89193002dabd82efd3db44271822ec06151c6 Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 28 Dec 2018 19:19:38 +0800 Subject: [PATCH 270/848] change: we now avoid generating the Content-Type response header when getting all response headers. The previous behavior assumed that nginx will always generate the Content-Type header if it is missing. However in some cases (such as the upstream response not having a Content-Type header), nginx will not generate the Content-Type header itself. Thus, generating the Content-Type when getting response headers may create an unexpected Content-Type header in the response. Signed-off-by: Thibault Charbonnier --- README.markdown | 37 ------- doc/HttpLuaModule.wiki | 35 ------- src/ngx_http_lua_headers.c | 33 ++---- src/ngx_http_lua_headers_out.c | 24 ++--- t/016-resp-header.t | 185 ++++++++++++++++++--------------- 5 files changed, 112 insertions(+), 202 deletions(-) diff --git a/README.markdown b/README.markdown index 4f31cb5243..96bd7a5992 100644 --- a/README.markdown +++ b/README.markdown @@ -4147,25 +4147,6 @@ to be returned when reading `ngx.header.Foo`. Note that `ngx.header` is not a normal Lua table and as such, it is not possible to iterate through it using the Lua `ipairs` function. -Note: if the current response does not have a Content-Type header, but you use -`ngx.header` to attempt retrieving it, a side effect will be that a -Content-Type header will be created and returned according to your nginx -configuration. - -For example: - -```nginx - - location = /t { - default_type text/html; - content_by_lua_block { - -- even if the response had no Content-Type header, - -- it will now have: Content-Type: text/html - ngx.say(ngx.header['Content-Type']) - } - } -``` - For reading *request* headers, use the [ngx.req.get_headers](#ngxreqget_headers) function instead. [Back to TOC](#nginx-api-for-lua) @@ -4191,24 +4172,6 @@ Returns a Lua table holding all the current response headers for the current req end ``` -Note: if the current response does not have a Content-Type header, but you use -`ngx.resp.get_headers` to attempt retrieving it, a side effect will -be that a Content-Type header will be created and returned according to your -nginx configuration. - -For example: - -```nginx - - location = /t { - default_type text/html; - content_by_lua_block { - -- the Content-Type header will be text/html - ngx.say(ngx.resp.get_headers()['Content-Type']) - } - } -``` - This function has the same signature as [ngx.req.get_headers](#ngxreqget_headers) except getting response headers instead of request headers. Note that a maximum of 100 response headers are parsed by default (including those with the same name) and that additional response headers are silently discarded to guard against potential denial of service attacks. Since `v0.10.13`, when the limit is exceeded, it will return a second value which is the string `"truncated"`. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 82a7f6a64f..8f97d0d74d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3433,24 +3433,6 @@ to be returned when reading ngx.header.Foo. Note that ngx.header is not a normal Lua table and as such, it is not possible to iterate through it using the Lua ipairs function. -Note: if the current response does not have a Content-Type header, but you use -ngx.header to attempt retrieving it, a side effect will be that a -Content-Type header will be created and returned according to your nginx -configuration. - -For example: - - - location = /t { - default_type text/html; - content_by_lua_block { - -- even if the response had no Content-Type header, - -- it will now have: Content-Type: text/html - ngx.say(ngx.header['Content-Type']) - } - } - - For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_headers]] function instead. == ngx.resp.get_headers == @@ -3472,23 +3454,6 @@ for k, v in pairs(h) do end -Note: if the current response does not have a Content-Type header, but you use -ngx.resp.get_headers, a side effect will be that a Content-Type -header will be created and returned according to your nginx configuration. - -For example: - - - location = /t { - default_type text/html; - content_by_lua_block { - -- even if the response had no Content-Type header, - -- it will now have: Content-Type: text/html - ngx.say(ngx.resp.get_headers()['Content-Type']) - } - } - - This function has the same signature as [[#ngx.req.get_headers|ngx.req.get_headers]] except getting response headers instead of request headers. Note that a maximum of 100 response headers are parsed by default (including those with the same name) and that additional response headers are silently discarded to guard against potential denial of service attacks. Since v0.10.13, when the limit is exceeded, it will return a second value which is the string `"truncated"`. diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index fb743dd159..e3f48bcbef 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -505,7 +505,6 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) ngx_table_elt_t *header; ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; - ngx_int_t rc; u_char *lowcase_key = NULL; size_t lowcase_key_sz = 0; ngx_uint_t i; @@ -562,17 +561,6 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) lua_setmetatable(L, -2); } - if (!r->headers_out.content_type.len) { - rc = ngx_http_lua_set_content_type(r, ctx); - if (rc != NGX_OK) { - return luaL_error(L, - "failed to set default content type: %d", - (int) rc); - } - - ctx->headers_set = 1; - } - #if 1 if (r->headers_out.content_type.len) { extra++; @@ -1405,21 +1393,12 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, break; case 12: - if (ngx_strncasecmp(key_buf, (u_char *) "Content-Type", 12) == 0) { - if (!r->headers_out.content_type.len) { - if (ngx_http_lua_set_content_type(r, ctx) != NGX_OK) { - *errmsg = "failed to set default content type"; - return NGX_ERROR; - } - - ctx->headers_set = 1; - } - - if (r->headers_out.content_type.len) { - values[0].data = r->headers_out.content_type.data; - values[0].len = r->headers_out.content_type.len; - return 1; - } + if (ngx_strncasecmp(key_buf, (u_char *) "Content-Type", 12) == 0 + && r->headers_out.content_type.len) + { + values[0].data = r->headers_out.content_type.data; + values[0].len = r->headers_out.content_type.len; + return 1; } break; diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index ee542ee125..421df90e88 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -536,7 +536,6 @@ ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, { ngx_table_elt_t *h; ngx_list_part_t *part; - ngx_int_t rc; ngx_uint_t i; unsigned found; @@ -555,23 +554,12 @@ ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, break; case 12: - if (ngx_strncasecmp(key->data, (u_char *) "Content-Type", 12) == 0) { - if (r->headers_out.content_type.len == 0) { - rc = ngx_http_lua_set_content_type(r, ctx); - if (rc != NGX_OK) { - return luaL_error(L, - "failed to set default content type: %d", - (int) rc); - } - - ctx->headers_set = 1; - } - - if (r->headers_out.content_type.len) { - lua_pushlstring(L, (char *) r->headers_out.content_type.data, - r->headers_out.content_type.len); - return 1; - } + if (ngx_strncasecmp(key->data, (u_char *) "Content-Type", 12) == 0 + && r->headers_out.content_type.len) + { + lua_pushlstring(L, (char *) r->headers_out.content_type.data, + r->headers_out.content_type.len); + return 1; } break; diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 6a41672b46..b2987a1ec9 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 55); +plan tests => repeat_each() * (blocks() * 3 + 58); #no_diff(); no_long_string(); @@ -1501,84 +1501,11 @@ test -=== TEST 69: return the matched content-type instead of default_type ---- http_config -types { - image/png png; -} ---- config -location /set/ { - default_type text/html; - content_by_lua_block { - ngx.say(ngx.header["content-type"]) - } -} ---- request -GET /set/hello.png ---- response_headers -Content-Type: image/png ---- response_body -image/png ---- no_error_log -[error] - - - -=== TEST 70: always return the matched content-type ---- config - location /set/ { - default_type "image/png"; - content_by_lua_block { - ngx.say(ngx.header["content-type"]) - ngx.say(ngx.header["content-type"]) - } - } ---- request -GET /set/hello.png ---- response_headers -Content-Type: image/png ---- response_body -image/png -image/png ---- no_error_log -[error] - - - -=== TEST 71: return the matched content-type after ngx.resp.get_headers() ---- http_config -types { - image/png png; -} ---- config - location /set/ { - default_type text/html; - content_by_lua_block { - local h, err = ngx.resp.get_headers() - if err then - ngx.log(ngx.ERR, "err: ", err) - return ngx.exit(500) - end - - ngx.say(h["content-type"]) - } - } ---- request -GET /set/hello.png ---- response_headers -Content-Type: image/png ---- response_body -image/png ---- no_error_log -[error] - - - -=== TEST 72: exceeding max header limit (default 100) +=== TEST 69: exceeding max header limit (default 100) --- config location /resp-header { content_by_lua_block { - for i = 1, 99 do + for i = 1, 100 do ngx.header["Foo" .. i] = "Foo" end @@ -1608,11 +1535,11 @@ lua exceeding response header limit 101 > 100 -=== TEST 73: NOT exceeding max header limit (default 100) +=== TEST 70: NOT exceeding max header limit (default 100) --- config location /resp-header { content_by_lua_block { - for i = 1, 98 do + for i = 1, 99 do ngx.header["Foo" .. i] = "Foo" end @@ -1640,11 +1567,11 @@ lua exceeding response header limit -=== TEST 74: exceeding max header limit (custom limit, 3) +=== TEST 71: exceeding max header limit (custom limit, 3) --- config location /resp-header { content_by_lua_block { - for i = 1, 2 do + for i = 1, 3 do ngx.header["Foo" .. i] = "Foo" end @@ -1674,11 +1601,11 @@ lua exceeding response header limit 4 > 3 -=== TEST 75: NOT exceeding max header limit (custom limit, 3) +=== TEST 72: NOT exceeding max header limit (custom limit, 3) --- config location /resp-header { content_by_lua_block { - for i = 1, 1 do + for i = 1, 2 do ngx.header["Foo" .. i] = "Foo" end @@ -1705,7 +1632,29 @@ lua exceeding response header limit -=== TEST 76: don't generate Content-Type when setting other response header +=== TEST 73: return nil if Content-Type is not set yet +--- config + location /t { + default_type text/html; + content_by_lua_block { + ngx.log(ngx.WARN, "Content-Type: ", ngx.header["content-type"]) + ngx.say("Content-Type: ", ngx.header["content-type"]) + } + } +--- request +GET /t +--- response_headers +Content-Type: text/html +--- response_body +Content-Type: nil +--- no_error_log +[error] +--- error_log +Content-Type: nil + + + +=== TEST 74: don't generate Content-Type when setting other response header --- config location = /backend { content_by_lua_block { @@ -1735,7 +1684,7 @@ blah: foo -=== TEST 77: don't generate Content-Type when getting other response header +=== TEST 75: don't generate Content-Type when getting other response header --- config location = /backend { content_by_lua_block { @@ -1764,7 +1713,37 @@ foo -=== TEST 78: generate default Content-Type when setting other response header +=== TEST 76: don't generate Content-Type when getting it +--- config + location = /backend { + content_by_lua_block { + ngx.say("foo") + } + header_filter_by_lua_block { + ngx.header.content_type = nil + } + } + + location /t { + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; + header_filter_by_lua_block { + ngx.log(ngx.WARN, "Content-Type: ", ngx.header["content-type"]) + } + } +--- request +GET /t +--- response_body +foo +--- response_headers +!Content-Type +--- no_error_log +[error] +--- error_log +Content-Type: nil + + + +=== TEST 77: generate default Content-Type when setting other response header --- config location = /t { default_type text/html; @@ -1785,6 +1764,42 @@ Content-Type: text/html +=== TEST 78: don't generate Content-Type when calling ngx.resp.get_headers() +--- config + location = /backend { + content_by_lua_block { + ngx.say("foo") + } + header_filter_by_lua_block { + ngx.header.content_type = nil + } + } + + location /t { + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/backend; + header_filter_by_lua_block { + local h, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return + end + + ngx.log(ngx.WARN, "Content-Type: ", h["content-type"]) + } + } +--- request +GET /t +--- response_body +foo +--- response_headers +!Content-Type +--- no_error_log +[error] +--- error_log +Content-Type: nil + + + === TEST 79: don't generate default Content-Type when Content-Type is cleared --- config location = /t { From e76292dee587ecbc6eb21f5aa3377fe38468456c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 7 Jan 2019 10:59:14 -0800 Subject: [PATCH 271/848] tests: reindexed the t/158-global-var.t suite. --- t/158-global-var.t | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/t/158-global-var.t b/t/158-global-var.t index 745e7eaebb..414a43aac9 100644 --- a/t/158-global-var.t +++ b/t/158-global-var.t @@ -186,6 +186,7 @@ body_filter_by_lua:3: in main chunk, old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] + === TEST 7: log_by_lua --- config location /t { @@ -211,7 +212,6 @@ log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] - === TEST 8: ssl_certificate_by_lua --- http_config server { @@ -304,7 +304,6 @@ ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] - === TEST 9: timer --- config location /t { @@ -336,7 +335,6 @@ content_by_lua\(nginx\.conf:56\):4: in\n\z/, "old foo: 1\n"] - === TEST 10: init_by_lua --- http_config init_by_lua_block { From 7c2b58ee7e66689ef176b251d43bfc84f7db26c6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Wed, 9 Jan 2019 01:53:51 +0800 Subject: [PATCH 272/848] change: we now avoid running init_by_lua* in signaller processes and when testing the nginx configuration. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_module.c | 4 ++ t/160-disable-init-by-lua.t | 131 ++++++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 t/160-disable-init-by-lua.t diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index bd54547e54..5f242eab4a 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -647,6 +647,10 @@ ngx_http_lua_init(ngx_conf_t *cf) #endif ngx_str_t name = ngx_string("host"); + if (ngx_process == NGX_PROCESS_SIGNALLER || ngx_test_config) { + return NGX_OK; + } + lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); lmcf->host_var_index = ngx_http_get_variable_index(cf, &name); diff --git a/t/160-disable-init-by-lua.t b/t/160-disable-init-by-lua.t new file mode 100644 index 0000000000..c0c040859b --- /dev/null +++ b/t/160-disable-init-by-lua.t @@ -0,0 +1,131 @@ +use Test::Nginx::Socket::Lua; + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 2); + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); + +my $html_dir = $ENV{TEST_NGINX_HTML_DIR}; +my $http_config = <<_EOC_; + init_by_lua_block { + function set_up_ngx_tmp_conf() + local conf = [[ + events { + worker_connections 64; + } + http { + init_by_lua_block { + ngx.log(ngx.ERR, "run init_by_lua") + } + } + ]] + + assert(os.execute("mkdir -p $html_dir/logs")) + + local conf_file = "$html_dir/nginx.conf" + local f, err = io.open(conf_file, "w") + if not f then + ngx.log(ngx.ERR, err) + return + end + + assert(f:write(conf)) + + return conf_file + end + + function get_ngx_bin_path() + local ffi = require "ffi" + ffi.cdef[[char **ngx_argv;]] + return ffi.string(ffi.C.ngx_argv[0]) + end + } +_EOC_ + +add_block_preprocessor(sub { + my $block = shift; + + if (!defined $block->http_config) { + $block->set_value("http_config", $http_config); + } + + if (!defined $block->request) { + $block->set_value("request", "GET /t"); + } +}); + +log_level("warn"); +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: ensure init_by_lua* is not run in signaller process +--- config + location = /t { + content_by_lua_block { + local conf_file = set_up_ngx_tmp_conf() + local nginx = get_ngx_bin_path() + + local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -s reopen" + local p, err = io.popen(cmd) + if not p then + ngx.log(ngx.ERR, err) + return + end + + local out, err = p:read("*a") + if not out then + ngx.log(ngx.ERR, err) + + else + ngx.log(ngx.WARN, out) + end + } + } +--- no_error_log eval +qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ + + + +=== TEST 2: init_by_lua* does not run when testing Nginx configuration +--- config + location = /t { + content_by_lua_block { + local conf_file = set_up_ngx_tmp_conf() + local nginx = get_ngx_bin_path() + + local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -t" + local p, err = io.popen(cmd) + if not p then + ngx.log(ngx.ERR, err) + return + end + + local out, err = p:read("*a") + if not out then + ngx.log(ngx.ERR, err) + + else + ngx.log(ngx.WARN, out) + end + + local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -T" + local p, err = io.popen(cmd) + if not p then + ngx.log(ngx.ERR, err) + return + end + + local out, err = p:read("*a") + if not out then + ngx.log(ngx.ERR, err) + + else + ngx.log(ngx.WARN, out) + end + } + } +--- no_error_log eval +qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ From 248060b81302431ccfbf8bcce999647814ecd2f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Wed, 23 Jan 2019 06:22:51 +0800 Subject: [PATCH 273/848] feature: errors are now logged when timers fail to run. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_timer.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index eb10a8fc36..e92c211458 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -350,6 +350,9 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) ngx_add_timer(ev, delay); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "created timer (co: %p delay: %M ms)", tctx->co, delay); + lua_pushinteger(L, 1); return 1; @@ -509,6 +512,10 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) ngx_add_timer(ev, tctx->delay); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "created next timer (co: %p delay: %M ms)", tctx->co, + tctx->delay); + return NGX_OK; nomem: @@ -580,6 +587,9 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) c = ngx_http_lua_create_fake_connection(tctx.pool); if (c == NULL) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "failed to create fake connection to run timer (co: %p)", + tctx.co); goto failed; } @@ -591,6 +601,9 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) r = ngx_http_lua_create_fake_request(c); if (r == NULL) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "failed to create fake request to run timer (co: %p)", + tctx.co); goto failed; } @@ -626,6 +639,8 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) ctx = ngx_http_lua_create_ctx(r); if (ctx == NULL) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "failed to create ctx to run timer (co: %p)", tctx.co); goto failed; } @@ -634,6 +649,9 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) pcln = ngx_pool_cleanup_add(r->pool, 0); if (pcln == NULL) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "failed to add vm cleanup to run timer (co: %p)", + tctx.co); goto failed; } @@ -647,6 +665,9 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) cln = ngx_http_cleanup_add(r, 0); if (cln == NULL) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "failed to add request cleanup to run timer (co: %p)", + tctx.co); goto failed; } From 2fb8f120c3b3eefab196ff4245cf227ddcf3eac1 Mon Sep 17 00:00:00 2001 From: bjoe2k4 Date: Tue, 16 Oct 2018 16:03:42 +0200 Subject: [PATCH 274/848] feature: added support for the Nginx builtin Link header. This allows for specifying multiple values in the Link header. Co-authored-by: tokers Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_headers_out.c | 6 + t/016-resp-header.t | 226 +++++++++++++++++++++++++-------- 2 files changed, 182 insertions(+), 50 deletions(-) diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 421df90e88..cf94bd0423 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -106,6 +106,12 @@ static ngx_http_lua_set_header_t ngx_http_lua_set_handlers[] = { offsetof(ngx_http_headers_out_t, cache_control), ngx_http_set_builtin_multi_header }, +#if defined(nginx_version) && nginx_version >= 1013009 + { ngx_string("Link"), + offsetof(ngx_http_headers_out_t, link), + ngx_http_set_builtin_multi_header }, +#endif + { ngx_null_string, 0, ngx_http_set_header } }; diff --git a/t/016-resp-header.t b/t/016-resp-header.t index b2987a1ec9..e38202b144 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 58); +plan tests => repeat_each() * (blocks() * 3 + 59); #no_diff(); no_long_string(); @@ -630,7 +630,46 @@ Cache-Control: private, no-store -=== TEST 32: set multi values to cache-control and override it with a single value +=== TEST 32: set single value to Link header +--- config + location = /t { + content_by_lua_block { + ngx.header.link = "; rel=preload" + ngx.say("Link: ", ngx.var.sent_http_link) + } + } +--- request +GET /t +--- response_headers +Link: ; rel=preload +--- response_body +Link: ; rel=preload + + + +=== TEST 33: set multi values to Link header +--- config + location = /t { + content_by_lua_block { + ngx.header.link = { + "; rel=preload", + "; rel=preload; as=style" + } + + ngx.say("Link: ", ngx.var.sent_http_link) + } + } +--- request +GET /t +--- response_headers +Link: ; rel=preload, ; rel=preload; as=style +--- response_body_like chop +^Link: ; rel=preload[;,] ; rel=preload; as=style$ +--- skip_nginx: 3: < 1.13.9 + + + +=== TEST 34: set multi values to cache-control and override it with a single value --- config location /lua { content_by_lua ' @@ -650,7 +689,30 @@ Cache-Control: no-cache -=== TEST 33: set multi values to cache-control and override it with multiple values +=== TEST 35: set multi values to Link header and override it with a single value +--- config + location /lua { + content_by_lua_block { + ngx.header.link = { + "; rel=preload", + "; rel=preload; as=style" + } + ngx.header.link = "; rel=preload" + ngx.say("Link: ", ngx.var.sent_http_link) + ngx.say("Link: ", ngx.header.link) + } + } +--- request + GET /lua +--- response_headers +Link: ; rel=preload +--- response_body +Link: ; rel=preload +Link: ; rel=preload + + + +=== TEST 36: set multi values to cache-control and override it with multiple values --- config location /lua { content_by_lua ' @@ -672,7 +734,37 @@ Cache-Control: no-cache[;,] blah[;,] foo$ -=== TEST 34: set the www-authenticate response header +=== TEST 37: set multi values to Link header and override it with multiple values +--- config + location /lua { + content_by_lua_block { + ngx.header.link = { + "; rel=preload", + "; rel=preload; as=style" + } + ngx.header.link = { + "; rel=preload", + "; rel=preload", + "; rel=preload; as=style" + } + ngx.say("Link: ", ngx.var.sent_http_link) + ngx.say("Link: ", table.concat(ngx.header.link, ", ")) + } + } +--- request + GET /lua +--- response_headers +Link: ; rel=preload, ; rel=preload, ; rel=preload; as=style +--- response_body_like chop +^Link: ; rel=preload[;,] ; rel=preload[;,] ; rel=preload; as=style +Link: ; rel=preload[;,] ; rel=preload[;,] ; rel=preload; as=style$ +--- no_error_log +[error] +--- skip_nginx: 4: < 1.13.9 + + + +=== TEST 38: set the www-authenticate response header --- config location /lua { content_by_lua ' @@ -689,7 +781,7 @@ WWW-Authenticate: blah -=== TEST 35: set and clear the www-authenticate response header +=== TEST 39: set and clear the www-authenticate response header --- config location /lua { content_by_lua ' @@ -707,7 +799,7 @@ Foo: nil -=== TEST 36: set multi values to cache-control and override it with multiple values (to reproduce a bug) +=== TEST 40: set multi values to cache-control and override it with multiple values (to reproduce a bug) --- config location /lua { content_by_lua ' @@ -727,7 +819,7 @@ Cache-Control: blah -=== TEST 37: set last-modified and return 304 +=== TEST 41: set last-modified and return 304 --- config location /lua { content_by_lua ' @@ -745,7 +837,7 @@ Last-Modified: Thu, 18 Nov 2010 11:27:35 GMT -=== TEST 38: set last-modified and return 200 +=== TEST 42: set last-modified and return 200 --- config location /lua { content_by_lua ' @@ -764,7 +856,7 @@ Thu, 18 Nov 2010 11:27:35 GMT -=== TEST 39: set response content-encoding header should bypass ngx_http_gzip_filter_module +=== TEST 43: set response content-encoding header should bypass ngx_http_gzip_filter_module --- config default_type text/plain; gzip on; @@ -790,7 +882,7 @@ Hello, world, my dear friend! -=== TEST 40: no transform underscores (write) +=== TEST 44: no transform underscores (write) --- config lua_transform_underscores_in_response_headers off; location = /t { @@ -810,7 +902,7 @@ nil -=== TEST 41: with transform underscores (write) +=== TEST 45: with transform underscores (write) --- config lua_transform_underscores_in_response_headers on; location = /t { @@ -830,7 +922,7 @@ Hello -=== TEST 42: github issue #199: underscores in lua variables +=== TEST 46: github issue #199: underscores in lua variables --- config location /read { content_by_lua ' @@ -863,7 +955,7 @@ something: hello -=== TEST 43: set multiple response header +=== TEST 47: set multiple response header --- config location /read { content_by_lua ' @@ -883,7 +975,7 @@ text/my-plain-50 -=== TEST 44: set multiple response header and then reset and then clear +=== TEST 48: set multiple response header and then reset and then clear --- config location /read { content_by_lua ' @@ -912,7 +1004,7 @@ ok -=== TEST 45: set response content-type header for multiple times +=== TEST 49: set response content-type header for multiple times --- config location /read { content_by_lua ' @@ -930,7 +1022,7 @@ Hi -=== TEST 46: set Last-Modified response header for multiple times +=== TEST 50: set Last-Modified response header for multiple times --- config location /read { content_by_lua ' @@ -948,7 +1040,7 @@ ok -=== TEST 47: set Last-Modified response header and then clear +=== TEST 51: set Last-Modified response header and then clear --- config location /read { content_by_lua ' @@ -966,7 +1058,7 @@ ok -=== TEST 48: github #20: segfault caused by the nasty optimization in the nginx core (write) +=== TEST 52: github #20: segfault caused by the nasty optimization in the nginx core (write) --- config location = /t/ { header_filter_by_lua ' @@ -988,7 +1080,7 @@ Location: http://localhost:$ServerPort/t/ -=== TEST 49: github #20: segfault caused by the nasty optimization in the nginx core (read) +=== TEST 53: github #20: segfault caused by the nasty optimization in the nginx core (read) --- config location = /t/ { header_filter_by_lua ' @@ -1010,7 +1102,7 @@ Location: http://localhost:$ServerPort/t/ -=== TEST 50: github #20: segfault caused by the nasty optimization in the nginx core (read Location) +=== TEST 54: github #20: segfault caused by the nasty optimization in the nginx core (read Location) --- config location = /t/ { header_filter_by_lua ' @@ -1033,7 +1125,7 @@ Foo: /t/ -=== TEST 51: github #20: segfault caused by the nasty optimization in the nginx core (set Foo and read Location) +=== TEST 55: github #20: segfault caused by the nasty optimization in the nginx core (set Foo and read Location) --- config location = /t/ { header_filter_by_lua ' @@ -1057,7 +1149,7 @@ Foo: /t/ -=== TEST 52: case sensitive cache-control header +=== TEST 56: case sensitive cache-control header --- config location /lua { content_by_lua ' @@ -1074,7 +1166,24 @@ Cache-Control: private -=== TEST 53: clear Cache-Control when there was no Cache-Control +=== TEST 57: case sensitive Link header +--- config + location /lua { + content_by_lua_block { + ngx.header["link"] = "; rel=preload" + ngx.say("Link: ", ngx.var.sent_http_link) + } + } +--- request + GET /lua +--- raw_response_headers_like chop +link: ; rel=preload +--- response_body +Link: ; rel=preload + + + +=== TEST 58: clear Cache-Control when there was no Cache-Control --- config location /lua { content_by_lua ' @@ -1091,7 +1200,24 @@ Cache-Control: nil -=== TEST 54: set response content-type header +=== TEST 59: clear Link header when there was no Link +--- config + location /lua { + content_by_lua_block { + ngx.header["Link"] = nil + ngx.say("Link: ", ngx.var.sent_http_link) + } + } +--- request + GET /lua +--- raw_response_headers_unlike eval +qr/Link/i +--- response_body +Link: nil + + + +=== TEST 60: set response content-type header --- config location /read { content_by_lua ' @@ -1110,7 +1236,7 @@ s = content_type -=== TEST 55: set a number header name +=== TEST 61: set a number header name --- config location /lua { content_by_lua ' @@ -1129,7 +1255,7 @@ s = content_type -=== TEST 56: set a number header name (in a table value) +=== TEST 62: set a number header name (in a table value) --- config location /lua { content_by_lua ' @@ -1148,7 +1274,7 @@ foo: 32 -=== TEST 57: random access resp headers +=== TEST 63: random access resp headers --- config location /resp-header { content_by_lua ' @@ -1188,7 +1314,7 @@ bar: baz -=== TEST 58: iterating through raw resp headers +=== TEST 64: iterating through raw resp headers --- config location /resp-header { content_by_lua ' @@ -1224,7 +1350,7 @@ bar: nil -=== TEST 59: removed response headers +=== TEST 65: removed response headers --- config location /resp-header { content_by_lua ' @@ -1257,7 +1383,7 @@ bar: baz -=== TEST 60: built-in Content-Type header +=== TEST 66: built-in Content-Type header --- config location = /t { content_by_lua ' @@ -1290,7 +1416,7 @@ my content_type: text/plain -=== TEST 61: built-in Content-Length header +=== TEST 67: built-in Content-Length header --- config location = /t { content_by_lua ' @@ -1323,7 +1449,7 @@ my content_length: 3 -=== TEST 62: built-in Connection header +=== TEST 68: built-in Connection header --- config location = /t { content_by_lua ' @@ -1354,7 +1480,7 @@ my connection: close -=== TEST 63: built-in Transfer-Encoding header (chunked) +=== TEST 69: built-in Transfer-Encoding header (chunked) --- config location = /t { content_by_lua ' @@ -1386,7 +1512,7 @@ my transfer-encoding: chunked -=== TEST 64: built-in Transfer-Encoding header (none) +=== TEST 70: built-in Transfer-Encoding header (none) --- config location = /t { content_by_lua ' @@ -1419,7 +1545,7 @@ my transfer_encoding: nil -=== TEST 65: set Location (no host) +=== TEST 71: set Location (no host) --- config location = /t { content_by_lua ' @@ -1438,7 +1564,7 @@ Location: /foo/bar -=== TEST 66: set Location (with host) +=== TEST 72: set Location (with host) --- config location = /t { content_by_lua ' @@ -1457,7 +1583,7 @@ Location: http://test.com/foo/bar -=== TEST 67: ngx.header["Content-Type"] with ngx_gzip +=== TEST 73: ngx.header["Content-Type"] with ngx_gzip --- config gzip on; gzip_min_length 1; @@ -1481,7 +1607,7 @@ Content-Type: text/html; charset=utf-8 -=== TEST 68: ngx.header["Content-Type"] with "; blah" +=== TEST 74: ngx.header["Content-Type"] with "; blah" --- config location = /test2 { content_by_lua ' @@ -1501,7 +1627,7 @@ test -=== TEST 69: exceeding max header limit (default 100) +=== TEST 75: exceeding max header limit (default 100) --- config location /resp-header { content_by_lua_block { @@ -1535,7 +1661,7 @@ lua exceeding response header limit 101 > 100 -=== TEST 70: NOT exceeding max header limit (default 100) +=== TEST 76: NOT exceeding max header limit (default 100) --- config location /resp-header { content_by_lua_block { @@ -1567,7 +1693,7 @@ lua exceeding response header limit -=== TEST 71: exceeding max header limit (custom limit, 3) +=== TEST 77: exceeding max header limit (custom limit, 3) --- config location /resp-header { content_by_lua_block { @@ -1601,7 +1727,7 @@ lua exceeding response header limit 4 > 3 -=== TEST 72: NOT exceeding max header limit (custom limit, 3) +=== TEST 78: NOT exceeding max header limit (custom limit, 3) --- config location /resp-header { content_by_lua_block { @@ -1632,7 +1758,7 @@ lua exceeding response header limit -=== TEST 73: return nil if Content-Type is not set yet +=== TEST 79: return nil if Content-Type is not set yet --- config location /t { default_type text/html; @@ -1654,7 +1780,7 @@ Content-Type: nil -=== TEST 74: don't generate Content-Type when setting other response header +=== TEST 80: don't generate Content-Type when setting other response header --- config location = /backend { content_by_lua_block { @@ -1684,7 +1810,7 @@ blah: foo -=== TEST 75: don't generate Content-Type when getting other response header +=== TEST 81: don't generate Content-Type when getting other response header --- config location = /backend { content_by_lua_block { @@ -1713,7 +1839,7 @@ foo -=== TEST 76: don't generate Content-Type when getting it +=== TEST 82: don't generate Content-Type when getting it --- config location = /backend { content_by_lua_block { @@ -1743,7 +1869,7 @@ Content-Type: nil -=== TEST 77: generate default Content-Type when setting other response header +=== TEST 83: generate default Content-Type when setting other response header --- config location = /t { default_type text/html; @@ -1764,7 +1890,7 @@ Content-Type: text/html -=== TEST 78: don't generate Content-Type when calling ngx.resp.get_headers() +=== TEST 84: don't generate Content-Type when calling ngx.resp.get_headers() --- config location = /backend { content_by_lua_block { @@ -1800,7 +1926,7 @@ Content-Type: nil -=== TEST 79: don't generate default Content-Type when Content-Type is cleared +=== TEST 85: don't generate default Content-Type when Content-Type is cleared --- config location = /t { default_type text/html; @@ -1820,7 +1946,7 @@ foo -=== TEST 80: don't generate default Content-Type when Content-Type is set +=== TEST 86: don't generate default Content-Type when Content-Type is set --- config location = /t { default_type text/html; From c2940c132bf7e37e6a28731864892dae34c81a34 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 19 Dec 2018 21:31:58 +0800 Subject: [PATCH 275/848] feature: added backlog queueing to cosocket connection pools. Now we can pass 'pool_size' and 'backlog' options in connect method to specify the pool size and the queue size of the connection pool. When there is more connect operations than the pool_size, new connect operation will be queued until former connect operation fails or connection finalizes. When there is more queueing connect operations than the backlog, new connect operation will be dropped. This pr also migrated connection pool creation to connect method. Note that the meaning of `active_connections` is changed from the previous setkeepalive mechanism. The connection pool is destroyed only if all related connections closed now. Signed-off-by: Thibault Charbonnier --- README.markdown | 69 +- doc/HttpLuaModule.wiki | 69 +- src/ngx_http_lua_socket_tcp.c | 1137 ++++++++++++++++++++-------- src/ngx_http_lua_socket_tcp.h | 29 +- t/068-socket-keepalive.t | 1316 ++++++++++++++++++++++++++++++++- 5 files changed, 2292 insertions(+), 328 deletions(-) diff --git a/README.markdown b/README.markdown index 96bd7a5992..adeadcdb53 100644 --- a/README.markdown +++ b/README.markdown @@ -7091,6 +7091,43 @@ An optional Lua table can be specified as the last argument to this method to sp * `pool` specify a custom name for the connection pool being used. If omitted, then the connection pool name will be generated from the string template `":"` or `""`. +* `pool_size` + specify the size of the connection pool. If omitted and no + `backlog` option was provided, no pool will be created. If omitted + but `backlog` was provided, the pool will be created with a default + size equal to the value of the [lua_socket_pool_size](#lua_socket_pool_size) + directive. + The connection pool holds up to `pool_size` alive connections + ready to be reused by subsequent calls to [connect](#tcpsockconnect), but + note that there is no upper limit to the total number of opened connections + outside of the pool. If you need to restrict the total number of opened + connections, specify the `backlog` option. + When the connection pool would exceed its size limit, the least recently used + (kept-alive) connection already in the pool will be closed to make room for + the current connection. + Note that the cosocket connection pool is per Nginx worker process rather + than per Nginx server instance, so the size limit specified here also applies + to every single Nginx worker process. Also note that the size of the connection + pool cannot be changed once it has been created. + This option was first introduced in the `v0.10.14` release. + +* `backlog` + if specified, this module will limit the total number of opened connections + for this pool. No more connections than `pool_size` can be opened + for this pool at any time. If the connection pool is full, subsequent + connect operations will be queued into a queue equal to this option's + value (the "backlog" queue). + If the number of queued connect operations is equal to `backlog`, + subsequent connect operations will fail and return `nil` plus the + error string `"too many waiting connect operations"`. + The queued connect operations will be resumed once the number of connections + in the pool is less than `pool_size`. + The queued connect operation will abort once they have been queued for more + than `connect_timeout`, controlled by + [settimeouts](#tcpsocksettimeouts), and will return `nil` plus + the error string `"timeout"`. + This option was first introduced in the `v0.10.14` release. + The support for the options table argument was first introduced in the `v0.5.7` release. This method was first introduced in the `v0.5.0rc1` release. @@ -7422,13 +7459,31 @@ Puts the current socket's connection immediately into the cosocket built-in conn The first optional argument, `timeout`, can be used to specify the maximal idle timeout (in milliseconds) for the current connection. If omitted, the default setting in the [lua_socket_keepalive_timeout](#lua_socket_keepalive_timeout) config directive will be used. If the `0` value is given, then the timeout interval is unlimited. -The second optional argument, `size`, can be used to specify the maximal number of connections allowed in the connection pool for the current server (i.e., the current host-port pair or the unix domain socket file path). Note that the size of the connection pool cannot be changed once the pool is created. When this argument is omitted, the default setting in the [lua_socket_pool_size](#lua_socket_pool_size) config directive will be used. - -When the connection pool exceeds the available size limit, the least recently used (idle) connection already in the pool will be closed to make room for the current connection. - -Note that the cosocket connection pool is per Nginx worker process rather than per Nginx server instance, so the size limit specified here also applies to every single Nginx worker process. - -Idle connections in the pool will be monitored for any exceptional events like connection abortion or unexpected incoming data on the line, in which cases the connection in question will be closed and removed from the pool. +The second optional argument `size` is considered deprecated since +the `v0.10.14` release of this module, in favor of the +`pool_size` option of the [connect](#tcpsockconnect) method. +Since the `v0.10.14` release, this option will only take effect if +the call to [connect](#tcpsockconnect) did not already create a connection +pool. +When this option takes effect (no connection pool was previously created by +[connect](#tcpsockconnect)), it will specify the size of the connection pool, +and create it. +If omitted (and no pool was previously created), the default size is the value +of the [lua_socket_pool_size](#lua_socket_pool_size) directive. +The connection pool holds up to `size` alive connections ready to be +reused by subsequent calls to [connect](#tcpsockconnect), but note that there +is no upper limit to the total number of opened connections outside of the +pool. +When the connection pool would exceed its size limit, the least recently used +(kept-alive) connection already in the pool will be closed to make room for +the current connection. +Note that the cosocket connection pool is per Nginx worker process rather +than per Nginx server instance, so the size limit specified here also applies +to every single Nginx worker process. Also note that the size of the connection +pool cannot be changed once it has been created. +If you need to restrict the total number of opened connections, specify both +the `pool_size` and `backlog` option in the call to +[connect](#tcpsockconnect). In case of success, this method returns `1`; otherwise, it returns `nil` and a string describing the error. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 8f97d0d74d..e11b35e715 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5979,6 +5979,43 @@ An optional Lua table can be specified as the last argument to this method to sp * pool : specify a custom name for the connection pool being used. If omitted, then the connection pool name will be generated from the string template ":" or "". +* pool_size +: specify the size of the connection pool. If omitted and no +: backlog option was provided, no pool will be created. If omitted +: but backlog was provided, the pool will be created with a default +: size equal to the value of the [[#lua_socket_pool_size|lua_socket_pool_size]] +: directive. +: The connection pool holds up to pool_size alive connections +: ready to be reused by subsequent calls to [[#tcpsock:connect|connect]], but +: note that there is no upper limit to the total number of opened connections +: outside of the pool. If you need to restrict the total number of opened +: connections, specify the backlog option. +: When the connection pool would exceed its size limit, the least recently used +: (kept-alive) connection already in the pool will be closed to make room for +: the current connection. +: Note that the cosocket connection pool is per Nginx worker process rather +: than per Nginx server instance, so the size limit specified here also applies +: to every single Nginx worker process. Also note that the size of the connection +: pool cannot be changed once it has been created. +: This option was first introduced in the v0.10.14 release. + +* backlog +: if specified, this module will limit the total number of opened connections +: for this pool. No more connections than pool_size can be opened +: for this pool at any time. If the connection pool is full, subsequent +: connect operations will be queued into a queue equal to this option's +: value (the "backlog" queue). +: If the number of queued connect operations is equal to backlog, +: subsequent connect operations will fail and return nil plus the +: error string "too many waiting connect operations". +: The queued connect operations will be resumed once the number of connections +: in the pool is less than pool_size. +: The queued connect operation will abort once they have been queued for more +: than connect_timeout, controlled by +: [[#tcpsock:settimeouts|settimeouts]], and will return nil plus +: the error string "timeout". +: This option was first introduced in the v0.10.14 release. + The support for the options table argument was first introduced in the v0.5.7 release. This method was first introduced in the v0.5.0rc1 release. @@ -6273,13 +6310,31 @@ Puts the current socket's connection immediately into the cosocket built-in conn The first optional argument, timeout, can be used to specify the maximal idle timeout (in milliseconds) for the current connection. If omitted, the default setting in the [[#lua_socket_keepalive_timeout|lua_socket_keepalive_timeout]] config directive will be used. If the 0 value is given, then the timeout interval is unlimited. -The second optional argument, size, can be used to specify the maximal number of connections allowed in the connection pool for the current server (i.e., the current host-port pair or the unix domain socket file path). Note that the size of the connection pool cannot be changed once the pool is created. When this argument is omitted, the default setting in the [[#lua_socket_pool_size|lua_socket_pool_size]] config directive will be used. - -When the connection pool exceeds the available size limit, the least recently used (idle) connection already in the pool will be closed to make room for the current connection. - -Note that the cosocket connection pool is per Nginx worker process rather than per Nginx server instance, so the size limit specified here also applies to every single Nginx worker process. - -Idle connections in the pool will be monitored for any exceptional events like connection abortion or unexpected incoming data on the line, in which cases the connection in question will be closed and removed from the pool. +The second optional argument size is considered deprecated since +the v0.10.14 release of this module, in favor of the +pool_size option of the [[#tcpsock:connect|connect]] method. +Since the v0.10.14 release, this option will only take effect if +the call to [[#tcpsock:connect|connect]] did not already create a connection +pool. +When this option takes effect (no connection pool was previously created by +[[#tcpsock:connect|connect]]), it will specify the size of the connection pool, +and create it. +If omitted (and no pool was previously created), the default size is the value +of the [[#lua_socket_pool_size|lua_socket_pool_size]] directive. +The connection pool holds up to size alive connections ready to be +reused by subsequent calls to [[#tcpsock:connect|connect]], but note that there +is no upper limit to the total number of opened connections outside of the +pool. +When the connection pool would exceed its size limit, the least recently used +(kept-alive) connection already in the pool will be closed to make room for +the current connection. +Note that the cosocket connection pool is per Nginx worker process rather +than per Nginx server instance, so the size limit specified here also applies +to every single Nginx worker process. Also note that the size of the connection +pool cannot be changed once it has been created. +If you need to restrict the total number of opened connections, specify both +the pool_size and backlog option in the call to +[[#tcpsock:connect|connect]]. In case of success, this method returns 1; otherwise, it returns nil and a string describing the error. diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index af7b28921e..fcaf51a0c7 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -100,12 +100,28 @@ static int ngx_http_lua_req_socket(lua_State *L); static void ngx_http_lua_req_socket_rev_handler(ngx_http_request_t *r); static int ngx_http_lua_socket_tcp_getreusedtimes(lua_State *L); static int ngx_http_lua_socket_tcp_setkeepalive(lua_State *L); +static void ngx_http_lua_socket_tcp_create_socket_pool(lua_State *L, + ngx_http_request_t *r, ngx_str_t key, ngx_int_t pool_size, + ngx_int_t backlog, ngx_http_lua_socket_pool_t **spool); static ngx_int_t ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, - lua_State *L, int key_index, ngx_http_lua_socket_tcp_upstream_t *u); static void ngx_http_lua_socket_keepalive_dummy_handler(ngx_event_t *ev); +static int ngx_http_lua_socket_tcp_connect_helper(lua_State *L, + ngx_http_lua_socket_tcp_upstream_t *u, ngx_http_request_t *r, + ngx_http_lua_ctx_t *ctx, u_char *host_ref, size_t host_len, in_port_t port, + unsigned resuming); +static void ngx_http_lua_socket_tcp_conn_op_timeout_handler( + ngx_event_t *ev); +static int ngx_http_lua_socket_tcp_conn_op_timeout_retval_handler( + ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); +static void ngx_http_lua_socket_tcp_resume_conn_op( + ngx_http_lua_socket_pool_t *spool); +static void ngx_http_lua_socket_tcp_conn_op_ctx_cleanup(void *data); +static void ngx_http_lua_socket_tcp_conn_op_resume_handler(ngx_event_t *ev); static ngx_int_t ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev); static void ngx_http_lua_socket_keepalive_rev_handler(ngx_event_t *ev); +static int ngx_http_lua_socket_tcp_conn_op_resume_retval_handler( + ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); static int ngx_http_lua_socket_tcp_upstream_destroy(lua_State *L); static int ngx_http_lua_socket_downstream_destroy(lua_State *L); static ngx_int_t ngx_http_lua_socket_push_input_data(ngx_http_request_t *r, @@ -118,6 +134,7 @@ static ngx_int_t ngx_http_lua_socket_add_input_buffer(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); static ngx_int_t ngx_http_lua_socket_insert_buffer(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, u_char *pat, size_t prefix); +static ngx_int_t ngx_http_lua_socket_tcp_conn_op_resume(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_socket_tcp_conn_resume(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_socket_tcp_read_resume(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_socket_tcp_write_resume(ngx_http_request_t *r); @@ -155,7 +172,8 @@ enum { enum { SOCKET_OP_CONNECT, SOCKET_OP_READ, - SOCKET_OP_WRITE + SOCKET_OP_WRITE, + SOCKET_OP_RESUME_CONN }; @@ -431,31 +449,417 @@ ngx_http_lua_socket_tcp(lua_State *L) } +static void +ngx_http_lua_socket_tcp_create_socket_pool(lua_State *L, ngx_http_request_t *r, + ngx_str_t key, ngx_int_t pool_size, ngx_int_t backlog, + ngx_http_lua_socket_pool_t **spool) +{ + u_char *p; + size_t size, key_len; + ngx_int_t i; + ngx_http_lua_socket_pool_t *sp; + ngx_http_lua_socket_pool_item_t *items; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket connection pool size: %i, backlog: %i", + pool_size, backlog); + + key_len = ngx_align(key.len + 1, sizeof(void *)); + + size = sizeof(ngx_http_lua_socket_pool_t) - 1 + key_len + + sizeof(ngx_http_lua_socket_pool_item_t) * pool_size; + + /* before calling this function, the Lua stack is: + * -1 key + * -2 pools + */ + sp = lua_newuserdata(L, size); + if (sp == NULL) { + luaL_error(L, "no memory"); + return; + } + + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + pool_udata_metatable_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + lua_setmetatable(L, -2); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket keepalive create connection pool for key" + " \"%V\"", &key); + + /* a new socket pool with metatable is push to the stack, so now we have: + * -1 sp + * -2 key + * -3 pools + * + * it is time to set pools[key] to sp. + */ + lua_rawset(L, -3); + + /* clean up the stack for consistency's sake */ + lua_pop(L, 1); + + sp->backlog = backlog; + sp->size = pool_size; + sp->connections = 0; + sp->lua_vm = ngx_http_lua_get_lua_vm(r, NULL); + + ngx_queue_init(&sp->cache_connect_op); + ngx_queue_init(&sp->wait_connect_op); + ngx_queue_init(&sp->cache); + ngx_queue_init(&sp->free); + + p = ngx_copy(sp->key, key.data, key.len); + *p++ = '\0'; + + items = (ngx_http_lua_socket_pool_item_t *) (sp->key + key_len); + + dd("items: %p", items); + + ngx_http_lua_assert((void *) items == ngx_align_ptr(items, sizeof(void *))); + + for (i = 0; i < pool_size; i++) { + ngx_queue_insert_head(&sp->free, &items[i].queue); + items[i].socket_pool = sp; + } + + *spool = sp; +} + + +static int +ngx_http_lua_socket_tcp_connect_helper(lua_State *L, + ngx_http_lua_socket_tcp_upstream_t *u, ngx_http_request_t *r, + ngx_http_lua_ctx_t *ctx, u_char *host_ref, size_t host_len, in_port_t port, + unsigned resuming) +{ + int n; + int host_size; + int saved_top; + ngx_int_t rc; + ngx_str_t host; + ngx_str_t *conn_op_host; + ngx_url_t url; + ngx_queue_t *q; + ngx_resolver_ctx_t *rctx, temp; + ngx_http_lua_co_ctx_t *coctx; + ngx_http_core_loc_conf_t *clcf; + ngx_http_lua_socket_pool_t *spool; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; + + spool = u->socket_pool; + if (spool != NULL) { + rc = ngx_http_lua_get_keepalive_peer(r, u); + + if (rc == NGX_OK) { + lua_pushinteger(L, 1); + return 1; + } + + /* rc == NGX_DECLINED */ + + spool->connections++; + + /* check if backlog is enabled and + * don't queue resuming connection operation */ + if (spool->backlog >= 0 && !resuming) { + + dd("lua tcp socket %s connections %ld", + spool->key, spool->connections); + + if (spool->connections > spool->size + spool->backlog) { + spool->connections--; + lua_pushnil(L); + lua_pushliteral(L, "too many waiting connect operations"); + return 2; + } + + if (spool->connections > spool->size) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, u->peer.log, 0, + "lua tcp socket queue connect operation for " + "connection pool \"%s\", connections: %i", + spool->key, spool->connections); + + host_size = sizeof(u_char) * + (ngx_max(host_len, NGX_INET_ADDRSTRLEN) + 1); + + if (!ngx_queue_empty(&spool->cache_connect_op)) { + q = ngx_queue_last(&spool->cache_connect_op); + ngx_queue_remove(q); + conn_op_ctx = ngx_queue_data( + q, ngx_http_lua_socket_tcp_conn_op_ctx_t, queue); + + conn_op_host = &conn_op_ctx->host; + if (host_len > conn_op_host->len + && host_len > NGX_INET_ADDRSTRLEN) + { + ngx_free(conn_op_host->data); + conn_op_host->data = ngx_alloc(host_size, + ngx_cycle->log); + if (conn_op_host->data == NULL) { + ngx_free(conn_op_ctx); + goto no_memory_and_not_resuming; + } + } + + } else { + conn_op_ctx = ngx_alloc( + sizeof(ngx_http_lua_socket_tcp_conn_op_ctx_t), + ngx_cycle->log); + if (conn_op_ctx == NULL) { + goto no_memory_and_not_resuming; + } + + conn_op_host = &conn_op_ctx->host; + conn_op_host->data = ngx_alloc(host_size, ngx_cycle->log); + if (conn_op_host->data == NULL) { + ngx_free(conn_op_ctx); + goto no_memory_and_not_resuming; + } + } + + conn_op_ctx->cleanup = NULL; + + ngx_memcpy(conn_op_host->data, host_ref, host_len); + conn_op_host->data[host_len] = '\0'; + conn_op_host->len = host_len; + + conn_op_ctx->port = port; + + u->write_co_ctx = ctx->cur_co_ctx; + + conn_op_ctx->u = u; + ngx_memzero(&conn_op_ctx->event, sizeof(ngx_event_t)); + conn_op_ctx->event.handler = + ngx_http_lua_socket_tcp_conn_op_timeout_handler; + conn_op_ctx->event.data = conn_op_ctx; + conn_op_ctx->event.log = ngx_cycle->log; + + ngx_add_timer(&conn_op_ctx->event, u->connect_timeout); + + ngx_queue_insert_tail(&spool->wait_connect_op, + &conn_op_ctx->queue); + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua tcp socket queued connect operation for " + "%d(ms), u: %p, ctx: %p", + u->connect_timeout, conn_op_ctx->u, conn_op_ctx); + + return lua_yield(L, 0); + } + } + + } /* end spool != NULL */ + + host.data = ngx_palloc(r->pool, host_len + 1); + if (host.data == NULL) { + return luaL_error(L, "no memory"); + } + + host.len = host_len; + + ngx_memcpy(host.data, host_ref, host_len); + host.data[host_len] = '\0'; + + ngx_memzero(&url, sizeof(ngx_url_t)); + url.url = host; + url.default_port = port; + url.no_resolve = 1; + + coctx = ctx->cur_co_ctx; + + if (ngx_parse_url(r->pool, &url) != NGX_OK) { + lua_pushnil(L); + + if (url.err) { + lua_pushfstring(L, "failed to parse host name \"%s\": %s", + url.url.data, url.err); + + } else { + lua_pushfstring(L, "failed to parse host name \"%s\"", + url.url.data); + } + + goto failed; + } + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket connect timeout: %M", u->connect_timeout); + + u->resolved = ngx_pcalloc(r->pool, sizeof(ngx_http_upstream_resolved_t)); + if (u->resolved == NULL) { + if (resuming) { + lua_pushnil(L); + lua_pushliteral(L, "no memory"); + goto failed; + } + + goto no_memory_and_not_resuming; + } + + if (url.addrs && url.addrs[0].sockaddr) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket network address given directly"); + + u->resolved->sockaddr = url.addrs[0].sockaddr; + u->resolved->socklen = url.addrs[0].socklen; + u->resolved->naddrs = 1; + u->resolved->host = url.addrs[0].name; + + } else { + u->resolved->host = host; + u->resolved->port = url.default_port; + } + + if (u->resolved->sockaddr) { + rc = ngx_http_lua_socket_resolve_retval_handler(r, u, L); + if (rc == NGX_AGAIN && !resuming) { + return lua_yield(L, 0); + } + + if (rc > 1) { + goto failed; + } + + return rc; + } + + clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); + + temp.name = host; + rctx = ngx_resolve_start(clcf->resolver, &temp); + if (rctx == NULL) { + u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; + lua_pushnil(L); + lua_pushliteral(L, "failed to start the resolver"); + goto failed; + } + + if (rctx == NGX_NO_RESOLVER) { + u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; + lua_pushnil(L); + lua_pushfstring(L, "no resolver defined to resolve \"%s\"", host.data); + goto failed; + } + + rctx->name = host; +#if !defined(nginx_version) || nginx_version < 1005008 + rctx->type = NGX_RESOLVE_A; +#endif + rctx->handler = ngx_http_lua_socket_resolve_handler; + rctx->data = u; + rctx->timeout = clcf->resolver_timeout; + + u->resolved->ctx = rctx; + u->write_co_ctx = ctx->cur_co_ctx; + + ngx_http_lua_cleanup_pending_operation(coctx); + coctx->cleanup = ngx_http_lua_tcp_resolve_cleanup; + coctx->data = u; + + saved_top = lua_gettop(L); + + if (ngx_resolve_name(rctx) != NGX_OK) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket fail to run resolver immediately"); + + u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; + + coctx->cleanup = NULL; + coctx->data = NULL; + + u->resolved->ctx = NULL; + lua_pushnil(L); + lua_pushfstring(L, "%s could not be resolved", host.data); + goto failed; + } + + if (u->conn_waiting) { + dd("resolved and already connecting"); + + if (resuming) { + return NGX_AGAIN; + } + + return lua_yield(L, 0); + } + + n = lua_gettop(L) - saved_top; + if (n) { + dd("errors occurred during resolving or connecting" + "or already connected"); + + if (n > 1) { + goto failed; + } + + return n; + } + + /* still resolving */ + + u->conn_waiting = 1; + u->write_prepare_retvals = ngx_http_lua_socket_resolve_retval_handler; + + dd("setting data to %p", u); + + if (ctx->entered_content_phase) { + r->write_event_handler = ngx_http_lua_content_wev_handler; + + } else { + r->write_event_handler = ngx_http_core_run_phases; + } + + if (resuming) { + return NGX_AGAIN; + } + + return lua_yield(L, 0); + +failed: + + if (spool != NULL) { + spool->connections--; + ngx_http_lua_socket_tcp_resume_conn_op(spool); + } + + return 2; + +no_memory_and_not_resuming: + + if (spool != NULL) { + spool->connections--; + ngx_http_lua_socket_tcp_resume_conn_op(spool); + } + + return luaL_error(L, "no memory"); +} + + static int ngx_http_lua_socket_tcp_connect(lua_State *L) { ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; - ngx_str_t host; int port; - ngx_resolver_ctx_t *rctx, temp; - ngx_http_core_loc_conf_t *clcf; - int saved_top; int n; u_char *p; size_t len; - ngx_url_t url; - ngx_int_t rc; ngx_http_lua_loc_conf_t *llcf; ngx_peer_connection_t *pc; int connect_timeout, send_timeout, read_timeout; unsigned custom_pool; int key_index; + ngx_int_t backlog; + ngx_int_t pool_size; + ngx_str_t key; const char *msg; - ngx_http_lua_co_ctx_t *coctx; ngx_http_lua_socket_tcp_upstream_t *u; + ngx_http_lua_socket_pool_t *spool; + n = lua_gettop(L); if (n != 2 && n != 3 && n != 4) { return luaL_error(L, "ngx.socket connect: expecting 2, 3, or 4 " @@ -483,13 +887,54 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) p = (u_char *) luaL_checklstring(L, 2, &len); + backlog = -1; key_index = 2; + pool_size = 0; custom_pool = 0; + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (lua_type(L, n) == LUA_TTABLE) { /* found the last optional option table */ + lua_getfield(L, n, "pool_size"); + + if (lua_isnumber(L, -1)) { + pool_size = (ngx_int_t) lua_tointeger(L, -1); + + if (pool_size <= 0) { + msg = lua_pushfstring(L, "bad \"pool_size\" option value: %i", + pool_size); + return luaL_argerror(L, n, msg); + } + + } else if (!lua_isnil(L, -1)) { + msg = lua_pushfstring(L, "bad \"pool_size\" option type: %s", + lua_typename(L, lua_type(L, -1))); + return luaL_argerror(L, n, msg); + } + + lua_pop(L, 1); + + lua_getfield(L, n, "backlog"); + + if (lua_isnumber(L, -1)) { + backlog = (ngx_int_t) lua_tointeger(L, -1); + + if (backlog < 0) { + msg = lua_pushfstring(L, "bad \"backlog\" option value: %i", + backlog); + return luaL_argerror(L, n, msg); + } + + /* use default value for pool size if only backlog specified */ + if (pool_size == 0) { + pool_size = llcf->pool_size; + } + } + + lua_pop(L, 1); + lua_getfield(L, n, "pool"); switch (lua_type(L, -1)) { @@ -600,12 +1045,8 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) ngx_memzero(u, sizeof(ngx_http_lua_socket_tcp_upstream_t)); - coctx = ctx->cur_co_ctx; - u->request = r; /* set the controlling request */ - llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); - u->conf = llcf; pc = &u->peer; @@ -646,163 +1087,28 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) u->read_timeout = u->conf->read_timeout; } - rc = ngx_http_lua_get_keepalive_peer(r, L, key_index, u); - - if (rc == NGX_OK) { - lua_pushinteger(L, 1); - return 1; - } - - if (rc == NGX_ERROR) { - lua_pushnil(L); - lua_pushliteral(L, "error in get keepalive peer"); - return 2; - } - - /* rc == NGX_DECLINED */ - - /* TODO: we should avoid this in-pool allocation */ - - host.data = ngx_palloc(r->pool, len + 1); - if (host.data == NULL) { - return luaL_error(L, "no memory"); - } - - host.len = len; - - ngx_memcpy(host.data, p, len); - host.data[len] = '\0'; - - ngx_memzero(&url, sizeof(ngx_url_t)); - - url.url.len = host.len; - url.url.data = host.data; - url.default_port = (in_port_t) port; - url.no_resolve = 1; - - if (ngx_parse_url(r->pool, &url) != NGX_OK) { - lua_pushnil(L); - - if (url.err) { - lua_pushfstring(L, "failed to parse host name \"%s\": %s", - host.data, url.err); - - } else { - lua_pushfstring(L, "failed to parse host name \"%s\"", host.data); - } - - return 2; - } - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket connect timeout: %M", u->connect_timeout); - - u->resolved = ngx_pcalloc(r->pool, sizeof(ngx_http_upstream_resolved_t)); - if (u->resolved == NULL) { - return luaL_error(L, "no memory"); - } - - if (url.addrs && url.addrs[0].sockaddr) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket network address given directly"); - - u->resolved->sockaddr = url.addrs[0].sockaddr; - u->resolved->socklen = url.addrs[0].socklen; - u->resolved->naddrs = 1; - u->resolved->host = url.addrs[0].name; - - } else { - u->resolved->host = host; - u->resolved->port = (in_port_t) port; - } - - if (u->resolved->sockaddr) { - rc = ngx_http_lua_socket_resolve_retval_handler(r, u, L); - if (rc == NGX_AGAIN) { - return lua_yield(L, 0); - } - - return rc; - } - - clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); - - temp.name = host; - rctx = ngx_resolve_start(clcf->resolver, &temp); - if (rctx == NULL) { - u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; - lua_pushnil(L); - lua_pushliteral(L, "failed to start the resolver"); - return 2; - } - - if (rctx == NGX_NO_RESOLVER) { - u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; - lua_pushnil(L); - lua_pushfstring(L, "no resolver defined to resolve \"%s\"", host.data); - return 2; - } - - rctx->name = host; -#if !defined(nginx_version) || nginx_version < 1005008 - rctx->type = NGX_RESOLVE_A; -#endif - rctx->handler = ngx_http_lua_socket_resolve_handler; - rctx->data = u; - rctx->timeout = clcf->resolver_timeout; - - u->resolved->ctx = rctx; - u->write_co_ctx = ctx->cur_co_ctx; - - ngx_http_lua_cleanup_pending_operation(coctx); - coctx->cleanup = ngx_http_lua_tcp_resolve_cleanup; - coctx->data = u; - - saved_top = lua_gettop(L); - - if (ngx_resolve_name(rctx) != NGX_OK) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket fail to run resolver immediately"); - - u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_RESOLVER; - - coctx->cleanup = NULL; - coctx->data = NULL; - - u->resolved->ctx = NULL; - lua_pushnil(L); - lua_pushfstring(L, "%s could not be resolved", host.data); - - return 2; - } - - if (u->conn_waiting) { - dd("resolved and already connecting"); - return lua_yield(L, 0); - } - - n = lua_gettop(L) - saved_top; - if (n) { - dd("errors occurred during resolving or connecting" - "or already connected"); - return n; - } - - /* still resolving */ + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask(socket_pool_key)); + lua_rawget(L, LUA_REGISTRYINDEX); /* table */ + lua_pushvalue(L, key_index); /* key */ - u->conn_waiting = 1; - u->write_prepare_retvals = ngx_http_lua_socket_resolve_retval_handler; + lua_rawget(L, -2); + spool = lua_touserdata(L, -1); + lua_pop(L, 1); - dd("setting data to %p", u); + if (spool != NULL) { + u->socket_pool = spool; - if (ctx->entered_content_phase) { - r->write_event_handler = ngx_http_lua_content_wev_handler; + } else if (pool_size > 0) { + lua_pushvalue(L, key_index); + key.data = (u_char *) lua_tolstring(L, -1, &key.len); - } else { - r->write_event_handler = ngx_http_core_run_phases; + ngx_http_lua_socket_tcp_create_socket_pool(L, r, key, pool_size, + backlog, &spool); + u->socket_pool = spool; } - return lua_yield(L, 0); + return ngx_http_lua_socket_tcp_connect_helper(L, u, r, ctx, p, + len, port, 0); } @@ -3571,6 +3877,257 @@ ngx_http_lua_socket_tcp_finalize_write_part(ngx_http_request_t *r, } +static void +ngx_http_lua_socket_tcp_conn_op_timeout_handler(ngx_event_t *ev) +{ + ngx_http_lua_socket_tcp_upstream_t *u; + ngx_http_lua_ctx_t *ctx; + ngx_connection_t *c; + ngx_http_request_t *r; + ngx_http_lua_co_ctx_t *coctx; + ngx_http_lua_loc_conf_t *llcf; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; + + conn_op_ctx = ev->data; + ngx_queue_remove(&conn_op_ctx->queue); + + u = conn_op_ctx->u; + r = u->request; + + coctx = u->write_co_ctx; + coctx->cleanup = NULL; + /* note that we store conn_op_ctx in coctx->data instead of u */ + coctx->data = conn_op_ctx; + u->write_co_ctx = NULL; + + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + + if (llcf->log_socket_errors) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "lua tcp socket queued connect timed out," + " when trying to connect to %V:%ud", + &conn_op_ctx->host, conn_op_ctx->port); + } + + ngx_queue_insert_head(&u->socket_pool->cache_connect_op, + &conn_op_ctx->queue); + u->socket_pool->connections--; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return; + } + + ctx->cur_co_ctx = coctx; + + ngx_http_lua_assert(coctx && (!ngx_http_lua_is_thread(ctx) + || coctx->co_ref >= 0)); + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket waking up the current request"); + + u->write_prepare_retvals = + ngx_http_lua_socket_tcp_conn_op_timeout_retval_handler; + + c = r->connection; + + if (ctx->entered_content_phase) { + (void) ngx_http_lua_socket_tcp_conn_op_resume(r); + + } else { + ctx->resume_handler = ngx_http_lua_socket_tcp_conn_op_resume; + ngx_http_core_run_phases(r); + } + + ngx_http_run_posted_requests(c); +} + + +static int +ngx_http_lua_socket_tcp_conn_op_timeout_retval_handler(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) +{ + lua_pushnil(L); + lua_pushliteral(L, "timeout"); + return 2; +} + + +static void +ngx_http_lua_socket_tcp_resume_conn_op(ngx_http_lua_socket_pool_t *spool) +{ + ngx_queue_t *q; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; + +#if (NGX_DEBUG) + ngx_http_lua_assert(spool->connections >= 0); + +#else + if (spool->connections < 0) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "lua tcp socket connections count mismatched for " + "connection pool \"%s\", connections: %i, size: %i", + spool->key, spool->connections, spool->size); + spool->connections = 0; + } +#endif + + /* we manually destroy wait_connect_op before triggering connect + * operation resumption, so that there is no resumption happens when Nginx + * is exiting. + */ + if (ngx_queue_empty(&spool->wait_connect_op)) { +#if (NGX_DEBUG) + ngx_http_lua_assert(!(spool->backlog >= 0 + && spool->connections > spool->size)); + +#else + if (spool->backlog >= 0 && spool->connections > spool->size) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "lua tcp socket connections count mismatched for " + "connection pool \"%s\", connections: %i, size: %i", + spool->key, spool->connections, spool->size); + spool->connections = spool->size; + } +#endif + + return; + } + + q = ngx_queue_head(&spool->wait_connect_op); + ngx_queue_remove(q); + conn_op_ctx = ngx_queue_data(q, ngx_http_lua_socket_tcp_conn_op_ctx_t, + queue); + ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua tcp socket post connect operation resumption " + "u: %p, ctx: %p for connection pool \"%s\", " + "connections: %i", + conn_op_ctx->u, conn_op_ctx, spool->key, spool->connections); + + if (conn_op_ctx->event.timer_set) { + ngx_del_timer(&conn_op_ctx->event); + } + + conn_op_ctx->event.handler = + ngx_http_lua_socket_tcp_conn_op_resume_handler; + + ngx_post_event((&conn_op_ctx->event), &ngx_posted_events); +} + + +static void +ngx_http_lua_socket_tcp_conn_op_ctx_cleanup(void *data) +{ + ngx_http_lua_socket_tcp_upstream_t *u; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx = data; + + u = conn_op_ctx->u; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "cleanup lua tcp socket conn_op_ctx: \"%V\"", + &u->request->uri); + + ngx_queue_insert_head(&u->socket_pool->cache_connect_op, + &conn_op_ctx->queue); +} + + +static void +ngx_http_lua_socket_tcp_conn_op_resume_handler(ngx_event_t *ev) +{ + ngx_http_lua_socket_tcp_upstream_t *u; + ngx_http_lua_ctx_t *ctx; + ngx_connection_t *c; + ngx_http_request_t *r; + ngx_http_cleanup_t *cln; + ngx_http_lua_co_ctx_t *coctx; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; + + conn_op_ctx = ev->data; + u = conn_op_ctx->u; + r = u->request; + + coctx = u->write_co_ctx; + coctx->cleanup = NULL; + /* note that we store conn_op_ctx in coctx->data instead of u */ + coctx->data = conn_op_ctx; + u->write_co_ctx = NULL; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + ngx_queue_insert_head(&u->socket_pool->cache_connect_op, + &conn_op_ctx->queue); + return; + } + + ctx->cur_co_ctx = coctx; + + ngx_http_lua_assert(coctx && (!ngx_http_lua_is_thread(ctx) + || coctx->co_ref >= 0)); + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket waking up the current request"); + + u->write_prepare_retvals = + ngx_http_lua_socket_tcp_conn_op_resume_retval_handler; + + c = r->connection; + + if (ctx->entered_content_phase) { + (void) ngx_http_lua_socket_tcp_conn_op_resume(r); + + } else { + cln = ngx_http_lua_cleanup_add(r, 0); + if (cln != NULL) { + cln->handler = ngx_http_lua_socket_tcp_conn_op_ctx_cleanup; + cln->data = conn_op_ctx; + conn_op_ctx->cleanup = &cln->handler; + } + + ctx->resume_handler = ngx_http_lua_socket_tcp_conn_op_resume; + ngx_http_core_run_phases(r); + } + + ngx_http_run_posted_requests(c); +} + + +static int +ngx_http_lua_socket_tcp_conn_op_resume_retval_handler(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) +{ + int nret; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_co_ctx_t *coctx; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return NGX_ERROR; + } + + coctx = ctx->cur_co_ctx; + dd("coctx: %p", coctx); + conn_op_ctx = coctx->data; + if (conn_op_ctx->cleanup != NULL) { + *conn_op_ctx->cleanup = NULL; + ngx_http_lua_cleanup_free(r, conn_op_ctx->cleanup); + conn_op_ctx->cleanup = NULL; + } + + /* decrease pending connect operation counter */ + u->socket_pool->connections--; + + nret = ngx_http_lua_socket_tcp_connect_helper(L, u, r, ctx, + conn_op_ctx->host.data, + conn_op_ctx->host.len, + conn_op_ctx->port, 1); + ngx_queue_insert_head(&u->socket_pool->cache_connect_op, + &conn_op_ctx->queue); + + return nret; +} + + static void ngx_http_lua_socket_tcp_finalize(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u) @@ -3621,21 +4178,21 @@ ngx_http_lua_socket_tcp_finalize(ngx_http_request_t *r, ngx_http_lua_socket_tcp_close_connection(c); u->peer.connection = NULL; - - if (!u->reused) { - return; - } + u->conn_closed = 1; spool = u->socket_pool; if (spool == NULL) { return; } - spool->active_connections--; + spool->connections--; - if (spool->active_connections == 0) { + if (spool->connections == 0) { ngx_http_lua_socket_free_pool(r->connection->log, spool); + return; } + + ngx_http_lua_socket_tcp_resume_conn_op(spool); } } @@ -4608,20 +5165,18 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) ngx_http_lua_socket_tcp_upstream_t *u; ngx_connection_t *c; ngx_http_lua_socket_pool_t *spool; - size_t size, key_len; ngx_str_t key; - ngx_uint_t i; ngx_queue_t *q; ngx_peer_connection_t *pc; - u_char *p; ngx_http_request_t *r; ngx_msec_t timeout; - ngx_uint_t pool_size; + ngx_int_t pool_size; int n; ngx_int_t rc; ngx_buf_t *b; + const char *msg; - ngx_http_lua_socket_pool_item_t *items, *item; + ngx_http_lua_socket_pool_item_t *item; n = lua_gettop(L); @@ -4632,18 +5187,6 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) luaL_checktype(L, 1, LUA_TTABLE); - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - socket_pool_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - - lua_rawgeti(L, 1, SOCKET_KEY_INDEX); - key.data = (u_char *) lua_tolstring(L, -1, &key.len); - if (key.data == NULL) { - lua_pushnil(L); - lua_pushliteral(L, "key not found"); - return 2; - } - lua_rawgeti(L, 1, SOCKET_CTX_INDEX); u = lua_touserdata(L, -1); lua_pop(L, 1); @@ -4654,7 +5197,7 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) return 2; } - /* stack: obj cache key */ + /* stack: obj timeout? size? */ pc = &u->peer; c = pc->connection; @@ -4711,12 +5254,27 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) "lua tcp socket set keepalive while process exiting, " "closing connection %p", c); - goto finalize; + ngx_http_lua_socket_tcp_finalize(r, u); + lua_pushinteger(L, 1); + return 1; } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, "lua tcp socket set keepalive: saving connection %p", c); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask(socket_pool_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + + /* stack: obj timeout? size? pools */ + + lua_rawgeti(L, 1, SOCKET_KEY_INDEX); + key.data = (u_char *) lua_tolstring(L, -1, &key.len); + if (key.data == NULL) { + lua_pushnil(L); + lua_pushliteral(L, "key not found"); + return 2; + } + dd("saving connection to key %s", lua_tostring(L, -1)); lua_pushvalue(L, -1); @@ -4724,7 +5282,7 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) spool = lua_touserdata(L, -1); lua_pop(L, 1); - /* stack: obj timeout? size? cache key */ + /* stack: obj timeout? size? pools cache_key */ llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); @@ -4738,68 +5296,31 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) pool_size = llcf->pool_size; } - if (pool_size == 0) { - lua_pushnil(L); - lua_pushliteral(L, "zero pool size"); - return 2; - } - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket connection pool size: %ui", pool_size); - - key_len = ngx_align(key.len + 1, sizeof(void *)); - - size = sizeof(ngx_http_lua_socket_pool_t) + key_len - 1 - + sizeof(ngx_http_lua_socket_pool_item_t) - * pool_size; - - spool = lua_newuserdata(L, size); - if (spool == NULL) { - return luaL_error(L, "no memory"); + if (pool_size <= 0) { + msg = lua_pushfstring(L, "bad \"pool_size\" option value: %i", + pool_size); + return luaL_argerror(L, n, msg); } - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - pool_udata_metatable_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_setmetatable(L, -2); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "lua tcp socket keepalive create connection pool for key" - " \"%s\"", lua_tostring(L, -2)); - - lua_rawset(L, -3); - - spool->active_connections = 0; - spool->lua_vm = ngx_http_lua_get_lua_vm(r, NULL); - - ngx_queue_init(&spool->cache); - ngx_queue_init(&spool->free); - - p = ngx_copy(spool->key, key.data, key.len); - *p++ = '\0'; - - items = (ngx_http_lua_socket_pool_item_t *) (spool->key + key_len); - - dd("items: %p", items); - - ngx_http_lua_assert((void *) items == ngx_align_ptr(items, - sizeof(void *))); - - for (i = 0; i < pool_size; i++) { - ngx_queue_insert_head(&spool->free, &items[i].queue); - items[i].socket_pool = spool; - } + ngx_http_lua_socket_tcp_create_socket_pool(L, r, key, pool_size, -1, + &spool); + /* we should always increase connections after getting connected, + * and decrease connections after getting closed. + * however, we don't create connection pool in previous connect method. + * so we increase connections here for backward compatibility. + */ + spool->connections++; } if (ngx_queue_empty(&spool->free)) { q = ngx_queue_last(&spool->cache); ngx_queue_remove(q); - spool->active_connections--; item = ngx_queue_data(q, ngx_http_lua_socket_pool_item_t, queue); ngx_http_lua_socket_tcp_close_connection(item->connection); + spool->connections--; } else { q = ngx_queue_head(&spool->free); @@ -4811,10 +5332,6 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) item->connection = c; ngx_queue_insert_head(&spool->cache, q); - if (!u->reused) { - spool->active_connections++; - } - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, pc->log, 0, "lua tcp socket clear current socket connection"); @@ -4879,55 +5396,37 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) } } -finalize: - #if 1 ngx_http_lua_socket_tcp_finalize(r, u); #endif + /* since we set u->peer->connection to NULL previously, the connect + * operation won't be resumed in the ngx_http_lua_socket_tcp_finalize. + * Therefore we need to resume it here. + */ + ngx_http_lua_socket_tcp_resume_conn_op(spool); + lua_pushinteger(L, 1); return 1; } static ngx_int_t -ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, lua_State *L, - int key_index, ngx_http_lua_socket_tcp_upstream_t *u) +ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u) { ngx_http_lua_socket_pool_item_t *item; ngx_http_lua_socket_pool_t *spool; ngx_http_cleanup_t *cln; ngx_queue_t *q; - int top; ngx_peer_connection_t *pc; ngx_connection_t *c; - top = lua_gettop(L); - - if (key_index < 0) { - key_index = top + key_index + 1; - } - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua tcp socket pool get keepalive peer"); pc = &u->peer; - - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - socket_pool_key)); - lua_rawget(L, LUA_REGISTRYINDEX); /* table */ - lua_pushvalue(L, key_index); /* key */ - lua_rawget(L, -2); - - spool = lua_touserdata(L, -1); - if (spool == NULL) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "lua tcp socket keepalive connection pool not found"); - lua_settop(L, top); - return NGX_DECLINED; - } - - u->socket_pool = spool; + spool = u->socket_pool; if (!ngx_queue_empty(&spool->cache)) { q = ngx_queue_head(&spool->cache); @@ -4972,7 +5471,6 @@ ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, lua_State *L, cln = ngx_http_lua_cleanup_add(r, 0); if (cln == NULL) { u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_ERROR; - lua_settop(L, top); return NGX_ERROR; } @@ -4981,16 +5479,12 @@ ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, lua_State *L, u->cleanup = &cln->handler; } - lua_settop(L, top); - return NGX_OK; } ngx_log_debug0(NGX_LOG_DEBUG_HTTP, pc->log, 0, "lua tcp socket keepalive: connection pool empty"); - lua_settop(L, top); - return NGX_DECLINED; } @@ -5062,13 +5556,15 @@ ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev) ngx_queue_remove(&item->queue); ngx_queue_insert_head(&spool->free, &item->queue); - spool->active_connections--; + spool->connections--; - dd("keepalive: active connections: %u", - (unsigned) spool->active_connections); + dd("keepalive: connections: %u", (unsigned) spool->connections); - if (spool->active_connections == 0) { + if (spool->connections == 0) { ngx_http_lua_socket_free_pool(ev->log, spool); + + } else { + ngx_http_lua_socket_tcp_resume_conn_op(spool); } return NGX_DECLINED; @@ -5086,8 +5582,7 @@ ngx_http_lua_socket_free_pool(ngx_log_t *log, ngx_http_lua_socket_pool_t *spool) L = spool->lua_vm; - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - socket_pool_key)); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask(socket_pool_key)); lua_rawget(L, LUA_REGISTRYINDEX); lua_pushstring(L, (char *) spool->key); lua_pushnil(L); @@ -5099,9 +5594,10 @@ ngx_http_lua_socket_free_pool(ngx_log_t *log, ngx_http_lua_socket_pool_t *spool) static void ngx_http_lua_socket_shutdown_pool_helper(ngx_http_lua_socket_pool_t *spool) { - ngx_queue_t *q; - ngx_connection_t *c; - ngx_http_lua_socket_pool_item_t *item; + ngx_queue_t *q; + ngx_connection_t *c; + ngx_http_lua_socket_pool_item_t *item; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; while (!ngx_queue_empty(&spool->cache)) { q = ngx_queue_head(&spool->cache); @@ -5115,7 +5611,29 @@ ngx_http_lua_socket_shutdown_pool_helper(ngx_http_lua_socket_pool_t *spool) ngx_queue_insert_head(&spool->free, q); } - spool->active_connections = 0; + while (!ngx_queue_empty(&spool->cache_connect_op)) { + q = ngx_queue_head(&spool->cache_connect_op); + ngx_queue_remove(q); + conn_op_ctx = ngx_queue_data(q, ngx_http_lua_socket_tcp_conn_op_ctx_t, + queue); + ngx_http_lua_socket_tcp_free_conn_op_ctx(conn_op_ctx); + } + + while (!ngx_queue_empty(&spool->wait_connect_op)) { + q = ngx_queue_head(&spool->wait_connect_op); + ngx_queue_remove(q); + conn_op_ctx = ngx_queue_data(q, ngx_http_lua_socket_tcp_conn_op_ctx_t, + queue); + + if (conn_op_ctx->event.timer_set) { + ngx_del_timer(&conn_op_ctx->event); + } + + ngx_http_lua_socket_tcp_free_conn_op_ctx(conn_op_ctx); + } + + /* spool->connections will be decreased down to zero in + * ngx_http_lua_socket_tcp_finalize */ } @@ -5369,6 +5887,13 @@ static ngx_int_t ngx_http_lua_socket_insert_buffer(ngx_http_request_t *r, } +static ngx_int_t +ngx_http_lua_socket_tcp_conn_op_resume(ngx_http_request_t *r) +{ + return ngx_http_lua_socket_tcp_resume_helper(r, SOCKET_OP_RESUME_CONN); +} + + static ngx_int_t ngx_http_lua_socket_tcp_conn_resume(ngx_http_request_t *r) { @@ -5393,13 +5918,14 @@ ngx_http_lua_socket_tcp_write_resume(ngx_http_request_t *r) static ngx_int_t ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) { - int nret; - lua_State *vm; - ngx_int_t rc; - ngx_uint_t nreqs; - ngx_connection_t *c; - ngx_http_lua_ctx_t *ctx; - ngx_http_lua_co_ctx_t *coctx; + int nret; + lua_State *vm; + ngx_int_t rc; + ngx_uint_t nreqs; + ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_co_ctx_t *coctx; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; ngx_http_lua_socket_tcp_retval_handler prepare_retvals; @@ -5419,15 +5945,22 @@ ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) dd("coctx: %p", coctx); - u = coctx->data; - switch (socket_op) { + + case SOCKET_OP_RESUME_CONN: + conn_op_ctx = coctx->data; + u = conn_op_ctx->u; + prepare_retvals = u->write_prepare_retvals; + break; + case SOCKET_OP_CONNECT: case SOCKET_OP_WRITE: + u = coctx->data; prepare_retvals = u->write_prepare_retvals; break; case SOCKET_OP_READ: + u = coctx->data; prepare_retvals = u->read_prepare_retvals; break; @@ -5441,6 +5974,15 @@ ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) "u:%p", prepare_retvals, u); nret = prepare_retvals(r, u, ctx->cur_co_ctx->co); + if (socket_op == SOCKET_OP_CONNECT + && nret > 1 + && !u->conn_closed + && u->socket_pool != NULL) + { + u->socket_pool->connections--; + ngx_http_lua_socket_tcp_resume_conn_op(u->socket_pool); + } + if (nret == NGX_AGAIN) { return NGX_DONE; } @@ -5487,6 +6029,11 @@ ngx_http_lua_tcp_resolve_cleanup(void *data) return; } + if (u->socket_pool != NULL) { + u->socket_pool->connections--; + ngx_http_lua_socket_tcp_resume_conn_op(u->socket_pool); + } + rctx = u->resolved->ctx; if (rctx == NULL) { return; diff --git a/src/ngx_http_lua_socket_tcp.h b/src/ngx_http_lua_socket_tcp.h index dbdee41c6e..091e4378fe 100644 --- a/src/ngx_http_lua_socket_tcp.h +++ b/src/ngx_http_lua_socket_tcp.h @@ -35,17 +35,39 @@ typedef void (*ngx_http_lua_socket_tcp_upstream_handler_pt) (ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); +typedef struct { + ngx_event_t event; + ngx_queue_t queue; + ngx_str_t host; + ngx_http_cleanup_pt *cleanup; + ngx_http_lua_socket_tcp_upstream_t *u; + in_port_t port; +} ngx_http_lua_socket_tcp_conn_op_ctx_t; + + +#define ngx_http_lua_socket_tcp_free_conn_op_ctx(conn_op_ctx) \ + ngx_free(conn_op_ctx->host.data); \ + ngx_free(conn_op_ctx) + + typedef struct { lua_State *lua_vm; - /* active connections == out-of-pool reused connections - * + in-pool connections */ - ngx_uint_t active_connections; + ngx_int_t size; + ngx_queue_t cache_connect_op; + ngx_queue_t wait_connect_op; + + /* connections == active connections + pending connect operations, + * while active connections == out-of-pool reused connections + * + in-pool connections */ + ngx_int_t connections; /* queues of ngx_http_lua_socket_pool_item_t: */ ngx_queue_t cache; ngx_queue_t free; + ngx_int_t backlog; + u_char key[1]; } ngx_http_lua_socket_pool_t; @@ -104,6 +126,7 @@ struct ngx_http_lua_socket_tcp_upstream_s { unsigned raw_downstream:1; unsigned read_closed:1; unsigned write_closed:1; + unsigned conn_closed:1; #if (NGX_HTTP_SSL) unsigned ssl_verify:1; unsigned ssl_session_reuse:1; diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index f052e9afce..6e42fac6e6 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -4,13 +4,14 @@ use Test::Nginx::Socket::Lua; #repeat_each(2); -plan tests => repeat_each() * (blocks() * 5 + 9); +plan tests => repeat_each() * (blocks() * 4 + 33); our $HtmlDir = html_dir; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_HTML_DIR} = $HtmlDir; $ENV{TEST_NGINX_REDIS_PORT} ||= 6379; +$ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{LUA_PATH} ||= '/usr/local/openresty-debug/lualib/?.lua;/usr/local/openresty/lualib/?.lua;;'; @@ -701,7 +702,35 @@ qr/lua tcp socket connection pool size: 25\b/] -=== TEST 10: sock:keepalive_timeout(0) means unlimited +=== TEST 10: setkeepalive() 'pool_size' should be greater than zero +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + content_by_lua_block { + local sock, err = ngx.socket.connect("127.0.0.1", ngx.var.port) + if not sock then + ngx.say(err) + return + end + + local ok, err = pcall(sock.setkeepalive, sock, 0, 0) + if not ok then + ngx.say(err) + return + end + ngx.say(ok) + } + } +--- request +GET /t +--- response_body +bad argument #3 to '?' (bad "pool_size" option value: 0) +--- no_error_log +[error] + + + +=== TEST 11: sock:keepalive_timeout(0) means unlimited --- config server_tokens off; location /t { @@ -776,7 +805,7 @@ qr/lua tcp socket connection pool size: 30\b/] -=== TEST 11: sanity (uds) +=== TEST 12: sanity (uds) --- http_config eval " lua_package_path '$::HtmlDir/?.lua;./?.lua'; @@ -858,7 +887,7 @@ received response of 119 bytes -=== TEST 12: github issue #108: ngx.locaiton.capture + redis.set_keepalive +=== TEST 13: github issue #108: ngx.locaiton.capture + redis.set_keepalive --- http_config eval qq{ lua_package_path "$::HtmlDir/?.lua;;"; @@ -905,7 +934,7 @@ lua tcp socket get keepalive peer: using connection -=== TEST 13: github issue #110: ngx.exit with HTTP_NOT_FOUND causes worker process to exit +=== TEST 14: github issue #110: ngx.exit with HTTP_NOT_FOUND causes worker process to exit --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -964,7 +993,7 @@ Not found, dear... -=== TEST 14: custom pools (different pool for the same host:port) - tcp +=== TEST 15: custom pools (different pool for the same host:port) - tcp --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1012,7 +1041,7 @@ lua tcp socket keepalive create connection pool for key "B" -=== TEST 15: custom pools (same pool for different host:port) - tcp +=== TEST 16: custom pools (same pool for different host:port) - tcp --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1059,7 +1088,7 @@ lua tcp socket get keepalive peer: using connection -=== TEST 16: custom pools (different pool for the same host:port) - unix +=== TEST 17: custom pools (different pool for the same host:port) - unix --- http_config eval " lua_package_path '$::HtmlDir/?.lua;./?.lua'; @@ -1119,7 +1148,7 @@ lua tcp socket keepalive create connection pool for key "B" -=== TEST 17: custom pools (same pool for the same path) - unix +=== TEST 18: custom pools (same pool for the same path) - unix --- http_config eval " lua_package_path '$::HtmlDir/?.lua;./?.lua'; @@ -1174,7 +1203,7 @@ lua tcp socket get keepalive peer: using connection -=== TEST 18: numeric pool option value +=== TEST 19: numeric pool option value --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1221,7 +1250,7 @@ lua tcp socket get keepalive peer: using connection -=== TEST 19: nil pool option value +=== TEST 20: nil pool option value --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1264,7 +1293,7 @@ connected: 1, reused: 0 -=== TEST 20: (bad) table pool option value +=== TEST 21: (bad) table pool option value --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1305,7 +1334,7 @@ bad argument #3 to 'connect' (bad "pool" option type: table) -=== TEST 21: (bad) boolean pool option value +=== TEST 22: (bad) boolean pool option value --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1346,7 +1375,7 @@ bad argument #3 to 'connect' (bad "pool" option type: boolean) -=== TEST 22: clear the redis store +=== TEST 23: clear the redis store --- config location /t { redis2_query flushall; @@ -1363,7 +1392,7 @@ bad argument #3 to 'connect' (bad "pool" option type: boolean) -=== TEST 23: bug in send(): clear the chain writer ctx +=== TEST 24: bug in send(): clear the chain writer ctx --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua';" --- config @@ -1478,7 +1507,7 @@ done -=== TEST 24: setkeepalive() with explicit nil args +=== TEST 25: setkeepalive() with explicit nil args --- config server_tokens off; location /t { @@ -1551,3 +1580,1258 @@ done "lua tcp socket keepalive timeout: 100 ms", qr/lua tcp socket connection pool size: 30\b/] --- timeout: 4 + + + +=== TEST 26: conn queuing: connect() verifies the options for connection pool +--- config + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local function check_opts_for_connect(opts) + local ok, err = pcall(function() + sock:connect("127.0.0.1", ngx.var.port, opts) + end) + if not ok then + ngx.say(err) + else + ngx.say("ok") + end + end + + check_opts_for_connect({pool_size = 'a'}) + check_opts_for_connect({pool_size = 0}) + check_opts_for_connect({backlog = -1}) + check_opts_for_connect({backlog = 0}) + } + } +--- request +GET /t +--- response_body_like +.+ 'connect' \(bad "pool_size" option type: string\) +.+ 'connect' \(bad "pool_size" option value: 0\) +.+ 'connect' \(bad "backlog" option value: -1\) +ok +--- no_error_log +[error] + + + +=== TEST 27: conn queuing: connect() can specify 'pool_size' which overrides setkeepalive() +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local function go() + local sock = ngx.socket.tcp() + local ok, err = sock:connect("127.0.0.1", port, {pool_size = 1}) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok, ", reused: ", sock:getreusedtimes()) + + local req = "flush_all\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local line, err, part = sock:receive() + if line then + ngx.say("received: ", line) + else + ngx.say("failed to receive a line: ", err, " [", part, "]") + end + + local ok, err = sock:setkeepalive(0, 20) + if not ok then + ngx.say("failed to set reusable: ", err) + end + end + + -- reuse ok + go() + go() + + local sock1 = ngx.socket.connect("127.0.0.1", port) + local sock2 = ngx.socket.connect("127.0.0.1", port) + local ok, err = sock1:setkeepalive(0, 20) + if not ok then + ngx.say(err) + end + local ok, err = sock2:setkeepalive(0, 20) + if not ok then + ngx.say(err) + end + + -- the pool_size is 1 instead of 20 + sock1 = ngx.socket.connect("127.0.0.1", port) + sock2 = ngx.socket.connect("127.0.0.1", port) + ngx.say("reused: ", sock1:getreusedtimes()) + ngx.say("reused: ", sock2:getreusedtimes()) + sock1:setkeepalive(0, 20) + sock2:setkeepalive(0, 20) + } + } +--- request +GET /t +--- response_body +connected: 1, reused: 0 +request sent: 11 +received: OK +connected: 1, reused: 1 +request sent: 11 +received: OK +reused: 1 +reused: 0 +--- no_error_log eval +["[error]", +"lua tcp socket keepalive: free connection pool for ", +"lua tcp socket connection pool size: 20"] +--- error_log eval +[qq{lua tcp socket keepalive create connection pool for key "127.0.0.1:$ENV{TEST_NGINX_MEMCACHED_PORT}"}, +"lua tcp socket connection pool size: 1", +] + + + +=== TEST 28: conn queuing: connect() can specify 'pool_size' for unix domain socket +--- http_config eval +" + server { + listen unix:$::HtmlDir/nginx.sock; + } +" +--- config + location /t { + content_by_lua_block { + local path = "unix:" .. "$TEST_NGINX_HTML_DIR/nginx.sock"; + local function go() + local sock = ngx.socket.tcp() + local ok, err = sock:connect(path, {pool_size = 1}) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok, ", reused: ", sock:getreusedtimes()) + + local ok, err = sock:setkeepalive(0, 20) + if not ok then + ngx.say("failed to set reusable: ", err) + end + end + + go() + go() + + local sock1 = ngx.socket.connect(path) + local sock2 = ngx.socket.connect(path) + local ok, err = sock1:setkeepalive(0, 20) + if not ok then + ngx.say(err) + end + local ok, err = sock2:setkeepalive(0, 20) + if not ok then + ngx.say(err) + end + + -- the pool_size is 1 instead of 20 + sock1 = ngx.socket.connect(path) + sock2 = ngx.socket.connect(path) + ngx.say("reused: ", sock1:getreusedtimes()) + ngx.say("reused: ", sock2:getreusedtimes()) + sock1:setkeepalive(0, 20) + sock2:setkeepalive(0, 20) + } + } +--- request +GET /t +--- response_body +connected: 1, reused: 0 +connected: 1, reused: 1 +reused: 1 +reused: 0 +--- no_error_log eval +["[error]", +"lua tcp socket keepalive: free connection pool for ", +"lua tcp socket connection pool size: 20"] +--- error_log eval +["lua tcp socket get keepalive peer: using connection", +'lua tcp socket keepalive create connection pool for key "unix:', +"lua tcp socket connection pool size: 1", +] + + + +=== TEST 29: conn queuing: connect() can specify 'pool_size' for custom pool +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local function go(pool) + local sock = ngx.socket.tcp() + local ok, err = sock:connect("127.0.0.1", port, {pool = pool, pool_size = 1}) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", pool, ", reused: ", sock:getreusedtimes()) + + local ok, err = sock:setkeepalive(0, 20) + if not ok then + ngx.say("failed to set reusable: ", err) + end + end + + go('A') + go('B') + go('A') + go('B') + + local sock1 = ngx.socket.connect("127.0.0.1", port, {pool = 'A'}) + local sock2 = ngx.socket.connect("127.0.0.1", port, {pool = 'A'}) + local ok, err = sock1:setkeepalive(0, 20) + if not ok then + ngx.say(err) + end + local ok, err = sock2:setkeepalive(0, 20) + if not ok then + ngx.say(err) + end + + -- the pool_size is 1 instead of 20 + sock1 = ngx.socket.connect("127.0.0.1", port, {pool = 'A'}) + sock2 = ngx.socket.connect("127.0.0.1", port, {pool = 'A'}) + ngx.say("reused: ", sock1:getreusedtimes()) + ngx.say("reused: ", sock2:getreusedtimes()) + sock1:setkeepalive(0, 20) + sock2:setkeepalive(0, 20) + } + } +--- request +GET /t +--- response_body +connected: A, reused: 0 +connected: B, reused: 0 +connected: A, reused: 1 +connected: B, reused: 1 +reused: 1 +reused: 0 +--- no_error_log eval +["[error]", +"lua tcp socket keepalive: free connection pool for ", +"lua tcp socket connection pool size: 20"] +--- error_log eval +[qq{lua tcp socket keepalive create connection pool for key "A"}, +qq{lua tcp socket keepalive create connection pool for key "B"}, +"lua tcp socket connection pool size: 1", +] + + + +=== TEST 30: conn queuing: connect() uses lua_socket_pool_size as default if 'backlog' is given +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + lua_socket_pool_size 1234; + + content_by_lua_block { + local port = ngx.var.port + local opts = {backlog = 0} + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.say(err) + else + ngx.say("ok") + end + } + } +--- request +GET /t +--- response_body +ok +--- error_log +lua tcp socket connection pool size: 1234 +--- no_error_log +[error] + + + +=== TEST 31: conn queuing: more connect operations than 'backlog' size +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 2, backlog = 0} + local sock = ngx.socket.connect("127.0.0.1", port, opts) + local not_reused_socket, err = ngx.socket.connect("127.0.0.1", port, opts) + if not not_reused_socket then + ngx.say(err) + return + end + -- burst + local ok, err = ngx.socket.connect("127.0.0.1", port, opts) + if not ok then + ngx.say(err) + end + + local ok, err = sock:setkeepalive() + if not ok then + ngx.say(err) + return + end + + ok, err = sock:connect("127.0.0.1", port, opts) + if not ok then + ngx.say(err) + end + ngx.say("reused: ", sock:getreusedtimes()) + -- both queue and pool is full + ok, err = ngx.socket.connect("127.0.0.1", port, opts) + if not ok then + ngx.say(err) + end + } + } +--- request +GET /t +--- response_body +too many waiting connect operations +reused: 1 +too many waiting connect operations +--- no_error_log +[error] + + + +=== TEST 32: conn queuing: once 'pool_size' is reached and pool has 'backlog' +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 2, backlog = 2} + local sock1 = ngx.socket.connect("127.0.0.1", port, opts) + + ngx.timer.at(0, function(premature) + local sock2, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock2 then + ngx.log(ngx.ERR, err) + return + end + + ngx.log(ngx.WARN, "start to handle timer") + ngx.sleep(0.1) + sock2:close() + -- resume connect operation + ngx.log(ngx.WARN, "continue to handle timer") + end) + + ngx.sleep(0.05) + ngx.log(ngx.WARN, "start to handle cosocket") + local sock3, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock3 then + ngx.say(err) + return + end + ngx.log(ngx.WARN, "continue to handle cosocket") + + local req = "flush_all\r\n" + local bytes, err = sock3:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local line, err, part = sock3:receive() + if line then + ngx.say("received: ", line) + else + ngx.say("failed to receive a line: ", err, " [", part, "]") + end + + local ok, err = sock3:setkeepalive() + if not ok then + ngx.say("failed to set reusable: ", err) + end + ngx.say("setkeepalive: OK") + } + } +--- request +GET /t +--- response_body +request sent: 11 +received: OK +setkeepalive: OK +--- no_error_log +[error] +--- error_log +lua tcp socket queue connect operation for connection pool "127.0.0.1 +--- grep_error_log eval: qr/(start|continue) to handle \w+/ +--- grep_error_log_out +start to handle timer +start to handle cosocket +continue to handle timer +continue to handle cosocket + + + +=== TEST 33: conn queuing: do not count failed connect operations +--- config + resolver $TEST_NGINX_RESOLVER ipv6=off; + resolver_timeout 3s; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 0} + + local sock = ngx.socket.tcp() + sock:settimeouts(100, 3000, 3000) + local ok, err = sock:connect("agentzh.org", 12345, opts) + if not ok then + ngx.say(err) + end + + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.say(err) + end + ngx.say("ok") + } + } +--- request +GET /t +--- error_log +lua tcp socket connect timed out, when connecting to +--- response_body +timeout +ok + + + +=== TEST 34: conn queuing: connect until backlog is reached +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 1} + local sock1 = ngx.socket.connect("127.0.0.1", port, opts) + + ngx.timer.at(0.01, function(premature) + ngx.log(ngx.WARN, "start to handle timer") + local sock2, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock2 then + ngx.log(ngx.ERR, err) + return + end + + ngx.sleep(0.02) + local ok, err = sock2:close() + if not ok then + ngx.log(ngx.ERR, err) + end + ngx.log(ngx.WARN, "continue to handle timer") + end) + + ngx.sleep(0.02) + local sock3, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock3 then + ngx.say(err) + end + local ok, err = sock1:setkeepalive() + if not ok then + ngx.say(err) + return + end + ngx.sleep(0.01) -- run sock2 + + ngx.log(ngx.WARN, "start to handle cosocket") + local sock3, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock3 then + ngx.say(err) + return + end + ngx.log(ngx.WARN, "continue to handle cosocket") + + local ok, err = sock3:setkeepalive() + if not ok then + ngx.say(err) + end + } + } +--- request +GET /t +--- response_body +too many waiting connect operations +--- no_error_log +[error] +--- error_log +lua tcp socket queue connect operation for connection pool "127.0.0.1 +--- grep_error_log eval: qr/queue connect operation for connection pool|(start|continue) to handle \w+/ +--- grep_error_log_out +start to handle timer +queue connect operation for connection pool +start to handle cosocket +queue connect operation for connection pool +continue to handle timer +continue to handle cosocket + + + +=== TEST 35: conn queuing: memory reuse for host in queueing connect operation ctx +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 3} + local sock = ngx.socket.connect("127.0.0.1", port, opts) + + ngx.timer.at(0.01, function(premature) + local sock, err = ngx.socket.connect("0.0.0.0", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local ok, err = sock:close() + if not ok then + ngx.log(ngx.ERR, err) + end + end) + + ngx.timer.at(0.015, function(premature) + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local ok, err = sock:close() + if not ok then + ngx.log(ngx.ERR, err) + end + end) + + ngx.timer.at(0.02, function(premature) + local sock, err = ngx.socket.connect("0.0.0.0", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local ok, err = sock:close() + if not ok then + ngx.log(ngx.ERR, err) + end + end) + + ngx.sleep(0.03) + local ok, err = sock:setkeepalive() + if not ok then + ngx.say(err) + return + end + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- no_error_log +[error] +--- grep_error_log eval: qr/queue connect operation for connection pool/ +--- grep_error_log_out +queue connect operation for connection pool +queue connect operation for connection pool +queue connect operation for connection pool + + + +=== TEST 36: conn queuing: connect() returns error after connect operation resumed +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 1} + local sock = ngx.socket.connect("127.0.0.1", port, opts) + + ngx.timer.at(0, function(premature) + local sock, err = ngx.socket.connect("", port, opts) + if not sock then + ngx.log(ngx.WARN, err) + end + end) + + ngx.sleep(0.01) + -- use 'close' to force parsing host instead of reusing conn + local ok, err = sock:close() + if not ok then + ngx.say(err) + return + end + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- no_error_log +[error] +--- error_log +failed to parse host name +--- grep_error_log eval: qr/queue connect operation for connection pool/ +--- grep_error_log_out +queue connect operation for connection pool + + + +=== TEST 37: conn queuing: in uthread +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 2} + + local conn_sock = function() + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.say(err) + return + end + ngx.say("start to handle uthread") + + ngx.sleep(0.01) + sock:close() + ngx.say("continue to handle other uthread") + end + + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local co1 = ngx.thread.spawn(conn_sock) + local co2 = ngx.thread.spawn(conn_sock) + local co3 = ngx.thread.spawn(conn_sock) + + local ok, err = sock:setkeepalive() + if not ok then + ngx.log(ngx.ERR, err) + end + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.thread.wait(co3) + ngx.say("all uthreads ok") + } + } +--- request +GET /t +--- response_body +too many waiting connect operations +start to handle uthread +continue to handle other uthread +start to handle uthread +continue to handle other uthread +all uthreads ok +--- no_error_log +[error] +--- grep_error_log eval: qr/queue connect operation for connection pool/ +--- grep_error_log_out +queue connect operation for connection pool +queue connect operation for connection pool + + + +=== TEST 38: conn queuing: in access_by_lua +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + access_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 2} + + local conn_sock = function() + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.say(err) + return + end + ngx.say("start to handle uthread") + + ngx.sleep(0.01) + sock:close() + ngx.say("continue to handle other uthread") + end + + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local co1 = ngx.thread.spawn(conn_sock) + local co2 = ngx.thread.spawn(conn_sock) + local co3 = ngx.thread.spawn(conn_sock) + + local ok, err = sock:setkeepalive() + if not ok then + ngx.log(ngx.ERR, err) + end + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.thread.wait(co3) + ngx.say("all uthreads ok") + } + } +--- request +GET /t +--- response_body +too many waiting connect operations +start to handle uthread +continue to handle other uthread +start to handle uthread +continue to handle other uthread +all uthreads ok +--- no_error_log +[error] +--- grep_error_log eval: qr/queue connect operation for connection pool/ +--- grep_error_log_out +queue connect operation for connection pool +queue connect operation for connection pool + + + +=== TEST 39: conn queuing: in rewrite_by_lua +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + rewrite_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 2} + + local conn_sock = function() + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.say(err) + return + end + ngx.say("start to handle uthread") + + ngx.sleep(0.01) + sock:close() + ngx.say("continue to handle other uthread") + end + + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local co1 = ngx.thread.spawn(conn_sock) + local co2 = ngx.thread.spawn(conn_sock) + local co3 = ngx.thread.spawn(conn_sock) + + local ok, err = sock:setkeepalive() + if not ok then + ngx.log(ngx.ERR, err) + end + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.thread.wait(co3) + ngx.say("all uthreads ok") + } + } +--- request +GET /t +--- response_body +too many waiting connect operations +start to handle uthread +continue to handle other uthread +start to handle uthread +continue to handle other uthread +all uthreads ok +--- no_error_log +[error] +--- grep_error_log eval: qr/queue connect operation for connection pool/ +--- grep_error_log_out +queue connect operation for connection pool +queue connect operation for connection pool + + + +=== TEST 40: conn queuing: in subrequest +--- config + set $port $TEST_NGINX_MEMCACHED_PORT; + + location /t { + content_by_lua_block { + local port = ngx.var.port + ngx.timer.at(0, function() + local opts = {pool_size = 1, backlog = 2} + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + ngx.sleep(0.1) + local ok, err = sock:setkeepalive() + if not ok then + ngx.log(ngx.ERR, err) + end + end) + + ngx.sleep(0.01) + local res1, res2, res3 = ngx.location.capture_multi{ + {"/conn"}, {"/conn"}, {"/conn"} + } + ngx.say(res1.body) + ngx.say(res2.body) + ngx.say(res3.body) + } + } + + location /conn { + content_by_lua_block { + local port = ngx.var.port + local sock, err = ngx.socket.connect("127.0.0.1", port) + if not sock then + ngx.print(err) + return + end + local ok, err = sock:setkeepalive() + if not ok then + ngx.print(err) + else + ngx.print("ok") + end + } + } +--- request +GET /t +--- response_body +ok +ok +too many waiting connect operations +--- no_error_log +[error] +--- grep_error_log eval: qr/queue connect operation for connection pool/ +--- grep_error_log_out +queue connect operation for connection pool +queue connect operation for connection pool + + + +=== TEST 41: conn queuing: timeouts when 'connect_timeout' is reached +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 1} + local sock1 = ngx.socket.connect("127.0.0.1", port, opts) + + local sock2 = ngx.socket.tcp() + sock2:settimeouts(10, 3000, 3000) + local ok, err = sock2:connect("127.0.0.1", port, opts) + if not ok then + ngx.say(err) + end + } + } +--- request +GET /t +--- response_body +timeout +--- error_log eval +"lua tcp socket queued connect timed out, when trying to connect to 127.0.0.1:$ENV{TEST_NGINX_MEMCACHED_PORT}" + + + +=== TEST 42: conn queuing: set timeout via lua_socket_connect_timeout +--- config + lua_socket_connect_timeout 10ms; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 1} + local sock1 = ngx.socket.connect("127.0.0.1", port, opts) + + local sock2 = ngx.socket.tcp() + local ok, err = sock2:connect("127.0.0.1", port, opts) + if not ok then + ngx.say(err) + end + } + } +--- request +GET /t +--- response_body +timeout +--- error_log eval +"lua tcp socket queued connect timed out, when trying to connect to 127.0.0.1:$ENV{TEST_NGINX_MEMCACHED_PORT}" + + + +=== TEST 43: conn queuing: client aborting while connect operation is queued +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool_size = 1, backlog = 1} + local sock1 = ngx.socket.connect("127.0.0.1", port, opts) + + local sock2 = ngx.socket.tcp() + sock2:settimeouts(3000, 3000, 3000) + local ok, err = sock2:connect("127.0.0.1", port, opts) + if not ok then + ngx.say(err) + end + } + } +--- request +GET /t +--- ignore_response +--- timeout: 0.1 +--- abort +--- no_error_log +[error] + + + +=== TEST 44: conn queuing: resume next connect operation if resumed connect failed immediately +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 2} + + local conn_sock = function(should_timeout) + local sock = ngx.socket.tcp() + local ok, err + if should_timeout then + ok, err = sock:connect("", port, opts) + else + ok, err = sock:connect("127.0.0.1", port, opts) + end + if not ok then + ngx.say(err) + return + end + ngx.say("connected in uthread") + sock:close() + end + + local sock, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock then + ngx.log(ngx.ERR, err) + return + end + + local co1 = ngx.thread.spawn(conn_sock, true) + local co2 = ngx.thread.spawn(conn_sock) + + local ok, err = sock:close() + if not ok then + ngx.log(ngx.ERR, err) + end + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +failed to parse host name "": no host +connected in uthread +ok +--- no_error_log +[error] + + + +=== TEST 45: conn queuing: resume connect operation if resumed connect failed (timeout) +--- config + resolver $TEST_NGINX_RESOLVER ipv6=off; + resolver_timeout 3s; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 1} + + local conn_sock = function(should_timeout) + local sock = ngx.socket.tcp() + local ok, err + if should_timeout then + sock:settimeouts(100, 3000, 3000) + ok, err = sock:connect("agentzh.org", 12345, opts) + else + ok, err = sock:connect("127.0.0.1", port, opts) + end + if not ok then + ngx.say(err) + return + end + ngx.say("connected in uthread") + sock:close() + end + + local co1 = ngx.thread.spawn(conn_sock, true) + local co2 = ngx.thread.spawn(conn_sock) + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +timeout +connected in uthread +ok +--- error_log +queue connect operation for connection pool "test" +lua tcp socket connect timed out, when connecting to + + + +=== TEST 46: conn queuing: resume connect operation if resumed connect failed (could not be resolved) +--- config + resolver 127.0.0.1 ipv6=off; + resolver_timeout 100ms; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 1} + + local conn_sock = function(should_timeout) + local sock = ngx.socket.tcp() + local ok, err + if should_timeout then + sock:settimeouts(100, 3000, 3000) + ok, err = sock:connect("agentzh.org", 12345, opts) + else + ok, err = sock:connect("127.0.0.1", port, opts) + end + if not ok then + ngx.say(err) + return + end + ngx.say("connected in uthread") + sock:close() + end + + local co1 = ngx.thread.spawn(conn_sock, true) + local co2 = ngx.thread.spawn(conn_sock) + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +agentzh.org could not be resolved (110: Operation timed out) +connected in uthread +ok +--- error_log +queue connect operation for connection pool "test" + + + +=== TEST 47: conn queuing: resume connect operation if resumed connect failed (connection refused) +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local port = ngx.var.port + local opts = {pool = "test", pool_size = 1, backlog = 1} + + local conn_sock = function(should_timeout) + local sock = ngx.socket.tcp() + local ok, err + if should_timeout then + sock:settimeouts(100, 3000, 3000) + ok, err = sock:connect("127.0.0.1", 62345, opts) + else + ok, err = sock:connect("127.0.0.1", port, opts) + end + if not ok then + ngx.say(err) + return + end + ngx.say("connected in uthread") + sock:close() + end + + local co1 = ngx.thread.spawn(conn_sock, true) + local co2 = ngx.thread.spawn(conn_sock) + + ngx.thread.wait(co1) + ngx.thread.wait(co2) + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +connection refused +connected in uthread +ok +--- error_log +queue connect operation for connection pool "test" + + + +=== TEST 48: conn queuing: resume connect operation if resumed connect failed (uthread aborted while resolving) +--- http_config + lua_package_path '../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;'; +--- config + resolver 127.0.0.1 ipv6=off; + resolver_timeout 100s; + set $port $TEST_NGINX_MEMCACHED_PORT; + + location /sub { + content_by_lua_block { + local semaphore = require "ngx.semaphore" + local sem = semaphore.new() + + local function f() + sem:wait(0.1) + ngx.exit(0) + end + + local opts = {pool = "test", pool_size = 1, backlog = 1} + local port = ngx.var.port + ngx.timer.at(0, function() + sem:post() + local sock2, err = ngx.socket.connect("127.0.0.1", port, opts) + package.loaded.for_timer_to_resume:post() + if not sock2 then + ngx.log(ngx.ALERT, "resume connect failed: ", err) + return + end + + ngx.log(ngx.INFO, "resume success") + end) + + ngx.thread.spawn(f) + local sock1, err = ngx.socket.connect("openresty.org", 80, opts) + if not sock1 then + ngx.say(err) + return + end + } + } + + location /t { + content_by_lua_block { + local semaphore = require "ngx.semaphore" + local for_timer_to_resume = semaphore.new() + package.loaded.for_timer_to_resume = for_timer_to_resume + + ngx.location.capture("/sub") + for_timer_to_resume:wait(0.1) + } + } +--- request +GET /t +--- no_error_log +[alert] +--- error_log +resume success + + + +=== TEST 49: conn queuing: resume connect operation if resumed connect failed (uthread killed while resolving) +--- config + resolver 127.0.0.1 ipv6=off; + resolver_timeout 100s; + set $port $TEST_NGINX_MEMCACHED_PORT; + + location /t { + content_by_lua_block { + local opts = {pool = "test", pool_size = 1, backlog = 1} + local port = ngx.var.port + + local function resolve() + local sock1, err = ngx.socket.connect("openresty.org", 80, opts) + if not sock1 then + ngx.say(err) + return + end + end + + local th = ngx.thread.spawn(resolve) + local ok, err = ngx.thread.kill(th) + if not ok then + ngx.log(ngx.ALERT, "kill thread failed: ", err) + return + end + + local sock2, err = ngx.socket.connect("127.0.0.1", port, opts) + if not sock2 then + ngx.log(ngx.ALERT, "resume connect failed: ", err) + return + end + + ngx.log(ngx.INFO, "resume success") + } + } +--- request +GET /t +--- no_error_log +[alert] +--- error_log +resume success From bd9964e7f3e3f89ce35b6ac3ee2570dbd19c8f39 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 29 Jan 2019 12:09:09 -0800 Subject: [PATCH 276/848] tests: fixed some test cases failing with the 1.15.8 nginx core. --- t/058-tcp-socket.t | 8 ++++---- t/124-init-worker.t | 3 +++ t/129-ssl-socket.t | 4 ++-- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 298503c2cc..edebf3267b 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -3720,10 +3720,10 @@ sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT } --- request GET /t ---- response_body -failed to connect: www.google.com could not be resolved -failed to connect: www.google.com could not be resolved -failed to connect: www.google.com could not be resolved +--- response_body_like +failed to connect: www.google.com could not be resolved(?: \(\d+: Operation timed out\))? +failed to connect: www.google.com could not be resolved(?: \(\d+: Operation timed out\))? +failed to connect: www.google.com could not be resolved(?: \(\d+: Operation timed out\))? hello! --- error_log eval qr{\[alert\] .*? send\(\) failed \(\d+: Operation not permitted\) while resolving} diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 9b2a91fcfa..182ab40212 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -550,6 +550,7 @@ second line received: Server: openresty else say("connect: ", ok, " ", err) end + done = true end local ok, err = ngx.timer.at(0, handler) @@ -600,6 +601,7 @@ qr/connect\(\) failed \(\d+: Connection refused\), context: ngx\.timer$/ else say("connect: ", ok, " ", err) end + done = true end local ok, err = ngx.timer.at(0, handler) @@ -650,6 +652,7 @@ qr/connect\(\) failed \(\d+: Connection refused\)/ else say("connect: ", ok, " ", err) end + done = true end local ok, err = ngx.timer.at(0, handler) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index d5068ce3df..287f9b250b 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1333,13 +1333,13 @@ failed to send http request: closed --- grep_error_log_out --- error_log eval [ -qr/\[crit\] .*?SSL_do_handshake\(\) failed .*?(unsupported protocol|no protocols available)/, +qr/\[(crit|error)\] .*?SSL_do_handshake\(\) failed .*?(unsupported protocol|no protocols available)/, 'lua ssl server name: "openresty.org"', ] --- no_error_log SSL reused session -[error] [alert] +[emerg] --- timeout: 5 From 2c8984934c74f7bbf3c365e67ec3273c0c13efd3 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 25 Jan 2019 14:44:55 -0800 Subject: [PATCH 277/848] travis: bumped the nginx core version to 1.15.8. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 4d0823d67c..589255d5d7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,8 +48,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.0.2q OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.13.6 OPENSSL_VER=1.1.0j OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2q OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0j OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" # to $OPENSSL_OPT. From aef03eb3317a2533d7fca3a06b07973a421a900c Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 14 Nov 2018 22:40:26 +0800 Subject: [PATCH 278/848] feature: added the C API to support the 'ngx.pipe' module provided by lua-resty-core. Signed-off-by: Yichun Zhang (agentzh) Signed-off-by: Thibault Charbonnier --- config | 32 + src/ngx_http_lua_initworkerby.c | 8 + src/ngx_http_lua_module.c | 6 + src/ngx_http_lua_pipe.c | 2487 +++++++++++++++++++++++++++++++ src/ngx_http_lua_pipe.h | 95 ++ 5 files changed, 2628 insertions(+) create mode 100644 src/ngx_http_lua_pipe.c create mode 100644 src/ngx_http_lua_pipe.h diff --git a/config b/config index cf887abf63..e1d5e3543f 100644 --- a/config +++ b/config @@ -362,6 +362,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl.c \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.c \ $ngx_addon_dir/src/ngx_http_lua_input_filters.c \ + $ngx_addon_dir/src/ngx_http_lua_pipe.c \ " HTTP_LUA_DEPS=" \ @@ -424,6 +425,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl.h \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.h \ $ngx_addon_dir/src/ngx_http_lua_input_filters.h \ + $ngx_addon_dir/src/ngx_http_lua_pipe.h \ " CFLAGS="$CFLAGS -DNDK_SET_VAR" @@ -525,6 +527,36 @@ CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" # ---------------------------------------- +ngx_feature="pipe2" +ngx_feature_libs= +ngx_feature_name="NGX_HTTP_LUA_HAVE_PIPE2" +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_test="int fd[2]; pipe2(fd, O_CLOEXEC|O_NONBLOCK);" +SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" +CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" + +. auto/feature + +CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" + +# ---------------------------------------- + +ngx_feature="signalfd" +ngx_feature_libs= +ngx_feature_name="NGX_HTTP_LUA_HAVE_SIGNALFD" +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_test="sigset_t set; signalfd(-1, &set, SFD_NONBLOCK|SFD_CLOEXEC);" +SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" +CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" + +. auto/feature + +CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" + +# ---------------------------------------- + if test -n "$ngx_module_link"; then ngx_module_type=HTTP_AUX_FILTER ngx_module_name=$ngx_addon_name diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index e15390ed59..76acdc3bb4 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -12,6 +12,7 @@ #include "ngx_http_lua_initworkerby.h" #include "ngx_http_lua_util.h" +#include "ngx_http_lua_pipe.h" static u_char *ngx_http_lua_log_init_worker_error(ngx_log_t *log, @@ -65,6 +66,13 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) return NGX_OK; } + +#ifdef HAVE_NGX_LUA_PIPE + if (ngx_http_lua_pipe_add_signal_handler(cycle) != NGX_OK) { + return NGX_ERROR; + } +#endif + #endif /* NGX_WIN32 */ #if (NGX_HTTP_LUA_HAVE_SA_RESTART) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 5f242eab4a..cb99fd781e 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -29,6 +29,7 @@ #include "ngx_http_lua_ssl_session_storeby.h" #include "ngx_http_lua_ssl_session_fetchby.h" #include "ngx_http_lua_headers.h" +#include "ngx_http_lua_pipe.h" static void *ngx_http_lua_create_main_conf(ngx_conf_t *cf); @@ -742,6 +743,11 @@ ngx_http_lua_init(ngx_conf_t *cf) cln->data = lmcf; cln->handler = ngx_http_lua_sema_mm_cleanup; + +#ifdef HAVE_NGX_LUA_PIPE + ngx_http_lua_pipe_init(); +#endif + #endif #if nginx_version >= 1011011 diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c new file mode 100644 index 0000000000..65251ff75d --- /dev/null +++ b/src/ngx_http_lua_pipe.c @@ -0,0 +1,2487 @@ + +/* + * Copyright (C) by OpenResty Inc. + */ + + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#include "ngx_http_lua_common.h" +#include "ngx_http_lua_input_filters.h" +#include "ngx_http_lua_util.h" +#include "ngx_http_lua_pipe.h" +#if (NGX_HTTP_LUA_HAVE_SIGNALFD) +#include +#endif + + +#ifdef HAVE_NGX_LUA_PIPE +static ngx_rbtree_node_t *ngx_http_lua_pipe_lookup_pid(ngx_rbtree_key_t key); +#if !(NGX_HTTP_LUA_HAVE_SIGNALFD) +static void ngx_http_lua_pipe_sigchld_handler(int signo, siginfo_t *siginfo, + void *ucontext); +#endif +static void ngx_http_lua_pipe_sigchld_event_handler(ngx_event_t *ev); +static ssize_t ngx_http_lua_pipe_fd_read(ngx_connection_t *c, u_char *buf, + size_t size); +static ssize_t ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, + size_t size); +static ngx_int_t ngx_http_lua_pipe_close_helper( + ngx_http_lua_pipe_ctx_t *pipe_ctx, ngx_event_t *ev, int forced); +static ngx_int_t ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe, + int forced); +static ngx_int_t ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe, + int forced); +static ngx_int_t ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe, + int forced); +static void ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc, + int forced); +static ngx_int_t ngx_http_lua_pipe_get_lua_ctx(ngx_http_request_t *r, + ngx_http_lua_ctx_t **ctx, u_char *errbuf, size_t *errbuf_size); +static void ngx_http_lua_pipe_put_error(ngx_http_lua_pipe_ctx_t *pipe_ctx, + u_char *errbuf, size_t *errbuf_size); +static void ngx_http_lua_pipe_put_data(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx, u_char **buf, size_t *buf_size); +static ngx_int_t ngx_http_lua_pipe_add_input_buffer(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx); +static ngx_int_t ngx_http_lua_pipe_read_all(void *data, ssize_t bytes); +static ngx_int_t ngx_http_lua_pipe_read_bytes(void *data, ssize_t bytes); +static ngx_int_t ngx_http_lua_pipe_read_line(void *data, ssize_t bytes); +static ngx_int_t ngx_http_lua_pipe_read_any(void *data, ssize_t bytes); +static ngx_int_t ngx_http_lua_pipe_read(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx); +static ngx_int_t ngx_http_lua_pipe_init_ctx( + ngx_http_lua_pipe_ctx_t **pipe_ctx_pt, int fd, ngx_pool_t *pool, + u_char *errbuf, size_t *errbuf_size); +static ngx_int_t ngx_http_lua_pipe_write(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx); +static int ngx_http_lua_pipe_read_stdout_retval( + ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L); +static int ngx_http_lua_pipe_read_stderr_retval( + ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L); +static int ngx_http_lua_pipe_read_retval_helper( + ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L, int from_stderr); +static int ngx_http_lua_pipe_write_retval(ngx_http_lua_ffi_pipe_proc_t *proc, + lua_State *L); +static int ngx_http_lua_pipe_wait_retval(ngx_http_lua_ffi_pipe_proc_t *proc, + lua_State *L); +static void ngx_http_lua_pipe_resume_helper(ngx_event_t *ev, + ngx_http_lua_co_ctx_t *wait_co_ctx); +static void ngx_http_lua_pipe_resume_read_stdout_handler(ngx_event_t *ev); +static void ngx_http_lua_pipe_resume_read_stderr_handler(ngx_event_t *ev); +static void ngx_http_lua_pipe_resume_write_handler(ngx_event_t *ev); +static void ngx_http_lua_pipe_resume_wait_handler(ngx_event_t *ev); +static ngx_int_t ngx_http_lua_pipe_resume(ngx_http_request_t *r); +static void ngx_http_lua_pipe_dummy_event_handler(ngx_event_t *ev); +static void ngx_http_lua_pipe_cleanup_helper( + ngx_http_lua_co_ctx_t *wait_co_ctx); +static void ngx_http_lua_pipe_proc_read_stdout_cleanup(void *data); +static void ngx_http_lua_pipe_proc_read_stderr_cleanup(void *data); +static void ngx_http_lua_pipe_proc_write_cleanup(void *data); +static void ngx_http_lua_pipe_proc_wait_cleanup(void *data); + + +static ngx_rbtree_t ngx_http_lua_pipe_rbtree; +static ngx_rbtree_node_t ngx_http_lua_pipe_proc_sentinel; + + +#if (NGX_HTTP_LUA_HAVE_SIGNALFD) +static int ngx_http_lua_signalfd; +static struct signalfd_siginfo ngx_http_lua_pipe_notification; + +#define ngx_http_lua_read_sigfd ngx_http_lua_signalfd + +#else +static int ngx_http_lua_sigchldfd[2]; +static u_char ngx_http_lua_pipe_notification[1]; + +#define ngx_http_lua_read_sigfd ngx_http_lua_sigchldfd[0] +#define ngx_http_lua_write_sigfd ngx_http_lua_sigchldfd[1] +#endif + + +static ngx_connection_t *ngx_http_lua_sigfd_conn = NULL; + + +/* The below signals are ignored by Nginx. + * We need to reset them for the spawned child processes. */ +ngx_http_lua_pipe_signal_t ngx_signals[] = { + { SIGSYS, "SIGSYS" }, + { SIGPIPE, "SIGPIPE" }, + { 0, NULL } +}; + + +enum { + PIPE_ERR_CLOSED = 1, + PIPE_ERR_SYSCALL, + PIPE_ERR_NOMEM, + PIPE_ERR_TIMEOUT, + PIPE_ERR_ADD_READ_EV, + PIPE_ERR_ADD_WRITE_EV +}; + + +enum { + PIPE_READ_ALL = 0, + PIPE_READ_BYTES, + PIPE_READ_LINE, + PIPE_READ_ANY +}; + + +#define REASON_EXIT "exit" +#define REASON_SIGNAL "signal" +#define REASON_UNKNOWN "unknown" + +#define REASON_RUNNING_CODE 0 +#define REASON_EXIT_CODE 1 +#define REASON_SIGNAL_CODE 2 +#define REASON_UNKNOWN_CODE 3 + + +void +ngx_http_lua_pipe_init(void) +{ + ngx_rbtree_init(&ngx_http_lua_pipe_rbtree, + &ngx_http_lua_pipe_proc_sentinel, ngx_rbtree_insert_value); +} + + +ngx_int_t +ngx_http_lua_pipe_add_signal_handler(ngx_cycle_t *cycle) +{ + ngx_event_t *rev; +#if (NGX_HTTP_LUA_HAVE_SIGNALFD) + sigset_t set; + +#else + int rc; + struct sigaction sa; +#endif + +#if (NGX_HTTP_LUA_HAVE_SIGNALFD) + if (sigemptyset(&set) != 0) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe init signal set failed"); + return NGX_ERROR; + } + + if (sigaddset(&set, SIGCHLD) != 0) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe add SIGCHLD to signal set failed"); + return NGX_ERROR; + } + + if (sigprocmask(SIG_BLOCK, &set, NULL) != 0) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe block SIGCHLD failed"); + return NGX_ERROR; + } + + ngx_http_lua_signalfd = signalfd(-1, &set, SFD_NONBLOCK|SFD_CLOEXEC); + if (ngx_http_lua_signalfd < 0) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe create signalfd instance failed"); + return NGX_ERROR; + } + +#else /* !(NGX_HTTP_LUA_HAVE_SIGNALFD) */ +# if (NGX_HTTP_LUA_HAVE_PIPE2) + rc = pipe2(ngx_http_lua_sigchldfd, O_NONBLOCK|O_CLOEXEC); +# else + rc = pipe(ngx_http_lua_sigchldfd); +# endif + + if (rc == -1) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe init SIGCHLD fd failed"); + return NGX_ERROR; + } + +# if !(NGX_HTTP_LUA_HAVE_PIPE2) + if (ngx_nonblocking(ngx_http_lua_read_sigfd) == -1) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, "lua pipe " + ngx_nonblocking_n " SIGCHLD read fd failed"); + goto failed; + } + + if (ngx_nonblocking(ngx_http_lua_write_sigfd) == -1) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, "lua pipe " + ngx_nonblocking_n " SIGCHLD write fd failed"); + goto failed; + } + + /* it's ok not to set the pipe fd with O_CLOEXEC. This requires + * extra syscall */ +# endif /* !(NGX_HTTP_LUA_HAVE_PIPE2) */ +#endif /* NGX_HTTP_LUA_HAVE_SIGNALFD */ + + ngx_http_lua_sigfd_conn = ngx_get_connection(ngx_http_lua_read_sigfd, + cycle->log); + if (ngx_http_lua_sigfd_conn == NULL) { + goto failed; + } + + ngx_http_lua_sigfd_conn->log = cycle->log; + ngx_http_lua_sigfd_conn->recv = ngx_http_lua_pipe_fd_read; + rev = ngx_http_lua_sigfd_conn->read; + rev->log = ngx_http_lua_sigfd_conn->log; + rev->handler = ngx_http_lua_pipe_sigchld_event_handler; + +#ifdef HAVE_SOCKET_CLOEXEC_PATCH + rev->skip_socket_leak_check = 1; +#endif + + if (ngx_handle_read_event(rev, 0) == NGX_ERROR) { + goto failed; + } + +#if !(NGX_HTTP_LUA_HAVE_SIGNALFD) + ngx_memzero(&sa, sizeof(struct sigaction)); + sa.sa_sigaction = ngx_http_lua_pipe_sigchld_handler; + sa.sa_flags = SA_SIGINFO; + + if (sigemptyset(&sa.sa_mask) != 0) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe init signal mask failed"); + goto failed; + } + + if (sigaction(SIGCHLD, &sa, NULL) == -1) { + ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, + "lua pipe sigaction(SIGCHLD) failed"); + goto failed; + } +#endif + + return NGX_OK; + +failed: + + if (ngx_http_lua_sigfd_conn != NULL) { + ngx_close_connection(ngx_http_lua_sigfd_conn); + ngx_http_lua_sigfd_conn = NULL; + } + + if (close(ngx_http_lua_read_sigfd) == -1) { + ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, + "lua pipe close the read sigfd failed"); + } + +#if !(NGX_HTTP_LUA_HAVE_SIGNALFD) + if (close(ngx_http_lua_write_sigfd) == -1) { + ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, + "lua pipe close the write sigfd failed"); + } +#endif + + return NGX_ERROR; +} + + +static ngx_rbtree_node_t * +ngx_http_lua_pipe_lookup_pid(ngx_rbtree_key_t key) +{ + ngx_rbtree_node_t *node, *sentinel; + + node = ngx_http_lua_pipe_rbtree.root; + sentinel = ngx_http_lua_pipe_rbtree.sentinel; + + while (node != sentinel) { + if (key < node->key) { + node = node->left; + continue; + } + + if (key > node->key) { + node = node->right; + continue; + } + + return node; + } + + return NULL; +} + + +#if !(NGX_HTTP_LUA_HAVE_SIGNALFD) +static void +ngx_http_lua_pipe_sigchld_handler(int signo, siginfo_t *siginfo, + void *ucontext) +{ + ngx_err_t err, saved_err; + ngx_int_t n; + + saved_err = ngx_errno; + + for ( ;; ) { + n = write(ngx_http_lua_write_sigfd, ngx_http_lua_pipe_notification, + sizeof(ngx_http_lua_pipe_notification)); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ngx_cycle->log, 0, + "lua pipe SIGCHLD fd write siginfo:%p", siginfo); + + if (n >= 0) { + break; + } + + err = ngx_errno; + + if (err != NGX_EINTR) { + if (err != NGX_EAGAIN) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, err, + "lua pipe SIGCHLD fd write failed"); + } + + break; + } + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ngx_cycle->log, err, + "lua pipe SIGCHLD fd write was interrupted"); + } + + ngx_set_errno(saved_err); +} +#endif + + +static void +ngx_http_lua_pipe_sigchld_event_handler(ngx_event_t *ev) +{ + int n; + int status; + ngx_pid_t pid; + ngx_connection_t *c = ev->data; + ngx_rbtree_node_t *node; + ngx_http_lua_pipe_node_t *pipe_node; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ngx_cycle->log, 0, + "lua pipe reaping children"); + + for ( ;; ) { +#if (NGX_HTTP_LUA_HAVE_SIGNALFD) + n = c->recv(c, (u_char *) &ngx_http_lua_pipe_notification, +#else + n = c->recv(c, ngx_http_lua_pipe_notification, +#endif + sizeof(ngx_http_lua_pipe_notification)); + + if (n <= 0) { + if (n == NGX_ERROR) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe SIGCHLD fd read failed"); + } + + break; + } + + for ( ;; ) { + pid = waitpid(-1, &status, WNOHANG); + + if (pid == 0) { + break; + } + + if (pid < 0) { + if (ngx_errno != NGX_ECHILD) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe waitpid failed"); + } + + break; + } + + /* This log is ported from Nginx's signal handler since we override + * or block it in this implementation. */ + ngx_log_error(NGX_LOG_NOTICE, ngx_cycle->log, 0, + "signal %d (SIGCHLD) received from %P", + SIGCHLD, pid); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe SIGCHLD fd read pid:%P status:%d", pid, + status); + + node = ngx_http_lua_pipe_lookup_pid(pid); + if (node != NULL) { + pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; + if (pipe_node->wait_co_ctx != NULL) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe resume process:%p waiting for %P", + pipe_node->proc, pid); + + /* + * We need the extra parentheses around the first argument + * of ngx_post_event() just to work around macro issues in + * nginx cores older than 1.7.12 (exclusive). + */ + ngx_post_event((&pipe_node->wait_co_ctx->sleep), + &ngx_posted_events); + } + + pipe_node->proc->pipe->dead = 1; + + if (WIFSIGNALED(status)) { + pipe_node->status = WTERMSIG(status); + pipe_node->reason_code = REASON_SIGNAL_CODE; + + } else if (WIFEXITED(status)) { + pipe_node->status = WEXITSTATUS(status); + pipe_node->reason_code = REASON_EXIT_CODE; + + } else { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "lua pipe unknown exit status %d from " + "process %P", status, pid); + pipe_node->status = status; + pipe_node->reason_code = REASON_UNKNOWN_CODE; + } + } + } + } +} + + +static ssize_t +ngx_http_lua_pipe_fd_read(ngx_connection_t *c, u_char *buf, size_t size) +{ + ssize_t n; + ngx_err_t err; + ngx_event_t *rev; + + rev = c->read; + + do { + n = read(c->fd, buf, size); + + err = ngx_errno; + + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, + "read: fd:%d %z of %uz", c->fd, n, size); + + if (n == 0) { + rev->ready = 0; + rev->eof = 1; + return 0; + } + + if (n > 0) { + if ((size_t) n < size + && !(ngx_event_flags & NGX_USE_GREEDY_EVENT)) + { + rev->ready = 0; + } + + return n; + } + + if (err == NGX_EAGAIN || err == NGX_EINTR) { + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, err, + "read() not ready"); + n = NGX_AGAIN; + + } else { + n = ngx_connection_error(c, err, "read() failed"); + break; + } + + } while (err == NGX_EINTR); + + rev->ready = 0; + + if (n == NGX_ERROR) { + rev->error = 1; + } + + return n; +} + + +static ssize_t +ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, size_t size) +{ + ssize_t n; + ngx_err_t err; + ngx_event_t *wev; + + wev = c->write; + + do { + n = write(c->fd, buf, size); + + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, + "write: fd:%d %z of %uz", c->fd, n, size); + + if (n >= 0) { + if ((size_t) n != size) { + wev->ready = 0; + } + + return n; + } + + err = ngx_errno; + + if (err == NGX_EAGAIN || err == NGX_EINTR) { + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, err, + "write() not ready"); + n = NGX_AGAIN; + + } else if (err != NGX_EPIPE) { + n = ngx_connection_error(c, err, "write() failed"); + break; + } + + } while (err == NGX_EINTR); + + wev->ready = 0; + + if (n == NGX_ERROR) { + wev->error = 1; + } + + return n; +} + + +int +ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, + const char *file, const char **argv, int merge_stderr, size_t buffer_size, + u_char *errbuf, size_t *errbuf_size) +{ + int rc; + int in[2]; + int out[2]; + int err[2]; + int stdin_fd, stdout_fd, stderr_fd; + int errlog_fd, temp_errlog_fd; + ngx_pid_t pid; + ssize_t pool_size; + ngx_pool_t *pool; + ngx_uint_t i; + ngx_listening_t *ls; + ngx_http_lua_pipe_t *pp; + ngx_rbtree_node_t *node; + ngx_http_lua_pipe_node_t *pipe_node; + struct sigaction sa; + ngx_http_lua_pipe_signal_t *sig; + sigset_t set; + + pool_size = ngx_align(NGX_MIN_POOL_SIZE + buffer_size * 2, + NGX_POOL_ALIGNMENT); + + pool = ngx_create_pool(pool_size, ngx_cycle->log); + if (pool == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no memory") + - errbuf; + return NGX_ERROR; + } + + pp = ngx_pcalloc(pool, sizeof(ngx_http_lua_pipe_t) + + offsetof(ngx_rbtree_node_t, color) + + sizeof(ngx_http_lua_pipe_node_t)); + if (pp == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no memory") + - errbuf; + goto free_pool; + } + + rc = pipe(in); + if (rc == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe failed: %s", + strerror(errno)) + - errbuf; + goto free_pool; + } + + rc = pipe(out); + if (rc == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe failed: %s", + strerror(errno)) + - errbuf; + goto close_in_fd; + } + + if (!merge_stderr) { + rc = pipe(err); + if (rc == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + "pipe failed: %s", strerror(errno)) + - errbuf; + goto close_in_out_fd; + } + } + + pid = fork(); + if (pid == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "fork failed: %s", + strerror(errno)) + - errbuf; + goto close_in_out_err_fd; + } + + if (pid == 0) { + +#if (NGX_HAVE_CPU_AFFINITY) + /* reset the CPU affinity mask */ + ngx_uint_t log_level; + ngx_cpuset_t child_cpu_affinity; + + if (ngx_process == NGX_PROCESS_WORKER + && ngx_get_cpu_affinity(ngx_worker) != NULL) + { + CPU_ZERO(&child_cpu_affinity); + + for (i = 0; i < (ngx_uint_t) ngx_min(ngx_ncpu, CPU_SETSIZE); i++) { + CPU_SET(i, &child_cpu_affinity); + } + + log_level = ngx_cycle->log->log_level; + ngx_cycle->log->log_level = NGX_LOG_WARN; + ngx_setaffinity(&child_cpu_affinity, ngx_cycle->log); + ngx_cycle->log->log_level = log_level; + } +#endif + + /* reset the handler of ignored signals to the default */ + for (sig = ngx_signals; sig->signo != 0; sig++) { + ngx_memzero(&sa, sizeof(struct sigaction)); + sa.sa_handler = SIG_DFL; + + if (sigemptyset(&sa.sa_mask) != 0) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child init signal mask failed"); + exit(EXIT_FAILURE); + } + + if (sigaction(sig->signo, &sa, NULL) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child reset signal handler for %s " + "failed", sig->signame); + exit(EXIT_FAILURE); + } + } + + /* reset signal mask */ + if (sigemptyset(&set) != 0) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child init signal set failed"); + exit(EXIT_FAILURE); + } + + if (sigprocmask(SIG_SETMASK, &set, NULL) != 0) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child reset signal mask failed"); + exit(EXIT_FAILURE); + } + + /* close listening socket fd */ + ls = ngx_cycle->listening.elts; + for (i = 0; i < ngx_cycle->listening.nelts; i++) { + if (ngx_close_socket(ls[i].fd) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_socket_errno, + "lua pipe child " ngx_close_socket_n + " %V failed", &ls[i].addr_text); + } + } + + /* close and dup pipefd */ + if (close(in[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe child failed to close the in[1] " + "pipe fd"); + } + + if (close(out[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe child failed to close the out[0] " + "pipe fd"); + } + + errlog_fd = ngx_cycle->log->file->fd; + if (errlog_fd == STDERR_FILENO) { + temp_errlog_fd = dup(errlog_fd); + + if (temp_errlog_fd == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child dup errlog fd failed"); + exit(EXIT_FAILURE); + } + + if (ngx_cloexec(temp_errlog_fd) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child new errlog fd " ngx_cloexec_n + " failed"); + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe child dup old errlog fd %d to new fd %d", + ngx_cycle->log->file->fd, temp_errlog_fd); + + ngx_cycle->log->file->fd = temp_errlog_fd; + } + + if (dup2(in[0], STDIN_FILENO) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child dup2 stdin failed"); + exit(EXIT_FAILURE); + } + + if (dup2(out[1], STDOUT_FILENO) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child dup2 stdout failed"); + exit(EXIT_FAILURE); + } + + if (merge_stderr) { + if (dup2(STDOUT_FILENO, STDERR_FILENO) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child dup2 stderr failed"); + exit(EXIT_FAILURE); + } + + } else { + if (close(err[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe child failed to close the err[0] " + "pipe fd"); + } + + if (dup2(err[1], STDERR_FILENO) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child dup2 stderr failed"); + exit(EXIT_FAILURE); + } + } + + if (execvp(file, (char * const *) argv) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child execvp() failed while executing %s", + file); + } + + exit(EXIT_FAILURE); + } + + /* parent process */ + if (close(in[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe: failed to close the in[0] pipe fd"); + } + + stdin_fd = in[1]; + + if (ngx_nonblocking(stdin_fd) == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + ngx_nonblocking_n " failed: %s", + strerror(errno)) + - errbuf; + goto close_in_out_err_fd; + } + + pp->stdin_fd = stdin_fd; + + if (close(out[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe: failed to close the out[1] pipe fd"); + } + + stdout_fd = out[0]; + + if (ngx_nonblocking(stdout_fd) == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + ngx_nonblocking_n " failed: %s", + strerror(errno)) + - errbuf; + goto close_in_out_err_fd; + } + + pp->stdout_fd = stdout_fd; + + if (!merge_stderr) { + if (close(err[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe: failed to close the err[1] pipe fd"); + } + + stderr_fd = err[0]; + + if (ngx_nonblocking(stderr_fd) == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + ngx_nonblocking_n " failed: %s", + strerror(errno)) + - errbuf; + goto close_in_out_err_fd; + } + + pp->stderr_fd = stderr_fd; + } + + node = (ngx_rbtree_node_t *) (pp + 1); + node->key = pid; + pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; + pipe_node->proc = proc; + ngx_rbtree_insert(&ngx_http_lua_pipe_rbtree, node); + + pp->node = node; + pp->pool = pool; + pp->merge_stderr = merge_stderr; + pp->buffer_size = buffer_size; + + proc->_pid = pid; + proc->write_timeout = 10000; + proc->stdout_read_timeout = 10000; + proc->stderr_read_timeout = 10000; + proc->wait_timeout = 10000; + proc->pipe = pp; + + ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe spawn process:%p pid:%P merge_stderr:%d " + "buffer_size:%uz", proc, pid, merge_stderr, buffer_size); + return NGX_OK; + +close_in_out_err_fd: + + if (!merge_stderr) { + if (close(err[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the err[0] pipe fd"); + } + + if (close(err[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the err[1] pipe fd"); + } + } + +close_in_out_fd: + + if (close(out[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the out[0] pipe fd"); + } + + if (close(out[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the out[1] pipe fd"); + } + +close_in_fd: + + if (close(in[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the in[0] pipe fd"); + } + + if (close(in[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the in[1] pipe fd"); + } + +free_pool: + + ngx_destroy_pool(pool); + return NGX_ERROR; +} + + +static ngx_int_t +ngx_http_lua_pipe_close_helper(ngx_http_lua_pipe_ctx_t *pipe_ctx, + ngx_event_t *ev, int forced) +{ + if (ev->handler != ngx_http_lua_pipe_dummy_event_handler && !forced) { + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe cannot close fd:%d without " + "forced pipe:%p ev:%p", pipe_ctx->c->fd, pipe_ctx, ev); + return NGX_ERROR; + } + + ngx_close_connection(pipe_ctx->c); + pipe_ctx->c = NULL; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe, int forced) +{ + ngx_event_t *wev; + + if (pipe->stdin_ctx == NULL) { + if (pipe->stdin_fd != -1) { + if (close(pipe->stdin_fd) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the stdin pipe fd"); + } + + pipe->stdin_fd = -1; + } + + } else if (pipe->stdin_ctx->c != NULL) { + wev = pipe->stdin_ctx->c->write; + return ngx_http_lua_pipe_close_helper(pipe->stdin_ctx, wev, forced); + } + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe, int forced) +{ + ngx_event_t *rev; + + if (pipe->stdout_ctx == NULL) { + if (pipe->stdout_fd != -1) { + if (close(pipe->stdout_fd) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the stdout pipe fd"); + } + + pipe->stdout_fd = -1; + } + + } else if (pipe->stdout_ctx->c != NULL) { + rev = pipe->stdout_ctx->c->read; + return ngx_http_lua_pipe_close_helper(pipe->stdout_ctx, rev, forced); + } + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe, int forced) +{ + ngx_event_t *rev; + + if (pipe->stderr_ctx == NULL) { + if (pipe->stderr_fd != -1) { + if (close(pipe->stderr_fd) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "failed to close the stderr pipe fd"); + } + + pipe->stderr_fd = -1; + } + + } else if (pipe->stderr_ctx->c != NULL) { + rev = pipe->stderr_ctx->c->read; + return ngx_http_lua_pipe_close_helper(pipe->stderr_ctx, rev, forced); + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_pipe_proc_shutdown_stdin(ngx_http_lua_ffi_pipe_proc_t *proc, + u_char *errbuf, size_t *errbuf_size) +{ + ngx_int_t rc; + ngx_http_lua_pipe_t *pipe; + + pipe = proc->pipe; + if (pipe == NULL || pipe->closed) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + rc = ngx_http_lua_pipe_close_stdin(pipe, 0); + if (rc != NGX_OK) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy writing") + - errbuf; + return NGX_ERROR; + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_pipe_proc_shutdown_stdout(ngx_http_lua_ffi_pipe_proc_t *proc, + u_char *errbuf, size_t *errbuf_size) +{ + ngx_int_t rc; + ngx_http_lua_pipe_t *pipe; + + pipe = proc->pipe; + if (pipe == NULL || pipe->closed) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + rc = ngx_http_lua_pipe_close_stdout(pipe, 0); + if (rc != NGX_OK) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy reading") + - errbuf; + return NGX_ERROR; + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_pipe_proc_shutdown_stderr(ngx_http_lua_ffi_pipe_proc_t *proc, + u_char *errbuf, size_t *errbuf_size) +{ + ngx_http_lua_pipe_t *pipe; + + pipe = proc->pipe; + if (pipe == NULL || pipe->closed) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + if (pipe->merge_stderr) { + /* stdout is used internally as stderr when merge_stderr is true */ + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "merged to stdout") + - errbuf; + return NGX_ERROR; + } + + if (ngx_http_lua_pipe_close_stderr(pipe, 0) != NGX_OK) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy reading") + - errbuf; + return NGX_ERROR; + } + + return NGX_OK; +} + + +static void +ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc, int forced) +{ + ngx_http_lua_pipe_t *pipe; + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe finalize process:%p pid:%P forced:%d", proc, + proc->_pid, forced); + pipe = proc->pipe; + + if (pipe->node) { + ngx_rbtree_delete(&ngx_http_lua_pipe_rbtree, pipe->node); + pipe->node = NULL; + } + + pipe->dead = 1; + + ngx_http_lua_pipe_close_stdin(pipe, forced); + ngx_http_lua_pipe_close_stdout(pipe, forced); + + if (!pipe->merge_stderr) { + ngx_http_lua_pipe_close_stderr(pipe, forced); + } + + pipe->closed = 1; +} + + +void +ngx_http_lua_ffi_pipe_proc_destroy(ngx_http_lua_ffi_pipe_proc_t *proc) +{ + ngx_http_lua_pipe_t *pipe; + + pipe = proc->pipe; + if (pipe == NULL) { + return; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe destroy process:%p pid:%P", proc, proc->_pid); + + if (!pipe->dead) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe kill process:%p pid:%P", proc, proc->_pid); + + if (kill(proc->_pid, SIGKILL) == -1) { + if (ngx_errno != ESRCH) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe failed to kill process:%p pid:%P"); + } + } + } + + ngx_http_lua_pipe_proc_finalize(proc, 1); + ngx_destroy_pool(pipe->pool); + proc->pipe = NULL; +} + + +static ngx_int_t +ngx_http_lua_pipe_get_lua_ctx(ngx_http_request_t *r, + ngx_http_lua_ctx_t **ctx, u_char *errbuf, size_t *errbuf_size) +{ + int rc; + + *ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return NGX_HTTP_LUA_FFI_NO_REQ_CTX; + } + + rc = ngx_http_lua_ffi_check_context(*ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_ACCESS + | NGX_HTTP_LUA_CONTEXT_CONTENT + | NGX_HTTP_LUA_CONTEXT_TIMER + | NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH, + errbuf, errbuf_size); + if (rc != NGX_OK) { + return NGX_HTTP_LUA_FFI_BAD_CONTEXT; + } + + return NGX_OK; +} + + +static void +ngx_http_lua_pipe_put_error(ngx_http_lua_pipe_ctx_t *pipe_ctx, u_char *errbuf, + size_t *errbuf_size) +{ + switch (pipe_ctx->err_type) { + + case PIPE_ERR_CLOSED: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + break; + + case PIPE_ERR_SYSCALL: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "%s", + strerror(pipe_ctx->pipe_errno)) + - errbuf; + break; + + case PIPE_ERR_NOMEM: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no memory") + - errbuf; + break; + + case PIPE_ERR_TIMEOUT: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "timeout") + - errbuf; + break; + + case PIPE_ERR_ADD_READ_EV: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + "failed to add read event") + - errbuf; + break; + + case PIPE_ERR_ADD_WRITE_EV: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + "failed to add write event") + - errbuf; + break; + + default: + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "unexpected err type: %d", pipe_ctx->err_type); + ngx_http_lua_assert(NULL); + } +} + + +static void +ngx_http_lua_pipe_put_data(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx, u_char **buf, size_t *buf_size) +{ + size_t size = 0; + size_t chunk_size; + size_t nbufs; + u_char *p; + ngx_buf_t *b; + ngx_chain_t *cl; + ngx_chain_t **ll; + + nbufs = 0; + ll = NULL; + + for (cl = pipe_ctx->bufs_in; cl; cl = cl->next) { + b = cl->buf; + chunk_size = b->last - b->pos; + + if (cl->next) { + ll = &cl->next; + } + + size += chunk_size; + + nbufs++; + } + + if (*buf_size < size) { + *buf = NULL; + *buf_size = size; + + return; + } + + *buf_size = size; + + p = *buf; + for (cl = pipe_ctx->bufs_in; cl; cl = cl->next) { + b = cl->buf; + chunk_size = b->last - b->pos; + p = ngx_cpymem(p, b->pos, chunk_size); + } + + if (nbufs > 1 && ll) { + *ll = pipe->free_bufs; + pipe->free_bufs = pipe_ctx->bufs_in; + pipe_ctx->bufs_in = pipe_ctx->buf_in; + } + + if (pipe_ctx->buffer.pos == pipe_ctx->buffer.last) { + pipe_ctx->buffer.pos = pipe_ctx->buffer.start; + pipe_ctx->buffer.last = pipe_ctx->buffer.start; + } + + if (pipe_ctx->bufs_in) { + pipe_ctx->buf_in->buf->last = pipe_ctx->buffer.pos; + pipe_ctx->buf_in->buf->pos = pipe_ctx->buffer.pos; + } +} + + +static ngx_int_t +ngx_http_lua_pipe_add_input_buffer(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx) +{ + ngx_chain_t *cl; + + cl = ngx_http_lua_chain_get_free_buf(ngx_cycle->log, pipe->pool, + &pipe->free_bufs, + pipe->buffer_size); + + if (cl == NULL) { + pipe_ctx->err_type = PIPE_ERR_NOMEM; + return NGX_ERROR; + } + + pipe_ctx->buf_in->next = cl; + pipe_ctx->buf_in = cl; + pipe_ctx->buffer = *cl->buf; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_lua_pipe_read_all(void *data, ssize_t bytes) +{ + ngx_http_lua_pipe_ctx_t *pipe_ctx = data; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua pipe read all"); + return ngx_http_lua_read_all(&pipe_ctx->buffer, pipe_ctx->buf_in, bytes, + ngx_cycle->log); +} + + +static ngx_int_t +ngx_http_lua_pipe_read_bytes(void *data, ssize_t bytes) +{ + ngx_int_t rc; + ngx_http_lua_pipe_ctx_t *pipe_ctx = data; + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe read bytes %z", bytes); + + rc = ngx_http_lua_read_bytes(&pipe_ctx->buffer, pipe_ctx->buf_in, + &pipe_ctx->rest, bytes, ngx_cycle->log); + if (rc == NGX_ERROR) { + pipe_ctx->err_type = PIPE_ERR_CLOSED; + return NGX_ERROR; + } + + return rc; +} + + +static ngx_int_t +ngx_http_lua_pipe_read_line(void *data, ssize_t bytes) +{ + ngx_int_t rc; + ngx_http_lua_pipe_ctx_t *pipe_ctx = data; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe read line"); + rc = ngx_http_lua_read_line(&pipe_ctx->buffer, pipe_ctx->buf_in, bytes, + ngx_cycle->log); + if (rc == NGX_ERROR) { + pipe_ctx->err_type = PIPE_ERR_CLOSED; + return NGX_ERROR; + } + + return rc; +} + + +static ngx_int_t +ngx_http_lua_pipe_read_any(void *data, ssize_t bytes) +{ + ngx_int_t rc; + ngx_http_lua_pipe_ctx_t *pipe_ctx = data; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua pipe read any"); + rc = ngx_http_lua_read_any(&pipe_ctx->buffer, pipe_ctx->buf_in, + &pipe_ctx->rest, bytes, ngx_cycle->log); + if (rc == NGX_ERROR) { + pipe_ctx->err_type = PIPE_ERR_CLOSED; + return NGX_ERROR; + } + + return rc; +} + + +static ngx_int_t +ngx_http_lua_pipe_read(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx) +{ + int rc; + int read; + size_t size; + ssize_t n; + ngx_buf_t *b; + ngx_event_t *rev; + ngx_connection_t *c; + + c = pipe_ctx->c; + rev = c->read; + b = &pipe_ctx->buffer; + read = 0; + + for ( ;; ) { + size = b->last - b->pos; + + if (size || pipe_ctx->eof) { + rc = pipe_ctx->input_filter(pipe_ctx->input_filter_ctx, size); + if (rc == NGX_ERROR) { + return NGX_ERROR; + } + + if (rc == NGX_OK) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe read done pipe:%p", pipe_ctx); + return NGX_OK; + } + + /* rc == NGX_AGAIN */ + continue; + } + + if (read && !rev->ready) { + break; + } + + size = b->end - b->last; + + if (size == 0) { + rc = ngx_http_lua_pipe_add_input_buffer(pipe, pipe_ctx); + if (rc == NGX_ERROR) { + return NGX_ERROR; + } + + b = &pipe_ctx->buffer; + size = (size_t) (b->end - b->last); + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe try to read data %uz pipe:%p", + size, pipe_ctx); + + n = c->recv(c, b->last, size); + read = 1; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe read data returned %z pipe:%p", n, pipe_ctx); + + if (n == NGX_AGAIN) { + break; + } + + if (n == 0) { + pipe_ctx->eof = 1; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe closed pipe:%p", pipe_ctx); + continue; + } + + if (n == NGX_ERROR) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, ngx_errno, + "lua pipe read data error pipe:%p", pipe_ctx); + + pipe_ctx->err_type = PIPE_ERR_SYSCALL; + pipe_ctx->pipe_errno = ngx_errno; + return NGX_ERROR; + } + + b->last += n; + } + + return NGX_AGAIN; +} + + +static ngx_int_t +ngx_http_lua_pipe_init_ctx(ngx_http_lua_pipe_ctx_t **pipe_ctx_pt, int fd, + ngx_pool_t *pool, u_char *errbuf, size_t *errbuf_size) +{ + ngx_connection_t *c; + + if (fd == -1) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + *pipe_ctx_pt = ngx_pcalloc(pool, sizeof(ngx_http_lua_pipe_ctx_t)); + if (*pipe_ctx_pt == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no memory") + - errbuf; + return NGX_ERROR; + } + + c = ngx_get_connection(fd, ngx_cycle->log); + if (c == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no connection") + - errbuf; + return NGX_ERROR; + } + + c->log = ngx_cycle->log; + c->recv = ngx_http_lua_pipe_fd_read; + c->read->handler = ngx_http_lua_pipe_dummy_event_handler; + c->read->log = c->log; + +#ifdef HAVE_SOCKET_CLOEXEC_PATCH + c->read->skip_socket_leak_check = 1; +#endif + + c->send = ngx_http_lua_pipe_fd_write; + c->write->handler = ngx_http_lua_pipe_dummy_event_handler; + c->write->log = c->log; + (*pipe_ctx_pt)->c = c; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe init pipe ctx:%p fd:*%d", *pipe_ctx_pt, fd); + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_pipe_proc_read(ngx_http_request_t *r, + ngx_http_lua_ffi_pipe_proc_t *proc, int from_stderr, int reader_type, + size_t length, u_char **buf, size_t *buf_size, u_char *errbuf, + size_t *errbuf_size) +{ + int rc; + ngx_msec_t timeout; + ngx_event_t *rev; + ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_pipe_ctx_t *pipe_ctx; + + rc = ngx_http_lua_pipe_get_lua_ctx(r, &ctx, errbuf, errbuf_size); + if (rc != NGX_OK) { + return rc; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe read process:%p pid:%P", proc, proc->_pid); + + pipe = proc->pipe; + if (pipe == NULL || pipe->closed) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + if (pipe->merge_stderr && from_stderr) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "merged to stdout") + - errbuf; + return NGX_ERROR; + } + + if (from_stderr) { + if (pipe->stderr_ctx == NULL) { + if (ngx_http_lua_pipe_init_ctx(&pipe->stderr_ctx, pipe->stderr_fd, + pipe->pool, errbuf, + errbuf_size) + != NGX_OK) + { + return NGX_ERROR; + } + + } else { + pipe->stderr_ctx->err_type = 0; + } + + pipe_ctx = pipe->stderr_ctx; + + } else { + if (pipe->stdout_ctx == NULL) { + if (ngx_http_lua_pipe_init_ctx(&pipe->stdout_ctx, pipe->stdout_fd, + pipe->pool, errbuf, + errbuf_size) + != NGX_OK) + { + return NGX_ERROR; + } + + } else { + pipe->stdout_ctx->err_type = 0; + } + + pipe_ctx = pipe->stdout_ctx; + } + + c = pipe_ctx->c; + if (c == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + rev = c->read; + if (rev->handler != ngx_http_lua_pipe_dummy_event_handler) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy reading") + - errbuf; + return NGX_ERROR; + } + + pipe_ctx->input_filter_ctx = pipe_ctx; + + switch (reader_type) { + + case PIPE_READ_ALL: + pipe_ctx->input_filter = ngx_http_lua_pipe_read_all; + break; + + case PIPE_READ_BYTES: + pipe_ctx->input_filter = ngx_http_lua_pipe_read_bytes; + break; + + case PIPE_READ_LINE: + pipe_ctx->input_filter = ngx_http_lua_pipe_read_line; + break; + + case PIPE_READ_ANY: + pipe_ctx->input_filter = ngx_http_lua_pipe_read_any; + break; + + default: + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "unexpected reader_type: %d", reader_type); + ngx_http_lua_assert(NULL); + } + + pipe_ctx->rest = length; + + if (pipe_ctx->bufs_in == NULL) { + pipe_ctx->bufs_in = + ngx_http_lua_chain_get_free_buf(ngx_cycle->log, pipe->pool, + &pipe->free_bufs, + pipe->buffer_size); + + if (pipe_ctx->bufs_in == NULL) { + pipe_ctx->err_type = PIPE_ERR_NOMEM; + goto error; + } + + pipe_ctx->buf_in = pipe_ctx->bufs_in; + pipe_ctx->buffer = *pipe_ctx->buf_in->buf; + } + + rc = ngx_http_lua_pipe_read(pipe, pipe_ctx); + if (rc == NGX_ERROR) { + goto error; + } + + if (rc == NGX_OK) { + ngx_http_lua_pipe_put_data(pipe, pipe_ctx, buf, buf_size); + return NGX_OK; + } + + /* rc == NGX_AGAIN */ + wait_co_ctx = ctx->cur_co_ctx; + + c->data = wait_co_ctx; + if (ngx_handle_read_event(rev, 0) != NGX_OK) { + pipe_ctx->err_type = PIPE_ERR_ADD_READ_EV; + goto error; + } + + wait_co_ctx->data = proc; + + if (from_stderr) { + rev->handler = ngx_http_lua_pipe_resume_read_stderr_handler; + wait_co_ctx->cleanup = ngx_http_lua_pipe_proc_read_stderr_cleanup; + timeout = proc->stderr_read_timeout; + + } else { + rev->handler = ngx_http_lua_pipe_resume_read_stdout_handler; + wait_co_ctx->cleanup = ngx_http_lua_pipe_proc_read_stdout_cleanup; + timeout = proc->stdout_read_timeout; + } + + if (timeout > 0) { + ngx_add_timer(rev, timeout); + ngx_log_debug5(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe add timer for reading: %d(ms) process:%p " + "pid:%P pipe:%p ev:%p", timeout, proc, proc->_pid, pipe, + rev); + } + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe read yielding process:%p pid:%P pipe:%p", proc, + proc->_pid, pipe); + + return NGX_AGAIN; + +error: + + if (pipe_ctx->bufs_in) { + ngx_http_lua_pipe_put_data(pipe, pipe_ctx, buf, buf_size); + ngx_http_lua_pipe_put_error(pipe_ctx, errbuf, errbuf_size); + return NGX_DECLINED; + } + + ngx_http_lua_pipe_put_error(pipe_ctx, errbuf, errbuf_size); + + return NGX_ERROR; +} + + +/* + * ngx_http_lua_ffi_pipe_get_read_result should only be called just after + * ngx_http_lua_ffi_pipe_proc_read, so we omit most of the sanity check already + * done in ngx_http_lua_ffi_pipe_proc_read. + */ +int +ngx_http_lua_ffi_pipe_get_read_result(ngx_http_request_t *r, + ngx_http_lua_ffi_pipe_proc_t *proc, int from_stderr, u_char **buf, + size_t *buf_size, u_char *errbuf, size_t *errbuf_size) +{ + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_pipe_ctx_t *pipe_ctx; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe get read result process:%p pid:%P", proc, + proc->_pid); + + pipe = proc->pipe; + pipe_ctx = from_stderr ? pipe->stderr_ctx : pipe->stdout_ctx; + + if (!pipe_ctx->err_type) { + ngx_http_lua_pipe_put_data(pipe, pipe_ctx, buf, buf_size); + return NGX_OK; + } + + if (pipe_ctx->bufs_in) { + ngx_http_lua_pipe_put_data(pipe, pipe_ctx, buf, buf_size); + ngx_http_lua_pipe_put_error(pipe_ctx, errbuf, errbuf_size); + return NGX_DECLINED; + } + + ngx_http_lua_pipe_put_error(pipe_ctx, errbuf, errbuf_size); + + return NGX_ERROR; +} + + +static ngx_int_t +ngx_http_lua_pipe_write(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx) +{ + size_t size; + ngx_int_t n; + ngx_buf_t *b; + ngx_connection_t *c; + + c = pipe_ctx->c; + b = pipe_ctx->buf_in->buf; + + for ( ;; ) { + size = b->last - b->pos; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe try to write data %uz pipe:%p", size, + pipe_ctx); + + n = c->send(c, b->pos, size); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe write returned %i pipe:%p", n, pipe_ctx); + + if (n >= 0) { + b->pos += n; + + if (b->pos == b->last) { + b->pos = b->start; + b->last = b->start; + + if (!pipe->free_bufs) { + pipe->free_bufs = pipe_ctx->buf_in; + + } else { + pipe->free_bufs->next = pipe_ctx->buf_in; + } + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe write done pipe:%p", pipe_ctx); + return NGX_OK; + } + + continue; + } + + /* NGX_ERROR || NGX_AGAIN */ + break; + } + + if (n == NGX_ERROR) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, ngx_errno, + "lua pipe write data error pipe:%p", pipe_ctx); + + if (ngx_errno == NGX_EPIPE) { + pipe_ctx->err_type = PIPE_ERR_CLOSED; + + } else { + pipe_ctx->err_type = PIPE_ERR_SYSCALL; + pipe_ctx->pipe_errno = ngx_errno; + } + + return NGX_ERROR; + } + + return NGX_AGAIN; +} + + +ssize_t +ngx_http_lua_ffi_pipe_proc_write(ngx_http_request_t *r, + ngx_http_lua_ffi_pipe_proc_t *proc, const u_char *data, size_t len, + u_char *errbuf, size_t *errbuf_size) +{ + int rc; + ngx_buf_t *b; + ngx_msec_t timeout; + ngx_chain_t *cl; + ngx_event_t *wev; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_pipe_ctx_t *pipe_ctx; + + rc = ngx_http_lua_pipe_get_lua_ctx(r, &ctx, errbuf, errbuf_size); + if (rc != NGX_OK) { + return rc; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe write process:%p pid:%P", proc, proc->_pid); + + pipe = proc->pipe; + if (pipe == NULL || pipe->closed) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + if (pipe->stdin_ctx == NULL) { + if (ngx_http_lua_pipe_init_ctx(&pipe->stdin_ctx, pipe->stdin_fd, + pipe->pool, errbuf, + errbuf_size) + != NGX_OK) + { + return NGX_ERROR; + } + + } else { + pipe->stdin_ctx->err_type = 0; + } + + pipe_ctx = pipe->stdin_ctx; + if (pipe_ctx->c == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "closed") - errbuf; + return NGX_ERROR; + } + + wev = pipe_ctx->c->write; + if (wev->handler != ngx_http_lua_pipe_dummy_event_handler) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy writing") + - errbuf; + return NGX_ERROR; + } + + pipe_ctx->rest = len; + + cl = ngx_http_lua_chain_get_free_buf(ngx_cycle->log, pipe->pool, + &pipe->free_bufs, len); + if (cl == NULL) { + pipe_ctx->err_type = PIPE_ERR_NOMEM; + goto error; + } + + pipe_ctx->buf_in = cl; + b = pipe_ctx->buf_in->buf; + b->last = ngx_copy(b->last, data, len); + + rc = ngx_http_lua_pipe_write(pipe, pipe_ctx); + if (rc == NGX_ERROR) { + goto error; + } + + if (rc == NGX_OK) { + return len; + } + + /* rc == NGX_AGAIN */ + wait_co_ctx = ctx->cur_co_ctx; + pipe_ctx->c->data = wait_co_ctx; + + wev->handler = ngx_http_lua_pipe_resume_write_handler; + if (ngx_handle_write_event(wev, 0) != NGX_OK) { + pipe_ctx->err_type = PIPE_ERR_ADD_WRITE_EV; + goto error; + } + + wait_co_ctx->data = proc; + wait_co_ctx->cleanup = ngx_http_lua_pipe_proc_write_cleanup; + timeout = proc->write_timeout; + + if (timeout > 0) { + ngx_add_timer(wev, timeout); + ngx_log_debug5(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe add timer for writing: %d(ms) process:%p " + "pid:%P pipe:%p ev:%p", timeout, proc, proc->_pid, pipe, + wev); + } + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe write yielding process:%p pid:%P pipe:%p", proc, + proc->_pid, pipe); + + return NGX_AGAIN; + +error: + + ngx_http_lua_pipe_put_error(pipe_ctx, errbuf, errbuf_size); + return NGX_ERROR; +} + + +/* + * ngx_http_lua_ffi_pipe_get_write_result should only be called just after + * ngx_http_lua_ffi_pipe_proc_write, so we omit most of the sanity check + * already done in ngx_http_lua_ffi_pipe_proc_write. + */ +ssize_t +ngx_http_lua_ffi_pipe_get_write_result(ngx_http_request_t *r, + ngx_http_lua_ffi_pipe_proc_t *proc, u_char *errbuf, size_t *errbuf_size) +{ + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_pipe_ctx_t *pipe_ctx; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe get write result process:%p pid:%P", proc, + proc->_pid); + + pipe = proc->pipe; + pipe_ctx = pipe->stdin_ctx; + + if (pipe_ctx->err_type) { + ngx_http_lua_pipe_put_error(pipe_ctx, errbuf, errbuf_size); + return NGX_ERROR; + } + + return pipe_ctx->rest; +} + + +int +ngx_http_lua_ffi_pipe_proc_wait(ngx_http_request_t *r, + ngx_http_lua_ffi_pipe_proc_t *proc, char **reason, int *status, + u_char *errbuf, size_t *errbuf_size) +{ + int rc; + ngx_rbtree_node_t *node; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_pipe_node_t *pipe_node; + + rc = ngx_http_lua_pipe_get_lua_ctx(r, &ctx, errbuf, errbuf_size); + if (rc != NGX_OK) { + return rc; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe wait process:%p pid:%P", proc, proc->_pid); + + pipe = proc->pipe; + if (pipe == NULL || pipe->closed) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "exited") - errbuf; + return NGX_ERROR; + } + + node = pipe->node; + pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; + if (pipe_node->wait_co_ctx) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy waiting") + - errbuf; + return NGX_ERROR; + } + + if (pipe_node->reason_code == REASON_RUNNING_CODE) { + wait_co_ctx = ctx->cur_co_ctx; + wait_co_ctx->data = proc; + ngx_memzero(&wait_co_ctx->sleep, sizeof(ngx_event_t)); + wait_co_ctx->sleep.handler = ngx_http_lua_pipe_resume_wait_handler; + wait_co_ctx->sleep.data = wait_co_ctx; + wait_co_ctx->sleep.log = r->connection->log; + wait_co_ctx->cleanup = ngx_http_lua_pipe_proc_wait_cleanup; + + pipe_node->wait_co_ctx = wait_co_ctx; + + if (proc->wait_timeout > 0) { + ngx_add_timer(&wait_co_ctx->sleep, proc->wait_timeout); + ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe add timer for waiting: %d(ms) process:%p " + "pid:%P ev:%p", proc->wait_timeout, proc, + proc->_pid, &wait_co_ctx->sleep); + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua pipe wait yielding process:%p pid:%P", proc, + proc->_pid); + + return NGX_AGAIN; + } + + *status = pipe_node->status; + + switch (pipe_node->reason_code) { + + case REASON_EXIT_CODE: + *reason = REASON_EXIT; + break; + + case REASON_SIGNAL_CODE: + *reason = REASON_SIGNAL; + break; + + default: + *reason = REASON_UNKNOWN; + } + + ngx_http_lua_pipe_proc_finalize(proc, 0); + + if (*status == 0) { + return NGX_OK; + } + + return NGX_DECLINED; +} + + +int +ngx_http_lua_ffi_pipe_proc_kill(ngx_http_lua_ffi_pipe_proc_t *proc, int signal, + u_char *errbuf, size_t *errbuf_size) +{ + ngx_pid_t pid; + ngx_http_lua_pipe_t *pipe; + + pipe = proc->pipe; + + if (pipe == NULL || pipe->dead) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "exited") - errbuf; + return NGX_ERROR; + } + + pid = proc->_pid; + + if (kill(pid, signal) == -1) { + switch (ngx_errno) { + case EINVAL: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "invalid signal") + - errbuf; + break; + + case ESRCH: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "exited") + - errbuf; + break; + + default: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "%s", + strerror(ngx_errno)) + - errbuf; + } + + return NGX_ERROR; + } + + return NGX_OK; +} + + +static int +ngx_http_lua_pipe_read_stdout_retval(ngx_http_lua_ffi_pipe_proc_t *proc, + lua_State *L) +{ + return ngx_http_lua_pipe_read_retval_helper(proc, L, 0); +} + + +static int +ngx_http_lua_pipe_read_stderr_retval(ngx_http_lua_ffi_pipe_proc_t *proc, + lua_State *L) +{ + return ngx_http_lua_pipe_read_retval_helper(proc, L, 1); +} + + +static int +ngx_http_lua_pipe_read_retval_helper(ngx_http_lua_ffi_pipe_proc_t *proc, + lua_State *L, int from_stderr) +{ + int rc; + ngx_msec_t timeout; + ngx_event_t *rev; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_pipe_ctx_t *pipe_ctx; + + pipe = proc->pipe; + if (from_stderr) { + pipe_ctx = pipe->stderr_ctx; + + } else { + pipe_ctx = pipe->stdout_ctx; + } + + if (pipe->timeout) { + pipe->timeout = 0; + pipe_ctx->err_type = PIPE_ERR_TIMEOUT; + return 0; + } + + rc = ngx_http_lua_pipe_read(pipe, pipe_ctx); + if (rc != NGX_AGAIN) { + return 0; + } + + rev = pipe_ctx->c->read; + + if (from_stderr) { + rev->handler = ngx_http_lua_pipe_resume_read_stderr_handler; + timeout = proc->stderr_read_timeout; + + } else { + rev->handler = ngx_http_lua_pipe_resume_read_stdout_handler; + timeout = proc->stdout_read_timeout; + } + + if (timeout > 0) { + ngx_add_timer(rev, timeout); + ngx_log_debug5(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe add timer for reading: %d(ms) proc:%p " + "pid:%P pipe:%p ev:%p", timeout, proc, proc->_pid, pipe, + rev); + } + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe read yielding process:%p pid:%P pipe:%p", proc, + proc->_pid, pipe); + + return NGX_AGAIN; +} + + +static int +ngx_http_lua_pipe_write_retval(ngx_http_lua_ffi_pipe_proc_t *proc, + lua_State *L) +{ + int rc; + ngx_msec_t timeout; + ngx_event_t *wev; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_pipe_ctx_t *pipe_ctx; + + pipe = proc->pipe; + pipe_ctx = pipe->stdin_ctx; + + if (pipe->timeout) { + pipe->timeout = 0; + pipe_ctx->err_type = PIPE_ERR_TIMEOUT; + return 0; + } + + rc = ngx_http_lua_pipe_write(pipe, pipe_ctx); + if (rc != NGX_AGAIN) { + return 0; + } + + wev = pipe_ctx->c->write; + wev->handler = ngx_http_lua_pipe_resume_write_handler; + timeout = proc->write_timeout; + + if (timeout > 0) { + ngx_add_timer(wev, timeout); + ngx_log_debug5(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe add timer for writing: %d(ms) proc:%p " + "pid:%P pipe:%p ev:%p", timeout, proc, proc->_pid, pipe, + wev); + } + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe write yielding process:%p pid:%P pipe:%p", proc, + proc->_pid, pipe); + + return NGX_AGAIN; +} + + +static int +ngx_http_lua_pipe_wait_retval(ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L) +{ + int nret; + ngx_rbtree_node_t *node; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_pipe_node_t *pipe_node; + + pipe = proc->pipe; + node = pipe->node; + pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; + pipe_node->wait_co_ctx = NULL; + + if (pipe->timeout) { + pipe->timeout = 0; + lua_pushboolean(L, 0); + lua_pushliteral(L, "timeout"); + return 2; + } + + ngx_http_lua_pipe_proc_finalize(pipe_node->proc, 0); + + if (pipe_node->status == 0) { + lua_pushboolean(L, 1); + lua_pushliteral(L, REASON_EXIT); + lua_pushinteger(L, pipe_node->status); + nret = 3; + + } else { + lua_pushboolean(L, 0); + + switch (pipe_node->reason_code) { + + case REASON_EXIT_CODE: + lua_pushliteral(L, REASON_EXIT); + break; + + case REASON_SIGNAL_CODE: + lua_pushliteral(L, REASON_SIGNAL); + break; + + default: + lua_pushliteral(L, REASON_UNKNOWN); + } + + lua_pushinteger(L, pipe_node->status); + nret = 3; + } + + return nret; +} + + +static void +ngx_http_lua_pipe_resume_helper(ngx_event_t *ev, + ngx_http_lua_co_ctx_t *wait_co_ctx) +{ + ngx_connection_t *c; + ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_ffi_pipe_proc_t *proc; + + if (ev->timedout) { + proc = wait_co_ctx->data; + pipe = proc->pipe; + pipe->timeout = 1; + ev->timedout = 0; + } + + if (ev->timer_set) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ev->log, 0, + "lua pipe del timer for ev:%p", ev); + ngx_del_timer(ev); + } + + if (ev->posted) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ev->log, 0, + "lua pipe del posted event for ev:%p", ev); + ngx_delete_posted_event(ev); + } + + ev->handler = ngx_http_lua_pipe_dummy_event_handler; + + r = ngx_http_lua_get_req(wait_co_ctx->co); + c = r->connection; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + ngx_http_lua_assert(ctx != NULL); + + ctx->cur_co_ctx = wait_co_ctx; + + if (ctx->entered_content_phase) { + (void) ngx_http_lua_pipe_resume(r); + + } else { + ctx->resume_handler = ngx_http_lua_pipe_resume; + ngx_http_core_run_phases(r); + } + + ngx_http_run_posted_requests(c); +} + + +static void +ngx_http_lua_pipe_resume_read_stdout_handler(ngx_event_t *ev) +{ + ngx_connection_t *c = ev->data; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_ffi_pipe_proc_t *proc; + + wait_co_ctx = c->data; + proc = wait_co_ctx->data; + pipe = proc->pipe; + pipe->retval_handler = ngx_http_lua_pipe_read_stdout_retval; + ngx_http_lua_pipe_resume_helper(ev, wait_co_ctx); +} + + +static void +ngx_http_lua_pipe_resume_read_stderr_handler(ngx_event_t *ev) +{ + ngx_connection_t *c = ev->data; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_ffi_pipe_proc_t *proc; + + wait_co_ctx = c->data; + proc = wait_co_ctx->data; + pipe = proc->pipe; + pipe->retval_handler = ngx_http_lua_pipe_read_stderr_retval; + ngx_http_lua_pipe_resume_helper(ev, wait_co_ctx); +} + + +static void +ngx_http_lua_pipe_resume_write_handler(ngx_event_t *ev) +{ + ngx_connection_t *c = ev->data; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_ffi_pipe_proc_t *proc; + + wait_co_ctx = c->data; + proc = wait_co_ctx->data; + pipe = proc->pipe; + pipe->retval_handler = ngx_http_lua_pipe_write_retval; + ngx_http_lua_pipe_resume_helper(ev, wait_co_ctx); +} + + +static void +ngx_http_lua_pipe_resume_wait_handler(ngx_event_t *ev) +{ + ngx_http_lua_co_ctx_t *wait_co_ctx = ev->data; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_ffi_pipe_proc_t *proc; + + proc = wait_co_ctx->data; + pipe = proc->pipe; + pipe->retval_handler = ngx_http_lua_pipe_wait_retval; + ngx_http_lua_pipe_resume_helper(ev, wait_co_ctx); +} + + +static ngx_int_t +ngx_http_lua_pipe_resume(ngx_http_request_t *r) +{ + int nret; + lua_State *vm; + ngx_int_t rc; + ngx_uint_t nreqs; + ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_pipe_t *pipe; + ngx_http_lua_ffi_pipe_proc_t *proc; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return NGX_ERROR; + } + + ctx->resume_handler = ngx_http_lua_wev_handler; + ctx->cur_co_ctx->cleanup = NULL; + + proc = ctx->cur_co_ctx->data; + pipe = proc->pipe; + nret = pipe->retval_handler(proc, ctx->cur_co_ctx->co); + if (nret == NGX_AGAIN) { + return NGX_DONE; + } + + c = r->connection; + vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; + + rc = ngx_http_lua_run_thread(vm, r, ctx, nret); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua run thread returned %d", rc); + + if (rc == NGX_AGAIN) { + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (rc == NGX_DONE) { + ngx_http_lua_finalize_request(r, NGX_DONE); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + /* rc == NGX_ERROR || rc >= NGX_OK */ + + if (ctx->entered_content_phase) { + ngx_http_lua_finalize_request(r, rc); + return NGX_DONE; + } + + return rc; +} + + +static void +ngx_http_lua_pipe_dummy_event_handler(ngx_event_t *ev) +{ + /* do nothing */ +} + + +static void +ngx_http_lua_pipe_cleanup_helper(ngx_http_lua_co_ctx_t *wait_co_ctx) +{ + if (wait_co_ctx->sleep.timer_set) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, wait_co_ctx->sleep.log, 0, + "lua pipe del timer for ev:%p", &wait_co_ctx->sleep); + ngx_del_timer(&wait_co_ctx->sleep); + } + + wait_co_ctx->cleanup = NULL; +} + + +static void +ngx_http_lua_pipe_proc_read_stdout_cleanup(void *data) +{ + ngx_event_t *rev; + ngx_connection_t *c; + ngx_http_lua_co_ctx_t *wait_co_ctx = data; + ngx_http_lua_ffi_pipe_proc_t *proc; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe proc read stdout cleanup"); + + proc = wait_co_ctx->data; + c = proc->pipe->stdout_ctx->c; + if (c) { + rev = c->read; + rev->handler = ngx_http_lua_pipe_dummy_event_handler; + } + + ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); +} + + +static void +ngx_http_lua_pipe_proc_read_stderr_cleanup(void *data) +{ + ngx_event_t *rev; + ngx_connection_t *c; + ngx_http_lua_co_ctx_t *wait_co_ctx = data; + ngx_http_lua_ffi_pipe_proc_t *proc; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe proc read stderr cleanup"); + + proc = wait_co_ctx->data; + c = proc->pipe->stderr_ctx->c; + if (c) { + rev = c->read; + rev->handler = ngx_http_lua_pipe_dummy_event_handler; + } + + ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); +} + + +static void +ngx_http_lua_pipe_proc_write_cleanup(void *data) +{ + ngx_event_t *wev; + ngx_connection_t *c; + ngx_http_lua_co_ctx_t *wait_co_ctx = data; + ngx_http_lua_ffi_pipe_proc_t *proc; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe proc write cleanup"); + + proc = wait_co_ctx->data; + c = proc->pipe->stdin_ctx->c; + if (c) { + wev = c->write; + wev->handler = ngx_http_lua_pipe_dummy_event_handler; + } + + ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); +} + + +static void +ngx_http_lua_pipe_proc_wait_cleanup(void *data) +{ + ngx_rbtree_node_t *node; + ngx_http_lua_co_ctx_t *wait_co_ctx = data; + ngx_http_lua_pipe_node_t *pipe_node; + ngx_http_lua_ffi_pipe_proc_t *proc; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe proc wait cleanup"); + + proc = wait_co_ctx->data; + node = proc->pipe->node; + pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; + pipe_node->wait_co_ctx = NULL; + + if (wait_co_ctx->sleep.posted) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe del posted event for ev:%p", + &wait_co_ctx->sleep); + ngx_delete_posted_event(&wait_co_ctx->sleep); + } + + ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); +} + + +#endif /* HAVE_NGX_LUA_PIPE */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_pipe.h b/src/ngx_http_lua_pipe.h new file mode 100644 index 0000000000..b5cea89320 --- /dev/null +++ b/src/ngx_http_lua_pipe.h @@ -0,0 +1,95 @@ + +/* + * Copyright (C) by OpenResty Inc. + */ + + +#ifndef _NGX_HTTP_LUA_PIPE_H_INCLUDED_ +#define _NGX_HTTP_LUA_PIPE_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +typedef ngx_int_t (*ngx_http_lua_pipe_input_filter)(void *data, ssize_t bytes); + + +typedef struct { + ngx_connection_t *c; + ngx_http_lua_pipe_input_filter input_filter; + void *input_filter_ctx; + size_t rest; + ngx_chain_t *buf_in; + ngx_chain_t *bufs_in; + ngx_buf_t buffer; + ngx_err_t pipe_errno; + unsigned err_type:16; + unsigned eof:1; +} ngx_http_lua_pipe_ctx_t; + + +typedef struct ngx_http_lua_pipe_s ngx_http_lua_pipe_t; + + +typedef struct { + ngx_pid_t _pid; + ngx_msec_t write_timeout; + ngx_msec_t stdout_read_timeout; + ngx_msec_t stderr_read_timeout; + ngx_msec_t wait_timeout; + /* pipe hides the implementation from the Lua binding */ + ngx_http_lua_pipe_t *pipe; +} ngx_http_lua_ffi_pipe_proc_t; + + +typedef int (*ngx_http_lua_pipe_retval_handler)( + ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L); + + +struct ngx_http_lua_pipe_s { + ngx_pool_t *pool; + ngx_chain_t *free_bufs; + ngx_rbtree_node_t *node; + int stdin_fd; + int stdout_fd; + int stderr_fd; + ngx_http_lua_pipe_ctx_t *stdin_ctx; + ngx_http_lua_pipe_ctx_t *stdout_ctx; + ngx_http_lua_pipe_ctx_t *stderr_ctx; + ngx_http_lua_pipe_retval_handler retval_handler; + size_t buffer_size; + unsigned closed:1; + unsigned dead:1; + unsigned timeout:1; + unsigned merge_stderr:1; +}; + + +typedef struct { + u_char color; + u_char reason_code; + int status; + ngx_http_lua_co_ctx_t *wait_co_ctx; + ngx_http_lua_ffi_pipe_proc_t *proc; +} ngx_http_lua_pipe_node_t; + + +typedef struct { + int signo; + char *signame; +} ngx_http_lua_pipe_signal_t; + + +#if !(NGX_WIN32) && !defined(NGX_LUA_NO_FFI_API) \ + && defined(HAVE_SOCKET_CLOEXEC_PATCH) +#define HAVE_NGX_LUA_PIPE 1 + + +void ngx_http_lua_pipe_init(void); +ngx_int_t ngx_http_lua_pipe_add_signal_handler(ngx_cycle_t *cycle); +#endif + + +#endif /* _NGX_HTTP_LUA_PIPE_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From 055bb17c6cf35258f06460d546468c114c591587 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Fri, 1 Feb 2019 06:15:56 +0700 Subject: [PATCH 279/848] bugfix: fixed a segfault introduced in 7c2b58e when testing an nginx configuration containing 'lua_shared_dict' directive(s). In commit 7c2b58e, we avoided running `init_by_lua*` in signaller processes and when testing the Nginx configuration. However, when the Nginx configuration contains `lua_shared_dict` directives, the execution of `init_by_lua*` will be postponded when testing the configuration. This case was not handled and led to the segmentation fault reported in #1462. --- src/ngx_http_lua_api.c | 2 +- t/160-disable-init-by-lua.t | 76 ++++++++++++++++++++++++++++++++----- 2 files changed, 67 insertions(+), 11 deletions(-) diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index 7b590e7a9f..ac014f2638 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -195,7 +195,7 @@ ngx_http_lua_shared_memory_init(ngx_shm_zone_t *shm_zone, void *data) lmcf->shm_zones_inited++; if (lmcf->shm_zones_inited == lmcf->shm_zones->nelts - && lmcf->init_handler) + && lmcf->init_handler && !ngx_test_config) { saved_cycle = ngx_cycle; ngx_cycle = ctx->cycle; diff --git a/t/160-disable-init-by-lua.t b/t/160-disable-init-by-lua.t index c0c040859b..c18f8e254b 100644 --- a/t/160-disable-init-by-lua.t +++ b/t/160-disable-init-by-lua.t @@ -9,17 +9,19 @@ $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); my $html_dir = $ENV{TEST_NGINX_HTML_DIR}; my $http_config = <<_EOC_; init_by_lua_block { - function set_up_ngx_tmp_conf() - local conf = [[ - events { - worker_connections 64; - } - http { - init_by_lua_block { - ngx.log(ngx.ERR, "run init_by_lua") + function set_up_ngx_tmp_conf(conf) + if conf == nil then + conf = [[ + events { + worker_connections 64; } - } - ]] + http { + init_by_lua_block { + ngx.log(ngx.ERR, "run init_by_lua") + } + } + ]] + end assert(os.execute("mkdir -p $html_dir/logs")) @@ -129,3 +131,57 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ } --- no_error_log eval qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ + + + +=== TEST 3: init_by_lua* does not run when testing Nginx configuration which contains 'lua_shared_dict' (GitHub #1462) +--- config + location = /t { + content_by_lua_block { + local conf = [[ + events { + worker_connections 64; + } + http { + lua_shared_dict test 64k; + init_by_lua_block { + ngx.log(ngx.ERR, "run init_by_lua with lua_shared_dict") + } + } + ]] + local conf_file = set_up_ngx_tmp_conf(conf) + local nginx = get_ngx_bin_path() + + local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -t" + local p, err = io.popen(cmd) + if not p then + ngx.log(ngx.ERR, err) + return + end + + local out, err = p:read("*a") + if not out then + ngx.log(ngx.ERR, err) + + else + ngx.log(ngx.WARN, out) + end + + local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -T" + local p, err = io.popen(cmd) + if not p then + ngx.log(ngx.ERR, err) + return + end + + local out, err = p:read("*a") + if not out then + ngx.log(ngx.ERR, err) + + else + ngx.log(ngx.WARN, out) + end + } + } +--- no_error_log eval +qr/\[error\] .*? init_by_lua:\d+: run init_by_lua with lua_shared_dict/ From f627f4c546b3c0c6fcab531901ece207b8b2dfd8 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 31 Jan 2019 16:08:31 -0800 Subject: [PATCH 280/848] tests: t/160-disable-init-by-lua.t: hardened test suite to prevent false positives. The test cases was not testing the result of the `nginx -t/-T/reopen` commands, and was thus not catching the segfault regression fixed by 055bb17, only that the string `run init_by_lua` was not present. We avoid testing the exit code due to incompatibilities between Lua versions in the os.execute/io.popen APIs. --- t/160-disable-init-by-lua.t | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/t/160-disable-init-by-lua.t b/t/160-disable-init-by-lua.t index c18f8e254b..541771e1b5 100644 --- a/t/160-disable-init-by-lua.t +++ b/t/160-disable-init-by-lua.t @@ -2,7 +2,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 2); +plan tests => repeat_each() * (blocks() * 3); $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -57,6 +57,7 @@ add_block_preprocessor(sub { } }); +env_to_nginx("PATH"); log_level("warn"); no_long_string(); run_tests(); @@ -86,6 +87,8 @@ __DATA__ end } } +--- error_log +failed (2: No such file or directory) --- no_error_log eval qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ @@ -129,6 +132,8 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ end } } +--- error_log +test is successful --- no_error_log eval qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ @@ -183,5 +188,7 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ end } } +--- error_log +test is successful --- no_error_log eval qr/\[error\] .*? init_by_lua:\d+: run init_by_lua with lua_shared_dict/ From b2af4d36906dea4404bbc9ca00800900bbbf6eb9 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 4 Feb 2019 17:53:05 -0800 Subject: [PATCH 281/848] bugfix: inlined Lua code snippets in nginx.conf failed to use the Lua source checksum as part of the Lua code cache key. thanks Oleg A. Mamontov for the report in #1428. Fix #1428. --- src/ngx_http_lua_cache.c | 6 +- src/ngx_http_lua_directive.c | 11 +-- t/025-codecache.t | 132 ++++++++++++++++++++++++++++++++++- 3 files changed, 141 insertions(+), 8 deletions(-) diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index 3e0f209f65..5b29527c68 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -156,7 +156,8 @@ ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, n = lua_gettop(L); - dd("XXX cache key: [%s]", cache_key); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, + "looking up Lua code cache with key '%s'", cache_key); rc = ngx_http_lua_cache_load_code(log, L, (char *) cache_key); if (rc == NGX_OK) { @@ -240,7 +241,8 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, dd("CACHE file key already pre-calculated"); } - dd("XXX cache key for file: [%s]", cache_key); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, + "looking up Lua code cache with key '%s'", cache_key); rc = ngx_http_lua_cache_load_code(log, L, (char *) cache_key); if (rc == NGX_OK) { diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index fb8c6dc5bf..a989c2671e 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1304,11 +1304,12 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, found: - ngx_snprintf(out, len, "=%*s(%*s:%d)%Z", - tag_len, tag, cf->conf_file->file.name.data - + cf->conf_file->file.name.len - p, - p, cf->conf_file->line); - *chunkname_len = len; + p = ngx_snprintf(out, len, "=%*s(%*s:%d)%Z", + tag_len, tag, cf->conf_file->file.name.data + + cf->conf_file->file.name.len - p, + p, cf->conf_file->line); + + *chunkname_len = p - out - 1; /* exclude the trailing '\0' byte */ return out; } diff --git a/t/025-codecache.t b/t/025-codecache.t index 0f6a08fc82..11d53acdbf 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 155; +plan tests => repeat_each() * 163; #$ENV{LUA_PATH} = $ENV{HOME} . '/work/JSON4Lua-0.9.30/json/?.lua'; @@ -1244,3 +1244,133 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, "decrementing the reference count for Lua VM: 1", "lua close the global Lua VM", ] + + + +=== TEST 32: make sure inline code keys are correct +GitHub issue #1428 +--- config +include ../html/a/proxy.conf; +include ../html/b/proxy.conf; +include ../html/c/proxy.conf; + +location /t { + echo_location /a/; + echo_location /b/; + echo_location /a/; + echo_location /c/; +} + +--- user_files +>>> a/proxy.conf +location /a/ { + content_by_lua_block { ngx.say("/a/ is called") } +} + +>>> b/proxy.conf +location /b/ { + content_by_lua_block { ngx.say("/b/ is called") } +} + +>>> c/proxy.conf +location /c/ { + content_by_lua_block { ngx.say("/b/ is called") } +} + +--- request +GET /t +--- response_body +/a/ is called +/b/ is called +/a/ is called +/b/ is called +--- grep_error_log eval: qr/looking up Lua code cache with key '.*?'/ +--- grep_error_log_out eval +[ +"looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +", +"looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' +looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +"] +--- log_level: debug +--- no_error_log +[error] + + + +=== TEST 33: make sure Lua code file keys are correct +GitHub issue #1428 +--- config +include ../html/a/proxy.conf; +include ../html/b/proxy.conf; +include ../html/c/proxy.conf; + +location /t { + echo_location /a/; + echo_location /b/; + echo_location /a/; + echo_location /c/; +} + +--- user_files +>>> a.lua +ngx.say("/a/ is called") + +>>> b.lua +ngx.say("/b/ is called") + +>>> c.lua +ngx.say("/b/ is called") + +>>> a/proxy.conf +location /a/ { + content_by_lua_file html/a.lua; +} + +>>> b/proxy.conf +location /b/ { + content_by_lua_file html/b.lua; +} + +>>> c/proxy.conf +location /c/ { + content_by_lua_file html/c.lua; +} + +--- request +GET /t +--- response_body +/a/ is called +/b/ is called +/a/ is called +/b/ is called +--- grep_error_log eval: qr/looking up Lua code cache with key '.*?'/ +--- grep_error_log_out eval +[ +"looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' +looking up Lua code cache with key 'nhlf_68f5f4e946c3efd1cc206452b807e8b6' +looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' +looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' +", +"looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' +looking up Lua code cache with key 'nhlf_68f5f4e946c3efd1cc206452b807e8b6' +looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' +looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' +looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' +looking up Lua code cache with key 'nhlf_68f5f4e946c3efd1cc206452b807e8b6' +looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' +looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' +" +] +--- log_level: debug +--- no_error_log +[error] From 9750a81731b5ab3fd29881eb1c1e8b0061a96e73 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 7 Feb 2019 11:32:11 -0800 Subject: [PATCH 282/848] tests: fixed new test failure due to openresty.org server changes. --- t/129-ssl-socket.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 287f9b250b..f2fa26b6cc 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -573,7 +573,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: agentzh.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET /en/linux-packages.html HTTP/1.1\\r\\nHost: openresty.com\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -602,7 +602,7 @@ GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 56 bytes. +sent http request: 80 bytes. received: HTTP/1.1 404 Not Found close: 1 nil From 70dd0bee4c69c5dd7f87f848606d94b0f721a7f5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 7 Feb 2019 11:42:20 -0800 Subject: [PATCH 283/848] tests: fixed a test case in 068-socket-keepalive.t which always fails when a local dns server is running. --- t/068-socket-keepalive.t | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index 6e42fac6e6..51e3cd6be6 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -2644,8 +2644,8 @@ lua tcp socket connect timed out, when connecting to === TEST 46: conn queuing: resume connect operation if resumed connect failed (could not be resolved) --- config - resolver 127.0.0.1 ipv6=off; - resolver_timeout 100ms; + resolver agentzh.org:12345 ipv6=off; + resolver_timeout 1s; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2657,7 +2657,7 @@ lua tcp socket connect timed out, when connecting to local sock = ngx.socket.tcp() local ok, err if should_timeout then - sock:settimeouts(100, 3000, 3000) + sock:settimeouts(1, 3000, 3000) ok, err = sock:connect("agentzh.org", 12345, opts) else ok, err = sock:connect("127.0.0.1", port, opts) From 01735c27fe2ffa9185bc3f9b8d0162f6b263717f Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 1 Feb 2019 11:49:20 -0800 Subject: [PATCH 284/848] tests: hardened a few flaky test cases from recent CI runs. --- t/087-udp-socket.t | 2 +- t/108-timer-safe.t | 4 ++-- t/159-sa-restart.t | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index 8bedc320a9..9e7eb2168a 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -555,7 +555,7 @@ lua udp socket read timed out local udp = socket.udp() - udp:settimeout(2000) -- 2 sec + udp:settimeout(5000) -- 5 sec local ok, err = udp:setpeername("$TEST_NGINX_RESOLVER", 53) if not ok then diff --git a/t/108-timer-safe.t b/t/108-timer-safe.t index 30589c7cfc..10687e78a1 100644 --- a/t/108-timer-safe.t +++ b/t/108-timer-safe.t @@ -489,7 +489,7 @@ delete thread 2 --- response_body hello world ---- wait: 0.1 +--- wait: 0.15 --- no_error_log [error] [alert] @@ -499,7 +499,7 @@ hello world [ "registered timer", qr/\[lua\] .*? my lua timer handler/, -qr/\[lua\] log_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-6])/, +qr/\[lua\] log_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-9]|8[0-6])/, "lua ngx.timer expired", "http lua close fake http connection" ] diff --git a/t/159-sa-restart.t b/t/159-sa-restart.t index 0fc63145d5..912338df95 100644 --- a/t/159-sa-restart.t +++ b/t/159-sa-restart.t @@ -11,7 +11,7 @@ add_block_preprocessor(sub { init_by_lua_block { function test_sa_restart() local signals = { - "HUP", + --"HUP", --"INFO", --"XCPU", --"USR1", From cf887654a2ac36b6747c54a11f50ca56f88d6dbd Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 7 Feb 2019 15:44:38 -0800 Subject: [PATCH 285/848] tests: switched to 127.0.0.2 instead of agentzh.org for timeout-related tests. --- .travis.yml | 2 ++ t/023-rewrite/client-abort.t | 4 ++-- t/023-rewrite/tcp-socket-timeout.t | 12 ++++++------ t/023-rewrite/tcp-socket.t | 4 ++-- t/023-rewrite/uthread-exit.t | 4 ++-- t/024-access/client-abort.t | 4 ++-- t/024-access/uthread-exit.t | 2 +- t/027-multi-capture.t | 2 +- t/030-uri-args.t | 4 ++-- t/058-tcp-socket.t | 6 +++--- t/065-tcp-socket-timeout.t | 22 +++++++++++----------- t/068-socket-keepalive.t | 8 ++++---- t/090-log-socket-errors.t | 10 +++++----- t/094-uthread-exit.t | 2 +- t/100-client-abort.t | 4 ++-- t/127-uthread-kill.t | 6 +++--- t/128-duplex-tcp-socket.t | 6 +++--- t/147-tcp-socket-timeouts.t | 4 ++-- 18 files changed, 54 insertions(+), 52 deletions(-) diff --git a/.travis.yml b/.travis.yml index 589255d5d7..bfb940dde7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -96,6 +96,8 @@ before_script: script: - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT + - sudo iptables -A OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP + - sudo iptables -A OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP - cd luajit2/ - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1) - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1) diff --git a/t/023-rewrite/client-abort.t b/t/023-rewrite/client-abort.t index 19c47872eb..0d6f2b39b3 100644 --- a/t/023-rewrite/client-abort.t +++ b/t/023-rewrite/client-abort.t @@ -199,7 +199,7 @@ bad things happen location = /sub { proxy_ignore_client_abort on; - proxy_pass http://agentzh.org:12345/; + proxy_pass http://127.0.0.2:12345/; } location = /sleep { @@ -240,7 +240,7 @@ client prematurely closed connection location = /sub { proxy_ignore_client_abort off; - proxy_pass http://agentzh.org:12345/; + proxy_pass http://127.0.0.2:12345/; } --- request GET /t diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index 329c04513c..e713bb5755 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -46,7 +46,7 @@ __DATA__ location /t1 { rewrite_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -63,7 +63,7 @@ GET /t1 failed to connect: timeout --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 @@ -79,7 +79,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 rewrite_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(150) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -114,7 +114,7 @@ lua tcp socket connect timeout: 150 rewrite_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(nil) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -149,7 +149,7 @@ lua tcp socket connect timeout: 102 rewrite_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(0) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -185,7 +185,7 @@ lua tcp socket connect timeout: 102 rewrite_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(-1) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index d05d80851d..b81f68e2fa 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -304,7 +304,7 @@ qr/connect\(\) failed \(\d+: Connection refused\)/ location /test { rewrite_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) ngx.say("connect: ", ok, " ", err) local bytes @@ -329,7 +329,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 diff --git a/t/023-rewrite/uthread-exit.t b/t/023-rewrite/uthread-exit.t index 54db17d44c..63e765dce3 100644 --- a/t/023-rewrite/uthread-exit.t +++ b/t/023-rewrite/uthread-exit.t @@ -311,7 +311,7 @@ exiting the user thread ngx.thread.spawn(f) ngx.say("after") local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -521,7 +521,7 @@ after ngx.say("after") local sock = ngx.socket.tcp() sock:settimeout(12000) - local ok, err = sock:connect("172.105.207.225", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/024-access/client-abort.t b/t/024-access/client-abort.t index 299d410311..83bf0a4df1 100644 --- a/t/024-access/client-abort.t +++ b/t/024-access/client-abort.t @@ -200,7 +200,7 @@ bad things happen location = /sub { proxy_ignore_client_abort on; - proxy_pass http://agentzh.org:12345/; + proxy_pass http://127.0.0.2:12345/; } location = /sleep { @@ -241,7 +241,7 @@ client prematurely closed connection location = /sub { proxy_ignore_client_abort off; - proxy_pass http://agentzh.org:12345/; + proxy_pass http://127.0.0.2:12345/; } --- request GET /t diff --git a/t/024-access/uthread-exit.t b/t/024-access/uthread-exit.t index b884e29b5a..57b6ea8208 100644 --- a/t/024-access/uthread-exit.t +++ b/t/024-access/uthread-exit.t @@ -502,7 +502,7 @@ after ngx.say("after") local sock = ngx.socket.tcp() sock:settimeout(12000) - local ok, err = sock:connect("172.105.207.225", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/027-multi-capture.t b/t/027-multi-capture.t index 2087940b5c..9299561687 100644 --- a/t/027-multi-capture.t +++ b/t/027-multi-capture.t @@ -743,7 +743,7 @@ proxy_cache_path conf/cache levels=1:2 keys_zone=STATIC:10m inactive=10m max_siz location = /proxy { proxy_cache STATIC; - proxy_pass http://agentzh.org:12345; + proxy_pass http://127.0.0.2:12345; proxy_cache_key $proxy_host$uri$args; proxy_cache_valid any 1s; #proxy_http_version 1.1; diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 7d63daf8d2..7af63c4c24 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -421,7 +421,7 @@ done ngx.req.set_uri_args("hello") ngx.req.set_uri("/bar", true); '; - proxy_pass http://agentzh.org:12345; + proxy_pass http://127.0.0.2:12345; } --- request GET /foo?world @@ -568,7 +568,7 @@ HTTP/1.0 ca%20t=%25 ngx.req.set_uri("/bar", true); ngx.exit(503) '; - proxy_pass http://agentzh.org:12345; + proxy_pass http://127.0.0.2:12345; } --- request GET /foo?world diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index edebf3267b..38f072673b 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -298,7 +298,7 @@ qr/connect\(\) failed \(\d+: Connection refused\)/ location /test { content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) ngx.say("connect: ", ok, " ", err) local bytes @@ -321,7 +321,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 @@ -3315,7 +3315,7 @@ close: 1 nil local thr = ngx.thread.spawn(function () sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 66ff6459b3..092094a3c9 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -51,7 +51,7 @@ __DATA__ location /t { content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -66,7 +66,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 @@ -81,7 +81,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 content_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(150) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -96,7 +96,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 150 -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 @@ -110,7 +110,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 content_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(nil) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -125,7 +125,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 102 -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 @@ -139,7 +139,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 content_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(0) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -154,7 +154,7 @@ GET /t failed to connect: timeout --- error_log lua tcp socket connect timeout: 102 -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 @@ -168,7 +168,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 content_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(-1) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return @@ -683,7 +683,7 @@ after location /t { content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("1: failed to connect: ", err) @@ -707,7 +707,7 @@ GET /t 2: connected: 1 --- error_log lua tcp socket connect timeout: 100 -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index 51e3cd6be6..85be758f36 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -2005,7 +2005,7 @@ continue to handle cosocket local sock = ngx.socket.tcp() sock:settimeouts(100, 3000, 3000) - local ok, err = sock:connect("agentzh.org", 12345, opts) + local ok, err = sock:connect("127.0.0.2", 12345, opts) if not ok then ngx.say(err) end @@ -2610,7 +2610,7 @@ ok local ok, err if should_timeout then sock:settimeouts(100, 3000, 3000) - ok, err = sock:connect("agentzh.org", 12345, opts) + ok, err = sock:connect("127.0.0.2", 12345, opts) else ok, err = sock:connect("127.0.0.1", port, opts) end @@ -2644,7 +2644,7 @@ lua tcp socket connect timed out, when connecting to === TEST 46: conn queuing: resume connect operation if resumed connect failed (could not be resolved) --- config - resolver agentzh.org:12345 ipv6=off; + resolver 127.0.0.2:12345 ipv6=off; resolver_timeout 1s; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2658,7 +2658,7 @@ lua tcp socket connect timed out, when connecting to local ok, err if should_timeout then sock:settimeouts(1, 3000, 3000) - ok, err = sock:connect("agentzh.org", 12345, opts) + ok, err = sock:connect("agentzh.org", 80, opts) else ok, err = sock:connect("127.0.0.1", port, opts) end diff --git a/t/090-log-socket-errors.t b/t/090-log-socket-errors.t index 16576b6fae..f5a9f80b78 100644 --- a/t/090-log-socket-errors.t +++ b/t/090-log-socket-errors.t @@ -28,7 +28,7 @@ __DATA__ lua_socket_log_errors off; content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) ngx.say(err) '; } @@ -50,7 +50,7 @@ timeout lua_socket_log_errors on; content_by_lua ' local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) ngx.say(err) '; } @@ -59,7 +59,7 @@ GET /t --- response_body timeout --- error_log -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 @@ -72,7 +72,7 @@ lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 lua_socket_read_timeout 1ms; content_by_lua ' local sock = ngx.socket.udp() - local ok, err = sock:setpeername("agentzh.org", 12345) + local ok, err = sock:setpeername("127.0.0.2", 12345) ok, err = sock:receive() ngx.say(err) '; @@ -95,7 +95,7 @@ lua udp socket read timed out lua_socket_read_timeout 1ms; content_by_lua ' local sock = ngx.socket.udp() - local ok, err = sock:setpeername("agentzh.org", 12345) + local ok, err = sock:setpeername("127.0.0.2", 12345) ok, err = sock:receive() ngx.say(err) '; diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index d4e4a7c7b1..032b02a2bf 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -501,7 +501,7 @@ after ngx.say("after") local sock = ngx.socket.tcp() sock:settimeout(12000) - local ok, err = sock:connect("172.105.207.225", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/100-client-abort.t b/t/100-client-abort.t index a5cba319e0..f5713874e7 100644 --- a/t/100-client-abort.t +++ b/t/100-client-abort.t @@ -195,7 +195,7 @@ bad things happen location = /sub { proxy_ignore_client_abort on; - proxy_pass http://agentzh.org:12345/; + proxy_pass http://127.0.0.2:12345/; } location = /sleep { @@ -236,7 +236,7 @@ client prematurely closed connection location = /sub { proxy_ignore_client_abort off; - proxy_pass http://agentzh.org:12345/; + proxy_pass http://127.0.0.2:12345/; } --- request GET /t diff --git a/t/127-uthread-kill.t b/t/127-uthread-kill.t index a919b3d1de..5542153cf2 100644 --- a/t/127-uthread-kill.t +++ b/t/127-uthread-kill.t @@ -138,12 +138,12 @@ lua clean up the timer for pending ngx.sleep === TEST 3: kill pending resolver --- config - resolver agentzh.org:12345; + resolver 127.0.0.2:12345; location /lua { content_by_lua ' local function f() local sock = ngx.socket.tcp() - sock:connect("some.agentzh.org", 12345) + sock:connect("some.127.0.0.2", 12345) end local t, err = ngx.thread.spawn(f) @@ -200,7 +200,7 @@ resolve name done: -2 sock:close() ready = true sock:settimeout(10000) - sock:connect("agentzh.org", 12345) + sock:connect("127.0.0.2", 12345) end local t, err = ngx.thread.spawn(f) diff --git a/t/128-duplex-tcp-socket.t b/t/128-duplex-tcp-socket.t index aed560c18b..22b9f036dd 100644 --- a/t/128-duplex-tcp-socket.t +++ b/t/128-duplex-tcp-socket.t @@ -396,7 +396,7 @@ F(ngx_http_lua_socket_tcp_finalize_write_part) { end sock:settimeout(300) - local ok, err = sock:connect("172.105.207.225", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) ngx.say("connect: ", ok, " ", err) local ok, err = sock:close() @@ -425,7 +425,7 @@ close: nil closed --- config server_tokens off; lua_socket_log_errors off; - resolver agentzh.org:12345; + resolver 127.0.0.2:12345; resolver_timeout 300ms; location /t { content_by_lua ' @@ -459,7 +459,7 @@ close: nil closed end sock:settimeout(300) - local ok, err = sock:connect("some2.agentzh.org", 12345) + local ok, err = sock:connect("some2.agentzh.org", 80) ngx.say("connect: ", ok, " ", err) local ok, err = sock:close() diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 2fa3f4e3c0..8566c0ba6f 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -323,7 +323,7 @@ lua tcp socket write timed out sock:settimeouts(100, 100, 100) - local ok, err = sock:connect("agentzh.org", 12345) + local ok, err = sock:connect("127.0.0.2", 12345) ngx.say("connect: ", ok, " ", err) local bytes @@ -346,7 +346,7 @@ send: nil closed receive: nil closed close: nil closed --- error_log -lua tcp socket connect timed out, when connecting to 172.105.207.225:12345 +lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 --- timeout: 10 From 4ebab2b8a3cbf6d79eeced86fa2ae1497f0d12e6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 8 Feb 2019 12:17:56 -0800 Subject: [PATCH 286/848] doc: emphasized that this module is a core component of OpenResty. --- README.markdown | 4 ++++ doc/HttpLuaModule.wiki | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/README.markdown b/README.markdown index adeadcdb53..c21894a194 100644 --- a/README.markdown +++ b/README.markdown @@ -10,6 +10,8 @@ ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. *This module is not distributed with the Nginx source.* See [the installation instructions](#installation). +This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) + Table of Contents ================= @@ -186,6 +188,8 @@ Synopsis Description =========== +This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) + This module embeds Lua, via [LuaJIT 2.0/2.1](http://luajit.org/luajit.html), into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. Unlike [Apache's mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html) and [Lighttpd's mod_magnet](http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet), Lua code executed using this module can be *100% non-blocking* on network traffic as long as the [Nginx API for Lua](#nginx-api-for-lua) provided by this module is used to handle diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index e11b35e715..4f13fdd21f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4,6 +4,8 @@ ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. ''This module is not distributed with the Nginx source.'' See [[#Installation|the installation instructions]]. +This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) + = Status = Production ready. @@ -130,6 +132,8 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t = Description = +This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) + This module embeds Lua, via [http://luajit.org/luajit.html LuaJIT 2.0/2.1], into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. Unlike [https://httpd.apache.org/docs/trunk/mod/mod_lua.html Apache's mod_lua] and [http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet Lighttpd's mod_magnet], Lua code executed using this module can be ''100% non-blocking'' on network traffic as long as the [[#Nginx API for Lua|Nginx API for Lua]] provided by this module is used to handle From fd90f4e8252e9d06419317fdf525b55c65e15a50 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 8 Feb 2019 12:20:14 -0800 Subject: [PATCH 287/848] doc: one followup tweak. --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index c21894a194..3df44953e0 100644 --- a/README.markdown +++ b/README.markdown @@ -8,10 +8,10 @@ Name ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. -*This module is not distributed with the Nginx source.* See [the installation instructions](#installation). - This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) +*This module is not distributed with the Nginx source.* See [the installation instructions](#installation). + Table of Contents ================= diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4f13fdd21f..9222811251 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2,10 +2,10 @@ ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. -''This module is not distributed with the Nginx source.'' See [[#Installation|the installation instructions]]. - This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) +''This module is not distributed with the Nginx source.'' See [[#Installation|the installation instructions]]. + = Status = Production ready. From 8edb21abcc4ee3ccc2d50c63bbc7c3ca4553b374 Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 14 Feb 2019 23:00:56 +0800 Subject: [PATCH 288/848] bugfix: only update connections count when cosockets are truly associated with the socket pool. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_socket_tcp.c | 21 ++++++---- t/068-socket-keepalive.t | 76 ++++++++++++++++++++++++++++++++++- 2 files changed, 89 insertions(+), 8 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index fcaf51a0c7..edc9b31843 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5304,12 +5304,6 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) ngx_http_lua_socket_tcp_create_socket_pool(L, r, key, pool_size, -1, &spool); - /* we should always increase connections after getting connected, - * and decrease connections after getting closed. - * however, we don't create connection pool in previous connect method. - * so we increase connections here for backward compatibility. - */ - spool->connections++; } if (ngx_queue_empty(&spool->free)) { @@ -5320,13 +5314,26 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) item = ngx_queue_data(q, ngx_http_lua_socket_pool_item_t, queue); ngx_http_lua_socket_tcp_close_connection(item->connection); - spool->connections--; + + /* only decrease the counter for connections which were counted */ + if (u->socket_pool != NULL) { + u->socket_pool->connections--; + } } else { q = ngx_queue_head(&spool->free); ngx_queue_remove(q); item = ngx_queue_data(q, ngx_http_lua_socket_pool_item_t, queue); + + /* we should always increase connections after getting connected, + * and decrease connections after getting closed. + * however, we don't create connection pool in previous connect method. + * so we increase connections here for backward compatibility. + */ + if (u->socket_pool == NULL) { + spool->connections++; + } } item->connection = c; diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index 85be758f36..a5f599c7df 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; #repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 33); +plan tests => repeat_each() * (blocks() * 4 + 31); our $HtmlDir = html_dir; @@ -2835,3 +2835,77 @@ GET /t [alert] --- error_log resume success + + + +=== TEST 50: conn queuing: increase the counter for connections created before creating the pool with setkeepalive() +--- config + set $port $TEST_NGINX_MEMCACHED_PORT; + + location /t { + content_by_lua_block { + local function connect() + local sock, err = ngx.socket.connect("127.0.0.1", ngx.var.port) + if not sock then + error("connect failed: " .. err) + end + + return sock + end + + local sock1 = connect() + local sock2 = connect() + assert(sock1:setkeepalive()) + assert(sock2:setkeepalive()) + + local sock1 = connect() + local sock2 = connect() + assert(sock1:close()) + assert(sock2:close()) + + ngx.say("ok") + } + } +--- request +GET /t +--- no_error_log +[error] +--- response_body +ok + + + +=== TEST 51: conn queuing: only decrease the counter for connections which were counted by the pool +--- config + set $port $TEST_NGINX_MEMCACHED_PORT; + + location /t { + content_by_lua_block { + local function connect() + local sock, err = ngx.socket.connect("127.0.0.1", ngx.var.port) + if not sock then + error("connect failed: " .. err) + end + + return sock + end + + local sock1 = connect() + local sock2 = connect() + assert(sock1:setkeepalive(1000, 1)) + assert(sock2:setkeepalive(1000, 1)) + + local sock1 = connect() + local sock2 = connect() + assert(sock1:close()) + assert(sock2:close()) + + ngx.say("ok") + } + } +--- request +GET /t +--- no_error_log +[error] +--- response_body +ok From 761ddca72deb4595ff93df7840dfa39854126123 Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 15 Feb 2019 16:01:41 +0800 Subject: [PATCH 289/848] bugfix: ensured we cleanup up aborted queued connect operations. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_socket_tcp.c | 81 +++++++++++++++++++++++++++-------- t/068-socket-keepalive.t | 47 ++++++++++++++++++++ 2 files changed, 110 insertions(+), 18 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index edc9b31843..87461e6ac2 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -140,6 +140,7 @@ static ngx_int_t ngx_http_lua_socket_tcp_read_resume(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_socket_tcp_write_resume(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op); +static void ngx_http_lua_tcp_queue_conn_op_cleanup(void *data); static void ngx_http_lua_tcp_resolve_cleanup(void *data); static void ngx_http_lua_coctx_cleanup(void *data); static void ngx_http_lua_socket_free_pool(ngx_log_t *log, @@ -630,6 +631,10 @@ ngx_http_lua_socket_tcp_connect_helper(lua_State *L, u->write_co_ctx = ctx->cur_co_ctx; conn_op_ctx->u = u; + ctx->cur_co_ctx->cleanup = + ngx_http_lua_tcp_queue_conn_op_cleanup; + ctx->cur_co_ctx->data = conn_op_ctx; + ngx_memzero(&conn_op_ctx->event, sizeof(ngx_event_t)); conn_op_ctx->event.handler = ngx_http_lua_socket_tcp_conn_op_timeout_handler; @@ -3977,25 +3982,10 @@ ngx_http_lua_socket_tcp_resume_conn_op(ngx_http_lua_socket_pool_t *spool) * is exiting. */ if (ngx_queue_empty(&spool->wait_connect_op)) { -#if (NGX_DEBUG) - ngx_http_lua_assert(!(spool->backlog >= 0 - && spool->connections > spool->size)); - -#else - if (spool->backlog >= 0 && spool->connections > spool->size) { - ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, - "lua tcp socket connections count mismatched for " - "connection pool \"%s\", connections: %i, size: %i", - spool->key, spool->connections, spool->size); - spool->connections = spool->size; - } -#endif - return; } q = ngx_queue_head(&spool->wait_connect_op); - ngx_queue_remove(q); conn_op_ctx = ngx_queue_data(q, ngx_http_lua_socket_tcp_conn_op_ctx_t, queue); ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, @@ -4034,27 +4024,52 @@ ngx_http_lua_socket_tcp_conn_op_ctx_cleanup(void *data) static void ngx_http_lua_socket_tcp_conn_op_resume_handler(ngx_event_t *ev) { - ngx_http_lua_socket_tcp_upstream_t *u; - ngx_http_lua_ctx_t *ctx; + ngx_queue_t *q; ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; ngx_http_request_t *r; ngx_http_cleanup_t *cln; ngx_http_lua_co_ctx_t *coctx; + ngx_http_lua_socket_pool_t *spool; + ngx_http_lua_socket_tcp_upstream_t *u; ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; conn_op_ctx = ev->data; u = conn_op_ctx->u; r = u->request; + spool = u->socket_pool; + + if (ngx_queue_empty(&spool->wait_connect_op)) { +#if (NGX_DEBUG) + ngx_http_lua_assert(!(spool->backlog >= 0 + && spool->connections > spool->size)); + +#else + if (spool->backlog >= 0 && spool->connections > spool->size) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "lua tcp socket connections count mismatched for " + "connection pool \"%s\", connections: %i, size: %i", + spool->key, spool->connections, spool->size); + spool->connections = spool->size; + } +#endif + + return; + } + + q = ngx_queue_head(&spool->wait_connect_op); + ngx_queue_remove(q); coctx = u->write_co_ctx; coctx->cleanup = NULL; /* note that we store conn_op_ctx in coctx->data instead of u */ coctx->data = conn_op_ctx; + /* clear ngx_http_lua_tcp_queue_conn_op_cleanup */ u->write_co_ctx = NULL; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { - ngx_queue_insert_head(&u->socket_pool->cache_connect_op, + ngx_queue_insert_head(&spool->cache_connect_op, &conn_op_ctx->queue); return; } @@ -6021,6 +6036,36 @@ ngx_http_lua_socket_tcp_resume_helper(ngx_http_request_t *r, int socket_op) } +static void +ngx_http_lua_tcp_queue_conn_op_cleanup(void *data) +{ + ngx_http_lua_co_ctx_t *coctx = data; + ngx_http_lua_socket_tcp_upstream_t *u; + ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx; + + conn_op_ctx = coctx->data; + u = conn_op_ctx->u; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua tcp socket abort queueing, conn_op_ctx: %p, u: %p", + conn_op_ctx, u); + + if (conn_op_ctx->event.posted) { + ngx_delete_posted_event(&conn_op_ctx->event); + + } else if (conn_op_ctx->event.timer_set) { + ngx_del_timer(&conn_op_ctx->event); + } + + ngx_queue_remove(&conn_op_ctx->queue); + ngx_queue_insert_head(&u->socket_pool->cache_connect_op, + &conn_op_ctx->queue); + + u->socket_pool->connections--; + ngx_http_lua_socket_tcp_resume_conn_op(u->socket_pool); +} + + static void ngx_http_lua_tcp_resolve_cleanup(void *data) { diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index a5f599c7df..fe85b8c81b 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -2909,3 +2909,50 @@ GET /t [error] --- response_body ok + + + +=== TEST 52: conn queuing: clean up pending connect operations which are in queue +--- config + set $port $TEST_NGINX_MEMCACHED_PORT; + + location /sub { + content_by_lua_block { + local opts = {pool = "test", pool_size = 1, backlog = 1} + local sock, err = ngx.socket.connect("127.0.0.1", ngx.var.port, opts) + if not sock then + ngx.say("connect failed: " .. err) + return + end + + local function f() + assert(ngx.socket.connect("127.0.0.1", ngx.var.port, opts)) + end + + local th = ngx.thread.spawn(f) + local ok, err = ngx.thread.kill(th) + if not ok then + ngx.log(ngx.ERR, "kill thread failed: ", err) + return + end + + sock:close() + } + } + + location /t { + content_by_lua_block { + ngx.location.capture("/sub") + -- let pending connect operation resumes first + ngx.sleep(0) + ngx.say("ok") + } + } +--- request +GET /t +--- no_error_log +[error] +--- error_log +lua tcp socket abort queueing +--- response_body +ok From c368af2f0784b1472741ad7be834ab4853bba1a8 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sun, 17 Feb 2019 16:45:16 -0800 Subject: [PATCH 290/848] tests: 159-sa-restart.t: avoided sending SIGWINCH to the test nginx worker process. The handler for this signal falls-through to a graceful shutdown in daemonized worker processes. --- t/159-sa-restart.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/159-sa-restart.t b/t/159-sa-restart.t index 912338df95..a0f0c0eeb1 100644 --- a/t/159-sa-restart.t +++ b/t/159-sa-restart.t @@ -20,7 +20,7 @@ add_block_preprocessor(sub { --"INT", "IO", "CHLD", - "WINCH", + --"WINCH", } for _, signame in ipairs(signals) do From ee47e55eb5c36ef8eb7937a232f3ec0fcd22ec33 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 20 Feb 2019 22:11:40 +0800 Subject: [PATCH 291/848] bugfix: ngx.pipe: proc:wait() returns false only when the sub-process exited abnormally. --- src/ngx_http_lua_pipe.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 65251ff75d..0857a04211 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -2162,7 +2162,7 @@ ngx_http_lua_pipe_wait_retval(ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L) if (pipe->timeout) { pipe->timeout = 0; - lua_pushboolean(L, 0); + lua_pushnil(L); lua_pushliteral(L, "timeout"); return 2; } From 83ca6b57ae5ede2b9ff987babff323e5814cebaf Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sat, 23 Feb 2019 22:14:12 -0800 Subject: [PATCH 292/848] doc: bumped version to 0.10.14. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 3df44953e0..7f46e64661 100644 --- a/README.markdown +++ b/README.markdown @@ -63,7 +63,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.13](https://github.com/openresty/lua-nginx-module/tags) released on 22 April 2018. +This document describes ngx_lua [v0.10.14](https://github.com/openresty/lua-nginx-module/tags) released on February 23rd, 2019. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 9222811251..66cc62e7c8 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -12,7 +12,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.13] released on 22 April 2018. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.14] released on February 23rd, 2019. = Synopsis = From a92cf7255cf5132fc887bfccc937eebb29b30a14 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 26 Feb 2019 16:48:40 -0800 Subject: [PATCH 293/848] tests: fixed failures due to recent CDN changes. --- t/023-rewrite/tcp-socket.t | 6 +++--- t/023-rewrite/uthread-exit.t | 4 ++-- t/024-access/uthread-exit.t | 4 ++-- t/058-tcp-socket.t | 6 +++--- t/094-uthread-exit.t | 4 ++-- t/124-init-worker.t | 7 +++---- 6 files changed, 15 insertions(+), 16 deletions(-) diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index b81f68e2fa..9a7d0ed8f4 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -250,11 +250,11 @@ attempt to send data on a closed socket: } --- request GET /t ---- response_body +--- response_body_like connected: 1 request sent: 56 -first line received: HTTP/1.1 200 OK -second line received: Server: openresty +first line received: HTTP\/1\.1 200 OK +second line received: (?:Date|Server): .*? --- no_error_log [error] diff --git a/t/023-rewrite/uthread-exit.t b/t/023-rewrite/uthread-exit.t index 63e765dce3..e25f7f8b0c 100644 --- a/t/023-rewrite/uthread-exit.t +++ b/t/023-rewrite/uthread-exit.t @@ -298,7 +298,7 @@ exiting the user thread === TEST 5: exit in user thread (entry thread is still pending on the DNS resolver for ngx.socket.tcp) --- config location /lua { - resolver agentzh.org; + resolver 127.0.0.2:12345; resolver_timeout 12s; rewrite_by_lua ' local function f() @@ -403,7 +403,7 @@ after === TEST 6: exit in user thread (entry thread is still pending on the DNS resolver for ngx.socket.udp) --- config location /lua { - resolver agentzh.org; + resolver 127.0.0.2:12345; #resolver 127.0.0.1; resolver_timeout 12s; rewrite_by_lua ' diff --git a/t/024-access/uthread-exit.t b/t/024-access/uthread-exit.t index 57b6ea8208..27572370d5 100644 --- a/t/024-access/uthread-exit.t +++ b/t/024-access/uthread-exit.t @@ -298,7 +298,7 @@ exiting the user thread === TEST 5: exit in user thread (entry thread is still pending on the DNS resolver for ngx.socket.tcp) --- config location /lua { - resolver agentzh.org; + resolver 127.0.0.2:12345; resolver_timeout 12s; access_by_lua ' local function f() @@ -394,7 +394,7 @@ after === TEST 6: exit in user thread (entry thread is still pending on the DNS resolver for ngx.socket.udp) --- config location /lua { - resolver agentzh.org; + resolver 127.0.0.2:12345; resolver_timeout 12s; access_by_lua ' local function f() diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 38f072673b..541beb176d 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -245,11 +245,11 @@ attempt to send data on a closed socket: --- request GET /t ---- response_body +--- response_body_like connected: 1 request sent: 56 -first line received: HTTP/1.1 200 OK -second line received: Server: openresty +first line received: HTTP\/1\.1 200 OK +second line received: (?:Date|Server): .*? --- no_error_log [error] --- timeout: 10 diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index 032b02a2bf..58d8d0b7db 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -297,7 +297,7 @@ exiting the user thread === TEST 5: exit in user thread (entry thread is still pending on the DNS resolver for ngx.socket.tcp) --- config location /lua { - resolver agentzh.org; + resolver 127.0.0.2:12345; resolver_timeout 12s; content_by_lua ' local function f() @@ -393,7 +393,7 @@ after === TEST 6: exit in user thread (entry thread is still pending on the DNS resolver for ngx.socket.udp) --- config location /lua { - resolver agentzh.org; + resolver 127.0.0.2:12345; resolver_timeout 12s; content_by_lua ' local function f() diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 182ab40212..b648d55e61 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -521,12 +521,11 @@ warn(): Thu, 01 Jan 1970 01:34:38 GMT } --- request GET /t ---- response_body -timer created +--- response_body_like connected: 1 request sent: 56 -first line received: HTTP/1.1 200 OK -second line received: Server: openresty +first line received: HTTP\/1\.1 200 OK +second line received: (?:Date|Server): .*? --- no_error_log [error] --- timeout: 10 From 15b01c66f737f2b6ea4dcd3dfb525c2f2c023fcb Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 9 Mar 2019 11:35:17 -0800 Subject: [PATCH 294/848] tests: fixed the tests using iscribblet.org which would time out now. --- t/091-coroutine.t | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/091-coroutine.t b/t/091-coroutine.t index 7e6918d116..7cf60f9675 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -178,7 +178,7 @@ cc3: 2 local urls = { "agentzh.org", - "iscribblet.org", + "openresty.com", "openresty.org" } @@ -204,7 +204,7 @@ cc3: 2 GET /lua --- response_body successfully connected to: agentzh.org -successfully connected to: iscribblet.org +successfully connected to: openresty.com successfully connected to: openresty.org *** All Done *** --- no_error_log @@ -377,7 +377,7 @@ GET /lua local urls = { "agentzh.org", - "iscribblet.org", + "openresty.com", "openresty.org" } @@ -398,7 +398,7 @@ GET /lua GET /lua --- response_body successfully connected to: agentzh.org -successfully connected to: iscribblet.org +successfully connected to: openresty.com successfully connected to: openresty.org *** All Done *** --- no_error_log From 2bc20ccb04a1e87dc9b22fb84173995ae33b0ed3 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Sat, 9 Mar 2019 11:22:34 +0500 Subject: [PATCH 295/848] travis-ci: switch to xenial image --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index bfb940dde7..f9c36774b1 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,5 @@ sudo: required -dist: trusty +dist: xenial os: linux From afeeb84b0840061d52e3e5686dd00142139ecd31 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sun, 10 Mar 2019 08:30:12 +0800 Subject: [PATCH 296/848] bugfix: ngx.pipe: fixed a segfault when error_log is configured with syslog. Fix https://github.com/openresty/lua-resty-shell/issues/7. --- src/ngx_http_lua_pipe.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 0857a04211..f2fc671bb6 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -703,8 +703,8 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, "pipe fd"); } - errlog_fd = ngx_cycle->log->file->fd; - if (errlog_fd == STDERR_FILENO) { + if (ngx_cycle->log->file && ngx_cycle->log->file->fd == STDERR_FILENO) { + errlog_fd = ngx_cycle->log->file->fd; temp_errlog_fd = dup(errlog_fd); if (temp_errlog_fd == -1) { From 12d991e182ec0e742b200405b65150cba8659e3d Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 26 Mar 2019 09:35:58 -0700 Subject: [PATCH 297/848] api: bumped API version to 0.10.15. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index 7f46e64661..720618b691 100644 --- a/README.markdown +++ b/README.markdown @@ -63,7 +63,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.14](https://github.com/openresty/lua-nginx-module/tags) released on February 23rd, 2019. +This document describes ngx_lua [v0.10.15](https://github.com/openresty/lua-nginx-module/tags) released on March 14th, 2019. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 66cc62e7c8..b073e2c4a2 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -12,7 +12,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.14] released on February 23rd, 2019. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.15] released on March 14th, 2019. = Synopsis = diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 3f75a4df7a..129eb9979c 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10014 +#define ngx_http_lua_version 10015 typedef struct { From 95a2e59bfb466e220d69d9953f29b12464ad687e Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Mon, 8 Apr 2019 16:24:32 -0700 Subject: [PATCH 298/848] tests: removed obsolete drizzle upstream block in init.t. --- t/000--init.t | 5 ----- 1 file changed, 5 deletions(-) diff --git a/t/000--init.t b/t/000--init.t index dbe2c33099..5865755372 100644 --- a/t/000--init.t +++ b/t/000--init.t @@ -10,11 +10,6 @@ $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_MYSQL_PORT} ||= 3306; our $http_config = <<'_EOC_'; - upstream database { - drizzle_server 127.0.0.1:$TEST_NGINX_MYSQL_PORT protocol=mysql - dbname=ngx_test user=ngx_test password=ngx_test; - } - lua_package_path "../lua-resty-mysql/lib/?.lua;;"; _EOC_ From 7ac6f3f67dad6ba9ce2eea85285c5177bcf9d589 Mon Sep 17 00:00:00 2001 From: uudiin <77389867@qq.com> Date: Tue, 9 Apr 2019 07:28:19 +0800 Subject: [PATCH 299/848] doc: fixed a typo in a comment from ngx_http_lua_common.h. --- src/ngx_http_lua_common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 0428e2d3fa..112256a22b 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -479,7 +479,7 @@ typedef struct { typedef struct ngx_http_lua_ctx_s { - /* for lua_coce_cache off: */ + /* for lua_code_cache off: */ ngx_http_lua_vm_state_t *vm_state; ngx_http_request_t *request; From 87f0a09a60e6a51c99cd896dc02229f9b87e2607 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 2 Apr 2019 15:42:33 -0700 Subject: [PATCH 300/848] feat: implemented the lua_load_resty_core directive which loads resty.core by default. --- README.markdown | 33 +++++++++++++++++++ doc/HttpLuaModule.wiki | 28 ++++++++++++++++ src/ngx_http_lua_common.h | 2 ++ src/ngx_http_lua_module.c | 12 +++++++ src/ngx_http_lua_util.c | 23 +++++++++++++ t/161-load-resty-core.t | 68 +++++++++++++++++++++++++++++++++++++++ 6 files changed, 166 insertions(+) create mode 100644 t/161-load-resty-core.t diff --git a/README.markdown b/README.markdown index 720618b691..04f259697a 100644 --- a/README.markdown +++ b/README.markdown @@ -1004,6 +1004,7 @@ See Also Directives ========== +* [lua_load_resty_core](#lua_load_resty_core) * [lua_capture_error_log](#lua_capture_error_log) * [lua_use_default_type](#lua_use_default_type) * [lua_malloc_trim](#lua_malloc_trim) @@ -1079,6 +1080,38 @@ how the result will be used. Below is a diagram showing the order in which direc [Back to TOC](#table-of-contents) +lua_load_resty_core +------------------- + +**syntax:** *lua_load_resty_core on|off* + +**default:** *lua_load_resty_core on* + +**context:** *http* + +Controls whether the `resty.core` module (from +[lua-resty-core](https://github.com/openresty/lua-resty-core)) should be loaded +or not. When enabled, this directive is equivalent to executing the following +when the Lua VM is created: + +```lua + + require "resty.core" +``` + +Note that usage of the `resty.core` module is recommended, as its +FFI implementation is both faster, safer, and more complete than the Lua C API +of the ngx_lua module. + +It must also be noted that the Lua C API of the ngx_lua module will eventually +be removed, and usage of the FFI-based API (i.e. the `resty.core` +module) will become mandatory. This directive only aims at providing a +temporary backwards-compatibility mode in case of edge-cases. + +This directive was first introduced in the `v0.10.15` release. + +[Back to TOC](#directives) + lua_capture_error_log --------------------- **syntax:** *lua_capture_error_log size* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index b073e2c4a2..98ff9834a0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -835,6 +835,34 @@ how the result will be used. Below is a diagram showing the order in which direc ![Lua Nginx Modules Directives](https://cloud.githubusercontent.com/assets/2137369/15272097/77d1c09e-1a37-11e6-97ef-d9767035fc3e.png) +== lua_load_resty_core == + +'''syntax:''' ''lua_load_resty_core on|off'' + +'''default:''' ''lua_load_resty_core on'' + +'''context:''' ''http'' + +Controls whether the resty.core module (from +[https://github.com/openresty/lua-resty-core lua-resty-core]) should be loaded +or not. When enabled, this directive is equivalent to executing the following +when the Lua VM is created: + + + require "resty.core" + + +Note that usage of the resty.core module is recommended, as its +FFI implementation is both faster, safer, and more complete than the Lua C API +of the ngx_lua module. + +It must also be noted that the Lua C API of the ngx_lua module will eventually +be removed, and usage of the FFI-based API (i.e. the resty.core +module) will become mandatory. This directive only aims at providing a +temporary backwards-compatibility mode in case of edge-cases. + +This directive was first introduced in the v0.10.15 release. + == lua_capture_error_log == '''syntax:''' ''lua_capture_error_log size'' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 112256a22b..dae524516f 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -182,6 +182,8 @@ struct ngx_http_lua_main_conf_s { ngx_cycle_t *cycle; ngx_pool_t *pool; + ngx_flag_t load_resty_core; + ngx_int_t max_pending_timers; ngx_int_t pending_timers; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index cb99fd781e..2a3f5d57b6 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -77,6 +77,13 @@ static ngx_conf_bitmask_t ngx_http_lua_ssl_protocols[] = { static ngx_command_t ngx_http_lua_cmds[] = { + { ngx_string("lua_load_resty_core"), + NGX_HTTP_MAIN_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_MAIN_CONF_OFFSET, + offsetof(ngx_http_lua_main_conf_t, load_resty_core), + NULL }, + { ngx_string("lua_max_running_timers"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, ngx_conf_set_num_slot, @@ -877,6 +884,7 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) */ lmcf->pool = cf->pool; + lmcf->load_resty_core = NGX_CONF_UNSET; lmcf->max_pending_timers = NGX_CONF_UNSET; lmcf->max_running_timers = NGX_CONF_UNSET; #if (NGX_PCRE) @@ -910,6 +918,10 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) { ngx_http_lua_main_conf_t *lmcf = conf; + if (lmcf->load_resty_core == NGX_CONF_UNSET) { + lmcf->load_resty_core = 1; + } + #if (NGX_PCRE) if (lmcf->regex_cache_max_entries == NGX_CONF_UNSET) { lmcf->regex_cache_max_entries = 1024; diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 72558624fb..c12262e84c 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -147,6 +147,13 @@ static int ngx_http_lua_get_raw_phase_context(lua_State *L); #define LUA_PATH_SEP ";" #endif + +#if !defined(LUA_DEFAULT_PATH) && (NGX_DEBUG) +#define LUA_DEFAULT_PATH "../lua-resty-core/lib/?.lua;" \ + "../lua-resty-lrucache/lib/?.lua" +#endif + + #define AUX_MARK "\1" @@ -3813,6 +3820,7 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log, ngx_pool_cleanup_t **pcln) { + int rc; lua_State *L; ngx_uint_t i; ngx_pool_cleanup_t *cln; @@ -3880,6 +3888,21 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, lua_pop(L, 2); } + if (lmcf->load_resty_core) { + lua_getglobal(L, "require"); + lua_pushstring(L, "resty.core"); + + rc = lua_pcall(L, 1, 1, 0); + if (rc != 0) { + ngx_log_error(NGX_LOG_ERR, log, 0, + "lua_load_resty_core failed to load the resty.core " + "module from https://github.com/openresty/lua-resty" + "-core; ensure you are using an OpenResty release " + "from https://openresty.org/en/download.html " + "(rc: %i, reason: %s)", rc, lua_tostring(L, -1)); + } + } + return L; } diff --git a/t/161-load-resty-core.t b/t/161-load-resty-core.t new file mode 100644 index 0000000000..41b18f0301 --- /dev/null +++ b/t/161-load-resty-core.t @@ -0,0 +1,68 @@ +use Test::Nginx::Socket::Lua; + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3); + +add_block_preprocessor(sub { + my $block = shift; + + if (!defined $block->request) { + $block->set_value("request", "GET /t"); + } + + if (!defined $block->no_error_log) { + $block->set_value("no_error_log", "[error]"); + } +}); + +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: lua_load_resty_core is enabled by default +--- config + location = /t { + content_by_lua_block { + local loaded_resty_core = package.loaded["resty.core"] + local resty_core = require "resty.core" + + ngx.say("resty.core loaded: ", loaded_resty_core == resty_core) + } + } +--- response_body +resty.core loaded: true + + + +=== TEST 2: lua_load_resty_core can be disabled +--- http_config + lua_load_resty_core off; +--- config + location = /t { + content_by_lua_block { + local loaded_resty_core = package.loaded["resty.core"] + + ngx.say("resty.core loaded: ", loaded_resty_core ~= nil) + } + } +--- response_body +resty.core loaded: false + + + +=== TEST 3: lua_load_resty_core is effective when using lua_shared_dict +--- http_config + lua_shared_dict dogs 128k; +--- config + location = /t { + content_by_lua_block { + local loaded_resty_core = package.loaded["resty.core"] + local resty_core = require "resty.core" + + ngx.say("resty.core loaded: ", loaded_resty_core == resty_core) + } + } +--- response_body +resty.core loaded: true From fccf383ae8a4b9d53487b0dfe1f3d92a11239a13 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 2 Apr 2019 16:43:10 -0700 Subject: [PATCH 301/848] tests: fixed default lua path to load resty.core from many test cases. --- t/004-require.t | 2 +- t/014-bugs.t | 10 +++--- t/023-rewrite/req-socket.t | 6 ++-- t/023-rewrite/socket-keepalive.t | 12 +++---- t/023-rewrite/tcp-socket.t | 20 +++++------ t/025-codecache.t | 20 +++++------ t/058-tcp-socket.t | 20 +++++------ t/063-abort.t | 62 ++++++++++++++++---------------- t/064-pcall.t | 4 +-- t/067-req-socket.t | 6 ++-- t/068-socket-keepalive.t | 30 ++++++++-------- t/087-udp-socket.t | 10 +++--- t/106-timer.t | 10 +++--- t/108-timer-safe.t | 10 +++--- 14 files changed, 111 insertions(+), 111 deletions(-) diff --git a/t/004-require.t b/t/004-require.t index 9fd28606cb..17e1d57c9f 100644 --- a/t/004-require.t +++ b/t/004-require.t @@ -24,7 +24,7 @@ __DATA__ === TEST 1: sanity --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /main { echo_location /load; diff --git a/t/014-bugs.t b/t/014-bugs.t index c0babf9ad2..dad084ec4d 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -43,7 +43,7 @@ __DATA__ === TEST 1: sanity --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /load { content_by_lua ' @@ -69,7 +69,7 @@ end === TEST 2: sanity --- http_config -lua_package_path '/home/agentz/rpm/BUILD/lua-yajl-1.1/build/?.so;/home/lz/luax/?.so;./?.so'; +lua_package_cpath '/home/agentz/rpm/BUILD/lua-yajl-1.1/build/?.so;/home/lz/luax/?.so;./?.so'; --- config location = '/report/listBidwordPrices4lzExtra.htm' { content_by_lua ' @@ -847,7 +847,7 @@ ok === TEST 37: resolving names with a trailing dot --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -865,7 +865,7 @@ GET /t === TEST 38: resolving names with a trailing dot --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua'; + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; server { listen 12354; @@ -892,7 +892,7 @@ args: foo=1&bar=2 === TEST 39: lua_code_cache off + setkeepalive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config lua_code_cache off; location = /t { diff --git a/t/023-rewrite/req-socket.t b/t/023-rewrite/req-socket.t index 87cbbbef26..9292385265 100644 --- a/t/023-rewrite/req-socket.t +++ b/t/023-rewrite/req-socket.t @@ -325,7 +325,7 @@ found the end of the stream === TEST 4: attempt to use the req socket across request boundary --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { rewrite_by_lua ' @@ -376,7 +376,7 @@ hello world === TEST 5: receive until on request_body - receiveuntil(1) on the last byte of the body See https://groups.google.com/group/openresty/browse_thread/thread/43cf01da3c681aba for details --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { rewrite_by_lua ' @@ -440,7 +440,7 @@ done === TEST 6: pipelined POST requests --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { rewrite_by_lua ' diff --git a/t/023-rewrite/socket-keepalive.t b/t/023-rewrite/socket-keepalive.t index 489a70fc93..8b532ead6b 100644 --- a/t/023-rewrite/socket-keepalive.t +++ b/t/023-rewrite/socket-keepalive.t @@ -27,7 +27,7 @@ __DATA__ === TEST 1: sanity --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -104,7 +104,7 @@ lua tcp socket get keepalive peer: using connection === TEST 2: free up the whole connection pool if no active connections --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -177,7 +177,7 @@ received: OK === TEST 3: upstream sockets close prematurely --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; keepalive_timeout 100ms; @@ -254,7 +254,7 @@ done === TEST 4: http keepalive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -815,7 +815,7 @@ lua tcp socket keepalive timeout: unlimited === TEST 11: sanity (uds) --- http_config eval " - lua_package_path '$::HtmlDir/?.lua;./?.lua'; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; server { listen unix:$::HtmlDir/nginx.sock; default_type 'text/plain'; @@ -953,7 +953,7 @@ lua tcp socket get keepalive peer: using connection === TEST 13: github issue #110: ngx.exit with HTTP_NOT_FOUND causes worker process to exit --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config error_page 404 /404.html; location /t { diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index 9a7d0ed8f4..5258487eba 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -1082,7 +1082,7 @@ close: 1 nil === TEST 19: cannot survive across request boundary (send) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -1143,7 +1143,7 @@ received: OK|failed to send request: closed)\$" === TEST 20: cannot survive across request boundary (receive) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -1214,7 +1214,7 @@ received: OK|failed to receive a line: closed \[nil\])$/ === TEST 21: cannot survive across request boundary (close) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -1285,7 +1285,7 @@ received: OK|failed to close: closed)$/ === TEST 22: cannot survive across request boundary (connect) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -2041,7 +2041,7 @@ close: 1 nil === TEST 33: bad request tries to connect --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2097,7 +2097,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):7: bad request/ === TEST 34: bad request tries to receive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2156,7 +2156,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 35: bad request tries to send --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2215,7 +2215,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 36: bad request tries to close --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2274,7 +2274,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 37: bad request tries to set keepalive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2333,7 +2333,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 38: bad request tries to receiveuntil --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { diff --git a/t/025-codecache.t b/t/025-codecache.t index 11d53acdbf..22b78b2136 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -193,7 +193,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 6: code cache explicitly off (affects require) + content_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /lua { lua_code_cache off; @@ -231,7 +231,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 7: code cache explicitly off (affects require) + content_by_lua_file --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /lua { lua_code_cache off; @@ -269,7 +269,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 8: code cache explicitly off (affects require) + set_by_lua_file --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /lua { lua_code_cache off; @@ -308,7 +308,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 9: code cache explicitly on (affects require) + set_by_lua_file --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /lua { lua_code_cache on; @@ -468,7 +468,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 14: no clear builtin lib "string" --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config lua_code_cache off; location /lua { @@ -504,7 +504,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 15: do not skip luarocks --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua'; + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; lua_code_cache off;" --- config location /main { @@ -554,7 +554,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 16: do not skip luarocks* --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua'; + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; lua_code_cache off;" --- config location /main { @@ -604,7 +604,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/ === TEST 17: clear _G table --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config lua_code_cache off; location /t { @@ -1050,7 +1050,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, === TEST 29: cosocket connection pool timeout (after Lua VM destroys) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config lua_code_cache off; location = /t { @@ -1118,7 +1118,7 @@ qr/\blua tcp socket keepalive: free connection pool [0-9A-F]+ for "127.0.0.1:/, === TEST 30: cosocket connection pool timeout (before Lua VM destroys) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config lua_code_cache off; location = /t { diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 541beb176d..837006c638 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -1057,7 +1057,7 @@ close: 1 nil === TEST 19: cannot survive across request boundary (send) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -1116,7 +1116,7 @@ received: OK|failed to send request: closed)\$" === TEST 20: cannot survive across request boundary (receive) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -1191,7 +1191,7 @@ received: OK|failed to receive a line: closed \[nil\])$/ === TEST 21: cannot survive across request boundary (close) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -1260,7 +1260,7 @@ received: OK|failed to close: closed)$/ === TEST 22: cannot survive across request boundary (connect) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -2672,7 +2672,7 @@ lua clean up the timer for pending ngx.sleep === TEST 44: bad request tries to connect --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2726,7 +2726,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):7: bad request/ === TEST 45: bad request tries to receive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2783,7 +2783,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 46: bad request tries to send --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2840,7 +2840,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 47: bad request tries to close --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2897,7 +2897,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 48: bad request tries to set keepalive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -2954,7 +2954,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 49: bad request tries to receiveuntil --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { diff --git a/t/063-abort.t b/t/063-abort.t index 411a07eee9..7c90eaafae 100644 --- a/t/063-abort.t +++ b/t/063-abort.t @@ -22,7 +22,7 @@ __DATA__ === TEST 1: ngx.exit(400) should abort print --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /memc_query { internal; @@ -68,7 +68,7 @@ GET /test?a === TEST 2: ngx.exit(400) should abort ngx.log --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /memc_query { internal; @@ -116,7 +116,7 @@ GET /test?a === TEST 3: ngx.exit(400) should abort ngx.location.capture --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /memc_query { internal; @@ -161,7 +161,7 @@ the "$memc_key" variable is not set === TEST 4: ngx.exit(400) should abort ngx.location.capture_multi --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /memc_query { internal; @@ -206,7 +206,7 @@ the "$memc_key" variable is not set === TEST 5: ngx.exit(400) should abort ngx.redirect --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -233,7 +233,7 @@ lua redirect to "/blah" with code 302 === TEST 6: ngx.exit(400) should abort ngx.exit --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -260,7 +260,7 @@ lua exit with code 503 === TEST 7: ngx.exit(400) should abort ngx.exec --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -287,7 +287,7 @@ lua exec "/blah?" === TEST 8: ngx.exit(400) should abort ngx.send_headers --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -314,7 +314,7 @@ lua send headers === TEST 9: ngx.exit(400) should abort ngx.print --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -341,7 +341,7 @@ lua print response === TEST 10: ngx.exit(400) should abort ngx.say --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -368,7 +368,7 @@ lua say response === TEST 11: ngx.exit(400) should abort ngx.flush --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -395,7 +395,7 @@ lua flush asynchronously === TEST 12: ngx.exit(400) should abort ngx.eof --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -422,7 +422,7 @@ lua send eof === TEST 13: ngx.exit(400) should abort ngx.re.match --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -449,7 +449,7 @@ lua compiling match regex "a" with options "jo" === TEST 14: ngx.exit(400) should abort ngx.re.gmatch --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -476,7 +476,7 @@ lua compiling gmatch regex "a" with options "jo" === TEST 15: ngx.exit(400) should abort ngx.re.sub --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -503,7 +503,7 @@ lua compiling sub regex "a" with options "jo" === TEST 16: ngx.exit(400) should abort ngx.re.gsub --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -530,7 +530,7 @@ lua compiling gsub regex "a" with options "jo" === TEST 17: ngx.exit(400) should abort ngx.shared.DICT (set) --- http_config eval - "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -560,7 +560,7 @@ foo = 56 === TEST 18: ngx.exit(400) should abort ngx.shared.DICT (replace) --- http_config eval - "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -590,7 +590,7 @@ foo = 56 === TEST 19: ngx.exit(400) should abort ngx.shared.DICT (incr) --- http_config eval - "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -620,7 +620,7 @@ foo = 88 === TEST 20: ngx.exit(400) should abort ngx.shared.DICT (get) --- http_config eval - "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_shared_dict dogs 1m; lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -649,7 +649,7 @@ fetching key "foo" in shared dict "dogs" === TEST 21: ngx.exit(400) should skip os.execute --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -679,7 +679,7 @@ GET /test === TEST 22: ngx.exit(400) should break pcall and skip os.execute --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -709,7 +709,7 @@ fetching key "foo" in shared dict "dogs" === TEST 23: ngx.exit(400) should break pcall and skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -739,7 +739,7 @@ GET /test === TEST 24: ngx.redirect() should break pcall and skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -769,7 +769,7 @@ GET /test === TEST 25: ngx.redirect() should skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -799,7 +799,7 @@ GET /test === TEST 26: ngx.exec() should break pcall and skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -832,7 +832,7 @@ foo === TEST 27: ngx.exec() should skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -865,7 +865,7 @@ foo === TEST 28: ngx.set_uri(uri, true) should break pcall and skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { rewrite_by_lua ' @@ -921,7 +921,7 @@ hello world === TEST 30: ngx.exit(400) should break xpcall and skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -955,7 +955,7 @@ GET /test === TEST 31: ngx.exec() should skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -988,7 +988,7 @@ foo === TEST 32: ngx.exec() should break pcall and skip os.execute (all in user module) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' diff --git a/t/064-pcall.t b/t/064-pcall.t index 4bf447e43a..9af2de7d92 100644 --- a/t/064-pcall.t +++ b/t/064-pcall.t @@ -22,7 +22,7 @@ __DATA__ === TEST 1: pcall works --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' @@ -58,7 +58,7 @@ $/s === TEST 2: xpcall works --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /test { content_by_lua ' diff --git a/t/067-req-socket.t b/t/067-req-socket.t index 229d5ccf41..6593360e4c 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -320,7 +320,7 @@ found the end of the stream === TEST 4: attempt to use the req socket across request boundary --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { content_by_lua ' @@ -369,7 +369,7 @@ hello world === TEST 5: receive until on request_body - receiveuntil(1) on the last byte of the body See https://groups.google.com/group/openresty/browse_thread/thread/43cf01da3c681aba for details --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { content_by_lua ' @@ -431,7 +431,7 @@ done === TEST 6: pipelined POST requests --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { content_by_lua ' diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index fe85b8c81b..e4c698887b 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -30,7 +30,7 @@ __DATA__ === TEST 1: sanity --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -98,7 +98,7 @@ lua tcp socket keepalive create connection pool for key "127.0.0.1:$ENV{TEST_NGI === TEST 2: free up the whole connection pool if no active connections --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -169,7 +169,7 @@ received: OK === TEST 3: upstream sockets close prematurely --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; keepalive_timeout 100ms; @@ -244,7 +244,7 @@ done === TEST 4: http keepalive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location /t { @@ -808,7 +808,7 @@ qr/lua tcp socket connection pool size: 30\b/] === TEST 12: sanity (uds) --- http_config eval " - lua_package_path '$::HtmlDir/?.lua;./?.lua'; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; server { listen unix:$::HtmlDir/nginx.sock; default_type 'text/plain'; @@ -936,7 +936,7 @@ lua tcp socket get keepalive peer: using connection === TEST 14: github issue #110: ngx.exit with HTTP_NOT_FOUND causes worker process to exit --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config error_page 404 /404.html; location /t { @@ -995,7 +995,7 @@ Not found, dear... === TEST 15: custom pools (different pool for the same host:port) - tcp --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1043,7 +1043,7 @@ lua tcp socket keepalive create connection pool for key "B" === TEST 16: custom pools (same pool for different host:port) - tcp --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1091,7 +1091,7 @@ lua tcp socket get keepalive peer: using connection === TEST 17: custom pools (different pool for the same host:port) - unix --- http_config eval " - lua_package_path '$::HtmlDir/?.lua;./?.lua'; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; server { listen unix:$::HtmlDir/nginx.sock; default_type 'text/plain'; @@ -1151,7 +1151,7 @@ lua tcp socket keepalive create connection pool for key "B" === TEST 18: custom pools (same pool for the same path) - unix --- http_config eval " - lua_package_path '$::HtmlDir/?.lua;./?.lua'; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; server { listen unix:$::HtmlDir/nginx.sock; default_type 'text/plain'; @@ -1205,7 +1205,7 @@ lua tcp socket get keepalive peer: using connection === TEST 19: numeric pool option value --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1252,7 +1252,7 @@ lua tcp socket get keepalive peer: using connection === TEST 20: nil pool option value --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1295,7 +1295,7 @@ connected: 1, reused: 0 === TEST 21: (bad) table pool option value --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1336,7 +1336,7 @@ bad argument #3 to 'connect' (bad "pool" option type: table) === TEST 22: (bad) boolean pool option value --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1394,7 +1394,7 @@ bad argument #3 to 'connect' (bad "pool" option type: boolean) === TEST 24: bug in send(): clear the chain writer ctx --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location /t { set $port $TEST_NGINX_REDIS_PORT; diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index 9e7eb2168a..dc6cad5ef8 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -824,7 +824,7 @@ probe syscall.socket.return, syscall.connect.return { === TEST 15: bad request tries to setpeer --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -881,7 +881,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 16: bad request tries to send --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -938,7 +938,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 17: bad request tries to receive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -995,7 +995,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 18: bad request tries to close --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { @@ -1052,7 +1052,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 19: bad request tries to receive --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config server_tokens off; location = /main { diff --git a/t/106-timer.t b/t/106-timer.t index bb5b73a8db..7b7c85ec72 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -320,7 +320,7 @@ qr/received: Server: \S+/, === TEST 6: tcp cosocket in timer handler (keep-alive connections) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -596,7 +596,7 @@ qr/\[lua\] log_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-6])/, === TEST 10: tcp cosocket in timer handler (keep-alive connections) - log_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -693,7 +693,7 @@ qr/go\(\): connected: 1, reused: \d+/, === TEST 11: tcp cosocket in timer handler (keep-alive connections) - header_filter_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -799,7 +799,7 @@ qr/go\(\): connected: 1, reused: \d+/, === TEST 12: tcp cosocket in timer handler (keep-alive connections) - body_filter_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -912,7 +912,7 @@ qr/go\(\): connected: 1, reused: \d+/, === TEST 13: tcp cosocket in timer handler (keep-alive connections) - set_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { diff --git a/t/108-timer-safe.t b/t/108-timer-safe.t index 10687e78a1..758d01a6f2 100644 --- a/t/108-timer-safe.t +++ b/t/108-timer-safe.t @@ -229,7 +229,7 @@ qr/received: Server: \S+/, === TEST 4: tcp cosocket in timer handler (keep-alive connections) --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -508,7 +508,7 @@ qr/\[lua\] log_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-9]|8[0- === TEST 8: tcp cosocket in timer handler (keep-alive connections) - log_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -606,7 +606,7 @@ qr/go\(\): connected: 1, reused: \d+/, === TEST 9: tcp cosocket in timer handler (keep-alive connections) - header_filter_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -713,7 +713,7 @@ qr/go\(\): connected: 1, reused: \d+/, === TEST 10: tcp cosocket in timer handler (keep-alive connections) - body_filter_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { @@ -827,7 +827,7 @@ qr/go\(\): connected: 1, reused: \d+/, === TEST 11: tcp cosocket in timer handler (keep-alive connections) - set_by_lua --- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua';" + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config location = /t { From 61e4d0aac8974b8fad1b5b93d0d3d694d257d328 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 2 Apr 2019 17:11:01 -0700 Subject: [PATCH 302/848] tests: fixed a few failing test cases due to loading resty.core. --- t/013-base64.t | 2 +- t/035-gmatch.t | 6 +++--- t/062-count.t | 2 +- t/123-lua-path.t | 4 ++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/t/013-base64.t b/t/013-base64.t index 1cc27de5e9..d1a8d8b094 100644 --- a/t/013-base64.t +++ b/t/013-base64.t @@ -242,4 +242,4 @@ GET /t --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log eval -qr/bad argument \#2 to 'encode_base64' \(boolean expected, got number\)|\[error\] .*? boolean argument only/ +qr/bad argument \#2 to 'encode_base64' \(boolean expected, got number\)|\[error\] .*? bad no_padding: boolean expected, got number/ diff --git a/t/035-gmatch.t b/t/035-gmatch.t index c526778db1..0dfeaeaf39 100644 --- a/t/035-gmatch.t +++ b/t/035-gmatch.t @@ -482,9 +482,9 @@ end GET /main --- response_body matched -sr failed: 500 ---- error_log -attempt to use ngx.re.gmatch iterator in a request that did not create it +matched +--- no_error_log +[error] diff --git a/t/062-count.t b/t/062-count.t index 2e6c5419e0..9b0afe8aac 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -283,7 +283,7 @@ n = 5 --- request GET /test --- response_body -n = 18 +n = 22 --- no_error_log [error] diff --git a/t/123-lua-path.t b/t/123-lua-path.t index 23681a9dbb..c058ebe6ff 100644 --- a/t/123-lua-path.t +++ b/t/123-lua-path.t @@ -37,7 +37,7 @@ env LUA_CPATH; --- request GET /lua --- response_body -/foo/bar/baz +../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;/foo/bar/baz /baz/bar/foo --- no_error_log @@ -61,7 +61,7 @@ env LUA_CPATH; --- request GET /lua --- response_body -/foo/bar/baz +../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;/foo/bar/baz /baz/bar/foo --- no_error_log From 7be0a4dc9325e35bcfe33e967ba0654e5fe80153 Mon Sep 17 00:00:00 2001 From: Keith Walsh Date: Wed, 24 Apr 2019 08:37:10 -0400 Subject: [PATCH 303/848] doc: fixed a typo in README.md. Signed-off-by: Thibault Charbonnier --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 04f259697a..ed0c72dae3 100644 --- a/README.markdown +++ b/README.markdown @@ -6738,7 +6738,7 @@ ngx.shared.DICT.flush_expired Flushes out the expired items in the dictionary, up to the maximal number specified by the optional `max_count` argument. When the `max_count` argument is given `0` or not given at all, then it means unlimited. Returns the number of items that have actually been flushed. -Unlike the [flush_all](#ngxshareddictflush_all) method, this method actually free up the memory used by the expired items. +Unlike the [flush_all](#ngxshareddictflush_all) method, this method actually frees up the memory used by the expired items. This feature was first introduced in the `v0.6.3` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 98ff9834a0..ff9b269282 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5667,7 +5667,7 @@ See also [[#ngx.shared.DICT.flush_expired|ngx.shared.DICT.flush_expired]] and [[ Flushes out the expired items in the dictionary, up to the maximal number specified by the optional max_count argument. When the max_count argument is given 0 or not given at all, then it means unlimited. Returns the number of items that have actually been flushed. -Unlike the [[#ngx.shared.DICT.flush_all|flush_all]] method, this method actually free up the memory used by the expired items. +Unlike the [[#ngx.shared.DICT.flush_all|flush_all]] method, this method actually frees up the memory used by the expired items. This feature was first introduced in the v0.6.3 release. From 86941a706bb8522aee973640305ed3e443ae57e6 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 22 Apr 2019 12:53:27 -0700 Subject: [PATCH 304/848] tests: 123-lua-path.t: fixed a few test cases failing in EC2 to-ngx_lua suite. In to-ngx_lua, the paths to lua-resty-core and lua-resty-lrucache are not included by default, but is still required for Nginx to start due to the default value of `lua_load_resty_core`. Hence, we always provide those paths via the `LUA_PATH` env variable. In non-to-* suites, those paths will thus be present twice, so the regexes take that into consideration. --- t/123-lua-path.t | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/t/123-lua-path.t b/t/123-lua-path.t index c058ebe6ff..e30132f3c5 100644 --- a/t/123-lua-path.t +++ b/t/123-lua-path.t @@ -11,7 +11,7 @@ repeat_each(2); plan tests => repeat_each() * (blocks() * 3 + 1); -$ENV{LUA_PATH} = "/foo/bar/baz"; +$ENV{LUA_PATH} = "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;/foo/bar/baz"; $ENV{LUA_CPATH} = "/baz/bar/foo"; #no_diff(); #no_long_string(); @@ -36,8 +36,8 @@ env LUA_CPATH; } --- request GET /lua ---- response_body -../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;/foo/bar/baz +--- response_body_like +(?:\.\.\/lua-resty-core\/lib\/\?\.lua;\.\.\/lua-resty-lrucache\/lib\/\?\.lua;){1,2}\/foo\/bar\/baz /baz/bar/foo --- no_error_log @@ -60,8 +60,8 @@ env LUA_CPATH; } --- request GET /lua ---- response_body -../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;/foo/bar/baz +--- response_body_like +(?:\.\.\/lua-resty-core\/lib\/\?\.lua;\.\.\/lua-resty-lrucache\/lib\/\?\.lua;){1,2}\/foo\/bar\/baz /baz/bar/foo --- no_error_log From f87d1f16516b9cc2a78690eee863d38310db5533 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Mon, 6 May 2019 08:36:45 +0800 Subject: [PATCH 305/848] bugfix: ngx.pipe: cleared the pending event after aborting the uthread. --- src/ngx_http_lua_pipe.c | 56 ++++++++++++++++------------------------- 1 file changed, 22 insertions(+), 34 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index f2fc671bb6..8c8221fdb5 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -77,8 +77,7 @@ static void ngx_http_lua_pipe_resume_write_handler(ngx_event_t *ev); static void ngx_http_lua_pipe_resume_wait_handler(ngx_event_t *ev); static ngx_int_t ngx_http_lua_pipe_resume(ngx_http_request_t *r); static void ngx_http_lua_pipe_dummy_event_handler(ngx_event_t *ev); -static void ngx_http_lua_pipe_cleanup_helper( - ngx_http_lua_co_ctx_t *wait_co_ctx); +static void ngx_http_lua_pipe_clear_event(ngx_event_t *ev); static void ngx_http_lua_pipe_proc_read_stdout_cleanup(void *data); static void ngx_http_lua_pipe_proc_read_stderr_cleanup(void *data); static void ngx_http_lua_pipe_proc_write_cleanup(void *data); @@ -2217,19 +2216,7 @@ ngx_http_lua_pipe_resume_helper(ngx_event_t *ev, ev->timedout = 0; } - if (ev->timer_set) { - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ev->log, 0, - "lua pipe del timer for ev:%p", ev); - ngx_del_timer(ev); - } - - if (ev->posted) { - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ev->log, 0, - "lua pipe del posted event for ev:%p", ev); - ngx_delete_posted_event(ev); - } - - ev->handler = ngx_http_lua_pipe_dummy_event_handler; + ngx_http_lua_pipe_clear_event(ev); r = ngx_http_lua_get_req(wait_co_ctx->co); c = r->connection; @@ -2377,15 +2364,21 @@ ngx_http_lua_pipe_dummy_event_handler(ngx_event_t *ev) static void -ngx_http_lua_pipe_cleanup_helper(ngx_http_lua_co_ctx_t *wait_co_ctx) +ngx_http_lua_pipe_clear_event(ngx_event_t *ev) { - if (wait_co_ctx->sleep.timer_set) { - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, wait_co_ctx->sleep.log, 0, - "lua pipe del timer for ev:%p", &wait_co_ctx->sleep); - ngx_del_timer(&wait_co_ctx->sleep); + ev->handler = ngx_http_lua_pipe_dummy_event_handler; + + if (ev->timer_set) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ev->log, 0, + "lua pipe del timer for ev:%p", ev); + ngx_del_timer(ev); } - wait_co_ctx->cleanup = NULL; + if (ev->posted) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ev->log, 0, + "lua pipe del posted event for ev:%p", ev); + ngx_delete_posted_event(ev); + } } @@ -2404,10 +2397,10 @@ ngx_http_lua_pipe_proc_read_stdout_cleanup(void *data) c = proc->pipe->stdout_ctx->c; if (c) { rev = c->read; - rev->handler = ngx_http_lua_pipe_dummy_event_handler; + ngx_http_lua_pipe_clear_event(rev); } - ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); + wait_co_ctx->cleanup = NULL; } @@ -2426,10 +2419,10 @@ ngx_http_lua_pipe_proc_read_stderr_cleanup(void *data) c = proc->pipe->stderr_ctx->c; if (c) { rev = c->read; - rev->handler = ngx_http_lua_pipe_dummy_event_handler; + ngx_http_lua_pipe_clear_event(rev); } - ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); + wait_co_ctx->cleanup = NULL; } @@ -2448,10 +2441,10 @@ ngx_http_lua_pipe_proc_write_cleanup(void *data) c = proc->pipe->stdin_ctx->c; if (c) { wev = c->write; - wev->handler = ngx_http_lua_pipe_dummy_event_handler; + ngx_http_lua_pipe_clear_event(wev); } - ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); + wait_co_ctx->cleanup = NULL; } @@ -2471,14 +2464,9 @@ ngx_http_lua_pipe_proc_wait_cleanup(void *data) pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; pipe_node->wait_co_ctx = NULL; - if (wait_co_ctx->sleep.posted) { - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua pipe del posted event for ev:%p", - &wait_co_ctx->sleep); - ngx_delete_posted_event(&wait_co_ctx->sleep); - } + ngx_http_lua_pipe_clear_event(&wait_co_ctx->sleep); - ngx_http_lua_pipe_cleanup_helper(wait_co_ctx); + wait_co_ctx->cleanup = NULL; } From 28cf5ce3b6ec8e7ab44eadac9cc1c3b6f5c387ba Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 3 May 2019 22:29:13 -0700 Subject: [PATCH 306/848] feature: added the FFI API for 'str_replace_char()'. --- src/ngx_http_lua_string.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index 239b2322ab..bdadeb47f0 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -762,6 +762,20 @@ ngx_http_lua_ffi_escape_uri(const u_char *src, size_t len, u_char *dst) ngx_http_lua_escape_uri(dst, (u_char *) src, len, NGX_ESCAPE_URI_COMPONENT); } + +void +ngx_http_lua_ffi_str_replace_char(u_char *buf, size_t len, const u_char find, + const u_char replace) +{ + while (len) { + if (*buf == find) { + *buf = replace; + } + + buf++; + len--; + } +} #endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From 441c7f626ae934a7d385a020ecce664a26792adf Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 20 May 2019 12:16:53 -0700 Subject: [PATCH 307/848] tests: re-enabled a test case for using ngx.re.match in init_by_lua*. --- t/034-match.t | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/t/034-match.t b/t/034-match.t index 473a11fff7..1bf45bce1a 100644 --- a/t/034-match.t +++ b/t/034-match.t @@ -9,7 +9,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 15); +plan tests => repeat_each() * (blocks() * 2 + 16); #no_diff(); no_long_string(); @@ -1129,27 +1129,33 @@ failed to match -=== TEST 48: init_by_lua +=== TEST 48: init_by_lua_block --- http_config - init_by_lua ' - package.loaded.m = ngx.re.match("hello, 1234", "(\\\\d+)") - '; + init_by_lua_block { + local m, err = ngx.re.match("hello, 1234", [[(\d+)]]) + if not m then + ngx.log(ngx.ERR, "failed to match: ", err) + else + package.loaded.m = m + end + } --- config location /re { - content_by_lua ' + content_by_lua_block { local m = package.loaded.m if m then ngx.say(m[0]) else ngx.say("not matched!") end - '; + } } --- request GET /re --- response_body 1234 ---- SKIP +--- no_error_log +[error] From 30678fd5b9bbe344920278916f73cc0eb946db19 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 22 May 2019 14:30:06 -0700 Subject: [PATCH 308/848] tests: ssl-socket.t: fixed a failing test case due to openresty.org's default cipher. --- t/129-ssl-socket.t | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index f2fa26b6cc..454852d7da 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1096,9 +1096,11 @@ close: 1 nil qr/^lua ssl save session: ([0-9A-F]+) lua ssl free session: ([0-9A-F]+) $/ ---- error_log -lua ssl server name: "openresty.org" -SSL: TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 +--- error_log eval +[ +'lua ssl server name: "openresty.org"', +qr/SSL: TLSv1\.2, cipher: "(?:ECDHE-RSA-AES(?:256|128)-GCM-SHA(?:384|256)|ECDHE-(?:RSA|ECDSA)-CHACHA20-POLY1305) TLSv1\.2/, +] --- no_error_log SSL reused session [error] From 1c756b385dc6812b8e7a90f9a690f16a775792c9 Mon Sep 17 00:00:00 2001 From: Luke Gorrie Date: Thu, 23 May 2019 00:06:01 +0200 Subject: [PATCH 309/848] doc: fixed a mistake in tcpsock:receiveany() example in README.md. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index ed0c72dae3..2c004bf2db 100644 --- a/README.markdown +++ b/README.markdown @@ -7310,7 +7310,7 @@ Timeout for the reading operation is controlled by the [lua_socket_read_timeout] ```lua sock:settimeouts(1000, 1000, 1000) -- one second timeout for connect/read/write - local data, err = sock:receiveany(10 * 1024 * 1024) -- read any data, at most 10K + local data, err = sock:receiveany(10 * 1024) -- read any data, at most 10K if not data then ngx.say("failed to read any data: ", err) return diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ff9b269282..c2a131b275 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -6178,7 +6178,7 @@ Timeout for the reading operation is controlled by the [[#lua_socket_read_timeou sock:settimeouts(1000, 1000, 1000) -- one second timeout for connect/read/write - local data, err = sock:receiveany(10 * 1024 * 1024) -- read any data, at most 10K + local data, err = sock:receiveany(10 * 1024) -- read any data, at most 10K if not data then ngx.say("failed to read any data: ", err) return From a7afd56c66284db64d2dde74e6d9c11d06668e38 Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 25 Apr 2019 11:14:04 +0800 Subject: [PATCH 310/848] feature: added support for specifying environment variables in ngx.pipe spawn() FFI API. Signed-off-by: Thibault Charbonnier --- config | 15 +++++++++++++++ src/ngx_http_lua_pipe.c | 31 ++++++++++++++++++++++++++++++- 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/config b/config index e1d5e3543f..3ef8f0749f 100644 --- a/config +++ b/config @@ -557,6 +557,21 @@ CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" # ---------------------------------------- +ngx_feature="execvpe" +ngx_feature_libs= +ngx_feature_name="NGX_HTTP_LUA_HAVE_EXECVPE" +ngx_feature_run=no +ngx_feature_incs= +ngx_feature_test='char* argv[] = {"/bin/sh"};execvpe("/bin/sh", argv, NULL);' +SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" +CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" + +. auto/feature + +CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" + +# ---------------------------------------- + if test -n "$ngx_module_link"; then ngx_module_type=HTTP_AUX_FILTER ngx_module_name=$ngx_addon_name diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 8c8221fdb5..f9ae6cd455 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -551,7 +551,7 @@ ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, size_t size) int ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, const char *file, const char **argv, int merge_stderr, size_t buffer_size, - u_char *errbuf, size_t *errbuf_size) + const char **environ, u_char *errbuf, size_t *errbuf_size) { int rc; int in[2]; @@ -571,6 +571,15 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, ngx_http_lua_pipe_signal_t *sig; sigset_t set; +#if !(NGX_HTTP_LUA_HAVE_EXECVPE) + if (environ != NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + "environ option not supported") + - errbuf; + return NGX_ERROR; + } +#endif + pool_size = ngx_align(NGX_MIN_POOL_SIZE + buffer_size * 2, NGX_POOL_ALIGNMENT); @@ -758,11 +767,31 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } } +#if (NGX_HTTP_LUA_HAVE_EXECVPE) + if (environ != NULL) { + if (execvpe(file, (char * const *) argv, (char * const *) environ) + == -1) + { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child execvpe() failed while " + "executing %s", file); + } + + } else { + if (execvp(file, (char * const *) argv) == -1) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe child execvp() failed while " + "executing %s", file); + } + } + +#else if (execvp(file, (char * const *) argv) == -1) { ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, "lua pipe child execvp() failed while executing %s", file); } +#endif exit(EXIT_FAILURE); } From 1882c30a087168051a8292d0fcd199c0408da814 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 27 May 2019 10:40:20 -0700 Subject: [PATCH 311/848] doc: we now work with nginx 1.14.x and 1.15.x. Thanks Antoine for the reminder in #1534. --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index 2c004bf2db..f979407117 100644 --- a/README.markdown +++ b/README.markdown @@ -252,6 +252,8 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.15.x (last tested: 1.15.8) +* 1.14.x * 1.13.x (last tested: 1.13.6) * 1.12.x * 1.11.x (last tested: 1.11.2) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index c2a131b275..5f7695a96e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -190,6 +190,8 @@ The Lua state (Lua VM instance) is shared across all the requests handled by a s The latest version of this module is compatible with the following versions of Nginx: +* 1.15.x (last tested: 1.15.8) +* 1.14.x * 1.13.x (last tested: 1.13.6) * 1.12.x * 1.11.x (last tested: 1.11.2) From 4102b82cd0971c93663fe89eabd75b73741c4213 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 28 May 2019 10:31:49 -0700 Subject: [PATCH 312/848] tests: 161-load-resty-core.t: added a test asserting the directive honors 'lua_package_path'. --- t/161-load-resty-core.t | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/t/161-load-resty-core.t b/t/161-load-resty-core.t index 41b18f0301..66bc4258b8 100644 --- a/t/161-load-resty-core.t +++ b/t/161-load-resty-core.t @@ -4,6 +4,8 @@ repeat_each(2); plan tests => repeat_each() * (blocks() * 3); +our $HtmlDir = html_dir; + add_block_preprocessor(sub { my $block = shift; @@ -66,3 +68,30 @@ resty.core loaded: false } --- response_body resty.core loaded: true + + + +=== TEST 4: lua_load_resty_core honors the lua_package_path directive +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;;';" +--- config + location = /t { + content_by_lua_block { + local loaded_resty_core = package.loaded["resty.core"] + local resty_core = require "resty.core" + + ngx.say("resty.core loaded: ", loaded_resty_core == resty_core) + + resty_core.go() + } + } +--- response_body +resty.core loaded: true +loaded from html dir +--- user_files +>>> resty/core.lua +return { + go = function () + ngx.say("loaded from html dir") + end +} From 33a5517c9c4e1a6d1415c2a59a51b0f75e3c8b04 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 2 Jun 2019 20:20:11 -0700 Subject: [PATCH 313/848] tweak: fixed some warnings from the clang static code analyzer. --- src/ngx_http_lua_semaphore.c | 4 ++++ src/ngx_http_lua_util.c | 1 - 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 0b70aea722..9e50b7b30e 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -86,6 +86,8 @@ ngx_http_lua_alloc_sema(void) lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, ngx_http_lua_module); + ngx_http_lua_assert(lmcf != NULL); + mm = lmcf->sema_mm; if (!ngx_queue_empty(&mm->free_queue)) { @@ -172,6 +174,8 @@ ngx_http_lua_sema_mm_cleanup(void *data) sem = ngx_queue_data(q, ngx_http_lua_sema_t, chain); block = sem->block; + ngx_http_lua_assert(block != NULL); + if (block->used == 0) { iter = (ngx_http_lua_sema_t *) (block + 1); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c12262e84c..47768c003b 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -269,7 +269,6 @@ ngx_http_lua_new_state(lua_State *parent_vm, ngx_cycle_t *cycle, lua_pushliteral(L, LUA_DEFAULT_PATH ";"); /* package default */ lua_getfield(L, -2, "path"); /* package default old */ - old_path = lua_tolstring(L, -1, &old_path_len); lua_concat(L, 2); /* package new */ lua_setfield(L, -2, "path"); /* package */ #endif From 3a5872969cce758eedbd0e750522afb877734195 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 4 Jun 2019 16:51:32 -0700 Subject: [PATCH 314/848] travis-ci: bumped OpenSSL to 1.0.2s and 1.1.0k. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index f9c36774b1..ef3083ca54 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,8 +48,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2q OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0j OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2s OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0k OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" # to $OPENSSL_OPT. From a38d9066787f6fe311135fc512004b1bc19ba1ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Wed, 5 Jun 2019 10:19:53 +0800 Subject: [PATCH 315/848] feature: added OpenSSL 1.1.1 support to ssl_session_fetch_by_lua*. See openresty/openresty#456. Signed-off-by: Thibault Charbonnier --- .travis.yml | 11 +- README.markdown | 15 ++- doc/HttpLuaModule.wiki | 15 ++- src/ngx_http_lua_module.c | 6 + src/ngx_http_lua_ssl_session_fetchby.c | 156 ++++++++++++++++++++----- src/ngx_http_lua_ssl_session_fetchby.h | 5 + 6 files changed, 162 insertions(+), 46 deletions(-) diff --git a/.travis.yml b/.travis.yml index ef3083ca54..1ce1371a7b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,10 +48,9 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2s OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0k OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d - # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" - # to $OPENSSL_OPT. + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2s OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0k OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.1c OPENSSL_PATCH_VER="" services: - memcache @@ -116,8 +115,8 @@ script: - cd .. - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch - - ./config no-threads shared enable-ssl3 enable-ssl3-method $OPENSSL_OPT -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) + - if [ -n "$OPENSSL_PATCH_VER" ]; then patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; fi + - ./config no-threads shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. diff --git a/README.markdown b/README.markdown index f979407117..a7c8ca9906 100644 --- a/README.markdown +++ b/README.markdown @@ -2623,15 +2623,18 @@ But do not forget to comment this line out before publishing your site to the wo If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. -If you are using OpenSSL libraries not provided by [OpenResty](https://openresty.org), -then you need to apply the following patch for OpenSSL 1.0.2h or later: +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +1.0.2, up to (and including) 1.1.0: - + -If you are not using the NGINX core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, then you need to -apply the following patch to the standard NGINX core 1.11.2 or later: +Similarly, if you are not using the NGINX core shipped with +[OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply +patches to the standard NGINX core: - + This directive was first introduced in the `v0.10.6` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 5f7695a96e..2e78bdc1bc 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2219,15 +2219,18 @@ But do not forget to comment this line out before publishing your site to the wo If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. -If you are using OpenSSL libraries not provided by [OpenResty](https://openresty.org), -then you need to apply the following patch for OpenSSL 1.0.2h or later: +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +1.0.2, up to (and including) 1.1.0: -https://github.com/openresty/openresty/blob/master/patches/openssl-1.0.2h-sess_set_get_cb_yield.patch +https://openresty.org/en/openssl-patches.html -If you are not using the NGINX core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, then you need to -apply the following patch to the standard NGINX core 1.11.2 or later: +Similarly, if you are not using the NGINX core shipped with +[OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply +patches to the standard NGINX core: -http://openresty.org/download/nginx-1.11.2-nonblocking_ssl_handshake_hooks.patch +https://openresty.org/en/nginx-ssl-patches.html This directive was first introduced in the v0.10.6 release. diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 2a3f5d57b6..331bd04152 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1077,6 +1077,12 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) return NGX_CONF_ERROR; #else +# ifdef HAVE_SSL_CLIENT_HELLO_CB_SUPPORT + SSL_CTX_set_client_hello_cb(sscf->ssl.ctx, + ngx_http_lua_ssl_client_hello_handler, + NULL); +# endif + SSL_CTX_sess_set_get_cb(sscf->ssl.ctx, ngx_http_lua_ssl_sess_fetch_handler); #endif diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 303efb47aa..7ff0168655 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -173,16 +173,12 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } -/* cached session fetching callback to be set with SSL_CTX_sess_set_get_cb */ -ngx_ssl_session_t * -ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, -#if OPENSSL_VERSION_NUMBER >= 0x10100003L - const -#endif - u_char *id, int len, int *copy) +static ngx_int_t +ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, + const u_char *id, int len) { lua_State *L; - ngx_int_t rc; + ngx_int_t rc, res = NGX_ERROR; ngx_connection_t *c, *fc = NULL; ngx_http_request_t *r = NULL; ngx_pool_cleanup_t *cln; @@ -191,11 +187,6 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, ngx_http_lua_srv_conf_t *lscf; ngx_http_core_loc_conf_t *clcf; - /* set copy to 0 as we expect OpenSSL to handle - * the memory of returned session */ - - *copy = 0; - c = ngx_ssl_get_connection(ssl_conn); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, @@ -217,17 +208,10 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, cctx->exit_code); dd("lua ssl sess_fetch done, finally"); - return cctx->session; + return NGX_OK; } -#ifdef SSL_ERROR_PENDING_SESSION - return SSL_magic_pending_session_ptr(); -#else - ngx_log_error(NGX_LOG_CRIT, c->log, 0, - "lua: cannot yield in sess get cb: " - "missing async sess get cb support in OpenSSL"); - return NULL; -#endif + return NGX_AGAIN; } dd("first time"); @@ -329,7 +313,7 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, "sess get cb exit code: %d", rc, cctx->exit_code); c->log->action = "SSL handshaking"; - return cctx->session; + return NGX_OK; } /* rc == NGX_DONE */ @@ -356,12 +340,13 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, *cctx->cleanup = ngx_http_lua_ssl_sess_fetch_aborted; -#ifdef SSL_ERROR_PENDING_SESSION - return SSL_magic_pending_session_ptr(); +#if defined(SSL_ERROR_PENDING_SESSION) \ + || defined(HAVE_SSL_CLIENT_HELLO_CB_SUPPORT) + + return NGX_AGAIN; + #else - ngx_log_error(NGX_LOG_CRIT, c->log, 0, - "lua: cannot yield in sess get cb: " - "missing async sess get cb support in OpenSSL"); + res = NGX_AGAIN; /* fall through to the "failed" label below */ #endif @@ -376,10 +361,125 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, ngx_http_lua_close_fake_connection(fc); } + return res; +} + + +#ifdef HAVE_SSL_CLIENT_HELLO_CB_SUPPORT +int +ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, + int *al, void *arg) +{ + int len; + ngx_int_t rc; + const u_char *id; + + len = SSL_client_hello_get0_session_id(ssl_conn, &id); + + if (len <= 0) { + return SSL_CLIENT_HELLO_SUCCESS; + } + + rc = ngx_http_lua_ssl_sess_fetch_helper(ssl_conn, id, len); + + if (rc == NGX_AGAIN) { + return SSL_CLIENT_HELLO_RETRY; + } + + return SSL_CLIENT_HELLO_SUCCESS; +} + + +ngx_ssl_session_t * +ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, + const u_char *id, int len, int *copy) +{ + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; + + /* set copy to 0 as we expect OpenSSL to handle + * the memory of returned session */ + + *copy = 0; + + c = ngx_ssl_get_connection(ssl_conn); + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + + if (cctx && cctx->done) { + return cctx->session; + } + return NULL; } +#else + +/* cached session fetching callback to be set with SSL_CTX_sess_set_get_cb */ +ngx_ssl_session_t * +ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, +#if OPENSSL_VERSION_NUMBER >= 0x10100003L + const +#endif + u_char *id, int len, int *copy) +{ + ngx_int_t rc; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; + + /* set copy to 0 as we expect OpenSSL to handle + * the memory of returned session */ + + *copy = 0; + + c = ngx_ssl_get_connection(ssl_conn); + + rc = ngx_http_lua_ssl_sess_fetch_helper(ssl_conn, id, len); + + if (rc == NGX_AGAIN) { + +#ifdef SSL_ERROR_PENDING_SESSION + + return SSL_magic_pending_session_ptr(); + +#else + + ngx_log_error(NGX_LOG_CRIT, c->log, 0, + "lua: cannot yield in sess get cb: " +# if OPENSSL_VERSION_NUMBER >= 0x1010100fL + "missing support for yielding during SSL handshake in " + "the nginx core; consider using the OpenResty releases " + "from https://openresty.org/en/download.html or apply " + "the nginx core patches yourself (see " + "https://openresty.org/en/nginx-ssl-patches.html)"); + +# else + "missing support for yielding during SSL handshake in " + "linked " OPENSSL_VERSION_TEXT "; consider using the " + "OpenResty releases from " + "https://openresty.org/en/download.html or apply " + "the OpenSSL patches yourself (see " + "https://openresty.org/en/openssl-patches.html)"); +# endif + + return NULL; + +#endif + } + + if (rc == NGX_ERROR) { + return NULL; + } + + /* rc == NGX_OK */ + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + return cctx->session; +} +#endif + + static void ngx_http_lua_ssl_sess_fetch_done(void *data) { diff --git a/src/ngx_http_lua_ssl_session_fetchby.h b/src/ngx_http_lua_ssl_session_fetchby.h index 50c6616dd2..7a73ed96e5 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.h +++ b/src/ngx_http_lua_ssl_session_fetchby.h @@ -30,8 +30,13 @@ ngx_ssl_session_t *ngx_http_lua_ssl_sess_fetch_handler( const #endif u_char *id, int len, int *copy); + +#ifdef HAVE_SSL_CLIENT_HELLO_CB_SUPPORT +int ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, + int *al, void *arg); #endif +#endif /* NGX_HTTP_SSL */ #endif /* _NGX_HTTP_LUA_SSL_SESSION_FETCHBY_H_INCLUDED_ */ From 59b6430f7b7c407e15b19035c566a56cfd2eb3e8 Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 31 May 2019 19:15:30 +0800 Subject: [PATCH 316/848] bugfix: LuaJIT sets currentline to -1 if it is unavailable. --- src/ngx_http_lua_log.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index 0042a5e2b4..3938da5507 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -194,7 +194,7 @@ log_wrapper(ngx_log_t *log, const char *ident, ngx_uint_t level, *p++ = ':'; p = ngx_snprintf(p, NGX_INT_T_LEN, "%d", - ar.currentline ? ar.currentline : ar.linedefined); + ar.currentline > 0 ? ar.currentline : ar.linedefined); *p++ = ':'; *p++ = ' '; From 32dd6a34a17d6f1f094552a45c204ef614f5fbb5 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 14 Jun 2019 17:21:34 -0700 Subject: [PATCH 317/848] style: fixed a minor alignment issue in ngx_http_lua_ssl_certby.c. --- src/ngx_http_lua_ssl_certby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 622d0a5c1a..5e73de2de6 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -116,7 +116,7 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, u_char *p; u_char *name; ngx_str_t *value; - ngx_http_lua_srv_conf_t *lscf = conf; + ngx_http_lua_srv_conf_t *lscf = conf; /* must specify a concrete handler */ if (cmd->post == NULL) { From 81344434aa625878cb33bf0090a8c951a0979a64 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 27 Jun 2019 17:08:15 -0700 Subject: [PATCH 318/848] doc: fixed a mistake in ngx.location.capture() example in README.md. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index a7c8ca9906..fa605e1077 100644 --- a/README.markdown +++ b/README.markdown @@ -3818,7 +3818,7 @@ The `args` option can also take plain query strings: ```lua ngx.location.capture('/foo?a=1', - { args = 'b=3&c=%3a' } } + { args = 'b=3&c=%3a' } ) ``` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 2e78bdc1bc..8223b5b013 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3126,7 +3126,7 @@ The args option can also take plain query strings: ngx.location.capture('/foo?a=1', - { args = 'b=3&c=%3a' } } + { args = 'b=3&c=%3a' } ) From b0c7373917c63af3a06474df0ff9f1a65939d3e1 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 2 Jul 2019 11:57:20 -0700 Subject: [PATCH 319/848] Revert "feature: added OpenSSL 1.1.1 support to ssl_session_fetch_by_lua*." This reverts commit a38d9066787f6fe311135fc512004b1bc19ba1ac. As put by @spacewander: > Previously, we used ClientHello callback to do ssl session fetching non-blockingly. However, this way can not handle an edge case: the ssl session resumption via session ticket might fail, and the client falls back to session ID resumption. The ClientHello callback is run too early to know if the client will fall back to use session ID resumption. Therefore, we have to take back the OpenSSL sess_set_get_cb_yield patch and upgrade it to adapt OpenSSL 1.1.1. --- The `ssl_session_(fetch|store)_by_lua*` handlers will be skipped for TLS 1.3 connections. See: https://github.com/openresty/lua-nginx-module/issues/1539 https://github.com/openresty/lua-nginx-module/pull/1540 https://github.com/openresty/lua-nginx-module/pull/1544 --- .travis.yml | 11 +- README.markdown | 15 +-- doc/HttpLuaModule.wiki | 15 +-- src/ngx_http_lua_module.c | 6 - src/ngx_http_lua_ssl_session_fetchby.c | 156 +++++-------------------- src/ngx_http_lua_ssl_session_fetchby.h | 5 - 6 files changed, 46 insertions(+), 162 deletions(-) diff --git a/.travis.yml b/.travis.yml index 1ce1371a7b..ef3083ca54 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,9 +48,10 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2s OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0k OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.1c OPENSSL_PATCH_VER="" + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2s OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0k OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d + # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" + # to $OPENSSL_OPT. services: - memcache @@ -115,8 +116,8 @@ script: - cd .. - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - - if [ -n "$OPENSSL_PATCH_VER" ]; then patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; fi - - ./config no-threads shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) + - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch + - ./config no-threads shared enable-ssl3 enable-ssl3-method $OPENSSL_OPT -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. diff --git a/README.markdown b/README.markdown index fa605e1077..24597d6ca0 100644 --- a/README.markdown +++ b/README.markdown @@ -2623,18 +2623,15 @@ But do not forget to comment this line out before publishing your site to the wo If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. -If you are not using one of the [OpenSSL -packages](https://openresty.org/en/linux-packages.html) provided by -[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -1.0.2, up to (and including) 1.1.0: +If you are using OpenSSL libraries not provided by [OpenResty](https://openresty.org), +then you need to apply the following patch for OpenSSL 1.0.2h or later: - + -Similarly, if you are not using the NGINX core shipped with -[OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply -patches to the standard NGINX core: +If you are not using the NGINX core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, then you need to +apply the following patch to the standard NGINX core 1.11.2 or later: - + This directive was first introduced in the `v0.10.6` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 8223b5b013..6038fa49d6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2219,18 +2219,15 @@ But do not forget to comment this line out before publishing your site to the wo If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. -If you are not using one of the [OpenSSL -packages](https://openresty.org/en/linux-packages.html) provided by -[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -1.0.2, up to (and including) 1.1.0: +If you are using OpenSSL libraries not provided by [OpenResty](https://openresty.org), +then you need to apply the following patch for OpenSSL 1.0.2h or later: -https://openresty.org/en/openssl-patches.html +https://github.com/openresty/openresty/blob/master/patches/openssl-1.0.2h-sess_set_get_cb_yield.patch -Similarly, if you are not using the NGINX core shipped with -[OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply -patches to the standard NGINX core: +If you are not using the NGINX core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, then you need to +apply the following patch to the standard NGINX core 1.11.2 or later: -https://openresty.org/en/nginx-ssl-patches.html +http://openresty.org/download/nginx-1.11.2-nonblocking_ssl_handshake_hooks.patch This directive was first introduced in the v0.10.6 release. diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 331bd04152..2a3f5d57b6 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1077,12 +1077,6 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) return NGX_CONF_ERROR; #else -# ifdef HAVE_SSL_CLIENT_HELLO_CB_SUPPORT - SSL_CTX_set_client_hello_cb(sscf->ssl.ctx, - ngx_http_lua_ssl_client_hello_handler, - NULL); -# endif - SSL_CTX_sess_set_get_cb(sscf->ssl.ctx, ngx_http_lua_ssl_sess_fetch_handler); #endif diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 7ff0168655..303efb47aa 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -173,12 +173,16 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } -static ngx_int_t -ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, - const u_char *id, int len) +/* cached session fetching callback to be set with SSL_CTX_sess_set_get_cb */ +ngx_ssl_session_t * +ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, +#if OPENSSL_VERSION_NUMBER >= 0x10100003L + const +#endif + u_char *id, int len, int *copy) { lua_State *L; - ngx_int_t rc, res = NGX_ERROR; + ngx_int_t rc; ngx_connection_t *c, *fc = NULL; ngx_http_request_t *r = NULL; ngx_pool_cleanup_t *cln; @@ -187,6 +191,11 @@ ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, ngx_http_lua_srv_conf_t *lscf; ngx_http_core_loc_conf_t *clcf; + /* set copy to 0 as we expect OpenSSL to handle + * the memory of returned session */ + + *copy = 0; + c = ngx_ssl_get_connection(ssl_conn); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, @@ -208,10 +217,17 @@ ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, cctx->exit_code); dd("lua ssl sess_fetch done, finally"); - return NGX_OK; + return cctx->session; } - return NGX_AGAIN; +#ifdef SSL_ERROR_PENDING_SESSION + return SSL_magic_pending_session_ptr(); +#else + ngx_log_error(NGX_LOG_CRIT, c->log, 0, + "lua: cannot yield in sess get cb: " + "missing async sess get cb support in OpenSSL"); + return NULL; +#endif } dd("first time"); @@ -313,7 +329,7 @@ ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, "sess get cb exit code: %d", rc, cctx->exit_code); c->log->action = "SSL handshaking"; - return NGX_OK; + return cctx->session; } /* rc == NGX_DONE */ @@ -340,13 +356,12 @@ ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, *cctx->cleanup = ngx_http_lua_ssl_sess_fetch_aborted; -#if defined(SSL_ERROR_PENDING_SESSION) \ - || defined(HAVE_SSL_CLIENT_HELLO_CB_SUPPORT) - - return NGX_AGAIN; - +#ifdef SSL_ERROR_PENDING_SESSION + return SSL_magic_pending_session_ptr(); #else - res = NGX_AGAIN; + ngx_log_error(NGX_LOG_CRIT, c->log, 0, + "lua: cannot yield in sess get cb: " + "missing async sess get cb support in OpenSSL"); /* fall through to the "failed" label below */ #endif @@ -361,125 +376,10 @@ ngx_http_lua_ssl_sess_fetch_helper(ngx_ssl_conn_t *ssl_conn, ngx_http_lua_close_fake_connection(fc); } - return res; -} - - -#ifdef HAVE_SSL_CLIENT_HELLO_CB_SUPPORT -int -ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, - int *al, void *arg) -{ - int len; - ngx_int_t rc; - const u_char *id; - - len = SSL_client_hello_get0_session_id(ssl_conn, &id); - - if (len <= 0) { - return SSL_CLIENT_HELLO_SUCCESS; - } - - rc = ngx_http_lua_ssl_sess_fetch_helper(ssl_conn, id, len); - - if (rc == NGX_AGAIN) { - return SSL_CLIENT_HELLO_RETRY; - } - - return SSL_CLIENT_HELLO_SUCCESS; -} - - -ngx_ssl_session_t * -ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, - const u_char *id, int len, int *copy) -{ - ngx_connection_t *c; - ngx_http_lua_ssl_ctx_t *cctx; - - /* set copy to 0 as we expect OpenSSL to handle - * the memory of returned session */ - - *copy = 0; - - c = ngx_ssl_get_connection(ssl_conn); - - cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); - - if (cctx && cctx->done) { - return cctx->session; - } - return NULL; } -#else - -/* cached session fetching callback to be set with SSL_CTX_sess_set_get_cb */ -ngx_ssl_session_t * -ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, -#if OPENSSL_VERSION_NUMBER >= 0x10100003L - const -#endif - u_char *id, int len, int *copy) -{ - ngx_int_t rc; - ngx_connection_t *c; - ngx_http_lua_ssl_ctx_t *cctx; - - /* set copy to 0 as we expect OpenSSL to handle - * the memory of returned session */ - - *copy = 0; - - c = ngx_ssl_get_connection(ssl_conn); - - rc = ngx_http_lua_ssl_sess_fetch_helper(ssl_conn, id, len); - - if (rc == NGX_AGAIN) { - -#ifdef SSL_ERROR_PENDING_SESSION - - return SSL_magic_pending_session_ptr(); - -#else - - ngx_log_error(NGX_LOG_CRIT, c->log, 0, - "lua: cannot yield in sess get cb: " -# if OPENSSL_VERSION_NUMBER >= 0x1010100fL - "missing support for yielding during SSL handshake in " - "the nginx core; consider using the OpenResty releases " - "from https://openresty.org/en/download.html or apply " - "the nginx core patches yourself (see " - "https://openresty.org/en/nginx-ssl-patches.html)"); - -# else - "missing support for yielding during SSL handshake in " - "linked " OPENSSL_VERSION_TEXT "; consider using the " - "OpenResty releases from " - "https://openresty.org/en/download.html or apply " - "the OpenSSL patches yourself (see " - "https://openresty.org/en/openssl-patches.html)"); -# endif - - return NULL; - -#endif - } - - if (rc == NGX_ERROR) { - return NULL; - } - - /* rc == NGX_OK */ - - cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); - return cctx->session; -} -#endif - - static void ngx_http_lua_ssl_sess_fetch_done(void *data) { diff --git a/src/ngx_http_lua_ssl_session_fetchby.h b/src/ngx_http_lua_ssl_session_fetchby.h index 7a73ed96e5..50c6616dd2 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.h +++ b/src/ngx_http_lua_ssl_session_fetchby.h @@ -30,13 +30,8 @@ ngx_ssl_session_t *ngx_http_lua_ssl_sess_fetch_handler( const #endif u_char *id, int len, int *copy); - -#ifdef HAVE_SSL_CLIENT_HELLO_CB_SUPPORT -int ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, - int *al, void *arg); #endif -#endif /* NGX_HTTP_SSL */ #endif /* _NGX_HTTP_LUA_SSL_SESSION_FETCHBY_H_INCLUDED_ */ From 4e974ba96150d0f5493f7e41c7523f1a3c2e0bef Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 2 Jul 2019 12:07:06 -0700 Subject: [PATCH 320/848] doc: updated ssl_session_fetch_by_lua_block documentation around OpenSSL and NGINX core patches. --- README.markdown | 15 +++++++++------ doc/HttpLuaModule.wiki | 15 +++++++++------ 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/README.markdown b/README.markdown index 24597d6ca0..d040c9a60c 100644 --- a/README.markdown +++ b/README.markdown @@ -2623,15 +2623,18 @@ But do not forget to comment this line out before publishing your site to the wo If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. -If you are using OpenSSL libraries not provided by [OpenResty](https://openresty.org), -then you need to apply the following patch for OpenSSL 1.0.2h or later: +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +in order to use this hook: - + -If you are not using the NGINX core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, then you need to -apply the following patch to the standard NGINX core 1.11.2 or later: +Similarly, if you are not using the NGINX core shipped with +[OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply +patches to the standard NGINX core: - + This directive was first introduced in the `v0.10.6` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 6038fa49d6..4dcb27b2ae 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2219,15 +2219,18 @@ But do not forget to comment this line out before publishing your site to the wo If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. -If you are using OpenSSL libraries not provided by [OpenResty](https://openresty.org), -then you need to apply the following patch for OpenSSL 1.0.2h or later: +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +in order to use this hook: -https://github.com/openresty/openresty/blob/master/patches/openssl-1.0.2h-sess_set_get_cb_yield.patch +https://openresty.org/en/openssl-patches.html -If you are not using the NGINX core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, then you need to -apply the following patch to the standard NGINX core 1.11.2 or later: +Similarly, if you are not using the NGINX core shipped with +[OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply +patches to the standard NGINX core: -http://openresty.org/download/nginx-1.11.2-nonblocking_ssl_handshake_hooks.patch +https://openresty.org/en/nginx-ssl-patches.html This directive was first introduced in the v0.10.6 release. From 65bdd4a838a72dc7d9dedc2add58b72aa3a5dfd1 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sun, 30 Jun 2019 14:56:11 -0700 Subject: [PATCH 321/848] tests: 139-ssl-cert-by.t: downgraded error log level of SSL handshake failures from 'crit' to 'info'. See NGINX commit https://github.com/nginx/nginx/commit/99d7bb690924e60e9e03096ac5e507111f7c182d which landed in 1.15.10. --- t/139-ssl-cert-by.t | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index bebbdb27ae..23af6ebafa 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -574,7 +574,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua_certificate_by_lua: handler return value: -1, cert cb exit code: 0', -qr/\[crit\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, 'lua exit with code -1', ] @@ -715,7 +715,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua_certificate_by_lua: cert cb exit code: 0', -qr/\[crit\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, 'lua exit with code -1', ] @@ -786,7 +786,7 @@ failed to do SSL handshake: handshake failed [ 'runtime error: ssl_certificate_by_lua:2: bad bad bad', 'lua_certificate_by_lua: handler return value: 500, cert cb exit code: 0', -qr/\[crit\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, qr/context: ssl_certificate_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, ] @@ -858,7 +858,7 @@ failed to do SSL handshake: handshake failed [ 'runtime error: ssl_certificate_by_lua:3: bad bad bad', 'lua_certificate_by_lua: cert cb exit code: 0', -qr/\[crit\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, ] --- no_error_log @@ -1046,7 +1046,7 @@ failed to do SSL handshake: handshake failed [ 'lua ssl server name: "test.com"', 'ssl_certificate_by_lua:1: API disabled in the context of ssl_certificate_by_lua*', -qr/\[crit\] .*?cert cb error/, +qr/\[info\] .*?cert cb error/, ] --- no_error_log @@ -1775,7 +1775,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ qr/\[alert\] .*? no ssl_certificate_by_lua\* defined in server test2\.com\b/, -qr/\[crit\] .*? SSL_do_handshake\(\) failed\b/, +qr/\[info\] .*? SSL_do_handshake\(\) failed\b/, ] @@ -1870,7 +1870,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ qr/\[alert\] .*? no ssl_certificate_by_lua\* defined in server ~test2\\\.com\b/, -qr/\[crit\] .*? SSL_do_handshake\(\) failed\b/, +qr/\[info\] .*? SSL_do_handshake\(\) failed\b/, ] From 2524330e59f0a385a9c77d4d1b957476dce7cb33 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 9 Jul 2019 11:00:03 -0700 Subject: [PATCH 322/848] travis-ci: bumped the NGINX core version to 1.17.1. --- .travis.yml | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index ef3083ca54..9a9cf29301 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,10 +48,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.0.2s OPENSSL_OPT="" OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.15.8 OPENSSL_VER=1.1.0k OPENSSL_OPT="" OPENSSL_PATCH_VER=1.1.0d - # TODO: when adding an OpenSSL version >= 1.1.1, please add "enable-tls1_3" - # to $OPENSSL_OPT. + - NGINX_VERSION=1.17.1 OPENSSL_VER=1.0.2s OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.0k OPENSSL_PATCH_VER=1.1.0d services: - memcache @@ -117,7 +115,7 @@ script: - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch - - ./config no-threads shared enable-ssl3 enable-ssl3-method $OPENSSL_OPT -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) + - ./config no-threads shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. From 277d382f51067c0550a8264cb278e1364ba40e9c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 23 May 2019 16:59:09 -0700 Subject: [PATCH 323/848] api: bumped API version to 0.10.16. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index d040c9a60c..3e9c9c1d41 100644 --- a/README.markdown +++ b/README.markdown @@ -63,7 +63,7 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.15](https://github.com/openresty/lua-nginx-module/tags) released on March 14th, 2019. +This document describes ngx_lua [v0.10.16](https://github.com/openresty/lua-nginx-module/tags), which is not released yet. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4dcb27b2ae..5910732d55 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -12,7 +12,7 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.15] released on March 14th, 2019. +This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.16], which is not released yet. = Synopsis = diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 129eb9979c..221e423ddb 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10015 +#define ngx_http_lua_version 10016 typedef struct { From 2014dd80fe4f299ad68c3b60a2c5f846b4286b6f Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 10 Jul 2019 22:41:29 -0700 Subject: [PATCH 324/848] bugfix: ensured the 'pcre_version' symbol is preserved as undefined when PCRE is statically linked. When using `--with-pcre=...`, NGINX is statically linked against libpcre.a. Since `pcre_version()` is unused, its symbol is stripped by the linker. Because lua-resty-core's `resty.core.regex` module needs it, we here ensure that the symbol is entered as undefined in our final binary. This should not be an issue for win32 builds since `--export-all-symbols` is already set. --- config | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/config b/config index 3ef8f0749f..ff44482bc4 100644 --- a/config +++ b/config @@ -588,5 +588,27 @@ else CORE_LIBS="$CORE_LIBS $ngx_module_libs" fi +# ---------------------------------------- + +if [ $PCRE != NO -a $PCRE != YES ]; then + # force pcre_version symbol to be undefined when PCRE is statically linked + + ngx_feature="force undefined symbols (--undefined)" + ngx_feature_libs="-Wl,--undefined=printf" + ngx_feature_name= + ngx_feature_run=no + ngx_feature_incs="#include " + ngx_feature_path= + ngx_feature_test='printf("hello");' + + . auto/feature + + if [ $ngx_found = yes ]; then + CORE_LIBS="$CORE_LIBS -Wl,--undefined=pcre_version" + fi +fi + +# ---------------------------------------- + #CFLAGS=$"$CFLAGS -DLUA_DEFAULT_PATH='\"/usr/local/openresty/lualib/?.lua\"'" #CFLAGS=$"$CFLAGS -DLUA_DEFAULT_CPATH='\"/usr/local/openresty/lualib/?.so\"'" From d3dbc0c8102a9978d649c99e3261d93aac547378 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 17 Jul 2019 10:56:21 -0700 Subject: [PATCH 325/848] bugfix: skipped 'ssl_session_store_by_lua*' and 'ssl_session_fetch_by_lua*' when using TLS 1.3. Previously, we used the OpenSSL 1.1.1 ClientHello callback to do ssl session fetching non-blockingly. However, this way cannot handle an edge case: the ssl session resumption via session ticket might fail, and the client fallbacks to session ID resumption. The ClientHello callback is run too early to know if the client will fallback to use session ID resumption. Therefore, we have to take back the OpenSSL sess_set_get_cb_yield patch and upgrade it to adapt OpenSSL 1.1.1, which was done in our openresty/openresty repository. Now, this means that for the time being, we must skip `ssl_session_(fetch|store)_by_lua*` for TLS 1.3. It is possible to support PSK with session ID in TLS 1.3., but we need to modify a number of functions to pass the result up, which will make the patch too complex to maintain. Since PSK with session ticket is supported, supporting PSK with session ID is not so profitable. If someone needs this feature, they can contribute it themselves. Thanks Yongjian Xu and crasyangel for their help. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_ssl_session_fetchby.c | 15 +++ src/ngx_http_lua_ssl_session_storeby.c | 15 +++ t/142-ssl-session-store.t | 62 ++++++++++++ t/143-ssl-session-fetch.t | 134 +++++++++++++++++++++++++ 4 files changed, 226 insertions(+) diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 303efb47aa..17e41a97de 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -181,6 +181,9 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, #endif u_char *id, int len, int *copy) { +#if defined(NGX_SSL_TLSv1_3) && defined(TLS1_3_VERSION) + int tls_version; +#endif lua_State *L; ngx_int_t rc; ngx_connection_t *c, *fc = NULL; @@ -198,6 +201,18 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, c = ngx_ssl_get_connection(ssl_conn); +#if defined(NGX_SSL_TLSv1_3) && defined(TLS1_3_VERSION) + tls_version = SSL_version(ssl_conn); + + if (tls_version >= TLS1_3_VERSION) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "ssl_session_fetch_by_lua*: skipped since " + "TLS version >= 1.3 (%xd)", tls_version); + + return 0; + } +#endif + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "ssl session fetch: connection reusable: %ud", c->reusable); diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index a02f729ced..f5016418e0 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -176,6 +176,9 @@ int ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, ngx_ssl_session_t *sess) { +#if defined(NGX_SSL_TLSv1_3) && defined(TLS1_3_VERSION) + int tls_version; +#endif const u_char *sess_id; unsigned int sess_id_len; lua_State *L; @@ -189,6 +192,18 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, c = ngx_ssl_get_connection(ssl_conn); +#if defined(NGX_SSL_TLSv1_3) && defined(TLS1_3_VERSION) + tls_version = SSL_version(ssl_conn); + + if (tls_version >= TLS1_3_VERSION) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "ssl_session_store_by_lua*: skipped since " + "TLS version >= 1.3 (%xd)", tls_version); + + return 0; + } +#endif + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "ssl session store: connection reusable: %ud", c->reusable); diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index cc3a664cfb..14e90574b0 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -901,3 +901,65 @@ ssl_session_store_by_lua_block:1: ssl session store by lua is running! --- no_error_log [error] [alert] + + + +=== TEST 13: ssl_session_store_by_lua* is skipped when using TLSv1.3 +--- skip_openssl: 6: < 1.1.1 +--- http_config + ssl_session_store_by_lua_block { ngx.log(ngx.ERR, "ssl_session_store_by_lua* is running!") } + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; + ssl_protocols TLSv1.3; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- error_log eval +qr/ssl_session_store_by_lua\*: skipped since TLS version >= 1\.3 \(\d+\)/ +--- no_error_log +[error] +[alert] +[emerg] diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 78a66cf9b1..bb97adc818 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -1219,3 +1219,137 @@ GET /t [error] [alert] [emerg] + + + +=== TEST 15: ssl_session_fetch_by_lua* is skipped when session ticket is provided +--- http_config + ssl_session_fetch_by_lua_block { ngx.log(ngx.ERR, "ssl_session_fetch_by_lua* is running!") } + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + package.loaded.session = sess + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- no_error_log +[warn] +[error] +[alert] +[emerg] + + + +=== TEST 16: ssl_session_fetch_by_lua* always runs when using SSLv3 (SSLv3 does not support session tickets) +--- http_config + ssl_session_fetch_by_lua_block { print("ssl_session_fetch_by_lua* is running!") } + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols SSLv3; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols SSLv3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + package.loaded.session = sess + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- grep_error_log eval: qr/ssl_session_fetch_by_lua_block:.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log_out eval +[ +qr/\A(?:reusable connection: [01]\n)+\z/s, +qr/^reusable connection: 1 +ssl session fetch: connection reusable: 1 +reusable connection: 0 +ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +/m, +qr/^reusable connection: 1 +ssl session fetch: connection reusable: 1 +reusable connection: 0 +ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +/m, +] +--- no_error_log +[error] +[alert] +[emerg] From 467f0134dbf53737a0cf26de7ccb2c35bd490d85 Mon Sep 17 00:00:00 2001 From: spacewander Date: Wed, 17 Jul 2019 13:30:51 -0700 Subject: [PATCH 326/848] tests: added tests asserting that 'ssl_certificate_by_lua*' and 'ssl_session_fetch_by_lua*' can yield when reading early data. Signed-off-by: Thibault Charbonnier --- t/139-ssl-cert-by.t | 67 ++++++++++++++++++++++++++++++++++ t/143-ssl-session-fetch.t | 76 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 143 insertions(+) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 23af6ebafa..8b5517c494 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -2075,3 +2075,70 @@ client socket file: --- no_error_log [error] [alert] + + + +=== TEST 24: ssl_certificate_by_lua* can yield when reading early data +--- skip_openssl: 6: < 1.1.1 +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_early_data on; + server_tokens off; + + ssl_certificate_by_lua_block { + local begin = ngx.now() + ngx.sleep(0.1) + print("elapsed in ssl_certificate_by_lua*: ", ngx.now() - begin) + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: boolean +--- grep_error_log eval +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, +--- grep_error_log_out eval +[ +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, +] +--- no_error_log +[error] +[alert] +[emerg] diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index bb97adc818..df9972cb46 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -1353,3 +1353,79 @@ ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, [error] [alert] [emerg] + + + +=== TEST 17: ssl_session_fetch_by_lua* can yield when reading early data +--- skip_openssl: 6: < 1.1.1 +--- http_config + ssl_session_fetch_by_lua_block { + local begin = ngx.now() + ngx.sleep(0.1) + print("elapsed in ssl_session_fetch_by_lua*: ", ngx.now() - begin) + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; + ssl_early_data on; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + package.loaded.session = sess + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- grep_error_log eval +qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, +--- grep_error_log_out eval +[ +'', +qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, +qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, +] +--- no_error_log +[error] +[alert] +[emerg] From de60e555a3463f78a05f91ac28641d623f556c2f Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 17 Jul 2019 13:38:34 -0700 Subject: [PATCH 327/848] travis: added OpenSSL 1.1.1c to the tests matrix. --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 9a9cf29301..e50de2cdbb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,6 +50,7 @@ env: matrix: - NGINX_VERSION=1.17.1 OPENSSL_VER=1.0.2s OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.0k OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.1c OPENSSL_PATCH_VER=1.1.1c services: - memcache From cf77a42c4ec4cc87734c0179fde972e7f2d204f7 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Sat, 13 Jul 2019 13:09:28 +0800 Subject: [PATCH 328/848] doc: removed TODO items that have been taken care of. Signed-off-by: Thibault Charbonnier --- README.markdown | 18 +----------------- doc/HttpLuaModule.wiki | 17 +---------------- 2 files changed, 2 insertions(+), 33 deletions(-) diff --git a/README.markdown b/README.markdown index 3e9c9c1d41..326b26e1be 100644 --- a/README.markdown +++ b/README.markdown @@ -858,28 +858,12 @@ TODO ==== * cosocket: implement LuaSocket's unconnected UDP API. -* port this module to the "datagram" subsystem of NGINX for implementing general UDP servers instead of HTTP -servers in Lua. For example, -```lua - - datagram { - server { - listen 1953; - handler_by_lua_block { - -- custom Lua code implementing the special UDP server... - } - } - } -``` -* shm: implement a "shared queue API" to complement the existing [shared dict](#lua_shared_dict) API. * cosocket: add support in the context of [init_by_lua*](#init_by_lua). * cosocket: implement the `bind()` method for stream-typed cosockets. -* cosocket: pool-based backend concurrency level control: implement automatic `connect` queueing when the backend concurrency exceeds its connection pool limit. * cosocket: review and merge aviramc's [patch](https://github.com/openresty/lua-nginx-module/pull/290) for adding the `bsdrecv` method. -* add new API function `ngx.resp.add_header` to emulate the standard `add_header` config directive. +* cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option * use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), [ngx.header.HEADER](#ngxheaderheader), and etc. -* add configure options for different strategies of handling the cosocket connection exceeding in the pools. * add directives to run Lua codes when nginx stops. * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 5910732d55..fd255de4c6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -698,27 +698,12 @@ phases. = TODO = * cosocket: implement LuaSocket's unconnected UDP API. -* port this module to the "datagram" subsystem of NGINX for implementing general UDP servers instead of HTTP -servers in Lua. For example, - - datagram { - server { - listen 1953; - handler_by_lua_block { - -- custom Lua code implementing the special UDP server... - } - } - } - -* shm: implement a "shared queue API" to complement the existing [[#lua_shared_dict|shared dict]] API. * cosocket: add support in the context of [[#init_by_lua|init_by_lua*]]. * cosocket: implement the bind() method for stream-typed cosockets. -* cosocket: pool-based backend concurrency level control: implement automatic connect queueing when the backend concurrency exceeds its connection pool limit. * cosocket: review and merge aviramc's [https://github.com/openresty/lua-nginx-module/pull/290 patch] for adding the bsdrecv method. -* add new API function ngx.resp.add_header to emulate the standard add_header config directive. +* cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [[#ngx.location.capture|ngx.location.capture]]'s extra_headers option * use ngx_hash_t to optimize the built-in header look-up process for [[#ngx.req.set_header|ngx.req.set_header]], [[#ngx.header.HEADER|ngx.header.HEADER]], and etc. -* add configure options for different strategies of handling the cosocket connection exceeding in the pools. * add directives to run Lua codes when nginx stops. * add ignore_resp_headers, ignore_resp_body, and ignore_resp options to [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. From b6875300b7bd2425fd6f93de1e55e47685425b18 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 17 Jul 2019 17:18:48 -0700 Subject: [PATCH 329/848] doc: mentioned that we support NGINX 1.17.x. --- README.markdown | 1 + doc/HttpLuaModule.wiki | 1 + 2 files changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index 326b26e1be..45a5ae2dcb 100644 --- a/README.markdown +++ b/README.markdown @@ -252,6 +252,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.17.x (last tested: 1.17.1) * 1.15.x (last tested: 1.15.8) * 1.14.x * 1.13.x (last tested: 1.13.6) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index fd255de4c6..59170d47c9 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -190,6 +190,7 @@ The Lua state (Lua VM instance) is shared across all the requests handled by a s The latest version of this module is compatible with the following versions of Nginx: +* 1.17.x (last tested: 1.17.1) * 1.15.x (last tested: 1.15.8) * 1.14.x * 1.13.x (last tested: 1.13.6) From 09484e8fda2f1c1970a898a8c08111af13d69930 Mon Sep 17 00:00:00 2001 From: wangfakang Date: Wed, 10 Jul 2019 17:56:49 +0800 Subject: [PATCH 330/848] bugfix: ensured 'init_worker_by_lua*' does not mutate another NGINX module's main_conf. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_initworkerby.c | 22 ++- t/124-init-worker.t | 22 +++ t/data/fake-merge-module/config | 3 + .../ngx_http_fake_merge_module.c | 187 ++++++++++++++++++ util/build.sh | 2 + 5 files changed, 235 insertions(+), 1 deletion(-) create mode 100644 t/data/fake-merge-module/config create mode 100644 t/data/fake-merge-module/ngx_http_fake_merge_module.c diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 76acdc3bb4..5e02a83860 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -86,7 +86,6 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) } conf_ctx = (ngx_http_conf_ctx_t *) cycle->conf_ctx[ngx_http_module.index]; - http_ctx.main_conf = conf_ctx->main_conf; top_clcf = conf_ctx->loc_conf[ngx_http_core_module.ctx_index]; top_llcf = conf_ctx->loc_conf[ngx_http_lua_module.ctx_index]; @@ -197,6 +196,12 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) return NGX_ERROR; } + http_ctx.main_conf = ngx_pcalloc(conf.pool, + sizeof(void *) * ngx_http_max_module); + if (http_ctx.main_conf == NULL) { + return NGX_ERROR; + } + #if defined(nginx_version) && nginx_version >= 1009011 modules = cycle->modules; #else @@ -210,6 +215,21 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) module = modules[i]->ctx; + if (module->create_main_conf) { + cur = module->create_main_conf(&conf); + if (cur == NULL) { + return NGX_ERROR; + } + + if (ngx_modules[i]->index == ngx_http_lua_module.index) { + ngx_memcpy(cur, + conf_ctx->main_conf[ngx_http_lua_module.ctx_index], + sizeof(ngx_http_lua_main_conf_t)); + } + + http_ctx.main_conf[modules[i]->ctx_index] = cur; + } + if (module->create_srv_conf) { cur = module->create_srv_conf(&conf); if (cur == NULL) { diff --git a/t/124-init-worker.t b/t/124-init-worker.t index b648d55e61..1a82ef23b2 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -955,3 +955,25 @@ qr/lua close the global Lua VM ([0-9A-F]+)$/, --- no_error_log [error] start privileged agent process + + + +=== TEST 25: ensure it does not mutate another module's main_conf (github issue #1553) +https://github.com/openresty/lua-nginx-module/issues/1553 +--- http_config + init_worker_by_lua_block { + return + } +--- config + location /t { + content_by_lua_block { + ngx.say("fake_var = ", ngx.var.fake_var) + } + } +--- request +GET /t +--- response_body +fake_var = 1 +--- no_error_log +[error] +[alert] diff --git a/t/data/fake-merge-module/config b/t/data/fake-merge-module/config new file mode 100644 index 0000000000..f6dbf60e7b --- /dev/null +++ b/t/data/fake-merge-module/config @@ -0,0 +1,3 @@ +ngx_addon_name=ngx_http_fake_merge_module +HTTP_MODULES="$HTTP_MODULES ngx_http_fake_merge_module" +NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fake_merge_module.c" diff --git a/t/data/fake-merge-module/ngx_http_fake_merge_module.c b/t/data/fake-merge-module/ngx_http_fake_merge_module.c new file mode 100644 index 0000000000..579f62332c --- /dev/null +++ b/t/data/fake-merge-module/ngx_http_fake_merge_module.c @@ -0,0 +1,187 @@ +/* + * This fake module was used to reproduce a bug in ngx_lua's init_worker_by_lua + * implementation. The bug would cause this module's main_conf->a to be reset + * to 0 due to init_worker_by_lua invoking merge_loc_conf with a brand new + * loc_conf, whose a member would be 0. + * The test case for this bug is in 124-init-worker.t. + */ + + +#include +#include +#include +#include + + +typedef struct { + ngx_flag_t a; +} ngx_http_fake_merge_main_conf_t; + + +typedef struct { + ngx_flag_t a; +} ngx_http_fake_merge_loc_conf_t; + + +static ngx_int_t ngx_http_fake_merge_var(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data); +static ngx_int_t ngx_http_fake_merge_add_variables(ngx_conf_t *cf); +static ngx_int_t ngx_http_fake_merge_init(ngx_conf_t *cf); +static void *ngx_http_fake_merge_create_main_conf(ngx_conf_t *cf); +static void *ngx_http_fake_merge_create_loc_conf(ngx_conf_t *cf); +static char *ngx_http_fake_merge_merge_loc_conf(ngx_conf_t *cf, void *prev, + void *conf); + + +static ngx_http_module_t ngx_http_fake_merge_module_ctx = { + ngx_http_fake_merge_init, /* preconfiguration */ + NULL, /* postconfiguration */ + + ngx_http_fake_merge_create_main_conf, /* create main configuration */ + NULL, /* init main configuration */ + + NULL, /* create server configuration */ + NULL, /* merge server configuration */ + + ngx_http_fake_merge_create_loc_conf, /* create location configuration */ + ngx_http_fake_merge_merge_loc_conf /* merge location configuration */ +}; + + +ngx_module_t ngx_http_fake_merge_module = { + NGX_MODULE_V1, + &ngx_http_fake_merge_module_ctx, /* module context */ + NULL, /* module directives */ + NGX_HTTP_MODULE, /* module type */ + NULL, /* init master */ + NULL, /* init module */ + NULL, /* init process */ + NULL, /* init thread */ + NULL, /* exit thread */ + NULL, /* exit process */ + NULL, /* exit master */ + NGX_MODULE_V1_PADDING +}; + + +static ngx_http_variable_t ngx_http_fake_merge_variables[] = { + + { ngx_string("fake_var"), NULL, + ngx_http_fake_merge_var, 0, + NGX_HTTP_VAR_NOCACHEABLE, 0 }, + + { ngx_null_string, NULL, NULL, 0, 0, 0 } +}; + + +static ngx_int_t +ngx_http_fake_merge_var(ngx_http_request_t *r, ngx_http_variable_value_t *v, + uintptr_t data) +{ + static char *str[] = {"0", "1"}; + ngx_http_fake_merge_main_conf_t *fmcf; + + fmcf = ngx_http_get_module_main_conf(r, ngx_http_fake_merge_module); + if (fmcf == NULL) { + return NGX_ERROR; + } + + v->len = 1; + v->data = (u_char *) str[fmcf->a]; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_fake_merge_add_variables(ngx_conf_t *cf) +{ + ngx_http_variable_t *var, *v; + + for (v = ngx_http_fake_merge_variables; v->name.len; v++) { + var = ngx_http_add_variable(cf, &v->name, v->flags); + if (var == NULL) { + return NGX_ERROR; + } + + var->get_handler = v->get_handler; + var->data = v->data; + } + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_fake_merge_init(ngx_conf_t *cf) +{ + ngx_http_fake_merge_loc_conf_t *flcf; + + flcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_fake_merge_module); + if (flcf == NULL) { + return NGX_ERROR; + } + + if (ngx_http_fake_merge_add_variables(cf) != NGX_OK) { + return NGX_ERROR; + } + + flcf->a = 1; + + return NGX_OK; +} + + +static void * +ngx_http_fake_merge_create_main_conf(ngx_conf_t *cf) +{ + ngx_http_fake_merge_main_conf_t *fmcf; + + fmcf = ngx_pcalloc(cf->pool, sizeof(ngx_http_fake_merge_main_conf_t)); + if (fmcf == NULL) { + return NULL; + } + + fmcf->a = NGX_CONF_UNSET; + + return fmcf; +} + + +static void * +ngx_http_fake_merge_create_loc_conf(ngx_conf_t *cf) +{ + ngx_http_fake_merge_loc_conf_t *flcf; + + flcf = ngx_pcalloc(cf->pool, sizeof(ngx_http_fake_merge_loc_conf_t)); + if (flcf == NULL) { + return NULL; + } + + flcf->a = NGX_CONF_UNSET; + + return flcf; +} + + +static char * +ngx_http_fake_merge_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) +{ + ngx_http_fake_merge_loc_conf_t *conf = child; + ngx_http_fake_merge_loc_conf_t *prev = parent; + ngx_http_fake_merge_main_conf_t *fmcf; + + ngx_conf_merge_value(conf->a, prev->a, 0); + + fmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_fake_merge_module); + if (fmcf == NULL) { + return NGX_CONF_ERROR; + } + + fmcf->a = conf->a; + + return NGX_CONF_OK; +} diff --git a/util/build.sh b/util/build.sh index 14e9fd46d9..5f9b1d5e07 100755 --- a/util/build.sh +++ b/util/build.sh @@ -23,6 +23,7 @@ force=$2 #--with-http_spdy_module \ add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" +add_fake_merge_module="--add-module=$root/t/data/fake-merge-module" time ngx-build $force $version \ --with-pcre-jit \ @@ -58,6 +59,7 @@ time ngx-build $force $version \ --add-module=$root/../stream-lua-nginx-module \ --add-module=$root/t/data/fake-module \ $add_fake_shm_module \ + $add_fake_merge_module \ --add-module=$root/t/data/fake-delayed-load-module \ --with-http_gunzip_module \ --with-http_dav_module \ From e42f9b5f6520d7d5f51b6742134377f66bcdf841 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 25 Jul 2019 10:39:17 -0700 Subject: [PATCH 331/848] doc: updated the section on OpenSSL and NGINX patches in the 'ssl_certificate_by_lua*' documentation. --- README.markdown | 28 ++++++++++++++++++++-------- doc/HttpLuaModule.wiki | 28 ++++++++++++++++++++-------- 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/README.markdown b/README.markdown index 45a5ae2dcb..de4a35ea5f 100644 --- a/README.markdown +++ b/README.markdown @@ -2529,14 +2529,25 @@ while starting NGINX: nginx: [emerg] no ssl configured for the server -This directive currently requires the following NGINX core patch to work correctly: +This directive requires OpenSSL 1.0.2e or greater. - +If you are using the [official pre-built +packages](http://openresty.org/en/linux-packages.html) for +[OpenResty](https://openresty.org/) 1.9.7.2 or later, then everything should +work out of the box. -The bundled version of the NGINX core in OpenResty 1.9.7.2 (or above) already has this -patch applied. +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +in order to use this directive: -Furthermore, one needs at least OpenSSL 1.0.2e for this directive to work. + + +Similarly, if you are not using the NGINX core shipped with +[OpenResty](https://openresty.org) 1.9.7.2 or later, you will need to apply +patches to the standard NGINX core: + + This directive was first introduced in the `v0.10.0` release. @@ -2611,7 +2622,7 @@ If you are using the [official pre-built packages](http://openresty.org/en/linux If you are not using one of the [OpenSSL packages](https://openresty.org/en/linux-packages.html) provided by [OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -in order to use this hook: +in order to use this directive: @@ -2623,8 +2634,9 @@ patches to the standard NGINX core: This directive was first introduced in the `v0.10.6` release. -Note that: this directive is only allowed to used in **http context** from the `v0.10.7` release -(because SSL session resumption happens before server name dispatch). +Note that this directive can only be used in the **http context** starting +with the `v0.10.7` release since SSL session resumption happens +before server name dispatch. [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 59170d47c9..ad829795b4 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2132,14 +2132,25 @@ while starting NGINX: nginx: [emerg] no ssl configured for the server -This directive currently requires the following NGINX core patch to work correctly: +This directive requires OpenSSL 1.0.2e or greater. -http://mailman.nginx.org/pipermail/nginx-devel/2016-January/007748.html +If you are using the [official pre-built +packages](http://openresty.org/en/linux-packages.html) for +[OpenResty](https://openresty.org/) 1.9.7.2 or later, then everything should +work out of the box. -The bundled version of the NGINX core in OpenResty 1.9.7.2 (or above) already has this -patch applied. +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +in order to use this directive: -Furthermore, one needs at least OpenSSL 1.0.2e for this directive to work. +https://openresty.org/en/openssl-patches.html + +Similarly, if you are not using the NGINX core shipped with +[OpenResty](https://openresty.org) 1.9.7.2 or later, you will need to apply +patches to the standard NGINX core: + +https://openresty.org/en/nginx-ssl-patches.html This directive was first introduced in the v0.10.0 release. @@ -2208,7 +2219,7 @@ If you are using the [official pre-built packages](http://openresty.org/en/linux If you are not using one of the [OpenSSL packages](https://openresty.org/en/linux-packages.html) provided by [OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -in order to use this hook: +in order to use this directive: https://openresty.org/en/openssl-patches.html @@ -2220,8 +2231,9 @@ https://openresty.org/en/nginx-ssl-patches.html This directive was first introduced in the v0.10.6 release. -Note that: this directive is only allowed to used in '''http context''' from the v0.10.7 release -(because SSL session resumption happens before server name dispatch). +Note that this directive can only be used in the '''http context''' starting +with the v0.10.7 release since SSL session resumption happens +before server name dispatch. == ssl_session_fetch_by_lua_file == From bf11f67d8cddec3554baf7c4c0142b2f1db984ab Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 10 Jun 2019 16:55:01 -0700 Subject: [PATCH 332/848] tests: codecache.t: added tests for code cache behavior with *_by_lua_file directives containing NGINX variables. --- t/025-codecache.t | 245 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 244 insertions(+), 1 deletion(-) diff --git a/t/025-codecache.t b/t/025-codecache.t index 22b78b2136..6290780e29 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 163; +plan tests => repeat_each() * 181; #$ENV{LUA_PATH} = $ENV{HOME} . '/work/JSON4Lua-0.9.30/json/?.lua'; @@ -1374,3 +1374,246 @@ looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' --- log_level: debug --- no_error_log [error] + + + +=== TEST 34: variables in rewrite_by_lua_file's file path +--- config + location ~ ^/lua/(.+)$ { + rewrite_by_lua_file html/$1.lua; + } + + location /main { + echo_location /lua/a; + echo_location /lua/b; + echo_location /lua/a; + echo_location /lua/a; + echo_location /lua/b; + } +--- user_files +>>> a.lua +ngx.say("a") +>>> b.lua +ngx.say("b") +--- request +GET /main +--- response_body +a +b +a +a +b +--- no_error_log +[error] + + + +=== TEST 35: variables in access_by_lua_file's file path +--- config + location ~ ^/lua/(.+)$ { + access_by_lua_file html/$1.lua; + + content_by_lua_block { + return + } + } + + location ~ ^/proxy/(.+)$ { + proxy_pass http://127.0.0.1:$server_port/lua/$1; + } + + location /main { + content_by_lua_block { + local res1, res2, res3, res4, res5 = ngx.location.capture_multi{ + { "/proxy/a" }, + { "/proxy/b" }, + { "/proxy/a" }, + { "/proxy/a" }, + { "/proxy/b" }, + } + + ngx.say(res1.body) + ngx.say(res2.body) + ngx.say(res3.body) + ngx.say(res4.body) + ngx.say(res5.body) + } + } +--- user_files +>>> a.lua +ngx.print("a") +>>> b.lua +ngx.print("b") +--- request +GET /main +--- response_body +a +b +a +a +b +--- no_error_log +[error] + + + +=== TEST 36: variables in content_by_lua_file's file path +--- config + location ~ ^/lua/(.+)$ { + content_by_lua_file html/$1.lua; + } + + location /main { + echo_location /lua/a; + echo_location /lua/b; + echo_location /lua/a; + echo_location /lua/a; + echo_location /lua/b; + } +--- user_files +>>> a.lua +ngx.say("a") +>>> b.lua +ngx.say("b") +--- request +GET /main +--- response_body +a +b +a +a +b +--- no_error_log +[error] + + + +=== TEST 37: variables in header_filter_by_lua_file's file path +--- config + location ~ ^/lua/(.+)$ { + return 200; + + header_filter_by_lua_file html/$1.lua; + } + + location ~ ^/proxy/(.+)$ { + proxy_pass http://127.0.0.1:$server_port/lua/$1; + } + + location /main { + content_by_lua_block { + local res1, res2, res3, res4, res5 = ngx.location.capture_multi{ + { "/proxy/a" }, + { "/proxy/b" }, + { "/proxy/a" }, + { "/proxy/a" }, + { "/proxy/b" }, + } + + ngx.say(res1.header.match) + ngx.say(res2.header.match) + ngx.say(res3.header.match) + ngx.say(res4.header.match) + ngx.say(res5.header.match) + } + } +--- user_files +>>> a.lua +ngx.header.match = "a" +>>> b.lua +ngx.header.match = "b" +--- request +GET /main +--- response_body +a +b +a +a +b +--- no_error_log +[error] + + + +=== TEST 38: variables in body_filter_by_lua_file's file path +--- config + location ~ ^/lua/(.+)$ { + echo hello; + + body_filter_by_lua_file html/$1.lua; + } + + location /main { + echo_location /lua/a; + echo_location /lua/b; + echo_location /lua/a; + echo_location /lua/a; + echo_location /lua/b; + } +--- user_files +>>> a.lua +ngx.arg[1] = "a\n" +ngx.arg[2] = true +>>> b.lua +ngx.arg[1] = "b\n" +ngx.arg[2] = true +--- request +GET /main +--- response_body +a +b +a +a +b +--- no_error_log +[error] + + + +=== TEST 39: variables in log_by_lua_file's file path +--- config + log_subrequest on; + + location ~ ^/lua/(.+)$ { + echo hello; + + log_by_lua_file html/$1.lua; + } + + location /main { + echo_location /lua/a; + echo_location /lua/b; + echo_location /lua/a; + echo_location /lua/a; + echo_location /lua/b; + } +--- user_files +>>> a.lua +ngx.log(ngx.NOTICE, "grep me: a") +>>> b.lua +ngx.log(ngx.NOTICE, "grep me: b") +--- request +GET /main +--- ignore_response_body +--- grep_error_log eval: qr/grep me: ([ab])/ +--- grep_error_log_out eval +[ +"grep me: a +grep me: b +grep me: a +grep me: a +grep me: b +", +"grep me: a +grep me: b +grep me: a +grep me: a +grep me: b +grep me: a +grep me: b +grep me: a +grep me: a +grep me: b +"] +--- no_error_log +[error] From 39c2ead3cb7efae4456494ea27a348fa1eca6c5c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 11 Jun 2019 16:55:01 -0700 Subject: [PATCH 333/848] optimize: improved code cache lookup performance by using 'luaL_ref()' to avoid the costly 'lj_str_new()'. --- src/ngx_http_lua_accessby.c | 2 + src/ngx_http_lua_balancer.c | 2 + src/ngx_http_lua_bodyfilterby.c | 2 + src/ngx_http_lua_cache.c | 95 +++++++++++++++++--------- src/ngx_http_lua_cache.h | 4 +- src/ngx_http_lua_common.h | 14 +++- src/ngx_http_lua_contentby.c | 2 + src/ngx_http_lua_directive.c | 3 + src/ngx_http_lua_headerfilterby.c | 2 + src/ngx_http_lua_logby.c | 2 + src/ngx_http_lua_module.c | 31 +++++++-- src/ngx_http_lua_rewriteby.c | 2 + src/ngx_http_lua_ssl_certby.c | 2 + src/ngx_http_lua_ssl_session_fetchby.c | 2 + src/ngx_http_lua_ssl_session_storeby.c | 2 + t/025-codecache.t | 78 ++++++++++++++------- 16 files changed, 179 insertions(+), 66 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index d3fe294917..1cbf629a16 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -179,6 +179,7 @@ ngx_http_lua_access_handler_inline(ngx_http_request_t *r) rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->access_src.value.data, llcf->access_src.value.len, + &llcf->access_src_ref, llcf->access_src_key, (const char *) llcf->access_chunkname); @@ -217,6 +218,7 @@ ngx_http_lua_access_handler_file(ngx_http_request_t *r) /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &llcf->access_src_ref, llcf->access_src_key); if (rc != NGX_OK) { if (rc < NGX_HTTP_SPECIAL_RESPONSE) { diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 3ecd592917..9277122544 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -66,6 +66,7 @@ ngx_http_lua_balancer_handler_file(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadfile(r->connection->log, L, lscf->balancer.src.data, + &lscf->balancer.src_ref, lscf->balancer.src_key); if (rc != NGX_OK) { return rc; @@ -87,6 +88,7 @@ ngx_http_lua_balancer_handler_inline(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, lscf->balancer.src.data, lscf->balancer.src.len, + &lscf->balancer.src_ref, lscf->balancer.src_key, "=balancer_by_lua"); if (rc != NGX_OK) { diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index f3af14c164..fbb3eb6077 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -162,6 +162,7 @@ ngx_http_lua_body_filter_inline(ngx_http_request_t *r, ngx_chain_t *in) rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->body_filter_src.value.data, llcf->body_filter_src.value.len, + &llcf->body_filter_src_ref, llcf->body_filter_src_key, "=body_filter_by_lua"); if (rc != NGX_OK) { @@ -209,6 +210,7 @@ ngx_http_lua_body_filter_file(ngx_http_request_t *r, ngx_chain_t *in) /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &llcf->body_filter_src_ref, llcf->body_filter_src_key); if (rc != NGX_OK) { return NGX_ERROR; diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index 5b29527c68..eb2f75db4a 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -33,7 +33,7 @@ * */ static ngx_int_t ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, - const char *key) + int *ref, const char *key) { #ifndef OPENRESTY_LUAJIT int rc; @@ -45,16 +45,45 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, code_cache_key)); lua_rawget(L, LUA_REGISTRYINDEX); /* sp++ */ - dd("Code cache table to load: %p", lua_topointer(L, -1)); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, + "code cache lookup (key='%s', ref=%d)", key, *ref); + + dd("code cache table to load: %p", lua_topointer(L, -1)); if (!lua_istable(L, -1)) { dd("Error: code cache table to load did not exist!!"); return NGX_ERROR; } - lua_getfield(L, -1, key); /* sp++ */ + ngx_http_lua_assert(key != NULL); + + if (*ref == LUA_NOREF) { + lua_getfield(L, -1, key); /* cache closure */ + + } else { + if (*ref == LUA_REFNIL) { + lua_getfield(L, -1, key); /* cache ref */ + + if (!lua_isnumber(L, -1)) { + goto not_found; + } + + *ref = lua_tonumber(L, -1); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, + "code cache setting ref (key='%s', ref=%d)", + key, *ref); + + lua_pop(L, 1); /* cache */ + } + + lua_rawgeti(L, -1, *ref); /* cache closure */ + } if (lua_isfunction(L, -1)) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, + "code cache hit (key='%s', ref=%d)", key, *ref); + #ifdef OPENRESTY_LUAJIT lua_remove(L, -2); /* sp-- */ return NGX_OK; @@ -83,6 +112,8 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, #endif /* OPENRESTY_LUAJIT */ } +not_found: + dd("Value associated with given key in code cache table is not code " "chunk: stack top=%d, top value type=%s\n", lua_gettop(L), lua_typename(L, -1)); @@ -90,6 +121,9 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, /* remove cache table and value from stack */ lua_pop(L, 2); /* sp-=2 */ + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0, + "code cache miss (key='%s', ref=%d)", key, *ref); + return NGX_DECLINED; } @@ -108,7 +142,7 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, * * */ static ngx_int_t -ngx_http_lua_cache_store_code(lua_State *L, const char *key) +ngx_http_lua_cache_store_code(lua_State *L, int *ref, const char *key) { #ifndef OPENRESTY_LUAJIT int rc; @@ -126,8 +160,22 @@ ngx_http_lua_cache_store_code(lua_State *L, const char *key) return NGX_ERROR; } + ngx_http_lua_assert(key != NULL); + lua_pushvalue(L, -2); /* closure cache closure */ - lua_setfield(L, -2, key); /* closure cache */ + + if (*ref == LUA_NOREF) { + /* cache closure by cache key */ + lua_setfield(L, -2, key); /* closure cache */ + + } else { + /* cache closure with reference */ + *ref = luaL_ref(L, -2); /* closure cache */ + + /* cache reference by cache key */ + lua_pushnumber(L, *ref); /* closure cache ref */ + lua_setfield(L, -2, key); /* closure cache */ + } /* remove cache table, leave closure factory at top of stack */ lua_pop(L, 1); /* closure */ @@ -147,7 +195,7 @@ ngx_http_lua_cache_store_code(lua_State *L, const char *key) ngx_int_t ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, - const u_char *src, size_t src_len, const u_char *cache_key, + const u_char *src, size_t src_len, int *cache_ref, const u_char *cache_key, const char *name) { int n; @@ -156,14 +204,8 @@ ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, n = lua_gettop(L); - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, - "looking up Lua code cache with key '%s'", cache_key); - - rc = ngx_http_lua_cache_load_code(log, L, (char *) cache_key); + rc = ngx_http_lua_cache_load_code(log, L, cache_ref, (char *) cache_key); if (rc == NGX_OK) { - /* code chunk loaded from cache, sp++ */ - dd("Code cache hit! cache key='%s', stack top=%d, script='%.*s'", - cache_key, lua_gettop(L), (int) src_len, src); return NGX_OK; } @@ -173,9 +215,6 @@ ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, /* rc == NGX_DECLINED */ - dd("Code cache missed! cache key='%s', stack top=%d, script='%.*s'", - cache_key, lua_gettop(L), (int) src_len, src); - /* load closure factory of inline script to the top of lua stack, sp++ */ rc = ngx_http_lua_clfactory_loadbuffer(L, (char *) src, src_len, name); @@ -198,7 +237,7 @@ ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, /* store closure factory and gen new closure at the top of lua stack to * code cache */ - rc = ngx_http_lua_cache_store_code(L, (char *) cache_key); + rc = ngx_http_lua_cache_store_code(L, cache_ref, (char *) cache_key); if (rc != NGX_OK) { err = "fail to generate new closure from the closure factory"; goto error; @@ -217,7 +256,7 @@ ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, ngx_int_t ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, - const u_char *script, const u_char *cache_key) + const u_char *script, int *cache_ref, const u_char *cache_key) { int n; ngx_int_t rc, errcode = NGX_ERROR; @@ -230,25 +269,22 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, /* calculate digest of script file path */ if (cache_key == NULL) { dd("CACHE file key not pre-calculated...calculating"); - p = ngx_copy(buf, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); + p = ngx_copy(buf, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); p = ngx_http_lua_digest_hex(p, script, ngx_strlen(script)); - *p = '\0'; + cache_key = buf; + *cache_ref = LUA_NOREF; } else { dd("CACHE file key already pre-calculated"); - } - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, - "looking up Lua code cache with key '%s'", cache_key); + ngx_http_lua_assert(cache_ref != NULL && *cache_ref != LUA_NOREF); + } - rc = ngx_http_lua_cache_load_code(log, L, (char *) cache_key); + rc = ngx_http_lua_cache_load_code(log, L, cache_ref, (char *) cache_key); if (rc == NGX_OK) { - /* code chunk loaded from cache, sp++ */ - dd("Code cache hit! cache key='%s', stack top=%d, file path='%s'", - cache_key, lua_gettop(L), script); return NGX_OK; } @@ -258,9 +294,6 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, /* rc == NGX_DECLINED */ - dd("Code cache missed! cache key='%s', stack top=%d, file path='%s'", - cache_key, lua_gettop(L), script); - /* load closure factory of script file to the top of lua stack, sp++ */ rc = ngx_http_lua_clfactory_loadfile(L, (char *) script); @@ -291,7 +324,7 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, /* store closure factory and gen new closure at the top of lua stack * to code cache */ - rc = ngx_http_lua_cache_store_code(L, (char *) cache_key); + rc = ngx_http_lua_cache_store_code(L, cache_ref, (char *) cache_key); if (rc != NGX_OK) { err = "fail to generate new closure from the closure factory"; goto error; diff --git a/src/ngx_http_lua_cache.h b/src/ngx_http_lua_cache.h index 52e6d2c94f..26edc3e6d4 100644 --- a/src/ngx_http_lua_cache.h +++ b/src/ngx_http_lua_cache.h @@ -13,10 +13,10 @@ ngx_int_t ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, - const u_char *src, size_t src_len, const u_char *cache_key, + const u_char *src, size_t src_len, int *cache_ref, const u_char *cache_key, const char *name); ngx_int_t ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, - const u_char *script, const u_char *cache_key); + const u_char *script, int *cache_ref, const u_char *cache_key); #endif /* _NGX_HTTP_LUA_CACHE_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index dae524516f..44f11b41c9 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -102,6 +102,7 @@ #if defined(NDK) && NDK typedef struct { size_t size; + int ref; u_char *key; ngx_str_t script; } ngx_http_lua_set_var_data_t; @@ -281,22 +282,25 @@ union ngx_http_lua_srv_conf_u { ngx_http_lua_srv_conf_handler_pt ssl_cert_handler; ngx_str_t ssl_cert_src; u_char *ssl_cert_src_key; + int ssl_cert_src_ref; ngx_http_lua_srv_conf_handler_pt ssl_sess_store_handler; ngx_str_t ssl_sess_store_src; u_char *ssl_sess_store_src_key; + int ssl_sess_store_src_ref; ngx_http_lua_srv_conf_handler_pt ssl_sess_fetch_handler; ngx_str_t ssl_sess_fetch_src; u_char *ssl_sess_fetch_src_key; + int ssl_sess_fetch_src_ref; } srv; #endif struct { + ngx_http_lua_srv_conf_handler_pt handler; ngx_str_t src; u_char *src_key; - - ngx_http_lua_srv_conf_handler_pt handler; + int src_ref; } balancer; }; @@ -333,6 +337,7 @@ typedef struct { file path */ u_char *rewrite_src_key; /* cached key for rewrite_src */ + int rewrite_src_ref; u_char *access_chunkname; ngx_http_complex_value_t access_src; /* access_by_lua @@ -340,6 +345,7 @@ typedef struct { file path */ u_char *access_src_key; /* cached key for access_src */ + int access_src_ref; u_char *content_chunkname; ngx_http_complex_value_t content_src; /* content_by_lua @@ -347,6 +353,7 @@ typedef struct { file path */ u_char *content_src_key; /* cached key for content_src */ + int content_src_ref; u_char *log_chunkname; @@ -354,6 +361,7 @@ typedef struct { file path */ u_char *log_src_key; /* cached key for log_src */ + int log_src_ref; ngx_http_complex_value_t header_filter_src; /* header_filter_by_lua inline script/script @@ -361,10 +369,12 @@ typedef struct { u_char *header_filter_src_key; /* cached key for header_filter_src */ + int header_filter_src_ref; ngx_http_complex_value_t body_filter_src; u_char *body_filter_src_key; + int body_filter_src_ref; ngx_msec_t keepalive_timeout; ngx_msec_t connect_timeout; diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 274c9ad33f..296ba9d35f 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -271,6 +271,7 @@ ngx_http_lua_content_handler_file(ngx_http_request_t *r) /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &llcf->content_src_ref, llcf->content_src_key); if (rc != NGX_OK) { if (rc < NGX_HTTP_SPECIAL_RESPONSE) { @@ -302,6 +303,7 @@ ngx_http_lua_content_handler_inline(ngx_http_request_t *r) rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->content_src.value.data, llcf->content_src.value.len, + &llcf->content_src_ref, llcf->content_src_key, (const char *) llcf->content_chunkname); diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index a989c2671e..c2a9fcb63a 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -278,6 +278,7 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) } filter_data->key = p; + filter_data->ref = LUA_REFNIL; p = ngx_copy(p, "set_by_lua", sizeof("set_by_lua") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); @@ -361,6 +362,7 @@ ngx_http_lua_filter_set_by_lua_inline(ngx_http_request_t *r, ngx_str_t *val, rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, filter_data->script.data, filter_data->script.len, + &filter_data->ref, filter_data->key, "=set_by_lua"); if (rc != NGX_OK) { return NGX_ERROR; @@ -413,6 +415,7 @@ ngx_http_lua_filter_set_by_lua_file(ngx_http_request_t *r, ngx_str_t *val, /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &filter_data->ref, filter_data->key); if (rc != NGX_OK) { return NGX_ERROR; diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index a0acd4a114..c9e770133e 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -172,6 +172,7 @@ ngx_http_lua_header_filter_inline(ngx_http_request_t *r) rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->header_filter_src.value.data, llcf->header_filter_src.value.len, + &llcf->header_filter_src_ref, llcf->header_filter_src_key, "=header_filter_by_lua"); if (rc != NGX_OK) { @@ -213,6 +214,7 @@ ngx_http_lua_header_filter_file(ngx_http_request_t *r) /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &llcf->header_filter_src_ref, llcf->header_filter_src_key); if (rc != NGX_OK) { return NGX_ERROR; diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index 32d1cba727..3da9b428b9 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -154,6 +154,7 @@ ngx_http_lua_log_handler_inline(ngx_http_request_t *r) rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->log_src.value.data, llcf->log_src.value.len, + &llcf->log_src_ref, llcf->log_src_key, (const char *) llcf->log_chunkname); if (rc != NGX_OK) { @@ -190,6 +191,7 @@ ngx_http_lua_log_handler_file(ngx_http_request_t *r) /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &llcf->log_src_ref, llcf->log_src_key); if (rc != NGX_OK) { return NGX_ERROR; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 2a3f5d57b6..712bfdb79d 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -986,6 +986,11 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) * lscf->balancer.src_key = NULL; */ + lscf->srv.ssl_cert_src_ref = LUA_REFNIL; + lscf->srv.ssl_sess_store_src_ref = LUA_REFNIL; + lscf->srv.ssl_sess_fetch_src_ref = LUA_REFNIL; + lscf->balancer.src_ref = LUA_REFNIL; + return lscf; } @@ -1003,6 +1008,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->srv.ssl_cert_src.len == 0) { conf->srv.ssl_cert_src = prev->srv.ssl_cert_src; + conf->srv.ssl_cert_src_ref = prev->srv.ssl_cert_src_ref; conf->srv.ssl_cert_src_key = prev->srv.ssl_cert_src_key; conf->srv.ssl_cert_handler = prev->srv.ssl_cert_handler; } @@ -1041,6 +1047,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->srv.ssl_sess_store_src.len == 0) { conf->srv.ssl_sess_store_src = prev->srv.ssl_sess_store_src; + conf->srv.ssl_sess_store_src_ref = prev->srv.ssl_sess_store_src_ref; conf->srv.ssl_sess_store_src_key = prev->srv.ssl_sess_store_src_key; conf->srv.ssl_sess_store_handler = prev->srv.ssl_sess_store_handler; } @@ -1063,6 +1070,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->srv.ssl_sess_fetch_src.len == 0) { conf->srv.ssl_sess_fetch_src = prev->srv.ssl_sess_fetch_src; + conf->srv.ssl_sess_fetch_src_ref = prev->srv.ssl_sess_fetch_src_ref; conf->srv.ssl_sess_fetch_src_key = prev->srv.ssl_sess_fetch_src_key; conf->srv.ssl_sess_fetch_handler = prev->srv.ssl_sess_fetch_handler; } @@ -1102,23 +1110,23 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) * conf->access_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->access_src_key = NULL * conf->rewrite_src = {{ 0, NULL }, NULL, NULL, NULL}; - * conf->rewrite_src_key = NULL + * conf->rewrite_src_key = NULL; * conf->rewrite_handler = NULL; * * conf->content_src = {{ 0, NULL }, NULL, NULL, NULL}; - * conf->content_src_key = NULL + * conf->content_src_key = NULL; * conf->content_handler = NULL; * * conf->log_src = {{ 0, NULL }, NULL, NULL, NULL}; - * conf->log_src_key = NULL + * conf->log_src_key = NULL; * conf->log_handler = NULL; * * conf->header_filter_src = {{ 0, NULL }, NULL, NULL, NULL}; - * conf->header_filter_src_key = NULL + * conf->header_filter_src_key = NULL; * conf->header_filter_handler = NULL; * * conf->body_filter_src = {{ 0, NULL }, NULL, NULL, NULL}; - * conf->body_filter_src_key = NULL + * conf->body_filter_src_key = NULL; * conf->body_filter_handler = NULL; * * conf->ssl = 0; @@ -1145,6 +1153,13 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) conf->transform_underscores_in_resp_headers = NGX_CONF_UNSET; conf->log_socket_errors = NGX_CONF_UNSET; + conf->rewrite_src_ref = LUA_REFNIL; + conf->access_src_ref = LUA_REFNIL; + conf->content_src_ref = LUA_REFNIL; + conf->header_filter_src_ref = LUA_REFNIL; + conf->body_filter_src_ref = LUA_REFNIL; + conf->log_src_ref = LUA_REFNIL; + #if (NGX_HTTP_SSL) conf->ssl_verify_depth = NGX_CONF_UNSET_UINT; #endif @@ -1162,6 +1177,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->rewrite_src.value.len == 0) { conf->rewrite_src = prev->rewrite_src; conf->rewrite_handler = prev->rewrite_handler; + conf->rewrite_src_ref = prev->rewrite_src_ref; conf->rewrite_src_key = prev->rewrite_src_key; conf->rewrite_chunkname = prev->rewrite_chunkname; } @@ -1169,6 +1185,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->access_src.value.len == 0) { conf->access_src = prev->access_src; conf->access_handler = prev->access_handler; + conf->access_src_ref = prev->access_src_ref; conf->access_src_key = prev->access_src_key; conf->access_chunkname = prev->access_chunkname; } @@ -1176,6 +1193,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->content_src.value.len == 0) { conf->content_src = prev->content_src; conf->content_handler = prev->content_handler; + conf->content_src_ref = prev->content_src_ref; conf->content_src_key = prev->content_src_key; conf->content_chunkname = prev->content_chunkname; } @@ -1183,6 +1201,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->log_src.value.len == 0) { conf->log_src = prev->log_src; conf->log_handler = prev->log_handler; + conf->log_src_ref = prev->log_src_ref; conf->log_src_key = prev->log_src_key; conf->log_chunkname = prev->log_chunkname; } @@ -1190,12 +1209,14 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->header_filter_src.value.len == 0) { conf->header_filter_src = prev->header_filter_src; conf->header_filter_handler = prev->header_filter_handler; + conf->header_filter_src_ref = prev->header_filter_src_ref; conf->header_filter_src_key = prev->header_filter_src_key; } if (conf->body_filter_src.value.len == 0) { conf->body_filter_src = prev->body_filter_src; conf->body_filter_handler = prev->body_filter_handler; + conf->body_filter_src_ref = prev->body_filter_src_ref; conf->body_filter_src_key = prev->body_filter_src_key; } diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 1de9968e5e..558d6291a8 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -184,6 +184,7 @@ ngx_http_lua_rewrite_handler_inline(ngx_http_request_t *r) rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->rewrite_src.value.data, llcf->rewrite_src.value.len, + &llcf->rewrite_src_ref, llcf->rewrite_src_key, (const char *) llcf->rewrite_chunkname); @@ -221,6 +222,7 @@ ngx_http_lua_rewrite_handler_file(ngx_http_request_t *r) /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &llcf->rewrite_src_ref, llcf->rewrite_src_key); if (rc != NGX_OK) { if (rc < NGX_HTTP_SPECIAL_RESPONSE) { diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 5e73de2de6..e2b21a096d 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -46,6 +46,7 @@ ngx_http_lua_ssl_cert_handler_file(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadfile(r->connection->log, L, lscf->srv.ssl_cert_src.data, + &lscf->srv.ssl_cert_src_ref, lscf->srv.ssl_cert_src_key); if (rc != NGX_OK) { return rc; @@ -67,6 +68,7 @@ ngx_http_lua_ssl_cert_handler_inline(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, lscf->srv.ssl_cert_src.data, lscf->srv.ssl_cert_src.len, + &lscf->srv.ssl_cert_src_ref, lscf->srv.ssl_cert_src_key, "=ssl_certificate_by_lua"); if (rc != NGX_OK) { diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 17e41a97de..73c24312a6 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -41,6 +41,7 @@ ngx_http_lua_ssl_sess_fetch_handler_file(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadfile(r->connection->log, L, lscf->srv.ssl_sess_fetch_src.data, + &lscf->srv.ssl_sess_fetch_src_ref, lscf->srv.ssl_sess_fetch_src_key); if (rc != NGX_OK) { return rc; @@ -63,6 +64,7 @@ ngx_http_lua_ssl_sess_fetch_handler_inline(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, lscf->srv.ssl_sess_fetch_src.data, lscf->srv.ssl_sess_fetch_src.len, + &lscf->srv.ssl_sess_fetch_src_ref, lscf->srv.ssl_sess_fetch_src_key, "=ssl_session_fetch_by_lua_block"); if (rc != NGX_OK) { diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index f5016418e0..d7902c8b9b 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -39,6 +39,7 @@ ngx_http_lua_ssl_sess_store_handler_file(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadfile(r->connection->log, L, lscf->srv.ssl_sess_store_src.data, + &lscf->srv.ssl_sess_store_src_ref, lscf->srv.ssl_sess_store_src_key); if (rc != NGX_OK) { return rc; @@ -61,6 +62,7 @@ ngx_http_lua_ssl_sess_store_handler_inline(ngx_http_request_t *r, rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, lscf->srv.ssl_sess_store_src.data, lscf->srv.ssl_sess_store_src.len, + &lscf->srv.ssl_sess_store_src_ref, lscf->srv.ssl_sess_store_src_key, "=ssl_session_store_by_lua_block"); if (rc != NGX_OK) { diff --git a/t/025-codecache.t b/t/025-codecache.t index 6290780e29..aa2e0c5195 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1284,22 +1284,36 @@ GET /t /b/ is called /a/ is called /b/ is called ---- grep_error_log eval: qr/looking up Lua code cache with key '.*?'/ +--- grep_error_log eval: qr/code cache .*/ --- grep_error_log_out eval [ -"looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +"code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache miss (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache miss (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache setting ref (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) ", -"looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6' -looking up Lua code cache with key '=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04' +"code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache miss (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache miss (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache setting ref (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) "] --- log_level: debug --- no_error_log @@ -1353,22 +1367,34 @@ GET /t /b/ is called /a/ is called /b/ is called ---- grep_error_log eval: qr/looking up Lua code cache with key '.*?'/ +--- grep_error_log eval: qr/code cache .*/ --- grep_error_log_out eval [ -"looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' -looking up Lua code cache with key 'nhlf_68f5f4e946c3efd1cc206452b807e8b6' -looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' -looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' +"code cache lookup (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=-1) +code cache miss (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=-1) +code cache lookup (key='nhlf_68f5f4e946c3efd1cc206452b807e8b6', ref=-1) +code cache miss (key='nhlf_68f5f4e946c3efd1cc206452b807e8b6', ref=-1) +code cache lookup (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache hit (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache lookup (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=-1) +code cache miss (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=-1) ", -"looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' -looking up Lua code cache with key 'nhlf_68f5f4e946c3efd1cc206452b807e8b6' -looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' -looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' -looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' -looking up Lua code cache with key 'nhlf_68f5f4e946c3efd1cc206452b807e8b6' -looking up Lua code cache with key 'nhlf_48a9a7def61143c003a7de1644e026e4' -looking up Lua code cache with key 'nhlf_042c9b3a136fbacbbd0e4b9ad10896b7' +"code cache lookup (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=-1) +code cache miss (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=-1) +code cache lookup (key='nhlf_68f5f4e946c3efd1cc206452b807e8b6', ref=-1) +code cache miss (key='nhlf_68f5f4e946c3efd1cc206452b807e8b6', ref=-1) +code cache lookup (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache hit (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache lookup (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=-1) +code cache miss (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=-1) +code cache lookup (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache hit (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache lookup (key='nhlf_68f5f4e946c3efd1cc206452b807e8b6', ref=2) +code cache hit (key='nhlf_68f5f4e946c3efd1cc206452b807e8b6', ref=2) +code cache lookup (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache hit (key='nhlf_48a9a7def61143c003a7de1644e026e4', ref=1) +code cache lookup (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=3) +code cache hit (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=3) " ] --- log_level: debug From 0c755ed892f4586536e7a0ad482e5a3feaa22ab9 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 12 Jun 2019 16:55:01 -0700 Subject: [PATCH 334/848] bugfix: ensured set_by_lua_file with nginx variables re-computes closures code cache key. --- src/ngx_http_lua_directive.c | 28 ++++++++++++++++------ t/025-codecache.t | 46 ++++++++++++++++++++++++++++++------ 2 files changed, 60 insertions(+), 14 deletions(-) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index c2a9fcb63a..97c9301041 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -301,7 +301,9 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) ngx_str_t target; ndk_set_var_t filter; - ngx_http_lua_set_var_data_t *filter_data; + ngx_http_lua_set_var_data_t *filter_data; + ngx_http_complex_value_t cv; + ngx_http_compile_complex_value_t ccv; /* * value[0] = "set_by_lua_file" @@ -324,16 +326,28 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) filter_data->size = filter.size; - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); + ccv.cf = cf; + ccv.value = &value[2]; + ccv.complex_value = &cv; + + if (ngx_http_compile_complex_value(&ccv) != NGX_OK) { return NGX_CONF_ERROR; } - filter_data->key = p; + if (cv.lengths == NULL) { + /* no variable found */ + p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); + if (p == NULL) { + return NGX_CONF_ERROR; + } - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[2].data, value[2].len); - *p = '\0'; + filter_data->key = p; + + p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); + p = ngx_http_lua_digest_hex(p, value[2].data, value[2].len); + *p = '\0'; + } ngx_str_null(&filter_data->script); diff --git a/t/025-codecache.t b/t/025-codecache.t index aa2e0c5195..c6f8c0ae1e 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 181; +plan tests => repeat_each() * 184; #$ENV{LUA_PATH} = $ENV{HOME} . '/work/JSON4Lua-0.9.30/json/?.lua'; @@ -1403,7 +1403,39 @@ code cache hit (key='nhlf_042c9b3a136fbacbbd0e4b9ad10896b7', ref=3) -=== TEST 34: variables in rewrite_by_lua_file's file path +=== TEST 34: variables in set_by_lua_file's file path +--- config + location ~ ^/lua/(.+)$ { + set_by_lua_file $res html/$1.lua; + echo $res; + } + + location /main { + echo_location /lua/a; + echo_location /lua/b; + echo_location /lua/a; + echo_location /lua/a; + echo_location /lua/b; + } +--- user_files +>>> a.lua +return "a" +>>> b.lua +return "b" +--- request +GET /main +--- response_body +a +b +a +a +b +--- no_error_log +[error] + + + +=== TEST 35: variables in rewrite_by_lua_file's file path --- config location ~ ^/lua/(.+)$ { rewrite_by_lua_file html/$1.lua; @@ -1434,7 +1466,7 @@ b -=== TEST 35: variables in access_by_lua_file's file path +=== TEST 36: variables in access_by_lua_file's file path --- config location ~ ^/lua/(.+)$ { access_by_lua_file html/$1.lua; @@ -1483,7 +1515,7 @@ b -=== TEST 36: variables in content_by_lua_file's file path +=== TEST 37: variables in content_by_lua_file's file path --- config location ~ ^/lua/(.+)$ { content_by_lua_file html/$1.lua; @@ -1514,7 +1546,7 @@ b -=== TEST 37: variables in header_filter_by_lua_file's file path +=== TEST 38: variables in header_filter_by_lua_file's file path --- config location ~ ^/lua/(.+)$ { return 200; @@ -1561,7 +1593,7 @@ b -=== TEST 38: variables in body_filter_by_lua_file's file path +=== TEST 39: variables in body_filter_by_lua_file's file path --- config location ~ ^/lua/(.+)$ { echo hello; @@ -1596,7 +1628,7 @@ b -=== TEST 39: variables in log_by_lua_file's file path +=== TEST 40: variables in log_by_lua_file's file path --- config log_subrequest on; From bd83c656d55c573caf186c0f2cd603191a724e33 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 13 Jun 2019 16:55:01 -0700 Subject: [PATCH 335/848] tests: codecache.t: moved and reinforced a test case ensuring *by_lua directives with identical Lua chunks produce distinct cache keys. --- t/025-codecache.t | 204 ++++++++++++++++++++++++++++++++++++++++++++- t/132-lua-blocks.t | 123 +-------------------------- 2 files changed, 204 insertions(+), 123 deletions(-) diff --git a/t/025-codecache.t b/t/025-codecache.t index c6f8c0ae1e..cc6c80e08e 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1,12 +1,16 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: use Test::Nginx::Socket::Lua; +use Cwd qw(abs_path realpath); +use File::Basename; repeat_each(2); -plan tests => repeat_each() * 184; +plan tests => repeat_each() * 198; #$ENV{LUA_PATH} = $ENV{HOME} . '/work/JSON4Lua-0.9.30/json/?.lua'; +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); +$ENV{TEST_NGINX_CERT_DIR} ||= dirname(realpath(abs_path(__FILE__))); no_long_string(); @@ -1675,3 +1679,201 @@ grep me: b "] --- no_error_log [error] + + + +=== TEST 41: same chunk from different directives produces different closures +--- http_config + ssl_session_fetch_by_lua_block { ngx.log(ngx.INFO, "hello") } + + ssl_session_store_by_lua_block { ngx.log(ngx.INFO, "hello") } + + upstream backend { + server 0.0.0.1; + balancer_by_lua_block { ngx.log(ngx.INFO, "hello") } + } + + server { + server_name test.com; + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; + + ssl_certificate_by_lua_block { ngx.log(ngx.INFO, "hello") } + + location /lua { + set_by_lua_block $res { ngx.log(ngx.INFO, "hello") } + + rewrite_by_lua_block { ngx.log(ngx.INFO, "hello") } + + access_by_lua_block { ngx.log(ngx.INFO, "hello") } + + content_by_lua_block { ngx.log(ngx.INFO, "hello") } + + header_filter_by_lua_block { ngx.log(ngx.INFO, "hello") } + + body_filter_by_lua_block { ngx.log(ngx.INFO, "hello") } + + log_by_lua_block { ngx.log(ngx.INFO, "hello") } + } + } +--- config + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + + location = /proxy { + proxy_pass http://backend; + } + + location = /t { + set $html_dir $TEST_NGINX_HTML_DIR; + + content_by_lua_block { + ngx.location.capture("/proxy") + + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("unix:" .. ngx.var.html_dir .. "/nginx.sock") + if not ok then + ngx.log(ngx.ERR, "failed to connect: ", err) + return + end + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.log(ngx.ERR, "failed to do SSL handshake: ", err) + return + end + package.loaded.session = sess + sock:close() + + local ok, err = sock:connect("unix:" .. ngx.var.html_dir .. "/nginx.sock") + if not ok then + ngx.log(ngx.ERR, "failed to connect: ", err) + return + end + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.log(ngx.ERR, "failed to do SSL handshake: ", err) + return + end + + local req = "GET /lua HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.log(ngx.ERR, "failed to send http request: ", err) + return + end + } + } +--- request +GET /t +--- ignore_response_body +--- grep_error_log eval: qr/code cache .*/ +--- grep_error_log_out eval +[ +"code cache lookup (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache miss (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache lookup (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +", +"code cache lookup (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache miss (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache lookup (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=1) +code cache hit (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=1) +code cache lookup (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=2) +code cache hit (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=2) +code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=5) +code cache hit (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=5) +code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=6) +code cache hit (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=6) +code cache lookup (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=7) +code cache hit (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=7) +code cache lookup (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=8) +code cache hit (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=8) +code cache lookup (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=9) +code cache hit (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=9) +code cache lookup (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=10) +code cache hit (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=10) +code cache lookup (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=11) +code cache hit (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=11) +code cache lookup (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) +code cache hit (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) +"] +--- error_log eval +[ +qr/balancer_by_lua:\d+: hello/, +qr/ssl_session_fetch_by_lua_block:\d+: hello/, +qr/ssl_certificate_by_lua:\d+: hello/, +qr/ssl_session_store_by_lua_block:\d+: hello/, +qr/set_by_lua:\d+: hello/, +qr/rewrite_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/access_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/content_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/header_filter_by_lua:\d+: hello/, +qr/body_filter_by_lua:\d+: hello/, +qr/log_by_lua\(nginx\.conf:\d+\):\d+: hello/, +] +--- log_level: debug +--- no_error_log +[error] diff --git a/t/132-lua-blocks.t b/t/132-lua-blocks.t index 292af58c37..76f4a6184d 100644 --- a/t/132-lua-blocks.t +++ b/t/132-lua-blocks.t @@ -10,7 +10,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 3 + 13); +plan tests => repeat_each() * (blocks() * 3 + 3); $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -606,124 +606,3 @@ GET /t --- response_body_like: } --- no_error_log [error] - - - -=== TEST 24: inline Lua code cache should not mess up across different phases ---- http_config - # The indent is enforced to generate the same hash tags - ssl_session_fetch_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - ssl_session_store_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - - upstream backend { - server 0.0.0.1; - balancer_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - } - - server { - listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; - server_name test.com; - - ssl_certificate_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - - ssl_certificate ../../cert/test.crt; - ssl_certificate_key ../../cert/test.key; - ssl_session_tickets off; - - location /foo { - set_by_lua_block $x { - ngx.log(ngx.INFO, "hello") - } - rewrite_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - access_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - content_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - header_filter_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - body_filter_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - log_by_lua_block { - ngx.log(ngx.INFO, "hello") - } - } - } ---- config - lua_ssl_trusted_certificate ../../cert/test.crt; - - location = /proxy { - proxy_pass http://backend; - } - - location /t { - content_by_lua_block { - ngx.location.capture("/proxy") - - local sock = ngx.socket.tcp() - sock:settimeout(2000) - local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") - if not ok then - ngx.say("failed to connect: ", err) - return - end - - local sess, err = sock:sslhandshake(nil, "test.com", true) - if not sess then - ngx.say("failed to do SSL handshake: ", err) - return - end - package.loaded.session = sess - sock:close() - - local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") - if not ok then - ngx.say("failed to connect: ", err) - return - end - - local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) - if not sess then - ngx.say("failed to do SSL handshake: ", err) - return - end - - local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" - local bytes, err = sock:send(req) - if not bytes then - ngx.say("failed to send http request: ", err) - return - end - } - } ---- request -GET /t ---- no_error_log -[error] ---- error_log eval -[ -qr/balancer_by_lua:\d+: hello/, -qr/ssl_session_fetch_by_lua_block:\d+: hello/, -qr/ssl_certificate_by_lua:\d+: hello/, -qr/ssl_session_store_by_lua_block:\d+: hello/, -qr/set_by_lua:\d+: hello/, -qr/rewrite_by_lua\(nginx\.conf:\d+\):\d+: hello/, -qr/access_by_lua\(nginx\.conf:\d+\):\d+: hello/, -qr/content_by_lua\(nginx\.conf:\d+\):\d+: hello/, -qr/header_filter_by_lua:\d+: hello/, -qr/body_filter_by_lua:\d+: hello/, -qr/log_by_lua\(nginx\.conf:\d+\):\d+: hello/, -] From f45908cd32025d107c84f16aed5f6829aabcf536 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 14 Jun 2019 16:55:01 -0700 Subject: [PATCH 336/848] refactor: reduced cache key size by removing unnecessary chunkname component. --- src/ngx_http_lua_balancer.c | 5 +- src/ngx_http_lua_directive.c | 31 ++-- src/ngx_http_lua_ssl_certby.c | 6 +- src/ngx_http_lua_ssl_session_fetchby.c | 6 +- src/ngx_http_lua_ssl_session_storeby.c | 6 +- t/025-codecache.t | 220 ++++++++++++------------- 6 files changed, 138 insertions(+), 136 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 9277122544..38565fea5b 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -176,14 +176,15 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->balancer.src = value[1]; p = ngx_palloc(cf->pool, - sizeof("balancer_by_lua") + NGX_HTTP_LUA_INLINE_KEY_LEN); + sizeof("balancer_by_lua_") + + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } lscf->balancer.src_key = p; - p = ngx_copy(p, "balancer_by_lua", sizeof("balancer_by_lua") - 1); + p = ngx_copy(p, "balancer_by_lua_", sizeof("balancer_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 97c9301041..8d6aeeec23 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -272,7 +272,7 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) filter_data->size = filter.size; p = ngx_palloc(cf->pool, - sizeof("set_by_lua") + NGX_HTTP_LUA_INLINE_KEY_LEN); + sizeof("set_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } @@ -280,7 +280,7 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) filter_data->key = p; filter_data->ref = LUA_REFNIL; - p = ngx_copy(p, "set_by_lua", sizeof("set_by_lua") - 1); + p = ngx_copy(p, "set_by_lua_", sizeof("set_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[2].data, value[2].len); *p = '\0'; @@ -515,14 +515,14 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->rewrite_src.value = value[1]; p = ngx_palloc(cf->pool, - chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + sizeof("rewrite_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } llcf->rewrite_src_key = p; - p = ngx_copy(p, chunkname, chunkname_len); + p = ngx_copy(p, "rewrite_by_lua_", sizeof("rewrite_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -629,14 +629,14 @@ ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->access_src.value = value[1]; p = ngx_palloc(cf->pool, - chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + sizeof("access_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } llcf->access_src_key = p; - p = ngx_copy(p, chunkname, chunkname_len); + p = ngx_copy(p, "access_by_lua_", sizeof("access_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -749,14 +749,14 @@ ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->content_src.value = value[1]; p = ngx_palloc(cf->pool, - chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + sizeof("content_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } llcf->content_src_key = p; - p = ngx_copy(p, chunkname, chunkname_len); + p = ngx_copy(p, "content_by_lua_", sizeof("content_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -870,14 +870,14 @@ ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) llcf->log_src.value = value[1]; p = ngx_palloc(cf->pool, - chunkname_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); + sizeof("log_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; } llcf->log_src_key = p; - p = ngx_copy(p, chunkname, chunkname_len); + p = ngx_copy(p, "log_by_lua_", sizeof("log_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -972,7 +972,7 @@ ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, llcf->header_filter_src.value = value[1]; p = ngx_palloc(cf->pool, - sizeof("header_filter_by_lua") + + sizeof("header_filter_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; @@ -980,8 +980,8 @@ ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, llcf->header_filter_src_key = p; - p = ngx_copy(p, "header_filter_by_lua", - sizeof("header_filter_by_lua") - 1); + p = ngx_copy(p, "header_filter_by_lua_", + sizeof("header_filter_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; @@ -1076,7 +1076,7 @@ ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, llcf->body_filter_src.value = value[1]; p = ngx_palloc(cf->pool, - sizeof("body_filter_by_lua") + + sizeof("body_filter_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN + 1); if (p == NULL) { return NGX_CONF_ERROR; @@ -1084,7 +1084,8 @@ ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, llcf->body_filter_src_key = p; - p = ngx_copy(p, "body_filter_by_lua", sizeof("body_filter_by_lua") - 1); + p = ngx_copy(p, "body_filter_by_lua_", + sizeof("body_filter_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index e2b21a096d..42b6dadce3 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -166,7 +166,7 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_cert_src = value[1]; p = ngx_palloc(cf->pool, - sizeof("ssl_certificate_by_lua") + + sizeof("ssl_certificate_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; @@ -174,8 +174,8 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_cert_src_key = p; - p = ngx_copy(p, "ssl_certificate_by_lua", - sizeof("ssl_certificate_by_lua") - 1); + p = ngx_copy(p, "ssl_certificate_by_lua_", + sizeof("ssl_certificate_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 73c24312a6..9e9e94ccd1 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -156,7 +156,7 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_sess_fetch_src = value[1]; p = ngx_palloc(cf->pool, - sizeof("ssl_session_fetch_by_lua") + + sizeof("ssl_session_fetch_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; @@ -164,8 +164,8 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_sess_fetch_src_key = p; - p = ngx_copy(p, "ssl_session_fetch_by_lua", - sizeof("ssl_session_fetch_by_lua") - 1); + p = ngx_copy(p, "ssl_session_fetch_by_lua_", + sizeof("ssl_session_fetch_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index d7902c8b9b..4aa1162b30 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -154,7 +154,7 @@ ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_sess_store_src = value[1]; p = ngx_palloc(cf->pool, - sizeof("ssl_session_store_by_lua") + + sizeof("ssl_session_store_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); if (p == NULL) { return NGX_CONF_ERROR; @@ -162,8 +162,8 @@ ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_sess_store_src_key = p; - p = ngx_copy(p, "ssl_session_store_by_lua", - sizeof("ssl_session_store_by_lua") - 1); + p = ngx_copy(p, "ssl_session_store_by_lua_", + sizeof("ssl_session_store_by_lua_") - 1); p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); *p = '\0'; diff --git a/t/025-codecache.t b/t/025-codecache.t index cc6c80e08e..e162b3bc2e 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1291,33 +1291,33 @@ GET /t --- grep_error_log eval: qr/code cache .*/ --- grep_error_log_out eval [ -"code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) -code cache miss (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) -code cache miss (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) -code cache setting ref (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +"code cache lookup (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache miss (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache lookup (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache miss (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache lookup (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache setting ref (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) ", -"code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) -code cache miss (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) -code cache miss (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) -code cache setting ref (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) -code cache lookup (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) -code cache hit (key='=content_by_lua(proxy.conf:2)nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +"code cache lookup (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache miss (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=-1) +code cache lookup (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache miss (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache lookup (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=-1) +code cache setting ref (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache lookup (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache lookup (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache hit (key='content_by_lua_nhli_3c7137b8371d10bc148c8f8bb3042ee6', ref=1) +code cache lookup (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) +code cache hit (key='content_by_lua_nhli_1dfe09105792ef65c8d576cc486d5e04', ref=2) "] --- log_level: debug --- no_error_log @@ -1774,91 +1774,91 @@ GET /t --- grep_error_log eval: qr/code cache .*/ --- grep_error_log_out eval [ -"code cache lookup (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) -code cache miss (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) -code cache lookup (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache lookup (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +"code cache lookup (key='content_by_lua_nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache miss (key='content_by_lua_nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache lookup (key='balancer_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='balancer_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_fetch_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_fetch_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='set_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='set_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='rewrite_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='rewrite_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='access_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='access_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='content_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='content_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='header_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='header_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='body_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='body_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) ", -"code cache lookup (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) -code cache miss (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) -code cache lookup (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache lookup (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache miss (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) -code cache lookup (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=1) -code cache hit (key='=content_by_lua(nginx.conf:120)nhli_56ca4388611109b6ecfdeada050c8024', ref=1) -code cache lookup (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=2) -code cache hit (key='balancer_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=2) -code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache lookup (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=5) -code cache hit (key='ssl_session_fetch_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=5) -code cache lookup (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache hit (key='ssl_certificate_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) -code cache lookup (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache hit (key='ssl_session_store_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) -code cache lookup (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=6) -code cache hit (key='set_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=6) -code cache lookup (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=7) -code cache hit (key='=rewrite_by_lua(nginx.conf:46)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=7) -code cache lookup (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=8) -code cache hit (key='=access_by_lua(nginx.conf:48)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=8) -code cache lookup (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=9) -code cache hit (key='=content_by_lua(nginx.conf:50)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=9) -code cache lookup (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=10) -code cache hit (key='header_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=10) -code cache lookup (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=11) -code cache hit (key='body_filter_by_luanhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=11) -code cache lookup (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) -code cache hit (key='=log_by_lua(nginx.conf:56)nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) +"code cache lookup (key='content_by_lua_nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache miss (key='content_by_lua_nhli_56ca4388611109b6ecfdeada050c8024', ref=-1) +code cache lookup (key='balancer_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='balancer_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_session_fetch_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='ssl_session_fetch_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='set_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='set_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='rewrite_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='rewrite_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='access_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='access_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='content_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='content_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='header_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='header_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='body_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='body_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache miss (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=-1) +code cache lookup (key='content_by_lua_nhli_56ca4388611109b6ecfdeada050c8024', ref=1) +code cache hit (key='content_by_lua_nhli_56ca4388611109b6ecfdeada050c8024', ref=1) +code cache lookup (key='balancer_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=2) +code cache hit (key='balancer_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=2) +code cache lookup (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='ssl_session_fetch_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=5) +code cache hit (key='ssl_session_fetch_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=5) +code cache lookup (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache hit (key='ssl_certificate_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=3) +code cache lookup (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache hit (key='ssl_session_store_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=4) +code cache lookup (key='set_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=6) +code cache hit (key='set_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=6) +code cache lookup (key='rewrite_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=7) +code cache hit (key='rewrite_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=7) +code cache lookup (key='access_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=8) +code cache hit (key='access_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=8) +code cache lookup (key='content_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=9) +code cache hit (key='content_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=9) +code cache lookup (key='header_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=10) +code cache hit (key='header_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=10) +code cache lookup (key='body_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=11) +code cache hit (key='body_filter_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=11) +code cache lookup (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) +code cache hit (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) "] --- error_log eval [ From 246ec8ac482d13770c545f2cf59355df50ce843b Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 14 Jun 2019 17:55:01 -0700 Subject: [PATCH 337/848] refactor: simplified Lua chunk and file cache key generation. --- src/ngx_http_lua_balancer.c | 40 ++-- src/ngx_http_lua_cache.c | 72 ++++++- src/ngx_http_lua_cache.h | 4 + src/ngx_http_lua_directive.c | 248 ++++++++----------------- src/ngx_http_lua_ssl_certby.c | 42 ++--- src/ngx_http_lua_ssl_session_fetchby.c | 42 ++--- src/ngx_http_lua_ssl_session_storeby.c | 42 ++--- 7 files changed, 211 insertions(+), 279 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 38565fea5b..0ee709a6b0 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -125,7 +125,7 @@ char * ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key = NULL; u_char *name; ngx_str_t *value; ngx_http_lua_srv_conf_t *lscf = conf; @@ -149,47 +149,35 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, if (cmd->post == ngx_http_lua_balancer_handler_file) { /* Lua code in an external file */ - name = ngx_http_lua_rebase_path(cf->pool, value[1].data, value[1].len); if (name == NULL) { return NGX_CONF_ERROR; } - lscf->balancer.src.data = name; - lscf->balancer.src.len = ngx_strlen(name); - - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->balancer.src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + lscf->balancer.src.data = name; + lscf->balancer.src.len = ngx_strlen(name); } else { - /* inlined Lua code */ - - lscf->balancer.src = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("balancer_by_lua_") + - NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "balancer_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->balancer.src_key = p; - - p = ngx_copy(p, "balancer_by_lua_", sizeof("balancer_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + lscf->balancer.src = value[1]; } + lscf->balancer.src_key = cache_key; + uscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_upstream_module); if (uscf->peer.init_upstream) { diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index eb2f75db4a..3e0003a4d4 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -19,6 +19,10 @@ #include "ngx_http_lua_util.h" +static u_char *ngx_http_lua_gen_file_cache_key_helper(u_char *out, + const u_char *src, size_t src_len); + + /** * Find code chunk associated with the given key in code cache, * and push it to the top of Lua stack if found. @@ -260,7 +264,6 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, { int n; ngx_int_t rc, errcode = NGX_ERROR; - u_char *p; u_char buf[NGX_HTTP_LUA_FILE_KEY_LEN + 1]; const char *err = NULL; @@ -270,11 +273,8 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, if (cache_key == NULL) { dd("CACHE file key not pre-calculated...calculating"); - p = ngx_copy(buf, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, script, ngx_strlen(script)); - *p = '\0'; - - cache_key = buf; + cache_key = ngx_http_lua_gen_file_cache_key_helper(buf, script, + ngx_strlen(script)); *cache_ref = LUA_NOREF; } else { @@ -341,4 +341,64 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, return errcode; } + +u_char * +ngx_http_lua_gen_chunk_cache_key(ngx_conf_t *cf, const char *tag, + const u_char *src, size_t src_len) +{ + u_char *p, *out; + size_t tag_len; + + tag_len = ngx_strlen(tag); + + out = ngx_palloc(cf->pool, tag_len + NGX_HTTP_LUA_INLINE_KEY_LEN + 2); + if (out == NULL) { + return NULL; + } + + p = ngx_copy(out, tag, tag_len); + p = ngx_copy(p, "_", 1); + p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); + p = ngx_http_lua_digest_hex(p, src, src_len); + *p = '\0'; + + return out; +} + + +static u_char * +ngx_http_lua_gen_file_cache_key_helper(u_char *out, const u_char *src, + size_t src_len) +{ + u_char *p; + + ngx_http_lua_assert(out != NULL); + + if (out == NULL) { + return NULL; + } + + p = ngx_copy(out, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); + p = ngx_http_lua_digest_hex(p, src, src_len); + *p = '\0'; + + return out; +} + + +u_char * +ngx_http_lua_gen_file_cache_key(ngx_conf_t *cf, const u_char *src, + size_t src_len) +{ + u_char *out; + + out = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_TAG_LEN + 1); + if (out == NULL) { + return NULL; + } + + return ngx_http_lua_gen_file_cache_key_helper(out, src, src_len); +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_cache.h b/src/ngx_http_lua_cache.h index 26edc3e6d4..9535279d8b 100644 --- a/src/ngx_http_lua_cache.h +++ b/src/ngx_http_lua_cache.h @@ -17,6 +17,10 @@ ngx_int_t ngx_http_lua_cache_loadbuffer(ngx_log_t *log, lua_State *L, const char *name); ngx_int_t ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, const u_char *script, int *cache_ref, const u_char *cache_key); +u_char *ngx_http_lua_gen_chunk_cache_key(ngx_conf_t *cf, const char *tag, + const u_char *src, size_t src_len); +u_char *ngx_http_lua_gen_file_cache_key(ngx_conf_t *cf, const u_char *src, + size_t src_len); #endif /* _NGX_HTTP_LUA_CACHE_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 8d6aeeec23..3186581a3a 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -244,7 +244,7 @@ ngx_http_lua_set_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, char * ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key; ngx_str_t *value; ngx_str_t target; ndk_set_var_t filter; @@ -269,23 +269,17 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } - filter_data->size = filter.size; - - p = ngx_palloc(cf->pool, - sizeof("set_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "set_by_lua", + value[2].data, + value[2].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - filter_data->key = p; + filter_data->key = cache_key; filter_data->ref = LUA_REFNIL; - - p = ngx_copy(p, "set_by_lua_", sizeof("set_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[2].data, value[2].len); - *p = '\0'; - filter_data->script = value[2]; + filter_data->size = filter.size; filter.data = filter_data; @@ -296,7 +290,7 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) char * ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key; ngx_str_t *value; ngx_str_t target; ndk_set_var_t filter; @@ -337,16 +331,13 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (cv.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[2].data, + value[2].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - filter_data->key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[2].data, value[2].len); - *p = '\0'; + filter_data->key = cache_key; } ngx_str_null(&filter_data->script); @@ -468,7 +459,7 @@ char * ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { size_t chunkname_len; - u_char *p, *chunkname; + u_char *cache_key = NULL, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -508,24 +499,16 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } - llcf->rewrite_chunkname = chunkname; - - /* Don't eval nginx variables for inline lua code */ - - llcf->rewrite_src.value = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("rewrite_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "rewrite_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - llcf->rewrite_src_key = p; - - p = ngx_copy(p, "rewrite_by_lua_", sizeof("rewrite_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + llcf->rewrite_src.value = value[1]; + llcf->rewrite_chunkname = chunkname; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -539,19 +522,15 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (llcf->rewrite_src.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - - llcf->rewrite_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; } } + llcf->rewrite_src_key = cache_key; llcf->rewrite_handler = (ngx_http_handler_pt) cmd->post; lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); @@ -586,7 +565,7 @@ char * ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { size_t chunkname_len; - u_char *p, *chunkname; + u_char *cache_key = NULL, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -622,24 +601,16 @@ ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } - llcf->access_chunkname = chunkname; - - /* Don't eval nginx variables for inline lua code */ - - llcf->access_src.value = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("access_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "access_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - llcf->access_src_key = p; - - p = ngx_copy(p, "access_by_lua_", sizeof("access_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + llcf->access_src.value = value[1]; + llcf->access_chunkname = chunkname; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -653,19 +624,15 @@ ngx_http_lua_access_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (llcf->access_src.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - - llcf->access_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; } } + llcf->access_src_key = cache_key; llcf->access_handler = (ngx_http_handler_pt) cmd->post; lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); @@ -700,8 +667,7 @@ char * ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { size_t chunkname_len; - u_char *p; - u_char *chunkname; + u_char *cache_key = NULL, *chunkname; ngx_str_t *value; ngx_http_core_loc_conf_t *clcf; ngx_http_lua_main_conf_t *lmcf; @@ -740,26 +706,16 @@ ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } - llcf->content_chunkname = chunkname; - - dd("chunkname: %s", chunkname); - - /* Don't eval nginx variables for inline lua code */ - - llcf->content_src.value = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("content_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "content_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - llcf->content_src_key = p; - - p = ngx_copy(p, "content_by_lua_", sizeof("content_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + llcf->content_src.value = value[1]; + llcf->content_chunkname = chunkname; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -773,19 +729,15 @@ ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (llcf->content_src.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - - llcf->content_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; } } + llcf->content_src_key = cache_key; llcf->content_handler = (ngx_http_handler_pt) cmd->post; lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); @@ -827,7 +779,7 @@ char * ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { size_t chunkname_len; - u_char *p, *chunkname; + u_char *cache_key = NULL, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -863,24 +815,16 @@ ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } - llcf->log_chunkname = chunkname; - - /* Don't eval nginx variables for inline lua code */ - - llcf->log_src.value = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("log_by_lua_") + NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "log_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - llcf->log_src_key = p; - - p = ngx_copy(p, "log_by_lua_", sizeof("log_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + llcf->log_src.value = value[1]; + llcf->log_chunkname = chunkname; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -894,19 +838,15 @@ ngx_http_lua_log_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (llcf->log_src.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - - llcf->log_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; } } + llcf->log_src_key = cache_key; llcf->log_handler = (ngx_http_handler_pt) cmd->post; lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); @@ -940,7 +880,7 @@ char * ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key = NULL; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -968,23 +908,15 @@ ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } if (cmd->post == ngx_http_lua_header_filter_inline) { - /* Don't eval nginx variables for inline lua code */ - llcf->header_filter_src.value = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("header_filter_by_lua_") + - NGX_HTTP_LUA_INLINE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "header_filter_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - llcf->header_filter_src_key = p; - - p = ngx_copy(p, "header_filter_by_lua_", - sizeof("header_filter_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + llcf->header_filter_src.value = value[1]; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -998,19 +930,15 @@ ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, if (llcf->header_filter_src.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - - llcf->header_filter_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; } } + llcf->header_filter_src_key = cache_key; llcf->header_filter_handler = (ngx_http_handler_pt) cmd->post; lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); @@ -1044,7 +972,7 @@ char * ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key = NULL; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -1072,23 +1000,15 @@ ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } if (cmd->post == ngx_http_lua_body_filter_inline) { - /* Don't eval nginx variables for inline lua code */ - llcf->body_filter_src.value = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("body_filter_by_lua_") + - NGX_HTTP_LUA_INLINE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, "body_filter_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - llcf->body_filter_src_key = p; - - p = ngx_copy(p, "body_filter_by_lua_", - sizeof("body_filter_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + llcf->body_filter_src.value = value[1]; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -1102,19 +1022,15 @@ ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, if (llcf->body_filter_src.lengths == NULL) { /* no variable found */ - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - - llcf->body_filter_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; } } + llcf->body_filter_src_key = cache_key; llcf->body_filter_handler = (ngx_http_output_body_filter_pt) cmd->post; lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 42b6dadce3..5ee1657070 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -115,7 +115,7 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, #else - u_char *p; + u_char *cache_key = NULL; u_char *name; ngx_str_t *value; ngx_http_lua_srv_conf_t *lscf = conf; @@ -139,48 +139,36 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, if (cmd->post == ngx_http_lua_ssl_cert_handler_file) { /* Lua code in an external file */ - name = ngx_http_lua_rebase_path(cf->pool, value[1].data, value[1].len); if (name == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_cert_src.data = name; - lscf->srv.ssl_cert_src.len = ngx_strlen(name); - - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_cert_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + lscf->srv.ssl_cert_src.data = name; + lscf->srv.ssl_cert_src.len = ngx_strlen(name); } else { - /* inlined Lua code */ - - lscf->srv.ssl_cert_src = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("ssl_certificate_by_lua_") + - NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, + "ssl_certificate_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_cert_src_key = p; - - p = ngx_copy(p, "ssl_certificate_by_lua_", - sizeof("ssl_certificate_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + lscf->srv.ssl_cert_src = value[1]; } + lscf->srv.ssl_cert_src_key = cache_key; + return NGX_CONF_OK; #endif /* OPENSSL_VERSION_NUMBER < 0x1000205fL */ diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 9e9e94ccd1..bd72eb813f 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -102,7 +102,7 @@ char * ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key = NULL; u_char *name; ngx_str_t *value; ngx_http_lua_srv_conf_t *lscf = conf; @@ -129,48 +129,36 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, if (cmd->post == ngx_http_lua_ssl_sess_fetch_handler_file) { /* Lua code in an external file */ - name = ngx_http_lua_rebase_path(cf->pool, value[1].data, value[1].len); if (name == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_sess_fetch_src.data = name; - lscf->srv.ssl_sess_fetch_src.len = ngx_strlen(name); - - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_sess_fetch_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + lscf->srv.ssl_sess_fetch_src.data = name; + lscf->srv.ssl_sess_fetch_src.len = ngx_strlen(name); } else { - /* inlined Lua code */ - - lscf->srv.ssl_sess_fetch_src = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("ssl_session_fetch_by_lua_") + - NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, + "ssl_session_fetch_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_sess_fetch_src_key = p; - - p = ngx_copy(p, "ssl_session_fetch_by_lua_", - sizeof("ssl_session_fetch_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + lscf->srv.ssl_sess_fetch_src = value[1]; } + lscf->srv.ssl_sess_fetch_src_key = cache_key; + return NGX_CONF_OK; } diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 4aa1162b30..3569bf6c27 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -100,7 +100,7 @@ char * ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *p; + u_char *cache_key = NULL; u_char *name; ngx_str_t *value; ngx_http_lua_srv_conf_t *lscf = conf; @@ -127,48 +127,36 @@ ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, if (cmd->post == ngx_http_lua_ssl_sess_store_handler_file) { /* Lua code in an external file */ - name = ngx_http_lua_rebase_path(cf->pool, value[1].data, value[1].len); if (name == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_sess_store_src.data = name; - lscf->srv.ssl_sess_store_src.len = ngx_strlen(name); - - p = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); - if (p == NULL) { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_sess_store_src_key = p; - - p = ngx_copy(p, NGX_HTTP_LUA_FILE_TAG, NGX_HTTP_LUA_FILE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + lscf->srv.ssl_sess_store_src.data = name; + lscf->srv.ssl_sess_store_src.len = ngx_strlen(name); } else { - /* inlined Lua code */ - - lscf->srv.ssl_sess_store_src = value[1]; - - p = ngx_palloc(cf->pool, - sizeof("ssl_session_store_by_lua_") + - NGX_HTTP_LUA_INLINE_KEY_LEN); - if (p == NULL) { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, + "ssl_session_store_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { return NGX_CONF_ERROR; } - lscf->srv.ssl_sess_store_src_key = p; - - p = ngx_copy(p, "ssl_session_store_by_lua_", - sizeof("ssl_session_store_by_lua_") - 1); - p = ngx_copy(p, NGX_HTTP_LUA_INLINE_TAG, NGX_HTTP_LUA_INLINE_TAG_LEN); - p = ngx_http_lua_digest_hex(p, value[1].data, value[1].len); - *p = '\0'; + /* Don't eval nginx variables for inline lua code */ + lscf->srv.ssl_sess_store_src = value[1]; } + lscf->srv.ssl_sess_store_src_key = cache_key; + return NGX_CONF_OK; } From 881bf91da6853446398cebc1b2d6a01f8d7939d6 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Sat, 13 Jul 2019 13:06:33 +0800 Subject: [PATCH 338/848] bugfix: ensured arguments of APIs mutating response headers do not contain '\r' and '\n' characters. Signed-off-by: Thibault Charbonnier --- README.markdown | 12 +++ doc/HttpLuaModule.wiki | 12 +++ src/ngx_http_lua_control.c | 2 + src/ngx_http_lua_headers_out.c | 4 + src/ngx_http_lua_util.h | 15 ++++ t/016-resp-header.t | 138 ++++++++++++++++++++++++++++++++- t/022-redirect.t | 76 +++++++++++++++++- 7 files changed, 257 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index de4a35ea5f..7aa6394509 100644 --- a/README.markdown +++ b/README.markdown @@ -4186,6 +4186,11 @@ to be returned when reading `ngx.header.Foo`. Note that `ngx.header` is not a normal Lua table and as such, it is not possible to iterate through it using the Lua `ipairs` function. +Note: `HEADER` and `VALUE` will be truncated if they +contain the `\r` or `\n` characters. The truncated values +will contain all characters up to (and excluding) the first occurrence of +`\r` or `\n`. + For reading *request* headers, use the [ngx.req.get_headers](#ngxreqget_headers) function instead. [Back to TOC](#nginx-api-for-lua) @@ -5151,6 +5156,13 @@ ngx.redirect Issue an `HTTP 301` or `302` redirection to `uri`. +Notice: the `uri` should not contains `\r` or `\n`, otherwise, the characters after `\r` or `\n` will be truncated, including the `\r` or `\n` bytes themself. + +The `uri` argument will be truncated if it contains the +`\r` or `\n` characters. The truncated value will contain +all characters up to (and excluding) the first occurrence of `\r` or +`\n`. + The optional `status` parameter specifies the HTTP status code to be used. The following status codes are supported right now: * `301` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ad829795b4..40107244fa 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3468,6 +3468,11 @@ to be returned when reading ngx.header.Foo. Note that ngx.header is not a normal Lua table and as such, it is not possible to iterate through it using the Lua ipairs function. +Note: HEADER and VALUE will be truncated if they +contain the \r or \n characters. The truncated values +will contain all characters up to (and excluding) the first occurrence of +\r or \n. + For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_headers]] function instead. == ngx.resp.get_headers == @@ -4302,6 +4307,13 @@ It is recommended that a coding style that combines this method call with the HTTP 301 or 302 redirection to uri. +Notice: the uri should not contains \r or \n, otherwise, the characters after \r or \n will be truncated, including the \r or \n bytes themself. + +The uri argument will be truncated if it contains the +\r or \n characters. The truncated value will contain +all characters up to (and excluding) the first occurrence of \r or +\n. + The optional status parameter specifies the HTTP status code to be used. The following status codes are supported right now: * 301 diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 5cd1d64c8e..9c5230156c 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -248,6 +248,8 @@ ngx_http_lua_ngx_redirect(lua_State *L) "the headers"); } + len = ngx_http_lua_safe_header_value_len(p, len); + uri = ngx_palloc(r->pool, len); if (uri == NULL) { return luaL_error(L, "no memory"); diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index cf94bd0423..4b59721eb0 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -491,6 +491,10 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, dd("set header value: %.*s", (int) value.len, value.data); + key.len = ngx_http_lua_safe_header_value_len(key.data, key.len); + + value.len = ngx_http_lua_safe_header_value_len(value.data, value.len); + hv.hash = ngx_hash_key_lc(key.data, key.len); hv.key = key; diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index f08f4815db..179ff3aa71 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -260,6 +260,21 @@ void ngx_http_lua_set_sa_restart(ngx_log_t *log); } +static ngx_inline size_t +ngx_http_lua_safe_header_value_len(u_char *str, size_t len) +{ + size_t i; + + for (i = 0; i < len; i++, str++) { + if (*str == '\r' || *str == '\n') { + return i; + } + } + + return len; +} + + static ngx_inline void ngx_http_lua_init_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) { diff --git a/t/016-resp-header.t b/t/016-resp-header.t index e38202b144..e12c4b2468 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 59); +plan tests => repeat_each() * (blocks() * 3 + 73); #no_diff(); no_long_string(); @@ -1963,3 +1963,139 @@ foo Content-Type: application/json --- no_error_log [error] + + + +=== TEST 87: truncates value after '\r' +--- config + location = /t { + content_by_lua_block { + ngx.header.header = "value\rfoo:bar\nbar:foo" + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +header: value +foo: +bar: +--- no_error_log +[error] + + + +=== TEST 88: truncates value after '\n' +--- config + location = /t { + content_by_lua_block { + ngx.header.header = "value\nfoo:bar\rbar:foo" + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +header: value +foo: +bar: +--- no_error_log +[error] + + + +=== TEST 89: truncates key after '\r' +--- config + location = /t { + content_by_lua_block { + ngx.header["header: value\rfoo:bar\nbar:foo"] = "xx" + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +header: value: xx +foo: +bar: +--- no_error_log +[error] + + + +=== TEST 90: truncates key after '\n' +--- config + location = /t { + content_by_lua_block { + ngx.header["header: value\nfoo:bar\rbar:foo"] = "xx" + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +header: value: xx +foo: +bar: +--- no_error_log +[error] + + + +=== TEST 91: truncates key after '\r' as the first character +--- config + location = /t { + content_by_lua_block { + ngx.header["\rheader: value\rfoo:bar\nbar:foo"] = "xx" + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +header: +foo: +bar: +--- no_error_log +[error] + + + +=== TEST 92: truncates key after '\n' as the first character +--- config + location = /t { + content_by_lua_block { + ngx.header["\nheader: value\nfoo:bar\rbar:foo"] = "xx" + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +header: +foo: +bar: +--- no_error_log +[error] + + + +=== TEST 93: truncates multiple values if they contain '\r' or '\n' +--- config + location = /t { + content_by_lua_block { + ngx.header["foo"] = { + "foo\nxx:bar", + "bar\rxxx:foo", + } + ngx.say("foo") + } + } +--- request +GET /t +--- response_headers +foo: foo, bar +xx: +xxx: +--- no_error_log +[error] diff --git a/t/022-redirect.t b/t/022-redirect.t index 09374d3e3b..a6cc91ea5b 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -9,7 +9,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 3 + 2); +plan tests => repeat_each() * (blocks() * 3 + 8); #no_diff(); #no_long_string(); @@ -320,3 +320,77 @@ GET /read --- response_headers Location: http://agentzh.org/foo?a=b&c=d --- error_code: 308 + + + +=== TEST 18: truncates uri after '\r' +--- config + location = /t { + content_by_lua_block { + ngx.redirect("http://agentzh.org/foo\rfoo:bar\nbar:foo"); + ngx.say("hi") + } + } +--- request +GET /t +--- response_headers +Location: http://agentzh.org/foo +foo: +bar: +--- response_body_like: 302 Found +--- error_code: 302 + + + +=== TEST 19: truncates uri after '\n' +--- config + location = /t { + content_by_lua_block { + ngx.redirect("http://agentzh.org/foo\nfoo:bar\rbar:foo"); + ngx.say("hi") + } + } +--- request +GET /t +--- response_headers +Location: http://agentzh.org/foo +foo: +bar: +--- response_body_like: 302 Found +--- error_code: 302 + + + +=== TEST 20: truncates uri with '\n' as the first character +--- config + location = /t { + content_by_lua_block { + ngx.redirect("\nfoo:http://agentzh.org/foo"); + ngx.say("hi") + } + } +--- request +GET /t +--- response_headers +Location: +foo: +--- response_body_like: 302 Found +--- error_code: 302 + + + +=== TEST 21: truncates uri with '\r' as the first character +--- config + location = /t { + content_by_lua_block { + ngx.redirect("\rfoo:http://agentzh.org/foo"); + ngx.say("hi") + } + } +--- request +GET /t +--- response_headers +Location: +foo: +--- response_body_like: 302 Found +--- error_code: 302 From c618d0d2391f493a3394a3978957ee3de449100c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sat, 27 Jul 2019 09:48:13 -0700 Subject: [PATCH 339/848] bugfix: fixed compilation errors encountered with ngx_stream_lua due to a recent commit. A follow-up commit to 39c2ead3cb7efae4456494ea27a348fa1eca6c5c. --- src/ngx_http_lua_module.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 712bfdb79d..678b2a3254 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -986,9 +986,12 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) * lscf->balancer.src_key = NULL; */ +#if (NGX_HTTP_SSL) lscf->srv.ssl_cert_src_ref = LUA_REFNIL; lscf->srv.ssl_sess_store_src_ref = LUA_REFNIL; lscf->srv.ssl_sess_fetch_src_ref = LUA_REFNIL; +#endif + lscf->balancer.src_ref = LUA_REFNIL; return lscf; From d313bf8dea8d697cffa8fd08224febd0db3d6046 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 26 Jul 2019 12:45:35 -0700 Subject: [PATCH 340/848] bugfix: ensured the 'pcre_version' symbol is also preserved on Darwin platforms. Follow-up commit to 2014dd80fe4f299ad68c3b60a2c5f846b4286b6f --- config | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/config b/config index ff44482bc4..5065a4e9af 100644 --- a/config +++ b/config @@ -592,7 +592,6 @@ fi if [ $PCRE != NO -a $PCRE != YES ]; then # force pcre_version symbol to be undefined when PCRE is statically linked - ngx_feature="force undefined symbols (--undefined)" ngx_feature_libs="-Wl,--undefined=printf" ngx_feature_name= @@ -606,6 +605,21 @@ if [ $PCRE != NO -a $PCRE != YES ]; then if [ $ngx_found = yes ]; then CORE_LIBS="$CORE_LIBS -Wl,--undefined=pcre_version" fi + + # for LLVM ld (Darwin) + ngx_feature="force undefined symbols (-all_load -U)" + ngx_feature_libs="-all_load -U printf" + ngx_feature_name= + ngx_feature_run=no + ngx_feature_incs="#include " + ngx_feature_path= + ngx_feature_test='printf("hello");' + + . auto/feature + + if [ $ngx_found = yes ]; then + CORE_LIBS="$CORE_LIBS -all_load -U pcre_version" + fi fi # ---------------------------------------- From 6c25b7bea9eae8ab14d3bd6ac4a68ad1a348a238 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sun, 28 Jul 2019 13:04:19 -0700 Subject: [PATCH 341/848] optimize: removed declaration of obsolete 'ngx_http_lua_inject_logby_ngx_api' function. This function was removed in 5078b14431f612693a4d468f200b792d128347d6. --- src/ngx_http_lua_logby.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ngx_http_lua_logby.h b/src/ngx_http_lua_logby.h index af8aaaa756..2b011c884e 100644 --- a/src/ngx_http_lua_logby.h +++ b/src/ngx_http_lua_logby.h @@ -14,7 +14,6 @@ ngx_int_t ngx_http_lua_log_handler(ngx_http_request_t *r); ngx_int_t ngx_http_lua_log_handler_inline(ngx_http_request_t *r); ngx_int_t ngx_http_lua_log_handler_file(ngx_http_request_t *r); -void ngx_http_lua_inject_logby_ngx_api(ngx_conf_t *cf, lua_State *L); #endif /* _NGX_HTTP_LUA_LOGBY_H_INCLUDED_ */ From eed28fca8d46dd0cf5d5486b6545e943a30c8415 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 29 Jul 2019 10:34:41 -0700 Subject: [PATCH 342/848] optimize: fixed the pre-allocated number of keys in the 'ngx.socket.tcp' table. --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 87461e6ac2..42abb7e4f2 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -304,7 +304,7 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* {{{tcp object metatable */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( tcp_socket_metatable_key)); - lua_createtable(L, 0 /* narr */, 12 /* nrec */); + lua_createtable(L, 0 /* narr */, 13 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_connect); lua_setfield(L, -2, "connect"); From ba182dd599d5383e53c5b8837c565f3ee42aa806 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Sun, 28 Jul 2019 13:05:23 -0700 Subject: [PATCH 343/848] style: fixed some minor styling issues to align with the meta_lua module. --- src/ddebug.h | 32 ++++++++++++++++---------------- src/ngx_http_lua_clfactory.c | 12 ++++++------ src/ngx_http_lua_common.h | 1 - src/ngx_http_lua_log_ringbuf.c | 3 +++ src/ngx_http_lua_module.c | 6 +++--- src/ngx_http_lua_regex.c | 2 +- src/ngx_http_lua_shdict.c | 2 +- src/ngx_http_lua_util.c | 8 ++++---- 8 files changed, 34 insertions(+), 32 deletions(-) diff --git a/src/ddebug.h b/src/ddebug.h index f2d4b738f9..7bf22e9bc1 100644 --- a/src/ddebug.h +++ b/src/ddebug.h @@ -17,8 +17,8 @@ # if (NGX_HAVE_VARIADIC_MACROS) -# define dd(...) fprintf(stderr, "lua *** %s: ", __func__); \ - fprintf(stderr, __VA_ARGS__); \ +# define dd(...) fprintf(stderr, "lua *** %s: ", __func__); \ + fprintf(stderr, __VA_ARGS__); \ fprintf(stderr, " at %s line %d.\n", __FILE__, __LINE__) # else @@ -54,22 +54,22 @@ dd(const char *fmt, ...) { #if defined(DDEBUG) && (DDEBUG) -#define dd_check_read_event_handler(r) \ - dd("r->read_event_handler = %s", \ - r->read_event_handler == ngx_http_block_reading ? \ - "ngx_http_block_reading" : \ - r->read_event_handler == ngx_http_test_reading ? \ - "ngx_http_test_reading" : \ - r->read_event_handler == ngx_http_request_empty_handler ? \ +#define dd_check_read_event_handler(r) \ + dd("r->read_event_handler = %s", \ + r->read_event_handler == ngx_http_block_reading ? \ + "ngx_http_block_reading" : \ + r->read_event_handler == ngx_http_test_reading ? \ + "ngx_http_test_reading" : \ + r->read_event_handler == ngx_http_request_empty_handler ? \ "ngx_http_request_empty_handler" : "UNKNOWN") -#define dd_check_write_event_handler(r) \ - dd("r->write_event_handler = %s", \ - r->write_event_handler == ngx_http_handler ? \ - "ngx_http_handler" : \ - r->write_event_handler == ngx_http_core_run_phases ? \ - "ngx_http_core_run_phases" : \ - r->write_event_handler == ngx_http_request_empty_handler ? \ +#define dd_check_write_event_handler(r) \ + dd("r->write_event_handler = %s", \ + r->write_event_handler == ngx_http_handler ? \ + "ngx_http_handler" : \ + r->write_event_handler == ngx_http_core_run_phases ? \ + "ngx_http_core_run_phases" : \ + r->write_event_handler == ngx_http_request_empty_handler ? \ "ngx_http_request_empty_handler" : "UNKNOWN") #else diff --git a/src/ngx_http_lua_clfactory.c b/src/ngx_http_lua_clfactory.c index 754ed8d2f1..045e46adf9 100644 --- a/src/ngx_http_lua_clfactory.c +++ b/src/ngx_http_lua_clfactory.c @@ -380,12 +380,12 @@ ngx_http_lua_clfactory_bytecode_prepare(lua_State *L, #if defined(DDEBUG) && (DDEBUG) { - dd("==LJ_BT_HEADER=="); - size_t i; - for (i = 0; i < LJ_HEADERSIZE; i++) { - dd("%ld: 0x%02X", i, (unsigned)(u_char) lf->begin_code.str[i]); - } - dd("==LJ_BT_HEADER_END=="); + dd("==LJ_BT_HEADER=="); + size_t i; + for (i = 0; i < LJ_HEADERSIZE; i++) { + dd("%ld: 0x%02X", i, (unsigned)(u_char) lf->begin_code.str[i]); + } + dd("==LJ_BT_HEADER_END=="); } #endif diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 44f11b41c9..31ff53ad01 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -628,7 +628,6 @@ typedef struct { ngx_str_t name; ngx_uint_t offset; ngx_http_lua_set_header_pt handler; - } ngx_http_lua_set_header_t; diff --git a/src/ngx_http_lua_log_ringbuf.c b/src/ngx_http_lua_log_ringbuf.c index 04640697ff..0227f63fb2 100644 --- a/src/ngx_http_lua_log_ringbuf.c +++ b/src/ngx_http_lua_log_ringbuf.c @@ -223,3 +223,6 @@ ngx_http_lua_log_ringbuf_read(ngx_http_lua_log_ringbuf_t *rb, int *log_level, return NGX_OK; } + + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 678b2a3254..afc2f93821 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1028,7 +1028,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) #ifdef LIBRESSL_VERSION_NUMBER ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "LibreSSL does not support ssl_certificate_by_lua*"); + "LibreSSL is not supported by ssl_certificate_by_lua*"); return NGX_CONF_ERROR; #else @@ -1060,7 +1060,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (sscf && sscf->ssl.ctx) { #ifdef LIBRESSL_VERSION_NUMBER ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "LibreSSL does not support " + "LibreSSL is not supported by " "ssl_session_store_by_lua*"); return NGX_CONF_ERROR; @@ -1083,7 +1083,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (sscf && sscf->ssl.ctx) { #ifdef LIBRESSL_VERSION_NUMBER ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "LibreSSL does not support " + "LibreSSL is not supported by " "ssl_session_fetch_by_lua*"); return NGX_CONF_ERROR; diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 8f3373b4d5..9593575d7c 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -2043,7 +2043,7 @@ ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc) if ((size_t) erroff == rc->pattern.len) { rc->err.len = ngx_snprintf(rc->err.data, rc->err.len, "pcre_compile() failed: %s in \"%V\"", - errstr, &rc->pattern) + errstr, &rc->pattern) - rc->err.data; } else { diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index b017bea658..0cda3b7b18 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -44,7 +44,7 @@ static int ngx_http_lua_shdict_llen(lua_State *L); static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L, - int index); + int index); #define NGX_HTTP_LUA_SHDICT_ADD 0x0001 diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 47768c003b..219bff6fd3 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -2588,9 +2588,9 @@ ngx_http_lua_process_args_option(ngx_http_request_t *r, lua_State *L, if (lua_isboolean(L, -1)) { if (lua_toboolean(L, -1)) { if (total_escape) { - p = (u_char *) ngx_http_lua_escape_uri(p, key, - key_len, - NGX_ESCAPE_URI_COMPONENT); + p = (u_char *) + ngx_http_lua_escape_uri(p, key, key_len, + NGX_ESCAPE_URI_COMPONENT); } else { dd("shortcut: no escape required"); @@ -3371,7 +3371,7 @@ ngx_http_lua_check_broken_connection(ngx_http_request_t *r, ngx_event_t *ev) char buf[1]; ngx_err_t err; ngx_int_t event; - ngx_connection_t *c; + ngx_connection_t *c; ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ev->log, 0, "http lua check client, write event:%d, \"%V\"", From 9fa316f50b980eb2e26a8271e9927359ba77b95e Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 30 Jul 2019 13:19:32 -0700 Subject: [PATCH 344/848] tweak: updated the debug log in 'ngx_http_lua_socket_tcp_conn_op_ctx_cleanup()' to align with the meta_lua module. --- src/ngx_http_lua_socket_tcp.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 42abb7e4f2..3173ab1b94 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4012,9 +4012,11 @@ ngx_http_lua_socket_tcp_conn_op_ctx_cleanup(void *data) ngx_http_lua_socket_tcp_conn_op_ctx_t *conn_op_ctx = data; u = conn_op_ctx->u; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, - "cleanup lua tcp socket conn_op_ctx: \"%V\"", - &u->request->uri); + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "cleanup lua tcp socket conn_op_ctx: %p, u: %p, " + "request: \"%V\"", + conn_op_ctx, u, &u->request->uri); ngx_queue_insert_head(&u->socket_pool->cache_connect_op, &conn_op_ctx->queue); From 6b3822b18c3ac0b2d4d61a02b32d10a6207e1a4d Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 30 Jul 2019 14:24:23 -0700 Subject: [PATCH 345/848] bugfix: added missing arguments to an 'ngx_log_error()' call in ngx_http_lua_pipe.c. --- src/ngx_http_lua_pipe.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index f9ae6cd455..c525e0b9b8 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -1133,7 +1133,8 @@ ngx_http_lua_ffi_pipe_proc_destroy(ngx_http_lua_ffi_pipe_proc_t *proc) if (kill(proc->_pid, SIGKILL) == -1) { if (ngx_errno != ESRCH) { ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, - "lua pipe failed to kill process:%p pid:%P"); + "lua pipe failed to kill process:%p pid:%P", + proc, proc->_pid); } } } From 2f270642bfd3fc252a278a48860ff168d65be703 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 5 Aug 2019 15:19:26 -0700 Subject: [PATCH 346/848] bugfix: fixed file cache key size allocation. Fix #1572 --- src/ngx_http_lua_cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index 3e0003a4d4..30f29fc4a7 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -392,7 +392,7 @@ ngx_http_lua_gen_file_cache_key(ngx_conf_t *cf, const u_char *src, { u_char *out; - out = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_TAG_LEN + 1); + out = ngx_palloc(cf->pool, NGX_HTTP_LUA_FILE_KEY_LEN + 1); if (out == NULL) { return NULL; } From a96e99ab1a770d261fa9eecbd9c7d112f39aa34b Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 31 Jul 2019 20:32:05 -0700 Subject: [PATCH 347/848] change: we now prevent compilation with PUC-Rio Lua; only LuaJIT 2.x is supported. --- config | 313 +++++++++++++------------------------- src/ngx_http_lua_common.h | 5 + 2 files changed, 108 insertions(+), 210 deletions(-) diff --git a/config b/config index 5065a4e9af..bf8bf5cb8a 100644 --- a/config +++ b/config @@ -1,13 +1,3 @@ -ngx_feature="Lua library" -ngx_feature_libs="-llua -lm" -ngx_feature_name= -ngx_feature_run=no -ngx_feature_incs="#include " -ngx_feature_path= -ngx_feature_test="#if LUA_VERSION_NUM != 501 -# error unsupported Lua language version -#endif -(void) luaL_newstate();" ngx_lua_opt_I= ngx_lua_opt_L= @@ -20,7 +10,7 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then ngx_lua_opt_I="-I$LUAJIT_INC" ngx_lua_opt_L="-L$LUAJIT_LIB" - # ensure that our -I$LUAJIT_INC and -L$LUAJIT_LIB is at the first. + # ensure that -I$LUAJIT_INC and -L$LUAJIT_LIB come first SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" SAVED_NGX_TEST_LD_OPT="$NGX_TEST_LD_OPT" @@ -35,14 +25,13 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" NGX_TEST_LD_OPT="$SAVED_NGX_TEST_LD_OPT" else - # attempt to link with -ldl, static linking on Linux requires it. ngx_feature="LuaJIT library in $LUAJIT_LIB and $LUAJIT_INC (specified by the LUAJIT_LIB and LUAJIT_INC env, with -ldl)" ngx_feature_path="$LUAJIT_INC" ngx_lua_opt_I="-I$LUAJIT_INC" ngx_lua_opt_L="-L$LUAJIT_LIB" - # ensure that our -I$LUAJIT_INC and -L$LUAJIT_LIB is at the first. + # ensure that -I$LUAJIT_INC and -L$LUAJIT_LIB come first SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" SAVED_NGX_TEST_LD_OPT="$NGX_TEST_LD_OPT" @@ -67,7 +56,7 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then ngx_lua_opt_I="-I$LUAJIT_INC" ngx_lua_opt_L="-L$LUAJIT_LIB" - # ensure that our -I$LUAJIT_INC and -L$LUAJIT_LIB is at the first. + # ensure that -I$LUAJIT_INC and -L$LUAJIT_LIB come first SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" SAVED_NGX_TEST_LD_OPT="$NGX_TEST_LD_OPT" @@ -89,7 +78,7 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then if [ $ngx_found = no ]; then cat << END - $0: error: ngx_http_lua_module requires the Lua or LuaJIT library and LUAJIT_LIB is defined as $LUAJIT_LIB and LUAJIT_INC (path for lua.h) $LUAJIT_INC, but we cannot find LuaJIT there. + $0: error: ngx_http_lua_module requires the LuaJIT library, but it could not be found where specified (LUAJIT_LIB=$LUAJIT_LIB, LUAJIT_INC=$LUAJIT_INC). END exit 1 fi @@ -98,7 +87,7 @@ END Darwin:*) case "$NGX_MACHINE" in amd64 | x86_64 | i386) - echo "adding extra linking options needed by LuaJIT" + echo "adding extra linking options needed by LuaJIT on $NGX_MACHINE" ngx_feature_libs="$ngx_feature_libs -pagezero_size 10000 -image_base 100000000" ;; @@ -111,176 +100,41 @@ END ;; esac else - if [ -n "$LUA_INC" -o -n "$LUA_LIB" ]; then - # explicitly set Lua paths - ngx_feature="Lua library in $LUA_LIB and $LUA_INC (specified by the LUA_LIB and LUA_INC env)" - ngx_feature_path="$LUA_INC" - ngx_lua_opt_I="-I$LUA_INC" - ngx_lua_opt_L="-L$LUA_LIB" - - # ensure that our -I$LUA_INC and -L$LUA_LIB is at the first. - SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" - CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" - SAVED_NGX_TEST_LD_OPT="$NGX_TEST_LD_OPT" - NGX_TEST_LD_OPT="$ngx_lua_opt_L $NGX_TEST_LD_OPT" - + # auto-discovery + if [ $ngx_found = no ]; then + # FreeBSD with luajit-2.0 from ports collection + ngx_feature="LuaJIT library in /usr/local/" + ngx_feature_path="/usr/local/include/luajit-2.0" if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R$LUA_LIB $ngx_lua_opt_L -llua -lm -ldl" + ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -lluajit-5.1 -lm" else - ngx_feature_libs="$ngx_lua_opt_L -llua -lm -ldl" + ngx_feature_libs="-L/usr/local/lib -lluajit-5.1 -lm" fi - . auto/feature + fi - # clean up - CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" - NGX_TEST_LD_OPT="$SAVED_NGX_TEST_LD_OPT" - - if [ $ngx_found = no ]; then - # retry without -ldl - - ngx_feature_path="$LUA_INC" - ngx_lua_opt_I="-I$LUA_INC" - ngx_lua_opt_L="-L$LUA_LIB" - - # ensure that our -I$LUA_INC and -L$LUA_LIB is at the first. - SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" - CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" - SAVED_NGX_TEST_LD_OPT="$NGX_TEST_LD_OPT" - NGX_TEST_LD_OPT="$ngx_lua_opt_L $NGX_TEST_LD_OPT" - - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R$LUA_LIB $ngx_lua_opt_L -llua -lm" - else - ngx_feature_libs="$ngx_lua_opt_L -llua -lm" - fi - - . auto/feature - - # clean up - CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" - NGX_TEST_LD_OPT="$SAVED_NGX_TEST_LD_OPT" - fi - - if [ $ngx_found = no ]; then - cat << END - $0: error: ngx_http_lua_module requires the Lua or LuaJIT library and LUA_LIB is defined as $LUA_LIB and LUA_INC (path for lua.h) is $LUA_INC, but we cannot find standard Lua there. -END - exit 1 + if [ $ngx_found = no ]; then + # Gentoo with LuaJIT-2.0, try with -ldl + ngx_feature="LuaJIT library in /usr/" + ngx_feature_path="/usr/include/luajit-2.0" + if [ $NGX_RPATH = YES ]; then + ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -lluajit-5.1 -ldl" + else + ngx_feature_libs="-L/usr/lib -lm -lluajit-5.1 -ldl" fi - else - # auto-discovery - ngx_feature="Lua library" - ngx_feature_libs="-llua -lm" . auto/feature + fi - if [ $ngx_found = no ]; then - # OpenBSD-5.2 - ngx_feature="Lua library in /usr/local/" - ngx_feature_path="/usr/local/include/lua-5.1" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -llua -lm" - else - ngx_feature_libs="-L/usr/local/lib -llua5.1 -lm" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # OpenBSD < 5.2 - ngx_feature="Lua library in /usr/local/" - ngx_feature_path="/usr/local/include" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -llua -lm" - else - ngx_feature_libs="-L/usr/local/lib -llua -lm" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # NetBSD - ngx_feature="Lua library in /usr/pkg/" - ngx_feature_path="/usr/pkg/include/" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/pkg/lib -L/usr/pkg/lib -lm -llua" - else - ngx_feature_libs="-L/usr/pkg/lib -lm -llua" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # MacPorts - ngx_feature="Lua library in /opt/local/" - ngx_feature_path="/opt/local/include" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/opt/local/lib -L/opt/local/lib -lm -llua" - else - ngx_feature_libs="-L/opt/local/lib -lm -llua" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # FreeBSD - ngx_feature="Lua library in /usr/local/*/lua51/" - ngx_feature_path="/usr/local/include/lua51" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/local/lib/lua51 -L/usr/local/lib/lua51 -llua -lm" - else - ngx_feature_libs="-L/usr/local/lib/lua51 -llua -lm" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # Debian - ngx_feature="Lua library in /usr/" - ngx_feature_path="/usr/include/lua5.1" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -llua5.1" - else - ngx_feature_libs="-L/usr/lib -lm -llua5.1" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # FreeBSD with luajit-2.0 from ports collection - ngx_feature="LuaJIT library in /usr/local/" - ngx_feature_path="/usr/local/include/luajit-2.0" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -lluajit-5.1 -lm" - else - ngx_feature_libs="-L/usr/local/lib -lluajit-5.1 -lm" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # Gentoo with LuaJIT-2.0, try with -ldl - ngx_feature="LuaJIT library in /usr/" - ngx_feature_path="/usr/include/luajit-2.0" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -lluajit-5.1 -ldl" - else - ngx_feature_libs="-L/usr/lib -lm -lluajit-5.1 -ldl" - fi - . auto/feature - fi - - if [ $ngx_found = no ]; then - # Gentoo with LuaJIT 2.0 - ngx_feature="LuaJIT library in /usr/" - ngx_feature_path="/usr/include/luajit-2.0" - if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -lluajit-5.1" - else - ngx_feature_libs="-L/usr/lib -lm -lluajit-5.1" - fi - . auto/feature + if [ $ngx_found = no ]; then + # Gentoo with LuaJIT 2.0 + ngx_feature="LuaJIT library in /usr/" + ngx_feature_path="/usr/include/luajit-2.0" + if [ $NGX_RPATH = YES ]; then + ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -lluajit-5.1" + else + ngx_feature_libs="-L/usr/lib -lm -lluajit-5.1" fi + . auto/feature fi fi @@ -289,19 +143,71 @@ ngx_module_libs= if [ $ngx_found = yes ]; then # this is a hack to persuade nginx's build system to favor - # the paths set by our user environments: + # the paths set by our user environment CFLAGS="$ngx_lua_opt_I $CFLAGS" NGX_LD_OPT="$ngx_lua_opt_L $NGX_LD_OPT" ngx_module_incs="$ngx_module_incs $ngx_feature_path" ngx_module_libs="$ngx_module_libs $ngx_feature_libs" else - cat << END - $0: error: ngx_http_lua_module requires the Lua library. + cat << END + $0: error: ngx_http_lua_module requires the LuaJIT library. END - exit 1 + exit 1 fi +# ---------------------------------------- + +ngx_feature="LuaJIT 2.x" +ngx_feature_name= +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_path= +ngx_feature_test='#if !defined(LUAJIT_VERSION_NUM) || (LUAJIT_VERSION_NUM < 20000) + # error unsupported LuaJIT version + #endif' +SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" +CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" + +. auto/feature + +if [ $ngx_found = no ]; then + cat << END + $0: error: unsupported LuaJIT version; ngx_http_lua_module requires LuaJIT 2.x. +END + exit 1 +fi + +CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" + +# ---------------------------------------- + +ngx_feature="LuaJIT has FFI" +ngx_feature_name= +ngx_feature_run=yes +ngx_feature_incs="#include + #include + #include " +ngx_feature_path= +ngx_feature_test='lua_State *L = luaL_newstate(); + assert(L != NULL); + luaopen_ffi(L);' +SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" +CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" + +. auto/feature + +if [ $ngx_found = no ]; then + cat << END + $0: error: unsupported LuaJIT build; ngx_http_lua_module requires LuaJIT with FFI enabled. +END + exit 1 +fi + +CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" + +# ---------------------------------------- + ngx_addon_name=ngx_http_lua_module HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_script.c \ @@ -428,7 +334,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_pipe.h \ " -CFLAGS="$CFLAGS -DNDK_SET_VAR" +# ---------------------------------------- ngx_feature="export symbols by default (-E)" ngx_feature_libs="-Wl,-E" @@ -444,6 +350,8 @@ if [ $ngx_found = yes ]; then CORE_LIBS="-Wl,-E $CORE_LIBS" fi +# ---------------------------------------- + # for Cygwin ngx_feature="export symbols by default (--export-all-symbols)" ngx_feature_libs="-Wl,--export-all-symbols" @@ -459,13 +367,7 @@ if [ $ngx_found = yes ]; then CORE_LIBS="-Wl,--export-all-symbols $CORE_LIBS" fi -NGX_DTRACE_PROVIDERS="$NGX_DTRACE_PROVIDERS $ngx_addon_dir/dtrace/ngx_lua_provider.d" -NGX_TAPSET_SRCS="$NGX_TAPSET_SRCS $ngx_addon_dir/tapset/ngx_lua.stp" - -USE_MD5=YES -USE_SHA1=YES - -CORE_INCS="$CORE_INCS $ngx_addon_dir/src/api" +# ---------------------------------------- ngx_feature="SO_PASSCRED" ngx_feature_libs= @@ -478,6 +380,8 @@ ngx_feature_test='setsockopt(1, SOL_SOCKET, SO_PASSCRED, NULL, 0);' . auto/feature +# ---------------------------------------- + ngx_feature="SA_RESTART" ngx_feature_libs= ngx_feature_name="NGX_HTTP_LUA_HAVE_SA_RESTART" @@ -489,26 +393,6 @@ ngx_feature_test='struct sigaction act; . auto/feature -ngx_feature="__attribute__(constructor)" -ngx_feature_libs= -ngx_feature_name="NGX_HTTP_LUA_HAVE_CONSTRUCTOR" -ngx_feature_run=yes -ngx_feature_incs="#include -int a = 2; -__attribute__((constructor)) -static void foo(void) -{ - a = 0; -} -" -ngx_feature_test="exit(a);" -SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" -CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" - -. auto/feature - -CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" - # ---------------------------------------- ngx_feature="malloc_trim" @@ -572,7 +456,7 @@ CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" # ---------------------------------------- -if test -n "$ngx_module_link"; then +if [ -n "$ngx_module_link" ]; then ngx_module_type=HTTP_AUX_FILTER ngx_module_name=$ngx_addon_name ngx_module_deps="$HTTP_LUA_DEPS" @@ -624,5 +508,14 @@ fi # ---------------------------------------- +USE_MD5=YES +USE_SHA1=YES + +NGX_DTRACE_PROVIDERS="$NGX_DTRACE_PROVIDERS $ngx_addon_dir/dtrace/ngx_lua_provider.d" +NGX_TAPSET_SRCS="$NGX_TAPSET_SRCS $ngx_addon_dir/tapset/ngx_lua.stp" + +CORE_INCS="$CORE_INCS $ngx_addon_dir/src/api" + +CFLAGS="$CFLAGS -DNDK_SET_VAR" #CFLAGS=$"$CFLAGS -DLUA_DEFAULT_PATH='\"/usr/local/openresty/lualib/?.lua\"'" #CFLAGS=$"$CFLAGS -DLUA_DEFAULT_CPATH='\"/usr/local/openresty/lualib/?.so\"'" diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 31ff53ad01..6e4c2ec7da 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -50,6 +50,11 @@ #endif +#if !defined(LUAJIT_VERSION_NUM) || (LUAJIT_VERSION_NUM < 20000) +# error unsupported LuaJIT version +#endif + + #if (!defined OPENSSL_NO_OCSP && defined SSL_CTRL_SET_TLSEXT_STATUS_REQ_CB) # define NGX_HTTP_LUA_USE_OCSP 1 #endif From 60736e686ac5ca5af9a5bf118cb9bd4a9126cefc Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 1 Aug 2019 14:41:12 -0700 Subject: [PATCH 348/848] change: 'resty.core' is now mandatorily loaded, and the 'lua_load_resty_core' directive is deprecated. --- README.markdown | 28 ++++------- doc/HttpLuaModule.wiki | 27 ++++------- src/ngx_http_lua_common.h | 2 - src/ngx_http_lua_directive.c | 11 +++++ src/ngx_http_lua_directive.h | 2 + src/ngx_http_lua_module.c | 38 ++++++++++----- src/ngx_http_lua_util.c | 38 +++++++-------- src/ngx_http_lua_util.h | 38 +++++++++++---- t/161-load-resty-core.t | 90 ++++++++++++++++++++++++++++-------- 9 files changed, 171 insertions(+), 103 deletions(-) diff --git a/README.markdown b/README.markdown index 7aa6394509..407a1d613a 100644 --- a/README.markdown +++ b/README.markdown @@ -1076,26 +1076,14 @@ lua_load_resty_core **context:** *http* -Controls whether the `resty.core` module (from -[lua-resty-core](https://github.com/openresty/lua-resty-core)) should be loaded -or not. When enabled, this directive is equivalent to executing the following -when the Lua VM is created: - -```lua - - require "resty.core" -``` - -Note that usage of the `resty.core` module is recommended, as its -FFI implementation is both faster, safer, and more complete than the Lua C API -of the ngx_lua module. - -It must also be noted that the Lua C API of the ngx_lua module will eventually -be removed, and usage of the FFI-based API (i.e. the `resty.core` -module) will become mandatory. This directive only aims at providing a -temporary backwards-compatibility mode in case of edge-cases. - -This directive was first introduced in the `v0.10.15` release. +This directive is deprecated since the `v0.10.16` release of this +module. The `resty.core` module from +[lua-resty-core](https://github.com/openresty/lua-resty-core) is now mandatorily +loaded during the Lua VM initialization. Specifying this directive will have no +effect. + +This directive was first introduced in the `v0.10.15` release and +used to optionally load the `resty.core` module. [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 40107244fa..ce6528e434 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -831,25 +831,14 @@ how the result will be used. Below is a diagram showing the order in which direc '''context:''' ''http'' -Controls whether the resty.core module (from -[https://github.com/openresty/lua-resty-core lua-resty-core]) should be loaded -or not. When enabled, this directive is equivalent to executing the following -when the Lua VM is created: - - - require "resty.core" - - -Note that usage of the resty.core module is recommended, as its -FFI implementation is both faster, safer, and more complete than the Lua C API -of the ngx_lua module. - -It must also be noted that the Lua C API of the ngx_lua module will eventually -be removed, and usage of the FFI-based API (i.e. the resty.core -module) will become mandatory. This directive only aims at providing a -temporary backwards-compatibility mode in case of edge-cases. - -This directive was first introduced in the v0.10.15 release. +This directive is deprecated since the v0.10.16 release of this +module. The resty.core module from +[https://github.com/openresty/lua-resty-core lua-resty-core] is now mandatorily +loaded during the Lua VM initialization. Specifying this directive will have no +effect. + +This directive was first introduced in the v0.10.15 release and +used to optionally load the resty.core module. == lua_capture_error_log == '''syntax:''' ''lua_capture_error_log size'' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 6e4c2ec7da..82907a479e 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -188,8 +188,6 @@ struct ngx_http_lua_main_conf_s { ngx_cycle_t *cycle; ngx_pool_t *pool; - ngx_flag_t load_resty_core; - ngx_int_t max_pending_timers; ngx_int_t pending_timers; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 3186581a3a..cdc573097b 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -179,6 +179,17 @@ ngx_http_lua_code_cache(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) } +char * +ngx_http_lua_load_resty_core(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_conf_log_error(NGX_LOG_WARN, cf, 0, + "lua_load_resty_core is deprecated (the lua-resty-core " + "library is required since ngx_lua v0.10.16)"); + + return NGX_CONF_OK; +} + + char * ngx_http_lua_package_cpath(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index 5abfe4dde2..c72c90dbe7 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -50,6 +50,8 @@ char *ngx_http_lua_init_worker_by_lua_block(ngx_conf_t *cf, char *ngx_http_lua_init_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_code_cache(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +char *ngx_http_lua_load_resty_core(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); #if defined(NDK) && NDK diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index afc2f93821..4a5d8cf1ef 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -79,9 +79,9 @@ static ngx_command_t ngx_http_lua_cmds[] = { { ngx_string("lua_load_resty_core"), NGX_HTTP_MAIN_CONF|NGX_CONF_FLAG, - ngx_conf_set_flag_slot, + ngx_http_lua_load_resty_core, NGX_HTTP_MAIN_CONF_OFFSET, - offsetof(ngx_http_lua_main_conf_t, load_resty_core), + 0, NULL }, { ngx_string("lua_max_running_timers"), @@ -787,14 +787,33 @@ ngx_http_lua_init(ngx_conf_t *cf) ngx_http_lua_hash_literal("content-length"); ngx_http_lua_location_hash = ngx_http_lua_hash_literal("location"); - lmcf->lua = ngx_http_lua_init_vm(NULL, cf->cycle, cf->pool, lmcf, - cf->log, NULL); - if (lmcf->lua == NULL) { - ngx_conf_log_error(NGX_LOG_ERR, cf, 0, - "failed to initialize Lua VM"); + rc = ngx_http_lua_init_vm(&lmcf->lua, NULL, cf->cycle, cf->pool, + lmcf, cf->log, NULL); + if (rc != NGX_OK) { + if (rc == NGX_DECLINED) { + ngx_http_lua_assert(lmcf->lua != NULL); + + ngx_conf_log_error(NGX_LOG_ALERT, cf, 0, + "failed to load the 'resty.core' module " + "(https://github.com/openresty/lua-resty" + "-core); ensure you are using an OpenResty " + "release from https://openresty.org/en/" + "download.html (reason: %s)", + lua_tostring(lmcf->lua, -1)); + + } else { + /* rc == NGX_ERROR */ + ngx_conf_log_error(NGX_LOG_ALERT, cf, 0, + "failed to initialize Lua VM"); + } + return NGX_ERROR; } + /* rc == NGX_OK */ + + ngx_http_lua_assert(lmcf->lua != NULL); + if (!lmcf->requires_shm && lmcf->init_handler) { saved_cycle = ngx_cycle; ngx_cycle = cf->cycle; @@ -884,7 +903,6 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) */ lmcf->pool = cf->pool; - lmcf->load_resty_core = NGX_CONF_UNSET; lmcf->max_pending_timers = NGX_CONF_UNSET; lmcf->max_running_timers = NGX_CONF_UNSET; #if (NGX_PCRE) @@ -918,10 +936,6 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) { ngx_http_lua_main_conf_t *lmcf = conf; - if (lmcf->load_resty_core == NGX_CONF_UNSET) { - lmcf->load_resty_core = 1; - } - #if (NGX_PCRE) if (lmcf->regex_cache_max_entries == NGX_CONF_UNSET) { lmcf->regex_cache_max_entries = 1024; diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 219bff6fd3..6d5b511aa2 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -3814,10 +3814,10 @@ ngx_http_lua_close_fake_connection(ngx_connection_t *c) } -lua_State * -ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, - ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log, - ngx_pool_cleanup_t **pcln) +ngx_int_t +ngx_http_lua_init_vm(lua_State **new_vm, lua_State *parent_vm, + ngx_cycle_t *cycle, ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, + ngx_log_t *log, ngx_pool_cleanup_t **pcln) { int rc; lua_State *L; @@ -3828,13 +3828,13 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, cln = ngx_pool_cleanup_add(pool, 0); if (cln == NULL) { - return NULL; + return NGX_ERROR; } /* create new Lua VM instance */ L = ngx_http_lua_new_state(parent_vm, cycle, lmcf, log); if (L == NULL) { - return NULL; + return NGX_ERROR; } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, "lua initialize the " @@ -3845,7 +3845,7 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, state = ngx_alloc(sizeof(ngx_http_lua_vm_state_t), log); if (state == NULL) { - return NULL; + return NGX_ERROR; } state->vm = L; state->count = 1; @@ -3878,7 +3878,8 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, for (i = 0; i < lmcf->preload_hooks->nelts; i++) { - ngx_http_lua_probe_register_preload_package(L, hook[i].package); + ngx_http_lua_probe_register_preload_package(L, + hook[i].package); lua_pushcfunction(L, hook[i].loader); lua_setfield(L, -2, (char *) hook[i].package); @@ -3887,22 +3888,17 @@ ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, lua_pop(L, 2); } - if (lmcf->load_resty_core) { - lua_getglobal(L, "require"); - lua_pushstring(L, "resty.core"); + *new_vm = L; - rc = lua_pcall(L, 1, 1, 0); - if (rc != 0) { - ngx_log_error(NGX_LOG_ERR, log, 0, - "lua_load_resty_core failed to load the resty.core " - "module from https://github.com/openresty/lua-resty" - "-core; ensure you are using an OpenResty release " - "from https://openresty.org/en/download.html " - "(rc: %i, reason: %s)", rc, lua_tostring(L, -1)); - } + lua_getglobal(L, "require"); + lua_pushstring(L, "resty.core"); + + rc = lua_pcall(L, 1, 1, 0); + if (rc != 0) { + return NGX_DECLINED; } - return L; + return NGX_OK; } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 179ff3aa71..c80bf758fb 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -136,9 +136,9 @@ ngx_http_lua_ffi_check_context(ngx_http_lua_ctx_t *ctx, unsigned flags, SSL_get_ex_data(ssl_conn, ngx_http_lua_ssl_ctx_index) -lua_State *ngx_http_lua_init_vm(lua_State *parent_vm, ngx_cycle_t *cycle, - ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log, - ngx_pool_cleanup_t **pcln); +ngx_int_t ngx_http_lua_init_vm(lua_State **new_vm, lua_State *parent_vm, + ngx_cycle_t *cycle, ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, + ngx_log_t *log, ngx_pool_cleanup_t **pcln); lua_State *ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *l, int *ref); @@ -289,7 +289,8 @@ ngx_http_lua_init_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) static ngx_inline ngx_http_lua_ctx_t * ngx_http_lua_create_ctx(ngx_http_request_t *r) { - lua_State *L; + ngx_int_t rc; + lua_State *L = NULL; ngx_http_lua_ctx_t *ctx; ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -311,14 +312,33 @@ ngx_http_lua_create_ctx(ngx_http_request_t *r) dd("lmcf: %p", lmcf); #endif - L = ngx_http_lua_init_vm(lmcf->lua, lmcf->cycle, r->pool, lmcf, - r->connection->log, &cln); - if (L == NULL) { - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "failed to initialize Lua VM"); + rc = ngx_http_lua_init_vm(&L, lmcf->lua, lmcf->cycle, r->pool, lmcf, + r->connection->log, &cln); + if (rc != NGX_OK) { + if (rc == NGX_DECLINED) { + ngx_http_lua_assert(L != NULL); + + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "failed to load the 'resty.core' module " + "(https://github.com/openresty/lua-resty" + "-core); ensure you are using an OpenResty " + "release from https://openresty.org/en/" + "download.html (reason: %s)", + lua_tostring(L, -1)); + + } else { + /* rc == NGX_ERROR */ + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "failed to initialize Lua VM"); + } + return NULL; } + /* rc == NGX_OK */ + + ngx_http_lua_assert(L != NULL); + if (lmcf->init_handler) { if (lmcf->init_handler(r->connection->log, lmcf, L) != NGX_OK) { /* an error happened */ diff --git a/t/161-load-resty-core.t b/t/161-load-resty-core.t index 66bc4258b8..444e3c8ee0 100644 --- a/t/161-load-resty-core.t +++ b/t/161-load-resty-core.t @@ -23,7 +23,7 @@ run_tests(); __DATA__ -=== TEST 1: lua_load_resty_core is enabled by default +=== TEST 1: resty.core is automatically loaded in the Lua VM --- config location = /t { content_by_lua_block { @@ -38,25 +38,9 @@ resty.core loaded: true -=== TEST 2: lua_load_resty_core can be disabled +=== TEST 2: resty.core is automatically loaded in the Lua VM with 'lua_code_cache off' --- http_config - lua_load_resty_core off; ---- config - location = /t { - content_by_lua_block { - local loaded_resty_core = package.loaded["resty.core"] - - ngx.say("resty.core loaded: ", loaded_resty_core ~= nil) - } - } ---- response_body -resty.core loaded: false - - - -=== TEST 3: lua_load_resty_core is effective when using lua_shared_dict ---- http_config - lua_shared_dict dogs 128k; + lua_code_cache off; --- config location = /t { content_by_lua_block { @@ -71,7 +55,7 @@ resty.core loaded: true -=== TEST 4: lua_load_resty_core honors the lua_package_path directive +=== TEST 3: resty.core loading honors the lua_package_path directive --- http_config eval "lua_package_path '$::HtmlDir/?.lua;;';" --- config @@ -95,3 +79,69 @@ return { ngx.say("loaded from html dir") end } + + + +=== TEST 4: resty.core not loading aborts the initialization +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;';" +--- config + location = /t { + return 200; + } +--- must_die +--- error_log eval +qr/\[alert\] .*? failed to load the 'resty\.core' module .*? \(reason: module 'resty\.core' not found:/ + + + +=== TEST 5: resty.core not loading produces an error with 'lua_code_cache off' +--- http_config + lua_code_cache off; + + init_by_lua_block { + package.path = "" + } +--- config + location = /t { + content_by_lua_block { + ngx.say("ok") + } + } +--- error_code: 500 +--- error_log eval +qr/\[error\] .*? failed to load the 'resty\.core' module .*? \(reason: module 'resty\.core' not found:/ +--- no_error_log eval +qr/\[alert\] .*? failed to load the 'resty\.core' module/ + + + +=== TEST 6: lua_load_resty_core logs a deprecation warning when specified (on) +--- http_config + lua_load_resty_core on; +--- config + location = /t { + return 200; + } +--- grep_error_log eval: qr/\[warn\] .*? lua_load_resty_core is deprecated.*/ +--- grep_error_log_out eval +[ +qr/\[warn\] .*? lua_load_resty_core is deprecated \(the lua-resty-core library is required since ngx_lua v0\.10\.16\) in .*?nginx\.conf:\d+/, +"" +] + + + +=== TEST 7: lua_load_resty_core logs a deprecation warning when specified (off) +--- http_config + lua_load_resty_core off; +--- config + location = /t { + return 200; + } +--- grep_error_log eval: qr/\[warn\] .*? lua_load_resty_core is deprecated.*/ +--- grep_error_log_out eval +[ +qr/\[warn\] .*? lua_load_resty_core is deprecated \(the lua-resty-core library is required since ngx_lua v0\.10\.16\) in .*?nginx\.conf:\d+/, +"" +] From 947fa0088b7a0387f43eb016436a2e4462eb5746 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 1 Aug 2019 19:27:35 -0700 Subject: [PATCH 349/848] change: retired the old CFunction API when newer FFI implementations are available. --- config | 6 - src/ngx_http_lua_args.c | 67 - src/ngx_http_lua_balancer.c | 5 +- src/ngx_http_lua_bodyfilterby.c | 3 - src/ngx_http_lua_common.h | 2 - src/ngx_http_lua_control.c | 114 +- src/ngx_http_lua_ctx.c | 67 - src/ngx_http_lua_ctx.h | 2 - src/ngx_http_lua_headerfilterby.c | 3 - src/ngx_http_lua_headers.c | 381 +---- src/ngx_http_lua_log.c | 2 - src/ngx_http_lua_logby.c | 3 - src/ngx_http_lua_misc.c | 161 -- src/ngx_http_lua_misc.h | 2 - src/ngx_http_lua_module.c | 9 +- src/ngx_http_lua_ndk.c | 118 +- src/ngx_http_lua_phase.c | 99 +- src/ngx_http_lua_phase.h | 13 - src/ngx_http_lua_pipe.h | 3 +- src/ngx_http_lua_regex.c | 2102 +----------------------- src/ngx_http_lua_regex.h | 24 - src/ngx_http_lua_req_method.c | 134 -- src/ngx_http_lua_req_method.h | 19 - src/ngx_http_lua_semaphore.c | 5 - src/ngx_http_lua_semaphore.h | 2 - src/ngx_http_lua_setby.c | 3 - src/ngx_http_lua_shdict.c | 1171 ++----------- src/ngx_http_lua_ssl_certby.c | 5 - src/ngx_http_lua_ssl_ocsp.c | 4 - src/ngx_http_lua_ssl_session_fetchby.c | 4 - src/ngx_http_lua_ssl_session_storeby.c | 4 - src/ngx_http_lua_string.c | 290 +--- src/ngx_http_lua_time.c | 238 +-- src/ngx_http_lua_time.h | 21 - src/ngx_http_lua_util.c | 109 +- src/ngx_http_lua_util.h | 6 +- src/ngx_http_lua_variable.c | 274 --- src/ngx_http_lua_variable.h | 20 - src/ngx_http_lua_worker.c | 83 +- src/ngx_http_lua_worker.h | 17 - t/001-set.t | 4 +- t/062-count.t | 14 +- t/158-global-var.t | 44 +- 43 files changed, 325 insertions(+), 5332 deletions(-) delete mode 100644 src/ngx_http_lua_phase.h delete mode 100644 src/ngx_http_lua_regex.h delete mode 100644 src/ngx_http_lua_req_method.h delete mode 100644 src/ngx_http_lua_time.h delete mode 100644 src/ngx_http_lua_variable.h delete mode 100644 src/ngx_http_lua_worker.h diff --git a/config b/config index bf8bf5cb8a..e40a76300c 100644 --- a/config +++ b/config @@ -278,17 +278,14 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_subrequest.h \ $ngx_addon_dir/src/ngx_http_lua_ndk.h \ $ngx_addon_dir/src/ngx_http_lua_control.h \ - $ngx_addon_dir/src/ngx_http_lua_time.h \ $ngx_addon_dir/src/ngx_http_lua_string.h \ $ngx_addon_dir/src/ngx_http_lua_misc.h \ - $ngx_addon_dir/src/ngx_http_lua_variable.h \ $ngx_addon_dir/src/ngx_http_lua_output.h \ $ngx_addon_dir/src/ngx_http_lua_headers.h \ $ngx_addon_dir/src/ngx_http_lua_uri.h \ $ngx_addon_dir/src/ngx_http_lua_req_body.h \ $ngx_addon_dir/src/ngx_http_lua_args.h \ $ngx_addon_dir/src/ngx_http_lua_ctx.h \ - $ngx_addon_dir/src/ngx_http_lua_regex.h \ $ngx_addon_dir/src/ngx_http_lua_common.h \ $ngx_addon_dir/src/ngx_http_lua_directive.h \ $ngx_addon_dir/src/ngx_http_lua_headers_out.h \ @@ -316,13 +313,10 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_initby.h \ $ngx_addon_dir/src/ngx_http_lua_initworkerby.h \ $ngx_addon_dir/src/ngx_http_lua_socket_udp.h \ - $ngx_addon_dir/src/ngx_http_lua_req_method.h \ - $ngx_addon_dir/src/ngx_http_lua_phase.h \ $ngx_addon_dir/src/ngx_http_lua_probe.h \ $ngx_addon_dir/src/ngx_http_lua_uthread.h \ $ngx_addon_dir/src/ngx_http_lua_timer.h \ $ngx_addon_dir/src/ngx_http_lua_config.h \ - $ngx_addon_dir/src/ngx_http_lua_worker.h \ $ngx_addon_dir/src/ngx_http_lua_ssl_certby.h \ $ngx_addon_dir/src/ngx_http_lua_lex.h \ $ngx_addon_dir/src/ngx_http_lua_balancer.h \ diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index 2712bcd7c2..6da3759212 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -16,7 +16,6 @@ static int ngx_http_lua_ngx_req_set_uri_args(lua_State *L); -static int ngx_http_lua_ngx_req_get_uri_args(lua_State *L); static int ngx_http_lua_ngx_req_get_post_args(lua_State *L); @@ -80,64 +79,6 @@ ngx_http_lua_ngx_req_set_uri_args(lua_State *L) } -static int -ngx_http_lua_ngx_req_get_uri_args(lua_State *L) -{ - ngx_http_request_t *r; - u_char *buf; - u_char *last; - int retval; - int n; - int max; - - n = lua_gettop(L); - - if (n != 0 && n != 1) { - return luaL_error(L, "expecting 0 or 1 arguments but seen %d", n); - } - - if (n == 1) { - max = luaL_checkinteger(L, 1); - lua_pop(L, 1); - - } else { - max = NGX_HTTP_LUA_MAX_ARGS; - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ngx_http_lua_check_fake_request(L, r); - - if (r->args.len == 0) { - lua_createtable(L, 0, 0); - return 1; - } - - /* we copy r->args over to buf to simplify - * unescaping query arg keys and values */ - - buf = ngx_palloc(r->pool, r->args.len); - if (buf == NULL) { - return luaL_error(L, "no memory"); - } - - lua_createtable(L, 0, 4); - - ngx_memcpy(buf, r->args.data, r->args.len); - - last = buf + r->args.len; - - retval = ngx_http_lua_parse_args(L, buf, last, max); - - ngx_pfree(r->pool, buf); - - return retval; -} - - static int ngx_http_lua_ngx_req_get_post_args(lua_State *L) { @@ -368,18 +309,11 @@ ngx_http_lua_inject_req_args_api(lua_State *L) lua_pushcfunction(L, ngx_http_lua_ngx_req_set_uri_args); lua_setfield(L, -2, "set_uri_args"); - lua_pushcfunction(L, ngx_http_lua_ngx_req_get_uri_args); - lua_setfield(L, -2, "get_uri_args"); - - lua_pushcfunction(L, ngx_http_lua_ngx_req_get_uri_args); - lua_setfield(L, -2, "get_query_args"); /* deprecated */ - lua_pushcfunction(L, ngx_http_lua_ngx_req_get_post_args); lua_setfield(L, -2, "get_post_args"); } -#ifndef NGX_LUA_NO_FFI_API size_t ngx_http_lua_ffi_req_get_querystring_len(ngx_http_request_t *r) { @@ -549,7 +483,6 @@ ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, return i; } -#endif /* NGX_LUA_NO_FFI_API */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 0ee709a6b0..f2db02d513 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -457,8 +457,6 @@ ngx_http_lua_balancer_save_session(ngx_peer_connection_t *pc, void *data) #endif -#ifndef NGX_LUA_NO_FFI_API - int ngx_http_lua_ffi_balancer_set_current_peer(ngx_http_request_t *r, const u_char *addr, size_t addr_len, int port, char **err) @@ -754,4 +752,5 @@ ngx_http_lua_ffi_balancer_get_last_failure(ngx_http_request_t *r, return bp->last_peer_state; } -#endif /* NGX_LUA_NO_FFI_API */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index fbb3eb6077..9fea183951 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -15,12 +15,9 @@ #include "ngx_http_lua_exception.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_pcrefix.h" -#include "ngx_http_lua_time.h" #include "ngx_http_lua_log.h" -#include "ngx_http_lua_regex.h" #include "ngx_http_lua_cache.h" #include "ngx_http_lua_headers.h" -#include "ngx_http_lua_variable.h" #include "ngx_http_lua_string.h" #include "ngx_http_lua_misc.h" #include "ngx_http_lua_consts.h" diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 82907a479e..ad5c5f8e74 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -140,10 +140,8 @@ typedef struct { #define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x1000 -#ifndef NGX_LUA_NO_FFI_API #define NGX_HTTP_LUA_FFI_NO_REQ_CTX -100 #define NGX_HTTP_LUA_FFI_BAD_CONTEXT -101 -#endif #if (NGX_PTR_SIZE >= 8 && !defined(_WIN64)) diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 9c5230156c..5ace17638f 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -18,7 +18,6 @@ static int ngx_http_lua_ngx_exec(lua_State *L); static int ngx_http_lua_ngx_redirect(lua_State *L); -static int ngx_http_lua_ngx_exit(lua_State *L); static int ngx_http_lua_on_abort(lua_State *L); @@ -35,14 +34,6 @@ ngx_http_lua_inject_control_api(ngx_log_t *log, lua_State *L) lua_pushcfunction(L, ngx_http_lua_ngx_exec); lua_setfield(L, -2, "exec"); - lua_pushcfunction(L, ngx_http_lua_ngx_exit); - lua_setfield(L, -2, "throw_error"); /* deprecated */ - - /* ngx.exit */ - - lua_pushcfunction(L, ngx_http_lua_ngx_exit); - lua_setfield(L, -2, "exit"); - /* ngx.on_abort */ lua_pushcfunction(L, ngx_http_lua_on_abort); @@ -297,108 +288,6 @@ ngx_http_lua_ngx_redirect(lua_State *L) } -static int -ngx_http_lua_ngx_exit(lua_State *L) -{ - ngx_int_t rc; - ngx_http_request_t *r; - ngx_http_lua_ctx_t *ctx; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "no request ctx found"); - } - - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_HEADER_FILTER - | NGX_HTTP_LUA_CONTEXT_BALANCER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); - - rc = (ngx_int_t) luaL_checkinteger(L, 1); - - if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH)) - { - -#if (NGX_HTTP_SSL) - - ctx->exit_code = rc; - ctx->exited = 1; - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua exit with code %i", rc); - - if (ctx->context == NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE) { - return 0; - } - - return lua_yield(L, 0); - -#else - - return luaL_error(L, "no SSL support"); - -#endif - } - - if (ctx->no_abort - && rc != NGX_ERROR - && rc != NGX_HTTP_CLOSE - && rc != NGX_HTTP_REQUEST_TIME_OUT - && rc != NGX_HTTP_CLIENT_CLOSED_REQUEST) - { - return luaL_error(L, "attempt to abort with pending subrequests"); - } - - if ((r->header_sent || ctx->header_sent) - && rc >= NGX_HTTP_SPECIAL_RESPONSE - && rc != NGX_HTTP_REQUEST_TIME_OUT - && rc != NGX_HTTP_CLIENT_CLOSED_REQUEST - && rc != NGX_HTTP_CLOSE) - { - if (rc != (ngx_int_t) r->headers_out.status) { - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "attempt to " - "set status %i via ngx.exit after sending out the " - "response status %ui", rc, r->headers_out.status); - } - - rc = NGX_HTTP_OK; - } - - dd("setting exit code: %d", (int) rc); - - ctx->exit_code = rc; - ctx->exited = 1; - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua exit with code %i", ctx->exit_code); - - if (ctx->context & (NGX_HTTP_LUA_CONTEXT_HEADER_FILTER - | NGX_HTTP_LUA_CONTEXT_BALANCER)) - { - return 0; - } - - dd("calling yield"); - return lua_yield(L, 0); -} - - static int ngx_http_lua_on_abort(lua_State *L) { @@ -457,7 +346,6 @@ ngx_http_lua_on_abort(lua_State *L) } -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, size_t *errlen) @@ -553,6 +441,6 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, return NGX_OK; } -#endif /* NGX_LUA_NO_FFI_API */ + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_ctx.c b/src/ngx_http_lua_ctx.c index a14bb4d495..74571378c0 100644 --- a/src/ngx_http_lua_ctx.c +++ b/src/ngx_http_lua_ctx.c @@ -25,71 +25,6 @@ static ngx_int_t ngx_http_lua_ngx_ctx_add_cleanup(ngx_http_request_t *r, static void ngx_http_lua_ngx_ctx_cleanup(void *data); -int -ngx_http_lua_ngx_get_ctx(lua_State *L) -{ - ngx_http_request_t *r; - ngx_http_lua_ctx_t *ctx; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request found"); - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "no request ctx found"); - } - - if (ctx->ctx_ref == LUA_NOREF) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua create ngx.ctx table for the current request"); - - lua_pushliteral(L, ngx_http_lua_ctx_tables_key); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_createtable(L, 0 /* narr */, 4 /* nrec */); - lua_pushvalue(L, -1); - ctx->ctx_ref = luaL_ref(L, -3); - - if (ngx_http_lua_ngx_ctx_add_cleanup(r, ctx->ctx_ref) != NGX_OK) { - return luaL_error(L, "no memory"); - } - - return 1; - } - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua fetching existing ngx.ctx table for the current " - "request"); - - lua_pushliteral(L, ngx_http_lua_ctx_tables_key); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_rawgeti(L, -1, ctx->ctx_ref); - - return 1; -} - - -int -ngx_http_lua_ngx_set_ctx(lua_State *L) -{ - ngx_http_request_t *r; - ngx_http_lua_ctx_t *ctx; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request found"); - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "no request ctx found"); - } - - return ngx_http_lua_ngx_set_ctx_helper(L, r, ctx, 3); -} - - int ngx_http_lua_ngx_set_ctx_helper(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, int index) @@ -130,7 +65,6 @@ ngx_http_lua_ngx_set_ctx_helper(lua_State *L, ngx_http_request_t *r, } -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r) { @@ -163,7 +97,6 @@ ngx_http_lua_ffi_set_ctx_ref(ngx_http_request_t *r, int ref) return NGX_OK; } -#endif /* NGX_LUA_NO_FFI_API */ static ngx_int_t diff --git a/src/ngx_http_lua_ctx.h b/src/ngx_http_lua_ctx.h index f73f73e5c1..7596238c07 100644 --- a/src/ngx_http_lua_ctx.h +++ b/src/ngx_http_lua_ctx.h @@ -12,8 +12,6 @@ #include "ngx_http_lua_common.h" -int ngx_http_lua_ngx_get_ctx(lua_State *L); -int ngx_http_lua_ngx_set_ctx(lua_State *L); int ngx_http_lua_ngx_set_ctx_helper(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, int index); diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index c9e770133e..4741c72267 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -13,12 +13,9 @@ #include "ngx_http_lua_exception.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_pcrefix.h" -#include "ngx_http_lua_time.h" #include "ngx_http_lua_log.h" -#include "ngx_http_lua_regex.h" #include "ngx_http_lua_cache.h" #include "ngx_http_lua_headers.h" -#include "ngx_http_lua_variable.h" #include "ngx_http_lua_string.h" #include "ngx_http_lua_misc.h" #include "ngx_http_lua_consts.h" diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index e3f48bcbef..a50a1ed333 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -20,17 +20,39 @@ static int ngx_http_lua_ngx_req_http_version(lua_State *L); static int ngx_http_lua_ngx_req_raw_header(lua_State *L); static int ngx_http_lua_ngx_req_header_set_helper(lua_State *L); -static int ngx_http_lua_ngx_header_get(lua_State *L); -static int ngx_http_lua_ngx_header_set(lua_State *L); -static int ngx_http_lua_ngx_req_get_headers(lua_State *L); -static int ngx_http_lua_ngx_req_header_clear(lua_State *L); -static int ngx_http_lua_ngx_req_header_set(lua_State *L); static int ngx_http_lua_ngx_resp_get_headers(lua_State *L); +static int ngx_http_lua_ngx_req_header_set(lua_State *L); #if nginx_version >= 1011011 void ngx_http_lua_ngx_raw_header_cleanup(void *data); #endif +void +ngx_http_lua_inject_resp_header_api(lua_State *L) +{ + lua_createtable(L, 0, 1); /* .resp */ + + lua_pushcfunction(L, ngx_http_lua_ngx_resp_get_headers); + lua_setfield(L, -2, "get_headers"); + + lua_setfield(L, -2, "resp"); +} + + +void +ngx_http_lua_inject_req_header_api(lua_State *L) +{ + lua_pushcfunction(L, ngx_http_lua_ngx_req_http_version); + lua_setfield(L, -2, "http_version"); + + lua_pushcfunction(L, ngx_http_lua_ngx_req_raw_header); + lua_setfield(L, -2, "raw_header"); + + lua_pushcfunction(L, ngx_http_lua_ngx_req_header_set); + lua_setfield(L, -2, "set_header"); +} + + static int ngx_http_lua_ngx_req_http_version(lua_State *L) { @@ -386,118 +408,6 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) } -static int -ngx_http_lua_ngx_req_get_headers(lua_State *L) -{ - ngx_list_part_t *part; - ngx_table_elt_t *header; - ngx_http_request_t *r; - ngx_uint_t i; - int n; - int max; - int raw = 0; - int count = 0; - int truncated = 0; - - n = lua_gettop(L); - - if (n >= 1) { - if (lua_isnil(L, 1)) { - max = NGX_HTTP_LUA_MAX_HEADERS; - - } else { - max = luaL_checkinteger(L, 1); - } - - if (n >= 2) { - raw = lua_toboolean(L, 2); - } - - } else { - max = NGX_HTTP_LUA_MAX_HEADERS; - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ngx_http_lua_check_fake_request(L, r); - - part = &r->headers_in.headers.part; - count = part->nelts; - while (part->next != NULL) { - part = part->next; - count += part->nelts; - } - - if (max > 0 && count > max) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua exceeding request header limit %d > %d", count, - max); - count = max; - truncated = 1; - } - - lua_createtable(L, 0, count); - - if (!raw) { - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - headers_metatable_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_setmetatable(L, -2); - } - - part = &r->headers_in.headers.part; - header = part->elts; - - for (i = 0; /* void */; i++) { - - dd("stack top: %d", lua_gettop(L)); - - if (i >= part->nelts) { - if (part->next == NULL) { - break; - } - - part = part->next; - header = part->elts; - i = 0; - } - - if (raw) { - lua_pushlstring(L, (char *) header[i].key.data, header[i].key.len); - - } else { - lua_pushlstring(L, (char *) header[i].lowcase_key, - header[i].key.len); - } - - /* stack: table key */ - - lua_pushlstring(L, (char *) header[i].value.data, - header[i].value.len); /* stack: table key value */ - - ngx_http_lua_set_multi_value_table(L, -3); - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua request header: \"%V: %V\"", - &header[i].key, &header[i].value); - - if (--count <= 0) { - break; - } - } /* for */ - - if (truncated) { - lua_pushliteral(L, "truncated"); - return 2; - } - - return 1; -} - - static int ngx_http_lua_ngx_resp_get_headers(lua_State *L) { @@ -674,198 +584,6 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) } -static int -ngx_http_lua_ngx_header_get(lua_State *L) -{ - ngx_http_request_t *r; - u_char *p, c; - ngx_str_t key; - ngx_uint_t i; - size_t len; - ngx_http_lua_loc_conf_t *llcf; - ngx_http_lua_ctx_t *ctx; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "no ctx found"); - } - - ngx_http_lua_check_fake_request(L, r); - - /* we skip the first argument that is the table */ - p = (u_char *) luaL_checklstring(L, 2, &len); - - dd("key: %.*s, len %d", (int) len, p, (int) len); - - llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); - if (llcf->transform_underscores_in_resp_headers - && memchr(p, '_', len) != NULL) - { - key.data = (u_char *) lua_newuserdata(L, len); - if (key.data == NULL) { - return luaL_error(L, "no memory"); - } - - /* replace "_" with "-" */ - for (i = 0; i < len; i++) { - c = p[i]; - if (c == '_') { - c = '-'; - } - key.data[i] = c; - } - - } else { - key.data = p; - } - - key.len = len; - - return ngx_http_lua_get_output_header(L, r, ctx, &key); -} - - -static int -ngx_http_lua_ngx_header_set(lua_State *L) -{ - ngx_http_request_t *r; - u_char *p; - ngx_str_t key; - ngx_str_t value; - ngx_uint_t i; - size_t len; - ngx_http_lua_ctx_t *ctx; - ngx_int_t rc; - ngx_uint_t n; - ngx_http_lua_loc_conf_t *llcf; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "no ctx"); - } - - ngx_http_lua_check_fake_request(L, r); - - if (r->header_sent || ctx->header_sent) { - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "attempt to " - "set ngx.header.HEADER after sending out " - "response headers"); - return 0; - } - - /* we skip the first argument that is the table */ - p = (u_char *) luaL_checklstring(L, 2, &len); - - dd("key: %.*s, len %d", (int) len, p, (int) len); - - key.data = ngx_palloc(r->pool, len + 1); - if (key.data == NULL) { - return luaL_error(L, "no memory"); - } - - ngx_memcpy(key.data, p, len); - key.data[len] = '\0'; - key.len = len; - - llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); - - if (llcf->transform_underscores_in_resp_headers) { - /* replace "_" with "-" */ - p = key.data; - for (i = 0; i < len; i++) { - if (p[i] == '_') { - p[i] = '-'; - } - } - } - - ctx->headers_set = 1; - - if (lua_type(L, 3) == LUA_TNIL) { - ngx_str_null(&value); - - } else if (lua_type(L, 3) == LUA_TTABLE) { - n = lua_objlen(L, 3); - if (n == 0) { - ngx_str_null(&value); - - } else { - for (i = 1; i <= n; i++) { - dd("header value table index %d", (int) i); - - lua_rawgeti(L, 3, i); - p = (u_char *) luaL_checklstring(L, -1, &len); - - value.data = ngx_palloc(r->pool, len); - if (value.data == NULL) { - return luaL_error(L, "no memory"); - } - - ngx_memcpy(value.data, p, len); - value.len = len; - - rc = ngx_http_lua_set_output_header(r, ctx, key, value, - i == 1 /* override */); - - if (rc == NGX_ERROR) { - return luaL_error(L, - "failed to set header %s (error: %d)", - key.data, (int) rc); - } - } - - return 0; - } - - } else { - p = (u_char *) luaL_checklstring(L, 3, &len); - value.data = ngx_palloc(r->pool, len); - if (value.data == NULL) { - return luaL_error(L, "no memory"); - } - - ngx_memcpy(value.data, p, len); - value.len = len; - } - - dd("key: %.*s, value: %.*s", - (int) key.len, key.data, (int) value.len, value.data); - - rc = ngx_http_lua_set_output_header(r, ctx, key, value, 1 /* override */); - - if (rc == NGX_ERROR) { - return luaL_error(L, "failed to set header %s (error: %d)", - key.data, (int) rc); - } - - return 0; -} - - -static int -ngx_http_lua_ngx_req_header_clear(lua_State *L) -{ - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one arguments, but seen %d", - lua_gettop(L)); - } - - lua_pushnil(L); - - return ngx_http_lua_ngx_req_header_set_helper(L); -} - - static int ngx_http_lua_ngx_req_header_set(lua_State *L) { @@ -998,49 +716,6 @@ ngx_http_lua_ngx_req_header_set_helper(lua_State *L) } -void -ngx_http_lua_inject_resp_header_api(lua_State *L) -{ - lua_newtable(L); /* .header */ - - lua_createtable(L, 0, 2); /* metatable for .header */ - lua_pushcfunction(L, ngx_http_lua_ngx_header_get); - lua_setfield(L, -2, "__index"); - lua_pushcfunction(L, ngx_http_lua_ngx_header_set); - lua_setfield(L, -2, "__newindex"); - lua_setmetatable(L, -2); - - lua_setfield(L, -2, "header"); - - lua_createtable(L, 0, 1); /* .resp */ - - lua_pushcfunction(L, ngx_http_lua_ngx_resp_get_headers); - lua_setfield(L, -2, "get_headers"); - - lua_setfield(L, -2, "resp"); -} - - -void -ngx_http_lua_inject_req_header_api(lua_State *L) -{ - lua_pushcfunction(L, ngx_http_lua_ngx_req_http_version); - lua_setfield(L, -2, "http_version"); - - lua_pushcfunction(L, ngx_http_lua_ngx_req_raw_header); - lua_setfield(L, -2, "raw_header"); - - lua_pushcfunction(L, ngx_http_lua_ngx_req_header_clear); - lua_setfield(L, -2, "clear_header"); - - lua_pushcfunction(L, ngx_http_lua_ngx_req_header_set); - lua_setfield(L, -2, "set_header"); - - lua_pushcfunction(L, ngx_http_lua_ngx_req_get_headers); - lua_setfield(L, -2, "get_headers"); -} - - void ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L) { @@ -1072,7 +747,6 @@ ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L) } -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_req_get_headers_count(ngx_http_request_t *r, int max, int *truncated) @@ -1462,7 +1136,6 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, return found; } -#endif /* NGX_LUA_NO_FFI_API */ #if nginx_version >= 1011011 diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index 3938da5507..5411831a18 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -340,7 +340,6 @@ ngx_http_lua_capture_log_handler(ngx_log_t *log, #endif -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_errlog_set_filter_level(int level, u_char *err, size_t *errlen) { @@ -451,6 +450,5 @@ ngx_http_lua_ffi_raw_log(ngx_http_request_t *r, int level, u_char *s, return NGX_OK; } -#endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index 3da9b428b9..422698af7b 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -15,12 +15,9 @@ #include "ngx_http_lua_exception.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_pcrefix.h" -#include "ngx_http_lua_time.h" #include "ngx_http_lua_log.h" -#include "ngx_http_lua_regex.h" #include "ngx_http_lua_cache.h" #include "ngx_http_lua_headers.h" -#include "ngx_http_lua_variable.h" #include "ngx_http_lua_string.h" #include "ngx_http_lua_misc.h" #include "ngx_http_lua_consts.h" diff --git a/src/ngx_http_lua_misc.c b/src/ngx_http_lua_misc.c index 145ca9b5e3..2300aa3e4c 100644 --- a/src/ngx_http_lua_misc.c +++ b/src/ngx_http_lua_misc.c @@ -12,28 +12,12 @@ #include "ngx_http_lua_misc.h" -#include "ngx_http_lua_ctx.h" #include "ngx_http_lua_util.h" -static int ngx_http_lua_ngx_get(lua_State *L); -static int ngx_http_lua_ngx_set(lua_State *L); static int ngx_http_lua_ngx_req_is_internal(lua_State *L); -void -ngx_http_lua_inject_misc_api(lua_State *L) -{ - /* ngx. getter and setter */ - lua_createtable(L, 0, 2); /* metatable for .ngx */ - lua_pushcfunction(L, ngx_http_lua_ngx_get); - lua_setfield(L, -2, "__index"); - lua_pushcfunction(L, ngx_http_lua_ngx_set); - lua_setfield(L, -2, "__newindex"); - lua_setmetatable(L, -2); -} - - void ngx_http_lua_inject_req_misc_api(lua_State *L) { @@ -57,150 +41,6 @@ ngx_http_lua_ngx_req_is_internal(lua_State *L) } -static int -ngx_http_lua_ngx_get(lua_State *L) -{ - int status; - ngx_http_request_t *r; - u_char *p; - size_t len; - ngx_http_lua_ctx_t *ctx; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - lua_pushnil(L); - return 1; - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - lua_pushnil(L); - return 1; - } - - p = (u_char *) luaL_checklstring(L, -1, &len); - - dd("ngx get %s", p); - - if (len == sizeof("status") - 1 - && ngx_strncmp(p, "status", sizeof("status") - 1) == 0) - { - ngx_http_lua_check_fake_request(L, r); - - if (r->err_status) { - status = r->err_status; - - } else if (r->headers_out.status) { - status = r->headers_out.status; - - } else if (r->http_version == NGX_HTTP_VERSION_9) { - status = 9; - - } else { - status = 0; - } - - lua_pushinteger(L, status); - return 1; - } - - if (len == sizeof("ctx") - 1 - && ngx_strncmp(p, "ctx", sizeof("ctx") - 1) == 0) - { - return ngx_http_lua_ngx_get_ctx(L); - } - - if (len == sizeof("is_subrequest") - 1 - && ngx_strncmp(p, "is_subrequest", sizeof("is_subrequest") - 1) == 0) - { - lua_pushboolean(L, r != r->main); - return 1; - } - - if (len == sizeof("headers_sent") - 1 - && ngx_strncmp(p, "headers_sent", sizeof("headers_sent") - 1) == 0) - { - ngx_http_lua_check_fake_request(L, r); - - dd("headers sent: %d", r->header_sent || ctx->header_sent); - - lua_pushboolean(L, r->header_sent || ctx->header_sent); - return 1; - } - - dd("key %s not matched", p); - - lua_pushnil(L); - return 1; -} - - -static int -ngx_http_lua_ngx_set(lua_State *L) -{ - ngx_http_request_t *r; - u_char *p; - size_t len; - - /* we skip the first argument that is the table */ - p = (u_char *) luaL_checklstring(L, 2, &len); - - if (len == sizeof("status") - 1 - && ngx_strncmp(p, "status", sizeof("status") - 1) == 0) - { - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - if (r->header_sent) { - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "attempt to set ngx.status after sending out " - "response headers"); - return 0; - } - - if (r->err_status) { - r->err_status = 0; - } - - ngx_http_lua_check_fake_request(L, r); - - /* get the value */ - r->headers_out.status = (ngx_uint_t) luaL_checknumber(L, 3); - - if (r->headers_out.status == 101) { - /* - * XXX work-around a bug in the Nginx core that 101 does - * not have a default status line - */ - - ngx_str_set(&r->headers_out.status_line, "101 Switching Protocols"); - - } else { - r->headers_out.status_line.len = 0; - } - - return 0; - } - - if (len == sizeof("ctx") - 1 - && ngx_strncmp(p, "ctx", sizeof("ctx") - 1) == 0) - { - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - return ngx_http_lua_ngx_set_ctx(L); - } - - lua_rawset(L, -3); - return 0; -} - - -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_get_resp_status(ngx_http_request_t *r) { @@ -313,7 +153,6 @@ ngx_http_lua_ffi_get_conf_env(u_char *name, u_char **env_buf, size_t *name_len) return NGX_DECLINED; } -#endif /* NGX_LUA_NO_FFI_API */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_misc.h b/src/ngx_http_lua_misc.h index c26d869b30..061ccf279a 100644 --- a/src/ngx_http_lua_misc.h +++ b/src/ngx_http_lua_misc.h @@ -12,8 +12,6 @@ #include "ngx_http_lua_common.h" -void ngx_http_lua_inject_misc_api(lua_State *L); - void ngx_http_lua_inject_req_misc_api(lua_State *L); diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 4a5d8cf1ef..efa8dc12bc 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -650,7 +650,7 @@ ngx_http_lua_init(ngx_conf_t *cf) volatile ngx_cycle_t *saved_cycle; ngx_http_core_main_conf_t *cmcf; ngx_http_lua_main_conf_t *lmcf; -#if !defined(NGX_LUA_NO_FFI_API) || nginx_version >= 1011011 +#if nginx_version >= 1011011 ngx_pool_cleanup_t *cln; #endif ngx_str_t name = ngx_string("host"); @@ -741,7 +741,6 @@ ngx_http_lua_init(ngx_conf_t *cf) } } -#ifndef NGX_LUA_NO_FFI_API /* add the cleanup of semaphores after the lua_close */ cln = ngx_pool_cleanup_add(cf->pool, 0); if (cln == NULL) { @@ -755,8 +754,6 @@ ngx_http_lua_init(ngx_conf_t *cf) ngx_http_lua_pipe_init(); #endif -#endif - #if nginx_version >= 1011011 cln = ngx_pool_cleanup_add(cf->pool, 0); if (cln == NULL) { @@ -867,9 +864,7 @@ ngx_http_lua_lowat_check(ngx_conf_t *cf, void *post, void *data) static void * ngx_http_lua_create_main_conf(ngx_conf_t *cf) { -#ifndef NGX_LUA_NO_FFI_API ngx_int_t rc; -#endif ngx_http_lua_main_conf_t *lmcf; @@ -918,14 +913,12 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) lmcf->malloc_trim_cycle = NGX_CONF_UNSET_UINT; #endif -#ifndef NGX_LUA_NO_FFI_API rc = ngx_http_lua_sema_mm_init(cf, lmcf); if (rc != NGX_OK) { return NULL; } dd("nginx Lua module main config structure initialized!"); -#endif return lmcf; } diff --git a/src/ngx_http_lua_ndk.c b/src/ngx_http_lua_ndk.c index 6344183723..5daeeeb9cb 100644 --- a/src/ngx_http_lua_ndk.c +++ b/src/ngx_http_lua_ndk.c @@ -19,93 +19,20 @@ static ndk_set_var_value_pt ngx_http_lookup_ndk_set_var_directive(u_char *name, size_t name_len); -static int ngx_http_lua_ndk_set_var_get(lua_State *L); -static int ngx_http_lua_ndk_set_var_set(lua_State *L); -static int ngx_http_lua_run_set_var_directive(lua_State *L); -int -ngx_http_lua_ndk_set_var_get(lua_State *L) -{ - ndk_set_var_value_pt func; - size_t len; - u_char *p; - - p = (u_char *) luaL_checklstring(L, 2, &len); - - dd("ndk.set_var metatable __index: %s", p); - - func = ngx_http_lookup_ndk_set_var_directive(p, len); - - if (func == NULL) { - return luaL_error(L, "ndk.set_var: directive \"%s\" not found " - "or does not use ndk_set_var_value", p); - } - - lua_pushvalue(L, -1); /* table key key */ - lua_pushvalue(L, -1); /* table key key key */ - lua_pushlightuserdata(L, (void *) func); /* table key key key func */ - lua_pushcclosure(L, ngx_http_lua_run_set_var_directive, 2); - /* table key key closure */ - lua_rawset(L, 1); /* table key */ - lua_rawget(L, 1); /* table closure */ - - return 1; -} - - -int -ngx_http_lua_ndk_set_var_set(lua_State *L) -{ - return luaL_error(L, "Not allowed"); -} - - -int -ngx_http_lua_run_set_var_directive(lua_State *L) +void +ngx_http_lua_inject_ndk_api(lua_State *L) { - ngx_int_t rc; - ndk_set_var_value_pt func; - ngx_str_t res; - ngx_http_variable_value_t arg; - u_char *p; - size_t len; - ngx_http_request_t *r; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - -#if 1 - ngx_memzero(&arg, sizeof(ngx_http_variable_value_t)); - - arg.valid = 1; -#endif - - arg.data = (u_char *) luaL_checklstring(L, 1, &len); - arg.len = len; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - p = (u_char *) luaL_checklstring(L, lua_upvalueindex(1), &len); - - dd("calling set_var func for %s", p); - - func = (ndk_set_var_value_pt) lua_touserdata(L, lua_upvalueindex(2)); - - rc = func(r, &res, &arg); - - if (rc != NGX_OK) { - return luaL_error(L, "calling directive %s failed with code %d", - p, (int) rc); - } + lua_createtable(L, 0, 1 /* nrec */); /* ndk.* */ - lua_pushlstring(L, (char *) res.data, res.len); + lua_getglobal(L, "package"); /* ndk package */ + lua_getfield(L, -1, "loaded"); /* ndk package loaded */ + lua_pushvalue(L, -3); /* ndk package loaded ndk */ + lua_setfield(L, -2, "ndk"); /* ndk package loaded */ + lua_pop(L, 2); - return 1; + lua_setglobal(L, "ndk"); } @@ -160,32 +87,6 @@ ngx_http_lookup_ndk_set_var_directive(u_char *name, } -void -ngx_http_lua_inject_ndk_api(lua_State *L) -{ - lua_createtable(L, 0, 1 /* nrec */); /* ndk.* */ - - lua_newtable(L); /* .set_var */ - - lua_createtable(L, 0, 2 /* nrec */); /* metatable for .set_var */ - lua_pushcfunction(L, ngx_http_lua_ndk_set_var_get); - lua_setfield(L, -2, "__index"); - lua_pushcfunction(L, ngx_http_lua_ndk_set_var_set); - lua_setfield(L, -2, "__newindex"); - lua_setmetatable(L, -2); - - lua_setfield(L, -2, "set_var"); - - lua_getglobal(L, "package"); /* ndk package */ - lua_getfield(L, -1, "loaded"); /* ndk package loaded */ - lua_pushvalue(L, -3); /* ndk package loaded ndk */ - lua_setfield(L, -2, "ndk"); /* ndk package loaded */ - lua_pop(L, 2); - - lua_setglobal(L, "ndk"); -} - - int ngx_http_lua_ffi_ndk_lookup_directive(const u_char *var_data, size_t var_len, ndk_set_var_value_pt *func) @@ -229,4 +130,5 @@ ngx_http_lua_ffi_ndk_set_var_get(ngx_http_request_t *r, #endif /* defined(NDK) && NDK */ + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_phase.c b/src/ngx_http_lua_phase.c index 304c88ae13..85f1e7a312 100644 --- a/src/ngx_http_lua_phase.c +++ b/src/ngx_http_lua_phase.c @@ -10,105 +10,9 @@ #include "ddebug.h" -#include "ngx_http_lua_phase.h" -#include "ngx_http_lua_util.h" -#include "ngx_http_lua_ctx.h" +#include "ngx_http_lua_common.h" -static int ngx_http_lua_ngx_get_phase(lua_State *L); - - -static int -ngx_http_lua_ngx_get_phase(lua_State *L) -{ - ngx_http_request_t *r; - ngx_http_lua_ctx_t *ctx; - - r = ngx_http_lua_get_req(L); - - /* If we have no request object, assume we are called from the "init" - * phase. */ - - if (r == NULL) { - lua_pushliteral(L, "init"); - return 1; - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "no request ctx found"); - } - - switch (ctx->context) { - case NGX_HTTP_LUA_CONTEXT_INIT_WORKER: - lua_pushliteral(L, "init_worker"); - break; - - case NGX_HTTP_LUA_CONTEXT_SET: - lua_pushliteral(L, "set"); - break; - - case NGX_HTTP_LUA_CONTEXT_REWRITE: - lua_pushliteral(L, "rewrite"); - break; - - case NGX_HTTP_LUA_CONTEXT_ACCESS: - lua_pushliteral(L, "access"); - break; - - case NGX_HTTP_LUA_CONTEXT_CONTENT: - lua_pushliteral(L, "content"); - break; - - case NGX_HTTP_LUA_CONTEXT_LOG: - lua_pushliteral(L, "log"); - break; - - case NGX_HTTP_LUA_CONTEXT_HEADER_FILTER: - lua_pushliteral(L, "header_filter"); - break; - - case NGX_HTTP_LUA_CONTEXT_BODY_FILTER: - lua_pushliteral(L, "body_filter"); - break; - - case NGX_HTTP_LUA_CONTEXT_TIMER: - lua_pushliteral(L, "timer"); - break; - - case NGX_HTTP_LUA_CONTEXT_BALANCER: - lua_pushliteral(L, "balancer"); - break; - - case NGX_HTTP_LUA_CONTEXT_SSL_CERT: - lua_pushliteral(L, "ssl_cert"); - break; - - case NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE: - lua_pushliteral(L, "ssl_session_store"); - break; - - case NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH: - lua_pushliteral(L, "ssl_session_fetch"); - break; - - default: - return luaL_error(L, "unknown phase: %#x", (int) ctx->context); - } - - return 1; -} - - -void -ngx_http_lua_inject_phase_api(lua_State *L) -{ - lua_pushcfunction(L, ngx_http_lua_ngx_get_phase); - lua_setfield(L, -2, "get_phase"); -} - - -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_get_phase(ngx_http_request_t *r, char **err) { @@ -122,7 +26,6 @@ ngx_http_lua_ffi_get_phase(ngx_http_request_t *r, char **err) return ctx->context; } -#endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_phase.h b/src/ngx_http_lua_phase.h deleted file mode 100644 index dcb670f221..0000000000 --- a/src/ngx_http_lua_phase.h +++ /dev/null @@ -1,13 +0,0 @@ -#ifndef _NGX_HTTP_LUA_PHASE_H_INCLUDED_ -#define _NGX_HTTP_LUA_PHASE_H_INCLUDED_ - - -#include "ngx_http_lua_common.h" - - -void ngx_http_lua_inject_phase_api(lua_State *L); - - -#endif /* _NGX_HTTP_LUA_PHASE_H_INCLUDED_ */ - -/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_pipe.h b/src/ngx_http_lua_pipe.h index b5cea89320..ecb86c8806 100644 --- a/src/ngx_http_lua_pipe.h +++ b/src/ngx_http_lua_pipe.h @@ -80,8 +80,7 @@ typedef struct { } ngx_http_lua_pipe_signal_t; -#if !(NGX_WIN32) && !defined(NGX_LUA_NO_FFI_API) \ - && defined(HAVE_SOCKET_CLOEXEC_PATCH) +#if !(NGX_WIN32) && defined(HAVE_SOCKET_CLOEXEC_PATCH) #define HAVE_NGX_LUA_PIPE 1 diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 9593575d7c..471339cc99 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -12,7 +12,6 @@ #if (NGX_PCRE) -#include "ngx_http_lua_regex.h" #include "ngx_http_lua_pcrefix.h" #include "ngx_http_lua_script.h" #include "ngx_http_lua_util.h" @@ -25,10 +24,8 @@ #endif -#define NGX_LUA_RE_COMPILE_ONCE (1<<0) #define NGX_LUA_RE_MODE_DFA (1<<1) #define NGX_LUA_RE_MODE_JIT (1<<2) -#define NGX_LUA_RE_MODE_DUPNAMES (1<<3) #define NGX_LUA_RE_NO_UTF8_CHECK (1<<4) #define NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT (100) @@ -37,1973 +34,60 @@ typedef struct { -#ifndef NGX_LUA_NO_FFI_API ngx_pool_t *pool; u_char *name_table; int name_count; int name_entry_size; -#endif - - int ncaptures; - int *captures; - - pcre *regex; - pcre_extra *regex_sd; - - ngx_http_lua_complex_value_t *replace; - -#ifndef NGX_LUA_NO_FFI_API - /* only for (stap) debugging, and may be an invalid pointer */ - const u_char *pattern; -#endif -} ngx_http_lua_regex_t; - - -typedef struct { - ngx_str_t pattern; - ngx_pool_t *pool; - ngx_int_t options; - - pcre *regex; - int captures; - ngx_str_t err; -} ngx_http_lua_regex_compile_t; - - -typedef struct { - ngx_http_cleanup_pt *cleanup; - ngx_http_request_t *request; - pcre *regex; - pcre_extra *regex_sd; - int ncaptures; - int *captures; - int captures_len; - uint8_t flags; -} ngx_http_lua_regex_ctx_t; - - -static int ngx_http_lua_ngx_re_gmatch_iterator(lua_State *L); -static ngx_uint_t ngx_http_lua_ngx_re_parse_opts(lua_State *L, - ngx_http_lua_regex_compile_t *re, ngx_str_t *opts, int narg); -static int ngx_http_lua_ngx_re_sub_helper(lua_State *L, unsigned global); -static int ngx_http_lua_ngx_re_match_helper(lua_State *L, int wantcaps); -static int ngx_http_lua_ngx_re_find(lua_State *L); -static int ngx_http_lua_ngx_re_match(lua_State *L); -static int ngx_http_lua_ngx_re_gmatch(lua_State *L); -static int ngx_http_lua_ngx_re_sub(lua_State *L); -static int ngx_http_lua_ngx_re_gsub(lua_State *L); -static void ngx_http_lua_regex_free_study_data(ngx_pool_t *pool, - pcre_extra *sd); -static ngx_int_t ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc); -static void ngx_http_lua_ngx_re_gmatch_cleanup(void *data); -static int ngx_http_lua_ngx_re_gmatch_gc(lua_State *L); -static void ngx_http_lua_re_collect_named_captures(lua_State *L, - int res_tb_idx, u_char *name_table, int name_count, int name_entry_size, - unsigned flags, ngx_str_t *subj); - - -#define ngx_http_lua_regex_exec(re, e, s, start, captures, size, opts) \ - pcre_exec(re, e, (const char *) (s)->data, (s)->len, start, opts, \ - captures, size) - - -#define ngx_http_lua_regex_dfa_exec(re, e, s, start, captures, size, ws, \ - wscount, opts) \ - pcre_dfa_exec(re, e, (const char *) (s)->data, (s)->len, start, opts, \ - captures, size, ws, wscount) - - -static int -ngx_http_lua_ngx_re_match(lua_State *L) -{ - return ngx_http_lua_ngx_re_match_helper(L, 1 /* want captures */); -} - - -static int -ngx_http_lua_ngx_re_find(lua_State *L) -{ - return ngx_http_lua_ngx_re_match_helper(L, 0 /* want captures */); -} - - -static int -ngx_http_lua_ngx_re_match_helper(lua_State *L, int wantcaps) -{ - /* u_char *p; */ - int res_tb_idx = 0; - ngx_http_request_t *r; - ngx_str_t subj; - ngx_str_t pat; - ngx_str_t opts; - ngx_http_lua_regex_t *re; - const char *msg; - ngx_int_t rc; - ngx_uint_t n; - int i; - ngx_int_t pos = 0; - int nargs; - int *cap = NULL; - int ovecsize; - int has_ctx = 0; - ngx_uint_t flags; - ngx_pool_t *pool, *old_pool; - ngx_http_lua_main_conf_t *lmcf; - u_char errstr[NGX_MAX_CONF_ERRSTR + 1]; - pcre_extra *sd = NULL; - int name_entry_size = 0, name_count; - u_char *name_table = NULL; - int exec_opts; - int group_id = 0; - - ngx_http_lua_regex_compile_t re_comp; - - nargs = lua_gettop(L); - - if (nargs != 2 && nargs != 3 && nargs != 4 && nargs != 5) { - return luaL_error(L, "expecting 2, 3, 4 or 5 arguments, " - "but got %d", nargs); - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - subj.data = (u_char *) luaL_checklstring(L, 1, &subj.len); - pat.data = (u_char *) luaL_checklstring(L, 2, &pat.len); - - ngx_memzero(&re_comp, sizeof(ngx_http_lua_regex_compile_t)); - - if (nargs >= 3) { - opts.data = (u_char *) luaL_checklstring(L, 3, &opts.len); - - if (nargs >= 4) { - if (!lua_isnil(L, 4)) { - luaL_checktype(L, 4, LUA_TTABLE); - has_ctx = 1; - - lua_getfield(L, 4, "pos"); - if (lua_isnumber(L, -1)) { - pos = (ngx_int_t) lua_tointeger(L, -1); - if (pos <= 0) { - pos = 0; - - } else { - pos--; /* 1-based on the Lua land */ - } - - } else if (lua_isnil(L, -1)) { - pos = 0; - - } else { - msg = lua_pushfstring(L, "bad pos field type in the ctx " - "table argument: %s", - luaL_typename(L, -1)); - - return luaL_argerror(L, 4, msg); - } - - lua_pop(L, 1); - } - } - - } else { - opts.data = (u_char *) ""; - opts.len = 0; - } - - if (nargs == 5) { - if (wantcaps) { - luaL_checktype(L, 5, LUA_TTABLE); - res_tb_idx = 5; - -#if 0 - /* clear the Lua table */ - lua_pushnil(L); - while (lua_next(L, res_tb_idx) != 0) { - lua_pop(L, 1); - lua_pushvalue(L, -1); - lua_pushnil(L); - lua_rawset(L, res_tb_idx); - } -#endif - - } else { - group_id = luaL_checkint(L, 5); - if (group_id < 0) { - group_id = 0; - } - } - } - - re_comp.options = 0; - - flags = ngx_http_lua_ngx_re_parse_opts(L, &re_comp, &opts, 3); - - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - if (flags & NGX_LUA_RE_COMPILE_ONCE) { - pool = lmcf->pool; - - dd("server pool %p", lmcf->pool); - - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - regex_cache_key)); - lua_rawget(L, LUA_REGISTRYINDEX); /* table */ - - lua_pushliteral(L, "m"); - lua_pushvalue(L, 2); /* table regex */ - - dd("options size: %d", (int) sizeof(re_comp.options)); - - lua_pushlstring(L, (char *) &re_comp.options, sizeof(re_comp.options)); - /* table regex opts */ - - lua_concat(L, 3); /* table key */ - lua_pushvalue(L, -1); /* table key key */ - - dd("regex cache key: %.*s", (int) (pat.len + sizeof(re_comp.options)), - lua_tostring(L, -1)); - - lua_rawget(L, -3); /* table key re */ - re = lua_touserdata(L, -1); - - lua_pop(L, 1); /* table key */ - - if (re) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua regex cache hit for match regex \"%s\" with " - "options \"%s\"", pat.data, opts.data); - - lua_pop(L, 2); - - dd("restoring regex %p, ncaptures %d, captures %p", re->regex, - re->ncaptures, re->captures); - - re_comp.regex = re->regex; - sd = re->regex_sd; - re_comp.captures = re->ncaptures; - cap = re->captures; - - if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; - - } else { - ovecsize = (re->ncaptures + 1) * 3; - } - - goto exec; - } - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua regex cache miss for match regex \"%s\" " - "with options \"%s\"", pat.data, opts.data); - - if (lmcf->regex_cache_entries >= lmcf->regex_cache_max_entries) { - - if (lmcf->regex_cache_entries == lmcf->regex_cache_max_entries) { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "lua exceeding regex cache max entries (%i)", - lmcf->regex_cache_max_entries); - - lmcf->regex_cache_entries++; - } - - pool = r->pool; - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - } - - } else { - pool = r->pool; - } - - dd("pool %p, r pool %p", pool, r->pool); - - re_comp.pattern = pat; - re_comp.err.len = NGX_MAX_CONF_ERRSTR; - re_comp.err.data = errstr; - re_comp.pool = pool; - - ngx_log_debug5(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua compiling match regex \"%s\" with options \"%s\" " - "(compile once: %d) (dfa mode: %d) (jit mode: %d)", - pat.data, opts.data, - (flags & NGX_LUA_RE_COMPILE_ONCE) != 0, - (flags & NGX_LUA_RE_MODE_DFA) != 0, - (flags & NGX_LUA_RE_MODE_JIT) != 0); - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - rc = ngx_http_lua_regex_compile(&re_comp); - - ngx_http_lua_pcre_malloc_done(old_pool); - - if (rc != NGX_OK) { - dd("compile failed"); - - lua_pushnil(L); - if (!wantcaps) { - lua_pushnil(L); - } - lua_pushlstring(L, (char *) re_comp.err.data, re_comp.err.len); - return wantcaps ? 2 : 3; - } - -#if (LUA_HAVE_PCRE_JIT) - - if (flags & NGX_LUA_RE_MODE_JIT) { - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - sd = pcre_study(re_comp.regex, PCRE_STUDY_JIT_COMPILE, &msg); - - if (sd && lmcf->jit_stack) { - pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); - } - - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - dd("sd = %p", sd); - - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } - - if (sd != NULL) { - int jitted; - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - pcre_fullinfo(re_comp.regex, sd, PCRE_INFO_JIT, &jitted); - - ngx_http_lua_pcre_malloc_done(old_pool); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre JIT compiling result: %d", jitted); - } -# endif /* !(NGX_DEBUG) */ - - } else { - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - sd = pcre_study(re_comp.regex, 0, &msg); - - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - dd("sd = %p", sd); - - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre_study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } -# endif /* NGX_DEBUG */ - } - -#else /* !(LUA_HAVE_PCRE_JIT) */ - - if (flags & NGX_LUA_RE_MODE_JIT) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "your pcre build does not have JIT support and " - "the \"j\" regex option is ignored"); - } - -#endif /* LUA_HAVE_PCRE_JIT */ - - if (sd && lmcf->regex_match_limit > 0) { - sd->flags |= PCRE_EXTRA_MATCH_LIMIT; - sd->match_limit = lmcf->regex_match_limit; - } - - dd("compile done, captures %d", (int) re_comp.captures); - - if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; - re_comp.captures = 0; - - } else { - ovecsize = (re_comp.captures + 1) * 3; - } - - dd("allocating cap with size: %d", (int) ovecsize); - - cap = ngx_palloc(pool, ovecsize * sizeof(int)); - - if (cap == NULL) { - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - msg = "no memory"; - goto error; - } - - if (flags & NGX_LUA_RE_COMPILE_ONCE) { - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua saving compiled regex (%d captures) into the cache " - "(entries %i)", re_comp.captures, - lmcf->regex_cache_entries); - - re = ngx_palloc(pool, sizeof(ngx_http_lua_regex_t)); - if (re == NULL) { - msg = "no memory"; - goto error; - } - - dd("saving regex %p, ncaptures %d, captures %p", re_comp.regex, - re_comp.captures, cap); - - re->regex = re_comp.regex; - re->regex_sd = sd; - re->ncaptures = re_comp.captures; - re->captures = cap; - re->replace = NULL; - - lua_pushlightuserdata(L, re); /* table key value */ - lua_rawset(L, -3); /* table */ - lua_pop(L, 1); - - if (lmcf) { - lmcf->regex_cache_entries++; - } - } - -exec: - - if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMECOUNT, - &name_count) != 0) - { - msg = "cannot acquire named subpattern count"; - goto error; - } - - if (name_count > 0) { - if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMEENTRYSIZE, - &name_entry_size) != 0) - { - msg = "cannot acquire named subpattern entry size"; - goto error; - } - - if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMETABLE, - &name_table) != 0) - { - msg = "cannot acquire named subpattern table"; - goto error; - } - } - - if (flags & NGX_LUA_RE_NO_UTF8_CHECK) { - exec_opts = PCRE_NO_UTF8_CHECK; - - } else { - exec_opts = 0; - } - - if (flags & NGX_LUA_RE_MODE_DFA) { - -#if LUA_HAVE_PCRE_DFA - - int ws[NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT]; - rc = ngx_http_lua_regex_dfa_exec(re_comp.regex, sd, &subj, - (int) pos, cap, ovecsize, ws, - sizeof(ws)/sizeof(ws[0]), exec_opts); - -#else /* LUA_HAVE_PCRE_DFA */ - - msg = "at least pcre 6.0 is required for the DFA mode"; - goto error; - -#endif /* LUA_HAVE_PCRE_DFA */ - - } else { - rc = ngx_http_lua_regex_exec(re_comp.regex, sd, &subj, (int) pos, cap, - ovecsize, exec_opts); - } - - if (rc == NGX_REGEX_NO_MATCHED) { - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "regex \"%V\" not matched on string \"%V\" starting " - "from %i", &pat, &subj, pos); - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - ngx_pfree(pool, re_comp.regex); - ngx_pfree(pool, cap); - } - - lua_pushnil(L); - return 1; - } - - if (rc < 0) { - msg = lua_pushfstring(L, ngx_regex_exec_n " failed: %d", (int) rc); - goto error; - } - - if (rc == 0) { - if (flags & NGX_LUA_RE_MODE_DFA) { - rc = 1; - - } else { - msg = "capture size too small"; - goto error; - } - } - - dd("rc = %d", (int) rc); - - if (has_ctx) { /* having ctx table */ - pos = cap[1]; - lua_pushinteger(L, (lua_Integer) (pos + 1)); - lua_setfield(L, 4, "pos"); - } - - if (!wantcaps) { - if (group_id > re_comp.captures) { - lua_pushnil(L); - lua_pushnil(L); - lua_pushliteral(L, "nth out of bound"); - return 3; - } - - if (group_id >= rc) { - lua_pushnil(L); - lua_pushnil(L); - return 2; - } - - { - int from, to; - - from = cap[group_id * 2] + 1; - to = cap[group_id * 2 + 1]; - if (from < 0 || to < 0) { - lua_pushnil(L); - lua_pushnil(L); - return 2; - } - - lua_pushinteger(L, from); - lua_pushinteger(L, to); - return 2; - } - } - - if (res_tb_idx == 0) { - lua_createtable(L, re_comp.captures || 1 /* narr */, - name_count /* nrec */); - res_tb_idx = lua_gettop(L); - } - - for (i = 0, n = 0; i <= re_comp.captures; i++, n += 2) { - dd("capture %d: %d %d", i, cap[n], cap[n + 1]); - if (i >= rc || cap[n] < 0) { - lua_pushboolean(L, 0); - - } else { - lua_pushlstring(L, (char *) &subj.data[cap[n]], - cap[n + 1] - cap[n]); - - dd("pushing capture %s at %d", lua_tostring(L, -1), (int) i); - } - - lua_rawseti(L, res_tb_idx, (int) i); - } - - if (name_count > 0) { - ngx_http_lua_re_collect_named_captures(L, res_tb_idx, name_table, - name_count, name_entry_size, - flags, &subj); - } - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - ngx_pfree(pool, re_comp.regex); - ngx_pfree(pool, cap); - } - - return 1; - -error: - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - if (re_comp.regex) { - ngx_pfree(pool, re_comp.regex); - } - - if (cap) { - ngx_pfree(pool, cap); - } - } - - lua_pushnil(L); - if (!wantcaps) { - lua_pushnil(L); - } - lua_pushstring(L, msg); - return wantcaps ? 2 : 3; -} - - -static int -ngx_http_lua_ngx_re_gmatch(lua_State *L) -{ - ngx_http_lua_main_conf_t *lmcf; - ngx_http_request_t *r; - ngx_str_t subj; - ngx_str_t pat; - ngx_str_t opts; - int ovecsize; - ngx_http_lua_regex_t *re; - ngx_http_lua_regex_ctx_t *ctx; - const char *msg; - int nargs; - ngx_int_t flags; - int *cap = NULL; - ngx_int_t rc; - ngx_pool_t *pool, *old_pool; - u_char errstr[NGX_MAX_CONF_ERRSTR + 1]; - pcre_extra *sd = NULL; - ngx_http_cleanup_t *cln; - - ngx_http_lua_regex_compile_t re_comp; - - nargs = lua_gettop(L); - - if (nargs != 2 && nargs != 3) { - return luaL_error(L, "expecting two or three arguments, but got %d", - nargs); - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - subj.data = (u_char *) luaL_checklstring(L, 1, &subj.len); - pat.data = (u_char *) luaL_checklstring(L, 2, &pat.len); - - if (nargs == 3) { - opts.data = (u_char *) luaL_checklstring(L, 3, &opts.len); - lua_pop(L, 1); - - } else { - opts.data = (u_char *) ""; - opts.len = 0; - } - - /* stack: subj regex */ - - re_comp.options = 0; - - flags = ngx_http_lua_ngx_re_parse_opts(L, &re_comp, &opts, 3); - - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - if (flags & NGX_LUA_RE_COMPILE_ONCE) { - pool = lmcf->pool; - - dd("server pool %p", lmcf->pool); - - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - regex_cache_key)); - lua_rawget(L, LUA_REGISTRYINDEX); /* table */ - - lua_pushliteral(L, "m"); - lua_pushvalue(L, 2); /* table regex */ - - dd("options size: %d", (int) sizeof(re_comp.options)); - - lua_pushlstring(L, (char *) &re_comp.options, - sizeof(re_comp.options)); /* table regex opts */ - - lua_concat(L, 3); /* table key */ - lua_pushvalue(L, -1); /* table key key */ - - dd("regex cache key: %.*s", (int) (pat.len + sizeof(re_comp.options)), - lua_tostring(L, -1)); - - lua_rawget(L, -3); /* table key re */ - re = lua_touserdata(L, -1); - - lua_pop(L, 1); /* table key */ - - if (re) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua regex cache hit for match regex \"%s\" " - "with options \"%s\"", pat.data, opts.data); - - lua_pop(L, 2); - - dd("restoring regex %p, ncaptures %d, captures %p", re->regex, - re->ncaptures, re->captures); - - re_comp.regex = re->regex; - sd = re->regex_sd; - re_comp.captures = re->ncaptures; - cap = re->captures; - - if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; - - } else { - ovecsize = (re->ncaptures + 1) * 3; - } - - goto compiled; - } - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua regex cache miss for match regex \"%s\" " - "with options \"%s\"", pat.data, opts.data); - - if (lmcf->regex_cache_entries >= lmcf->regex_cache_max_entries) { - - if (lmcf->regex_cache_entries == lmcf->regex_cache_max_entries) { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "lua exceeding regex cache max entries (%i)", - lmcf->regex_cache_max_entries); - - lmcf->regex_cache_entries++; - } - - pool = r->pool; - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - } - - } else { - pool = r->pool; - } - - re_comp.pattern = pat; - re_comp.err.len = NGX_MAX_CONF_ERRSTR; - re_comp.err.data = errstr; - re_comp.pool = pool; - - ngx_log_debug5(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua compiling gmatch regex \"%s\" with options \"%s\" " - "(compile once: %d) (dfa mode: %d) (jit mode: %d)", - pat.data, opts.data, - (flags & NGX_LUA_RE_COMPILE_ONCE) != 0, - (flags & NGX_LUA_RE_MODE_DFA) != 0, - (flags & NGX_LUA_RE_MODE_JIT) != 0); - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - rc = ngx_http_lua_regex_compile(&re_comp); - - ngx_http_lua_pcre_malloc_done(old_pool); - - if (rc != NGX_OK) { - dd("compile failed"); - - lua_pushnil(L); - lua_pushlstring(L, (char *) re_comp.err.data, re_comp.err.len); - return 2; - } - -#if LUA_HAVE_PCRE_JIT - - if (flags & NGX_LUA_RE_MODE_JIT) { - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - sd = pcre_study(re_comp.regex, PCRE_STUDY_JIT_COMPILE, &msg); - - if (sd && lmcf->jit_stack) { - pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); - } - - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - dd("sd = %p", sd); - - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre_study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } - - if (sd != NULL) { - int jitted; - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - pcre_fullinfo(re_comp.regex, sd, PCRE_INFO_JIT, &jitted); - - ngx_http_lua_pcre_malloc_done(old_pool); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre JIT compiling result: %d", jitted); - } -# endif /* NGX_DEBUG */ - - } else { - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - sd = pcre_study(re_comp.regex, 0, &msg); - - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - dd("sd = %p", sd); - - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } -# endif /* NGX_DEBUG */ - } - -#else /* LUA_HAVE_PCRE_JIT */ - - if (flags & NGX_LUA_RE_MODE_JIT) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "your pcre build does not have JIT support and " - "the \"j\" regex option is ignored"); - } - -#endif /* LUA_HAVE_PCRE_JIT */ - - if (sd && lmcf->regex_match_limit > 0) { - sd->flags |= PCRE_EXTRA_MATCH_LIMIT; - sd->match_limit = lmcf->regex_match_limit; - } - - dd("compile done, captures %d", re_comp.captures); - - if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; - re_comp.captures = 0; - - } else { - ovecsize = (re_comp.captures + 1) * 3; - } - - cap = ngx_palloc(pool, ovecsize * sizeof(int)); - if (cap == NULL) { - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - msg = "no memory"; - goto error; - } - - if (flags & NGX_LUA_RE_COMPILE_ONCE) { - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua saving compiled regex (%d captures) into the cache " - "(entries %i)", re_comp.captures, - lmcf->regex_cache_entries); - - re = ngx_palloc(pool, sizeof(ngx_http_lua_regex_t)); - if (re == NULL) { - msg = "no memory"; - goto error; - } - - dd("saving regex %p, ncaptures %d, captures %p", re_comp.regex, - re_comp.captures, cap); - - re->regex = re_comp.regex; - re->regex_sd = sd; - re->ncaptures = re_comp.captures; - re->captures = cap; - re->replace = NULL; - - lua_pushlightuserdata(L, re); /* table key value */ - lua_rawset(L, -3); /* table */ - lua_pop(L, 1); - - if (lmcf) { - lmcf->regex_cache_entries++; - } - } - -compiled: - - lua_settop(L, 1); - - ctx = lua_newuserdata(L, sizeof(ngx_http_lua_regex_ctx_t)); - - ctx->request = r; - ctx->regex = re_comp.regex; - ctx->regex_sd = sd; - ctx->ncaptures = re_comp.captures; - ctx->captures = cap; - ctx->captures_len = ovecsize; - ctx->flags = (uint8_t) flags; - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ - lua_pushcfunction(L, ngx_http_lua_ngx_re_gmatch_gc); - lua_setfield(L, -2, "__gc"); - lua_setmetatable(L, -2); - - cln = ngx_http_cleanup_add(r, 0); - if (cln == NULL) { - msg = "no memory"; - goto error; - } - - cln->handler = ngx_http_lua_ngx_re_gmatch_cleanup; - cln->data = ctx; - ctx->cleanup = &cln->handler; - - } else { - ctx->cleanup = NULL; - } - - lua_pushinteger(L, 0); - - /* upvalues in order: subj ctx offset */ - lua_pushcclosure(L, ngx_http_lua_ngx_re_gmatch_iterator, 3); - - return 1; - -error: - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - if (re_comp.regex) { - ngx_pfree(pool, re_comp.regex); - } - - if (cap) { - ngx_pfree(pool, cap); - } - } - - lua_pushnil(L); - lua_pushstring(L, msg); - return 2; -} - - -static int -ngx_http_lua_ngx_re_gmatch_iterator(lua_State *L) -{ - ngx_http_lua_regex_ctx_t *ctx; - ngx_http_request_t *r; - int *cap; - ngx_int_t rc; - ngx_uint_t n; - int i; - ngx_str_t subj; - int offset; - const char *msg = NULL; - int name_entry_size = 0, name_count; - u_char *name_table = NULL; - int exec_opts; - - /* upvalues in order: subj ctx offset */ - - subj.data = (u_char *) lua_tolstring(L, lua_upvalueindex(1), &subj.len); - ctx = (ngx_http_lua_regex_ctx_t *) lua_touserdata(L, lua_upvalueindex(2)); - offset = (int) lua_tointeger(L, lua_upvalueindex(3)); - - if (offset < 0) { - lua_pushnil(L); - return 1; - } - - cap = ctx->captures; - - dd("offset %d, r %p, subj %s", (int) offset, ctx->request, subj.data); - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - if (r != ctx->request || r->pool != ctx->request->pool) { - return luaL_error(L, "attempt to use ngx.re.gmatch iterator in a " - "request that did not create it"); - } - - dd("regex exec..."); - - if (pcre_fullinfo(ctx->regex, NULL, PCRE_INFO_NAMECOUNT, - &name_count) != 0) - { - msg = "cannot acquire named subpattern count"; - goto error; - } - - if (name_count > 0) { - if (pcre_fullinfo(ctx->regex, NULL, PCRE_INFO_NAMEENTRYSIZE, - &name_entry_size) != 0) - { - msg = "cannot acquire named subpattern entry size"; - goto error; - } - - if (pcre_fullinfo(ctx->regex, NULL, PCRE_INFO_NAMETABLE, - &name_table) != 0) - { - msg = "cannot acquire named subpattern table"; - goto error; - } - } - - if (ctx->flags & NGX_LUA_RE_NO_UTF8_CHECK) { - exec_opts = PCRE_NO_UTF8_CHECK; - - } else { - exec_opts = 0; - } - - if (ctx->flags & NGX_LUA_RE_MODE_DFA) { - -#if LUA_HAVE_PCRE_DFA - - int ws[NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT]; - - rc = ngx_http_lua_regex_dfa_exec(ctx->regex, ctx->regex_sd, &subj, - offset, cap, ctx->captures_len, ws, - sizeof(ws)/sizeof(ws[0]), exec_opts); - -#else /* LUA_HAVE_PCRE_DFA */ - msg = "at least pcre 6.0 is required for the DFA mode"; - goto error; - -#endif /* LUA_HAVE_PCRE_DFA */ - - } else { - rc = ngx_http_lua_regex_exec(ctx->regex, ctx->regex_sd, &subj, - offset, cap, ctx->captures_len, - exec_opts); - } - - if (rc == NGX_REGEX_NO_MATCHED) { - /* set upvalue "offset" to -1 */ - lua_pushinteger(L, -1); - lua_replace(L, lua_upvalueindex(3)); - - if (!(ctx->flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (ctx->regex_sd) { - ngx_http_lua_regex_free_study_data(r->pool, ctx->regex_sd); - ctx->regex_sd = NULL; - } - - ngx_pfree(r->pool, cap); - } - - lua_pushnil(L); - return 1; - } - - if (rc < 0) { - msg = lua_pushfstring(L, ngx_regex_exec_n " failed: %d", (int) rc); - goto error; - } - - if (rc == 0) { - if (ctx->flags & NGX_LUA_RE_MODE_DFA) { - rc = 1; - - } else { - goto error; - } - } - - dd("rc = %d", (int) rc); - - lua_createtable(L, ctx->ncaptures || 1 /* narr */, name_count /* nrec */); - - for (i = 0, n = 0; i <= ctx->ncaptures; i++, n += 2) { - dd("capture %d: %d %d", i, cap[n], cap[n + 1]); - if (i >= rc || cap[n] < 0) { - lua_pushboolean(L, 0); - - } else { - lua_pushlstring(L, (char *) &subj.data[cap[n]], - cap[n + 1] - cap[n]); - - dd("pushing capture %s at %d", lua_tostring(L, -1), (int) i); - } - - lua_rawseti(L, -2, (int) i); - } - - if (name_count > 0) { - ngx_http_lua_re_collect_named_captures(L, lua_gettop(L), name_table, - name_count, name_entry_size, - ctx->flags, &subj); - } - - offset = cap[1]; - if (offset == cap[0]) { - offset++; - } - - if (offset > (ssize_t) subj.len) { - offset = -1; - - if (!(ctx->flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (ctx->regex_sd) { - ngx_http_lua_regex_free_study_data(r->pool, ctx->regex_sd); - ctx->regex_sd = NULL; - } - - ngx_pfree(r->pool, cap); - } - } - - lua_pushinteger(L, offset); - lua_replace(L, lua_upvalueindex(3)); - - return 1; - -error: - - lua_pushinteger(L, -1); - lua_replace(L, lua_upvalueindex(3)); - - if (!(ctx->flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (ctx->regex_sd) { - ngx_http_lua_regex_free_study_data(r->pool, ctx->regex_sd); - ctx->regex_sd = NULL; - } - - ngx_pfree(r->pool, cap); - } - - lua_pushnil(L); - lua_pushstring(L, msg); - return 2; -} - - -static ngx_uint_t -ngx_http_lua_ngx_re_parse_opts(lua_State *L, ngx_http_lua_regex_compile_t *re, - ngx_str_t *opts, int narg) -{ - u_char *p; - const char *msg; - ngx_uint_t flags; - - flags = 0; - p = opts->data; - - while (*p != '\0') { - switch (*p) { - case 'i': - re->options |= NGX_REGEX_CASELESS; - break; - - case 's': - re->options |= PCRE_DOTALL; - break; - - case 'm': - re->options |= PCRE_MULTILINE; - break; - - case 'u': - re->options |= PCRE_UTF8; - break; - - case 'U': - re->options |= PCRE_UTF8; - flags |= NGX_LUA_RE_NO_UTF8_CHECK; - break; - - case 'x': - re->options |= PCRE_EXTENDED; - break; - - case 'o': - flags |= NGX_LUA_RE_COMPILE_ONCE; - break; - - case 'j': - flags |= NGX_LUA_RE_MODE_JIT; - break; - - case 'd': - flags |= NGX_LUA_RE_MODE_DFA; - break; - - case 'a': - re->options |= PCRE_ANCHORED; - break; - -#if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 12) - case 'D': - re->options |= PCRE_DUPNAMES; - flags |= NGX_LUA_RE_MODE_DUPNAMES; - break; - - case 'J': - re->options |= PCRE_JAVASCRIPT_COMPAT; - break; -#endif - - default: - msg = lua_pushfstring(L, "unknown flag \"%c\" (flags \"%s\")", - *p, opts->data); - return luaL_argerror(L, narg, msg); - } - - p++; - } - - /* pcre does not support JIT for DFA mode yet, - * so if DFA mode is specified, we turn off JIT automatically - * */ - if ((flags & NGX_LUA_RE_MODE_JIT) && (flags & NGX_LUA_RE_MODE_DFA)) { - flags &= ~NGX_LUA_RE_MODE_JIT; - } - - return flags; -} - - -static int -ngx_http_lua_ngx_re_sub(lua_State *L) -{ - return ngx_http_lua_ngx_re_sub_helper(L, 0 /* global */); -} - - -static int -ngx_http_lua_ngx_re_gsub(lua_State *L) -{ - return ngx_http_lua_ngx_re_sub_helper(L, 1 /* global */); -} - - -static int -ngx_http_lua_ngx_re_sub_helper(lua_State *L, unsigned global) -{ - ngx_http_lua_regex_t *re; - ngx_http_request_t *r; - ngx_str_t subj; - ngx_str_t pat; - ngx_str_t opts; - ngx_str_t tpl; - ngx_http_lua_main_conf_t *lmcf; - ngx_pool_t *pool, *old_pool; - const char *msg; - ngx_int_t rc; - ngx_uint_t n; - ngx_int_t i; - int nargs; - int *cap = NULL; - int ovecsize; - int type; - unsigned func; - int offset; - int cp_offset; - size_t count; - luaL_Buffer luabuf; - ngx_int_t flags; - u_char *p; - u_char errstr[NGX_MAX_CONF_ERRSTR + 1]; - pcre_extra *sd = NULL; - int name_entry_size = 0, name_count; - u_char *name_table = NULL; - int exec_opts; - - ngx_http_lua_regex_compile_t re_comp; - ngx_http_lua_complex_value_t *ctpl = NULL; - ngx_http_lua_compile_complex_value_t ccv; - - nargs = lua_gettop(L); - - if (nargs != 3 && nargs != 4) { - return luaL_error(L, "expecting three or four arguments, but got %d", - nargs); - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - subj.data = (u_char *) luaL_checklstring(L, 1, &subj.len); - pat.data = (u_char *) luaL_checklstring(L, 2, &pat.len); - - func = 0; - - type = lua_type(L, 3); - switch (type) { - case LUA_TFUNCTION: - func = 1; - tpl.len = 0; - tpl.data = (u_char *) ""; - break; - - case LUA_TNUMBER: - case LUA_TSTRING: - tpl.data = (u_char *) lua_tolstring(L, 3, &tpl.len); - break; - - default: - msg = lua_pushfstring(L, "string, number, or function expected, " - "got %s", lua_typename(L, type)); - return luaL_argerror(L, 3, msg); - } - - ngx_memzero(&re_comp, sizeof(ngx_http_lua_regex_compile_t)); - - if (nargs == 4) { - opts.data = (u_char *) luaL_checklstring(L, 4, &opts.len); - lua_pop(L, 1); - - } else { /* nargs == 3 */ - opts.data = (u_char *) ""; - opts.len = 0; - } - - /* stack: subj regex repl */ - - re_comp.options = 0; - - flags = ngx_http_lua_ngx_re_parse_opts(L, &re_comp, &opts, 4); - - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - if (flags & NGX_LUA_RE_COMPILE_ONCE) { - pool = lmcf->pool; - - dd("server pool %p", lmcf->pool); - - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - regex_cache_key)); - lua_rawget(L, LUA_REGISTRYINDEX); /* table */ - - lua_pushliteral(L, "s"); - lua_pushinteger(L, tpl.len); - lua_pushliteral(L, ":"); - lua_pushvalue(L, 2); - - if (tpl.len != 0) { - lua_pushvalue(L, 3); - } - - dd("options size: %d", (int) sizeof(re_comp.options)); - - lua_pushlstring(L, (char *) &re_comp.options, sizeof(re_comp.options)); - /* table regex opts */ - - if (tpl.len == 0) { - lua_concat(L, 5); /* table key */ - - } else { - lua_concat(L, 6); /* table key */ - } - - lua_pushvalue(L, -1); /* table key key */ - - dd("regex cache key: %.*s", (int) (pat.len + sizeof(re_comp.options)), - lua_tostring(L, -1)); - - lua_rawget(L, -3); /* table key re */ - re = lua_touserdata(L, -1); - - lua_pop(L, 1); /* table key */ - - if (re) { - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua regex cache hit for sub regex \"%s\" with " - "options \"%s\" and replace \"%s\"", - pat.data, opts.data, - func ? (u_char *) "" : tpl.data); - - lua_pop(L, 2); - - dd("restoring regex %p, ncaptures %d, captures %p", re->regex, - re->ncaptures, re->captures); - - re_comp.regex = re->regex; - sd = re->regex_sd; - re_comp.captures = re->ncaptures; - cap = re->captures; - ctpl = re->replace; - - if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; - - } else { - ovecsize = (re->ncaptures + 1) * 3; - } - - goto exec; - } - - ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua regex cache miss for %ssub regex \"%s\" with " - "options \"%s\" and replace \"%s\"", - global ? "g" : "", pat.data, opts.data, - func ? (u_char *) "" : tpl.data); - - if (lmcf->regex_cache_entries >= lmcf->regex_cache_max_entries) { - - if (lmcf->regex_cache_entries == lmcf->regex_cache_max_entries) { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "lua exceeding regex cache max entries (%i)", - lmcf->regex_cache_max_entries); - - lmcf->regex_cache_entries++; - } - - pool = r->pool; - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - } - - } else { - pool = r->pool; - } - - re_comp.pattern = pat; - re_comp.err.len = NGX_MAX_CONF_ERRSTR; - re_comp.err.data = errstr; - re_comp.pool = pool; - - dd("compiling regex"); - - ngx_log_debug6(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua compiling %ssub regex \"%s\" with options \"%s\" " - "(compile once: %d) (dfa mode: %d) (jit mode: %d)", - global ? "g" : "", pat.data, opts.data, - (flags & NGX_LUA_RE_COMPILE_ONCE) != 0, - (flags & NGX_LUA_RE_MODE_DFA) != 0, - (flags & NGX_LUA_RE_MODE_JIT) != 0); - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - rc = ngx_http_lua_regex_compile(&re_comp); - - ngx_http_lua_pcre_malloc_done(old_pool); - - if (rc != NGX_OK) { - dd("compile failed"); - - lua_pushnil(L); - lua_pushnil(L); - lua_pushlstring(L, (char *) re_comp.err.data, re_comp.err.len); - return 3; - } - -#if LUA_HAVE_PCRE_JIT - - if (flags & NGX_LUA_RE_MODE_JIT) { - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - sd = pcre_study(re_comp.regex, PCRE_STUDY_JIT_COMPILE, &msg); - - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - dd("sd = %p", sd); - - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } - - if (sd != NULL) { - int jitted; - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - pcre_fullinfo(re_comp.regex, sd, PCRE_INFO_JIT, &jitted); - - ngx_http_lua_pcre_malloc_done(old_pool); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre JIT compiling result: %d", jitted); - } -# endif /* NGX_DEBUG */ - - } else { - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - sd = pcre_study(re_comp.regex, 0, &msg); - - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - dd("sd = %p", sd); - - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "pcre_study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } -# endif /* NGX_DEBUG */ - } - -#else /* LUA_HAVE_PCRE_JIT */ - - if (flags & NGX_LUA_RE_MODE_JIT) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "your pcre build does not have JIT support and " - "the \"j\" regex option is ignored"); - } - -#endif /* LUA_HAVE_PCRE_JIT */ - - if (sd && lmcf->regex_match_limit > 0) { - sd->flags |= PCRE_EXTRA_MATCH_LIMIT; - sd->match_limit = lmcf->regex_match_limit; - } - - dd("compile done, captures %d", re_comp.captures); - - if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; - re_comp.captures = 0; - - } else { - ovecsize = (re_comp.captures + 1) * 3; - } - - cap = ngx_palloc(pool, ovecsize * sizeof(int)); - if (cap == NULL) { - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - msg = "no memory"; - goto error; - } - if (func) { - ctpl = NULL; - - } else { - ctpl = ngx_palloc(pool, sizeof(ngx_http_lua_complex_value_t)); - if (ctpl == NULL) { - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - msg = "no memory"; - goto error; - } - - if ((flags & NGX_LUA_RE_COMPILE_ONCE) && tpl.len != 0) { - /* copy the string buffer pointed to by tpl.data from Lua VM */ - p = ngx_palloc(pool, tpl.len + 1); - if (p == NULL) { - flags &= ~NGX_LUA_RE_COMPILE_ONCE; - msg = "no memory"; - goto error; - } - - ngx_memcpy(p, tpl.data, tpl.len); - p[tpl.len] = '\0'; - - tpl.data = p; - } - - ngx_memzero(&ccv, sizeof(ngx_http_lua_compile_complex_value_t)); - ccv.pool = pool; - ccv.log = r->connection->log; - ccv.value = &tpl; - ccv.complex_value = ctpl; - - if (ngx_http_lua_compile_complex_value(&ccv) != NGX_OK) { - ngx_pfree(pool, cap); - ngx_pfree(pool, ctpl); - - if ((flags & NGX_LUA_RE_COMPILE_ONCE) && tpl.len != 0) { - ngx_pfree(pool, tpl.data); - } - - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - ngx_pfree(pool, re_comp.regex); - - lua_pushnil(L); - lua_pushnil(L); - lua_pushliteral(L, "failed to compile the replacement template"); - return 3; - } - } - - if (flags & NGX_LUA_RE_COMPILE_ONCE) { - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua saving compiled sub regex (%d captures) into " - "the cache (entries %i)", re_comp.captures, - lmcf->regex_cache_entries); - - re = ngx_palloc(pool, sizeof(ngx_http_lua_regex_t)); - if (re == NULL) { - msg = "no memory"; - goto error; - } - - dd("saving regex %p, ncaptures %d, captures %p", re_comp.regex, - re_comp.captures, cap); - - re->regex = re_comp.regex; - re->regex_sd = sd; - re->ncaptures = re_comp.captures; - re->captures = cap; - re->replace = ctpl; - - lua_pushlightuserdata(L, re); /* table key value */ - lua_rawset(L, -3); /* table */ - lua_pop(L, 1); - - if (lmcf) { - lmcf->regex_cache_entries++; - } - } - -exec: - - count = 0; - offset = 0; - cp_offset = 0; - - if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMECOUNT, - &name_count) != 0) - { - msg = "cannot acquire named subpattern count"; - goto error; - } - - if (name_count > 0) { - if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMEENTRYSIZE, - &name_entry_size) != 0) - { - msg = "cannot acquire named subpattern entry size"; - goto error; - } - - if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMETABLE, - &name_table) != 0) - { - msg = "cannot acquire named subpattern table"; - goto error; - } - } - - if (flags & NGX_LUA_RE_NO_UTF8_CHECK) { - exec_opts = PCRE_NO_UTF8_CHECK; - - } else { - exec_opts = 0; - } - - for (;;) { - if (flags & NGX_LUA_RE_MODE_DFA) { - -#if LUA_HAVE_PCRE_DFA - - int ws[NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT]; - rc = ngx_http_lua_regex_dfa_exec(re_comp.regex, sd, &subj, - offset, cap, ovecsize, ws, - sizeof(ws)/sizeof(ws[0]), - exec_opts); - -#else /* LUA_HAVE_PCRE_DFA */ - - msg = "at least pcre 6.0 is required for the DFA mode"; - goto error; - -#endif /* LUA_HAVE_PCRE_DFA */ - - } else { - rc = ngx_http_lua_regex_exec(re_comp.regex, sd, &subj, offset, cap, - ovecsize, exec_opts); - } - - if (rc == NGX_REGEX_NO_MATCHED) { - break; - } - - if (rc < 0) { - msg = lua_pushfstring(L, ngx_regex_exec_n " failed: %d", - (int) rc); - goto error; - } - - if (rc == 0) { - if (flags & NGX_LUA_RE_MODE_DFA) { - rc = 1; - - } else { - msg = "capture size too small"; - goto error; - } - } - - dd("rc = %d", (int) rc); - - count++; - - if (count == 1) { - luaL_buffinit(L, &luabuf); - } - - if (func) { - lua_pushvalue(L, 3); - - lua_createtable(L, re_comp.captures || 1 /* narr */, - name_count /* nrec */); - - for (i = 0, n = 0; i <= re_comp.captures; i++, n += 2) { - dd("capture %d: %d %d", (int) i, cap[n], cap[n + 1]); - if (i >= rc || cap[n] < 0) { - lua_pushboolean(L, 0); - - } else { - lua_pushlstring(L, (char *) &subj.data[cap[n]], - cap[n + 1] - cap[n]); - - dd("pushing capture %s at %d", lua_tostring(L, -1), - (int) i); - } - - lua_rawseti(L, -2, (int) i); - } - - if (name_count > 0) { - ngx_http_lua_re_collect_named_captures(L, lua_gettop(L), - name_table, - name_count, - name_entry_size, - flags, &subj); - } - - dd("stack size at call: %d", lua_gettop(L)); - - lua_call(L, 1 /* nargs */, 1 /* nresults */); - type = lua_type(L, -1); - switch (type) { - case LUA_TNUMBER: - case LUA_TSTRING: - tpl.data = (u_char *) lua_tolstring(L, -1, &tpl.len); - break; - - default: - msg = lua_pushfstring(L, "string or number expected to be " - "returned by the replace " - "function, got %s", - lua_typename(L, type)); - return luaL_argerror(L, 3, msg); - } - - lua_insert(L, 1); - - luaL_addlstring(&luabuf, (char *) &subj.data[cp_offset], - cap[0] - cp_offset); - - luaL_addlstring(&luabuf, (char *) tpl.data, tpl.len); - - lua_remove(L, 1); - - cp_offset = cap[1]; - offset = cp_offset; - if (offset == cap[0]) { - offset++; - if (offset > (ssize_t) subj.len) { - break; - } - } - - if (global) { - continue; - } - - break; - } - - rc = ngx_http_lua_complex_value(r, &subj, cp_offset, rc, cap, ctpl, - &luabuf); - - if (rc != NGX_OK) { - msg = lua_pushfstring(L, "failed to eval the template for " - "replacement: \"%s\"", tpl.data); - goto error; - } - - cp_offset = cap[1]; - offset = cp_offset; - if (offset == cap[0]) { - offset++; - if (offset > (ssize_t) subj.len) { - break; - } - } - - if (global) { - continue; - } - - break; - } - - if (count == 0) { - dd("no match, just the original subject"); - lua_settop(L, 1); - - } else { - if (offset < (int) subj.len) { - dd("adding trailer: %s (len %d)", &subj.data[cp_offset], - (int) (subj.len - cp_offset)); - - - luaL_addlstring(&luabuf, (char *) &subj.data[cp_offset], - subj.len - cp_offset); - } - - luaL_pushresult(&luabuf); - - dd("the dst string: %s", lua_tostring(L, -1)); - } - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - if (re_comp.regex) { - ngx_pfree(pool, re_comp.regex); - } - - if (ctpl) { - ngx_pfree(pool, ctpl); - } - - if (cap) { - ngx_pfree(pool, cap); - } - } - - lua_pushinteger(L, count); - return 2; - -error: - - if (!(flags & NGX_LUA_RE_COMPILE_ONCE)) { - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } - - if (re_comp.regex) { - ngx_pfree(pool, re_comp.regex); - } - - if (ctpl) { - ngx_pfree(pool, ctpl); - } - - if (cap) { - ngx_pfree(pool, cap); - } - } - - lua_pushnil(L); - lua_pushnil(L); - lua_pushstring(L, msg); - return 3; -} - - -ngx_int_t -ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, - size_t *errstr_size) -{ -#if LUA_HAVE_PCRE_JIT - - ngx_http_lua_main_conf_t *lmcf; - ngx_pool_t *pool, *old_pool; - - lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, - ngx_http_lua_module); - - if (size < NGX_LUA_RE_MIN_JIT_STACK_SIZE) { - size = NGX_LUA_RE_MIN_JIT_STACK_SIZE; - } - - pool = lmcf->pool; - - dd("server pool %p", lmcf->pool); - - if (lmcf->jit_stack) { - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - pcre_jit_stack_free(lmcf->jit_stack); - - ngx_http_lua_pcre_malloc_done(old_pool); - } - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - lmcf->jit_stack = pcre_jit_stack_alloc(NGX_LUA_RE_MIN_JIT_STACK_SIZE, - size); - - ngx_http_lua_pcre_malloc_done(old_pool); + int ncaptures; + int *captures; - if (lmcf->jit_stack == NULL) { - *errstr_size = ngx_snprintf(errstr, *errstr_size, - "pcre jit stack allocation failed") - - errstr; - return NGX_ERROR; - } + pcre *regex; + pcre_extra *regex_sd; - return NGX_OK; + ngx_http_lua_complex_value_t *replace; -#else /* LUA_HAVE_PCRE_JIT */ + /* only for (stap) debugging, and may be an invalid pointer */ + const u_char *pattern; +} ngx_http_lua_regex_t; - *errstr_size = ngx_snprintf(errstr, *errstr_size, - "no pcre jit support found") - errstr; - return NGX_ERROR; -#endif /* LUA_HAVE_PCRE_JIT */ -} +typedef struct { + ngx_str_t pattern; + ngx_pool_t *pool; + ngx_int_t options; + pcre *regex; + int captures; + ngx_str_t err; +} ngx_http_lua_regex_compile_t; -void -ngx_http_lua_inject_regex_api(lua_State *L) -{ - /* ngx.re */ - lua_createtable(L, 0, 5 /* nrec */); /* .re */ +typedef struct { + ngx_http_request_t *request; + pcre *regex; + pcre_extra *regex_sd; + int ncaptures; + int *captures; + int captures_len; + uint8_t flags; +} ngx_http_lua_regex_ctx_t; - lua_pushcfunction(L, ngx_http_lua_ngx_re_find); - lua_setfield(L, -2, "find"); - lua_pushcfunction(L, ngx_http_lua_ngx_re_match); - lua_setfield(L, -2, "match"); +static void ngx_http_lua_regex_free_study_data(ngx_pool_t *pool, + pcre_extra *sd); +static ngx_int_t ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc); - lua_pushcfunction(L, ngx_http_lua_ngx_re_gmatch); - lua_setfield(L, -2, "gmatch"); - lua_pushcfunction(L, ngx_http_lua_ngx_re_sub); - lua_setfield(L, -2, "sub"); +#define ngx_http_lua_regex_exec(re, e, s, start, captures, size, opts) \ + pcre_exec(re, e, (const char *) (s)->data, (s)->len, start, opts, \ + captures, size) - lua_pushcfunction(L, ngx_http_lua_ngx_re_gsub); - lua_setfield(L, -2, "gsub"); - lua_setfield(L, -2, "re"); -} +#define ngx_http_lua_regex_dfa_exec(re, e, s, start, captures, size, ws, \ + wscount, opts) \ + pcre_dfa_exec(re, e, (const char *) (s)->data, (s)->len, start, opts, \ + captures, size, ws, wscount) static void @@ -2077,108 +161,61 @@ ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc) } -static void -ngx_http_lua_ngx_re_gmatch_cleanup(void *data) +ngx_int_t +ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, + size_t *errstr_size) { - ngx_http_lua_regex_ctx_t *ctx = data; +#if LUA_HAVE_PCRE_JIT - if (ctx) { - if (ctx->regex_sd) { - ngx_http_lua_regex_free_study_data(ctx->request->pool, - ctx->regex_sd); - ctx->regex_sd = NULL; - } + ngx_http_lua_main_conf_t *lmcf; + ngx_pool_t *pool, *old_pool; - if (ctx->cleanup) { - *ctx->cleanup = NULL; - ctx->cleanup = NULL; - } + lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, + ngx_http_lua_module); - ctx->request = NULL; + if (size < NGX_LUA_RE_MIN_JIT_STACK_SIZE) { + size = NGX_LUA_RE_MIN_JIT_STACK_SIZE; } - return; -} + pool = lmcf->pool; + dd("server pool %p", lmcf->pool); -static int -ngx_http_lua_ngx_re_gmatch_gc(lua_State *L) -{ - ngx_http_lua_regex_ctx_t *ctx; + if (lmcf->jit_stack) { + old_pool = ngx_http_lua_pcre_malloc_init(pool); - ctx = lua_touserdata(L, 1); + pcre_jit_stack_free(lmcf->jit_stack); - if (ctx && ctx->cleanup) { - ngx_http_lua_ngx_re_gmatch_cleanup(ctx); + ngx_http_lua_pcre_malloc_done(old_pool); } - return 0; -} - - -static void -ngx_http_lua_re_collect_named_captures(lua_State *L, int res_tb_idx, - u_char *name_table, int name_count, int name_entry_size, unsigned flags, - ngx_str_t *subj) -{ - int i, n; - size_t len; - u_char *name_entry; - char *name; - - for (i = 0; i < name_count; i++) { - dd("top: %d", lua_gettop(L)); - - name_entry = &name_table[i * name_entry_size]; - n = (name_entry[0] << 8) | name_entry[1]; - name = (char *) &name_entry[2]; - - lua_rawgeti(L, -1, n); - if (lua_isnil(L, -1)) { - lua_pop(L, 1); - continue; - } - - if (flags & NGX_LUA_RE_MODE_DUPNAMES) { - /* unmatched groups are not stored in tables in DUPNAMES mode */ - if (!lua_toboolean(L, -1)) { - lua_pop(L, 1); - continue; - } + old_pool = ngx_http_lua_pcre_malloc_init(pool); - lua_getfield(L, -2, name); /* big_tb cap small_tb */ + lmcf->jit_stack = pcre_jit_stack_alloc(NGX_LUA_RE_MIN_JIT_STACK_SIZE, + size); - if (lua_isnil(L, -1)) { - lua_pop(L, 1); + ngx_http_lua_pcre_malloc_done(old_pool); - /* assuming named submatches are usually unique */ - lua_createtable(L, 1 /* narr */, 0 /* nrec */); - lua_pushstring(L, name); - lua_pushvalue(L, -2); /* big_tb cap small_tb key small_tb */ - lua_rawset(L, res_tb_idx); /* big_tb cap small_tb */ - len = 0; + if (lmcf->jit_stack == NULL) { + *errstr_size = ngx_snprintf(errstr, *errstr_size, + "pcre jit stack allocation failed") + - errstr; + return NGX_ERROR; + } - } else { - len = lua_objlen(L, -1); - } + return NGX_OK; - lua_pushvalue(L, -2); /* big_tb cap small_tb cap */ - lua_rawseti(L, -2, (int) len + 1); /* big_tb cap small_tb */ - lua_pop(L, 2); +#else /* LUA_HAVE_PCRE_JIT */ - } else { - lua_pushstring(L, name); /* big_tb cap key */ - lua_pushvalue(L, -2); /* big_tb cap key cap */ - lua_rawset(L, res_tb_idx); /* big_tb cap */ - lua_pop(L, 1); - } + *errstr_size = ngx_snprintf(errstr, *errstr_size, + "no pcre jit support found") + - errstr; + return NGX_ERROR; - dd("top 2: %d", lua_gettop(L)); - } +#endif /* LUA_HAVE_PCRE_JIT */ } -#ifndef NGX_LUA_NO_FFI_API ngx_http_lua_regex_t * ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, int flags, int pcre_opts, u_char *errstr, @@ -2540,11 +577,12 @@ ngx_http_lua_ffi_max_regex_cache_size(void) if (lmcf == NULL) { return 0; } + return (uint32_t) lmcf->regex_cache_max_entries; } -#endif /* NGX_LUA_NO_FFI_API */ #endif /* NGX_PCRE */ + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_regex.h b/src/ngx_http_lua_regex.h deleted file mode 100644 index 03dffb80b9..0000000000 --- a/src/ngx_http_lua_regex.h +++ /dev/null @@ -1,24 +0,0 @@ - -/* - * Copyright (C) Yichun Zhang (agentzh) - */ - - -#ifndef _NGX_HTTP_LUA_REGEX_H_INCLUDED_ -#define _NGX_HTTP_LUA_REGEX_H_INCLUDED_ - - -#include "ngx_http_lua_common.h" -#include "ngx_http_lua_script.h" - - -#if (NGX_PCRE) -void ngx_http_lua_inject_regex_api(lua_State *L); -ngx_int_t ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, - size_t *errstr_size); -#endif - - -#endif /* _NGX_HTTP_LUA_REGEX_H_INCLUDED_ */ - -/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_req_method.c b/src/ngx_http_lua_req_method.c index a4969b99f6..a41e5af593 100644 --- a/src/ngx_http_lua_req_method.c +++ b/src/ngx_http_lua_req_method.c @@ -10,142 +10,9 @@ #include "ddebug.h" -#include "ngx_http_lua_req_method.h" #include "ngx_http_lua_subrequest.h" -#include "ngx_http_lua_util.h" -static int ngx_http_lua_ngx_req_get_method(lua_State *L); -static int ngx_http_lua_ngx_req_set_method(lua_State *L); - - -void -ngx_http_lua_inject_req_method_api(lua_State *L) -{ - lua_pushcfunction(L, ngx_http_lua_ngx_req_get_method); - lua_setfield(L, -2, "get_method"); - - lua_pushcfunction(L, ngx_http_lua_ngx_req_set_method); - lua_setfield(L, -2, "set_method"); -} - - -static int -ngx_http_lua_ngx_req_get_method(lua_State *L) -{ - int n; - ngx_http_request_t *r; - - n = lua_gettop(L); - if (n != 0) { - return luaL_error(L, "only one argument expected but got %d", n); - } - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "request object not found"); - } - - ngx_http_lua_check_fake_request(L, r); - - lua_pushlstring(L, (char *) r->method_name.data, r->method_name.len); - return 1; -} - - -static int -ngx_http_lua_ngx_req_set_method(lua_State *L) -{ - int n; - int method; - ngx_http_request_t *r; - - n = lua_gettop(L); - if (n != 1) { - return luaL_error(L, "only one argument expected but got %d", n); - } - - method = luaL_checkint(L, 1); - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "request object not found"); - } - - ngx_http_lua_check_fake_request(L, r); - - switch (method) { - case NGX_HTTP_GET: - r->method_name = ngx_http_lua_get_method; - break; - - case NGX_HTTP_POST: - r->method_name = ngx_http_lua_post_method; - break; - - case NGX_HTTP_PUT: - r->method_name = ngx_http_lua_put_method; - break; - - case NGX_HTTP_HEAD: - r->method_name = ngx_http_lua_head_method; - break; - - case NGX_HTTP_DELETE: - r->method_name = ngx_http_lua_delete_method; - break; - - case NGX_HTTP_OPTIONS: - r->method_name = ngx_http_lua_options_method; - break; - - case NGX_HTTP_MKCOL: - r->method_name = ngx_http_lua_mkcol_method; - break; - - case NGX_HTTP_COPY: - r->method_name = ngx_http_lua_copy_method; - break; - - case NGX_HTTP_MOVE: - r->method_name = ngx_http_lua_move_method; - break; - - case NGX_HTTP_PROPFIND: - r->method_name = ngx_http_lua_propfind_method; - break; - - case NGX_HTTP_PROPPATCH: - r->method_name = ngx_http_lua_proppatch_method; - break; - - case NGX_HTTP_LOCK: - r->method_name = ngx_http_lua_lock_method; - break; - - case NGX_HTTP_UNLOCK: - r->method_name = ngx_http_lua_unlock_method; - break; - - case NGX_HTTP_PATCH: - r->method_name = ngx_http_lua_patch_method; - break; - - case NGX_HTTP_TRACE: - r->method_name = ngx_http_lua_trace_method; - break; - - default: - return luaL_error(L, "unsupported HTTP method: %d", method); - } - - r->method = method; - - return 0; -} - - -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_req_get_method(ngx_http_request_t *r) { @@ -247,7 +114,6 @@ ngx_http_lua_ffi_req_set_method(ngx_http_request_t *r, int method) r->method = method; return NGX_OK; } -#endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_req_method.h b/src/ngx_http_lua_req_method.h deleted file mode 100644 index f8aa1af39a..0000000000 --- a/src/ngx_http_lua_req_method.h +++ /dev/null @@ -1,19 +0,0 @@ - -/* - * Copyright (C) Yichun Zhang (agentzh) - */ - - -#ifndef _NGX_HTTP_LUA_METHOD_H_INCLUDED_ -#define _NGX_HTTP_LUA_METHOD_H_INCLUDED_ - - -#include "ngx_http_lua_common.h" - - -void ngx_http_lua_inject_req_method_api(lua_State *L); - - -#endif /* _NGX_HTTP_LUA_METHOD_H_INCLUDED_ */ - -/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 9e50b7b30e..38389a063a 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -7,9 +7,6 @@ */ -#ifndef NGX_LUA_NO_FFI_API - - #ifndef DDEBUG #define DDEBUG 0 #endif @@ -577,6 +574,4 @@ ngx_http_lua_ffi_sema_gc(ngx_http_lua_sema_t *sem) } -#endif /* NGX_LUA_NO_FFI_API */ - /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_semaphore.h b/src/ngx_http_lua_semaphore.h index 65ba8fff34..826f86a3f6 100644 --- a/src/ngx_http_lua_semaphore.h +++ b/src/ngx_http_lua_semaphore.h @@ -41,11 +41,9 @@ typedef struct ngx_http_lua_sema_s { } ngx_http_lua_sema_t; -#ifndef NGX_LUA_NO_FFI_API void ngx_http_lua_sema_mm_cleanup(void *data); ngx_int_t ngx_http_lua_sema_mm_init(ngx_conf_t *cf, ngx_http_lua_main_conf_t *lmcf); -#endif #endif /* _NGX_HTTP_LUA_SEMAPHORE_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_setby.c b/src/ngx_http_lua_setby.c index d4288b50c2..f00468c903 100644 --- a/src/ngx_http_lua_setby.c +++ b/src/ngx_http_lua_setby.c @@ -15,10 +15,7 @@ #include "ngx_http_lua_exception.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_pcrefix.h" -#include "ngx_http_lua_time.h" #include "ngx_http_lua_log.h" -#include "ngx_http_lua_regex.h" -#include "ngx_http_lua_variable.h" #include "ngx_http_lua_string.h" #include "ngx_http_lua_misc.h" #include "ngx_http_lua_consts.h" diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 0cda3b7b18..85d36e3de3 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -15,23 +15,11 @@ #include "ngx_http_lua_api.h" -static int ngx_http_lua_shdict_set(lua_State *L); -static int ngx_http_lua_shdict_safe_set(lua_State *L); -static int ngx_http_lua_shdict_get(lua_State *L); -static int ngx_http_lua_shdict_get_stale(lua_State *L); -static int ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale); static int ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n); static ngx_int_t ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, u_char *kdata, size_t klen, ngx_http_lua_shdict_node_t **sdp); -static int ngx_http_lua_shdict_set_helper(lua_State *L, int flags); -static int ngx_http_lua_shdict_add(lua_State *L); -static int ngx_http_lua_shdict_safe_add(lua_State *L); -static int ngx_http_lua_shdict_replace(lua_State *L); -static int ngx_http_lua_shdict_incr(lua_State *L); -static int ngx_http_lua_shdict_delete(lua_State *L); -static int ngx_http_lua_shdict_flush_all(lua_State *L); static int ngx_http_lua_shdict_flush_expired(lua_State *L); static int ngx_http_lua_shdict_get_keys(lua_State *L); static int ngx_http_lua_shdict_lpush(lua_State *L); @@ -331,34 +319,7 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_createtable(L, 0, lmcf->shdict_zones->nelts /* nrec */); /* ngx.shared */ - lua_createtable(L, 0 /* narr */, 18 /* nrec */); /* shared mt */ - - lua_pushcfunction(L, ngx_http_lua_shdict_get); - lua_setfield(L, -2, "get"); - - lua_pushcfunction(L, ngx_http_lua_shdict_get_stale); - lua_setfield(L, -2, "get_stale"); - - lua_pushcfunction(L, ngx_http_lua_shdict_set); - lua_setfield(L, -2, "set"); - - lua_pushcfunction(L, ngx_http_lua_shdict_safe_set); - lua_setfield(L, -2, "safe_set"); - - lua_pushcfunction(L, ngx_http_lua_shdict_add); - lua_setfield(L, -2, "add"); - - lua_pushcfunction(L, ngx_http_lua_shdict_safe_add); - lua_setfield(L, -2, "safe_add"); - - lua_pushcfunction(L, ngx_http_lua_shdict_replace); - lua_setfield(L, -2, "replace"); - - lua_pushcfunction(L, ngx_http_lua_shdict_incr); - lua_setfield(L, -2, "incr"); - - lua_pushcfunction(L, ngx_http_lua_shdict_delete); - lua_setfield(L, -2, "delete"); + lua_createtable(L, 0 /* narr */, 22 /* nrec */); /* shared mt */ lua_pushcfunction(L, ngx_http_lua_shdict_lpush); lua_setfield(L, -2, "lpush"); @@ -375,9 +336,6 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_pushcfunction(L, ngx_http_lua_shdict_llen); lua_setfield(L, -2, "llen"); - lua_pushcfunction(L, ngx_http_lua_shdict_flush_all); - lua_setfield(L, -2, "flush_all"); - lua_pushcfunction(L, ngx_http_lua_shdict_flush_expired); lua_setfield(L, -2, "flush_expired"); @@ -416,20 +374,6 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) } -static int -ngx_http_lua_shdict_get(lua_State *L) -{ - return ngx_http_lua_shdict_get_helper(L, 0 /* stale */); -} - - -static int -ngx_http_lua_shdict_get_stale(lua_State *L) -{ - return ngx_http_lua_shdict_get_helper(L, 1 /* stale */); -} - - static ngx_inline ngx_shm_zone_t * ngx_http_lua_shdict_get_zone(lua_State *L, int index) { @@ -446,1062 +390,197 @@ ngx_http_lua_shdict_get_zone(lua_State *L, int index) zone = *zone_udata; return zone; -} - - -static int -ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) -{ - int n; - ngx_str_t name; - ngx_str_t key; - uint32_t hash; - ngx_int_t rc; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_http_lua_shdict_node_t *sd; - ngx_str_t value; - int value_type; - double num; - u_char c; - ngx_shm_zone_t *zone; - uint32_t user_flags = 0; - - n = lua_gettop(L); - - if (n != 2) { - return luaL_error(L, "expecting exactly two arguments, " - "but only seen %d", n); - } - - if (lua_type(L, 1) != LUA_TTABLE) { - return luaL_error(L, "bad \"zone\" argument"); - } - - zone = ngx_http_lua_shdict_get_zone(L, 1); - if (zone == NULL) { - return luaL_error(L, "bad \"zone\" argument"); - } - - ctx = zone->data; - name = ctx->name; - - if (lua_isnil(L, 2)) { - lua_pushnil(L); - lua_pushliteral(L, "nil key"); - return 2; - } - - key.data = (u_char *) luaL_checklstring(L, 2, &key.len); - - if (key.len == 0) { - lua_pushnil(L); - lua_pushliteral(L, "empty key"); - return 2; - } - - if (key.len > 65535) { - lua_pushnil(L); - lua_pushliteral(L, "key too long"); - return 2; - } - - hash = ngx_crc32_short(key.data, key.len); - -#if (NGX_DEBUG) - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "fetching key \"%V\" in shared dict \"%V\"", &key, &name); -#endif /* NGX_DEBUG */ - - ngx_shmtx_lock(&ctx->shpool->mutex); - -#if 1 - if (!get_stale) { - ngx_http_lua_shdict_expire(ctx, 1); - } -#endif - - rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); - - dd("shdict lookup returns %d", (int) rc); - - if (rc == NGX_DECLINED || (rc == NGX_DONE && !get_stale)) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - lua_pushnil(L); - return 1; - } - - /* rc == NGX_OK || (rc == NGX_DONE && get_stale) */ - - value_type = sd->value_type; - - dd("data: %p", sd->data); - dd("key len: %d", (int) sd->key_len); - - value.data = sd->data + sd->key_len; - value.len = (size_t) sd->value_len; - - switch (value_type) { - - case SHDICT_TSTRING: - - lua_pushlstring(L, (char *) value.data, value.len); - break; - - case SHDICT_TNUMBER: - - if (value.len != sizeof(double)) { - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - return luaL_error(L, "bad lua number value size found for key %s " - "in shared_dict %s: %lu", key.data, name.data, - (unsigned long) value.len); - } - - ngx_memcpy(&num, value.data, sizeof(double)); - - lua_pushnumber(L, num); - break; - - case SHDICT_TBOOLEAN: - - if (value.len != sizeof(u_char)) { - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - return luaL_error(L, "bad lua boolean value size found for key %s " - "in shared_dict %s: %lu", key.data, name.data, - (unsigned long) value.len); - } - - c = *value.data; - - lua_pushboolean(L, c ? 1 : 0); - break; - - case SHDICT_TLIST: - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushnil(L); - lua_pushliteral(L, "value is a list"); - return 2; - - default: - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - return luaL_error(L, "bad value type found for key %s in " - "shared_dict %s: %d", key.data, name.data, - value_type); - } - - user_flags = sd->user_flags; - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - if (get_stale) { - - /* always return value, flags, stale */ - - if (user_flags) { - lua_pushinteger(L, (lua_Integer) user_flags); - - } else { - lua_pushnil(L); - } - - lua_pushboolean(L, rc == NGX_DONE); - return 3; - } - - if (user_flags) { - lua_pushinteger(L, (lua_Integer) user_flags); - return 2; - } - - return 1; -} - - -static int -ngx_http_lua_shdict_delete(lua_State *L) -{ - int n; - - n = lua_gettop(L); - - if (n != 2) { - return luaL_error(L, "expecting 2 arguments, " - "but only seen %d", n); - } - - lua_pushnil(L); - - return ngx_http_lua_shdict_set_helper(L, 0); -} - - -static int -ngx_http_lua_shdict_flush_all(lua_State *L) -{ - ngx_queue_t *q; - ngx_http_lua_shdict_node_t *sd; - int n; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_shm_zone_t *zone; - - n = lua_gettop(L); - - if (n != 1) { - return luaL_error(L, "expecting 1 argument, but seen %d", n); - } - - luaL_checktype(L, 1, LUA_TTABLE); - - zone = ngx_http_lua_shdict_get_zone(L, 1); - if (zone == NULL) { - return luaL_error(L, "bad user data for the ngx_shm_zone_t pointer"); - } - - ctx = zone->data; - - ngx_shmtx_lock(&ctx->shpool->mutex); - - for (q = ngx_queue_head(&ctx->sh->lru_queue); - q != ngx_queue_sentinel(&ctx->sh->lru_queue); - q = ngx_queue_next(q)) - { - sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - sd->expires = 1; - } - - ngx_http_lua_shdict_expire(ctx, 0); - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - return 0; -} - - -static int -ngx_http_lua_shdict_flush_expired(lua_State *L) -{ - ngx_queue_t *q, *prev, *list_queue, *lq; - ngx_http_lua_shdict_node_t *sd; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_shm_zone_t *zone; - ngx_time_t *tp; - int freed = 0; - int attempts = 0; - ngx_rbtree_node_t *node; - uint64_t now; - int n; - ngx_http_lua_shdict_list_node_t *lnode; - - n = lua_gettop(L); - - if (n != 1 && n != 2) { - return luaL_error(L, "expecting 1 or 2 argument(s), but saw %d", n); - } - - luaL_checktype(L, 1, LUA_TTABLE); - - zone = ngx_http_lua_shdict_get_zone(L, 1); - if (zone == NULL) { - return luaL_error(L, "bad user data for the ngx_shm_zone_t pointer"); - } - - if (n == 2) { - attempts = luaL_checkint(L, 2); - } - - ctx = zone->data; - - ngx_shmtx_lock(&ctx->shpool->mutex); - - if (ngx_queue_empty(&ctx->sh->lru_queue)) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - lua_pushnumber(L, 0); - return 1; - } - - tp = ngx_timeofday(); - - now = (uint64_t) tp->sec * 1000 + tp->msec; - - q = ngx_queue_last(&ctx->sh->lru_queue); - - while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { - prev = ngx_queue_prev(q); - - sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - - if (sd->expires != 0 && sd->expires <= now) { - - if (sd->value_type == SHDICT_TLIST) { - list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); - - for (lq = ngx_queue_head(list_queue); - lq != ngx_queue_sentinel(list_queue); - lq = ngx_queue_next(lq)) - { - lnode = ngx_queue_data(lq, ngx_http_lua_shdict_list_node_t, - queue); - - ngx_slab_free_locked(ctx->shpool, lnode); - } - } - - ngx_queue_remove(q); - - node = (ngx_rbtree_node_t *) - ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); - - ngx_rbtree_delete(&ctx->sh->rbtree, node); - ngx_slab_free_locked(ctx->shpool, node); - freed++; - - if (attempts && freed == attempts) { - break; - } - } - - q = prev; - } - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushnumber(L, freed); - return 1; -} - - -/* - * This trades CPU for memory. This is potentially slow. O(2n) - */ - -static int -ngx_http_lua_shdict_get_keys(lua_State *L) -{ - ngx_queue_t *q, *prev; - ngx_http_lua_shdict_node_t *sd; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_shm_zone_t *zone; - ngx_time_t *tp; - int total = 0; - int attempts = 1024; - uint64_t now; - int n; - - n = lua_gettop(L); - - if (n != 1 && n != 2) { - return luaL_error(L, "expecting 1 or 2 argument(s), " - "but saw %d", n); - } - - luaL_checktype(L, 1, LUA_TTABLE); - - zone = ngx_http_lua_shdict_get_zone(L, 1); - if (zone == NULL) { - return luaL_error(L, "bad user data for the ngx_shm_zone_t pointer"); - } - - if (n == 2) { - attempts = luaL_checkint(L, 2); - } - - ctx = zone->data; - - ngx_shmtx_lock(&ctx->shpool->mutex); - - if (ngx_queue_empty(&ctx->sh->lru_queue)) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - lua_createtable(L, 0, 0); - return 1; - } - - tp = ngx_timeofday(); - - now = (uint64_t) tp->sec * 1000 + tp->msec; - - /* first run through: get total number of elements we need to allocate */ - - q = ngx_queue_last(&ctx->sh->lru_queue); - - while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { - prev = ngx_queue_prev(q); - - sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - - if (sd->expires == 0 || sd->expires > now) { - total++; - if (attempts && total == attempts) { - break; - } - } - - q = prev; - } - - lua_createtable(L, total, 0); - - /* second run through: add keys to table */ - - total = 0; - q = ngx_queue_last(&ctx->sh->lru_queue); - - while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { - prev = ngx_queue_prev(q); - - sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - - if (sd->expires == 0 || sd->expires > now) { - lua_pushlstring(L, (char *) sd->data, sd->key_len); - lua_rawseti(L, -2, ++total); - if (attempts && total == attempts) { - break; - } - } - - q = prev; - } - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - /* table is at top of stack */ - return 1; -} - - -static int -ngx_http_lua_shdict_add(lua_State *L) -{ - return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_ADD); -} - - -static int -ngx_http_lua_shdict_safe_add(lua_State *L) -{ - return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_ADD - |NGX_HTTP_LUA_SHDICT_SAFE_STORE); -} - - -static int -ngx_http_lua_shdict_replace(lua_State *L) -{ - return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_REPLACE); -} - - -static int -ngx_http_lua_shdict_set(lua_State *L) -{ - return ngx_http_lua_shdict_set_helper(L, 0); -} - - -static int -ngx_http_lua_shdict_safe_set(lua_State *L) -{ - return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_SAFE_STORE); -} - - -static int -ngx_http_lua_shdict_set_helper(lua_State *L, int flags) -{ - int i, n; - ngx_str_t key; - uint32_t hash; - ngx_int_t rc; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_http_lua_shdict_node_t *sd; - ngx_str_t value; - int value_type; - double num; - u_char c; - lua_Number exptime = 0; - u_char *p; - ngx_rbtree_node_t *node; - ngx_time_t *tp; - ngx_shm_zone_t *zone; - int forcible = 0; - /* indicates whether to foricibly override other - * valid entries */ - int32_t user_flags = 0; - ngx_queue_t *queue, *q; - - n = lua_gettop(L); - - if (n != 3 && n != 4 && n != 5) { - return luaL_error(L, "expecting 3, 4 or 5 arguments, " - "but only seen %d", n); - } - - if (lua_type(L, 1) != LUA_TTABLE) { - return luaL_error(L, "bad \"zone\" argument"); - } - - zone = ngx_http_lua_shdict_get_zone(L, 1); - if (zone == NULL) { - return luaL_error(L, "bad \"zone\" argument"); - } - - ctx = zone->data; - - if (lua_isnil(L, 2)) { - lua_pushnil(L); - lua_pushliteral(L, "nil key"); - return 2; - } - - key.data = (u_char *) luaL_checklstring(L, 2, &key.len); - - if (key.len == 0) { - lua_pushnil(L); - lua_pushliteral(L, "empty key"); - return 2; - } - - if (key.len > 65535) { - lua_pushnil(L); - lua_pushliteral(L, "key too long"); - return 2; - } - - hash = ngx_crc32_short(key.data, key.len); - - value_type = lua_type(L, 3); - - switch (value_type) { - - case SHDICT_TSTRING: - value.data = (u_char *) lua_tolstring(L, 3, &value.len); - break; - - case SHDICT_TNUMBER: - value.len = sizeof(double); - num = lua_tonumber(L, 3); - value.data = (u_char *) # - break; - - case SHDICT_TBOOLEAN: - value.len = sizeof(u_char); - c = lua_toboolean(L, 3) ? 1 : 0; - value.data = &c; - break; - - case LUA_TNIL: - if (flags & (NGX_HTTP_LUA_SHDICT_ADD|NGX_HTTP_LUA_SHDICT_REPLACE)) { - lua_pushnil(L); - lua_pushliteral(L, "attempt to add or replace nil values"); - return 2; - } - - ngx_str_null(&value); - break; - - default: - lua_pushnil(L); - lua_pushliteral(L, "bad value type"); - return 2; - } - - if (n >= 4) { - exptime = luaL_checknumber(L, 4); - if (exptime < 0) { - return luaL_error(L, "bad \"exptime\" argument"); - } - } - - if (n == 5) { - user_flags = (uint32_t) luaL_checkinteger(L, 5); - } - - ngx_shmtx_lock(&ctx->shpool->mutex); - -#if 1 - ngx_http_lua_shdict_expire(ctx, 1); -#endif - - rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); - - dd("shdict lookup returned %d", (int) rc); - - if (flags & NGX_HTTP_LUA_SHDICT_REPLACE) { - - if (rc == NGX_DECLINED || rc == NGX_DONE) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 0); - lua_pushliteral(L, "not found"); - lua_pushboolean(L, forcible); - return 3; - } - - /* rc == NGX_OK */ - - goto replace; - } - - if (flags & NGX_HTTP_LUA_SHDICT_ADD) { - - if (rc == NGX_OK) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 0); - lua_pushliteral(L, "exists"); - lua_pushboolean(L, forcible); - return 3; - } - - if (rc == NGX_DONE) { - /* exists but expired */ - - dd("go to replace"); - goto replace; - } - - /* rc == NGX_DECLINED */ - - dd("go to insert"); - goto insert; - } - - if (rc == NGX_OK || rc == NGX_DONE) { - - if (value_type == LUA_TNIL) { - goto remove; - } - -replace: - - if (value.data - && value.len == (size_t) sd->value_len - && sd->value_type != SHDICT_TLIST) - { - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict set: found old entry and value " - "size matched, reusing it"); - - ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); - - sd->key_len = (u_short) key.len; - - if (exptime > 0) { - tp = ngx_timeofday(); - sd->expires = (uint64_t) tp->sec * 1000 + tp->msec - + (uint64_t) (exptime * 1000); - - } else { - sd->expires = 0; - } - - sd->user_flags = user_flags; - - sd->value_len = (uint32_t) value.len; - - dd("setting value type to %d", value_type); - - sd->value_type = (uint8_t) value_type; - - p = ngx_copy(sd->data, key.data, key.len); - ngx_memcpy(p, value.data, value.len); - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 1); - lua_pushnil(L); - lua_pushboolean(L, forcible); - return 3; - } - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict set: found old entry but value size " - "NOT matched, removing it first"); - -remove: - - if (sd->value_type == SHDICT_TLIST) { - queue = ngx_http_lua_shdict_get_list_head(sd, key.len); - - for (q = ngx_queue_head(queue); - q != ngx_queue_sentinel(queue); - q = ngx_queue_next(q)) - { - p = (u_char *) ngx_queue_data(q, - ngx_http_lua_shdict_list_node_t, - queue); - - ngx_slab_free_locked(ctx->shpool, p); - } - } - - ngx_queue_remove(&sd->queue); - - node = (ngx_rbtree_node_t *) - ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); - - ngx_rbtree_delete(&ctx->sh->rbtree, node); - - ngx_slab_free_locked(ctx->shpool, node); - - } - -insert: - - /* rc == NGX_DECLINED or value size unmatch */ - - if (value.data == NULL) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 1); - lua_pushnil(L); - lua_pushboolean(L, 0); - return 3; - } - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict set: creating a new entry"); - - n = offsetof(ngx_rbtree_node_t, color) - + offsetof(ngx_http_lua_shdict_node_t, data) - + key.len - + value.len; - - dd("overhead = %d", (int) (offsetof(ngx_rbtree_node_t, color) - + offsetof(ngx_http_lua_shdict_node_t, data))); - - node = ngx_slab_alloc_locked(ctx->shpool, n); - - if (node == NULL) { - - if (flags & NGX_HTTP_LUA_SHDICT_SAFE_STORE) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 0); - lua_pushliteral(L, "no memory"); - return 2; - } - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict set: overriding non-expired items " - "due to memory shortage for entry \"%V\"", &key); - - for (i = 0; i < 30; i++) { - if (ngx_http_lua_shdict_expire(ctx, 0) == 0) { - break; - } - - forcible = 1; - - node = ngx_slab_alloc_locked(ctx->shpool, n); - if (node != NULL) { - goto allocated; - } - } - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 0); - lua_pushliteral(L, "no memory"); - lua_pushboolean(L, forcible); - return 3; - } - -allocated: - - sd = (ngx_http_lua_shdict_node_t *) &node->color; - - node->key = hash; - sd->key_len = (u_short) key.len; - - if (exptime > 0) { - tp = ngx_timeofday(); - sd->expires = (uint64_t) tp->sec * 1000 + tp->msec - + (uint64_t) (exptime * 1000); - - } else { - sd->expires = 0; - } - - sd->user_flags = user_flags; - - sd->value_len = (uint32_t) value.len; - - dd("setting value type to %d", value_type); - - sd->value_type = (uint8_t) value_type; - - p = ngx_copy(sd->data, key.data, key.len); - ngx_memcpy(p, value.data, value.len); - - ngx_rbtree_insert(&ctx->sh->rbtree, node); - - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 1); - lua_pushnil(L); - lua_pushboolean(L, forcible); - return 3; -} - - -static int -ngx_http_lua_shdict_incr(lua_State *L) -{ - int i, n; - ngx_str_t key; - uint32_t hash; - ngx_int_t rc; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_http_lua_shdict_node_t *sd; - double num; - double init = 0; - u_char *p; - ngx_shm_zone_t *zone; - double value; - ngx_rbtree_node_t *node; - /* indicates whether to foricibly override other - * valid entries */ - int forcible = 0; - ngx_queue_t *queue, *q; +} + + +static int +ngx_http_lua_shdict_flush_expired(lua_State *L) +{ + ngx_queue_t *q, *prev, *list_queue, *lq; + ngx_http_lua_shdict_node_t *sd; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_shm_zone_t *zone; + ngx_time_t *tp; + int freed = 0; + int attempts = 0; + ngx_rbtree_node_t *node; + uint64_t now; + int n; + ngx_http_lua_shdict_list_node_t *lnode; n = lua_gettop(L); - if (n != 3 && n != 4) { - return luaL_error(L, "expecting 3 or 4 arguments, but only seen %d", n); + if (n != 1 && n != 2) { + return luaL_error(L, "expecting 1 or 2 argument(s), but saw %d", n); } - if (lua_type(L, 1) != LUA_TTABLE) { - return luaL_error(L, "bad \"zone\" argument"); - } + luaL_checktype(L, 1, LUA_TTABLE); zone = ngx_http_lua_shdict_get_zone(L, 1); if (zone == NULL) { return luaL_error(L, "bad user data for the ngx_shm_zone_t pointer"); } - ctx = zone->data; - - if (lua_isnil(L, 2)) { - lua_pushnil(L); - lua_pushliteral(L, "nil key"); - return 2; - } - - key.data = (u_char *) luaL_checklstring(L, 2, &key.len); - - if (key.len == 0) { - lua_pushnil(L); - lua_pushliteral(L, "empty key"); - return 2; - } - - if (key.len > 65535) { - lua_pushnil(L); - lua_pushliteral(L, "key too long"); - return 2; + if (n == 2) { + attempts = luaL_checkint(L, 2); } - hash = ngx_crc32_short(key.data, key.len); + ctx = zone->data; - value = luaL_checknumber(L, 3); + ngx_shmtx_lock(&ctx->shpool->mutex); - if (n == 4) { - init = luaL_checknumber(L, 4); + if (ngx_queue_empty(&ctx->sh->lru_queue)) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + lua_pushnumber(L, 0); + return 1; } - dd("looking up key %.*s in shared dict %.*s", (int) key.len, key.data, - (int) ctx->name.len, ctx->name.data); - - ngx_shmtx_lock(&ctx->shpool->mutex); + tp = ngx_timeofday(); -#if 1 - ngx_http_lua_shdict_expire(ctx, 1); -#endif + now = (uint64_t) tp->sec * 1000 + tp->msec; - rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); + q = ngx_queue_last(&ctx->sh->lru_queue); - dd("shdict lookup returned %d", (int) rc); + while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { + prev = ngx_queue_prev(q); - if (rc == NGX_DECLINED || rc == NGX_DONE) { + sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - if (n == 3) { - ngx_shmtx_unlock(&ctx->shpool->mutex); + if (sd->expires != 0 && sd->expires <= now) { - lua_pushnil(L); - lua_pushliteral(L, "not found"); - return 2; - } + if (sd->value_type == SHDICT_TLIST) { + list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); - /* add value */ - num = value + init; + for (lq = ngx_queue_head(list_queue); + lq != ngx_queue_sentinel(list_queue); + lq = ngx_queue_next(lq)) + { + lnode = ngx_queue_data(lq, ngx_http_lua_shdict_list_node_t, + queue); - if (rc == NGX_DONE) { + ngx_slab_free_locked(ctx->shpool, lnode); + } + } - /* found an expired item */ + ngx_queue_remove(q); - if ((size_t) sd->value_len == sizeof(double) - && sd->value_type != SHDICT_TLIST) - { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict incr: found old entry and " - "value size matched, reusing it"); + node = (ngx_rbtree_node_t *) + ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); - ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); + ngx_rbtree_delete(&ctx->sh->rbtree, node); + ngx_slab_free_locked(ctx->shpool, node); + freed++; - dd("go to setvalue"); - goto setvalue; + if (attempts && freed == attempts) { + break; } - - dd("go to remove"); - goto remove; } - dd("go to insert"); - goto insert; - } - - /* rc == NGX_OK */ - - if (sd->value_type != SHDICT_TNUMBER || sd->value_len != sizeof(double)) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushnil(L); - lua_pushliteral(L, "not a number"); - return 2; + q = prev; } - ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); - - dd("setting value type to %d", (int) sd->value_type); - - p = sd->data + key.len; - - ngx_memcpy(&num, p, sizeof(double)); - num += value; - - ngx_memcpy(p, (double *) &num, sizeof(double)); - ngx_shmtx_unlock(&ctx->shpool->mutex); - lua_pushnumber(L, num); - lua_pushnil(L); - return 2; + lua_pushnumber(L, freed); + return 1; +} -remove: - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict incr: found old entry but value size " - "NOT matched, removing it first"); +/* + * This trades CPU for memory. This is potentially slow. O(2n) + */ - if (sd->value_type == SHDICT_TLIST) { - queue = ngx_http_lua_shdict_get_list_head(sd, key.len); +static int +ngx_http_lua_shdict_get_keys(lua_State *L) +{ + ngx_queue_t *q, *prev; + ngx_http_lua_shdict_node_t *sd; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_shm_zone_t *zone; + ngx_time_t *tp; + int total = 0; + int attempts = 1024; + uint64_t now; + int n; - for (q = ngx_queue_head(queue); - q != ngx_queue_sentinel(queue); - q = ngx_queue_next(q)) - { - p = (u_char *) ngx_queue_data(q, - ngx_http_lua_shdict_list_node_t, - queue); + n = lua_gettop(L); - ngx_slab_free_locked(ctx->shpool, p); - } + if (n != 1 && n != 2) { + return luaL_error(L, "expecting 1 or 2 argument(s), " + "but saw %d", n); } - ngx_queue_remove(&sd->queue); + luaL_checktype(L, 1, LUA_TTABLE); - node = (ngx_rbtree_node_t *) - ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); + zone = ngx_http_lua_shdict_get_zone(L, 1); + if (zone == NULL) { + return luaL_error(L, "bad user data for the ngx_shm_zone_t pointer"); + } - ngx_rbtree_delete(&ctx->sh->rbtree, node); + if (n == 2) { + attempts = luaL_checkint(L, 2); + } - ngx_slab_free_locked(ctx->shpool, node); + ctx = zone->data; -insert: + ngx_shmtx_lock(&ctx->shpool->mutex); - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict incr: creating a new entry"); + if (ngx_queue_empty(&ctx->sh->lru_queue)) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + lua_createtable(L, 0, 0); + return 1; + } - n = offsetof(ngx_rbtree_node_t, color) - + offsetof(ngx_http_lua_shdict_node_t, data) - + key.len - + sizeof(double); + tp = ngx_timeofday(); - node = ngx_slab_alloc_locked(ctx->shpool, n); + now = (uint64_t) tp->sec * 1000 + tp->msec; - if (node == NULL) { + /* first run through: get total number of elements we need to allocate */ - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict incr: overriding non-expired items " - "due to memory shortage for entry \"%V\"", &key); + q = ngx_queue_last(&ctx->sh->lru_queue); - for (i = 0; i < 30; i++) { - if (ngx_http_lua_shdict_expire(ctx, 0) == 0) { - break; - } + while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { + prev = ngx_queue_prev(q); - forcible = 1; + sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - node = ngx_slab_alloc_locked(ctx->shpool, n); - if (node != NULL) { - goto allocated; + if (sd->expires == 0 || sd->expires > now) { + total++; + if (attempts && total == attempts) { + break; } } - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 0); - lua_pushliteral(L, "no memory"); - lua_pushboolean(L, forcible); - return 3; + q = prev; } -allocated: - - sd = (ngx_http_lua_shdict_node_t *) &node->color; - - node->key = hash; - - sd->key_len = (u_short) key.len; - - sd->value_len = (uint32_t) sizeof(double); - - ngx_rbtree_insert(&ctx->sh->rbtree, node); - - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); + lua_createtable(L, total, 0); -setvalue: + /* second run through: add keys to table */ - sd->user_flags = 0; + total = 0; + q = ngx_queue_last(&ctx->sh->lru_queue); - sd->expires = 0; + while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { + prev = ngx_queue_prev(q); - dd("setting value type to %d", LUA_TNUMBER); + sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); - sd->value_type = (uint8_t) LUA_TNUMBER; + if (sd->expires == 0 || sd->expires > now) { + lua_pushlstring(L, (char *) sd->data, sd->key_len); + lua_rawseti(L, -2, ++total); + if (attempts && total == attempts) { + break; + } + } - p = ngx_copy(sd->data, key.data, key.len); - ngx_memcpy(p, (double *) &num, sizeof(double)); + q = prev; + } ngx_shmtx_unlock(&ctx->shpool->mutex); - lua_pushnumber(L, num); - lua_pushnil(L); - lua_pushboolean(L, forcible); - return 3; + /* table is at top of stack */ + return 1; } @@ -2217,7 +1296,6 @@ ngx_http_lua_find_zone(u_char *name_data, size_t name_len) } -#ifndef NGX_LUA_NO_FFI_API ngx_shm_zone_t * ngx_http_lua_ffi_shdict_udata_to_zone(void *zone_udata) { @@ -3021,7 +2099,4 @@ ngx_http_lua_ffi_shdict_free_space(ngx_shm_zone_t *zone) # endif /* nginx_version >= 1011007 */ -#endif /* NGX_LUA_NO_FFI_API */ - - /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 5ee1657070..21ac296fe5 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -540,8 +540,6 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) } -#ifndef NGX_LUA_NO_FFI_API - int ngx_http_lua_ffi_ssl_get_tls1_version(ngx_http_request_t *r, char **err) { @@ -1311,7 +1309,4 @@ ngx_http_lua_ffi_set_priv_key(ngx_http_request_t *r, } -#endif /* NGX_LUA_NO_FFI_API */ - - #endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_ssl_ocsp.c b/src/ngx_http_lua_ssl_ocsp.c index 9ec8b5098c..d1805b23dd 100644 --- a/src/ngx_http_lua_ssl_ocsp.c +++ b/src/ngx_http_lua_ssl_ocsp.c @@ -16,8 +16,6 @@ #include "ngx_http_lua_common.h" -#ifndef NGX_LUA_NO_FFI_API - #ifdef NGX_HTTP_LUA_USE_OCSP static int ngx_http_lua_ssl_empty_status_callback(ngx_ssl_conn_t *ssl_conn, void *data); @@ -500,7 +498,5 @@ ngx_http_lua_ffi_ssl_set_ocsp_status_resp(ngx_http_request_t *r, #endif /* NGX_HTTP_LUA_USE_OCSP */ } -#endif /* NGX_LUA_NO_FFI_API */ - #endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index bd72eb813f..4b15d61dd3 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -562,8 +562,6 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) } -#ifndef NGX_LUA_NO_FFI_API - /* de-serialized a SSL session and set it back to the request at lua context */ int ngx_http_lua_ffi_ssl_set_serialized_session(ngx_http_request_t *r, @@ -615,7 +613,5 @@ ngx_http_lua_ffi_ssl_set_serialized_session(ngx_http_request_t *r, return NGX_OK; } -#endif /* NGX_LUA_NO_FFI_API */ - #endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 3569bf6c27..99662b3a3e 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -433,8 +433,6 @@ ngx_http_lua_ssl_sess_store_by_chunk(lua_State *L, ngx_http_request_t *r) } -#ifndef NGX_LUA_NO_FFI_API - /* serialize a session from lua context into buf. * the memory allocation of buf should be handled externally. */ int @@ -620,7 +618,5 @@ ngx_http_lua_ffi_ssl_get_session_id_size(ngx_http_request_t *r, return 2 * cctx->session_id.len; } -#endif /* NGX_LUA_NO_FFI_API */ - #endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index bdadeb47f0..7092ff02f4 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -27,25 +27,9 @@ #endif -#ifndef SHA_DIGEST_LENGTH -#define SHA_DIGEST_LENGTH 20 -#endif - - static uintptr_t ngx_http_lua_ngx_escape_sql_str(u_char *dst, u_char *src, - size_t size); -static int ngx_http_lua_ngx_escape_uri(lua_State *L); -static int ngx_http_lua_ngx_unescape_uri(lua_State *L); + size_t size); static int ngx_http_lua_ngx_quote_sql_str(lua_State *L); -static int ngx_http_lua_ngx_md5(lua_State *L); -static int ngx_http_lua_ngx_md5_bin(lua_State *L); - -#if (NGX_HAVE_SHA1) -static int ngx_http_lua_ngx_sha1_bin(lua_State *L); -#endif - -static int ngx_http_lua_ngx_decode_base64(lua_State *L); -static int ngx_http_lua_ngx_encode_base64(lua_State *L); static int ngx_http_lua_ngx_crc32_short(lua_State *L); static int ngx_http_lua_ngx_crc32_long(lua_State *L); static int ngx_http_lua_ngx_encode_args(lua_State *L); @@ -58,12 +42,6 @@ static int ngx_http_lua_ngx_hmac_sha1(lua_State *L); void ngx_http_lua_inject_string_api(lua_State *L) { - lua_pushcfunction(L, ngx_http_lua_ngx_escape_uri); - lua_setfield(L, -2, "escape_uri"); - - lua_pushcfunction(L, ngx_http_lua_ngx_unescape_uri); - lua_setfield(L, -2, "unescape_uri"); - lua_pushcfunction(L, ngx_http_lua_ngx_encode_args); lua_setfield(L, -2, "encode_args"); @@ -73,23 +51,6 @@ ngx_http_lua_inject_string_api(lua_State *L) lua_pushcfunction(L, ngx_http_lua_ngx_quote_sql_str); lua_setfield(L, -2, "quote_sql_str"); - lua_pushcfunction(L, ngx_http_lua_ngx_decode_base64); - lua_setfield(L, -2, "decode_base64"); - - lua_pushcfunction(L, ngx_http_lua_ngx_encode_base64); - lua_setfield(L, -2, "encode_base64"); - - lua_pushcfunction(L, ngx_http_lua_ngx_md5_bin); - lua_setfield(L, -2, "md5_bin"); - - lua_pushcfunction(L, ngx_http_lua_ngx_md5); - lua_setfield(L, -2, "md5"); - -#if (NGX_HAVE_SHA1) - lua_pushcfunction(L, ngx_http_lua_ngx_sha1_bin); - lua_setfield(L, -2, "sha1_bin"); -#endif - lua_pushcfunction(L, ngx_http_lua_ngx_crc32_short); lua_setfield(L, -2, "crc32_short"); @@ -103,75 +64,6 @@ ngx_http_lua_inject_string_api(lua_State *L) } -static int -ngx_http_lua_ngx_escape_uri(lua_State *L) -{ - size_t len, dlen; - uintptr_t escape; - u_char *src, *dst; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - if (lua_isnil(L, 1)) { - lua_pushliteral(L, ""); - return 1; - } - - src = (u_char *) luaL_checklstring(L, 1, &len); - - if (len == 0) { - return 1; - } - - escape = 2 * ngx_http_lua_escape_uri(NULL, src, len, - NGX_ESCAPE_URI_COMPONENT); - - if (escape) { - dlen = escape + len; - dst = lua_newuserdata(L, dlen); - ngx_http_lua_escape_uri(dst, src, len, NGX_ESCAPE_URI_COMPONENT); - lua_pushlstring(L, (char *) dst, dlen); - } - - return 1; -} - - -static int -ngx_http_lua_ngx_unescape_uri(lua_State *L) -{ - size_t len, dlen; - u_char *p; - u_char *src, *dst; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - if (lua_isnil(L, 1)) { - lua_pushliteral(L, ""); - return 1; - } - - src = (u_char *) luaL_checklstring(L, 1, &len); - - /* the unescaped string can only be smaller */ - dlen = len; - - p = lua_newuserdata(L, dlen); - - dst = p; - - ngx_http_lua_unescape_uri(&dst, &src, len, NGX_UNESCAPE_URI_COMPONENT); - - lua_pushlstring(L, (char *) p, dst - p); - - return 1; -} - - static int ngx_http_lua_ngx_quote_sql_str(lua_State *L) { @@ -319,138 +211,6 @@ ngx_http_lua_ngx_escape_sql_str(u_char *dst, u_char *src, size_t size) } -static int -ngx_http_lua_ngx_md5(lua_State *L) -{ - u_char *src; - size_t slen; - - ngx_md5_t md5; - u_char md5_buf[MD5_DIGEST_LENGTH]; - u_char hex_buf[2 * sizeof(md5_buf)]; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - if (lua_isnil(L, 1)) { - src = (u_char *) ""; - slen = 0; - - } else { - src = (u_char *) luaL_checklstring(L, 1, &slen); - } - - ngx_md5_init(&md5); - ngx_md5_update(&md5, src, slen); - ngx_md5_final(md5_buf, &md5); - - ngx_hex_dump(hex_buf, md5_buf, sizeof(md5_buf)); - - lua_pushlstring(L, (char *) hex_buf, sizeof(hex_buf)); - - return 1; -} - - -static int -ngx_http_lua_ngx_md5_bin(lua_State *L) -{ - u_char *src; - size_t slen; - - ngx_md5_t md5; - u_char md5_buf[MD5_DIGEST_LENGTH]; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - if (lua_isnil(L, 1)) { - src = (u_char *) ""; - slen = 0; - - } else { - src = (u_char *) luaL_checklstring(L, 1, &slen); - } - - dd("slen: %d", (int) slen); - - ngx_md5_init(&md5); - ngx_md5_update(&md5, src, slen); - ngx_md5_final(md5_buf, &md5); - - lua_pushlstring(L, (char *) md5_buf, sizeof(md5_buf)); - - return 1; -} - - -#if (NGX_HAVE_SHA1) -static int -ngx_http_lua_ngx_sha1_bin(lua_State *L) -{ - u_char *src; - size_t slen; - - ngx_sha1_t sha; - u_char sha_buf[SHA_DIGEST_LENGTH]; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - if (lua_isnil(L, 1)) { - src = (u_char *) ""; - slen = 0; - - } else { - src = (u_char *) luaL_checklstring(L, 1, &slen); - } - - dd("slen: %d", (int) slen); - - ngx_sha1_init(&sha); - ngx_sha1_update(&sha, src, slen); - ngx_sha1_final(sha_buf, &sha); - - lua_pushlstring(L, (char *) sha_buf, sizeof(sha_buf)); - - return 1; -} -#endif - - -static int -ngx_http_lua_ngx_decode_base64(lua_State *L) -{ - ngx_str_t p, src; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - if (lua_type(L, 1) != LUA_TSTRING) { - return luaL_error(L, "string argument only"); - } - - src.data = (u_char *) luaL_checklstring(L, 1, &src.len); - - p.len = ngx_base64_decoded_length(src.len); - - p.data = lua_newuserdata(L, p.len); - - if (ngx_decode_base64(&p, &src) == NGX_OK) { - lua_pushlstring(L, (char *) p.data, p.len); - - } else { - lua_pushnil(L); - } - - return 1; -} - - static void ngx_http_lua_encode_base64(ngx_str_t *dst, ngx_str_t *src, int no_padding) { @@ -496,51 +256,6 @@ ngx_http_lua_encode_base64(ngx_str_t *dst, ngx_str_t *src, int no_padding) } -static size_t -ngx_http_lua_base64_encoded_length(size_t n, int no_padding) -{ - return no_padding ? (n * 8 + 5) / 6 : ngx_base64_encoded_length(n); -} - - -static int -ngx_http_lua_ngx_encode_base64(lua_State *L) -{ - int n; - int no_padding = 0; - ngx_str_t p, src; - - n = lua_gettop(L); - if (n != 1 && n != 2) { - return luaL_error(L, "expecting one or two arguments"); - } - - if (lua_isnil(L, 1)) { - src.data = (u_char *) ""; - src.len = 0; - - } else { - src.data = (u_char *) luaL_checklstring(L, 1, &src.len); - } - - if (n == 2) { - /* get the 2nd optional argument */ - luaL_checktype(L, 2, LUA_TBOOLEAN); - no_padding = lua_toboolean(L, 2); - } - - p.len = ngx_http_lua_base64_encoded_length(src.len, no_padding); - - p.data = lua_newuserdata(L, p.len); - - ngx_http_lua_encode_base64(&p, &src, no_padding); - - lua_pushlstring(L, (char *) p.data, p.len); - - return 1; -} - - static int ngx_http_lua_ngx_crc32_short(lua_State *L) { @@ -657,7 +372,6 @@ ngx_http_lua_ngx_hmac_sha1(lua_State *L) #endif -#ifndef NGX_LUA_NO_FFI_API void ngx_http_lua_ffi_md5_bin(const u_char *src, size_t len, u_char *dst) { @@ -776,6 +490,6 @@ ngx_http_lua_ffi_str_replace_char(u_char *buf, size_t len, const u_char find, len--; } } -#endif + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_time.c b/src/ngx_http_lua_time.c index bbb6f2b704..61324f37a2 100644 --- a/src/ngx_http_lua_time.c +++ b/src/ngx_http_lua_time.c @@ -11,244 +11,9 @@ #include "ddebug.h" -#include "ngx_http_lua_time.h" -#include "ngx_http_lua_util.h" +#include "ngx_http_lua_common.h" -static int ngx_http_lua_ngx_today(lua_State *L); -static int ngx_http_lua_ngx_time(lua_State *L); -static int ngx_http_lua_ngx_now(lua_State *L); -static int ngx_http_lua_ngx_localtime(lua_State *L); -static int ngx_http_lua_ngx_utctime(lua_State *L); -static int ngx_http_lua_ngx_cookie_time(lua_State *L); -static int ngx_http_lua_ngx_http_time(lua_State *L); -static int ngx_http_lua_ngx_parse_http_time(lua_State *L); -static int ngx_http_lua_ngx_update_time(lua_State *L); -static int ngx_http_lua_ngx_req_start_time(lua_State *L); - - -static int -ngx_http_lua_ngx_today(lua_State *L) -{ - time_t now; - ngx_tm_t tm; - u_char buf[sizeof("2010-11-19") - 1]; - - now = ngx_time(); - ngx_gmtime(now + ngx_cached_time->gmtoff * 60, &tm); - - ngx_sprintf(buf, "%04d-%02d-%02d", tm.ngx_tm_year, tm.ngx_tm_mon, - tm.ngx_tm_mday); - - lua_pushlstring(L, (char *) buf, sizeof(buf)); - - return 1; -} - - -static int -ngx_http_lua_ngx_localtime(lua_State *L) -{ - ngx_tm_t tm; - - u_char buf[sizeof("2010-11-19 20:56:31") - 1]; - - ngx_gmtime(ngx_time() + ngx_cached_time->gmtoff * 60, &tm); - - ngx_sprintf(buf, "%04d-%02d-%02d %02d:%02d:%02d", tm.ngx_tm_year, - tm.ngx_tm_mon, tm.ngx_tm_mday, tm.ngx_tm_hour, tm.ngx_tm_min, - tm.ngx_tm_sec); - - lua_pushlstring(L, (char *) buf, sizeof(buf)); - - return 1; -} - - -static int -ngx_http_lua_ngx_time(lua_State *L) -{ - lua_pushnumber(L, (lua_Number) ngx_time()); - - return 1; -} - - -static int -ngx_http_lua_ngx_now(lua_State *L) -{ - ngx_time_t *tp; - - tp = ngx_timeofday(); - - lua_pushnumber(L, (lua_Number) (tp->sec + tp->msec / 1000.0L)); - - return 1; -} - - -static int -ngx_http_lua_ngx_update_time(lua_State *L) -{ - ngx_time_update(); - return 0; -} - - -static int -ngx_http_lua_ngx_utctime(lua_State *L) -{ - ngx_tm_t tm; - u_char buf[sizeof("2010-11-19 20:56:31") - 1]; - - ngx_gmtime(ngx_time(), &tm); - - ngx_sprintf(buf, "%04d-%02d-%02d %02d:%02d:%02d", tm.ngx_tm_year, - tm.ngx_tm_mon, tm.ngx_tm_mday, tm.ngx_tm_hour, tm.ngx_tm_min, - tm.ngx_tm_sec); - - lua_pushlstring(L, (char *) buf, sizeof(buf)); - - return 1; -} - - -static int -ngx_http_lua_ngx_cookie_time(lua_State *L) -{ - time_t t; - u_char *p; - - u_char buf[sizeof("Mon, 28 Sep 1970 06:00:00 GMT") - 1]; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - t = (time_t) luaL_checknumber(L, 1); - - p = buf; - p = ngx_http_cookie_time(p, t); - - lua_pushlstring(L, (char *) buf, p - buf); - - return 1; -} - - -static int -ngx_http_lua_ngx_http_time(lua_State *L) -{ - time_t t; - u_char *p; - - u_char buf[sizeof("Mon, 28 Sep 1970 06:00:00 GMT") - 1]; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - t = (time_t) luaL_checknumber(L, 1); - - p = buf; - p = ngx_http_time(p, t); - - lua_pushlstring(L, (char *) buf, p - buf); - - return 1; -} - - -static int -ngx_http_lua_ngx_parse_http_time(lua_State *L) -{ - u_char *p; - size_t len; - time_t time; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument"); - } - - p = (u_char *) luaL_checklstring(L, 1, &len); - - time = ngx_http_parse_time(p, len); - if (time == NGX_ERROR) { - lua_pushnil(L); - return 1; - } - - lua_pushnumber(L, (lua_Number) time); - - return 1; -} - - -static int -ngx_http_lua_ngx_req_start_time(lua_State *L) -{ - ngx_http_request_t *r; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request found"); - } - - lua_pushnumber(L, (lua_Number) (r->start_sec + r->start_msec / 1000.0L)); - return 1; -} - - -void -ngx_http_lua_inject_time_api(lua_State *L) -{ - lua_pushcfunction(L, ngx_http_lua_ngx_utctime); - lua_setfield(L, -2, "utctime"); - - lua_pushcfunction(L, ngx_http_lua_ngx_time); - lua_setfield(L, -2, "get_now_ts"); /* deprecated */ - - lua_pushcfunction(L, ngx_http_lua_ngx_localtime); - lua_setfield(L, -2, "get_now"); /* deprecated */ - - lua_pushcfunction(L, ngx_http_lua_ngx_localtime); - lua_setfield(L, -2, "localtime"); - - lua_pushcfunction(L, ngx_http_lua_ngx_time); - lua_setfield(L, -2, "time"); - - lua_pushcfunction(L, ngx_http_lua_ngx_now); - lua_setfield(L, -2, "now"); - - lua_pushcfunction(L, ngx_http_lua_ngx_update_time); - lua_setfield(L, -2, "update_time"); - - lua_pushcfunction(L, ngx_http_lua_ngx_today); - lua_setfield(L, -2, "get_today"); /* deprecated */ - - lua_pushcfunction(L, ngx_http_lua_ngx_today); - lua_setfield(L, -2, "today"); - - lua_pushcfunction(L, ngx_http_lua_ngx_cookie_time); - lua_setfield(L, -2, "cookie_time"); - - lua_pushcfunction(L, ngx_http_lua_ngx_http_time); - lua_setfield(L, -2, "http_time"); - - lua_pushcfunction(L, ngx_http_lua_ngx_parse_http_time); - lua_setfield(L, -2, "parse_http_time"); -} - - -void -ngx_http_lua_inject_req_time_api(lua_State *L) -{ - lua_pushcfunction(L, ngx_http_lua_ngx_req_start_time); - lua_setfield(L, -2, "start_time"); -} - - -#ifndef NGX_LUA_NO_FFI_API double ngx_http_lua_ffi_now(void) { @@ -343,7 +108,6 @@ ngx_http_lua_ffi_parse_http_time(const u_char *str, size_t len, /* ngx_http_parse_time doesn't modify 'str' actually */ *time = ngx_http_parse_time((u_char *) str, len); } -#endif /* NGX_LUA_NO_FFI_API */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_time.h b/src/ngx_http_lua_time.h deleted file mode 100644 index 291f784ce8..0000000000 --- a/src/ngx_http_lua_time.h +++ /dev/null @@ -1,21 +0,0 @@ - -/* - * Copyright (C) Xiaozhe Wang (chaoslawful) - * Copyright (C) Yichun Zhang (agentzh) - */ - - -#ifndef _NGX_HTTP_LUA_TIME_H_INCLUDED_ -#define _NGX_HTTP_LUA_TIME_H_INCLUDED_ - - -#include "ngx_http_lua_common.h" - - -void ngx_http_lua_inject_time_api(lua_State *L); -void ngx_http_lua_inject_req_time_api(lua_State *L); - - -#endif /* _NGX_HTTP_LUA_TIME_H_INCLUDED_ */ - -/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 6d5b511aa2..6dd110af4b 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -16,22 +16,18 @@ #include "ngx_http_lua_util.h" #include "ngx_http_lua_exception.h" #include "ngx_http_lua_pcrefix.h" -#include "ngx_http_lua_regex.h" #include "ngx_http_lua_args.h" #include "ngx_http_lua_uri.h" #include "ngx_http_lua_req_body.h" #include "ngx_http_lua_headers.h" #include "ngx_http_lua_output.h" -#include "ngx_http_lua_time.h" #include "ngx_http_lua_control.h" #include "ngx_http_lua_ndk.h" #include "ngx_http_lua_subrequest.h" #include "ngx_http_lua_log.h" -#include "ngx_http_lua_variable.h" #include "ngx_http_lua_string.h" #include "ngx_http_lua_misc.h" #include "ngx_http_lua_consts.h" -#include "ngx_http_lua_req_method.h" #include "ngx_http_lua_shdict.h" #include "ngx_http_lua_coroutine.h" #include "ngx_http_lua_socket_tcp.h" @@ -41,13 +37,11 @@ #include "ngx_http_lua_headerfilterby.h" #include "ngx_http_lua_bodyfilterby.h" #include "ngx_http_lua_logby.h" -#include "ngx_http_lua_phase.h" #include "ngx_http_lua_probe.h" #include "ngx_http_lua_uthread.h" #include "ngx_http_lua_contentby.h" #include "ngx_http_lua_timer.h" #include "ngx_http_lua_config.h" -#include "ngx_http_lua_worker.h" #include "ngx_http_lua_socket_tcp.h" #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_ssl.h" @@ -105,6 +99,10 @@ static ngx_int_t ngx_http_lua_send_http10_headers(ngx_http_request_t *r, static void ngx_http_lua_init_registry(lua_State *L, ngx_log_t *log); static void ngx_http_lua_init_globals(lua_State *L, ngx_cycle_t *cycle, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log); +#ifdef OPENRESTY_LUAJIT +static void ngx_http_lua_inject_global_write_guard(lua_State *L, + ngx_log_t *log); +#endif static void ngx_http_lua_set_path(ngx_cycle_t *cycle, lua_State *L, int tab_idx, const char *fieldname, const char *path, const char *default_path, ngx_log_t *log); @@ -744,7 +742,7 @@ static void ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log) { - lua_createtable(L, 0 /* narr */, 117 /* nrec */); /* ngx.* */ + lua_createtable(L, 0 /* narr */, 113 /* nrec */); /* ngx.* */ lua_pushcfunction(L, ngx_http_lua_get_raw_phase_context); lua_setfield(L, -2, "_phase_ctx"); @@ -756,30 +754,20 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_http_lua_inject_log_api(L); ngx_http_lua_inject_output_api(L); - ngx_http_lua_inject_time_api(L); ngx_http_lua_inject_string_api(L); ngx_http_lua_inject_control_api(log, L); ngx_http_lua_inject_subrequest_api(L); ngx_http_lua_inject_sleep_api(L); - ngx_http_lua_inject_phase_api(L); - -#if (NGX_PCRE) - ngx_http_lua_inject_regex_api(L); -#endif ngx_http_lua_inject_req_api(log, L); ngx_http_lua_inject_resp_header_api(L); ngx_http_lua_create_headers_metatable(log, L); - ngx_http_lua_inject_variable_api(L); ngx_http_lua_inject_shdict_api(lmcf, L); ngx_http_lua_inject_socket_tcp_api(log, L); ngx_http_lua_inject_socket_udp_api(log, L); ngx_http_lua_inject_uthread_api(log, L); ngx_http_lua_inject_timer_api(L); ngx_http_lua_inject_config_api(L); - ngx_http_lua_inject_worker_api(L); - - ngx_http_lua_inject_misc_api(L); lua_getglobal(L, "package"); /* ngx package */ lua_getfield(L, -1, "loaded"); /* ngx package loaded */ @@ -790,53 +778,56 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, lua_setglobal(L, "ngx"); ngx_http_lua_inject_coroutine_api(log, L); +} + #ifdef OPENRESTY_LUAJIT - { - int rc; - - const char buf[] = - "local ngx_log = ngx.log\n" - "local ngx_WARN = ngx.WARN\n" - "local tostring = tostring\n" - "local ngx_get_phase = ngx.get_phase\n" - "local traceback = require 'debug'.traceback\n" - "local function newindex(table, key, value)\n" - "rawset(table, key, value)\n" - "local phase = ngx_get_phase()\n" - "if phase == 'init_worker' or phase == 'init' then\n" - "return\n" - "end\n" - "ngx_log(ngx_WARN, 'writing a global lua variable " - "(\\'', tostring(key), '\\') which may lead to " - "race conditions between concurrent requests, so " - "prefer the use of \\'local\\' variables', " - "traceback('', 2))\n" +static void +ngx_http_lua_inject_global_write_guard(lua_State *L, ngx_log_t *log) +{ + int rc; + + const char buf[] = + "local ngx_log = ngx.log\n" + "local ngx_WARN = ngx.WARN\n" + "local tostring = tostring\n" + "local ngx_get_phase = ngx.get_phase\n" + "local traceback = require 'debug'.traceback\n" + "local function newindex(table, key, value)\n" + "rawset(table, key, value)\n" + "local phase = ngx_get_phase()\n" + "if phase == 'init_worker' or phase == 'init' then\n" + "return\n" "end\n" - "setmetatable(_G, { __newindex = newindex })\n" - ; + "ngx_log(ngx_WARN, 'writing a global Lua variable " + "(\\'', tostring(key), '\\') which may lead to " + "race conditions between concurrent requests, so " + "prefer the use of \\'local\\' variables', " + "traceback('', 2))\n" + "end\n" + "setmetatable(_G, { __newindex = newindex })\n" + ; - rc = luaL_loadbuffer(L, buf, sizeof(buf) - 1, "=_G write guard"); + rc = luaL_loadbuffer(L, buf, sizeof(buf) - 1, "=_G write guard"); - if (rc != 0) { - ngx_log_error(NGX_LOG_ERR, log, 0, - "failed to load Lua code (%i): %s", - rc, lua_tostring(L, -1)); + if (rc != 0) { + ngx_log_error(NGX_LOG_ERR, log, 0, + "failed to load Lua code (%i): %s", + rc, lua_tostring(L, -1)); - lua_pop(L, 1); - return; - } + lua_pop(L, 1); + return; + } - rc = lua_pcall(L, 0, 0, 0); - if (rc != 0) { - ngx_log_error(NGX_LOG_ERR, log, 0, - "failed to run Lua code (%i): %s", - rc, lua_tostring(L, -1)); - lua_pop(L, 1); - } + rc = lua_pcall(L, 0, 0, 0); + if (rc != 0) { + ngx_log_error(NGX_LOG_ERR, log, 0, + "failed to run Lua code (%i): %s", + rc, lua_tostring(L, -1)); + lua_pop(L, 1); } -#endif } +#endif void @@ -2189,15 +2180,13 @@ ngx_http_lua_inject_req_api(ngx_log_t *log, lua_State *L) { /* ngx.req table */ - lua_createtable(L, 0 /* narr */, 24 /* nrec */); /* .req */ + lua_createtable(L, 0 /* narr */, 23 /* nrec */); /* .req */ ngx_http_lua_inject_req_header_api(L); ngx_http_lua_inject_req_uri_api(log, L); ngx_http_lua_inject_req_args_api(L); ngx_http_lua_inject_req_body_api(L); ngx_http_lua_inject_req_socket_api(L); - ngx_http_lua_inject_req_method_api(L); - ngx_http_lua_inject_req_time_api(L); ngx_http_lua_inject_req_misc_api(L); lua_setfield(L, -2, "req"); @@ -3898,6 +3887,10 @@ ngx_http_lua_init_vm(lua_State **new_vm, lua_State *parent_vm, return NGX_DECLINED; } +#ifdef OPENRESTY_LUAJIT + ngx_http_lua_inject_global_write_guard(L, log); +#endif + return NGX_OK; } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index c80bf758fb..98edb69349 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -23,12 +23,10 @@ #endif -#ifndef NGX_LUA_NO_FFI_API typedef struct { ngx_http_lua_ffi_str_t key; ngx_http_lua_ffi_str_t value; } ngx_http_lua_ffi_table_elt_t; -#endif /* NGX_LUA_NO_FFI_API */ /* char whose address we use as the key in Lua vm registry for @@ -41,7 +39,7 @@ extern char ngx_http_lua_code_cache_key; /* char whose address we use as the key in Lua vm registry for - * regex cache table */ + * regex cache table */ extern char ngx_http_lua_regex_cache_key; /* char whose address we use as the key in Lua vm registry for @@ -100,7 +98,6 @@ extern char ngx_http_lua_headers_metatable_key; } -#ifndef NGX_LUA_NO_FFI_API static ngx_inline ngx_int_t ngx_http_lua_ffi_check_context(ngx_http_lua_ctx_t *ctx, unsigned flags, u_char *err, size_t *errlen) @@ -116,7 +113,6 @@ ngx_http_lua_ffi_check_context(ngx_http_lua_ctx_t *ctx, unsigned flags, return NGX_OK; } -#endif #define ngx_http_lua_check_fake_request(L, r) \ diff --git a/src/ngx_http_lua_variable.c b/src/ngx_http_lua_variable.c index e489d9e57e..573629ad21 100644 --- a/src/ngx_http_lua_variable.c +++ b/src/ngx_http_lua_variable.c @@ -11,282 +11,9 @@ #include "ddebug.h" -#include "ngx_http_lua_variable.h" #include "ngx_http_lua_util.h" -static int ngx_http_lua_var_get(lua_State *L); -static int ngx_http_lua_var_set(lua_State *L); - - -void -ngx_http_lua_inject_variable_api(lua_State *L) -{ - /* {{{ register reference maps */ - lua_newtable(L); /* ngx.var */ - - lua_createtable(L, 0, 2 /* nrec */); /* metatable for .var */ - lua_pushcfunction(L, ngx_http_lua_var_get); - lua_setfield(L, -2, "__index"); - lua_pushcfunction(L, ngx_http_lua_var_set); - lua_setfield(L, -2, "__newindex"); - lua_setmetatable(L, -2); - - lua_setfield(L, -2, "var"); -} - - -/** - * Get nginx internal variables content - * - * @retval Always return a string or nil on Lua stack. Return nil when failed - * to get content, and actual content string when found the specified variable. - * @seealso ngx_http_lua_var_set - * */ -static int -ngx_http_lua_var_get(lua_State *L) -{ - ngx_http_request_t *r; - u_char *p, *lowcase; - size_t len; - ngx_uint_t hash; - ngx_str_t name; - ngx_http_variable_value_t *vv; - -#if (NGX_PCRE) - u_char *val; - ngx_uint_t n; - LUA_NUMBER index; - int *cap; -#endif - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ngx_http_lua_check_fake_request(L, r); - -#if (NGX_PCRE) - if (lua_type(L, -1) == LUA_TNUMBER) { - /* it is a regex capturing variable */ - - index = lua_tonumber(L, -1); - - if (index <= 0) { - lua_pushnil(L); - return 1; - } - - n = (ngx_uint_t) index * 2; - - dd("n = %d, ncaptures = %d", (int) n, (int) r->ncaptures); - - if (r->captures == NULL - || r->captures_data == NULL - || n >= r->ncaptures) - { - lua_pushnil(L); - return 1; - } - - /* n >= 0 && n < r->ncaptures */ - - cap = r->captures; - - p = r->captures_data; - - val = &p[cap[n]]; - - lua_pushlstring(L, (const char *) val, (size_t) (cap[n + 1] - cap[n])); - - return 1; - } -#endif - - if (lua_type(L, -1) != LUA_TSTRING) { - return luaL_error(L, "bad variable name"); - } - - p = (u_char *) lua_tolstring(L, -1, &len); - - lowcase = lua_newuserdata(L, len); - - hash = ngx_hash_strlow(lowcase, p, len); - - name.len = len; - name.data = lowcase; - - vv = ngx_http_get_variable(r, &name, hash); - - if (vv == NULL || vv->not_found) { - lua_pushnil(L); - return 1; - } - - lua_pushlstring(L, (const char *) vv->data, (size_t) vv->len); - return 1; -} - - -/** - * Set nginx internal variable content - * - * @retval Always return a boolean on Lua stack. Return true when variable - * content was modified successfully, false otherwise. - * @seealso ngx_http_lua_var_get - * */ -static int -ngx_http_lua_var_set(lua_State *L) -{ - ngx_http_variable_t *v; - ngx_http_variable_value_t *vv; - ngx_http_core_main_conf_t *cmcf; - u_char *p, *lowcase, *val; - size_t len; - ngx_str_t name; - ngx_uint_t hash; - ngx_http_request_t *r; - int value_type; - const char *msg; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request object found"); - } - - ngx_http_lua_check_fake_request(L, r); - - /* we skip the first argument that is the table */ - - /* we read the variable name */ - - if (lua_type(L, 2) != LUA_TSTRING) { - return luaL_error(L, "bad variable name"); - } - - p = (u_char *) lua_tolstring(L, 2, &len); - - lowcase = lua_newuserdata(L, len + 1); - - hash = ngx_hash_strlow(lowcase, p, len); - lowcase[len] = '\0'; - - name.len = len; - name.data = lowcase; - - /* we read the variable new value */ - - value_type = lua_type(L, 3); - switch (value_type) { - case LUA_TNUMBER: - case LUA_TSTRING: - p = (u_char *) luaL_checklstring(L, 3, &len); - - val = ngx_palloc(r->pool, len); - if (val == NULL) { - return luaL_error(L, "memory allocation error"); - } - - ngx_memcpy(val, p, len); - - break; - - case LUA_TNIL: - /* undef the variable */ - - val = NULL; - len = 0; - - break; - - default: - msg = lua_pushfstring(L, "string, number, or nil expected, " - "but got %s", lua_typename(L, value_type)); - return luaL_argerror(L, 1, msg); - } - - /* we fetch the variable itself */ - - cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module); - - v = ngx_hash_find(&cmcf->variables_hash, hash, name.data, name.len); - - if (v) { - if (!(v->flags & NGX_HTTP_VAR_CHANGEABLE)) { - return luaL_error(L, "variable \"%s\" not changeable", lowcase); - } - - if (v->set_handler) { - - dd("set variables with set_handler"); - - vv = ngx_palloc(r->pool, sizeof(ngx_http_variable_value_t)); - if (vv == NULL) { - return luaL_error(L, "no memory"); - } - - if (value_type == LUA_TNIL) { - vv->valid = 0; - vv->not_found = 1; - vv->no_cacheable = 0; - vv->data = NULL; - vv->len = 0; - - } else { - vv->valid = 1; - vv->not_found = 0; - vv->no_cacheable = 0; - - vv->data = val; - vv->len = len; - } - - v->set_handler(r, vv, v->data); - - return 0; - } - - if (v->flags & NGX_HTTP_VAR_INDEXED) { - vv = &r->variables[v->index]; - - dd("set indexed variable"); - - if (value_type == LUA_TNIL) { - vv->valid = 0; - vv->not_found = 1; - vv->no_cacheable = 0; - - vv->data = NULL; - vv->len = 0; - - } else { - vv->valid = 1; - vv->not_found = 0; - vv->no_cacheable = 0; - - vv->data = val; - vv->len = len; - } - - return 0; - } - - return luaL_error(L, "variable \"%s\" cannot be assigned a value", - lowcase); - } - - /* variable not found */ - - return luaL_error(L, "variable \"%s\" not found for writing; " - "maybe it is a built-in variable that is not changeable " - "or you forgot to use \"set $%s '';\" " - "in the config file to define it first", - lowcase, lowcase); -} - - -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_var_get(ngx_http_request_t *r, u_char *name_data, size_t name_len, u_char *lowcase_buf, int capture_id, u_char **value, @@ -503,7 +230,6 @@ ngx_http_lua_ffi_var_set(ngx_http_request_t *r, u_char *name_data, *errlen = ngx_snprintf(errbuf, *errlen, "no memory") - errbuf; return NGX_ERROR; } -#endif /* NGX_LUA_NO_FFI_API */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_variable.h b/src/ngx_http_lua_variable.h deleted file mode 100644 index 550e4d3888..0000000000 --- a/src/ngx_http_lua_variable.h +++ /dev/null @@ -1,20 +0,0 @@ - -/* - * Copyright (C) Xiaozhe Wang (chaoslawful) - * Copyright (C) Yichun Zhang (agentzh) - */ - - -#ifndef _NGX_HTTP_LUA_VARIABLE_H_INCLUDED_ -#define _NGX_HTTP_LUA_VARIABLE_H_INCLUDED_ - - -#include "ngx_http_lua_common.h" - - -void ngx_http_lua_inject_variable_api(lua_State *L); - - -#endif /* _NGX_HTTP_LUA_VARIABLE_H_INCLUDED_ */ - -/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 33cfa11968..8389dbeaec 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -10,88 +10,9 @@ #include "ddebug.h" -#include "ngx_http_lua_worker.h" - - #define NGX_PROCESS_PRIVILEGED_AGENT 99 -static int ngx_http_lua_ngx_worker_exiting(lua_State *L); -static int ngx_http_lua_ngx_worker_pid(lua_State *L); -static int ngx_http_lua_ngx_worker_id(lua_State *L); -static int ngx_http_lua_ngx_worker_count(lua_State *L); - - -void -ngx_http_lua_inject_worker_api(lua_State *L) -{ - lua_createtable(L, 0 /* narr */, 4 /* nrec */); /* ngx.worker. */ - - lua_pushcfunction(L, ngx_http_lua_ngx_worker_exiting); - lua_setfield(L, -2, "exiting"); - - lua_pushcfunction(L, ngx_http_lua_ngx_worker_pid); - lua_setfield(L, -2, "pid"); - - lua_pushcfunction(L, ngx_http_lua_ngx_worker_id); - lua_setfield(L, -2, "id"); - - lua_pushcfunction(L, ngx_http_lua_ngx_worker_count); - lua_setfield(L, -2, "count"); - - lua_setfield(L, -2, "worker"); -} - - -static int -ngx_http_lua_ngx_worker_exiting(lua_State *L) -{ - lua_pushboolean(L, ngx_exiting); - return 1; -} - - -static int -ngx_http_lua_ngx_worker_pid(lua_State *L) -{ - lua_pushinteger(L, (lua_Integer) ngx_pid); - return 1; -} - - -static int -ngx_http_lua_ngx_worker_id(lua_State *L) -{ -#if (nginx_version >= 1009001) - if (ngx_process != NGX_PROCESS_WORKER - && ngx_process != NGX_PROCESS_SINGLE) - { - lua_pushnil(L); - return 1; - } - - lua_pushinteger(L, (lua_Integer) ngx_worker); -#else - lua_pushnil(L); -#endif - return 1; -} - - -static int -ngx_http_lua_ngx_worker_count(lua_State *L) -{ - ngx_core_conf_t *ccf; - - ccf = (ngx_core_conf_t *) ngx_get_conf(ngx_cycle->conf_ctx, - ngx_core_module); - - lua_pushinteger(L, (lua_Integer) ccf->worker_processes); - return 1; -} - - -#ifndef NGX_LUA_NO_FFI_API int ngx_http_lua_ffi_worker_pid(void) { @@ -201,4 +122,6 @@ ngx_http_lua_ffi_process_signal_graceful_exit(void) { ngx_quit = 1; } -#endif + + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_worker.h b/src/ngx_http_lua_worker.h deleted file mode 100644 index 30803949f3..0000000000 --- a/src/ngx_http_lua_worker.h +++ /dev/null @@ -1,17 +0,0 @@ - -/* - * Copyright (C) Yichun Zhang (agentzh) - */ - - -#ifndef _NGX_HTTP_LUA_WORKER_H_INCLUDED_ -#define _NGX_HTTP_LUA_WORKER_H_INCLUDED_ - - -#include "ngx_http_lua_common.h" - - -void ngx_http_lua_inject_worker_api(lua_State *L); - - -#endif /* _NGX_HTTP_LUA_WORKER_H_INCLUDED_ */ diff --git a/t/001-set.t b/t/001-set.t index 9e8ee7b382..62dec1a7f9 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -613,9 +613,9 @@ GET /lua \z --- no_error_log [error] ---- grep_error_log eval: qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ +--- grep_error_log eval: qr/(old foo: \d+|writing a global Lua variable \('\w+'\))/ --- grep_error_log_out eval -["writing a global lua variable \('foo'\)\n", "old foo: 1\n"] +["writing a global Lua variable \('foo'\)\n", "old foo: 1\n"] diff --git a/t/062-count.t b/t/062-count.t index 9b0afe8aac..b481763abf 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -34,7 +34,7 @@ __DATA__ --- request GET /test --- response_body -ngx: 117 +ngx: 113 --- no_error_log [error] @@ -55,7 +55,7 @@ ngx: 117 --- request GET /test --- response_body -117 +113 --- no_error_log [error] @@ -83,7 +83,7 @@ GET /test --- request GET /test --- response_body -n = 117 +n = 113 --- no_error_log [error] @@ -123,7 +123,7 @@ n = 1 --- request GET /test --- response_body -n = 24 +n = 23 --- no_error_log [error] @@ -145,7 +145,7 @@ n = 24 --- request GET /test --- response_body -n = 24 +n = 23 --- no_error_log [error] @@ -172,7 +172,7 @@ n = 24 --- request GET /test --- response_body -n = 24 +n = 23 --- no_error_log [error] @@ -305,7 +305,7 @@ GET /t --- response_body_like: 404 Not Found --- error_code: 404 --- error_log -ngx. entry count: 117 +ngx. entry count: 113 diff --git a/t/158-global-var.t b/t/158-global-var.t index 414a43aac9..258ccdbce0 100644 --- a/t/158-global-var.t +++ b/t/158-global-var.t @@ -60,9 +60,9 @@ __DATA__ --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|set_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|set_by_lua:\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] @@ -83,9 +83,9 @@ set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -106,9 +106,9 @@ rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -129,9 +129,9 @@ access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) content_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -154,9 +154,9 @@ content_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|1)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] @@ -179,9 +179,9 @@ header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|2)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ --- grep_error_log_out eval -[qr/\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\[warn\] .*?writing a global Lua variable \('foo'\) body_filter_by_lua:3: in main chunk, old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] @@ -205,9 +205,9 @@ old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] --- response_body_like chomp \A(?:nil|1)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk)/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk)/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] @@ -297,9 +297,9 @@ log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]done\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk)/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk)/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] @@ -328,9 +328,9 @@ ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in\b)/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in\b)/ --- grep_error_log_out eval -[qr/\A\[warn\] .*?writing a global lua variable \('foo'\) +[qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) content_by_lua\(nginx\.conf:56\):4: in\n\z/, "old foo: 1\n"] @@ -476,9 +476,9 @@ setting global variable --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ +qr/(old foo: \d+|writing a global Lua variable \('\w+'\))/ --- grep_error_log_out eval -["writing a global lua variable \('foo'\)\n", "old foo: 1\n"] +["writing a global Lua variable \('foo'\)\n", "old foo: 1\n"] @@ -503,6 +503,6 @@ qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ --- error_code: 502 --- error_log eval qr/\[crit\].*?\Qconnect() to 0.0.0.1:80 failed\E/ ---- grep_error_log eval: qr/(old foo: \d+|writing a global lua variable \('\w+'\))/ +--- grep_error_log eval: qr/(old foo: \d+|writing a global Lua variable \('\w+'\))/ --- grep_error_log_out eval -["writing a global lua variable \('foo'\)\n", "old foo: 1\n"] +["writing a global Lua variable \('foo'\)\n", "old foo: 1\n"] From c5135a051162e58a30fe65af97b14d7fb7dac354 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 2 Aug 2019 14:16:00 -0700 Subject: [PATCH 350/848] change: removed compatibility code for unsupported NGINX versions (< 1.6.0). --- src/ngx_http_lua_accessby.c | 5 -- src/ngx_http_lua_bodyfilterby.c | 4 -- src/ngx_http_lua_common.h | 4 +- src/ngx_http_lua_consts.c | 2 - src/ngx_http_lua_contentby.c | 4 -- src/ngx_http_lua_directive.c | 10 +-- src/ngx_http_lua_headers.c | 16 ++--- src/ngx_http_lua_headers.h | 2 +- src/ngx_http_lua_headers_in.c | 6 -- src/ngx_http_lua_initworkerby.c | 27 +------- src/ngx_http_lua_module.c | 39 +++--------- src/ngx_http_lua_req_body.c | 8 --- src/ngx_http_lua_rewriteby.c | 5 -- src/ngx_http_lua_shdict.c | 2 - src/ngx_http_lua_socket_tcp.c | 86 ++++---------------------- src/ngx_http_lua_socket_udp.c | 74 ++++------------------ src/ngx_http_lua_ssl_certby.c | 19 +----- src/ngx_http_lua_ssl_session_fetchby.c | 19 +----- src/ngx_http_lua_ssl_session_storeby.c | 19 +----- src/ngx_http_lua_subrequest.c | 8 +-- src/ngx_http_lua_timer.c | 19 +----- src/ngx_http_lua_util.c | 18 ------ src/ngx_http_lua_util.h | 5 -- 23 files changed, 57 insertions(+), 344 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 1cbf629a16..fabe59a3ac 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -145,11 +145,6 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) ngx_http_lua_generic_phase_post_read); if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) { -#if (nginx_version < 1002006) || \ - (nginx_version >= 1003000 && nginx_version < 1003009) - r->main->count--; -#endif - return rc; } diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 9fea183951..75608695e1 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -316,11 +316,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) return NGX_ERROR; } -#if nginx_version >= 1001004 ngx_chain_update_chains(r->pool, -#else - ngx_chain_update_chains( -#endif &ctx->free_bufs, &ctx->busy_bufs, &out, (ngx_buf_tag_t) &ngx_http_lua_module); diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index ad5c5f8e74..455581d742 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -64,7 +64,7 @@ #endif #ifndef NGX_HAVE_SHA1 -# if (nginx_version >= 1011002) +# if defined(nginx_version) && (nginx_version >= 1011002) # define NGX_HAVE_SHA1 1 # endif #endif @@ -256,7 +256,7 @@ struct ngx_http_lua_main_conf_s { of reqeusts */ ngx_uint_t malloc_trim_req_count; -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) /* the following 2 fields are only used by ngx.req.raw_headers() for now */ ngx_buf_t **busy_buf_ptrs; ngx_int_t busy_buf_ptr_count; diff --git a/src/ngx_http_lua_consts.c b/src/ngx_http_lua_consts.c index 809e712e7a..47ba998576 100644 --- a/src/ngx_http_lua_consts.c +++ b/src/ngx_http_lua_consts.c @@ -119,10 +119,8 @@ ngx_http_lua_inject_http_consts(lua_State *L) lua_pushinteger(L, NGX_HTTP_MOVED_TEMPORARILY); lua_setfield(L, -2, "HTTP_MOVED_TEMPORARILY"); -#if defined(nginx_version) && nginx_version >= 8042 lua_pushinteger(L, NGX_HTTP_SEE_OTHER); lua_setfield(L, -2, "HTTP_SEE_OTHER"); -#endif lua_pushinteger(L, NGX_HTTP_PERMANENT_REDIRECT); lua_setfield(L, -2, "HTTP_PERMANENT_REDIRECT"); diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 296ba9d35f..56a6192282 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -202,10 +202,6 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) ngx_http_lua_content_phase_post_read); if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) { -#if (nginx_version < 1002006) || \ - (nginx_version >= 1003000 && nginx_version < 1003009) - r->main->count--; -#endif return rc; } diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index cdc573097b..42431f8362 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -479,10 +479,6 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) dd("enter"); -#if defined(nginx_version) && nginx_version >= 8042 && nginx_version <= 8053 - return "does not work with " NGINX_VER; -#endif - /* must specify a content handler */ if (cmd->post == NULL) { return NGX_CONF_ERROR; @@ -1437,12 +1433,12 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, ngx_uint_t start_line; ngx_str_t *word; ngx_buf_t *b; -#if nginx_version >= 1009002 +#if defined(nginx_version) && (nginx_version >= 1009002) ngx_buf_t *dump; #endif b = cf->conf_file->buffer; -#if nginx_version >= 1009002 +#if defined(nginx_version) && (nginx_version >= 1009002) dump = cf->conf_file->dump; #endif start = b->pos; @@ -1513,7 +1509,7 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, b->last = b->start + len + n; start = b->start; -#if nginx_version >= 1009002 +#if defined(nginx_version) && (nginx_version >= 1009002) if (dump) { dump->last = ngx_cpymem(dump->last, b->start + len, size); } diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index a50a1ed333..6c248d3b47 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -22,7 +22,7 @@ static int ngx_http_lua_ngx_req_raw_header(lua_State *L); static int ngx_http_lua_ngx_req_header_set_helper(lua_State *L); static int ngx_http_lua_ngx_resp_get_headers(lua_State *L); static int ngx_http_lua_ngx_req_header_set(lua_State *L); -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) void ngx_http_lua_ngx_raw_header_cleanup(void *data); #endif @@ -102,7 +102,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) size_t size; ngx_buf_t *b, *first = NULL; ngx_int_t i, j; -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) ngx_buf_t **bb; ngx_chain_t *cl; ngx_http_lua_main_conf_t *lmcf; @@ -123,7 +123,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) return luaL_error(L, "no request object found"); } -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); #endif @@ -143,7 +143,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) dd("hc->nbusy: %d", (int) hc->nbusy); if (hc->nbusy) { -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) dd("hc->busy: %p %p %p %p", hc->busy->buf->start, hc->busy->buf->pos, hc->busy->buf->last, hc->busy->buf->end); #else @@ -185,7 +185,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) dd("size: %d", (int) size); if (hc->nbusy) { -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) if (hc->nbusy > lmcf->busy_buf_ptr_count) { if (lmcf->busy_buf_ptrs) { ngx_free(lmcf->busy_buf_ptrs); @@ -208,7 +208,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) #endif b = NULL; -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) bb = lmcf->busy_buf_ptrs; for (i = hc->nbusy; i > 0; i--) { b = bb[i - 1]; @@ -291,7 +291,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) if (hc->nbusy) { -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) bb = lmcf->busy_buf_ptrs; for (i = hc->nbusy - 1; i >= 0; i--) { b = bb[i]; @@ -1138,7 +1138,7 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, } -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) void ngx_http_lua_ngx_raw_header_cleanup(void *data) { diff --git a/src/ngx_http_lua_headers.h b/src/ngx_http_lua_headers.h index ee4d21c1ae..1cb0b1dec7 100644 --- a/src/ngx_http_lua_headers.h +++ b/src/ngx_http_lua_headers.h @@ -15,7 +15,7 @@ void ngx_http_lua_inject_resp_header_api(lua_State *L); void ngx_http_lua_inject_req_header_api(lua_State *L); void ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L); -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) void ngx_http_lua_ngx_raw_header_cleanup(void *data); #endif diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index c52cd13db1..4a5a5b6146 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -56,13 +56,10 @@ static ngx_http_lua_set_header_t ngx_http_lua_set_handlers[] = { offsetof(ngx_http_headers_in_t, if_modified_since), ngx_http_set_builtin_header }, -#if defined(nginx_version) && nginx_version >= 9002 { ngx_string("If-Unmodified-Since"), offsetof(ngx_http_headers_in_t, if_unmodified_since), ngx_http_set_builtin_header }, -#endif -#if defined(nginx_version) && nginx_version >= 1003003 { ngx_string("If-Match"), offsetof(ngx_http_headers_in_t, if_match), ngx_http_set_builtin_header }, @@ -70,7 +67,6 @@ static ngx_http_lua_set_header_t ngx_http_lua_set_handlers[] = { { ngx_string("If-None-Match"), offsetof(ngx_http_headers_in_t, if_none_match), ngx_http_set_builtin_header }, -#endif { ngx_string("User-Agent"), offsetof(ngx_http_headers_in_t, user_agent), @@ -104,11 +100,9 @@ static ngx_http_lua_set_header_t ngx_http_lua_set_handlers[] = { offsetof(ngx_http_headers_in_t, expect), ngx_http_set_builtin_header }, -#if defined(nginx_version) && nginx_version >= 1003013 { ngx_string("Upgrade"), offsetof(ngx_http_headers_in_t, upgrade), ngx_http_set_builtin_header }, -#endif #if (NGX_HTTP_GZIP) { ngx_string("Accept-Encoding"), diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 5e02a83860..dfd4e9cd23 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -114,12 +114,8 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) ngx_memcpy(fake_cycle, cycle, sizeof(ngx_cycle_t)); -#if defined(nginx_version) && nginx_version >= 9007 - ngx_queue_init(&fake_cycle->reusable_connections_queue); -#endif - if (ngx_array_init(&fake_cycle->listening, cycle->pool, cycle->listening.nelts || 1, sizeof(ngx_listening_t)) @@ -128,8 +124,6 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) goto failed; } -#if defined(nginx_version) && nginx_version >= 1003007 - if (ngx_array_init(&fake_cycle->paths, cycle->pool, cycle->paths.nelts || 1, sizeof(ngx_path_t *)) != NGX_OK) @@ -137,8 +131,6 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) goto failed; } -#endif - part = &cycle->open_files.part; ofile = part->elts; @@ -302,26 +294,11 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && nginx_version >= 1003014 - -# if nginx_version >= 1009000 - +#if defined(nginx_version) && (nginx_version >= 1009000) ngx_set_connection_log(r->connection, clcf->error_log); -# else - - ngx_http_set_connection_log(r->connection, clcf->error_log); - -# endif - #else - - c->log->file = clcf->error_log->file; - - if (!(c->log->log_level & NGX_LOG_DEBUG_CONNECTION)) { - c->log->log_level = clcf->error_log->log_level; - } - + ngx_http_set_connection_log(r->connection, clcf->error_log); #endif ctx = ngx_http_lua_create_ctx(r); diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index efa8dc12bc..ef99b0dc42 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -58,7 +58,7 @@ static ngx_conf_post_t ngx_http_lua_lowat_post = static volatile ngx_cycle_t *ngx_http_lua_prev_cycle = NULL; -#if (NGX_HTTP_SSL) && defined(nginx_version) && nginx_version >= 1001013 +#if (NGX_HTTP_SSL) static ngx_conf_bitmask_t ngx_http_lua_ssl_protocols[] = { { ngx_string("SSLv2"), NGX_SSL_SSLv2 }, @@ -515,8 +515,6 @@ static ngx_command_t ngx_http_lua_cmds[] = { #if (NGX_HTTP_SSL) -# if defined(nginx_version) && nginx_version >= 1001013 - { ngx_string("lua_ssl_protocols"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE, ngx_conf_set_bitmask_slot, @@ -524,8 +522,6 @@ static ngx_command_t ngx_http_lua_cmds[] = { offsetof(ngx_http_lua_loc_conf_t, ssl_protocols), &ngx_http_lua_ssl_protocols }, -# endif - { ngx_string("lua_ssl_ciphers"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, ngx_conf_set_str_slot, @@ -650,7 +646,7 @@ ngx_http_lua_init(ngx_conf_t *cf) volatile ngx_cycle_t *saved_cycle; ngx_http_core_main_conf_t *cmcf; ngx_http_lua_main_conf_t *lmcf; -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) ngx_pool_cleanup_t *cln; #endif ngx_str_t name = ngx_string("host"); @@ -754,7 +750,7 @@ ngx_http_lua_init(ngx_conf_t *cf) ngx_http_lua_pipe_init(); #endif -#if nginx_version >= 1011011 +#if defined(nginx_version) && (nginx_version >= 1011011) cln = ngx_pool_cleanup_add(cf->pool, 0); if (cln == NULL) { return NGX_ERROR; @@ -1232,15 +1228,11 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) #if (NGX_HTTP_SSL) -# if defined(nginx_version) && nginx_version >= 1001013 - ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3 |NGX_SSL_TLSv1|NGX_SSL_TLSv1_1 |NGX_SSL_TLSv1_2)); -# endif - ngx_conf_merge_str_value(conf->ssl_ciphers, prev->ssl_ciphers, "DEFAULT"); @@ -1328,26 +1320,13 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) return NGX_ERROR; } - if (llcf->ssl_trusted_certificate.len) { - -#if defined(nginx_version) && nginx_version >= 1003007 - - if (ngx_ssl_trusted_certificate(cf, llcf->ssl, - &llcf->ssl_trusted_certificate, - llcf->ssl_verify_depth) - != NGX_OK) - { - return NGX_ERROR; - } - -#else - - ngx_log_error(NGX_LOG_CRIT, cf->log, 0, "at least nginx 1.3.7 is " - "required for the \"lua_ssl_trusted_certificate\" " - "directive"); + if (llcf->ssl_trusted_certificate.len + && ngx_ssl_trusted_certificate(cf, llcf->ssl, + &llcf->ssl_trusted_certificate, + llcf->ssl_verify_depth) + != NGX_OK) + { return NGX_ERROR; - -#endif } dd("ssl crl: %.*s", (int) llcf->ssl_crl.len, llcf->ssl_crl.data); diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index e6bf3c192e..5fe4564e89 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -114,11 +114,6 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) rc = ngx_http_read_client_request_body(r, ngx_http_lua_req_body_post_read); -#if (nginx_version < 1002006) || \ - (nginx_version >= 1003000 && nginx_version < 1003009) - r->main->count--; -#endif - if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { ctx->exit_code = rc; ctx->exited = 1; @@ -130,11 +125,8 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) return lua_yield(L, 0); } -#if (nginx_version >= 1002006 && nginx_version < 1003000) || \ - nginx_version >= 1003009 r->main->count--; dd("decrement r->main->count: %d", (int) r->main->count); -#endif if (rc == NGX_AGAIN) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 558d6291a8..70a065f2c2 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -149,11 +149,6 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) ngx_http_lua_generic_phase_post_read); if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) { -#if (nginx_version < 1002006) || \ - (nginx_version >= 1003000 && nginx_version < 1003009) - r->main->count--; -#endif - return rc; } diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 85d36e3de3..2fd2a94c13 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -115,9 +115,7 @@ ngx_http_lua_shdict_init_zone(ngx_shm_zone_t *shm_zone, void *data) ngx_sprintf(ctx->shpool->log_ctx, " in lua_shared_dict zone \"%V\"%Z", &shm_zone->shm.name); -#if defined(nginx_version) && nginx_version >= 1005013 ctx->shpool->log_nomem = 0; -#endif return NGX_OK; } diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 3173ab1b94..d3453d456e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -749,9 +749,6 @@ ngx_http_lua_socket_tcp_connect_helper(lua_State *L, } rctx->name = host; -#if !defined(nginx_version) || nginx_version < 1005008 - rctx->type = NGX_RESOLVE_A; -#endif rctx->handler = ngx_http_lua_socket_resolve_handler; rctx->data = u; rctx->timeout = clcf->resolver_timeout; @@ -1135,12 +1132,8 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) ngx_http_lua_socket_tcp_upstream_t *u; u_char *p; size_t len; -#if defined(nginx_version) && nginx_version >= 1005008 socklen_t socklen; struct sockaddr *sockaddr; -#else - struct sockaddr_in *sin; -#endif ngx_uint_t i; unsigned waiting; @@ -1195,36 +1188,20 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) #if (NGX_DEBUG) { -# if defined(nginx_version) && nginx_version >= 1005008 - u_char text[NGX_SOCKADDR_STRLEN]; - ngx_str_t addr; -# else - in_addr_t addr; -# endif - ngx_uint_t i; - -# if defined(nginx_version) && nginx_version >= 1005008 - addr.data = text; + u_char text[NGX_SOCKADDR_STRLEN]; + ngx_str_t addr; + ngx_uint_t i; - for (i = 0; i < ctx->naddrs; i++) { - addr.len = ngx_sock_ntop(ur->addrs[i].sockaddr, ur->addrs[i].socklen, - text, NGX_SOCKADDR_STRLEN, 0); + addr.data = text; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "name was resolved to %V", &addr); - } -# else - for (i = 0; i < ctx->naddrs; i++) { - dd("addr i: %d %p", (int) i, &ctx->addrs[i]); + for (i = 0; i < ctx->naddrs; i++) { + addr.len = ngx_sock_ntop(ur->addrs[i].sockaddr, + ur->addrs[i].socklen, text, + NGX_SOCKADDR_STRLEN, 0); - addr = ntohl(ctx->addrs[i]); - - ngx_log_debug4(NGX_LOG_DEBUG_HTTP, c->log, 0, - "name was resolved to %ud.%ud.%ud.%ud", - (addr >> 24) & 0xff, (addr >> 16) & 0xff, - (addr >> 8) & 0xff, addr & 0xff); - } -# endif + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "name was resolved to %V", &addr); + } } #endif @@ -1239,7 +1216,6 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) dd("selected addr index: %d", (int) i); -#if defined(nginx_version) && nginx_version >= 1005008 socklen = ur->addrs[i].socklen; sockaddr = ngx_palloc(r->pool, socklen); @@ -1268,30 +1244,6 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) ur->sockaddr = sockaddr; ur->socklen = socklen; -#else - /* for nginx older than 1.5.8 */ - - len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1; - - p = ngx_pnalloc(r->pool, len + sizeof(struct sockaddr_in)); - if (p == NULL) { - goto nomem; - } - - sin = (struct sockaddr_in *) &p[len]; - ngx_memzero(sin, sizeof(struct sockaddr_in)); - - len = ngx_inet_ntop(AF_INET, &ur->addrs[i], p, NGX_INET_ADDRSTRLEN); - len = ngx_sprintf(&p[len], ":%d", ur->port) - p; - - sin->sin_family = AF_INET; - sin->sin_port = htons(ur->port); - sin->sin_addr.s_addr = ur->addrs[i]; - - ur->sockaddr = (struct sockaddr *) sin; - ur->socklen = sizeof(struct sockaddr_in); -#endif - ur->host.data = p; ur->host.len = len; ur->naddrs = 1; @@ -2054,11 +2006,7 @@ ngx_http_lua_socket_prepare_error_retvals(ngx_http_request_t *r, } else { if (u->socket_errno) { -#if defined(nginx_version) && nginx_version >= 9000 p = ngx_strerror(u->socket_errno, errstr, sizeof(errstr)); -#else - p = ngx_strerror_r(u->socket_errno, errstr, sizeof(errstr)); -#endif /* for compatibility with LuaSocket */ ngx_strlow(errstr, errstr, p - errstr); lua_pushlstring(L, (char *) errstr, p - errstr); @@ -3383,11 +3331,7 @@ ngx_http_lua_socket_send(ngx_http_request_t *r, } -#if defined(nginx_version) && nginx_version >= 1001004 ngx_chain_update_chains(r->pool, -#else - ngx_chain_update_chains( -#endif &ctx->free_bufs, &ctx->busy_bufs, &u->request_bufs, (ngx_buf_tag_t) &ngx_http_lua_module); @@ -4919,13 +4863,11 @@ ngx_http_lua_req_socket(lua_State *L) } #endif -#if nginx_version >= 1003009 if (!raw && r->headers_in.chunked) { lua_pushnil(L); lua_pushliteral(L, "chunked request bodies not supported yet"); return 2; } -#endif ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { @@ -4939,11 +4881,6 @@ ngx_http_lua_req_socket(lua_State *L) c = r->connection; if (raw) { -#if !defined(nginx_version) || nginx_version < 1003013 - lua_pushnil(L); - lua_pushliteral(L, "nginx version too old"); - return 2; -#else if (r->request_body) { if (r->request_body->rest > 0) { lua_pushnil(L); @@ -4991,7 +4928,6 @@ ngx_http_lua_req_socket(lua_State *L) ctx->acquired_raw_req_socket = 1; r->keepalive = 0; r->lingering_close = 1; -#endif } else { /* request body reader */ diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index cbc32d63ec..f35e00e67b 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -367,9 +367,6 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) } rctx->name = host; -#if !defined(nginx_version) || nginx_version < 1005008 - rctx->type = NGX_RESOLVE_A; -#endif rctx->handler = ngx_http_lua_socket_resolve_handler; rctx->data = u; rctx->timeout = clcf->resolver_timeout; @@ -437,12 +434,8 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) ngx_http_lua_socket_udp_upstream_t *u; u_char *p; size_t len; -#if defined(nginx_version) && nginx_version >= 1005008 socklen_t socklen; struct sockaddr *sockaddr; -#else - struct sockaddr_in *sin; -#endif ngx_uint_t i; unsigned waiting; @@ -502,36 +495,20 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) #if (NGX_DEBUG) { -# if defined(nginx_version) && nginx_version >= 1005008 - u_char text[NGX_SOCKADDR_STRLEN]; - ngx_str_t addr; -# else - in_addr_t addr; -# endif - ngx_uint_t i; - -# if defined(nginx_version) && nginx_version >= 1005008 - addr.data = text; + u_char text[NGX_SOCKADDR_STRLEN]; + ngx_str_t addr; + ngx_uint_t i; - for (i = 0; i < ctx->naddrs; i++) { - addr.len = ngx_sock_ntop(ur->addrs[i].sockaddr, ur->addrs[i].socklen, - text, NGX_SOCKADDR_STRLEN, 0); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "name was resolved to %V", &addr); - } -# else - for (i = 0; i < ctx->naddrs; i++) { - dd("addr i: %d %p", (int) i, &ctx->addrs[i]); + addr.data = text; - addr = ntohl(ctx->addrs[i]); + for (i = 0; i < ctx->naddrs; i++) { + addr.len = ngx_sock_ntop(ur->addrs[i].sockaddr, + ur->addrs[i].socklen, text, + NGX_SOCKADDR_STRLEN, 0); - ngx_log_debug4(NGX_LOG_DEBUG_HTTP, c->log, 0, - "name was resolved to %ud.%ud.%ud.%ud", - (addr >> 24) & 0xff, (addr >> 16) & 0xff, - (addr >> 8) & 0xff, addr & 0xff); - } -# endif + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "name was resolved to %V", &addr); + } } #endif @@ -546,7 +523,6 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) dd("selected addr index: %d", (int) i); -#if defined(nginx_version) && nginx_version >= 1005008 socklen = ur->addrs[i].socklen; sockaddr = ngx_palloc(r->pool, socklen); @@ -575,30 +551,6 @@ ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) ur->sockaddr = sockaddr; ur->socklen = socklen; -#else - /* for nginx older than 1.5.8 */ - - len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1; - - p = ngx_pnalloc(r->pool, len + sizeof(struct sockaddr_in)); - if (p == NULL) { - goto nomem; - } - - sin = (struct sockaddr_in *) &p[len]; - ngx_memzero(sin, sizeof(struct sockaddr_in)); - - len = ngx_inet_ntop(AF_INET, &ur->addrs[i], p, NGX_INET_ADDRSTRLEN); - len = ngx_sprintf(&p[len], ":%d", ur->port) - p; - - sin->sin_family = AF_INET; - sin->sin_port = htons(ur->port); - sin->sin_addr.s_addr = ur->addrs[i]; - - ur->sockaddr = (struct sockaddr *) sin; - ur->socklen = sizeof(struct sockaddr_in); -#endif - ur->host.data = p; ur->host.len = len; ur->naddrs = 1; @@ -763,11 +715,7 @@ ngx_http_lua_socket_error_retval_handler(ngx_http_request_t *r, } else { if (u->socket_errno) { -#if defined(nginx_version) && nginx_version >= 9000 p = ngx_strerror(u->socket_errno, errstr, sizeof(errstr)); -#else - p = ngx_strerror_r(u->socket_errno, errstr, sizeof(errstr)); -#endif /* for compatibility with LuaSocket */ ngx_strlow(errstr, errstr, p - errstr); lua_pushlstring(L, (char *) errstr, p - errstr); diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 21ac296fe5..6384d38a70 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -245,26 +245,11 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && nginx_version >= 1003014 - -# if nginx_version >= 1009000 - +#if defined(nginx_version) && (nginx_version >= 1009000) ngx_set_connection_log(fc, clcf->error_log); -# else - - ngx_http_set_connection_log(fc, clcf->error_log); - -# endif - #else - - fc->log->file = clcf->error_log->file; - - if (!(fc->log->log_level & NGX_LOG_DEBUG_CONNECTION)) { - fc->log->log_level = clcf->error_log->log_level; - } - + ngx_http_set_connection_log(fc, clcf->error_log); #endif if (cctx == NULL) { diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 4b15d61dd3..eddb80f213 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -267,26 +267,11 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && nginx_version >= 1003014 - -# if nginx_version >= 1009000 - +#if defined(nginx_version) && (nginx_version >= 1009000) ngx_set_connection_log(fc, clcf->error_log); -# else - - ngx_http_set_connection_log(fc, clcf->error_log); - -# endif - #else - - fc->log->file = clcf->error_log->file; - - if (!(fc->log->log_level & NGX_LOG_DEBUG_CONNECTION)) { - fc->log->log_level = clcf->error_log->log_level; - } - + ngx_http_set_connection_log(fc, clcf->error_log); #endif if (cctx == NULL) { diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 99662b3a3e..34e7474b82 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -229,26 +229,11 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && nginx_version >= 1003014 - -# if nginx_version >= 1009000 - +#if defined(nginx_version) && (nginx_version >= 1009000) ngx_set_connection_log(fc, clcf->error_log); -# else - - ngx_http_set_connection_log(fc, clcf->error_log); - -# endif - #else - - fc->log->file = clcf->error_log->file; - - if (!(fc->log->log_level & NGX_LOG_DEBUG_CONNECTION)) { - fc->log->log_level = clcf->error_log->log_level; - } - + ngx_http_set_connection_log(fc, clcf->error_log); #endif if (cctx == NULL) { diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index f312b65e0a..174460c5c4 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1081,11 +1081,7 @@ ngx_http_lua_post_subrequest(ngx_http_request_t *r, void *data, ngx_int_t rc) if (ctx->body) { -#if defined(nginx_version) && nginx_version >= 1001004 ngx_chain_update_chains(r->pool, -#else - ngx_chain_update_chains( -#endif &pr_ctx->free_bufs, &pr_ctx->busy_bufs, &ctx->body, (ngx_buf_tag_t) &ngx_http_lua_module); @@ -1457,7 +1453,7 @@ ngx_http_lua_subrequest(ngx_http_request_t *r, ngx_http_request_t *sr; ngx_http_core_srv_conf_t *cscf; -#if nginx_version >= 1009005 +#if defined(nginx_version) && (nginx_version >= 1009005) if (r->subrequests == 0) { #if defined(NGX_DTRACE) && NGX_DTRACE @@ -1576,7 +1572,7 @@ ngx_http_lua_subrequest(ngx_http_request_t *r, sr->uri_changes = NGX_HTTP_MAX_URI_CHANGES + 1; -#if nginx_version >= 1009005 +#if defined(nginx_version) && (nginx_version >= 1009005) sr->subrequests = r->subrequests - 1; #endif diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index e92c211458..ad88b9512f 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -613,26 +613,11 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && nginx_version >= 1003014 - -# if nginx_version >= 1009000 - +#if defined(nginx_version) && (nginx_version >= 1009000) ngx_set_connection_log(r->connection, clcf->error_log); -# else - - ngx_http_set_connection_log(r->connection, clcf->error_log); - -# endif - #else - - c->log->file = clcf->error_log->file; - - if (!(c->log->log_level & NGX_LOG_DEBUG_CONNECTION)) { - c->log->log_level = clcf->error_log->log_level; - } - + ngx_http_set_connection_log(r->connection, clcf->error_log); #endif dd("lmcf: %p", lmcf); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 6dd110af4b..5f18c89465 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -537,16 +537,6 @@ ngx_http_lua_send_chain_link(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, } } -#if defined(nginx_version) && nginx_version <= 8004 - - /* earlier versions of nginx does not allow subrequests - to send last_buf themselves */ - if (r != r->main) { - return NGX_OK; - } - -#endif - ctx->eof = 1; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, @@ -629,11 +619,7 @@ ngx_http_lua_output_filter(ngx_http_request_t *r, ngx_chain_t *in) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); -#if nginx_version >= 1001004 ngx_chain_update_chains(r->pool, -#else - ngx_chain_update_chains( -#endif &ctx->free_bufs, &ctx->busy_bufs, &in, (ngx_buf_tag_t) &ngx_http_lua_module); @@ -942,9 +928,7 @@ ngx_http_lua_generic_phase_post_read(ngx_http_request_t *r) ctx->read_body_done = 1; -#if defined(nginx_version) && nginx_version >= 8011 r->main->count--; -#endif if (ctx->waiting_more_body) { ctx->waiting_more_body = 0; @@ -2767,9 +2751,7 @@ ngx_http_lua_open_and_stat_file(u_char *name, ngx_open_file_info_t *of, of->uniq = ngx_file_uniq(&fi); of->mtime = ngx_file_mtime(&fi); of->size = ngx_file_size(&fi); -#if defined(nginx_version) && nginx_version >= 1000001 of->fs_size = ngx_file_fs_size(&fi); -#endif of->is_dir = ngx_is_dir(&fi); of->is_file = ngx_is_file(&fi); of->is_link = ngx_is_link(&fi); diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 98edb69349..8cc7a874a7 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -67,11 +67,6 @@ extern char ngx_http_lua_headers_metatable_key; #endif -#if defined(nginx_version) && nginx_version < 1000000 -#define ngx_memmove(dst, src, n) (void) memmove(dst, src, n) -#endif - - #define ngx_http_lua_context_name(c) \ ((c) == NGX_HTTP_LUA_CONTEXT_SET ? "set_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_REWRITE ? "rewrite_by_lua*" \ From d154e5a7c1cbb7810d0224862e577fc8a6b8e3ac Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 1 Aug 2019 12:34:26 -0700 Subject: [PATCH 351/848] doc: readme: removed all mentions of PUC-Rio Lua and fixed styling and grammatical issues. * Removed mentions of PUC-Rio Lua, or hints that this module may work with it. * Clarified that this module only supports LuaJIT 2.x starting with v0.10.16. * Clarified that the terms "Lua" and "LuaJIT" are used interchangeably in this document. * Consistently capitalized "Nginx". * Many styling and grammatical fixes. --- README.markdown | 579 ++++++++++++++++++++++++++++------------- doc/HttpLuaModule.wiki | 569 +++++++++++++++++++++++++++------------- 2 files changed, 785 insertions(+), 363 deletions(-) diff --git a/README.markdown b/README.markdown index 407a1d613a..7902550d5e 100644 --- a/README.markdown +++ b/README.markdown @@ -8,9 +8,11 @@ Name ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. -This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) +This module is a core component of [OpenResty](https://openresty.org). If you are using this module, +then you are essentially using OpenResty. -*This module is not distributed with the Nginx source.* See [the installation instructions](#installation). +*This module is not distributed with the Nginx source.* See +[the installation instructions](#installation). Table of Contents ================= @@ -30,7 +32,7 @@ Table of Contents * [Chinese Mailing List](#chinese-mailing-list) * [Code Repository](#code-repository) * [Bugs and Patches](#bugs-and-patches) -* [Lua/LuaJIT bytecode support](#lualuajit-bytecode-support) +* [LuaJIT bytecode support](#luajit-bytecode-support) * [System Environment Variable Support](#system-environment-variable-support) * [HTTP 1.0 support](#http-10-support) * [Statically Linking Pure Lua Modules](#statically-linking-pure-lua-modules) @@ -54,6 +56,7 @@ Table of Contents * [Nginx API for Lua](#nginx-api-for-lua) * [Obsolete Sections](#obsolete-sections) * [Special PCRE Sequences](#special-pcre-sequences) + * [Lua/LuaJIT bytecode support](#lualuajit-bytecode-support) Status ====== @@ -63,7 +66,9 @@ Production ready. Version ======= -This document describes ngx_lua [v0.10.16](https://github.com/openresty/lua-nginx-module/tags), which is not released yet. +This document describes ngx_lua +[v0.10.16](https://github.com/openresty/lua-nginx-module/tags), which is not +released yet. Synopsis ======== @@ -188,14 +193,27 @@ Synopsis Description =========== -This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) +This module embeds [LuaJIT 2.0/2.1](https://luajit.org/luajit.html) into Nginx. +It is a core component of [OpenResty](https://openresty.org). If you are using +this module, then you are essentially using OpenResty. -This module embeds Lua, via [LuaJIT 2.0/2.1](http://luajit.org/luajit.html), into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. +Since version `v0.10.16` of this module, the standard Lua +interpreter (also known as "PUC-Rio Lua") is not supported anymore. This +document interchangeably uses the terms "Lua" and "LuaJIT" to refer to the +LuaJIT interpreter. -Unlike [Apache's mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html) and [Lighttpd's mod_magnet](http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet), Lua code executed using this module can be *100% non-blocking* on network traffic as long as the [Nginx API for Lua](#nginx-api-for-lua) provided by this module is used to handle -requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services. +By leveraging Nginx's subrequests, this module allows the integration of the +powerful Lua threads (known as Lua "coroutines") into the Nginx event model. -At least the following Lua libraries and Nginx modules can be used with this ngx_lua module: +Unlike [Apache's mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html) +and [Lighttpd's mod_magnet](http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet), +Lua code executed using this module can be *100% non-blocking* on network +traffic as long as the [Nginx API for Lua](#nginx-api-for-lua) provided by +this module is used to handle requests to upstream services such as MySQL, +PostgreSQL, Memcached, Redis, or upstream HTTP web services. + +At least the following Lua libraries and Nginx modules can be used with this +module: * [lua-resty-memcached](https://github.com/openresty/lua-resty-memcached) * [lua-resty-mysql](https://github.com/openresty/lua-resty-mysql) @@ -214,15 +232,26 @@ At least the following Lua libraries and Nginx modules can be used with this ngx * [ngx_proxy](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) * [ngx_fastcgi](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html) -Almost all the Nginx modules can be used with this ngx_lua module by means of [ngx.location.capture](#ngxlocationcapture) or [ngx.location.capture_multi](#ngxlocationcapture_multi) but it is recommended to use those `lua-resty-*` libraries instead of creating subrequests to access the Nginx upstream modules because the former is usually much more flexible and memory-efficient. +Almost any Nginx modules can be used with this ngx_lua module by means of +[ngx.location.capture](#ngxlocationcapture) or +[ngx.location.capture_multi](#ngxlocationcapture_multi) but it is +recommended to use those `lua-resty-*` libraries instead of creating +subrequests to access the Nginx upstream modules because the former is usually +much more flexible and memory-efficient. -The Lua interpreter or LuaJIT instance is shared across all the requests in a single nginx worker process but request contexts are segregated using lightweight Lua coroutines. +The Lua interpreter (also known as "Lua State" or "LuaJIT VM instance") is +shared across all the requests in a single Nginx worker process to minimize +memory use. Request contexts are segregated using lightweight Lua coroutines. -Loaded Lua modules persist in the nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads. +Loaded Lua modules persist in the Nginx worker process level resulting in a +small memory footprint in Lua even when under heavy loads. -This module is plugged into NGINX's "http" subsystem so it can only speaks downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, and etc). -If you want to do generic TCP communications with the downstream clients, then you should use the [ngx_stream_lua](https://github.com/openresty/stream-lua-nginx-module#readme) module instead -which has a compatible Lua API. +This module is plugged into Nginx's "http" subsystem so it can only speaks +downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, +WebSockets, etc...). If you want to do generic TCP communications with the +downstream clients, then you should use the +[ngx_stream_lua](https://github.com/openresty/stream-lua-nginx-module#readme) +module instead, which offers a compatible Lua API. [Back to TOC](#table-of-contents) @@ -231,7 +260,7 @@ Typical Uses Just to name a few: -* Mashup'ing and processing outputs of various nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua, +* Mashup'ing and processing outputs of various Nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua, * doing arbitrarily complex access control and security checks in Lua before requests actually reach the upstream backends, * manipulating response headers in an arbitrary way (by Lua) * fetching backend information from external storage backends (like redis, memcached, mysql, postgresql) and use that information to choose which upstream backend to access on-the-fly, @@ -239,11 +268,14 @@ Just to name a few: * doing very complex URL dispatch in Lua at rewrite phase, * using Lua to implement advanced caching mechanism for Nginx's subrequests and arbitrary locations. -The possibilities are unlimited as the module allows bringing together various elements within Nginx as well as exposing the power of the Lua language to the user. The module provides the full flexibility of scripting while offering performance levels comparable with native C language programs both in terms of CPU time as well as memory footprint. This is particularly the case when LuaJIT 2.x is enabled. - -Other scripting language implementations typically struggle to match this performance level. +The possibilities are unlimited as the module allows bringing together various +elements within Nginx as well as exposing the power of the Lua language to the +user. The module provides the full flexibility of scripting while offering +performance levels comparable with native C language programs both in terms of +CPU time as well as memory footprint thanks to LuaJIT 2.x. -The Lua state (Lua VM instance) is shared across all the requests handled by a single nginx worker process to minimize memory use. +Other scripting language implementations typically struggle to match this +performance level. [Back to TOC](#table-of-contents) @@ -271,20 +303,32 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. Installation ============ -It is *highly* recommended to use [OpenResty releases](http://openresty.org) which integrate Nginx, ngx_lua, OpenResty's LuaJIT 2.1 branch version, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. +It is *highly* recommended to use [OpenResty releases](https://openresty.org) +which bundle Nginx, ngx_lua (this module), LuaJIT, as well as other powerful +companion Nginx modules and Lua libraries. + +It is discouraged to build this module with Nginx yourself since it is tricky +to set up exactly right. + +Note that Nginx, LuaJIT, and OpenSSL official releases have various limitations +and long standing bugs that can cause some of this module's features to be +disabled, not work properly, or run slower. Official OpenResty releases are +recommended because they bundle [OpenResty's optimized LuaJIT 2.1 fork](https://github.com/openresty/luajit2) and +[Nginx/OpenSSL +patches](https://github.com/openresty/openresty/tree/master/patches). Alternatively, ngx_lua can be manually compiled into Nginx: -1. LuaJIT can be downloaded from the [latest release of OpenResty's LuaJIT branch version](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.0 and 2.1 releases are also supported, although the performance will be significantly lower in many cases. +1. LuaJIT can be downloaded from the [latest release of OpenResty's LuaJIT fork](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above.. 1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags). 1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags). -1. Download the latest version of Nginx [HERE](http://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) +1. Download the latest version of Nginx [HERE](https://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) Build the source with this module: ```bash - wget 'http://nginx.org/download/nginx-1.13.6.tar.gz' + wget 'https://nginx.org/download/nginx-1.13.6.tar.gz' tar -xzvf nginx-1.13.6.tar.gz cd nginx-1.13.6/ @@ -296,13 +340,9 @@ Build the source with this module: export LUAJIT_LIB=/path/to/luajit/lib export LUAJIT_INC=/path/to/luajit/include/luajit-2.1 - # or tell where to find Lua if using Lua instead: - #export LUA_LIB=/path/to/lua/lib - #export LUA_INC=/path/to/lua/include - # Here we assume Nginx is to be installed under /opt/nginx/. ./configure --prefix=/opt/nginx \ - --with-ld-opt="-Wl,-rpath,/path/to/luajit-or-lua/lib" \ + --with-ld-opt="-Wl,-rpath,/path/to/luajit/lib" \ --add-module=/path/to/ngx_devel_kit \ --add-module=/path/to/lua-nginx-module @@ -322,7 +362,7 @@ Building as a dynamic module ---------------------------- Starting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the `--add-dynamic-module=PATH` option instead of `--add-module=PATH` on the -`./configure` command line above. And then you can explicitly load the module in your `nginx.conf` via the [load_module](http://nginx.org/en/docs/ngx_core_module.html#load_module) +`./configure` command line above. And then you can explicitly load the module in your `nginx.conf` via the [load_module](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive, for example, ```nginx @@ -336,16 +376,14 @@ directive, for example, C Macro Configurations ---------------------- -While building this module either via OpenResty or with the NGINX core, you can define the following C macros via the C compiler options: +While building this module either via OpenResty or with the Nginx core, you can define the following C macros via the C compiler options: * `NGX_LUA_USE_ASSERT` When defined, will enable assertions in the ngx_lua C code base. Recommended for debugging or testing builds. It can introduce some (small) runtime overhead when enabled. This macro was first introduced in the `v0.9.10` release. * `NGX_LUA_ABORT_AT_PANIC` - When the Lua/LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the `v0.9.8` release. -* `NGX_LUA_NO_FFI_API` - Excludes pure C API functions for FFI-based Lua API for NGINX (as required by [lua-resty-core](https://github.com/openresty/lua-resty-core#readme), for example). Enabling this macro can make the resulting binary code size smaller. + When the LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the `v0.9.8` release. -To enable one or more of these macros, just pass extra C compiler options to the `./configure` script of either NGINX or OpenResty. For instance, +To enable one or more of these macros, just pass extra C compiler options to the `./configure` script of either Nginx or OpenResty. For instance, ./configure --with-cc-opt="-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC" @@ -389,12 +427,10 @@ Please submit bug reports, wishlists, or patches by [Back to TOC](#table-of-contents) -Lua/LuaJIT bytecode support -=========================== +LuaJIT bytecode support +======================= -As from the `v0.5.0rc32` release, all `*_by_lua_file` configure directives (such as [content_by_lua_file](#content_by_lua_file)) support loading Lua 5.1 and LuaJIT 2.0/2.1 raw bytecode files directly. - -Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if you are using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: +As from the `v0.5.0rc32` release, all `*_by_lua_file` configure directives (such as [content_by_lua_file](#content_by_lua_file)) support loading LuaJIT 2.0/2.1 raw bytecode files directly: ```bash @@ -410,24 +446,29 @@ The `-bg` option can be used to include debug information in the LuaJIT bytecode Please refer to the official LuaJIT documentation on the `-b` option for more details: - + -Also, the bytecode files generated by LuaJIT 2.1 is *not* compatible with LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added in ngx_lua v0.9.3. +Note that the bytecode files generated by LuaJIT 2.1 is *not* compatible with +LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added +in ngx_lua v0.9.3. -Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked to LuaJIT 2.0/2.1 or vice versa, will result in an error message, such as that below, being logged into the Nginx `error.log` file: +Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked +to LuaJIT 2.0/2.1 (or vice versa) will result in an Nginx error message such as +the one below: [error] 13909#0: *1 failed to load Lua inlined code: bad byte-code header in /path/to/test_file.luac -Loading bytecode files via the Lua primitives like `require` and `dofile` should always work as expected. +Loading bytecode files via the Lua primitives like `require` and +`dofile` should always work as expected. [Back to TOC](#table-of-contents) System Environment Variable Support =================================== -If you want to access the system environment variable, say, `foo`, in Lua via the standard Lua API [os.getenv](http://www.lua.org/manual/5.1/manual.html#pdf-os.getenv), then you should also list this environment variable name in your `nginx.conf` file via the [env directive](http://nginx.org/en/docs/ngx_core_module.html#env). For example, +If you want to access the system environment variable, say, `foo`, in Lua via the standard Lua API [os.getenv](https://www.lua.org/manual/5.1/manual.html#pdf-os.getenv), then you should also list this environment variable name in your `nginx.conf` file via the [env directive](https://nginx.org/en/docs/ngx_core_module.html#env). For example, ```nginx @@ -455,9 +496,12 @@ To force `curl` to send HTTP 1.0 requests, use the `-0` option. Statically Linking Pure Lua Modules =================================== -When LuaJIT 2.x is used, it is possible to statically link the bytecode of pure Lua modules into the Nginx executable. +With LuaJIT 2.x, it is possible to statically link the bytecode of pure Lua +modules into the Nginx executable. -Basically you use the `luajit` executable to compile `.lua` Lua module files to `.o` object files containing the exported bytecode data, and then link the `.o` files directly in your Nginx build. +You can use the `luajit` executable to compile `.lua` Lua +module files to `.o` object files containing the exported bytecode +data, and then link the `.o` files directly in your Nginx build. Below is a trivial example to demonstrate this. Consider that we have the following `.lua` file named `foo.lua`: @@ -475,7 +519,10 @@ Below is a trivial example to demonstrate this. Consider that we have the follow And then we compile this `.lua` file to `foo.o` file: - /path/to/luajit/bin/luajit -bg foo.lua foo.o +```bash + + /path/to/luajit/bin/luajit -bg foo.lua foo.o +``` What matters here is the name of the `.lua` file, which determines how you use this module later on the Lua land. The file name `foo.o` does not matter at all except the `.o` file extension (which tells `luajit` what output format is used). If you want to strip the Lua debug information from the resulting bytecode, you can just specify the `-b` option above instead of `-bg`. @@ -514,7 +561,7 @@ When you have multiple `.lua` files to compile and link, then just specify their ./configure --with-ld-opt="/path/to/foo.o /path/to/bar.o" ... ``` -If you have just too many `.o` files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your `.o` files, as in +If you have too many `.o` files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your `.o` files, as in ```bash @@ -536,9 +583,17 @@ where `/path/to/lib` is the path of the directory containing the `libmyluafiles. Data Sharing within an Nginx Worker =================================== -To globally share data among all the requests handled by the same nginx worker process, encapsulate the shared data into a Lua module, use the Lua `require` builtin to import the module, and then manipulate the shared data in Lua. This works because required Lua modules are loaded only once and all coroutines will share the same copy of the module (both its code and data). Note however that Lua global variables (note, not module-level variables) WILL NOT persist between requests because of the one-coroutine-per-request isolation design. +To globally share data among all the requests handled by the same Nginx worker +process, encapsulate the shared data into a Lua module, use the Lua +`require` builtin to import the module, and then manipulate the +shared data in Lua. This works because required Lua modules are loaded only +once and all coroutines will share the same copy of the module (both its code +and data). + +Note that the use of global Lua variables is *strongly discouraged*, as it may +lead to unexpected race conditions between concurrent requests. -Here is a complete small example: +Here is a small example on sharing data within an Nginx worker via a Lua module: ```lua @@ -571,16 +626,16 @@ and then accessing it from `nginx.conf`: ``` The `mydata` module in this example will only be loaded and run on the first request to the location `/lua`, -and all subsequent requests to the same nginx worker process will use the reloaded instance of the +and all subsequent requests to the same Nginx worker process will use the reloaded instance of the module as well as the same copy of the data in it, until a `HUP` signal is sent to the Nginx master process to force a reload. This data sharing technique is essential for high performance Lua applications based on this module. -Note that this data sharing is on a *per-worker* basis and not on a *per-server* basis. That is, when there are multiple nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers. +Note that this data sharing is on a *per-worker* basis and not on a *per-server* basis. That is, when there are multiple Nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers. -It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each nginx worker process as +It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each Nginx worker process as long as there is *no* nonblocking I/O operations (including [ngx.sleep](#ngxsleep)) in the middle of your calculations. As long as you do not give the -control back to the nginx event loop and ngx_lua's light thread +control back to the Nginx event loop and ngx_lua's light thread scheduler (even implicitly), there can never be any race conditions in between. For this reason, always be very careful when you want to share changeable data on the worker level. Buggy optimizations can easily lead to hard-to-debug @@ -589,8 +644,8 @@ race conditions under load. If server-wide data sharing is required, then use one or more of the following approaches: 1. Use the [ngx.shared.DICT](#ngxshareddict) API provided by this module. -1. Use only a single nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine). -1. Use data storage mechanisms such as `memcached`, `redis`, `MySQL` or `PostgreSQL`. [The OpenResty bundle](http://openresty.org) associated with this module comes with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms. +1. Use only a single Nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine). +1. Use data storage mechanisms such as `memcached`, `redis`, `MySQL` or `PostgreSQL`. [The OpenResty official releases](https://openresty.org) come with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms. [Back to TOC](#table-of-contents) @@ -601,6 +656,7 @@ Known Issues TCP socket connect operation issues ----------------------------------- + The [tcpsock:connect](#tcpsockconnect) method may indicate `success` despite connection failures such as with `Connection Refused` errors. However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation. @@ -611,13 +667,15 @@ This issue is due to limitations in the Nginx event model and only appears to af Lua Coroutine Yielding/Resuming ------------------------------- + * Because Lua's `dofile` and `require` builtins are currently implemented as C functions in LuaJIT 2.0/2.1, if the Lua file being loaded by `dofile` or `require` invokes [ngx.location.capture*](#ngxlocationcapture), [ngx.exec](#ngxexec), [ngx.exit](#ngxexit), or other API functions requiring yielding in the *top-level* scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file. [Back to TOC](#table-of-contents) Lua Variable Scope ------------------ -Care must be taken when importing modules and this form should be used: + +Care must be taken when importing modules, and this form should be used: ```lua @@ -654,7 +712,6 @@ It is therefore *highly* recommended to always declare such within an appropriat local function foo() return 123 end ``` - To find all instances of Lua global variables in your Lua code, run the [lua-releng tool](https://github.com/openresty/nginx-devel-utils/blob/master/lua-releng) across all `.lua` source files: $ lua-releng @@ -671,7 +728,8 @@ This tool will guarantee that local variables in the Lua module functions are al Locations Configured by Subrequest Directives of Other Modules -------------------------------------------------------------- -The [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) directives cannot capture locations that include the [add_before_body](http://nginx.org/en/docs/http/ngx_http_addition_module.html#add_before_body), [add_after_body](http://nginx.org/en/docs/http/ngx_http_addition_module.html#add_after_body), [auth_request](http://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request), [echo_location](http://github.com/openresty/echo-nginx-module#echo_location), [echo_location_async](http://github.com/openresty/echo-nginx-module#echo_location_async), [echo_subrequest](http://github.com/openresty/echo-nginx-module#echo_subrequest), or [echo_subrequest_async](http://github.com/openresty/echo-nginx-module#echo_subrequest_async) directives. + +The [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) directives cannot capture locations that include the [add_before_body](http://nginx.org/en/docs/http/ngx_http_addition_module.html#add_before_body), [add_after_body](http://nginx.org/en/docs/http/ngx_http_addition_module.html#add_after_body), [auth_request](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request), [echo_location](http://github.com/openresty/echo-nginx-module#echo_location), [echo_location_async](http://github.com/openresty/echo-nginx-module#echo_location_async), [echo_subrequest](http://github.com/openresty/echo-nginx-module#echo_subrequest), or [echo_subrequest_async](http://github.com/openresty/echo-nginx-module#echo_subrequest_async) directives. ```nginx @@ -700,11 +758,11 @@ will not work as expected. Cosockets Not Available Everywhere ---------------------------------- -Due to internal limitations in the nginx core, the cosocket API is disabled in the following contexts: [set_by_lua*](#set_by_lua), [log_by_lua*](#log_by_lua), [header_filter_by_lua*](#header_filter_by_lua), and [body_filter_by_lua](#body_filter_by_lua). +Due to internal limitations in the Nginx core, the cosocket API is disabled in the following contexts: [set_by_lua*](#set_by_lua), [log_by_lua*](#log_by_lua), [header_filter_by_lua*](#header_filter_by_lua), and [body_filter_by_lua](#body_filter_by_lua). -The cosockets are currently also disabled in the [init_by_lua*](#init_by_lua) and [init_worker_by_lua*](#init_worker_by_lua) directive contexts but we may add support for these contexts in the future because there is no limitation in the nginx core (or the limitation might be worked around). +The cosockets are currently also disabled in the [init_by_lua*](#init_by_lua) and [init_worker_by_lua*](#init_worker_by_lua) directive contexts but we may add support for these contexts in the future because there is no limitation in the Nginx core (or the limitation might be worked around). -There exists a work-around, however, when the original context does *not* need to wait for the cosocket results. That is, creating a zero-delay timer via the [ngx.timer.at](#ngxtimerat) API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. +There exists a workaround, however, when the original context does *not* need to wait for the cosocket results. That is, creating a zero-delay timer via the [ngx.timer.at](#ngxtimerat) API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. [Back to TOC](#table-of-contents) @@ -713,7 +771,7 @@ Special Escaping Sequences **NOTE** Following the `v0.9.17` release, this pitfall can be avoided by using the `*_by_lua_block {}` configuration directives. -PCRE sequences such as `\d`, `\s`, or `\w`, require special attention because in string literals, the backslash character, `\`, is stripped out by both the Lua language parser and by the nginx config file parser before processing if not within a `*_by_lua_block {}` directive. So the following snippet will not work as expected: +PCRE sequences such as `\d`, `\s`, or `\w`, require special attention because in string literals, the backslash character, `\`, is stripped out by both the Lua language parser and by the Nginx config file parser before processing if not within a `*_by_lua_block {}` directive. So the following snippet will not work as expected: ```nginx @@ -816,7 +874,6 @@ As noted earlier, PCRE sequences presented within `*_by_lua_block {}` directives # evaluates to "1234" ``` - [Back to TOC](#table-of-contents) Mixing with SSI Not Supported @@ -865,7 +922,7 @@ TODO * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option * use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), [ngx.header.HEADER](#ngxheaderheader), and etc. -* add directives to run Lua codes when nginx stops. +* add directives to run Lua codes when Nginx stops. * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add `stat` mode similar to [mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html). @@ -878,7 +935,7 @@ Changes The changes made in every release of this module are listed in the change logs of the OpenResty bundle: - + [Back to TOC](#table-of-contents) @@ -936,7 +993,7 @@ To run specific test files: To run a specific test block in a particular test file, add the line `--- ONLY` to the test block you want to run, and then use the `prove` utility to run that `.t` file. -There are also various testing modes based on mockeagain, valgrind, and etc. Refer to the [Test::Nginx documentation](http://search.cpan.org/perldoc?Test::Nginx) for more details for various advanced testing modes. See also the test reports for the Nginx test cluster running on Amazon EC2: . +There are also various testing modes based on mockeagain, valgrind, and etc. Refer to the [Test::Nginx documentation](https://search.cpan.org/perldoc?Test::Nginx) for more details for various advanced testing modes. See also the test reports for the Nginx test cluster running on Amazon EC2: . [Back to TOC](#table-of-contents) @@ -964,26 +1021,26 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND See Also ======== -* [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme) for an official port of this module for the NGINX "stream" subsystem (doing generic downstream TCP communications). +* [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme) for an official port of this module for the Nginx "stream" subsystem (doing generic downstream TCP communications). * [lua-resty-memcached](https://github.com/openresty/lua-resty-memcached) library based on ngx_lua cosocket. * [lua-resty-redis](https://github.com/openresty/lua-resty-redis) library based on ngx_lua cosocket. * [lua-resty-mysql](https://github.com/openresty/lua-resty-mysql) library based on ngx_lua cosocket. * [lua-resty-upload](https://github.com/openresty/lua-resty-upload) library based on ngx_lua cosocket. * [lua-resty-dns](https://github.com/openresty/lua-resty-dns) library based on ngx_lua cosocket. * [lua-resty-websocket](https://github.com/openresty/lua-resty-websocket) library for both WebSocket server and client, based on ngx_lua cosocket. -* [lua-resty-string](https://github.com/openresty/lua-resty-string) library based on [LuaJIT FFI](http://luajit.org/ext_ffi.html). +* [lua-resty-string](https://github.com/openresty/lua-resty-string) library based on [LuaJIT FFI](https://luajit.org/ext_ffi.html). * [lua-resty-lock](https://github.com/openresty/lua-resty-lock) library for a nonblocking simple lock API. * [lua-resty-cookie](https://github.com/cloudflare/lua-resty-cookie) library for HTTP cookie manipulation. -* [Routing requests to different MySQL queries based on URI arguments](http://openresty.org/#RoutingMySQLQueriesBasedOnURIArgs) -* [Dynamic Routing Based on Redis and Lua](http://openresty.org/#DynamicRoutingBasedOnRedis) -* [Using LuaRocks with ngx_lua](http://openresty.org/#UsingLuaRocks) +* [Routing requests to different MySQL queries based on URI arguments](https://openresty.org/#RoutingMySQLQueriesBasedOnURIArgs) +* [Dynamic Routing Based on Redis and Lua](https://openresty.org/#DynamicRoutingBasedOnRedis) +* [Using LuaRocks with ngx_lua](https://openresty.org/#UsingLuaRocks) * [Introduction to ngx_lua](https://github.com/openresty/lua-nginx-module/wiki/Introduction) * [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) * [echo-nginx-module](http://github.com/openresty/echo-nginx-module) * [drizzle-nginx-module](http://github.com/openresty/drizzle-nginx-module) * [postgres-nginx-module](https://github.com/FRiCKLE/ngx_postgres) * [memc-nginx-module](http://github.com/openresty/memc-nginx-module) -* [The OpenResty bundle](http://openresty.org) +* [The OpenResty bundle](https://openresty.org) * [Nginx Systemtap Toolkit](https://github.com/openresty/nginx-systemtap-toolkit) [Back to TOC](#table-of-contents) @@ -1089,14 +1146,15 @@ used to optionally load the `resty.core` module. lua_capture_error_log --------------------- + **syntax:** *lua_capture_error_log size* **default:** *none* **context:** *http* -Enables a buffer of the specified `size` for capturing all the nginx error log message data (not just those produced -by this module or the nginx http subsystem, but everything) without touching files or disks. +Enables a buffer of the specified `size` for capturing all the Nginx error log message data (not just those produced +by this module or the Nginx http subsystem, but everything) without touching files or disks. You can use units like `k` and `m` in the `size` value, as in @@ -1121,15 +1179,15 @@ also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough. -Note that the log level specified in the standard [error_log](http://nginx.org/r/error_log) directive +Note that the log level specified in the standard [error_log](https://nginx.org/r/error_log) directive *does* have effect on this capturing facility. It only captures log -messages of a level no lower than the specified log level in the [error_log](http://nginx.org/r/error_log) directive. +messages of a level no lower than the specified log level in the [error_log](https://nginx.org/r/error_log) directive. The user can still choose to set an even higher filtering log level on the fly via the Lua API function [errlog.set_filter_level](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_filter_level). -So it is more flexible than the static [error_log](http://nginx.org/r/error_log) directive. +So it is more flexible than the static [error_log](https://nginx.org/r/error_log) directive. It is worth noting that there is no way to capture the debugging logs -without building OpenResty or NGINX with the `./configure` +without building OpenResty or Nginx with the `./configure` option `--with-debug`. And enabling debugging logs is strongly discouraged in production builds due to high overhead. @@ -1139,13 +1197,14 @@ This directive was first introduced in the `v0.10.9` release. lua_use_default_type -------------------- + **syntax:** *lua_use_default_type on | off* **default:** *lua_use_default_type on* **context:** *http, server, location, location if* -Specifies whether to use the MIME type specified by the [default_type](http://nginx.org/en/docs/http/ngx_http_core_module.html#default_type) directive for the default value of the `Content-Type` response header. Deactivate this directive if a default `Content-Type` response header for Lua request handlers is not desired. +Specifies whether to use the MIME type specified by the [default_type](https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type) directive for the default value of the `Content-Type` response header. Deactivate this directive if a default `Content-Type` response header for Lua request handlers is not desired. This directive is turned on by default. @@ -1155,6 +1214,7 @@ This directive was first introduced in the `v0.9.1` release. lua_malloc_trim --------------- + **syntax:** *lua_malloc_trim <request-count>* **default:** *lua_malloc_trim 1000* @@ -1162,7 +1222,7 @@ lua_malloc_trim **context:** *http* Asks the underlying `libc` runtime library to release its cached free memory back to the operating system every -`N` requests processed by the NGINX core. By default, `N` is 1000. You can configure the request count +`N` requests processed by the Nginx core. By default, `N` is 1000. You can configure the request count by using your own numbers. Smaller numbers mean more frequent releases, which may introduce higher CPU time consumption and smaller memory footprint while larger numbers usually lead to less CPU time overhead and relatively larger memory footprint. Just tune the number for your own use cases. @@ -1174,8 +1234,8 @@ Configuring the argument to `0` essentially turns off the periodical memory trim lua_malloc_trim 0; # turn off trimming completely ``` -The current implementation uses an NGINX log phase handler to do the request counting. So the appearance of the -[log_subrequest on](http://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest) directives in `nginx.conf` +The current implementation uses an Nginx log phase handler to do the request counting. So the appearance of the +[log_subrequest on](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest) directives in `nginx.conf` may make the counting faster when subrequests are involved. By default, only "main requests" count. Note that this directive does *not* affect the memory allocated by LuaJIT's own allocator based on the `mmap` @@ -1223,6 +1283,7 @@ development as it has a significant negative impact on overall performance. For lua_regex_cache_max_entries --------------------------- + **syntax:** *lua_regex_cache_max_entries <num>* **default:** *lua_regex_cache_max_entries 1024* @@ -1247,6 +1308,7 @@ Do not activate the `o` option for regular expressions (and/or `replace` string lua_regex_match_limit --------------------- + **syntax:** *lua_regex_match_limit <num>* **default:** *lua_regex_match_limit 0* @@ -1353,9 +1415,9 @@ You can also initialize the [lua_shared_dict](#lua_shared_dict) shm storage at t But note that, the [lua_shared_dict](#lua_shared_dict)'s shm storage will not be cleared through a config reload (via the `HUP` signal, for example). So if you do *not* want to re-initialize the shm storage in your `init_by_lua` code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your `init_by_lua` code. -Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the [Copy-on-write (COW)](http://en.wikipedia.org/wiki/Copy-on-write) feature provided by many operating systems among all the worker processes, thus saving a lot of memory. +Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the [Copy-on-write (COW)](https://en.wikipedia.org/wiki/Copy-on-write) feature provided by many operating systems among all the worker processes, thus saving a lot of memory. -Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [Lua Variable Scope](#lua-variable-scope) section for more details). The recommended way is to use proper [Lua module](http://www.lua.org/manual/5.1/manual.html#5.3) files (but do not use the standard Lua function [module()](http://www.lua.org/manual/5.1/manual.html#pdf-module) to define Lua modules because it pollutes the global namespace as well) and call [require()](http://www.lua.org/manual/5.1/manual.html#pdf-require) to load your own module files in `init_by_lua` or other contexts ([require()](http://www.lua.org/manual/5.1/manual.html#pdf-require) does cache the loaded Lua modules in the global `package.loaded` table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). +Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [Lua Variable Scope](#lua-variable-scope) section for more details). The recommended way is to use proper [Lua module](https://www.lua.org/manual/5.1/manual.html#5.3) files (but do not use the standard Lua function [module()](https://www.lua.org/manual/5.1/manual.html#pdf-module) to define Lua modules because it pollutes the global namespace as well) and call [require()](https://www.lua.org/manual/5.1/manual.html#pdf-require) to load your own module files in `init_by_lua` or other contexts ([require()](https://www.lua.org/manual/5.1/manual.html#pdf-require) does cache the loaded Lua modules in the global `package.loaded` table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). Only a small set of the [Nginx API for Lua](#nginx-api-for-lua) is supported in this context: @@ -1383,7 +1445,7 @@ init_by_lua_block Similar to the [init_by_lua](#init_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1408,7 +1470,7 @@ init_by_lua_file **phase:** *loading-config* -Equivalent to [init_by_lua](#init_by_lua), except that the file specified by `` contains the Lua code or [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [init_by_lua](#init_by_lua), except that the file specified by `` contains the Lua code or [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -1476,7 +1538,7 @@ init_worker_by_lua_block Similar to the [init_worker_by_lua](#init_worker_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1584,7 +1646,7 @@ set_by_lua_block Similar to the [set_by_lua](#set_by_lua) directive except that 1. this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping), and 1. this directive does not support extra arguments after the Lua script as in [set_by_lua](#set_by_lua). @@ -1604,13 +1666,14 @@ This directive was first introduced in the `v0.9.17` release. set_by_lua_file --------------- + **syntax:** *set_by_lua_file $res <path-to-lua-script-file> [$arg1 $arg2 ...]* **context:** *server, server if, location, location if* **phase:** *rewrite* -Equivalent to [set_by_lua](#set_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [set_by_lua](#set_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variable interpolation is supported in the `` argument string of this directive. But special care must be taken for injection attacks. @@ -1654,7 +1717,7 @@ content_by_lua_block Similar to the [content_by_lua](#content_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1679,7 +1742,7 @@ content_by_lua_file **phase:** *content* -Equivalent to [content_by_lua](#content_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [content_by_lua](#content_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1811,7 +1874,7 @@ can be implemented in ngx_lua as: Just as any other rewrite phase handlers, [rewrite_by_lua](#rewrite_by_lua) also runs in subrequests. -Note that when calling `ngx.exit(ngx.OK)` within a [rewrite_by_lua](#rewrite_by_lua) handler, the nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [rewrite_by_lua](#rewrite_by_lua) handler, calling [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. +Note that when calling `ngx.exit(ngx.OK)` within a [rewrite_by_lua](#rewrite_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [rewrite_by_lua](#rewrite_by_lua) handler, calling [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. If the [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s [rewrite](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) directive is used to change the URI and initiate location re-lookups (internal redirections), then any [rewrite_by_lua](#rewrite_by_lua) or [rewrite_by_lua_file](#rewrite_by_lua_file) code sequences within the current location will not be executed. For example, @@ -1843,7 +1906,7 @@ rewrite_by_lua_block Similar to the [rewrite_by_lua](#rewrite_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1868,7 +1931,7 @@ rewrite_by_lua_file **phase:** *rewrite tail* -Equivalent to [rewrite_by_lua](#rewrite_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [rewrite_by_lua](#rewrite_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1953,11 +2016,11 @@ can be implemented in ngx_lua as: As with other access phase handlers, [access_by_lua](#access_by_lua) will *not* run in subrequests. -Note that when calling `ngx.exit(ngx.OK)` within a [access_by_lua](#access_by_lua) handler, the nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [access_by_lua](#access_by_lua) handler, calling [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. +Note that when calling `ngx.exit(ngx.OK)` within a [access_by_lua](#access_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [access_by_lua](#access_by_lua) handler, calling [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. Starting from the `v0.9.20` release, you can use the [access_by_lua_no_postpone](#access_by_lua_no_postpone) directive to control when to run this handler inside the "access" request-processing phase -of NGINX. +of Nginx. [Back to TOC](#directives) @@ -1972,7 +2035,7 @@ access_by_lua_block Similar to the [access_by_lua](#access_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1997,7 +2060,7 @@ access_by_lua_file **phase:** *access tail* -Equivalent to [access_by_lua](#access_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [access_by_lua](#access_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -2056,7 +2119,7 @@ header_filter_by_lua_block Similar to the [header_filter_by_lua](#header_filter_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -2081,7 +2144,7 @@ header_filter_by_lua_file **phase:** *output-header-filter* -Equivalent to [header_filter_by_lua](#header_filter_by_lua), except that the file specified by `` contains the Lua code, or as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [header_filter_by_lua](#header_filter_by_lua), except that the file specified by `` contains the Lua code, or as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2168,7 +2231,7 @@ When the Lua code may change the length of the response body, then it is require } ``` -Note that the following API functions are currently disabled within this context due to the limitations in NGINX output filter's current implementation: +Note that the following API functions are currently disabled within this context due to the limitations in Nginx output filter's current implementation: * Output API functions (e.g., [ngx.say](#ngxsay) and [ngx.send_headers](#ngxsend_headers)) * Control API functions (e.g., [ngx.exit](#ngxexit) and [ngx.exec](#ngxexec)) @@ -2192,7 +2255,7 @@ body_filter_by_lua_block Similar to the [body_filter_by_lua](#body_filter_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -2217,7 +2280,7 @@ body_filter_by_lua_file **phase:** *output-body-filter* -Equivalent to [body_filter_by_lua](#body_filter_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [body_filter_by_lua](#body_filter_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2303,7 +2366,7 @@ log_by_lua_block Similar to the [log_by_lua](#log_by_lua) directive except that this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an NGINX string literal (which requires +inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -2328,7 +2391,7 @@ log_by_lua_file **phase:** *log* -Equivalent to [log_by_lua](#log_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [log_by_lua](#log_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2367,13 +2430,13 @@ For instance, } ``` -The resulting Lua load balancer can work with any existing nginx upstream modules -like [ngx_proxy](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and -[ngx_fastcgi](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). +The resulting Lua load balancer can work with any existing Nginx upstream modules +like [ngx_proxy](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) and +[ngx_fastcgi](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html). Also, the Lua load balancer can work with the standard upstream connection pool mechanism, -i.e., the standard [keepalive](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. -Just ensure that the [keepalive](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive +i.e., the standard [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. +Just ensure that the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive is used *after* this `balancer_by_lua_block` directive in a single `upstream {}` configuration block. The Lua load balancer can totally ignore the list of servers defined in the `upstream {}` block @@ -2382,8 +2445,8 @@ and select peer from a completely dynamic server list (even changing per request from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. The Lua code handler registered by this directive might get called more than once in a single -downstream request when the nginx upstream mechanism retries the request on conditions -specified by directives like the [proxy_next_upstream](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) +downstream request when the Nginx upstream mechanism retries the request on conditions +specified by directives like the [proxy_next_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) directive. This Lua code execution context does not support yielding, so Lua APIs that may yield @@ -2405,7 +2468,7 @@ balancer_by_lua_file **phase:** *content* -Equivalent to [balancer_by_lua_block](#balancer_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [balancer_by_lua_block](#balancer_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2450,7 +2513,7 @@ ssl_certificate_by_lua_block **phase:** *right-before-SSL-handshake* -This directive runs user Lua code when NGINX is about to start the SSL handshake for the downstream +This directive runs user Lua code when Nginx is about to start the SSL handshake for the downstream SSL (https) connections. It is particularly useful for setting the SSL certificate chain and the corresponding private key on a per-request @@ -2472,9 +2535,9 @@ library are particularly useful in this context. You can use the Lua API offered to manipulate the SSL certificate chain and private key for the current SSL connection being initiated. -This Lua handler does not run at all, however, when NGINX/OpenSSL successfully resumes +This Lua handler does not run at all, however, when Nginx/OpenSSL successfully resumes the SSL session via SSL session IDs or TLS session tickets for the current SSL connection. In -other words, this Lua handler only runs when NGINX has to initiate a full SSL handshake. +other words, this Lua handler only runs when Nginx has to initiate a full SSL handshake. Below is a trivial example using the [ngx.ssl](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md) module @@ -2507,8 +2570,8 @@ This Lua code execution context *does* support yielding, so Lua APIs that may yi (like cosockets, sleeping, and "light threads") are enabled in this context. -Note, however, you still need to configure the [ssl_certificate](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate) and -[ssl_certificate_key](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key) +Note, however, you still need to configure the [ssl_certificate](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate) and +[ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key) directives even though you will not use this static certificate and private key at all. This is because the NGINX core requires their appearance otherwise you are seeing the following error while starting NGINX: @@ -2520,7 +2583,7 @@ while starting NGINX: This directive requires OpenSSL 1.0.2e or greater. If you are using the [official pre-built -packages](http://openresty.org/en/linux-packages.html) for +packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.9.7.2 or later, then everything should work out of the box. @@ -2531,9 +2594,9 @@ in order to use this directive: -Similarly, if you are not using the NGINX core shipped with +Similarly, if you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.9.7.2 or later, you will need to apply -patches to the standard NGINX core: +patches to the standard Nginx core: @@ -2550,7 +2613,7 @@ ssl_certificate_by_lua_file **phase:** *right-before-SSL-handshake* -Equivalent to [ssl_certificate_by_lua_block](#ssl_certificate_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [ssl_certificate_by_lua_block](#ssl_certificate_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2594,7 +2657,7 @@ When [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block) is specified at th this hook usually runs before [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block). When the SSL session is found and successfully loaded for the current SSL connection, SSL session resumption will happen and thus bypass the [ssl_certificate_by_lua*](#ssl_certificate_by_lua_block) -hook completely. In this case, NGINX also bypasses the [ssl_session_store_by_lua*](#ssl_session_store_by_lua_block) +hook completely. In this case, Nginx also bypasses the [ssl_session_store_by_lua*](#ssl_session_store_by_lua_block) hook, for obvious reasons. To easily test this hook locally with a modern web browser, you can temporarily put the following line @@ -2604,7 +2667,7 @@ in your https server block to disable the TLS session ticket support: But do not forget to comment this line out before publishing your site to the world. -If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) +If you are using the [official pre-built packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. If you are not using one of the [OpenSSL @@ -2614,9 +2677,9 @@ in order to use this directive: -Similarly, if you are not using the NGINX core shipped with +Similarly, if you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply -patches to the standard NGINX core: +patches to the standard Nginx core: @@ -2637,7 +2700,7 @@ ssl_session_fetch_by_lua_file **phase:** *right-before-SSL-handshake* -Equivalent to [ssl_session_fetch_by_lua_block](#ssl_session_fetch_by_lua_block), except that the file specified by `` contains the Lua code, or rather, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [ssl_session_fetch_by_lua_block](#ssl_session_fetch_by_lua_block), except that the file specified by `` contains the Lua code, or rather, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2694,7 +2757,7 @@ ssl_session_store_by_lua_file **phase:** *right-after-SSL-handshake* -Equivalent to [ssl_session_store_by_lua_block](#ssl_session_store_by_lua_block), except that the file specified by `` contains the Lua code, or rather, the [Lua/LuaJIT bytecode](#lualuajit-bytecode-support) to be executed. +Equivalent to [ssl_session_store_by_lua_block](#ssl_session_store_by_lua_block), except that the file specified by `` contains the Lua code, or rather, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2718,7 +2781,7 @@ lua_shared_dict Declares a shared memory zone, ``, to serve as storage for the shm based Lua dictionary `ngx.shared.`. -Shared memory zones are always shared by all the nginx worker processes in the current nginx server instance. +Shared memory zones are always shared by all the Nginx worker processes in the current Nginx server instance. The `` argument accepts size units such as `k` and `m`: @@ -2837,7 +2900,7 @@ Default to 30 connections for every pool. When the connection pool exceeds the available size limit, the least recently used (idle) connection already in the pool will be closed to make room for the current connection. -Note that the cosocket connection pool is per nginx worker process rather than per nginx server instance, so size limit specified here also applies to every single nginx worker process. +Note that the cosocket connection pool is per Nginx worker process rather than per Nginx server instance, so size limit specified here also applies to every single Nginx worker process. This directive was first introduced in the `v0.5.0rc1` release. @@ -2869,7 +2932,7 @@ lua_socket_log_errors **context:** *http, server, location* -This directive can be used to toggle error logging when a failure occurs for the TCP or UDP cosockets. If you are already doing proper error handling and logging in your Lua code, then it is recommended to turn this directive off to prevent data flushing in your nginx error log files (which is usually rather expensive). +This directive can be used to toggle error logging when a failure occurs for the TCP or UDP cosockets. If you are already doing proper error handling and logging in your Lua code, then it is recommended to turn this directive off to prevent data flushing in your Nginx error log files (which is usually rather expensive). This directive was first introduced in the `v0.5.13` release. @@ -3106,9 +3169,9 @@ lua_sa_restart **context:** *http* -When enabled, this module will set the `SA_RESTART` flag on nginx workers signal dispositions. +When enabled, this module will set the `SA_RESTART` flag on Nginx workers signal dispositions. -This allows Lua I/O primitives to not be interrupted by nginx's handling of various signals. +This allows Lua I/O primitives to not be interrupted by Nginx's handling of various signals. This directive was first introduced in the `v0.10.14` release. @@ -3272,6 +3335,7 @@ Nginx API for Lua Introduction ------------ + The various `*_by_lua`, `*_by_lua_block` and `*_by_lua_file` configuration directives serve as gateways to the Lua API within the `nginx.conf` file. The Nginx Lua API described below can only be called within the user Lua code run in the context of these configuration directives. The API is exposed to Lua in the form of two standard packages `ngx` and `ndk`. These packages are in the default global scope within ngx_lua and are always available within ngx_lua directives. @@ -3291,7 +3355,7 @@ The packages can be introduced into external Lua modules like this: return _M ``` -Use of the [package.seeall](http://www.lua.org/manual/5.1/manual.html#pdf-package.seeall) flag is strongly discouraged due to its various bad side-effects. +Use of the [package.seeall](https://www.lua.org/manual/5.1/manual.html#pdf-package.seeall) flag is strongly discouraged due to its various bad side-effects. It is also possible to directly require the packages in external Lua modules: @@ -3309,6 +3373,7 @@ Network I/O operations in user code should only be done through the Nginx Lua AP ngx.arg ------- + **syntax:** *val = ngx.arg\[index\]* **context:** *set_by_lua*, body_filter_by_lua** @@ -3346,6 +3411,7 @@ The data chunk and "eof" flag passed to the downstream Nginx output filters can ngx.var.VARIABLE ---------------- + **syntax:** *ngx.var.VAR_NAME* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** @@ -3358,7 +3424,7 @@ Read and write Nginx variable values. ngx.var.some_nginx_variable_name = value ``` -Note that only already defined nginx variables can be written to. +Note that only already defined Nginx variables can be written to. For example: ```nginx @@ -3372,9 +3438,9 @@ For example: } ``` -That is, nginx variables cannot be created on-the-fly. +That is, Nginx variables cannot be created on-the-fly. -Some special nginx variables like `$args` and `$limit_rate` can be assigned a value, +Some special Nginx variables like `$args` and `$limit_rate` can be assigned a value, many others are not, like `$query_string`, `$arg_PARAMETER`, and `$http_NAME`. Nginx regex group capturing variables `$1`, `$2`, `$3`, and etc, can be read by this @@ -3397,7 +3463,7 @@ Setting `ngx.var.Foo` to a `nil` value will unset the `$Foo` Nginx variable. to prevent (temporary) memory leaking within the current request's lifetime. Another way of caching the result is to use the [ngx.ctx](#ngxctx) table. -Undefined NGINX variables are evaluated to `nil` while uninitialized (but defined) NGINX variables are evaluated to an empty Lua string. +Undefined Nginx variables are evaluated to `nil` while uninitialized (but defined) Nginx variables are evaluated to an empty Lua string. This API requires a relatively expensive metamethod call and it is recommended to avoid using it on hot code paths. @@ -3405,6 +3471,7 @@ This API requires a relatively expensive metamethod call and it is recommended t Core constants -------------- + **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```lua @@ -3431,6 +3498,7 @@ The `ngx.DECLINED` constant was first introduced in the `v0.5.0rc19` release. HTTP method constants --------------------- + **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -3457,6 +3525,7 @@ These constants are usually used in [ngx.location.capture](#ngxlocationcapture) HTTP status constants --------------------- + **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```nginx @@ -3502,6 +3571,7 @@ HTTP status constants Nginx log level constants ------------------------- + **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** ```lua @@ -3523,11 +3593,12 @@ These constants are usually used by the [ngx.log](#ngxlog) method. print ----- + **syntax:** *print(...)* **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Writes argument values into the nginx `error.log` file with the `ngx.NOTICE` log level. +Writes argument values into the Nginx `error.log` file with the `ngx.NOTICE` log level. It is equivalent to @@ -3544,6 +3615,7 @@ There is a hard coded `2048` byte limitation on error message lengths in the Ngi ngx.ctx ------- + **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua** This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). @@ -3688,13 +3760,14 @@ That is, let the caller pass the `ctx` table explicitly via a function argument. ngx.location.capture -------------------- + **syntax:** *res = ngx.location.capture(uri, options?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** Issues a synchronous but still non-blocking *Nginx Subrequest* using `uri`. -Nginx's subrequests provide a powerful way to make non-blocking internal requests to other locations configured with disk file directory or *any* other nginx C modules like `ngx_proxy`, `ngx_fastcgi`, `ngx_memc`, +Nginx's subrequests provide a powerful way to make non-blocking internal requests to other locations configured with disk file directory or *any* other Nginx C modules like `ngx_proxy`, `ngx_fastcgi`, `ngx_memc`, `ngx_postgres`, `ngx_drizzle`, and even ngx_lua itself and etc etc etc. Also note that subrequests just mimic the HTTP interface but there is *no* extra HTTP/TCP traffic *nor* IPC involved. Everything works internally, efficiently, on the C level. @@ -3743,7 +3816,7 @@ URI query strings can be concatenated to URI itself, for instance, ``` Named locations like `@foo` are not allowed due to a limitation in -the nginx core. Use normal locations combined with the `internal` directive to +the Nginx core. Use normal locations combined with the `internal` directive to prepare internal-only locations. An optional option table can be fed as the second @@ -3760,7 +3833,7 @@ argument, which supports the options: * `vars` take a Lua table which holds the values to set the specified Nginx variables in the subrequest as this option's value. This option was first introduced in the `v0.3.1rc31` release. * `copy_all_vars` - specify whether to copy over all the Nginx variable values of the current request to the subrequest in question. modifications of the nginx variables in the subrequest will not affect the current (parent) request. This option was first introduced in the `v0.3.1rc31` release. + specify whether to copy over all the Nginx variable values of the current request to the subrequest in question. modifications of the Nginx variables in the subrequest will not affect the current (parent) request. This option was first introduced in the `v0.3.1rc31` release. * `share_all_vars` specify whether to share all the Nginx variables of the subrequest with the current (parent) request. modifications of the Nginx variables in the subrequest will affect the current (parent) request. Enabling this option may lead to hard-to-debug issues due to bad side-effects and is considered bad and harmful. Only enable this option when you completely know what you are doing. * `always_forward_body` @@ -3809,7 +3882,7 @@ The `args` option can also take plain query strings: This is functionally identical to the previous examples. -The `share_all_vars` option controls whether to share nginx variables among the current request and its subrequests. +The `share_all_vars` option controls whether to share Nginx variables among the current request and its subrequests. If this option is set to `true`, then the current request and associated subrequests will share the same Nginx variable scope. Hence, changes to Nginx variables made by a subrequest will affect the current request. Care should be taken in using this option as variable scope sharing can have unexpected side effects. The `args`, `vars`, or `copy_all_vars` options are generally preferable instead. @@ -3980,6 +4053,7 @@ Please also refer to restrictions on capturing locations configured by [subreque ngx.location.capture_multi -------------------------- + **syntax:** *res1, res2, ... = ngx.location.capture_multi({ {uri, options?}, {uri, options?}, ... })* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -4045,6 +4119,7 @@ Please also refer to restrictions on capturing locations configured by [subreque ngx.status ---------- + **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** Read and write the current request's response status. This should be called @@ -4056,7 +4131,7 @@ before sending out the response headers. status = ngx.status ``` -Setting `ngx.status` after the response header is sent out has no effect but leaving an error message in your nginx's error log file: +Setting `ngx.status` after the response header is sent out has no effect but leaving an error message in your Nginx's error log file: attempt to set ngx.status after sending out response headers @@ -4066,6 +4141,7 @@ Setting `ngx.status` after the response header is sent out has no effect but lea ngx.header.HEADER ----------------- + **syntax:** *ngx.header.HEADER = VALUE* **syntax:** *value = ngx.header.HEADER* @@ -4185,6 +4261,7 @@ For reading *request* headers, use the [ngx.req.get_headers](#ngxreqget_headers) ngx.resp.get_headers -------------------- + **syntax:** *headers, err = ngx.resp.get_headers(max_headers?, raw?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** @@ -4214,12 +4291,13 @@ This API was first introduced in the `v0.9.5` release. ngx.req.is_internal ------------------- + **syntax:** *is_internal = ngx.req.is_internal()* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** Returns a boolean indicating whether the current request is an "internal request", i.e., -a request initiated from inside the current nginx server instead of from the client side. +a request initiated from inside the current Nginx server instead of from the client side. Subrequests are all internal requests and so are requests after internal redirects. @@ -4229,6 +4307,7 @@ This API was first introduced in the `v0.9.20` release. ngx.req.start_time ------------------ + **syntax:** *secs = ngx.req.start_time()* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** @@ -4250,6 +4329,7 @@ See also [ngx.now](#ngxnow) and [ngx.update_time](#ngxupdate_time). ngx.req.http_version -------------------- + **syntax:** *num = ngx.req.http_version()* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua** @@ -4264,6 +4344,7 @@ This method was first introduced in the `v0.7.17` release. ngx.req.raw_header ------------------ + **syntax:** *str = ngx.req.raw_header(no_request_line?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua** @@ -4312,6 +4393,7 @@ This method does not work in HTTP/2 requests yet. ngx.req.get_method ------------------ + **syntax:** *method_name = ngx.req.get_method()* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, balancer_by_lua*, log_by_lua** @@ -4328,6 +4410,7 @@ See also [ngx.req.set_method](#ngxreqset_method). ngx.req.set_method ------------------ + **syntax:** *ngx.req.set_method(method_id)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua** @@ -4344,6 +4427,7 @@ See also [ngx.req.get_method](#ngxreqget_method). ngx.req.set_uri --------------- + **syntax:** *ngx.req.set_uri(uri, jump?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua** @@ -4354,7 +4438,7 @@ The optional boolean `jump` argument can trigger location rematch (or location j Location jump will not be triggered otherwise, and only the current request's URI will be modified, which is also the default behavior. This function will return but with no returned values when the `jump` argument is `false` or absent altogether. -For example, the following nginx config snippet +For example, the following Nginx config snippet ```nginx @@ -4443,6 +4527,7 @@ This interface was first introduced in the `v0.3.1rc14` release. ngx.req.set_uri_args -------------------- + **syntax:** *ngx.req.set_uri_args(args)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua** @@ -4480,6 +4565,7 @@ See also [ngx.req.set_uri](#ngxreqset_uri). ngx.req.get_uri_args -------------------- + **syntax:** *args, err = ngx.req.get_uri_args(max_args?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** @@ -4542,7 +4628,7 @@ That is, they will take Lua boolean values `true`. However, they are different f Empty key arguments are discarded. `GET /test?=hello&=world` will yield an empty output for instance. -Updating query arguments via the nginx variable `$args` (or `ngx.var.args` in Lua) at runtime is also supported: +Updating query arguments via the Nginx variable `$args` (or `ngx.var.args` in Lua) at runtime is also supported: ```lua @@ -4584,6 +4670,7 @@ Removing the `max_args` cap is strongly discouraged. ngx.req.get_post_args --------------------- + **syntax:** *args, err = ngx.req.get_post_args(max_args?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** @@ -4694,6 +4781,7 @@ Removing the `max_args` cap is strongly discouraged. ngx.req.get_headers ------------------- + **syntax:** *headers, err = ngx.req.get_headers(max_headers?, raw?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** @@ -4777,6 +4865,7 @@ The `__index` metamethod will not be added when the `raw` argument is set to `tr ngx.req.set_header ------------------ + **syntax:** *ngx.req.set_header(header_name, header_value)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua** @@ -4828,6 +4917,7 @@ is equivalent to ngx.req.clear_header -------------------- + **syntax:** *ngx.req.clear_header(header_name)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua** @@ -4838,6 +4928,7 @@ Clears the current request's request header named `header_name`. None of the cur ngx.req.read_body ----------------- + **syntax:** *ngx.req.read_body()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -4869,6 +4960,7 @@ This function was first introduced in the `v0.3.1rc17` release. ngx.req.discard_body -------------------- + **syntax:** *ngx.req.discard_body()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -4887,6 +4979,7 @@ See also [ngx.req.read_body](#ngxreqread_body). ngx.req.get_body_data --------------------- + **syntax:** *data = ngx.req.get_body_data()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, log_by_lua** @@ -4915,6 +5008,7 @@ See also [ngx.req.get_body_file](#ngxreqget_body_file). ngx.req.get_body_file --------------------- + **syntax:** *file_name = ngx.req.get_body_file()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -4937,6 +5031,7 @@ See also [ngx.req.get_body_data](#ngxreqget_body_data). ngx.req.set_body_data --------------------- + **syntax:** *ngx.req.set_body_data(data)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -4955,6 +5050,7 @@ See also [ngx.req.set_body_file](#ngxreqset_body_file). ngx.req.set_body_file --------------------- + **syntax:** *ngx.req.set_body_file(file_name, auto_clean?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -4977,6 +5073,7 @@ See also [ngx.req.set_body_data](#ngxreqset_body_data). ngx.req.init_body ----------------- + **syntax:** *ngx.req.init_body(buffer_size?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5008,6 +5105,7 @@ This function was first introduced in the `v0.5.11` release. ngx.req.append_body ------------------- + **syntax:** *ngx.req.append_body(data_chunk)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5028,6 +5126,7 @@ See also [ngx.req.init_body](#ngxreqinit_body). ngx.req.finish_body ------------------- + **syntax:** *ngx.req.finish_body()* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5044,6 +5143,7 @@ See also [ngx.req.init_body](#ngxreqinit_body). ngx.req.socket -------------- + **syntax:** *tcpsock, err = ngx.req.socket()* **syntax:** *tcpsock, err = ngx.req.socket(raw)* @@ -5063,7 +5163,7 @@ Since the `v0.9.0` release, this function accepts an optional boolean `raw` argu When the `raw` argument is `true`, it is required that no pending data from any previous [ngx.say](#ngxsay), [ngx.print](#ngxprint), or [ngx.send_headers](#ngxsend_headers) calls exists. So if you have these downstream output calls previously, you should call [ngx.flush(true)](#ngxflush) before calling `ngx.req.socket(true)` to ensure that there is no pending output data. If the request body has not been read yet, then this "raw socket" can also be used to read the request body. -You can use the "raw request socket" returned by `ngx.req.socket(true)` to implement fancy protocols like [WebSocket](http://en.wikipedia.org/wiki/WebSocket), or just emit your own raw HTTP response header or body data. You can refer to the [lua-resty-websocket library](https://github.com/openresty/lua-resty-websocket) for a real world example. +You can use the "raw request socket" returned by `ngx.req.socket(true)` to implement fancy protocols like [WebSocket](https://en.wikipedia.org/wiki/WebSocket), or just emit your own raw HTTP response header or body data. You can refer to the [lua-resty-websocket library](https://github.com/openresty/lua-resty-websocket) for a real world example. This function was first introduced in the `v0.5.0rc1` release. @@ -5071,6 +5171,7 @@ This function was first introduced in the `v0.5.0rc1` release. ngx.exec -------- + **syntax:** *ngx.exec(uri, args?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5138,6 +5239,7 @@ It is recommended that a coding style that combines this method call with the `r ngx.redirect ------------ + **syntax:** *ngx.redirect(uri, status?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5234,6 +5336,7 @@ It is recommended that a coding style that combines this method call with the `r ngx.send_headers ---------------- + **syntax:** *ok, err = ngx.send_headers()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5249,6 +5352,7 @@ before content is output with [ngx.say](#ngxsay) or [ngx.print](#ngxprint) or wh ngx.headers_sent ---------------- + **syntax:** *value = ngx.headers_sent* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5261,6 +5365,7 @@ This API was first introduced in ngx_lua v0.3.1rc6. ngx.print --------- + **syntax:** *ok, err = ngx.print(...)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5302,6 +5407,7 @@ Please note that both `ngx.print` and [ngx.say](#ngxsay) will always invoke the ngx.say ------- + **syntax:** *ok, err = ngx.say(...)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5312,6 +5418,7 @@ Just as [ngx.print](#ngxprint) but also emit a trailing newline. ngx.log ------- + **syntax:** *ngx.log(log_level, ...)* **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5328,6 +5435,7 @@ There is a hard coded `2048` byte limitation on error message lengths in the Ngi ngx.flush --------- + **syntax:** *ok, err = ngx.flush(wait?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5348,11 +5456,12 @@ Since `v0.8.3` this function returns `1` on success, or returns `nil` and a stri ngx.exit -------- + **syntax:** *ngx.exit(status)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -When `status >= 200` (i.e., `ngx.HTTP_OK` and above), it will interrupt the execution of the current request and return status code to nginx. +When `status >= 200` (i.e., `ngx.HTTP_OK` and above), it will interrupt the execution of the current request and return status code to Nginx. When `status == 0` (i.e., `ngx.OK`), it will only quit the current phase handler (or the content handler if the [content_by_lua*](#content_by_lua) directive is used) and continue to run later phases (if any) for the current request. @@ -5403,6 +5512,7 @@ an asynchronous operation and will return immediately. This behavior may change ngx.eof ------- + **syntax:** *ok, err = ngx.eof()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -5438,6 +5548,7 @@ Since `v0.8.3` this function returns `1` on success, or returns `nil` and a stri ngx.sleep --------- + **syntax:** *ngx.sleep(seconds)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -5454,6 +5565,7 @@ This method was introduced in the `0.5.0rc30` release. ngx.escape_uri -------------- + **syntax:** *newstr = ngx.escape_uri(str)* **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5464,6 +5576,7 @@ Escape `str` as a URI component. ngx.unescape_uri ---------------- + **syntax:** *newstr = ngx.unescape_uri(str)* **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua** @@ -5487,6 +5600,7 @@ gives the output ngx.encode_args --------------- + **syntax:** *str = ngx.encode_args(table)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua** @@ -5544,6 +5658,7 @@ This method was first introduced in the `v0.3.1rc27` release. ngx.decode_args --------------- + **syntax:** *table, err = ngx.decode_args(str, max_args?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5567,6 +5682,7 @@ This method was introduced in the `v0.5.0rc29`. ngx.encode_base64 ----------------- + **syntax:** *newstr = ngx.encode_base64(str, no_padding?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5579,6 +5695,7 @@ Since the `0.9.16` release, an optional boolean-typed `no_padding` argument can ngx.decode_base64 ----------------- + **syntax:** *newstr = ngx.decode_base64(str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5589,6 +5706,7 @@ Decodes the `str` argument as a base64 digest to the raw form. Returns `nil` if ngx.crc32_short --------------- + **syntax:** *intval = ngx.crc32_short(str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5605,6 +5723,7 @@ This API was first introduced in the `v0.3.1rc8` release. ngx.crc32_long -------------- + **syntax:** *intval = ngx.crc32_long(str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5621,11 +5740,12 @@ This API was first introduced in the `v0.3.1rc8` release. ngx.hmac_sha1 ------------- + **syntax:** *digest = ngx.hmac_sha1(secret_key, str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Computes the [HMAC-SHA1](http://en.wikipedia.org/wiki/HMAC) digest of the argument `str` and turns the result using the secret key ``. +Computes the [HMAC-SHA1](https://en.wikipedia.org/wiki/HMAC) digest of the argument `str` and turns the result using the secret key ``. The raw binary form of the `HMAC-SHA1` digest will be generated, use [ngx.encode_base64](#ngxencode_base64), for example, to encode the result to a textual representation if desired. @@ -5653,6 +5773,7 @@ This function was first introduced in the `v0.3.1rc29` release. ngx.md5 ------- + **syntax:** *digest = ngx.md5(str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5680,6 +5801,7 @@ See [ngx.md5_bin](#ngxmd5_bin) if the raw binary MD5 digest is required. ngx.md5_bin ----------- + **syntax:** *digest = ngx.md5_bin(str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5692,6 +5814,7 @@ See [ngx.md5](#ngxmd5) if the hexadecimal form of the MD5 digest is required. ngx.sha1_bin ------------ + **syntax:** *digest = ngx.sha1_bin(str)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5706,6 +5829,7 @@ This function was first introduced in the `v0.5.0rc6`. ngx.quote_sql_str ----------------- + **syntax:** *quoted_value = ngx.quote_sql_str(raw_value)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5716,11 +5840,12 @@ Returns a quoted SQL string literal according to the MySQL quoting rules. ngx.today --------- + **syntax:** *str = ngx.today()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Returns current date (in the format `yyyy-mm-dd`) from the nginx cached time (no syscall involved unlike Lua's date library). +Returns current date (in the format `yyyy-mm-dd`) from the Nginx cached time (no syscall involved unlike Lua's date library). This is the local time. @@ -5728,11 +5853,12 @@ This is the local time. ngx.time -------- + **syntax:** *secs = ngx.time()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Returns the elapsed seconds from the epoch for the current time stamp from the nginx cached time (no syscall involved unlike Lua's date library). +Returns the elapsed seconds from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). Updates of the Nginx time cache can be forced by calling [ngx.update_time](#ngxupdate_time) first. @@ -5740,11 +5866,12 @@ Updates of the Nginx time cache can be forced by calling [ngx.update_time](#ngxu ngx.now ------- + **syntax:** *secs = ngx.now()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the nginx cached time (no syscall involved unlike Lua's date library). +Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). You can forcibly update the Nginx time cache by calling [ngx.update_time](#ngxupdate_time) first. @@ -5754,6 +5881,7 @@ This API was first introduced in `v0.3.1rc32`. ngx.update_time --------------- + **syntax:** *ngx.update_time()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5766,11 +5894,12 @@ This API was first introduced in `v0.3.1rc32`. ngx.localtime ------------- + **syntax:** *str = ngx.localtime()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the nginx cached time (no syscall involved unlike Lua's [os.date](http://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). +Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the Nginx cached time (no syscall involved unlike Lua's [os.date](https://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). This is the local time. @@ -5778,11 +5907,12 @@ This is the local time. ngx.utctime ----------- + **syntax:** *str = ngx.utctime()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the nginx cached time (no syscall involved unlike Lua's [os.date](http://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). +Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the Nginx cached time (no syscall involved unlike Lua's [os.date](https://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). This is the UTC time. @@ -5790,6 +5920,7 @@ This is the UTC time. ngx.cookie_time --------------- + **syntax:** *str = ngx.cookie_time(sec)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5806,6 +5937,7 @@ Returns a formatted string can be used as the cookie expiration time. The parame ngx.http_time ------------- + **syntax:** *str = ngx.http_time(sec)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5822,6 +5954,7 @@ Returns a formated string can be used as the http header time (for example, bein ngx.parse_http_time ------------------- + **syntax:** *sec = ngx.parse_http_time(str)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5840,16 +5973,18 @@ Parse the http time string (as returned by [ngx.http_time](#ngxhttp_time)) into ngx.is_subrequest ----------------- + **syntax:** *value = ngx.is_subrequest* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** -Returns `true` if the current request is an nginx subrequest, or `false` otherwise. +Returns `true` if the current request is an Nginx subrequest, or `false` otherwise. [Back to TOC](#nginx-api-for-lua) ngx.re.match ------------ + **syntax:** *captures, err = ngx.re.match(subject, regex, options?, ctx?, res_table?)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -5992,7 +6127,7 @@ The `ctx` table argument combined with the `a` regex modifier can be used to con Note that, the `options` argument is not optional when the `ctx` argument is specified and that the empty Lua string (`""`) must be used as placeholder for `options` if no meaningful regex options are required. -This method requires the PCRE library enabled in Nginx. ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). +This method requires the PCRE library enabled in Nginx ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). To confirm that PCRE JIT is enabled, activate the Nginx debug log by adding the `--with-debug` option to Nginx or OpenResty's `./configure` script. Then, enable the "debug" error log level in `error_log` directive. The following message will be generated if PCRE JIT is enabled: @@ -6008,11 +6143,12 @@ This feature was introduced in the `v0.2.1rc11` release. ngx.re.find ----------- + **syntax:** *from, to, err = ngx.re.find(subject, regex, options?, ctx?, nth?)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Similar to [ngx.re.match](#ngxrematch) but only returns the beginning index (`from`) and end index (`to`) of the matched substring. The returned indexes are 1-based and can be fed directly into the [string.sub](http://www.lua.org/manual/5.1/manual.html#pdf-string.sub) API function to obtain the matched substring. +Similar to [ngx.re.match](#ngxrematch) but only returns the beginning index (`from`) and end index (`to`) of the matched substring. The returned indexes are 1-based and can be fed directly into the [string.sub](https://www.lua.org/manual/5.1/manual.html#pdf-string.sub) API function to obtain the matched substring. In case of errors (like bad regexes or any PCRE runtime errors), this API function returns two `nil` values followed by a string describing the error. @@ -6062,6 +6198,7 @@ This API function was first introduced in the `v0.9.2` release. ngx.re.gmatch ------------- + **syntax:** *iterator, err = ngx.re.gmatch(subject, regex, options?)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6132,7 +6269,7 @@ The optional `options` argument takes exactly the same semantics as the [ngx.re. The current implementation requires that the iterator returned should only be used in a single request. That is, one should *not* assign it to a variable belonging to persistent namespace like a Lua package. -This method requires the PCRE library enabled in Nginx. ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). +This method requires the PCRE library enabled in Nginx ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). This feature was first introduced in the `v0.2.1rc12` release. @@ -6140,6 +6277,7 @@ This feature was first introduced in the `v0.2.1rc12` release. ngx.re.sub ---------- + **syntax:** *newstr, n, err = ngx.re.sub(subject, regex, replace, options?)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6198,7 +6336,7 @@ When the `replace` argument is of type "function", then it will be invoked with The dollar sign characters in the return value of the `replace` function argument are not special at all. -This method requires the PCRE library enabled in Nginx. ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). +This method requires the PCRE library enabled in Nginx ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). This feature was first introduced in the `v0.2.1rc13` release. @@ -6206,6 +6344,7 @@ This feature was first introduced in the `v0.2.1rc13` release. ngx.re.gsub ----------- + **syntax:** *newstr, n, err = ngx.re.gsub(subject, regex, replace, options?)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6236,7 +6375,7 @@ Here is some examples: -- n == 2 ``` -This method requires the PCRE library enabled in Nginx. ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). +This method requires the PCRE library enabled in Nginx ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). This feature was first introduced in the `v0.2.1rc15` release. @@ -6244,6 +6383,7 @@ This feature was first introduced in the `v0.2.1rc15` release. ngx.shared.DICT --------------- + **syntax:** *dict = ngx.shared.DICT* **syntax:** *dict = ngx.shared\[name_var\]* @@ -6252,7 +6392,7 @@ ngx.shared.DICT Fetching the shm-based Lua dictionary object for the shared memory zone named `DICT` defined by the [lua_shared_dict](#lua_shared_dict) directive. -Shared memory zones are always shared by all the nginx worker processes in the current nginx server instance. +Shared memory zones are always shared by all the Nginx worker processes in the current Nginx server instance. The resulting object `dict` has the following methods: @@ -6278,7 +6418,7 @@ The resulting object `dict` has the following methods: * [capacity](#ngxshareddictcapacity) * [free_space](#ngxshareddictfree_space) -All these methods are *atomic* operations, that is, safe from concurrent accesses from multiple nginx worker processes for the same `lua_shared_dict` zone. +All these methods are *atomic* operations, that is, safe from concurrent accesses from multiple Nginx worker processes for the same `lua_shared_dict` zone. Here is an example: @@ -6330,6 +6470,7 @@ This feature was first introduced in the `v0.3.1rc22` release. ngx.shared.DICT.get ------------------- + **syntax:** *value, flags = ngx.shared.DICT:get(key)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6368,6 +6509,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.get_stale ------------------------- + **syntax:** *value, flags, stale = ngx.shared.DICT:get_stale(key)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6386,6 +6528,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.set ------------------- + **syntax:** *success, err, forcible = ngx.shared.DICT:set(key, value, exptime?, flags?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6434,6 +6577,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.safe_set ------------------------ + **syntax:** *ok, err = ngx.shared.DICT:safe_set(key, value, exptime?, flags?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6448,6 +6592,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.add ------------------- + **syntax:** *success, err, forcible = ngx.shared.DICT:add(key, value, exptime?, flags?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6464,6 +6609,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.safe_add ------------------------ + **syntax:** *ok, err = ngx.shared.DICT:safe_add(key, value, exptime?, flags?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6478,6 +6624,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.replace ----------------------- + **syntax:** *success, err, forcible = ngx.shared.DICT:replace(key, value, exptime?, flags?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6494,6 +6641,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.delete ---------------------- + **syntax:** *ngx.shared.DICT:delete(key)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6510,6 +6658,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.incr -------------------- + **syntax:** *newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?, init_ttl?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6564,6 +6713,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.lpush --------------------- + **syntax:** *length, err = ngx.shared.DICT:lpush(key, value)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6582,6 +6732,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.rpush --------------------- + **syntax:** *length, err = ngx.shared.DICT:rpush(key, value)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6596,6 +6747,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.lpop -------------------- + **syntax:** *val, err = ngx.shared.DICT:lpop(key)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6612,6 +6764,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.rpop -------------------- + **syntax:** *val, err = ngx.shared.DICT:rpop(key)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6628,6 +6781,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.llen -------------------- + **syntax:** *len, err = ngx.shared.DICT:llen(key)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6644,6 +6798,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.ttl ------------------- + **syntax:** *ttl, err = ngx.shared.DICT:ttl(key)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6681,6 +6836,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.expire ---------------------- + **syntax:** *success, err = ngx.shared.DICT:expire(key, exptime)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6720,6 +6876,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.flush_all ------------------------- + **syntax:** *ngx.shared.DICT:flush_all()* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6734,6 +6891,7 @@ See also [ngx.shared.DICT.flush_expired](#ngxshareddictflush_expired) and [ngx.s ngx.shared.DICT.flush_expired ----------------------------- + **syntax:** *flushed = ngx.shared.DICT:flush_expired(max_count?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6750,6 +6908,7 @@ See also [ngx.shared.DICT.flush_all](#ngxshareddictflush_all) and [ngx.shared.DI ngx.shared.DICT.get_keys ------------------------ + **syntax:** *keys = ngx.shared.DICT:get_keys(max_count?)* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6766,6 +6925,7 @@ This feature was first introduced in the `v0.7.3` release. ngx.shared.DICT.capacity ------------------------ + **syntax:** *capacity_bytes = ngx.shared.DICT:capacity()* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6789,7 +6949,7 @@ This feature was first introduced in the `v0.10.11` release. **Note:** This method requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. -This feature requires at least nginx core version `0.7.3`. +This feature requires at least Nginx core version `0.7.3`. See also [ngx.shared.DICT](#ngxshareddict). @@ -6797,6 +6957,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.shared.DICT.free_space -------------------------- + **syntax:** *free_page_bytes = ngx.shared.DICT:free_space()* **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -6805,7 +6966,7 @@ ngx.shared.DICT.free_space Retrieves the free page size in bytes for the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). -**Note:** The memory for ngx.shared.DICT is allocated via the nginx slab allocator which has each slot for +**Note:** The memory for ngx.shared.DICT is allocated via the Nginx slab allocator which has each slot for data size ranges like \~8, 9\~16, 17\~32, ..., 1025\~2048, 2048\~ bytes. And pages are assigned to a slot if there is no room in already assigned pages for the slot. @@ -6831,7 +6992,7 @@ This feature was first introduced in the `v0.10.11` release. **Note:** This method requires the `resty.core.shdict` or `resty.core` modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. -This feature requires at least nginx core version `1.11.7`. +This feature requires at least Nginx core version `1.11.7`. See also [ngx.shared.DICT](#ngxshareddict). @@ -6839,6 +7000,7 @@ See also [ngx.shared.DICT](#ngxshareddict). ngx.socket.udp -------------- + **syntax:** *udpsock = ngx.socket.udp()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -6861,6 +7023,7 @@ See also [ngx.socket.tcp](#ngxsockettcp). udpsock:setpeername ------------------- + **syntax:** *ok, err = udpsock:setpeername(host, port)* **syntax:** *ok, err = udpsock:setpeername("unix:/path/to/unix-domain.socket")* @@ -6922,6 +7085,7 @@ This method was first introduced in the `v0.5.7` release. udpsock:send ------------ + **syntax:** *ok, err = udpsock:send(data)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -6938,6 +7102,7 @@ This feature was first introduced in the `v0.5.7` release. udpsock:receive --------------- + **syntax:** *data, err = udpsock:receive(size?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -6973,6 +7138,7 @@ This feature was first introduced in the `v0.5.7` release. udpsock:close ------------- + **syntax:** *ok, err = udpsock:close()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -6987,6 +7153,7 @@ This feature was first introduced in the `v0.5.7` release. udpsock:settimeout ------------------ + **syntax:** *udpsock:settimeout(time)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7011,6 +7178,7 @@ This API function was first added to the `v0.10.1` release. ngx.socket.tcp -------------- + **syntax:** *tcpsock = ngx.socket.tcp()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7032,7 +7200,7 @@ Creates and returns a TCP or stream-oriented unix domain socket object (also kno It is intended to be compatible with the TCP API of the [LuaSocket](http://w3.impa.br/~diego/software/luasocket/tcp.html) library but is 100% nonblocking out of the box. Also, we introduce some new APIs to provide more functionalities. -The cosocket object created by this API function has exactly the same lifetime as the Lua handler creating it. So never pass the cosocket object to any other Lua handler (including ngx.timer callback functions) and never share the cosocket object between different NGINX requests. +The cosocket object created by this API function has exactly the same lifetime as the Lua handler creating it. So never pass the cosocket object to any other Lua handler (including ngx.timer callback functions) and never share the cosocket object between different Nginx requests. For every cosocket object's underlying connection, if you do not explicitly close it (via [close](#tcpsockclose)) or put it back to the connection @@ -7057,6 +7225,7 @@ See also [ngx.socket.udp](#ngxsocketudp). tcpsock:connect --------------- + **syntax:** *ok, err = tcpsock:connect(host, port, options_table?)* **syntax:** *ok, err = tcpsock:connect("unix:/path/to/unix-domain.socket", options_table?)* @@ -7175,6 +7344,7 @@ This method was first introduced in the `v0.5.0rc1` release. tcpsock:sslhandshake -------------------- + **syntax:** *session, err = tcpsock:sslhandshake(reused_session?, server_name?, ssl_verify?, send_status_req?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7220,6 +7390,7 @@ This method was first introduced in the `v0.9.11` release. tcpsock:send ------------ + **syntax:** *bytes, err = tcpsock:send(data)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7250,6 +7421,7 @@ This feature was first introduced in the `v0.5.0rc1` release. tcpsock:receive --------------- + **syntax:** *data, err, partial = tcpsock:receive(size)* **syntax:** *data, err, partial = tcpsock:receive(pattern?)* @@ -7294,6 +7466,7 @@ This feature was first introduced in the `v0.5.0rc1` release. tcpsock:receiveany ------------------ + **syntax:** *data, err = tcpsock:receiveany(max)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7328,6 +7501,7 @@ This feature was first introduced in the `v0.10.14` release. tcpsock:receiveuntil -------------------- + **syntax:** *iterator = tcpsock:receiveuntil(pattern, options?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7427,6 +7601,7 @@ This method was first introduced in the `v0.5.0rc1` release. tcpsock:close ------------- + **syntax:** *ok, err = tcpsock:close()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7443,6 +7618,7 @@ This feature was first introduced in the `v0.5.0rc1` release. tcpsock:settimeout ------------------ + **syntax:** *tcpsock:settimeout(time)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7459,6 +7635,7 @@ This feature was first introduced in the `v0.5.0rc1` release. tcpsock:settimeouts ------------------- + **syntax:** *tcpsock:settimeouts(connect_timeout, send_timeout, read_timeout)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7478,6 +7655,7 @@ This feature was first introduced in the `v0.10.7` release. tcpsock:setoption ----------------- + **syntax:** *tcpsock:setoption(option, value?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7490,6 +7668,7 @@ This feature was first introduced in the `v0.5.0rc1` release. tcpsock:setkeepalive -------------------- + **syntax:** *ok, err = tcpsock:setkeepalive(timeout?, size?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7536,6 +7715,7 @@ This feature was first introduced in the `v0.5.0rc1` release. tcpsock:getreusedtimes ---------------------- + **syntax:** *count, err = tcpsock:getreusedtimes()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7550,6 +7730,7 @@ This feature was first introduced in the `v0.5.0rc1` release. ngx.socket.connect ------------------ + **syntax:** *tcpsock, err = ngx.socket.connect(host, port)* **syntax:** *tcpsock, err = ngx.socket.connect("unix:/path/to/unix-domain.socket")* @@ -7576,6 +7757,7 @@ This feature was first introduced in the `v0.5.0rc1` release. ngx.get_phase ------------- + **syntax:** *str = ngx.get_phase()* **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -7617,6 +7799,7 @@ This API was first introduced in the `v0.5.10` release. ngx.thread.spawn ---------------- + **syntax:** *co = ngx.thread.spawn(func, arg1, arg2, ...)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7755,6 +7938,7 @@ This API was first enabled in the `v0.7.0` release. ngx.thread.wait --------------- + **syntax:** *ok, res1, res2, ... = ngx.thread.wait(thread1, thread2, ...)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** @@ -7858,13 +8042,14 @@ This API was first enabled in the `v0.7.0` release. ngx.thread.kill --------------- + **syntax:** *ok, err = ngx.thread.kill(thread)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.** Kills a running "light thread" created by [ngx.thread.spawn](#ngxthreadspawn). Returns a true value when successful or `nil` and a string describing the error otherwise. -According to the current implementation, only the parent coroutine (or "light thread") can kill a thread. Also, a running "light thread" with pending NGINX subrequests (initiated by [ngx.location.capture](#ngxlocationcapture) for example) cannot be killed due to a limitation in the NGINX core. +According to the current implementation, only the parent coroutine (or "light thread") can kill a thread. Also, a running "light thread" with pending Nginx subrequests (initiated by [ngx.location.capture](#ngxlocationcapture) for example) cannot be killed due to a limitation in the Nginx core. This API was first enabled in the `v0.9.9` release. @@ -7872,6 +8057,7 @@ This API was first enabled in the `v0.9.9` release. ngx.on_abort ------------ + **syntax:** *ok, err = ngx.on_abort(callback)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** @@ -7912,6 +8098,7 @@ See also [lua_check_client_abort](#lua_check_client_abort). ngx.timer.at ------------ + **syntax:** *hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -8041,6 +8228,7 @@ This API was first introduced in the `v0.8.0` release. ngx.timer.every --------------- + **syntax:** *hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -8060,6 +8248,7 @@ This API was first introduced in the `v0.10.9` release. ngx.timer.running_count ----------------------- + **syntax:** *count = ngx.timer.running_count()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -8072,6 +8261,7 @@ This directive was first introduced in the `v0.9.20` release. ngx.timer.pending_count ----------------------- + **syntax:** *count = ngx.timer.pending_count()* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** @@ -8084,11 +8274,12 @@ This directive was first introduced in the `v0.9.20` release. ngx.config.subsystem -------------------- + **syntax:** *subsystem = ngx.config.subsystem* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** -This string field indicates the current NGINX subsystem the current Lua environment is based on. For this module, this field always takes the string value `"http"`. For +This string field indicates the current Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value `"http"`. For [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme), however, this field takes the value `"stream"`. This field was first introduced in the `0.10.1`. @@ -8097,6 +8288,7 @@ This field was first introduced in the `0.10.1`. ngx.config.debug ---------------- + **syntax:** *debug = ngx.config.debug* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** @@ -8114,7 +8306,7 @@ ngx.config.prefix **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** -Returns the Nginx server "prefix" path, as determined by the `-p` command-line option when running the nginx executable, or the path specified by the `--prefix` command-line option when building Nginx with the `./configure` script. +Returns the Nginx server "prefix" path, as determined by the `-p` command-line option when running the Nginx executable, or the path specified by the `--prefix` command-line option when building Nginx with the `./configure` script. This function was first introduced in the `0.9.2`. @@ -8140,7 +8332,7 @@ ngx.config.nginx_configure **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua** -This function returns a string for the NGINX `./configure` command's arguments string. +This function returns a string for the Nginx `./configure` command's arguments string. This API was first introduced in the `0.9.5` release. @@ -8193,7 +8385,7 @@ ngx.worker.count **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** Returns the total number of the Nginx worker processes (i.e., the value configured -by the [worker_processes](http://nginx.org/en/docs/ngx_core_module.html#worker_processes) +by the [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes) directive in `nginx.conf`). This API was first introduced in the `0.9.20` release. @@ -8212,7 +8404,7 @@ Returns the ordinal number of the current Nginx worker processes (starting from So if the total number of workers is `N`, then this method may return a number between 0 and `N - 1` (inclusive). -This function returns meaningful values only for NGINX 1.9.1+. With earlier versions of NGINX, it +This function returns meaningful values only for Nginx 1.9.1+. With earlier versions of Nginx, it always returns `nil`. See also [ngx.worker.count](#ngxworkercount). @@ -8223,11 +8415,12 @@ This API was first introduced in the `0.9.20` release. ngx.semaphore ------------- + **syntax:** *local semaphore = require "ngx.semaphore"* This is a Lua module that implements a classic-style semaphore API for efficient synchronizations among different "light threads". Sharing the same semaphore among different "light threads" created in different (request) -contexts are also supported as long as the "light threads" reside in the same NGINX worker process +contexts are also supported as long as the "light threads" reside in the same Nginx worker process and the [lua_code_cache](#lua_code_cache) directive is turned on (which is the default). This Lua module does not ship with this ngx_lua module itself rather it is shipped with @@ -8244,6 +8437,7 @@ This feature requires at least ngx_lua `v0.10.0`. ngx.balancer ------------ + **syntax:** *local balancer = require "ngx.balancer"* This is a Lua module that provides a Lua API to allow defining completely dynamic load balancers @@ -8263,6 +8457,7 @@ This feature requires at least ngx_lua `v0.10.0`. ngx.ssl ------- + **syntax:** *local ssl = require "ngx.ssl"* This Lua module provides API functions to control the SSL handshake process in contexts like @@ -8281,6 +8476,7 @@ This feature requires at least ngx_lua `v0.10.0`. ngx.ocsp -------- + **syntax:** *local ocsp = require "ngx.ocsp"* This Lua module provides API to perform OCSP queries, OCSP response validations, and @@ -8303,11 +8499,12 @@ This feature requires at least ngx_lua `v0.10.0`. ndk.set_var.DIRECTIVE --------------------- + **syntax:** *res = ndk.set_var.DIRECTIVE_NAME* **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -This mechanism allows calling other nginx C modules' directives that are implemented by [Nginx Devel Kit](https://github.com/simplresty/ngx_devel_kit) (NDK)'s set_var submodule's `ndk_set_var_value`. +This mechanism allows calling other Nginx C modules' directives that are implemented by [Nginx Devel Kit](https://github.com/simplresty/ngx_devel_kit) (NDK)'s set_var submodule's `ndk_set_var_value`. For example, the following [set-misc-nginx-module](http://github.com/openresty/set-misc-nginx-module) directives can be invoked this way: @@ -8344,13 +8541,14 @@ This feature requires the [ngx_devel_kit](https://github.com/simplresty/ngx_deve coroutine.create ---------------- + **syntax:** *co = coroutine.create(f)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Creates a user Lua coroutines with a Lua function, and returns a coroutine object. -Similar to the standard Lua [coroutine.create](http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.create) API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [coroutine.create](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.create) API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [init_by_lua*](#init_by_lua) since the `0.9.2`. @@ -8360,13 +8558,14 @@ This API was first introduced in the `v0.6.0` release. coroutine.resume ---------------- + **syntax:** *ok, ... = coroutine.resume(co, ...)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Resumes the executation of a user Lua coroutine object previously yielded or just created. -Similar to the standard Lua [coroutine.resume](http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.resume) API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [coroutine.resume](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.resume) API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [init_by_lua*](#init_by_lua) since the `0.9.2`. @@ -8376,13 +8575,14 @@ This API was first introduced in the `v0.6.0` release. coroutine.yield --------------- + **syntax:** *... = coroutine.yield(...)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Yields the execution of the current user Lua coroutine. -Similar to the standard Lua [coroutine.yield](http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.yield) API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [coroutine.yield](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.yield) API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [init_by_lua*](#init_by_lua) since the `0.9.2`. @@ -8392,11 +8592,12 @@ This API was first introduced in the `v0.6.0` release. coroutine.wrap -------------- + **syntax:** *co = coroutine.wrap(f)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Similar to the standard Lua [coroutine.wrap](http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.wrap) API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [coroutine.wrap](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.wrap) API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [init_by_lua*](#init_by_lua) since the `0.9.2`. @@ -8406,11 +8607,12 @@ This API was first introduced in the `v0.6.0` release. coroutine.running ----------------- + **syntax:** *co = coroutine.running()* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Identical to the standard Lua [coroutine.running](http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running) API. +Identical to the standard Lua [coroutine.running](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running) API. This API was first usable in the context of [init_by_lua*](#init_by_lua) since the `0.9.2`. @@ -8420,11 +8622,12 @@ This API was first enabled in the `v0.6.0` release. coroutine.status ---------------- + **syntax:** *status = coroutine.status(co)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Identical to the standard Lua [coroutine.status](http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.status) API. +Identical to the standard Lua [coroutine.status](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.status) API. This API was first usable in the context of [init_by_lua*](#init_by_lua) since the `0.9.2`. @@ -8443,3 +8646,13 @@ Special PCRE Sequences ---------------------- This section has been renamed to [Special Escaping Sequences](#special-escaping-sequences). + +[Back to TOC](#table-of-contents) + +Lua/LuaJIT bytecode support +--------------------------- + +This section has been renamed to +[LuaJIT bytecode support](#luajit-bytecode-support). As of version +`v0.10.16` of this module, the standard Lua interpreter (also known +as "PUC-Rio Lua") is not supported anymore. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ce6528e434..1993b05a5d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2,9 +2,11 @@ ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. -This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) +This module is a core component of [https://openresty.org OpenResty]. If you are using this module, +then you are essentially using OpenResty. -''This module is not distributed with the Nginx source.'' See [[#Installation|the installation instructions]]. +''This module is not distributed with the Nginx source.'' See +[[#Installation|the installation instructions]]. = Status = @@ -12,7 +14,9 @@ Production ready. = Version = -This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.16], which is not released yet. +This document describes ngx_lua +[https://github.com/openresty/lua-nginx-module/tags v0.10.16], which is not +released yet. = Synopsis = @@ -132,14 +136,27 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/t = Description = -This module is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) +This module embeds [https://luajit.org/luajit.html LuaJIT 2.0/2.1] into Nginx. +It is a core component of [https://openresty.org OpenResty]. If you are using +this module, then you are essentially using OpenResty. -This module embeds Lua, via [http://luajit.org/luajit.html LuaJIT 2.0/2.1], into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model. +Since version v0.10.16 of this module, the standard Lua +interpreter (also known as "PUC-Rio Lua") is not supported anymore. This +document interchangeably uses the terms "Lua" and "LuaJIT" to refer to the +LuaJIT interpreter. -Unlike [https://httpd.apache.org/docs/trunk/mod/mod_lua.html Apache's mod_lua] and [http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet Lighttpd's mod_magnet], Lua code executed using this module can be ''100% non-blocking'' on network traffic as long as the [[#Nginx API for Lua|Nginx API for Lua]] provided by this module is used to handle -requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services. +By leveraging Nginx's subrequests, this module allows the integration of the +powerful Lua threads (known as Lua "coroutines") into the Nginx event model. -At least the following Lua libraries and Nginx modules can be used with this ngx_lua module: +Unlike [https://httpd.apache.org/docs/trunk/mod/mod_lua.html Apache's mod_lua] +and [http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet Lighttpd's mod_magnet], +Lua code executed using this module can be ''100% non-blocking'' on network +traffic as long as the [[#Nginx API for Lua|Nginx API for Lua]] provided by +this module is used to handle requests to upstream services such as MySQL, +PostgreSQL, Memcached, Redis, or upstream HTTP web services. + +At least the following Lua libraries and Nginx modules can be used with this +module: * [https://github.com/openresty/lua-resty-memcached lua-resty-memcached] * [https://github.com/openresty/lua-resty-mysql lua-resty-mysql] @@ -158,21 +175,32 @@ At least the following Lua libraries and Nginx modules can be used with this ngx * [[HttpProxyModule|ngx_proxy]] * [[HttpFastcgiModule|ngx_fastcgi]] -Almost all the Nginx modules can be used with this ngx_lua module by means of [[#ngx.location.capture|ngx.location.capture]] or [[#ngx.location.capture_multi|ngx.location.capture_multi]] but it is recommended to use those lua-resty-* libraries instead of creating subrequests to access the Nginx upstream modules because the former is usually much more flexible and memory-efficient. +Almost any Nginx modules can be used with this ngx_lua module by means of +[[#ngx.location.capture|ngx.location.capture]] or +[[#ngx.location.capture_multi|ngx.location.capture_multi]] but it is +recommended to use those lua-resty-* libraries instead of creating +subrequests to access the Nginx upstream modules because the former is usually +much more flexible and memory-efficient. -The Lua interpreter or LuaJIT instance is shared across all the requests in a single nginx worker process but request contexts are segregated using lightweight Lua coroutines. +The Lua interpreter (also known as "Lua State" or "LuaJIT VM instance") is +shared across all the requests in a single Nginx worker process to minimize +memory use. Request contexts are segregated using lightweight Lua coroutines. -Loaded Lua modules persist in the nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads. +Loaded Lua modules persist in the Nginx worker process level resulting in a +small memory footprint in Lua even when under heavy loads. -This module is plugged into NGINX's "http" subsystem so it can only speaks downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, and etc). -If you want to do generic TCP communications with the downstream clients, then you should use the [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua] module instead -which has a compatible Lua API. +This module is plugged into Nginx's "http" subsystem so it can only speaks +downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, +WebSockets, etc...). If you want to do generic TCP communications with the +downstream clients, then you should use the +[https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua] +module instead, which offers a compatible Lua API. = Typical Uses = Just to name a few: -* Mashup'ing and processing outputs of various nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua, +* Mashup'ing and processing outputs of various Nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua, * doing arbitrarily complex access control and security checks in Lua before requests actually reach the upstream backends, * manipulating response headers in an arbitrary way (by Lua) * fetching backend information from external storage backends (like redis, memcached, mysql, postgresql) and use that information to choose which upstream backend to access on-the-fly, @@ -180,11 +208,14 @@ Just to name a few: * doing very complex URL dispatch in Lua at rewrite phase, * using Lua to implement advanced caching mechanism for Nginx's subrequests and arbitrary locations. -The possibilities are unlimited as the module allows bringing together various elements within Nginx as well as exposing the power of the Lua language to the user. The module provides the full flexibility of scripting while offering performance levels comparable with native C language programs both in terms of CPU time as well as memory footprint. This is particularly the case when LuaJIT 2.x is enabled. - -Other scripting language implementations typically struggle to match this performance level. +The possibilities are unlimited as the module allows bringing together various +elements within Nginx as well as exposing the power of the Lua language to the +user. The module provides the full flexibility of scripting while offering +performance levels comparable with native C language programs both in terms of +CPU time as well as memory footprint thanks to LuaJIT 2.x. -The Lua state (Lua VM instance) is shared across all the requests handled by a single nginx worker process to minimize memory use. +Other scripting language implementations typically struggle to match this +performance level. = Nginx Compatibility = @@ -206,19 +237,32 @@ Nginx cores older than 1.6.0 (exclusive) are *not* supported. = Installation = -It is *highly* recommended to use [http://openresty.org OpenResty releases] which integrate Nginx, ngx_lua, OpenResty's LuaJIT 2.1 branch version, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower. The same applies to LuaJIT as well. OpenResty includes its own version of LuaJIT which gets specifically optimized and enhanced for the OpenResty environment. +It is *highly* recommended to use [https://openresty.org OpenResty releases] +which bundle Nginx, ngx_lua (this module), LuaJIT, as well as other powerful +companion Nginx modules and Lua libraries. + +It is discouraged to build this module with Nginx yourself since it is tricky +to set up exactly right. + +Note that Nginx, LuaJIT, and OpenSSL official releases have various limitations +and long standing bugs that can cause some of this module's features to be +disabled, not work properly, or run slower. Official OpenResty releases are +recommended because they bundle [https://github.com/openresty/luajit2 +OpenResty's optimized LuaJIT 2.1 fork] and +[https://github.com/openresty/openresty/tree/master/patches Nginx/OpenSSL +patches]. Alternatively, ngx_lua can be manually compiled into Nginx: -# LuaJIT can be downloaded from the [https://github.com/openresty/luajit2/releases latest release of OpenResty's LuaJIT branch version]. The official LuaJIT 2.0 and 2.1 releases are also supported, although the performance will be significantly lower in many cases. +# LuaJIT can be downloaded from the [https://github.com/openresty/luajit2/releases latest release of OpenResty's LuaJIT fork]. The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above.. # Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE]. # Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE]. -# Download the latest version of Nginx [http://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) +# Download the latest version of Nginx [https://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) Build the source with this module: - wget 'http://nginx.org/download/nginx-1.13.6.tar.gz' + wget 'https://nginx.org/download/nginx-1.13.6.tar.gz' tar -xzvf nginx-1.13.6.tar.gz cd nginx-1.13.6/ @@ -230,13 +274,9 @@ Build the source with this module: export LUAJIT_LIB=/path/to/luajit/lib export LUAJIT_INC=/path/to/luajit/include/luajit-2.1 - # or tell where to find Lua if using Lua instead: - #export LUA_LIB=/path/to/lua/lib - #export LUA_INC=/path/to/lua/include - # Here we assume Nginx is to be installed under /opt/nginx/. ./configure --prefix=/opt/nginx \ - --with-ld-opt="-Wl,-rpath,/path/to/luajit-or-lua/lib" \ + --with-ld-opt="-Wl,-rpath,/path/to/luajit/lib" \ --add-module=/path/to/ngx_devel_kit \ --add-module=/path/to/lua-nginx-module @@ -253,7 +293,7 @@ Build the source with this module: == Building as a dynamic module == Starting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the --add-dynamic-module=PATH option instead of --add-module=PATH on the -./configure command line above. And then you can explicitly load the module in your nginx.conf via the [load_module](http://nginx.org/en/docs/ngx_core_module.html#load_module) +./configure command line above. And then you can explicitly load the module in your nginx.conf via the [load_module](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive, for example, @@ -263,16 +303,14 @@ load_module /path/to/modules/ngx_http_lua_module.so; == C Macro Configurations == -While building this module either via OpenResty or with the NGINX core, you can define the following C macros via the C compiler options: +While building this module either via OpenResty or with the Nginx core, you can define the following C macros via the C compiler options: * NGX_LUA_USE_ASSERT : When defined, will enable assertions in the ngx_lua C code base. Recommended for debugging or testing builds. It can introduce some (small) runtime overhead when enabled. This macro was first introduced in the v0.9.10 release. * NGX_LUA_ABORT_AT_PANIC -: When the Lua/LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the v0.9.8 release. -* NGX_LUA_NO_FFI_API -: Excludes pure C API functions for FFI-based Lua API for NGINX (as required by [https://github.com/openresty/lua-resty-core#readme lua-resty-core], for example). Enabling this macro can make the resulting binary code size smaller. +: When the LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the v0.9.8 release. -To enable one or more of these macros, just pass extra C compiler options to the ./configure script of either NGINX or OpenResty. For instance, +To enable one or more of these macros, just pass extra C compiler options to the ./configure script of either Nginx or OpenResty. For instance, ./configure --with-cc-opt="-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC" @@ -299,11 +337,9 @@ Please submit bug reports, wishlists, or patches by # creating a ticket on the [https://github.com/openresty/lua-nginx-module/issues GitHub Issue Tracker], # or posting to the [[#Community|OpenResty community]]. -= Lua/LuaJIT bytecode support = - -As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as [[#content_by_lua_file|content_by_lua_file]]) support loading Lua 5.1 and LuaJIT 2.0/2.1 raw bytecode files directly. += LuaJIT bytecode support = -Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if you are using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown: +As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as [[#content_by_lua_file|content_by_lua_file]]) support loading LuaJIT 2.0/2.1 raw bytecode files directly: /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc @@ -317,21 +353,26 @@ The -bg option can be used to include debug information in the LuaJ Please refer to the official LuaJIT documentation on the -b option for more details: -http://luajit.org/running.html#opt_b +https://luajit.org/running.html#opt_b -Also, the bytecode files generated by LuaJIT 2.1 is ''not'' compatible with LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added in ngx_lua v0.9.3. +Note that the bytecode files generated by LuaJIT 2.1 is ''not'' compatible with +LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added +in ngx_lua v0.9.3. -Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked to LuaJIT 2.0/2.1 or vice versa, will result in an error message, such as that below, being logged into the Nginx error.log file: +Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked +to LuaJIT 2.0/2.1 (or vice versa) will result in an Nginx error message such as +the one below: [error] 13909#0: *1 failed to load Lua inlined code: bad byte-code header in /path/to/test_file.luac -Loading bytecode files via the Lua primitives like require and dofile should always work as expected. +Loading bytecode files via the Lua primitives like require and +dofile should always work as expected. = System Environment Variable Support = -If you want to access the system environment variable, say, foo, in Lua via the standard Lua API [http://www.lua.org/manual/5.1/manual.html#pdf-os.getenv os.getenv], then you should also list this environment variable name in your nginx.conf file via the [http://nginx.org/en/docs/ngx_core_module.html#env env directive]. For example, +If you want to access the system environment variable, say, foo, in Lua via the standard Lua API [https://www.lua.org/manual/5.1/manual.html#pdf-os.getenv os.getenv], then you should also list this environment variable name in your nginx.conf file via the [https://nginx.org/en/docs/ngx_core_module.html#env env directive]. For example, env foo; @@ -352,9 +393,12 @@ To force curl to send HTTP 1.0 requests, use the -0 op = Statically Linking Pure Lua Modules = -When LuaJIT 2.x is used, it is possible to statically link the bytecode of pure Lua modules into the Nginx executable. +With LuaJIT 2.x, it is possible to statically link the bytecode of pure Lua +modules into the Nginx executable. -Basically you use the luajit executable to compile .lua Lua module files to .o object files containing the exported bytecode data, and then link the .o files directly in your Nginx build. +You can use the luajit executable to compile .lua Lua +module files to .o object files containing the exported bytecode +data, and then link the .o files directly in your Nginx build. Below is a trivial example to demonstrate this. Consider that we have the following .lua file named foo.lua: @@ -371,7 +415,9 @@ Below is a trivial example to demonstrate this. Consider that we have the follow And then we compile this .lua file to foo.o file: + /path/to/luajit/bin/luajit -bg foo.lua foo.o + What matters here is the name of the .lua file, which determines how you use this module later on the Lua land. The file name foo.o does not matter at all except the .o file extension (which tells luajit what output format is used). If you want to strip the Lua debug information from the resulting bytecode, you can just specify the -b option above instead of -bg. @@ -406,7 +452,7 @@ When you have multiple .lua files to compile and link, then just sp ./configure --with-ld-opt="/path/to/foo.o /path/to/bar.o" ... -If you have just too many .o files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your .o files, as in +If you have too many .o files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your .o files, as in ar rcus libmyluafiles.a *.o @@ -423,9 +469,17 @@ where /path/to/lib is the path of the directory containing the require builtin to import the module, and then manipulate the shared data in Lua. This works because required Lua modules are loaded only once and all coroutines will share the same copy of the module (both its code and data). Note however that Lua global variables (note, not module-level variables) WILL NOT persist between requests because of the one-coroutine-per-request isolation design. +To globally share data among all the requests handled by the same Nginx worker +process, encapsulate the shared data into a Lua module, use the Lua +require builtin to import the module, and then manipulate the +shared data in Lua. This works because required Lua modules are loaded only +once and all coroutines will share the same copy of the module (both its code +and data). + +Note that the use of global Lua variables is *strongly discouraged*, as it may +lead to unexpected race conditions between concurrent requests. -Here is a complete small example: +Here is a small example on sharing data within an Nginx worker via a Lua module: -- mydata.lua @@ -456,16 +510,16 @@ and then accessing it from nginx.conf: The mydata module in this example will only be loaded and run on the first request to the location /lua, -and all subsequent requests to the same nginx worker process will use the reloaded instance of the +and all subsequent requests to the same Nginx worker process will use the reloaded instance of the module as well as the same copy of the data in it, until a HUP signal is sent to the Nginx master process to force a reload. This data sharing technique is essential for high performance Lua applications based on this module. -Note that this data sharing is on a ''per-worker'' basis and not on a ''per-server'' basis. That is, when there are multiple nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers. +Note that this data sharing is on a ''per-worker'' basis and not on a ''per-server'' basis. That is, when there are multiple Nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers. -It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each nginx worker process as +It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each Nginx worker process as long as there is ''no'' nonblocking I/O operations (including [[#ngx.sleep|ngx.sleep]]) in the middle of your calculations. As long as you do not give the -control back to the nginx event loop and ngx_lua's light thread +control back to the Nginx event loop and ngx_lua's light thread scheduler (even implicitly), there can never be any race conditions in between. For this reason, always be very careful when you want to share changeable data on the worker level. Buggy optimizations can easily lead to hard-to-debug @@ -474,12 +528,13 @@ race conditions under load. If server-wide data sharing is required, then use one or more of the following approaches: # Use the [[#ngx.shared.DICT|ngx.shared.DICT]] API provided by this module. -# Use only a single nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine). -# Use data storage mechanisms such as memcached, redis, MySQL or PostgreSQL. [http://openresty.org The OpenResty bundle] associated with this module comes with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms. +# Use only a single Nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine). +# Use data storage mechanisms such as memcached, redis, MySQL or PostgreSQL. [https://openresty.org The OpenResty official releases] come with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms. = Known Issues = == TCP socket connect operation issues == + The [[#tcpsock:connect|tcpsock:connect]] method may indicate success despite connection failures such as with Connection Refused errors. However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation. @@ -487,10 +542,12 @@ However, later attempts to manipulate the cosocket object will fail and return t This issue is due to limitations in the Nginx event model and only appears to affect Mac OS X. == Lua Coroutine Yielding/Resuming == + * Because Lua's dofile and require builtins are currently implemented as C functions in LuaJIT 2.0/2.1, if the Lua file being loaded by dofile or require invokes [[#ngx.location.capture|ngx.location.capture*]], [[#ngx.exec|ngx.exec]], [[#ngx.exit|ngx.exit]], or other API functions requiring yielding in the *top-level* scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file. == Lua Variable Scope == -Care must be taken when importing modules and this form should be used: + +Care must be taken when importing modules, and this form should be used: local xxx = require('xxx') @@ -524,7 +581,6 @@ It is therefore *highly* recommended to always declare such within an appropriat local function foo() return 123 end - To find all instances of Lua global variables in your Lua code, run the [https://github.com/openresty/nginx-devel-utils/blob/master/lua-releng lua-releng tool] across all .lua source files: $ lua-releng @@ -538,7 +594,8 @@ The output says that the line 1489 of file lib/foo/bar.lua writes t This tool will guarantee that local variables in the Lua module functions are all declared with the local keyword, otherwise a runtime exception will be thrown. It prevents undesirable race conditions while accessing such variables. See [[#Data_Sharing_within_an_Nginx_Worker|Data Sharing within an Nginx Worker]] for the reasons behind this. == Locations Configured by Subrequest Directives of Other Modules == -The [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] directives cannot capture locations that include the [[HttpAdditionModule#add_before_body|add_before_body]], [[HttpAdditionModule#add_after_body|add_after_body]], [http://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request auth_request], [[HttpEchoModule#echo_location|echo_location]], [[HttpEchoModule#echo_location_async|echo_location_async]], [[HttpEchoModule#echo_subrequest|echo_subrequest]], or [[HttpEchoModule#echo_subrequest_async|echo_subrequest_async]] directives. + +The [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] directives cannot capture locations that include the [[HttpAdditionModule#add_before_body|add_before_body]], [[HttpAdditionModule#add_after_body|add_after_body]], [https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request auth_request], [[HttpEchoModule#echo_location|echo_location]], [[HttpEchoModule#echo_location_async|echo_location_async]], [[HttpEchoModule#echo_subrequest|echo_subrequest]], or [[HttpEchoModule#echo_subrequest_async|echo_subrequest_async]] directives. location /foo { @@ -562,17 +619,17 @@ will not work as expected. == Cosockets Not Available Everywhere == -Due to internal limitations in the nginx core, the cosocket API is disabled in the following contexts: [[#set_by_lua|set_by_lua*]], [[#log_by_lua|log_by_lua*]], [[#header_filter_by_lua|header_filter_by_lua*]], and [[#body_filter_by_lua|body_filter_by_lua]]. +Due to internal limitations in the Nginx core, the cosocket API is disabled in the following contexts: [[#set_by_lua|set_by_lua*]], [[#log_by_lua|log_by_lua*]], [[#header_filter_by_lua|header_filter_by_lua*]], and [[#body_filter_by_lua|body_filter_by_lua]]. -The cosockets are currently also disabled in the [[#init_by_lua|init_by_lua*]] and [[#init_worker_by_lua|init_worker_by_lua*]] directive contexts but we may add support for these contexts in the future because there is no limitation in the nginx core (or the limitation might be worked around). +The cosockets are currently also disabled in the [[#init_by_lua|init_by_lua*]] and [[#init_worker_by_lua|init_worker_by_lua*]] directive contexts but we may add support for these contexts in the future because there is no limitation in the Nginx core (or the limitation might be worked around). -There exists a work-around, however, when the original context does *not* need to wait for the cosocket results. That is, creating a zero-delay timer via the [[#ngx.timer.at|ngx.timer.at]] API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. +There exists a workaround, however, when the original context does *not* need to wait for the cosocket results. That is, creating a zero-delay timer via the [[#ngx.timer.at|ngx.timer.at]] API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer. == Special Escaping Sequences == '''NOTE''' Following the v0.9.17 release, this pitfall can be avoided by using the *_by_lua_block {} configuration directives. -PCRE sequences such as \d, \s, or \w, require special attention because in string literals, the backslash character, \, is stripped out by both the Lua language parser and by the nginx config file parser before processing if not within a *_by_lua_block {} directive. So the following snippet will not work as expected: +PCRE sequences such as \d, \s, or \w, require special attention because in string literals, the backslash character, \, is stripped out by both the Lua language parser and by the Nginx config file parser before processing if not within a *_by_lua_block {} directive. So the following snippet will not work as expected: # nginx.conf @@ -668,7 +725,6 @@ As noted earlier, PCRE sequences presented within *_by_lua_block {} # evaluates to "1234" - == Mixing with SSI Not Supported == Mixing SSI with ngx_lua in the same Nginx request is not supported at all. Just use ngx_lua exclusively. Everything you can do with SSI can be done atop ngx_lua anyway and it can be more efficient when using ngx_lua. @@ -705,7 +761,7 @@ phases. * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [[#ngx.location.capture|ngx.location.capture]]'s extra_headers option * use ngx_hash_t to optimize the built-in header look-up process for [[#ngx.req.set_header|ngx.req.set_header]], [[#ngx.header.HEADER|ngx.header.HEADER]], and etc. -* add directives to run Lua codes when nginx stops. +* add directives to run Lua codes when Nginx stops. * add ignore_resp_headers, ignore_resp_body, and ignore_resp options to [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add stat mode similar to [https://httpd.apache.org/docs/trunk/mod/mod_lua.html mod_lua]. @@ -715,7 +771,7 @@ phases. The changes made in every release of this module are listed in the change logs of the OpenResty bundle: -http://openresty.org/#Changes +https://openresty.org/#Changes = Test Suite = @@ -770,7 +826,7 @@ To run specific test files: To run a specific test block in a particular test file, add the line --- ONLY to the test block you want to run, and then use the prove utility to run that .t file. -There are also various testing modes based on mockeagain, valgrind, and etc. Refer to the [http://search.cpan.org/perldoc?Test::Nginx Test::Nginx documentation] for more details for various advanced testing modes. See also the test reports for the Nginx test cluster running on Amazon EC2: http://qa.openresty.org. +There are also various testing modes based on mockeagain, valgrind, and etc. Refer to the [https://search.cpan.org/perldoc?Test::Nginx Test::Nginx documentation] for more details for various advanced testing modes. See also the test reports for the Nginx test cluster running on Amazon EC2: https://qa.openresty.org. = Copyright and License = @@ -792,26 +848,26 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND = See Also = -* [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module] for an official port of this module for the NGINX "stream" subsystem (doing generic downstream TCP communications). +* [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module] for an official port of this module for the Nginx "stream" subsystem (doing generic downstream TCP communications). * [https://github.com/openresty/lua-resty-memcached lua-resty-memcached] library based on ngx_lua cosocket. * [https://github.com/openresty/lua-resty-redis lua-resty-redis] library based on ngx_lua cosocket. * [https://github.com/openresty/lua-resty-mysql lua-resty-mysql] library based on ngx_lua cosocket. * [https://github.com/openresty/lua-resty-upload lua-resty-upload] library based on ngx_lua cosocket. * [https://github.com/openresty/lua-resty-dns lua-resty-dns] library based on ngx_lua cosocket. * [https://github.com/openresty/lua-resty-websocket lua-resty-websocket] library for both WebSocket server and client, based on ngx_lua cosocket. -* [https://github.com/openresty/lua-resty-string lua-resty-string] library based on [http://luajit.org/ext_ffi.html LuaJIT FFI]. +* [https://github.com/openresty/lua-resty-string lua-resty-string] library based on [https://luajit.org/ext_ffi.html LuaJIT FFI]. * [https://github.com/openresty/lua-resty-lock lua-resty-lock] library for a nonblocking simple lock API. * [https://github.com/cloudflare/lua-resty-cookie lua-resty-cookie] library for HTTP cookie manipulation. -* [http://openresty.org/#RoutingMySQLQueriesBasedOnURIArgs Routing requests to different MySQL queries based on URI arguments] -* [http://openresty.org/#DynamicRoutingBasedOnRedis Dynamic Routing Based on Redis and Lua] -* [http://openresty.org/#UsingLuaRocks Using LuaRocks with ngx_lua] +* [https://openresty.org/#RoutingMySQLQueriesBasedOnURIArgs Routing requests to different MySQL queries based on URI arguments] +* [https://openresty.org/#DynamicRoutingBasedOnRedis Dynamic Routing Based on Redis and Lua] +* [https://openresty.org/#UsingLuaRocks Using LuaRocks with ngx_lua] * [https://github.com/openresty/lua-nginx-module/wiki/Introduction Introduction to ngx_lua] * [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] * [[HttpEchoModule]] * [[HttpDrizzleModule]] * [https://github.com/FRiCKLE/ngx_postgres postgres-nginx-module] * [[HttpMemcModule]] -* [http://openresty.org The OpenResty bundle] +* [https://openresty.org The OpenResty bundle] * [https://github.com/openresty/nginx-systemtap-toolkit Nginx Systemtap Toolkit] = Directives = @@ -841,14 +897,15 @@ This directive was first introduced in the v0.10.15 release and used to optionally load the resty.core module. == lua_capture_error_log == + '''syntax:''' ''lua_capture_error_log size'' '''default:''' ''none'' '''context:''' ''http'' -Enables a buffer of the specified size for capturing all the nginx error log message data (not just those produced -by this module or the nginx http subsystem, but everything) without touching files or disks. +Enables a buffer of the specified size for capturing all the Nginx error log message data (not just those produced +by this module or the Nginx http subsystem, but everything) without touching files or disks. You can use units like `k` and `m` in the size value, as in @@ -872,34 +929,36 @@ also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough. -Note that the log level specified in the standard [http://nginx.org/r/error_log error_log] directive +Note that the log level specified in the standard [https://nginx.org/r/error_log error_log] directive ''does'' have effect on this capturing facility. It only captures log -messages of a level no lower than the specified log level in the [http://nginx.org/r/error_log error_log] directive. +messages of a level no lower than the specified log level in the [https://nginx.org/r/error_log error_log] directive. The user can still choose to set an even higher filtering log level on the fly via the Lua API function [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/errlog.md#set_filter_level errlog.set_filter_level]. -So it is more flexible than the static [http://nginx.org/r/error_log error_log] directive. +So it is more flexible than the static [https://nginx.org/r/error_log error_log] directive. It is worth noting that there is no way to capture the debugging logs -without building OpenResty or NGINX with the ./configure +without building OpenResty or Nginx with the ./configure option --with-debug. And enabling debugging logs is strongly discouraged in production builds due to high overhead. This directive was first introduced in the v0.10.9 release. == lua_use_default_type == + '''syntax:''' ''lua_use_default_type on | off'' '''default:''' ''lua_use_default_type on'' '''context:''' ''http, server, location, location if'' -Specifies whether to use the MIME type specified by the [http://nginx.org/en/docs/http/ngx_http_core_module.html#default_type default_type] directive for the default value of the Content-Type response header. Deactivate this directive if a default Content-Type response header for Lua request handlers is not desired. +Specifies whether to use the MIME type specified by the [https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type default_type] directive for the default value of the Content-Type response header. Deactivate this directive if a default Content-Type response header for Lua request handlers is not desired. This directive is turned on by default. This directive was first introduced in the v0.9.1 release. == lua_malloc_trim == + '''syntax:''' ''lua_malloc_trim '' '''default:''' ''lua_malloc_trim 1000'' @@ -907,7 +966,7 @@ This directive was first introduced in the v0.9.1 release. '''context:''' ''http'' Asks the underlying libc runtime library to release its cached free memory back to the operating system every -N requests processed by the NGINX core. By default, N is 1000. You can configure the request count +N requests processed by the Nginx core. By default, N is 1000. You can configure the request count by using your own numbers. Smaller numbers mean more frequent releases, which may introduce higher CPU time consumption and smaller memory footprint while larger numbers usually lead to less CPU time overhead and relatively larger memory footprint. Just tune the number for your own use cases. @@ -918,8 +977,8 @@ Configuring the argument to 0 essentially turns off the periodical lua_malloc_trim 0; # turn off trimming completely -The current implementation uses an NGINX log phase handler to do the request counting. So the appearance of the -[http://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest log_subrequest on] directives in nginx.conf +The current implementation uses an Nginx log phase handler to do the request counting. So the appearance of the +[https://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest log_subrequest on] directives in nginx.conf may make the counting faster when subrequests are involved. By default, only "main requests" count. Note that this directive does *not* affect the memory allocated by LuaJIT's own allocator based on the mmap @@ -961,6 +1020,7 @@ discouraged for production use and should only be used during development as it has a significant negative impact on overall performance. For example, the performance of a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. == lua_regex_cache_max_entries == + '''syntax:''' ''lua_regex_cache_max_entries '' '''default:''' ''lua_regex_cache_max_entries 1024'' @@ -982,6 +1042,7 @@ If you are using the ngx.re.* implementation of [lua-resty-core](ht Do not activate the o option for regular expressions (and/or replace string arguments for [[#ngx.re.sub|ngx.re.sub]] and [[#ngx.re.gsub|ngx.re.gsub]]) that are generated ''on the fly'' and give rise to infinite variations to avoid hitting the specified limit. == lua_regex_match_limit == + '''syntax:''' ''lua_regex_match_limit '' '''default:''' ''lua_regex_match_limit 0'' @@ -1077,9 +1138,9 @@ You can also initialize the [[#lua_shared_dict|lua_shared_dict]] shm storage at But note that, the [[#lua_shared_dict|lua_shared_dict]]'s shm storage will not be cleared through a config reload (via the HUP signal, for example). So if you do ''not'' want to re-initialize the shm storage in your init_by_lua code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your init_by_lua code. -Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the [http://en.wikipedia.org/wiki/Copy-on-write Copy-on-write (COW)] feature provided by many operating systems among all the worker processes, thus saving a lot of memory. +Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the [https://en.wikipedia.org/wiki/Copy-on-write Copy-on-write (COW)] feature provided by many operating systems among all the worker processes, thus saving a lot of memory. -Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [[#Lua_Variable_Scope|Lua Variable Scope]] section for more details). The recommended way is to use proper [http://www.lua.org/manual/5.1/manual.html#5.3 Lua module] files (but do not use the standard Lua function [http://www.lua.org/manual/5.1/manual.html#pdf-module module()] to define Lua modules because it pollutes the global namespace as well) and call [http://www.lua.org/manual/5.1/manual.html#pdf-require require()] to load your own module files in init_by_lua or other contexts ([http://www.lua.org/manual/5.1/manual.html#pdf-require require()] does cache the loaded Lua modules in the global package.loaded table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). +Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [[#Lua_Variable_Scope|Lua Variable Scope]] section for more details). The recommended way is to use proper [https://www.lua.org/manual/5.1/manual.html#5.3 Lua module] files (but do not use the standard Lua function [https://www.lua.org/manual/5.1/manual.html#pdf-module module()] to define Lua modules because it pollutes the global namespace as well) and call [https://www.lua.org/manual/5.1/manual.html#pdf-require require()] to load your own module files in init_by_lua or other contexts ([https://www.lua.org/manual/5.1/manual.html#pdf-require require()] does cache the loaded Lua modules in the global package.loaded table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). Only a small set of the [[#Nginx API for Lua|Nginx API for Lua]] is supported in this context: @@ -1104,7 +1165,7 @@ This directive was first introduced in the v0.5.5 release. Similar to the [[#init_by_lua|init_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1125,7 +1186,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''loading-config'' -Equivalent to [[#init_by_lua|init_by_lua]], except that the file specified by contains the Lua code or [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#init_by_lua|init_by_lua]], except that the file specified by contains the Lua code or [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -1186,7 +1247,7 @@ This hook no longer runs in the cache manager and cache loader processes since t Similar to the [[#init_worker_by_lua|init_worker_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1282,7 +1343,7 @@ This directive requires the [https://github.com/simplresty/ngx_devel_kit ngx_dev Similar to the [[#set_by_lua|set_by_lua]] directive except that # this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping), and # this directive does not support extra arguments after the Lua script as in [[#set_by_lua|set_by_lua]]. @@ -1298,13 +1359,14 @@ No special escaping is required in the Lua code block. This directive was first introduced in the v0.9.17 release. == set_by_lua_file == + '''syntax:''' ''set_by_lua_file $res [$arg1 $arg2 ...]'' '''context:''' ''server, server if, location, location if'' '''phase:''' ''rewrite'' -Equivalent to [[#set_by_lua|set_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#set_by_lua|set_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variable interpolation is supported in the argument string of this directive. But special care must be taken for injection attacks. @@ -1342,7 +1404,7 @@ Do not use this directive and other content handler directives in the same locat Similar to the [[#content_by_lua|content_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1363,7 +1425,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''content'' -Equivalent to [[#content_by_lua|content_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#content_by_lua|content_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1486,7 +1548,7 @@ can be implemented in ngx_lua as: Just as any other rewrite phase handlers, [[#rewrite_by_lua|rewrite_by_lua]] also runs in subrequests. -Note that when calling ngx.exit(ngx.OK) within a [[#rewrite_by_lua|rewrite_by_lua]] handler, the nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#rewrite_by_lua|rewrite_by_lua]] handler, calling [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. +Note that when calling ngx.exit(ngx.OK) within a [[#rewrite_by_lua|rewrite_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#rewrite_by_lua|rewrite_by_lua]] handler, calling [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. If the [[HttpRewriteModule]]'s [[HttpRewriteModule#rewrite|rewrite]] directive is used to change the URI and initiate location re-lookups (internal redirections), then any [[#rewrite_by_lua|rewrite_by_lua]] or [[#rewrite_by_lua_file|rewrite_by_lua_file]] code sequences within the current location will not be executed. For example, @@ -1514,7 +1576,7 @@ The rewrite_by_lua code will always run at the end of the rew Similar to the [[#rewrite_by_lua|rewrite_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1535,7 +1597,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''rewrite tail'' -Equivalent to [[#rewrite_by_lua|rewrite_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#rewrite_by_lua|rewrite_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1614,11 +1676,11 @@ can be implemented in ngx_lua as: As with other access phase handlers, [[#access_by_lua|access_by_lua]] will ''not'' run in subrequests. -Note that when calling ngx.exit(ngx.OK) within a [[#access_by_lua|access_by_lua]] handler, the nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#access_by_lua|access_by_lua]] handler, calling [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. +Note that when calling ngx.exit(ngx.OK) within a [[#access_by_lua|access_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#access_by_lua|access_by_lua]] handler, calling [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. Starting from the v0.9.20 release, you can use the [[#access_by_lua_no_postpone|access_by_lua_no_postpone]] directive to control when to run this handler inside the "access" request-processing phase -of NGINX. +of Nginx. == access_by_lua_block == @@ -1630,7 +1692,7 @@ of NGINX. Similar to the [[#access_by_lua|access_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1651,7 +1713,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''access tail'' -Equivalent to [[#access_by_lua|access_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#access_by_lua|access_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1703,7 +1765,7 @@ This directive was first introduced in the v0.2.1rc20 release. Similar to the [[#header_filter_by_lua|header_filter_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1724,7 +1786,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''output-header-filter'' -Equivalent to [[#header_filter_by_lua|header_filter_by_lua]], except that the file specified by contains the Lua code, or as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#header_filter_by_lua|header_filter_by_lua]], except that the file specified by contains the Lua code, or as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -1804,7 +1866,7 @@ When the Lua code may change the length of the response body, then it is require } -Note that the following API functions are currently disabled within this context due to the limitations in NGINX output filter's current implementation: +Note that the following API functions are currently disabled within this context due to the limitations in Nginx output filter's current implementation: * Output API functions (e.g., [[#ngx.say|ngx.say]] and [[#ngx.send_headers|ngx.send_headers]]) * Control API functions (e.g., [[#ngx.exit|ngx.exit]] and [[#ngx.exec|ngx.exec]]) @@ -1825,7 +1887,7 @@ This directive was first introduced in the v0.5.0rc32 release. Similar to the [[#body_filter_by_lua|body_filter_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1846,7 +1908,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''output-body-filter'' -Equivalent to [[#body_filter_by_lua|body_filter_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#body_filter_by_lua|body_filter_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -1925,7 +1987,7 @@ This directive was first introduced in the v0.5.0rc31 release. Similar to the [[#log_by_lua|log_by_lua]] directive except that this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires +inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires special character escaping). For instance, @@ -1946,7 +2008,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''log'' -Equivalent to [[#log_by_lua|log_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#log_by_lua|log_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -1981,13 +2043,13 @@ For instance, } -The resulting Lua load balancer can work with any existing nginx upstream modules -like [http://nginx.org/en/docs/http/ngx_http_proxy_module.html ngx_proxy] and -[http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html ngx_fastcgi]. +The resulting Lua load balancer can work with any existing Nginx upstream modules +like [https://nginx.org/en/docs/http/ngx_http_proxy_module.html ngx_proxy] and +[https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html ngx_fastcgi]. Also, the Lua load balancer can work with the standard upstream connection pool mechanism, -i.e., the standard [http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive keepalive] directive. -Just ensure that the [http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive keepalive] directive +i.e., the standard [https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive keepalive] directive. +Just ensure that the [https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive keepalive] directive is used *after* this balancer_by_lua_block directive in a single upstream {} configuration block. The Lua load balancer can totally ignore the list of servers defined in the upstream {} block @@ -1996,8 +2058,8 @@ and select peer from a completely dynamic server list (even changing per request from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. The Lua code handler registered by this directive might get called more than once in a single -downstream request when the nginx upstream mechanism retries the request on conditions -specified by directives like the [http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream proxy_next_upstream] +downstream request when the Nginx upstream mechanism retries the request on conditions +specified by directives like the [https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream proxy_next_upstream] directive. This Lua code execution context does not support yielding, so Lua APIs that may yield @@ -2016,7 +2078,7 @@ This directive was first introduced in the v0.10.0 release. '''phase:''' ''content'' -Equivalent to [[#balancer_by_lua_block|balancer_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#balancer_by_lua_block|balancer_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -2055,7 +2117,7 @@ This also applies to [[#access_by_lua|access_by_lua*]]. '''phase:''' ''right-before-SSL-handshake'' -This directive runs user Lua code when NGINX is about to start the SSL handshake for the downstream +This directive runs user Lua code when Nginx is about to start the SSL handshake for the downstream SSL (https) connections. It is particularly useful for setting the SSL certificate chain and the corresponding private key on a per-request @@ -2077,9 +2139,9 @@ library are particularly useful in this context. You can use the Lua API offered to manipulate the SSL certificate chain and private key for the current SSL connection being initiated. -This Lua handler does not run at all, however, when NGINX/OpenSSL successfully resumes +This Lua handler does not run at all, however, when Nginx/OpenSSL successfully resumes the SSL session via SSL session IDs or TLS session tickets for the current SSL connection. In -other words, this Lua handler only runs when NGINX has to initiate a full SSL handshake. +other words, this Lua handler only runs when Nginx has to initiate a full SSL handshake. Below is a trivial example using the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md ngx.ssl] module @@ -2111,8 +2173,8 @@ This Lua code execution context *does* support yielding, so Lua APIs that may yi (like cosockets, sleeping, and "light threads") are enabled in this context. -Note, however, you still need to configure the [http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate ssl_certificate] and -[http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key ssl_certificate_key] +Note, however, you still need to configure the [https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate ssl_certificate] and +[https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key ssl_certificate_key] directives even though you will not use this static certificate and private key at all. This is because the NGINX core requires their appearance otherwise you are seeing the following error while starting NGINX: @@ -2124,7 +2186,7 @@ while starting NGINX: This directive requires OpenSSL 1.0.2e or greater. If you are using the [official pre-built -packages](http://openresty.org/en/linux-packages.html) for +packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.9.7.2 or later, then everything should work out of the box. @@ -2135,9 +2197,9 @@ in order to use this directive: https://openresty.org/en/openssl-patches.html -Similarly, if you are not using the NGINX core shipped with +Similarly, if you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.9.7.2 or later, you will need to apply -patches to the standard NGINX core: +patches to the standard Nginx core: https://openresty.org/en/nginx-ssl-patches.html @@ -2151,7 +2213,7 @@ This directive was first introduced in the v0.10.0 release. '''phase:''' ''right-before-SSL-handshake'' -Equivalent to [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -2192,7 +2254,7 @@ When [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]] is specified at t this hook usually runs before [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]]. When the SSL session is found and successfully loaded for the current SSL connection, SSL session resumption will happen and thus bypass the [[#ssl_certificate_by_lua_block|ssl_certificate_by_lua*]] -hook completely. In this case, NGINX also bypasses the [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]] +hook completely. In this case, Nginx also bypasses the [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]] hook, for obvious reasons. To easily test this hook locally with a modern web browser, you can temporarily put the following line @@ -2202,7 +2264,7 @@ in your https server block to disable the TLS session ticket support: But do not forget to comment this line out before publishing your site to the world. -If you are using the [official pre-built packages](http://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) +If you are using the [official pre-built packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.11.2.1 or later, then everything should work out of the box. If you are not using one of the [OpenSSL @@ -2212,9 +2274,9 @@ in order to use this directive: https://openresty.org/en/openssl-patches.html -Similarly, if you are not using the NGINX core shipped with +Similarly, if you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.11.2.1 or later, you will need to apply -patches to the standard NGINX core: +patches to the standard Nginx core: https://openresty.org/en/nginx-ssl-patches.html @@ -2232,7 +2294,7 @@ before server name dispatch. '''phase:''' ''right-before-SSL-handshake'' -Equivalent to [[#ssl_session_fetch_by_lua_block|ssl_session_fetch_by_lua_block]], except that the file specified by contains the Lua code, or rather, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#ssl_session_fetch_by_lua_block|ssl_session_fetch_by_lua_block]], except that the file specified by contains the Lua code, or rather, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -2283,7 +2345,7 @@ Note that: this directive is only allowed to used in '''http context''' from the '''phase:''' ''right-after-SSL-handshake'' -Equivalent to [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua_block]], except that the file specified by contains the Lua code, or rather, the [[#Lua/LuaJIT bytecode support|Lua/LuaJIT bytecode]] to be executed. +Equivalent to [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua_block]], except that the file specified by contains the Lua code, or rather, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -2304,7 +2366,7 @@ Note that: this directive is only allowed to used in '''http context''' from the Declares a shared memory zone, , to serve as storage for the shm based Lua dictionary ngx.shared.. -Shared memory zones are always shared by all the nginx worker processes in the current nginx server instance. +Shared memory zones are always shared by all the Nginx worker processes in the current Nginx server instance. The argument accepts size units such as k and m: @@ -2404,7 +2466,7 @@ Default to 30 connections for every pool. When the connection pool exceeds the available size limit, the least recently used (idle) connection already in the pool will be closed to make room for the current connection. -Note that the cosocket connection pool is per nginx worker process rather than per nginx server instance, so size limit specified here also applies to every single nginx worker process. +Note that the cosocket connection pool is per Nginx worker process rather than per Nginx server instance, so size limit specified here also applies to every single Nginx worker process. This directive was first introduced in the v0.5.0rc1 release. @@ -2430,7 +2492,7 @@ This directive was first introduced in the v0.5.0rc1 release. '''context:''' ''http, server, location'' -This directive can be used to toggle error logging when a failure occurs for the TCP or UDP cosockets. If you are already doing proper error handling and logging in your Lua code, then it is recommended to turn this directive off to prevent data flushing in your nginx error log files (which is usually rather expensive). +This directive can be used to toggle error logging when a failure occurs for the TCP or UDP cosockets. If you are already doing proper error handling and logging in your Lua code, then it is recommended to turn this directive off to prevent data flushing in your Nginx error log files (which is usually rather expensive). This directive was first introduced in the v0.5.13 release. @@ -2627,9 +2689,9 @@ This directive was first introduced in the v0.8.0 release. '''context:''' ''http'' -When enabled, this module will set the `SA_RESTART` flag on nginx workers signal dispositions. +When enabled, this module will set the `SA_RESTART` flag on Nginx workers signal dispositions. -This allows Lua I/O primitives to not be interrupted by nginx's handling of various signals. +This allows Lua I/O primitives to not be interrupted by Nginx's handling of various signals. This directive was first introduced in the v0.10.14 release. @@ -2638,6 +2700,7 @@ This directive was first introduced in the v0.10.14 release. == Introduction == + The various *_by_lua, *_by_lua_block and *_by_lua_file configuration directives serve as gateways to the Lua API within the nginx.conf file. The Nginx Lua API described below can only be called within the user Lua code run in the context of these configuration directives. The API is exposed to Lua in the form of two standard packages ngx and ndk. These packages are in the default global scope within ngx_lua and are always available within ngx_lua directives. @@ -2656,7 +2719,7 @@ The packages can be introduced into external Lua modules like this: return _M -Use of the [http://www.lua.org/manual/5.1/manual.html#pdf-package.seeall package.seeall] flag is strongly discouraged due to its various bad side-effects. +Use of the [https://www.lua.org/manual/5.1/manual.html#pdf-package.seeall package.seeall] flag is strongly discouraged due to its various bad side-effects. It is also possible to directly require the packages in external Lua modules: @@ -2670,6 +2733,7 @@ The ability to require these packages was introduced in the v0.2.1rc19io library but huge file reading and writing should be avoided wherever possible as they may block the Nginx process significantly. Delegating all network and disk I/O operations to Nginx's subrequests (via the [[#ngx.location.capture|ngx.location.capture]] method and similar) is strongly recommended for maximum performance. == ngx.arg == + '''syntax:''' ''val = ngx.arg[index]'' '''context:''' ''set_by_lua*, body_filter_by_lua*'' @@ -2702,6 +2766,7 @@ When this table is used in the context of [[#body_filter_by_lua|body_filter_by_l The data chunk and "eof" flag passed to the downstream Nginx output filters can also be overridden by assigning values directly to the corresponding table elements. When setting nil or an empty Lua string value to ngx.arg[1], no data chunk will be passed to the downstream Nginx output filters at all. == ngx.var.VARIABLE == + '''syntax:''' ''ngx.var.VAR_NAME'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' @@ -2713,7 +2778,7 @@ Read and write Nginx variable values. ngx.var.some_nginx_variable_name = value -Note that only already defined nginx variables can be written to. +Note that only already defined Nginx variables can be written to. For example: @@ -2726,9 +2791,9 @@ For example: } -That is, nginx variables cannot be created on-the-fly. +That is, Nginx variables cannot be created on-the-fly. -Some special nginx variables like $args and $limit_rate can be assigned a value, +Some special Nginx variables like $args and $limit_rate can be assigned a value, many others are not, like $query_string, $arg_PARAMETER, and $http_NAME. Nginx regex group capturing variables $1, $2, $3, and etc, can be read by this @@ -2749,11 +2814,12 @@ Setting ngx.var.Foo to a nil value will unset the nil while uninitialized (but defined) NGINX variables are evaluated to an empty Lua string. +Undefined Nginx variables are evaluated to nil while uninitialized (but defined) Nginx variables are evaluated to an empty Lua string. This API requires a relatively expensive metamethod call and it is recommended to avoid using it on hot code paths. == Core constants == + '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -2775,6 +2841,7 @@ The ngx.null constant is a NULL light userdata usually The ngx.DECLINED constant was first introduced in the v0.5.0rc19 release. == HTTP method constants == + '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -2798,6 +2865,7 @@ The ngx.DECLINED constant was first introduced in the v0.5.0r These constants are usually used in [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] method calls. == HTTP status constants == + '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -2839,6 +2907,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture == Nginx log level constants == + '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -2856,11 +2925,12 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture These constants are usually used by the [[#ngx.log|ngx.log]] method. == print == + '''syntax:''' ''print(...)'' '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Writes argument values into the nginx error.log file with the ngx.NOTICE log level. +Writes argument values into the Nginx error.log file with the ngx.NOTICE log level. It is equivalent to @@ -2873,6 +2943,7 @@ Lua nil arguments are accepted and result in literal "nil"2048 byte limitation on error message lengths in the Nginx core. This limit includes trailing newlines and leading time stamps. If the message size exceeds this limit, Nginx will truncate the message text accordingly. This limit can be manually modified by editing the NGX_MAX_ERROR_STR macro definition in the src/core/ngx_log.h file in the Nginx source tree. == ngx.ctx == + '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*'' This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). @@ -3005,13 +3076,14 @@ return _M That is, let the caller pass the ctx table explicitly via a function argument. == ngx.location.capture == + '''syntax:''' ''res = ngx.location.capture(uri, options?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' Issues a synchronous but still non-blocking ''Nginx Subrequest'' using uri. -Nginx's subrequests provide a powerful way to make non-blocking internal requests to other locations configured with disk file directory or ''any'' other nginx C modules like ngx_proxy, ngx_fastcgi, ngx_memc, +Nginx's subrequests provide a powerful way to make non-blocking internal requests to other locations configured with disk file directory or ''any'' other Nginx C modules like ngx_proxy, ngx_fastcgi, ngx_memc, ngx_postgres, ngx_drizzle, and even ngx_lua itself and etc etc etc. Also note that subrequests just mimic the HTTP interface but there is ''no'' extra HTTP/TCP traffic ''nor'' IPC involved. Everything works internally, efficiently, on the C level. @@ -3057,7 +3129,7 @@ URI query strings can be concatenated to URI itself, for instance, Named locations like @foo are not allowed due to a limitation in -the nginx core. Use normal locations combined with the internal directive to +the Nginx core. Use normal locations combined with the internal directive to prepare internal-only locations. An optional option table can be fed as the second @@ -3074,7 +3146,7 @@ argument, which supports the options: * vars : take a Lua table which holds the values to set the specified Nginx variables in the subrequest as this option's value. This option was first introduced in the v0.3.1rc31 release. * copy_all_vars -: specify whether to copy over all the Nginx variable values of the current request to the subrequest in question. modifications of the nginx variables in the subrequest will not affect the current (parent) request. This option was first introduced in the v0.3.1rc31 release. +: specify whether to copy over all the Nginx variable values of the current request to the subrequest in question. modifications of the Nginx variables in the subrequest will not affect the current (parent) request. This option was first introduced in the v0.3.1rc31 release. * share_all_vars : specify whether to share all the Nginx variables of the subrequest with the current (parent) request. modifications of the Nginx variables in the subrequest will affect the current (parent) request. Enabling this option may lead to hard-to-debug issues due to bad side-effects and is considered bad and harmful. Only enable this option when you completely know what you are doing. * always_forward_body @@ -3119,7 +3191,7 @@ The args option can also take plain query strings: This is functionally identical to the previous examples. -The share_all_vars option controls whether to share nginx variables among the current request and its subrequests. +The share_all_vars option controls whether to share Nginx variables among the current request and its subrequests. If this option is set to true, then the current request and associated subrequests will share the same Nginx variable scope. Hence, changes to Nginx variables made by a subrequest will affect the current request. Care should be taken in using this option as variable scope sharing can have unexpected side effects. The args, vars, or copy_all_vars options are generally preferable instead. @@ -3282,6 +3354,7 @@ The limit can be manually modified if required by editing the definition of the Please also refer to restrictions on capturing locations configured by [[#Locations_Configured_by_Subrequest_Directives_of_Other_Modules|subrequest directives of other modules]]. == ngx.location.capture_multi == + '''syntax:''' ''res1, res2, ... = ngx.location.capture_multi({ {uri, options?}, {uri, options?}, ... })'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -3341,6 +3414,7 @@ of this function. Logically speaking, the [[#ngx.location.capture|ngx.location.c Please also refer to restrictions on capturing locations configured by [[#Locations_Configured_by_Subrequest_Directives_of_Other_Modules|subrequest directives of other modules]]. == ngx.status == + '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Read and write the current request's response status. This should be called @@ -3351,13 +3425,14 @@ before sending out the response headers. status = ngx.status -Setting ngx.status after the response header is sent out has no effect but leaving an error message in your nginx's error log file: +Setting ngx.status after the response header is sent out has no effect but leaving an error message in your Nginx's error log file: attempt to set ngx.status after sending out response headers == ngx.header.HEADER == + '''syntax:''' ''ngx.header.HEADER = VALUE'' '''syntax:''' ''value = ngx.header.HEADER'' @@ -3465,6 +3540,7 @@ will contain all characters up to (and excluding) the first occurrence of For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_headers]] function instead. == ngx.resp.get_headers == + '''syntax:''' ''headers, err = ngx.resp.get_headers(max_headers?, raw?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' @@ -3490,18 +3566,20 @@ Note that a maximum of 100 response headers are parsed by default (including tho This API was first introduced in the v0.9.5 release. == ngx.req.is_internal == + '''syntax:''' ''is_internal = ngx.req.is_internal()'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' Returns a boolean indicating whether the current request is an "internal request", i.e., -a request initiated from inside the current nginx server instead of from the client side. +a request initiated from inside the current Nginx server instead of from the client side. Subrequests are all internal requests and so are requests after internal redirects. This API was first introduced in the v0.9.20 release. == ngx.req.start_time == + '''syntax:''' ''secs = ngx.req.start_time()'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' @@ -3519,6 +3597,7 @@ This function was first introduced in the v0.7.7 release. See also [[#ngx.now|ngx.now]] and [[#ngx.update_time|ngx.update_time]]. == ngx.req.http_version == + '''syntax:''' ''num = ngx.req.http_version()'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*'' @@ -3530,6 +3609,7 @@ Current possible values are 2.0, 1.0, 1.1, and 0.9. Returns nil for This method was first introduced in the v0.7.17 release. == ngx.req.raw_header == + '''syntax:''' ''str = ngx.req.raw_header(no_request_line?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*'' @@ -3573,6 +3653,7 @@ This method was first introduced in the v0.7.17 release. This method does not work in HTTP/2 requests yet. == ngx.req.get_method == + '''syntax:''' ''method_name = ngx.req.get_method()'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, balancer_by_lua*, log_by_lua*'' @@ -3586,6 +3667,7 @@ This method was first introduced in the v0.5.6 release. See also [[#ngx.req.set_method|ngx.req.set_method]]. == ngx.req.set_method == + '''syntax:''' ''ngx.req.set_method(method_id)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*'' @@ -3599,6 +3681,7 @@ This method was first introduced in the v0.5.6 release. See also [[#ngx.req.get_method|ngx.req.get_method]]. == ngx.req.set_uri == + '''syntax:''' ''ngx.req.set_uri(uri, jump?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*'' @@ -3609,7 +3692,7 @@ The optional boolean jump argument can trigger location rematch (or Location jump will not be triggered otherwise, and only the current request's URI will be modified, which is also the default behavior. This function will return but with no returned values when the jump argument is false or absent altogether. -For example, the following nginx config snippet +For example, the following Nginx config snippet rewrite ^ /foo last; @@ -3685,6 +3768,7 @@ or This interface was first introduced in the v0.3.1rc14 release. == ngx.req.set_uri_args == + '''syntax:''' ''ngx.req.set_uri_args(args)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*'' @@ -3716,6 +3800,7 @@ This interface was first introduced in the v0.3.1rc13 release. See also [[#ngx.req.set_uri|ngx.req.set_uri]]. == ngx.req.get_uri_args == + '''syntax:''' ''args, err = ngx.req.get_uri_args(max_args?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' @@ -3773,7 +3858,7 @@ That is, they will take Lua boolean values true. However, they are Empty key arguments are discarded. GET /test?=hello&=world will yield an empty output for instance. -Updating query arguments via the nginx variable $args (or ngx.var.args in Lua) at runtime is also supported: +Updating query arguments via the Nginx variable $args (or ngx.var.args in Lua) at runtime is also supported: ngx.var.args = "a=3&b=42" @@ -3808,6 +3893,7 @@ This argument can be set to zero to remove the limit and to process all request Removing the max_args cap is strongly discouraged. == ngx.req.get_post_args == + '''syntax:''' ''args, err = ngx.req.get_post_args(max_args?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' @@ -3906,6 +3992,7 @@ This argument can be set to zero to remove the limit and to process all request Removing the max_args cap is strongly discouraged. == ngx.req.get_headers == + '''syntax:''' ''headers, err = ngx.req.get_headers(max_headers?, raw?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' @@ -3979,6 +4066,7 @@ Also, by default, an __index metamethod is added to the resulting L The __index metamethod will not be added when the raw argument is set to true. == ngx.req.set_header == + '''syntax:''' ''ngx.req.set_header(header_name, header_value)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*'' @@ -4022,6 +4110,7 @@ is equivalent to == ngx.req.clear_header == + '''syntax:''' ''ngx.req.clear_header(header_name)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*'' @@ -4029,6 +4118,7 @@ is equivalent to Clears the current request's request header named header_name. None of the current request's existing subrequests will be affected but subsequently initiated subrequests will inherit the change by default. == ngx.req.read_body == + '''syntax:''' ''ngx.req.read_body()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4056,6 +4146,7 @@ In cases where current request may have a request body and the request body data This function was first introduced in the v0.3.1rc17 release. == ngx.req.discard_body == + '''syntax:''' ''ngx.req.discard_body()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4071,6 +4162,7 @@ This function was first introduced in the v0.3.1rc17 release. See also [[#ngx.req.read_body|ngx.req.read_body]]. == ngx.req.get_body_data == + '''syntax:''' ''data = ngx.req.get_body_data()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, log_by_lua*'' @@ -4096,6 +4188,7 @@ This function was first introduced in the v0.3.1rc17 release. See also [[#ngx.req.get_body_file|ngx.req.get_body_file]]. == ngx.req.get_body_file == + '''syntax:''' ''file_name = ngx.req.get_body_file()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4115,6 +4208,7 @@ This function was first introduced in the v0.3.1rc17 release. See also [[#ngx.req.get_body_data|ngx.req.get_body_data]]. == ngx.req.set_body_data == + '''syntax:''' ''ngx.req.set_body_data(data)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4130,6 +4224,7 @@ This function was first introduced in the v0.3.1rc18 release. See also [[#ngx.req.set_body_file|ngx.req.set_body_file]]. == ngx.req.set_body_file == + '''syntax:''' ''ngx.req.set_body_file(file_name, auto_clean?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4149,6 +4244,7 @@ This function was first introduced in the v0.3.1rc18 release. See also [[#ngx.req.set_body_data|ngx.req.set_body_data]]. == ngx.req.init_body == + '''syntax:''' ''ngx.req.init_body(buffer_size?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4176,6 +4272,7 @@ This function can be used with [[#ngx.req.append_body|ngx.req.append_body]], [[# This function was first introduced in the v0.5.11 release. == ngx.req.append_body == + '''syntax:''' ''ngx.req.append_body(data_chunk)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4193,6 +4290,7 @@ This function was first introduced in the v0.5.11 release. See also [[#ngx.req.init_body|ngx.req.init_body]]. == ngx.req.finish_body == + '''syntax:''' ''ngx.req.finish_body()'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4206,6 +4304,7 @@ This function was first introduced in the v0.5.11 release. See also [[#ngx.req.init_body|ngx.req.init_body]]. == ngx.req.socket == + '''syntax:''' ''tcpsock, err = ngx.req.socket()'' '''syntax:''' ''tcpsock, err = ngx.req.socket(raw)'' @@ -4225,11 +4324,12 @@ Since the v0.9.0 release, this function accepts an optional boolean When the raw argument is true, it is required that no pending data from any previous [[#ngx.say|ngx.say]], [[#ngx.print|ngx.print]], or [[#ngx.send_headers|ngx.send_headers]] calls exists. So if you have these downstream output calls previously, you should call [[#ngx.flush|ngx.flush(true)]] before calling ngx.req.socket(true) to ensure that there is no pending output data. If the request body has not been read yet, then this "raw socket" can also be used to read the request body. -You can use the "raw request socket" returned by ngx.req.socket(true) to implement fancy protocols like [http://en.wikipedia.org/wiki/WebSocket WebSocket], or just emit your own raw HTTP response header or body data. You can refer to the [https://github.com/openresty/lua-resty-websocket lua-resty-websocket library] for a real world example. +You can use the "raw request socket" returned by ngx.req.socket(true) to implement fancy protocols like [https://en.wikipedia.org/wiki/WebSocket WebSocket], or just emit your own raw HTTP response header or body data. You can refer to the [https://github.com/openresty/lua-resty-websocket lua-resty-websocket library] for a real world example. This function was first introduced in the v0.5.0rc1 release. == ngx.exec == + '''syntax:''' ''ngx.exec(uri, args?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4290,6 +4390,7 @@ outputs by either [[#ngx.print|ngx.print]] or [[#ngx.say|ngx.say]]. It is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.exec(...) be adopted when this method call is used in contexts other than [[#header_filter_by_lua|header_filter_by_lua*]] to reinforce the fact that the request processing is being terminated. == ngx.redirect == + '''syntax:''' ''ngx.redirect(uri, status?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4374,6 +4475,7 @@ outputs by either [[#ngx.print|ngx.print]] or [[#ngx.say|ngx.say]]. It is recommended that a coding style that combines this method call with the return statement, i.e., return ngx.redirect(...) be adopted when this method call is used in contexts other than [[#header_filter_by_lua|header_filter_by_lua*]] to reinforce the fact that the request processing is being terminated. == ngx.send_headers == + '''syntax:''' ''ok, err = ngx.send_headers()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4386,6 +4488,7 @@ Note that there is normally no need to manually send out response headers as ngx before content is output with [[#ngx.say|ngx.say]] or [[#ngx.print|ngx.print]] or when [[#content_by_lua|content_by_lua*]] exits normally. == ngx.headers_sent == + '''syntax:''' ''value = ngx.headers_sent'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4395,6 +4498,7 @@ Returns true if the response headers have been sent (by ngx_lua), a This API was first introduced in ngx_lua v0.3.1rc6. == ngx.print == + '''syntax:''' ''ok, err = ngx.print(...)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4431,6 +4535,7 @@ This is an asynchronous call and will return immediately without waiting for all Please note that both ngx.print and [[#ngx.say|ngx.say]] will always invoke the whole Nginx output body filter chain, which is an expensive operation. So be careful when calling either of these two in a tight loop; buffer the data yourself in Lua and save the calls. == ngx.say == + '''syntax:''' ''ok, err = ngx.say(...)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4438,6 +4543,7 @@ Please note that both ngx.print and [[#ngx.say|ngx.say]] will alway Just as [[#ngx.print|ngx.print]] but also emit a trailing newline. == ngx.log == + '''syntax:''' ''ngx.log(log_level, ...)'' '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4451,6 +4557,7 @@ The log_level argument can take constants like ngx.ERR There is a hard coded 2048 byte limitation on error message lengths in the Nginx core. This limit includes trailing newlines and leading time stamps. If the message size exceeds this limit, Nginx will truncate the message text accordingly. This limit can be manually modified by editing the NGX_MAX_ERROR_STR macro definition in the src/core/ngx_log.h file in the Nginx source tree. == ngx.flush == + '''syntax:''' ''ok, err = ngx.flush(wait?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4468,11 +4575,12 @@ Note that ngx.flush is not functional when in the HTTP 1.0 output b Since v0.8.3 this function returns 1 on success, or returns nil and a string describing the error otherwise. == ngx.exit == + '''syntax:''' ''ngx.exit(status)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -When status >= 200 (i.e., ngx.HTTP_OK and above), it will interrupt the execution of the current request and return status code to nginx. +When status >= 200 (i.e., ngx.HTTP_OK and above), it will interrupt the execution of the current request and return status code to Nginx. When status == 0 (i.e., ngx.OK), it will only quit the current phase handler (or the content handler if the [[#content_by_lua|content_by_lua*]] directive is used) and continue to run later phases (if any) for the current request. @@ -4517,6 +4625,7 @@ When being used in the contexts of [[#header_filter_by_lua|header_filter_by_lua* an asynchronous operation and will return immediately. This behavior may change in future and it is recommended that users always use return in combination as suggested above. == ngx.eof == + '''syntax:''' ''ok, err = ngx.eof()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -4547,6 +4656,7 @@ A better way to do background jobs is to use the [[#ngx.timer.at|ngx.timer.at]] Since v0.8.3 this function returns 1 on success, or returns nil and a string describing the error otherwise. == ngx.sleep == + '''syntax:''' ''ngx.sleep(seconds)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -4560,6 +4670,7 @@ Since the 0.7.20 release, The 0 time argument can also This method was introduced in the 0.5.0rc30 release. == ngx.escape_uri == + '''syntax:''' ''newstr = ngx.escape_uri(str)'' '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4567,6 +4678,7 @@ This method was introduced in the 0.5.0rc30 release. Escape str as a URI component. == ngx.unescape_uri == + '''syntax:''' ''newstr = ngx.unescape_uri(str)'' '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*'' @@ -4586,6 +4698,7 @@ gives the output == ngx.encode_args == + '''syntax:''' ''str = ngx.encode_args(table)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*'' @@ -4637,6 +4750,7 @@ If the argument value is false, then the effect is equivalent to th This method was first introduced in the v0.3.1rc27 release. == ngx.decode_args == + '''syntax:''' ''table, err = ngx.decode_args(str, max_args?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4656,6 +4770,7 @@ Removing the max_args cap is strongly discouraged. This method was introduced in the v0.5.0rc29. == ngx.encode_base64 == + '''syntax:''' ''newstr = ngx.encode_base64(str, no_padding?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4665,6 +4780,7 @@ Encodes str to a base64 digest. Since the 0.9.16 release, an optional boolean-typed no_padding argument can be specified to control whether the base64 padding should be appended to the resulting digest (default to false, i.e., with padding enabled). == ngx.decode_base64 == + '''syntax:''' ''newstr = ngx.decode_base64(str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4672,6 +4788,7 @@ Since the 0.9.16 release, an optional boolean-typed no_paddin Decodes the str argument as a base64 digest to the raw form. Returns nil if str is not well formed. == ngx.crc32_short == + '''syntax:''' ''intval = ngx.crc32_short(str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4685,6 +4802,7 @@ Behind the scene, it is just a thin wrapper around the ngx_crc32_shortv0.3.1rc8 release. == ngx.crc32_long == + '''syntax:''' ''intval = ngx.crc32_long(str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4698,11 +4816,12 @@ Behind the scene, it is just a thin wrapper around the ngx_crc32_longv0.3.1rc8 release. == ngx.hmac_sha1 == + '''syntax:''' ''digest = ngx.hmac_sha1(secret_key, str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Computes the [http://en.wikipedia.org/wiki/HMAC HMAC-SHA1] digest of the argument str and turns the result using the secret key . +Computes the [https://en.wikipedia.org/wiki/HMAC HMAC-SHA1] digest of the argument str and turns the result using the secret key . The raw binary form of the HMAC-SHA1 digest will be generated, use [[#ngx.encode_base64|ngx.encode_base64]], for example, to encode the result to a textual representation if desired. @@ -4726,6 +4845,7 @@ This API requires the OpenSSL library enabled in the Nginx build (usually by pas This function was first introduced in the v0.3.1rc29 release. == ngx.md5 == + '''syntax:''' ''digest = ngx.md5(str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4749,6 +4869,7 @@ yields the output See [[#ngx.md5_bin|ngx.md5_bin]] if the raw binary MD5 digest is required. == ngx.md5_bin == + '''syntax:''' ''digest = ngx.md5_bin(str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4758,6 +4879,7 @@ Returns the binary form of the MD5 digest of the str argument. See [[#ngx.md5|ngx.md5]] if the hexadecimal form of the MD5 digest is required. == ngx.sha1_bin == + '''syntax:''' ''digest = ngx.sha1_bin(str)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4769,6 +4891,7 @@ This function requires SHA-1 support in the Nginx build. (This usually just mean This function was first introduced in the v0.5.0rc6. == ngx.quote_sql_str == + '''syntax:''' ''quoted_value = ngx.quote_sql_str(raw_value)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4776,35 +4899,39 @@ This function was first introduced in the v0.5.0rc6. Returns a quoted SQL string literal according to the MySQL quoting rules. == ngx.today == + '''syntax:''' ''str = ngx.today()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Returns current date (in the format yyyy-mm-dd) from the nginx cached time (no syscall involved unlike Lua's date library). +Returns current date (in the format yyyy-mm-dd) from the Nginx cached time (no syscall involved unlike Lua's date library). This is the local time. == ngx.time == + '''syntax:''' ''secs = ngx.time()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Returns the elapsed seconds from the epoch for the current time stamp from the nginx cached time (no syscall involved unlike Lua's date library). +Returns the elapsed seconds from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). Updates of the Nginx time cache can be forced by calling [[#ngx.update_time|ngx.update_time]] first. == ngx.now == + '''syntax:''' ''secs = ngx.now()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the nginx cached time (no syscall involved unlike Lua's date library). +Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). You can forcibly update the Nginx time cache by calling [[#ngx.update_time|ngx.update_time]] first. This API was first introduced in v0.3.1rc32. == ngx.update_time == + '''syntax:''' ''ngx.update_time()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4814,24 +4941,27 @@ Forcibly updates the Nginx current time cache. This call involves a syscall and This API was first introduced in v0.3.1rc32. == ngx.localtime == + '''syntax:''' ''str = ngx.localtime()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the nginx cached time (no syscall involved unlike Lua's [http://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). +Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the Nginx cached time (no syscall involved unlike Lua's [https://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). This is the local time. == ngx.utctime == + '''syntax:''' ''str = ngx.utctime()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the nginx cached time (no syscall involved unlike Lua's [http://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). +Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the Nginx cached time (no syscall involved unlike Lua's [https://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). This is the UTC time. == ngx.cookie_time == + '''syntax:''' ''str = ngx.cookie_time(sec)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4844,6 +4974,7 @@ Returns a formatted string can be used as the cookie expiration time. The parame == ngx.http_time == + '''syntax:''' ''str = ngx.http_time(sec)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4856,6 +4987,7 @@ Returns a formated string can be used as the http header time (for example, bein == ngx.parse_http_time == + '''syntax:''' ''sec = ngx.parse_http_time(str)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -4870,13 +5002,15 @@ Parse the http time string (as returned by [[#ngx.http_time|ngx.http_time]]) int == ngx.is_subrequest == + '''syntax:''' ''value = ngx.is_subrequest'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' -Returns true if the current request is an nginx subrequest, or false otherwise. +Returns true if the current request is an Nginx subrequest, or false otherwise. == ngx.re.match == + '''syntax:''' ''captures, err = ngx.re.match(subject, regex, options?, ctx?, res_table?)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5011,7 +5145,7 @@ The ctx table argument combined with the a regex modif Note that, the options argument is not optional when the ctx argument is specified and that the empty Lua string ("") must be used as placeholder for options if no meaningful regex options are required. -This method requires the PCRE library enabled in Nginx. ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). +This method requires the PCRE library enabled in Nginx ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). To confirm that PCRE JIT is enabled, activate the Nginx debug log by adding the --with-debug option to Nginx or OpenResty's ./configure script. Then, enable the "debug" error log level in error_log directive. The following message will be generated if PCRE JIT is enabled: @@ -5024,11 +5158,12 @@ Starting from the 0.9.4 release, this function also accepts a 5th a This feature was introduced in the v0.2.1rc11 release. == ngx.re.find == + '''syntax:''' ''from, to, err = ngx.re.find(subject, regex, options?, ctx?, nth?)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Similar to [[#ngx.re.match|ngx.re.match]] but only returns the beginning index (from) and end index (to) of the matched substring. The returned indexes are 1-based and can be fed directly into the [http://www.lua.org/manual/5.1/manual.html#pdf-string.sub string.sub] API function to obtain the matched substring. +Similar to [[#ngx.re.match|ngx.re.match]] but only returns the beginning index (from) and end index (to) of the matched substring. The returned indexes are 1-based and can be fed directly into the [https://www.lua.org/manual/5.1/manual.html#pdf-string.sub string.sub] API function to obtain the matched substring. In case of errors (like bad regexes or any PCRE runtime errors), this API function returns two nil values followed by a string describing the error. @@ -5073,6 +5208,7 @@ Since the 0.9.3 release, an optional 5th argument, nth This API function was first introduced in the v0.9.2 release. == ngx.re.gmatch == + '''syntax:''' ''iterator, err = ngx.re.gmatch(subject, regex, options?)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5141,11 +5277,12 @@ The optional options argument takes exactly the same semantics as t The current implementation requires that the iterator returned should only be used in a single request. That is, one should ''not'' assign it to a variable belonging to persistent namespace like a Lua package. -This method requires the PCRE library enabled in Nginx. ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). +This method requires the PCRE library enabled in Nginx ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). This feature was first introduced in the v0.2.1rc12 release. == ngx.re.sub == + '''syntax:''' ''newstr, n, err = ngx.re.sub(subject, regex, replace, options?)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5200,11 +5337,12 @@ When the replace argument is of type "function", then it will be in The dollar sign characters in the return value of the replace function argument are not special at all. -This method requires the PCRE library enabled in Nginx. ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). +This method requires the PCRE library enabled in Nginx ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). This feature was first introduced in the v0.2.1rc13 release. == ngx.re.gsub == + '''syntax:''' ''newstr, n, err = ngx.re.gsub(subject, regex, replace, options?)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5233,11 +5371,12 @@ Here is some examples: -- n == 2 -This method requires the PCRE library enabled in Nginx. ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). +This method requires the PCRE library enabled in Nginx ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). This feature was first introduced in the v0.2.1rc15 release. == ngx.shared.DICT == + '''syntax:''' ''dict = ngx.shared.DICT'' '''syntax:''' ''dict = ngx.shared[name_var]'' @@ -5246,7 +5385,7 @@ This feature was first introduced in the v0.2.1rc15 release. Fetching the shm-based Lua dictionary object for the shared memory zone named DICT defined by the [[#lua_shared_dict|lua_shared_dict]] directive. -Shared memory zones are always shared by all the nginx worker processes in the current nginx server instance. +Shared memory zones are always shared by all the Nginx worker processes in the current Nginx server instance. The resulting object dict has the following methods: @@ -5272,7 +5411,7 @@ The resulting object dict has the following methods: * [[#ngx.shared.DICT.capacity|capacity]] * [[#ngx.shared.DICT.free_space|free_space]] -All these methods are ''atomic'' operations, that is, safe from concurrent accesses from multiple nginx worker processes for the same lua_shared_dict zone. +All these methods are ''atomic'' operations, that is, safe from concurrent accesses from multiple Nginx worker processes for the same lua_shared_dict zone. Here is an example: @@ -5319,6 +5458,7 @@ The contents in the dictionary storage will be lost, however, when the Nginx ser This feature was first introduced in the v0.3.1rc22 release. == ngx.shared.DICT.get == + '''syntax:''' ''value, flags = ngx.shared.DICT:get(key)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5352,6 +5492,7 @@ This feature was first introduced in the v0.3.1rc22 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.get_stale == + '''syntax:''' ''value, flags, stale = ngx.shared.DICT:get_stale(key)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5367,6 +5508,7 @@ This method was first introduced in the 0.8.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.set == + '''syntax:''' ''success, err, forcible = ngx.shared.DICT:set(key, value, exptime?, flags?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5410,6 +5552,7 @@ Please note that while internally the key-value pair is set atomically, the atom See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.safe_set == + '''syntax:''' ''ok, err = ngx.shared.DICT:safe_set(key, value, exptime?, flags?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5421,6 +5564,7 @@ This feature was first introduced in the v0.7.18 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.add == + '''syntax:''' ''success, err, forcible = ngx.shared.DICT:add(key, value, exptime?, flags?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5434,6 +5578,7 @@ This feature was first introduced in the v0.3.1rc22 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.safe_add == + '''syntax:''' ''ok, err = ngx.shared.DICT:safe_add(key, value, exptime?, flags?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5445,6 +5590,7 @@ This feature was first introduced in the v0.7.18 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.replace == + '''syntax:''' ''success, err, forcible = ngx.shared.DICT:replace(key, value, exptime?, flags?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5458,6 +5604,7 @@ This feature was first introduced in the v0.3.1rc22 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.delete == + '''syntax:''' ''ngx.shared.DICT:delete(key)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5471,6 +5618,7 @@ This feature was first introduced in the v0.3.1rc22 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.incr == + '''syntax:''' ''newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?, init_ttl?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5521,6 +5669,7 @@ The optional init_ttl parameter was introduced in the v0.10.1 See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.lpush == + '''syntax:''' ''length, err = ngx.shared.DICT:lpush(key, value)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5536,6 +5685,7 @@ This feature was first introduced in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.rpush == + '''syntax:''' ''length, err = ngx.shared.DICT:rpush(key, value)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5547,6 +5697,7 @@ This feature was first introduced in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.lpop == + '''syntax:''' ''val, err = ngx.shared.DICT:lpop(key)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5560,6 +5711,7 @@ This feature was first introduced in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.rpop == + '''syntax:''' ''val, err = ngx.shared.DICT:rpop(key)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5573,6 +5725,7 @@ This feature was first introduced in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.llen == + '''syntax:''' ''len, err = ngx.shared.DICT:llen(key)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5586,6 +5739,7 @@ This feature was first introduced in the v0.10.6 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.ttl == + '''syntax:''' ''ttl, err = ngx.shared.DICT:ttl(key)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5619,6 +5773,7 @@ This feature was first introduced in the v0.10.11 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.expire == + '''syntax:''' ''success, err = ngx.shared.DICT:expire(key, exptime)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5654,6 +5809,7 @@ This feature was first introduced in the v0.10.11 release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.flush_all == + '''syntax:''' ''ngx.shared.DICT:flush_all()'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5665,6 +5821,7 @@ This feature was first introduced in the v0.5.0rc17 release. See also [[#ngx.shared.DICT.flush_expired|ngx.shared.DICT.flush_expired]] and [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.flush_expired == + '''syntax:''' ''flushed = ngx.shared.DICT:flush_expired(max_count?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5678,6 +5835,7 @@ This feature was first introduced in the v0.6.3 release. See also [[#ngx.shared.DICT.flush_all|ngx.shared.DICT.flush_all]] and [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.get_keys == + '''syntax:''' ''keys = ngx.shared.DICT:get_keys(max_count?)'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5691,6 +5849,7 @@ By default, only the first 1024 keys (if any) are returned. When the v0.7.3 release. == ngx.shared.DICT.capacity == + '''syntax:''' ''capacity_bytes = ngx.shared.DICT:capacity()'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5713,11 +5872,12 @@ This feature was first introduced in the v0.10.11 release. '''Note:''' This method requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. -This feature requires at least nginx core version 0.7.3. +This feature requires at least Nginx core version 0.7.3. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.shared.DICT.free_space == + '''syntax:''' ''free_page_bytes = ngx.shared.DICT:free_space()'' '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -5726,7 +5886,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. Retrieves the free page size in bytes for the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. -'''Note:''' The memory for ngx.shared.DICT is allocated via the nginx slab allocator which has each slot for +'''Note:''' The memory for ngx.shared.DICT is allocated via the Nginx slab allocator which has each slot for data size ranges like \~8, 9\~16, 17\~32, ..., 1025\~2048, 2048\~ bytes. And pages are assigned to a slot if there is no room in already assigned pages for the slot. @@ -5751,11 +5911,12 @@ This feature was first introduced in the v0.10.11 release. '''Note:''' This method requires the resty.core.shdict or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. -This feature requires at least nginx core version 1.11.7. +This feature requires at least Nginx core version 1.11.7. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. == ngx.socket.udp == + '''syntax:''' ''udpsock = ngx.socket.udp()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -5775,6 +5936,7 @@ This feature was first introduced in the v0.5.7 release. See also [[#ngx.socket.tcp|ngx.socket.tcp]]. == udpsock:setpeername == + '''syntax:''' ''ok, err = udpsock:setpeername(host, port)'' '''syntax:''' ''ok, err = udpsock:setpeername("unix:/path/to/unix-domain.socket")'' @@ -5830,6 +5992,7 @@ Calling this method on an already connected socket object will cause the origina This method was first introduced in the v0.5.7 release. == udpsock:send == + '''syntax:''' ''ok, err = udpsock:send(data)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -5843,6 +6006,7 @@ The input argument data can either be a Lua string or a (nested) Lu This feature was first introduced in the v0.5.7 release. == udpsock:receive == + '''syntax:''' ''data, err = udpsock:receive(size?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -5874,6 +6038,7 @@ It is important here to call the [[#udpsock:settimeout|settimeout]] method ''bef This feature was first introduced in the v0.5.7 release. == udpsock:close == + '''syntax:''' ''ok, err = udpsock:close()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -5885,6 +6050,7 @@ Socket objects that have not invoked this method (and associated connections) wi This feature was first introduced in the v0.5.7 release. == udpsock:settimeout == + '''syntax:''' ''udpsock:settimeout(time)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -5903,6 +6069,7 @@ socket, then this API name is preferred. This API function was first added to the v0.10.1 release. == ngx.socket.tcp == + '''syntax:''' ''tcpsock = ngx.socket.tcp()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -5924,7 +6091,7 @@ Creates and returns a TCP or stream-oriented unix domain socket object (also kno It is intended to be compatible with the TCP API of the [http://w3.impa.br/~diego/software/luasocket/tcp.html LuaSocket] library but is 100% nonblocking out of the box. Also, we introduce some new APIs to provide more functionalities. -The cosocket object created by this API function has exactly the same lifetime as the Lua handler creating it. So never pass the cosocket object to any other Lua handler (including ngx.timer callback functions) and never share the cosocket object between different NGINX requests. +The cosocket object created by this API function has exactly the same lifetime as the Lua handler creating it. So never pass the cosocket object to any other Lua handler (including ngx.timer callback functions) and never share the cosocket object between different Nginx requests. For every cosocket object's underlying connection, if you do not explicitly close it (via [[#tcpsock:close|close]]) or put it back to the connection @@ -5946,6 +6113,7 @@ This feature was first introduced in the v0.5.0rc1 release. See also [[#ngx.socket.udp|ngx.socket.udp]]. == tcpsock:connect == + '''syntax:''' ''ok, err = tcpsock:connect(host, port, options_table?)'' '''syntax:''' ''ok, err = tcpsock:connect("unix:/path/to/unix-domain.socket", options_table?)'' @@ -6057,6 +6225,7 @@ The support for the options table argument was first introduced in the v0. This method was first introduced in the v0.5.0rc1 release. == tcpsock:sslhandshake == + '''syntax:''' ''session, err = tcpsock:sslhandshake(reused_session?, server_name?, ssl_verify?, send_status_req?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6099,6 +6268,7 @@ immediately. This method was first introduced in the v0.9.11 release. == tcpsock:send == + '''syntax:''' ''bytes, err = tcpsock:send(data)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6125,6 +6295,7 @@ In case of any connection errors, this method always automatically closes the cu This feature was first introduced in the v0.5.0rc1 release. == tcpsock:receive == + '''syntax:''' ''data, err, partial = tcpsock:receive(size)'' '''syntax:''' ''data, err, partial = tcpsock:receive(pattern?)'' @@ -6165,6 +6336,7 @@ Since the v0.8.8 release, this method no longer automatically close This feature was first introduced in the v0.5.0rc1 release. == tcpsock:receiveany == + '''syntax:''' ''data, err = tcpsock:receiveany(max)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6195,6 +6367,7 @@ This method doesn't automatically close the current connection when the read tim This feature was first introduced in the v0.10.14 release. == tcpsock:receiveuntil == + '''syntax:''' ''iterator = tcpsock:receiveuntil(pattern, options?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6287,6 +6460,7 @@ Since the v0.8.8 release, this method no longer automatically close This method was first introduced in the v0.5.0rc1 release. == tcpsock:close == + '''syntax:''' ''ok, err = tcpsock:close()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6300,6 +6474,7 @@ Socket objects that have not invoked this method (and associated connections) wi This feature was first introduced in the v0.5.0rc1 release. == tcpsock:settimeout == + '''syntax:''' ''tcpsock:settimeout(time)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6313,6 +6488,7 @@ Note that this method does ''not'' affect the [[#lua_socket_keepalive_timeout|lu This feature was first introduced in the v0.5.0rc1 release. == tcpsock:settimeouts == + '''syntax:''' ''tcpsock:settimeouts(connect_timeout, send_timeout, read_timeout)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6329,6 +6505,7 @@ Note that this method does ''not'' affect the [[#lua_socket_keepalive_timeout|lu This feature was first introduced in the v0.10.7 release. == tcpsock:setoption == + '''syntax:''' ''tcpsock:setoption(option, value?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6338,6 +6515,7 @@ This function is added for [http://w3.impa.br/~diego/software/luasocket/tcp.html This feature was first introduced in the v0.5.0rc1 release. == tcpsock:setkeepalive == + '''syntax:''' ''ok, err = tcpsock:setkeepalive(timeout?, size?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6381,6 +6559,7 @@ This method also makes the current cosocket object enter the "closed" state, so This feature was first introduced in the v0.5.0rc1 release. == tcpsock:getreusedtimes == + '''syntax:''' ''count, err = tcpsock:getreusedtimes()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6392,6 +6571,7 @@ If the current connection does not come from the built-in connection pool, then This feature was first introduced in the v0.5.0rc1 release. == ngx.socket.connect == + '''syntax:''' ''tcpsock, err = ngx.socket.connect(host, port)'' '''syntax:''' ''tcpsock, err = ngx.socket.connect("unix:/path/to/unix-domain.socket")'' @@ -6414,6 +6594,7 @@ There is no way to use the [[#tcpsock:settimeout|settimeout]] method to specify This feature was first introduced in the v0.5.0rc1 release. == ngx.get_phase == + '''syntax:''' ''str = ngx.get_phase()'' '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -6452,6 +6633,7 @@ Retrieves the current running phase name. Possible return values are This API was first introduced in the v0.5.10 release. == ngx.thread.spawn == + '''syntax:''' ''co = ngx.thread.spawn(func, arg1, arg2, ...)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6585,6 +6767,7 @@ Then it will generate the output This API was first enabled in the v0.7.0 release. == ngx.thread.wait == + '''syntax:''' ''ok, res1, res2, ... = ngx.thread.wait(thread1, thread2, ...)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' @@ -6683,17 +6866,19 @@ And it will generate the following output: This API was first enabled in the v0.7.0 release. == ngx.thread.kill == + '''syntax:''' ''ok, err = ngx.thread.kill(thread)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*'' Kills a running "light thread" created by [[#ngx.thread.spawn|ngx.thread.spawn]]. Returns a true value when successful or nil and a string describing the error otherwise. -According to the current implementation, only the parent coroutine (or "light thread") can kill a thread. Also, a running "light thread" with pending NGINX subrequests (initiated by [[#ngx.location.capture|ngx.location.capture]] for example) cannot be killed due to a limitation in the NGINX core. +According to the current implementation, only the parent coroutine (or "light thread") can kill a thread. Also, a running "light thread" with pending Nginx subrequests (initiated by [[#ngx.location.capture|ngx.location.capture]] for example) cannot be killed due to a limitation in the Nginx core. This API was first enabled in the v0.9.9 release. == ngx.on_abort == + '''syntax:''' ''ok, err = ngx.on_abort(callback)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' @@ -6730,6 +6915,7 @@ This API was first introduced in the v0.7.4 release. See also [[#lua_check_client_abort|lua_check_client_abort]]. == ngx.timer.at == + '''syntax:''' ''hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -6854,6 +7040,7 @@ You can pass most of the standard Lua values (nils, booleans, numbers, strings, This API was first introduced in the v0.8.0 release. == ngx.timer.every == + '''syntax:''' ''hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -6870,6 +7057,7 @@ This API also respect the [[#lua_max_pending_timers|lua_max_pending_timers]] and This API was first introduced in the v0.10.9 release. == ngx.timer.running_count == + '''syntax:''' ''count = ngx.timer.running_count()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -6879,6 +7067,7 @@ Returns the number of timers currently running. This directive was first introduced in the v0.9.20 release. == ngx.timer.pending_count == + '''syntax:''' ''count = ngx.timer.pending_count()'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' @@ -6888,16 +7077,18 @@ Returns the number of pending timers. This directive was first introduced in the v0.9.20 release. == ngx.config.subsystem == + '''syntax:''' ''subsystem = ngx.config.subsystem'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' -This string field indicates the current NGINX subsystem the current Lua environment is based on. For this module, this field always takes the string value "http". For +This string field indicates the current Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value "http". For [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module], however, this field takes the value "stream". This field was first introduced in the 0.10.1. == ngx.config.debug == + '''syntax:''' ''debug = ngx.config.debug'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' @@ -6912,7 +7103,7 @@ This field was first introduced in the 0.8.7. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' -Returns the Nginx server "prefix" path, as determined by the -p command-line option when running the nginx executable, or the path specified by the --prefix command-line option when building Nginx with the ./configure script. +Returns the Nginx server "prefix" path, as determined by the -p command-line option when running the Nginx executable, or the path specified by the --prefix command-line option when building Nginx with the ./configure script. This function was first introduced in the 0.9.2. @@ -6932,7 +7123,7 @@ This API was first introduced in the 0.9.3 release. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*'' -This function returns a string for the NGINX ./configure command's arguments string. +This function returns a string for the Nginx ./configure command's arguments string. This API was first introduced in the 0.9.5 release. @@ -6973,7 +7164,7 @@ This API was first introduced in the 0.9.5 release. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' Returns the total number of the Nginx worker processes (i.e., the value configured -by the [http://nginx.org/en/docs/ngx_core_module.html#worker_processes worker_processes] +by the [https://nginx.org/en/docs/ngx_core_module.html#worker_processes worker_processes] directive in nginx.conf). This API was first introduced in the 0.9.20 release. @@ -6989,7 +7180,7 @@ Returns the ordinal number of the current Nginx worker processes (starting from So if the total number of workers is N, then this method may return a number between 0 and N - 1 (inclusive). -This function returns meaningful values only for NGINX 1.9.1+. With earlier versions of NGINX, it +This function returns meaningful values only for Nginx 1.9.1+. With earlier versions of Nginx, it always returns nil. See also [[#ngx.worker.count|ngx.worker.count]]. @@ -6997,11 +7188,12 @@ See also [[#ngx.worker.count|ngx.worker.count]]. This API was first introduced in the 0.9.20 release. == ngx.semaphore == + '''syntax:''' ''local semaphore = require "ngx.semaphore"'' This is a Lua module that implements a classic-style semaphore API for efficient synchronizations among different "light threads". Sharing the same semaphore among different "light threads" created in different (request) -contexts are also supported as long as the "light threads" reside in the same NGINX worker process +contexts are also supported as long as the "light threads" reside in the same Nginx worker process and the [[#lua_code_cache|lua_code_cache]] directive is turned on (which is the default). This Lua module does not ship with this ngx_lua module itself rather it is shipped with @@ -7015,6 +7207,7 @@ for more details. This feature requires at least ngx_lua v0.10.0. == ngx.balancer == + '''syntax:''' ''local balancer = require "ngx.balancer"'' This is a Lua module that provides a Lua API to allow defining completely dynamic load balancers @@ -7031,6 +7224,7 @@ for more details. This feature requires at least ngx_lua v0.10.0. == ngx.ssl == + '''syntax:''' ''local ssl = require "ngx.ssl"'' This Lua module provides API functions to control the SSL handshake process in contexts like @@ -7046,6 +7240,7 @@ for this ngx.ssl Lua module for more details. This feature requires at least ngx_lua v0.10.0. == ngx.ocsp == + '''syntax:''' ''local ocsp = require "ngx.ocsp"'' This Lua module provides API to perform OCSP queries, OCSP response validations, and @@ -7065,11 +7260,12 @@ for this ngx.ocsp Lua module for more details. This feature requires at least ngx_lua v0.10.0. == ndk.set_var.DIRECTIVE == + '''syntax:''' ''res = ndk.set_var.DIRECTIVE_NAME'' '''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -This mechanism allows calling other nginx C modules' directives that are implemented by [https://github.com/simplresty/ngx_devel_kit Nginx Devel Kit] (NDK)'s set_var submodule's ndk_set_var_value. +This mechanism allows calling other Nginx C modules' directives that are implemented by [https://github.com/simplresty/ngx_devel_kit Nginx Devel Kit] (NDK)'s set_var submodule's ndk_set_var_value. For example, the following [[HttpSetMiscModule]] directives can be invoked this way: @@ -7102,72 +7298,78 @@ Similarly, the following directives provided by [[HttpEncryptedSessionModule]] c This feature requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] module. == coroutine.create == + '''syntax:''' ''co = coroutine.create(f)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Creates a user Lua coroutines with a Lua function, and returns a coroutine object. -Similar to the standard Lua [http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.create coroutine.create] API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.create coroutine.create] API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since the 0.9.2. This API was first introduced in the v0.6.0 release. == coroutine.resume == + '''syntax:''' ''ok, ... = coroutine.resume(co, ...)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Resumes the executation of a user Lua coroutine object previously yielded or just created. -Similar to the standard Lua [http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.resume coroutine.resume] API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.resume coroutine.resume] API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since the 0.9.2. This API was first introduced in the v0.6.0 release. == coroutine.yield == + '''syntax:''' ''... = coroutine.yield(...)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Yields the execution of the current user Lua coroutine. -Similar to the standard Lua [http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.yield coroutine.yield] API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.yield coroutine.yield] API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since the 0.9.2. This API was first introduced in the v0.6.0 release. == coroutine.wrap == + '''syntax:''' ''co = coroutine.wrap(f)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Similar to the standard Lua [http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.wrap coroutine.wrap] API, but works in the context of the Lua coroutines created by ngx_lua. +Similar to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.wrap coroutine.wrap] API, but works in the context of the Lua coroutines created by ngx_lua. This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since the 0.9.2. This API was first introduced in the v0.6.0 release. == coroutine.running == + '''syntax:''' ''co = coroutine.running()'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Identical to the standard Lua [http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running coroutine.running] API. +Identical to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running coroutine.running] API. This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since the 0.9.2. This API was first enabled in the v0.6.0 release. == coroutine.status == + '''syntax:''' ''status = coroutine.status(co)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Identical to the standard Lua [http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.status coroutine.status] API. +Identical to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.status coroutine.status] API. This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since the 0.9.2. @@ -7180,3 +7382,10 @@ This section is just holding obsolete documentation sections that have been eith == Special PCRE Sequences == This section has been renamed to [[#Special Escaping Sequences|Special Escaping Sequences]]. + +== Lua/LuaJIT bytecode support == + +This section has been renamed to +[[#LuaJIT bytecode support|LuaJIT bytecode support]]. As of version +v0.10.16 of this module, the standard Lua interpreter (also known +as "PUC-Rio Lua") is not supported anymore. From 8573c1c5d88a9d0438a4c51f8b74163e47dc1be1 Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 25 Apr 2019 13:25:02 +0800 Subject: [PATCH 352/848] feature: ngx.pipe: allowed for specifying timeouts in resty.core 'ngx_pipe.spawn()'. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_pipe.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index c525e0b9b8..8ff308039b 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -862,10 +862,6 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, pp->buffer_size = buffer_size; proc->_pid = pid; - proc->write_timeout = 10000; - proc->stdout_read_timeout = 10000; - proc->stderr_read_timeout = 10000; - proc->wait_timeout = 10000; proc->pipe = pp; ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, From 4b9e53f74812cb9694397a4f3c3795989fc2962e Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 25 Apr 2019 14:57:29 +0800 Subject: [PATCH 353/848] feature: ngx.pipe: allowed for calling resty.core 'ngx_pipe.shutdown()' on a sub-process when a light thread is waiting on it. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_pipe.c | 120 +++++++++++++++++++--------------------- 1 file changed, 56 insertions(+), 64 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 8ff308039b..c0a2e346ad 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -30,16 +30,12 @@ static ssize_t ngx_http_lua_pipe_fd_read(ngx_connection_t *c, u_char *buf, size_t size); static ssize_t ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, size_t size); -static ngx_int_t ngx_http_lua_pipe_close_helper( - ngx_http_lua_pipe_ctx_t *pipe_ctx, ngx_event_t *ev, int forced); -static ngx_int_t ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe, - int forced); -static ngx_int_t ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe, - int forced); -static ngx_int_t ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe, - int forced); -static void ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc, - int forced); +static void ngx_http_lua_pipe_close_helper( + ngx_http_lua_pipe_ctx_t *pipe_ctx, ngx_event_t *ev); +static void ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe); +static void ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe); +static void ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe); +static void ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc); static ngx_int_t ngx_http_lua_pipe_get_lua_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t **ctx, u_char *errbuf, size_t *errbuf_size); static void ngx_http_lua_pipe_put_error(ngx_http_lua_pipe_ctx_t *pipe_ctx, @@ -121,7 +117,8 @@ enum { PIPE_ERR_NOMEM, PIPE_ERR_TIMEOUT, PIPE_ERR_ADD_READ_EV, - PIPE_ERR_ADD_WRITE_EV + PIPE_ERR_ADD_WRITE_EV, + PIPE_ERR_ABORTED }; @@ -914,26 +911,27 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } -static ngx_int_t +static void ngx_http_lua_pipe_close_helper(ngx_http_lua_pipe_ctx_t *pipe_ctx, - ngx_event_t *ev, int forced) + ngx_event_t *ev) { - if (ev->handler != ngx_http_lua_pipe_dummy_event_handler && !forced) { - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua pipe cannot close fd:%d without " - "forced pipe:%p ev:%p", pipe_ctx->c->fd, pipe_ctx, ev); - return NGX_ERROR; + if (ev->handler != ngx_http_lua_pipe_dummy_event_handler) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe abort blocking operation pipe_ctx:%p ev:%p", + pipe_ctx, ev); + + pipe_ctx->err_type = PIPE_ERR_ABORTED; + ngx_post_event(ev, &ngx_posted_events); + return; } ngx_close_connection(pipe_ctx->c); pipe_ctx->c = NULL; - - return NGX_OK; } -static ngx_int_t -ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe, int forced) +static void +ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe) { ngx_event_t *wev; @@ -949,15 +947,13 @@ ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe, int forced) } else if (pipe->stdin_ctx->c != NULL) { wev = pipe->stdin_ctx->c->write; - return ngx_http_lua_pipe_close_helper(pipe->stdin_ctx, wev, forced); + ngx_http_lua_pipe_close_helper(pipe->stdin_ctx, wev); } - - return NGX_OK; } -static ngx_int_t -ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe, int forced) +static void +ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe) { ngx_event_t *rev; @@ -973,15 +969,13 @@ ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe, int forced) } else if (pipe->stdout_ctx->c != NULL) { rev = pipe->stdout_ctx->c->read; - return ngx_http_lua_pipe_close_helper(pipe->stdout_ctx, rev, forced); + ngx_http_lua_pipe_close_helper(pipe->stdout_ctx, rev); } - - return NGX_OK; } -static ngx_int_t -ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe, int forced) +static void +ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe) { ngx_event_t *rev; @@ -997,10 +991,8 @@ ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe, int forced) } else if (pipe->stderr_ctx->c != NULL) { rev = pipe->stderr_ctx->c->read; - return ngx_http_lua_pipe_close_helper(pipe->stderr_ctx, rev, forced); + ngx_http_lua_pipe_close_helper(pipe->stderr_ctx, rev); } - - return NGX_OK; } @@ -1008,7 +1000,6 @@ int ngx_http_lua_ffi_pipe_proc_shutdown_stdin(ngx_http_lua_ffi_pipe_proc_t *proc, u_char *errbuf, size_t *errbuf_size) { - ngx_int_t rc; ngx_http_lua_pipe_t *pipe; pipe = proc->pipe; @@ -1017,12 +1008,7 @@ ngx_http_lua_ffi_pipe_proc_shutdown_stdin(ngx_http_lua_ffi_pipe_proc_t *proc, return NGX_ERROR; } - rc = ngx_http_lua_pipe_close_stdin(pipe, 0); - if (rc != NGX_OK) { - *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy writing") - - errbuf; - return NGX_ERROR; - } + ngx_http_lua_pipe_close_stdin(pipe); return NGX_OK; } @@ -1032,7 +1018,6 @@ int ngx_http_lua_ffi_pipe_proc_shutdown_stdout(ngx_http_lua_ffi_pipe_proc_t *proc, u_char *errbuf, size_t *errbuf_size) { - ngx_int_t rc; ngx_http_lua_pipe_t *pipe; pipe = proc->pipe; @@ -1041,12 +1026,7 @@ ngx_http_lua_ffi_pipe_proc_shutdown_stdout(ngx_http_lua_ffi_pipe_proc_t *proc, return NGX_ERROR; } - rc = ngx_http_lua_pipe_close_stdout(pipe, 0); - if (rc != NGX_OK) { - *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy reading") - - errbuf; - return NGX_ERROR; - } + ngx_http_lua_pipe_close_stdout(pipe); return NGX_OK; } @@ -1071,24 +1051,20 @@ ngx_http_lua_ffi_pipe_proc_shutdown_stderr(ngx_http_lua_ffi_pipe_proc_t *proc, return NGX_ERROR; } - if (ngx_http_lua_pipe_close_stderr(pipe, 0) != NGX_OK) { - *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "pipe busy reading") - - errbuf; - return NGX_ERROR; - } + ngx_http_lua_pipe_close_stderr(pipe); return NGX_OK; } static void -ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc, int forced) +ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc) { ngx_http_lua_pipe_t *pipe; - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua pipe finalize process:%p pid:%P forced:%d", proc, - proc->_pid, forced); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe finalize process:%p pid:%P", + proc, proc->_pid); pipe = proc->pipe; if (pipe->node) { @@ -1098,11 +1074,11 @@ ngx_http_lua_pipe_proc_finalize(ngx_http_lua_ffi_pipe_proc_t *proc, int forced) pipe->dead = 1; - ngx_http_lua_pipe_close_stdin(pipe, forced); - ngx_http_lua_pipe_close_stdout(pipe, forced); + ngx_http_lua_pipe_close_stdin(pipe); + ngx_http_lua_pipe_close_stdout(pipe); if (!pipe->merge_stderr) { - ngx_http_lua_pipe_close_stderr(pipe, forced); + ngx_http_lua_pipe_close_stderr(pipe); } pipe->closed = 1; @@ -1135,7 +1111,7 @@ ngx_http_lua_ffi_pipe_proc_destroy(ngx_http_lua_ffi_pipe_proc_t *proc) } } - ngx_http_lua_pipe_proc_finalize(proc, 1); + ngx_http_lua_pipe_proc_finalize(proc); ngx_destroy_pool(pipe->pool); proc->pipe = NULL; } @@ -1205,6 +1181,10 @@ ngx_http_lua_pipe_put_error(ngx_http_lua_pipe_ctx_t *pipe_ctx, u_char *errbuf, - errbuf; break; + case PIPE_ERR_ABORTED: + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "aborted") - errbuf; + break; + default: ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, "unexpected err type: %d", pipe_ctx->err_type); @@ -2005,7 +1985,7 @@ ngx_http_lua_ffi_pipe_proc_wait(ngx_http_request_t *r, *reason = REASON_UNKNOWN; } - ngx_http_lua_pipe_proc_finalize(proc, 0); + ngx_http_lua_pipe_proc_finalize(proc); if (*status == 0) { return NGX_OK; @@ -2096,6 +2076,12 @@ ngx_http_lua_pipe_read_retval_helper(ngx_http_lua_ffi_pipe_proc_t *proc, return 0; } + if (pipe_ctx->err_type == PIPE_ERR_ABORTED) { + ngx_close_connection(pipe_ctx->c); + pipe_ctx->c = NULL; + return 0; + } + rc = ngx_http_lua_pipe_read(pipe, pipe_ctx); if (rc != NGX_AGAIN) { return 0; @@ -2147,6 +2133,12 @@ ngx_http_lua_pipe_write_retval(ngx_http_lua_ffi_pipe_proc_t *proc, return 0; } + if (pipe_ctx->err_type == PIPE_ERR_ABORTED) { + ngx_close_connection(pipe_ctx->c); + pipe_ctx->c = NULL; + return 0; + } + rc = ngx_http_lua_pipe_write(pipe, pipe_ctx); if (rc != NGX_AGAIN) { return 0; @@ -2192,7 +2184,7 @@ ngx_http_lua_pipe_wait_retval(ngx_http_lua_ffi_pipe_proc_t *proc, lua_State *L) return 2; } - ngx_http_lua_pipe_proc_finalize(pipe_node->proc, 0); + ngx_http_lua_pipe_proc_finalize(pipe_node->proc); if (pipe_node->status == 0) { lua_pushboolean(L, 1); From 189ba24ef4ebdeced25fe2994fbf4b03113225fb Mon Sep 17 00:00:00 2001 From: spacewander Date: Fri, 31 May 2019 19:05:56 +0800 Subject: [PATCH 354/848] feature: when a timer fails to run we now log the location of its function's definition. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_timer.c | 61 +++++++++++------- t/106-timer.t | 133 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 169 insertions(+), 25 deletions(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index ad88b9512f..99a461da89 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -16,6 +16,9 @@ #include "ngx_http_lua_probe.h" +#define NGX_HTTP_LUA_TIMER_ERRBUF_SIZE 128 + + typedef struct { void **main_conf; void **srv_conf; @@ -559,12 +562,20 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) ngx_http_lua_main_conf_t *lmcf; ngx_http_core_loc_conf_t *clcf; + lua_Debug ar; + u_char *p; + u_char errbuf[NGX_HTTP_LUA_TIMER_ERRBUF_SIZE]; + const char *source; + const char *errmsg; + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua ngx.timer expired"); ngx_memcpy(&tctx, ev->data, sizeof(ngx_http_lua_timer_ctx_t)); ngx_free(ev); + ngx_http_lua_assert(tctx.co_ref && tctx.co); + lmcf = tctx.lmcf; lmcf->pending_timers--; @@ -579,17 +590,17 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) } if (lmcf->running_timers >= lmcf->max_running_timers) { - ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, - "%i lua_max_running_timers are not enough", - lmcf->max_running_timers); + p = ngx_snprintf(errbuf, NGX_HTTP_LUA_TIMER_ERRBUF_SIZE - 1, + "%i lua_max_running_timers are not enough", + lmcf->max_running_timers); + *p = '\0'; + errmsg = (const char *) errbuf; goto failed; } c = ngx_http_lua_create_fake_connection(tctx.pool); if (c == NULL) { - ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, - "failed to create fake connection to run timer (co: %p)", - tctx.co); + errmsg = "could not create fake connection"; goto failed; } @@ -601,9 +612,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) r = ngx_http_lua_create_fake_request(c); if (r == NULL) { - ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, - "failed to create fake request to run timer (co: %p)", - tctx.co); + errmsg = "could not create fake request"; goto failed; } @@ -624,8 +633,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) ctx = ngx_http_lua_create_ctx(r); if (ctx == NULL) { - ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, - "failed to create ctx to run timer (co: %p)", tctx.co); + errmsg = "could not create ctx"; goto failed; } @@ -634,9 +642,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) pcln = ngx_pool_cleanup_add(r->pool, 0); if (pcln == NULL) { - ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, - "failed to add vm cleanup to run timer (co: %p)", - tctx.co); + errmsg = "could not add vm cleanup"; goto failed; } @@ -650,9 +656,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) cln = ngx_http_cleanup_add(r, 0); if (cln == NULL) { - ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, - "failed to add request cleanup to run timer (co: %p)", - tctx.co); + errmsg = "could not add request cleanup"; goto failed; } @@ -709,14 +713,25 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) failed: - if (tctx.co_ref && tctx.co) { - lua_pushlightuserdata(tctx.co, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(tctx.co, LUA_REGISTRYINDEX); - luaL_unref(tctx.co, -1, tctx.co_ref); - lua_settop(tctx.co, 0); + /* co stack: func [args] */ + lua_getinfo(tctx.co, ">Sf", &ar); + + source = ar.source; + + if (source == NULL) { + source = "(unknown)"; } + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "lua failed to run timer with function defined at %s:%d: %s", + source, ar.linedefined, errmsg); + + lua_pushlightuserdata(tctx.co, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(tctx.co, LUA_REGISTRYINDEX); + luaL_unref(tctx.co, -1, tctx.co_ref); + lua_settop(tctx.co, 0); + if (tctx.vm_state) { ngx_http_lua_cleanup_vm(tctx.vm_state); } diff --git a/t/106-timer.t b/t/106-timer.t index 7b7c85ec72..485886495f 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -12,7 +12,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 8 + 72); +plan tests => repeat_each() * (blocks() * 8 + 63); #no_diff(); no_long_string(); @@ -1932,7 +1932,7 @@ registered timer --- error_log eval [ -qr/\[alert\] .*? 1 lua_max_running_timers are not enough/, +qr/\[alert\] .*? lua failed to run timer with function defined at =content_by_lua\(nginx.conf:\d+\):10: 1 lua_max_running_timers are not enough/, "lua ngx.timer expired", "http lua close fake http connection", ] @@ -2193,3 +2193,132 @@ ok --- error_log Bad bad bad --- skip_nginx: 4: < 1.7.1 + + + +=== TEST 33: log function location when failed to run a timer +--- http_config + lua_max_running_timers 1; +--- config + location /t { + content_by_lua_block { + local function g() + ngx.sleep(0.01) + end + + local function f() + ngx.sleep(0.01) + end + + local ok, err = ngx.timer.at(0, f) + if not ok then + ngx.say("failed to create timer f: ", err) + return + end + + local ok, err = ngx.timer.at(0, g) + if not ok then + ngx.say("failed to create timer g: ", err) + return + end + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- wait: 0.1 +--- error_log eval +qr/\[alert\] .*? lua failed to run timer with function defined at =content_by_lua\(nginx.conf:\d+\):2: 1 lua_max_running_timers are not enough/ +--- no_error_log +[crit] +[error] + + + +=== TEST 34: log function location when failed to run a timer (anonymous function) +--- http_config + lua_max_running_timers 1; +--- config + location /t { + content_by_lua_block { + local function f() + ngx.sleep(0.01) + end + + local ok, err = ngx.timer.at(0, f) + if not ok then + ngx.say("failed to set timer f: ", err) + return + end + + local ok, err = ngx.timer.at(0, function() + ngx.sleep(0.01) + end) + + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- wait: 0.1 +--- error_log eval +qr/\[alert\] .*? lua failed to run timer with function defined at =content_by_lua\(nginx.conf:\d+\):12: 1 lua_max_running_timers are not enough/ +--- no_error_log +[crit] +[error] + + + +=== TEST 35: log function location when failed to run a timer (lua file) +--- user_files +>>> test.lua +local _M = {} + +function _M.run() + ngx.sleep(0.01) +end + +return _M +--- http_config + lua_package_path '$TEST_NGINX_HTML_DIR/?.lua;./?.lua;;'; + lua_max_running_timers 1; +--- config + location /t { + content_by_lua_block { + local test = require "test" + + local ok, err = ngx.timer.at(0, test.run) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + local ok, err = ngx.timer.at(0, test.run) + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- wait: 0.1 +--- no_error_log +[crit] +[error] +--- error_log eval +qr/\[alert\] .*? lua failed to run timer with function defined at @.+\/test.lua:3: 1 lua_max_running_timers are not enough/ From 97431d814114b5f94c5076557fbab3e0624b900f Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 9 Aug 2019 13:06:30 -0700 Subject: [PATCH 355/848] bugfix: fixed compilation with NGINX cores < 1.11.11 A follow-up commit to 947fa00 in which removed the `NGX_LUA_NO_FFI_API` macro. Doing so prevents the `cln` variable from being declared, which it previously was before even with such older NGINX core versions. --- src/ngx_http_lua_module.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index ef99b0dc42..6603b1965e 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -646,9 +646,7 @@ ngx_http_lua_init(ngx_conf_t *cf) volatile ngx_cycle_t *saved_cycle; ngx_http_core_main_conf_t *cmcf; ngx_http_lua_main_conf_t *lmcf; -#if defined(nginx_version) && (nginx_version >= 1011011) ngx_pool_cleanup_t *cln; -#endif ngx_str_t name = ngx_string("host"); if (ngx_process == NGX_PROCESS_SIGNALLER || ngx_test_config) { From 97f0101e525b8f0c0f125ab50e381c936f0c2f10 Mon Sep 17 00:00:00 2001 From: Tom Thorogood Date: Mon, 12 Aug 2019 16:12:20 -0700 Subject: [PATCH 356/848] feature: enabled the 'ngx.thread' and 'ngx.socket.udp' APIs in ssl_session_fetch_by_lua*. Signed-off-by: Thibault Charbonnier --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- src/ngx_http_lua_socket_udp.c | 6 +- src/ngx_http_lua_uthread.c | 6 +- t/143-ssl-session-fetch.t | 293 ++++++++++++++++++++++++++++++++++ 5 files changed, 303 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index 7902550d5e..f085848da2 100644 --- a/README.markdown +++ b/README.markdown @@ -8045,7 +8045,7 @@ ngx.thread.kill **syntax:** *ok, err = ngx.thread.kill(thread)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** Kills a running "light thread" created by [ngx.thread.spawn](#ngxthreadspawn). Returns a true value when successful or `nil` and a string describing the error otherwise. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 1993b05a5d..e158c1cc27 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -6869,7 +6869,7 @@ This API was first enabled in the v0.7.0 release. '''syntax:''' ''ok, err = ngx.thread.kill(thread)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' Kills a running "light thread" created by [[#ngx.thread.spawn|ngx.thread.spawn]]. Returns a true value when successful or nil and a string describing the error otherwise. diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index f35e00e67b..0e55344ea8 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -143,7 +143,8 @@ ngx_http_lua_socket_udp(lua_State *L) | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT); + | NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); lua_createtable(L, 3 /* narr */, 1 /* nrec */); lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( @@ -205,7 +206,8 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT); + | NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); luaL_checktype(L, 1, LUA_TTABLE); diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index 43517a049d..2e0c3d8625 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -128,7 +128,8 @@ ngx_http_lua_uthread_wait(lua_State *L) | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT); + | NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); coctx = ctx->cur_co_ctx; @@ -226,7 +227,8 @@ ngx_http_lua_uthread_kill(lua_State *L) | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT); + | NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); coctx = ctx->cur_co_ctx; diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index df9972cb46..183e6faa68 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -1429,3 +1429,296 @@ qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, [error] [alert] [emerg] + + + +=== TEST 18: cosocket (UDP) +--- http_config + ssl_session_fetch_by_lua_block { + local sock = ngx.socket.udp() + + sock:settimeout(1000) + + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = sock:send(req) + if not ok then + ngx.log(ngx.ERR, "failed to send flush_all to memc: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + ngx.log(ngx.INFO, "received memc reply of ", #res, " bytes") + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + package.loaded.session = sess + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- grep_error_log eval: qr/received memc reply of \d+ bytes/ +--- grep_error_log_out eval +[ +'', +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +] +--- no_error_log +[alert] +[error] +[emerg] + + + +=== TEST 19: uthread (kill) +--- http_config + ssl_session_fetch_by_lua_block { + local function f() + ngx.log(ngx.INFO, "uthread: hello from f()") + ngx.sleep(1) + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return + end + + collectgarbage() + + local ok, err = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.ERR, "failed to kill thread: ", err) + return + end + + ngx.log(ngx.INFO, "uthread: killed") + + local ok, err = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.INFO, "uthread: failed to kill: ", err) + end + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + package.loaded.session = sess + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out eval +[ +'', +'uthread: hello from f() +uthread: killed +uthread: failed to kill: already waited or killed +', +'uthread: hello from f() +uthread: killed +uthread: failed to kill: already waited or killed +' +] +--- no_error_log +[alert] +[error] +[emerg] + + + +=== TEST 20: uthread (wait) +--- http_config + ssl_session_fetch_by_lua_block { + local function f() + ngx.log(ngx.INFO, "uthread: hello from f()") + ngx.sleep(0.001) + return 32 + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return + end + + collectgarbage() + + local ok, res = ngx.thread.wait(t) + if not ok then + ngx.log(ngx.ERR, "failed to wait on thread: ", res) + return + end + + ngx.log(ngx.INFO, "uthread: ", res) + + local ok, err = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.INFO, "uthread: failed to kill: ", err) + end + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_session_tickets off; + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(5000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + package.loaded.session = sess + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +close: 1 nil +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out eval +[ +'', +'uthread: hello from f() +uthread: 32 +uthread: failed to kill: already waited or killed +', +'uthread: hello from f() +uthread: 32 +uthread: failed to kill: already waited or killed +' +] +--- no_error_log +[alert] +[error] +[emerg] From c658e22606c504b96e6408554cac8c1fefcfd0ea Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 12 Aug 2019 16:13:15 -0700 Subject: [PATCH 357/848] tests: 139-ssl-cert.t: added test cases asserting support for 'ngx.thread.kill()' and 'ngx.socket.udp' in 'ssl_certificate_by_lua*'. --- t/139-ssl-cert-by.t | 174 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 174 insertions(+) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 8b5517c494..2344b63b1e 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -2142,3 +2142,177 @@ qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, [error] [alert] [emerg] + + + +=== TEST 25: cosocket (UDP) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_tokens off; + + ssl_certificate_by_lua_block { + local sock = ngx.socket.udp() + + sock:settimeout(1000) + + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = sock:send(req) + if not ok then + ngx.log(ngx.ERR, "failed to send flush_all to memc: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + ngx.log(ngx.INFO, "received memc reply of ", #res, " bytes") + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +--- no_error_log +[error] +[alert] +[emerg] +--- grep_error_log eval: qr/received memc reply of \d+ bytes/ +--- grep_error_log_out eval +[ +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +] + + + +=== TEST 26: uthread (kill) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_tokens off; + + ssl_certificate_by_lua_block { + local function f() + ngx.log(ngx.INFO, "uthread: hello from f()") + ngx.sleep(1) + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return ngx.exit(ngx.ERROR) + end + + local ok, res = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.ERR, "failed to kill thread: ", res) + return + end + + ngx.log(ngx.INFO, "uthread: killed") + + local ok, err = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.INFO, "uthread: failed to kill: ", err) + end + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +--- no_error_log +[error] +[alert] +[emerg] +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out +uthread: hello from f() +uthread: killed +uthread: failed to kill: already waited or killed From 9a892b8d5dc1fe3eb21882b318a1e3260330ad5c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 17 Jan 2018 22:20:44 -0800 Subject: [PATCH 358/848] bugfix: coroutine.wrap: propagate errors to the parent coroutine. --- src/ngx_http_lua_common.h | 7 + src/ngx_http_lua_coroutine.c | 64 ++++-- src/ngx_http_lua_util.c | 75 ++++--- t/062-count.t | 2 +- t/091-coroutine.t | 396 ++++++++++++++++++++++++++++++++++- 5 files changed, 503 insertions(+), 41 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 455581d742..5d998d8d2f 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -482,6 +482,13 @@ struct ngx_http_lua_co_ctx_s { the ngx.thread.spawn() call */ unsigned sem_resume_status:1; + + unsigned is_wrap:1; /* set when creating coroutines via + coroutine.wrap */ + + unsigned propagate_error:1; /* set when propagating an error + from a coroutine to its + parent */ }; diff --git a/src/ngx_http_lua_coroutine.c b/src/ngx_http_lua_coroutine.c index 99a24235a3..5c469c0fa9 100644 --- a/src/ngx_http_lua_coroutine.c +++ b/src/ngx_http_lua_coroutine.c @@ -25,6 +25,7 @@ static int ngx_http_lua_coroutine_create(lua_State *L); +static int ngx_http_lua_coroutine_wrap(lua_State *L); static int ngx_http_lua_coroutine_resume(lua_State *L); static int ngx_http_lua_coroutine_yield(lua_State *L); static int ngx_http_lua_coroutine_status(lua_State *L); @@ -62,6 +63,45 @@ ngx_http_lua_coroutine_create(lua_State *L) } +static int +ngx_http_lua_coroutine_wrap_runner(lua_State *L) +{ + /* retrieve closure and insert it at the bottom of + * the stack for coroutine.resume() */ + lua_pushvalue(L, lua_upvalueindex(1)); + lua_insert(L, 1); + + return ngx_http_lua_coroutine_resume(L); +} + + +static int +ngx_http_lua_coroutine_wrap(lua_State *L) +{ + ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_co_ctx_t *coctx = NULL; + + r = ngx_http_lua_get_req(L); + if (r == NULL) { + return luaL_error(L, "no request found"); + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return luaL_error(L, "no request ctx found"); + } + + ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx); + + coctx->is_wrap = 1; + + lua_pushcclosure(L, ngx_http_lua_coroutine_wrap_runner, 1); + + return 1; +} + + int ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t **pcoctx) @@ -250,7 +290,7 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) int rc; /* new coroutine table */ - lua_createtable(L, 0 /* narr */, 14 /* nrec */); + lua_createtable(L, 0 /* narr */, 16 /* nrec */); /* get old coroutine table */ lua_getglobal(L, "coroutine"); @@ -262,6 +302,9 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) lua_getfield(L, -1, "create"); lua_setfield(L, -3, "_create"); + lua_getfield(L, -1, "wrap"); + lua_setfield(L, -3, "_wrap"); + lua_getfield(L, -1, "resume"); lua_setfield(L, -3, "_resume"); @@ -277,6 +320,9 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) lua_pushcfunction(L, ngx_http_lua_coroutine_create); lua_setfield(L, -2, "__create"); + lua_pushcfunction(L, ngx_http_lua_coroutine_wrap); + lua_setfield(L, -2, "__wrap"); + lua_pushcfunction(L, ngx_http_lua_coroutine_resume); lua_setfield(L, -2, "__resume"); @@ -291,7 +337,7 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) /* inject coroutine APIs */ { const char buf[] = - "local keys = {'create', 'yield', 'resume', 'status'}\n" + "local keys = {'create', 'yield', 'resume', 'status', 'wrap'}\n" #ifdef OPENRESTY_LUAJIT "local get_req = require 'thread.exdata'\n" #else @@ -321,24 +367,18 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) "return std(...)\n" "end\n" "end\n" - "local create, resume = coroutine.create, coroutine.resume\n" - "coroutine.wrap = function(f)\n" - "local co = create(f)\n" - "return function(...) return select(2, resume(co, ...)) end\n" - "end\n" - "package.loaded.coroutine = coroutine"; - + "package.loaded.coroutine = coroutine" #if 0 "debug.sethook(function () collectgarbage() end, 'rl', 1)" #endif ; - rc = luaL_loadbuffer(L, buf, sizeof(buf) - 1, "=coroutine.wrap"); + rc = luaL_loadbuffer(L, buf, sizeof(buf) - 1, "=coroutine_api"); } if (rc != 0) { ngx_log_error(NGX_LOG_ERR, log, 0, - "failed to load Lua code for coroutine.wrap(): %i: %s", + "failed to load Lua code for coroutine_api: %i: %s", rc, lua_tostring(L, -1)); lua_pop(L, 1); @@ -348,7 +388,7 @@ ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L) rc = lua_pcall(L, 0, 0, 0); if (rc != 0) { ngx_log_error(NGX_LOG_ERR, log, 0, - "failed to run the Lua code for coroutine.wrap(): %i: %s", + "failed to run the Lua code for coroutine_api: %i: %s", rc, lua_tostring(L, -1)); lua_pop(L, 1); } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 5f18c89465..c7ed643a0c 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1018,8 +1018,6 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, /* set Lua VM panic handler */ lua_atpanic(L, ngx_http_lua_atpanic); - dd("ctx = %p", ctx); - NGX_LUA_EXCEPTION_TRY { if (ctx->cur_co_ctx->thread_spawn_yielded) { @@ -1031,19 +1029,15 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, for ( ;; ) { - dd("calling lua_resume: vm %p, nret %d", ctx->cur_co_ctx->co, - (int) nrets); + dd("ctx: %p, co: %p, co status: %d, co is_wrap: %d", + ctx, ctx->cur_co_ctx->co, ctx->cur_co_ctx->co_status, + ctx->cur_co_ctx->is_wrap); #if (NGX_PCRE) /* XXX: work-around to nginx regex subsystem */ old_pool = ngx_http_lua_pcre_malloc_init(r->pool); #endif - /* run code */ - dd("ctx: %p", ctx); - dd("cur co: %p", ctx->cur_co_ctx->co); - dd("cur co status: %d", ctx->cur_co_ctx->co_status); - orig_coctx = ctx->cur_co_ctx; #ifdef NGX_LUA_USE_ASSERT @@ -1055,10 +1049,19 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, #if DDEBUG if (lua_gettop(orig_coctx->co) > 0) { - dd("top elem: %s", luaL_typename(orig_coctx->co, -1)); + dd("co top elem: %s", luaL_typename(orig_coctx->co, -1)); + } + + if (orig_coctx->propagate_error) { + dd("co propagate_error: %d", orig_coctx->propagate_error); } #endif + if (orig_coctx->propagate_error) { + orig_coctx->propagate_error = 0; + goto propagate_error; + } + ngx_http_lua_assert(orig_coctx->co_top + nrets == lua_gettop(orig_coctx->co)); @@ -1203,12 +1206,6 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, next_coctx = ctx->cur_co_ctx->parent_co_ctx; next_co = next_coctx->co; - /* - * prepare return values for coroutine.resume - * (true plus any retvals) - */ - lua_pushboolean(next_co, 1); - if (nrets) { dd("moving %d return values to next co", nrets); lua_xmove(ctx->cur_co_ctx->co, next_co, nrets); @@ -1217,7 +1214,15 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, #endif } - nrets++; /* add the true boolean value */ + if (!ctx->cur_co_ctx->is_wrap) { + /* + * prepare return values for coroutine.resume + * (true plus any retvals) + */ + lua_pushboolean(next_co, 1); + lua_insert(next_co, 1); + nrets++; /* add the true boolean value */ + } ctx->cur_co_ctx = next_coctx; @@ -1328,12 +1333,6 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, next_co = next_coctx->co; - /* - * ended successful, coroutine.resume returns true plus - * any return values - */ - lua_pushboolean(next_co, success); - if (nrets) { lua_xmove(ctx->cur_co_ctx->co, next_co, nrets); } @@ -1343,7 +1342,16 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, ctx->uthreads--; } - nrets++; + if (!ctx->cur_co_ctx->is_wrap) { + /* + * ended successful, coroutine.resume returns true plus + * any return values + */ + lua_pushboolean(next_co, success); + lua_insert(next_co, 1); + nrets++; + } + ctx->cur_co_ctx = next_coctx; ngx_http_lua_probe_info("set parent running"); @@ -1399,6 +1407,10 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, ctx->cur_co_ctx); trace = lua_tostring(L, -1); +propagate_error: + + ngx_http_lua_assert(err != NULL && msg != NULL && trace != NULL); + if (ctx->cur_co_ctx->is_uthread) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, @@ -1488,16 +1500,25 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, next_coctx->co_status = NGX_HTTP_LUA_CO_RUNNING; + ctx->cur_co_ctx = next_coctx; + + if (orig_coctx->is_wrap) { + /* + * coroutine.wrap propagates errors + * to the parent + */ + next_coctx->propagate_error = 1; + continue; + } + /* * ended with error, coroutine.resume returns false plus * err msg */ lua_pushboolean(next_co, 0); - lua_xmove(ctx->cur_co_ctx->co, next_co, 1); + lua_xmove(orig_coctx->co, next_co, 1); nrets = 2; - ctx->cur_co_ctx = next_coctx; - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "lua coroutine: %s: %s\n%s", err, msg, trace); diff --git a/t/062-count.t b/t/062-count.t index b481763abf..ffb769a50d 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -391,7 +391,7 @@ probe process("$LIBLUA_PATH").function("rehashtab") { --- stap_out2 3 --- response_body -coroutine: 14 +coroutine: 16 --- no_error_log [error] diff --git a/t/091-coroutine.t b/t/091-coroutine.t index 7cf60f9675..c9ee72c7c4 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 5); +plan tests => repeat_each() * (blocks() * 3 + 14); $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; @@ -1318,3 +1318,397 @@ co yield: 2 --- no_error_log [error] + + + +=== TEST 32: coroutine.wrap propagates errors to parent coroutine +--- config + location = /t { + content_by_lua_block { + local co = coroutine.wrap(function() + print("in wrapped coroutine") + error("something went wrong") + end) + + co() + + ngx.say("ok") + } + } +--- request +GET /t +--- error_code: 500 +--- response_body_unlike +ok +--- error_log eval +[ + qr/\[notice\] .*? in wrapped coroutine/, + qr/\[error\] .*? lua entry thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):\d+: something went wrong/, + "stack traceback:", + "coroutine 0:", + "coroutine 1:" +] + + + +=== TEST 33: coroutine.wrap propagates nested errors to parent coroutine +Note: in this case, both the error message and the traceback are constructed +from co1's stack level. +--- config + location = /t { + content_by_lua_block { + local co1 = coroutine.wrap(function() + error("something went wrong in co1") + end) + + local co2 = coroutine.wrap(function() + co1() + end) + + co2() + } + } +--- request +GET /t +--- error_code: 500 +--- ignore_response +--- error_log eval +[ + qr/\[error\] .*? lua entry thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):\d+: something went wrong in co1/, + "stack traceback:", + "coroutine 0:", + "coroutine 1:", + "coroutine 2:" +] + + + +=== TEST 34: coroutine.wrap propagates nested errors with stack level to parent coroutine +Note: in this case, the error message is constructed at the entry thread stack +level, and the traceback is constructed from co1's stack level. +--- config + location = /t { + content_by_lua_block { + local co1 = coroutine.wrap(function() + error("something went wrong in co1", 2) + end) + + local co2 = coroutine.wrap(function() + co1() + end) + + co2() + } + } +--- request +GET /t +--- error_code: 500 +--- error_log eval +[ + qr/\[error\] .*? lua entry thread aborted: runtime error: something went wrong in co1/, + "stack traceback:", + "coroutine 0:", + "coroutine 1:", + "coroutine 2:" +] + + + +=== TEST 35: coroutine.wrap runtime errors do not log errors +--- config + location = /t { + content_by_lua_block { + local co = coroutine.wrap(function() + print("in wrapped coroutine") + error("something went wrong") + end) + + co() + } + } +--- request +GET /t +--- error_code: 500 +--- ignore_response +--- no_error_log eval +[ + qr/\[error\] .*? lua coroutine: runtime error:/, + "[crit]", + "[emerg]", + "[warn]", +] + + + +=== TEST 36: coroutine.wrap does not return status boolean on yield +--- config + location = /t { + content_by_lua_block { + local co = coroutine.wrap(function() + coroutine.yield("ok", "err") + end) + + local ret1, ret2 = co() + + ngx.say(ret1, ", ", ret2) + } + } +--- request +GET /t +--- response_body +ok, err +--- no_error_log +[error] + + + +=== TEST 37: coroutine.wrap does not return status boolean on done +--- config + location = /t { + content_by_lua_block { + local co = coroutine.wrap(function() end) + + local ret1 = co() + + ngx.say(ret1) + } + } +--- request +GET /t +--- response_body +nil +--- no_error_log +[error]g + + + +=== TEST 38: coroutine.wrap does not return status boolean on error +--- SKIP: not supported +--- config + location = /t { + content_by_lua_block { + local co = coroutine.wrap(function() + error("something went wrong") + end) + + local ret1, ret2 = pcall(co) + + ngx.say(ret1, ", ", ret2) + } + } +--- request +GET /t +--- response_body +false, something went wrong +--- no_error_log +[error] + + + +=== TEST 39: coroutine.wrap creates different function refs +--- config + location = /t { + content_by_lua_block { + local f = function() end + local co = coroutine.wrap(f) + local co2 = coroutine.wrap(f) + + ngx.say("co == co2: ", co == co2) + } + } +--- request +GET /t +--- response_body +co == co2: false +--- no_error_log +[error] + + + +=== TEST 40: coroutine.wrap supports yielding and resuming +--- config + location = /t { + content_by_lua_block { + function f() + local cnt = 0 + for i = 1, 20 do + ngx.say("co yield: ", cnt) + coroutine.yield() + cnt = cnt + 1 + end + end + + local f = coroutine.wrap(f) + for i = 1, 3 do + ngx.say("co resume") + f() + end + } + } +--- request +GET /t +--- response_body +co resume +co yield: 0 +co resume +co yield: 1 +co resume +co yield: 2 +--- no_error_log +[error] + + + +=== TEST 41: coroutine.wrap return values +--- config + location = /t { + content_by_lua_block { + function f() + local cnt = 0 + for i = 1, 20 do + coroutine.yield(cnt, cnt + 1) + cnt = cnt + 1 + end + end + + local f = coroutine.wrap(f) + for i = 1, 3 do + ngx.say("co resume") + local ret1, ret2 = f() + ngx.say("co yield: ", ret1, ", ", ret2) + end + } + } +--- request +GET /t +--- response_body +co resume +co yield: 0, 1 +co resume +co yield: 1, 2 +co resume +co yield: 2, 3 +--- no_error_log +[error] + + + +=== TEST 42: coroutine.wrap arguments +--- config + location = /t { + content_by_lua_block { + function f(step) + local cnt = 0 + for i = 1, 20 do + ngx.say("co yield: ", cnt) + coroutine.yield() + cnt = cnt + step + end + end + + local f = coroutine.wrap(f) + for i = 1, 3 do + ngx.say("co resume") + f(i) + end + } + } +--- request +GET /t +--- response_body +co resume +co yield: 0 +co resume +co yield: 1 +co resume +co yield: 2 +--- no_error_log +[error] + + + +=== TEST 43: coroutine.wrap in header_filter_by_lua (orig coroutine.wrap) +--- config + location = /t { + return 200; + + header_filter_by_lua_block { + function f() + local cnt = 0 + for i = 1, 20 do + print("co yield: ", cnt) + coroutine.yield() + cnt = cnt + 1 + end + end + + local f = coroutine.wrap(f) + for i = 1, 3 do + print("co resume.") + f() + end + } + } +--- request +GET /t +--- ignore_response +--- grep_error_log eval: qr/co (?:yield: \d+|resume\.)/ +--- grep_error_log_out +co resume. +co yield: 0 +co resume. +co yield: 1 +co resume. +co yield: 2 +--- no_error_log +[error] + + + +=== TEST 44: coroutine.wrap in header_filter_by_lua propagates errors (orig coroutine.wrap) +--- config + location = /t { + return 200; + + header_filter_by_lua_block { + local co = coroutine.wrap(function() + print("in wrapped coroutine") + error("something went wrong") + end) + + local err = co() + + ngx.log(ngx.CRIT, "err: ", err) + } + } +--- request +GET /t +--- ignore_response +--- error_log eval +[ + qr/\[notice\] .*? in wrapped coroutine/, + qr/\[error\] .*? failed to run header_filter_by_lua\*: header_filter_by_lua:\d+: header_filter_by_lua:\d+: something went wrong/, + "stack traceback:", + "in function 'co'" +] + + + +=== TEST 45: coroutine.wrap in init_by_lua propagates errors (orig coroutine.wrap) +--- http_config + init_by_lua_block { + local co = coroutine.wrap(function() + print("in wrapped coroutine") + error("something went wrong") + end) + + local err = co() + + ngx.log(ngx.CRIT, "err: ", err) + } +--- config + +--- must_die +--- grep_error_log eval: qr/init_by_lua error: .*? something went wrong/ +--- grep_error_log_out +init_by_lua error: init_by_lua:7: init_by_lua:4: something went wrong From 2a190736a58674086c3a27bf71a7993383fffb55 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 8 Aug 2019 15:32:45 -0700 Subject: [PATCH 359/848] change: removed error logging on coroutine runtime errors. --- src/ngx_http_lua_util.c | 44 +++++++++++++++++++++++++-------------- t/091-coroutine.t | 46 ++++++++++++++++++++++++++++++++++------- 2 files changed, 66 insertions(+), 24 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c7ed643a0c..5cd5bac5d8 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1020,6 +1020,14 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, NGX_LUA_EXCEPTION_TRY { + /* + * silence a -Werror=clobbered warning with gcc 5.4 + * due to above setjmp + */ + err = NULL; + msg = NULL; + trace = NULL; + if (ctx->cur_co_ctx->thread_spawn_yielded) { ngx_http_lua_probe_info("thread spawn yielded"); @@ -1389,29 +1397,33 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, ctx->cur_co_ctx = orig_coctx; } - if (lua_isstring(ctx->cur_co_ctx->co, -1)) { - dd("user custom error msg"); - msg = lua_tostring(ctx->cur_co_ctx->co, -1); - - } else { - msg = "unknown reason"; - } - ngx_http_lua_cleanup_pending_operation(ctx->cur_co_ctx); ngx_http_lua_probe_coroutine_done(r, ctx->cur_co_ctx->co, 0); ctx->cur_co_ctx->co_status = NGX_HTTP_LUA_CO_DEAD; - ngx_http_lua_thread_traceback(L, ctx->cur_co_ctx->co, - ctx->cur_co_ctx); - trace = lua_tostring(L, -1); + if (orig_coctx->is_uthread + || orig_coctx->is_wrap + || ngx_http_lua_is_entry_thread(ctx)) + { + ngx_http_lua_thread_traceback(L, orig_coctx->co, orig_coctx); + trace = lua_tostring(L, -1); + + if (lua_isstring(orig_coctx->co, -1)) { + msg = lua_tostring(orig_coctx->co, -1); + dd("user custom error msg: %s", msg); -propagate_error: + } else { + msg = "unknown reason"; + } + } - ngx_http_lua_assert(err != NULL && msg != NULL && trace != NULL); +propagate_error: if (ctx->cur_co_ctx->is_uthread) { + ngx_http_lua_assert(err != NULL && msg != NULL + && trace != NULL); ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "lua user thread aborted: %s: %s\n%s", @@ -1462,6 +1474,9 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, } if (ngx_http_lua_is_entry_thread(ctx)) { + ngx_http_lua_assert(err != NULL && msg != NULL + && trace != NULL); + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "lua entry thread aborted: %s: %s\n%s", err, msg, trace); @@ -1519,9 +1534,6 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, lua_xmove(orig_coctx->co, next_co, 1); nrets = 2; - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "lua coroutine: %s: %s\n%s", err, msg, trace); - /* try resuming on the new coroutine again */ continue; } diff --git a/t/091-coroutine.t b/t/091-coroutine.t index c9ee72c7c4..5af1ea3a19 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -467,21 +467,25 @@ done === TEST 10: thread traceback (multi-thread) +Note: only coroutine.wrap propagates errors to the parent coroutine +(and thus produces a traceback) --- config location /lua { - content_by_lua ' + content_by_lua_block { local f = function(cr) coroutine.resume(cr) end -- emit a error local g = function() unknown.unknown = 1 end - local l1 = coroutine.create(f) - local l2 = coroutine.create(g) - coroutine.resume(l1, l2) + local l1 = coroutine.wrap(f) + local l2 = coroutine.wrap(g) + local l3 = coroutine.wrap(function() l1(l2) end) + l3() ngx.say("hello") - '; + } } --- request GET /lua ---- response_body +--- error_code: 500 +--- response_body_unlike hello --- error_log eval ["stack traceback:", "coroutine 0:", "coroutine 1:", "coroutine 2:"] @@ -904,8 +908,8 @@ qr/^child: resume: falsecontent_by_lua\(nginx\.conf:\d+\):4: bad child: status: dead parent: status: running $/s ---- error_log eval -qr/lua coroutine: runtime error: content_by_lua\(nginx\.conf:\d+\):4: bad/ +--- no_error_log +[error] @@ -1712,3 +1716,29 @@ GET /t --- grep_error_log eval: qr/init_by_lua error: .*? something went wrong/ --- grep_error_log_out init_by_lua error: init_by_lua:7: init_by_lua:4: something went wrong + + + +=== TEST 46: coroutine.resume runtime errors do not log errors +--- config + location = /t { + content_by_lua_block { + local function f() + error("something went wrong") + end + + local ret1, ret2 = coroutine.resume(coroutine.create(f)) + ngx.say(ret1) + ngx.say(ret2) + } + } +--- request +GET /t +--- response_body_like +false +content_by_lua\(nginx.conf:\d+\):\d+: something went wrong +--- no_error_log eval +[ + qr/\[error\] .*? lua coroutine: runtime error:", + "stack traceback:", +] From 315330ac099a35f5f17a0d453b7c001fe1181148 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 26 Aug 2019 17:19:02 -0700 Subject: [PATCH 360/848] bugfix: config: ensured LuaJIT FFI check and static PCRE builds succeed on all platforms. - fix the LuaJIT FFI check (tested on Linux and Darwin) - fix static libpcre builds on Darwin - ensure previous config scripts do not make the LuaJIT check fail --- config | 95 ++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 52 insertions(+), 43 deletions(-) diff --git a/config b/config index e40a76300c..6431fa38fc 100644 --- a/config +++ b/config @@ -1,5 +1,11 @@ ngx_lua_opt_I= ngx_lua_opt_L= +luajit_ld_opt= + +ngx_feature_name= +ngx_feature_run=no +ngx_feature_incs= +ngx_feature_test= if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then # explicitly set LuaJIT paths @@ -30,6 +36,7 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then ngx_feature_path="$LUAJIT_INC" ngx_lua_opt_I="-I$LUAJIT_INC" ngx_lua_opt_L="-L$LUAJIT_LIB" + luajit_ld_opt="-lm -ldl" # ensure that -I$LUAJIT_INC and -L$LUAJIT_LIB come first SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" @@ -38,9 +45,9 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then NGX_TEST_LD_OPT="$ngx_lua_opt_L $NGX_TEST_LD_OPT" if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R$LUAJIT_LIB $ngx_lua_opt_L -lluajit-5.1 -lm -ldl" + ngx_feature_libs="-R$LUAJIT_LIB $ngx_lua_opt_L -lluajit-5.1 $luajit_ld_opt" else - ngx_feature_libs="$ngx_lua_opt_L -lluajit-5.1 -lm -ldl" + ngx_feature_libs="$ngx_lua_opt_L -lluajit-5.1 $luajit_ld_opt" fi . auto/feature @@ -55,6 +62,7 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then ngx_feature_path="$LUAJIT_INC" ngx_lua_opt_I="-I$LUAJIT_INC" ngx_lua_opt_L="-L$LUAJIT_LIB" + luajit_ld_opt="-lm" # ensure that -I$LUAJIT_INC and -L$LUAJIT_LIB come first SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" @@ -63,9 +71,9 @@ if [ -n "$LUAJIT_INC" -o -n "$LUAJIT_LIB" ]; then NGX_TEST_LD_OPT="$ngx_lua_opt_L $NGX_TEST_LD_OPT" if [ $NGX_RPATH = YES ]; then - ngx_feature_libs="-R$LUAJIT_LIB $ngx_lua_opt_L -lluajit-5.1 -lm" + ngx_feature_libs="-R$LUAJIT_LIB $ngx_lua_opt_L -lluajit-5.1 $luajit_ld_opt" else - ngx_feature_libs="$ngx_lua_opt_L -lluajit-5.1 -lm" + ngx_feature_libs="$ngx_lua_opt_L -lluajit-5.1 $luajit_ld_opt" fi . auto/feature @@ -88,6 +96,7 @@ END case "$NGX_MACHINE" in amd64 | x86_64 | i386) echo "adding extra linking options needed by LuaJIT on $NGX_MACHINE" + luajit_ld_opt="$luajit_ld_opt -pagezero_size 10000 -image_base 100000000" ngx_feature_libs="$ngx_feature_libs -pagezero_size 10000 -image_base 100000000" ;; @@ -105,6 +114,8 @@ else # FreeBSD with luajit-2.0 from ports collection ngx_feature="LuaJIT library in /usr/local/" ngx_feature_path="/usr/local/include/luajit-2.0" + luajit_ld_opt="-lm" + LUAJIT_LIB="/usr/local/lib" if [ $NGX_RPATH = YES ]; then ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -lluajit-5.1 -lm" else @@ -117,6 +128,8 @@ else # Gentoo with LuaJIT-2.0, try with -ldl ngx_feature="LuaJIT library in /usr/" ngx_feature_path="/usr/include/luajit-2.0" + luajit_ld_opt="-lm -ldl" + LUAJIT_LIB="/usr/lib" if [ $NGX_RPATH = YES ]; then ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -lluajit-5.1 -ldl" else @@ -129,6 +142,8 @@ else # Gentoo with LuaJIT 2.0 ngx_feature="LuaJIT library in /usr/" ngx_feature_path="/usr/include/luajit-2.0" + luajit_ld_opt="-lm" + LUAJIT_LIB="/usr/lib" if [ $NGX_RPATH = YES ]; then ngx_feature_libs="-R/usr/lib -L/usr/lib -lm -lluajit-5.1" else @@ -159,15 +174,11 @@ fi # ---------------------------------------- ngx_feature="LuaJIT 2.x" -ngx_feature_name= ngx_feature_run=no ngx_feature_incs="#include " -ngx_feature_path= ngx_feature_test='#if !defined(LUAJIT_VERSION_NUM) || (LUAJIT_VERSION_NUM < 20000) # error unsupported LuaJIT version #endif' -SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" -CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" . auto/feature @@ -178,22 +189,17 @@ END exit 1 fi -CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" - # ---------------------------------------- ngx_feature="LuaJIT has FFI" -ngx_feature_name= +ngx_feature_libs="$LUAJIT_LIB/libluajit-5.1.a $luajit_ld_opt" ngx_feature_run=yes ngx_feature_incs="#include #include #include " -ngx_feature_path= -ngx_feature_test='lua_State *L = luaL_newstate(); +ngx_feature_test="lua_State *L = luaL_newstate(); assert(L != NULL); - luaopen_ffi(L);' -SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" -CC_TEST_FLAGS="$ngx_lua_opt_I $CC_TEST_FLAGS" + luaopen_ffi(L);" . auto/feature @@ -204,8 +210,6 @@ END exit 1 fi -CC_TEST_FLAGS="$SAVED_CC_TEST_FLAGS" - # ---------------------------------------- ngx_addon_name=ngx_http_lua_module @@ -468,36 +472,41 @@ fi # ---------------------------------------- -if [ $PCRE != NO -a $PCRE != YES ]; then - # force pcre_version symbol to be undefined when PCRE is statically linked - ngx_feature="force undefined symbols (--undefined)" - ngx_feature_libs="-Wl,--undefined=printf" - ngx_feature_name= - ngx_feature_run=no - ngx_feature_incs="#include " - ngx_feature_path= - ngx_feature_test='printf("hello");' +if [ $USE_PCRE = YES -o $PCRE != NONE ] && [ $PCRE != NO -a $PCRE != YES ]; then + # force pcre_version symbol to be required when PCRE is statically linked + case "$NGX_PLATFORM" in + Darwin:*) + ngx_feature="require defined symbols (-u)" + ngx_feature_name= + ngx_feature_path= + ngx_feature_libs="-Wl,-u,_strerror" + ngx_feature_run=no + ngx_feature_incs="#include " + ngx_feature_test='printf("hello");' - . auto/feature + . auto/feature - if [ $ngx_found = yes ]; then - CORE_LIBS="$CORE_LIBS -Wl,--undefined=pcre_version" - fi + if [ $ngx_found = yes ]; then + CORE_LIBS="-Wl,-u,_pcre_version $CORE_LIBS" + fi + ;; - # for LLVM ld (Darwin) - ngx_feature="force undefined symbols (-all_load -U)" - ngx_feature_libs="-all_load -U printf" - ngx_feature_name= - ngx_feature_run=no - ngx_feature_incs="#include " - ngx_feature_path= - ngx_feature_test='printf("hello");' + *) + ngx_feature="require defined symbols (--require-defined)" + ngx_feature_name= + ngx_feature_path= + ngx_feature_libs="-Wl,--require-defined=strerror" + ngx_feature_run=no + ngx_feature_incs="#include " + ngx_feature_test='printf("hello");' - . auto/feature + . auto/feature - if [ $ngx_found = yes ]; then - CORE_LIBS="$CORE_LIBS -all_load -U pcre_version" - fi + if [ $ngx_found = yes ]; then + CORE_LIBS="-Wl,--require-defined=pcre_version $CORE_LIBS" + fi + ;; + esac fi # ---------------------------------------- From 70484bcbe4a0dcc67137df6387b3a7bed205fff8 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 15 Aug 2019 17:57:37 -0700 Subject: [PATCH 361/848] tweaks: ported adjustments and missing test cases from the ngx_stream_lua module. --- src/ngx_http_lua_util.c | 4 +-- t/058-tcp-socket.t | 58 ++++++++++++++++++++++++++++++++++------- t/091-coroutine.t | 8 +++--- t/161-load-resty-core.t | 29 ++++++++++++++++----- 4 files changed, 77 insertions(+), 22 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 5cd5bac5d8..b3817fddd7 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1352,7 +1352,7 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, if (!ctx->cur_co_ctx->is_wrap) { /* - * ended successful, coroutine.resume returns true plus + * ended successfully, coroutine.resume returns true plus * any return values */ lua_pushboolean(next_co, success); @@ -1520,7 +1520,7 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, if (orig_coctx->is_wrap) { /* * coroutine.wrap propagates errors - * to the parent + * to its parent coroutine */ next_coctx->propagate_error = 1; continue; diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 837006c638..6ac67e0849 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 219; +plan tests => repeat_each() * 222; our $HtmlDir = html_dir; @@ -3643,7 +3643,45 @@ lua http cleanup reuse -=== TEST 60: options_table is nil +=== TEST 60: setkeepalive on socket already shutdown +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local ok, err = sock:close() + if not ok then + ngx.log(ngx.ERR, "failed to close socket: ", err) + return + end + + local ok, err = sock:setkeepalive() + if not ok then + ngx.log(ngx.ERR, "failed to setkeepalive: ", err) + end + } + } +--- request +GET /t +--- response_body +connected: 1 +--- error_log +failed to setkeepalive: closed + + + +=== TEST 61: options_table is nil --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -3693,7 +3731,7 @@ close: 1 nil -=== TEST 61: resolver send query failing immediately in connect() +=== TEST 62: resolver send query failing immediately in connect() this case did not clear coctx->cleanup properly and would lead to memory invalid accesses. this test case requires the following iptables rule to work properly: @@ -3730,7 +3768,7 @@ qr{\[alert\] .*? send\(\) failed \(\d+: Operation not permitted\) while resolvin -=== TEST 62: the upper bound of port range should be 2^16 - 1 +=== TEST 63: the upper bound of port range should be 2^16 - 1 --- config location /t { content_by_lua_block { @@ -3749,7 +3787,7 @@ failed to connect: bad port number: 65536 -=== TEST 63: send boolean and nil +=== TEST 64: send boolean and nil --- config location /t { set $port $TEST_NGINX_SERVER_PORT; @@ -3811,7 +3849,7 @@ received: truefalsenil -=== TEST 64: receiveany method in cosocket +=== TEST 65: receiveany method in cosocket --- config server_tokens off; location = /t { @@ -3900,7 +3938,7 @@ lua tcp socket read any -=== TEST 65: receiveany send data after read side closed +=== TEST 66: receiveany send data after read side closed --- config server_tokens off; location = /t { @@ -3944,7 +3982,7 @@ GET /t -=== TEST 66: receiveany with limited, max <= 0 +=== TEST 67: receiveany with limited, max <= 0 --- config location = /t { set $port $TEST_NGINX_SERVER_PORT; @@ -3980,7 +4018,7 @@ GET /t -=== TEST 67: receiveany with limited, max is larger than data +=== TEST 68: receiveany with limited, max is larger than data --- config server_tokens off; location = /t { @@ -4049,7 +4087,7 @@ lua tcp socket calling receiveany() method to read at most 128 bytes -=== TEST 68: receiveany with limited, max is smaller than data +=== TEST 69: receiveany with limited, max is smaller than data --- config server_tokens off; location = /t { diff --git a/t/091-coroutine.t b/t/091-coroutine.t index 5af1ea3a19..b9b3d9d38d 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -1533,7 +1533,7 @@ co == co2: false --- config location = /t { content_by_lua_block { - function f() + local function f() local cnt = 0 for i = 1, 20 do ngx.say("co yield: ", cnt) @@ -1567,7 +1567,7 @@ co yield: 2 --- config location = /t { content_by_lua_block { - function f() + local function f() local cnt = 0 for i = 1, 20 do coroutine.yield(cnt, cnt + 1) @@ -1601,7 +1601,7 @@ co yield: 2, 3 --- config location = /t { content_by_lua_block { - function f(step) + local function f(step) local cnt = 0 for i = 1, 20 do ngx.say("co yield: ", cnt) @@ -1637,7 +1637,7 @@ co yield: 2 return 200; header_filter_by_lua_block { - function f() + local function f() local cnt = 0 for i = 1, 20 do print("co yield: ", cnt) diff --git a/t/161-load-resty-core.t b/t/161-load-resty-core.t index 444e3c8ee0..e6a4d0471a 100644 --- a/t/161-load-resty-core.t +++ b/t/161-load-resty-core.t @@ -38,7 +38,24 @@ resty.core loaded: true -=== TEST 2: resty.core is automatically loaded in the Lua VM with 'lua_code_cache off' +=== TEST 2: resty.core is automatically loaded in the Lua VM when 'lua_shared_dict' is used +--- http_config + lua_shared_dict dogs 128k; +--- config + location = /t { + content_by_lua_block { + local loaded_resty_core = package.loaded["resty.core"] + local resty_core = require "resty.core" + + ngx.say("resty.core loaded: ", loaded_resty_core == resty_core) + } + } +--- response_body +resty.core loaded: true + + + +=== TEST 3: resty.core is automatically loaded in the Lua VM with 'lua_code_cache off' --- http_config lua_code_cache off; --- config @@ -55,7 +72,7 @@ resty.core loaded: true -=== TEST 3: resty.core loading honors the lua_package_path directive +=== TEST 4: resty.core loading honors the lua_package_path directive --- http_config eval "lua_package_path '$::HtmlDir/?.lua;;';" --- config @@ -82,7 +99,7 @@ return { -=== TEST 4: resty.core not loading aborts the initialization +=== TEST 5: resty.core not loading aborts the initialization --- http_config eval "lua_package_path '$::HtmlDir/?.lua;';" --- config @@ -95,7 +112,7 @@ qr/\[alert\] .*? failed to load the 'resty\.core' module .*? \(reason: module 'r -=== TEST 5: resty.core not loading produces an error with 'lua_code_cache off' +=== TEST 6: resty.core not loading produces an error with 'lua_code_cache off' --- http_config lua_code_cache off; @@ -116,7 +133,7 @@ qr/\[alert\] .*? failed to load the 'resty\.core' module/ -=== TEST 6: lua_load_resty_core logs a deprecation warning when specified (on) +=== TEST 7: lua_load_resty_core logs a deprecation warning when specified (on) --- http_config lua_load_resty_core on; --- config @@ -132,7 +149,7 @@ qr/\[warn\] .*? lua_load_resty_core is deprecated \(the lua-resty-core library i -=== TEST 7: lua_load_resty_core logs a deprecation warning when specified (off) +=== TEST 8: lua_load_resty_core logs a deprecation warning when specified (off) --- http_config lua_load_resty_core off; --- config From 4c943b9447784439153a2f4adec69f2fd6520d10 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 22 Aug 2019 15:55:52 -0700 Subject: [PATCH 362/848] tests: 087-udp-socket.t: removed a duplicated test case. --- t/087-udp-socket.t | 59 +--------------------------------------------- 1 file changed, 1 insertion(+), 58 deletions(-) diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index dc6cad5ef8..f6ef26067d 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (3 * blocks() + 14); +plan tests => repeat_each() * (3 * blocks() + 13); our $HtmlDir = html_dir; @@ -1006,63 +1006,6 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; - content_by_lua ' - local test = require "test" - if ngx.var.arg_reset then - local sock = test.new_sock() - local ok, err = sock:setpeername("127.0.0.1", ngx.var.port) - if not ok then - ngx.say("failed to set peer: ", err) - else - ngx.say("peer set") - end - return - end - local sock = test.get_sock() - sock:send("a") - '; - } ---- user_files ->>> test.lua -module("test", package.seeall) - -local sock - -function new_sock() - sock = ngx.socket.udp() - return sock -end - -function get_sock() - return sock -end ---- request -GET /main ---- response_body_like eval -qr/^peer set - Date: Thu, 15 Aug 2019 17:55:09 -0700 Subject: [PATCH 363/848] doc: readme: minor updates to align with the ngx_stream_lua module README. --- README.markdown | 19 ++++++++++--------- doc/HttpLuaModule.wiki | 19 ++++++++++--------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/README.markdown b/README.markdown index f085848da2..0d4e0dd8eb 100644 --- a/README.markdown +++ b/README.markdown @@ -319,10 +319,10 @@ patches](https://github.com/openresty/openresty/tree/master/patches). Alternatively, ngx_lua can be manually compiled into Nginx: -1. LuaJIT can be downloaded from the [latest release of OpenResty's LuaJIT fork](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above.. -1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags). -1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags). -1. Download the latest version of Nginx [HERE](https://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) +1. LuaJIT can be downloaded from the [latest release of OpenResty's LuaJIT fork](https://github.com/openresty/luajit2/releases). The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above +1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags) +1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags) +1. Download the latest supported version of Nginx [HERE](https://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) Build the source with this module: @@ -413,7 +413,8 @@ The [openresty](https://groups.google.com/group/openresty) mailing list is for C Code Repository =============== -The code repository of this project is hosted on github at [openresty/lua-nginx-module](https://github.com/openresty/lua-nginx-module). +The code repository of this project is hosted on GitHub at +[openresty/lua-nginx-module](https://github.com/openresty/lua-nginx-module). [Back to TOC](#table-of-contents) @@ -7625,7 +7626,7 @@ tcpsock:settimeout Set the timeout value in milliseconds for subsequent socket operations ([connect](#tcpsockconnect), [receive](#tcpsockreceive), and iterators returned from [receiveuntil](#tcpsockreceiveuntil)). -Settings done by this method takes priority over those config directives, i.e., [lua_socket_connect_timeout](#lua_socket_connect_timeout), [lua_socket_send_timeout](#lua_socket_send_timeout), and [lua_socket_read_timeout](#lua_socket_read_timeout). +Settings done by this method take priority over those specified via config directives (i.e. [lua_socket_connect_timeout](#lua_socket_connect_timeout), [lua_socket_send_timeout](#lua_socket_send_timeout), and [lua_socket_read_timeout](#lua_socket_read_timeout)). Note that this method does *not* affect the [lua_socket_keepalive_timeout](#lua_socket_keepalive_timeout) setting; the `timeout` argument to the [setkeepalive](#tcpsocksetkeepalive) method should be used for this purpose instead. @@ -7640,12 +7641,12 @@ tcpsock:settimeouts **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** -Sets the connect timeout thresold, send timeout threshold, and read timeout threshold, respetively, in milliseconds, for subsequent socket +Respectively sets the connect, send, and read timeout thresholds (in milliseconds) for subsequent socket operations ([connect](#tcpsockconnect), [send](#tcpsocksend), [receive](#tcpsockreceive), and iterators returned from [receiveuntil](#tcpsockreceiveuntil)). -Settings done by this method takes priority over those config directives, i.e., [lua_socket_connect_timeout](#lua_socket_connect_timeout), [lua_socket_send_timeout](#lua_socket_send_timeout), and [lua_socket_read_timeout](#lua_socket_read_timeout). +Settings done by this method take priority over those specified via config directives (i.e. [lua_socket_connect_timeout](#lua_socket_connect_timeout), [lua_socket_send_timeout](#lua_socket_send_timeout), and [lua_socket_read_timeout](#lua_socket_read_timeout)). -You are recommended to use [settimeouts](#tcpsocksettimeouts) instead of [settimeout](#tcpsocksettimeout). +It is recommended to use [settimeouts](#tcpsocksettimeouts) instead of [settimeout](#tcpsocksettimeout). Note that this method does *not* affect the [lua_socket_keepalive_timeout](#lua_socket_keepalive_timeout) setting; the `timeout` argument to the [setkeepalive](#tcpsocksetkeepalive) method should be used for this purpose instead. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index e158c1cc27..762a7c9b6a 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -254,10 +254,10 @@ patches]. Alternatively, ngx_lua can be manually compiled into Nginx: -# LuaJIT can be downloaded from the [https://github.com/openresty/luajit2/releases latest release of OpenResty's LuaJIT fork]. The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above.. -# Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE]. -# Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE]. -# Download the latest version of Nginx [https://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) +# LuaJIT can be downloaded from the [https://github.com/openresty/luajit2/releases latest release of OpenResty's LuaJIT fork]. The official LuaJIT 2.x releases are also supported, although performance will be significantly lower for reasons elaborated above +# Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE] +# Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE] +# Download the latest supported version of Nginx [https://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) Build the source with this module: @@ -328,7 +328,8 @@ The [https://groups.google.com/group/openresty openresty] mailing list is for Ch = Code Repository = -The code repository of this project is hosted on github at [https://github.com/openresty/lua-nginx-module openresty/lua-nginx-module]. +The code repository of this project is hosted on GitHub at +[https://github.com/openresty/lua-nginx-module openresty/lua-nginx-module]. = Bugs and Patches = @@ -6481,7 +6482,7 @@ This feature was first introduced in the v0.5.0rc1 release. Set the timeout value in milliseconds for subsequent socket operations ([[#tcpsock:connect|connect]], [[#tcpsock:receive|receive]], and iterators returned from [[#tcpsock:receiveuntil|receiveuntil]]). -Settings done by this method takes priority over those config directives, i.e., [[#lua_socket_connect_timeout|lua_socket_connect_timeout]], [[#lua_socket_send_timeout|lua_socket_send_timeout]], and [[#lua_socket_read_timeout|lua_socket_read_timeout]]. +Settings done by this method take priority over those specified via config directives (i.e. [[#lua_socket_connect_timeout|lua_socket_connect_timeout]], [[#lua_socket_send_timeout|lua_socket_send_timeout]], and [[#lua_socket_read_timeout|lua_socket_read_timeout]]). Note that this method does ''not'' affect the [[#lua_socket_keepalive_timeout|lua_socket_keepalive_timeout]] setting; the timeout argument to the [[#tcpsock:setkeepalive|setkeepalive]] method should be used for this purpose instead. @@ -6493,12 +6494,12 @@ This feature was first introduced in the v0.5.0rc1 release. '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' -Sets the connect timeout thresold, send timeout threshold, and read timeout threshold, respetively, in milliseconds, for subsequent socket +Respectively sets the connect, send, and read timeout thresholds (in milliseconds) for subsequent socket operations ([[#tcpsock:connect|connect]], [[#tcpsock:send|send]], [[#tcpsock:receive|receive]], and iterators returned from [[#tcpsock:receiveuntil|receiveuntil]]). -Settings done by this method takes priority over those config directives, i.e., [[#lua_socket_connect_timeout|lua_socket_connect_timeout]], [[#lua_socket_send_timeout|lua_socket_send_timeout]], and [[#lua_socket_read_timeout|lua_socket_read_timeout]]. +Settings done by this method take priority over those specified via config directives (i.e. [[#lua_socket_connect_timeout|lua_socket_connect_timeout]], [[#lua_socket_send_timeout|lua_socket_send_timeout]], and [[#lua_socket_read_timeout|lua_socket_read_timeout]]). -You are recommended to use [[#tcpsock:settimeouts|settimeouts]] instead of [[#tcpsock:settimeout|settimeout]]. +It is recommended to use [[#tcpsock:settimeouts|settimeouts]] instead of [[#tcpsock:settimeout|settimeout]]. Note that this method does ''not'' affect the [[#lua_socket_keepalive_timeout|lua_socket_keepalive_timeout]] setting; the timeout argument to the [[#tcpsock:setkeepalive|setkeepalive]] method should be used for this purpose instead. From fd1e0f89e495bcec9e930cb8c3525e953baa6f4a Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 28 Aug 2019 13:59:05 -0700 Subject: [PATCH 364/848] feat: config: added a check for Lua language version 5.1. --- config | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/config b/config index 6431fa38fc..e15cb49593 100644 --- a/config +++ b/config @@ -176,9 +176,9 @@ fi ngx_feature="LuaJIT 2.x" ngx_feature_run=no ngx_feature_incs="#include " -ngx_feature_test='#if !defined(LUAJIT_VERSION_NUM) || (LUAJIT_VERSION_NUM < 20000) +ngx_feature_test="#if !defined(LUAJIT_VERSION_NUM) || LUAJIT_VERSION_NUM < 20000 # error unsupported LuaJIT version - #endif' + #endif" . auto/feature @@ -191,6 +191,24 @@ fi # ---------------------------------------- +ngx_feature="Lua language 5.1" +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_test="#if !defined(LUA_VERSION_NUM) || LUA_VERSION_NUM != 501 + # error unsupported Lua language version + #endif" + +. auto/feature + +if [ $ngx_found = no ]; then + cat << END + $0: error: unsupported Lua language version; ngx_http_lua_module requires Lua 5.1. +END + exit 1 +fi + +# ---------------------------------------- + ngx_feature="LuaJIT has FFI" ngx_feature_libs="$LUAJIT_LIB/libluajit-5.1.a $luajit_ld_opt" ngx_feature_run=yes From 50dabbc3ae2e4123a004ba68c92ef889026df0b4 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 12 Sep 2019 10:55:23 -0700 Subject: [PATCH 365/848] travis-ci: bumped OpenSSL versions to 1.0.2t, 1.1.0l, and 1.1.1d. --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index e50de2cdbb..b7f6b2412f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,9 +48,9 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.17.1 OPENSSL_VER=1.0.2s OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.0k OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.1c OPENSSL_PATCH_VER=1.1.1c + - NGINX_VERSION=1.17.1 OPENSSL_VER=1.0.2t OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c services: - memcache From d062170e94141b236483d86067cdb0d0a596353d Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 10 Sep 2019 11:45:11 -0700 Subject: [PATCH 366/848] feature: implemented the 'ngx.crc32_short' and 'ngx.crc32_long' C FFI APIs. --- src/ngx_http_lua_string.c | 58 ++++++++++----------------------------- 1 file changed, 14 insertions(+), 44 deletions(-) diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index 7092ff02f4..1bd4f2d99c 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -30,8 +30,6 @@ static uintptr_t ngx_http_lua_ngx_escape_sql_str(u_char *dst, u_char *src, size_t size); static int ngx_http_lua_ngx_quote_sql_str(lua_State *L); -static int ngx_http_lua_ngx_crc32_short(lua_State *L); -static int ngx_http_lua_ngx_crc32_long(lua_State *L); static int ngx_http_lua_ngx_encode_args(lua_State *L); static int ngx_http_lua_ngx_decode_args(lua_State *L); #if (NGX_OPENSSL) @@ -51,12 +49,6 @@ ngx_http_lua_inject_string_api(lua_State *L) lua_pushcfunction(L, ngx_http_lua_ngx_quote_sql_str); lua_setfield(L, -2, "quote_sql_str"); - lua_pushcfunction(L, ngx_http_lua_ngx_crc32_short); - lua_setfield(L, -2, "crc32_short"); - - lua_pushcfunction(L, ngx_http_lua_ngx_crc32_long); - lua_setfield(L, -2, "crc32_long"); - #if (NGX_OPENSSL) lua_pushcfunction(L, ngx_http_lua_ngx_hmac_sha1); lua_setfield(L, -2, "hmac_sha1"); @@ -256,42 +248,6 @@ ngx_http_lua_encode_base64(ngx_str_t *dst, ngx_str_t *src, int no_padding) } -static int -ngx_http_lua_ngx_crc32_short(lua_State *L) -{ - u_char *p; - size_t len; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument, but got %d", - lua_gettop(L)); - } - - p = (u_char *) luaL_checklstring(L, 1, &len); - - lua_pushnumber(L, (lua_Number) ngx_crc32_short(p, len)); - return 1; -} - - -static int -ngx_http_lua_ngx_crc32_long(lua_State *L) -{ - u_char *p; - size_t len; - - if (lua_gettop(L) != 1) { - return luaL_error(L, "expecting one argument, but got %d", - lua_gettop(L)); - } - - p = (u_char *) luaL_checklstring(L, 1, &len); - - lua_pushnumber(L, (lua_Number) ngx_crc32_long(p, len)); - return 1; -} - - static int ngx_http_lua_ngx_encode_args(lua_State *L) { @@ -414,6 +370,20 @@ ngx_http_lua_ffi_sha1_bin(const u_char *src, size_t len, u_char *dst) } +unsigned int +ngx_http_lua_ffi_crc32_short(const u_char *src, size_t len) +{ + return ngx_crc32_short((u_char *) src, len); +} + + +unsigned int +ngx_http_lua_ffi_crc32_long(const u_char *src, size_t len) +{ + return ngx_crc32_long((u_char *) src, len); +} + + size_t ngx_http_lua_ffi_encode_base64(const u_char *src, size_t slen, u_char *dst, int no_padding) From 69780f676900fb861cc4fa116e6dbe28d805da74 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 24 Sep 2019 12:33:25 -0700 Subject: [PATCH 367/848] bugfix: config: fallback to '--undefined' option for ld 2.25 and below. --- config | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/config b/config index e15cb49593..32290d4c69 100644 --- a/config +++ b/config @@ -522,6 +522,22 @@ if [ $USE_PCRE = YES -o $PCRE != NONE ] && [ $PCRE != NO -a $PCRE != YES ]; then if [ $ngx_found = yes ]; then CORE_LIBS="-Wl,--require-defined=pcre_version $CORE_LIBS" + + else + # ld 2.25 and below (--required-defined was introduced in 2.26) + ngx_feature="require defined symbols fallback (--undefined)" + ngx_feature_name= + ngx_feature_path= + ngx_feature_libs="-Wl,--undefined=strerror" + ngx_feature_run=no + ngx_feature_incs="#include " + ngx_feature_test='printf("hello");' + + . auto/feature + + if [ $ngx_found = yes ]; then + CORE_LIBS="-Wl,--undefined=pcre_version $CORE_LIBS" + fi fi ;; esac From 4e593c335ffff5af1f3465bc6a71500951222331 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 27 Sep 2019 11:25:09 -0700 Subject: [PATCH 368/848] travis-ci: bumped the NGINX core to 1.17.4. --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index b7f6b2412f..377b15652e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,9 +48,9 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.17.1 OPENSSL_VER=1.0.2t OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.1 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c + - NGINX_VERSION=1.17.4 OPENSSL_VER=1.0.2t OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.17.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.17.4 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c services: - memcache From 478bae6fc14359fe5b07f66f9fa721efe3eb0a9b Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 27 Sep 2019 16:20:47 -0700 Subject: [PATCH 369/848] doc: readme: mentioned that the latest tested NGINX core is 1.17.4. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 0d4e0dd8eb..323f4f0c0a 100644 --- a/README.markdown +++ b/README.markdown @@ -284,7 +284,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: -* 1.17.x (last tested: 1.17.1) +* 1.17.x (last tested: 1.17.4) * 1.15.x (last tested: 1.15.8) * 1.14.x * 1.13.x (last tested: 1.13.6) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 762a7c9b6a..af4c0c1598 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -221,7 +221,7 @@ performance level. The latest version of this module is compatible with the following versions of Nginx: -* 1.17.x (last tested: 1.17.1) +* 1.17.x (last tested: 1.17.4) * 1.15.x (last tested: 1.15.8) * 1.14.x * 1.13.x (last tested: 1.13.6) From d1e55394ea70be0a95d2e429fce489fbb263a50c Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 3 Sep 2019 12:12:57 -0700 Subject: [PATCH 370/848] misc: removed dead code, guard non-OpenResty LuaJIT definitions, and fixed styling issues. --- src/api/ngx_http_lua_api.h | 2 +- src/ngx_http_lua_balancer.c | 6 +- src/ngx_http_lua_clfactory.c | 4 +- src/ngx_http_lua_common.h | 131 ++++++++++++++------------------- src/ngx_http_lua_directive.c | 6 +- src/ngx_http_lua_headers_in.c | 2 +- src/ngx_http_lua_log_ringbuf.c | 2 +- src/ngx_http_lua_pipe.c | 4 +- src/ngx_http_lua_semaphore.c | 2 +- src/ngx_http_lua_socket_tcp.c | 6 +- src/ngx_http_lua_socket_udp.c | 2 +- src/ngx_http_lua_ssl_certby.c | 2 +- src/ngx_http_lua_subrequest.c | 6 ++ src/ngx_http_lua_util.c | 11 +-- src/ngx_http_lua_util.h | 127 ++++++++++++++++---------------- 15 files changed, 143 insertions(+), 170 deletions(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 221e423ddb..02c242299a 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10016 +#define ngx_http_lua_version 10016 typedef struct { diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index f2db02d513..f71a3e00ad 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -54,8 +54,8 @@ static ngx_int_t ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data); static ngx_int_t ngx_http_lua_balancer_by_chunk(lua_State *L, ngx_http_request_t *r); -void ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, - ngx_uint_t state); +static void ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, + void *data, ngx_uint_t state); ngx_int_t @@ -399,7 +399,7 @@ ngx_http_lua_balancer_by_chunk(lua_State *L, ngx_http_request_t *r) } -void +static void ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, ngx_uint_t state) { diff --git a/src/ngx_http_lua_clfactory.c b/src/ngx_http_lua_clfactory.c index 045e46adf9..e8a8a0e1f7 100644 --- a/src/ngx_http_lua_clfactory.c +++ b/src/ngx_http_lua_clfactory.c @@ -288,12 +288,12 @@ typedef enum { NGX_LUA_TEXT_FILE, NGX_LUA_BT_LUA, - NGX_LUA_BT_LJ + NGX_LUA_BT_LJ, } ngx_http_lua_clfactory_file_type_e; enum { - NGX_LUA_READER_BUFSIZE = 4096 + NGX_LUA_READER_BUFSIZE = 4096, }; diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 5d998d8d2f..30dc52a4a0 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -22,34 +22,44 @@ #include -#if (NGX_PCRE) - -#include +#if defined(NDK) && NDK +#include -#if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) -# define LUA_HAVE_PCRE_JIT 1 -#else -# define LUA_HAVE_PCRE_JIT 0 +typedef struct { + size_t size; + int ref; + u_char *key; + ngx_str_t script; +} ngx_http_lua_set_var_data_t; #endif + +#ifdef NGX_LUA_USE_ASSERT +#include +# define ngx_http_lua_assert(a) assert(a) +#else +# define ngx_http_lua_assert(a) #endif -#if !defined(nginx_version) || (nginx_version < 1006000) -#error at least nginx 1.6.0 is required but found an older version +#if (NGX_PCRE) +#include +# if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) +# define LUA_HAVE_PCRE_JIT 1 +# else +# define LUA_HAVE_PCRE_JIT 0 +# endif #endif -#if defined(NDK) && NDK -#include +#if !defined(nginx_version) || (nginx_version < 1006000) +# error at least nginx 1.6.0 is required but found an older version #endif - #if LUA_VERSION_NUM != 501 # error unsupported Lua language version #endif - #if !defined(LUAJIT_VERSION_NUM) || (LUAJIT_VERSION_NUM < 20000) # error unsupported LuaJIT version #endif @@ -60,26 +70,25 @@ #endif #ifndef NGX_HTTP_PERMANENT_REDIRECT -# define NGX_HTTP_PERMANENT_REDIRECT 308 +# define NGX_HTTP_PERMANENT_REDIRECT 308 #endif #ifndef NGX_HAVE_SHA1 # if defined(nginx_version) && (nginx_version >= 1011002) -# define NGX_HAVE_SHA1 1 +# define NGX_HAVE_SHA1 1 # endif #endif - #ifndef MD5_DIGEST_LENGTH -#define MD5_DIGEST_LENGTH 16 +# define MD5_DIGEST_LENGTH 16 #endif +#ifndef NGX_HTTP_LUA_MAX_ARGS +# define NGX_HTTP_LUA_MAX_ARGS 100 +#endif -#ifdef NGX_LUA_USE_ASSERT -# include -# define ngx_http_lua_assert(a) assert(a) -#else -# define ngx_http_lua_assert(a) +#ifndef NGX_HTTP_LUA_MAX_HEADERS +# define NGX_HTTP_LUA_MAX_HEADERS 100 #endif @@ -104,26 +113,6 @@ (NGX_HTTP_LUA_FILE_TAG_LEN + 2 * MD5_DIGEST_LENGTH) -#if defined(NDK) && NDK -typedef struct { - size_t size; - int ref; - u_char *key; - ngx_str_t script; -} ngx_http_lua_set_var_data_t; -#endif - - -#ifndef NGX_HTTP_LUA_MAX_ARGS -#define NGX_HTTP_LUA_MAX_ARGS 100 -#endif - - -#ifndef NGX_HTTP_LUA_MAX_HEADERS -#define NGX_HTTP_LUA_MAX_HEADERS 100 -#endif - - /* must be within 16 bit */ #define NGX_HTTP_LUA_CONTEXT_SET 0x0001 #define NGX_HTTP_LUA_CONTEXT_REWRITE 0x0002 @@ -145,30 +134,38 @@ typedef struct { #if (NGX_PTR_SIZE >= 8 && !defined(_WIN64)) -#define ngx_http_lua_lightudata_mask(ludata) \ - ((void *) ((uintptr_t) (&ngx_http_lua_##ludata) & ((1UL << 47) - 1))) - +# define ngx_http_lua_lightudata_mask(ludata) \ + ((void *) ((uintptr_t) (&ngx_http_lua_##ludata) & ((1UL << 47) - 1))) #else -#define ngx_http_lua_lightudata_mask(ludata) (&ngx_http_lua_##ludata) +# define ngx_http_lua_lightudata_mask(ludata) \ + (&ngx_http_lua_##ludata) #endif -typedef struct ngx_http_lua_main_conf_s ngx_http_lua_main_conf_t; -typedef union ngx_http_lua_srv_conf_u ngx_http_lua_srv_conf_t; +typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; +typedef struct ngx_http_lua_sema_mm_s ngx_http_lua_sema_mm_t; -typedef struct ngx_http_lua_balancer_peer_data_s - ngx_http_lua_balancer_peer_data_t; +typedef union ngx_http_lua_srv_conf_u ngx_http_lua_srv_conf_t; +typedef struct ngx_http_lua_main_conf_s ngx_http_lua_main_conf_t; -typedef struct ngx_http_lua_sema_mm_s ngx_http_lua_sema_mm_t; +typedef struct ngx_http_lua_header_val_s ngx_http_lua_header_val_t; + +typedef struct ngx_http_lua_posted_thread_s ngx_http_lua_posted_thread_t; +typedef struct ngx_http_lua_balancer_peer_data_s + ngx_http_lua_balancer_peer_data_t; typedef ngx_int_t (*ngx_http_lua_main_conf_handler_pt)(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, lua_State *L); + typedef ngx_int_t (*ngx_http_lua_srv_conf_handler_pt)(ngx_http_request_t *r, ngx_http_lua_srv_conf_t *lscf, lua_State *L); +typedef ngx_int_t (*ngx_http_lua_set_header_pt)(ngx_http_request_t *r, + ngx_http_lua_header_val_t *hv, ngx_str_t *value); + typedef struct { u_char *package; @@ -198,11 +195,9 @@ struct ngx_http_lua_main_conf_s { ngx_int_t regex_cache_entries; ngx_int_t regex_cache_max_entries; ngx_int_t regex_match_limit; - -#if (LUA_HAVE_PCRE_JIT) +# if (LUA_HAVE_PCRE_JIT) pcre_jit_stack *jit_stack; -#endif - +# endif #endif ngx_array_t *shm_zones; /* of ngx_shm_zone_t* */ @@ -298,10 +293,10 @@ union ngx_http_lua_srv_conf_u { #endif struct { - ngx_http_lua_srv_conf_handler_pt handler; - ngx_str_t src; - u_char *src_key; - int src_ref; + ngx_http_lua_srv_conf_handler_pt handler; + ngx_str_t src; + u_char *src_key; + int src_ref; } balancer; }; @@ -398,7 +393,7 @@ typedef enum { NGX_HTTP_LUA_USER_CORO_NOP = 0, NGX_HTTP_LUA_USER_CORO_RESUME = 1, NGX_HTTP_LUA_USER_CORO_YIELD = 2, - NGX_HTTP_LUA_USER_THREAD_RESUME = 3 + NGX_HTTP_LUA_USER_THREAD_RESUME = 3, } ngx_http_lua_user_coro_op_t; @@ -411,21 +406,12 @@ typedef enum { } ngx_http_lua_co_status_t; -typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; - -typedef struct ngx_http_lua_posted_thread_s ngx_http_lua_posted_thread_t; - struct ngx_http_lua_posted_thread_s { ngx_http_lua_co_ctx_t *co_ctx; ngx_http_lua_posted_thread_t *next; }; -enum { - NGX_HTTP_LUA_SUBREQ_TRUNCATED = 1 -}; - - struct ngx_http_lua_co_ctx_s { void *data; /* user state for cosockets */ @@ -615,13 +601,6 @@ typedef struct ngx_http_lua_ctx_s { } ngx_http_lua_ctx_t; -typedef struct ngx_http_lua_header_val_s ngx_http_lua_header_val_t; - - -typedef ngx_int_t (*ngx_http_lua_set_header_pt)(ngx_http_request_t *r, - ngx_http_lua_header_val_t *hv, ngx_str_t *value); - - struct ngx_http_lua_header_val_s { ngx_http_complex_value_t value; ngx_uint_t hash; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 42431f8362..d50c037ecb 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -66,7 +66,7 @@ enum { FOUND_RIGHT_LBRACKET, FOUND_COMMENT_LINE, FOUND_DOUBLE_QUOTED, - FOUND_SINGLE_QUOTED + FOUND_SINGLE_QUOTED, }; @@ -1272,7 +1272,7 @@ ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd) ngx_array_t *saved; enum { parse_block = 0, - parse_param + parse_param, } type; if (cf->conf_file->file.fd != NGX_INVALID_FILE) { @@ -1422,7 +1422,7 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, ngx_http_lua_block_parser_ctx_t *ctx) { enum { - OVEC_SIZE = 2 + OVEC_SIZE = 2, }; int i, rc; int ovec[OVEC_SIZE]; diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 4a5a5b6146..26739fa3eb 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -340,7 +340,7 @@ ngx_http_lua_validate_host(ngx_str_t *host, ngx_pool_t *pool, ngx_uint_t alloc) enum { sw_usual = 0, sw_literal, - sw_rest + sw_rest, } state; dot_pos = host->len; diff --git a/src/ngx_http_lua_log_ringbuf.c b/src/ngx_http_lua_log_ringbuf.c index 0227f63fb2..40fbff1e08 100644 --- a/src/ngx_http_lua_log_ringbuf.c +++ b/src/ngx_http_lua_log_ringbuf.c @@ -17,7 +17,7 @@ typedef struct { enum { - HEADER_LEN = sizeof(ngx_http_lua_log_ringbuf_header_t) + HEADER_LEN = sizeof(ngx_http_lua_log_ringbuf_header_t), }; diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index c0a2e346ad..596412c6ec 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -118,7 +118,7 @@ enum { PIPE_ERR_TIMEOUT, PIPE_ERR_ADD_READ_EV, PIPE_ERR_ADD_WRITE_EV, - PIPE_ERR_ABORTED + PIPE_ERR_ABORTED, }; @@ -126,7 +126,7 @@ enum { PIPE_READ_ALL = 0, PIPE_READ_BYTES, PIPE_READ_LINE, - PIPE_READ_ANY + PIPE_READ_ANY, }; diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 38389a063a..a3b79398e7 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -37,7 +37,7 @@ void ngx_http_lua_ffi_sema_gc(ngx_http_lua_sema_t *sem); enum { SEMAPHORE_WAIT_SUCC = 0, - SEMAPHORE_WAIT_TIMEOUT = 1 + SEMAPHORE_WAIT_TIMEOUT = 1, }; diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index d3453d456e..b3a986c5bc 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -174,7 +174,7 @@ enum { SOCKET_OP_CONNECT, SOCKET_OP_READ, SOCKET_OP_WRITE, - SOCKET_OP_RESUME_CONN + SOCKET_OP_RESUME_CONN, }; @@ -1915,8 +1915,8 @@ ngx_http_lua_ssl_handshake_retval_handler(ngx_http_request_t *r, } else { *ud = ssl_session; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua ssl save session: %p", ssl_session); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua ssl save session: %p", ssl_session); /* set up the __gc metamethod */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 0e55344ea8..3a0d204319 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -63,7 +63,7 @@ static void ngx_http_lua_udp_socket_cleanup(void *data); enum { SOCKET_CTX_INDEX = 1, - SOCKET_TIMEOUT_INDEX = 2 + SOCKET_TIMEOUT_INDEX = 2, }; diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 6384d38a70..dc7a95d686 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -26,7 +26,7 @@ enum { NGX_HTTP_LUA_ADDR_TYPE_UNIX = 0, NGX_HTTP_LUA_ADDR_TYPE_INET = 1, - NGX_HTTP_LUA_ADDR_TYPE_INET6 = 2 + NGX_HTTP_LUA_ADDR_TYPE_INET6 = 2, }; diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 174460c5c4..14ad3e3079 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -27,6 +27,7 @@ #define ngx_http_lua_method_name(m) { sizeof(m) - 1, (u_char *) m " " } + ngx_str_t ngx_http_lua_get_method = ngx_http_lua_method_name("GET"); ngx_str_t ngx_http_lua_put_method = ngx_http_lua_method_name("PUT"); ngx_str_t ngx_http_lua_post_method = ngx_http_lua_method_name("POST"); @@ -81,6 +82,11 @@ static ngx_int_t ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, ngx_http_request_t *r); +enum { + NGX_HTTP_LUA_SUBREQ_TRUNCATED = 1, +}; + + /* ngx.location.capture is just a thin wrapper around * ngx.location.capture_multi */ static int diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index b3817fddd7..074957b6e7 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -84,7 +84,6 @@ char ngx_http_lua_code_cache_key; -char ngx_http_lua_regex_cache_key; char ngx_http_lua_socket_pool_key; char ngx_http_lua_coroutines_key; char ngx_http_lua_headers_metatable_key; @@ -691,14 +690,6 @@ ngx_http_lua_init_registry(lua_State *L, ngx_log_t *log) lua_createtable(L, 0, 8 /* nrec */); lua_rawset(L, LUA_REGISTRYINDEX); -#if (NGX_PCRE) - /* create the registry entry for the Lua precompiled regex object cache */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - regex_cache_key)); - lua_createtable(L, 0, 16 /* nrec */); - lua_rawset(L, LUA_REGISTRYINDEX); -#endif - /* {{{ register table to cache user code: * { [(string)cache_key] = } */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( @@ -2065,7 +2056,7 @@ ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, enum { sw_usual = 0, sw_quoted, - sw_quoted_second + sw_quoted_second, } state; d = *dst; diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 8cc7a874a7..64da4de9e1 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -19,54 +19,19 @@ #ifndef NGX_UNESCAPE_URI_COMPONENT -#define NGX_UNESCAPE_URI_COMPONENT 0 +# define NGX_UNESCAPE_URI_COMPONENT 0 #endif -typedef struct { - ngx_http_lua_ffi_str_t key; - ngx_http_lua_ffi_str_t value; -} ngx_http_lua_ffi_table_elt_t; - - -/* char whose address we use as the key in Lua vm registry for - * user code cache table */ -extern char ngx_http_lua_code_cache_key; +#ifndef NGX_HTTP_SWITCHING_PROTOCOLS +# define NGX_HTTP_SWITCHING_PROTOCOLS 101 +#endif /* key in Lua vm registry for all the "ngx.ctx" tables */ #define ngx_http_lua_ctx_tables_key "ngx_lua_ctx_tables" -/* char whose address we use as the key in Lua vm registry for - * regex cache table */ -extern char ngx_http_lua_regex_cache_key; - -/* char whose address we use as the key in Lua vm registry for - * socket connection pool table */ -extern char ngx_http_lua_socket_pool_key; - -/* char whose address we use as the key for the coroutine parent relationship */ -extern char ngx_http_lua_coroutine_parents_key; - -/* coroutine anchoring table key in Lua VM registry */ -extern char ngx_http_lua_coroutines_key; - -/* key to the metatable for ngx.req.get_headers() and ngx.resp.get_headers() */ -extern char ngx_http_lua_headers_metatable_key; - - -#ifndef ngx_str_set -#define ngx_str_set(str, text) \ - (str)->len = sizeof(text) - 1; (str)->data = (u_char *) text -#endif - - -#ifndef NGX_HTTP_SWITCHING_PROTOCOLS -#define NGX_HTTP_SWITCHING_PROTOCOLS 101 -#endif - - #define ngx_http_lua_context_name(c) \ ((c) == NGX_HTTP_LUA_CONTEXT_SET ? "set_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_REWRITE ? "rewrite_by_lua*" \ @@ -93,23 +58,6 @@ extern char ngx_http_lua_headers_metatable_key; } -static ngx_inline ngx_int_t -ngx_http_lua_ffi_check_context(ngx_http_lua_ctx_t *ctx, unsigned flags, - u_char *err, size_t *errlen) -{ - if (!(ctx->context & flags)) { - *errlen = ngx_snprintf(err, *errlen, - "API disabled in the context of %s", - ngx_http_lua_context_name((ctx)->context)) - - err; - - return NGX_DECLINED; - } - - return NGX_OK; -} - - #define ngx_http_lua_check_fake_request(L, r) \ if ((r)->connection->fd == (ngx_socket_t) -1) { \ return luaL_error(L, "API disabled in the current context"); \ @@ -123,10 +71,64 @@ ngx_http_lua_ffi_check_context(ngx_http_lua_ctx_t *ctx, unsigned flags, } +#define ngx_http_lua_check_if_abortable(L, ctx) \ + if ((ctx)->no_abort) { \ + return luaL_error(L, "attempt to abort with pending subrequests"); \ + } + + #define ngx_http_lua_ssl_get_ctx(ssl_conn) \ SSL_get_ex_data(ssl_conn, ngx_http_lua_ssl_ctx_index) +#define ngx_http_lua_hash_literal(s) \ + ngx_http_lua_hash_str((u_char *) s, sizeof(s) - 1) + + +#ifndef ngx_str_set +# define ngx_str_set(str, text) \ + (str)->len = sizeof(text) - 1; (str)->data = (u_char *) text +#endif + + +typedef struct { + ngx_http_lua_ffi_str_t key; + ngx_http_lua_ffi_str_t value; +} ngx_http_lua_ffi_table_elt_t; + + +/* char whose address we use as the key in Lua vm registry for + * user code cache table */ +extern char ngx_http_lua_code_cache_key; + +/* char whose address we use as the key in Lua vm registry for + * socket connection pool table */ +extern char ngx_http_lua_socket_pool_key; + +/* coroutine anchoring table key in Lua VM registry */ +extern char ngx_http_lua_coroutines_key; + +/* key to the metatable for ngx.req.get_headers() and ngx.resp.get_headers() */ +extern char ngx_http_lua_headers_metatable_key; + + +static ngx_inline ngx_int_t +ngx_http_lua_ffi_check_context(ngx_http_lua_ctx_t *ctx, unsigned flags, + u_char *err, size_t *errlen) +{ + if (!(ctx->context & flags)) { + *errlen = ngx_snprintf(err, *errlen, + "API disabled in the context of %s", + ngx_http_lua_context_name((ctx)->context)) + - err; + + return NGX_DECLINED; + } + + return NGX_OK; +} + + ngx_int_t ngx_http_lua_init_vm(lua_State **new_vm, lua_State *parent_vm, ngx_cycle_t *cycle, ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log, ngx_pool_cleanup_t **pcln); @@ -245,11 +247,6 @@ void ngx_http_lua_cleanup_free(ngx_http_request_t *r, void ngx_http_lua_set_sa_restart(ngx_log_t *log); #endif -#define ngx_http_lua_check_if_abortable(L, ctx) \ - if ((ctx)->no_abort) { \ - return luaL_error(L, "attempt to abort with pending subrequests"); \ - } - static ngx_inline size_t ngx_http_lua_safe_header_value_len(u_char *str, size_t len) @@ -370,7 +367,9 @@ ngx_http_lua_get_lua_vm(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) } +#ifndef OPENRESTY_LUAJIT #define ngx_http_lua_req_key "__ngx_req" +#endif static ngx_inline ngx_http_request_t * @@ -402,6 +401,7 @@ ngx_http_lua_set_req(lua_State *L, ngx_http_request_t *r) } +#ifndef OPENRESTY_LUAJIT static ngx_inline void ngx_http_lua_get_globals_table(lua_State *L) { @@ -414,10 +414,7 @@ ngx_http_lua_set_globals_table(lua_State *L) { lua_replace(L, LUA_GLOBALSINDEX); } - - -#define ngx_http_lua_hash_literal(s) \ - ngx_http_lua_hash_str((u_char *) s, sizeof(s) - 1) +#endif /* OPENRESTY_LUAJIT */ static ngx_inline ngx_uint_t From 771eb231e59770f544055633992f3cac10c890d9 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 26 Sep 2019 17:02:42 -0700 Subject: [PATCH 371/848] misc: ngx_http_lua_util.h: removed 'ngx_str_set' definition as it is defined in NGINX 1.6.0+ NGINX 1.6.0+ (the minimum NGINX core version supported as of this commit) already defines it. --- src/ngx_http_lua_util.h | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 64da4de9e1..13bfd273e7 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -85,12 +85,6 @@ ngx_http_lua_hash_str((u_char *) s, sizeof(s) - 1) -#ifndef ngx_str_set -# define ngx_str_set(str, text) \ - (str)->len = sizeof(text) - 1; (str)->data = (u_char *) text -#endif - - typedef struct { ngx_http_lua_ffi_str_t key; ngx_http_lua_ffi_str_t value; From a3ac3f557eed7efa3460d0870684f18d232adf5f Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 26 Sep 2019 17:12:11 -0700 Subject: [PATCH 372/848] style: updated 'nginx_version' guard macros assuming it is always defined. No need to check if `nginx_version` is defined, as of NGINX 1.6.0+ (today's minimum supported core version), always is. --- src/ngx_http_lua_api.c | 2 +- src/ngx_http_lua_common.h | 6 +++--- src/ngx_http_lua_directive.c | 6 +++--- src/ngx_http_lua_headers.c | 16 ++++++++-------- src/ngx_http_lua_headers.h | 2 +- src/ngx_http_lua_headers_out.c | 2 +- src/ngx_http_lua_initworkerby.c | 4 ++-- src/ngx_http_lua_module.c | 2 +- src/ngx_http_lua_ndk.c | 2 +- src/ngx_http_lua_shdict.c | 4 ++-- src/ngx_http_lua_socket_tcp.c | 6 +++--- src/ngx_http_lua_ssl_certby.c | 2 +- src/ngx_http_lua_ssl_session_fetchby.c | 2 +- src/ngx_http_lua_ssl_session_storeby.c | 2 +- src/ngx_http_lua_subrequest.c | 4 ++-- src/ngx_http_lua_timer.c | 2 +- 16 files changed, 32 insertions(+), 32 deletions(-) diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index ac014f2638..b69d627dce 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -175,7 +175,7 @@ ngx_http_lua_shared_memory_init(ngx_shm_zone_t *shm_zone, void *data) } zone->shm = shm_zone->shm; -#if defined(nginx_version) && nginx_version >= 1009000 +#if (nginx_version >= 1009000) zone->noreuse = shm_zone->noreuse; #endif diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 30dc52a4a0..c8c383c6e4 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -52,7 +52,7 @@ typedef struct { #endif -#if !defined(nginx_version) || (nginx_version < 1006000) +#if (nginx_version < 1006000) # error at least nginx 1.6.0 is required but found an older version #endif @@ -74,7 +74,7 @@ typedef struct { #endif #ifndef NGX_HAVE_SHA1 -# if defined(nginx_version) && (nginx_version >= 1011002) +# if (nginx_version >= 1011002) # define NGX_HAVE_SHA1 1 # endif #endif @@ -251,7 +251,7 @@ struct ngx_http_lua_main_conf_s { of reqeusts */ ngx_uint_t malloc_trim_req_count; -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) /* the following 2 fields are only used by ngx.req.raw_headers() for now */ ngx_buf_t **busy_buf_ptrs; ngx_int_t busy_buf_ptr_count; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index d50c037ecb..80c4b7be58 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1433,12 +1433,12 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, ngx_uint_t start_line; ngx_str_t *word; ngx_buf_t *b; -#if defined(nginx_version) && (nginx_version >= 1009002) +#if (nginx_version >= 1009002) ngx_buf_t *dump; #endif b = cf->conf_file->buffer; -#if defined(nginx_version) && (nginx_version >= 1009002) +#if (nginx_version >= 1009002) dump = cf->conf_file->dump; #endif start = b->pos; @@ -1509,7 +1509,7 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, b->last = b->start + len + n; start = b->start; -#if defined(nginx_version) && (nginx_version >= 1009002) +#if (nginx_version >= 1009002) if (dump) { dump->last = ngx_cpymem(dump->last, b->start + len, size); } diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 6c248d3b47..10fbbe8ed8 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -22,7 +22,7 @@ static int ngx_http_lua_ngx_req_raw_header(lua_State *L); static int ngx_http_lua_ngx_req_header_set_helper(lua_State *L); static int ngx_http_lua_ngx_resp_get_headers(lua_State *L); static int ngx_http_lua_ngx_req_header_set(lua_State *L); -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) void ngx_http_lua_ngx_raw_header_cleanup(void *data); #endif @@ -102,7 +102,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) size_t size; ngx_buf_t *b, *first = NULL; ngx_int_t i, j; -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) ngx_buf_t **bb; ngx_chain_t *cl; ngx_http_lua_main_conf_t *lmcf; @@ -123,7 +123,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) return luaL_error(L, "no request object found"); } -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); #endif @@ -143,7 +143,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) dd("hc->nbusy: %d", (int) hc->nbusy); if (hc->nbusy) { -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) dd("hc->busy: %p %p %p %p", hc->busy->buf->start, hc->busy->buf->pos, hc->busy->buf->last, hc->busy->buf->end); #else @@ -185,7 +185,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) dd("size: %d", (int) size); if (hc->nbusy) { -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) if (hc->nbusy > lmcf->busy_buf_ptr_count) { if (lmcf->busy_buf_ptrs) { ngx_free(lmcf->busy_buf_ptrs); @@ -208,7 +208,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) #endif b = NULL; -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) bb = lmcf->busy_buf_ptrs; for (i = hc->nbusy; i > 0; i--) { b = bb[i - 1]; @@ -291,7 +291,7 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) if (hc->nbusy) { -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) bb = lmcf->busy_buf_ptrs; for (i = hc->nbusy - 1; i >= 0; i--) { b = bb[i]; @@ -1138,7 +1138,7 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, } -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) void ngx_http_lua_ngx_raw_header_cleanup(void *data) { diff --git a/src/ngx_http_lua_headers.h b/src/ngx_http_lua_headers.h index 1cb0b1dec7..679136169a 100644 --- a/src/ngx_http_lua_headers.h +++ b/src/ngx_http_lua_headers.h @@ -15,7 +15,7 @@ void ngx_http_lua_inject_resp_header_api(lua_State *L); void ngx_http_lua_inject_req_header_api(lua_State *L); void ngx_http_lua_create_headers_metatable(ngx_log_t *log, lua_State *L); -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) void ngx_http_lua_ngx_raw_header_cleanup(void *data); #endif diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 4b59721eb0..659a89f4f9 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -106,7 +106,7 @@ static ngx_http_lua_set_header_t ngx_http_lua_set_handlers[] = { offsetof(ngx_http_headers_out_t, cache_control), ngx_http_set_builtin_multi_header }, -#if defined(nginx_version) && nginx_version >= 1013009 +#if (nginx_version >= 1013009) { ngx_string("Link"), offsetof(ngx_http_headers_out_t, link), ngx_http_set_builtin_multi_header }, diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index dfd4e9cd23..5b34528076 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -194,7 +194,7 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) return NGX_ERROR; } -#if defined(nginx_version) && nginx_version >= 1009011 +#if (nginx_version >= 1009011) modules = cycle->modules; #else modules = ngx_modules; @@ -294,7 +294,7 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && (nginx_version >= 1009000) +#if (nginx_version >= 1009000) ngx_set_connection_log(r->connection, clcf->error_log); #else diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 6603b1965e..7957ade545 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -748,7 +748,7 @@ ngx_http_lua_init(ngx_conf_t *cf) ngx_http_lua_pipe_init(); #endif -#if defined(nginx_version) && (nginx_version >= 1011011) +#if (nginx_version >= 1011011) cln = ngx_pool_cleanup_add(cf->pool, 0); if (cln == NULL) { return NGX_ERROR; diff --git a/src/ngx_http_lua_ndk.c b/src/ngx_http_lua_ndk.c index 5daeeeb9cb..72ff7de815 100644 --- a/src/ngx_http_lua_ndk.c +++ b/src/ngx_http_lua_ndk.c @@ -46,7 +46,7 @@ ngx_http_lookup_ndk_set_var_directive(u_char *name, ngx_module_t **modules; ngx_command_t *cmd; -#if defined(nginx_version) && nginx_version >= 1009011 +#if (nginx_version >= 1009011) modules = ngx_cycle->modules; #else modules = ngx_modules; diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 2fd2a94c13..07effa88e2 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2079,7 +2079,7 @@ ngx_http_lua_ffi_shdict_capacity(ngx_shm_zone_t *zone) } -# if nginx_version >= 1011007 +#if (nginx_version >= 1011007) size_t ngx_http_lua_ffi_shdict_free_space(ngx_shm_zone_t *zone) { @@ -2094,7 +2094,7 @@ ngx_http_lua_ffi_shdict_free_space(ngx_shm_zone_t *zone) return bytes; } -# endif /* nginx_version >= 1011007 */ +#endif /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index b3a986c5bc..efd5875fc0 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1839,7 +1839,7 @@ ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) goto failed; } -#if defined(nginx_version) && nginx_version >= 1007000 +#if (nginx_version >= 1007000) if (u->ssl_name.len && ngx_ssl_check_host(c, &u->ssl_name) != NGX_OK) @@ -3758,7 +3758,7 @@ ngx_http_lua_socket_tcp_finalize_read_part(ngx_http_request_t *r, ngx_del_event(c->read, NGX_READ_EVENT, NGX_CLOSE_EVENT); } -#if defined(nginx_version) && nginx_version >= 1007005 +#if (nginx_version >= 1007005) if (c->read->posted) { #else if (c->read->prev) { @@ -3811,7 +3811,7 @@ ngx_http_lua_socket_tcp_finalize_write_part(ngx_http_request_t *r, ngx_del_event(c->write, NGX_WRITE_EVENT, NGX_CLOSE_EVENT); } -#if defined(nginx_version) && nginx_version >= 1007005 +#if (nginx_version >= 1007005) if (c->write->posted) { #else if (c->write->prev) { diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index dc7a95d686..6106c6946c 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -245,7 +245,7 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && (nginx_version >= 1009000) +#if (nginx_version >= 1009000) ngx_set_connection_log(fc, clcf->error_log); #else diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index eddb80f213..c54e8eadd8 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -267,7 +267,7 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && (nginx_version >= 1009000) +#if (nginx_version >= 1009000) ngx_set_connection_log(fc, clcf->error_log); #else diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 34e7474b82..1067b3bb2e 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -229,7 +229,7 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && (nginx_version >= 1009000) +#if (nginx_version >= 1009000) ngx_set_connection_log(fc, clcf->error_log); #else diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 14ad3e3079..19a88d995f 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1459,7 +1459,7 @@ ngx_http_lua_subrequest(ngx_http_request_t *r, ngx_http_request_t *sr; ngx_http_core_srv_conf_t *cscf; -#if defined(nginx_version) && (nginx_version >= 1009005) +#if (nginx_version >= 1009005) if (r->subrequests == 0) { #if defined(NGX_DTRACE) && NGX_DTRACE @@ -1578,7 +1578,7 @@ ngx_http_lua_subrequest(ngx_http_request_t *r, sr->uri_changes = NGX_HTTP_MAX_URI_CHANGES + 1; -#if defined(nginx_version) && (nginx_version >= 1009005) +#if (nginx_version >= 1009005) sr->subrequests = r->subrequests - 1; #endif diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 99a461da89..347187ce3c 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -622,7 +622,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); -#if defined(nginx_version) && (nginx_version >= 1009000) +#if (nginx_version >= 1009000) ngx_set_connection_log(r->connection, clcf->error_log); #else From 7c5b11d73785fa72291e3280e293b3e0cfab57db Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 2 Dec 2019 14:51:32 -0800 Subject: [PATCH 373/848] tests: 124-init-worker.t: fixed a few failing test cases. --- t/124-init-worker.t | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 1a82ef23b2..1e73aa6fcd 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -791,6 +791,10 @@ lua close the global Lua VM \1 lua close the global Lua VM \2 in the cache helper process \d+ lua close the global Lua VM \2 lua close the global Lua VM \2 +|lua close the global Lua VM ([0-9A-F]+) +lua close the global Lua VM \3 in the cache helper process \d+ +lua close the global Lua VM \3 +lua close the global Lua VM \3 in the cache helper process \d+ )(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- no_error_log @@ -829,6 +833,10 @@ lua close the global Lua VM \1 lua close the global Lua VM \2 in the cache helper process \d+ lua close the global Lua VM \2 lua close the global Lua VM \2 +|lua close the global Lua VM ([0-9A-F]+) +lua close the global Lua VM \3 in the cache helper process \d+ +lua close the global Lua VM \3 +lua close the global Lua VM \3 in the cache helper process \d+ )(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- no_error_log @@ -868,6 +876,10 @@ lua close the global Lua VM \1 lua close the global Lua VM \2 in the cache helper process \d+ lua close the global Lua VM \2 lua close the global Lua VM \2 +|lua close the global Lua VM ([0-9A-F]+) +lua close the global Lua VM \3 in the cache helper process \d+ +lua close the global Lua VM \3 +lua close the global Lua VM \3 in the cache helper process \d+ )(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- error_log eval From 760f7073da87da23bb4f024943089edfad35c72e Mon Sep 17 00:00:00 2001 From: spacewander Date: Thu, 25 Oct 2018 10:15:11 +0800 Subject: [PATCH 374/848] feature: implemented the FFI interface for the 'ngx_req.add_header()' API. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_headers.c | 70 ++++++++++++++++++++++++++++++-------- 1 file changed, 55 insertions(+), 15 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 10fbbe8ed8..60b58d44c9 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -955,11 +955,15 @@ ngx_http_lua_ffi_set_resp_header(ngx_http_request_t *r, const u_char *key_data, int -ngx_http_lua_ffi_req_header_set_single_value(ngx_http_request_t *r, - const u_char *key, size_t key_len, const u_char *value, size_t value_len) +ngx_http_lua_ffi_req_set_header(ngx_http_request_t *r, const u_char *key, + size_t key_len, const u_char *value, size_t value_len, + ngx_http_lua_ffi_str_t *mvals, size_t mvals_len, int override, + char **errmsg) { - ngx_str_t k; - ngx_str_t v; + u_char *p; + size_t len; + ngx_uint_t i; + ngx_str_t k, v; if (r->connection->fd == (ngx_socket_t) -1) { /* fake request */ return NGX_HTTP_LUA_FFI_BAD_CONTEXT; @@ -971,35 +975,71 @@ ngx_http_lua_ffi_req_header_set_single_value(ngx_http_request_t *r, k.data = ngx_palloc(r->pool, key_len + 1); if (k.data == NULL) { - return NGX_ERROR; + goto nomem; } + ngx_memcpy(k.data, key, key_len); k.data[key_len] = '\0'; - k.len = key_len; - if (value_len == 0) { + if (mvals) { + if (mvals_len > 0) { + for (i = 0; i < mvals_len; i++) { + p = mvals[i].data; + len = mvals[i].len; + + v.data = ngx_palloc(r->pool, len + 1); + if (v.data == NULL) { + goto nomem; + } + + ngx_memcpy(v.data, p, len); + v.data[len] = '\0'; + v.len = len; + + if (ngx_http_lua_set_input_header(r, k, v, override) + != NGX_OK) + { + goto failed; + } + } + + return NGX_OK; + } + v.data = NULL; v.len = 0; - } else { + } else if (value) { v.data = ngx_palloc(r->pool, value_len + 1); if (v.data == NULL) { - return NGX_ERROR; + goto nomem; } + ngx_memcpy(v.data, value, value_len); v.data[value_len] = '\0'; - } + v.len = value_len; - v.len = value_len; + } else { + v.data = NULL; + v.len = 0; + } - if (ngx_http_lua_set_input_header(r, k, v, 1 /* override */) - != NGX_OK) - { - return NGX_ERROR; + if (ngx_http_lua_set_input_header(r, k, v, override) != NGX_OK) { + goto failed; } return NGX_OK; + +nomem: + + *errmsg = "no memory"; + return NGX_ERROR; + +failed: + + *errmsg = "failed to set header"; + return NGX_ERROR; } From 8f518e7d92314b961cada4c9ba9e45d15f7b25f7 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 6 Jan 2020 14:26:38 -0800 Subject: [PATCH 375/848] travis-ci: bumped OpenSSL to 1.0.2u. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 377b15652e..61e3a2439b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,7 +48,7 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.17.4 OPENSSL_VER=1.0.2t OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.17.4 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.17.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.17.4 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c From e3286454308863f4ea8d0090c900da3ab74bd1fa Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 13 Jan 2020 16:32:49 -0800 Subject: [PATCH 376/848] change: we now ignore 'lua_regex_*' directives when NGINX is compiled without PCRE support. --- src/ngx_http_lua_directive.c | 24 ++++++++++++++++++++++++ src/ngx_http_lua_directive.h | 4 ++++ src/ngx_http_lua_module.c | 14 ++++++++++---- 3 files changed, 38 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 80c4b7be58..9958714187 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -232,6 +232,30 @@ ngx_http_lua_package_path(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) } +char * +ngx_http_lua_regex_cache_max_entries(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ +#if (NGX_PCRE) + return ngx_conf_set_num_slot(cf, cmd, conf); +#else + return NGX_CONF_OK; +#endif +} + + +char * +ngx_http_lua_regex_match_limit(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ +#if (NGX_PCRE) + return ngx_conf_set_num_slot(cf, cmd, conf); +#else + return NGX_CONF_OK; +#endif +} + + #if defined(NDK) && NDK char * ngx_http_lua_set_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index c72c90dbe7..8555e2f364 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -17,6 +17,10 @@ char *ngx_http_lua_package_cpath(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_package_path(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +char *ngx_http_lua_regex_cache_max_entries(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); +char *ngx_http_lua_regex_match_limit(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); char *ngx_http_lua_content_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 7957ade545..5303ae76f8 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -119,21 +119,27 @@ static ngx_command_t ngx_http_lua_cmds[] = { offsetof(ngx_http_lua_main_conf_t, set_sa_restart), NULL }, -#if (NGX_PCRE) { ngx_string("lua_regex_cache_max_entries"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, - ngx_conf_set_num_slot, + ngx_http_lua_regex_cache_max_entries, NGX_HTTP_MAIN_CONF_OFFSET, +#if (NGX_PCRE) offsetof(ngx_http_lua_main_conf_t, regex_cache_max_entries), +#else + 0, +#endif NULL }, { ngx_string("lua_regex_match_limit"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, - ngx_conf_set_num_slot, + ngx_http_lua_regex_match_limit, NGX_HTTP_MAIN_CONF_OFFSET, +#if (NGX_PCRE) offsetof(ngx_http_lua_main_conf_t, regex_match_limit), - NULL }, +#else + 0, #endif + NULL }, { ngx_string("lua_package_cpath"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, From c2390abb584e7aa69894e7d8a6acc7dd7879a753 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 17 Jan 2020 14:46:53 -0800 Subject: [PATCH 377/848] Revert "bugfix: ensured 'init_worker_by_lua*' does not mutate another NGINX module's main_conf." This reverts commit 09484e8fda2f1c1970a898a8c08111af13d69930. --- src/ngx_http_lua_initworkerby.c | 22 +-- t/124-init-worker.t | 22 --- t/data/fake-merge-module/config | 3 - .../ngx_http_fake_merge_module.c | 187 ------------------ util/build.sh | 2 - 5 files changed, 1 insertion(+), 235 deletions(-) delete mode 100644 t/data/fake-merge-module/config delete mode 100644 t/data/fake-merge-module/ngx_http_fake_merge_module.c diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 5b34528076..840f14778e 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -86,6 +86,7 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) } conf_ctx = (ngx_http_conf_ctx_t *) cycle->conf_ctx[ngx_http_module.index]; + http_ctx.main_conf = conf_ctx->main_conf; top_clcf = conf_ctx->loc_conf[ngx_http_core_module.ctx_index]; top_llcf = conf_ctx->loc_conf[ngx_http_lua_module.ctx_index]; @@ -188,12 +189,6 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) return NGX_ERROR; } - http_ctx.main_conf = ngx_pcalloc(conf.pool, - sizeof(void *) * ngx_http_max_module); - if (http_ctx.main_conf == NULL) { - return NGX_ERROR; - } - #if (nginx_version >= 1009011) modules = cycle->modules; #else @@ -207,21 +202,6 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) module = modules[i]->ctx; - if (module->create_main_conf) { - cur = module->create_main_conf(&conf); - if (cur == NULL) { - return NGX_ERROR; - } - - if (ngx_modules[i]->index == ngx_http_lua_module.index) { - ngx_memcpy(cur, - conf_ctx->main_conf[ngx_http_lua_module.ctx_index], - sizeof(ngx_http_lua_main_conf_t)); - } - - http_ctx.main_conf[modules[i]->ctx_index] = cur; - } - if (module->create_srv_conf) { cur = module->create_srv_conf(&conf); if (cur == NULL) { diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 1e73aa6fcd..4340893878 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -967,25 +967,3 @@ qr/lua close the global Lua VM ([0-9A-F]+)$/, --- no_error_log [error] start privileged agent process - - - -=== TEST 25: ensure it does not mutate another module's main_conf (github issue #1553) -https://github.com/openresty/lua-nginx-module/issues/1553 ---- http_config - init_worker_by_lua_block { - return - } ---- config - location /t { - content_by_lua_block { - ngx.say("fake_var = ", ngx.var.fake_var) - } - } ---- request -GET /t ---- response_body -fake_var = 1 ---- no_error_log -[error] -[alert] diff --git a/t/data/fake-merge-module/config b/t/data/fake-merge-module/config deleted file mode 100644 index f6dbf60e7b..0000000000 --- a/t/data/fake-merge-module/config +++ /dev/null @@ -1,3 +0,0 @@ -ngx_addon_name=ngx_http_fake_merge_module -HTTP_MODULES="$HTTP_MODULES ngx_http_fake_merge_module" -NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fake_merge_module.c" diff --git a/t/data/fake-merge-module/ngx_http_fake_merge_module.c b/t/data/fake-merge-module/ngx_http_fake_merge_module.c deleted file mode 100644 index 579f62332c..0000000000 --- a/t/data/fake-merge-module/ngx_http_fake_merge_module.c +++ /dev/null @@ -1,187 +0,0 @@ -/* - * This fake module was used to reproduce a bug in ngx_lua's init_worker_by_lua - * implementation. The bug would cause this module's main_conf->a to be reset - * to 0 due to init_worker_by_lua invoking merge_loc_conf with a brand new - * loc_conf, whose a member would be 0. - * The test case for this bug is in 124-init-worker.t. - */ - - -#include -#include -#include -#include - - -typedef struct { - ngx_flag_t a; -} ngx_http_fake_merge_main_conf_t; - - -typedef struct { - ngx_flag_t a; -} ngx_http_fake_merge_loc_conf_t; - - -static ngx_int_t ngx_http_fake_merge_var(ngx_http_request_t *r, - ngx_http_variable_value_t *v, uintptr_t data); -static ngx_int_t ngx_http_fake_merge_add_variables(ngx_conf_t *cf); -static ngx_int_t ngx_http_fake_merge_init(ngx_conf_t *cf); -static void *ngx_http_fake_merge_create_main_conf(ngx_conf_t *cf); -static void *ngx_http_fake_merge_create_loc_conf(ngx_conf_t *cf); -static char *ngx_http_fake_merge_merge_loc_conf(ngx_conf_t *cf, void *prev, - void *conf); - - -static ngx_http_module_t ngx_http_fake_merge_module_ctx = { - ngx_http_fake_merge_init, /* preconfiguration */ - NULL, /* postconfiguration */ - - ngx_http_fake_merge_create_main_conf, /* create main configuration */ - NULL, /* init main configuration */ - - NULL, /* create server configuration */ - NULL, /* merge server configuration */ - - ngx_http_fake_merge_create_loc_conf, /* create location configuration */ - ngx_http_fake_merge_merge_loc_conf /* merge location configuration */ -}; - - -ngx_module_t ngx_http_fake_merge_module = { - NGX_MODULE_V1, - &ngx_http_fake_merge_module_ctx, /* module context */ - NULL, /* module directives */ - NGX_HTTP_MODULE, /* module type */ - NULL, /* init master */ - NULL, /* init module */ - NULL, /* init process */ - NULL, /* init thread */ - NULL, /* exit thread */ - NULL, /* exit process */ - NULL, /* exit master */ - NGX_MODULE_V1_PADDING -}; - - -static ngx_http_variable_t ngx_http_fake_merge_variables[] = { - - { ngx_string("fake_var"), NULL, - ngx_http_fake_merge_var, 0, - NGX_HTTP_VAR_NOCACHEABLE, 0 }, - - { ngx_null_string, NULL, NULL, 0, 0, 0 } -}; - - -static ngx_int_t -ngx_http_fake_merge_var(ngx_http_request_t *r, ngx_http_variable_value_t *v, - uintptr_t data) -{ - static char *str[] = {"0", "1"}; - ngx_http_fake_merge_main_conf_t *fmcf; - - fmcf = ngx_http_get_module_main_conf(r, ngx_http_fake_merge_module); - if (fmcf == NULL) { - return NGX_ERROR; - } - - v->len = 1; - v->data = (u_char *) str[fmcf->a]; - v->valid = 1; - v->no_cacheable = 0; - v->not_found = 0; - - return NGX_OK; -} - - -static ngx_int_t -ngx_http_fake_merge_add_variables(ngx_conf_t *cf) -{ - ngx_http_variable_t *var, *v; - - for (v = ngx_http_fake_merge_variables; v->name.len; v++) { - var = ngx_http_add_variable(cf, &v->name, v->flags); - if (var == NULL) { - return NGX_ERROR; - } - - var->get_handler = v->get_handler; - var->data = v->data; - } - - return NGX_OK; -} - - -static ngx_int_t -ngx_http_fake_merge_init(ngx_conf_t *cf) -{ - ngx_http_fake_merge_loc_conf_t *flcf; - - flcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_fake_merge_module); - if (flcf == NULL) { - return NGX_ERROR; - } - - if (ngx_http_fake_merge_add_variables(cf) != NGX_OK) { - return NGX_ERROR; - } - - flcf->a = 1; - - return NGX_OK; -} - - -static void * -ngx_http_fake_merge_create_main_conf(ngx_conf_t *cf) -{ - ngx_http_fake_merge_main_conf_t *fmcf; - - fmcf = ngx_pcalloc(cf->pool, sizeof(ngx_http_fake_merge_main_conf_t)); - if (fmcf == NULL) { - return NULL; - } - - fmcf->a = NGX_CONF_UNSET; - - return fmcf; -} - - -static void * -ngx_http_fake_merge_create_loc_conf(ngx_conf_t *cf) -{ - ngx_http_fake_merge_loc_conf_t *flcf; - - flcf = ngx_pcalloc(cf->pool, sizeof(ngx_http_fake_merge_loc_conf_t)); - if (flcf == NULL) { - return NULL; - } - - flcf->a = NGX_CONF_UNSET; - - return flcf; -} - - -static char * -ngx_http_fake_merge_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) -{ - ngx_http_fake_merge_loc_conf_t *conf = child; - ngx_http_fake_merge_loc_conf_t *prev = parent; - ngx_http_fake_merge_main_conf_t *fmcf; - - ngx_conf_merge_value(conf->a, prev->a, 0); - - fmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_fake_merge_module); - if (fmcf == NULL) { - return NGX_CONF_ERROR; - } - - fmcf->a = conf->a; - - return NGX_CONF_OK; -} diff --git a/util/build.sh b/util/build.sh index 5f9b1d5e07..14e9fd46d9 100755 --- a/util/build.sh +++ b/util/build.sh @@ -23,7 +23,6 @@ force=$2 #--with-http_spdy_module \ add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" -add_fake_merge_module="--add-module=$root/t/data/fake-merge-module" time ngx-build $force $version \ --with-pcre-jit \ @@ -59,7 +58,6 @@ time ngx-build $force $version \ --add-module=$root/../stream-lua-nginx-module \ --add-module=$root/t/data/fake-module \ $add_fake_shm_module \ - $add_fake_merge_module \ --add-module=$root/t/data/fake-delayed-load-module \ --with-http_gunzip_module \ --with-http_dav_module \ From d328e5620eaebe3e61fb7ce795a39ede5ce81f46 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 22 Jan 2020 14:20:25 -0800 Subject: [PATCH 378/848] tests: 025-codecache.t: updated a test case to avoid error logs in stderr. --- t/025-codecache.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/025-codecache.t b/t/025-codecache.t index e162b3bc2e..3e01bb88cf 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1689,7 +1689,7 @@ grep me: b ssl_session_store_by_lua_block { ngx.log(ngx.INFO, "hello") } upstream backend { - server 0.0.0.1; + server unix:$TEST_NGINX_HTML_DIR/nginx.sock; balancer_by_lua_block { ngx.log(ngx.INFO, "hello") } } From c06793c3e1431b24332b8b0239cc43df43361941 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 22 Jan 2020 14:22:12 -0800 Subject: [PATCH 379/848] travis-ci: bumped the NGINX core to 1.17.8. --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 61e3a2439b..384d105cd4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,9 +48,9 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 matrix: - - NGINX_VERSION=1.17.4 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.17.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.4 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c + - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c services: - memcache From 028c0d4951c7ad99dabf32b3a931d8caac7a5c63 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 22 Jan 2020 15:16:33 -0800 Subject: [PATCH 380/848] doc: mentioned that 1.17.8 is the latest NGINX core version tested. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 323f4f0c0a..2adfd1a775 100644 --- a/README.markdown +++ b/README.markdown @@ -284,7 +284,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: -* 1.17.x (last tested: 1.17.4) +* 1.17.x (last tested: 1.17.8) * 1.15.x (last tested: 1.15.8) * 1.14.x * 1.13.x (last tested: 1.13.6) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index af4c0c1598..cf9db47320 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -221,7 +221,7 @@ performance level. The latest version of this module is compatible with the following versions of Nginx: -* 1.17.x (last tested: 1.17.4) +* 1.17.x (last tested: 1.17.8) * 1.15.x (last tested: 1.15.8) * 1.14.x * 1.13.x (last tested: 1.13.6) From 1d9298d6439ee0a24ddbea37264e10a8ff5ee9a9 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 5 Feb 2020 15:00:02 -0800 Subject: [PATCH 381/848] bugfix: fixed use of an uninitialized value in ngx_http_lua_set_by_lua_file. Caught by Valgrind: ==1136586== Use of uninitialised value of size 8 ==1136586== at 0x48B3C55: lj_tab_getinth (lj_tab.c:411) ==1136586== by 0x48CBC42: lua_rawgeti (lj_api.c:823) ==1136586== by 0x5823D6: ngx_http_lua_cache_load_code (ngx_http_lua_cache.c:84) ==1136586== by 0x5829F3: ngx_http_lua_cache_loadfile (ngx_http_lua_cache.c:286) ==1136586== by 0x576A9E: ngx_http_lua_filter_set_by_lua_file (ngx_http_lua_directive.c:457) ==1136586== by 0x549361: ndk_set_var_multi_value_data_code (ndk_set_var.c:215) ==1136586== by 0x4FA637: ngx_http_rewrite_handler (ngx_http_rewrite_module.c:180) ==1136586== by 0x481491: ngx_http_core_rewrite_phase (ngx_http_core_module.c:922) ==1136586== by 0x4812ED: ngx_http_core_run_phases (ngx_http_core_module.c:868) ==1136586== by 0x48125A: ngx_http_handler (ngx_http_core_module.c:851) ==1136586== by 0x4907F3: ngx_http_process_request (ngx_http_request.c:2055) ==1136586== by 0x48F250: ngx_http_process_request_headers (ngx_http_request.c:1480) ==1136586== by 0x48E62F: ngx_http_process_request_line (ngx_http_request.c:1151) ==1136586== by 0x48CEA3: ngx_http_wait_request_handler (ngx_http_request.c:500) ==1136586== by 0x46BA45: ngx_epoll_process_events (ngx_epoll_module.c:901) ==1136586== by 0x459066: ngx_process_events_and_timers (ngx_event.c:257) ==1136586== by 0x467988: ngx_single_process_cycle (ngx_process_cycle.c:333) ==1136586== by 0x4236A9: main (nginx.c:382) ==1136586== { Memcheck:Value8 fun:lj_tab_getinth fun:lua_rawgeti fun:ngx_http_lua_cache_load_code fun:ngx_http_lua_cache_loadfile fun:ngx_http_lua_filter_set_by_lua_file fun:ndk_set_var_multi_value_data_code fun:ngx_http_rewrite_handler fun:ngx_http_core_rewrite_phase fun:ngx_http_core_run_phases fun:ngx_http_handler fun:ngx_http_process_request fun:ngx_http_process_request_headers fun:ngx_http_process_request_line fun:ngx_http_wait_request_handler fun:ngx_epoll_process_events fun:ngx_process_events_and_timers fun:ngx_single_process_cycle fun:main } --- src/ngx_http_lua_directive.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 9958714187..0a88366fe6 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -353,6 +353,7 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } + filter_data->ref = LUA_REFNIL; filter_data->size = filter.size; ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); From 743773bfb0bc5d79305efb2de295554e1560eb53 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 6 Feb 2020 13:49:58 -0800 Subject: [PATCH 382/848] tests: fixed a newly failing test due to openresty.org now rejecting TLSv1 connections. --- t/129-ssl-socket.t | 39 +++++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 454852d7da..58d1b9c876 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1,16 +1,18 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: use Test::Nginx::Socket::Lua; +use Cwd qw(abs_path realpath); +use File::Basename; repeat_each(2); plan tests => repeat_each() * 211; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); - $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_SERVER_SSL_PORT} ||= 12345; +$ENV{TEST_NGINX_CERT_DIR} ||= dirname(realpath(abs_path(__FILE__))); #log_level 'warn'; log_level 'debug'; @@ -1190,20 +1192,31 @@ SSL reused session === TEST 15: explicit ssl protocol configuration +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols TLSv1; + + location / { + content_by_lua_block { + ngx.exit(200) + } + } + } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_protocols TLSv1; - location /t { - #set $port 5000; - set $port $TEST_NGINX_MEMCACHED_PORT; + location /t { content_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(2000) do - local ok, err = sock:connect("openresty.org", 443) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -1211,7 +1224,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "openresty.org") + local session, err = sock:sslhandshake(nil, "test.com") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1219,7 +1232,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: test.com\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1242,14 +1255,13 @@ SSL reused session collectgarbage() '; } - --- request GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 58 bytes. -received: HTTP/1.1 302 Moved Temporarily +sent http request: 53 bytes. +received: HTTP/1.1 200 OK close: 1 nil --- log_level: debug @@ -1259,13 +1271,12 @@ qr/^lua ssl save session: ([0-9A-F]+) lua ssl free session: ([0-9A-F]+) $/ --- error_log eval -['lua ssl server name: "openresty.org"', -qr/SSL: TLSv1, cipher: "ECDHE-RSA-AES128-SHA (SSLv3|TLSv1)/] +['lua ssl server name: "test.com"', +qr/SSL: TLSv1, cipher: "ECDHE-RSA-AES256-SHA (SSLv3|TLSv1)/] --- no_error_log SSL reused session [error] [alert] ---- timeout: 5 From 8669e0985d683553aa9af34066eb8c292926d6c5 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 12 Feb 2020 15:17:20 -0800 Subject: [PATCH 383/848] bugfix: ngx.pipe: ensured reading while a process died returns the "closed" error. As per the ngx.pipe documentation: If the reading operation is aborted by the shutdown method, it will return nil and the error string "aborted". [...] Reading from an exited process's stream will return nil and the error string "closed". However, prior to this patch, a process that exits while reads are pending would make `std*_read_*()` calls return the "aborted" error, when they should have returned "closed". This was caught by the MOCKEAGAIN=r testing mode: t/pipe-stdout.t .. 1/18 Failed test 'TEST 28: read process while waiting process in other request - response_body - response is expected @@ -1,3 +1,3 @@ -hello world +aborted exit 2 --- src/ngx_http_lua_pipe.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 596412c6ec..1a91b9514f 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -30,7 +30,7 @@ static ssize_t ngx_http_lua_pipe_fd_read(ngx_connection_t *c, u_char *buf, size_t size); static ssize_t ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, size_t size); -static void ngx_http_lua_pipe_close_helper( +static void ngx_http_lua_pipe_close_helper(ngx_http_lua_pipe_t *pipe, ngx_http_lua_pipe_ctx_t *pipe_ctx, ngx_event_t *ev); static void ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe); static void ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe); @@ -912,15 +912,21 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, static void -ngx_http_lua_pipe_close_helper(ngx_http_lua_pipe_ctx_t *pipe_ctx, - ngx_event_t *ev) +ngx_http_lua_pipe_close_helper(ngx_http_lua_pipe_t *pipe, + ngx_http_lua_pipe_ctx_t *pipe_ctx, ngx_event_t *ev) { if (ev->handler != ngx_http_lua_pipe_dummy_event_handler) { ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua pipe abort blocking operation pipe_ctx:%p ev:%p", pipe_ctx, ev); - pipe_ctx->err_type = PIPE_ERR_ABORTED; + if (pipe->dead) { + pipe_ctx->err_type = PIPE_ERR_CLOSED; + + } else { + pipe_ctx->err_type = PIPE_ERR_ABORTED; + } + ngx_post_event(ev, &ngx_posted_events); return; } @@ -947,7 +953,7 @@ ngx_http_lua_pipe_close_stdin(ngx_http_lua_pipe_t *pipe) } else if (pipe->stdin_ctx->c != NULL) { wev = pipe->stdin_ctx->c->write; - ngx_http_lua_pipe_close_helper(pipe->stdin_ctx, wev); + ngx_http_lua_pipe_close_helper(pipe, pipe->stdin_ctx, wev); } } @@ -969,7 +975,7 @@ ngx_http_lua_pipe_close_stdout(ngx_http_lua_pipe_t *pipe) } else if (pipe->stdout_ctx->c != NULL) { rev = pipe->stdout_ctx->c->read; - ngx_http_lua_pipe_close_helper(pipe->stdout_ctx, rev); + ngx_http_lua_pipe_close_helper(pipe, pipe->stdout_ctx, rev); } } @@ -991,7 +997,7 @@ ngx_http_lua_pipe_close_stderr(ngx_http_lua_pipe_t *pipe) } else if (pipe->stderr_ctx->c != NULL) { rev = pipe->stderr_ctx->c->read; - ngx_http_lua_pipe_close_helper(pipe->stderr_ctx, rev); + ngx_http_lua_pipe_close_helper(pipe, pipe->stderr_ctx, rev); } } From 9259e7a3435976b07bc20997c9fb7c9b9e257357 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Thu, 13 Feb 2020 13:15:37 -0800 Subject: [PATCH 384/848] tests: 139-ssl-cert-by.t: relaxed timing test cases for tvw test mode. --- t/139-ssl-cert-by.t | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 2344b63b1e..17a4927cce 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -218,7 +218,7 @@ close: 1 nil --- error_log eval [ 'lua ssl server name: "test.com"', -qr/elapsed in ssl cert by lua: 0.(?:09|1[01])\d+,/, +qr/elapsed in ssl cert by lua: 0.(?:09|1\d)\d+,/, ] --- no_error_log @@ -2131,12 +2131,12 @@ GET /t connected: 1 ssl handshake: boolean --- grep_error_log eval -qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1\d)\d+,/, --- grep_error_log_out eval [ -qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, -qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, -qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1[01])\d+,/, +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1\d)\d+,/, +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1\d)\d+,/, +qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1\d)\d+,/, ] --- no_error_log [error] From 6b79cd44132bb565f2778979cdbbf34a5c4919dd Mon Sep 17 00:00:00 2001 From: Zhenzhen Zhao <229194950@qq.com> Date: Sun, 16 Feb 2020 11:33:33 +0800 Subject: [PATCH 385/848] doc: minor grammar typo fix (#1642) (#1643) --- doc/HttpLuaModule.wiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index cf9db47320..886734b7a6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4662,7 +4662,7 @@ Since v0.8.3 this function returns 1 on success, or re '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' -Sleeps for the specified seconds without blocking. One can specify time resolution up to 0.001 seconds (i.e., one milliseconds). +Sleeps for the specified seconds without blocking. One can specify time resolution up to 0.001 seconds (i.e., one millisecond). Behind the scene, this method makes use of the Nginx timers. From 52f0d95fa4cc715d4ef437624b5431be6d465970 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Mon, 17 Feb 2020 01:13:53 +0800 Subject: [PATCH 386/848] doc: sync the minor change (in the last commit) to README.markdown from the HttpLuaModule.wiki file. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 2adfd1a775..e3f8720b90 100644 --- a/README.markdown +++ b/README.markdown @@ -5554,7 +5554,7 @@ ngx.sleep **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** -Sleeps for the specified seconds without blocking. One can specify time resolution up to 0.001 seconds (i.e., one milliseconds). +Sleeps for the specified seconds without blocking. One can specify time resolution up to 0.001 seconds (i.e., one millisecond). Behind the scene, this method makes use of the Nginx timers. From 4c4bb0fcbd12fc79e5adc5fad5d454dc03be875f Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 25 Feb 2020 14:07:28 -0800 Subject: [PATCH 387/848] bugfix: fixed use of uninitialized cache_key in ngx_http_set_by_lua_file. Caught by running the 025-codecache.t test suite in HUP reload mode. --- src/ngx_http_lua_directive.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 0a88366fe6..8f7e7fbef3 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -325,7 +325,7 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) char * ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *cache_key; + u_char *cache_key = NULL; ngx_str_t *value; ngx_str_t target; ndk_set_var_t filter; @@ -353,9 +353,6 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } - filter_data->ref = LUA_REFNIL; - filter_data->size = filter.size; - ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); ccv.cf = cf; ccv.value = &value[2]; @@ -372,10 +369,12 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) if (cache_key == NULL) { return NGX_CONF_ERROR; } - - filter_data->key = cache_key; } + filter_data->key = cache_key; + filter_data->ref = LUA_REFNIL; + filter_data->size = filter.size; + ngx_str_null(&filter_data->script); filter.data = filter_data; From ec3b219e64f04590569c3c8397da8cb01ddd44fe Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 9 Mar 2020 14:33:42 -0700 Subject: [PATCH 388/848] tests: fixed a newly failing test due to changes in openresty.org accepted ciphers. --- t/129-ssl-socket.t | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 58d1b9c876..d7f9d6bc02 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1112,20 +1112,31 @@ SSL reused session === TEST 14: explicit cipher configuration +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols TLSv1; + + location / { + content_by_lua_block { + ngx.exit(200) + } + } + } --- config server_tokens off; - resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_ciphers ECDHE-RSA-AES256-SHA; - location /t { - #set $port 5000; - set $port $TEST_NGINX_MEMCACHED_PORT; + location /t { content_by_lua ' local sock = ngx.socket.tcp() sock:settimeout(2000) do - local ok, err = sock:connect("openresty.org", 443) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -1133,7 +1144,7 @@ SSL reused session ngx.say("connected: ", ok) - local session, err = sock:sslhandshake(nil, "openresty.org") + local session, err = sock:sslhandshake(nil, "test.com") if not session then ngx.say("failed to do SSL handshake: ", err) return @@ -1141,7 +1152,7 @@ SSL reused session ngx.say("ssl handshake: ", type(session)) - local req = "GET / HTTP/1.1\\r\\nHost: openresty.org\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: test.com\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -1164,14 +1175,13 @@ SSL reused session collectgarbage() '; } - --- request GET /t --- response_body connected: 1 ssl handshake: userdata -sent http request: 58 bytes. -received: HTTP/1.1 302 Moved Temporarily +sent http request: 53 bytes. +received: HTTP/1.1 200 OK close: 1 nil --- log_level: debug @@ -1181,8 +1191,8 @@ qr/^lua ssl save session: ([0-9A-F]+) lua ssl free session: ([0-9A-F]+) $/ --- error_log eval -['lua ssl server name: "openresty.org"', -qr/SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-SHA (SSLv3|TLSv1)/] +['lua ssl server name: "test.com"', +qr/SSL: TLSv\d(?:\.\d)?, cipher: "ECDHE-RSA-AES256-SHA (SSLv3|TLSv1)/] --- no_error_log SSL reused session [error] From 43220429e5411d3461445e25031864e6edffbebe Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 11 Mar 2020 10:54:45 -0700 Subject: [PATCH 389/848] tests: fixed numbering thanks to the reindex script. --- t/087-udp-socket.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index f6ef26067d..b3d5e824f0 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -1050,7 +1050,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ -=== TEST 20: the upper bound of port range should be 2^16 - 1 +=== TEST 19: the upper bound of port range should be 2^16 - 1 --- config location /t { content_by_lua_block { @@ -1070,7 +1070,7 @@ failed to connect: bad port number: 65536 -=== TEST 21: send boolean and nil +=== TEST 20: send boolean and nil --- config server_tokens off; location /t { From be864c2afd08222e28bd8d1e9bc05f8d94112a8d Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Wed, 18 Mar 2020 17:29:07 -0700 Subject: [PATCH 390/848] travis-ci: upgraded OpenSSL to 1.1.1e. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 384d105cd4..6de5511dff 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,7 +50,7 @@ env: matrix: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1d OPENSSL_PATCH_VER=1.1.1c + - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1e OPENSSL_PATCH_VER=1.1.1e services: - memcache From 38f587a35ef72b8cbb50114797fa875307ff0bfe Mon Sep 17 00:00:00 2001 From: doujiang Date: Fri, 20 Mar 2020 05:17:20 +0800 Subject: [PATCH 391/848] bugfix: ensured arguments of APIs mutating uri or request/response headers do not contain unsafe characters. (#1654) Signed-off-by: Thibault Charbonnier --- README.markdown | 20 +++--- doc/HttpLuaModule.wiki | 20 +++--- src/ngx_http_lua_control.c | 4 +- src/ngx_http_lua_headers_in.c | 6 ++ src/ngx_http_lua_headers_out.c | 8 ++- src/ngx_http_lua_uri.c | 59 ++++++++++++++++ src/ngx_http_lua_util.c | 119 +++++++++++++++++++++++++++++++++ src/ngx_http_lua_util.h | 17 +---- t/016-resp-header.t | 64 +++++++++++------- t/022-redirect.t | 58 +++++++++++----- t/028-req-header.t | 78 ++++++++++++++++++++- t/030-uri-args.t | 38 ++++++++++- t/113-req-header-cookie.t | 22 ++++++ 13 files changed, 433 insertions(+), 80 deletions(-) diff --git a/README.markdown b/README.markdown index e3f8720b90..6839efc6bd 100644 --- a/README.markdown +++ b/README.markdown @@ -4251,10 +4251,8 @@ to be returned when reading `ngx.header.Foo`. Note that `ngx.header` is not a normal Lua table and as such, it is not possible to iterate through it using the Lua `ipairs` function. -Note: `HEADER` and `VALUE` will be truncated if they -contain the `\r` or `\n` characters. The truncated values -will contain all characters up to (and excluding) the first occurrence of -`\r` or `\n`. +Note: this function throws a Lua error if `HEADER` or +`VALUE` contain unsafe characters (control characters). For reading *request* headers, use the [ngx.req.get_headers](#ngxreqget_headers) function instead. @@ -4499,6 +4497,9 @@ which is functionally equivalent to } ``` +Note: this function throws a Lua error if the `uri` argument +contains unsafe characters (control characters). + Note that it is not possible to use this interface to rewrite URI arguments and that [ngx.req.set_uri_args](#ngxreqset_uri_args) should be used for this instead. For instance, Nginx config ```nginx @@ -4914,6 +4915,9 @@ is equivalent to ngx.req.clear_header("X-Foo") ``` +Note: this function throws a Lua error if `header_name` or +`header_value` contain unsafe characters (control characters). + [Back to TOC](#nginx-api-for-lua) ngx.req.clear_header @@ -5247,12 +5251,8 @@ ngx.redirect Issue an `HTTP 301` or `302` redirection to `uri`. -Notice: the `uri` should not contains `\r` or `\n`, otherwise, the characters after `\r` or `\n` will be truncated, including the `\r` or `\n` bytes themself. - -The `uri` argument will be truncated if it contains the -`\r` or `\n` characters. The truncated value will contain -all characters up to (and excluding) the first occurrence of `\r` or -`\n`. +Note: this function throws a Lua error if the `uri` argument +contains unsafe characters (control characters). The optional `status` parameter specifies the HTTP status code to be used. The following status codes are supported right now: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 886734b7a6..91409d2109 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3533,10 +3533,8 @@ to be returned when reading ngx.header.Foo. Note that ngx.header is not a normal Lua table and as such, it is not possible to iterate through it using the Lua ipairs function. -Note: HEADER and VALUE will be truncated if they -contain the \r or \n characters. The truncated values -will contain all characters up to (and excluding) the first occurrence of -\r or \n. +Note: this function throws a Lua error if HEADER or +VALUE contain unsafe characters (control characters). For reading ''request'' headers, use the [[#ngx.req.get_headers|ngx.req.get_headers]] function instead. @@ -3746,6 +3744,9 @@ which is functionally equivalent to } +Note: this function throws a Lua error if the uri argument +contains unsafe characters (control characters). + Note that it is not possible to use this interface to rewrite URI arguments and that [[#ngx.req.set_uri_args|ngx.req.set_uri_args]] should be used for this instead. For instance, Nginx config @@ -4110,6 +4111,9 @@ is equivalent to ngx.req.clear_header("X-Foo") +Note: this function throws a Lua error if header_name or +header_value contain unsafe characters (control characters). + == ngx.req.clear_header == '''syntax:''' ''ngx.req.clear_header(header_name)'' @@ -4398,12 +4402,8 @@ It is recommended that a coding style that combines this method call with the HTTP 301 or 302 redirection to uri. -Notice: the uri should not contains \r or \n, otherwise, the characters after \r or \n will be truncated, including the \r or \n bytes themself. - -The uri argument will be truncated if it contains the -\r or \n characters. The truncated value will contain -all characters up to (and excluding) the first occurrence of \r or -\n. +Note: this function throws a Lua error if the uri argument +contains unsafe characters (control characters). The optional status parameter specifies the HTTP status code to be used. The following status codes are supported right now: diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 5ace17638f..2fd3d17478 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -239,7 +239,9 @@ ngx_http_lua_ngx_redirect(lua_State *L) "the headers"); } - len = ngx_http_lua_safe_header_value_len(p, len); + if (ngx_http_lua_check_header_safe(r, p, len) != NGX_OK) { + return luaL_error(L, "attempt to use unsafe uri"); + } uri = ngx_palloc(r->pool, len); if (uri == NULL) { diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 26739fa3eb..a284f028bf 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -658,6 +658,12 @@ ngx_http_lua_set_input_header(ngx_http_request_t *r, ngx_str_t key, dd("set header value: %.*s", (int) value.len, value.data); + if (ngx_http_lua_check_header_safe(r, key.data, key.len) != NGX_OK + || ngx_http_lua_check_header_safe(r, value.data, value.len) != NGX_OK) + { + return NGX_ERROR; + } + hv.hash = ngx_hash_key_lc(key.data, key.len); hv.key = key; diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 659a89f4f9..87f114b820 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -491,9 +491,11 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, dd("set header value: %.*s", (int) value.len, value.data); - key.len = ngx_http_lua_safe_header_value_len(key.data, key.len); - - value.len = ngx_http_lua_safe_header_value_len(value.data, value.len); + if (ngx_http_lua_check_header_safe(r, key.data, key.len) != NGX_OK + || ngx_http_lua_check_header_safe(r, value.data, value.len) != NGX_OK) + { + return NGX_ERROR; + } hv.hash = ngx_hash_key_lc(key.data, key.len); hv.key = key; diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index 3559b5c0cc..6818ba85c4 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -16,6 +16,8 @@ static int ngx_http_lua_ngx_req_set_uri(lua_State *L); +static ngx_int_t ngx_http_lua_check_uri_safe(ngx_http_request_t *r, + u_char *str, size_t len); void @@ -55,6 +57,10 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return luaL_error(L, "attempt to use zero-length uri"); } + if (ngx_http_lua_check_uri_safe(r, p, len) != NGX_OK) { + return luaL_error(L, "attempt to use unsafe uri"); + } + if (n == 2) { luaL_checktype(L, 2, LUA_TBOOLEAN); @@ -107,4 +113,57 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return 0; } + +static ngx_inline ngx_int_t +ngx_http_lua_check_uri_safe(ngx_http_request_t *r, u_char *str, size_t len) +{ + size_t i, buf_len; + u_char c; + u_char *buf, *src = str; + + /* %00-%1F, " ", %7F */ + + static uint32_t unsafe[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0x00000001, /* 0000 0000 0000 0000 0000 0000 0000 0001 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000 /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + }; + + for (i = 0; i < len; i++, str++) { + c = *str; + if (unsafe[c >> 5] & (1 << (c & 0x1f))) { + buf_len = ngx_http_lua_escape_log(NULL, src, len); + buf = ngx_palloc(r->pool, buf_len); + if (buf == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_log(buf, src, len); + + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "unsafe byte \"0x%uxd\" in uri \"%*s\"", + (unsigned) c, buf_len, buf); + + ngx_pfree(r->pool, buf); + + return NGX_ERROR; + } + } + + return NGX_OK; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 074957b6e7..42bfb91eaf 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4261,4 +4261,123 @@ ngx_http_lua_set_sa_restart(ngx_log_t *log) #endif +size_t +ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size) +{ + size_t n; + u_char c; + static u_char hex[] = "0123456789ABCDEF"; + + static uint32_t escape[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0x00000004, /* 0000 0000 0000 0000 0000 0000 0000 0100 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x10000000, /* 0001 0000 0000 0000 0000 0000 0000 0000 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + }; + + if (dst == NULL) { + + /* find the number of characters to be escaped */ + + n = 0; + + while (size) { + c = *src; + if (escape[c >> 5] & (1 << (c & 0x1f))) { + n += 4; + + } else { + n++; + } + + src++; + size--; + } + + return n; + } + + while (size) { + c = *src; + if (escape[c >> 5] & (1 << (c & 0x1f))) { + *dst++ = '\\'; + *dst++ = 'x'; + *dst++ = hex[*src >> 4]; + *dst++ = hex[*src & 0xf]; + src++; + + } else { + *dst++ = *src++; + } + + size--; + } + + return 0; +} + + +ngx_inline ngx_int_t +ngx_http_lua_check_header_safe(ngx_http_request_t *r, u_char *str, size_t len) +{ + size_t i, buf_len; + u_char c; + u_char *buf, *src = str; + + /* %00-%1F, %7F */ + + static uint32_t unsafe[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000 /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + }; + + for (i = 0; i < len; i++, str++) { + c = *str; + if (unsafe[c >> 5] & (1 << (c & 0x1f))) { + buf_len = ngx_http_lua_escape_log(NULL, src, len); + buf = ngx_palloc(r->pool, buf_len); + if (buf == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_log(buf, src, len); + + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "unsafe byte \"0x%uxd\" in header \"%*s\"", + (unsigned) c, buf_len, buf); + + ngx_pfree(r->pool, buf); + + return NGX_ERROR; + } + } + + return NGX_OK; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 13bfd273e7..7e62dccbf2 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -241,20 +241,9 @@ void ngx_http_lua_cleanup_free(ngx_http_request_t *r, void ngx_http_lua_set_sa_restart(ngx_log_t *log); #endif - -static ngx_inline size_t -ngx_http_lua_safe_header_value_len(u_char *str, size_t len) -{ - size_t i; - - for (i = 0; i < len; i++, str++) { - if (*str == '\r' || *str == '\n') { - return i; - } - } - - return len; -} +size_t ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size); +ngx_int_t ngx_http_lua_check_header_safe(ngx_http_request_t *r, u_char *str, + size_t len); static ngx_inline void diff --git a/t/016-resp-header.t b/t/016-resp-header.t index e12c4b2468..3ae22c803f 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 73); +plan tests => repeat_each() * (blocks() * 3 + 80); #no_diff(); no_long_string(); @@ -1966,7 +1966,7 @@ Content-Type: application/json -=== TEST 87: truncates value after '\r' +=== TEST 87: value contains '\r' --- config location = /t { content_by_lua_block { @@ -1976,16 +1976,18 @@ Content-Type: application/json } --- request GET /t +--- error_code: 500 --- response_headers -header: value +header: foo: bar: ---- no_error_log -[error] +--- error_log +unsafe byte "0xd" in header "value\x0Dfoo:bar\x0Abar:foo" +failed to set header -=== TEST 88: truncates value after '\n' +=== TEST 88: value contains '\n' --- config location = /t { content_by_lua_block { @@ -1995,16 +1997,18 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers -header: value +header: foo: bar: ---- no_error_log -[error] +--- error_log +unsafe byte "0xa" in header "value\x0Afoo:bar\x0Dbar:foo" +failed to set header -=== TEST 89: truncates key after '\r' +=== TEST 89: header name contains '\r' --- config location = /t { content_by_lua_block { @@ -2014,12 +2018,14 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers -header: value: xx +header: foo: bar: ---- no_error_log -[error] +--- error_log +unsafe byte "0xd" in header "header: value\x0Dfoo:bar\x0Abar:foo" +failed to set header @@ -2033,16 +2039,18 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers -header: value: xx +header: foo: bar: ---- no_error_log -[error] +--- error_log +unsafe byte "0xa" in header "header: value\x0Afoo:bar\x0Dbar:foo" +failed to set header -=== TEST 91: truncates key after '\r' as the first character +=== TEST 91: header name: '\r' as the first character --- config location = /t { content_by_lua_block { @@ -2052,16 +2060,18 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers header: foo: bar: ---- no_error_log -[error] +--- error_log +unsafe byte "0xd" in header "\x0Dheader: value\x0Dfoo:bar\x0Abar:foo" +failed to set header -=== TEST 92: truncates key after '\n' as the first character +=== TEST 92: header name: '\n' as the first character --- config location = /t { content_by_lua_block { @@ -2071,12 +2081,14 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers header: foo: bar: ---- no_error_log -[error] +--- error_log +unsafe byte "0xa" in header "\x0Aheader: value\x0Afoo:bar\x0Dbar:foo" +failed to set header @@ -2093,9 +2105,11 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers -foo: foo, bar +foo: xx: xxx: ---- no_error_log -[error] +--- error_log +unsafe byte "0xa" in header "foo\x0Axx:bar" +failed to set header diff --git a/t/022-redirect.t b/t/022-redirect.t index a6cc91ea5b..c3ae35421c 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -9,7 +9,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 3 + 8); +plan tests => repeat_each() * (blocks() * 3 + 14); #no_diff(); #no_long_string(); @@ -323,7 +323,7 @@ Location: http://agentzh.org/foo?a=b&c=d -=== TEST 18: truncates uri after '\r' +=== TEST 18: uri contains '\r' --- config location = /t { content_by_lua_block { @@ -333,16 +333,18 @@ Location: http://agentzh.org/foo?a=b&c=d } --- request GET /t +--- error_code: 500 --- response_headers -Location: http://agentzh.org/foo +Location: foo: bar: ---- response_body_like: 302 Found ---- error_code: 302 +--- error_log +unsafe byte "0xd" in header "http://agentzh.org/foo\x0Dfoo:bar\x0Abar:foo" +attempt to use unsafe uri -=== TEST 19: truncates uri after '\n' +=== TEST 19: uri contains '\n' --- config location = /t { content_by_lua_block { @@ -352,16 +354,18 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers -Location: http://agentzh.org/foo +Location: foo: bar: ---- response_body_like: 302 Found ---- error_code: 302 +--- error_log +unsafe byte "0xa" in header "http://agentzh.org/foo\x0Afoo:bar\x0Dbar:foo" +attempt to use unsafe uri -=== TEST 20: truncates uri with '\n' as the first character +=== TEST 20: uri prefix '\n' --- config location = /t { content_by_lua_block { @@ -371,15 +375,17 @@ bar: } --- request GET /t +--- error_code: 500 --- response_headers Location: foo: ---- response_body_like: 302 Found ---- error_code: 302 +--- error_log +unsafe byte "0xa" in header "\x0Afoo:http://agentzh.org/foo" +attempt to use unsafe uri -=== TEST 21: truncates uri with '\r' as the first character +=== TEST 21: uri prefix '\r' --- config location = /t { content_by_lua_block { @@ -389,8 +395,30 @@ foo: } --- request GET /t +--- error_code: 500 --- response_headers Location: foo: ---- response_body_like: 302 Found ---- error_code: 302 +--- error_log +unsafe byte "0xd" in header "\x0Dfoo:http://agentzh.org/foo" +attempt to use unsafe uri + + + +=== TEST 22: uri with invalid characters escapes '"' and '\' characters +--- config + location = /t { + content_by_lua_block { + ngx.redirect("\rhttp\\://\"agentzh.org\"/foo"); + ngx.say("hi") + } + } +--- request +GET /t +--- error_code: 500 +--- response_headers +Location: +foo: +--- error_log +unsafe byte "0xd" in header "\x0Dhttp\x5C://\x22agentzh.org\x22/foo" +attempt to use unsafe uri diff --git a/t/028-req-header.t b/t/028-req-header.t index 8ddbb29796..d9b9fdd111 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (2 * blocks() + 44); +plan tests => repeat_each() * (2 * blocks() + 48); #no_diff(); no_long_string(); @@ -2030,3 +2030,79 @@ localhost new --- no_error_log [error] + + + +=== TEST 62: set input header with unsafe key +--- config + location /req-header { + rewrite_by_lua_block { + ngx.req.set_header("Foo\rfoo", "new value"); + } + + echo "Foo: $http_foo"; + } +--- request +GET /req-header +--- error_code: 500 +--- error_log +unsafe byte "0xd" in header "Foo\x0Dfoo" +failed to set header + + + +=== TEST 63: set input header with unsafe value +--- config + location /req-header { + rewrite_by_lua_block { + ngx.req.set_header("Foo", "new\nvalue"); + } + + echo "Foo: $http_foo"; + } +--- request +GET /req-header +--- error_code: 500 +--- error_log +unsafe byte "0xa" in header "new\x0Avalue" +failed to set header + + + +=== TEST 64: set input header with multiple unsafe values +--- config + location /req-header { + rewrite_by_lua_block { + ngx.req.set_header("Foo", { "new\nvalue", "foo\tbar" } ); + } + + content_by_lua_block { + ngx.say(table.concat(ngx.req.get_headers()["foo"], ", "), ".") + } + } +--- request +GET /req-header +--- error_code: 500 +--- error_log +unsafe byte "0xa" in header "new\x0Avalue" +failed to set header + + + +=== TEST 65: unsafe value errors escape '"' and '\' characters +--- config + location /req-header { + rewrite_by_lua_block { + ngx.req.set_header("Foo", "\"new\nvalue\\\""); + } + + content_by_lua_block { + ngx.say(table.concat(ngx.req.get_headers()["foo"], ", "), ".") + } + } +--- request +GET /req-header +--- error_code: 500 +--- error_log +unsafe byte "0xa" in header "\x22new\x0Avalue\x5C\x22" +failed to set header diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 7af63c4c24..05a6de4893 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,7 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 18); +plan tests => repeat_each() * (blocks() * 2 + 20); no_root_location(); @@ -1554,3 +1554,39 @@ GET /lua args: foo=%2C%24%40%7C%60&bar=-_.!~*'() --- no_error_log [error] + + + +=== TEST 58: set_uri with unsafe uri (with \t) +--- config + location /t { + content_by_lua_block { + local new_uri = "/foo\tbar" + ngx.req.set_uri(new_uri) + ngx.say(ngx.var.uri) + } + } +--- request + GET /t +--- error_code: 500 +--- error_log +unsafe byte "0x9" in uri "/foo\x09bar" +attempt to use unsafe uri + + + +=== TEST 59: set_uri with unsafe uri (with " ") +--- config + location /t { + content_by_lua_block { + local new_uri = "/foo bar" + ngx.req.set_uri(new_uri) + ngx.say(ngx.var.uri) + } + } +--- request + GET /t +--- error_code: 500 +--- error_log +unsafe byte "0x20" in uri "/foo bar" +attempt to use unsafe uri diff --git a/t/113-req-header-cookie.t b/t/113-req-header-cookie.t index 4a9305342b..63330504ab 100644 --- a/t/113-req-header-cookie.t +++ b/t/113-req-header-cookie.t @@ -247,3 +247,25 @@ Cookie: boo=123; foo=bar --- no_error_log [error] + + + +=== TEST 7: set multiple custom cookies (with unsafe values) +--- config + location /t { + rewrite_by_lua_block { + ngx.req.set_header("Cookie", {"boo=123\nfoo", "foo=bar\rbar"}) + } + echo "Cookie foo: $cookie_foo"; + echo "Cookie baz: $cookie_baz"; + echo "Cookie boo: $cookie_boo"; + echo "Cookie: $http_cookie"; + } +--- request +GET /t +--- error_code: 500 +--- error_log +unsafe byte "0xa" in header "boo=123\x0Afoo" +failed to set header +--- no_error_log +[crit] From 3a37c9c61852bf37a6b9956c06a7954233f8d804 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 19 Mar 2020 14:23:40 -0700 Subject: [PATCH 392/848] bugfix: FFI availability feature test was broken on Windows. bugfix: we now avoided passing -DLUA_DEFAULT_PATH=... and -DLUA_DEFAULT_CPATH=... options via CFLAGS. This is know to cause problems in some cases. openresty/openresty#587. bugfix: some LuaJIT related feature tests may cause unwanted C compiler errors due to `#endif;`. --- config | 15 +++++++++------ src/ngx_http_lua_common.h | 2 ++ 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/config b/config index 32290d4c69..7465a94932 100644 --- a/config +++ b/config @@ -178,7 +178,8 @@ ngx_feature_run=no ngx_feature_incs="#include " ngx_feature_test="#if !defined(LUAJIT_VERSION_NUM) || LUAJIT_VERSION_NUM < 20000 # error unsupported LuaJIT version - #endif" + #endif + " . auto/feature @@ -196,7 +197,8 @@ ngx_feature_run=no ngx_feature_incs="#include " ngx_feature_test="#if !defined(LUA_VERSION_NUM) || LUA_VERSION_NUM != 501 # error unsupported Lua language version - #endif" + #endif + " . auto/feature @@ -210,8 +212,8 @@ fi # ---------------------------------------- ngx_feature="LuaJIT has FFI" -ngx_feature_libs="$LUAJIT_LIB/libluajit-5.1.a $luajit_ld_opt" -ngx_feature_run=yes +ngx_feature_libs="$ngx_module_libs" +ngx_feature_run=no ngx_feature_incs="#include #include #include " @@ -295,6 +297,7 @@ HTTP_LUA_SRCS=" \ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ddebug.h \ + $ngx_addon_dir/src/ngx_http_lua_autoconf.h \ $ngx_addon_dir/src/ngx_http_lua_script.h \ $ngx_addon_dir/src/ngx_http_lua_log.h \ $ngx_addon_dir/src/ngx_http_lua_subrequest.h \ @@ -554,5 +557,5 @@ NGX_TAPSET_SRCS="$NGX_TAPSET_SRCS $ngx_addon_dir/tapset/ngx_lua.stp" CORE_INCS="$CORE_INCS $ngx_addon_dir/src/api" CFLAGS="$CFLAGS -DNDK_SET_VAR" -#CFLAGS=$"$CFLAGS -DLUA_DEFAULT_PATH='\"/usr/local/openresty/lualib/?.lua\"'" -#CFLAGS=$"$CFLAGS -DLUA_DEFAULT_CPATH='\"/usr/local/openresty/lualib/?.so\"'" + +echo "/* DO NOT EDIT! This file was automatically generated by config */" > "$ngx_addon_dir/src/ngx_http_lua_autoconf.h" diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index c8c383c6e4..781a2454a1 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -9,6 +9,8 @@ #define _NGX_HTTP_LUA_COMMON_H_INCLUDED_ +#include "ngx_http_lua_autoconf.h" + #include #include #include From 79b3bd3cf2236d0d379bd5e65be05afaea2a067c Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 19 Mar 2020 14:33:50 -0700 Subject: [PATCH 393/848] config: more followup fixes in autoconf feature tests. --- config | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/config b/config index 7465a94932..18014aa97c 100644 --- a/config +++ b/config @@ -216,10 +216,12 @@ ngx_feature_libs="$ngx_module_libs" ngx_feature_run=no ngx_feature_incs="#include #include - #include " + #include + " ngx_feature_test="lua_State *L = luaL_newstate(); assert(L != NULL); - luaopen_ffi(L);" + luaopen_ffi(L); + " . auto/feature From ada56de5711ef53567a377adcad5173a431617fc Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 19 Mar 2020 14:40:44 -0700 Subject: [PATCH 394/848] updated .gitignore. --- .gitignore | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.gitignore b/.gitignore index e20be8dff2..11d444fe4c 100644 --- a/.gitignore +++ b/.gitignore @@ -171,3 +171,9 @@ tthread addr2line hup theaders +src/ngx_http_lua_autoconf.h +src/autoconf.h +src/filters.c +src/filters.h +src/ringbuf.c +src/ringbuf.h From 3908769d39b40b6973cc2e2002cff8d5b3169b13 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 19 Mar 2020 23:10:53 -0700 Subject: [PATCH 395/848] travis: upgraded pcre to 8.44. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 6de5511dff..822ed6c823 100644 --- a/.travis.yml +++ b/.travis.yml @@ -34,7 +34,7 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_VER=8.41 + - PCRE_VER=8.44 - PCRE_PREFIX=/opt/pcre - PCRE_LIB=$PCRE_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include @@ -64,7 +64,7 @@ before_install: install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache http://ftp.cs.stanford.edu/pub/exim/pcre/pcre-$PCRE_VER.tar.gz; fi + - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://ftp.pcre.org/pub/pcre/pcre-$PCRE_VER.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz; fi - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty From f6ff2890cbf8e19dcc73d2bc9b0a71fdd94a4983 Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Fri, 20 Mar 2020 16:35:40 -0700 Subject: [PATCH 396/848] misc: avoided warnings with old gcc versions due to incorrect inline definition and renamed functions for clarity. --- src/ngx_http_lua_control.c | 2 +- src/ngx_http_lua_headers_in.c | 4 +-- src/ngx_http_lua_headers_out.c | 4 +-- src/ngx_http_lua_uri.c | 6 ++-- src/ngx_http_lua_util.c | 52 -------------------------------- src/ngx_http_lua_util.h | 54 ++++++++++++++++++++++++++++++++-- t/016-resp-header.t | 14 ++++----- t/022-redirect.t | 10 +++---- t/028-req-header.t | 8 ++--- t/030-uri-args.t | 4 +-- t/113-req-header-cookie.t | 2 +- 11 files changed, 79 insertions(+), 81 deletions(-) diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 2fd3d17478..3d92d88e23 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -239,7 +239,7 @@ ngx_http_lua_ngx_redirect(lua_State *L) "the headers"); } - if (ngx_http_lua_check_header_safe(r, p, len) != NGX_OK) { + if (ngx_http_lua_check_unsafe_header(r, p, len) != NGX_OK) { return luaL_error(L, "attempt to use unsafe uri"); } diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index a284f028bf..f7b18bbd33 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -658,8 +658,8 @@ ngx_http_lua_set_input_header(ngx_http_request_t *r, ngx_str_t key, dd("set header value: %.*s", (int) value.len, value.data); - if (ngx_http_lua_check_header_safe(r, key.data, key.len) != NGX_OK - || ngx_http_lua_check_header_safe(r, value.data, value.len) != NGX_OK) + if (ngx_http_lua_check_unsafe_header(r, key.data, key.len) != NGX_OK + || ngx_http_lua_check_unsafe_header(r, value.data, value.len) != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 87f114b820..cbdf410f3c 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -491,8 +491,8 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, dd("set header value: %.*s", (int) value.len, value.data); - if (ngx_http_lua_check_header_safe(r, key.data, key.len) != NGX_OK - || ngx_http_lua_check_header_safe(r, value.data, value.len) != NGX_OK) + if (ngx_http_lua_check_unsafe_header(r, key.data, key.len) != NGX_OK + || ngx_http_lua_check_unsafe_header(r, value.data, value.len) != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index 6818ba85c4..037efa77f1 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -16,7 +16,7 @@ static int ngx_http_lua_ngx_req_set_uri(lua_State *L); -static ngx_int_t ngx_http_lua_check_uri_safe(ngx_http_request_t *r, +static ngx_inline ngx_int_t ngx_http_lua_check_unsafe_uri(ngx_http_request_t *r, u_char *str, size_t len); @@ -57,7 +57,7 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return luaL_error(L, "attempt to use zero-length uri"); } - if (ngx_http_lua_check_uri_safe(r, p, len) != NGX_OK) { + if (ngx_http_lua_check_unsafe_uri(r, p, len) != NGX_OK) { return luaL_error(L, "attempt to use unsafe uri"); } @@ -115,7 +115,7 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) static ngx_inline ngx_int_t -ngx_http_lua_check_uri_safe(ngx_http_request_t *r, u_char *str, size_t len) +ngx_http_lua_check_unsafe_uri(ngx_http_request_t *r, u_char *str, size_t len) { size_t i, buf_len; u_char c; diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 42bfb91eaf..983f476bb4 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4328,56 +4328,4 @@ ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size) } -ngx_inline ngx_int_t -ngx_http_lua_check_header_safe(ngx_http_request_t *r, u_char *str, size_t len) -{ - size_t i, buf_len; - u_char c; - u_char *buf, *src = str; - - /* %00-%1F, %7F */ - - static uint32_t unsafe[] = { - 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ - - /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - - /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - - /* ~}| {zyx wvut srqp onml kjih gfed cba` */ - 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ - - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - 0x00000000 /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - }; - - for (i = 0; i < len; i++, str++) { - c = *str; - if (unsafe[c >> 5] & (1 << (c & 0x1f))) { - buf_len = ngx_http_lua_escape_log(NULL, src, len); - buf = ngx_palloc(r->pool, buf_len); - if (buf == NULL) { - return NGX_ERROR; - } - - ngx_http_lua_escape_log(buf, src, len); - - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "unsafe byte \"0x%uxd\" in header \"%*s\"", - (unsigned) c, buf_len, buf); - - ngx_pfree(r->pool, buf); - - return NGX_ERROR; - } - } - - return NGX_OK; -} - - /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 7e62dccbf2..d5c4aeb120 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -242,8 +242,6 @@ void ngx_http_lua_set_sa_restart(ngx_log_t *log); #endif size_t ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size); -ngx_int_t ngx_http_lua_check_header_safe(ngx_http_request_t *r, u_char *str, - size_t len); static ngx_inline void @@ -491,6 +489,58 @@ ngx_inet_get_port(struct sockaddr *sa) #endif +static ngx_inline ngx_int_t +ngx_http_lua_check_unsafe_header(ngx_http_request_t *r, u_char *str, size_t len) +{ + size_t i, buf_len; + u_char c; + u_char *buf, *src = str; + + /* %00-%1F, %7F */ + + static uint32_t unsafe[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000 /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + }; + + for (i = 0; i < len; i++, str++) { + c = *str; + if (unsafe[c >> 5] & (1 << (c & 0x1f))) { + buf_len = ngx_http_lua_escape_log(NULL, src, len); + buf = ngx_palloc(r->pool, buf_len); + if (buf == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_log(buf, src, len); + + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "unsafe byte \"0x%uxd\" in header \"%*s\"", + (unsigned) c, buf_len, buf); + + ngx_pfree(r->pool, buf); + + return NGX_ERROR; + } + } + + return NGX_OK; +} + + extern ngx_uint_t ngx_http_lua_location_hash; extern ngx_uint_t ngx_http_lua_content_length_hash; diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 3ae22c803f..bb6013f3bf 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -1966,7 +1966,7 @@ Content-Type: application/json -=== TEST 87: value contains '\r' +=== TEST 87: unsafe header value (with '\r') --- config location = /t { content_by_lua_block { @@ -1987,7 +1987,7 @@ failed to set header -=== TEST 88: value contains '\n' +=== TEST 88: unsafe header value (with '\n') --- config location = /t { content_by_lua_block { @@ -2008,7 +2008,7 @@ failed to set header -=== TEST 89: header name contains '\r' +=== TEST 89: unsafe header name (with '\r') --- config location = /t { content_by_lua_block { @@ -2029,7 +2029,7 @@ failed to set header -=== TEST 90: truncates key after '\n' +=== TEST 90: unsafe header name (with '\n') --- config location = /t { content_by_lua_block { @@ -2050,7 +2050,7 @@ failed to set header -=== TEST 91: header name: '\r' as the first character +=== TEST 91: unsafe header name (with prefix '\r') --- config location = /t { content_by_lua_block { @@ -2071,7 +2071,7 @@ failed to set header -=== TEST 92: header name: '\n' as the first character +=== TEST 92: unsafe header name (with prefix '\n') --- config location = /t { content_by_lua_block { @@ -2092,7 +2092,7 @@ failed to set header -=== TEST 93: truncates multiple values if they contain '\r' or '\n' +=== TEST 93: multiple unsafe header values (with '\n' and '\r') --- config location = /t { content_by_lua_block { diff --git a/t/022-redirect.t b/t/022-redirect.t index c3ae35421c..dbfdc6821d 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -323,7 +323,7 @@ Location: http://agentzh.org/foo?a=b&c=d -=== TEST 18: uri contains '\r' +=== TEST 18: unsafe uri (with '\r') --- config location = /t { content_by_lua_block { @@ -344,7 +344,7 @@ attempt to use unsafe uri -=== TEST 19: uri contains '\n' +=== TEST 19: unsafe uri (with '\n') --- config location = /t { content_by_lua_block { @@ -365,7 +365,7 @@ attempt to use unsafe uri -=== TEST 20: uri prefix '\n' +=== TEST 20: unsafe uri (with prefix '\n') --- config location = /t { content_by_lua_block { @@ -385,7 +385,7 @@ attempt to use unsafe uri -=== TEST 21: uri prefix '\r' +=== TEST 21: unsafe uri (with prefix '\r') --- config location = /t { content_by_lua_block { @@ -405,7 +405,7 @@ attempt to use unsafe uri -=== TEST 22: uri with invalid characters escapes '"' and '\' characters +=== TEST 22: unsafe uri logging escapes '"' and '\' characters --- config location = /t { content_by_lua_block { diff --git a/t/028-req-header.t b/t/028-req-header.t index d9b9fdd111..2313f77188 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -2033,7 +2033,7 @@ new -=== TEST 62: set input header with unsafe key +=== TEST 62: unsafe header name (with '\r') --- config location /req-header { rewrite_by_lua_block { @@ -2051,7 +2051,7 @@ failed to set header -=== TEST 63: set input header with unsafe value +=== TEST 63: unsafe header value (with '\n') --- config location /req-header { rewrite_by_lua_block { @@ -2069,7 +2069,7 @@ failed to set header -=== TEST 64: set input header with multiple unsafe values +=== TEST 64: multiple unsafe header values (with '\n' and '\t') --- config location /req-header { rewrite_by_lua_block { @@ -2089,7 +2089,7 @@ failed to set header -=== TEST 65: unsafe value errors escape '"' and '\' characters +=== TEST 65: unsafe names/values logging escapes '"' and '\' characters --- config location /req-header { rewrite_by_lua_block { diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 05a6de4893..b21bac0c18 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1557,7 +1557,7 @@ args: foo=%2C%24%40%7C%60&bar=-_.!~*'() -=== TEST 58: set_uri with unsafe uri (with \t) +=== TEST 58: set_uri with unsafe uri (with '\t') --- config location /t { content_by_lua_block { @@ -1575,7 +1575,7 @@ attempt to use unsafe uri -=== TEST 59: set_uri with unsafe uri (with " ") +=== TEST 59: set_uri with unsafe uri (with ' ') --- config location /t { content_by_lua_block { diff --git a/t/113-req-header-cookie.t b/t/113-req-header-cookie.t index 63330504ab..04ef521bae 100644 --- a/t/113-req-header-cookie.t +++ b/t/113-req-header-cookie.t @@ -250,7 +250,7 @@ Cookie: boo=123; foo=bar -=== TEST 7: set multiple custom cookies (with unsafe values) +=== TEST 7: set multiple custom cookies with unsafe values (with '\n' and 'r') --- config location /t { rewrite_by_lua_block { From f45d4e999c914daa01b8f37e3a5ab584ef44c011 Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 24 Feb 2020 16:50:45 +0800 Subject: [PATCH 397/848] bugfix: overrided the input header only for the first time When multiple values are associated to the header, we should override the header only for the first time, otherwise the following value will override the previous one. --- src/ngx_http_lua_headers.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 60b58d44c9..b638277bd7 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -997,7 +997,7 @@ ngx_http_lua_ffi_req_set_header(ngx_http_request_t *r, const u_char *key, v.data[len] = '\0'; v.len = len; - if (ngx_http_lua_set_input_header(r, k, v, override) + if (ngx_http_lua_set_input_header(r, k, v, override && i == 0) != NGX_OK) { goto failed; From f991f3e10fb592f12b7116c609bb0614618adc93 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Mon, 30 Mar 2020 18:25:55 +0800 Subject: [PATCH 398/848] travis: fixed the warnings from build config validation. --- .travis.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/.travis.yml b/.travis.yml index 822ed6c823..0a76655352 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,4 +1,3 @@ -sudo: required dist: xenial os: linux @@ -47,14 +46,14 @@ env: - LD_LIBRARY_PATH=$LUAJIT_LIB:$LD_LIBRARY_PATH - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 - matrix: + jobs: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1e OPENSSL_PATCH_VER=1.1.1e services: - - memcache - - redis-server + - memcached + - redis - mysql before_install: From 66026039d11e32a55a2bc019d2da9b5cebe9e46a Mon Sep 17 00:00:00 2001 From: Chris Kuehl Date: Thu, 1 Nov 2018 09:59:28 -0700 Subject: [PATCH 399/848] doc: clarify ngx.timer.every callback argument conventions, #1409. --- README.markdown | 3 +++ doc/HttpLuaModule.wiki | 3 +++ 2 files changed, 6 insertions(+) diff --git a/README.markdown b/README.markdown index 6839efc6bd..c150b2a749 100644 --- a/README.markdown +++ b/README.markdown @@ -8239,6 +8239,9 @@ Similar to the [ngx.timer.at](#ngxtimerat) API function, but 1. `delay` *cannot* be zero, 1. timer will be created every `delay` seconds until the current Nginx worker process starts exiting. +Like [ngx.timer.at](#ngxtimerat), the `callback` argument will be called +automatically with the arguments `premature`, `user_arg1`, `user_arg2`, etc. + When success, returns a "conditional true" value (but not a `true`). Otherwise, returns a "conditional false" value and a string describing the error. This API also respect the [lua_max_pending_timers](#lua_max_pending_timers) and [lua_max_running_timers](#lua_max_running_timers). diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 91409d2109..378c23ad6b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7051,6 +7051,9 @@ Similar to the [[#ngx.timer.at|ngx.timer.at]] API function, but # delay ''cannot'' be zero, # timer will be created every delay seconds until the current Nginx worker process starts exiting. +Like [[#ngx.timer.at|ngx.timer.at]], the callback argument will be called +automatically with the arguments premature, user_arg1, user_arg2, etc. + When success, returns a "conditional true" value (but not a true). Otherwise, returns a "conditional false" value and a string describing the error. This API also respect the [[#lua_max_pending_timers|lua_max_pending_timers]] and [[#lua_max_running_timers|lua_max_running_timers]]. From 6d270c32f1f17fea8e9d36c4e758e7e59c8b4d6e Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 31 Mar 2020 11:50:47 -0700 Subject: [PATCH 400/848] doc: fixed the return value name in 'ngx.worker.id()' documentation. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index c150b2a749..ac563a9044 100644 --- a/README.markdown +++ b/README.markdown @@ -8399,7 +8399,7 @@ This API was first introduced in the `0.9.20` release. ngx.worker.id ------------- -**syntax:** *count = ngx.worker.id()* +**syntax:** *id = ngx.worker.id()* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_worker_by_lua** diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 378c23ad6b..12b1b40dd9 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7175,7 +7175,7 @@ This API was first introduced in the 0.9.20 release. == ngx.worker.id == -'''syntax:''' ''count = ngx.worker.id()'' +'''syntax:''' ''id = ngx.worker.id()'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_worker_by_lua*'' From beb5c6ba03b654c3e435e9a7c6a05d44d1550861 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 31 Mar 2020 18:07:39 +0800 Subject: [PATCH 401/848] bugfix: allow the use of spaces in 'ngx.req.set_uri()'. Signed-off-by: Thibault Charbonnier --- src/ngx_http_lua_control.c | 4 +-- src/ngx_http_lua_headers_in.c | 6 ++-- src/ngx_http_lua_headers_out.c | 6 ++-- src/ngx_http_lua_uri.c | 58 ++-------------------------------- src/ngx_http_lua_util.h | 7 ++-- t/016-resp-header.t | 14 ++++---- t/022-redirect.t | 20 ++++++------ t/028-req-header.t | 8 ++--- t/030-uri-args.t | 39 +++++++++++++++++++---- t/113-req-header-cookie.t | 2 +- 10 files changed, 71 insertions(+), 93 deletions(-) diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 3d92d88e23..a468285331 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -239,8 +239,8 @@ ngx_http_lua_ngx_redirect(lua_State *L) "the headers"); } - if (ngx_http_lua_check_unsafe_header(r, p, len) != NGX_OK) { - return luaL_error(L, "attempt to use unsafe uri"); + if (ngx_http_lua_check_unsafe_string(r, p, len, "redirect uri") != NGX_OK) { + return luaL_error(L, "attempt to set unsafe redirect uri"); } uri = ngx_palloc(r->pool, len); diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index f7b18bbd33..34f2fb667f 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -658,8 +658,10 @@ ngx_http_lua_set_input_header(ngx_http_request_t *r, ngx_str_t key, dd("set header value: %.*s", (int) value.len, value.data); - if (ngx_http_lua_check_unsafe_header(r, key.data, key.len) != NGX_OK - || ngx_http_lua_check_unsafe_header(r, value.data, value.len) != NGX_OK) + if (ngx_http_lua_check_unsafe_string(r, key.data, key.len, + "header name") != NGX_OK + || ngx_http_lua_check_unsafe_string(r, value.data, value.len, + "header value") != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index cbdf410f3c..a0a9012643 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -491,8 +491,10 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, dd("set header value: %.*s", (int) value.len, value.data); - if (ngx_http_lua_check_unsafe_header(r, key.data, key.len) != NGX_OK - || ngx_http_lua_check_unsafe_header(r, value.data, value.len) != NGX_OK) + if (ngx_http_lua_check_unsafe_string(r, key.data, key.len, + "header name") != NGX_OK + || ngx_http_lua_check_unsafe_string(r, value.data, value.len, + "header value") != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index 037efa77f1..569cb50270 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -16,8 +16,6 @@ static int ngx_http_lua_ngx_req_set_uri(lua_State *L); -static ngx_inline ngx_int_t ngx_http_lua_check_unsafe_uri(ngx_http_request_t *r, - u_char *str, size_t len); void @@ -57,8 +55,8 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return luaL_error(L, "attempt to use zero-length uri"); } - if (ngx_http_lua_check_unsafe_uri(r, p, len) != NGX_OK) { - return luaL_error(L, "attempt to use unsafe uri"); + if (ngx_http_lua_check_unsafe_string(r, p, len, "uri") != NGX_OK) { + return luaL_error(L, "attempt to set unsafe uri"); } if (n == 2) { @@ -114,56 +112,4 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) } -static ngx_inline ngx_int_t -ngx_http_lua_check_unsafe_uri(ngx_http_request_t *r, u_char *str, size_t len) -{ - size_t i, buf_len; - u_char c; - u_char *buf, *src = str; - - /* %00-%1F, " ", %7F */ - - static uint32_t unsafe[] = { - 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ - - /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ - 0x00000001, /* 0000 0000 0000 0000 0000 0000 0000 0001 */ - - /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - - /* ~}| {zyx wvut srqp onml kjih gfed cba` */ - 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ - - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - 0x00000000 /* 0000 0000 0000 0000 0000 0000 0000 0000 */ - }; - - for (i = 0; i < len; i++, str++) { - c = *str; - if (unsafe[c >> 5] & (1 << (c & 0x1f))) { - buf_len = ngx_http_lua_escape_log(NULL, src, len); - buf = ngx_palloc(r->pool, buf_len); - if (buf == NULL) { - return NGX_ERROR; - } - - ngx_http_lua_escape_log(buf, src, len); - - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "unsafe byte \"0x%uxd\" in uri \"%*s\"", - (unsigned) c, buf_len, buf); - - ngx_pfree(r->pool, buf); - - return NGX_ERROR; - } - } - - return NGX_OK; -} - - /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index d5c4aeb120..a0ab2de357 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -490,7 +490,8 @@ ngx_inet_get_port(struct sockaddr *sa) static ngx_inline ngx_int_t -ngx_http_lua_check_unsafe_header(ngx_http_request_t *r, u_char *str, size_t len) +ngx_http_lua_check_unsafe_string(ngx_http_request_t *r, u_char *str, size_t len, + const char *name) { size_t i, buf_len; u_char c; @@ -528,8 +529,8 @@ ngx_http_lua_check_unsafe_header(ngx_http_request_t *r, u_char *str, size_t len) ngx_http_lua_escape_log(buf, src, len); ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "unsafe byte \"0x%uxd\" in header \"%*s\"", - (unsigned) c, buf_len, buf); + "unsafe byte \"0x%uxd\" in %s \"%*s\"", + (unsigned) c, name, buf_len, buf); ngx_pfree(r->pool, buf); diff --git a/t/016-resp-header.t b/t/016-resp-header.t index bb6013f3bf..3d38d8f21b 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -1982,7 +1982,7 @@ header: foo: bar: --- error_log -unsafe byte "0xd" in header "value\x0Dfoo:bar\x0Abar:foo" +unsafe byte "0xd" in header value "value\x0Dfoo:bar\x0Abar:foo" failed to set header @@ -2003,7 +2003,7 @@ header: foo: bar: --- error_log -unsafe byte "0xa" in header "value\x0Afoo:bar\x0Dbar:foo" +unsafe byte "0xa" in header value "value\x0Afoo:bar\x0Dbar:foo" failed to set header @@ -2024,7 +2024,7 @@ header: foo: bar: --- error_log -unsafe byte "0xd" in header "header: value\x0Dfoo:bar\x0Abar:foo" +unsafe byte "0xd" in header name "header: value\x0Dfoo:bar\x0Abar:foo" failed to set header @@ -2045,7 +2045,7 @@ header: foo: bar: --- error_log -unsafe byte "0xa" in header "header: value\x0Afoo:bar\x0Dbar:foo" +unsafe byte "0xa" in header name "header: value\x0Afoo:bar\x0Dbar:foo" failed to set header @@ -2066,7 +2066,7 @@ header: foo: bar: --- error_log -unsafe byte "0xd" in header "\x0Dheader: value\x0Dfoo:bar\x0Abar:foo" +unsafe byte "0xd" in header name "\x0Dheader: value\x0Dfoo:bar\x0Abar:foo" failed to set header @@ -2087,7 +2087,7 @@ header: foo: bar: --- error_log -unsafe byte "0xa" in header "\x0Aheader: value\x0Afoo:bar\x0Dbar:foo" +unsafe byte "0xa" in header name "\x0Aheader: value\x0Afoo:bar\x0Dbar:foo" failed to set header @@ -2111,5 +2111,5 @@ foo: xx: xxx: --- error_log -unsafe byte "0xa" in header "foo\x0Axx:bar" +unsafe byte "0xa" in header value "foo\x0Axx:bar" failed to set header diff --git a/t/022-redirect.t b/t/022-redirect.t index dbfdc6821d..fb894d87ce 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -339,8 +339,8 @@ Location: foo: bar: --- error_log -unsafe byte "0xd" in header "http://agentzh.org/foo\x0Dfoo:bar\x0Abar:foo" -attempt to use unsafe uri +unsafe byte "0xd" in redirect uri "http://agentzh.org/foo\x0Dfoo:bar\x0Abar:foo" +attempt to set unsafe redirect uri @@ -360,8 +360,8 @@ Location: foo: bar: --- error_log -unsafe byte "0xa" in header "http://agentzh.org/foo\x0Afoo:bar\x0Dbar:foo" -attempt to use unsafe uri +unsafe byte "0xa" in redirect uri "http://agentzh.org/foo\x0Afoo:bar\x0Dbar:foo" +attempt to set unsafe redirect uri @@ -380,8 +380,8 @@ GET /t Location: foo: --- error_log -unsafe byte "0xa" in header "\x0Afoo:http://agentzh.org/foo" -attempt to use unsafe uri +unsafe byte "0xa" in redirect uri "\x0Afoo:http://agentzh.org/foo" +attempt to set unsafe redirect uri @@ -400,8 +400,8 @@ GET /t Location: foo: --- error_log -unsafe byte "0xd" in header "\x0Dfoo:http://agentzh.org/foo" -attempt to use unsafe uri +unsafe byte "0xd" in redirect uri "\x0Dfoo:http://agentzh.org/foo" +attempt to set unsafe redirect uri @@ -420,5 +420,5 @@ GET /t Location: foo: --- error_log -unsafe byte "0xd" in header "\x0Dhttp\x5C://\x22agentzh.org\x22/foo" -attempt to use unsafe uri +unsafe byte "0xd" in redirect uri "\x0Dhttp\x5C://\x22agentzh.org\x22/foo" +attempt to set unsafe redirect uri diff --git a/t/028-req-header.t b/t/028-req-header.t index 2313f77188..fbd4e9eac9 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -2046,7 +2046,7 @@ new GET /req-header --- error_code: 500 --- error_log -unsafe byte "0xd" in header "Foo\x0Dfoo" +unsafe byte "0xd" in header name "Foo\x0Dfoo" failed to set header @@ -2064,7 +2064,7 @@ failed to set header GET /req-header --- error_code: 500 --- error_log -unsafe byte "0xa" in header "new\x0Avalue" +unsafe byte "0xa" in header value "new\x0Avalue" failed to set header @@ -2084,7 +2084,7 @@ failed to set header GET /req-header --- error_code: 500 --- error_log -unsafe byte "0xa" in header "new\x0Avalue" +unsafe byte "0xa" in header value "new\x0Avalue" failed to set header @@ -2104,5 +2104,5 @@ failed to set header GET /req-header --- error_code: 500 --- error_log -unsafe byte "0xa" in header "\x22new\x0Avalue\x5C\x22" +unsafe byte "0xa" in header value "\x22new\x0Avalue\x5C\x22" failed to set header diff --git a/t/030-uri-args.t b/t/030-uri-args.t index b21bac0c18..02fe6e1042 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,7 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 20); +plan tests => repeat_each() * (blocks() * 2 + 21); no_root_location(); @@ -1571,15 +1571,15 @@ args: foo=%2C%24%40%7C%60&bar=-_.!~*'() --- error_code: 500 --- error_log unsafe byte "0x9" in uri "/foo\x09bar" -attempt to use unsafe uri +attempt to set unsafe uri -=== TEST 59: set_uri with unsafe uri (with ' ') +=== TEST 59: set_uri with unsafe uri (with '\0') --- config location /t { content_by_lua_block { - local new_uri = "/foo bar" + local new_uri = '\0foo' ngx.req.set_uri(new_uri) ngx.say(ngx.var.uri) } @@ -1588,5 +1588,32 @@ attempt to use unsafe uri GET /t --- error_code: 500 --- error_log -unsafe byte "0x20" in uri "/foo bar" -attempt to use unsafe uri +unsafe byte "0x0" in uri "\x00foo" +attempt to set unsafe uri + + + +=== TEST 60: set_uri with safe uri (with ' ') +--- config + location /t { + rewrite_by_lua_block { + local new_uri = "/foo bar" + ngx.req.set_uri(new_uri) + } + + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT; + } + + location /foo { + content_by_lua_block { + ngx.say("request_uri: ", ngx.var.request_uri) + ngx.say("uri: ", ngx.var.uri) + } + } +--- request + GET /t +--- response_body +request_uri: /foo%20bar +uri: /foo bar +--- no_error_log +[error] diff --git a/t/113-req-header-cookie.t b/t/113-req-header-cookie.t index 04ef521bae..3baa899c3b 100644 --- a/t/113-req-header-cookie.t +++ b/t/113-req-header-cookie.t @@ -265,7 +265,7 @@ Cookie: boo=123; foo=bar GET /t --- error_code: 500 --- error_log -unsafe byte "0xa" in header "boo=123\x0Afoo" +unsafe byte "0xa" in header value "boo=123\x0Afoo" failed to set header --- no_error_log [crit] From efd284106b7f84ee47b4e94aedb490dba9a2756e Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Tue, 31 Mar 2020 17:50:02 -0700 Subject: [PATCH 402/848] travis-ci: bumped OpenSSL to 1.1.1f and fixed older versions downloads. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 0a76655352..2ca9a75037 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,7 @@ env: jobs: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1e OPENSSL_PATCH_VER=1.1.1e + - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1f OPENSSL_PATCH_VER=1.1.1f services: - memcached @@ -64,7 +64,7 @@ before_install: install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://ftp.pcre.org/pub/pcre/pcre-$PCRE_VER.tar.gz; fi - - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz; fi + - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx From 896638287afd83f060d3f8e9a51b53dbd26d0ae7 Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Tue, 14 Apr 2020 08:32:40 +0800 Subject: [PATCH 403/848] style: fixed a minor styling issue in 'ngx_http_lua_initworkerby.c'. --- src/ngx_http_lua_initworkerby.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 840f14778e..fa094a2e8b 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -67,11 +67,11 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) return NGX_OK; } -#ifdef HAVE_NGX_LUA_PIPE +# ifdef HAVE_NGX_LUA_PIPE if (ngx_http_lua_pipe_add_signal_handler(cycle) != NGX_OK) { return NGX_ERROR; } -#endif +# endif #endif /* NGX_WIN32 */ @@ -351,3 +351,6 @@ ngx_http_lua_log_init_worker_error(ngx_log_t *log, u_char *buf, size_t len) return ngx_snprintf(buf, len, ", context: init_worker_by_lua*"); } + + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From 6e48a804f7e3968e3f083e878a5be62f519c6e37 Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Tue, 19 May 2020 03:07:55 +0800 Subject: [PATCH 404/848] style: ngx_http_lua_socket_tcp.c: ensured two blank lines between functions. --- src/ngx_http_lua_socket_tcp.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index efd5875fc0..30a4e7b282 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5082,6 +5082,7 @@ ngx_http_lua_req_socket_rev_handler(ngx_http_request_t *r) u->read_event_handler(r, u); } + static int ngx_http_lua_socket_tcp_getreusedtimes(lua_State *L) { From 50c1912ef7d0076d506c73f5dad0d129d4c1de94 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Tue, 19 May 2020 03:08:36 +0800 Subject: [PATCH 405/848] misc: ngx_http_lua_logby.c: removed duplicate headers. --- src/ngx_http_lua_logby.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index 422698af7b..408708cfa5 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -22,8 +22,6 @@ #include "ngx_http_lua_misc.h" #include "ngx_http_lua_consts.h" #include "ngx_http_lua_shdict.h" -#include "ngx_http_lua_util.h" -#include "ngx_http_lua_exception.h" #if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) #include #endif From febd730062a80593cadef29fff28f3b4a40c786a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Tue, 19 May 2020 03:39:03 +0800 Subject: [PATCH 406/848] travis-ci: bumped OpenSSL to 1.1.1g. This version contains a high-severity bugfix: https://www.openssl.org/news/secadv/20200421.txt --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 2ca9a75037..b7fa83a91a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,7 @@ env: jobs: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1f OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1g OPENSSL_PATCH_VER=1.1.1f services: - memcached From 8c9968bd471829c94a5a1856d1cf2dc175044181 Mon Sep 17 00:00:00 2001 From: erankor Date: Fri, 29 May 2020 18:51:42 +0300 Subject: [PATCH 407/848] doc: fixed a few typos/wording issues in readme. (#1712) --- README.markdown | 12 ++++++------ doc/HttpLuaModule.wiki | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/README.markdown b/README.markdown index ac563a9044..ee87baeee7 100644 --- a/README.markdown +++ b/README.markdown @@ -1386,7 +1386,7 @@ Usually you can pre-load Lua modules at server start-up by means of this hook an location = /api { content_by_lua_block { -- the following require() will just return - -- the alrady loaded module from package.loaded: + -- the already loaded module from package.loaded: ngx.say(require "cjson".encode{dog = 5, cat = 6}) } } @@ -1875,7 +1875,7 @@ can be implemented in ngx_lua as: Just as any other rewrite phase handlers, [rewrite_by_lua](#rewrite_by_lua) also runs in subrequests. -Note that when calling `ngx.exit(ngx.OK)` within a [rewrite_by_lua](#rewrite_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [rewrite_by_lua](#rewrite_by_lua) handler, calling [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. +Note that when calling `ngx.exit(ngx.OK)` within a [rewrite_by_lua](#rewrite_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [rewrite_by_lua](#rewrite_by_lua) handler, call [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. If the [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s [rewrite](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) directive is used to change the URI and initiate location re-lookups (internal redirections), then any [rewrite_by_lua](#rewrite_by_lua) or [rewrite_by_lua_file](#rewrite_by_lua_file) code sequences within the current location will not be executed. For example, @@ -2017,7 +2017,7 @@ can be implemented in ngx_lua as: As with other access phase handlers, [access_by_lua](#access_by_lua) will *not* run in subrequests. -Note that when calling `ngx.exit(ngx.OK)` within a [access_by_lua](#access_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [access_by_lua](#access_by_lua) handler, calling [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. +Note that when calling `ngx.exit(ngx.OK)` within a [access_by_lua](#access_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [access_by_lua](#access_by_lua) handler, call [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. Starting from the `v0.9.20` release, you can use the [access_by_lua_no_postpone](#access_by_lua_no_postpone) directive to control when to run this handler inside the "access" request-processing phase @@ -6882,7 +6882,7 @@ ngx.shared.DICT.flush_all **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Flushes out all the items in the dictionary. This method does not actuall free up all the memory blocks in the dictionary but just marks all the existing items as expired. +Flushes out all the items in the dictionary. This method does not actually free up all the memory blocks in the dictionary but just marks all the existing items as expired. This feature was first introduced in the `v0.5.0rc17` release. @@ -7818,7 +7818,7 @@ All the Lua code chunks running by [rewrite_by_lua](#rewrite_by_lua), [access_by By default, the corresponding Nginx handler (e.g., [rewrite_by_lua](#rewrite_by_lua) handler) will not terminate until 1. both the "entry thread" and all the user "light threads" terminates, -1. a "light thread" (either the "entry thread" or a user "light thread" aborts by calling [ngx.exit](#ngxexit), [ngx.exec](#ngxexec), [ngx.redirect](#ngxredirect), or [ngx.req.set_uri(uri, true)](#ngxreqset_uri), or +1. a "light thread" (either the "entry thread" or a user "light thread") aborts by calling [ngx.exit](#ngxexit), [ngx.exec](#ngxexec), [ngx.redirect](#ngxredirect), or [ngx.req.set_uri(uri, true)](#ngxreqset_uri), or 1. the "entry thread" terminates with a Lua error. When the user "light thread" terminates with a Lua error, however, it will not abort other running "light threads" like the "entry thread" does. @@ -8283,7 +8283,7 @@ ngx.config.subsystem **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** -This string field indicates the current Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value `"http"`. For +This string field indicates the Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value `"http"`. For [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme), however, this field takes the value `"stream"`. This field was first introduced in the `0.10.1`. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 12b1b40dd9..97f7a4a0ea 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1110,7 +1110,7 @@ Usually you can pre-load Lua modules at server start-up by means of this hook an location = /api { content_by_lua_block { -- the following require() will just return - -- the alrady loaded module from package.loaded: + -- the already loaded module from package.loaded: ngx.say(require "cjson".encode{dog = 5, cat = 6}) } } @@ -1549,7 +1549,7 @@ can be implemented in ngx_lua as: Just as any other rewrite phase handlers, [[#rewrite_by_lua|rewrite_by_lua]] also runs in subrequests. -Note that when calling ngx.exit(ngx.OK) within a [[#rewrite_by_lua|rewrite_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#rewrite_by_lua|rewrite_by_lua]] handler, calling [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. +Note that when calling ngx.exit(ngx.OK) within a [[#rewrite_by_lua|rewrite_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#rewrite_by_lua|rewrite_by_lua]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. If the [[HttpRewriteModule]]'s [[HttpRewriteModule#rewrite|rewrite]] directive is used to change the URI and initiate location re-lookups (internal redirections), then any [[#rewrite_by_lua|rewrite_by_lua]] or [[#rewrite_by_lua_file|rewrite_by_lua_file]] code sequences within the current location will not be executed. For example, @@ -1677,7 +1677,7 @@ can be implemented in ngx_lua as: As with other access phase handlers, [[#access_by_lua|access_by_lua]] will ''not'' run in subrequests. -Note that when calling ngx.exit(ngx.OK) within a [[#access_by_lua|access_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#access_by_lua|access_by_lua]] handler, calling [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. +Note that when calling ngx.exit(ngx.OK) within a [[#access_by_lua|access_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#access_by_lua|access_by_lua]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. Starting from the v0.9.20 release, you can use the [[#access_by_lua_no_postpone|access_by_lua_no_postpone]] directive to control when to run this handler inside the "access" request-processing phase @@ -5815,7 +5815,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Flushes out all the items in the dictionary. This method does not actuall free up all the memory blocks in the dictionary but just marks all the existing items as expired. +Flushes out all the items in the dictionary. This method does not actually free up all the memory blocks in the dictionary but just marks all the existing items as expired. This feature was first introduced in the v0.5.0rc17 release. @@ -6652,7 +6652,7 @@ All the Lua code chunks running by [[#rewrite_by_lua|rewrite_by_lua]], [[#access By default, the corresponding Nginx handler (e.g., [[#rewrite_by_lua|rewrite_by_lua]] handler) will not terminate until # both the "entry thread" and all the user "light threads" terminates, -# a "light thread" (either the "entry thread" or a user "light thread" aborts by calling [[#ngx.exit|ngx.exit]], [[#ngx.exec|ngx.exec]], [[#ngx.redirect|ngx.redirect]], or [[#ngx.req.set_uri|ngx.req.set_uri(uri, true)]], or +# a "light thread" (either the "entry thread" or a user "light thread") aborts by calling [[#ngx.exit|ngx.exit]], [[#ngx.exec|ngx.exec]], [[#ngx.redirect|ngx.redirect]], or [[#ngx.req.set_uri|ngx.req.set_uri(uri, true)]], or # the "entry thread" terminates with a Lua error. When the user "light thread" terminates with a Lua error, however, it will not abort other running "light threads" like the "entry thread" does. @@ -7086,7 +7086,7 @@ This directive was first introduced in the v0.9.20 release. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' -This string field indicates the current Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value "http". For +This string field indicates the Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value "http". For [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module], however, this field takes the value "stream". This field was first introduced in the 0.10.1. From 678988696a9f3d8412f1f14a757ab07e126a9497 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 13 Jun 2020 10:57:08 +0800 Subject: [PATCH 408/848] bugfix: set_by_lua_block allowed more than one arg (in addition to the block). --- src/ngx_http_lua_module.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 5303ae76f8..9816d86441 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -230,10 +230,10 @@ static ngx_command_t ngx_http_lua_cmds[] = { (void *) ngx_http_lua_init_worker_by_file }, #if defined(NDK) && NDK - /* set_by_lua $res { inline Lua code } [$arg1 [$arg2 [...]]] */ + /* set_by_lua_block $res { inline Lua code } */ { ngx_string("set_by_lua_block"), NGX_HTTP_SRV_CONF|NGX_HTTP_SIF_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF - |NGX_CONF_1MORE|NGX_CONF_BLOCK, + |NGX_CONF_TAKE1|NGX_CONF_BLOCK, ngx_http_lua_set_by_lua_block, NGX_HTTP_LOC_CONF_OFFSET, 0, From 23d2bc10a16fc56fef36df3344d4cf5ffa1681f1 Mon Sep 17 00:00:00 2001 From: zhuizhuhaomeng <151090888@qq.com> Date: Sat, 25 Apr 2020 16:41:11 +0800 Subject: [PATCH 409/848] bugfix: ngx.req.set_uri_args() threw an exception with wrong argument info. --- src/ngx_http_lua_args.c | 2 +- t/030-uri-args.t | 71 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 71 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index 6da3759212..45ee6023ca 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -64,7 +64,7 @@ ngx_http_lua_ngx_req_set_uri_args(lua_State *L) default: msg = lua_pushfstring(L, "string, number, or table expected, " - "but got %s", luaL_typename(L, 2)); + "but got %s", luaL_typename(L, 1)); return luaL_argerror(L, 1, msg); } diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 02fe6e1042..951355e527 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,7 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 21); +plan tests => repeat_each() * (blocks() * 2 + 24); no_root_location(); @@ -1617,3 +1617,72 @@ request_uri: /foo%20bar uri: /foo bar --- no_error_log [error] + + + +=== TEST 61: set_uri_args with boolean +--- config + location /bar { + echo $query_string; + } + location /foo { + #set $args 'hello'; + rewrite_by_lua_block { + ngx.req.set_uri_args(true) + ngx.req.set_uri("/bar", true) + } + proxy_pass http://127.0.0.2:12345; + } +--- request + GET /foo?world +--- response_body_like: 500 Internal Server Error +--- log_level: debug +--- error_code: 500 +--- error_log +bad argument #1 to 'set_uri_args' (string, number, or table expected, but got boolean) + + + +=== TEST 62: set_uri_args with nil +--- config + location /bar { + echo $query_string; + } + location /foo { + #set $args 'hello'; + rewrite_by_lua_block { + ngx.req.set_uri_args(nil) + ngx.req.set_uri("/bar", true) + } + proxy_pass http://127.0.0.2:12345; + } +--- request + GET /foo?world +--- response_body_like: 500 Internal Server Error +--- log_level: debug +--- error_code: 500 +--- error_log +bad argument #1 to 'set_uri_args' (string, number, or table expected, but got nil) + + + +=== TEST 63: set_uri_args with userdata +--- config + location /bar { + echo $query_string; + } + location /foo { + #set $args 'hello'; + rewrite_by_lua_block { + ngx.req.set_uri_args(ngx.null) + ngx.req.set_uri("/bar", true) + } + proxy_pass http://127.0.0.2:12345; + } +--- request + GET /foo?world +--- response_body_like: 500 Internal Server Error +--- log_level: debug +--- error_code: 500 +--- error_log +bad argument #1 to 'set_uri_args' (string, number, or table expected, but got userdata) From d278043fb0f9856dac00a428c90f237ae43ffc9a Mon Sep 17 00:00:00 2001 From: wanghuizzz Date: Fri, 17 Apr 2020 01:48:46 -0400 Subject: [PATCH 410/848] doc: added docs for working around memory fragmentation issues to the shdict:set() method. --- README.markdown | 4 ++++ doc/HttpLuaModule.wiki | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/README.markdown b/README.markdown index ee87baeee7..e36a4c83d6 100644 --- a/README.markdown +++ b/README.markdown @@ -6548,6 +6548,10 @@ The optional `flags` argument specifies a user flags value associated with the e When it fails to allocate memory for the current key-value item, then `set` will try removing existing items in the storage according to the Least-Recently Used (LRU) algorithm. Note that, LRU takes priority over expiration time here. If up to tens of existing items have been removed and the storage left is still insufficient (either due to the total capacity limit specified by [lua_shared_dict](#lua_shared_dict) or memory segmentation), then the `err` return value will be `no memory` and `success` will be `false`. +If the sizes of items in the dictionary are not multiples or even powers of a certain value (like 2), it is easier to encounter `no memory` error because of memory fragmentation. It is recommended to use different dictionaries for different sizes of items. + +When you encounter `no memory` error, you can also evict more least-recently-used items by retrying this method call more times to to make room for the current item. + If this method succeeds in storing the current item by forcibly removing other not-yet-expired items in the dictionary via LRU, the `forcible` return value will be `true`. If it stores the item without forcibly removing other valid items, then the return value `forcible` will be `false`. The first argument to this method must be the dictionary object itself, for example, diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 97f7a4a0ea..a3b6ddab46 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5528,6 +5528,10 @@ The optional flags argument specifies a user flags value associated When it fails to allocate memory for the current key-value item, then set will try removing existing items in the storage according to the Least-Recently Used (LRU) algorithm. Note that, LRU takes priority over expiration time here. If up to tens of existing items have been removed and the storage left is still insufficient (either due to the total capacity limit specified by [[#lua_shared_dict|lua_shared_dict]] or memory segmentation), then the err return value will be no memory and success will be false. +If the sizes of items in the dictionary are not multiples or even powers of a certain value (like 2), it is easier to encounter no memory error because of memory fragmentation. It is recommended to use different dictionaries for different sizes of items. + +When you encounter no memory error, you can also evict more least-recently-used items by retrying this method call more times to to make room for the current item. + If this method succeeds in storing the current item by forcibly removing other not-yet-expired items in the dictionary via LRU, the forcible return value will be true. If it stores the item without forcibly removing other valid items, then the return value forcible will be false. The first argument to this method must be the dictionary object itself, for example, From 8ac6cc7285994026645deb8d24f185e64db8c2d8 Mon Sep 17 00:00:00 2001 From: rainingmaster <312841925@qq.com> Date: Sat, 4 Apr 2020 16:58:16 +0800 Subject: [PATCH 411/848] doc: made the code examples more realistic (better for direct copy&paste use). --- README.markdown | 49 ++++++++++++++++++++++++++++-------------- doc/HttpLuaModule.wiki | 49 ++++++++++++++++++++++++++++-------------- 2 files changed, 66 insertions(+), 32 deletions(-) diff --git a/README.markdown b/README.markdown index e36a4c83d6..21917d4bf4 100644 --- a/README.markdown +++ b/README.markdown @@ -1512,6 +1512,8 @@ This hook is often used to create per-worker reoccurring timers (via the [ngx.ti return end end + + -- do something in timer end local hdl, err = new_timer(delay, check) @@ -1519,6 +1521,8 @@ This hook is often used to create per-worker reoccurring timers (via the [ngx.ti log(ERR, "failed to create timer: ", err) return end + + -- other job in init_worker_by_lua '; ``` @@ -6292,13 +6296,13 @@ When the `replace` is a string, then it is treated as a special template for str ```lua local newstr, n, err = ngx.re.sub("hello, 1234", "([0-9])[0-9]", "[$0][$1]") - if newstr then - -- newstr == "hello, [12][1]34" - -- n == 1 - else + if not newstr then ngx.log(ngx.ERR, "error: ", err) return end + + -- newstr == "hello, [12][1]34" + -- n == 1 ``` where `$0` referring to the whole substring matched by the pattern and `$1` referring to the first parenthesized capturing substring. @@ -6308,8 +6312,8 @@ Curly braces can also be used to disambiguate variable names from the background ```lua local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "${0}00") - -- newstr == "hello, 100234" - -- n == 1 + -- newstr == "hello, 100234" + -- n == 1 ``` Literal dollar sign characters (`$`) in the `replace` string argument can be escaped by another dollar sign, for instance, @@ -6317,8 +6321,8 @@ Literal dollar sign characters (`$`) in the `replace` string argument can be esc ```lua local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "$$") - -- newstr == "hello, $234" - -- n == 1 + -- newstr == "hello, $234" + -- n == 1 ``` Do not use backlashes to escape dollar signs; it will not work as expected. @@ -6330,9 +6334,10 @@ When the `replace` argument is of type "function", then it will be invoked with local func = function (m) return "[" .. m[0] .. "][" .. m[1] .. "]" end + local newstr, n, err = ngx.re.sub("hello, 1234", "( [0-9] ) [0-9]", func, "x") - -- newstr == "hello, [12][1]34" - -- n == 1 + -- newstr == "hello, [12][1]34" + -- n == 1 ``` The dollar sign characters in the return value of the `replace` function argument are not special at all. @@ -6357,13 +6362,13 @@ Here is some examples: ```lua local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", "[$0,$1]", "i") - if newstr then - -- newstr == "[hello,h], [world,w]" - -- n == 2 - else + if not newstr then ngx.log(ngx.ERR, "error: ", err) return end + + -- newstr == "[hello,h], [world,w]" + -- n == 2 ``` ```lua @@ -6372,8 +6377,8 @@ Here is some examples: return "[" .. m[0] .. "," .. m[1] .. "]" end local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", func, "i") - -- newstr == "[hello,h], [world,w]" - -- n == 2 + -- newstr == "[hello,h], [world,w]" + -- n == 2 ``` This method requires the PCRE library enabled in Nginx ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)). @@ -7078,6 +7083,9 @@ Since the `v0.7.18` release, connecting to a datagram unix domain socket file is ngx.say("failed to connect to the datagram unix domain socket: ", err) return end + + -- do something after connect + -- such as sock:send or sock:receive ``` assuming the datagram service is listening on the unix domain socket file `/tmp/some-datagram-service.sock` and the client socket will use the "autobind" feature on Linux. @@ -7282,6 +7290,9 @@ Connecting to a Unix Domain Socket file is also possible: ngx.say("failed to connect to the memcached unix domain socket: ", err) return end + + -- do something after connect + -- such as sock:send or sock:receive ``` assuming memcached (or something else) is listening on the unix domain socket file `/tmp/memcached.sock`. @@ -8160,6 +8171,8 @@ Here is a simple example: ngx.log(ngx.ERR, "failed to create timer: ", err) return end + + -- other job in log_by_lua_block } } ``` @@ -8180,6 +8193,8 @@ One can also create infinite re-occurring timers, for instance, a timer getting ngx.log(ngx.ERR, "failed to create the timer: ", err) return end + + -- do something in timer end local ok, err = ngx.timer.at(delay, handler) @@ -8187,6 +8202,8 @@ One can also create infinite re-occurring timers, for instance, a timer getting ngx.log(ngx.ERR, "failed to create the timer: ", err) return end + + -- do other jobs ``` It is recommended, however, to use the [ngx.timer.every](#ngxtimerevery) API function diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a3b6ddab46..ee3d9c122c 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1224,6 +1224,8 @@ This hook is often used to create per-worker reoccurring timers (via the [[#ngx. return end end + + -- do something in timer end local hdl, err = new_timer(delay, check) @@ -1231,6 +1233,8 @@ This hook is often used to create per-worker reoccurring timers (via the [[#ngx. log(ERR, "failed to create timer: ", err) return end + + -- other job in init_worker_by_lua '; @@ -5296,13 +5300,13 @@ When the replace is a string, then it is treated as a special templ local newstr, n, err = ngx.re.sub("hello, 1234", "([0-9])[0-9]", "[$0][$1]") - if newstr then - -- newstr == "hello, [12][1]34" - -- n == 1 - else + if not newstr then ngx.log(ngx.ERR, "error: ", err) return end + + -- newstr == "hello, [12][1]34" + -- n == 1 where $0 referring to the whole substring matched by the pattern and $1 referring to the first parenthesized capturing substring. @@ -5311,16 +5315,16 @@ Curly braces can also be used to disambiguate variable names from the background local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "${0}00") - -- newstr == "hello, 100234" - -- n == 1 + -- newstr == "hello, 100234" + -- n == 1 Literal dollar sign characters ($) in the replace string argument can be escaped by another dollar sign, for instance, local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "$$") - -- newstr == "hello, $234" - -- n == 1 + -- newstr == "hello, $234" + -- n == 1 Do not use backlashes to escape dollar signs; it will not work as expected. @@ -5331,9 +5335,10 @@ When the replace argument is of type "function", then it will be in local func = function (m) return "[" .. m[0] .. "][" .. m[1] .. "]" end + local newstr, n, err = ngx.re.sub("hello, 1234", "( [0-9] ) [0-9]", func, "x") - -- newstr == "hello, [12][1]34" - -- n == 1 + -- newstr == "hello, [12][1]34" + -- n == 1 The dollar sign characters in the return value of the replace function argument are not special at all. @@ -5354,13 +5359,13 @@ Here is some examples: local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", "[$0,$1]", "i") - if newstr then - -- newstr == "[hello,h], [world,w]" - -- n == 2 - else + if not newstr then ngx.log(ngx.ERR, "error: ", err) return end + + -- newstr == "[hello,h], [world,w]" + -- n == 2 @@ -5368,8 +5373,8 @@ Here is some examples: return "[" .. m[0] .. "," .. m[1] .. "]" end local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", func, "i") - -- newstr == "[hello,h], [world,w]" - -- n == 2 + -- newstr == "[hello,h], [world,w]" + -- n == 2 This method requires the PCRE library enabled in Nginx ([[#Special Escaping Sequences|Known Issue With Special Escaping Sequences]]). @@ -5988,6 +5993,9 @@ Since the v0.7.18 release, connecting to a datagram unix domain soc ngx.say("failed to connect to the datagram unix domain socket: ", err) return end + + -- do something after connect + -- such as sock:send or sock:receive assuming the datagram service is listening on the unix domain socket file /tmp/some-datagram-service.sock and the client socket will use the "autobind" feature on Linux. @@ -6167,6 +6175,9 @@ Connecting to a Unix Domain Socket file is also possible: ngx.say("failed to connect to the memcached unix domain socket: ", err) return end + + -- do something after connect + -- such as sock:send or sock:receive assuming memcached (or something else) is listening on the unix domain socket file /tmp/memcached.sock. @@ -6976,6 +6987,8 @@ Here is a simple example: ngx.log(ngx.ERR, "failed to create timer: ", err) return end + + -- other job in log_by_lua_block } } @@ -6995,6 +7008,8 @@ One can also create infinite re-occurring timers, for instance, a timer getting ngx.log(ngx.ERR, "failed to create the timer: ", err) return end + + -- do something in timer end local ok, err = ngx.timer.at(delay, handler) @@ -7002,6 +7017,8 @@ One can also create infinite re-occurring timers, for instance, a timer getting ngx.log(ngx.ERR, "failed to create the timer: ", err) return end + + -- do other jobs It is recommended, however, to use the [[#ngx.timer.every|ngx.timer.every]] API function From c86819f0b0972d2d05679c790ecbbbfa73339ae9 Mon Sep 17 00:00:00 2001 From: Yuchen Wu Date: Fri, 19 Jun 2020 13:59:43 -0700 Subject: [PATCH 412/848] doc: updated the docs on the limits of subrequests to reflect recent changes in the nginx core. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 21917d4bf4..77ca7ce32f 100644 --- a/README.markdown +++ b/README.markdown @@ -4044,7 +4044,7 @@ in gzipped responses that cannot be handled properly in Lua code. Original reque When the `body` option is not specified and the `always_forward_body` option is false (the default value), the `POST` and `PUT` subrequests will inherit the request bodies of the parent request (if any). -There is a hard-coded upper limit on the number of concurrent subrequests possible for every main request. In older versions of Nginx, the limit was `50` concurrent subrequests and in more recent versions, Nginx `1.1.x` onwards, this was increased to `200` concurrent subrequests. When this limit is exceeded, the following error message is added to the `error.log` file: +There is a hard-coded upper limit on the number of subrequests possible for every main request. In older versions of Nginx, the limit was `50` concurrent subrequests and in more recent versions, Nginx `1.9.5` onwards, the same limit is changed to limit the depth of recursive subrequests. When this limit is exceeded, the following error message is added to the `error.log` file: [error] 13983#0: *1 subrequests cycle while processing "/uri" diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ee3d9c122c..db9367a6ca 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3348,7 +3348,7 @@ in gzipped responses that cannot be handled properly in Lua code. Original reque When the body option is not specified and the always_forward_body option is false (the default value), the POST and PUT subrequests will inherit the request bodies of the parent request (if any). -There is a hard-coded upper limit on the number of concurrent subrequests possible for every main request. In older versions of Nginx, the limit was 50 concurrent subrequests and in more recent versions, Nginx 1.1.x onwards, this was increased to 200 concurrent subrequests. When this limit is exceeded, the following error message is added to the error.log file: +There is a hard-coded upper limit on the number of subrequests possible for every main request. In older versions of Nginx, the limit was 50 concurrent subrequests and in more recent versions, Nginx 1.9.5 onwards, the same limit is changed to limit the depth of recursive subrequests. When this limit is exceeded, the following error message is added to the error.log file: [error] 13983#0: *1 subrequests cycle while processing "/uri" From f655b0e19a34c206ce8a2dddbb1b93b9293c221d Mon Sep 17 00:00:00 2001 From: rainingmaster <312841925@qq.com> Date: Wed, 10 Jun 2020 11:55:52 +0800 Subject: [PATCH 413/848] feature: add mergify.yml to manage our pr --- .mergify.yml | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 .mergify.yml diff --git a/.mergify.yml b/.mergify.yml new file mode 100644 index 0000000000..c89be4fb54 --- /dev/null +++ b/.mergify.yml @@ -0,0 +1,35 @@ +--- +pull_request_rules: + - name: warn on conflicts + conditions: + - conflict + actions: + comment: + message: This pull request is now in conflict :( + label: + add: + - conflict + - name: remove conflict label if not needed + conditions: + - -conflict + actions: + label: + remove: + - conflict + - name: add label needs-test-cases + conditions: + - files~=^src/ + - -files~=^t/ + actions: + label: + add: + - needs-test-cases + - name: remove label needs-test-cases + conditions: + - label=needs-test-cases + - files~=^src/ + - files~=^t/ + actions: + label: + remove: + - needs-test-cases From 99a5a6bcb4d72d07467f1dd9c82b7a8d2c6055ec Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 8 Jun 2020 14:27:40 +0800 Subject: [PATCH 414/848] test: used the iptable -I option to insert rule to the head of the filter chain. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index b7fa83a91a..fc2e4c20bf 100644 --- a/.travis.yml +++ b/.travis.yml @@ -94,8 +94,8 @@ before_script: script: - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT - - sudo iptables -A OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP - - sudo iptables -A OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP + - sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP + - sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP - cd luajit2/ - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1) - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1) From 785fb57094e3dbb161962875800e9cf92ce3717b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 6 Jun 2020 14:51:07 +0800 Subject: [PATCH 415/848] feature: ngx.req.set_uri_args() now automatically escapes control and whitespace characters if the query-string is provided directly. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 7 +- doc/HttpLuaModule.wiki | 7 +- src/ngx_http_lua_args.c | 88 +++++++++++++++++++++++- t/030-uri-args-with-ctrl.t | 137 +++++++++++++++++++++++++++++++++++++ 4 files changed, 236 insertions(+), 3 deletions(-) create mode 100644 t/030-uri-args-with-ctrl.t diff --git a/README.markdown b/README.markdown index 77ca7ce32f..c822b3d498 100644 --- a/README.markdown +++ b/README.markdown @@ -4552,7 +4552,12 @@ or a Lua table holding the query arguments' key-value pairs, as in ngx.req.set_uri_args({ a = 3, b = "hello world" }) ``` -where in the latter case, this method will escape argument keys and values according to the URI escaping rule. +In the former case, i.e., when the whole query-strng is provided directly, +the input Lua string should already be well-formed with the URI encoding. +For security considerations, his method will autoamticaly escape any control and +whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. + +In the latter case, this method will escape argument keys and values according to the URI escaping rule. Multi-value arguments are also supported: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index db9367a6ca..6f99701605 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3791,7 +3791,12 @@ or a Lua table holding the query arguments' key-value pairs, as in ngx.req.set_uri_args({ a = 3, b = "hello world" }) -where in the latter case, this method will escape argument keys and values according to the URI escaping rule. +In the former case, i.e., when the whole query-strng is provided directly, +the input Lua string should already be well-formed with the URI encoding. +For security considerations, his method will autoamticaly escape any control and +whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. + +In the latter case, this method will escape argument keys and values according to the URI escaping rule. Multi-value arguments are also supported: diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index 45ee6023ca..7b7d5eae6c 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -19,6 +19,66 @@ static int ngx_http_lua_ngx_req_set_uri_args(lua_State *L); static int ngx_http_lua_ngx_req_get_post_args(lua_State *L); +uintptr_t +ngx_http_lua_escape_args(u_char *dst, u_char *src, size_t size) +{ + ngx_uint_t n; + static u_char hex[] = "0123456789ABCDEF"; + + /* %00-%20 %7F*/ + + static uint32_t escape[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0x00000001, /* 0000 0000 0000 0000 0000 0000 0000 0001 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + }; + + if (dst == NULL) { + + /* find the number of the characters to be escaped */ + + n = 0; + + while (size) { + if (escape[*src >> 5] & (1 << (*src & 0x1f))) { + n++; + } + src++; + size--; + } + + return (uintptr_t) n; + } + + while (size) { + if (escape[*src >> 5] & (1 << (*src & 0x1f))) { + *dst++ = '%'; + *dst++ = hex[*src >> 4]; + *dst++ = hex[*src & 0xf]; + src++; + + } else { + *dst++ = *src++; + } + size--; + } + + return (uintptr_t) dst; +} + + static int ngx_http_lua_ngx_req_set_uri_args(lua_State *L) { @@ -27,6 +87,7 @@ ngx_http_lua_ngx_req_set_uri_args(lua_State *L) const char *msg; size_t len; u_char *p; + uintptr_t escape; if (lua_gettop(L) != 1) { return luaL_error(L, "expecting 1 argument but seen %d", @@ -42,7 +103,6 @@ ngx_http_lua_ngx_req_set_uri_args(lua_State *L) switch (lua_type(L, 1)) { case LUA_TNUMBER: - case LUA_TSTRING: p = (u_char *) lua_tolstring(L, 1, &len); args.data = ngx_palloc(r->pool, len); @@ -55,6 +115,32 @@ ngx_http_lua_ngx_req_set_uri_args(lua_State *L) args.len = len; break; + case LUA_TSTRING: + p = (u_char *) lua_tolstring(L, 1, &len); + + escape = ngx_http_lua_escape_args(NULL, p, len); + if (escape > 0) { + args.len = len + 2 * escape; + args.data = ngx_palloc(r->pool, args.len); + if (args.data == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_args(args.data, p, len); + + } else { + args.data = ngx_palloc(r->pool, len); + if (args.data == NULL) { + return luaL_error(L, "no memory"); + } + + ngx_memcpy(args.data, p, len); + + args.len = len; + } + + break; + case LUA_TTABLE: ngx_http_lua_process_args_option(r, L, 1, &args); diff --git a/t/030-uri-args-with-ctrl.t b/t/030-uri-args-with-ctrl.t new file mode 100644 index 0000000000..e74da38eeb --- /dev/null +++ b/t/030-uri-args-with-ctrl.t @@ -0,0 +1,137 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: +use Test::Nginx::Socket::Lua; + +log_level('warn'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 2 + 3); + +no_root_location(); + +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: rewrite args (string with \r) +--- config + location /foo { + rewrite_by_lua_block { + ngx.req.set_uri_args("a\rb") + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/echo; + } + location /echo { + content_by_lua_block { + ngx.say(ngx.var.request_uri); + } + } +--- request +GET /foo?world +--- error_code: 200 +--- response_body +/echo?a%0Db + + + +=== TEST 2: rewrite args (string with \n) +--- config + location /foo { + rewrite_by_lua_block { + ngx.req.set_uri_args("a\nb") + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/echo; + } + location /echo { + content_by_lua_block { + ngx.say(ngx.var.request_uri); + } + } +--- request +GET /foo?world +--- response_body +/echo?a%0Ab + + + +=== TEST 3: rewrite args (string with \0) +--- config + location /foo { + rewrite_by_lua_block { + ngx.req.set_uri_args("a\0b") + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/echo; + } + location /echo { + content_by_lua_block { + ngx.say(ngx.var.request_uri); + } + } +--- request +GET /foo?world +--- response_body +/echo?a%00b + + + +=== TEST 4: rewrite args (string arg with 'lang=中文') +ngx.req.set_uri_args with string argument should be carefully encoded. +For backward compatibility, we are allowed to pass such parameters. +--- config + location /foo { + rewrite_by_lua_block { + ngx.req.set_uri_args("lang=中文") + } + content_by_lua_block { + ngx.say(ngx.var.arg_lang) + } + } +--- request +GET /foo?world +--- response_body +中文 +--- no_error_log +[error] + + + +=== TEST 5: rewrite args (string arg with '语言=chinese') +ngx.req.set_uri_args with string argument should be carefully encoded. +For backward compatibility, we are allowed to pass such parameters. +--- config + location /foo { + rewrite_by_lua_block { + ngx.req.set_uri_args("语言=chinese") + } + content_by_lua_block { + ngx.say(ngx.var.arg_语言) + } + } +--- request +GET /foo?world +--- response_body +chinese +--- no_error_log +[error] + + + +=== TEST 6: rewrite args (string arg with '语言=中文') +ngx.req.set_uri_args with string argument should be carefully encoded. +For backward compatibility, we are allowed to pass such parameters. +--- config + location /foo { + rewrite_by_lua_block { + ngx.req.set_uri_args("语言=中文") + } + content_by_lua_block { + ngx.say(ngx.var.arg_语言) + } + } +--- request +GET /foo?world +--- response_body +中文 +--- no_error_log +[error] From c6ffcfbf375b044792db17c739df8e5dcf99298a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 6 Jun 2020 10:47:39 +0800 Subject: [PATCH 416/848] tests: added test cases for the nginx core patch static_mod_escape_loc_hdr. See https://github.com/openresty/openresty/blob/master/patches/nginx-1.17.8-static_mod_escape_loc_hdr.patch and https://github.com/openresty/openresty/commit/6985198d. --- src/ngx_http_lua_uri.c | 4 -- t/030-uri-args.t | 16 +++--- t/162-static-module-location.t | 95 ++++++++++++++++++++++++++++++++++ 3 files changed, 102 insertions(+), 13 deletions(-) create mode 100644 t/162-static-module-location.t diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index 569cb50270..b605bf2b89 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -55,10 +55,6 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return luaL_error(L, "attempt to use zero-length uri"); } - if (ngx_http_lua_check_unsafe_string(r, p, len, "uri") != NGX_OK) { - return luaL_error(L, "attempt to set unsafe uri"); - } - if (n == 2) { luaL_checktype(L, 2, LUA_TBOOLEAN); diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 951355e527..b9dba2b736 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,7 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 24); +plan tests => repeat_each() * (blocks() * 2 + 22); no_root_location(); @@ -1568,10 +1568,9 @@ args: foo=%2C%24%40%7C%60&bar=-_.!~*'() } --- request GET /t ---- error_code: 500 ---- error_log -unsafe byte "0x9" in uri "/foo\x09bar" -attempt to set unsafe uri +--- error_code: 200 +--- response_body eval +qr#/foo\tbar# @@ -1586,10 +1585,9 @@ attempt to set unsafe uri } --- request GET /t ---- error_code: 500 ---- error_log -unsafe byte "0x0" in uri "\x00foo" -attempt to set unsafe uri +--- error_code: 200 +--- response_body eval +qr/\0foo/ diff --git a/t/162-static-module-location.t b/t/162-static-module-location.t new file mode 100644 index 0000000000..1dce330e77 --- /dev/null +++ b/t/162-static-module-location.t @@ -0,0 +1,95 @@ +use Test::Nginx::Socket::Lua; + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3); + +our $HtmlDir = html_dir; + +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: decoded url contains '\0' and '\r\n' +--- config + server_tokens off; + location = /t { + rewrite_by_lua_block { + ngx.req.read_body(); + local args, _ = ngx.req.get_post_args(); + ngx.req.set_uri(args["url"], true); + } + } +--- request +POST /t +url=%00%0a%0dset-cookie:1234567 +--- error_code: 301 +--- response_headers +Location: %00%0A%0Dset-cookie:1234567/ +--- response_body_like +.*301 Moved Permanently.* + + + +=== TEST 2: uri contain chinese characters +--- config + server_tokens off; +--- user_files +>>> t/中文/foo.txt +Hello, world +--- request +GET /t/中文 +--- error_code: 301 +--- response_headers +Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/ +--- response_body_like +.*301 Moved Permanently.* + + + +=== TEST 3: uri contain chinese characters with args +--- config + server_tokens off; +--- user_files +>>> t/中文/foo.txt +Hello, world +--- request +GET /t/中文?q=name +--- error_code: 301 +--- response_headers +Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/?q=name +--- response_body_like +.*301 Moved Permanently.* + + + +=== TEST 4: uri already encoded +--- config + server_tokens off; +--- user_files +>>> t/中文/foo.txt +Hello, world +--- request +GET /t/%E4%B8%AD%E6%96%87 +--- error_code: 301 +--- response_headers +Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/ +--- response_body_like +.*301 Moved Permanently.* + + + +=== TEST 5: uri already encoded with args +--- config + server_tokens off; +--- user_files +>>> t/中文/foo.txt +Hello, world +--- request +GET /t/%E4%B8%AD%E6%96%87?q=name +--- error_code: 301 +--- response_headers +Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/?q=name +--- response_body_like +.*301 Moved Permanently.* From 2a6c9a63e2ef4720509408f9893e8b8aefd0f223 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 27 Jun 2020 17:05:44 -0700 Subject: [PATCH 417/848] doc: minor typo fixes. --- doc/HttpLuaModule.wiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 6f99701605..3f44adfa3e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3791,9 +3791,9 @@ or a Lua table holding the query arguments' key-value pairs, as in ngx.req.set_uri_args({ a = 3, b = "hello world" }) -In the former case, i.e., when the whole query-strng is provided directly, +In the former case, i.e., when the whole query-string is provided directly, the input Lua string should already be well-formed with the URI encoding. -For security considerations, his method will autoamticaly escape any control and +For security considerations, his method will automatically escape any control and whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. In the latter case, this method will escape argument keys and values according to the URI escaping rule. From be35318272782313e11674304c91d7657742ee41 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 27 Jun 2020 17:09:26 -0700 Subject: [PATCH 418/848] doc: more typo fixes. --- doc/HttpLuaModule.wiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 3f44adfa3e..7a624d4031 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3793,7 +3793,7 @@ or a Lua table holding the query arguments' key-value pairs, as in In the former case, i.e., when the whole query-string is provided directly, the input Lua string should already be well-formed with the URI encoding. -For security considerations, his method will automatically escape any control and +For security considerations, this method will automatically escape any control and whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. In the latter case, this method will escape argument keys and values according to the URI escaping rule. From e1e335fd7f72bf2f1dad20800c33f79ff3e0d1c4 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 7 Jun 2020 10:17:41 +0800 Subject: [PATCH 419/848] security: ngx.req.set_header(): now we always escape bytes in header names and header values which are prohibited by RFC 7230. See https://tools.ietf.org/html/rfc7230#section-3.2.6 for more details. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 9 +++- doc/HttpLuaModule.wiki | 5 ++ src/ngx_http_lua_headers_in.c | 14 +++--- src/ngx_http_lua_headers_out.c | 10 ++-- src/ngx_http_lua_util.c | 80 +++++++++++++++++++++++++++++- src/ngx_http_lua_util.h | 6 +++ t/016-resp-header.t | 57 +++++++++------------ t/028-req-header.t | 91 +++++++++++++++++++++++++++------- t/030-uri-args.t | 8 +-- t/113-req-header-cookie.t | 13 ++--- 10 files changed, 219 insertions(+), 74 deletions(-) diff --git a/README.markdown b/README.markdown index c822b3d498..9aa0e4c4e3 100644 --- a/README.markdown +++ b/README.markdown @@ -4552,9 +4552,9 @@ or a Lua table holding the query arguments' key-value pairs, as in ngx.req.set_uri_args({ a = 3, b = "hello world" }) ``` -In the former case, i.e., when the whole query-strng is provided directly, +In the former case, i.e., when the whole query-string is provided directly, the input Lua string should already be well-formed with the URI encoding. -For security considerations, his method will autoamticaly escape any control and +For security considerations, this method will automatically escape any control and whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. In the latter case, this method will escape argument keys and values according to the URI escaping rule. @@ -4883,6 +4883,11 @@ ngx.req.set_header Set the current request's request header named `header_name` to value `header_value`, overriding any existing ones. +The input Lua string `header_name` and `header_value` should already be well-formed with the URI encoding. +For security considerations, this method will automatically escape " ", """, "(", ")", ",", "/", ":", ";", "?", +"<", "=", ">", "?", "@", "[", "]", "\", "{", "}", 0x00-0x1F, 0x7F-0xFF in `header_name` and automatically escape +"0x00-0x08, 0x0A-0x0F, 0x7F in `header_value`. + By default, all the subrequests subsequently initiated by [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) will inherit the new header. Here is an example of setting the `Content-Type` header: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 7a624d4031..bfacf86cb0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4084,6 +4084,11 @@ The __index metamethod will not be added when the raw Set the current request's request header named header_name to value header_value, overriding any existing ones. +The input Lua string `header_name` and `header_value` should already be well-formed with the URI encoding. +For security considerations, this method will automatically escape " ", """, "(", ")", ",", "/", ":", ";", "?", +"<", "=", ">", "?", "@", "[", "]", "\", "{", "}", 0x00-0x1F, 0x7F-0xFF in `header_name` and automatically escape +"0x00-0x08, 0x0A-0x0F, 0x7F in `header_value`. + By default, all the subrequests subsequently initiated by [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] will inherit the new header. Here is an example of setting the Content-Type header: diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 34f2fb667f..713818efc2 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -653,16 +653,18 @@ ngx_http_lua_set_input_header(ngx_http_request_t *r, ngx_str_t key, { ngx_http_lua_header_val_t hv; ngx_http_lua_set_header_t *handlers = ngx_http_lua_set_handlers; - + ngx_int_t rc; ngx_uint_t i; dd("set header value: %.*s", (int) value.len, value.data); - if (ngx_http_lua_check_unsafe_string(r, key.data, key.len, - "header name") != NGX_OK - || ngx_http_lua_check_unsafe_string(r, value.data, value.len, - "header value") != NGX_OK) - { + rc = ngx_http_lua_copy_escaped_header(r, &key, 1); + if (rc != NGX_OK) { + return NGX_ERROR; + } + + rc = ngx_http_lua_copy_escaped_header(r, &value, 0); + if (rc != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index a0a9012643..a3b0f9e90e 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -491,11 +491,11 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, dd("set header value: %.*s", (int) value.len, value.data); - if (ngx_http_lua_check_unsafe_string(r, key.data, key.len, - "header name") != NGX_OK - || ngx_http_lua_check_unsafe_string(r, value.data, value.len, - "header value") != NGX_OK) - { + if (ngx_http_lua_copy_escaped_header(r, &key, 1) != NGX_OK) { + return NGX_ERROR; + } + + if (ngx_http_lua_copy_escaped_header(r, &value, 0) != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 983f476bb4..0154796b34 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -2008,8 +2008,52 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) /* mail_auth is the same as memcached */ + /* " ", """, "(", ")", ",", "/", ":", ";", "?", + * "<", "=", ">", "?", "@", "[", "]", "\", "{", + * "}", %00-%1F, %7F-%FF + */ + + static uint32_t header_name[] = { + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0xfc009305, /* 1111 1100 0000 0000 1001 0011 0000 0101 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x38000001, /* 0011 1000 0000 0000 0000 0000 0000 0001 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0xa8000000, /* 1010 1000 0000 0000 0000 0000 0000 0000 */ + + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + }; + + /* "%00-%08, %0A-%0F, %7F */ + + static uint32_t header_value[] = { + 0xfffffdff, /* 1111 1111 1111 1111 1111 1101 1111 1111 */ + + /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + + /* ~}| {zyx wvut srqp onml kjih gfed cba` */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ + + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ + }; + static uint32_t *map[] = - { uri, args, uri_component, html, refresh, memcached, memcached }; + { uri, args, uri_component, html, refresh, memcached, memcached, + header_name, header_value }; escape = map[type]; @@ -4328,4 +4372,38 @@ ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size) } +ngx_int_t +ngx_http_lua_copy_escaped_header(ngx_http_request_t *r, + ngx_str_t *dst, int is_name) +{ + size_t escape; + size_t len; + u_char *data; + int type; + + type = is_name + ? NGX_HTTP_LUA_ESCAPE_HEADER_NAME : NGX_HTTP_LUA_ESCAPE_HEADER_VALUE; + + data = dst->data; + len = dst->len; + + escape = ngx_http_lua_escape_uri(NULL, data, len, type); + if (escape > 0) { + /* + * we allocate space for the trailling '\0' char here because nginx + * header values must be null-terminated + */ + dst->data = ngx_palloc(r->pool, len + 2 * escape + 1); + if (dst->data == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_uri(dst->data, data, len, type); + dst->len = len + 2 * escape; + dst->data[dst->len] = '\0'; + } + + return NGX_OK; +} + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index a0ab2de357..92f3ba4737 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -27,6 +27,9 @@ # define NGX_HTTP_SWITCHING_PROTOCOLS 101 #endif +#define NGX_HTTP_LUA_ESCAPE_HEADER_NAME 7 + +#define NGX_HTTP_LUA_ESCAPE_HEADER_VALUE 8 /* key in Lua vm registry for all the "ngx.ctx" tables */ #define ngx_http_lua_ctx_tables_key "ngx_lua_ctx_tables" @@ -169,6 +172,9 @@ void ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, uintptr_t ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type); +ngx_int_t ngx_http_lua_copy_escaped_header(ngx_http_request_t *r, + ngx_str_t *dst, int is_name); + void ngx_http_lua_inject_req_api(ngx_log_t *log, lua_State *L); void ngx_http_lua_process_args_option(ngx_http_request_t *r, diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 3d38d8f21b..febd276fef 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 80); +plan tests => repeat_each() * (blocks() * 3 + 77); #no_diff(); no_long_string(); @@ -51,7 +51,7 @@ Content-Type: text/html -=== TEST 3: set response content-type header +=== TEST 3: set response content-length header --- config location /read { content_by_lua ' @@ -1976,14 +1976,12 @@ Content-Type: application/json } --- request GET /t ---- error_code: 500 --- response_headers -header: +header: value%0Dfoo:bar%0Abar:foo foo: bar: ---- error_log -unsafe byte "0xd" in header value "value\x0Dfoo:bar\x0Abar:foo" -failed to set header +--- no_error_log +[error] @@ -1997,14 +1995,12 @@ failed to set header } --- request GET /t ---- error_code: 500 --- response_headers -header: +header: value%0Afoo:bar%0Dbar:foo foo: bar: ---- error_log -unsafe byte "0xa" in header value "value\x0Afoo:bar\x0Dbar:foo" -failed to set header +--- no_error_log +[error] @@ -2018,14 +2014,13 @@ failed to set header } --- request GET /t ---- error_code: 500 --- response_headers +header%3A%20value%0Dfoo%3Abar%0Abar%3Afoo: xx header: foo: bar: ---- error_log -unsafe byte "0xd" in header name "header: value\x0Dfoo:bar\x0Abar:foo" -failed to set header +--- no_error_log +[error] @@ -2039,14 +2034,13 @@ failed to set header } --- request GET /t ---- error_code: 500 --- response_headers +header%3A%20value%0Afoo%3Abar%0Dbar%3Afoo: xx header: foo: bar: ---- error_log -unsafe byte "0xa" in header name "header: value\x0Afoo:bar\x0Dbar:foo" -failed to set header +--- no_error_log +[error] @@ -2060,14 +2054,13 @@ failed to set header } --- request GET /t ---- error_code: 500 --- response_headers +%0Dheader%3A%20value%0Dfoo%3Abar%0Abar%3Afoo: xx header: foo: bar: ---- error_log -unsafe byte "0xd" in header name "\x0Dheader: value\x0Dfoo:bar\x0Abar:foo" -failed to set header +--- no_error_log +[error] @@ -2081,14 +2074,13 @@ failed to set header } --- request GET /t ---- error_code: 500 --- response_headers +%0Aheader%3A%20value%0Afoo%3Abar%0Dbar%3Afoo: xx header: foo: bar: ---- error_log -unsafe byte "0xa" in header name "\x0Aheader: value\x0Afoo:bar\x0Dbar:foo" -failed to set header +--- no_error_log +[error] @@ -2105,11 +2097,10 @@ failed to set header } --- request GET /t ---- error_code: 500 --- response_headers -foo: xx: xxx: ---- error_log -unsafe byte "0xa" in header value "foo\x0Axx:bar" -failed to set header +--- raw_response_headers_like chomp +foo: foo%0Axx:bar\r\nfoo: bar%0Dxxx:foo\r\n +--- no_error_log +[error] diff --git a/t/028-req-header.t b/t/028-req-header.t index fbd4e9eac9..c2c58eb52e 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -2044,10 +2044,10 @@ new } --- request GET /req-header ---- error_code: 500 ---- error_log -unsafe byte "0xd" in header name "Foo\x0Dfoo" -failed to set header +--- response_body +Foo: +--- no_error_log +[error] @@ -2062,10 +2062,10 @@ failed to set header } --- request GET /req-header ---- error_code: 500 ---- error_log -unsafe byte "0xa" in header value "new\x0Avalue" -failed to set header +--- response_body +Foo: new%0Avalue +--- no_error_log +[error] @@ -2082,10 +2082,10 @@ failed to set header } --- request GET /req-header ---- error_code: 500 ---- error_log -unsafe byte "0xa" in header value "new\x0Avalue" -failed to set header +--- response_body +new%0Avalue, foo bar. +--- no_error_log +[error] @@ -2097,12 +2097,69 @@ failed to set header } content_by_lua_block { - ngx.say(table.concat(ngx.req.get_headers()["foo"], ", "), ".") + ngx.say(ngx.req.get_headers()["foo"]) } } --- request GET /req-header ---- error_code: 500 ---- error_log -unsafe byte "0xa" in header value "\x22new\x0Avalue\x5C\x22" -failed to set header +--- response_body +"new%0Avalue\" +--- no_error_log +[error] + + + +=== TEST 66: add request headers with '\r\n' +--- config + location /bar { + access_by_lua_block { + ngx.req.set_header("Foo\r", "123\r\n") + } + proxy_pass http://127.0.0.1:$server_port/foo; + } + + location = /foo { + echo $echo_client_request_headers; + } +--- request +GET /bar +--- response_body_like chomp +\bFoo%0D: 123%0D%0A\b + + + +=== TEST 67: add request headers with '\0' +--- config + location /bar { + access_by_lua_block { + ngx.req.set_header("Foo", "\0") + } + proxy_pass http://127.0.0.1:$server_port/foo; + } + + location = /foo { + echo $echo_client_request_headers; + } +--- request +GET /bar +--- response_body_like chomp +\bFoo: %00\b + + + +=== TEST 68: add request headers with '中文' +--- config + location /bar { + access_by_lua_block { + ngx.req.set_header("Foo中文", "ab中文a") + } + proxy_pass http://127.0.0.1:$server_port/foo; + } + + location = /foo { + echo $echo_client_request_headers; + } +--- request +GET /bar +--- response_body_like chomp +\bFoo%E4%B8%AD%E6%96%87: ab中文a\r\n diff --git a/t/030-uri-args.t b/t/030-uri-args.t index b9dba2b736..d4cb6d17ef 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,7 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 22); +plan tests => repeat_each() * (blocks() * 2 + 21); no_root_location(); @@ -1568,9 +1568,9 @@ args: foo=%2C%24%40%7C%60&bar=-_.!~*'() } --- request GET /t ---- error_code: 200 ---- response_body eval -qr#/foo\tbar# +--- response +/foo bar +--- no_error_log diff --git a/t/113-req-header-cookie.t b/t/113-req-header-cookie.t index 3baa899c3b..944549cd74 100644 --- a/t/113-req-header-cookie.t +++ b/t/113-req-header-cookie.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (4 * blocks()); +plan tests => repeat_each() * (3 * blocks() + 6); #no_diff(); no_long_string(); @@ -263,9 +263,10 @@ Cookie: boo=123; foo=bar } --- request GET /t ---- error_code: 500 ---- error_log -unsafe byte "0xa" in header value "boo=123\x0Afoo" -failed to set header +--- response_body +Cookie foo: bar%0Dbar +Cookie baz: +Cookie boo: 123%0Afoo +Cookie: boo=123%0Afoo; foo=bar%0Dbar --- no_error_log -[crit] +[error] From 8746f64051feea83faa1aa1236928c579dd0848b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 28 Jun 2020 18:08:21 +0800 Subject: [PATCH 420/848] doc: ngx.req.set_uri_args: fixed byte ranges for automatic escaping. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 9aa0e4c4e3..d8c008c72f 100644 --- a/README.markdown +++ b/README.markdown @@ -4555,7 +4555,7 @@ or a Lua table holding the query arguments' key-value pairs, as in In the former case, i.e., when the whole query-string is provided directly, the input Lua string should already be well-formed with the URI encoding. For security considerations, this method will automatically escape any control and -whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. +whitespace characters (ASCII code 0x00 ~ 0x20 and 0x7F) in the Lua string. In the latter case, this method will escape argument keys and values according to the URI escaping rule. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index bfacf86cb0..e11f36af01 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3794,7 +3794,7 @@ or a Lua table holding the query arguments' key-value pairs, as in In the former case, i.e., when the whole query-string is provided directly, the input Lua string should already be well-formed with the URI encoding. For security considerations, this method will automatically escape any control and -whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string. +whitespace characters (ASCII code 0x00 ~ 0x20 and 0x7F) in the Lua string. In the latter case, this method will escape argument keys and values according to the URI escaping rule. From 5d15172687f0d8a2b10c43b4ac7a46f993a88be8 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 26 Jun 2020 10:59:03 +0800 Subject: [PATCH 421/848] feature: ngx.req.set_uri(): added the 'binary' optional boolean arg to allow arbitrary binary data in the unencoded URI. Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 14 +++++- doc/HttpLuaModule.wiki | 14 +++++- src/ngx_http_lua_control.c | 16 ++++++- src/ngx_http_lua_uri.c | 31 ++++++++++-- src/ngx_http_lua_util.h | 26 +++------- t/022-redirect.t | 17 +++---- t/030-uri-args.t | 87 +++++++++++++++++++++++++++++++++- t/162-static-module-location.t | 2 +- 8 files changed, 166 insertions(+), 41 deletions(-) diff --git a/README.markdown b/README.markdown index d8c008c72f..e61f2c9377 100644 --- a/README.markdown +++ b/README.markdown @@ -4431,7 +4431,7 @@ See also [ngx.req.get_method](#ngxreqget_method). ngx.req.set_uri --------------- -**syntax:** *ngx.req.set_uri(uri, jump?)* +**syntax:** *ngx.req.set_uri(uri, jump?, binary?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua** @@ -4527,6 +4527,18 @@ or ngx.req.set_uri("/foo", true) ``` +Starting from `0.10.16` of this module, this function accepts an +optional boolean `binary` argument to allow arbitrary binary URI +data. By default, this `binary` argument is false and this function +will throw out a Lua error such as the one below when the `uri` +argument contains any control characters (ASCII Code 0 ~ 0x08, 0x0A ~ 0x1F and 0x7F). + + + [error] 23430#23430: *1 lua entry thread aborted: runtime error: + content_by_lua(nginx.conf:44):3: ngx.req.set_uri unsafe byte "0x00" + in "\x00foo" (maybe you want to set the 'binary' argument?) + + This interface was first introduced in the `v0.3.1rc14` release. [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index e11f36af01..40ef150a48 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3685,7 +3685,7 @@ See also [[#ngx.req.get_method|ngx.req.get_method]]. == ngx.req.set_uri == -'''syntax:''' ''ngx.req.set_uri(uri, jump?)'' +'''syntax:''' ''ngx.req.set_uri(uri, jump?, binary?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*'' @@ -3771,6 +3771,18 @@ or ngx.req.set_uri("/foo", true) +Starting from 0.10.16 of this module, this function accepts an +optional boolean binary argument to allow arbitrary binary URI +data. By default, this binary argument is false and this function +will throw out a Lua error such as the one below when the uri +argument contains any control characters (ASCII Code 0 ~ 0x08, 0x0A ~ 0x1F and 0x7F). + + + [error] 23430#23430: *1 lua entry thread aborted: runtime error: + content_by_lua(nginx.conf:44):3: ngx.req.set_uri unsafe byte "0x00" + in "\x00foo" (maybe you want to set the 'binary' argument?) + + This interface was first introduced in the v0.3.1rc14 release. == ngx.req.set_uri_args == diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index a468285331..ae5ca0c5d4 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -186,9 +186,12 @@ ngx_http_lua_ngx_redirect(lua_State *L) int n; u_char *p; u_char *uri; + u_char byte; size_t len; ngx_table_elt_t *h; ngx_http_request_t *r; + size_t buf_len; + u_char *buf; n = lua_gettop(L); @@ -239,8 +242,17 @@ ngx_http_lua_ngx_redirect(lua_State *L) "the headers"); } - if (ngx_http_lua_check_unsafe_string(r, p, len, "redirect uri") != NGX_OK) { - return luaL_error(L, "attempt to set unsafe redirect uri"); + if (ngx_http_lua_check_unsafe_uri_bytes(r, p, len, &byte) != NGX_OK) { + buf_len = ngx_http_lua_escape_log(NULL, p, len) + 1; + buf = ngx_palloc(r->pool, buf_len); + if (buf == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_log(buf, p, len); + buf[buf_len - 1] = '\0'; + return luaL_error(L, "unsafe byte \"0x%02x\" in redirect uri \"%s\"", + byte, buf); } uri = ngx_palloc(r->pool, len); diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index b605bf2b89..0f88846fef 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -32,14 +32,18 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) ngx_http_request_t *r; size_t len; u_char *p; + u_char byte; int n; int jump = 0; + int binary = 0; ngx_http_lua_ctx_t *ctx; + size_t buf_len; + u_char *buf; n = lua_gettop(L); - if (n != 1 && n != 2) { - return luaL_error(L, "expecting 1 or 2 arguments but seen %d", n); + if (n < 1 || n > 3) { + return luaL_error(L, "expecting 1, 2 or 3 arguments but seen %d", n); } r = ngx_http_lua_get_req(L); @@ -55,8 +59,29 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return luaL_error(L, "attempt to use zero-length uri"); } - if (n == 2) { + if (n >= 3) { + luaL_checktype(L, 3, LUA_TBOOLEAN); + binary = lua_toboolean(L, 3); + } + + if (!binary + && ngx_http_lua_check_unsafe_uri_bytes(r, p, len, &byte) != NGX_OK) + { + buf_len = ngx_http_lua_escape_log(NULL, p, len) + 1; + buf = ngx_palloc(r->pool, buf_len); + if (buf == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_escape_log(buf, p, len); + buf[buf_len - 1] = '\0'; + + return luaL_error(L, "unsafe byte \"0x%02x\" in uri \"%s\" " + "(maybe you want to set the 'binary' argument?)", + byte, buf); + } + if (n >= 2) { luaL_checktype(L, 2, LUA_TBOOLEAN); jump = lua_toboolean(L, 2); diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 92f3ba4737..15e19853e3 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -496,17 +496,16 @@ ngx_inet_get_port(struct sockaddr *sa) static ngx_inline ngx_int_t -ngx_http_lua_check_unsafe_string(ngx_http_request_t *r, u_char *str, size_t len, - const char *name) +ngx_http_lua_check_unsafe_uri_bytes(ngx_http_request_t *r, u_char *str, + size_t len, u_char *byte) { - size_t i, buf_len; + size_t i; u_char c; - u_char *buf, *src = str; - /* %00-%1F, %7F */ + /* %00-%08, %0A-%1F, %7F */ static uint32_t unsafe[] = { - 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ + 0xfffffdff, /* 1111 1111 1111 1111 1111 1101 1111 1111 */ /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ @@ -526,20 +525,7 @@ ngx_http_lua_check_unsafe_string(ngx_http_request_t *r, u_char *str, size_t len, for (i = 0; i < len; i++, str++) { c = *str; if (unsafe[c >> 5] & (1 << (c & 0x1f))) { - buf_len = ngx_http_lua_escape_log(NULL, src, len); - buf = ngx_palloc(r->pool, buf_len); - if (buf == NULL) { - return NGX_ERROR; - } - - ngx_http_lua_escape_log(buf, src, len); - - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "unsafe byte \"0x%uxd\" in %s \"%*s\"", - (unsigned) c, name, buf_len, buf); - - ngx_pfree(r->pool, buf); - + *byte = c; return NGX_ERROR; } } diff --git a/t/022-redirect.t b/t/022-redirect.t index fb894d87ce..ef80174ca5 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -9,7 +9,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 3 + 14); +plan tests => repeat_each() * (blocks() * 3 + 9); #no_diff(); #no_long_string(); @@ -339,8 +339,7 @@ Location: foo: bar: --- error_log -unsafe byte "0xd" in redirect uri "http://agentzh.org/foo\x0Dfoo:bar\x0Abar:foo" -attempt to set unsafe redirect uri +unsafe byte "0x0d" in redirect uri "http://agentzh.org/foo\x0Dfoo:bar\x0Abar:foo" @@ -360,8 +359,7 @@ Location: foo: bar: --- error_log -unsafe byte "0xa" in redirect uri "http://agentzh.org/foo\x0Afoo:bar\x0Dbar:foo" -attempt to set unsafe redirect uri +unsafe byte "0x0a" in redirect uri "http://agentzh.org/foo\x0Afoo:bar\x0Dbar:foo" @@ -380,8 +378,7 @@ GET /t Location: foo: --- error_log -unsafe byte "0xa" in redirect uri "\x0Afoo:http://agentzh.org/foo" -attempt to set unsafe redirect uri +unsafe byte "0x0a" in redirect uri "\x0Afoo:http://agentzh.org/foo" @@ -400,8 +397,7 @@ GET /t Location: foo: --- error_log -unsafe byte "0xd" in redirect uri "\x0Dfoo:http://agentzh.org/foo" -attempt to set unsafe redirect uri +unsafe byte "0x0d" in redirect uri "\x0Dfoo:http://agentzh.org/foo" @@ -420,5 +416,4 @@ GET /t Location: foo: --- error_log -unsafe byte "0xd" in redirect uri "\x0Dhttp\x5C://\x22agentzh.org\x22/foo" -attempt to set unsafe redirect uri +unsafe byte "0x0d" in redirect uri "\x0Dhttp\x5C://\x22agentzh.org\x22/foo" diff --git a/t/030-uri-args.t b/t/030-uri-args.t index d4cb6d17ef..f163f80d1b 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -9,7 +9,9 @@ log_level('warn'); repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 21); + +plan tests => repeat_each() * (blocks() * 2 + 23); + no_root_location(); @@ -1579,7 +1581,7 @@ args: foo=%2C%24%40%7C%60&bar=-_.!~*'() location /t { content_by_lua_block { local new_uri = '\0foo' - ngx.req.set_uri(new_uri) + ngx.req.set_uri(new_uri, false, true) ngx.say(ngx.var.uri) } } @@ -1684,3 +1686,84 @@ bad argument #1 to 'set_uri_args' (string, number, or table expected, but got ni --- error_code: 500 --- error_log bad argument #1 to 'set_uri_args' (string, number, or table expected, but got userdata) + + + +=== TEST 64: set_uri binary option with unsafe uri +explict specify binary option to true +--- config + location /t { + rewrite_by_lua_block { + local new_uri = "/foo\r\nbar" + ngx.req.set_uri(new_uri, false, true) + } + + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT; + } + + location /foo { + content_by_lua_block { + ngx.say("request_uri: ", ngx.var.request_uri) + ngx.say("uri: ", ngx.var.uri) + } + } +--- request + GET /t +--- response_body eval +["request_uri: /foo%0D%0Abar\nuri: /foo\r\nbar\n", "request_uri: /foo%0D%0Abar\nuri: /foo\r\nbar\n"] +--- no_error_log +[error] + + + +=== TEST 65: set_uri binary option with unsafe uri +explict specify binary option to false +--- config + location /t { + rewrite_by_lua_block { + local new_uri = "/foo\r\nbar" + ngx.req.set_uri(new_uri, false, false) + } + + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT; + } + + location /foo { + content_by_lua_block { + ngx.say("request_uri: ", ngx.var.request_uri) + ngx.say("uri: ", ngx.var.uri) + } + } +--- request + GET /t +--- error_code: 500 +--- error_log eval +qr{\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: rewrite_by_lua\(nginx.conf:\d+\):\d+: unsafe byte "0x0d" in uri "/foo\\x0D\\x0Abar" \(maybe you want to set the 'binary' argument\?\)} + + + +=== TEST 66: set_uri binary option with safe uri +explict specify binary option to false +--- config + location /t { + rewrite_by_lua_block { + local new_uri = "/foo bar" + ngx.req.set_uri(new_uri, false, true) + } + + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT; + } + + location /foo { + content_by_lua_block { + ngx.say("request_uri: ", ngx.var.request_uri) + ngx.say("uri: ", ngx.var.uri) + } + } +--- request + GET /t +--- response_body +request_uri: /foo%20bar +uri: /foo bar +--- no_error_log +[error] diff --git a/t/162-static-module-location.t b/t/162-static-module-location.t index 1dce330e77..222760c188 100644 --- a/t/162-static-module-location.t +++ b/t/162-static-module-location.t @@ -18,7 +18,7 @@ __DATA__ rewrite_by_lua_block { ngx.req.read_body(); local args, _ = ngx.req.get_post_args(); - ngx.req.set_uri(args["url"], true); + ngx.req.set_uri(args["url"], true, true); } } --- request From bb045f2d8d3192150b3c26f94a4ca5aaa9646e80 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 19 May 2020 10:56:31 +0800 Subject: [PATCH 422/848] feature: ngx.escape_uri: added new optional 'type' argument not_component argument for pure URI escaping (still defauls to URI component escaping). Signed-off-by: Yichun Zhang (agentzh) --- README.markdown | 12 ++++- doc/HttpLuaModule.wiki | 12 ++++- src/ngx_http_lua_string.c | 11 +++-- src/ngx_http_lua_util.c | 6 +-- t/006-escape.t | 99 ++++++++++++++++++++++++++++++++++++++- 5 files changed, 127 insertions(+), 13 deletions(-) diff --git a/README.markdown b/README.markdown index e61f2c9377..3895591fc0 100644 --- a/README.markdown +++ b/README.markdown @@ -5593,11 +5593,19 @@ This method was introduced in the `0.5.0rc30` release. ngx.escape_uri -------------- -**syntax:** *newstr = ngx.escape_uri(str)* +**syntax:** *newstr = ngx.escape_uri(str, type?)* **context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Escape `str` as a URI component. +Since `v0.10.16`, this function accepts an optional `type` argument. +It accepts the following values (defaults to `2`): + +* `0`: escapes `str` as a full URI. And the characters +` ` (space), `#`, `%`, +`?`, 0x00 ~ 0x1F, 0x7F ~ 0xFF will be escaped. +* `2`: escape `str` as a URI component. All characters except +alphabetic characters, digits, `-`, `.`, _, +`~` will be encoded as `%XX`. [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 40ef150a48..a9a7002370 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4698,11 +4698,19 @@ This method was introduced in the 0.5.0rc30 release. == ngx.escape_uri == -'''syntax:''' ''newstr = ngx.escape_uri(str)'' +'''syntax:''' ''newstr = ngx.escape_uri(str, type?)'' '''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Escape str as a URI component. +Since `v0.10.16`, this function accepts an optional type argument. +It accepts the following values (defaults to `2`): + +* `0`: escapes str as a full URI. And the characters + (space), #, %, +`?`, 0x00 ~ 0x1F, 0x7F ~ 0xFF will be escaped. +* `2`: escape str as a URI component. All characters except +alphabetic characters, digits, -, ., _, +~ will be encoded as `%XX`. == ngx.unescape_uri == diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index 1bd4f2d99c..a94e181745 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -433,17 +433,18 @@ ngx_http_lua_ffi_unescape_uri(const u_char *src, size_t len, u_char *dst) size_t -ngx_http_lua_ffi_uri_escaped_length(const u_char *src, size_t len) +ngx_http_lua_ffi_uri_escaped_length(const u_char *src, size_t len, + int type) { - return len + 2 * ngx_http_lua_escape_uri(NULL, (u_char *) src, len, - NGX_ESCAPE_URI_COMPONENT); + return len + 2 * ngx_http_lua_escape_uri(NULL, (u_char *) src, len, type); } void -ngx_http_lua_ffi_escape_uri(const u_char *src, size_t len, u_char *dst) +ngx_http_lua_ffi_escape_uri(const u_char *src, size_t len, u_char *dst, + int type) { - ngx_http_lua_escape_uri(dst, (u_char *) src, len, NGX_ESCAPE_URI_COMPONENT); + ngx_http_lua_escape_uri(dst, (u_char *) src, len, type); } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 0154796b34..272906cb96 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1892,13 +1892,13 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ /* ?>=< ;:98 7654 3210 /.-, +*)( '&%$ #"! */ - 0xfc00886d, /* 1111 1100 0000 0000 1000 1000 0110 1101 */ + 0x80000029, /* 1000 0000 0000 0000 0000 0000 0010 1001 */ /* _^]\ [ZYX WVUT SRQP ONML KJIH GFED CBA@ */ - 0x78000000, /* 0111 1000 0000 0000 0000 0000 0000 0000 */ + 0x00000000, /* 0000 0000 0000 0000 0000 0000 0000 0000 */ /* ~}| {zyx wvut srqp onml kjih gfed cba` */ - 0xa8000000, /* 1010 1000 0000 0000 0000 0000 0000 0000 */ + 0x80000000, /* 1000 0000 0000 0000 0000 0000 0000 0000 */ 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ 0xffffffff, /* 1111 1111 1111 1111 1111 1111 1111 1111 */ diff --git a/t/006-escape.t b/t/006-escape.t index a21d3cb509..7c357eefe8 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 2); +plan tests => repeat_each() * (blocks() * 2 + 4); no_long_string(); @@ -197,3 +197,100 @@ GET /lua %2C%24%40%7C%60 --- no_error_log [error] + + + +=== TEST 15: escape type argument +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.escape_uri("https://www.google.com", 0)) + ngx.say(ngx.escape_uri("https://www.google.com/query?q=test", 0)) + ngx.say(ngx.escape_uri("https://www.google.com/query?\r\nq=test", 0)) + ngx.say(ngx.escape_uri("-_.~!*'();:@&=+$,/?#", 0)) + ngx.say(ngx.escape_uri("<>[]{}\\\" ", 0)) + } + } +--- request +GET /lua +--- response_body +https://www.google.com +https://www.google.com/query%3Fq=test +https://www.google.com/query%3F%0D%0Aq=test +-_.~!*'();:@&=+$,/%3F%23 +<>[]{}\"%20 +--- no_error_log +[error] + + + +=== TEST 16: escape type argument +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 0)) + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 1)) + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 2)) + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 3)) + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 4)) + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 5)) + ngx.say(ngx.escape_uri("https://www.google.com/?t=abc@ :", 6)) + } + } +--- request +GET /lua +--- response_body +https://www.google.com/%3Ft=abc@%20: +https://www.google.com/%3Ft=abc@%20: +https%3A%2F%2Fwww.google.com%2F%3Ft%3Dabc%40%20%3A +https://www.google.com/?t=abc@%20: +https://www.google.com/?t=abc@%20: +https://www.google.com/?t=abc@%20: +https://www.google.com/?t=abc@%20: +--- no_error_log +[error] + + + +=== TEST 17: escape type out of range +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.escape_uri("https://www.google.com", -1)) + } + } +--- request +GET /lua +--- error_code: 500 +--- error_log eval +qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" \-1 out of range/ + + + +=== TEST 18: escape type out of range +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.escape_uri("https://www.google.com", 10)) + } + } +--- request +GET /lua +--- error_code: 500 +--- error_log eval +qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" 10 out of range/ + + + +=== TEST 19: escape type not integer +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.escape_uri("https://www.google.com", true)) + } + } +--- request +GET /lua +--- error_code: 500 +--- error_log eval +qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" is not number/ From aa548f53852d93b5f0d26c7366721acf62dd84e7 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 29 Jun 2020 22:31:21 -0700 Subject: [PATCH 423/848] tests: fixed an error message to reflect new changes in lua-resty-core. --- t/006-escape.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/006-escape.t b/t/006-escape.t index 7c357eefe8..aadf167552 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -293,4 +293,4 @@ qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" 10 o GET /lua --- error_code: 500 --- error_log eval -qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" is not number/ +qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" is not a number/ From c2565fe799408c31bf445530a0e68c9bdb2de1fa Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 29 Jun 2020 13:37:41 +0800 Subject: [PATCH 424/848] tests: added test cases for sending HUP signal to nginx processes without the master process. Signed-off-by: Yichun Zhang (agentzh) --- t/163-signal.t | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 t/163-signal.t diff --git a/t/163-signal.t b/t/163-signal.t new file mode 100644 index 0000000000..ee2de6ef3c --- /dev/null +++ b/t/163-signal.t @@ -0,0 +1,51 @@ +# vi:ft= + +use Test::Nginx::Socket::Lua; + +plan tests => 2 * blocks(); + +no_long_string(); + + +run_tests(); + +__DATA__ + +=== TEST 1: SIGHUP followed by SIGQUIT +--- config + location = /t { + content_by_lua_block { + local pid = ngx.worker.pid() + os.execute("kill -HUP " .. pid) + ngx.sleep(0.01) + + os.execute("kill -QUIT " .. pid) + } + } +--- request +GET /t +--- ignore_response +--- wait: 0.1 +--- error_log eval +qr/\[notice\] \d+#\d+: exit$/ +--- no_error_log eval +qr/\[notice\] \d+#\d+: reconfiguring/ + + + +=== TEST 2: exit after receiving SIGHUP in single process mode +--- config + location = /t { + content_by_lua_block { + local pid = ngx.worker.pid() + os.execute("kill -HUP " .. pid) + } + } +--- request +GET /t +--- ignore_response +--- wait: 0.1 +--- error_log eval +qr/\[notice\] \d+#\d+: exit$/ +--- no_error_log eval +qr/\[notice\] \d+#\d+: reconfiguring/ From 6913b1b1e63284c3b6822bb7f9cabc0d7e3a62dc Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 2 Jul 2020 19:34:27 -0700 Subject: [PATCH 425/848] doc: updated the Version section to reflect the release date. --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 3895591fc0..c650a0b374 100644 --- a/README.markdown +++ b/README.markdown @@ -67,8 +67,8 @@ Version ======= This document describes ngx_lua -[v0.10.16](https://github.com/openresty/lua-nginx-module/tags), which is not -released yet. +[v0.10.16](https://github.com/openresty/lua-nginx-module/tags), which was released +on 2 July, 2020. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a9a7002370..ec379ad3b7 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -15,8 +15,8 @@ Production ready. = Version = This document describes ngx_lua -[https://github.com/openresty/lua-nginx-module/tags v0.10.16], which is not -released yet. +[https://github.com/openresty/lua-nginx-module/tags v0.10.16], which was released +on 2 July, 2020. = Synopsis = From 9ab38e8ee35fc08a57636b1b6190dca70b0076fa Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 23 Mar 2020 19:40:47 -0700 Subject: [PATCH 426/848] bugfix: prevented request smuggling in the ngx.location.capture API. Signed-off-by: Yichun Zhang (agentzh) --- src/ngx_http_lua_subrequest.c | 196 +++++-------- t/020-subrequest.t | 520 +++++++++++++++++++++++++++++++++- 2 files changed, 585 insertions(+), 131 deletions(-) diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 19a88d995f..b866955569 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -57,8 +57,6 @@ static ngx_str_t ngx_http_lua_content_length_header_key = ngx_string("Content-Length"); -static ngx_int_t ngx_http_lua_set_content_length_header(ngx_http_request_t *r, - off_t len); static ngx_int_t ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, int forward_body, ngx_http_request_body_t *body, unsigned vars_action, @@ -79,7 +77,7 @@ static void ngx_http_lua_cancel_subreq(ngx_http_request_t *r); static ngx_int_t ngx_http_post_request_to_head(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_copy_in_file_request_body(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, - ngx_http_request_t *r); + ngx_http_request_t *pr, int pr_not_chunked); enum { @@ -634,8 +632,8 @@ ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, unsigned vars_action, ngx_array_t *extra_vars) { ngx_http_request_t *r; - ngx_int_t rc; ngx_http_core_main_conf_t *cmcf; + int pr_not_chunked = 0; size_t size; r = sr->parent; @@ -645,46 +643,32 @@ ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, if (body) { sr->request_body = body; - rc = ngx_http_lua_set_content_length_header(sr, - body->buf - ? ngx_buf_size(body->buf) - : 0); - - if (rc != NGX_OK) { - return NGX_ERROR; - } - } else if (!always_forward_body && method != NGX_HTTP_PUT && method != NGX_HTTP_POST && r->headers_in.content_length_n > 0) { - rc = ngx_http_lua_set_content_length_header(sr, 0); - if (rc != NGX_OK) { - return NGX_ERROR; - } - -#if 1 sr->request_body = NULL; -#endif } else { - if (ngx_http_lua_copy_request_headers(sr, r) != NGX_OK) { - return NGX_ERROR; + if (!r->headers_in.chunked) { + pr_not_chunked = 1; } - if (sr->request_body) { + if (sr->request_body && sr->request_body->temp_file) { /* deep-copy the request body */ - if (sr->request_body->temp_file) { - if (ngx_http_lua_copy_in_file_request_body(sr) != NGX_OK) { - return NGX_ERROR; - } + if (ngx_http_lua_copy_in_file_request_body(sr) != NGX_OK) { + return NGX_ERROR; } } } + if (ngx_http_lua_copy_request_headers(sr, r, pr_not_chunked) != NGX_OK) { + return NGX_ERROR; + } + sr->method = method; switch (method) { @@ -1134,100 +1118,6 @@ ngx_http_lua_post_subrequest(ngx_http_request_t *r, void *data, ngx_int_t rc) } -static ngx_int_t -ngx_http_lua_set_content_length_header(ngx_http_request_t *r, off_t len) -{ - ngx_table_elt_t *h, *header; - u_char *p; - ngx_list_part_t *part; - ngx_http_request_t *pr; - ngx_uint_t i; - - r->headers_in.content_length_n = len; - - if (ngx_list_init(&r->headers_in.headers, r->pool, 20, - sizeof(ngx_table_elt_t)) != NGX_OK) - { - return NGX_ERROR; - } - - h = ngx_list_push(&r->headers_in.headers); - if (h == NULL) { - return NGX_ERROR; - } - - h->key = ngx_http_lua_content_length_header_key; - h->lowcase_key = ngx_pnalloc(r->pool, h->key.len); - if (h->lowcase_key == NULL) { - return NGX_ERROR; - } - - ngx_strlow(h->lowcase_key, h->key.data, h->key.len); - - r->headers_in.content_length = h; - - p = ngx_palloc(r->pool, NGX_OFF_T_LEN); - if (p == NULL) { - return NGX_ERROR; - } - - h->value.data = p; - - h->value.len = ngx_sprintf(h->value.data, "%O", len) - h->value.data; - - h->hash = ngx_http_lua_content_length_hash; - -#if 0 - dd("content length hash: %lu == %lu", (unsigned long) h->hash, - ngx_hash_key_lc((u_char *) "Content-Length", - sizeof("Content-Length") - 1)); -#endif - - dd("r content length: %.*s", - (int) r->headers_in.content_length->value.len, - r->headers_in.content_length->value.data); - - pr = r->parent; - - if (pr == NULL) { - return NGX_OK; - } - - /* forward the parent request's all other request headers */ - - part = &pr->headers_in.headers.part; - header = part->elts; - - for (i = 0; /* void */; i++) { - - if (i >= part->nelts) { - if (part->next == NULL) { - break; - } - - part = part->next; - header = part->elts; - i = 0; - } - - if (header[i].key.len == sizeof("Content-Length") - 1 - && ngx_strncasecmp(header[i].key.data, (u_char *) "Content-Length", - sizeof("Content-Length") - 1) == 0) - { - continue; - } - - if (ngx_http_lua_set_input_header(r, header[i].key, - header[i].value, 0) == NGX_ERROR) - { - return NGX_ERROR; - } - } - - return NGX_OK; -} - - static void ngx_http_lua_handle_subreq_responses(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) @@ -1742,11 +1632,17 @@ ngx_http_lua_copy_in_file_request_body(ngx_http_request_t *r) static ngx_int_t -ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, ngx_http_request_t *r) +ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, + ngx_http_request_t *pr, int pr_not_chunked) { - ngx_table_elt_t *header; + ngx_table_elt_t *clh, *header; ngx_list_part_t *part; ngx_uint_t i; + u_char *p; + off_t len; + + dd("before: parent req headers count: %d", + (int) pr->headers_in.headers.part.nelts); if (ngx_list_init(&sr->headers_in.headers, sr->pool, 20, sizeof(ngx_table_elt_t)) != NGX_OK) @@ -1754,10 +1650,46 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, ngx_http_request_t *r) return NGX_ERROR; } - dd("before: parent req headers count: %d", - (int) r->headers_in.headers.part.nelts); + if (sr->request_body && !pr_not_chunked) { + + /* craft our own Content-Length */ + + len = sr->request_body->buf ? ngx_buf_size(sr->request_body->buf) : 0; + + clh = ngx_list_push(&sr->headers_in.headers); + if (clh == NULL) { + return NGX_ERROR; + } - part = &r->headers_in.headers.part; + clh->hash = ngx_http_lua_content_length_hash; + clh->key = ngx_http_lua_content_length_header_key; + clh->lowcase_key = ngx_pnalloc(sr->pool, clh->key.len); + if (clh->lowcase_key == NULL) { + return NGX_ERROR; + } + + ngx_strlow(clh->lowcase_key, clh->key.data, clh->key.len); + + p = ngx_palloc(sr->pool, NGX_OFF_T_LEN); + if (p == NULL) { + return NGX_ERROR; + } + + clh->value.data = p; + clh->value.len = ngx_sprintf(clh->value.data, "%O", len) + - clh->value.data; + + sr->headers_in.content_length = clh; + sr->headers_in.content_length_n = len; + + dd("sr crafted content-length: %.*s", + (int) sr->headers_in.content_length->value.len, + sr->headers_in.content_length->value.data); + } + + /* copy the parent request's headers */ + + part = &pr->headers_in.headers.part; header = part->elts; for (i = 0; /* void */; i++) { @@ -1772,7 +1704,14 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, ngx_http_request_t *r) i = 0; } - dd("setting request header %.*s: %.*s", (int) header[i].key.len, + if (!pr_not_chunked && header[i].key.len == sizeof("Content-Length") - 1 + && ngx_strncasecmp(header[i].key.data, (u_char *) "Content-Length", + sizeof("Content-Length") - 1) == 0) + { + continue; + } + + dd("sr copied req header %.*s: %.*s", (int) header[i].key.len, header[i].key.data, (int) header[i].value.len, header[i].value.data); @@ -1784,9 +1723,10 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, ngx_http_request_t *r) } dd("after: parent req headers count: %d", - (int) r->headers_in.headers.part.nelts); + (int) pr->headers_in.headers.part.nelts); return NGX_OK; } + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 5a386dbe1a..18befbd761 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -14,6 +14,7 @@ repeat_each(2); plan tests => repeat_each() * (blocks() * 3 + 23); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); #no_diff(); no_long_string(); @@ -210,7 +211,7 @@ GET -=== TEST 8: PUT (nobody, proxy method) +=== TEST 8: PUT (with body, proxy method) --- config location /other { default_type 'foo/bar'; @@ -242,7 +243,7 @@ hello -=== TEST 9: PUT (nobody, no proxy method) +=== TEST 9: PUT (with body, no proxy method) --- config location /other { default_type 'foo/bar'; @@ -271,7 +272,7 @@ hello -=== TEST 10: PUT (nobody, no proxy method) +=== TEST 10: PUT (no body, no proxy method) --- config location /other { default_type 'foo/bar'; @@ -2877,3 +2878,516 @@ DELETE /file.txt, response status: 204 --- no_error_log [error] --- error_code: 200 + + + +=== TEST 77: avoid request smuggling 1/4 (default capture + smuggle in header) +--- http_config + upstream backend { + server unix:$TEST_NGINX_HTML_DIR/nginx.sock; + keepalive 32; + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + content_by_lua_block { + ngx.say("method: ", ngx.var.request_method, + ", uri: ", ngx.var.uri, + ", X: ", ngx.var.http_x) + } + } + } +--- config + location /proxy { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://backend/foo; + } + + location /capture { + server_tokens off; + more_clear_headers Date; + + content_by_lua_block { + local res = ngx.location.capture("/proxy") + ngx.print(res.body) + } + } + + location /t { + content_by_lua_block { + local req = [[ +GET /capture HTTP/1.1 +Host: test.com +Content-Length: 37 +Transfer-Encoding: chunked + +0 + +GET /capture HTTP/1.1 +Host: test.com +X: GET /bar HTTP/1.0 + +]] + + local sock = ngx.socket.tcp() + sock:settimeout(1000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send req: ", err) + return + end + + ngx.say("req bytes: ", bytes) + + local n_resp = 0 + + local reader = sock:receiveuntil("\r\n") + while true do + local line, err = reader() + if line then + ngx.say(line) + if line == "0" then + n_resp = n_resp + 1 + end + + if n_resp >= 2 then + break + end + + else + ngx.say("err: ", err) + break + end + end + + sock:close() + } + } +--- request +GET /t +--- response_body +req bytes: 146 +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +1f +method: GET, uri: /foo, X: nil + +0 + +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +2d +method: GET, uri: /foo, X: GET /bar HTTP/1.0 + +0 +--- no_error_log +[error] + + + +=== TEST 78: avoid request smuggling 2/4 (POST capture + smuggle in body) +--- http_config + upstream backend { + server unix:$TEST_NGINX_HTML_DIR/nginx.sock; + keepalive 32; + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + content_by_lua_block { + ngx.say("method: ", ngx.var.request_method, + ", uri: ", ngx.var.uri) + } + } + } +--- config + location /proxy { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://backend/foo; + } + + location /capture { + server_tokens off; + more_clear_headers Date; + + content_by_lua_block { + ngx.req.read_body() + local res = ngx.location.capture("/proxy", { method = ngx.HTTP_POST }) + ngx.print(res.body) + } + } + + location /t { + content_by_lua_block { + local req = [[ +GET /capture HTTP/1.1 +Host: test.com +Content-Length: 57 +Transfer-Encoding: chunked + +0 + +POST /capture HTTP/1.1 +Host: test.com +Content-Length: 60 + +POST /bar HTTP/1.1 +Host: test.com +Content-Length: 5 + +hello + +]] + + local sock = ngx.socket.tcp() + sock:settimeout(1000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send req: ", err) + return + end + + ngx.say("req bytes: ", bytes) + + local n_resp = 0 + + local reader = sock:receiveuntil("\r\n") + while true do + local line, err = reader() + if line then + ngx.say(line) + if line == "0" then + n_resp = n_resp + 1 + end + + if n_resp >= 2 then + break + end + + else + ngx.say("err: ", err) + break + end + end + + sock:close() + } + } +--- request +GET /t +--- response_body +req bytes: 205 +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +18 +method: POST, uri: /foo + +0 + +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +18 +method: POST, uri: /foo + +0 +--- no_error_log +[error] + + + +=== TEST 79: avoid request smuggling 3/4 (POST capture w/ always_forward_body + smuggle in body) +--- http_config + upstream backend { + server unix:$TEST_NGINX_HTML_DIR/nginx.sock; + keepalive 32; + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + content_by_lua_block { + ngx.say("method: ", ngx.var.request_method, + ", uri: ", ngx.var.uri) + } + } + } +--- config + location /proxy { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://backend/foo; + } + + location /capture { + server_tokens off; + more_clear_headers Date; + + content_by_lua_block { + ngx.req.read_body() + local res = ngx.location.capture("/proxy", { + method = ngx.HTTP_POST, + always_forward_body = true + }) + ngx.print(res.body) + } + } + + location /t { + content_by_lua_block { + local req = [[ +GET /capture HTTP/1.1 +Host: test.com +Content-Length: 57 +Transfer-Encoding: chunked + +0 + +POST /capture HTTP/1.1 +Host: test.com +Content-Length: 60 + +POST /bar HTTP/1.1 +Host: test.com +Content-Length: 5 + +hello + +]] + + local sock = ngx.socket.tcp() + sock:settimeout(1000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send req: ", err) + return + end + + ngx.say("req bytes: ", bytes) + + local n_resp = 0 + + local reader = sock:receiveuntil("\r\n") + while true do + local line, err = reader() + if line then + ngx.say(line) + if line == "0" then + n_resp = n_resp + 1 + end + + if n_resp >= 2 then + break + end + + else + ngx.say("err: ", err) + break + end + end + + sock:close() + } + } +--- request +GET /t +--- response_body +req bytes: 205 +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +18 +method: POST, uri: /foo + +0 + +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +18 +method: POST, uri: /foo + +0 +--- no_error_log +[error] + + + +=== TEST 80: avoid request smuggling 4/4 (POST capture w/ body + smuggle in body) +--- http_config + upstream backend { + server unix:$TEST_NGINX_HTML_DIR/nginx.sock; + keepalive 32; + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + content_by_lua_block { + ngx.say("method: ", ngx.var.request_method, + ", uri: ", ngx.var.uri) + } + } + } +--- config + location /proxy { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://backend/foo; + } + + location /capture { + server_tokens off; + more_clear_headers Date; + + content_by_lua_block { + ngx.req.read_body() + local res = ngx.location.capture("/proxy", { + method = ngx.HTTP_POST, + always_forward_body = true, + body = ngx.req.get_body_data() + }) + ngx.print(res.body) + } + } + + location /t { + content_by_lua_block { + local req = [[ +GET /capture HTTP/1.1 +Host: test.com +Content-Length: 57 +Transfer-Encoding: chunked + +0 + +POST /capture HTTP/1.1 +Host: test.com +Content-Length: 60 + +POST /bar HTTP/1.1 +Host: test.com +Content-Length: 5 + +hello + +]] + + local sock = ngx.socket.tcp() + sock:settimeout(1000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send req: ", err) + return + end + + ngx.say("req bytes: ", bytes) + + local n_resp = 0 + + local reader = sock:receiveuntil("\r\n") + while true do + local line, err = reader() + if line then + ngx.say(line) + if line == "0" then + n_resp = n_resp + 1 + end + + if n_resp >= 2 then + break + end + + else + ngx.say("err: ", err) + break + end + end + + sock:close() + } + } +--- request +GET /t +--- response_body +req bytes: 205 +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +18 +method: POST, uri: /foo + +0 + +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Transfer-Encoding: chunked +Connection: keep-alive + +18 +method: POST, uri: /foo + +0 +--- no_error_log +[error] From 2d23bc4f0a29ed79aaaa754c11bffb1080aa44ba Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 3 Jul 2020 12:23:40 -0700 Subject: [PATCH 427/848] bumped version to 0.10.17. --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.markdown b/README.markdown index c650a0b374..c629fe02d0 100644 --- a/README.markdown +++ b/README.markdown @@ -67,8 +67,8 @@ Version ======= This document describes ngx_lua -[v0.10.16](https://github.com/openresty/lua-nginx-module/tags), which was released -on 2 July, 2020. +[v0.10.17](https://github.com/openresty/lua-nginx-module/tags), which was released +on 3 July, 2020. Synopsis ======== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ec379ad3b7..4030ee89e8 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -15,8 +15,8 @@ Production ready. = Version = This document describes ngx_lua -[https://github.com/openresty/lua-nginx-module/tags v0.10.16], which was released -on 2 July, 2020. +[https://github.com/openresty/lua-nginx-module/tags v0.10.17], which was released +on 3 July, 2020. = Synopsis = diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 02c242299a..ad07bd863d 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10016 +#define ngx_http_lua_version 10017 typedef struct { From c48eeed50927df85766c84e92c44eec2a7b99bb0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 3 Jul 2020 16:19:33 -0700 Subject: [PATCH 428/848] tests: 161-load-resty-core.t: added vim mode line. --- t/161-load-resty-core.t | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/161-load-resty-core.t b/t/161-load-resty-core.t index e6a4d0471a..2c2643068d 100644 --- a/t/161-load-resty-core.t +++ b/t/161-load-resty-core.t @@ -1,3 +1,5 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + use Test::Nginx::Socket::Lua; repeat_each(2); From c366f3a9bdd5a838b562d85c33cd33f0fdb23c82 Mon Sep 17 00:00:00 2001 From: Dmitry Meyer Date: Sun, 5 Jul 2020 21:48:04 +0300 Subject: [PATCH 429/848] doc: fixed an unclosed tag breaking the markup. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index c629fe02d0..ecaec83a5a 100644 --- a/README.markdown +++ b/README.markdown @@ -5604,7 +5604,7 @@ It accepts the following values (defaults to `2`): ` ` (space), `#`, `%`, `?`, 0x00 ~ 0x1F, 0x7F ~ 0xFF will be escaped. * `2`: escape `str` as a URI component. All characters except -alphabetic characters, digits, `-`, `.`, _, +alphabetic characters, digits, `-`, `.`, `_`, `~` will be encoded as `%XX`. [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4030ee89e8..215a5d5196 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4709,7 +4709,7 @@ It accepts the following values (defaults to `2`): (space), #, %, `?`, 0x00 ~ 0x1F, 0x7F ~ 0xFF will be escaped. * `2`: escape str as a URI component. All characters except -alphabetic characters, digits, -, ., _, +alphabetic characters, digits, -, ., _, ~ will be encoded as `%XX`. == ngx.unescape_uri == From a4d63f7f7fbfb6f5397d750aa968bd490f43776e Mon Sep 17 00:00:00 2001 From: spacewander Date: Tue, 30 Jun 2020 15:15:07 +0800 Subject: [PATCH 430/848] refactor: used 'yieldable context' to simpify the context list. --- src/ngx_http_lua_coroutine.c | 28 ++++------------------------ src/ngx_http_lua_pipe.c | 7 +------ src/ngx_http_lua_semaphore.c | 6 +----- src/ngx_http_lua_sleep.c | 7 +------ src/ngx_http_lua_socket_tcp.c | 14 ++------------ src/ngx_http_lua_socket_udp.c | 14 ++------------ src/ngx_http_lua_uthread.c | 14 ++------------ src/ngx_http_lua_util.h | 8 ++++++++ 8 files changed, 21 insertions(+), 77 deletions(-) diff --git a/src/ngx_http_lua_coroutine.c b/src/ngx_http_lua_coroutine.c index 5c469c0fa9..3a43b51142 100644 --- a/src/ngx_http_lua_coroutine.c +++ b/src/ngx_http_lua_coroutine.c @@ -113,12 +113,7 @@ ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, luaL_argcheck(L, lua_isfunction(L, 1) && !lua_iscfunction(L, 1), 1, "Lua function expected"); - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); vm = ngx_http_lua_get_lua_vm(r, ctx); @@ -194,12 +189,7 @@ ngx_http_lua_coroutine_resume(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); p_coctx = ctx->cur_co_ctx; if (p_coctx == NULL) { @@ -255,12 +245,7 @@ ngx_http_lua_coroutine_yield(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); coctx = ctx->cur_co_ctx; @@ -417,12 +402,7 @@ ngx_http_lua_coroutine_status(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); coctx = ngx_http_lua_get_co_ctx(co, ctx); if (coctx == NULL) { diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 1a91b9514f..c0be9c9e05 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -1134,12 +1134,7 @@ ngx_http_lua_pipe_get_lua_ctx(ngx_http_request_t *r, return NGX_HTTP_LUA_FFI_NO_REQ_CTX; } - rc = ngx_http_lua_ffi_check_context(*ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH, + rc = ngx_http_lua_ffi_check_context(*ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE, errbuf, errbuf_size); if (rc != NGX_OK) { return NGX_HTTP_LUA_FFI_BAD_CONTEXT; diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index a3b79398e7..8ca6efb345 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -379,11 +379,7 @@ ngx_http_lua_ffi_sema_wait(ngx_http_request_t *r, return NGX_ERROR; } - rc = ngx_http_lua_ffi_check_context(ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT, + rc = ngx_http_lua_ffi_check_context(ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE, err, errlen); if (rc != NGX_OK) { diff --git a/src/ngx_http_lua_sleep.c b/src/ngx_http_lua_sleep.c index 09ea0f6d40..b8601fee95 100644 --- a/src/ngx_http_lua_sleep.c +++ b/src/ngx_http_lua_sleep.c @@ -52,12 +52,7 @@ ngx_http_lua_ngx_sleep(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); coctx = ctx->cur_co_ctx; if (coctx == NULL) { diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 30a4e7b282..472fff0f87 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -431,12 +431,7 @@ ngx_http_lua_socket_tcp(lua_State *L) return luaL_error(L, "no ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); lua_createtable(L, 5 /* narr */, 1 /* nrec */); lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( @@ -878,12 +873,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) return luaL_error(L, "no ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); luaL_checktype(L, 1, LUA_TTABLE); diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 3a0d204319..f9d8fc0307 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -139,12 +139,7 @@ ngx_http_lua_socket_udp(lua_State *L) return luaL_error(L, "no ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); lua_createtable(L, 3 /* narr */, 1 /* nrec */); lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( @@ -202,12 +197,7 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) return luaL_error(L, "no ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); luaL_checktype(L, 1, LUA_TTABLE); diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index 2e0c3d8625..68babc0871 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -124,12 +124,7 @@ ngx_http_lua_uthread_wait(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); coctx = ctx->cur_co_ctx; @@ -223,12 +218,7 @@ ngx_http_lua_uthread_kill(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); coctx = ctx->cur_co_ctx; diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 15e19853e3..2d928f96cf 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -31,6 +31,14 @@ #define NGX_HTTP_LUA_ESCAPE_HEADER_VALUE 8 +#define NGX_HTTP_LUA_CONTEXT_YIELDABLE NGX_HTTP_LUA_CONTEXT_REWRITE \ + | NGX_HTTP_LUA_CONTEXT_ACCESS \ + | NGX_HTTP_LUA_CONTEXT_CONTENT \ + | NGX_HTTP_LUA_CONTEXT_TIMER \ + | NGX_HTTP_LUA_CONTEXT_SSL_CERT \ + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH + + /* key in Lua vm registry for all the "ngx.ctx" tables */ #define ngx_http_lua_ctx_tables_key "ngx_lua_ctx_tables" From 635b6e864c385bc16125bd43a2ffe8b4560ca9fa Mon Sep 17 00:00:00 2001 From: Guy Lewin Date: Tue, 7 Jul 2020 04:11:53 -0400 Subject: [PATCH 431/848] feature: supported receiveany on ngx.req.socket(true?) socks (#1623) --- README.markdown | 4 +- doc/HttpLuaModule.wiki | 4 +- src/ngx_http_lua_socket_tcp.c | 10 +++- t/062-count.t | 4 +- t/067-req-socket.t | 70 ++++++++++++++++++++++++++ t/116-raw-req-socket.t | 95 ++++++++++++++++++++++++++++++++++- 6 files changed, 178 insertions(+), 9 deletions(-) diff --git a/README.markdown b/README.markdown index ecaec83a5a..4046dfe15f 100644 --- a/README.markdown +++ b/README.markdown @@ -5181,7 +5181,7 @@ ngx.req.socket **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** -Returns a read-only cosocket object that wraps the downstream connection. Only [receive](#tcpsockreceive) and [receiveuntil](#tcpsockreceiveuntil) methods are supported on this object. +Returns a read-only cosocket object that wraps the downstream connection. Only [receive](#tcpsockreceive), [receiveany](#tcpsockreceiveany) and [receiveuntil](#tcpsockreceiveuntil) methods are supported on this object. In case of error, `nil` will be returned as well as a string describing the error. @@ -5190,7 +5190,7 @@ The socket object returned by this method is usually used to read the current re If any request body data has been pre-read into the Nginx core request header buffer, the resulting cosocket object will take care of this to avoid potential data loss resulting from such pre-reading. Chunked request bodies are not yet supported in this API. -Since the `v0.9.0` release, this function accepts an optional boolean `raw` argument. When this argument is `true`, this function returns a full-duplex cosocket object wrapping around the raw downstream connection socket, upon which you can call the [receive](#tcpsockreceive), [receiveuntil](#tcpsockreceiveuntil), and [send](#tcpsocksend) methods. +Since the `v0.9.0` release, this function accepts an optional boolean `raw` argument. When this argument is `true`, this function returns a full-duplex cosocket object wrapping around the raw downstream connection socket, upon which you can call the [receive](#tcpsockreceive), [receiveany](#tcpsockreceiveany), [receiveuntil](#tcpsockreceiveuntil), and [send](#tcpsocksend) methods. When the `raw` argument is `true`, it is required that no pending data from any previous [ngx.say](#ngxsay), [ngx.print](#ngxprint), or [ngx.send_headers](#ngxsend_headers) calls exists. So if you have these downstream output calls previously, you should call [ngx.flush(true)](#ngxflush) before calling `ngx.req.socket(true)` to ensure that there is no pending output data. If the request body has not been read yet, then this "raw socket" can also be used to read the request body. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 215a5d5196..37d8a1367b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4342,7 +4342,7 @@ See also [[#ngx.req.init_body|ngx.req.init_body]]. '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' -Returns a read-only cosocket object that wraps the downstream connection. Only [[#tcpsock:receive|receive]] and [[#tcpsock:receiveuntil|receiveuntil]] methods are supported on this object. +Returns a read-only cosocket object that wraps the downstream connection. Only [[#tcpsock:receive|receive]], [[#tcpsock:receiveany|receiveany]] and [[#tcpsock:receiveuntil|receiveuntil]] methods are supported on this object. In case of error, nil will be returned as well as a string describing the error. @@ -4351,7 +4351,7 @@ The socket object returned by this method is usually used to read the current re If any request body data has been pre-read into the Nginx core request header buffer, the resulting cosocket object will take care of this to avoid potential data loss resulting from such pre-reading. Chunked request bodies are not yet supported in this API. -Since the v0.9.0 release, this function accepts an optional boolean raw argument. When this argument is true, this function returns a full-duplex cosocket object wrapping around the raw downstream connection socket, upon which you can call the [[#tcpsock:receive|receive]], [[#tcpsock:receiveuntil|receiveuntil]], and [[#tcpsock:send|send]] methods. +Since the v0.9.0 release, this function accepts an optional boolean raw argument. When this argument is true, this function returns a full-duplex cosocket object wrapping around the raw downstream connection socket, upon which you can call the [[#tcpsock:receive|receive]], [[#tcpsock:receiveany|receiveany]], [[#tcpsock:receiveuntil|receiveuntil]], and [[#tcpsock:send|send]] methods. When the raw argument is true, it is required that no pending data from any previous [[#ngx.say|ngx.say]], [[#ngx.print|ngx.print]], or [[#ngx.send_headers|ngx.send_headers]] calls exists. So if you have these downstream output calls previously, you should call [[#ngx.flush|ngx.flush(true)]] before calling ngx.req.socket(true) to ensure that there is no pending output data. If the request body has not been read yet, then this "raw socket" can also be used to read the request body. diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 472fff0f87..c2776d83a7 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -255,11 +255,14 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* {{{req socket object metatable */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( req_socket_metatable_key)); - lua_createtable(L, 0 /* narr */, 5 /* nrec */); + lua_createtable(L, 0 /* narr */, 6 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_receive); lua_setfield(L, -2, "receive"); + lua_pushcfunction(L, ngx_http_lua_socket_tcp_receiveany); + lua_setfield(L, -2, "receiveany"); + lua_pushcfunction(L, ngx_http_lua_socket_tcp_receiveuntil); lua_setfield(L, -2, "receiveuntil"); @@ -278,11 +281,14 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* {{{raw req socket object metatable */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( raw_req_socket_metatable_key)); - lua_createtable(L, 0 /* narr */, 6 /* nrec */); + lua_createtable(L, 0 /* narr */, 7 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_receive); lua_setfield(L, -2, "receive"); + lua_pushcfunction(L, ngx_http_lua_socket_tcp_receiveany); + lua_setfield(L, -2, "receiveany"); + lua_pushcfunction(L, ngx_http_lua_socket_tcp_receiveuntil); lua_setfield(L, -2, "receiveuntil"); diff --git a/t/062-count.t b/t/062-count.t index ffb769a50d..841923b86d 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -259,7 +259,7 @@ n = 10 POST /test hello world --- response_body -n = 5 +n = 6 --- no_error_log [error] @@ -512,7 +512,7 @@ n = 6 --- request GET /test --- response_body -n = 6 +n = 7 --- no_error_log [error] diff --git a/t/067-req-socket.t b/t/067-req-socket.t index 6593360e4c..c5b2631e5e 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -1096,3 +1096,73 @@ done --- grep_error_log_out lua finalize socket GC cycle done + + + +=== TEST 18: receiveany +--- config + location = /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = sock:connect("127.0.0.1", ngx.var.server_port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send("POST /back HTTP/1.0\r\nHost: localhost\r\nContent-Length: 1024\r\n\r\nabc") + if not bytes then + ngx.say("failed to send: ", err) + end + + ngx.sleep(0.2) + + local bytes, err = sock:send("hello world\n") + if not bytes then + ngx.say("failed to send: ", err) + end + + local reader = sock:receiveuntil("\r\n\r\n") + local header, err = reader() + if not header then + ngx.say("failed to receive header: ", err) + return + end + + local line, err = sock:receive() + if not line then + ngx.say("failed to receive line: ", err) + return + end + ngx.say("received: ", line) + } + } + + location = /back { + content_by_lua_block { + ngx.send_headers() + ngx.flush(true) + + local sock, err = ngx.req.socket() + + if not sock then + ngx.say("failed to get socket: ", err) + return nil + end + + local data, err = sock:receiveany(4096) + if not data then + ngx.say("err: ", err) + return nil + end + + ngx.say("received: ", data) + } + } + +--- request +GET /t +--- response_body +received: received: abc +--- no_error_log +[error] diff --git a/t/116-raw-req-socket.t b/t/116-raw-req-socket.t index ab06ee4a31..4dfb92664e 100644 --- a/t/116-raw-req-socket.t +++ b/t/116-raw-req-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 40; +plan tests => repeat_each() * 43; our $HtmlDir = html_dir; @@ -876,3 +876,96 @@ request body: hey, hello world --- no_error_log [error] [alert] + + + +=== TEST 16: receiveany +--- config + server_tokens off; + location = /t { + #set $port 5000; + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local req = "GET /mysock HTTP/1.1\r\nUpgrade: mysock\r\nHost: localhost\r\nConnection: close\r\n\r\nhello" + -- req = "OK" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + -- Will return to I/O loop, causing receiveany() in /mysock location to be called + ngx.sleep(1) + + local bytes, err = sock:send(", world") + if not bytes then + ngx.say("failed to send packet 1: ", err) + return + end + + local reader = sock:receiveuntil("\r\n\r\n") + local data, err, partial = reader() + if not data then + ngx.say("no response header found") + return + end + + local msg, err = sock:receive() + if not msg then + ngx.say("failed to receive: ", err) + return + end + + ngx.say("msg: ", msg) + + ok, err = sock:close() + if not ok then + ngx.say("failed to close socket: ", err) + return + end + } + } + + location = /mysock { + content_by_lua_block { + ngx.status = 101 + ngx.send_headers() + ngx.flush(true) + ngx.req.read_body() + local sock, err = ngx.req.socket(true) + if not sock then + ngx.log(ngx.ERR, "server: failed to get raw req socket: ", err) + return + end + + local data, err = sock:receiveany(1024) + if not data then + ngx.log(ngx.ERR, "server: failed to receive: ", err) + return + end + + local bytes, err = sock:send("1: received: " .. data .. "\n") + if not bytes then + ngx.log(ngx.ERR, "server: failed to send: ", err) + return + end + } + more_clear_headers Date; + } + +--- request +GET /t +--- response_body +msg: 1: received: hello +--- no_error_log +[error] From 4368a8ea2723e4d55d9ea362e758dc9bf0111ebe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Fri, 10 Jul 2020 17:20:02 +0800 Subject: [PATCH 432/848] bugfix: macro contains operator should be wrapped inside a bracket. The bracket is missing in the previous refactor. --- src/ngx_http_lua_util.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 2d928f96cf..57c0814bfb 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -31,12 +31,12 @@ #define NGX_HTTP_LUA_ESCAPE_HEADER_VALUE 8 -#define NGX_HTTP_LUA_CONTEXT_YIELDABLE NGX_HTTP_LUA_CONTEXT_REWRITE \ +#define NGX_HTTP_LUA_CONTEXT_YIELDABLE (NGX_HTTP_LUA_CONTEXT_REWRITE \ | NGX_HTTP_LUA_CONTEXT_ACCESS \ | NGX_HTTP_LUA_CONTEXT_CONTENT \ | NGX_HTTP_LUA_CONTEXT_TIMER \ | NGX_HTTP_LUA_CONTEXT_SSL_CERT \ - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH) /* key in Lua vm registry for all the "ngx.ctx" tables */ From af748420195c6fbe05e991d4d97b91eb08af120e Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Thu, 16 Jul 2020 14:36:19 +0800 Subject: [PATCH 433/848] test: fix: add lua-resty-string to lua_package_path (#1748) --- .travis.yml | 1 + t/000--init.t | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index fc2e4c20bf..bf433bef5e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -86,6 +86,7 @@ install: - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql + - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 diff --git a/t/000--init.t b/t/000--init.t index 5865755372..0dd08fe188 100644 --- a/t/000--init.t +++ b/t/000--init.t @@ -10,7 +10,8 @@ $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_MYSQL_PORT} ||= 3306; our $http_config = <<'_EOC_'; - lua_package_path "../lua-resty-mysql/lib/?.lua;;"; + # lua-resty-string is required for lua-resty-mysql + lua_package_path "../lua-resty-mysql/lib/?.lua;../lua-resty-string/lib/?.lua;;"; _EOC_ no_shuffle(); From b050e1c944d38e0956da851b2c40ce9381a63e88 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 19 Jul 2020 17:17:51 +0800 Subject: [PATCH 434/848] sytle: added blank line after code block. (#1752) --- src/ngx_http_lua_args.c | 2 ++ src/ngx_http_lua_directive.c | 2 ++ src/ngx_http_lua_headers.c | 1 + src/ngx_http_lua_headers_in.c | 3 +++ src/ngx_http_lua_initworkerby.c | 1 + src/ngx_http_lua_log.c | 1 + src/ngx_http_lua_req_body.c | 3 +++ src/ngx_http_lua_socket_tcp.c | 1 + src/ngx_http_lua_string.c | 3 +++ src/ngx_http_lua_util.c | 3 +++ 10 files changed, 20 insertions(+) diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index 7b7d5eae6c..1280846545 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -55,6 +55,7 @@ ngx_http_lua_escape_args(u_char *dst, u_char *src, size_t size) if (escape[*src >> 5] & (1 << (*src & 0x1f))) { n++; } + src++; size--; } @@ -72,6 +73,7 @@ ngx_http_lua_escape_args(u_char *dst, u_char *src, size_t size) } else { *dst++ = *src++; } + size--; } diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 8f7e7fbef3..6b7ccc7cbe 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1378,6 +1378,7 @@ ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd) if (dst == NULL) { return NGX_CONF_ERROR; } + dst->len = len; dst->len--; /* skip the trailing '}' block terminator */ @@ -1385,6 +1386,7 @@ ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd) if (p == NULL) { return NGX_CONF_ERROR; } + dst->data = p; for (i = 0; i < cf->args->nelts; i++) { diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index b638277bd7..75985a1130 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -500,6 +500,7 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) } else { lua_pushliteral(L, "close"); } + lua_rawset(L, -3); if (r->chunked) { diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 713818efc2..fff8bafb92 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -359,6 +359,7 @@ ngx_http_lua_validate_host(ngx_str_t *host, ngx_pool_t *pool, ngx_uint_t alloc) if (dot_pos == i - 1) { return NGX_DECLINED; } + dot_pos = i; break; @@ -754,6 +755,7 @@ ngx_http_lua_rm_header_helper(ngx_list_t *l, ngx_list_part_t *cur, if (part->next == NULL) { return NGX_ERROR; } + part = part->next; } @@ -798,6 +800,7 @@ ngx_http_lua_rm_header_helper(ngx_list_t *l, ngx_list_part_t *cur, if (part->next == NULL) { return NGX_ERROR; } + part = part->next; } diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index fa094a2e8b..94de796ade 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -148,6 +148,7 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) if (part->next == NULL) { break; } + part = part->next; ofile = part->elts; i = 0; diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index 5411831a18..e56b9e598f 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -121,6 +121,7 @@ log_wrapper(ngx_log_t *log, const char *ident, ngx_uint_t level, if (*p == '/' || *p == '\\') { name.data = p + 1; } + p++; } diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 5fe4564e89..d6e5640a0b 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -443,6 +443,7 @@ ngx_http_lua_ngx_req_set_body_data(lua_State *L) if (b->start == NULL) { return luaL_error(L, "no memory"); } + b->end = b->start + body.len; b->pos = b->start; @@ -454,6 +455,7 @@ ngx_http_lua_ngx_req_set_body_data(lua_State *L) if (rb->bufs == NULL) { return luaL_error(L, "no memory"); } + rb->bufs->next = NULL; b = ngx_create_temp_buf(r->pool, body.len); @@ -899,6 +901,7 @@ ngx_http_lua_ngx_req_set_body_file(lua_State *L) if (rb->bufs == NULL) { return luaL_error(L, "no memory"); } + rb->bufs->next = NULL; b = ngx_calloc_buf(r->pool); diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index c2776d83a7..c99551a963 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4890,6 +4890,7 @@ ngx_http_lua_req_socket(lua_State *L) if (rb == NULL) { return luaL_error(L, "no memory"); } + r->request_body = rb; } diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index a94e181745..4c755f67f3 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -133,6 +133,7 @@ ngx_http_lua_ngx_escape_sql_str(u_char *dst, u_char *src, size_t size) break; } } + src++; size--; } @@ -192,9 +193,11 @@ ngx_http_lua_ngx_escape_sql_str(u_char *dst, u_char *src, size_t size) *dst++ = *src; break; } + } else { *dst++ = *src; } + src++; size--; } /* while (size) */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 272906cb96..4b4f2c7c6f 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -2067,6 +2067,7 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) if (escape[*src >> 5] & (1 << (*src & 0x1f))) { n++; } + src++; size--; } @@ -2084,6 +2085,7 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) } else { *dst++ = *src++; } + size--; } @@ -3886,6 +3888,7 @@ ngx_http_lua_init_vm(lua_State **new_vm, lua_State *parent_vm, if (state == NULL) { return NGX_ERROR; } + state->vm = L; state->count = 1; From 1669904d53e03762604defdd97db7e7d357dd7a3 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 20 Jul 2020 12:34:07 +0800 Subject: [PATCH 435/848] * optimize: avoided use of lua_tolstring in ngx_http_lua_calc_strlen_in_table, ngx_http_lua_copy_str_in_table, ngx_http_lua_socket_udp_send, log_wrapper and ngx_http_lua_ngx_echo. --- src/ngx_http_lua_common.h | 9 ++ src/ngx_http_lua_log.c | 7 ++ src/ngx_http_lua_output.c | 13 ++- src/ngx_http_lua_output.h | 44 +++++++++ src/ngx_http_lua_socket_tcp.c | 15 ++- src/ngx_http_lua_socket_udp.c | 22 +++-- t/009-log.t | 20 ++++ t/058-tcp-socket.t | 173 +++++++++++++++++++++++++++++++++- t/087-udp-socket.t | 104 +++++++++++++++++++- t/164-say.t | 55 +++++++++++ 10 files changed, 449 insertions(+), 13 deletions(-) create mode 100644 t/164-say.t diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 781a2454a1..1232984911 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -44,6 +44,15 @@ typedef struct { #endif +/** + * max positive +1.7976931348623158e+308 + * min positive +2.2250738585072014e-308 + */ +#ifndef NGX_DOUBLE_LEN +#define NGX_DOUBLE_LEN 25 +#endif + + #if (NGX_PCRE) #include # if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) diff --git a/src/ngx_http_lua_log.c b/src/ngx_http_lua_log.c index e56b9e598f..43ab82095a 100644 --- a/src/ngx_http_lua_log.c +++ b/src/ngx_http_lua_log.c @@ -14,6 +14,7 @@ #include "ngx_http_lua_log.h" #include "ngx_http_lua_util.h" #include "ngx_http_lua_log_ringbuf.h" +#include "ngx_http_lua_output.h" static int ngx_http_lua_print(lua_State *L); @@ -143,6 +144,9 @@ log_wrapper(ngx_log_t *log, const char *ident, ngx_uint_t level, type = lua_type(L, i); switch (type) { case LUA_TNUMBER: + size += ngx_http_lua_get_num_len(L, i); + break; + case LUA_TSTRING: lua_tolstring(L, i, &len); size += len; @@ -211,6 +215,9 @@ log_wrapper(ngx_log_t *log, const char *ident, ngx_uint_t level, type = lua_type(L, i); switch (type) { case LUA_TNUMBER: + p = ngx_http_lua_write_num(L, i, p); + break; + case LUA_TSTRING: q = (u_char *) lua_tolstring(L, i, &len); p = ngx_copy(p, q, len); diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index 0fe8840f95..7c7403f667 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -93,6 +93,9 @@ ngx_http_lua_ngx_echo(lua_State *L, unsigned newline) switch (type) { case LUA_TNUMBER: + size += ngx_http_lua_get_num_len(L, i); + break; + case LUA_TSTRING: lua_tolstring(L, i, &len); @@ -173,6 +176,9 @@ ngx_http_lua_ngx_echo(lua_State *L, unsigned newline) type = lua_type(L, i); switch (type) { case LUA_TNUMBER: + b->last = ngx_http_lua_write_num(L, i, b->last); + break; + case LUA_TSTRING: p = lua_tolstring(L, i, &len); b->last = ngx_copy(b->last, (u_char *) p, len); @@ -301,8 +307,10 @@ ngx_http_lua_calc_strlen_in_table(lua_State *L, int index, int arg_i, switch (type) { case LUA_TNUMBER: - case LUA_TSTRING: + size += ngx_http_lua_get_num_len(L, -1); + break; + case LUA_TSTRING: lua_tolstring(L, -1, &len); size += len; break; @@ -396,6 +404,9 @@ ngx_http_lua_copy_str_in_table(lua_State *L, int index, u_char *dst) type = lua_type(L, -1); switch (type) { case LUA_TNUMBER: + dst = ngx_http_lua_write_num(L, -1, dst); + break; + case LUA_TSTRING: p = (u_char *) lua_tolstring(L, -1, &len); dst = ngx_copy(dst, p, len); diff --git a/src/ngx_http_lua_output.h b/src/ngx_http_lua_output.h index 109a4b44b8..ec1db6d050 100644 --- a/src/ngx_http_lua_output.h +++ b/src/ngx_http_lua_output.h @@ -23,6 +23,50 @@ ngx_int_t ngx_http_lua_flush_resume_helper(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx); +/* Get the maximum possible length, not the actual length */ +static ngx_inline size_t +ngx_http_lua_get_num_len(lua_State *L, int idx) +{ + double num; + + num = (double) lua_tonumber(L, idx); + if (num == (double) (long) num) { + return NGX_INT64_LEN; + } + + return NGX_DOUBLE_LEN; +} + + +static ngx_inline u_char * +ngx_http_lua_write_num(lua_State *L, int idx, u_char *dst) +{ + double num; + int n; + + num = (double) lua_tonumber(L, idx); + if (num == (double) (long) num) { + dst = ngx_snprintf(dst, NGX_INT64_LEN, "%l", (long) num); + + } else { + /** + * The maximum number of significant digits is 14 in lua. + * Please refer to lj_strfmt.c for more details. + */ + n = snprintf((char *) dst, NGX_DOUBLE_LEN, "%.14g", num); + if (n < 0) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "snprintf(\"%f\") failed"); + + } else { + dst += n; + } + } + + return dst; +} + + #endif /* _NGX_HTTP_LUA_OUTPUT_H_INCLUDED_ */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index c99551a963..964774041e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2741,11 +2741,15 @@ ngx_http_lua_socket_tcp_send(lua_State *L) type = lua_type(L, 2); switch (type) { case LUA_TNUMBER: + len = ngx_http_lua_get_num_len(L, 2); + break; + case LUA_TSTRING: lua_tolstring(L, 2, &len); break; case LUA_TTABLE: + /* The maximum possible length, not the actual length */ len = ngx_http_lua_calc_strlen_in_table(L, 2, 2, 1 /* strict */); break; @@ -2789,13 +2793,16 @@ ngx_http_lua_socket_tcp_send(lua_State *L) switch (type) { case LUA_TNUMBER: + b->last = ngx_http_lua_write_num(L, 2, b->last); + break; + case LUA_TSTRING: - p = (u_char *) lua_tolstring(L, -1, &len); + p = (u_char *) lua_tolstring(L, 2, &len); b->last = ngx_copy(b->last, (u_char *) p, len); break; case LUA_TTABLE: - b->last = ngx_http_lua_copy_str_in_table(L, -1, b->last); + b->last = ngx_http_lua_copy_str_in_table(L, 2, b->last); break; case LUA_TNIL: @@ -2827,6 +2834,10 @@ ngx_http_lua_socket_tcp_send(lua_State *L) u->request_bufs = cl; + lua_assert(b->last - b->start <= len); + + len = b->last - b->start; + u->request_len = len; /* mimic ngx_http_upstream_init_request here */ diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index f9d8fc0307..fd3e074270 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -727,6 +727,7 @@ ngx_http_lua_socket_udp_send(lua_State *L) ssize_t n; ngx_http_request_t *r; u_char *p; + u_char *str; size_t len; ngx_http_lua_socket_udp_upstream_t *u; int type; @@ -781,11 +782,15 @@ ngx_http_lua_socket_udp_send(lua_State *L) type = lua_type(L, 2); switch (type) { case LUA_TNUMBER: + len = ngx_http_lua_get_num_len(L, 2); + break; + case LUA_TSTRING: lua_tolstring(L, 2, &len); break; case LUA_TTABLE: + /* The maximum possible length, not the actual length */ len = ngx_http_lua_calc_strlen_in_table(L, 2, 2, 1 /* strict */); break; @@ -812,29 +817,29 @@ ngx_http_lua_socket_udp_send(lua_State *L) } query.data = lua_newuserdata(L, len); - query.len = len; + p = query.data; switch (type) { case LUA_TNUMBER: + p = ngx_http_lua_write_num(L, 2, p); + break; + case LUA_TSTRING: - p = (u_char *) lua_tolstring(L, 2, &len); - ngx_memcpy(query.data, (u_char *) p, len); + str = (u_char *) lua_tolstring(L, 2, &len); + p = ngx_cpymem(p, (u_char *) str, len); break; case LUA_TTABLE: - (void) ngx_http_lua_copy_str_in_table(L, 2, query.data); + p = ngx_http_lua_copy_str_in_table(L, 2, p); break; case LUA_TNIL: - p = query.data; *p++ = 'n'; *p++ = 'i'; *p++ = 'l'; break; case LUA_TBOOLEAN: - p = query.data; - if (lua_toboolean(L, 2)) { *p++ = 't'; *p++ = 'r'; @@ -855,6 +860,9 @@ ngx_http_lua_socket_udp_send(lua_State *L) return luaL_error(L, "impossible to reach here"); } + query.len = p - query.data; + ngx_http_lua_assert(query.len <= len); + u->ft_type = 0; /* mimic ngx_http_upstream_init_request here */ diff --git a/t/009-log.t b/t/009-log.t index cb3895b8b2..c4597698ef 100644 --- a/t/009-log.t +++ b/t/009-log.t @@ -548,3 +548,23 @@ ok [error] --- error_log eval "2: hello\0world, client: " + + + +=== TEST 27: test log-level STDERR +Note: maximum number of digits after the decimal-point character is 13 +--- config + location /log { + content_by_lua_block { + ngx.say("before log") + ngx.log(ngx.STDERR, 3.14159265357939723846) + ngx.say("after log") + } + } +--- request +GET /log +--- response_body +before log +after log +--- error_log eval +qr/\[\] \S+: \S+ \[lua\] content_by_lua\(nginx\.conf:\d+\):3: 3.1415926535794/ diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 6ac67e0849..cac464d830 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 222; +plan tests => repeat_each() * 228; our $HtmlDir = html_dir; @@ -1493,7 +1493,7 @@ GET /t -=== TEST 25: send tables of string fragments +=== TEST 25: send tables of string fragments (with integers too) --- config server_tokens off; location /t { @@ -4158,3 +4158,172 @@ orld [error] --- error_log lua tcp socket calling receiveany() method to read at most 7 bytes + + + +=== TEST 70: send tables of string fragments (with floating point number too) +--- config + server_tokens off; + location /t { + #set $port 5000; + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = {"GET", " ", "/foo", " HTTP/", 1, ".", 0, "\r\n", + "Host: localhost\r\n", "Connection: close\r\n", + "Foo: ", 3.1415926, "\r\n", + "\r\n"} + -- req = "OK" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent: ", bytes) + + while true do + local line, err, part = sock:receive() + if line then + ngx.say("received: ", line) + + else + ngx.say("failed to receive a line: ", err, " [", part, "]") + break + end + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + content_by_lua_block { + ngx.say(ngx.req.get_headers()["Foo"]) + } + more_clear_headers Date; + } +--- request +GET /t +--- response_body +connected: 1 +request sent: 73 +received: HTTP/1.1 200 OK +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 10 +received: Connection: close +received: +received: 3.1415926 +failed to receive a line: closed [] +close: 1 nil +--- no_error_log +[error] + + + +=== TEST 71: send numbers +the maximum number of significant digits is 14 in lua +--- config + server_tokens off; + location /t { + #set $port 5000; + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = {"GET", " ", "/foo", " HTTP/", 1, ".", 0, "\r\n", + "Host: localhost\r\n", "Connection: close\r\n", + "Foo: "} + -- req = "OK" + + local total_bytes = 0; + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + total_bytes = total_bytes + bytes; + + bytes, err = sock:send(3.14159265357939723846) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + total_bytes = total_bytes + bytes; + + bytes, err = sock:send(31415926) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + total_bytes = total_bytes + bytes; + + bytes, err = sock:send("\r\n\r\n") + if not bytes then + ngx.say("failed to send request: ", err) + return + end + total_bytes = total_bytes + bytes; + + ngx.say("request sent: ", total_bytes) + + while true do + local line, err, part = sock:receive() + if line then + ngx.say("received: ", line) + + else + ngx.say("failed to receive a line: ", err, " [", part, "]") + break + end + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + content_by_lua_block { + ngx.say(ngx.req.get_headers()["Foo"]) + } + more_clear_headers Date; + } +--- request +GET /t +--- response_body +connected: 1 +request sent: 87 +received: HTTP/1.1 200 OK +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 24 +received: Connection: close +received: +received: 3.141592653579431415926 +failed to receive a line: closed [] +close: 1 nil +--- no_error_log +[error] diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index b3d5e824f0..fc467d769a 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (3 * blocks() + 13); +plan tests => repeat_each() * (3 * blocks() + 15); our $HtmlDir = html_dir; @@ -1117,3 +1117,105 @@ qr/send: fd:\d+ 4 of 4 send: fd:\d+ 5 of 5 send: fd:\d+ 3 of 3/ --- log_level: debug + + + +=== TEST 21: send numbers +Note: maximum number of digits after the decimal-point character is 13 +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local socket = ngx.socket + local udp = socket.udp() + local port = ngx.var.port + udp:settimeout(1000) -- 1 sec + + local ok, err = udp:setpeername("127.0.0.1", ngx.var.port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local function send(data) + local bytes, err = udp:send(data) + if not bytes then + ngx.say("failed to send: ", err) + return + end + ngx.say("sent ok") + end + + send(123456) + send(3.141926) + send(3.141592653579397238) + } + } +--- request +GET /t +--- response_body +sent ok +sent ok +sent ok +--- no_error_log +[error] +--- grep_error_log eval +qr/send: fd:\d+ \d+ of \d+/ +--- grep_error_log_out eval +qr/send: fd:\d+ 6 of 6 +send: fd:\d+ 8 of 8 +send: fd:\d+ 15 of 15/ +--- log_level: debug + + + +=== TEST 22: send tables of string framents (with numbers too) +the maximum number of significant digits is 14 in lua +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + + content_by_lua_block { + local socket = ngx.socket + local udp = socket.udp() + local port = ngx.var.port + udp:settimeout(1000) -- 1 sec + + local ok, err = udp:setpeername("127.0.0.1", ngx.var.port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local function send(data) + local bytes, err = udp:send(data) + if not bytes then + ngx.say("failed to send: ", err) + return + end + ngx.say("sent ok") + end + + send({"integer: ", 1234567890123}) + send({"float: ", 3.1419265}) + send({"float: ", 3.141592653579397238}) + } + } +--- request +GET /t +--- response_body +sent ok +sent ok +sent ok +--- no_error_log +[error] +--- grep_error_log eval +qr/send: fd:\d+ \d+ of \d+/ +--- grep_error_log_out eval +qr/send: fd:\d+ 22 of 22 +send: fd:\d+ 16 of 16 +send: fd:\d+ 22 of 22/ +--- log_level: debug diff --git a/t/164-say.t b/t/164-say.t new file mode 100644 index 0000000000..6114d2459e --- /dev/null +++ b/t/164-say.t @@ -0,0 +1,55 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +repeat_each(2); + +plan tests => blocks() * repeat_each() * 2; + +run_tests(); + +__DATA__ + +=== TEST 1: ngx.say (integer) +--- config + location /lua { + content_by_lua_block { + ngx.say(2) + } + } +--- request +GET /lua +--- response_body +2 + + + +=== TEST 2: ngx.say (floating point number) +the maximum number of significant digits is 14 in lua +--- config + location /lua { + content_by_lua_block { + ngx.say(3.1415926) + ngx.say(3.14159265357939723846) + } + } +--- request +GET /lua +--- response_body +3.1415926 +3.1415926535794 + + + +=== TEST 3: ngx.say (table with number) +--- config + location /lua { + content_by_lua_block { + local data = {123," ", 3.1415926} + ngx.say(data) + } + } +--- request +GET /lua +--- response_body +123 3.1415926 From 393c95470089225a258f4b530b595f43302d6deb Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Thu, 23 Jul 2020 11:28:00 +0800 Subject: [PATCH 436/848] feature: added exit_worker_by* to run Lua code upon nginx worker process exit. (#1682) Co-authored-by: letian Co-authored-by: jinhua.tan --- .travis.yml | 2 +- README.markdown | 115 +++++++++++++------ config | 2 + doc/HttpLuaModule.wiki | 106 +++++++++++------ src/ngx_http_lua_common.h | 4 + src/ngx_http_lua_directive.c | 59 ++++++++++ src/ngx_http_lua_directive.h | 4 + src/ngx_http_lua_exitworkerby.c | 111 ++++++++++++++++++ src/ngx_http_lua_exitworkerby.h | 25 ++++ src/ngx_http_lua_module.c | 17 ++- src/ngx_http_lua_util.h | 1 + t/089-phase.t | 27 ++++- t/162-exit-worker.t | 195 ++++++++++++++++++++++++++++++++ t/163-exit-worker-hup.t | 86 ++++++++++++++ 14 files changed, 681 insertions(+), 73 deletions(-) create mode 100644 src/ngx_http_lua_exitworkerby.c create mode 100644 src/ngx_http_lua_exitworkerby.h create mode 100644 t/162-exit-worker.t create mode 100644 t/163-exit-worker-hup.t diff --git a/.travis.yml b/.travis.yml index bf433bef5e..dd4be41a31 100644 --- a/.travis.yml +++ b/.travis.yml @@ -83,7 +83,7 @@ install: - git clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core + - git clone -b exit_worker https://github.com/rainingmaster/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string diff --git a/README.markdown b/README.markdown index 4046dfe15f..bd29909b53 100644 --- a/README.markdown +++ b/README.markdown @@ -1064,6 +1064,8 @@ Directives * [init_worker_by_lua](#init_worker_by_lua) * [init_worker_by_lua_block](#init_worker_by_lua_block) * [init_worker_by_lua_file](#init_worker_by_lua_file) +* [exit_worker_by_lua_block](#exit_worker_by_lua_block) +* [exit_worker_by_lua_file](#exit_worker_by_lua_file) * [set_by_lua](#set_by_lua) * [set_by_lua_block](#set_by_lua_block) * [set_by_lua_file](#set_by_lua_file) @@ -1578,6 +1580,47 @@ This hook no longer runs in the cache manager and cache loader processes since t [Back to TOC](#directives) +exit_worker_by_lua_block +------------------------ + +**syntax:** *exit_worker_by_lua_block { lua-script }* + +**context:** *http* + +**phase:** *exiting-worker* + +Runs the specified Lua code upon every Nginx worker process's exit when the master process is enabled. When the master process is disabled, this hook will run before the Nginx process exits. + +This hook is often used to release resources allocated by each worker (e.g. resources allocated by [init_worker_by_lua*](#init_worker_by_lua)), or to prevent workers from exiting abnormally. + +For example, + +```nginx + + exit_worker_by_lua_block { + print("log from exit_worker_by_lua_block") + } +``` + +This directive was first introduced in the `v0.10.18` release. + +[Back to TOC](#directives) + +exit_worker_by_lua_file +----------------------- + +**syntax:** *exit_worker_by_lua_file <path-to-lua-script-file>* + +**context:** *http* + +**phase:** *exiting-worker* + +Similar to [exit_worker_by_lua_block](#exit_worker_by_lua_block), but accepts the file path to a Lua source file or Lua bytecode file. + +This directive was first introduced in the `v0.10.18` release. + +[Back to TOC](#directives) + set_by_lua ---------- @@ -3577,7 +3620,7 @@ HTTP status constants Nginx log level constants ------------------------- -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** ```lua @@ -3601,7 +3644,7 @@ print **syntax:** *print(...)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Writes argument values into the Nginx `error.log` file with the `ngx.NOTICE` log level. @@ -3621,7 +3664,7 @@ There is a hard coded `2048` byte limitation on error message lengths in the Ngi ngx.ctx ------- -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, exit_worker_by_lua** This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). @@ -5448,7 +5491,7 @@ ngx.log **syntax:** *ngx.log(log_level, ...)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Log arguments concatenated to error.log with the given logging level. @@ -5595,7 +5638,7 @@ ngx.escape_uri **syntax:** *newstr = ngx.escape_uri(str, type?)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Since `v0.10.16`, this function accepts an optional `type` argument. It accepts the following values (defaults to `2`): @@ -5614,7 +5657,7 @@ ngx.unescape_uri **syntax:** *newstr = ngx.unescape_uri(str)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, exit_worker_by_lua** Unescape `str` as an escaped URI component. @@ -5878,7 +5921,7 @@ ngx.today **syntax:** *str = ngx.today()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns current date (in the format `yyyy-mm-dd`) from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -5891,7 +5934,7 @@ ngx.time **syntax:** *secs = ngx.time()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns the elapsed seconds from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -5904,7 +5947,7 @@ ngx.now **syntax:** *secs = ngx.now()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -5919,7 +5962,7 @@ ngx.update_time **syntax:** *ngx.update_time()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Forcibly updates the Nginx current time cache. This call involves a syscall and thus has some overhead, so do not abuse it. @@ -5932,7 +5975,7 @@ ngx.localtime **syntax:** *str = ngx.localtime()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the Nginx cached time (no syscall involved unlike Lua's [os.date](https://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). @@ -5945,7 +5988,7 @@ ngx.utctime **syntax:** *str = ngx.utctime()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the Nginx cached time (no syscall involved unlike Lua's [os.date](https://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). @@ -5958,7 +6001,7 @@ ngx.cookie_time **syntax:** *str = ngx.cookie_time(sec)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns a formatted string can be used as the cookie expiration time. The parameter `sec` is the time stamp in seconds (like those returned from [ngx.time](#ngxtime)). @@ -5975,7 +6018,7 @@ ngx.http_time **syntax:** *str = ngx.http_time(sec)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns a formated string can be used as the http header time (for example, being used in `Last-Modified` header). The parameter `sec` is the time stamp in seconds (like those returned from [ngx.time](#ngxtime)). @@ -5992,7 +6035,7 @@ ngx.parse_http_time **syntax:** *sec = ngx.parse_http_time(str)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Parse the http time string (as returned by [ngx.http_time](#ngxhttp_time)) into seconds. Returns the seconds or `nil` if the input string is in bad forms. @@ -6022,7 +6065,7 @@ ngx.re.match **syntax:** *captures, err = ngx.re.match(subject, regex, options?, ctx?, res_table?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Matches the `subject` string using the Perl compatible regular expression `regex` with the optional `options`. @@ -6181,7 +6224,7 @@ ngx.re.find **syntax:** *from, to, err = ngx.re.find(subject, regex, options?, ctx?, nth?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Similar to [ngx.re.match](#ngxrematch) but only returns the beginning index (`from`) and end index (`to`) of the matched substring. The returned indexes are 1-based and can be fed directly into the [string.sub](https://www.lua.org/manual/5.1/manual.html#pdf-string.sub) API function to obtain the matched substring. @@ -6236,7 +6279,7 @@ ngx.re.gmatch **syntax:** *iterator, err = ngx.re.gmatch(subject, regex, options?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Similar to [ngx.re.match](#ngxrematch), but returns a Lua iterator instead, so as to let the user programmer iterate all the matches over the `` string argument with the PCRE `regex`. @@ -6315,7 +6358,7 @@ ngx.re.sub **syntax:** *newstr, n, err = ngx.re.sub(subject, regex, replace, options?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Substitutes the first match of the Perl compatible regular expression `regex` on the `subject` argument string with the string or function argument `replace`. The optional `options` argument has exactly the same meaning as in [ngx.re.match](#ngxrematch). @@ -6383,7 +6426,7 @@ ngx.re.gsub **syntax:** *newstr, n, err = ngx.re.gsub(subject, regex, replace, options?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Just like [ngx.re.sub](#ngxresub), but does global substitution. @@ -6424,7 +6467,7 @@ ngx.shared.DICT **syntax:** *dict = ngx.shared\[name_var\]* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Fetching the shm-based Lua dictionary object for the shared memory zone named `DICT` defined by the [lua_shared_dict](#lua_shared_dict) directive. @@ -7806,7 +7849,7 @@ ngx.get_phase **syntax:** *str = ngx.get_phase()* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Retrieves the current running phase name. Possible return values are @@ -7838,6 +7881,8 @@ Retrieves the current running phase name. Possible return values are for the context of [log_by_lua*](#log_by_lua). * `timer` for the context of user callback functions for [ngx.timer.*](#ngxtimerat). +* `exit_worker` + for the context of [exit_worker_by_lua*](#exit_worker_by_lua). This API was first introduced in the `v0.5.10` release. @@ -8147,7 +8192,7 @@ ngx.timer.at **syntax:** *hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Creates an Nginx timer with a user callback function as well as optional user arguments. @@ -8283,7 +8328,7 @@ ngx.timer.every **syntax:** *hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Similar to the [ngx.timer.at](#ngxtimerat) API function, but @@ -8306,7 +8351,7 @@ ngx.timer.running_count **syntax:** *count = ngx.timer.running_count()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns the number of timers currently running. @@ -8319,7 +8364,7 @@ ngx.timer.pending_count **syntax:** *count = ngx.timer.pending_count()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** Returns the number of pending timers. @@ -8332,7 +8377,7 @@ ngx.config.subsystem **syntax:** *subsystem = ngx.config.subsystem* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** This string field indicates the Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value `"http"`. For [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme), however, this field takes the value `"stream"`. @@ -8346,7 +8391,7 @@ ngx.config.debug **syntax:** *debug = ngx.config.debug* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** This boolean field indicates whether the current Nginx is a debug build, i.e., being built by the `./configure` option `--with-debug`. @@ -8359,7 +8404,7 @@ ngx.config.prefix **syntax:** *prefix = ngx.config.prefix()* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** Returns the Nginx server "prefix" path, as determined by the `-p` command-line option when running the Nginx executable, or the path specified by the `--prefix` command-line option when building Nginx with the `./configure` script. @@ -8372,7 +8417,7 @@ ngx.config.nginx_version **syntax:** *ver = ngx.config.nginx_version* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** This field take an integral value indicating the version number of the current Nginx core being used. For example, the version number `1.4.3` results in the Lua number 1004003. @@ -8411,7 +8456,7 @@ ngx.worker.exiting **syntax:** *exiting = ngx.worker.exiting()* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** This function returns a boolean value indicating whether the current Nginx worker process already starts exiting. Nginx worker process exiting happens on Nginx server quit or configuration reload (aka HUP reload). @@ -8424,7 +8469,7 @@ ngx.worker.pid **syntax:** *pid = ngx.worker.pid()* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** This function returns a Lua number for the process ID (PID) of the current Nginx worker process. This API is more efficient than `ngx.var.pid` and can be used in contexts where the [ngx.var.VARIABLE](#ngxvarvariable) API cannot be used (like [init_worker_by_lua](#init_worker_by_lua)). @@ -8437,7 +8482,7 @@ ngx.worker.count **syntax:** *count = ngx.worker.count()* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua** Returns the total number of the Nginx worker processes (i.e., the value configured by the [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes) @@ -8452,7 +8497,7 @@ ngx.worker.id **syntax:** *id = ngx.worker.id()* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_worker_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_worker_by_lua*, exit_worker_by_lua** Returns the ordinal number of the current Nginx worker processes (starting from number 0). @@ -8557,7 +8602,7 @@ ndk.set_var.DIRECTIVE **syntax:** *res = ndk.set_var.DIRECTIVE_NAME* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** This mechanism allows calling other Nginx C modules' directives that are implemented by [Nginx Devel Kit](https://github.com/simplresty/ngx_devel_kit) (NDK)'s set_var submodule's `ndk_set_var_value`. diff --git a/config b/config index 18014aa97c..6cb951bb42 100644 --- a/config +++ b/config @@ -278,6 +278,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_bodyfilterby.c \ $ngx_addon_dir/src/ngx_http_lua_initby.c \ $ngx_addon_dir/src/ngx_http_lua_initworkerby.c \ + $ngx_addon_dir/src/ngx_http_lua_exitworkerby.c \ $ngx_addon_dir/src/ngx_http_lua_socket_udp.c \ $ngx_addon_dir/src/ngx_http_lua_req_method.c \ $ngx_addon_dir/src/ngx_http_lua_phase.c \ @@ -339,6 +340,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_bodyfilterby.h \ $ngx_addon_dir/src/ngx_http_lua_initby.h \ $ngx_addon_dir/src/ngx_http_lua_initworkerby.h \ + $ngx_addon_dir/src/ngx_http_lua_exitworkerby.h \ $ngx_addon_dir/src/ngx_http_lua_socket_udp.h \ $ngx_addon_dir/src/ngx_http_lua_probe.h \ $ngx_addon_dir/src/ngx_http_lua_uthread.h \ diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 37d8a1367b..04a2694972 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1281,6 +1281,40 @@ This directive was first introduced in the v0.9.5 release. This hook no longer runs in the cache manager and cache loader processes since the v0.10.12 release. +== exit_worker_by_lua_block == + +'''syntax:''' ''exit_worker_by_lua_block { lua-script }'' + +'''context:''' ''http'' + +'''phase:''' ''exiting-worker'' + +Runs the specified Lua code upon every Nginx worker process's exit when the master process is enabled. When the master process is disabled, this hook will run before the Nginx process exits. + +This hook is often used to release resources allocated by each worker (e.g. resources allocated by [[#init_worker_by_lua|init_worker_by_lua*]]), or to prevent workers from exiting abnormally. + +For example, + + + exit_worker_by_lua_block { + print("log from exit_worker_by_lua_block") + } + + +This directive was first introduced in the v0.10.18 release. + +== exit_worker_by_lua_file == + +'''syntax:''' ''exit_worker_by_lua_file '' + +'''context:''' ''http'' + +'''phase:''' ''exiting-worker'' + +Similar to [[#exit_worker_by_lua_block|exit_worker_by_lua_block]], but accepts the file path to a Lua source file or Lua bytecode file. + +This directive was first introduced in the v0.10.18 release. + == set_by_lua == '''syntax:''' ''set_by_lua $res [$arg1 $arg2 ...]'' @@ -2913,7 +2947,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture == Nginx log level constants == -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' ngx.STDERR @@ -2933,7 +2967,7 @@ These constants are usually used by the [[#ngx.log|ngx.log]] method. '''syntax:''' ''print(...)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Writes argument values into the Nginx error.log file with the ngx.NOTICE log level. @@ -2949,7 +2983,7 @@ There is a hard coded 2048 byte limitation on error message lengths == ngx.ctx == -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, exit_worker_by_lua*'' This table can be used to store per-request Lua context data and has a life time identical to the current request (as with the Nginx variables). @@ -4573,7 +4607,7 @@ Just as [[#ngx.print|ngx.print]] but also emit a trailing newline. '''syntax:''' ''ngx.log(log_level, ...)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Log arguments concatenated to error.log with the given logging level. @@ -4700,7 +4734,7 @@ This method was introduced in the 0.5.0rc30 release. '''syntax:''' ''newstr = ngx.escape_uri(str, type?)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Since `v0.10.16`, this function accepts an optional type argument. It accepts the following values (defaults to `2`): @@ -4716,7 +4750,7 @@ alphabetic characters, digits, -, ., _, '''syntax:''' ''newstr = ngx.unescape_uri(str)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, exit_worker_by_lua*'' Unescape str as an escaped URI component. @@ -4937,7 +4971,7 @@ Returns a quoted SQL string literal according to the MySQL quoting rules. '''syntax:''' ''str = ngx.today()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns current date (in the format yyyy-mm-dd) from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -4947,7 +4981,7 @@ This is the local time. '''syntax:''' ''secs = ngx.time()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns the elapsed seconds from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -4957,7 +4991,7 @@ Updates of the Nginx time cache can be forced by calling [[#ngx.update_time|ngx. '''syntax:''' ''secs = ngx.now()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -4969,7 +5003,7 @@ This API was first introduced in v0.3.1rc32. '''syntax:''' ''ngx.update_time()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Forcibly updates the Nginx current time cache. This call involves a syscall and thus has some overhead, so do not abuse it. @@ -4979,7 +5013,7 @@ This API was first introduced in v0.3.1rc32. '''syntax:''' ''str = ngx.localtime()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the Nginx cached time (no syscall involved unlike Lua's [https://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). @@ -4989,7 +5023,7 @@ This is the local time. '''syntax:''' ''str = ngx.utctime()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the Nginx cached time (no syscall involved unlike Lua's [https://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). @@ -4999,7 +5033,7 @@ This is the UTC time. '''syntax:''' ''str = ngx.cookie_time(sec)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns a formatted string can be used as the cookie expiration time. The parameter sec is the time stamp in seconds (like those returned from [[#ngx.time|ngx.time]]). @@ -5012,7 +5046,7 @@ Returns a formatted string can be used as the cookie expiration time. The parame '''syntax:''' ''str = ngx.http_time(sec)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns a formated string can be used as the http header time (for example, being used in Last-Modified header). The parameter sec is the time stamp in seconds (like those returned from [[#ngx.time|ngx.time]]). @@ -5025,7 +5059,7 @@ Returns a formated string can be used as the http header time (for example, bein '''syntax:''' ''sec = ngx.parse_http_time(str)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Parse the http time string (as returned by [[#ngx.http_time|ngx.http_time]]) into seconds. Returns the seconds or nil if the input string is in bad forms. @@ -5048,7 +5082,7 @@ Returns true if the current request is an Nginx subrequest, or subject string using the Perl compatible regular expression regex with the optional options. @@ -5196,7 +5230,7 @@ This feature was introduced in the v0.2.1rc11 release. '''syntax:''' ''from, to, err = ngx.re.find(subject, regex, options?, ctx?, nth?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Similar to [[#ngx.re.match|ngx.re.match]] but only returns the beginning index (from) and end index (to) of the matched substring. The returned indexes are 1-based and can be fed directly into the [https://www.lua.org/manual/5.1/manual.html#pdf-string.sub string.sub] API function to obtain the matched substring. @@ -5246,7 +5280,7 @@ This API function was first introduced in the v0.9.2 release. '''syntax:''' ''iterator, err = ngx.re.gmatch(subject, regex, options?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Similar to [[#ngx.re.match|ngx.re.match]], but returns a Lua iterator instead, so as to let the user programmer iterate all the matches over the string argument with the PCRE regex. @@ -5320,7 +5354,7 @@ This feature was first introduced in the v0.2.1rc12 release. '''syntax:''' ''newstr, n, err = ngx.re.sub(subject, regex, replace, options?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Substitutes the first match of the Perl compatible regular expression regex on the subject argument string with the string or function argument replace. The optional options argument has exactly the same meaning as in [[#ngx.re.match|ngx.re.match]]. @@ -5381,7 +5415,7 @@ This feature was first introduced in the v0.2.1rc13 release. '''syntax:''' ''newstr, n, err = ngx.re.gsub(subject, regex, replace, options?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Just like [[#ngx.re.sub|ngx.re.sub]], but does global substitution. @@ -5417,7 +5451,7 @@ This feature was first introduced in the v0.2.1rc15 release. '''syntax:''' ''dict = ngx.shared[name_var]'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Fetching the shm-based Lua dictionary object for the shared memory zone named DICT defined by the [[#lua_shared_dict|lua_shared_dict]] directive. @@ -6643,7 +6677,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''str = ngx.get_phase()'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Retrieves the current running phase name. Possible return values are @@ -6675,6 +6709,8 @@ Retrieves the current running phase name. Possible return values are : for the context of [[#log_by_lua|log_by_lua*]]. * timer : for the context of user callback functions for [[#ngx.timer.at|ngx.timer.*]]. +* exit_worker +: for the context of [[#exit_worker_by_lua|exit_worker_by_lua*]]. This API was first introduced in the v0.5.10 release. @@ -6964,7 +7000,7 @@ See also [[#lua_check_client_abort|lua_check_client_abort]]. '''syntax:''' ''hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Creates an Nginx timer with a user callback function as well as optional user arguments. @@ -7095,7 +7131,7 @@ This API was first introduced in the v0.8.0 release. '''syntax:''' ''hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Similar to the [[#ngx.timer.at|ngx.timer.at]] API function, but @@ -7115,7 +7151,7 @@ This API was first introduced in the v0.10.9 release. '''syntax:''' ''count = ngx.timer.running_count()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns the number of timers currently running. @@ -7125,7 +7161,7 @@ This directive was first introduced in the v0.9.20 release. '''syntax:''' ''count = ngx.timer.pending_count()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' Returns the number of pending timers. @@ -7135,7 +7171,7 @@ This directive was first introduced in the v0.9.20 release. '''syntax:''' ''subsystem = ngx.config.subsystem'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' This string field indicates the Nginx subsystem the current Lua environment is based on. For this module, this field always takes the string value "http". For [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module], however, this field takes the value "stream". @@ -7146,7 +7182,7 @@ This field was first introduced in the 0.10.1. '''syntax:''' ''debug = ngx.config.debug'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' This boolean field indicates whether the current Nginx is a debug build, i.e., being built by the ./configure option --with-debug. @@ -7156,7 +7192,7 @@ This field was first introduced in the 0.8.7. '''syntax:''' ''prefix = ngx.config.prefix()'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' Returns the Nginx server "prefix" path, as determined by the -p command-line option when running the Nginx executable, or the path specified by the --prefix command-line option when building Nginx with the ./configure script. @@ -7166,7 +7202,7 @@ This function was first introduced in the 0.9.2. '''syntax:''' ''ver = ngx.config.nginx_version'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' This field take an integral value indicating the version number of the current Nginx core being used. For example, the version number 1.4.3 results in the Lua number 1004003. @@ -7196,7 +7232,7 @@ This API was first introduced in the 0.9.3 release. '''syntax:''' ''exiting = ngx.worker.exiting()'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' This function returns a boolean value indicating whether the current Nginx worker process already starts exiting. Nginx worker process exiting happens on Nginx server quit or configuration reload (aka HUP reload). @@ -7206,7 +7242,7 @@ This API was first introduced in the 0.9.3 release. '''syntax:''' ''pid = ngx.worker.pid()'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' This function returns a Lua number for the process ID (PID) of the current Nginx worker process. This API is more efficient than ngx.var.pid and can be used in contexts where the [[#ngx.var.VARIABLE|ngx.var.VARIABLE]] API cannot be used (like [[#init_worker_by_lua|init_worker_by_lua]]). @@ -7216,7 +7252,7 @@ This API was first introduced in the 0.9.5 release. '''syntax:''' ''count = ngx.worker.count()'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_by_lua*, init_worker_by_lua*, exit_worker_by_lua*'' Returns the total number of the Nginx worker processes (i.e., the value configured by the [https://nginx.org/en/docs/ngx_core_module.html#worker_processes worker_processes] @@ -7228,7 +7264,7 @@ This API was first introduced in the 0.9.20 release. '''syntax:''' ''id = ngx.worker.id()'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_worker_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, init_worker_by_lua*, exit_worker_by_lua*'' Returns the ordinal number of the current Nginx worker processes (starting from number 0). @@ -7318,7 +7354,7 @@ This feature requires at least ngx_lua v0.10.0. '''syntax:''' ''res = ndk.set_var.DIRECTIVE_NAME'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' This mechanism allows calling other Nginx C modules' directives that are implemented by [https://github.com/simplresty/ngx_devel_kit Nginx Devel Kit] (NDK)'s set_var submodule's ndk_set_var_value. diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 1232984911..57c031f276 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -138,6 +138,7 @@ typedef struct { #define NGX_HTTP_LUA_CONTEXT_SSL_CERT 0x0400 #define NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE 0x0800 #define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x1000 +#define NGX_HTTP_LUA_CONTEXT_EXIT_WORKER 0x2000 #define NGX_HTTP_LUA_FFI_NO_REQ_CTX -100 @@ -226,6 +227,9 @@ struct ngx_http_lua_main_conf_s { ngx_http_lua_main_conf_handler_pt init_worker_handler; ngx_str_t init_worker_src; + ngx_http_lua_main_conf_handler_pt exit_worker_handler; + ngx_str_t exit_worker_src; + ngx_http_lua_balancer_peer_data_t *balancer_peer_data; /* neither yielding nor recursion is possible in * balancer_by_lua*, so there cannot be any races among diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 6b7ccc7cbe..1ec641e079 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -23,6 +23,7 @@ #include "ngx_http_lua_bodyfilterby.h" #include "ngx_http_lua_initby.h" #include "ngx_http_lua_initworkerby.h" +#include "ngx_http_lua_exitworkerby.h" #include "ngx_http_lua_shdict.h" #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_lex.h" @@ -1200,6 +1201,64 @@ ngx_http_lua_init_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } +char * +ngx_http_lua_exit_worker_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ + char *rv; + ngx_conf_t save; + + save = *cf; + cf->handler = ngx_http_lua_exit_worker_by_lua; + cf->handler_conf = conf; + + rv = ngx_http_lua_conf_lua_block_parse(cf, cmd); + + *cf = save; + + return rv; +} + + +char * +ngx_http_lua_exit_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ + u_char *name; + ngx_str_t *value; + ngx_http_lua_main_conf_t *lmcf = conf; + + /* must specify a content handler */ + if (cmd->post == NULL) { + return NGX_CONF_ERROR; + } + + if (lmcf->exit_worker_handler) { + return "is duplicate"; + } + + value = cf->args->elts; + + lmcf->exit_worker_handler = (ngx_http_lua_main_conf_handler_pt) cmd->post; + + if (cmd->post == ngx_http_lua_exit_worker_by_file) { + name = ngx_http_lua_rebase_path(cf->pool, value[1].data, + value[1].len); + if (name == NULL) { + return NGX_CONF_ERROR; + } + + lmcf->exit_worker_src.data = name; + lmcf->exit_worker_src.len = ngx_strlen(name); + + } else { + lmcf->exit_worker_src = value[1]; + } + + return NGX_CONF_OK; +} + + #if defined(NDK) && NDK static ngx_int_t ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index 8555e2f364..adfba12761 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -53,6 +53,10 @@ char *ngx_http_lua_init_worker_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_init_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +char *ngx_http_lua_exit_worker_by_lua_block(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf); +char *ngx_http_lua_exit_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); char *ngx_http_lua_code_cache(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_load_resty_core(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); diff --git a/src/ngx_http_lua_exitworkerby.c b/src/ngx_http_lua_exitworkerby.c new file mode 100644 index 0000000000..cd59dc4bed --- /dev/null +++ b/src/ngx_http_lua_exitworkerby.c @@ -0,0 +1,111 @@ + +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#include "ngx_http_lua_exitworkerby.h" +#include "ngx_http_lua_util.h" + + +void +ngx_http_lua_exit_worker(ngx_cycle_t *cycle) +{ + ngx_http_lua_main_conf_t *lmcf; + ngx_connection_t *c = NULL; + ngx_http_request_t *r = NULL; + ngx_http_lua_ctx_t *ctx; + ngx_http_conf_ctx_t *conf_ctx; + + lmcf = ngx_http_cycle_get_module_main_conf(cycle, ngx_http_lua_module); + if (lmcf == NULL + || lmcf->exit_worker_handler == NULL + || lmcf->lua == NULL +#if !(NGX_WIN32) + || (ngx_process == NGX_PROCESS_HELPER +# ifdef HAVE_PRIVILEGED_PROCESS_PATCH + && !ngx_is_privileged_agent +# endif + ) +#endif /* NGX_WIN32 */ + ) + { + return; + } + + conf_ctx = ((ngx_http_conf_ctx_t *) cycle->conf_ctx[ngx_http_module.index]); + + c = ngx_http_lua_create_fake_connection(NULL); + if (c == NULL) { + goto failed; + } + + c->log = ngx_cycle->log; + + r = ngx_http_lua_create_fake_request(c); + if (r == NULL) { + goto failed; + } + + r->main_conf = conf_ctx->main_conf; + r->srv_conf = conf_ctx->srv_conf; + r->loc_conf = conf_ctx->loc_conf; + + ctx = ngx_http_lua_create_ctx(r); + if (ctx == NULL) { + goto failed; + } + + ctx->context = NGX_HTTP_LUA_CONTEXT_EXIT_WORKER; + ctx->cur_co_ctx = NULL; + + ngx_http_lua_set_req(lmcf->lua, r); + + (void) lmcf->exit_worker_handler(cycle->log, lmcf, lmcf->lua); + + ngx_destroy_pool(c->pool); + return; + +failed: + + if (c) { + ngx_http_lua_close_fake_connection(c); + } + + return; +} + + +ngx_int_t +ngx_http_lua_exit_worker_by_inline(ngx_log_t *log, + ngx_http_lua_main_conf_t *lmcf, lua_State *L) +{ + int status; + + status = luaL_loadbuffer(L, (char *) lmcf->exit_worker_src.data, + lmcf->exit_worker_src.len, "=exit_worker_by_lua") + || ngx_http_lua_do_call(log, L); + + return ngx_http_lua_report(log, L, status, "exit_worker_by_lua"); +} + + +ngx_int_t +ngx_http_lua_exit_worker_by_file(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, + lua_State *L) +{ + int status; + + status = luaL_loadfile(L, (char *) lmcf->exit_worker_src.data) + || ngx_http_lua_do_call(log, L); + + return ngx_http_lua_report(log, L, status, "exit_worker_by_lua_file"); +} + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_exitworkerby.h b/src/ngx_http_lua_exitworkerby.h new file mode 100644 index 0000000000..3a4274c042 --- /dev/null +++ b/src/ngx_http_lua_exitworkerby.h @@ -0,0 +1,25 @@ + +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + + +#ifndef _NGX_HTTP_LUA_EXITWORKERBY_H_INCLUDED_ +#define _NGX_HTTP_LUA_EXITWORKERBY_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +ngx_int_t ngx_http_lua_exit_worker_by_inline(ngx_log_t *log, + ngx_http_lua_main_conf_t *lmcf, lua_State *L); + +ngx_int_t ngx_http_lua_exit_worker_by_file(ngx_log_t *log, + ngx_http_lua_main_conf_t *lmcf, lua_State *L); + +void ngx_http_lua_exit_worker(ngx_cycle_t *cycle); + + +#endif /* _NGX_HTTP_LUA_EXITWORKERBY_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 9816d86441..098fe87add 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -22,6 +22,7 @@ #include "ngx_http_lua_bodyfilterby.h" #include "ngx_http_lua_initby.h" #include "ngx_http_lua_initworkerby.h" +#include "ngx_http_lua_exitworkerby.h" #include "ngx_http_lua_probe.h" #include "ngx_http_lua_semaphore.h" #include "ngx_http_lua_balancer.h" @@ -229,6 +230,20 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, (void *) ngx_http_lua_init_worker_by_file }, + { ngx_string("exit_worker_by_lua_block"), + NGX_HTTP_MAIN_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS, + ngx_http_lua_exit_worker_by_lua_block, + NGX_HTTP_MAIN_CONF_OFFSET, + 0, + (void *) ngx_http_lua_exit_worker_by_inline }, + + { ngx_string("exit_worker_by_lua_file"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, + ngx_http_lua_exit_worker_by_lua, + NGX_HTTP_MAIN_CONF_OFFSET, + 0, + (void *) ngx_http_lua_exit_worker_by_file }, + #if defined(NDK) && NDK /* set_by_lua_block $res { inline Lua code } */ { ngx_string("set_by_lua_block"), @@ -636,7 +651,7 @@ ngx_module_t ngx_http_lua_module = { ngx_http_lua_init_worker, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ - NULL, /* exit process */ + ngx_http_lua_exit_worker, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 57c0814bfb..52d804bff0 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -53,6 +53,7 @@ : (c) == NGX_HTTP_LUA_CONTEXT_BODY_FILTER ? "body_filter_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_TIMER ? "ngx.timer" \ : (c) == NGX_HTTP_LUA_CONTEXT_INIT_WORKER ? "init_worker_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_EXIT_WORKER ? "exit_worker_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_BALANCER ? "balancer_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CERT ? "ssl_certificate_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE ? \ diff --git a/t/089-phase.t b/t/089-phase.t index 94b7619ec9..538dff8723 100644 --- a/t/089-phase.t +++ b/t/089-phase.t @@ -8,7 +8,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 1); +plan tests => repeat_each() * (blocks() * 2 + 2); #no_diff(); #no_long_string(); @@ -176,3 +176,28 @@ GET /lua init_worker --- no_error_log [error] + + + +=== TEST 11: get_phase in exit_worker_by_lua +--- http_config + exit_worker_by_lua_block { + local phase = ngx.get_phase() + ngx.log(ngx.ERR, phase) + ngx.log(ngx.ERR, "exiting now") + } +--- config + location /lua { + content_by_lua_block { + ngx.say("ok") + } + } +--- request +GET /lua +--- response_body +ok +--- shutdown_error_log eval +[ +qr/exit_worker_by_lua:\d+: exit_worker/, +qr/exiting now$/, +] diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t new file mode 100644 index 0000000000..cb0edf7076 --- /dev/null +++ b/t/162-exit-worker.t @@ -0,0 +1,195 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +master_on(); +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 2 + 8); + +#log_level("warn"); +no_long_string(); +our $HtmlDir = html_dir; + +run_tests(); + +__DATA__ + +=== TEST 1: simple exit_worker_by_lua_block +--- http_config + exit_worker_by_lua_block { + ngx.log(ngx.NOTICE, "log from exit_worker_by_lua_block") + } +--- config + location /t { + echo "ok"; + } +--- request +GET /t +--- response_body +ok +--- shutdown_error_log +log from exit_worker_by_lua_block + + + +=== TEST 2: simple exit_worker_by_lua_file +--- http_config + exit_worker_by_lua_file html/exit_worker.lua; +--- config + location /t { + echo "ok"; + } +--- user_files +>>> exit_worker.lua +ngx.log(ngx.NOTICE, "log from exit_worker_by_lua_file") +--- request +GET /t +--- response_body +ok +--- shutdown_error_log +log from exit_worker_by_lua_file + + + +=== TEST 3: exit_worker_by_lua (require a global table) +--- http_config eval + qq{lua_package_path '$::HtmlDir/?.lua;;'; + exit_worker_by_lua_block { + foo = require("foo") + ngx.log(ngx.NOTICE, foo.bar) + }} +--- config + location /t { + content_by_lua_block { + foo = require("foo") + foo.bar = "hello, world" + ngx.say("ok") + } + } +--- user_files +>>> foo.lua +return {} +--- request +GET /t +--- response_body +ok +--- shutdown_error_log +hello, world + + + +=== TEST 4: exit_worker_by_lua single process ngx.timer not work +--- http_config + exit_worker_by_lua_block { + local function bar() + ngx.log(ngx.ERR, "run the timer!" + end + + local ok, err = ngx.timer.at(0, bar) + if not ok then + ngx.log(ngx.ERR, "failed to create timer: ", err) + else + ngx.log(ngx.NOTICE, "success") + end + } +--- config + location /t { + echo "ok"; + } +--- request +GET /t +--- response_body +ok +--- no_error_log +[error] + + + +=== TEST 5: exit_worker_by_lua use shdict +--- http_config + lua_shared_dict dog 1m; + exit_worker_by_lua_block { + local dog = ngx.shared.dog + local val, err = dog:get("foo") + if not val then + ngx.log(ngx.ERR, "failed get shdict: ", err) + else + ngx.log(ngx.NOTICE, "get val: ", val) + end + } +--- config + location /t { + content_by_lua_block { + local dog = ngx.shared.dog + dog:set("foo", 100) + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- shutdown_error_log +get val: 100 + + + +=== TEST 6: skip in cache processes (with exit worker and privileged agent) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + init_by_lua_block { + assert(require "ngx.process".enable_privileged_agent()) + } + + exit_worker_by_lua_block { + local process = require "ngx.process" + ngx.log(ngx.INFO, "hello from exit worker by lua, process type: ", process.type()) + } +--- config + location = /t { + return 200; + } +--- request + GET /t +--- no_error_log +[error] +--- shutdown_error_log eval +[ +qr/cache loader process \d+ exited/, +qr/cache manager process \d+ exited/, +qr/hello from exit worker by lua, process type: worker/, +qr/hello from exit worker by lua, process type: privileged agent/, +qr/privileged agent process \d+ exited/, +] + + + +=== TEST 7: skipin cache processes (with init worker but without privileged agent) +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;../lua-resty-lrucache/lib/?.lua;;"; + + proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:1m; + + exit_worker_by_lua_block { + local process = require "ngx.process" + ngx.log(ngx.INFO, "hello from exit worker by lua, process type: ", process.type()) + } +--- config + location = /t { + return 200; + } +--- request + GET /t +--- no_error_log +[error] +start privileged agent process +--- shutdown_error_log eval +[ +qr/cache loader process \d+ exited/, +qr/cache manager process \d+ exited/, +qr/hello from exit worker by lua, process type: worker/, +] diff --git a/t/163-exit-worker-hup.t b/t/163-exit-worker-hup.t new file mode 100644 index 0000000000..3f516aba98 --- /dev/null +++ b/t/163-exit-worker-hup.t @@ -0,0 +1,86 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_CHECK_LEAK}) { + $SkipReason = "unavailable for the hup tests"; + + } else { + $ENV{TEST_NGINX_USE_HUP} = 1; + undef $ENV{TEST_NGINX_USE_STAP}; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 2 + 2); + +no_long_string(); + +worker_connections(1024); +run_tests(); + +__DATA__ + +=== TEST 1: simple exit_worker_by_lua_block with hup +--- http_config + exit_worker_by_lua_block { + ngx.log(ngx.NOTICE, "log from exit_worker_by_lua_block") + } +--- config + location /t { + content_by_lua_block { + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- shutdown_error_log +log from exit_worker_by_lua_block + + + +=== TEST 2: exit after worker_shutdown_timeout +--- main_config + worker_shutdown_timeout 1; +--- http_config + exit_worker_by_lua_block { + ngx.log(ngx.NOTICE, "log from exit_worker_by_lua_block") + } + + server { + listen 12345; + + location = /t { + echo 'hello world'; + } + } +--- config + location /t { + content_by_lua_block { + ngx.timer.at(0, function () + local sock = ngx.socket.tcp() + sock:connect("127.0.0.1", 12345) + local reader = sock:receiveuntil("unknow") + ngx.log(ngx.NOTICE, "reading to block the exiting") + reader() + end) + + ngx.sleep(0) + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- error_log +reading to block the exiting +--- shutdown_error_log +log from exit_worker_by_lua_block From fd25474917f0912c0f5054182ef63bc270467510 Mon Sep 17 00:00:00 2001 From: rainingmaster <312841925@qq.com> Date: Fri, 24 Jul 2020 11:41:28 +0800 Subject: [PATCH 437/848] test: use official repo instead of third pard repo --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index dd4be41a31..bf433bef5e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -83,7 +83,7 @@ install: - git clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - - git clone -b exit_worker https://github.com/rainingmaster/lua-resty-core.git ../lua-resty-core + - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string From c0e460a5387e816f4eddc561c0c1f04ab6294a62 Mon Sep 17 00:00:00 2001 From: ArchangelSDY Date: Thu, 30 Jul 2020 19:37:47 +0800 Subject: [PATCH 438/848] feature: add FFI interface to verify SSL client certificate (#1666) --- src/ngx_http_lua_ssl_certby.c | 136 +++++++++++++++++++++ t/140-ssl-c-api.t | 214 ++++++++++++++++++++++++++++++++++ 2 files changed, 350 insertions(+) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 6106c6946c..2e09a03857 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1294,4 +1294,140 @@ ngx_http_lua_ffi_set_priv_key(ngx_http_request_t *r, } +static int +ngx_http_lua_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) +{ + /* + * we never terminate handshake here and user can later use + * $ssl_client_verify to check verification result. + * + * this is consistent with Nginx behavior. + */ + return 1; +} + + +int +ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, + int depth, char **err) +{ + ngx_http_lua_ctx_t *ctx; + ngx_ssl_conn_t *ssl_conn; + ngx_http_ssl_srv_conf_t *sscf; + STACK_OF(X509) *chain = ca_certs; + STACK_OF(X509_NAME) *name_chain = NULL; + X509 *x509 = NULL; + X509_NAME *subject = NULL; + X509_STORE *ca_store = NULL; +#ifdef OPENSSL_IS_BORINGSSL + size_t i; +#else + int i; +#endif + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + *err = "no request ctx found"; + return NGX_ERROR; + } + + if (!(ctx->context & NGX_HTTP_LUA_CONTEXT_SSL_CERT)) { + *err = "API disabled in the current context"; + return NGX_ERROR; + } + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + /* enable verify */ + + SSL_set_verify(ssl_conn, SSL_VERIFY_PEER, ngx_http_lua_ssl_verify_callback); + + /* set depth */ + + if (depth < 0) { + sscf = ngx_http_get_module_srv_conf(r, ngx_http_ssl_module); + if (sscf != NULL) { + depth = sscf->verify_depth; + + } else { + /* same as the default value of ssl_verify_depth */ + depth = 1; + } + } + + SSL_set_verify_depth(ssl_conn, depth); + + /* set CA chain */ + + if (chain != NULL) { + ca_store = X509_STORE_new(); + if (ca_store == NULL) { + *err = "X509_STORE_new() failed"; + return NGX_ERROR; + } + + /* construct name chain */ + + name_chain = sk_X509_NAME_new_null(); + if (name_chain == NULL) { + *err = "sk_X509_NAME_new_null() failed"; + goto failed; + } + + for (i = 0; i < sk_X509_num(chain); i++) { + x509 = sk_X509_value(chain, i); + if (x509 == NULL) { + *err = "sk_X509_value() failed"; + goto failed; + } + + /* add subject to name chain, which will be sent to client */ + subject = X509_NAME_dup(X509_get_subject_name(x509)); + if (subject == NULL) { + *err = "X509_get_subject_name() failed"; + goto failed; + } + + if (!sk_X509_NAME_push(name_chain, subject)) { + *err = "sk_X509_NAME_push() failed"; + X509_NAME_free(subject); + goto failed; + } + + /* add to trusted CA store */ + if (X509_STORE_add_cert(ca_store, x509) == 0) { + *err = "X509_STORE_add_cert() failed"; + goto failed; + } + } + + if (SSL_set0_verify_cert_store(ssl_conn, ca_store) == 0) { + *err = "SSL_set0_verify_cert_store() failed"; + goto failed; + } + + SSL_set_client_CA_list(ssl_conn, name_chain); + } + + return NGX_OK; + +failed: + + sk_X509_NAME_free(name_chain); + + X509_STORE_free(ca_store); + + return NGX_ERROR; +} + + #endif /* NGX_HTTP_SSL */ diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index a77f5d4c05..1c59135492 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -63,6 +63,10 @@ ffi.cdef[[ void ngx_http_lua_ffi_free_priv_key(void *cdata); int ngx_http_lua_ffi_ssl_clear_certs(void *r, char **err); + + int ngx_http_lua_ffi_ssl_verify_client(void *r, void *cdata, + int depth, char **err); + ]] _EOC_ } @@ -812,3 +816,213 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] + + + +=== TEST 6: verify client with CA certificates +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + require "defines" + local ffi = require "ffi" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + local f = assert(io.open("t/cert/test.crt", "rb")) + local cert_data = f:read("*all") + f:close() + + local cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not cert then + ngx.log(ngx.ERR, "failed to parse PEM cert: ", + ffi.string(errmsg[0])) + return + end + + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, cert, 1, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to verify client: ", + ffi.string(errmsg[0])) + return + end + + ffi.C.ngx_http_lua_ffi_free_cert(cert) + } + + ssl_certificate ../../cert/test2.crt; + ssl_certificate_key ../../cert/test2.key; + + location / { + default_type 'text/plain'; + content_by_lua_block { + print('client certificate subject: ', ngx.var.ssl_client_s_dn) + ngx.say(ngx.var.ssl_client_verify) + } + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_certificate ../../cert/test.crt; + proxy_ssl_certificate_key ../../cert/test.key; + proxy_ssl_session_reuse off; + } + +--- request +GET /t +--- response_body +SUCCESS + +--- error_log +client certificate subject: emailAddress=agentzh@gmail.com,CN=test.com + +--- no_error_log +[error] +[alert] + + + +=== TEST 7: verify client without CA certificates +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + require "defines" + local ffi = require "ffi" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, nil, -1, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to verify client: ", + ffi.string(errmsg[0])) + return + end + } + + ssl_certificate ../../cert/test2.crt; + ssl_certificate_key ../../cert/test2.key; + + location / { + default_type 'text/plain'; + content_by_lua_block { + print('client certificate subject: ', ngx.var.ssl_client_s_dn) + ngx.say(ngx.var.ssl_client_verify) + } + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_certificate ../../cert/test.crt; + proxy_ssl_certificate_key ../../cert/test.key; + proxy_ssl_session_reuse off; + } + +--- request +GET /t +--- response_body +FAILED:self signed certificate + +--- error_log +client certificate subject: emailAddress=agentzh@gmail.com,CN=test.com + +--- no_error_log +[error] +[alert] + + + +=== TEST 8: verify client but client provides no certificate +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + require "defines" + local ffi = require "ffi" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + local f = assert(io.open("t/cert/test.crt", "rb")) + local cert_data = f:read("*all") + f:close() + + local cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not cert then + ngx.log(ngx.ERR, "failed to parse PEM cert: ", + ffi.string(errmsg[0])) + return + end + + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, cert, 1, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to verify client: ", + ffi.string(errmsg[0])) + return + end + + ffi.C.ngx_http_lua_ffi_free_cert(cert) + } + + ssl_certificate ../../cert/test2.crt; + ssl_certificate_key ../../cert/test2.key; + + location / { + default_type 'text/plain'; + content_by_lua_block { + print('client certificate subject: ', ngx.var.ssl_client_s_dn) + ngx.say(ngx.var.ssl_client_verify) + } + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_session_reuse off; + } + +--- request +GET /t +--- response_body +NONE + +--- error_log +client certificate subject: nil + +--- no_error_log +[error] +[alert] From e526caeeb2bf963d261bef2889940153fdb06b8a Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Thu, 30 Jul 2020 19:47:15 +0800 Subject: [PATCH 439/848] feature: added the `ngx_http_lua_ffi_balancer_recreate_request` FFI function to allow recreation of request buffer in balancer phase. (#1734) --- src/ngx_http_lua_balancer.c | 39 +++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index f71a3e00ad..eff9dc13f6 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -753,4 +753,43 @@ ngx_http_lua_ffi_balancer_get_last_failure(ngx_http_request_t *r, } +int +ngx_http_lua_ffi_balancer_recreate_request(ngx_http_request_t *r, + char **err) +{ + ngx_http_lua_ctx_t *ctx; + ngx_http_upstream_t *u; + + if (r == NULL) { + *err = "no request found"; + return NGX_ERROR; + } + + u = r->upstream; + + if (u == NULL) { + *err = "no upstream found"; + return NGX_ERROR; + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + *err = "no ctx found"; + return NGX_ERROR; + } + + if ((ctx->context & NGX_HTTP_LUA_CONTEXT_BALANCER) == 0) { + *err = "API disabled in the current context"; + return NGX_ERROR; + } + + /* u->create_request can not be NULL since we are in balancer phase */ + ngx_http_lua_assert(u->create_request != NULL); + + *err = NULL; + + return u->create_request(r); +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From bdfc4bfc072796dfd20141af672007f5119d2d1e Mon Sep 17 00:00:00 2001 From: Soojin Nam Date: Fri, 31 Jul 2020 00:38:12 +0900 Subject: [PATCH 440/848] doc: removed the useless semicolon in the example lua code (#1757) Co-authored-by: bob-floyd --- README.markdown | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/README.markdown b/README.markdown index bd29909b53..3500800096 100644 --- a/README.markdown +++ b/README.markdown @@ -1402,14 +1402,14 @@ You can also initialize the [lua_shared_dict](#lua_shared_dict) shm storage at t lua_shared_dict dogs 1m; init_by_lua_block { - local dogs = ngx.shared.dogs; + local dogs = ngx.shared.dogs dogs:set("Tom", 56) } server { location = /api { content_by_lua_block { - local dogs = ngx.shared.dogs; + local dogs = ngx.shared.dogs ngx.say(dogs:get("Tom")) } } @@ -1659,8 +1659,8 @@ a time. However, a workaround is possible using the [ngx.var.VARIABLE](#ngxvarva local a = 32 local b = 56 - ngx.var.diff = a - b; -- write to $diff directly - return a + b; -- return the $sum value normally + ngx.var.diff = a - b -- write to $diff directly + return a + b -- return the $sum value normally '; echo "sum = $sum, diff = $diff"; @@ -1874,7 +1874,7 @@ The right way of doing this is as follows: rewrite_by_lua ' ngx.var.b = tonumber(ngx.var.a) + 1 if tonumber(ngx.var.b) == 13 then - return ngx.redirect("/bar"); + return ngx.redirect("/bar") end '; @@ -3480,7 +3480,7 @@ For example: location /foo { set $my_var ''; # this line is required to create $my_var at config time content_by_lua_block { - ngx.var.my_var = 123; + ngx.var.my_var = 123 ... } } @@ -3948,7 +3948,7 @@ This option is set to `false` by default set $dog 'hello'; content_by_lua_block { res = ngx.location.capture("/other", - { share_all_vars = true }); + { share_all_vars = true }) ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) @@ -3976,7 +3976,7 @@ The `copy_all_vars` option provides a copy of the parent request's Nginx variabl set $dog 'hello'; content_by_lua_block { res = ngx.location.capture("/other", - { copy_all_vars = true }); + { copy_all_vars = true }) ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) @@ -4014,7 +4014,7 @@ unescaping them in the Nginx config file. set $cat ''; content_by_lua_block { res = ngx.location.capture("/other", - { vars = { dog = "hello", cat = 32 }}); + { vars = { dog = "hello", cat = 32 }}) ngx.print(res.body) } @@ -4042,8 +4042,8 @@ The `ctx` option can be used to specify a custom Lua table to serve as the [ngx. local ctx = {} res = ngx.location.capture("/sub", { ctx = ctx }) - ngx.say(ctx.foo); - ngx.say(ngx.ctx.foo); + ngx.say(ctx.foo) + ngx.say(ngx.ctx.foo) } } ``` @@ -4061,13 +4061,13 @@ It is also possible to use this `ctx` option to share the same [ngx.ctx](#ngxctx location /sub { content_by_lua_block { - ngx.ctx.foo = "bar"; + ngx.ctx.foo = "bar" } } location /lua { content_by_lua_block { res = ngx.location.capture("/sub", { ctx = ngx.ctx }) - ngx.say(ngx.ctx.foo); + ngx.say(ngx.ctx.foo) } } ``` @@ -4245,14 +4245,14 @@ Setting a slot to `nil` effectively removes it from the response headers: ```lua - ngx.header["X-My-Header"] = nil; + ngx.header["X-My-Header"] = nil ``` The same applies to assigning an empty table: ```lua - ngx.header["X-My-Header"] = {}; + ngx.header["X-My-Header"] = {} ``` Setting `ngx.header.HEADER` after sending out response headers (either explicitly with [ngx.send_headers](#ngxsend_headers) or implicitly with [ngx.print](#ngxprint) and similar) will log an error message. @@ -5254,9 +5254,9 @@ Does an internal redirect to `uri` with `args` and is similar to the [echo_exec] ```lua - ngx.exec('/some-location'); - ngx.exec('/some-location', 'a=3&b=5&c=6'); - ngx.exec('/some-location?a=3&b=5', 'c=6'); + ngx.exec('/some-location') + ngx.exec('/some-location', 'a=3&b=5&c=6') + ngx.exec('/some-location?a=3&b=5', 'c=6') ``` The optional second `args` can be used to specify extra URI query arguments, for example: @@ -5285,7 +5285,7 @@ Named locations are also supported but the second `args` argument will be ignore location /foo { content_by_lua_block { - ngx.exec("@bar", "a=goodbye"); + ngx.exec("@bar", "a=goodbye") } } @@ -5373,7 +5373,7 @@ is equivalent to the following Lua code ```lua - return ngx.redirect('/foo'); -- Lua code + return ngx.redirect('/foo') -- Lua code ``` while @@ -8626,7 +8626,7 @@ For instance, ```lua - local res = ndk.set_var.set_escape_uri('a/b'); + local res = ndk.set_var.set_escape_uri('a/b') -- now res == 'a%2fb' ``` From f6b486b24e6f35e8eb918141918c110c5ab0b5f2 Mon Sep 17 00:00:00 2001 From: Jeffrey 'jf' Lim Date: Fri, 31 Jul 2020 11:01:44 +0800 Subject: [PATCH 441/848] doc: fixed the wrong phase for lua_need_request_body (#1758) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 3500800096..48c75e62e6 100644 --- a/README.markdown +++ b/README.markdown @@ -2535,7 +2535,7 @@ lua_need_request_body **phase:** *depends on usage* -Determines whether to force the request body data to be read before running rewrite/access/access_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code. +Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code. To read the request body data within the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable, [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) must have the same value as [client_max_body_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size). Because when the content length exceeds [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) but less than [client_max_body_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size), Nginx will buffer the data into a temporary file on the disk, which will lead to empty value in the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 04a2694972..6ff2330620 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2133,7 +2133,7 @@ This directive was first introduced in the v0.10.0 release. '''phase:''' ''depends on usage'' -Determines whether to force the request body data to be read before running rewrite/access/access_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned on or the [[#ngx.req.read_body|ngx.req.read_body]] function should be called within the Lua code. +Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned on or the [[#ngx.req.read_body|ngx.req.read_body]] function should be called within the Lua code. To read the request body data within the [[HttpCoreModule#$request_body|$request_body]] variable, [[HttpCoreModule#client_body_buffer_size|client_body_buffer_size]] must have the same value as [[HttpCoreModule#client_max_body_size|client_max_body_size]]. Because when the content length exceeds [[HttpCoreModule#client_body_buffer_size|client_body_buffer_size]] but less than [[HttpCoreModule#client_max_body_size|client_max_body_size]], Nginx will buffer the data into a temporary file on the disk, which will lead to empty value in the [[HttpCoreModule#$request_body|$request_body]] variable. From 83bfe91572d77bc8a882504fce57247cc1186d43 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 6 Aug 2020 15:14:44 -0700 Subject: [PATCH 442/848] doc: added a blog post link. --- README.markdown | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index 48c75e62e6..0240c042a1 100644 --- a/README.markdown +++ b/README.markdown @@ -1435,6 +1435,8 @@ You should be very careful about potential security vulnerabilities in your Lua This directive was first introduced in the `v0.5.5` release. +See also the blog post ["How OpenResty and Nginx Shared Memory Zones Consume RAM "](https://blog.openresty.com/en/how-nginx-shm-consume-ram/). + [Back to TOC](#directives) init_by_lua_block From 49937fb69aed6945263d1e1b3fe6fcc8e8277152 Mon Sep 17 00:00:00 2001 From: sjnam Date: Thu, 6 Aug 2020 10:08:46 +0900 Subject: [PATCH 443/848] doc: removed the useless semicolon in the example lua code --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 42 +++++++++++++++++++++--------------------- 2 files changed, 23 insertions(+), 23 deletions(-) diff --git a/README.markdown b/README.markdown index 0240c042a1..df079cab22 100644 --- a/README.markdown +++ b/README.markdown @@ -4207,9 +4207,9 @@ The header names are matched case-insensitively. ```lua -- equivalent to ngx.header["Content-Type"] = 'text/plain' - ngx.header.content_type = 'text/plain'; + ngx.header.content_type = 'text/plain' - ngx.header["X-My-Header"] = 'blah blah'; + ngx.header["X-My-Header"] = 'blah blah' ``` Multi-value headers can be set this way: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 6ff2330620..20bf7013c1 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1123,14 +1123,14 @@ You can also initialize the [[#lua_shared_dict|lua_shared_dict]] shm storage at lua_shared_dict dogs 1m; init_by_lua_block { - local dogs = ngx.shared.dogs; + local dogs = ngx.shared.dogs dogs:set("Tom", 56) } server { location = /api { content_by_lua_block { - local dogs = ngx.shared.dogs; + local dogs = ngx.shared.dogs ngx.say(dogs:get("Tom")) } } @@ -1351,8 +1351,8 @@ a time. However, a workaround is possible using the [[#ngx.var.VARIABLE|ngx.var. local a = 32 local b = 56 - ngx.var.diff = a - b; -- write to $diff directly - return a + b; -- return the $sum value normally + ngx.var.diff = a - b -- write to $diff directly + return a + b -- return the $sum value normally '; echo "sum = $sum, diff = $diff"; @@ -1541,7 +1541,7 @@ The right way of doing this is as follows: rewrite_by_lua ' ngx.var.b = tonumber(ngx.var.a) + 1 if tonumber(ngx.var.b) == 13 then - return ngx.redirect("/bar"); + return ngx.redirect("/bar") end '; @@ -2824,7 +2824,7 @@ For example: location /foo { set $my_var ''; # this line is required to create $my_var at config time content_by_lua_block { - ngx.var.my_var = 123; + ngx.var.my_var = 123 ... } } @@ -3247,7 +3247,7 @@ This option is set to false by default set $dog 'hello'; content_by_lua_block { res = ngx.location.capture("/other", - { share_all_vars = true }); + { share_all_vars = true }) ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) @@ -3311,7 +3311,7 @@ unescaping them in the Nginx config file. set $cat ''; content_by_lua_block { res = ngx.location.capture("/other", - { vars = { dog = "hello", cat = 32 }}); + { vars = { dog = "hello", cat = 32 }}) ngx.print(res.body) } @@ -3338,8 +3338,8 @@ The ctx option can be used to specify a custom Lua table to serve a local ctx = {} res = ngx.location.capture("/sub", { ctx = ctx }) - ngx.say(ctx.foo); - ngx.say(ngx.ctx.foo); + ngx.say(ctx.foo) + ngx.say(ngx.ctx.foo) } } @@ -3356,13 +3356,13 @@ It is also possible to use this ctx option to share the same [[#ngx location /sub { content_by_lua_block { - ngx.ctx.foo = "bar"; + ngx.ctx.foo = "bar" } } location /lua { content_by_lua_block { res = ngx.location.capture("/sub", { ctx = ngx.ctx }) - ngx.say(ngx.ctx.foo); + ngx.say(ngx.ctx.foo) } } @@ -3486,9 +3486,9 @@ The header names are matched case-insensitively. -- equivalent to ngx.header["Content-Type"] = 'text/plain' - ngx.header.content_type = 'text/plain'; + ngx.header.content_type = 'text/plain' - ngx.header["X-My-Header"] = 'blah blah'; + ngx.header["X-My-Header"] = 'blah blah' Multi-value headers can be set this way: @@ -3521,13 +3521,13 @@ is equivalent to Setting a slot to nil effectively removes it from the response headers: - ngx.header["X-My-Header"] = nil; + ngx.header["X-My-Header"] = nil The same applies to assigning an empty table: - ngx.header["X-My-Header"] = {}; + ngx.header["X-My-Header"] = {} Setting ngx.header.HEADER after sending out response headers (either explicitly with [[#ngx.send_headers|ngx.send_headers]] or implicitly with [[#ngx.print|ngx.print]] and similar) will log an error message. @@ -4403,8 +4403,8 @@ Does an internal redirect to uri with args and is simi ngx.exec('/some-location'); - ngx.exec('/some-location', 'a=3&b=5&c=6'); - ngx.exec('/some-location?a=3&b=5', 'c=6'); + ngx.exec('/some-location', 'a=3&b=5&c=6') + ngx.exec('/some-location?a=3&b=5', 'c=6') The optional second args can be used to specify extra URI query arguments, for example: @@ -4430,7 +4430,7 @@ Named locations are also supported but the second args argument wil location /foo { content_by_lua_block { - ngx.exec("@bar", "a=goodbye"); + ngx.exec("@bar", "a=goodbye") } } @@ -4509,7 +4509,7 @@ This method is similar to the [[HttpRewriteModule#rewrite|rewrite]] directive wi is equivalent to the following Lua code - return ngx.redirect('/foo'); -- Lua code + return ngx.redirect('/foo') -- Lua code while @@ -7377,7 +7377,7 @@ For example, the following [[HttpSetMiscModule]] directives can be invoked this For instance, - local res = ndk.set_var.set_escape_uri('a/b'); + local res = ndk.set_var.set_escape_uri('a/b') -- now res == 'a%2fb' From eeee4069e0e6d03ef135fa290020460868717d2b Mon Sep 17 00:00:00 2001 From: qltk Date: Tue, 11 Aug 2020 22:12:24 +0800 Subject: [PATCH 444/848] doc: ngx.var.VARIABLE is available in the balancer_by_lua* context (#1761) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- t/045-ngx-var.t | 52 +++++++++++++++++++++++++++++++++++++++++- 3 files changed, 53 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index df079cab22..f9ed3e0626 100644 --- a/README.markdown +++ b/README.markdown @@ -3464,7 +3464,7 @@ ngx.var.VARIABLE **syntax:** *ngx.var.VAR_NAME* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** Read and write Nginx variable values. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 20bf7013c1..ec98979fd4 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2808,7 +2808,7 @@ The data chunk and "eof" flag passed to the downstream Nginx output filters can '''syntax:''' ''ngx.var.VAR_NAME'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' Read and write Nginx variable values. diff --git a/t/045-ngx-var.t b/t/045-ngx-var.t index 6475f1e809..81fcef6458 100644 --- a/t/045-ngx-var.t +++ b/t/045-ngx-var.t @@ -7,7 +7,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 7); +plan tests => repeat_each() * (blocks() * 2 + 9); #no_diff(); #no_long_string(); @@ -228,3 +228,53 @@ GET /test?hello --- error_log variable "query_string" not changeable --- error_code: 500 + + + +=== TEST 12: get a variable in balancer_by_lua_block +--- http_config + upstream balancer { + server 127.0.0.1; + balancer_by_lua_block { + local balancer = require "ngx.balancer" + local host = "127.0.0.1" + local port = ngx.var.port; + local ok, err = balancer.set_current_peer(host, port) + if not ok then + ngx.log(ngx.ERR, "failed to set the current peer: ", err) + return ngx.exit(500) + end + } + } + server { + # this is the real entry point + listen 8091; + location / { + content_by_lua_block{ + ngx.print("this is backend peer 8091") + } + } + } + server { + # this is the real entry point + listen 8092; + location / { + content_by_lua_block{ + ngx.print("this is backend peer 8092") + } + } + } +--- config + location =/balancer { + set $port ''; + set_by_lua_block $port { + local args, _ = ngx.req.get_uri_args() + local port = args['port'] + return port + } + proxy_pass http://balancer; + } +--- pipelined_requests eval +["GET /balancer?port=8091", "GET /balancer?port=8092"] +--- response_body eval +["this is backend peer 8091", "this is backend peer 8092"] From 3e24a95a27f64b41901511ebd632f8654e69d8fc Mon Sep 17 00:00:00 2001 From: syz Date: Tue, 11 Aug 2020 22:16:10 +0800 Subject: [PATCH 445/848] bugfix: the body size may overflow since the size_t is only int32 in 32-bit system (#1756) --- src/ngx_http_lua_socket_tcp.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 964774041e..cc0ab2d6d9 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2396,7 +2396,7 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, ngx_connection_t *c; ngx_buf_t *b; ngx_event_t *rev; - size_t size; + off_t size; ssize_t n; unsigned read; off_t preread = 0; @@ -2503,7 +2503,7 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, } b = &u->buffer; - size = (size_t) (b->end - b->last); + size = b->end - b->last; } if (u->raw_downstream) { @@ -2511,8 +2511,8 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, if (preread) { - if ((off_t) size > preread) { - size = (size_t) preread; + if (size > preread) { + size = preread; } ngx_http_lua_probe_req_socket_consume_preread(r, @@ -2553,8 +2553,8 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, preread = r->request_body->rest; } - if ((off_t) size > preread) { - size = (size_t) preread; + if (size > preread) { + size = preread; } ngx_http_lua_probe_req_socket_consume_preread(r, @@ -2573,8 +2573,8 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, continue; } - if (size > (size_t) r->request_body->rest) { - size = (size_t) r->request_body->rest; + if (size > r->request_body->rest) { + size = r->request_body->rest; } } From b0875fa8f8244f85a273586ab636a0f4e8594338 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 11 Aug 2020 16:39:37 -0700 Subject: [PATCH 446/848] doc: added more blog post links. --- README.markdown | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index f9ed3e0626..9395d50cae 100644 --- a/README.markdown +++ b/README.markdown @@ -1022,6 +1022,8 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND See Also ======== +Other related modules and libraries: + * [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme) for an official port of this module for the Nginx "stream" subsystem (doing generic downstream TCP communications). * [lua-resty-memcached](https://github.com/openresty/lua-resty-memcached) library based on ngx_lua cosocket. * [lua-resty-redis](https://github.com/openresty/lua-resty-redis) library based on ngx_lua cosocket. @@ -1044,6 +1046,12 @@ See Also * [The OpenResty bundle](https://openresty.org) * [Nginx Systemtap Toolkit](https://github.com/openresty/nginx-systemtap-toolkit) +Blog posts: + +* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) + [Back to TOC](#table-of-contents) Directives @@ -1435,7 +1443,10 @@ You should be very careful about potential security vulnerabilities in your Lua This directive was first introduced in the `v0.5.5` release. -See also the blog post ["How OpenResty and Nginx Shared Memory Zones Consume RAM "](https://blog.openresty.com/en/how-nginx-shm-consume-ram/). +See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: + +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) [Back to TOC](#directives) From 21cfd8ab26b0aec0520f2d707770cb9e34680a5b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 11 Aug 2020 17:02:27 -0700 Subject: [PATCH 447/848] doc: ported recent changes in mardkown to the wiki file. --- doc/HttpLuaModule.wiki | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ec98979fd4..65558e9e6f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -849,6 +849,8 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND = See Also = +Other related modules and libraries: + * [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module] for an official port of this module for the Nginx "stream" subsystem (doing generic downstream TCP communications). * [https://github.com/openresty/lua-resty-memcached lua-resty-memcached] library based on ngx_lua cosocket. * [https://github.com/openresty/lua-resty-redis lua-resty-redis] library based on ngx_lua cosocket. @@ -871,6 +873,12 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND * [https://openresty.org The OpenResty bundle] * [https://github.com/openresty/nginx-systemtap-toolkit Nginx Systemtap Toolkit] +Blog posts: + +* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) + = Directives = @@ -1156,6 +1164,11 @@ You should be very careful about potential security vulnerabilities in your Lua This directive was first introduced in the v0.5.5 release. +See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: + +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) + == init_by_lua_block == '''syntax:''' ''init_by_lua_block { lua-script }'' @@ -3274,7 +3287,7 @@ The copy_all_vars option provides a copy of the parent request's Ng set $dog 'hello'; content_by_lua_block { res = ngx.location.capture("/other", - { copy_all_vars = true }); + { copy_all_vars = true }) ngx.print(res.body) ngx.say(ngx.var.uri, ": ", ngx.var.dog) @@ -4402,7 +4415,7 @@ This function was first introduced in the v0.5.0rc1 release. Does an internal redirect to uri with args and is similar to the [[HttpEchoModule#echo_exec|echo_exec]] directive of the [[HttpEchoModule]]. - ngx.exec('/some-location'); + ngx.exec('/some-location') ngx.exec('/some-location', 'a=3&b=5&c=6') ngx.exec('/some-location?a=3&b=5', 'c=6') From d6a29d2611094ff4421d42bf96662dc2fcd59dc9 Mon Sep 17 00:00:00 2001 From: rainingmaster <312841925@qq.com> Date: Fri, 24 Jul 2020 12:19:15 +0800 Subject: [PATCH 448/848] feature(socket.tcp): enhance the logic of parameter verification in connect --- src/ngx_http_lua_socket_tcp.c | 33 +++++++- t/023-rewrite/unix-socket.t | 2 +- t/058-tcp-socket.t | 46 +++++++++++- t/059-unix-socket.t | 138 +++++++++++++++++++++++++++++++++- t/068-socket-keepalive.t | 62 ++++++++++++++- 5 files changed, 272 insertions(+), 9 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index cc0ab2d6d9..d3690edcf1 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -970,8 +970,16 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) n--; } - if (n == 3) { - port = luaL_checkinteger(L, 3); + /* most popular suit: host:port */ + if (n == 3 && lua_isnumber(L, 3)) { + + /* Hit the following parameter combination: + * sock:connect("127.0.0.1", port) + * sock:connect("127.0.0.1", port, opts) + * sock:connect("unix:/path", port) + * sock:connect("unix:/path", port, opts) */ + + port = (int) lua_tointeger(L, 3); if (port < 0 || port > 65535) { lua_pushnil(L); @@ -987,8 +995,27 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) dd("socket key: %s", lua_tostring(L, -1)); - } else { /* n == 2 */ + } else if (len >= 5 && ngx_strncasecmp(p, (u_char *) "unix:", 5) == 0) { + + /* Hit the following parameter combination: + * sock:connect("unix:/path") + * sock:connect("unix:/path", nil) + * sock:connect("unix:/path", opts) + * sock:connect("unix:/path", nil, opts) */ + port = 0; + + } else { + + /* Ban the following parameter combination: + * sock:connect("127.0.0.1") + * sock:connect("127.0.0.1", nil) + * sock:connect("127.0.0.1", opts) + * sock:connect("127.0.0.1", nil, opts) */ + + lua_pushnil(L); + lua_pushfstring(L, "missing the port number"); + return 2; } if (!custom_pool) { diff --git a/t/023-rewrite/unix-socket.t b/t/023-rewrite/unix-socket.t index 8a5f00019b..91826fc3eb 100644 --- a/t/023-rewrite/unix-socket.t +++ b/t/023-rewrite/unix-socket.t @@ -79,7 +79,7 @@ qr{\[crit\] .*? connect\(\) to unix:/tmp/nosuchfile\.sock failed} --- request GET /test --- response_body -failed to connect: failed to parse host name "/tmp/test-nginx.sock": invalid host +failed to connect: missing the port number diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index cac464d830..593e49461f 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 228; +plan tests => repeat_each() * 231; our $HtmlDir = html_dir; @@ -3029,7 +3029,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):16: bad request/ --- user_files >>> myfoo.lua local sock = ngx.socket.tcp() -local ok, err = sock:connect("agentzh.org") +local ok, err = sock:connect("agentzh.org", 12345) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err) return @@ -3059,7 +3059,7 @@ runtime error: attempt to yield across C-call boundary end local function err() local sock = ngx.socket.tcp() - local ok, err = sock:connect("agentzh.org") + local ok, err = sock:connect("agentzh.org", 12345) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err) return @@ -4327,3 +4327,43 @@ failed to receive a line: closed [] close: 1 nil --- no_error_log [error] + + + +=== TEST 72: port is not number +--- config + server_tokens off; + location = /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(500) + + local ok, err = sock:connect("127.0.0.1") + if not ok then + ngx.say("connect failed: ", err) + end + + local ok, err = sock:connect("127.0.0.1", nil) + if not ok then + ngx.say("connect failed: ", err) + end + + local ok, err = sock:connect("127.0.0.1", {}) + if not ok then + ngx.say("connect failed: ", err) + end + + ngx.say("finish") + } + } + +--- request +GET /t +--- response_body +connect failed: missing the port number +connect failed: missing the port number +connect failed: missing the port number +finish +--- no_error_log +[error] diff --git a/t/059-unix-socket.t b/t/059-unix-socket.t index b06ba6e2ee..bd83006baf 100644 --- a/t/059-unix-socket.t +++ b/t/059-unix-socket.t @@ -64,7 +64,7 @@ qr{\[crit\] .*? connect\(\) to unix:/tmp/nosuchfile\.sock failed} --- request GET /test --- response_body -failed to connect: failed to parse host name "/tmp/test-nginx.sock": invalid host +failed to connect: missing the port number @@ -201,3 +201,139 @@ received: received: foo failed to receive a line: closed close: 1 nil + + + +=== TEST 5: port will be ignored +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + default_type 'text/plain'; + + server_tokens off; + location /foo { + content_by_lua 'ngx.say("foo")'; + more_clear_headers Date; + } + } +--- config + location /test { + content_by_lua ' + local sock = ngx.socket.tcp() + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock", 80) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n" + -- req = "OK" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent: ", bytes) + + while true do + print("calling receive") + local line, err = sock:receive() + if line then + ngx.say("received: ", line) + + else + ngx.say("failed to receive a line: ", err) + break + end + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + '; + } +--- request + GET /test +--- response_body +connected: 1 +request sent: 57 +received: HTTP/1.1 200 OK +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +failed to receive a line: closed +close: 1 nil + + + +=== TEST 6: second parameter is nil +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + default_type 'text/plain'; + + server_tokens off; + location /foo { + content_by_lua 'ngx.say("foo")'; + more_clear_headers Date; + } + } +--- config + location /test { + content_by_lua ' + local sock = ngx.socket.tcp() + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock", nil) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n" + -- req = "OK" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent: ", bytes) + + while true do + print("calling receive") + local line, err = sock:receive() + if line then + ngx.say("received: ", line) + + else + ngx.say("failed to receive a line: ", err) + break + end + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + '; + } +--- request + GET /test +--- response_body +connected: 1 +request sent: 57 +received: HTTP/1.1 200 OK +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +failed to receive a line: closed +close: 1 nil diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index e4c698887b..d67da3bd1e 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; #repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 31); +plan tests => repeat_each() * (blocks() * 4 + 34); our $HtmlDir = html_dir; @@ -2956,3 +2956,63 @@ GET /t lua tcp socket abort queueing --- response_body ok + + + +=== TEST 53: custom pools in third parameters for unix domain socket +--- http_config eval +" + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; + server { + listen unix:$::HtmlDir/nginx.sock; + default_type 'text/plain'; + + server_tokens off; + location /foo { + echo foo; + more_clear_headers Date; + } + } +" +--- config + location /t { + set $port $TEST_NGINX_MEMCACHED_PORT; + content_by_lua ' + local test = require "test" + local path = "$TEST_NGINX_HTML_DIR/nginx.sock"; + test.go(path, "A") + test.go(path, "B") + '; + } +--- user_files +>>> test.lua +module("test", package.seeall) + +function go(path, pool) + local sock = ngx.socket.tcp() + local ok, err = sock:connect("unix:" .. path, nil, {pool = pool}) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok, ", reused: ", sock:getreusedtimes()) + + local ok, err = sock:setkeepalive() + if not ok then + ngx.say("failed to set reusable: ", err) + end +end +--- request +GET /t +--- response_body +connected: 1, reused: 0 +connected: 1, reused: 0 +--- no_error_log eval +["[error]", +"lua tcp socket keepalive: free connection pool for ", +"lua tcp socket get keepalive peer: using connection" +] +--- error_log +lua tcp socket keepalive create connection pool for key "A" +lua tcp socket keepalive create connection pool for key "B" From 218af5ee90adeffc8d2cd6eb82d3dc908f9f3f6d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Fri, 21 Aug 2020 20:50:18 +0800 Subject: [PATCH 449/848] feature: shared ngx.ctx among SSL_* phases and the following phases. (#1514) --- README.markdown | 8 ++++ doc/HttpLuaModule.wiki | 8 ++++ src/ngx_http_lua_ctx.c | 62 ++++++++++++++++++++++---- src/ngx_http_lua_ssl.h | 4 ++ src/ngx_http_lua_ssl_certby.c | 2 + src/ngx_http_lua_ssl_session_fetchby.c | 2 + src/ngx_http_lua_ssl_session_storeby.c | 2 + src/ngx_http_lua_util.h | 1 + 8 files changed, 80 insertions(+), 9 deletions(-) diff --git a/README.markdown b/README.markdown index 9395d50cae..93e899f19e 100644 --- a/README.markdown +++ b/README.markdown @@ -3769,6 +3769,14 @@ Then `GET /orig` will give rather than the original `"hello"` value. +Because HTTP request is created after SSL handshake, the `ngx.ctx` created +in [ssl_certificate_by_lua*](#ssl_certificate_by_lua), [ssl_session_store_by_lua*](#ssl_session_store_by_lua) and [ssl_session_fetch_by_lua*](#ssl_session_fetch_by_lua) +is not available in the following phases like [rewrite_by_lua*](#rewrite_by_lua). + +Since `dev`, the `ngx.ctx` created during a SSL handshake +will be inherited by the requests which share the same TCP connection established by the handshake. +Note that overwrite values in `ngx.ctx` in the http request phases (like `rewrite_by_lua*`) will only take affect in the current http request. + Arbitrary data values, including Lua closures and nested tables, can be inserted into this "magic" table. It also allows the registration of custom meta methods. Overriding `ngx.ctx` with a new Lua table is also supported, for example, diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 65558e9e6f..15cee7112d 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3082,6 +3082,14 @@ Then GET /orig will give rather than the original "hello" value. +Because HTTP request is created after SSL handshake, the ngx.ctx created +in [[#ssl_certificate_by_lua|ssl_certificate_by_lua*]], [[#ssl_session_store_by_lua|ssl_session_store_by_lua*]] and [[#ssl_session_fetch_by_lua|ssl_session_fetch_by_lua*]] +is not available in the following phases like [[#rewrite_by_lua|rewrite_by_lua*]]. + +Since dev, the ngx.ctx created during a SSL handshake +will be inherited by the requests which share the same TCP connection established by the handshake. +Note that overwrite values in ngx.ctx in the http request phases (like `rewrite_by_lua*`) will only take affect in the current http request. + Arbitrary data values, including Lua closures and nested tables, can be inserted into this "magic" table. It also allows the registration of custom meta methods. Overriding ngx.ctx with a new Lua table is also supported, for example, diff --git a/src/ngx_http_lua_ctx.c b/src/ngx_http_lua_ctx.c index 74571378c0..1709aefa78 100644 --- a/src/ngx_http_lua_ctx.c +++ b/src/ngx_http_lua_ctx.c @@ -11,6 +11,7 @@ #include "ngx_http_lua_util.h" +#include "ngx_http_lua_ssl.h" #include "ngx_http_lua_ctx.h" @@ -21,7 +22,7 @@ typedef struct { static ngx_int_t ngx_http_lua_ngx_ctx_add_cleanup(ngx_http_request_t *r, - int ref); + ngx_pool_t *pool, int ref); static void ngx_http_lua_ngx_ctx_cleanup(void *data); @@ -29,6 +30,8 @@ int ngx_http_lua_ngx_set_ctx_helper(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, int index) { + ngx_pool_t *pool; + if (index < 0) { index = lua_gettop(L) + index + 1; } @@ -43,7 +46,8 @@ ngx_http_lua_ngx_set_ctx_helper(lua_State *L, ngx_http_request_t *r, ctx->ctx_ref = luaL_ref(L, -2); lua_pop(L, 1); - if (ngx_http_lua_ngx_ctx_add_cleanup(r, ctx->ctx_ref) != NGX_OK) { + pool = r->pool; + if (ngx_http_lua_ngx_ctx_add_cleanup(r, pool, ctx->ctx_ref) != NGX_OK) { return luaL_error(L, "no memory"); } @@ -66,32 +70,71 @@ ngx_http_lua_ngx_set_ctx_helper(lua_State *L, ngx_http_request_t *r, int -ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r) +ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r, int *in_ssl_phase, + int *ssl_ctx_ref) { - ngx_http_lua_ctx_t *ctx; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_ssl_ctx_t *ssl_ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { return NGX_HTTP_LUA_FFI_NO_REQ_CTX; } - return ctx->ctx_ref; + if (ctx->ctx_ref >= 0 || in_ssl_phase == NULL) { + return ctx->ctx_ref; + } + + *in_ssl_phase = ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE); + *ssl_ctx_ref = LUA_NOREF; + + if (r->connection->ssl != NULL) { + ssl_ctx = ngx_http_lua_ssl_get_ctx(r->connection->ssl->connection); + + if (ssl_ctx != NULL) { + *ssl_ctx_ref = ssl_ctx->ctx_ref; + } + } + + return LUA_NOREF; } int ngx_http_lua_ffi_set_ctx_ref(ngx_http_request_t *r, int ref) { - ngx_http_lua_ctx_t *ctx; + ngx_pool_t *pool; + ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_ssl_ctx_t *ssl_ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { return NGX_HTTP_LUA_FFI_NO_REQ_CTX; } + if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE)) + { + ssl_ctx = ngx_http_lua_ssl_get_ctx(r->connection->ssl->connection); + if (ssl_ctx == NULL) { + return NGX_ERROR; + } + + ssl_ctx->ctx_ref = ref; + c = ngx_ssl_get_connection(r->connection->ssl->connection); + pool = c->pool; + + } else { + pool = r->pool; + } + ctx->ctx_ref = ref; - if (ngx_http_lua_ngx_ctx_add_cleanup(r, ref) != NGX_OK) { + if (ngx_http_lua_ngx_ctx_add_cleanup(r, pool, ref) != NGX_OK) { return NGX_ERROR; } @@ -100,7 +143,8 @@ ngx_http_lua_ffi_set_ctx_ref(ngx_http_request_t *r, int ref) static ngx_int_t -ngx_http_lua_ngx_ctx_add_cleanup(ngx_http_request_t *r, int ref) +ngx_http_lua_ngx_ctx_add_cleanup(ngx_http_request_t *r, ngx_pool_t *pool, + int ref) { lua_State *L; ngx_pool_cleanup_t *cln; @@ -111,7 +155,7 @@ ngx_http_lua_ngx_ctx_add_cleanup(ngx_http_request_t *r, int ref) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); L = ngx_http_lua_get_lua_vm(r, ctx); - cln = ngx_pool_cleanup_add(r->pool, + cln = ngx_pool_cleanup_add(pool, sizeof(ngx_http_lua_ngx_ctx_cleanup_data_t)); if (cln == NULL) { return NGX_ERROR; diff --git a/src/ngx_http_lua_ssl.h b/src/ngx_http_lua_ssl.h index acb8c4b16c..f50ecc5194 100644 --- a/src/ngx_http_lua_ssl.h +++ b/src/ngx_http_lua_ssl.h @@ -27,6 +27,10 @@ typedef struct { int exit_code; /* exit code for openssl's set_cert_cb callback */ + int ctx_ref; /* reference to anchor + request ctx data in lua + registry */ + unsigned done:1; unsigned aborted:1; diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 2e09a03857..c6fc8832f5 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -257,6 +257,8 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) if (cctx == NULL) { goto failed; /* error */ } + + cctx->ctx_ref = LUA_NOREF; } cctx->exit_code = 1; /* successful by default */ diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index c54e8eadd8..3ac9272693 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -279,6 +279,8 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, if (cctx == NULL) { goto failed; /* error */ } + + cctx->ctx_ref = LUA_NOREF; } cctx->exit_code = 1; /* successful by default */ diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 1067b3bb2e..10803522a9 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -241,6 +241,8 @@ ngx_http_lua_ssl_sess_store_handler(ngx_ssl_conn_t *ssl_conn, if (cctx == NULL) { goto failed; /* error */ } + + cctx->ctx_ref = LUA_NOREF; } #if OPENSSL_VERSION_NUMBER >= 0x1000200fL diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 52d804bff0..fa24f4072b 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -15,6 +15,7 @@ #include "ngx_http_lua_common.h" +#include "ngx_http_lua_ssl.h" #include "ngx_http_lua_api.h" From e86d41f50a6cf7b71a6ff5d61e8ad96356e39c0b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Mon, 31 Aug 2020 09:12:42 +0800 Subject: [PATCH 450/848] bugfix: added missing SSL feature macro (#1776) --- src/ngx_http_lua_ctx.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ctx.c b/src/ngx_http_lua_ctx.c index 1709aefa78..dc84013493 100644 --- a/src/ngx_http_lua_ctx.c +++ b/src/ngx_http_lua_ctx.c @@ -74,7 +74,9 @@ ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r, int *in_ssl_phase, int *ssl_ctx_ref) { ngx_http_lua_ctx_t *ctx; +#if (NGX_HTTP_SSL) ngx_http_lua_ssl_ctx_t *ssl_ctx; +#endif ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { @@ -90,6 +92,7 @@ ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r, int *in_ssl_phase, | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE); *ssl_ctx_ref = LUA_NOREF; +#if (NGX_HTTP_SSL) if (r->connection->ssl != NULL) { ssl_ctx = ngx_http_lua_ssl_get_ctx(r->connection->ssl->connection); @@ -97,6 +100,7 @@ ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r, int *in_ssl_phase, *ssl_ctx_ref = ssl_ctx->ctx_ref; } } +#endif return LUA_NOREF; } @@ -106,15 +110,18 @@ int ngx_http_lua_ffi_set_ctx_ref(ngx_http_request_t *r, int ref) { ngx_pool_t *pool; - ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; +#if (NGX_HTTP_SSL) + ngx_connection_t *c; ngx_http_lua_ssl_ctx_t *ssl_ctx; +#endif ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { return NGX_HTTP_LUA_FFI_NO_REQ_CTX; } +#if (NGX_HTTP_SSL) if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE)) @@ -132,6 +139,10 @@ ngx_http_lua_ffi_set_ctx_ref(ngx_http_request_t *r, int ref) pool = r->pool; } +#else + pool = r->pool; +#endif + ctx->ctx_ref = ref; if (ngx_http_lua_ngx_ctx_add_cleanup(r, pool, ref) != NGX_OK) { From b4f464a909f1d9272f2a73b8b7558a6f7e490f55 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 1 Sep 2020 14:41:21 -0700 Subject: [PATCH 451/848] doc: added a link to new blog post "Introduction to Lua-Land CPU Flame Graphs". --- README.markdown | 13 +++++++------ doc/HttpLuaModule.wiki | 13 +++++++------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/README.markdown b/README.markdown index 93e899f19e..b42576d00f 100644 --- a/README.markdown +++ b/README.markdown @@ -1022,6 +1022,13 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND See Also ======== +Blog posts: + +* [Introduction to Lua-Land CPU Flame Graphs](https://blog.openresty.com/en/lua-cpu-flame-graph/) +* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) + Other related modules and libraries: * [ngx_stream_lua_module](https://github.com/openresty/stream-lua-nginx-module#readme) for an official port of this module for the Nginx "stream" subsystem (doing generic downstream TCP communications). @@ -1046,12 +1053,6 @@ Other related modules and libraries: * [The OpenResty bundle](https://openresty.org) * [Nginx Systemtap Toolkit](https://github.com/openresty/nginx-systemtap-toolkit) -Blog posts: - -* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) - [Back to TOC](#table-of-contents) Directives diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 15cee7112d..7f2ee763b8 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -849,6 +849,13 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND = See Also = +Blog posts: + +* [Introduction to Lua-Land CPU Flame Graphs](https://blog.openresty.com/en/lua-cpu-flame-graph/) +* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) + Other related modules and libraries: * [https://github.com/openresty/stream-lua-nginx-module#readme ngx_stream_lua_module] for an official port of this module for the Nginx "stream" subsystem (doing generic downstream TCP communications). @@ -873,12 +880,6 @@ Other related modules and libraries: * [https://openresty.org The OpenResty bundle] * [https://github.com/openresty/nginx-systemtap-toolkit Nginx Systemtap Toolkit] -Blog posts: - -* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) - = Directives = From 2db0e82469210344f4333dde4e21432b69d42bc3 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 1 Sep 2020 14:46:32 -0700 Subject: [PATCH 452/848] doc: minor link tweaks. --- doc/HttpLuaModule.wiki | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 7f2ee763b8..d1bca2c0b9 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -851,10 +851,10 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND Blog posts: -* [Introduction to Lua-Land CPU Flame Graphs](https://blog.openresty.com/en/lua-cpu-flame-graph/) -* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) +* [Introduction to Lua-Land CPU Flame Graphs](https://blog.openresty.com/en/lua-cpu-flame-graph/?src=gh_ngxlua) +* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem?src=gh_ngxlua) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/?src=gh_ngxlua) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/?src=gh_ngxlua) Other related modules and libraries: @@ -1167,8 +1167,8 @@ This directive was first introduced in the v0.5.5 release. See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/src=gh_ngxlua) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/src=gh_ngxlua) == init_by_lua_block == From b1a3b5d16abf329f1d236ecf38554f961c7b924f Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 1 Sep 2020 14:47:16 -0700 Subject: [PATCH 453/848] style: removed a line-trailing space found by ngx-releng. --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index d3690edcf1..9ff3f57571 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1010,7 +1010,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) /* Ban the following parameter combination: * sock:connect("127.0.0.1") * sock:connect("127.0.0.1", nil) - * sock:connect("127.0.0.1", opts) + * sock:connect("127.0.0.1", opts) * sock:connect("127.0.0.1", nil, opts) */ lua_pushnil(L); From 9416eadfaa8793c7ab0e89afe917824762c19aa5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 1 Sep 2020 14:47:32 -0700 Subject: [PATCH 454/848] doc: minor link tweaks. --- README.markdown | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index b42576d00f..942cca61d7 100644 --- a/README.markdown +++ b/README.markdown @@ -1024,10 +1024,10 @@ See Also Blog posts: -* [Introduction to Lua-Land CPU Flame Graphs](https://blog.openresty.com/en/lua-cpu-flame-graph/) -* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem) -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) +* [Introduction to Lua-Land CPU Flame Graphs](https://blog.openresty.com/en/lua-cpu-flame-graph/?src=gh_ngxlua) +* [How OpenResty and Nginx Allocate and Manage Memory](https://blog.openresty.com/en//how-or-alloc-mem?src=gh_ngxlua) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/?src=gh_ngxlua) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/?src=gh_ngxlua) Other related modules and libraries: @@ -1446,8 +1446,8 @@ This directive was first introduced in the `v0.5.5` release. See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/src=gh_ngxlua) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/src=gh_ngxlua) [Back to TOC](#directives) From 172c2145e170aecc6a5bc1e1e06456b38201d986 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 1 Sep 2020 15:08:20 -0700 Subject: [PATCH 455/848] doc: fixed broken links. --- README.markdown | 10 +++++----- doc/HttpLuaModule.wiki | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/README.markdown b/README.markdown index 942cca61d7..dd737310b1 100644 --- a/README.markdown +++ b/README.markdown @@ -328,9 +328,9 @@ Build the source with this module: ```bash - wget 'https://nginx.org/download/nginx-1.13.6.tar.gz' - tar -xzvf nginx-1.13.6.tar.gz - cd nginx-1.13.6/ + wget 'https://openresty.org/download/nginx-1.17.8.tar.gz' + tar -xzvf nginx-1.17.8.tar.gz + cd nginx-1.17.8/ # tell nginx's build system where to find LuaJIT 2.0: export LUAJIT_LIB=/path/to/luajit/lib @@ -1446,8 +1446,8 @@ This directive was first introduced in the `v0.5.5` release. See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/src=gh_ngxlua) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/src=gh_ngxlua) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/?src=gh_ngxlua) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/?src=gh_ngxlua) [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index d1bca2c0b9..3aa7b36b6f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -262,9 +262,9 @@ Alternatively, ngx_lua can be manually compiled into Nginx: Build the source with this module: - wget 'https://nginx.org/download/nginx-1.13.6.tar.gz' - tar -xzvf nginx-1.13.6.tar.gz - cd nginx-1.13.6/ + wget 'https://openresty.org/download/nginx-1.17.8.tar.gz' + tar -xzvf nginx-1.17.8.tar.gz + cd nginx-1.17.8/ # tell nginx's build system where to find LuaJIT 2.0: export LUAJIT_LIB=/path/to/luajit/lib @@ -1167,8 +1167,8 @@ This directive was first introduced in the v0.5.5 release. See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: -* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/src=gh_ngxlua) -* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/src=gh_ngxlua) +* [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/?src=gh_ngxlua) +* [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/?src=gh_ngxlua) == init_by_lua_block == From 97b91899c996a459c88d313a0151663369525627 Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Fri, 18 Sep 2020 18:26:28 +0800 Subject: [PATCH 456/848] bugfix: when recreating balancer buffer, remove existing buffers first (#1770) --- src/ngx_http_lua_balancer.c | 7 +++++++ t/138-balancer.t | 40 +++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index eff9dc13f6..e4ac57a301 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -788,6 +788,13 @@ ngx_http_lua_ffi_balancer_recreate_request(ngx_http_request_t *r, *err = NULL; + if (u->request_bufs != NULL && u->request_bufs != r->request_body->bufs) { + /* u->request_bufs already contains a valid request buffer + * remove it from chain first + */ + u->request_bufs = u->request_bufs->next; + } + return u->create_request(r); } diff --git a/t/138-balancer.t b/t/138-balancer.t index 151db7be44..238dea8fa3 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -524,3 +524,43 @@ http next upstream, 2 --- no_error_log failed to set more tries: reduced tries due to limit [alert] + + + +=== TEST 17: recreate_request buffer bugfix +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + server { + listen 127.0.0.1:8888; + + location / { + return 200 "it works"; + } + } + + upstream foo { + server 127.0.0.1:8888 max_fails=0; + server 127.0.0.1:8889 max_fails=0 weight=9999; + + balancer_by_lua_block { + local bal = require "ngx.balancer" + + assert(bal.recreate_request()) + } + } + +--- config + location = /t { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://foo; + } +--- request +GET /t +--- error_code: 200 +--- error_log +connect() failed (111: Connection refused) while connecting to upstream +--- no_error_log +upstream sent more data than specified in "Content-Length" header while reading upstream +[alert] From db1999816e0a71ed67506776818fdc13335a614f Mon Sep 17 00:00:00 2001 From: Alexander Drozdov Date: Mon, 28 Sep 2020 11:02:55 +0300 Subject: [PATCH 457/848] bugfix: the format in argerror messages are incorrect (#1787) lua_pushfstring() doesn't accept '%i' format. We should use '%d' instead. --- src/ngx_http_lua_socket_tcp.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 9ff3f57571..4df69ea0d2 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -901,7 +901,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) pool_size = (ngx_int_t) lua_tointeger(L, -1); if (pool_size <= 0) { - msg = lua_pushfstring(L, "bad \"pool_size\" option value: %i", + msg = lua_pushfstring(L, "bad \"pool_size\" option value: %d", pool_size); return luaL_argerror(L, n, msg); } @@ -920,7 +920,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) backlog = (ngx_int_t) lua_tointeger(L, -1); if (backlog < 0) { - msg = lua_pushfstring(L, "bad \"backlog\" option value: %i", + msg = lua_pushfstring(L, "bad \"backlog\" option value: %d", backlog); return luaL_argerror(L, n, msg); } @@ -5286,7 +5286,7 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) } if (pool_size <= 0) { - msg = lua_pushfstring(L, "bad \"pool_size\" option value: %i", + msg = lua_pushfstring(L, "bad \"pool_size\" option value: %d", pool_size); return luaL_argerror(L, n, msg); } From 4d2a04a08147c2d0ba2ec9632b649264b7d006a9 Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 28 Sep 2020 18:03:51 +0800 Subject: [PATCH 458/848] test: increased time unit to make test less error-prone --- t/108-timer-safe.t | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/108-timer-safe.t b/t/108-timer-safe.t index 758d01a6f2..387f203c3d 100644 --- a/t/108-timer-safe.t +++ b/t/108-timer-safe.t @@ -81,16 +81,16 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:4[4-9]|5[0-6]) local begin = ngx.now() local function f() print("my lua timer handler") - ngx.sleep(0.02) + ngx.sleep(0.2) print("elapsed: ", ngx.now() - begin) end - local ok, err = ngx.timer.at(0.05, f) + local ok, err = ngx.timer.at(0.5, f) if not ok then ngx.say("failed to set timer: ", err) return end ngx.say("registered timer") - ngx.sleep(0.05) + ngx.sleep(0.5) '; } --- request @@ -116,7 +116,7 @@ registered timer --- error_log eval [ qr/\[lua\] .*? my lua timer handler/, -qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-6])/, +qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:6[4-9]|7[0-6])/, "lua ngx.timer expired", "http lua close fake http connection" ] From 85fb9b83950dbfc35ec0b4e6b9950acd86255fc1 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 2 Oct 2020 16:14:00 +0800 Subject: [PATCH 459/848] test: make the 030-uri-args.t more robust since there is no fixed order in lua table. (#1797) --- t/030-uri-args.t | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/t/030-uri-args.t b/t/030-uri-args.t index f163f80d1b..4391b919f6 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -690,8 +690,8 @@ err: API disabled in the context of set_by_lua* } --- request GET /lua ---- response_body -a=bar&b=foo +--- response_body eval +qr/a=bar&b=foo|b=foo&a=bar/ @@ -1105,8 +1105,8 @@ CORE::join("", @k); } --- request GET /foo?world ---- response_body -HTTP/1.0 a=3&b=5&b=6 +--- response_body eval +qr/HTTP\/1.0 (a=3&b=5&b=6|b=5&b=6&a=3|b=6&b=5&a=3)/ @@ -1552,8 +1552,8 @@ GET /lua } --- request GET /lua ---- response_body -args: foo=%2C%24%40%7C%60&bar=-_.!~*'() +--- response_body eval +qr/(\Qargs: foo=%2C%24%40%7C%60&bar=-_.!~*'()\E)|(\Qargs: bar=-_.!~*'()&foo=%2C%24%40%7C%60\E)/ --- no_error_log [error] From ac9836f97b5c44eeb21606b7d6e8463839058a17 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 6 Oct 2020 14:15:41 +0800 Subject: [PATCH 460/848] bugfix: double values larger than int32_t were incorrectly printed out as 64-bit integers. --- src/ngx_http_lua_output.h | 15 +++++++---- t/164-say.t | 56 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_output.h b/src/ngx_http_lua_output.h index ec1db6d050..e60f4c7e30 100644 --- a/src/ngx_http_lua_output.h +++ b/src/ngx_http_lua_output.h @@ -30,8 +30,8 @@ ngx_http_lua_get_num_len(lua_State *L, int idx) double num; num = (double) lua_tonumber(L, idx); - if (num == (double) (long) num) { - return NGX_INT64_LEN; + if (num == (double) (int32_t) num) { + return NGX_INT32_LEN; } return NGX_DOUBLE_LEN; @@ -45,11 +45,16 @@ ngx_http_lua_write_num(lua_State *L, int idx, u_char *dst) int n; num = (double) lua_tonumber(L, idx); - if (num == (double) (long) num) { - dst = ngx_snprintf(dst, NGX_INT64_LEN, "%l", (long) num); + /* + * luajit format number with only 14 significant digits. + * To be consistent with lujit, don't use (double) (long) below + * or integer greater than 99,999,999,999,999 will different from luajit. + */ + if (num == (double) (int32_t) num) { + dst = ngx_snprintf(dst, NGX_INT64_LEN, "%D", (int32_t) num); } else { - /** + /* * The maximum number of significant digits is 14 in lua. * Please refer to lj_strfmt.c for more details. */ diff --git a/t/164-say.t b/t/164-say.t index 6114d2459e..40ec1ff1b4 100644 --- a/t/164-say.t +++ b/t/164-say.t @@ -53,3 +53,59 @@ GET /lua GET /lua --- response_body 123 3.1415926 + + + +=== TEST 4: ngx.say min int32 -2147483648 +--- config + location /lua { + content_by_lua_block { + ngx.say(-2147483648) + } + } +--- request +GET /lua +--- response_body +-2147483648 + + + +=== TEST 5: ngx.say big integer 2147483647 +--- config + location /lua { + content_by_lua_block { + ngx.say(2147483647) + } + } +--- request +GET /lua +--- response_body +2147483647 + + + +=== TEST 6: ngx.say big integer -9223372036854775808 +--- config + location /lua { + content_by_lua_block { + ngx.say(-9223372036854775808) + } + } +--- request +GET /lua +--- response_body +-9.2233720368548e+18 + + + +=== TEST 7: ngx.say big integer 18446744073709551615 +--- config + location /lua { + content_by_lua_block { + ngx.say(18446744073709551615) + } + } +--- request +GET /lua +--- response_body +1.844674407371e+19 From ef9e7507a57491067c8bf369328e0d3cf1049170 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 7 Oct 2020 01:46:51 -0700 Subject: [PATCH 461/848] updated .gitignore a bit. --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 11d444fe4c..5a8d398bb8 100644 --- a/.gitignore +++ b/.gitignore @@ -177,3 +177,4 @@ src/filters.c src/filters.h src/ringbuf.c src/ringbuf.h +src/pipe.[ch] From ce30a088649f203049bbb0f73af606bbdbc21ee0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 8 Oct 2020 00:24:15 -0700 Subject: [PATCH 462/848] feature: added new C API ngx_http_lua_pcre_version to wrap the pcre_version() C API in libpcre. --- config | 55 ---------------------------------------- src/ngx_http_lua_regex.c | 7 +++++ 2 files changed, 7 insertions(+), 55 deletions(-) diff --git a/config b/config index 6cb951bb42..4b32d38318 100644 --- a/config +++ b/config @@ -497,61 +497,6 @@ fi # ---------------------------------------- -if [ $USE_PCRE = YES -o $PCRE != NONE ] && [ $PCRE != NO -a $PCRE != YES ]; then - # force pcre_version symbol to be required when PCRE is statically linked - case "$NGX_PLATFORM" in - Darwin:*) - ngx_feature="require defined symbols (-u)" - ngx_feature_name= - ngx_feature_path= - ngx_feature_libs="-Wl,-u,_strerror" - ngx_feature_run=no - ngx_feature_incs="#include " - ngx_feature_test='printf("hello");' - - . auto/feature - - if [ $ngx_found = yes ]; then - CORE_LIBS="-Wl,-u,_pcre_version $CORE_LIBS" - fi - ;; - - *) - ngx_feature="require defined symbols (--require-defined)" - ngx_feature_name= - ngx_feature_path= - ngx_feature_libs="-Wl,--require-defined=strerror" - ngx_feature_run=no - ngx_feature_incs="#include " - ngx_feature_test='printf("hello");' - - . auto/feature - - if [ $ngx_found = yes ]; then - CORE_LIBS="-Wl,--require-defined=pcre_version $CORE_LIBS" - - else - # ld 2.25 and below (--required-defined was introduced in 2.26) - ngx_feature="require defined symbols fallback (--undefined)" - ngx_feature_name= - ngx_feature_path= - ngx_feature_libs="-Wl,--undefined=strerror" - ngx_feature_run=no - ngx_feature_incs="#include " - ngx_feature_test='printf("hello");' - - . auto/feature - - if [ $ngx_found = yes ]; then - CORE_LIBS="-Wl,--undefined=pcre_version $CORE_LIBS" - fi - fi - ;; - esac -fi - -# ---------------------------------------- - USE_MD5=YES USE_SHA1=YES diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 471339cc99..e4f36bc44e 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -582,6 +582,13 @@ ngx_http_lua_ffi_max_regex_cache_size(void) } +const char * +ngx_http_lua_ffi_pcre_version(void) +{ + return pcre_version(); +} + + #endif /* NGX_PCRE */ From 4695419bd3ed6451f97a0df12316b3d361032d34 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 8 Oct 2020 00:37:57 -0700 Subject: [PATCH 463/848] bumped the API version to 0.10.18. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index ad07bd863d..c532b95e7a 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10017 +#define ngx_http_lua_version 10018 typedef struct { From 4de59f5dac5d67938408a1d5d263d235f2d6c9e0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 7 Oct 2020 00:39:24 -0700 Subject: [PATCH 464/848] feature: now we recycle lua thread GC objects for our "light threads" and added lua_thread_cache_max_entries directive to configure the cache size. --- .travis.yml | 2 +- README.markdown | 22 +++++ doc/HttpLuaModule.wiki | 18 ++++ src/ngx_http_lua_common.h | 12 +++ src/ngx_http_lua_control.c | 14 +-- src/ngx_http_lua_coroutine.c | 19 +++- src/ngx_http_lua_coroutine.h | 2 +- src/ngx_http_lua_module.c | 57 ++++++++++- src/ngx_http_lua_timer.c | 97 ++++--------------- src/ngx_http_lua_uthread.c | 12 +-- src/ngx_http_lua_util.c | 177 +++++++++++++++++++---------------- src/ngx_http_lua_util.h | 105 +++++++++++++++++++++ t/002-content.t | 10 +- t/005-exit.t | 1 + t/106-timer.t | 14 ++- t/165-thread-cache.t | 79 ++++++++++++++++ 16 files changed, 451 insertions(+), 190 deletions(-) create mode 100644 t/165-thread-cache.t diff --git a/.travis.yml b/.travis.yml index bf433bef5e..2a7514986b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -88,7 +88,7 @@ install: - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 + - git clone -b reset-thread https://github.com/openresty/luajit2.git luajit2 before_script: - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;' diff --git a/README.markdown b/README.markdown index dd737310b1..71a79d1975 100644 --- a/README.markdown +++ b/README.markdown @@ -1063,6 +1063,7 @@ Directives * [lua_use_default_type](#lua_use_default_type) * [lua_malloc_trim](#lua_malloc_trim) * [lua_code_cache](#lua_code_cache) +* [lua_thread_cache_max_entries](#lua_thread_cache_max_entries) * [lua_regex_cache_max_entries](#lua_regex_cache_max_entries) * [lua_regex_match_limit](#lua_regex_match_limit) * [lua_package_path](#lua_package_path) @@ -1293,6 +1294,27 @@ development as it has a significant negative impact on overall performance. For [Back to TOC](#directives) +lua_thread_cache_max_entries +---------------------------- + +**syntax:** *lua_thread_cache_max_entries <num>* + +**default:** *lua_thread_cache_max_entries 1024* + +**context:** *http* + +Specifies the maximum number of entries allowed in the worker process level lua thread object cache. + +This cache recycles the lua thread GC objects among all our "light threads". + +A zero value of `` disables the cache. + +Note that this feature requires OpenResty's LuaJIT with the new C API `lua_resetthread`. + +This feature was first introduced in verson `v0.10.9`. + +[Back to TOC](#directives) + lua_regex_cache_max_entries --------------------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 3aa7b36b6f..29e4b30ac9 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1029,6 +1029,24 @@ Disabling the Lua code cache is strongly discouraged for production use and should only be used during development as it has a significant negative impact on overall performance. For example, the performance of a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache. +== lua_thread_cache_max_entries == + +'''syntax:''' ''lua_thread_cache_max_entries '' + +'''default:''' ''lua_thread_cache_max_entries 1024'' + +'''context:''' ''http'' + +Specifies the maximum number of entries allowed in the worker process level lua thread object cache. + +This cache recycles the lua thread GC objects among all our "light threads". + +A zero value of `` disables the cache. + +Note that this feature requires OpenResty's LuaJIT with the new C API `lua_resetthread`. + +This feature was first introduced in verson `v0.10.9`. + == lua_regex_cache_max_entries == '''syntax:''' ''lua_regex_cache_max_entries '' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 57c031f276..37894871b5 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -185,6 +185,13 @@ typedef struct { } ngx_http_lua_preload_hook_t; +typedef struct { + int ref; + lua_State *co; + ngx_queue_t queue; +} ngx_http_lua_thread_ref_t; + + struct ngx_http_lua_main_conf_s { lua_State *lua; ngx_pool_cleanup_t *vm_cleanup; @@ -203,6 +210,8 @@ struct ngx_http_lua_main_conf_s { ngx_connection_t *watcher; /* for watching the process exit event */ + ngx_int_t lua_thread_cache_max_entries; + #if (NGX_PCRE) ngx_int_t regex_cache_entries; ngx_int_t regex_cache_max_entries; @@ -276,6 +285,9 @@ struct ngx_http_lua_main_conf_s { ngx_flag_t set_sa_restart; + ngx_queue_t free_lua_threads; /* of ngx_http_lua_thread_ref_t */ + ngx_queue_t cached_lua_threads; /* of ngx_http_lua_thread_ref_t */ + unsigned requires_header_filter:1; unsigned requires_body_filter:1; unsigned requires_capture_filter:1; diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index ae5ca0c5d4..9b51c4af37 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -305,6 +305,7 @@ ngx_http_lua_ngx_redirect(lua_State *L) static int ngx_http_lua_on_abort(lua_State *L) { + int co_ref; ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx = NULL; @@ -335,18 +336,9 @@ ngx_http_lua_on_abort(lua_State *L) return 2; } - ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx); - - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_pushvalue(L, -2); - - dd("on_wait thread 1: %p", lua_tothread(L, -1)); - - coctx->co_ref = luaL_ref(L, -2); - lua_pop(L, 1); + ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx, &co_ref); + coctx->co_ref = co_ref; coctx->is_uthread = 1; ctx->on_abort_co_ctx = coctx; diff --git a/src/ngx_http_lua_coroutine.c b/src/ngx_http_lua_coroutine.c index 3a43b51142..e8ed061a3c 100644 --- a/src/ngx_http_lua_coroutine.c +++ b/src/ngx_http_lua_coroutine.c @@ -59,7 +59,7 @@ ngx_http_lua_coroutine_create(lua_State *L) return luaL_error(L, "no request ctx found"); } - return ngx_http_lua_coroutine_create_helper(L, r, ctx, NULL); + return ngx_http_lua_coroutine_create_helper(L, r, ctx, NULL, NULL); } @@ -92,7 +92,7 @@ ngx_http_lua_coroutine_wrap(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx); + ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx, NULL); coctx->is_wrap = 1; @@ -104,11 +104,12 @@ ngx_http_lua_coroutine_wrap(lua_State *L) int ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, - ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t **pcoctx) + ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t **pcoctx, int *co_ref) { lua_State *vm; /* the Lua VM */ lua_State *co; /* new coroutine to be created */ ngx_http_lua_co_ctx_t *coctx; /* co ctx for the new coroutine */ + ngx_http_lua_main_conf_t *lmcf; luaL_argcheck(L, lua_isfunction(L, 1) && !lua_iscfunction(L, 1), 1, "Lua function expected"); @@ -120,7 +121,13 @@ ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, /* create new coroutine on root Lua state, so it always yields * to main Lua thread */ - co = lua_newthread(vm); + if (co_ref == NULL) { + co = lua_newthread(vm); + + } else { + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + *co_ref = ngx_http_lua_new_cached_thread(vm, &co, lmcf, 0); + } ngx_http_lua_probe_user_coroutine_create(r, L, co); @@ -151,6 +158,10 @@ ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, lua_xmove(vm, L, 1); /* move coroutine from main thread to L */ + if (co_ref) { + lua_pop(vm, 1); /* pop coroutines */ + } + lua_pushvalue(L, 1); /* copy entry function to top of L*/ lua_xmove(L, co, 1); /* move entry function from L to co */ diff --git a/src/ngx_http_lua_coroutine.h b/src/ngx_http_lua_coroutine.h index 8b7bc9057a..03e32f3878 100644 --- a/src/ngx_http_lua_coroutine.h +++ b/src/ngx_http_lua_coroutine.h @@ -15,7 +15,7 @@ void ngx_http_lua_inject_coroutine_api(ngx_log_t *log, lua_State *L); int ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, - ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t **pcoctx); + ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t **pcoctx, int *co_ref); #endif /* _NGX_HTTP_LUA_COROUTINE_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 098fe87add..c7c5086e52 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -85,6 +85,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, NULL }, + { ngx_string("lua_thread_cache_max_entries"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, + ngx_conf_set_num_slot, + NGX_HTTP_MAIN_CONF_OFFSET, + offsetof(ngx_http_lua_main_conf_t, lua_thread_cache_max_entries), + NULL }, + { ngx_string("lua_max_running_timers"), NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, ngx_conf_set_num_slot, @@ -793,6 +800,13 @@ ngx_http_lua_init(ngx_conf_t *cf) "the OpenResty releases from https://openresty.org/" "en/download.html)"); } +#elif !defined(HAVE_LUA_RESETTHREAD) + ngx_log_error(NGX_LOG_ALERT, cf->log, 0, + "detected an old version of OpenResty's LuaJIT missing " + "the lua_resetthread API and thus the " + "performance will be compromised; please upgrade to the " + "latest version of OpenResty's LuaJIT: " + "https://github.com/openresty/luajit2"); #endif ngx_http_lua_content_length_hash = @@ -915,6 +929,7 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) lmcf->pool = cf->pool; lmcf->max_pending_timers = NGX_CONF_UNSET; lmcf->max_running_timers = NGX_CONF_UNSET; + lmcf->lua_thread_cache_max_entries = NGX_CONF_UNSET; #if (NGX_PCRE) lmcf->regex_cache_max_entries = NGX_CONF_UNSET; lmcf->regex_match_limit = NGX_CONF_UNSET; @@ -942,7 +957,27 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) static char * ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) { - ngx_http_lua_main_conf_t *lmcf = conf; +#ifdef HAVE_LUA_RESETTHREAD + ngx_int_t i, n; + ngx_http_lua_thread_ref_t *trefs; +#endif + + ngx_http_lua_main_conf_t *lmcf = conf; + + if (lmcf->lua_thread_cache_max_entries < 0) { + lmcf->lua_thread_cache_max_entries = 1024; + +#ifndef HAVE_LUA_RESETTHREAD + + } else if (lmcf->lua_thread_cache_max_entries > 0) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "lua_thread_cache_max_entries has no effect when " + "LuaJIT has no support for the lua_resetthread API " + "(you forgot to use OpenResty's LuaJIT?)"); + return NGX_CONF_ERROR; + +#endif + } #if (NGX_PCRE) if (lmcf->regex_cache_max_entries == NGX_CONF_UNSET) { @@ -976,6 +1011,26 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) lmcf->cycle = cf->cycle; + ngx_queue_init(&lmcf->free_lua_threads); + ngx_queue_init(&lmcf->cached_lua_threads); + +#ifdef HAVE_LUA_RESETTHREAD + n = lmcf->lua_thread_cache_max_entries; + + if (n > 0) { + trefs = ngx_palloc(cf->pool, n * sizeof(ngx_http_lua_thread_ref_t)); + if (trefs == NULL) { + return NGX_CONF_ERROR; + } + + for (i = 0; i < n; i++) { + trefs[i].ref = LUA_NOREF; + trefs[i].co = NULL; + ngx_queue_insert_head(&lmcf->free_lua_threads, &trefs[i].queue); + } + } +#endif + return NGX_CONF_OK; } diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 347187ce3c..a9e35b95f7 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -132,7 +132,8 @@ ngx_http_lua_ngx_timer_every(lua_State *L) static int ngx_http_lua_ngx_timer_helper(lua_State *L, int every) { - int nargs, co_ref; + int nargs; + int co_ref; u_char *p; lua_State *vm; /* the main thread */ lua_State *co; @@ -217,33 +218,22 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) vm = ngx_http_lua_get_lua_vm(r, ctx); - co = lua_newthread(vm); + co_ref = ngx_http_lua_new_cached_thread(vm, &co, lmcf, 1); + + /* vm stack: coroutines thread */ /* L stack: time func [args] */ ngx_http_lua_probe_user_coroutine_create(r, L, co); -#ifndef OPENRESTY_LUAJIT - lua_createtable(co, 0, 0); /* the new globals table */ - - /* co stack: global_tb */ - - lua_createtable(co, 0, 1); /* the metatable */ - ngx_http_lua_get_globals_table(co); - lua_setfield(co, -2, "__index"); - lua_setmetatable(co, -2); - - /* co stack: global_tb */ - - ngx_http_lua_set_globals_table(co); -#endif - /* co stack: */ dd("stack top: %d", lua_gettop(L)); lua_xmove(vm, L, 1); /* move coroutine from main thread to L */ + lua_pop(vm, 1); /* pop coroutines */ + /* L stack: time func [args] thread */ /* vm stack: empty */ @@ -263,19 +253,6 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) /* co stack: func */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - - /* L stack: time func [args] thread coroutines */ - - lua_pushvalue(L, -2); - - /* L stack: time func [args] thread coroutines thread */ - - co_ref = luaL_ref(L, -2); - lua_pop(L, 1); - /* L stack: time func [args] thread */ if (nargs > 2) { @@ -369,10 +346,7 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) ngx_free(ev); } - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - luaL_unref(L, -1, co_ref); + ngx_http_lua_free_thread(r, L, co_ref, co, lmcf); return luaL_error(L, "no memory"); } @@ -397,22 +371,7 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) vm = old_tctx->vm_state ? old_tctx->vm_state->vm : lmcf->lua; - co = lua_newthread(vm); - -#ifndef OPENRESTY_LUAJIT - lua_createtable(co, 0, 0); /* the new globals table */ - - /* co stack: global_tb */ - - lua_createtable(co, 0, 1); /* the metatable */ - ngx_http_lua_get_globals_table(co); - lua_setfield(co, -2, "__index"); - lua_setmetatable(co, -2); - - /* co stack: global_tb */ - - ngx_http_lua_set_globals_table(co); -#endif + co_ref = ngx_http_lua_new_cached_thread(vm, &co, lmcf, 1); /* co stack: */ @@ -420,6 +379,8 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) lua_xmove(vm, L, 1); /* move coroutine from main thread to L */ + lua_pop(vm, 1); /* pop coroutines */ + /* L stack: func [args] thread */ /* vm stack: empty */ @@ -439,18 +400,7 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) /* co stack: func */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - - /* L stack: func [args] thread coroutines */ - - lua_pushvalue(L, -2); - - /* L stack: func [args] thread coroutines thread */ - - co_ref = luaL_ref(L, -2); - lua_pop(L, 2); + lua_pop(L, 1); /* pop thread */ /* L stack: func [args] */ @@ -533,14 +483,7 @@ ngx_http_lua_timer_copy(ngx_http_lua_timer_ctx_t *old_tctx) /* L stack: func [args] */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - luaL_unref(L, -1, co_ref); - - /* L stack: func [args] coroutines */ - - lua_pop(L, 1); + ngx_http_lua_free_thread(NULL, L, co_ref, co, lmcf); return NGX_ERROR; } @@ -550,7 +493,7 @@ static void ngx_http_lua_timer_handler(ngx_event_t *ev) { int n; - lua_State *L; + lua_State *L = NULL; ngx_int_t rc; ngx_connection_t *c = NULL; ngx_http_request_t *r = NULL; @@ -726,17 +669,15 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) "lua failed to run timer with function defined at %s:%d: %s", source, ar.linedefined, errmsg); - lua_pushlightuserdata(tctx.co, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(tctx.co, LUA_REGISTRYINDEX); - luaL_unref(tctx.co, -1, tctx.co_ref); - lua_settop(tctx.co, 0); + if (L != NULL) { + ngx_http_lua_free_thread(r, L, tctx.co_ref, tctx.co, lmcf); + } - if (tctx.vm_state) { + if (tctx.vm_state != NULL) { ngx_http_lua_cleanup_vm(tctx.vm_state); } - if (c) { + if (c != NULL) { ngx_http_lua_close_fake_connection(c); } else if (tctx.pool) { diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index 68babc0871..d510dcf67b 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -49,7 +49,7 @@ ngx_http_lua_inject_uthread_api(ngx_log_t *log, lua_State *L) static int ngx_http_lua_uthread_spawn(lua_State *L) { - int n; + int n, co_ref; ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx = NULL; @@ -66,22 +66,16 @@ ngx_http_lua_uthread_spawn(lua_State *L) return luaL_error(L, "no request ctx found"); } - ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx); + ngx_http_lua_coroutine_create_helper(L, r, ctx, &coctx, &co_ref); /* anchor the newly created coroutine into the Lua registry */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_pushvalue(L, -2); - coctx->co_ref = luaL_ref(L, -2); - lua_pop(L, 1); - if (n > 1) { lua_replace(L, 1); lua_xmove(L, coctx->co, n - 1); } + coctx->co_ref = co_ref; coctx->is_uthread = 1; ctx->uthreads++; diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 4b4f2c7c6f..6330c724e5 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -331,43 +331,71 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) { int base; lua_State *co; + ngx_queue_t *q; + + ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_thread_ref_t *tref; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua creating new thread"); - base = lua_gettop(L); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); + if (L == lmcf->lua && !ngx_queue_empty(&lmcf->cached_lua_threads)) { + q = ngx_queue_head(&lmcf->cached_lua_threads); + tref = ngx_queue_data(q, ngx_http_lua_thread_ref_t, queue); + + ngx_http_lua_assert(tref->ref != LUA_NOREF); + ngx_http_lua_assert(tref->co != NULL); + + co = tref->co; + *ref = tref->ref; + + tref->co = NULL; + tref->ref = LUA_NOREF; + + ngx_queue_remove(q); + ngx_queue_insert_head(&lmcf->free_lua_threads, q); - co = lua_newthread(L); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua reusing cached lua thread %p (ref %d)", co, *ref); + + } else { + base = lua_gettop(L); + + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + + co = lua_newthread(L); #ifndef OPENRESTY_LUAJIT - /* {{{ inherit coroutine's globals to main thread's globals table - * for print() function will try to find tostring() in current - * globals table. - */ - /* new globals table for coroutine */ - ngx_http_lua_create_new_globals_table(co, 0, 0); - - lua_createtable(co, 0, 1); - ngx_http_lua_get_globals_table(co); - lua_setfield(co, -2, "__index"); - lua_setmetatable(co, -2); - - ngx_http_lua_set_globals_table(co); - /* }}} */ + /* {{{ inherit coroutine's globals to main thread's globals table + * for print() function will try to find tostring() in current + * globals table. + */ + /* new globals table for coroutine */ + ngx_http_lua_create_new_globals_table(co, 0, 0); + + lua_createtable(co, 0, 1); + ngx_http_lua_get_globals_table(co); + lua_setfield(co, -2, "__index"); + lua_setmetatable(co, -2); + + ngx_http_lua_set_globals_table(co); + /* }}} */ #endif /* OPENRESTY_LUAJIT */ - *ref = luaL_ref(L, -2); + *ref = luaL_ref(L, -2); - if (*ref == LUA_NOREF) { - lua_settop(L, base); /* restore main thread stack */ - return NULL; + if (*ref == LUA_NOREF) { + lua_settop(L, base); /* restore main thread stack */ + return NULL; + } + + lua_settop(L, base); } - lua_settop(L, base); return co; } @@ -376,6 +404,12 @@ void ngx_http_lua_del_thread(ngx_http_request_t *r, lua_State *L, ngx_http_lua_ctx_t *ctx, ngx_http_lua_co_ctx_t *coctx) { +#ifdef HAVE_LUA_RESETTHREAD + ngx_queue_t *q; + ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_thread_ref_t *tref; +#endif + if (coctx->co_ref == LUA_NOREF) { return; } @@ -383,17 +417,42 @@ ngx_http_lua_del_thread(ngx_http_request_t *r, lua_State *L, ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua deleting light thread"); - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); - luaL_unref(L, -1, coctx->co_ref); +#ifdef HAVE_LUA_RESETTHREAD + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + + if (L == lmcf->lua && !ngx_queue_empty(&lmcf->free_lua_threads)) { + lua_resetthread(L, coctx->co); + q = ngx_queue_head(&lmcf->free_lua_threads); + tref = ngx_queue_data(q, ngx_http_lua_thread_ref_t, queue); + + ngx_http_lua_assert(tref->ref == LUA_NOREF); + ngx_http_lua_assert(tref->co == NULL); + + tref->ref = coctx->co_ref; + tref->co = coctx->co; + + ngx_queue_remove(q); + ngx_queue_insert_head(&lmcf->cached_lua_threads, q); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua caching unused lua thread %p (ref %d)", coctx->co, + coctx->co_ref); + + } else { +#endif + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + luaL_unref(L, -1, coctx->co_ref); + lua_pop(L, 1); +#ifdef HAVE_LUA_RESETTHREAD + } +#endif + coctx->co_ref = LUA_NOREF; coctx->co_status = NGX_HTTP_LUA_CO_DEAD; - - lua_pop(L, 1); } @@ -3253,18 +3312,11 @@ static void ngx_http_lua_finalize_threads(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, lua_State *L) { -#ifdef NGX_LUA_USE_ASSERT - int top; -#endif - int inited = 0, ref; + int ref; ngx_uint_t i; ngx_list_part_t *part; ngx_http_lua_co_ctx_t *cc, *coctx; -#ifdef NGX_LUA_USE_ASSERT - top = lua_gettop(L); -#endif - #if 1 coctx = ctx->on_abort_co_ctx; if (coctx && coctx->co_ref != LUA_NOREF) { @@ -3275,17 +3327,7 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, ctx->uthreads--; } - ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); - - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - inited = 1; - - luaL_unref(L, -1, coctx->co_ref); - coctx->co_ref = LUA_NOREF; - - coctx->co_status = NGX_HTTP_LUA_CO_DEAD; + ngx_http_lua_del_thread(r, L, ctx, coctx); ctx->on_abort_co_ctx = NULL; } #endif @@ -3313,21 +3355,8 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, if (ref != LUA_NOREF) { ngx_http_lua_cleanup_pending_operation(coctx); - ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); - - if (!inited) { - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - inited = 1; - } - - ngx_http_lua_assert(lua_gettop(L) - top == 1); - - luaL_unref(L, -1, ref); - coctx->co_ref = LUA_NOREF; + ngx_http_lua_del_thread(r, L, ctx, coctx); - coctx->co_status = NGX_HTTP_LUA_CO_DEAD; ctx->uthreads--; } } @@ -3342,25 +3371,7 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, ref = coctx->co_ref; if (ref != LUA_NOREF) { ngx_http_lua_cleanup_pending_operation(coctx); - - ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); - - if (!inited) { - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - coroutines_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - inited = 1; - } - - ngx_http_lua_assert(lua_gettop(L) - top == 1); - - luaL_unref(L, -1, coctx->co_ref); - coctx->co_ref = LUA_NOREF; - coctx->co_status = NGX_HTTP_LUA_CO_DEAD; - } - - if (inited) { - lua_pop(L, 1); + ngx_http_lua_del_thread(r, L, ctx, coctx); } } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index fa24f4072b..b92665e693 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -544,6 +544,111 @@ ngx_http_lua_check_unsafe_uri_bytes(ngx_http_request_t *r, u_char *str, } +static ngx_inline void +ngx_http_lua_free_thread(ngx_http_request_t *r, lua_State *L, int co_ref, + lua_State *co, ngx_http_lua_main_conf_t *lmcf) +{ +#ifdef HAVE_LUA_RESETTHREAD + ngx_queue_t *q; + ngx_http_lua_thread_ref_t *tref ; + + if (L == lmcf->lua && !ngx_queue_empty(&lmcf->free_lua_threads)) { + lua_resetthread(L, co); + + q = ngx_queue_head(&lmcf->free_lua_threads); + tref = ngx_queue_data(q, ngx_http_lua_thread_ref_t, queue); + + ngx_http_lua_assert(tref->ref == LUA_NOREF); + ngx_http_lua_assert(tref->co == NULL); + + tref->ref = co_ref; + tref->co = co; + + ngx_queue_remove(q); + ngx_queue_insert_head(&lmcf->cached_lua_threads, q); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, + r != NULL ? r->connection->log : ngx_cycle->log, 0, + "lua caching unused lua thread %p (ref %d)", co, + co_ref); + + return; + } +#endif + + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + luaL_unref(L, -1, co_ref); + lua_pop(L, 1); +} + + +static ngx_inline int +ngx_http_lua_new_cached_thread(lua_State *L, lua_State **out_co, + ngx_http_lua_main_conf_t *lmcf, int set_globals) +{ + int co_ref; + lua_State *co; + ngx_queue_t *q; + ngx_http_lua_thread_ref_t *tref; + + if (L == lmcf->lua && !ngx_queue_empty(&lmcf->cached_lua_threads)) { + q = ngx_queue_head(&lmcf->cached_lua_threads); + tref = ngx_queue_data(q, ngx_http_lua_thread_ref_t, queue); + + ngx_http_lua_assert(tref->ref != LUA_NOREF); + ngx_http_lua_assert(tref->co != NULL); + + co = tref->co; + co_ref = tref->ref; + + tref->co = NULL; + tref->ref = LUA_NOREF; + + ngx_queue_remove(q); + ngx_queue_insert_head(&lmcf->free_lua_threads, q); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua reusing cached lua thread %p (ref %d)", co, co_ref); + + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + lua_rawgeti(L, -1, co_ref); + + } else { + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + co = lua_newthread(L); + lua_pushvalue(L, -1); + co_ref = luaL_ref(L, -3); + +#ifndef OPENRESTY_LUAJIT + if (set_globals) { + lua_createtable(co, 0, 0); /* the new globals table */ + + /* co stack: global_tb */ + + lua_createtable(co, 0, 1); /* the metatable */ + ngx_http_lua_get_globals_table(co); + lua_setfield(co, -2, "__index"); + lua_setmetatable(co, -2); + + /* co stack: global_tb */ + + ngx_http_lua_set_globals_table(co); + } +#endif + } + + *out_co = co; + + return co_ref; +} + + extern ngx_uint_t ngx_http_lua_location_hash; extern ngx_uint_t ngx_http_lua_content_length_hash; diff --git a/t/002-content.t b/t/002-content.t index bb569f51e8..d6a3600444 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -10,7 +10,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 23); +plan tests => repeat_each() * (blocks() * 2 + 24); #no_diff(); #no_long_string(); @@ -36,6 +36,14 @@ GET /lua Hello, Lua! --- no_error_log [error] +--- grep_error_log eval: qr/lua caching unused lua thread|lua reusing cached lua thread/ +--- grep_error_log_out eval +[ + "lua caching unused lua thread\n", + "lua reusing cached lua thread +lua caching unused lua thread +", +] diff --git a/t/005-exit.t b/t/005-exit.t index a416a753bc..ef4e0f3031 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -3,6 +3,7 @@ use Test::Nginx::Socket::Lua; #repeat_each(20000); +#repeat_each(200); repeat_each(2); #master_on(); #workers(1); diff --git a/t/106-timer.t b/t/106-timer.t index 485886495f..cc4bddd67b 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -12,7 +12,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 8 + 63); +plan tests => repeat_each() * (blocks() * 8 + 64); #no_diff(); no_long_string(); @@ -73,6 +73,18 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:4[4-9]|5[0-6]) "http lua close fake http connection", "timer prematurely expired: false", ] +--- grep_error_log eval: qr/lua caching unused lua thread|lua reusing cached lua thread/ +--- grep_error_log_out eval +[ + "lua caching unused lua thread +lua caching unused lua thread +", + "lua reusing cached lua thread +lua reusing cached lua thread +lua caching unused lua thread +lua caching unused lua thread +", +] diff --git a/t/165-thread-cache.t b/t/165-thread-cache.t new file mode 100644 index 0000000000..0adc6c6250 --- /dev/null +++ b/t/165-thread-cache.t @@ -0,0 +1,79 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_on(); +#workers(2); +#log_level('warn'); + +repeat_each(2); +#repeat_each(1); + +plan tests => repeat_each() * (blocks() * 4); + +#no_diff(); +#no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: thread cache size == 1 +--- http_config + lua_thread_cache_max_entries 1; + +--- config + location /lua { + # NOTE: the newline escape sequence must be double-escaped, as nginx config + # parser will unescape first! + content_by_lua ' + local ok, err = ngx.print("Hello, Lua!\\n") + if not ok then + ngx.log(ngx.ERR, "print failed: ", err) + end + '; + } +--- request +GET /lua +--- response_body +Hello, Lua! +--- no_error_log +[error] +--- grep_error_log eval: qr/lua caching unused lua thread|lua reusing cached lua thread/ +--- grep_error_log_out eval +[ + "lua caching unused lua thread\n", + "lua reusing cached lua thread +lua caching unused lua thread +", +] + + + +=== TEST 2: thread cache size == 0 +--- http_config + lua_thread_cache_max_entries 0; + +--- config + location /lua { + # NOTE: the newline escape sequence must be double-escaped, as nginx config + # parser will unescape first! + content_by_lua ' + local ok, err = ngx.print("Hello, Lua!\\n") + if not ok then + ngx.log(ngx.ERR, "print failed: ", err) + end + '; + } +--- request +GET /lua +--- response_body +Hello, Lua! +--- no_error_log +[error] +--- grep_error_log eval: qr/lua caching unused lua thread|lua reusing cached lua thread/ +--- grep_error_log_out eval +[ + "", + "", +] From 021b75b88996d60b5899f4f0ae9677af872843f1 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 8 Oct 2020 13:59:08 -0700 Subject: [PATCH 465/848] travis: fixed the luajit branch. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 2a7514986b..bf433bef5e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -88,7 +88,7 @@ install: - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - - git clone -b reset-thread https://github.com/openresty/luajit2.git luajit2 + - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 before_script: - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;' From b28112130306370a1902e4f43a6c1dff90bacbed Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 7 Oct 2020 22:52:57 -0700 Subject: [PATCH 466/848] optimize: now we make use of the new lua_getexdata2 and lua_setexdata2 to attach coctx to the Lua thread objects directly which save the lookup overhead which can become the bottleneck when there are many Lua light threads around in a single request handler. --- src/ngx_http_lua_accessby.c | 2 ++ src/ngx_http_lua_contentby.c | 2 ++ src/ngx_http_lua_coroutine.c | 1 + src/ngx_http_lua_module.c | 12 +++++++++++- src/ngx_http_lua_rewriteby.c | 2 ++ src/ngx_http_lua_ssl_certby.c | 2 ++ src/ngx_http_lua_ssl_session_fetchby.c | 2 ++ src/ngx_http_lua_timer.c | 2 ++ src/ngx_http_lua_util.c | 4 ++++ src/ngx_http_lua_util.h | 9 +++++++++ 10 files changed, 37 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index fabe59a3ac..58c251443e 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -287,6 +287,8 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) ctx->cur_co_ctx->co_top = 1; #endif + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + /* }}} */ /* {{{ register request cleanup hooks */ diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 56a6192282..76e6a074da 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -79,6 +79,8 @@ ngx_http_lua_content_by_chunk(lua_State *L, ngx_http_request_t *r) ctx->cur_co_ctx->co_top = 1; #endif + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + /* {{{ register request cleanup hooks */ if (ctx->cleanup == NULL) { cln = ngx_http_cleanup_add(r, 0); diff --git a/src/ngx_http_lua_coroutine.c b/src/ngx_http_lua_coroutine.c index e8ed061a3c..65eeef2e8b 100644 --- a/src/ngx_http_lua_coroutine.c +++ b/src/ngx_http_lua_coroutine.c @@ -148,6 +148,7 @@ ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, #ifdef OPENRESTY_LUAJIT ngx_http_lua_set_req(co, r); + ngx_http_lua_attach_co_ctx_to_L(co, coctx); #else /* make new coroutine share globals of the parent coroutine. * NOTE: globals don't have to be separated! */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index c7c5086e52..3747f02630 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -800,13 +800,23 @@ ngx_http_lua_init(ngx_conf_t *cf) "the OpenResty releases from https://openresty.org/" "en/download.html)"); } -#elif !defined(HAVE_LUA_RESETTHREAD) +#else +# if !defined(HAVE_LUA_RESETTHREAD) ngx_log_error(NGX_LOG_ALERT, cf->log, 0, "detected an old version of OpenResty's LuaJIT missing " "the lua_resetthread API and thus the " "performance will be compromised; please upgrade to the " "latest version of OpenResty's LuaJIT: " "https://github.com/openresty/luajit2"); +# endif +# if !defined(HAVE_LUA_EXDATA2) + ngx_log_error(NGX_LOG_ALERT, cf->log, 0, + "detected an old version of OpenResty's LuaJIT missing " + "the exdata2 API and thus the " + "performance will be compromised; please upgrade to the " + "latest version of OpenResty's LuaJIT: " + "https://github.com/openresty/luajit2"); +# endif #endif ngx_http_lua_content_length_hash = diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 70a065f2c2..d1eabeccdd 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -287,6 +287,8 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) ctx->cur_co_ctx->co_top = 1; #endif + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + /* }}} */ /* {{{ register request cleanup hooks */ diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index c6fc8832f5..11f0d739a7 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -491,6 +491,8 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) ctx->cur_co_ctx->co_top = 1; #endif + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + /* register request cleanup hooks */ if (ctx->cleanup == NULL) { cln = ngx_http_cleanup_add(r, 0); diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 3ac9272693..e04658ed79 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -513,6 +513,8 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) ctx->cur_co_ctx->co_top = 1; #endif + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + /* register request cleanup hooks */ if (ctx->cleanup == NULL) { cln = ngx_http_cleanup_add(r, 0); diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index a9e35b95f7..95d1933356 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -621,6 +621,8 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) /* save the request in coroutine globals table */ ngx_http_lua_set_req(tctx.co, r); + ngx_http_lua_attach_co_ctx_to_L(tctx.co, ctx->cur_co_ctx); + lmcf->running_timers++; lua_pushboolean(tctx.co, tctx.premature); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 6330c724e5..95109ae9af 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -3168,6 +3168,9 @@ ngx_http_lua_param_set(lua_State *L) ngx_http_lua_co_ctx_t * ngx_http_lua_get_co_ctx(lua_State *L, ngx_http_lua_ctx_t *ctx) { +#ifdef HAVE_LUA_EXDATA2 + return (ngx_http_lua_co_ctx_t *) lua_getexdata2(L); +#else ngx_uint_t i; ngx_list_part_t *part; ngx_http_lua_co_ctx_t *coctx; @@ -3203,6 +3206,7 @@ ngx_http_lua_get_co_ctx(lua_State *L, ngx_http_lua_ctx_t *ctx) } return NULL; +#endif } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index b92665e693..a20c916035 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -398,6 +398,15 @@ ngx_http_lua_set_req(lua_State *L, ngx_http_request_t *r) } +static ngx_inline void +ngx_http_lua_attach_co_ctx_to_L(lua_State *L, ngx_http_lua_co_ctx_t *coctx) +{ +#ifdef HAVE_LUA_EXDATA2 + lua_setexdata2(L, (void *) coctx); +#endif +} + + #ifndef OPENRESTY_LUAJIT static ngx_inline void ngx_http_lua_get_globals_table(lua_State *L) From 2d82f078a17c5e3d9717774f5ab589b985d54b08 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 8 Oct 2020 14:34:52 -0700 Subject: [PATCH 467/848] bugfix: we now only recycle light threads which are "entry threads" otherwise it is not safe to recycle user created threads since we have no idea whether the user may still hold refs to them. --- src/ngx_http_lua_uthread.c | 2 ++ src/ngx_http_lua_util.c | 5 ++++- src/ngx_http_lua_util.h | 8 +++++++- 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index d510dcf67b..5ed534c7f1 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -91,6 +91,8 @@ ngx_http_lua_uthread_spawn(lua_State *L) coctx->parent_co_ctx = ctx->cur_co_ctx; ctx->cur_co_ctx = coctx; + ngx_http_lua_attach_co_ctx_to_L(coctx->co, coctx); + ngx_http_lua_probe_user_thread_spawn(r, L, coctx->co); dd("yielding with arg %s, top=%d, index-1:%s", luaL_typename(L, -1), diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 95109ae9af..da702a3210 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -422,7 +422,10 @@ ngx_http_lua_del_thread(ngx_http_request_t *r, lua_State *L, #ifdef HAVE_LUA_RESETTHREAD lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - if (L == lmcf->lua && !ngx_queue_empty(&lmcf->free_lua_threads)) { + if (ctx != NULL + && coctx->co == ctx->entry_co_ctx.co + && L == lmcf->lua && !ngx_queue_empty(&lmcf->free_lua_threads)) + { lua_resetthread(L, coctx->co); q = ngx_queue_head(&lmcf->free_lua_threads); tref = ngx_queue_data(q, ngx_http_lua_thread_ref_t, queue); diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index a20c916035..a1faafa600 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -560,8 +560,14 @@ ngx_http_lua_free_thread(ngx_http_request_t *r, lua_State *L, int co_ref, #ifdef HAVE_LUA_RESETTHREAD ngx_queue_t *q; ngx_http_lua_thread_ref_t *tref ; + ngx_http_lua_ctx_t *ctx; - if (L == lmcf->lua && !ngx_queue_empty(&lmcf->free_lua_threads)) { + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx != NULL + && L == ctx->entry_co_ctx.co + && L == lmcf->lua + && !ngx_queue_empty(&lmcf->free_lua_threads)) + { lua_resetthread(L, co); q = ngx_queue_head(&lmcf->free_lua_threads); From d8b408a7f12345026d277b92961d9363ea1f7d1e Mon Sep 17 00:00:00 2001 From: timebug Date: Mon, 28 Sep 2020 15:21:42 +0800 Subject: [PATCH 468/848] bugfix: build error with nginx 1.7.4 or older version --- src/ngx_http_lua_sleep.c | 11 ++++++++++- src/ngx_http_lua_socket_tcp.c | 11 ++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_sleep.c b/src/ngx_http_lua_sleep.c index b8601fee95..770ce9beac 100644 --- a/src/ngx_http_lua_sleep.c +++ b/src/ngx_http_lua_sleep.c @@ -159,11 +159,20 @@ ngx_http_lua_sleep_cleanup(void *data) } #ifdef HAVE_POSTED_DELAYED_EVENTS_PATCH +#if (nginx_version >= 1007005) if (coctx->sleep.posted) { +#else + if (coctx->sleep.prev) { +#endif ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "lua clean up the posted event for pending ngx.sleep"); - ngx_delete_posted_event(&coctx->sleep); + /* + * We need the extra parentheses around the argument + * of ngx_delete_posted_event() just to work around macro issues in + * nginx cores older than 1.7.5 (exclusive). + */ + ngx_delete_posted_event((&coctx->sleep)); } #endif } diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 4df69ea0d2..ae068a4dba 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -6024,8 +6024,17 @@ ngx_http_lua_tcp_queue_conn_op_cleanup(void *data) "lua tcp socket abort queueing, conn_op_ctx: %p, u: %p", conn_op_ctx, u); +#if (nginx_version >= 1007005) if (conn_op_ctx->event.posted) { - ngx_delete_posted_event(&conn_op_ctx->event); +#else + if (conn_op_ctx->event.prev) { +#endif + /* + * We need the extra parentheses around the argument + * of ngx_delete_posted_event() just to work around macro issues in + * nginx cores older than 1.7.5 (exclusive). + */ + ngx_delete_posted_event((&conn_op_ctx->event)); } else if (conn_op_ctx->event.timer_set) { ngx_del_timer(&conn_op_ctx->event); From 2c5b1893348d02aa8a2f0e4bbdc7c30798776401 Mon Sep 17 00:00:00 2001 From: Yan Zhu Date: Wed, 22 Jul 2020 12:50:40 +0800 Subject: [PATCH 469/848] feature: implemented the new ngx.ssl.server_port() API to get server port. --- src/ngx_http_lua_ssl_certby.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 11f0d739a7..6a2f1dc580 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -858,6 +858,39 @@ ngx_http_lua_ffi_ssl_server_name(ngx_http_request_t *r, char **name, } +int +ngx_http_lua_ffi_ssl_server_port(ngx_http_request_t *r, + unsigned short *server_port, char **err) +{ + ngx_ssl_conn_t *ssl_conn; + ngx_connection_t *c; + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + c = ngx_ssl_get_connection(ssl_conn); + + switch (c->local_sockaddr->sa_family) { + + case AF_UNIX: + *err = "unix domain has no port"; + return NGX_ERROR; + + default: + *server_port = (unsigned short) ngx_inet_get_port(c->local_sockaddr); + return NGX_OK; + } +} + + int ngx_http_lua_ffi_ssl_raw_client_addr(ngx_http_request_t *r, char **addr, size_t *addrlen, int *addrtype, char **err) From 0bccfb30fb4a69ec28d441632b93323e555beb33 Mon Sep 17 00:00:00 2001 From: hzzb Date: Fri, 9 Oct 2020 15:18:58 +0800 Subject: [PATCH 470/848] style: added "static" for the "ngx_http_lua_module_ctx" (#1794) --- src/ngx_http_lua_module.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 3747f02630..7358a95639 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -633,7 +633,7 @@ static ngx_command_t ngx_http_lua_cmds[] = { }; -ngx_http_module_t ngx_http_lua_module_ctx = { +static ngx_http_module_t ngx_http_lua_module_ctx = { NULL, /* preconfiguration */ ngx_http_lua_init, /* postconfiguration */ From df4033d13cd5e055a7c87bd985843787151087a6 Mon Sep 17 00:00:00 2001 From: Johnny Date: Fri, 9 Oct 2020 15:41:39 +0800 Subject: [PATCH 471/848] travis-ci: bumped the NGINX core to 1.19.2. (#1793) --- .travis.yml | 5 +++-- src/ngx_http_lua_ssl_certby.c | 2 ++ src/ngx_http_lua_ssl_session_fetchby.c | 2 ++ t/124-init-worker.t | 4 ++++ t/129-ssl-socket.t | 20 ++++++++++++----- t/139-ssl-cert-by.t | 11 ++++++--- t/143-ssl-session-fetch.t | 31 +++++++++++++++++++++++++- 7 files changed, 63 insertions(+), 12 deletions(-) diff --git a/.travis.yml b/.travis.yml index bf433bef5e..97eb1525b3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,8 +48,9 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.17.8 OPENSSL_VER=1.1.1g OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.19.3 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.1g OPENSSL_PATCH_VER=1.1.1f services: - memcached diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 6a2f1dc580..dfaa9f15a3 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -215,7 +215,9 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) dd("first time"); +#if (nginx_version < 1017009) ngx_reusable_connection(c, 0); +#endif hc = c->data; diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index e04658ed79..f69cabb198 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -237,7 +237,9 @@ ngx_http_lua_ssl_sess_fetch_handler(ngx_ssl_conn_t *ssl_conn, dd("first time"); +#if (nginx_version < 1017009) ngx_reusable_connection(c, 0); +#endif hc = c->data; diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 4340893878..94081ad2cf 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -795,6 +795,10 @@ lua close the global Lua VM \2 lua close the global Lua VM \3 in the cache helper process \d+ lua close the global Lua VM \3 lua close the global Lua VM \3 in the cache helper process \d+ +|lua close the global Lua VM ([0-9A-F]+) +lua close the global Lua VM \4 in the cache helper process \d+ +lua close the global Lua VM \4 in the cache helper process \d+ +lua close the global Lua VM \4 )(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- no_error_log diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index d7f9d6bc02..3b9b34c33b 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -772,7 +772,7 @@ $::DSTRootCertificate" GET /t --- response_body eval qr{connected: 1 -failed to do SSL handshake: (22: certificate chain too long|20: unable to get local issuer certificate) +failed to do SSL handshake: (22: certificate chain too long|20: unable to get local issuer certificate|21: unable to verify the first certificate) failed to send http request: closed } @@ -781,7 +781,7 @@ failed to send http request: closed --- grep_error_log_out --- error_log eval ['lua ssl server name: "openresty.org"', -qr/lua ssl certificate verify error: \((22: certificate chain too long|20: unable to get local issuer certificate)\)/] +qr/lua ssl certificate verify error: \((22: certificate chain too long|20: unable to get local issuer certificate|21: unable to verify the first certificate)\)/] --- no_error_log SSL reused session [alert] @@ -852,7 +852,7 @@ $::DSTRootCertificate" GET /t --- response_body eval qr/connected: 1 -failed to do SSL handshake: (22: certificate chain too long|20: unable to get local issuer certificate) +failed to do SSL handshake: (22: certificate chain too long|20: unable to get local issuer certificate|21: unable to verify the first certificate) failed to send http request: closed / @@ -1938,10 +1938,18 @@ SSL reused session --- request GET /t ---- response_body -connected: 1 +--- response_body eval +# Since nginx version 1.19.1, invalidity date is considerd a non-critical CRL +# entry extension, in other words, revoke still works even if CRL has expired. +$Test::Nginx::Util::NginxVersion >= 1.019001 ? + +"connected: 1 +failed to do SSL handshake: 23: certificate revoked +failed to send http request: closed\n" : + +"connected: 1 failed to do SSL handshake: 12: CRL has expired -failed to send http request: closed +failed to send http request: closed\n"; --- user_files eval ">>> test.key diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 17a4927cce..d7e20bafea 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -119,11 +119,16 @@ lua ssl server name: "test.com" [alert] --- grep_error_log eval: qr/ssl_certificate_by_lua:.*?,|\bssl cert: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval -qr/reusable connection: 1 +# Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) +# before call ssl callback function +$Test::Nginx::Util::NginxVersion >= 1.017009 ? +qr/reusable connection: 0 +ssl cert: connection reusable: 0 +ssl_certificate_by_lua:1: ssl cert by lua is running!,/ +: qr /reusable connection: 1 ssl cert: connection reusable: 1 reusable connection: 0 -ssl_certificate_by_lua:1: ssl cert by lua is running!, -/ +ssl_certificate_by_lua:1: ssl cert by lua is running!,/ diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 183e6faa68..084e7d596c 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -86,6 +86,21 @@ close: 1 nil --- grep_error_log eval: qr/ssl_session_fetch_by_lua_block:.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval +# Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) +# before call ssl callback function +$Test::Nginx::Util::NginxVersion >= 1.017009 ? +[ +qr/\A(?:reusable connection: [01]\n)+\z/s, +qr/^reusable connection: 0 +ssl session fetch: connection reusable: 0 +ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +/m, +qr/^reusable connection: 0 +ssl session fetch: connection reusable: 0 +ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +/m, +] +: [ qr/\A(?:reusable connection: [01]\n)+\z/s, qr/^reusable connection: 1 @@ -99,7 +114,6 @@ reusable connection: 0 ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, /m, ] - --- no_error_log [error] [alert] @@ -1336,6 +1350,21 @@ ssl handshake: userdata close: 1 nil --- grep_error_log eval: qr/ssl_session_fetch_by_lua_block:.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval +# Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) +# before call ssl callback function +$Test::Nginx::Util::NginxVersion >= 1.017009 ? +[ +qr/\A(?:reusable connection: [01]\n)+\z/s, +qr/^reusable connection: 0 +ssl session fetch: connection reusable: 0 +ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +/m, +qr/^reusable connection: 0 +ssl session fetch: connection reusable: 0 +ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +/m, +] +: [ qr/\A(?:reusable connection: [01]\n)+\z/s, qr/^reusable connection: 1 From ff81634233e8663df583face9eae02cd5b57502b Mon Sep 17 00:00:00 2001 From: syz Date: Sun, 30 Aug 2020 15:38:14 +0800 Subject: [PATCH 472/848] optimize: added the "ev" to ngx_posted_delayed_events instead of the rbtree for 0 delay timer, so that we can save an epoll wait in such case. --- src/ngx_http_lua_timer.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 95d1933356..db6ec079d3 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -328,6 +328,16 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) lmcf->pending_timers++; +#ifdef HAVE_POSTED_DELAYED_EVENTS_PATCH + if (delay == 0) { + dd("posting 0 sec sleep event to head of delayed queue"); + ngx_post_event(ev, &ngx_posted_delayed_events); + + lua_pushinteger(L, 1); + return 1; + } +#endif + ngx_add_timer(ev, delay); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, From 57ebdd34396571f8e6f705da01335445a10dadeb Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Sat, 10 Oct 2020 12:10:51 +0800 Subject: [PATCH 473/848] bugfix: should still use timer instead of delayed_events when worker process is exiting, otherwise the new created 0 delay timer may have no chance to run. --- src/ngx_http_lua_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index db6ec079d3..bf07df6812 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -329,7 +329,7 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) lmcf->pending_timers++; #ifdef HAVE_POSTED_DELAYED_EVENTS_PATCH - if (delay == 0) { + if (delay == 0 && !ngx_exiting) { dd("posting 0 sec sleep event to head of delayed queue"); ngx_post_event(ev, &ngx_posted_delayed_events); From 7367a2314c960facb59751a2677f704ae3eca738 Mon Sep 17 00:00:00 2001 From: LubinLew Date: Mon, 28 Sep 2020 17:53:16 +0800 Subject: [PATCH 474/848] bugfix: ngx.resp.get_headers(): may return a wrong (like: negative) value of the Content-Length field when it is larger than 2^32. --- src/ngx_http_lua_headers.c | 18 +++++++++++++++++- t/016-resp-header.t | 39 +++++++++++++++++++++++++++++++++++++- 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 75985a1130..4759107e77 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -424,6 +424,8 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) int count = 0; int truncated = 0; int extra = 0; + u_char *p = NULL; + size_t len = 0; n = lua_gettop(L); @@ -485,7 +487,21 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) { extra++; lua_pushliteral(L, "content-length"); - lua_pushfstring(L, "%d", (int) r->headers_out.content_length_n); + if (r->headers_out.content_length_n > NGX_MAX_INT32_VALUE) { + p = ngx_palloc(r->pool, NGX_OFF_T_LEN); + if (p == NULL) { + return luaL_error(L, "no memory"); + } + + len = ngx_snprintf(p, NGX_OFF_T_LEN, "%O", + r->headers_out.content_length_n) - p; + + lua_pushfstring(L, "%s", (char *) p, len); + + } else { + lua_pushfstring(L, "%d", (int) r->headers_out.content_length_n); + } + lua_rawset(L, -3); } diff --git a/t/016-resp-header.t b/t/016-resp-header.t index febd276fef..c7b5addee4 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 + 77); +plan tests => repeat_each() * (blocks() * 3 + 79); #no_diff(); no_long_string(); @@ -2104,3 +2104,40 @@ xxx: foo: foo%0Axx:bar\r\nfoo: bar%0Dxxx:foo\r\n --- no_error_log [error] + + + +=== TEST 94: fix negative content-length number(#1791) +--- config + location = /big-upstream { + content_by_lua_block { + ngx.header['Content-Length'] = math.pow(2, 33) - 1 + ngx.say('hi') + } + } + + location = /t { + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/big-upstream; + proxy_buffering off; + + header_filter_by_lua_block { + local hs, err = ngx.resp.get_headers() + if err then + ngx.log(ngx.ERR, "err: ", err) + return ngx.exit(500) + end + + print("my Content-Length: ", hs["Content-Length"]) + + ngx.header['Content-Length'] = 3 + } + } +--- request + GET /t +--- response_body +hi +--- no_error_log +[alert] +--- error_log +my Content-Length: 8589934591 +upstream prematurely closed connection while sending to client From fdf752daf8d846ab37bbb1ef4e6fdca627147fd6 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 9 Oct 2020 22:18:08 -0700 Subject: [PATCH 475/848] feature: added undocumented API for attaching arbitrary key-value pair data to a cosocket TCP connecton, which can survive connection pool reuse. This API may change in the future without notice. --- src/ngx_http_lua_socket_tcp.c | 244 ++++++++++++++++++++++++++++++++++ src/ngx_http_lua_socket_tcp.h | 23 ++++ 2 files changed, 267 insertions(+) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index ae068a4dba..e2c2cf2f01 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -221,6 +221,9 @@ static char ngx_http_lua_ssl_session_metatable_key; #endif +#define ngx_http_lua_tcp_socket_metatable_literal_key "__tcp_cosocket_mt" + + void ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) { @@ -355,6 +358,12 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_pushvalue(L, -1); lua_setfield(L, -2, "__index"); lua_rawset(L, LUA_REGISTRYINDEX); + + lua_pushliteral(L, ngx_http_lua_tcp_socket_metatable_literal_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + tcp_socket_metatable_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ /* {{{upstream userdata metatable */ @@ -5382,6 +5391,8 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) item->socklen = pc->socklen; ngx_memcpy(&item->sockaddr, pc->sockaddr, pc->socklen); item->reused = u->reused; + item->udata_queue = u->udata_queue; + u->udata_queue = NULL; if (c->read->ready) { rc = ngx_http_lua_socket_keepalive_close_handler(c->read); @@ -5457,6 +5468,8 @@ ngx_http_lua_get_keepalive_peer(ngx_http_request_t *r, pc->cached = 1; u->reused = item->reused + 1; + u->udata_queue = item->udata_queue; + item->udata_queue = NULL; #if 1 u->write_event_handler = ngx_http_lua_socket_dummy_handler; @@ -6151,4 +6164,235 @@ ngx_http_lua_cleanup_conn_pools(lua_State *L) lua_pop(L, 1); } + +int +ngx_http_lua_ffi_socket_tcp_init_udata_queue( + ngx_http_lua_socket_tcp_upstream_t *u, int capacity, char **err_msg) +{ + int i, max_size; + ngx_pool_t *pool; + ngx_http_lua_socket_udata_queue_t *udata_queue; + ngx_http_lua_socket_node_t *node; + + pool = u->peer.connection->pool; + + if (u->udata_queue == NULL) { + max_size = capacity; + if (max_size == 0) { + max_size = 4; + } + + udata_queue = ngx_palloc(pool, + sizeof(ngx_http_lua_socket_udata_queue_t) + + sizeof(ngx_http_lua_socket_node_t) * max_size); + + if (udata_queue == NULL) { + *err_msg = "no memory"; + return NGX_ERROR; + } + + udata_queue->pool = pool; + udata_queue->capacity = capacity; + udata_queue->len = 0; + ngx_queue_init(&udata_queue->queue); + ngx_queue_init(&udata_queue->free); + + node = (ngx_http_lua_socket_node_t *) (udata_queue + 1); + + for (i = 0; i < max_size; i++) { + ngx_queue_insert_head(&udata_queue->free, &node->queue); + node++; + } + + u->udata_queue = udata_queue; + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "init udata_queue %uD, cosocket %p udata %p", + capacity, u, udata_queue); + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_socket_tcp_count_udata(ngx_http_lua_socket_tcp_upstream_t *u) +{ + /* return NGX_ERROR (-1) for missing udata_queue to + * distinguish it from empty udata_queue */ + if (u->udata_queue == NULL) { + return NGX_ERROR; + } + + return u->udata_queue->len; +} + + +int +ngx_http_lua_ffi_socket_tcp_add_udata(ngx_http_lua_socket_tcp_upstream_t *u, + uint64_t key, uint64_t value, uint64_t *evicted_key, + uint64_t *evicted_value, char **err_msg) +{ + int evicted = 0; + ngx_pool_t *pool; + ngx_http_lua_socket_node_t *node = NULL; + ngx_queue_t *q, *uqueue; + + pool = u->peer.connection->pool; + + if (u->udata_queue == NULL) { + *err_msg = "no udata queue"; + return NGX_ERROR; + } + + uqueue = &u->udata_queue->queue; + + for (q = ngx_queue_head(uqueue); + q != ngx_queue_sentinel(uqueue); + q = ngx_queue_next(q)) + { + node = ngx_queue_data(q, ngx_http_lua_socket_node_t, queue); + + if (node->key == key) { + /* key exists */ + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "found %uD, cosocket %p udata %p", + key, u, u->udata_queue); + ngx_queue_remove(q); + node->value = value; + + break; + } + } + + if (q == ngx_queue_sentinel(uqueue)) { + + if (u->udata_queue->capacity + && u->udata_queue->capacity == u->udata_queue->len) + { + /* evict key */ + q = ngx_queue_last(uqueue); + node = ngx_queue_data(q, ngx_http_lua_socket_node_t, queue); + ngx_queue_remove(q); + ngx_log_debug4(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "evict %uD for %uD, cosocket %p udata %p", + node->key, key, u, u->udata_queue); + *evicted_key = node->key; + *evicted_value = node->value; + evicted = 1; + + } else { + /* insert key */ + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "insert %uD, cosocket %p udata %p", + key, u, u->udata_queue); + + if (!ngx_queue_empty(&u->udata_queue->free)) { + q = ngx_queue_head(&u->udata_queue->free); + node = ngx_queue_data(q, ngx_http_lua_socket_node_t, queue); + ngx_queue_remove(q); + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, + 0, "reuse free node %p, cosocket %p udata %p", + node, u, u->udata_queue); + + } else { + node = ngx_palloc(pool, sizeof(ngx_http_lua_socket_node_t)); + if (node == NULL) { + goto nomem; + } + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, + 0, "allocate new node %p, cosocket %p udata %p", + node, u, u->udata_queue); + } + + u->udata_queue->len++; + } + + node->key = key; + node->value = value; + } + + ngx_queue_insert_head(uqueue, &node->queue); + return evicted ? NGX_DONE : NGX_OK; + +nomem: + + *err_msg = "no memory"; + return NGX_ERROR; +} + + +int +ngx_http_lua_ffi_socket_tcp_get_udata(ngx_http_lua_socket_tcp_upstream_t *u, + uint64_t key, uint64_t *value, char **err_msg) +{ + ngx_http_lua_socket_node_t *node; + ngx_queue_t *q, *uqueue; + + if (u->udata_queue == NULL) { + *err_msg = "no udata queue"; + return NGX_ERROR; + } + + uqueue = &u->udata_queue->queue; + + for (q = ngx_queue_head(uqueue); + q != ngx_queue_sentinel(uqueue); + q = ngx_queue_next(q)) + { + node = ngx_queue_data(q, ngx_http_lua_socket_node_t, queue); + + if (node->key == key) { + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "found %uD, cosocket %p udata %p", + key, u, u->udata_queue); + ngx_queue_remove(q); + ngx_queue_insert_head(uqueue, &node->queue); + *value = node->value; + return NGX_OK; + } + } + + *err_msg = "not found"; + return NGX_ERROR; +} + + +int +ngx_http_lua_ffi_socket_tcp_del_udata(ngx_http_lua_socket_tcp_upstream_t *u, + uint64_t key, char **err_msg) +{ + ngx_http_lua_socket_node_t *node; + ngx_queue_t *q, *uqueue; + + if (u->udata_queue == NULL) { + *err_msg = "no udata queue"; + return NGX_ERROR; + } + + uqueue = &u->udata_queue->queue; + + for (q = ngx_queue_head(uqueue); + q != ngx_queue_sentinel(uqueue); + q = ngx_queue_next(q)) + { + node = ngx_queue_data(q, ngx_http_lua_socket_node_t, queue); + + if (node->key == key) { + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, u->request->connection->log, 0, + "delete %uD, cosocket %p udata %p", + key, u, u->udata_queue); + ngx_queue_remove(q); + ngx_queue_insert_head(&u->udata_queue->free, &node->queue); + u->udata_queue->len--; + return NGX_OK; + } + } + + *err_msg = "not found"; + return NGX_ERROR; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_socket_tcp.h b/src/ngx_http_lua_socket_tcp.h index 091e4378fe..a0a5a5181a 100644 --- a/src/ngx_http_lua_socket_tcp.h +++ b/src/ngx_http_lua_socket_tcp.h @@ -26,6 +26,10 @@ typedef struct ngx_http_lua_socket_tcp_upstream_s ngx_http_lua_socket_tcp_upstream_t; +typedef struct ngx_http_lua_socket_udata_queue_s + ngx_http_lua_socket_udata_queue_t; + + typedef int (*ngx_http_lua_socket_tcp_retval_handler)(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); @@ -79,6 +83,8 @@ struct ngx_http_lua_socket_tcp_upstream_s { ngx_http_lua_socket_tcp_upstream_handler_pt read_event_handler; ngx_http_lua_socket_tcp_upstream_handler_pt write_event_handler; + ngx_http_lua_socket_udata_queue_t *udata_queue; + ngx_http_lua_socket_pool_t *socket_pool; ngx_http_lua_loc_conf_t *conf; @@ -166,9 +172,26 @@ typedef struct { ngx_uint_t reused; + ngx_http_lua_socket_udata_queue_t *udata_queue; } ngx_http_lua_socket_pool_item_t; +struct ngx_http_lua_socket_udata_queue_s { + ngx_pool_t *pool; + ngx_queue_t queue; + ngx_queue_t free; + int len; + int capacity; +}; + + +typedef struct { + ngx_queue_t queue; + uint64_t key; + uint64_t value; +} ngx_http_lua_socket_node_t; + + void ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L); void ngx_http_lua_inject_req_socket_api(lua_State *L); void ngx_http_lua_cleanup_conn_pools(lua_State *L); From 186c8473822875dddf8510348d9c0dfe135a56ff Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 19 Oct 2020 21:48:22 -0700 Subject: [PATCH 476/848] debug: added more debug info to monitor the main lua stack size in timer creation code. --- src/ngx_http_lua_timer.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index bf07df6812..13cc887845 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -340,8 +340,9 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) ngx_add_timer(ev, delay); - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "created timer (co: %p delay: %M ms)", tctx->co, delay); + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "created timer (co: %p delay: %M ms): sz=%d", tctx->co, + delay, lua_gettop(L)); lua_pushinteger(L, 1); return 1; From e16d217a113f0e50b5256859bec1e51b75b29f22 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 19 Oct 2020 21:48:56 -0700 Subject: [PATCH 477/848] optimize: removed useless code for luajit without the lua_resetthread API. --- src/ngx_http_lua_util.c | 5 +++++ src/ngx_http_lua_util.h | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index da702a3210..ef60ebc46a 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -331,6 +331,8 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) { int base; lua_State *co; + +#ifdef HAVE_LUA_RESETTHREAD ngx_queue_t *q; ngx_http_lua_main_conf_t *lmcf; @@ -361,6 +363,9 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) "lua reusing cached lua thread %p (ref %d)", co, *ref); } else { +#else + { +#endif base = lua_gettop(L); lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index a1faafa600..c5b8f9ec4e 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -605,6 +605,8 @@ ngx_http_lua_new_cached_thread(lua_State *L, lua_State **out_co, { int co_ref; lua_State *co; + +#ifdef HAVE_LUA_RESETTHREAD ngx_queue_t *q; ngx_http_lua_thread_ref_t *tref; @@ -633,6 +635,9 @@ ngx_http_lua_new_cached_thread(lua_State *L, lua_State **out_co, lua_rawgeti(L, -1, co_ref); } else { +#else + { +#endif lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); From de1c3c63b8862cbf1d6a45743d858922ecd0134c Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 19 Oct 2020 23:04:36 -0700 Subject: [PATCH 478/848] bugfix: entry threads would leak when exceeding the lua_max_running_timers limit. this regression had appeared in commit 4de59f5da or v0.10.18rc2. --- src/ngx_http_lua_timer.c | 12 ++++++++++++ src/ngx_http_lua_util.c | 37 +++++++++++++++++++++++++++++++++++-- src/ngx_http_lua_util.h | 14 +++++++++++++- t/106-timer.t | 17 ++++++++++++++++- 4 files changed, 76 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 13cc887845..a181b886c1 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -682,6 +682,18 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) "lua failed to run timer with function defined at %s:%d: %s", source, ar.linedefined, errmsg); +#if 1 + if (L == NULL) { + if (tctx.vm_state != NULL) { + L = tctx.vm_state->vm; + } + + if (L == NULL) { + L = lmcf->lua; + } + } +#endif + if (L != NULL) { ngx_http_lua_free_thread(r, L, tctx.co_ref, tctx.co, lmcf); } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index ef60ebc46a..8d4d2fb896 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -362,6 +362,27 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua reusing cached lua thread %p (ref %d)", co, *ref); +#if 0 + { + int n = 0; + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + coroutines_key)); + lua_rawget(L, LUA_REGISTRYINDEX); + lua_pushnil(L); /* first key */ + while (lua_next(L, -2) != 0) { + if (!lua_isnil(L, -1) && !lua_isnil(L, -2)) { + n++; + } + lua_pop(L, 1); + } + lua_pop(L, 1); + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "! lua reusing cached lua thread %p (ref %d, n %d)", + co, *ref, n); + } +#endif + } else { #else { @@ -393,6 +414,10 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) *ref = luaL_ref(L, -2); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, + ngx_cycle->log, 0, "lua ref lua thread %p (ref %d)", co, + *ref); + if (*ref == LUA_NOREF) { lua_settop(L, base); /* restore main thread stack */ return NULL; @@ -419,8 +444,9 @@ ngx_http_lua_del_thread(ngx_http_request_t *r, lua_State *L, return; } - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua deleting light thread"); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua deleting light thread %p (ref %d)", coctx->co, + coctx->co_ref); ngx_http_lua_probe_thread_delete(r, coctx->co, ctx); @@ -450,6 +476,10 @@ ngx_http_lua_del_thread(ngx_http_request_t *r, lua_State *L, } else { #endif + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http lua unref thread %p: %d", coctx->co, + coctx->co_ref); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); @@ -3329,6 +3359,9 @@ ngx_http_lua_finalize_threads(ngx_http_request_t *r, ngx_list_part_t *part; ngx_http_lua_co_ctx_t *cc, *coctx; + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http lua finalize threads"); + #if 1 coctx = ctx->on_abort_co_ctx; if (coctx && coctx->co_ref != LUA_NOREF) { diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index c5b8f9ec4e..5679853653 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -562,7 +562,11 @@ ngx_http_lua_free_thread(ngx_http_request_t *r, lua_State *L, int co_ref, ngx_http_lua_thread_ref_t *tref ; ngx_http_lua_ctx_t *ctx; - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, + r == NULL ? ngx_cycle->log : r->connection->log, 0, + "lua freeing light thread %p (ref %d)", co, co_ref); + + ctx = r != NULL ? ngx_http_get_module_ctx(r, ngx_http_lua_module) : NULL; if (ctx != NULL && L == ctx->entry_co_ctx.co && L == lmcf->lua @@ -591,9 +595,14 @@ ngx_http_lua_free_thread(ngx_http_request_t *r, lua_State *L, int co_ref, } #endif + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, + r != NULL ? r->connection->log : ngx_cycle->log, 0, + "lua unref lua thread %p (ref %d)", co, co_ref); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); + luaL_unref(L, -1, co_ref); lua_pop(L, 1); } @@ -645,6 +654,9 @@ ngx_http_lua_new_cached_thread(lua_State *L, lua_State **out_co, lua_pushvalue(L, -1); co_ref = luaL_ref(L, -3); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua ref lua thread %p (ref %d)", co, co_ref); + #ifndef OPENRESTY_LUAJIT if (set_globals) { lua_createtable(co, 0, 0); /* the new globals table */ diff --git a/t/106-timer.t b/t/106-timer.t index cc4bddd67b..8109a2abae 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -230,8 +230,21 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:1[4-9]|2[0-6]?) === TEST 5: tcp cosocket in timer handler (short connections) --- config server_tokens off; + + location = /gc { + content_by_lua_block { + local c = collectgarbage("count") + ngx.say("before: ", c) + collectgarbage("collect") + c = collectgarbage("count") + ngx.say("after: ", c) + } + } + location = /t { content_by_lua ' + collectgarbage() + -- ngx.say("gc size: ", collectgarbage("count")) local begin = ngx.now() local function fail(...) ngx.log(ngx.ERR, ...) @@ -281,6 +294,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:1[4-9]|2[0-6]?) ngx.say("failed to set timer: ", err) return end + -- ngx.sleep(0.1) ngx.say("registered timer") '; } @@ -1892,6 +1906,7 @@ trace: [m][f][g] --- config location /t { content_by_lua ' + collectgarbage() local s = "" local function fail(...) @@ -1944,7 +1959,7 @@ registered timer --- error_log eval [ -qr/\[alert\] .*? lua failed to run timer with function defined at =content_by_lua\(nginx.conf:\d+\):10: 1 lua_max_running_timers are not enough/, +qr/\[alert\] .*? lua failed to run timer with function defined at =content_by_lua\(nginx.conf:\d+\):11: 1 lua_max_running_timers are not enough/, "lua ngx.timer expired", "http lua close fake http connection", ] From 7bccbbff627d296f490e8f3b66bb298a0a22911d Mon Sep 17 00:00:00 2001 From: jiahao Date: Thu, 22 Oct 2020 19:10:20 +0800 Subject: [PATCH 479/848] tests: skip t/163-signal.t under hup reload test mode. --- t/163-signal.t | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/t/163-signal.t b/t/163-signal.t index ee2de6ef3c..5a6b880b09 100644 --- a/t/163-signal.t +++ b/t/163-signal.t @@ -1,6 +1,14 @@ # vi:ft= -use Test::Nginx::Socket::Lua; +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_USE_HUP}) { + $SkipReason = "unavailable under hup test mode"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); plan tests => 2 * blocks(); From 8ae3e73f2019db31f047aa8e68172ae57e76a306 Mon Sep 17 00:00:00 2001 From: jiahao Date: Thu, 22 Oct 2020 17:07:38 +0800 Subject: [PATCH 480/848] tests: --- shutdown_error_log should be excluded from repeat_each() when counting plans. --- t/089-phase.t | 2 +- t/154-semaphore.t | 2 +- t/162-exit-worker.t | 2 +- t/163-exit-worker-hup.t | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/t/089-phase.t b/t/089-phase.t index 538dff8723..a7405f2d54 100644 --- a/t/089-phase.t +++ b/t/089-phase.t @@ -8,7 +8,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 2); +plan tests => repeat_each() * (blocks() * 2 + 1) + 2; #no_diff(); #no_long_string(); diff --git a/t/154-semaphore.t b/t/154-semaphore.t index 875f181ce7..c7b00e03f6 100644 --- a/t/154-semaphore.t +++ b/t/154-semaphore.t @@ -8,7 +8,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3) + blocks(); +plan tests => repeat_each() * (blocks() * 3) + 1; no_long_string(); #no_diff(); diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index cb0edf7076..ef2082f3b4 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -5,7 +5,7 @@ use Test::Nginx::Socket::Lua; master_on(); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 8); +plan tests => repeat_each() * (blocks() * 2 + 2) + 12; #log_level("warn"); no_long_string(); diff --git a/t/163-exit-worker-hup.t b/t/163-exit-worker-hup.t index 3f516aba98..a54d81db14 100644 --- a/t/163-exit-worker-hup.t +++ b/t/163-exit-worker-hup.t @@ -16,7 +16,7 @@ use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 2); +plan tests => repeat_each() * (blocks() * 2 + 1) + 2; no_long_string(); From ea6b07cd65e6fa64763a3f66f89c24cb2aee0d95 Mon Sep 17 00:00:00 2001 From: poorsea Date: Thu, 24 Sep 2020 22:56:42 -0700 Subject: [PATCH 481/848] feature: implemented the new tcpsok:setoption() function for setting options for the TCP socket. --- README.markdown | 94 ++++++++++++++++++- doc/HttpLuaModule.wiki | 88 ++++++++++++++++- src/ngx_http_lua_socket_tcp.c | 171 +++++++++++++++++++++++++++++++--- t/062-count.t | 2 +- 4 files changed, 337 insertions(+), 18 deletions(-) diff --git a/README.markdown b/README.markdown index 71a79d1975..a967948a48 100644 --- a/README.markdown +++ b/README.markdown @@ -7789,14 +7789,104 @@ This feature was first introduced in the `v0.10.7` release. tcpsock:setoption ----------------- -**syntax:** *tcpsock:setoption(option, value?)* +**syntax:** *ok, err = tcpsock:setoption(option, value?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** -This function is added for [LuaSocket](http://w3.impa.br/~diego/software/luasocket/tcp.html) API compatibility and does nothing for now. Its functionality will be implemented in future. +This function is added for [LuaSocket](http://w3.impa.br/~diego/software/luasocket/tcp.html) API compatibility and does nothing for now. Its functionality is implemented `v0.10.18`. This feature was first introduced in the `v0.5.0rc1` release. +In case of success, it returns `true`. Otherwise, it returns nil and a string describing the error. + +The `option` is a string with the option name, and the value depends on the option being set: + +* `keepalive` + + Setting this option to true enables sending of keep-alive messages on + connection-oriented sockets. Make sure the `connect` function + had been called before, for example, + + ```lua + + local ok, err = tcpsock:setoption("keepalive", true) + if not ok then + ngx.say("setoption keepalive failed: ", err) + end + ``` +* `reuseaddr` + + Enabling this option indicates that the rules used in validating addresses + supplied in a call to bind should allow reuse of local addresses. Make sure + the `connect` function had been called before, for example, + + ```lua + + local ok, err = tcpsock:setoption("reuseaddr", 0) + if not ok then + ngx.say("setoption reuseaddr failed: ", err) + end + ``` +* `tcp-nodelay` + + Setting this option to true disables the Nagle's algorithm for the connection. + Make sure the `connect` function had been called before, for example, + + ```lua + + local ok, err = tcpsock:setoption("tcp-nodelay", true) + if not ok then + ngx.say("setoption tcp-nodelay failed: ", err) + end + ``` +* `sndbuf` + + Sets the maximum socket send buffer in bytes. The kernel doubles this value + (to allow space for bookkeeping overhead) when it is set using setsockopt(). + Make sure the `connect` function had been called before, for example, + + ```lua + + local ok, err = tcpsock:setoption("sndbuf", 1024 * 10) + if not ok then + ngx.say("setoption sndbuf failed: ", err) + end + ``` +* `rcvbuf` + + Sets the maximum socket receive buffer in bytes. The kernel doubles this value + (to allow space for bookkeeping overhead) when it is set using setsockopt. Make + sure the `connect` function had been called before, for example, + + ```lua + + local ok, err = tcpsock:setoption("rcvbuf", 1024 * 10) + if not ok then + ngx.say("setoption rcvbuf failed: ", err) + end + ``` + +NOTE: Once the option is set, it will become effective until the connection is closed. If you know the connection is from the connection pool and all the in-pool connections already have called the setoption() method with the desired socket option state, then you can just skip calling setoption() again to avoid the overhead of repeated calls, for example, + +```lua + + local count, err = tcpsock:getreusedtimes() + if not count then + ngx.say("getreusedtimes failed: ", err) + return + end + + if count == 0 then + local ok, err = tcpsock:setoption("rcvbuf", 1024 * 10) + if not ok then + ngx.say("setoption rcvbuf failed: ", err) + return + end + end +``` + +These options described above are supported in `v0.10.18`, and more options will be implemented in future. + [Back to TOC](#nginx-api-for-lua) tcpsock:setkeepalive diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 29e4b30ac9..73d476514c 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -6626,14 +6626,98 @@ This feature was first introduced in the v0.10.7 release. == tcpsock:setoption == -'''syntax:''' ''tcpsock:setoption(option, value?)'' +'''syntax:''' ''ok, err = tcpsock:setoption(option, value?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' -This function is added for [http://w3.impa.br/~diego/software/luasocket/tcp.html LuaSocket] API compatibility and does nothing for now. Its functionality will be implemented in future. +This function is added for [http://w3.impa.br/~diego/software/luasocket/tcp.html LuaSocket] API compatibility and does nothing for now. Its functionality is implemented v0.10.18. This feature was first introduced in the v0.5.0rc1 release. +In case of success, it returns true. Otherwise, it returns nil and a string describing the error. + +The option is a string with the option name, and the value depends on the option being set: + +* keepalive + +: Setting this option to true enables sending of keep-alive messages on +: connection-oriented sockets. Make sure the connect function +: had been called before, for example, + + + local ok, err = tcpsock:setoption("keepalive", true) + if not ok then + ngx.say("setoption keepalive failed: ", err) + end + +* reuseaddr + +: Enabling this option indicates that the rules used in validating addresses +: supplied in a call to bind should allow reuse of local addresses. Make sure +: the connect function had been called before, for example, + + + local ok, err = tcpsock:setoption("reuseaddr", 0) + if not ok then + ngx.say("setoption reuseaddr failed: ", err) + end + +* tcp-nodelay + +: Setting this option to true disables the Nagle's algorithm for the connection. +: Make sure the connect function had been called before, for example, + + + local ok, err = tcpsock:setoption("tcp-nodelay", true) + if not ok then + ngx.say("setoption tcp-nodelay failed: ", err) + end + +* sndbuf + +: Sets the maximum socket send buffer in bytes. The kernel doubles this value +: (to allow space for bookkeeping overhead) when it is set using setsockopt(). +: Make sure the connect function had been called before, for example, + + + local ok, err = tcpsock:setoption("sndbuf", 1024 * 10) + if not ok then + ngx.say("setoption sndbuf failed: ", err) + end + +* rcvbuf + +: Sets the maximum socket receive buffer in bytes. The kernel doubles this value +: (to allow space for bookkeeping overhead) when it is set using setsockopt. Make +: sure the connect function had been called before, for example, + + + local ok, err = tcpsock:setoption("rcvbuf", 1024 * 10) + if not ok then + ngx.say("setoption rcvbuf failed: ", err) + end + + +NOTE: Once the option is set, it will become effective until the connection is closed. If you know the connection is from the connection pool and all the in-pool connections already have called the setoption() method with the desired socket option state, then you can just skip calling setoption() again to avoid the overhead of repeated calls, for example, + + + local count, err = tcpsock:getreusedtimes() + if not count then + ngx.say("getreusedtimes failed: ", err) + return + end + + if count == 0 then + local ok, err = tcpsock:setoption("rcvbuf", 1024 * 10) + if not ok then + ngx.say("setoption rcvbuf failed: ", err) + return + end + end + + +These options described above are supported in v0.10.18, and more options will be implemented in future. + == tcpsock:setkeepalive == '''syntax:''' ''ok, err = tcpsock:setkeepalive(timeout?, size?)'' diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index e2c2cf2f01..00159398ed 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -28,7 +28,6 @@ static int ngx_http_lua_socket_tcp_receive(lua_State *L); static int ngx_http_lua_socket_tcp_receiveany(lua_State *L); static int ngx_http_lua_socket_tcp_send(lua_State *L); static int ngx_http_lua_socket_tcp_close(lua_State *L); -static int ngx_http_lua_socket_tcp_setoption(lua_State *L); static int ngx_http_lua_socket_tcp_settimeout(lua_State *L); static int ngx_http_lua_socket_tcp_settimeouts(lua_State *L); static void ngx_http_lua_socket_tcp_handler(ngx_event_t *ev); @@ -178,6 +177,15 @@ enum { }; +enum { + NGX_HTTP_LUA_SOCKOPT_KEEPALIVE = 1, + NGX_HTTP_LUA_SOCKOPT_REUSEADDR, + NGX_HTTP_LUA_SOCKOPT_TCP_NODELAY, + NGX_HTTP_LUA_SOCKOPT_SNDBUF, + NGX_HTTP_LUA_SOCKOPT_RCVBUF, +}; + + #define ngx_http_lua_socket_check_busy_connecting(r, u, L) \ if ((u)->conn_waiting) { \ lua_pushnil(L); \ @@ -313,7 +321,7 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* {{{tcp object metatable */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( tcp_socket_metatable_key)); - lua_createtable(L, 0 /* narr */, 13 /* nrec */); + lua_createtable(L, 0 /* narr */, 14 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_connect); lua_setfield(L, -2, "connect"); @@ -340,9 +348,6 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_pushcfunction(L, ngx_http_lua_socket_tcp_close); lua_setfield(L, -2, "close"); - lua_pushcfunction(L, ngx_http_lua_socket_tcp_setoption); - lua_setfield(L, -2, "setoption"); - lua_pushcfunction(L, ngx_http_lua_socket_tcp_settimeout); lua_setfield(L, -2, "settimeout"); /* ngx socket mt */ @@ -3105,14 +3110,6 @@ ngx_http_lua_socket_tcp_close(lua_State *L) } -static int -ngx_http_lua_socket_tcp_setoption(lua_State *L) -{ - /* TODO */ - return 0; -} - - static int ngx_http_lua_socket_tcp_settimeout(lua_State *L) { @@ -6395,4 +6392,152 @@ ngx_http_lua_ffi_socket_tcp_del_udata(ngx_http_lua_socket_tcp_upstream_t *u, } +int +ngx_http_lua_ffi_socket_tcp_getoption(ngx_http_lua_socket_tcp_upstream_t *u, + int option, int *val, u_char *err, size_t *errlen) +{ + socklen_t len; + int fd, rc; + + if (u == NULL || u->peer.connection == NULL) { + *errlen = ngx_snprintf(err, *errlen, "closed") - err; + return NGX_ERROR; + } + + fd = u->peer.connection->fd; + + if (fd == (ngx_socket_t) -1) { + *errlen = ngx_snprintf(err, *errlen, "invalid socket fd") - err; + return NGX_ERROR; + } + + len = sizeof(int); + + switch (option) { + case NGX_HTTP_LUA_SOCKOPT_KEEPALIVE: + rc = getsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, (void *) val, &len); + break; + + case NGX_HTTP_LUA_SOCKOPT_REUSEADDR: + rc = getsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (void *) val, &len); + break; + + case NGX_HTTP_LUA_SOCKOPT_TCP_NODELAY: + rc = getsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (void *) val, &len); + break; + + case NGX_HTTP_LUA_SOCKOPT_SNDBUF: + rc = getsockopt(fd, SOL_SOCKET, SO_RCVBUF, (void *) val, &len); + break; + + case NGX_HTTP_LUA_SOCKOPT_RCVBUF: + rc = getsockopt(fd, SOL_SOCKET, SO_SNDBUF, (void *) val, &len); + break; + + default: + *errlen = ngx_snprintf(err, *errlen, "unsupported option %d", option) + - err; + return NGX_ERROR; + } + + if (rc == -1) { + *errlen = ngx_strerror(ngx_errno, err, NGX_MAX_ERROR_STR) - err; + return NGX_ERROR; + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_socket_tcp_setoption(ngx_http_lua_socket_tcp_upstream_t *u, + int option, int val, u_char *err, size_t *errlen) +{ + socklen_t len; + int fd, rc; + + if (u == NULL || u->peer.connection == NULL) { + *errlen = ngx_snprintf(err, *errlen, "closed") - err; + return NGX_ERROR; + } + + fd = u->peer.connection->fd; + + if (fd == (ngx_socket_t) -1) { + *errlen = ngx_snprintf(err, *errlen, "invalid socket fd") - err; + return NGX_ERROR; + } + + len = sizeof(int); + + switch (option) { + case NGX_HTTP_LUA_SOCKOPT_KEEPALIVE: + rc = setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, + (const void *) &val, len); + break; + + case NGX_HTTP_LUA_SOCKOPT_REUSEADDR: + rc = setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, + (const void *) &val, len); + break; + + case NGX_HTTP_LUA_SOCKOPT_TCP_NODELAY: + rc = setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, + (const void *) &val, len); + break; + + case NGX_HTTP_LUA_SOCKOPT_SNDBUF: + rc = setsockopt(fd, SOL_SOCKET, SO_RCVBUF, + (const void *) &val, len); + break; + + case NGX_HTTP_LUA_SOCKOPT_RCVBUF: + rc = setsockopt(fd, SOL_SOCKET, SO_SNDBUF, + (const void *) &val, len); + break; + + default: + *errlen = ngx_snprintf(err, *errlen, "unsupported option: %d", option) + - err; + return NGX_ERROR; + } + + if (rc == -1) { + *errlen = ngx_strerror(ngx_errno, err, NGX_MAX_ERROR_STR) - err; + return NGX_ERROR; + } + + return NGX_OK; +} + + +/* just hack the fd for testing bad case, it will also return the original fd */ +int +ngx_http_lua_ffi_socket_tcp_hack_fd(ngx_http_lua_socket_tcp_upstream_t *u, + int fd, u_char *err, size_t *errlen) +{ + int rc; + + if (u == NULL || u->peer.connection == NULL) { + *errlen = ngx_snprintf(err, *errlen, "closed") - err; + return -1; + } + + rc = u->peer.connection->fd; + if (rc == (ngx_socket_t) -1) { + *errlen = ngx_snprintf(err, *errlen, "invalid socket fd") - err; + return -1; + } + + /* return the original fd value directly when the new fd is invalid */ + if (fd < 0) { + return rc; + } + + u->peer.connection->fd = fd; + + return rc; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/062-count.t b/t/062-count.t index 841923b86d..807e618e21 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -459,7 +459,7 @@ worker: 4 --- request GET /test --- response_body -n = 13 +n = 14 --- no_error_log [error] From 843e2030161ca84dc0637911abd209bf00019413 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 23 Oct 2020 17:47:46 +0800 Subject: [PATCH 482/848] bugfix: ngx.flush never blocks on http v2. --- src/ngx_http_lua_output.c | 3 ++- src/ngx_http_lua_util.c | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index 7c7403f667..c0aadffb4f 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -559,7 +559,8 @@ ngx_http_lua_ngx_flush(lua_State *L) wev = r->connection->write; - if (wait && (r->connection->buffered & NGX_HTTP_LOWLEVEL_BUFFERED + if (wait && (r->connection->buffered + & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED) || wev->delayed)) { ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 8d4d2fb896..c98a51f005 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1733,7 +1733,7 @@ ngx_http_lua_wev_handler(ngx_http_request_t *r) return NGX_DONE; } - if (c->buffered & NGX_HTTP_LOWLEVEL_BUFFERED) { + if (c->buffered & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED)) { rc = ngx_http_lua_flush_pending_output(r, ctx); dd("flush pending output returned %d, c->error: %d", (int) rc, @@ -1885,7 +1885,7 @@ ngx_http_lua_flush_pending_output(ngx_http_request_t *r, return rc; } - if (c->buffered & NGX_HTTP_LOWLEVEL_BUFFERED) { + if (c->buffered & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED)) { clcf = ngx_http_get_module_loc_conf(r->main, ngx_http_core_module); From 50de32e8509a0c04d14daf87ea86e8fdcda067e0 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 25 Oct 2020 17:54:13 -0700 Subject: [PATCH 483/848] travis: upgraded openssl 1.1.1 to 1.1.1h. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 97eb1525b3..e2b73087d3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,7 +50,7 @@ env: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.3 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.1g OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.1h OPENSSL_PATCH_VER=1.1.1f services: - memcached From 8b9d7fdaefecd1cb7569af59caefacbe34df6d0c Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 25 Oct 2020 17:57:29 -0700 Subject: [PATCH 484/848] travis: openssl: enable threads. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index e2b73087d3..d9990ddd9d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -117,7 +117,7 @@ script: - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch - - ./config no-threads shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) + - ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. From 65d87507489b31b86419b0fcecc1c75ce46d0ece Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 27 Oct 2020 11:26:38 +0800 Subject: [PATCH 485/848] bugfix: wrong pipelined request body length when request body is parent's request body. (#1814) --- src/ngx_http_lua_subrequest.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index b866955569..a798f9bdab 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1637,6 +1637,7 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, { ngx_table_elt_t *clh, *header; ngx_list_part_t *part; + ngx_chain_t *in; ngx_uint_t i; u_char *p; off_t len; @@ -1653,8 +1654,11 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, if (sr->request_body && !pr_not_chunked) { /* craft our own Content-Length */ + len = 0; - len = sr->request_body->buf ? ngx_buf_size(sr->request_body->buf) : 0; + for (in = sr->request_body->bufs; in; in = in->next) { + len += ngx_buf_size(in->buf); + } clh = ngx_list_push(&sr->headers_in.headers); if (clh == NULL) { From 4653f2182e5986042755ae451422429f899e634b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 27 Oct 2020 19:37:48 +0800 Subject: [PATCH 486/848] bugfix: fixed segfault caused by an incorrect format token introduced in commit 3e24a95a. --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 00159398ed..55bd203d9c 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2627,7 +2627,7 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, #endif ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket try to recv data %uz: \"%V?%V\"", + "lua tcp socket try to recv data %O: \"%V?%V\"", size, &r->uri, &r->args); n = c->recv(c, b->last, size); From 411a9aed7a176cf3b28be52de518c85279aa3cf1 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 29 Oct 2020 21:52:13 +0800 Subject: [PATCH 487/848] tests: 124-init-worker.t: made the test robust by changing the match pattern for TEST 22. (#1817) --- t/124-init-worker.t | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 94081ad2cf..b7e17a07f1 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -884,6 +884,10 @@ lua close the global Lua VM \2 lua close the global Lua VM \3 in the cache helper process \d+ lua close the global Lua VM \3 lua close the global Lua VM \3 in the cache helper process \d+ +|lua close the global Lua VM ([0-9A-F]+) +lua close the global Lua VM \4 in the cache helper process \d+ +lua close the global Lua VM \4 in the cache helper process \d+ +lua close the global Lua VM \4 )(?:lua close the global Lua VM [0-9A-F]+ )*\z/ --- error_log eval From e497ffc683242813a6c47cfbd77681511d75c9b7 Mon Sep 17 00:00:00 2001 From: jiahao Date: Thu, 29 Oct 2020 20:54:33 +0800 Subject: [PATCH 488/848] bugfix: ngx.resp.set_headers(): may cause an error when Content-Length field is larger than 2^32. --- src/ngx_http_lua_headers_in.c | 2 +- src/ngx_http_lua_headers_out.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index fff8bafb92..7626d1f949 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -563,7 +563,7 @@ ngx_http_set_content_length_header(ngx_http_request_t *r, return ngx_http_clear_content_length_header(r, hv, value); } - len = ngx_atosz(value->data, value->len); + len = ngx_atoof(value->data, value->len); if (len == NGX_ERROR) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index a3b0f9e90e..6e1879c50f 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -450,7 +450,7 @@ ngx_http_set_content_length_header(ngx_http_request_t *r, return ngx_http_clear_content_length_header(r, hv, value); } - len = ngx_atosz(value->data, value->len); + len = ngx_atoof(value->data, value->len); if (len == NGX_ERROR) { return NGX_ERROR; } From 98f482bfb1f394f0cd83bcefe3bd7a674d29b860 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 29 Oct 2020 23:41:34 +0800 Subject: [PATCH 489/848] bugfix: process may crash when a timer run failed and a function with arguments run inside this timer. (#1819) used the uninitialized "ar" value due to the unexpected Lua stack and lua_getinfo will be failed to set the "ar" value. --- src/ngx_http_lua_timer.c | 2 ++ t/106-timer.t | 46 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index a181b886c1..71886761a4 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -670,6 +670,8 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) failed: /* co stack: func [args] */ + lua_pushvalue(tctx.co, 1); + /* co stack: func [args] func */ lua_getinfo(tctx.co, ">Sf", &ar); source = ar.source; diff --git a/t/106-timer.t b/t/106-timer.t index 8109a2abae..f0fe193ea7 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -2349,3 +2349,49 @@ ok [error] --- error_log eval qr/\[alert\] .*? lua failed to run timer with function defined at @.+\/test.lua:3: 1 lua_max_running_timers are not enough/ + + + +=== TEST 36: log function location when failed to run a timer with args (lua file) +--- user_files +>>> test.lua +local _M = {} + +function _M.run(premature, arg) + ngx.sleep(0.01) +end + +return _M +--- http_config + lua_package_path '$TEST_NGINX_HTML_DIR/?.lua;./?.lua;;'; + lua_max_running_timers 1; +--- config + location /t { + content_by_lua_block { + local test = require "test" + + local ok, err = ngx.timer.at(0, test.run, "arg") + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + local ok, err = ngx.timer.at(0, test.run, "arg") + if not ok then + ngx.say("failed to set timer: ", err) + return + end + + ngx.say("ok") + } + } +--- request +GET /t +--- response_body +ok +--- wait: 0.1 +--- no_error_log +[crit] +[error] +--- error_log eval +qr/\[alert\] .*? lua failed to run timer with function defined at @.+\/test.lua:3: 1 lua_max_running_timers are not enough/ From 26809e75b964fba0eb101bbacf64c9ec2c7ad4c3 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 30 Oct 2020 11:38:39 +0800 Subject: [PATCH 490/848] tests: 106-timer.t: fixed test plan count. --- t/106-timer.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/106-timer.t b/t/106-timer.t index f0fe193ea7..2996762ad4 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -12,7 +12,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 8 + 64); +plan tests => repeat_each() * (blocks() * 8 + 61); #no_diff(); no_long_string(); From 15197e7bbae89ce1c726061d04211262322f2712 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 30 Oct 2020 12:52:50 +0800 Subject: [PATCH 491/848] bugfix: accessed memory out of range when calling lua_pushfstring with none zero-terminated string introduced in commit 7367a231. --- src/ngx_http_lua_headers.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 4759107e77..d8d5edc8b2 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -496,7 +496,7 @@ ngx_http_lua_ngx_resp_get_headers(lua_State *L) len = ngx_snprintf(p, NGX_OFF_T_LEN, "%O", r->headers_out.content_length_n) - p; - lua_pushfstring(L, "%s", (char *) p, len); + lua_pushlstring(L, (char *) p, len); } else { lua_pushfstring(L, "%d", (int) r->headers_out.content_length_n); From 83a61d6b4592f13fddd401504ccdf6af83840119 Mon Sep 17 00:00:00 2001 From: Yichun Zhang Date: Sun, 1 Nov 2020 06:12:11 -0800 Subject: [PATCH 492/848] optimize: post zombie thread: avoided the linear search. (#1823) This function can be very expensive when there are a lot of zombie threads in the current context. --- src/ngx_http_lua_common.h | 1 + src/ngx_http_lua_coroutine.c | 1 + src/ngx_http_lua_util.c | 11 ++++++++--- src/ngx_http_lua_util.h | 2 ++ 4 files changed, 12 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 37894871b5..0751a1123f 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -446,6 +446,7 @@ struct ngx_http_lua_co_ctx_s { ngx_http_lua_co_ctx_t *parent_co_ctx; ngx_http_lua_posted_thread_t *zombie_child_threads; + ngx_http_lua_posted_thread_t **next_zombie_child_thread; ngx_http_cleanup_pt cleanup; diff --git a/src/ngx_http_lua_coroutine.c b/src/ngx_http_lua_coroutine.c index 65eeef2e8b..1580c5b28c 100644 --- a/src/ngx_http_lua_coroutine.c +++ b/src/ngx_http_lua_coroutine.c @@ -140,6 +140,7 @@ ngx_http_lua_coroutine_create_helper(lua_State *L, ngx_http_request_t *r, } else { ngx_memzero(coctx, sizeof(ngx_http_lua_co_ctx_t)); + coctx->next_zombie_child_thread = &coctx->zombie_child_threads; coctx->co_ref = LUA_NOREF; } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c98a51f005..36d6aa4cbd 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -986,6 +986,9 @@ ngx_http_lua_reset_ctx(ngx_http_request_t *r, lua_State *L, ngx_memzero(&ctx->entry_co_ctx, sizeof(ngx_http_lua_co_ctx_t)); + ctx->entry_co_ctx.next_zombie_child_thread = + &ctx->entry_co_ctx.zombie_child_threads; + ctx->entry_co_ctx.co_ref = LUA_NOREF; ctx->entered_rewrite_phase = 0; @@ -3268,6 +3271,7 @@ ngx_http_lua_create_co_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) ngx_memzero(coctx, sizeof(ngx_http_lua_co_ctx_t)); + coctx->next_zombie_child_thread = &coctx->zombie_child_threads; coctx->co_ref = LUA_NOREF; return coctx; @@ -3425,7 +3429,6 @@ static ngx_int_t ngx_http_lua_post_zombie_thread(ngx_http_request_t *r, ngx_http_lua_co_ctx_t *parent, ngx_http_lua_co_ctx_t *thread) { - ngx_http_lua_posted_thread_t **p; ngx_http_lua_posted_thread_t *pt; pt = ngx_palloc(r->pool, sizeof(ngx_http_lua_posted_thread_t)); @@ -3436,9 +3439,10 @@ ngx_http_lua_post_zombie_thread(ngx_http_request_t *r, pt->co_ctx = thread; pt->next = NULL; - for (p = &parent->zombie_child_threads; *p; p = &(*p)->next) { /* void */ } + ngx_http_lua_assert(parent->next_zombie_child_thread != NULL); - *p = pt; + *parent->next_zombie_child_thread = pt; + parent->next_zombie_child_thread = &pt->next; return NGX_OK; } @@ -3458,6 +3462,7 @@ ngx_http_lua_cleanup_zombie_child_uthreads(ngx_http_request_t *r, } coctx->zombie_child_threads = NULL; + coctx->next_zombie_child_thread = &coctx->zombie_child_threads; } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 5679853653..1768e5e28e 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -266,6 +266,8 @@ ngx_http_lua_init_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) ngx_memzero(ctx, sizeof(ngx_http_lua_ctx_t)); ctx->ctx_ref = LUA_NOREF; ctx->entry_co_ctx.co_ref = LUA_NOREF; + ctx->entry_co_ctx.next_zombie_child_thread = + &ctx->entry_co_ctx.zombie_child_threads; ctx->resume_handler = ngx_http_lua_wev_handler; ctx->request = r; } From 7105adaa523adedec80f0aaa13388b88d08988f8 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Tue, 3 Nov 2020 14:58:18 +0800 Subject: [PATCH 493/848] bumped the API version to 0.10.19. (#1824) --- README.markdown | 11 ++++++----- doc/HttpLuaModule.wiki | 11 ++++++----- src/api/ngx_http_lua_api.h | 2 +- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/README.markdown b/README.markdown index a967948a48..6a37359bec 100644 --- a/README.markdown +++ b/README.markdown @@ -67,8 +67,8 @@ Version ======= This document describes ngx_lua -[v0.10.17](https://github.com/openresty/lua-nginx-module/tags), which was released -on 3 July, 2020. +[v0.10.19](https://github.com/openresty/lua-nginx-module/tags), which was released +on 3 Nov, 2020. Synopsis ======== @@ -284,6 +284,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.19.x (last tested: 1.19.3) * 1.17.x (last tested: 1.17.8) * 1.15.x (last tested: 1.15.8) * 1.14.x @@ -328,9 +329,9 @@ Build the source with this module: ```bash - wget 'https://openresty.org/download/nginx-1.17.8.tar.gz' - tar -xzvf nginx-1.17.8.tar.gz - cd nginx-1.17.8/ + wget 'https://openresty.org/download/nginx-1.19.3.tar.gz' + tar -xzvf nginx-1.19.3.tar.gz + cd nginx-1.19.3/ # tell nginx's build system where to find LuaJIT 2.0: export LUAJIT_LIB=/path/to/luajit/lib diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 73d476514c..7e454d17ee 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -15,8 +15,8 @@ Production ready. = Version = This document describes ngx_lua -[https://github.com/openresty/lua-nginx-module/tags v0.10.17], which was released -on 3 July, 2020. +[https://github.com/openresty/lua-nginx-module/tags v0.10.19], which was released +on 3 Nov, 2020. = Synopsis = @@ -221,6 +221,7 @@ performance level. The latest version of this module is compatible with the following versions of Nginx: +* 1.19.x (last tested: 1.19.3) * 1.17.x (last tested: 1.17.8) * 1.15.x (last tested: 1.15.8) * 1.14.x @@ -262,9 +263,9 @@ Alternatively, ngx_lua can be manually compiled into Nginx: Build the source with this module: - wget 'https://openresty.org/download/nginx-1.17.8.tar.gz' - tar -xzvf nginx-1.17.8.tar.gz - cd nginx-1.17.8/ + wget 'https://openresty.org/download/nginx-1.19.3.tar.gz' + tar -xzvf nginx-1.19.3.tar.gz + cd nginx-1.19.3/ # tell nginx's build system where to find LuaJIT 2.0: export LUAJIT_LIB=/path/to/luajit/lib diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index c532b95e7a..8a86ef0593 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10018 +#define ngx_http_lua_version 10019 typedef struct { From d59afff27aab0a40b7da589f795960203bed6ae5 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sun, 29 Nov 2020 19:09:51 -0800 Subject: [PATCH 494/848] doc: added our tutorial videos. --- README.markdown | 20 ++++++++++++++++++++ doc/HttpLuaModule.wiki | 16 ++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/README.markdown b/README.markdown index 6a37359bec..5bb986991f 100644 --- a/README.markdown +++ b/README.markdown @@ -20,6 +20,7 @@ Table of Contents * [Name](#name) * [Status](#status) * [Version](#version) +* [Videos](#videos) * [Synopsis](#synopsis) * [Description](#description) * [Typical Uses](#typical-uses) @@ -70,6 +71,25 @@ This document describes ngx_lua [v0.10.19](https://github.com/openresty/lua-nginx-module/tags), which was released on 3 Nov, 2020. +Videos +====== + +* YouTube video "[Hello World HTTP Example with OpenResty/Lua](https://youtu.be/eSfYLvVQMxw)" + + [![Hello World HTTP Example with OpenResty/Lua](https://img.youtube.com/vi/eSfYLvVQMxw/0.jpg)](https://youtu.be/eSfYLvVQMxw) + +* YouTube video "[Write Your Own Lua Modules in OpenResty/Nginx Applications](https://youtu.be/vfYxOMl5LVY)" + + [![Write Your Own Lua Modules in OpenResty/Nginx Applications](https://img.youtube.com/vi/vfYxOMl5LVY/0.jpg)](https://youtu.be/vfYxOMl5LVY) + +* YouTube video "[OpenResty's resty Command-Line Utility Demo](https://youtu.be/L1c7aw4mSOo)" + + [![OpenResty's resty Command-Line Utility Demo](https://img.youtube.com/vi/L1c7aw4mSOo/0.jpg)](https://youtu.be/L1c7aw4mSOo) + +You are welcome to subscribe to our [official YouTube channel, OpenResty](https://www.youtube.com/channel/UCXVmwF-UCScv2ftsGoMqxhw). + +[Back to TOC](#table-of-contents) + Synopsis ======== ```nginx diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 7e454d17ee..4ff13f5541 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -18,6 +18,22 @@ This document describes ngx_lua [https://github.com/openresty/lua-nginx-module/tags v0.10.19], which was released on 3 Nov, 2020. += Videos = + +* YouTube video "[Hello World HTTP Example with OpenResty/Lua](https://youtu.be/eSfYLvVQMxw)" + + [![Hello World HTTP Example with OpenResty/Lua](https://img.youtube.com/vi/eSfYLvVQMxw/0.jpg)](https://youtu.be/eSfYLvVQMxw) + +* YouTube video "[Write Your Own Lua Modules in OpenResty/Nginx Applications](https://youtu.be/vfYxOMl5LVY)" + + [![Write Your Own Lua Modules in OpenResty/Nginx Applications](https://img.youtube.com/vi/vfYxOMl5LVY/0.jpg)](https://youtu.be/vfYxOMl5LVY) + +* YouTube video "[OpenResty's resty Command-Line Utility Demo](https://youtu.be/L1c7aw4mSOo)" + + [![OpenResty's resty Command-Line Utility Demo](https://img.youtube.com/vi/L1c7aw4mSOo/0.jpg)](https://youtu.be/L1c7aw4mSOo) + +You are welcome to subscribe to our [official YouTube channel, OpenResty](https://www.youtube.com/channel/UCXVmwF-UCScv2ftsGoMqxhw). + = Synopsis = # set search paths for pure Lua external libraries (';;' is the default path): From 99ea68a1353fb00b8b5deffb06272e216d46459c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Thu, 3 Dec 2020 08:51:40 +0800 Subject: [PATCH 495/848] bugfix: we closed listener's fd which was closed. (#1832) fix #1806 fix #1830 --- src/ngx_http_lua_pipe.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index c0be9c9e05..c555d7bc94 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -688,8 +688,10 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, /* close listening socket fd */ ls = ngx_cycle->listening.elts; for (i = 0; i < ngx_cycle->listening.nelts; i++) { - if (ngx_close_socket(ls[i].fd) == -1) { - ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_socket_errno, + if (ls[i].fd != (ngx_socket_t) -1 && + ngx_close_socket(ls[i].fd) == -1) + { + ngx_log_error(NGX_LOG_WARN, ngx_cycle->log, ngx_socket_errno, "lua pipe child " ngx_close_socket_n " %V failed", &ls[i].addr_text); } From b9b39645ff8e0a99ba61738c74f7ad7b905f385e Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Mon, 7 Dec 2020 15:30:18 -0800 Subject: [PATCH 496/848] doc: added 1 more video. --- README.markdown | 4 ++++ doc/HttpLuaModule.wiki | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/README.markdown b/README.markdown index 5bb986991f..c2c9d812fe 100644 --- a/README.markdown +++ b/README.markdown @@ -86,6 +86,10 @@ Videos [![OpenResty's resty Command-Line Utility Demo](https://img.youtube.com/vi/L1c7aw4mSOo/0.jpg)](https://youtu.be/L1c7aw4mSOo) +* YouTube video "[Measure Execution Time of Lua Code Correctly in OpenResty](https://youtu.be/VkRYW_qLoME)" + + [![Measure Execution Time of Lua Code Correctly in OpenResty](https://img.youtube.com/vi/VkRYW_qLoME/0.jpg)](https://youtu.be/VkRYW_qLoME) + You are welcome to subscribe to our [official YouTube channel, OpenResty](https://www.youtube.com/channel/UCXVmwF-UCScv2ftsGoMqxhw). [Back to TOC](#table-of-contents) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4ff13f5541..ec1ff1867b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -32,6 +32,10 @@ on 3 Nov, 2020. [![OpenResty's resty Command-Line Utility Demo](https://img.youtube.com/vi/L1c7aw4mSOo/0.jpg)](https://youtu.be/L1c7aw4mSOo) +* YouTube video "[Measure Execution Time of Lua Code Correctly in OpenResty](https://youtu.be/VkRYW_qLoME)" + + [![Measure Execution Time of Lua Code Correctly in OpenResty](https://img.youtube.com/vi/VkRYW_qLoME/0.jpg)](https://youtu.be/VkRYW_qLoME) + You are welcome to subscribe to our [official YouTube channel, OpenResty](https://www.youtube.com/channel/UCXVmwF-UCScv2ftsGoMqxhw). = Synopsis = From 60e95eff355c948085a4b5960257cc64237f9d2e Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Wed, 9 Dec 2020 19:10:59 -0800 Subject: [PATCH 497/848] doc: added new youtube video on luajit bytecode precompilation. --- README.markdown | 8 ++++++++ doc/HttpLuaModule.wiki | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/README.markdown b/README.markdown index c2c9d812fe..01f1773a8a 100644 --- a/README.markdown +++ b/README.markdown @@ -90,6 +90,10 @@ Videos [![Measure Execution Time of Lua Code Correctly in OpenResty](https://img.youtube.com/vi/VkRYW_qLoME/0.jpg)](https://youtu.be/VkRYW_qLoME) +* YouTube video "[Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup](https://youtu.be/EP7c0BM2yNo)" + + [![Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup](https://img.youtube.com/vi/EP7c0BM2yNo/0.jpg)](https://youtu.be/EP7c0BM2yNo) + You are welcome to subscribe to our [official YouTube channel, OpenResty](https://www.youtube.com/channel/UCXVmwF-UCScv2ftsGoMqxhw). [Back to TOC](#table-of-contents) @@ -456,6 +460,10 @@ Please submit bug reports, wishlists, or patches by LuaJIT bytecode support ======================= +Watch YouTube video "[Measure Execution Time of Lua Code Correctly in OpenResty](https://youtu.be/VkRYW_qLoME)" + +[![Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup](https://img.youtube.com/vi/EP7c0BM2yNo/0.jpg)](https://youtu.be/EP7c0BM2yNo) + As from the `v0.5.0rc32` release, all `*_by_lua_file` configure directives (such as [content_by_lua_file](#content_by_lua_file)) support loading LuaJIT 2.0/2.1 raw bytecode files directly: ```bash diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ec1ff1867b..a8426d30d4 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -36,6 +36,10 @@ on 3 Nov, 2020. [![Measure Execution Time of Lua Code Correctly in OpenResty](https://img.youtube.com/vi/VkRYW_qLoME/0.jpg)](https://youtu.be/VkRYW_qLoME) +* YouTube video "[Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup](https://youtu.be/EP7c0BM2yNo)" + + [![Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup](https://img.youtube.com/vi/EP7c0BM2yNo/0.jpg)](https://youtu.be/EP7c0BM2yNo) + You are welcome to subscribe to our [official YouTube channel, OpenResty](https://www.youtube.com/channel/UCXVmwF-UCScv2ftsGoMqxhw). = Synopsis = @@ -361,6 +365,10 @@ Please submit bug reports, wishlists, or patches by = LuaJIT bytecode support = +Watch YouTube video "[Measure Execution Time of Lua Code Correctly in OpenResty](https://youtu.be/VkRYW_qLoME)" + +[![Precompile Lua Modules into LuaJIT Bytecode to Speedup OpenResty Startup](https://img.youtube.com/vi/EP7c0BM2yNo/0.jpg)](https://youtu.be/EP7c0BM2yNo) + As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as [[#content_by_lua_file|content_by_lua_file]]) support loading LuaJIT 2.0/2.1 raw bytecode files directly: From 8f12a28e2e9c2eecae58dc21cf3056d78b0a41a9 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 11 Dec 2020 08:53:13 +0800 Subject: [PATCH 498/848] updated openssl to 1.1.1i. (#1836) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index d9990ddd9d..342568294e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,7 +50,7 @@ env: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.3 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.1h OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.1i OPENSSL_PATCH_VER=1.1.1f services: - memcached From 7bba6d1954bb1db8d6dcef22516f3ed9f238497f Mon Sep 17 00:00:00 2001 From: Hawker Date: Mon, 14 Dec 2020 10:56:53 +0800 Subject: [PATCH 499/848] bugfix: fixed memory leak in debug log. --- src/ngx_http_lua_logby.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index 408708cfa5..b47058be19 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -69,6 +69,9 @@ ngx_http_lua_log_handler(ngx_http_request_t *r) #if (NGX_HTTP_LUA_HAVE_MALLOC_TRIM) ngx_uint_t trim_cycle, trim_nreq; ngx_http_lua_main_conf_t *lmcf; +#if (NGX_DEBUG) + ngx_int_t trim_ret; +#endif #endif ngx_http_lua_loc_conf_t *llcf; ngx_http_lua_ctx_t *ctx; @@ -88,8 +91,9 @@ ngx_http_lua_log_handler(ngx_http_request_t *r) lmcf->malloc_trim_req_count = 0; #if (NGX_DEBUG) + trim_ret = malloc_trim(1); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "malloc_trim(1) returned %d", malloc_trim(1)); + "malloc_trim(1) returned %d", trim_ret); #else (void) malloc_trim(1); #endif From 138c1b96423aa26defe00fe64dd5760ef17e5ad8 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Wed, 18 Sep 2019 02:01:03 +0500 Subject: [PATCH 500/848] src/ngx_http_lua_ssl_certby.c: fix possible null pointer dereference found by coverity deref_ptr: Directly dereferencing pointer c. 419 if (c->addr_text.len) { 420 p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); 421 len -= p - buf; 422 buf = p; 423 } 424 CID 149837 (#1 of 1): Dereference before null check (REVERSE_INULL)check_after_deref: Null-checking c suggests that it may be null, but it has already been dereferenced on all paths leading to the check. 425 if (c && c->listening && c->listening->addr_text.len) { 426 p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); 427 /* len -= p - buf; */ 428 buf = p; 429 } --- src/ngx_http_lua_ssl_certby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index dfaa9f15a3..6ed2f3f1cb 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -420,7 +420,7 @@ ngx_http_lua_log_ssl_cert_error(ngx_log_t *log, u_char *buf, size_t len) c = log->data; - if (c->addr_text.len) { + if (c && c->addr_text.len) { p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); len -= p - buf; buf = p; From 23e40f3fd29499fef94d9a173bc29ac5dda197fd Mon Sep 17 00:00:00 2001 From: John Bampton Date: Tue, 29 Dec 2020 16:55:36 +1000 Subject: [PATCH 501/848] docs: fix spelling (#1841) --- t/030-uri-args.t | 6 +++--- t/129-ssl-socket.t | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 4391b919f6..0633476c8d 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1690,7 +1690,7 @@ bad argument #1 to 'set_uri_args' (string, number, or table expected, but got us === TEST 64: set_uri binary option with unsafe uri -explict specify binary option to true +explicit specify binary option to true --- config location /t { rewrite_by_lua_block { @@ -1717,7 +1717,7 @@ explict specify binary option to true === TEST 65: set_uri binary option with unsafe uri -explict specify binary option to false +explicit specify binary option to false --- config location /t { rewrite_by_lua_block { @@ -1743,7 +1743,7 @@ qr{\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: rewrite_by_ === TEST 66: set_uri binary option with safe uri -explict specify binary option to false +explicit specify binary option to false --- config location /t { rewrite_by_lua_block { diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 3b9b34c33b..fc2cc31f07 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1939,7 +1939,7 @@ SSL reused session --- request GET /t --- response_body eval -# Since nginx version 1.19.1, invalidity date is considerd a non-critical CRL +# Since nginx version 1.19.1, invalidity date is considered a non-critical CRL # entry extension, in other words, revoke still works even if CRL has expired. $Test::Nginx::Util::NginxVersion >= 1.019001 ? From 4d2d47ce561c22245d47cd988dcee81397de402b Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Mon, 11 Jan 2021 14:08:28 +0500 Subject: [PATCH 502/848] chore: fix possible null pointer dereference found by Coverity (#1845) --- src/ngx_http_lua_ssl_session_storeby.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 10803522a9..2cfc8f6559 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -328,15 +328,18 @@ ngx_http_lua_log_ssl_sess_store_error(ngx_log_t *log, u_char *buf, size_t len) c = log->data; - if (c->addr_text.len) { - p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); - len -= p - buf; - buf = p; - } + if (c != NULL) { + if (c->addr_text.len) { + p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); + len -= p - buf; + buf = p; + } - if (c && c->listening && c->listening->addr_text.len) { - p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); - buf = p; + if (c->listening && c->listening->addr_text.len) { + p = ngx_snprintf(buf, len, ", server: %V", \ + &c->listening->addr_text); + buf = p; + } } return buf; From 9a3176e93f504ab0ac33c0d5683d34546e17c876 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Mon, 11 Jan 2021 16:10:43 +0500 Subject: [PATCH 503/848] chore: fix possible null pointer dereference found by Coverity (#1844) --- src/ngx_http_lua_ssl_session_fetchby.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index f69cabb198..8d8c42b213 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -442,15 +442,18 @@ ngx_http_lua_log_ssl_sess_fetch_error(ngx_log_t *log, u_char *buf, size_t len) c = log->data; - if (c->addr_text.len) { - p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); - len -= p - buf; - buf = p; - } + if (c != NULL) { + if (c->addr_text.len) { + p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); + len -= p - buf; + buf = p; + } - if (c && c->listening && c->listening->addr_text.len) { - p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); - buf = p; + if (c->listening && c->listening->addr_text.len) { + p = ngx_snprintf(buf, len, ", server: %V", \ + &c->listening->addr_text); + buf = p; + } } return buf; From 88a52c44af56476f68c95c9200a380cd82f696dd Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Tue, 12 Jan 2021 07:52:15 +0500 Subject: [PATCH 504/848] chore: fix possible null pointer dereference found by Coverity (#1846) 734 buf = p; 735 deref_ptr: Directly dereferencing pointer c. 736 if (c->addr_text.len) { 737 p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); 738 len -= p - buf; 739 buf = p; 740 } 741 CID 149839 (#1 of 1): Dereference before null check (REVERSE_INULL)check_after_deref: Null-checking c suggests that it may be null, but it has already been dereferenced on all paths leading to the check. 742 if (c && c->listening && c->listening->addr_text.len) { 743 p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); 744 /* len -= p - buf; */ 745 buf = p; 746 } --- src/ngx_http_lua_timer.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 71886761a4..353007def4 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -733,16 +733,19 @@ ngx_http_lua_log_timer_error(ngx_log_t *log, u_char *buf, size_t len) len -= p - buf; buf = p; - if (c->addr_text.len) { - p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); - len -= p - buf; - buf = p; - } + if (c != NULL) { + if (c->addr_text.len) { + p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); + len -= p - buf; + buf = p; + } - if (c && c->listening && c->listening->addr_text.len) { - p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); - /* len -= p - buf; */ - buf = p; + if (c->listening && c->listening->addr_text.len) { + p = ngx_snprintf(buf, len, ", server: %V", + &c->listening->addr_text); + /* len -= p - buf; */ + buf = p; + } } return buf; From 2abb142d2b40ad2cc765ecd1a21fe6abaaf54b10 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Sun, 28 Mar 2021 23:20:08 +0800 Subject: [PATCH 505/848] tests: fixed typo in 162-exit-worker.t. (#1866) --- t/162-exit-worker.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index ef2082f3b4..8f2efc21e5 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -83,7 +83,7 @@ hello, world --- http_config exit_worker_by_lua_block { local function bar() - ngx.log(ngx.ERR, "run the timer!" + ngx.log(ngx.ERR, "run the timer!") end local ok, err = ngx.timer.at(0, bar) From eb944dedd28231a99d8a2a71cbc392ce2dad36c1 Mon Sep 17 00:00:00 2001 From: Suika <872957+suikabreaker@users.noreply.github.com> Date: Sun, 28 Mar 2021 23:22:08 +0800 Subject: [PATCH 506/848] bugfix: correct the way to handle invalid quote in ngx.escape_uri/ngx.req.get_{uri,post}_args (#1862) Co-authored-by: zhouxumin --- README.markdown | 15 ++++ src/ngx_http_lua_util.c | 160 ++++++++++++---------------------------- t/006-escape.t | 34 +++++++++ 3 files changed, 97 insertions(+), 112 deletions(-) diff --git a/README.markdown b/README.markdown index 01f1773a8a..c819da91fb 100644 --- a/README.markdown +++ b/README.markdown @@ -5750,6 +5750,21 @@ gives the output b r56 7 +Invalid escaping sequences are handled in a conventional way: `%`s are left unchanged. Also, characters that should not appear in escaped string are simply left unchanged. + +For example, + +```lua + + ngx.say(ngx.unescape_uri("try %search%%20%again%")) +``` + +gives the output + + + try %search% %again% + +(Note that `%20` following `%` got unescaped, even it can be considered a part of invalid sequence.) [Back to TOC](#nginx-api-for-lua) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 36d6aa4cbd..5c374fb9cc 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -2192,137 +2192,73 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) return (uintptr_t) dst; } +static int +ngx_http_lua_util_hex2int(char xdigit) +{ + if (isdigit(xdigit)) { + return xdigit - '0'; + } + + xdigit = tolower(xdigit); + if (xdigit <= 'f' && xdigit >= 'a') { + return xdigit - 'a' + 10; + } + + return -1; +} /* XXX we also decode '+' to ' ' */ void ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, ngx_uint_t type) { - u_char *d, *s, ch, c, decoded; - enum { - sw_usual = 0, - sw_quoted, - sw_quoted_second, - } state; - - d = *dst; - s = *src; - - state = 0; - decoded = 0; + u_char *d = *dst, *s = *src, *de = (*dst+size); + int isuri = type & NGX_UNESCAPE_URI; + int isredirect = type & NGX_UNESCAPE_REDIRECT; while (size--) { + u_char curr = *s++; - ch = *s++; - - switch (state) { - case sw_usual: - if (ch == '?' - && (type & (NGX_UNESCAPE_URI|NGX_UNESCAPE_REDIRECT))) - { - *d++ = ch; - goto done; - } - - if (ch == '%') { - state = sw_quoted; - break; - } - - if (ch == '+') { - *d++ = ' '; - break; - } - - *d++ = ch; - break; - - case sw_quoted: - - if (ch >= '0' && ch <= '9') { - decoded = (u_char) (ch - '0'); - state = sw_quoted_second; - break; - } - - c = (u_char) (ch | 0x20); - if (c >= 'a' && c <= 'f') { - decoded = (u_char) (c - 'a' + 10); - state = sw_quoted_second; - break; - } - - /* the invalid quoted character */ - - state = sw_usual; - - *d++ = ch; - + if (curr == '?' && + (type & (NGX_UNESCAPE_URI | NGX_UNESCAPE_REDIRECT))) + { + *d++ = '?'; break; - case sw_quoted_second: - - state = sw_usual; - - if (ch >= '0' && ch <= '9') { - ch = (u_char) ((decoded << 4) + ch - '0'); - - if (type & NGX_UNESCAPE_REDIRECT) { - if (ch > '%' && ch < 0x7f) { - *d++ = ch; - break; - } - - *d++ = '%'; *d++ = *(s - 2); *d++ = *(s - 1); - break; - } - - *d++ = ch; - - break; + } else if (curr == '%') { + u_char ch; + if (size < 2 || !(isxdigit(s[0]) && isxdigit(s[1]))) { + *d++ = '%'; + continue; } - - c = (u_char) (ch | 0x20); - if (c >= 'a' && c <= 'f') { - ch = (u_char) ((decoded << 4) + c - 'a' + 10); - - if (type & NGX_UNESCAPE_URI) { - if (ch == '?') { - *d++ = ch; - goto done; - } - - *d++ = ch; - break; - } - - if (type & NGX_UNESCAPE_REDIRECT) { - if (ch == '?') { - *d++ = ch; - goto done; - } - - if (ch > '%' && ch < 0x7f) { - *d++ = ch; - break; - } - - *d++ = '%'; *d++ = *(s - 2); *d++ = *(s - 1); - break; - } - + /* we can be sure here they must be hex digits */ + ch = ngx_http_lua_util_hex2int(s[0]) * 16 + + ngx_http_lua_util_hex2int(s[1]); + + if ((isuri || isredirect) && ch == '?') { *d++ = ch; - break; + } else if (isredirect && (ch <= '%' || ch >= 0x7f)) { + *d++ = '%'; + continue; } + *d++ = ch; + s += 2; + size -= 2; - /* the invalid quoted character */ + } else if (curr == '+') { + *d++ = ' '; + continue; - break; + } else { + *d++ = curr; } } - -done: + + /* a safe guard if dst need to be null-terminated */ + if (d != de) { + *d = '\0'; + } *dst = d; *src = s; diff --git a/t/006-escape.t b/t/006-escape.t index aadf167552..266b23516c 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -294,3 +294,37 @@ GET /lua --- error_code: 500 --- error_log eval qr/\[error\] \d+#\d+: \*\d+ lua entry thread aborted: runtime error: "type" is not a number/ + + + +=== TEST 20: invalid unescape sequences +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.unescape_uri("%ua%%20%au")) + } + } +--- request +GET /lua +--- response_body +%ua% %au + + + +=== TEST 21: invalid unescape sequences where remain length less than 2 +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.unescape_uri("%a")) -- first charactor is good + ngx.say(ngx.unescape_uri("%u")) -- first charactor is bad + ngx.say(ngx.unescape_uri("%")) + ngx.say(ngx.unescape_uri("good%20job%")) + } + } +--- request +GET /lua +--- response_body +%a +%u +% +good job% From 1c1bcfeb9276c6dfc9b038a6fa779e7e4b605bc9 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Sun, 4 Apr 2021 17:08:36 +0800 Subject: [PATCH 507/848] travis-ci: bumped the NGINX core to 1.19.9. (#1870) --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 342568294e..87f6d4c382 100644 --- a/.travis.yml +++ b/.travis.yml @@ -48,9 +48,9 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.19.3 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.19.3 OPENSSL_VER=1.1.1i OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.19.9 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h + - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1i OPENSSL_PATCH_VER=1.1.1f services: - memcached From a43cc398a867df86f453dd9d21284c2d4bd6da13 Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Mon, 19 Apr 2021 12:49:34 +0800 Subject: [PATCH 508/848] doc: sync doc from README to ensure wiki2markdown.pl is work (#1875) --- README.markdown | 4 +++- doc/HttpLuaModule.wiki | 16 ++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index c819da91fb..aac0deea06 100644 --- a/README.markdown +++ b/README.markdown @@ -5750,9 +5750,10 @@ gives the output b r56 7 + Invalid escaping sequences are handled in a conventional way: `%`s are left unchanged. Also, characters that should not appear in escaped string are simply left unchanged. -For example, +For example, ```lua @@ -5764,6 +5765,7 @@ gives the output try %search% %again% + (Note that `%20` following `%` got unescaped, even it can be considered a part of invalid sequence.) [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a8426d30d4..47519a5870 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4835,6 +4835,22 @@ gives the output b r56 7 +Invalid escaping sequences are handled in a conventional way: `%`s are left unchanged. Also, characters that should not appear in escaped string are simply left unchanged. + +For example, + + + ngx.say(ngx.unescape_uri("try %search%%20%again%")) + + +gives the output + + + try %search% %again% + + +(Note that `%20` following `%` got unescaped, even it can be considered a part of invalid sequence.) + == ngx.encode_args == '''syntax:''' ''str = ngx.encode_args(table)'' From 5b90b1fae70c323fa0fe9b6859895cdd7e2bcc91 Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Mon, 19 Apr 2021 12:52:01 +0800 Subject: [PATCH 509/848] doc: recommend to renew ngx.timer.at regularly (#1876) --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index aac0deea06..e99243b312 100644 --- a/README.markdown +++ b/README.markdown @@ -8501,6 +8501,8 @@ user "light threads" ([ngx.thread.*](#ngxthreadspawn)), [ngx.exit](#ngxexit), [n (like [ngx.say](#ngxsay), [ngx.print](#ngxprint), and [ngx.flush](#ngxflush)) are explicitly disabled in this context. +You must notice that each timer will be based on a fake request (this fake request is also based on a fake connection). Because Nginx's memory release is based on the connection closure, if you run a lot of APIs that apply for memory resources in a timer, such as [tcpsock:connect](#tcpsockconnect), will cause the accumulation of memory resources. So it is recommended to create a new timer after running several times to release memory resources. + You can pass most of the standard Lua values (nils, booleans, numbers, strings, tables, closures, file handles, and etc) into the timer callback, either explicitly as user arguments or implicitly as upvalues for the callback closure. There are several exceptions, however: you *cannot* pass any thread objects returned by [coroutine.create](#coroutinecreate) and [ngx.thread.spawn](#ngxthreadspawn) or any cosocket objects returned by [ngx.socket.tcp](#ngxsockettcp), [ngx.socket.udp](#ngxsocketudp), and [ngx.req.socket](#ngxreqsocket) because these objects' lifetime is bound to the request context creating them while the timer callback is detached from the creating request's context (by design) and runs in its own (fake) request context. If you try to share the thread or cosocket objects across the boundary of the creating request, then you will get the "no co ctx found" error (for threads) or "bad request" (for cosockets). It is fine, however, to create all these objects inside your timer callback. This API was first introduced in the `v0.8.0` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 47519a5870..456f8aa252 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7292,6 +7292,8 @@ user "light threads" ([[#ngx.thread.spawn|ngx.thread.*]]), [[#ngx.exit|ngx.exit] (like [[#ngx.say|ngx.say]], [[#ngx.print|ngx.print]], and [[#ngx.flush|ngx.flush]]) are explicitly disabled in this context. +You must notice that each timer will be based on a fake request (this fake request is also based on a fake connection). Because Nginx's memory release is based on the connection closure, if you run a lot of APIs that apply for memory resources in a timer, such as [[#tcpsock:connect|tcpsock:connect]], will cause the accumulation of memory resources. So it is recommended to create a new timer after running several times to release memory resources. + You can pass most of the standard Lua values (nils, booleans, numbers, strings, tables, closures, file handles, and etc) into the timer callback, either explicitly as user arguments or implicitly as upvalues for the callback closure. There are several exceptions, however: you ''cannot'' pass any thread objects returned by [[#coroutine.create|coroutine.create]] and [[#ngx.thread.spawn|ngx.thread.spawn]] or any cosocket objects returned by [[#ngx.socket.tcp|ngx.socket.tcp]], [[#ngx.socket.udp|ngx.socket.udp]], and [[#ngx.req.socket|ngx.req.socket]] because these objects' lifetime is bound to the request context creating them while the timer callback is detached from the creating request's context (by design) and runs in its own (fake) request context. If you try to share the thread or cosocket objects across the boundary of the creating request, then you will get the "no co ctx found" error (for threads) or "bad request" (for cosockets). It is fine, however, to create all these objects inside your timer callback. This API was first introduced in the v0.8.0 release. From dcc77fae754bab21016706704443192a2fbd2ce1 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Thu, 22 Apr 2021 18:24:56 -0700 Subject: [PATCH 510/848] doc: documented the scope of ngx.ctx for timer handlers. thanks Jason Hoch for the suggestion. --- README.markdown | 5 +++++ doc/HttpLuaModule.wiki | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/README.markdown b/README.markdown index e99243b312..c06419bdf6 100644 --- a/README.markdown +++ b/README.markdown @@ -8505,6 +8505,11 @@ You must notice that each timer will be based on a fake request (this fake reque You can pass most of the standard Lua values (nils, booleans, numbers, strings, tables, closures, file handles, and etc) into the timer callback, either explicitly as user arguments or implicitly as upvalues for the callback closure. There are several exceptions, however: you *cannot* pass any thread objects returned by [coroutine.create](#coroutinecreate) and [ngx.thread.spawn](#ngxthreadspawn) or any cosocket objects returned by [ngx.socket.tcp](#ngxsockettcp), [ngx.socket.udp](#ngxsocketudp), and [ngx.req.socket](#ngxreqsocket) because these objects' lifetime is bound to the request context creating them while the timer callback is detached from the creating request's context (by design) and runs in its own (fake) request context. If you try to share the thread or cosocket objects across the boundary of the creating request, then you will get the "no co ctx found" error (for threads) or "bad request" (for cosockets). It is fine, however, to create all these objects inside your timer callback. +Please note that the timer Lua handler has its own copy of the `ngx.ctx` magic +table. It won't share the same `ngx.ctx` with the Lua handler creating the timer. +If you need to pass data from the timer creator to the timer handler, please +use the extra parameters of `ngx.timer.at()`. + This API was first introduced in the `v0.8.0` release. [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 456f8aa252..7bc9fbe545 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7296,6 +7296,11 @@ You must notice that each timer will be based on a fake request (this fake reque You can pass most of the standard Lua values (nils, booleans, numbers, strings, tables, closures, file handles, and etc) into the timer callback, either explicitly as user arguments or implicitly as upvalues for the callback closure. There are several exceptions, however: you ''cannot'' pass any thread objects returned by [[#coroutine.create|coroutine.create]] and [[#ngx.thread.spawn|ngx.thread.spawn]] or any cosocket objects returned by [[#ngx.socket.tcp|ngx.socket.tcp]], [[#ngx.socket.udp|ngx.socket.udp]], and [[#ngx.req.socket|ngx.req.socket]] because these objects' lifetime is bound to the request context creating them while the timer callback is detached from the creating request's context (by design) and runs in its own (fake) request context. If you try to share the thread or cosocket objects across the boundary of the creating request, then you will get the "no co ctx found" error (for threads) or "bad request" (for cosockets). It is fine, however, to create all these objects inside your timer callback. +Please note that the timer Lua handler has its own copy of the `ngx.ctx` magic +table. It won't share the same `ngx.ctx` with the Lua handler creating the timer. +If you need to pass data from the timer creator to the timer handler, please +use the extra parameters of `ngx.timer.at()`. + This API was first introduced in the v0.8.0 release. == ngx.timer.every == From b8d5b379f8cc1a418a4a54b72a54a35eeb6768af Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 23 Apr 2021 11:21:12 +0800 Subject: [PATCH 511/848] tests: added a new rule about 'ngx_pass_open_channel()' to valgrind suppression file. (#1874) --- valgrind.suppress | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/valgrind.suppress b/valgrind.suppress index 7fff542ffe..8761fb350e 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -207,3 +207,30 @@ fun:ngx_epoll_init fun:ngx_event_process_init } +{ + + Memcheck:Param + sendmsg(msg.msg_iov[0]) + fun:__sendmsg_nocancel + fun:ngx_write_channel + fun:ngx_pass_open_channel + fun:ngx_start_worker_processes +} +{ + + Memcheck:Param + sendmsg(msg.msg_iov[0]) + fun:__sendmsg_nocancel + fun:ngx_write_channel + fun:ngx_pass_open_channel + fun:ngx_start_cache_manager_processes +} +{ + + Memcheck:Param + sendmsg(msg.msg_iov[0]) + fun:__sendmsg_nocancel + fun:ngx_write_channel + fun:ngx_pass_open_channel + fun:ngx_start_privileged_agent_processes +} From 14ef2cb25cce0c588f4a714d0fdd6d29bcdbaf0b Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Fri, 23 Apr 2021 12:54:32 -0700 Subject: [PATCH 512/848] doc: added sample config directives for internal redirects, as reported by Jason Hoch. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index c06419bdf6..0d91171657 100644 --- a/README.markdown +++ b/README.markdown @@ -3798,7 +3798,7 @@ Then `GET /main` will give the output Here, modification of the `ngx.ctx.blah` entry in the subrequest does not affect the one in the parent request. This is because they have two separate versions of `ngx.ctx.blah`. -Internal redirection will destroy the original request `ngx.ctx` data (if any) and the new request will have an empty `ngx.ctx` table. For instance, +Internal redirects (triggered by nginx configuration directives like `error_page`, `try_files`, `index` and etc) will destroy the original request `ngx.ctx` data (if any) and the new request will have an empty `ngx.ctx` table. For instance, ```nginx diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 7bc9fbe545..037ec3efa2 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3105,7 +3105,7 @@ Then GET /main will give the output Here, modification of the ngx.ctx.blah entry in the subrequest does not affect the one in the parent request. This is because they have two separate versions of ngx.ctx.blah. -Internal redirection will destroy the original request ngx.ctx data (if any) and the new request will have an empty ngx.ctx table. For instance, +Internal redirects (triggered by nginx configuration directives like `error_page`, `try_files`, `index` and etc) will destroy the original request ngx.ctx data (if any) and the new request will have an empty ngx.ctx table. For instance, location /new { From d3a8626421a86dc8802db1c74bdc4d3b2f065709 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 25 Apr 2021 12:01:21 +0800 Subject: [PATCH 513/848] tests: suspected memory leak of 160-disable-init-by-lua.t when test with TEST_NGINX_CHECK_LEAK=1 because forgot to close the file. (#1877) --- t/160-disable-init-by-lua.t | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/t/160-disable-init-by-lua.t b/t/160-disable-init-by-lua.t index 541771e1b5..dd0098d0d5 100644 --- a/t/160-disable-init-by-lua.t +++ b/t/160-disable-init-by-lua.t @@ -33,6 +33,7 @@ my $http_config = <<_EOC_; end assert(f:write(conf)) + f:close() return conf_file end @@ -85,6 +86,8 @@ __DATA__ else ngx.log(ngx.WARN, out) end + p:close() + collectgarbage("collect") } } --- error_log @@ -115,6 +118,7 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ else ngx.log(ngx.WARN, out) end + p:close() local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -T" local p, err = io.popen(cmd) @@ -130,6 +134,8 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ else ngx.log(ngx.WARN, out) end + p:close() + collectgarbage("collect") } } --- error_log @@ -171,6 +177,7 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ else ngx.log(ngx.WARN, out) end + p:close() local cmd = nginx .. " -p $TEST_NGINX_HTML_DIR -c " .. conf_file .. " -T" local p, err = io.popen(cmd) @@ -186,6 +193,8 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ else ngx.log(ngx.WARN, out) end + p:close() + collectgarbage("collect") } } --- error_log From dabc8e881f62086767b3ed137a0dfec3c50fd0de Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 6 May 2021 13:31:40 +0800 Subject: [PATCH 514/848] change: changed from graceful shutdown to abort when lua_resume return LUA_ERRMEM. If the worker voluntary enters into graceful shutdown state, the master will not respawn worker with the same worker id until the worker exit. when reuseport is on, the new connections assigned to the voluntary graceful shutdown worker will not be accepted. when reuseport is off, the new connections will be accepted by other processes which are still working. But if all workers are in voluntary shutdown state, the new connections will not be accepted. So the worker should not enter into the voluntary graceful shutdown state when LUA_ERRMEM occurs. --- src/ngx_http_lua_util.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 5c374fb9cc..5fe9b5063b 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1471,8 +1471,9 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, break; case LUA_ERRMEM: - err = "memory allocation error"; - ngx_quit = 1; + err = "[lua] memory allocation error"; + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, err); + abort(); break; case LUA_ERRERR: From 9007d673e28938f5dfa7720438991e22b794d225 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 7 May 2021 16:44:37 +0800 Subject: [PATCH 515/848] bumped the API version to 0.10.20. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 8a86ef0593..43f90bd0b1 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10019 +#define ngx_http_lua_version 10020 typedef struct { From 68f6a1498e6a1a8ed20e902b2f44be8d35f43697 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 7 May 2021 22:24:10 +0800 Subject: [PATCH 516/848] travis-ci: upgraded openssl from 1.1.1i to 1.1.1k. (#1868) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 87f6d4c382..feee2075b9 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,7 +50,7 @@ env: - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.9 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1i OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1k OPENSSL_PATCH_VER=1.1.1f services: - memcached From f72792234c56e10b9ef565015df034ab55a6cfc1 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 14 May 2021 00:37:40 +0800 Subject: [PATCH 517/848] doc: stripped line trailing whitespaces. (#1881) * stripped the trailing whitespaces introduced in ad2b404d. * stripped the trailing whitespaces introduced in 88a52c44. --- src/ngx_http_lua_timer.c | 2 +- src/ngx_http_lua_util.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 353007def4..cafce52cb6 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -741,7 +741,7 @@ ngx_http_lua_log_timer_error(ngx_log_t *log, u_char *buf, size_t len) } if (c->listening && c->listening->addr_text.len) { - p = ngx_snprintf(buf, len, ", server: %V", + p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); /* len -= p - buf; */ buf = p; diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 5fe9b5063b..f8630efbe4 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -2204,7 +2204,7 @@ ngx_http_lua_util_hex2int(char xdigit) if (xdigit <= 'f' && xdigit >= 'a') { return xdigit - 'a' + 10; } - + return -1; } @@ -2235,7 +2235,7 @@ ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, /* we can be sure here they must be hex digits */ ch = ngx_http_lua_util_hex2int(s[0]) * 16 + ngx_http_lua_util_hex2int(s[1]); - + if ((isuri || isredirect) && ch == '?') { *d++ = ch; break; @@ -2255,7 +2255,7 @@ ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, *d++ = curr; } } - + /* a safe guard if dst need to be null-terminated */ if (d != de) { *d = '\0'; From 6ed8e4b0ff6a736d0d35b76b5d51a9d4b056ef01 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Thu, 10 Jun 2021 20:56:29 +0800 Subject: [PATCH 518/848] feature: allow to be compiled with LibreSSL 3.0+ (#1888) --- src/ngx_http_lua_ssl_certby.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 6ed2f3f1cb..1ee2d2882b 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1350,6 +1350,13 @@ int ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, int depth, char **err) { +#ifdef LIBRESSL_VERSION_NUMBER + + *err = "LibreSSL not supported"; + return NGX_ERROR; + +#else + ngx_http_lua_ctx_t *ctx; ngx_ssl_conn_t *ssl_conn; ngx_http_ssl_srv_conf_t *sscf; @@ -1466,6 +1473,7 @@ ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, X509_STORE_free(ca_store); return NGX_ERROR; +#endif } From f1b97e4507a4093e015efc164a37b191640dd746 Mon Sep 17 00:00:00 2001 From: tzssangglass Date: Thu, 10 Jun 2021 20:56:56 +0800 Subject: [PATCH 519/848] feature: support environ in ngx.pipe on mac (#1882) --- src/ngx_http_lua_pipe.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index c555d7bc94..9b50d0aef5 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -545,6 +545,21 @@ ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, size_t size) } +#if !(NGX_HTTP_LUA_HAVE_EXECVPE) +static int +ngx_http_lua_execvpe(const char *program, char **argv, char **envp) +{ + char **saved = environ; + int rc; + + environ = envp; + rc = execvp(program, argv); + environ = saved; + return rc; +} +#endif + + int ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, const char *file, const char **argv, int merge_stderr, size_t buffer_size, @@ -568,15 +583,6 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, ngx_http_lua_pipe_signal_t *sig; sigset_t set; -#if !(NGX_HTTP_LUA_HAVE_EXECVPE) - if (environ != NULL) { - *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, - "environ option not supported") - - errbuf; - return NGX_ERROR; - } -#endif - pool_size = ngx_align(NGX_MIN_POOL_SIZE + buffer_size * 2, NGX_POOL_ALIGNMENT); @@ -766,9 +772,13 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } } -#if (NGX_HTTP_LUA_HAVE_EXECVPE) if (environ != NULL) { +#if (NGX_HTTP_LUA_HAVE_EXECVPE) if (execvpe(file, (char * const *) argv, (char * const *) environ) +#else + if (ngx_http_lua_execvpe(file, (char * const *) argv, + (char * const *) environ) +#endif == -1) { ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, @@ -784,14 +794,6 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } } -#else - if (execvp(file, (char * const *) argv) == -1) { - ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, - "lua pipe child execvp() failed while executing %s", - file); - } -#endif - exit(EXIT_FAILURE); } From 276f16af49f5dd9c85242bcfd879ce95b02555e8 Mon Sep 17 00:00:00 2001 From: Zhefeng Chen <38037704+catbro666@users.noreply.github.com> Date: Fri, 18 Jun 2021 18:01:40 +0800 Subject: [PATCH 520/848] =?UTF-8?q?feature:=20add=20lua=5Fssl=5Fconf=5Fcom?= =?UTF-8?q?mand=20directive=20for=20setting=20arbitrary=20Ope=E2=80=A6=20(?= =?UTF-8?q?#1890)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit add lua_ssl_conf_command directive for setting arbitrary OpenSSL configuration parameter particularly the TLSv1.3 ciphersuites --- README.markdown | 29 ++++ doc/HttpLuaModule.wiki | 27 ++++ src/ngx_http_lua_common.h | 3 + src/ngx_http_lua_module.c | 44 ++++++ t/129-ssl-socket.t | 272 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 375 insertions(+) diff --git a/README.markdown b/README.markdown index 0d91171657..7ecc5ae82e 100644 --- a/README.markdown +++ b/README.markdown @@ -1153,6 +1153,7 @@ Directives * [lua_ssl_protocols](#lua_ssl_protocols) * [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate) * [lua_ssl_verify_depth](#lua_ssl_verify_depth) +* [lua_ssl_conf_command](#lua_ssl_conf_command) * [lua_http10_buffering](#lua_http10_buffering) * [rewrite_by_lua_no_postpone](#rewrite_by_lua_no_postpone) * [access_by_lua_no_postpone](#access_by_lua_no_postpone) @@ -3138,6 +3139,34 @@ See also [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate). [Back to TOC](#directives) +lua_ssl_conf_command +-------------------- + +**syntax:** *lua_ssl_conf_command <command>* + +**default:** *no* + +**context:** *http, server, location* + +Sets arbitrary OpenSSL configuration [commands](https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html). + +The directive is supported when using OpenSSL 1.0.2 or higher and nginx 1.19.4 or higher. According to the specify command, higher OpenSSL version may be needed. + +Several `lua_ssl_conf_command` directives can be specified on the same level: + +``` +ssl_conf_command Options PrioritizeChaCha; +ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; +``` + +Configuration commands are applied after OpenResty own configuration for SSL, so they can be used to override anything set by OpenResty. + +Note though that configuring OpenSSL directly with `lua_ssl_conf_command` might result in a behaviour OpenResty does not expect, and should be done with care. + +This directive was first introduced in the `v0.10.21` release. + +[Back to TOC](#directives) + lua_http10_buffering -------------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 037ec3efa2..777958d183 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2664,6 +2664,33 @@ This directive was first introduced in the v0.9.11 release. See also [[#lua_ssl_trusted_certificate|lua_ssl_trusted_certificate]]. +== lua_ssl_conf_command == + +'''syntax:''' ''lua_ssl_conf_command '' + +'''default:''' ''no'' + +'''context:''' ''http, server, location'' + +Sets arbitrary OpenSSL configuration [https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html commands]. + +The directive is supported when using OpenSSL 1.0.2 or higher and nginx 1.19.4 or higher. According to the specify command, higher OpenSSL version may be needed. + +Several lua_ssl_conf_command directives can be specified on the same level: + + +ssl_conf_command Options PrioritizeChaCha; +ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; + + +Configuration commands are applied after OpenResty own configuration for SSL, so they can be used to override anything set by OpenResty. + +Note though that configuring OpenSSL directly with lua_ssl_conf_command might result in a behaviour OpenResty does not expect, and should be done with care. + +This directive was first introduced in the v0.10.21 release. + + + == lua_http10_buffering == '''syntax:''' ''lua_http10_buffering on|off'' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 0751a1123f..9a271e52ee 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -336,6 +336,9 @@ typedef struct { ngx_uint_t ssl_verify_depth; ngx_str_t ssl_trusted_certificate; ngx_str_t ssl_crl; +#if (nginx_version >= 1019004) + ngx_array_t *ssl_conf_commands; +#endif #endif ngx_flag_t force_read_body; /* whether force request body to diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 7358a95639..f1de894639 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -47,6 +47,10 @@ static char *ngx_http_lua_lowat_check(ngx_conf_t *cf, void *post, void *data); #if (NGX_HTTP_SSL) static ngx_int_t ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf); +#if (nginx_version >= 1019004) +static char *ngx_http_lua_ssl_conf_command_check(ngx_conf_t *cf, void *post, + void *data); +#endif #endif static char *ngx_http_lua_malloc_trim(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); @@ -73,6 +77,11 @@ static ngx_conf_bitmask_t ngx_http_lua_ssl_protocols[] = { { ngx_null_string, 0 } }; +#if (nginx_version >= 1019004) +static ngx_conf_post_t ngx_http_lua_ssl_conf_command_post = + { ngx_http_lua_ssl_conf_command_check }; +#endif + #endif @@ -620,6 +629,14 @@ static ngx_command_t ngx_http_lua_cmds[] = { offsetof(ngx_http_lua_loc_conf_t, ssl_crl), NULL }, +#if (nginx_version >= 1019004) + { ngx_string("lua_ssl_conf_command"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, + ngx_conf_set_keyval_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_lua_loc_conf_t, ssl_conf_commands), + &ngx_http_lua_ssl_conf_command_post }, +#endif #endif /* NGX_HTTP_SSL */ { ngx_string("lua_malloc_trim"), @@ -1252,6 +1269,9 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) #if (NGX_HTTP_SSL) conf->ssl_verify_depth = NGX_CONF_UNSET_UINT; +#if (nginx_version >= 1019004) + conf->ssl_conf_commands = NGX_CONF_UNSET_PTR; +#endif #endif return conf; @@ -1326,6 +1346,11 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); +#if (nginx_version >= 1019004) + ngx_conf_merge_ptr_value(conf->ssl_conf_commands, prev->ssl_conf_commands, + NULL); +#endif + if (ngx_http_lua_set_ssl(cf, conf) != NGX_OK) { return NGX_CONF_ERROR; } @@ -1419,9 +1444,28 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) return NGX_ERROR; } +#if (nginx_version >= 1019004) + if (ngx_ssl_conf_commands(cf, conf->ssl, conf->ssl_conf_commands) + != NGX_OK) { + return NGX_ERROR; + } +#endif + return NGX_OK; } +#if (nginx_version >= 1019004) +static char * +ngx_http_lua_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data) +{ +#ifndef SSL_CONF_FLAG_FILE + return "is not supported on this platform"; +#endif + + return NGX_CONF_OK; +} +#endif + #endif /* NGX_HTTP_SSL */ diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index fc2cc31f07..f3e1fed37e 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -2561,3 +2561,275 @@ qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including t --- no_error_log [alert] --- timeout: 10 + + +=== TEST 32: default cipher -TLSv1.3 +--- skip_openssl: 8: < 1.1.1 +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols TLSv1.3; + + location / { + content_by_lua_block { + ngx.exit(200) + } + } + } +--- config + server_tokens off; + lua_ssl_protocols TLSv1.3; + + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + do + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local session, err = sock:sslhandshake(nil, "test.com") + if not session then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(session)) + + local req = "GET / HTTP/1.1\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + local line, err = sock:receive() + if not line then + ngx.say("failed to receive response status line: ", err) + return + end + + ngx.say("received: ", line) + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 53 bytes. +received: HTTP/1.1 200 OK +close: 1 nil + +--- log_level: debug +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ +--- grep_error_log_out eval +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +$/ +--- error_log eval +['lua ssl server name: "test.com"', +qr/SSL: TLSv1.3, cipher: "TLS_AES_256_GCM_SHA384 TLSv1.3/] +--- no_error_log +SSL reused session +[error] +[alert] +--- timeout: 10 + + + +=== TEST 33: explicit cipher configuration - TLSv1.3 +--- skip_openssl: 8: < 1.1.1 +--- skip_nginx: 8: < 1.19.4 +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols TLSv1.3; + + location / { + content_by_lua_block { + ngx.exit(200) + } + } + } +--- config + server_tokens off; + lua_ssl_protocols TLSv1.3; + lua_ssl_conf_command Ciphersuites TLS_AES_128_GCM_SHA256; + + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + do + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local session, err = sock:sslhandshake(nil, "test.com") + if not session then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(session)) + + local req = "GET / HTTP/1.1\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + local line, err = sock:receive() + if not line then + ngx.say("failed to receive response status line: ", err) + return + end + + ngx.say("received: ", line) + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 53 bytes. +received: HTTP/1.1 200 OK +close: 1 nil + +--- log_level: debug +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ +--- grep_error_log_out eval +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +$/ +--- error_log eval +['lua ssl server name: "test.com"', +qr/SSL: TLSv1.3, cipher: "TLS_AES_128_GCM_SHA256 TLSv1.3/] +--- no_error_log +SSL reused session +[error] +[alert] +--- timeout: 10 + + + +=== TEST 34: explicit cipher configuration not in the default list - TLSv1.3 +--- skip_openssl: 8: < 1.1.1 +--- skip_nginx: 8: < 1.19.4 +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols TLSv1.3; + + location / { + content_by_lua_block { + ngx.exit(200) + } + } + } +--- config + server_tokens off; + lua_ssl_protocols TLSv1.3; + lua_ssl_conf_command Ciphersuites TLS_AES_128_CCM_SHA256; + + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + do + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local session, err = sock:sslhandshake(nil, "test.com") + if not session then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(session)) + + local req = "GET / HTTP/1.1\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + local line, err = sock:receive() + if not line then + ngx.say("failed to receive response status line: ", err) + return + end + + ngx.say("received: ", line) + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- log_level: debug +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ +--- grep_error_log_out +--- error_log eval +[ +qr/\[info\] .*?SSL_do_handshake\(\) failed .*?no shared cipher/, +'lua ssl server name: "test.com"', +] +--- no_error_log +SSL reused session +[alert] +[emerg] +--- timeout: 10 From 6c1a2e95b86150ebb89a473b0abbadac71e44cd5 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Sat, 19 Jun 2021 11:25:31 +0800 Subject: [PATCH 521/848] bugfix: fixed typo introduced by 276f16af. (#1891) --- src/ngx_http_lua_module.c | 2 +- t/129-ssl-socket.t | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index f1de894639..2faeb9eb03 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1445,7 +1445,7 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) } #if (nginx_version >= 1019004) - if (ngx_ssl_conf_commands(cf, conf->ssl, conf->ssl_conf_commands) + if (ngx_ssl_conf_commands(cf, llcf->ssl, llcf->ssl_conf_commands) != NGX_OK) { return NGX_ERROR; } diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index f3e1fed37e..8fd393d66a 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -6,7 +6,7 @@ use File::Basename; repeat_each(2); -plan tests => repeat_each() * 211; +plan tests => repeat_each() * blocks() * 7 - 6; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; From 223139d208c69cbe16ef6e1fb5472f4d3d75545a Mon Sep 17 00:00:00 2001 From: Zhefeng Chen <38037704+catbro666@users.noreply.github.com> Date: Sat, 19 Jun 2021 19:18:49 +0800 Subject: [PATCH 522/848] chore: fix typos introduced by PR #1890 (#1892) --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 7ecc5ae82e..0ab8e27ec6 100644 --- a/README.markdown +++ b/README.markdown @@ -3155,8 +3155,8 @@ The directive is supported when using OpenSSL 1.0.2 or higher and nginx 1.19.4 o Several `lua_ssl_conf_command` directives can be specified on the same level: ``` -ssl_conf_command Options PrioritizeChaCha; -ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; +lua_ssl_conf_command Options PrioritizeChaCha; +lua_ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; ``` Configuration commands are applied after OpenResty own configuration for SSL, so they can be used to override anything set by OpenResty. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 777958d183..e4fe192dd1 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2679,8 +2679,8 @@ The directive is supported when using OpenSSL 1.0.2 or higher and nginx 1.19.4 o Several lua_ssl_conf_command directives can be specified on the same level: -ssl_conf_command Options PrioritizeChaCha; -ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; +lua_ssl_conf_command Options PrioritizeChaCha; +lua_ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; Configuration commands are applied after OpenResty own configuration for SSL, so they can be used to override anything set by OpenResty. From bdd48a154c9b21013c294b366928457ef2a391b1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sun, 27 Jun 2021 12:26:18 +0800 Subject: [PATCH 523/848] bugfix: we didn't exclude the ssl_verify_callback when compiling with LibreSSL (#1894) --- src/ngx_http_lua_ssl_certby.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 1ee2d2882b..b561122b13 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1333,6 +1333,7 @@ ngx_http_lua_ffi_set_priv_key(ngx_http_request_t *r, } +#ifndef LIBRESSL_VERSION_NUMBER static int ngx_http_lua_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) { @@ -1344,6 +1345,7 @@ ngx_http_lua_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) */ return 1; } +#endif int From ccb0f8cc1087397278ea21ac094a8235658081ef Mon Sep 17 00:00:00 2001 From: doujiang Date: Wed, 30 Jun 2021 15:21:28 +0800 Subject: [PATCH 524/848] change: removed the openssl 1.0.2 in CI since it has been EOL ove a year, fix #1897. (#1899) --- .travis.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index feee2075b9..702c94e981 100644 --- a/.travis.yml +++ b/.travis.yml @@ -47,8 +47,6 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 jobs: - - NGINX_VERSION=1.17.8 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - - NGINX_VERSION=1.19.9 OPENSSL_VER=1.0.2u OPENSSL_PATCH_VER=1.0.2h - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1k OPENSSL_PATCH_VER=1.1.1f From 53bd821990d2d50af55d417fe5733a9648117f68 Mon Sep 17 00:00:00 2001 From: jiahao Date: Wed, 7 Jul 2021 16:41:28 +0800 Subject: [PATCH 525/848] travis-ci: remove clang compiler mode from travis. --- .travis.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 702c94e981..055fbe5685 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,7 +6,6 @@ language: c compiler: - gcc - - clang addons: apt: From 66a4d417d654404ade32fcc600ff1381e5e02f9c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8D=C3=B5=C3=91=C3=BE?= Date: Sun, 22 Nov 2020 18:45:37 -0800 Subject: [PATCH 526/848] add privileged agent connections --- src/ngx_http_lua_worker.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 8389dbeaec..d9331feedc 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -98,7 +98,7 @@ ngx_http_lua_ffi_get_process_type(void) int -ngx_http_lua_ffi_enable_privileged_agent(char **err) +ngx_http_lua_ffi_enable_privileged_agent(char **err, unsigned int connections) { #ifdef HAVE_PRIVILEGED_PROCESS_PATCH ngx_core_conf_t *ccf; @@ -107,6 +107,7 @@ ngx_http_lua_ffi_enable_privileged_agent(char **err) ngx_core_module); ccf->privileged_agent = 1; + ccf->privileged_agent_connections = connections; return NGX_OK; From b721656a9127255003b696b42ccc871c7ec18d59 Mon Sep 17 00:00:00 2001 From: wangyao Date: Mon, 19 Jul 2021 18:32:41 +0800 Subject: [PATCH 527/848] bugfix: add nginx version condition for privileged agent connections. (#1903) --- src/ngx_http_lua_worker.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index d9331feedc..0ca2d414e3 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -96,9 +96,13 @@ ngx_http_lua_ffi_get_process_type(void) return ngx_process; } - +#if defined(nginx_version) && nginx_version >= 1019003 int ngx_http_lua_ffi_enable_privileged_agent(char **err, unsigned int connections) +#else +int +ngx_http_lua_ffi_enable_privileged_agent(char **err) +#endif { #ifdef HAVE_PRIVILEGED_PROCESS_PATCH ngx_core_conf_t *ccf; @@ -107,7 +111,9 @@ ngx_http_lua_ffi_enable_privileged_agent(char **err, unsigned int connections) ngx_core_module); ccf->privileged_agent = 1; +#if defined(nginx_version) && nginx_version >= 1019003 ccf->privileged_agent_connections = connections; +#endif return NGX_OK; From 263bd0ca70e5dc3910871eb7eba5aba273db9cf6 Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Wed, 18 Aug 2021 10:24:01 +0800 Subject: [PATCH 528/848] bugfix: should not allow to create timer in the exit process phase (#1907) Since the exit process phase runs after all timers have been processed. --- README.markdown | 15 ++++++++++----- doc/HttpLuaModule.wiki | 6 ++++-- src/ngx_http_lua_timer.c | 9 +++++++++ t/162-exit-worker.t | 8 ++++---- 4 files changed, 27 insertions(+), 11 deletions(-) diff --git a/README.markdown b/README.markdown index 0ab8e27ec6..1eab5d4d97 100644 --- a/README.markdown +++ b/README.markdown @@ -1672,6 +1672,8 @@ For example, } ``` +It's not allowed to create a timer (even a 0-delay timer) here since it runs after all timers have been processed. + This directive was first introduced in the `v0.10.18` release. [Back to TOC](#directives) @@ -3154,9 +3156,10 @@ The directive is supported when using OpenSSL 1.0.2 or higher and nginx 1.19.4 o Several `lua_ssl_conf_command` directives can be specified on the same level: -``` -lua_ssl_conf_command Options PrioritizeChaCha; -lua_ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; +```nginx + + lua_ssl_conf_command Options PrioritizeChaCha; + lua_ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256; ``` Configuration commands are applied after OpenResty own configuration for SSL, so they can be used to override anything set by OpenResty. @@ -3165,6 +3168,8 @@ Note though that configuring OpenSSL directly with `lua_ssl_conf_command` might This directive was first introduced in the `v0.10.21` release. + + [Back to TOC](#directives) lua_http10_buffering @@ -8405,7 +8410,7 @@ ngx.timer.at **syntax:** *hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Creates an Nginx timer with a user callback function as well as optional user arguments. @@ -8548,7 +8553,7 @@ ngx.timer.every **syntax:** *hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** Similar to the [ngx.timer.at](#ngxtimerat) API function, but diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index e4fe192dd1..49353bb9d6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1362,6 +1362,8 @@ For example, } +It's not allowed to create a timer (even a 0-delay timer) here since it runs after all timers have been processed. + This directive was first introduced in the v0.10.18 release. == exit_worker_by_lua_file == @@ -7196,7 +7198,7 @@ See also [[#lua_check_client_abort|lua_check_client_abort]]. '''syntax:''' ''hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Creates an Nginx timer with a user callback function as well as optional user arguments. @@ -7334,7 +7336,7 @@ This API was first introduced in the v0.8.0 release. '''syntax:''' ''hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' Similar to the [[#ngx.timer.at|ngx.timer.at]] API function, but diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index cafce52cb6..fc6a3ae9a3 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -174,6 +174,15 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + /* + * Since nginx has been confirmed that all timers have been cleaned up when + * exit worker is executed, all timers will no longer be executed in exit + * worker phase. + * Reference https://github.com/nginx/nginx/blob/f02e2a734ef472f0dcf83ab2 + * e8ce96d1acead8a5/src/os/unix/ngx_process_cycle.c#L715 + */ + ngx_http_lua_check_context(L, ctx, ~NGX_HTTP_LUA_CONTEXT_EXIT_WORKER); + if (ngx_exiting && delay > 0) { lua_pushnil(L); lua_pushliteral(L, "process exiting"); diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index 8f2efc21e5..5d92c4fe1a 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -5,7 +5,7 @@ use Test::Nginx::Socket::Lua; master_on(); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 2) + 12; +plan tests => repeat_each() * (blocks() * 2 + 2) + 11; #log_level("warn"); no_long_string(); @@ -79,7 +79,7 @@ hello, world -=== TEST 4: exit_worker_by_lua single process ngx.timer not work +=== TEST 4: ngx.timer is not allow --- http_config exit_worker_by_lua_block { local function bar() @@ -101,8 +101,8 @@ hello, world GET /t --- response_body ok ---- no_error_log -[error] +--- shutdown_error_log +API disabled in the context of exit_worker_by_lua* From 89818721fefa3bac9aec99dd622ee4547a9794d9 Mon Sep 17 00:00:00 2001 From: kingluo Date: Thu, 15 Jul 2021 21:26:43 +0800 Subject: [PATCH 529/848] feature: implement the new ngx.run_worker_thread API to run Lua function in a seperated worker thread. --- README.markdown | 149 ++++++++ config | 2 + doc/HttpLuaModule.wiki | 136 +++++++ src/ngx_http_lua_common.h | 2 + src/ngx_http_lua_module.c | 13 + src/ngx_http_lua_util.c | 6 + src/ngx_http_lua_worker_thread.c | 531 ++++++++++++++++++++++++++ src/ngx_http_lua_worker_thread.h | 20 + t/062-count.t | 8 +- t/166-worker-thread.t | 629 +++++++++++++++++++++++++++++++ util/build.sh | 1 + 11 files changed, 1493 insertions(+), 4 deletions(-) create mode 100644 src/ngx_http_lua_worker_thread.c create mode 100644 src/ngx_http_lua_worker_thread.h create mode 100644 t/166-worker-thread.t diff --git a/README.markdown b/README.markdown index 1eab5d4d97..3295177b6d 100644 --- a/README.markdown +++ b/README.markdown @@ -1162,6 +1162,7 @@ Directives * [lua_max_pending_timers](#lua_max_pending_timers) * [lua_max_running_timers](#lua_max_running_timers) * [lua_sa_restart](#lua_sa_restart) +* [lua_worker_thread_vm_pool_size](#lua_worker_thread_vm_pool_size) The basic building blocks of scripting Nginx with Lua are directives. Directives are used to specify when the user Lua code is run and @@ -3328,6 +3329,25 @@ This directive was first introduced in the `v0.10.14` release. [Back to TOC](#directives) +lua_worker_thread_vm_pool_size +------------------------------ + +**syntax:** *lua_worker_thread_vm_pool_size <size>* + +**default:** *lua_worker_thread_vm_pool_size 100* + +**context:** *http* + +Specifies the size limit of the Lua VM pool (default 100) that will be used in the [ngx.run_worker_thread](#ngxrun_worker_thread) API. + +Also, it is not allowed to create Lua VMs that exceeds the pool size limit. + +The Lua VM in the VM pool is used to execute Lua code in seperate thread. + +The pool is global at Nginx worker level. And it is used to reuse Lua VMs between requests. + +[Back to TOC](#directives) + Nginx API for Lua ================= @@ -3480,6 +3500,7 @@ Nginx API for Lua * [coroutine.wrap](#coroutinewrap) * [coroutine.running](#coroutinerunning) * [coroutine.status](#coroutinestatus) +* [ngx.run_worker_thread](#ngxrun_worker_thread) [Back to TOC](#table-of-contents) @@ -8960,6 +8981,134 @@ This API was first enabled in the `v0.6.0` release. [Back to TOC](#nginx-api-for-lua) +ngx.run_worker_thread +--------------------- + +**syntax:** *ok, res1, res2, ... = ngx.run_worker_thread(threadpool, module_name, func_name, arg1, arg2, ...)* + +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** + +**This API is still experimental and may change in the future without notice.** + +**This API is available only for Linux.** + +Wrap the [nginx worker thread](http://nginx.org/en/docs/dev/development_guide.html#threads) to execute lua function. The caller coroutine would yield until the function returns. + +Note that no ngx_lua API can be used in the `function_name` function of the `module` module since it is invoked in a separate thread. + +The first argument `threadpool` specifies the Nginx thread pool name defined by [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool). + +The second argument `module_name` specifies the lua module name to execute in the worker thread, which would return a lua table. The module must be inside the package path, e.g. + +```nginx + + lua_package_path '/opt/openresty/?.lua;;'; +``` + +The third argument `func_name` specifies the function field in the module table as the second argument. + +The type of `arg`s must be one of type below: + +* boolean +* number +* string +* nil +* table (the table may be recursive, and contains members of types above.) + +The `ok` is in boolean type, which indicate the C land error (failed to get thread from thread pool, pcall the module function failed, .etc). If `ok` is `false`, the `res1` is the error string. + +The return values (res1, ...) are returned by invocation of the module function. Normally, the `res1` should be in boolean type, so that the caller could inspect the error. + +This API is useful when you need to execute the below types of tasks: + +* CPU bound task, e.g. do md5 calculation +* File I/O task +* Call `os.execute()` or blocking C API via `ffi` +* Call external Lua library not based on cosocket or nginx + +Example1: do md5 calculation. + +```nginx + + location /calc_md5 { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ret, md5_or_err = ngx.run_worker_thread("testpool", "calc_md5", "md5", ngx.var.arg_str) + if not ok then + ngx.say(ret) + return + end + if not ret then + ngx.say(md5_or_err) + return + end + ngx.say(md5_or_err) + } + } +``` + +`calc_md5.lua` + +```lua + + local resty_md5 = require "resty.md5" + local resty_str = require "resty.string" + + local function md5(str) + local md5 = resty_md5:new() + if not md5 then + return false, "md5 new error" + end + + local ok = md5:update(str) + if not ok then + return false, "md5 update error" + end + + local digest = md5:final() + return true, resty_str.to_hex(digest) + end + return {md5=md5} +``` + +Example2: write logs into the log file. + +```nginx + + location /write_log_file { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool", "write_log_file", "log", ngx.var.arg_str) + if not ok then + ngx.say(ok, " : ", err) + return + end + ngx.say(ok) + } + } +``` + +`write_log_file.lua` + +```lua + + local function log(str) + local file, err = io.open("/tmp/tmp.log", "a") + if not file then + return false, err + end + file:write(str) + file:flush() + file:close() + return true + end + return {log=log} +``` + +[Back to TOC](#nginx-api-for-lua) + Obsolete Sections ================= diff --git a/config b/config index 4b32d38318..4763949fc0 100644 --- a/config +++ b/config @@ -296,6 +296,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.c \ $ngx_addon_dir/src/ngx_http_lua_input_filters.c \ $ngx_addon_dir/src/ngx_http_lua_pipe.c \ + $ngx_addon_dir/src/ngx_http_lua_worker_thread.c \ " HTTP_LUA_DEPS=" \ @@ -355,6 +356,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.h \ $ngx_addon_dir/src/ngx_http_lua_input_filters.h \ $ngx_addon_dir/src/ngx_http_lua_pipe.h \ + $ngx_addon_dir/src/ngx_http_lua_worker_thread.h \ " # ---------------------------------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 49353bb9d6..4b9bd1eb03 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2824,6 +2824,22 @@ This allows Lua I/O primitives to not be interrupted by Nginx's handling of vari This directive was first introduced in the v0.10.14 release. +== lua_worker_thread_vm_pool_size == + +'''syntax:''' ''lua_worker_thread_vm_pool_size '' + +'''default:''' ''lua_worker_thread_vm_pool_size 100'' + +'''context:''' ''http'' + +Specifies the size limit of the Lua VM pool (default 100) that will be used in the [ngx.run_worker_thread](#ngxrun_worker_thread) API. + +Also, it is not allowed to create Lua VMs that exceeds the pool size limit. + +The Lua VM in the VM pool is used to execute Lua code in seperate thread. + +The pool is global at Nginx worker level. And it is used to reuse Lua VMs between requests. + = Nginx API for Lua = @@ -7671,6 +7687,126 @@ This API was first usable in the context of [[#init_by_lua|init_by_lua*]] since This API was first enabled in the v0.6.0 release. +== ngx.run_worker_thread == + +'''syntax:''' ''ok, res1, res2, ... = ngx.run_worker_thread(threadpool, module_name, func_name, arg1, arg2, ...)'' + +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' + +'''This API is still experimental and may change in the future without notice.''' + +'''This API is available only for Linux.''' + +Wrap the [http://nginx.org/en/docs/dev/development_guide.html#threads nginx worker thread] to execute lua function. The caller coroutine would yield until the function returns. + +Note that no ngx_lua API can be used in the `function_name` function of the `module` module since it is invoked in a separate thread. + +The first argument `threadpool` specifies the Nginx thread pool name defined by [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool). + +The second argument module_name specifies the lua module name to execute in the worker thread, which would return a lua table. The module must be inside the package path, e.g. + + +lua_package_path '/opt/openresty/?.lua;;'; + + +The third argument func_name specifies the function field in the module table as the second argument. + +The type of args must be one of type below: + +* boolean +* number +* string +* nil +* table (the table may be recursive, and contains members of types above.) + +The ok is in boolean type, which indicate the C land error (failed to get thread from thread pool, pcall the module function failed, .etc). If ok is false, the res1 is the error string. + +The return values (res1, ...) are returned by invocation of the module function. Normally, the res1 should be in boolean type, so that the caller could inspect the error. + +This API is useful when you need to execute the below types of tasks: + +* CPU bound task, e.g. do md5 calculation +* File I/O task +* Call os.execute() or blocking C API via ffi +* Call external Lua library not based on cosocket or nginx + +Example1: do md5 calculation. + + +location /calc_md5 { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ret, md5_or_err = ngx.run_worker_thread("testpool", "calc_md5", "md5", ngx.var.arg_str) + if not ok then + ngx.say(ret) + return + end + if not ret then + ngx.say(md5_or_err) + return + end + ngx.say(md5_or_err) + } +} + + +calc_md5.lua + + +local resty_md5 = require "resty.md5" +local resty_str = require "resty.string" + +local function md5(str) + local md5 = resty_md5:new() + if not md5 then + return false, "md5 new error" + end + + local ok = md5:update(str) + if not ok then + return false, "md5 update error" + end + + local digest = md5:final() + return true, resty_str.to_hex(digest) +end +return {md5=md5} + + +Example2: write logs into the log file. + + +location /write_log_file { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool", "write_log_file", "log", ngx.var.arg_str) + if not ok then + ngx.say(ok, " : ", err) + return + end + ngx.say(ok) + } +} + + +write_log_file.lua + + +local function log(str) + local file, err = io.open("/tmp/tmp.log", "a") + if not file then + return false, err + end + file:write(str) + file:flush() + file:close() + return true +end +return {log=log} + + = Obsolete Sections = This section is just holding obsolete documentation sections that have been either renamed or removed so that existing links over the web are still valid. diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 9a271e52ee..027ff71044 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -288,6 +288,8 @@ struct ngx_http_lua_main_conf_s { ngx_queue_t free_lua_threads; /* of ngx_http_lua_thread_ref_t */ ngx_queue_t cached_lua_threads; /* of ngx_http_lua_thread_ref_t */ + ngx_uint_t worker_thread_vm_pool_size; + unsigned requires_header_filter:1; unsigned requires_body_filter:1; unsigned requires_capture_filter:1; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 2faeb9eb03..79c7e35766 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -646,6 +646,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, NULL }, + { ngx_string("lua_worker_thread_vm_pool_size"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, + ngx_conf_set_num_slot, + NGX_HTTP_MAIN_CONF_OFFSET, + offsetof(ngx_http_lua_main_conf_t, worker_thread_vm_pool_size), + NULL }, + ngx_null_command }; @@ -975,6 +982,8 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) return NULL; } + lmcf->worker_thread_vm_pool_size = NGX_CONF_UNSET; + dd("nginx Lua module main config structure initialized!"); return lmcf; @@ -1058,6 +1067,10 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) } #endif + if (lmcf->worker_thread_vm_pool_size == NGX_CONF_UNSET_UINT) { + lmcf->worker_thread_vm_pool_size = 100; + } + return NGX_CONF_OK; } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index f8630efbe4..1a04bf025a 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -46,6 +46,9 @@ #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_ssl.h" #include "ngx_http_lua_log_ringbuf.h" +#if (NGX_THREADS) +#include "ngx_http_lua_worker_thread.h" +#endif #if 1 @@ -842,6 +845,9 @@ ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_http_lua_inject_uthread_api(log, L); ngx_http_lua_inject_timer_api(L); ngx_http_lua_inject_config_api(L); +#if (NGX_THREADS) + ngx_http_lua_inject_worker_thread_api(log, L); +#endif lua_getglobal(L, "package"); /* ngx package */ lua_getfield(L, -1, "loaded"); /* ngx package loaded */ diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c new file mode 100644 index 0000000000..92bb77d4cd --- /dev/null +++ b/src/ngx_http_lua_worker_thread.c @@ -0,0 +1,531 @@ +/* + * Copyright (C) Yichun Zhang (agentzh) + * Copyright (C) Jinhua Luo (kingluo) + * I hereby assign copyright in this code to the lua-nginx-module project, + * to be licensed under the same terms as the rest of the code. + */ + + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#include "ngx_http_lua_worker_thread.h" +#include "ngx_http_lua_util.h" + + +#if (NGX_THREADS) + + +#include +#include + + +typedef struct ngx_http_lua_task_ctx_s { + lua_State *vm; + struct ngx_http_lua_task_ctx_s *next; +} ngx_http_lua_task_ctx_t; + + +typedef struct { + ngx_http_lua_task_ctx_t *ctx; + ngx_http_lua_co_ctx_t *wait_co_ctx; + int n_args; + int rc; + int is_abort:1; +} ngx_http_lua_worker_thread_ctx_t; + + +static ngx_http_lua_task_ctx_t dummy_ctx; +static ngx_http_lua_task_ctx_t *ctxpool = &dummy_ctx; +static ngx_uint_t worker_thread_vm_count; + + +/* + * Re-implement ngx_thread_task_alloc to avoid alloc from request pool + * since the request may exit before worker thread finish. + * And we may implement a memory pool for this allocation in the future + * to avoid memory fragmentation. + */ +static ngx_thread_task_t * +ngx_http_lua_thread_task_alloc(size_t size) +{ + ngx_thread_task_t *task; + + task = ngx_calloc(sizeof(ngx_thread_task_t) + size, ngx_cycle->log); + if (task == NULL) { + return NULL; + } + + task->ctx = task + 1; + + return task; +} + + +static void +ngx_http_lua_thread_task_free(void *ctx) +{ + ngx_thread_task_t *task = ctx; + ngx_free(task - 1); +} + + +static ngx_http_lua_task_ctx_t * +ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) +{ + ngx_http_lua_task_ctx_t *ctx = NULL; + + size_t path_len; + const char *path; + size_t cpath_len; + const char *cpath; + lua_State *vm; + + ngx_http_lua_main_conf_t *lmcf; + + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + + if (ctxpool->next == NULL) { + if (worker_thread_vm_count >= lmcf->worker_thread_vm_pool_size) { + return NULL; + } + + ctx = ngx_calloc(sizeof(ngx_http_lua_task_ctx_t), ngx_cycle->log); + if (ctx == NULL) { + return NULL; + } + + vm = luaL_newstate(); + + if (vm == NULL) { + ngx_free(ctx); + return NULL; + } + + worker_thread_vm_count++; + + ctx->vm = vm; + + luaL_openlibs(vm); + + /* copy package.path and package.cpath */ + lua_getglobal(L, "package"); + lua_getfield(L, -1, "path"); + path = lua_tolstring(L, -1, &path_len); + lua_getfield(L, -2, "cpath"); + cpath = lua_tolstring(L, -1, &cpath_len); + + lua_getglobal(vm, "package"); + lua_pushlstring(vm, path, path_len); + lua_setfield(vm, -2, "path"); + lua_pushlstring(vm, cpath, cpath_len); + lua_setfield(vm, -2, "cpath"); + + /* pop path, cpath and "package" table from L */ + lua_pop(L, 3); + + /* pop the "package" table */ + lua_pop(vm, 1); + + } else { + ctx = ctxpool->next; + ctxpool->next = ctx->next; + ctx->next = NULL; + } + + return ctx; +} + + +static void +ngx_http_lua_free_task_ctx(ngx_http_lua_task_ctx_t *ctx) +{ + ctx->next = ctxpool->next; + ctxpool->next = ctx; + + /* clean Lua stack */ + lua_settop(ctx->vm, 0); +} + + +static int +ngx_http_lua_xcopy(lua_State *from, lua_State *to, int idx, + const int allow_nil) +{ + size_t len = 0; + const char *str; + int typ; + int top_from, top_to; + + typ = lua_type(from, idx); + switch (typ) { + case LUA_TBOOLEAN: + lua_pushboolean(to, lua_toboolean(from, idx)); + return LUA_TBOOLEAN; + + case LUA_TLIGHTUSERDATA: + lua_pushlightuserdata(to, lua_touserdata(from, idx)); + return LUA_TLIGHTUSERDATA; + + case LUA_TNUMBER: + lua_pushnumber(to, lua_tonumber(from, idx)); + return LUA_TNUMBER; + + case LUA_TSTRING: + str = lua_tolstring(from, idx, &len); + lua_pushlstring(to, str, len); + return LUA_TSTRING; + + case LUA_TTABLE: + top_from = lua_gettop(from); + top_to = lua_gettop(to); + + lua_newtable(to); + + /* to positive number */ + if (idx < 0) { + idx = lua_gettop(from) + idx + 1; + } + + lua_pushnil(from); + + while (lua_next(from, idx) != 0) { + if (ngx_http_lua_xcopy(from, to, -2, 0) != LUA_TNONE + && ngx_http_lua_xcopy(from, to, -1, 0) != LUA_TNONE) + { + lua_rawset(to, -3); + + } else { + lua_settop(from, top_from); + lua_settop(to, top_to); + return LUA_TNONE; + } + + lua_pop(from, 1); + } + + return LUA_TTABLE; + + case LUA_TNIL: + if (allow_nil) { + lua_pushnil(to); + return LUA_TNIL; + } + /* fall through */ + + /* + * ignore unsupported values: + * LUA_TNONE + * LUA_TFUNCTION + * LUA_TUSERDATA + * LUA_TTHREAD + */ + default: + return LUA_TNONE; + } +} + + +/* executed in a separate thread */ +static void +ngx_http_lua_worker_thread_handler(void *data, ngx_log_t *log) +{ + ngx_http_lua_worker_thread_ctx_t *ctx = data; + lua_State *vm = ctx->ctx->vm; + + ngx_http_lua_assert(lua_gettop(vm) == ctx->n_args); + + ctx->rc = lua_pcall(vm, ctx->n_args, LUA_MULTRET, 0); +} + + +/* executed in nginx event loop */ +static void +ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) +{ + ngx_http_lua_worker_thread_ctx_t *worker_thread_ctx; + lua_State *L; + ngx_http_request_t *r; + ngx_connection_t *c; + int nresults; + size_t len; + const char *str; + int i; + int rc; + ngx_http_lua_ctx_t *ctx; + lua_State *vm; + int saved_top; + + worker_thread_ctx = ev->data; + + if (worker_thread_ctx->is_abort) { + goto failed; + } + + L = worker_thread_ctx->wait_co_ctx->co; + + r = ngx_http_lua_get_req(L); + if (r == NULL) { + goto failed; + } + + c = r->connection; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + goto failed; + } + + vm = worker_thread_ctx->ctx->vm; + + if (worker_thread_ctx->rc != 0) { + str = lua_tolstring(vm, 1, &len); + lua_pushboolean(L, 0); + lua_pushlstring(L, str, len); + nresults = 2; + + } else { + /* copying return values */ + saved_top = lua_gettop(L); + lua_pushboolean(L, 1); + nresults = lua_gettop(vm) + 1; + for (i = 1; i < nresults; i++) { + if (ngx_http_lua_xcopy(vm, L, i, 1) == LUA_TNONE) { + lua_settop(L, saved_top); + lua_pushboolean(L, 0); + lua_pushstring(L, "unsupported return value"); + nresults = 2; + break; + } + } + } + + ctx->cur_co_ctx = worker_thread_ctx->wait_co_ctx; + ctx->cur_co_ctx->cleanup = NULL; + + ngx_http_lua_free_task_ctx(worker_thread_ctx->ctx); + ngx_http_lua_thread_task_free(worker_thread_ctx); + + /* resume the caller coroutine */ + + vm = ngx_http_lua_get_lua_vm(r, ctx); + + rc = ngx_http_lua_run_thread(vm, r, ctx, nresults); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua run thread returned %d", rc); + + if (rc == NGX_AGAIN) { + ngx_http_lua_run_posted_threads(c, vm, r, ctx, c->requests); + return; + } + + if (rc == NGX_DONE) { + ngx_http_lua_finalize_request(r, NGX_DONE); + ngx_http_lua_run_posted_threads(c, vm, r, ctx, c->requests); + return; + } + + /* rc == NGX_ERROR || rc >= NGX_OK */ + + if (ctx->entered_content_phase) { + ngx_http_lua_finalize_request(r, rc); + return; + } + + return; + +failed: + + ngx_http_lua_free_task_ctx(worker_thread_ctx->ctx); + ngx_http_lua_thread_task_free(worker_thread_ctx); + return; +} + + +static void +ngx_http_lua_worker_thread_cleanup(void *data) +{ + ngx_http_lua_co_ctx_t *ctx = data; + ngx_http_lua_worker_thread_ctx_t *worker_thread_ctx = ctx->data; + worker_thread_ctx->is_abort = 1; +} + + +/* It's not easy to use pure ffi here, using the Lua C API for now. */ +static int +ngx_http_lua_run_worker_thread(lua_State *L) +{ + ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + int n_args; + ngx_str_t thread_pool_name; + ngx_thread_pool_t *thread_pool; + ngx_http_lua_task_ctx_t *tctx; + lua_State *vm; + size_t len; + const char *mod_name; + const char *func_name; + int rc; + const char *err; + int i; + ngx_thread_task_t *task; + ngx_http_lua_worker_thread_ctx_t *worker_thread_ctx; + + r = ngx_http_lua_get_req(L); + if (r == NULL) { + return luaL_error(L, "no request found"); + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return luaL_error(L, "no ctx found"); + } + + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); + + n_args = lua_gettop(L); + + if (n_args < 3) { + lua_pushboolean(L, 0); + lua_pushstring(L, "expecting at least 3 arguments"); + return 2; + } + + thread_pool_name.data = (u_char *) + lua_tolstring(L, 1, &thread_pool_name.len); + + if (thread_pool_name.data == NULL) { + lua_pushboolean(L, 0); + lua_pushstring(L, "threadpool should be a string"); + return 2; + } + + thread_pool = ngx_thread_pool_get((ngx_cycle_t *) ngx_cycle, + &thread_pool_name); + + if (thread_pool == NULL) { + lua_pushboolean(L, 0); + lua_pushfstring(L, "thread pool %s not found", thread_pool_name.data); + return 2; + } + + mod_name = lua_tolstring(L, 2, &len); + if (mod_name == NULL) { + lua_pushboolean(L, 0); + lua_pushstring(L, "module name should be a string"); + return 2; + } + + func_name = lua_tolstring(L, 3, NULL); + if (func_name == NULL) { + lua_pushboolean(L, 0); + lua_pushstring(L, "function name should be a string"); + return 2; + } + + /* get vm */ + tctx = ngx_http_lua_get_task_ctx(L, r); + if (tctx == NULL) { + lua_pushboolean(L, 0); + lua_pushstring(L, "no available Lua vm"); + return 2; + } + + vm = tctx->vm; + + ngx_http_lua_assert(lua_gettop(vm) == 0); + + /* push function from module require */ + lua_getfield(vm, LUA_GLOBALSINDEX, "require"); + lua_pushlstring(vm, mod_name, len); + rc = lua_pcall(vm, 1, 1, 0); + + if (rc != 0) { + err = lua_tolstring(vm, 1, &len); + lua_pushboolean(L, 0); + lua_pushlstring(L, err, len); + ngx_http_lua_free_task_ctx(tctx); + return 2; + } + + if (!lua_istable(vm, -1)) { + lua_pushboolean(L, 0); + lua_pushstring(L, "invalid lua module"); + ngx_http_lua_free_task_ctx(tctx); + return 2; + } + + lua_getfield(vm, -1, func_name); + if (!lua_isfunction(vm, -1)) { + lua_pushboolean(L, 0); + lua_pushstring(L, "invalid function"); + ngx_http_lua_free_task_ctx(tctx); + return 2; + } + + /* remove the table returned by require */ + lua_remove(vm, 1); + + /* copying passed arguments */ + for (i = 4; i <= n_args; i++) { + if (ngx_http_lua_xcopy(L, vm, i, 1) == LUA_TNONE) { + lua_pushboolean(L, 0); + lua_pushstring(L, "unsupported argument type"); + ngx_http_lua_free_task_ctx(tctx); + return 2; + } + } + + /* post task */ + task = ngx_http_lua_thread_task_alloc( + sizeof(ngx_http_lua_worker_thread_ctx_t)); + + if (task == NULL) { + ngx_http_lua_free_task_ctx(tctx); + lua_pushboolean(L, 0); + lua_pushstring(L, "no memory"); + return 2; + } + + worker_thread_ctx = task->ctx; + + worker_thread_ctx->ctx = tctx; + worker_thread_ctx->wait_co_ctx = ctx->cur_co_ctx; + + ctx->cur_co_ctx->cleanup = ngx_http_lua_worker_thread_cleanup; + ctx->cur_co_ctx->data = worker_thread_ctx; + + worker_thread_ctx->n_args = n_args - 3; + worker_thread_ctx->rc = 0; + worker_thread_ctx->is_abort = 0; + + task->handler = ngx_http_lua_worker_thread_handler; + task->event.handler = ngx_http_lua_worker_thread_event_handler; + task->event.data = worker_thread_ctx; + + if (ngx_thread_task_post(thread_pool, task) != NGX_OK) { + ngx_http_lua_free_task_ctx(tctx); + ngx_http_lua_thread_task_free(task); + lua_pushboolean(L, 0); + lua_pushstring(L, "ngx_thread_task_post failed"); + return 2; + } + + return lua_yield(L, 0); +} + + +void +ngx_http_lua_inject_worker_thread_api(ngx_log_t *log, lua_State *L) +{ + lua_pushcfunction(L, ngx_http_lua_run_worker_thread); + lua_setfield(L, -2, "run_worker_thread"); +} + +#endif + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_worker_thread.h b/src/ngx_http_lua_worker_thread.h new file mode 100644 index 0000000000..d08fb903a7 --- /dev/null +++ b/src/ngx_http_lua_worker_thread.h @@ -0,0 +1,20 @@ +/* + * Copyright (C) Yichun Zhang (agentzh) + * Copyright (C) Jinhua Luo (kingluo) + * I hereby assign copyright in this code to the lua-nginx-module project, + * to be licensed under the same terms as the rest of the code. + */ + +#ifndef _NGX_HTTP_LUA_WORKER_THREAD_H_INCLUDED_ +#define _NGX_HTTP_LUA_WORKER_THREAD_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +void ngx_http_lua_inject_worker_thread_api(ngx_log_t *log, lua_State *L); + + +#endif /* _NGX_HTTP_LUA_WORKER_THREAD_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/062-count.t b/t/062-count.t index 807e618e21..598441c513 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -34,7 +34,7 @@ __DATA__ --- request GET /test --- response_body -ngx: 113 +ngx: 114 --- no_error_log [error] @@ -55,7 +55,7 @@ ngx: 113 --- request GET /test --- response_body -113 +114 --- no_error_log [error] @@ -83,7 +83,7 @@ GET /test --- request GET /test --- response_body -n = 113 +n = 114 --- no_error_log [error] @@ -305,7 +305,7 @@ GET /t --- response_body_like: 404 Not Found --- error_code: 404 --- error_log -ngx. entry count: 113 +ngx. entry count: 114 diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t new file mode 100644 index 0000000000..4037615b46 --- /dev/null +++ b/t/166-worker-thread.t @@ -0,0 +1,629 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_on(); +#workers(2); +#log_level('warn'); + +repeat_each(1); + +plan tests => repeat_each() * (blocks() * 2); + +our $HtmlDir = html_dir; + +our $HttpConfig = qq{ + lua_package_path "$::HtmlDir/?.lua;./?.lua;;"; + lua_worker_thread_vm_pool_size 1; +}; + +#no_diff(); +#no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: hello from worker thread +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : hello + + + +=== TEST 2: thread_pool not found +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +false : thread pool testpool not found + + + +=== TEST 3: pass table +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ok_or_err = ngx.run_worker_thread("testpool", "hello", "hello", {["hello"]="world", [1]={["embed"]=1}}) + ngx.say(ok, " , ", ok_or_err) + } +} +--- user_files +>>> hello.lua +local function hello(arg1) + if arg1.hello == "world" and arg1[1].embed == 1 then + return true + end + return false +end +return {hello=hello} +--- request +GET /hello +--- response_body +true , true + + + +=== TEST 4: expecting at least 3 arguments +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool") + ngx.say(ok, " : ", err) + } +} +--- request +GET /hello +--- response_body +false : expecting at least 3 arguments + + + +=== TEST 5: base64 +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, base64 = ngx.run_worker_thread("testpool", "hello", "enc", "hello") + ngx.say(ok, " , ", base64 == "aGVsbG8=") + } +} +--- user_files +>>> hello.lua +local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/' + +local function enc(data) + return ((data:gsub('.', function(x) + local r,b='',x:byte() + for i=8,1,-1 do r=r..(b%2^i-b%2^(i-1)>0 and '1' or '0') end + return r; + end)..'0000'):gsub('%d%d%d?%d?%d?%d?', function(x) + if (#x < 6) then return '' end + local c=0 + for i=1,6 do c=c+(x:sub(i,i)=='1' and 2^(6-i) or 0) end + return b:sub(c+1,c+1) + end)..({ '', '==', '=' })[#data%3+1]) +end + +return {enc=enc} +--- request +GET /hello +--- response_body +true , true + + + +=== TEST 6: return table +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ret = ngx.run_worker_thread("testpool", "hello", "hello") + if ret.hello == "world" and ret[1].embed == 1 then + ngx.say(ok, " , ", true) + end + } +} +--- user_files +>>> hello.lua +local function hello() + return {["hello"]="world", [1]={["embed"]=1}} +end +return {hello=hello} +--- request +GET /hello +--- response_body +true , true + + + +=== TEST 7: unsupported argument type +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local function dummy() end + local ok, err = ngx.run_worker_thread("testpool", "hello", "hello", dummy) + ngx.say(ok, " : ", err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +false : unsupported argument type + + + +=== TEST 8: multiple return values +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, res1, res2 = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", res1, " , ", res2) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello", 200 +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : hello , 200 + + + +=== TEST 9: access ngx.* api +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", err) + } +} +--- user_files +>>> hello.lua +local function hello() + ngx.sleep(1) + return "ok" +end +return {hello=hello} +--- request +GET /hello +--- response_body_like +false : .*attempt to index global 'ngx' \(a nil value\) + + + +=== TEST 10: module not found +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", err) + } +} +--- request +GET /hello +--- response_body_like +false : module 'hello' not found.* + + + +=== TEST 11: the number of Lua VM exceeds the pool size +--- main_config + thread_pool testpool threads=100; +--- http_config eval: $::HttpConfig +--- config +location /foo { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} + +location /bar { + default_type 'text/plain'; + + content_by_lua_block { + local ok, foobar_or_err = ngx.run_worker_thread("testpool", "foobar", "foobar") + ngx.say(ok, " : ", foobar_or_err) + } +} + +location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local function t(path) + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local req = "GET " .. path .. " HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + local ret, err, part = sock:receive("*a") + local _, idx = string.find(ret, "\r\n\r\n"); + idx = idx + 1 + ngx.print(string.sub(ret, idx)) + ok, err = sock:close() + end + + local t1 = ngx.thread.spawn(t, "/foo") + local t2 = ngx.thread.spawn(t, "/bar") + ngx.thread.wait(t1) + ngx.thread.wait(t2) + } +} +--- user_files +>>> hello.lua +local function hello() + os.execute("sleep 3") + return "hello" +end +return {hello=hello} +>>> foobar.lua +local function foobar() + return "foobar" +end +return {foobar=foobar} +--- request +GET /t +--- response_body eval +"false : no available Lua vm\ntrue : hello\n" +--- timeout: 10 + + + +=== TEST 12: kill uthread before worker thread callback +--- main_config + thread_pool testpool threads=100; +--- http_config eval: $::HttpConfig +--- config +location /foo { + default_type 'text/plain'; + + content_by_lua_block { + local function t() + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + end + local t1 = ngx.thread.spawn(t) + if ngx.var.arg_kill == "kill" then + ngx.thread.kill(t1) + ngx.say("killed") + end + } +} + +location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local function t(path) + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local req = "GET " .. path .. " HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + local ret, err, part = sock:receive("*a") + local _, idx = string.find(ret, "\r\n\r\n"); + idx = idx + 1 + ngx.print(string.sub(ret, idx)) + ok, err = sock:close() + end + + local t1 = ngx.thread.spawn(t, "/foo?kill=kill") + ngx.thread.wait(t1) + ngx.sleep(4) + local t2 = ngx.thread.spawn(t, "/foo") + ngx.thread.wait(t2) + } +} +--- user_files +>>> hello.lua +local function hello() + os.execute("sleep 1") + return "hello" +end +return {hello=hello} +>>> foobar.lua +local function foobar() + return "foobar" +end +return {foobar=foobar} +--- request +GET /t +--- response_body eval +"killed\ntrue : hello\n" +--- timeout: 10 + + + +=== TEST 13: exit before worker thread callback +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local function t() + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + end + ngx.thread.spawn(t) + ngx.exit(200) + } +} +--- user_files +>>> hello.lua +local function hello() + os.execute("sleep 3") + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +--- timeout: 10 + + + +=== TEST 14: unsupported argument type in nested table +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local function dummy() end + local ok, err = ngx.run_worker_thread("testpool", "hello", "hello", + {["hello"]="world", [1]={["embed"]=1, ["dummy"]=dummy}}) + ngx.say(ok, " : ", err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +false : unsupported argument type + + + +=== TEST 15: return table with unsupported type +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ret = ngx.run_worker_thread("testpool", "hello", "hello") + if ok == false then + ngx.say("false", " , ", ret) + end + if ret.hello == "world" and ret[1].embed == 1 then + ngx.say(ok, " , ", true) + end + } +} +--- user_files +>>> hello.lua +local function hello() + local function dummy() end + return {["hello"]="world", [1]={["embed"]=1, ["dummy"]=dummy}} +end +return {hello=hello} +--- request +GET /hello +--- response_body +false , unsupported return value + + + +=== TEST 16: the type of module name is not string +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local function dummy() end + local ok, err = ngx.run_worker_thread("testpool", dummy, "hello") + ngx.say(ok, " : ", err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +false : module name should be a string + + + +=== TEST 17: the type of function name is not string +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local function dummy() end + local ok, err = ngx.run_worker_thread("testpool", "hello", dummy) + ngx.say(ok, " : ", err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +false : function name should be a string + + + +=== TEST 18: the type of thread pool name is not string +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local function dummy() end + local ok, err = ngx.run_worker_thread(dummy, "hello", "hello") + ngx.say(ok, " : ", err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +false : threadpool should be a string diff --git a/util/build.sh b/util/build.sh index 14e9fd46d9..0bf7bae0b4 100755 --- a/util/build.sh +++ b/util/build.sh @@ -25,6 +25,7 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" time ngx-build $force $version \ + --with-threads \ --with-pcre-jit \ --with-ipv6 \ --with-cc-opt="-I$PCRE_INC -I$OPENSSL_INC" \ From c297cf69f20db81378093479ea4ad631c9145df3 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 30 Aug 2021 16:08:29 +0800 Subject: [PATCH 530/848] tests: improve test cases robustness. 162-static-module-location.t runs failed when set env TEST_NGINX_RANDOMIZE to 1. 025-codecache.t 106-timer.t 129-ssl-socket.t run failed when set env TEST_NGINX_USE_VALGRIND to 1. 138-balancer.t runs failed on fedora32 with kernel 5.9.14-100.fc32.x86_64. --- .travis.yml | 1 + t/025-codecache.t | 2 +- t/106-timer.t | 6 +++--- t/129-ssl-socket.t | 1 + t/138-balancer.t | 3 +++ t/162-static-module-location.t | 16 ++++++++-------- 6 files changed, 17 insertions(+), 12 deletions(-) diff --git a/.travis.yml b/.travis.yml index 055fbe5685..95f42b1275 100644 --- a/.travis.yml +++ b/.travis.yml @@ -95,6 +95,7 @@ script: - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT - sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP - sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP + - sudo ip route add prohibit 0.0.0.1/32 - cd luajit2/ - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1) - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1) diff --git a/t/025-codecache.t b/t/025-codecache.t index 3e01bb88cf..cf62ff0182 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1096,7 +1096,7 @@ function go(port) ngx.say("failed to receive a line: ", err, " [", part, "]") end - local ok, err = sock:setkeepalive(1) + local ok, err = sock:setkeepalive(10) if not ok then ngx.say("failed to set reusable: ", err) end diff --git a/t/106-timer.t b/t/106-timer.t index 2996762ad4..acd6e4c800 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -118,7 +118,7 @@ F(ngx_http_lua_timer_handler) { registered timer foo = 3 ---- wait: 0.1 +--- wait: 0.2 --- no_error_log [error] [alert] @@ -603,7 +603,7 @@ delete thread 2 --- response_body hello world ---- wait: 0.12 +--- wait: 0.3 --- no_error_log [error] [alert] @@ -613,7 +613,7 @@ hello world [ "registered timer", qr/\[lua\] .*? my lua timer handler/, -qr/\[lua\] log_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-6])/, +qr/\[lua\] log_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.0(?:6[4-9]|7[0-9]|8[1-3])/, "lua ngx.timer expired", "http lua close fake http connection" ] diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 8fd393d66a..3fecf3ee2f 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -2563,6 +2563,7 @@ qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including t --- timeout: 10 + === TEST 32: default cipher -TLSv1.3 --- skip_openssl: 8: < 1.1.1 --- http_config diff --git a/t/138-balancer.t b/t/138-balancer.t index 238dea8fa3..2355b59303 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -7,6 +7,9 @@ use Test::Nginx::Socket::Lua; #workers(2); #log_level('warn'); +#connect 0.0.0.1 on newer kernel won't return EINVAL +#so add an route with cmd: sudo ip route add prohibit 0.0.0.1/32 + repeat_each(2); plan tests => repeat_each() * (blocks() * 4 + 9); diff --git a/t/162-static-module-location.t b/t/162-static-module-location.t index 222760c188..fde93c71ba 100644 --- a/t/162-static-module-location.t +++ b/t/162-static-module-location.t @@ -41,8 +41,8 @@ Hello, world --- request GET /t/中文 --- error_code: 301 ---- response_headers -Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/ +--- response_headers_like +Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ --- response_body_like .*301 Moved Permanently.* @@ -57,8 +57,8 @@ Hello, world --- request GET /t/中文?q=name --- error_code: 301 ---- response_headers -Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/?q=name +--- response_headers_like +Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name --- response_body_like .*301 Moved Permanently.* @@ -73,8 +73,8 @@ Hello, world --- request GET /t/%E4%B8%AD%E6%96%87 --- error_code: 301 ---- response_headers -Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/ +--- response_headers_like +Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ --- response_body_like .*301 Moved Permanently.* @@ -89,7 +89,7 @@ Hello, world --- request GET /t/%E4%B8%AD%E6%96%87?q=name --- error_code: 301 ---- response_headers -Location: http://localhost:1984/t/%E4%B8%AD%E6%96%87/?q=name +--- response_headers_like +Location: http://localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name --- response_body_like .*301 Moved Permanently.* From a9cd21e943f3a9dde9a0f86dbd692194552b7af4 Mon Sep 17 00:00:00 2001 From: jiahao Date: Mon, 30 Aug 2021 14:26:48 +0800 Subject: [PATCH 531/848] travis-ci: upgraded openssl to 1.1.1l. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 95f42b1275..d6718104e8 100644 --- a/.travis.yml +++ b/.travis.yml @@ -47,7 +47,7 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1k OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f services: - memcached From 86514c1e44aa70dbb0436bf823453eaf63a77fab Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Thu, 2 Sep 2021 22:15:10 +0800 Subject: [PATCH 532/848] bugfix: fixed missing 'const' qualifier causing compilation failure on freebsd. (#1914) --- src/ngx_http_lua_pipe.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 9b50d0aef5..b610045bb0 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -547,12 +547,13 @@ ngx_http_lua_pipe_fd_write(ngx_connection_t *c, u_char *buf, size_t size) #if !(NGX_HTTP_LUA_HAVE_EXECVPE) static int -ngx_http_lua_execvpe(const char *program, char **argv, char **envp) +ngx_http_lua_execvpe(const char *program, char * const argv[], + char * const envp[]) { char **saved = environ; int rc; - environ = envp; + environ = (char **) envp; rc = execvp(program, argv); environ = saved; return rc; From 3820d0e118f834916fff66c66f2a29196832dcf8 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 3 Sep 2021 09:34:43 +0800 Subject: [PATCH 533/848] bugfix: buffer bloat and CPU 100% when download large file was filtered by body_filter_by_lua. (#1909) When http body was change by lua script, ngx_http_lua_body_filter_param_set will mark the origin buf as consumed with code "cl->buf->pos = cl->buf->last". Because the buf was marked as consumed, the function ngx_output_chain will continue to write data to the output filter. When kernel socket send buffer is full, write to the socket will return EAGAIN. And the output buf will be cached in the output chain. The output chain becomes longer and longer. Function ngx_chain_update_chains do a linear iteration of the output chain to put the new buf to the end of the chain. In the end, there are thousands of buffs in the output chain which causes the CPU utilization becomes 100% and memory continue to grow up. nginx.conf: location /download { alias download; header_filter_by_lua_block { ngx.header.content_length = nil } body_filter_by_lua_block { ngx.arg[1]=ngx.arg[1] } } create a 2G file with the following cmd: mkdir -p /usr/local/openresty/nginx/download dd if=/dev/zero of=/usr/local/openresty/nginx/download/2G count=4096000 reproduce with the following cmd: curl -o /dev/null http://127.0.0.1/download/2G typical call stacks: 42706f: ngx_chain_update_chains[14] 47e993: ngx_http_chunked_body_filter[14] 483024: ngx_http_gzip_body_filter[14] 4864d3: ngx_http_ssi_body_filter[14] 489668: ngx_http_charset_body_filter[14] 48b86c: ngx_http_addition_body_filter[14] 48bf9c: ngx_http_gunzip_body_filter[14] 48d75e: ngx_http_trailers_filter[14] 4fb63c: ngx_http_lua_capture_body_filter[14] 509cb9: ngx_http_lua_body_filter[14] 42739e: ngx_output_chain[14] 48e1aa: ngx_http_copy_filter[14] 45ea2b: ngx_http_output_filter[14] 496569: ngx_http_static_handler[14] 45ec6e: ngx_http_core_content_phase[14] 459a25: ngx_http_core_run_phases[14] 463ed3: ngx_http_process_request[14] 4643ff: ngx_http_process_request_headers[14] 4647a6: ngx_http_process_request_line[14] 44c1fe: ngx_epoll_process_events[14] 4431c3: ngx_process_events_and_timers[14] 44a65a: ngx_worker_process_cycle[14] 449074: ngx_spawn_process[14] 44aaac: ngx_start_worker_processes[14] 44b213: ngx_master_process_cycle[14] 422df2: main[14] 23493: __libc_start_main[2] 422e5e: _start[14] 42706f: ngx_chain_update_chains[14] 509ce7: ngx_http_lua_body_filter[14] 42739e: ngx_output_chain[14] 48e1aa: ngx_http_copy_filter[14] 45ea2b: ngx_http_output_filter[14] 496569: ngx_http_static_handler[14] 45ec6e: ngx_http_core_content_phase[14] 459a25: ngx_http_core_run_phases[14] 463ed3: ngx_http_process_request[14] 4643ff: ngx_http_process_request_headers[14] 4647a6: ngx_http_process_request_line[14] 44c1fe: ngx_epoll_process_events[14] 4431c3: ngx_process_events_and_timers[14] 44a65a: ngx_worker_process_cycle[14] 449074: ngx_spawn_process[14] 44aaac: ngx_start_worker_processes[14] 44b213: ngx_master_process_cycle[14] 422df2: main[14] 23493: __libc_start_main[2] 422e5e: _start[14] 47d51f: ngx_http_write_filter[14] 47e973: ngx_http_chunked_body_filter[14] 483024: ngx_http_gzip_body_filter[14] 4864d3: ngx_http_ssi_body_filter[14] 489668: ngx_http_charset_body_filter[14] 48b86c: ngx_http_addition_body_filter[14] 48bf9c: ngx_http_gunzip_body_filter[14] 48d75e: ngx_http_trailers_filter[14] 4fb63c: ngx_http_lua_capture_body_filter[14] 509cb9: ngx_http_lua_body_filter[14] 42739e: ngx_output_chain[14] 48e1aa: ngx_http_copy_filter[14] 45ea2b: ngx_http_output_filter[14] 496569: ngx_http_static_handler[14] 45ec6e: ngx_http_core_content_phase[14] 459a25: ngx_http_core_run_phases[14] 463ed3: ngx_http_process_request[14] 4643ff: ngx_http_process_request_headers[14] 4647a6: ngx_http_process_request_line[14] 44c1fe: ngx_epoll_process_events[14] 4431c3: ngx_process_events_and_timers[14] 44a65a: ngx_worker_process_cycle[14] 449074: ngx_spawn_process[14] 44aaac: ngx_start_worker_processes[14] 44b213: ngx_master_process_cycle[14] 422df2: main[14] 23493: __libc_start_main[2] 422e5e: _start[14] --- .travis.yml | 4 ++ src/ngx_http_lua_bodyfilterby.c | 105 ++++++++++++++++++++++++-------- src/ngx_http_lua_common.h | 3 + src/ngx_http_lua_output.c | 5 ++ t/082-body-filter-2.t | 55 +++++++++++++++++ 5 files changed, 148 insertions(+), 24 deletions(-) create mode 100644 t/082-body-filter-2.t diff --git a/.travis.yml b/.travis.yml index d6718104e8..5b3e09be3b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,9 @@ dist: xenial +branches: + only: + - "master" + os: linux language: c diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 75608695e1..a85de31689 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -235,18 +235,15 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) uint16_t old_context; ngx_http_cleanup_t *cln; ngx_chain_t *out; + ngx_chain_t *cl, *ln; ngx_http_lua_main_conf_t *lmcf; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua body filter for user lua code, uri \"%V\"", &r->uri); - if (in == NULL) { - return ngx_http_next_body_filter(r, in); - } - llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); - if (llcf->body_filter_handler == NULL) { + if (llcf->body_filter_handler == NULL || r->header_only) { dd("no body filter handler found"); return ngx_http_next_body_filter(r, in); } @@ -269,7 +266,48 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) in->buf->file_pos = in->buf->file_last; } - return NGX_OK; + in = NULL; + + /* continue to call ngx_http_next_body_filter to process cached data */ + } + + if (in != NULL + && ngx_chain_add_copy(r->pool, &ctx->filter_in_bufs, in) != NGX_OK) { + return NGX_ERROR; + } + + if (ctx->filter_busy_bufs != NULL + && (r->connection->buffered + & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED))) + { + /* Socket write buffer was full on last write. + * Try to write the remain data, if still can not write + * do not execute body_filter_by_lua otherwise the `in` chain will be + * replaced by new content from lua and buf of `in` mark as consumed. + * And then ngx_output_chain will call the filter chain again which + * make all the data cached in the memory and long ngx_chain_t link + * cause CPU 100%. + */ + rc = ngx_http_next_body_filter(r, NULL); + + if (rc == NGX_ERROR) { + return rc; + } + + out = NULL; + ngx_chain_update_chains(r->pool, + &ctx->free_bufs, &ctx->filter_busy_bufs, &out, + (ngx_buf_tag_t) &ngx_http_lua_module); + if (rc != NGX_OK + && ctx->filter_busy_bufs != NULL + && (r->connection->buffered + & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED))) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "waiting body filter busy buffer to be sent"); + return NGX_AGAIN; + } + + /* continue to process bufs in ctx->filter_in_bufs */ } if (ctx->cleanup == NULL) { @@ -286,38 +324,57 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) old_context = ctx->context; ctx->context = NGX_HTTP_LUA_CONTEXT_BODY_FILTER; - dd("calling body filter handler"); - rc = llcf->body_filter_handler(r, in); + in = ctx->filter_in_bufs; + ctx->filter_in_bufs = NULL; - dd("calling body filter handler returned %d", (int) rc); + if (in != NULL) { + dd("calling body filter handler"); + rc = llcf->body_filter_handler(r, in); - ctx->context = old_context; + dd("calling body filter handler returned %d", (int) rc); - if (rc != NGX_OK) { - return NGX_ERROR; - } + ctx->context = old_context; - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - out = lmcf->body_filter_chain; + if (rc != NGX_OK) { + return NGX_ERROR; + } - if (in == out) { - return ngx_http_next_body_filter(r, in); - } + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + + /* lmcf->body_filter_chain is the new buffer chain if + * body_filter_by_lua set new body content via ngx.arg[1] = new_content + * otherwise it is the original `in` buffer chain. + */ + out = lmcf->body_filter_chain; + + if (in != out) { + /* content of body was replaced in + * ngx_http_lua_body_filter_param_set and the buffers was marked + * as consumed. + */ + for (cl = in; cl != NULL; cl = ln) { + ln = cl->next; + ngx_free_chain(r->pool, cl); + } - if (out == NULL) { - /* do not forward NULL to the next filters because the input is - * not NULL */ - return NGX_OK; + if (out == NULL) { + /* do not forward NULL to the next filters because the input is + * not NULL */ + return NGX_OK; + } + } + + } else { + out = NULL; } - /* in != out */ rc = ngx_http_next_body_filter(r, out); if (rc == NGX_ERROR) { return NGX_ERROR; } ngx_chain_update_chains(r->pool, - &ctx->free_bufs, &ctx->busy_bufs, &out, + &ctx->free_bufs, &ctx->filter_busy_bufs, &out, (ngx_buf_tag_t) &ngx_http_lua_module); return rc; diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 027ff71044..b3b6f59a0f 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -548,6 +548,9 @@ typedef struct ngx_http_lua_ctx_s { ngx_chain_t *busy_bufs; ngx_chain_t *free_recv_bufs; + ngx_chain_t *filter_in_bufs; /* for the body filter */ + ngx_chain_t *filter_busy_bufs; /* for the body filter */ + ngx_http_cleanup_pt *cleanup; ngx_http_cleanup_t *free_cleanup; /* free list of cleanup records */ diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index c0aadffb4f..8d8c71d89e 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -244,6 +244,11 @@ ngx_http_lua_ngx_echo(lua_State *L, unsigned newline) return 2; } + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "%s has %sbusy bufs", + newline ? "lua say response" : "lua print response", + ctx->busy_bufs != NULL ? "" : "no "); + dd("downstream write: %d, buf len: %d", (int) rc, (int) (b->last - b->pos)); diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t new file mode 100644 index 0000000000..27baf400c2 --- /dev/null +++ b/t/082-body-filter-2.t @@ -0,0 +1,55 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +BEGIN { + $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + $ENV{MOCKEAGAIN}='w' +} + +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_process_enabled(1); +#log_level('warn'); + +log_level('debug'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 5); + +#no_diff(); +no_long_string(); + +run_tests(); + +__DATA__ + +=== TEST 1: check ctx->busy_bufs +--- config + location /t { + postpone_output 1; + content_by_lua_block { + for i = 1, 5 do + ngx.say(i, ": Hello World!") + end + } + + body_filter_by_lua_block { + ngx.arg[1] = ngx.arg[1] + } + } +--- request +GET /t +--- response_body +1: Hello World! +2: Hello World! +3: Hello World! +4: Hello World! +5: Hello World! + +--- error_log +waiting body filter busy buffer to be sent +lua say response has busy bufs +--- no_error_log +[error] From 776f829102ede4bf059dde4ef444e3cffd19b5d4 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 13 Sep 2021 09:41:47 +0800 Subject: [PATCH 534/848] style: fixed some of the code style issues and add code style checking for the ci. (#1916) --- .travis.yml | 4 +++- src/ngx_http_lua_bodyfilterby.c | 6 ++++-- src/ngx_http_lua_module.c | 3 ++- src/ngx_http_lua_pipe.c | 2 +- src/ngx_http_lua_ssl_session_fetchby.c | 2 +- src/ngx_http_lua_ssl_session_storeby.c | 2 +- src/ngx_http_lua_util.c | 12 +++++++++--- src/ngx_http_lua_worker_thread.c | 2 +- 8 files changed, 22 insertions(+), 11 deletions(-) diff --git a/.travis.yml b/.travis.yml index 5b3e09be3b..b4082f30a2 100644 --- a/.travis.yml +++ b/.travis.yml @@ -96,6 +96,9 @@ before_script: - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;' script: + - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH + - ngx-releng > check.txt || true + - lines=`wc -l check.txt | awk '{print $1}'`; if [ $lines -gt 13 ]; then cat check.txt; exit 1; fi - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT - sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP - sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP @@ -123,7 +126,6 @@ script: - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. - - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH - export NGX_BUILD_CC=$CC - sh util/build.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - nginx -V diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index a85de31689..902488911e 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -272,7 +272,8 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) } if (in != NULL - && ngx_chain_add_copy(r->pool, &ctx->filter_in_bufs, in) != NGX_OK) { + && ngx_chain_add_copy(r->pool, &ctx->filter_in_bufs, in) != NGX_OK) + { return NGX_ERROR; } @@ -301,7 +302,8 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) if (rc != NGX_OK && ctx->filter_busy_bufs != NULL && (r->connection->buffered - & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED))) { + & (NGX_HTTP_LOWLEVEL_BUFFERED | NGX_LOWLEVEL_BUFFERED))) + { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "waiting body filter busy buffer to be sent"); return NGX_AGAIN; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 79c7e35766..baf8ad4a40 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1459,7 +1459,8 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) #if (nginx_version >= 1019004) if (ngx_ssl_conf_commands(cf, llcf->ssl, llcf->ssl_conf_commands) - != NGX_OK) { + != NGX_OK) + { return NGX_ERROR; } #endif diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index b610045bb0..9ea0ba1c41 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -550,8 +550,8 @@ static int ngx_http_lua_execvpe(const char *program, char * const argv[], char * const envp[]) { + int rc; char **saved = environ; - int rc; environ = (char **) envp; rc = execvp(program, argv); diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 8d8c42b213..6584e6ab58 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -450,7 +450,7 @@ ngx_http_lua_log_ssl_sess_fetch_error(ngx_log_t *log, u_char *buf, size_t len) } if (c->listening && c->listening->addr_text.len) { - p = ngx_snprintf(buf, len, ", server: %V", \ + p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); buf = p; } diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 2cfc8f6559..0654d93715 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -336,7 +336,7 @@ ngx_http_lua_log_ssl_sess_store_error(ngx_log_t *log, u_char *buf, size_t len) } if (c->listening && c->listening->addr_text.len) { - p = ngx_snprintf(buf, len, ", server: %V", \ + p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); buf = p; } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 1a04bf025a..38e1b68384 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -376,8 +376,10 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) if (!lua_isnil(L, -1) && !lua_isnil(L, -2)) { n++; } + lua_pop(L, 1); } + lua_pop(L, 1); ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, @@ -2199,6 +2201,7 @@ ngx_http_lua_escape_uri(u_char *dst, u_char *src, size_t size, ngx_uint_t type) return (uintptr_t) dst; } + static int ngx_http_lua_util_hex2int(char xdigit) { @@ -2214,14 +2217,15 @@ ngx_http_lua_util_hex2int(char xdigit) return -1; } + /* XXX we also decode '+' to ' ' */ void ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, ngx_uint_t type) { - u_char *d = *dst, *s = *src, *de = (*dst+size); - int isuri = type & NGX_UNESCAPE_URI; - int isredirect = type & NGX_UNESCAPE_REDIRECT; + u_char *d = *dst, *s = *src, *de = (*dst + size); + int isuri = type & NGX_UNESCAPE_URI; + int isredirect = type & NGX_UNESCAPE_REDIRECT; while (size--) { u_char curr = *s++; @@ -2245,10 +2249,12 @@ ngx_http_lua_unescape_uri(u_char **dst, u_char **src, size_t size, if ((isuri || isredirect) && ch == '?') { *d++ = ch; break; + } else if (isredirect && (ch <= '%' || ch >= 0x7f)) { *d++ = '%'; continue; } + *d++ = ch; s += 2; size -= 2; diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 92bb77d4cd..080de53776 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -153,7 +153,7 @@ ngx_http_lua_free_task_ctx(ngx_http_lua_task_ctx_t *ctx) static int ngx_http_lua_xcopy(lua_State *from, lua_State *to, int idx, - const int allow_nil) + const int allow_nil) { size_t len = 0; const char *str; From fef25815f1de5789a92fd7df1837195637d2f105 Mon Sep 17 00:00:00 2001 From: Zhefeng Chen Date: Mon, 20 Sep 2021 17:57:38 +0800 Subject: [PATCH 535/848] feature: implemented the ssl_client_hello_by_lua_block and ssl_client_hello_by_lua_file directives for controlling the NGINX downstream SSL handshake dynamically with Lua. --- README.markdown | 314 ++- config | 2 + doc/HttpLuaModule.wiki | 305 ++- src/ngx_http_lua_common.h | 34 +- src/ngx_http_lua_control.c | 2 + src/ngx_http_lua_ctx.c | 2 + src/ngx_http_lua_module.c | 59 + src/ngx_http_lua_ssl.h | 2 + src/ngx_http_lua_ssl_client_helloby.c | 705 +++++++ src/ngx_http_lua_ssl_client_helloby.h | 35 + src/ngx_http_lua_util.h | 13 +- t/166-ssl-client-hello.t | 2567 +++++++++++++++++++++++++ 12 files changed, 3837 insertions(+), 203 deletions(-) create mode 100644 src/ngx_http_lua_ssl_client_helloby.c create mode 100644 src/ngx_http_lua_ssl_client_helloby.h create mode 100644 t/166-ssl-client-hello.t diff --git a/README.markdown b/README.markdown index 3295177b6d..28717882b7 100644 --- a/README.markdown +++ b/README.markdown @@ -1133,6 +1133,8 @@ Directives * [balancer_by_lua_block](#balancer_by_lua_block) * [balancer_by_lua_file](#balancer_by_lua_file) * [lua_need_request_body](#lua_need_request_body) +* [ssl_client_hello_by_lua_block](#ssl_client_hello_by_lua_block) +* [ssl_client_hello_by_lua_file](#ssl_client_hello_by_lua_file) * [ssl_certificate_by_lua_block](#ssl_certificate_by_lua_block) * [ssl_certificate_by_lua_file](#ssl_certificate_by_lua_file) * [ssl_session_fetch_by_lua_block](#ssl_session_fetch_by_lua_block) @@ -2625,6 +2627,132 @@ This also applies to [access_by_lua*](#access_by_lua). [Back to TOC](#directives) +ssl_client_hello_by_lua_block +----------------------------- + +**syntax:** *ssl_client_hello_by_lua_block { lua-script }* + +**context:** *http, server* + +**phase:** *right-after-client-hello-message-was-processed* + +This directive runs user Lua code when Nginx is about to post-process the SSL client hello message for the downstream +SSL (https) connections. + +It is particularly useful for dynamically setting the SSL protocols according to the SNI. + +It is also useful to do some custom operations according to the per-connection information in the client hello message. + +For example, one can parse custom client hello extension and do the corresponding handling in pure Lua. + +This Lua handler will always run whether the SSL session is resumed (via SSL session IDs or TLS session tickets) or not. +While the `ssl_certificate_by_lua*` Lua handler will only runs when initiating a full SSL handshake. + +The [ngx.ssl.clienthello](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienthello.md) Lua modules +provided by the [lua-resty-core](https://github.com/openresty/lua-resty-core/#readme) +library are particularly useful in this context. + +Note that this handler runs in extremelly early stage of SSL handshake, before the SSL client hello extensions are parsed. +So you can not use some Lua API like `ssl.server_name()` which is dependent on the later stage's processing. + +Also note that only the directive in default server is valid for several virtual servers with the same IP address and port. + +Below is a trivial example using the +[ngx.ssl.clienthello](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienthello.md) module +at the same time: + +```nginx + + server { + listen 443 ssl; + server_name test.com; + ssl_certificate /path/to/cert.crt; + ssl_certificate_key /path/to/key.key; + ssl_client_hello_by_lua_block { + local ssl_clt = require "ngx.ssl.clienthello" + local host, err = ssl_clt.get_client_hello_server_name() + if host == "test.com" then + ssl_clt.set_protocols({"TLSv1", "TLSv1.1"}) + elseif host == "test2.com" then + ssl_clt.set_protocols({"TLSv1.2", "TLSv1.3"}) + elseif not host then + ngx.log(ngx.ERR, "failed to get the SNI name: ", err) + ngx.exit(ngx.ERROR) + else + ngx.log(ngx.ERR, "unknown SNI name: ", host) + ngx.exit(ngx.ERROR) + end + } + ... + } + server { + listen 443 ssl; + server_name test2.com; + ssl_certificate /path/to/cert.crt; + ssl_certificate_key /path/to/key.key; + ... + } +``` + +See more information in the [ngx.ssl.clienthello](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienthello.md) +Lua modules' official documentation. + +Uncaught Lua exceptions in the user Lua code immediately abort the current SSL session, so does the +[ngx.exit](#ngxexit) call with an error code like `ngx.ERROR`. + +This Lua code execution context *does* support yielding, so Lua APIs that may yield +(like cosockets, sleeping, and "light threads") +are enabled in this context + +Note, you need to configure the [ssl_certificate](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate) +and [ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key) +to avoid the following error while starting NGINX: + + + nginx: [emerg] no ssl configured for the server + + +This directive requires OpenSSL 1.1.1 or greater. + +If you are using the [official pre-built +packages](https://openresty.org/en/linux-packages.html) for +[OpenResty](https://openresty.org/) 1.19.9.2 or later, then everything should +work out of the box. + +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +in order to use this directive: + + + +Similarly, if you are not using the Nginx core shipped with +[OpenResty](https://openresty.org) 1.19.9.2 or later, you will need to apply +patches to the standard Nginx core: + + + +This directive was first introduced in the `v0.10.21` release. + +[Back to TOC](#directives) + +ssl_client_hello_by_lua_file +---------------------------- + +**syntax:** *ssl_client_hello_by_lua_file <path-to-lua-script-file>* + +**context:** *http, server* + +**phase:** *right-after-client-hello-message-was-processed* + +Equivalent to [ssl_client_hello_by_lua_block](#ssl_client_hello_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. + +When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. + +This directive was first introduced in the `v0.10.21` release. + +[Back to TOC](#directives) + ssl_certificate_by_lua_block ---------------------------- @@ -3644,7 +3772,7 @@ This API requires a relatively expensive metamethod call and it is recommended t Core constants -------------- -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** ```lua @@ -3671,7 +3799,7 @@ The `ngx.DECLINED` constant was first introduced in the `v0.5.0rc19` release. HTTP method constants --------------------- -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** ngx.HTTP_GET @@ -3698,7 +3826,7 @@ These constants are usually used in [ngx.location.capture](#ngxlocationcapture) HTTP status constants --------------------- -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** ```nginx @@ -3744,7 +3872,7 @@ HTTP status constants Nginx log level constants ------------------------- -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** ```lua @@ -3768,7 +3896,7 @@ print **syntax:** *print(...)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Writes argument values into the Nginx `error.log` file with the `ngx.NOTICE` log level. @@ -3881,7 +4009,7 @@ Then `GET /orig` will give rather than the original `"hello"` value. Because HTTP request is created after SSL handshake, the `ngx.ctx` created -in [ssl_certificate_by_lua*](#ssl_certificate_by_lua), [ssl_session_store_by_lua*](#ssl_session_store_by_lua) and [ssl_session_fetch_by_lua*](#ssl_session_fetch_by_lua) +in [ssl_certificate_by_lua*](#ssl_certificate_by_lua), [ssl_session_store_by_lua*](#ssl_session_store_by_lua), [ssl_session_fetch_by_lua*](#ssl_session_fetch_by_lua) and [ssl_client_hello_by_lua*](#ssl_client_hello_by_lua) is not available in the following phases like [rewrite_by_lua*](#rewrite_by_lua). Since `dev`, the `ngx.ctx` created during a SSL handshake @@ -5623,7 +5751,7 @@ ngx.log **syntax:** *ngx.log(log_level, ...)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Log arguments concatenated to error.log with the given logging level. @@ -5661,7 +5789,7 @@ ngx.exit **syntax:** *ngx.exit(status)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** When `status >= 200` (i.e., `ngx.HTTP_OK` and above), it will interrupt the execution of the current request and return status code to Nginx. @@ -5753,7 +5881,7 @@ ngx.sleep **syntax:** *ngx.sleep(seconds)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Sleeps for the specified seconds without blocking. One can specify time resolution up to 0.001 seconds (i.e., one millisecond). @@ -5770,7 +5898,7 @@ ngx.escape_uri **syntax:** *newstr = ngx.escape_uri(str, type?)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Since `v0.10.16`, this function accepts an optional `type` argument. It accepts the following values (defaults to `2`): @@ -5789,7 +5917,7 @@ ngx.unescape_uri **syntax:** *newstr = ngx.unescape_uri(str)* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Unescape `str` as an escaped URI component. @@ -5830,7 +5958,7 @@ ngx.encode_args **syntax:** *str = ngx.encode_args(table)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_client_hello_by_lua** Encode the Lua table to a query args string according to the URI encoded rules. @@ -5888,7 +6016,7 @@ ngx.decode_args **syntax:** *table, err = ngx.decode_args(str, max_args?)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Decodes a URI encoded query-string into a Lua table. This is the inverse function of [ngx.encode_args](#ngxencode_args). @@ -5912,7 +6040,7 @@ ngx.encode_base64 **syntax:** *newstr = ngx.encode_base64(str, no_padding?)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Encodes `str` to a base64 digest. @@ -5925,7 +6053,7 @@ ngx.decode_base64 **syntax:** *newstr = ngx.decode_base64(str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Decodes the `str` argument as a base64 digest to the raw form. Returns `nil` if `str` is not well formed. @@ -5936,7 +6064,7 @@ ngx.crc32_short **syntax:** *intval = ngx.crc32_short(str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Calculates the CRC-32 (Cyclic Redundancy Code) digest for the `str` argument. @@ -5953,7 +6081,7 @@ ngx.crc32_long **syntax:** *intval = ngx.crc32_long(str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Calculates the CRC-32 (Cyclic Redundancy Code) digest for the `str` argument. @@ -5970,7 +6098,7 @@ ngx.hmac_sha1 **syntax:** *digest = ngx.hmac_sha1(secret_key, str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Computes the [HMAC-SHA1](https://en.wikipedia.org/wiki/HMAC) digest of the argument `str` and turns the result using the secret key ``. @@ -6003,7 +6131,7 @@ ngx.md5 **syntax:** *digest = ngx.md5(str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Returns the hexadecimal representation of the MD5 digest of the `str` argument. @@ -6031,7 +6159,7 @@ ngx.md5_bin **syntax:** *digest = ngx.md5_bin(str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Returns the binary form of the MD5 digest of the `str` argument. @@ -6044,7 +6172,7 @@ ngx.sha1_bin **syntax:** *digest = ngx.sha1_bin(str)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Returns the binary form of the SHA-1 digest of the `str` argument. @@ -6059,7 +6187,7 @@ ngx.quote_sql_str **syntax:** *quoted_value = ngx.quote_sql_str(raw_value)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Returns a quoted SQL string literal according to the MySQL quoting rules. @@ -6070,7 +6198,7 @@ ngx.today **syntax:** *str = ngx.today()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns current date (in the format `yyyy-mm-dd`) from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -6083,7 +6211,7 @@ ngx.time **syntax:** *secs = ngx.time()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns the elapsed seconds from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -6096,7 +6224,7 @@ ngx.now **syntax:** *secs = ngx.now()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -6111,7 +6239,7 @@ ngx.update_time **syntax:** *ngx.update_time()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Forcibly updates the Nginx current time cache. This call involves a syscall and thus has some overhead, so do not abuse it. @@ -6124,7 +6252,7 @@ ngx.localtime **syntax:** *str = ngx.localtime()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the Nginx cached time (no syscall involved unlike Lua's [os.date](https://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). @@ -6137,7 +6265,7 @@ ngx.utctime **syntax:** *str = ngx.utctime()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns the current time stamp (in the format `yyyy-mm-dd hh:mm:ss`) of the Nginx cached time (no syscall involved unlike Lua's [os.date](https://www.lua.org/manual/5.1/manual.html#pdf-os.date) function). @@ -6150,7 +6278,7 @@ ngx.cookie_time **syntax:** *str = ngx.cookie_time(sec)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns a formatted string can be used as the cookie expiration time. The parameter `sec` is the time stamp in seconds (like those returned from [ngx.time](#ngxtime)). @@ -6167,7 +6295,7 @@ ngx.http_time **syntax:** *str = ngx.http_time(sec)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns a formated string can be used as the http header time (for example, being used in `Last-Modified` header). The parameter `sec` is the time stamp in seconds (like those returned from [ngx.time](#ngxtime)). @@ -6184,7 +6312,7 @@ ngx.parse_http_time **syntax:** *sec = ngx.parse_http_time(str)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Parse the http time string (as returned by [ngx.http_time](#ngxhttp_time)) into seconds. Returns the seconds or `nil` if the input string is in bad forms. @@ -6214,7 +6342,7 @@ ngx.re.match **syntax:** *captures, err = ngx.re.match(subject, regex, options?, ctx?, res_table?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Matches the `subject` string using the Perl compatible regular expression `regex` with the optional `options`. @@ -6373,7 +6501,7 @@ ngx.re.find **syntax:** *from, to, err = ngx.re.find(subject, regex, options?, ctx?, nth?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Similar to [ngx.re.match](#ngxrematch) but only returns the beginning index (`from`) and end index (`to`) of the matched substring. The returned indexes are 1-based and can be fed directly into the [string.sub](https://www.lua.org/manual/5.1/manual.html#pdf-string.sub) API function to obtain the matched substring. @@ -6428,7 +6556,7 @@ ngx.re.gmatch **syntax:** *iterator, err = ngx.re.gmatch(subject, regex, options?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Similar to [ngx.re.match](#ngxrematch), but returns a Lua iterator instead, so as to let the user programmer iterate all the matches over the `` string argument with the PCRE `regex`. @@ -6507,7 +6635,7 @@ ngx.re.sub **syntax:** *newstr, n, err = ngx.re.sub(subject, regex, replace, options?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Substitutes the first match of the Perl compatible regular expression `regex` on the `subject` argument string with the string or function argument `replace`. The optional `options` argument has exactly the same meaning as in [ngx.re.match](#ngxrematch). @@ -6575,7 +6703,7 @@ ngx.re.gsub **syntax:** *newstr, n, err = ngx.re.gsub(subject, regex, replace, options?)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Just like [ngx.re.sub](#ngxresub), but does global substitution. @@ -6616,7 +6744,7 @@ ngx.shared.DICT **syntax:** *dict = ngx.shared\[name_var\]* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Fetching the shm-based Lua dictionary object for the shared memory zone named `DICT` defined by the [lua_shared_dict](#lua_shared_dict) directive. @@ -6701,7 +6829,7 @@ ngx.shared.DICT.get **syntax:** *value, flags = ngx.shared.DICT:get(key)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Retrieving the value in the dictionary [ngx.shared.DICT](#ngxshareddict) for the key `key`. If the key does not exist or has expired, then `nil` will be returned. @@ -6740,7 +6868,7 @@ ngx.shared.DICT.get_stale **syntax:** *value, flags, stale = ngx.shared.DICT:get_stale(key)* -**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Similar to the [get](#ngxshareddictget) method but returns the value even if the key has already expired. @@ -6759,7 +6887,7 @@ ngx.shared.DICT.set **syntax:** *success, err, forcible = ngx.shared.DICT:set(key, value, exptime?, flags?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Unconditionally sets a key-value pair into the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns three values: @@ -6812,7 +6940,7 @@ ngx.shared.DICT.safe_set **syntax:** *ok, err = ngx.shared.DICT:safe_set(key, value, exptime?, flags?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Similar to the [set](#ngxshareddictset) method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return `nil` and the string "no memory". @@ -6827,7 +6955,7 @@ ngx.shared.DICT.add **syntax:** *success, err, forcible = ngx.shared.DICT:add(key, value, exptime?, flags?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Just like the [set](#ngxshareddictset) method, but only stores the key-value pair into the dictionary [ngx.shared.DICT](#ngxshareddict) if the key does *not* exist. @@ -6844,7 +6972,7 @@ ngx.shared.DICT.safe_add **syntax:** *ok, err = ngx.shared.DICT:safe_add(key, value, exptime?, flags?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Similar to the [add](#ngxshareddictadd) method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return `nil` and the string "no memory". @@ -6859,7 +6987,7 @@ ngx.shared.DICT.replace **syntax:** *success, err, forcible = ngx.shared.DICT:replace(key, value, exptime?, flags?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Just like the [set](#ngxshareddictset) method, but only stores the key-value pair into the dictionary [ngx.shared.DICT](#ngxshareddict) if the key *does* exist. @@ -6876,7 +7004,7 @@ ngx.shared.DICT.delete **syntax:** *ngx.shared.DICT:delete(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Unconditionally removes the key-value pair from the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). @@ -6893,7 +7021,7 @@ ngx.shared.DICT.incr **syntax:** *newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?, init_ttl?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** **optional requirement:** `resty.core.shdict` or `resty.core` @@ -6948,7 +7076,7 @@ ngx.shared.DICT.lpush **syntax:** *length, err = ngx.shared.DICT:lpush(key, value)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Inserts the specified (numerical or string) `value` at the head of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the number of elements in the list after the push operation. @@ -6967,7 +7095,7 @@ ngx.shared.DICT.rpush **syntax:** *length, err = ngx.shared.DICT:rpush(key, value)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Similar to the [lpush](#ngxshareddictlpush) method, but inserts the specified (numerical or string) `value` at the tail of the list named `key`. @@ -6982,7 +7110,7 @@ ngx.shared.DICT.lpop **syntax:** *val, err = ngx.shared.DICT:lpop(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Removes and returns the first element of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). @@ -6999,7 +7127,7 @@ ngx.shared.DICT.rpop **syntax:** *val, err = ngx.shared.DICT:rpop(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Removes and returns the last element of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). @@ -7016,7 +7144,7 @@ ngx.shared.DICT.llen **syntax:** *len, err = ngx.shared.DICT:llen(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Returns the number of elements in the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). @@ -7033,7 +7161,7 @@ ngx.shared.DICT.ttl **syntax:** *ttl, err = ngx.shared.DICT:ttl(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** **requires:** `resty.core.shdict` or `resty.core` @@ -7071,7 +7199,7 @@ ngx.shared.DICT.expire **syntax:** *success, err = ngx.shared.DICT:expire(key, exptime)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** **requires:** `resty.core.shdict` or `resty.core` @@ -7111,7 +7239,7 @@ ngx.shared.DICT.flush_all **syntax:** *ngx.shared.DICT:flush_all()* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Flushes out all the items in the dictionary. This method does not actually free up all the memory blocks in the dictionary but just marks all the existing items as expired. @@ -7126,7 +7254,7 @@ ngx.shared.DICT.flush_expired **syntax:** *flushed = ngx.shared.DICT:flush_expired(max_count?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Flushes out the expired items in the dictionary, up to the maximal number specified by the optional `max_count` argument. When the `max_count` argument is given `0` or not given at all, then it means unlimited. Returns the number of items that have actually been flushed. @@ -7143,7 +7271,7 @@ ngx.shared.DICT.get_keys **syntax:** *keys = ngx.shared.DICT:get_keys(max_count?)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Fetch a list of the keys from the dictionary, up to ``. @@ -7160,7 +7288,7 @@ ngx.shared.DICT.capacity **syntax:** *capacity_bytes = ngx.shared.DICT:capacity()* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** **requires:** `resty.core.shdict` or `resty.core` @@ -7192,7 +7320,7 @@ ngx.shared.DICT.free_space **syntax:** *free_page_bytes = ngx.shared.DICT:free_space()* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** **requires:** `resty.core.shdict` or `resty.core` @@ -7235,7 +7363,7 @@ ngx.socket.udp **syntax:** *udpsock = ngx.socket.udp()* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Creates and returns a UDP or datagram-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: @@ -7260,7 +7388,7 @@ udpsock:setpeername **syntax:** *ok, err = udpsock:setpeername("unix:/path/to/unix-domain.socket")* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Attempts to connect a UDP socket object to a remote server or to a datagram unix domain socket file. Because the datagram protocol is actually connection-less, this method does not really establish a "connection", but only just set the name of the remote peer for subsequent read/write operations. @@ -7323,7 +7451,7 @@ udpsock:send **syntax:** *ok, err = udpsock:send(data)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Sends data on the current UDP or datagram unix domain socket object. @@ -7340,7 +7468,7 @@ udpsock:receive **syntax:** *data, err = udpsock:receive(size?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Receives data from the UDP or datagram unix domain socket object with an optional receive buffer size argument, `size`. @@ -7376,7 +7504,7 @@ udpsock:close **syntax:** *ok, err = udpsock:close()* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Closes the current UDP or datagram unix domain socket. It returns the `1` in case of success and returns `nil` with a string describing the error otherwise. @@ -7391,7 +7519,7 @@ udpsock:settimeout **syntax:** *udpsock:settimeout(time)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Set the timeout value in milliseconds for subsequent socket operations (like [receive](#udpsockreceive)). @@ -7416,7 +7544,7 @@ ngx.socket.tcp **syntax:** *tcpsock = ngx.socket.tcp()* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Creates and returns a TCP or stream-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: @@ -7465,7 +7593,7 @@ tcpsock:connect **syntax:** *ok, err = tcpsock:connect("unix:/path/to/unix-domain.socket", options_table?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Attempts to connect a TCP socket object to a remote server or to a stream unix domain socket file without blocking. @@ -7585,7 +7713,7 @@ tcpsock:sslhandshake **syntax:** *session, err = tcpsock:sslhandshake(reused_session?, server_name?, ssl_verify?, send_status_req?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Does SSL/TLS handshake on the currently established connection. @@ -7631,7 +7759,7 @@ tcpsock:send **syntax:** *bytes, err = tcpsock:send(data)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Sends data without blocking on the current TCP or Unix Domain Socket connection. @@ -7664,7 +7792,7 @@ tcpsock:receive **syntax:** *data, err, partial = tcpsock:receive(pattern?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Receives data from the connected socket according to the reading pattern or size. @@ -7707,7 +7835,7 @@ tcpsock:receiveany **syntax:** *data, err = tcpsock:receiveany(max)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Returns any data received by the connected socket, at most `max` bytes. @@ -7742,7 +7870,7 @@ tcpsock:receiveuntil **syntax:** *iterator = tcpsock:receiveuntil(pattern, options?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** This method returns an iterator Lua function that can be called to read the data stream until it sees the specified pattern or an error occurs. @@ -7842,7 +7970,7 @@ tcpsock:close **syntax:** *ok, err = tcpsock:close()* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Closes the current TCP or stream unix domain socket. It returns the `1` in case of success and returns `nil` with a string describing the error otherwise. @@ -7859,7 +7987,7 @@ tcpsock:settimeout **syntax:** *tcpsock:settimeout(time)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Set the timeout value in milliseconds for subsequent socket operations ([connect](#tcpsockconnect), [receive](#tcpsockreceive), and iterators returned from [receiveuntil](#tcpsockreceiveuntil)). @@ -7876,7 +8004,7 @@ tcpsock:settimeouts **syntax:** *tcpsock:settimeouts(connect_timeout, send_timeout, read_timeout)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Respectively sets the connect, send, and read timeout thresholds (in milliseconds) for subsequent socket operations ([connect](#tcpsockconnect), [send](#tcpsocksend), [receive](#tcpsockreceive), and iterators returned from [receiveuntil](#tcpsockreceiveuntil)). @@ -7896,7 +8024,7 @@ tcpsock:setoption **syntax:** *ok, err = tcpsock:setoption(option, value?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** This function is added for [LuaSocket](http://w3.impa.br/~diego/software/luasocket/tcp.html) API compatibility and does nothing for now. Its functionality is implemented `v0.10.18`. @@ -7999,7 +8127,7 @@ tcpsock:setkeepalive **syntax:** *ok, err = tcpsock:setkeepalive(timeout?, size?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Puts the current socket's connection immediately into the cosocket built-in connection pool and keep it alive until other [connect](#tcpsockconnect) method calls request it or the associated maximal idle timeout is expired. @@ -8046,7 +8174,7 @@ tcpsock:getreusedtimes **syntax:** *count, err = tcpsock:getreusedtimes()* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** This method returns the (successfully) reused times for the current connection. In case of error, it returns `nil` and a string describing the error. @@ -8088,7 +8216,7 @@ ngx.get_phase **syntax:** *str = ngx.get_phase()* -**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Retrieves the current running phase name. Possible return values are @@ -8102,6 +8230,8 @@ Retrieves the current running phase name. Possible return values are for the context of [ssl_session_fetch_by_lua*](#ssl_session_fetch_by_lua_block). * `ssl_session_store` for the context of [ssl_session_store_by_lua*](#ssl_session_store_by_lua_block). +* `ssl_client_hello` + for the context of [ssl_client_hello_by_lua*](#ssl_client_hello_by_lua_block). * `set` for the context of [set_by_lua*](#set_by_lua). * `rewrite` @@ -8132,7 +8262,7 @@ ngx.thread.spawn **syntax:** *co = ngx.thread.spawn(func, arg1, arg2, ...)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Spawns a new user "light thread" with the Lua function `func` as well as those optional arguments `arg1`, `arg2`, and etc. Returns a Lua thread (or Lua coroutine) object represents this "light thread". @@ -8271,7 +8401,7 @@ ngx.thread.wait **syntax:** *ok, res1, res2, ... = ngx.thread.wait(thread1, thread2, ...)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Waits on one or more child "light threads" and returns the results of the first "light thread" that terminates (either successfully or with an error). @@ -8375,7 +8505,7 @@ ngx.thread.kill **syntax:** *ok, err = ngx.thread.kill(thread)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** Kills a running "light thread" created by [ngx.thread.spawn](#ngxthreadspawn). Returns a true value when successful or `nil` and a string describing the error otherwise. @@ -8431,7 +8561,7 @@ ngx.timer.at **syntax:** *hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Creates an Nginx timer with a user callback function as well as optional user arguments. @@ -8574,7 +8704,7 @@ ngx.timer.every **syntax:** *hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Similar to the [ngx.timer.at](#ngxtimerat) API function, but @@ -8597,7 +8727,7 @@ ngx.timer.running_count **syntax:** *count = ngx.timer.running_count()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns the number of timers currently running. @@ -8610,7 +8740,7 @@ ngx.timer.pending_count **syntax:** *count = ngx.timer.pending_count()* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** Returns the number of pending timers. @@ -8848,7 +8978,7 @@ ndk.set_var.DIRECTIVE **syntax:** *res = ndk.set_var.DIRECTIVE_NAME* -**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua** +**context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** This mechanism allows calling other Nginx C modules' directives that are implemented by [Nginx Devel Kit](https://github.com/simplresty/ngx_devel_kit) (NDK)'s set_var submodule's `ndk_set_var_value`. @@ -8890,7 +9020,7 @@ coroutine.create **syntax:** *co = coroutine.create(f)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Creates a user Lua coroutines with a Lua function, and returns a coroutine object. @@ -8907,7 +9037,7 @@ coroutine.resume **syntax:** *ok, ... = coroutine.resume(co, ...)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Resumes the executation of a user Lua coroutine object previously yielded or just created. @@ -8924,7 +9054,7 @@ coroutine.yield **syntax:** *... = coroutine.yield(...)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Yields the execution of the current user Lua coroutine. @@ -8941,7 +9071,7 @@ coroutine.wrap **syntax:** *co = coroutine.wrap(f)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Similar to the standard Lua [coroutine.wrap](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.wrap) API, but works in the context of the Lua coroutines created by ngx_lua. @@ -8956,7 +9086,7 @@ coroutine.running **syntax:** *co = coroutine.running()* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Identical to the standard Lua [coroutine.running](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running) API. @@ -8971,7 +9101,7 @@ coroutine.status **syntax:** *status = coroutine.status(co)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** Identical to the standard Lua [coroutine.status](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.status) API. diff --git a/config b/config index 4763949fc0..14870a04e8 100644 --- a/config +++ b/config @@ -286,6 +286,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_timer.c \ $ngx_addon_dir/src/ngx_http_lua_config.c \ $ngx_addon_dir/src/ngx_http_lua_worker.c \ + $ngx_addon_dir/src/ngx_http_lua_ssl_client_helloby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_certby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_ocsp.c \ $ngx_addon_dir/src/ngx_http_lua_lex.c \ @@ -347,6 +348,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_uthread.h \ $ngx_addon_dir/src/ngx_http_lua_timer.h \ $ngx_addon_dir/src/ngx_http_lua_config.h \ + $ngx_addon_dir/src/ngx_http_lua_ssl_client_helloby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl_certby.h \ $ngx_addon_dir/src/ngx_http_lua_lex.h \ $ngx_addon_dir/src/ngx_http_lua_balancer.h \ diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4b9bd1eb03..2a647e879e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2211,6 +2211,125 @@ It is recommended however, to use the [[#ngx.req.read_body|ngx.req.read_body]] a This also applies to [[#access_by_lua|access_by_lua*]]. +== ssl_client_hello_by_lua_block == + +'''syntax:''' ''ssl_client_hello_by_lua_block { lua-script }'' + +'''context:''' ''http, server'' + +'''phase:''' ''right-after-client-hello-message-was-processed'' + +This directive runs user Lua code when Nginx is about to post-process the SSL client hello message for the downstream +SSL (https) connections. + +It is particularly useful for dynamically setting the SSL protocols according to the SNI. + +It is also useful to do some custom operations according to the per-connection information in the client hello message. + +For example, one can parse custom client hello extension and do the corresponding handling in pure Lua. + +This Lua handler will always run whether the SSL session is resumed (via SSL session IDs or TLS session tickets) or not. +While the ssl_certificate_by_lua* Lua handler will only runs when initiating a full SSL handshake. + +The [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienthello.md ngx.ssl.clienthello] Lua modules +provided by the [https://github.com/openresty/lua-resty-core/#readme lua-resty-core] +library are particularly useful in this context. + +Note that this handler runs in extremelly early stage of SSL handshake, before the SSL client hello extensions are parsed. +So you can not use some Lua API like ssl.server_name() which is dependent on the later stage's processing. + +Also note that only the directive in default server is valid for several virtual servers with the same IP address and port. + +Below is a trivial example using the +[https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienthello.md ngx.ssl.clienthello] module +at the same time: + + +server { + listen 443 ssl; + server_name test.com; + ssl_certificate /path/to/cert.crt; + ssl_certificate_key /path/to/key.key; + ssl_client_hello_by_lua_block { + local ssl_clt = require "ngx.ssl.clienthello" + local host, err = ssl_clt.get_client_hello_server_name() + if host == "test.com" then + ssl_clt.set_protocols({"TLSv1", "TLSv1.1"}) + elseif host == "test2.com" then + ssl_clt.set_protocols({"TLSv1.2", "TLSv1.3"}) + elseif not host then + ngx.log(ngx.ERR, "failed to get the SNI name: ", err) + ngx.exit(ngx.ERROR) + else + ngx.log(ngx.ERR, "unknown SNI name: ", host) + ngx.exit(ngx.ERROR) + end + } + ... +} +server { + listen 443 ssl; + server_name test2.com; + ssl_certificate /path/to/cert.crt; + ssl_certificate_key /path/to/key.key; + ... +} + + +See more information in the [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienthello.md ngx.ssl.clienthello] +Lua modules' official documentation. + +Uncaught Lua exceptions in the user Lua code immediately abort the current SSL session, so does the +[[#ngx.exit|ngx.exit]] call with an error code like ngx.ERROR. + +This Lua code execution context *does* support yielding, so Lua APIs that may yield +(like cosockets, sleeping, and "light threads") +are enabled in this context + +Note, you need to configure the [https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate ssl_certificate] +and [https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key ssl_certificate_key] +to avoid the following error while starting NGINX: + + + nginx: [emerg] no ssl configured for the server + + +This directive requires OpenSSL 1.1.1 or greater. + +If you are using the [official pre-built +packages](https://openresty.org/en/linux-packages.html) for +[OpenResty](https://openresty.org/) 1.19.9.2 or later, then everything should +work out of the box. + +If you are not using one of the [OpenSSL +packages](https://openresty.org/en/linux-packages.html) provided by +[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL +in order to use this directive: + +https://openresty.org/en/openssl-patches.html + +Similarly, if you are not using the Nginx core shipped with +[OpenResty](https://openresty.org) 1.19.9.2 or later, you will need to apply +patches to the standard Nginx core: + +https://openresty.org/en/nginx-ssl-patches.html + +This directive was first introduced in the v0.10.21 release. + +== ssl_client_hello_by_lua_file == + +'''syntax:''' ''ssl_client_hello_by_lua_file '' + +'''context:''' ''http, server'' + +'''phase:''' ''right-after-client-hello-message-was-processed'' + +Equivalent to [[#ssl_client_hello_by_lua_block|ssl_client_hello_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. + +When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. + +This directive was first introduced in the v0.10.21 release. + == ssl_certificate_by_lua_block == '''syntax:''' ''ssl_certificate_by_lua_block { lua-script }'' @@ -2965,7 +3084,7 @@ This API requires a relatively expensive metamethod call and it is recommended t == Core constants == -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, *log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' ngx.OK (0) @@ -2987,7 +3106,7 @@ The ngx.DECLINED constant was first introduced in the v0.5.0r == HTTP method constants == -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' ngx.HTTP_GET @@ -3011,7 +3130,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture == HTTP status constants == -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' value = ngx.HTTP_CONTINUE (100) (first added in the v0.9.20 release) @@ -3053,7 +3172,7 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture == Nginx log level constants == -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' ngx.STDERR @@ -3073,7 +3192,7 @@ These constants are usually used by the [[#ngx.log|ngx.log]] method. '''syntax:''' ''print(...)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Writes argument values into the Nginx error.log file with the ngx.NOTICE log level. @@ -3176,7 +3295,7 @@ Then GET /orig will give rather than the original "hello" value. Because HTTP request is created after SSL handshake, the ngx.ctx created -in [[#ssl_certificate_by_lua|ssl_certificate_by_lua*]], [[#ssl_session_store_by_lua|ssl_session_store_by_lua*]] and [[#ssl_session_fetch_by_lua|ssl_session_fetch_by_lua*]] +in [[#ssl_certificate_by_lua|ssl_certificate_by_lua*]], [[#ssl_session_store_by_lua|ssl_session_store_by_lua*]], [[#ssl_session_fetch_by_lua|ssl_session_fetch_by_lua*]] and [[#ssl_client_hello_by_lua|ssl_client_hello_by_lua*]] is not available in the following phases like [[#rewrite_by_lua|rewrite_by_lua*]]. Since dev, the ngx.ctx created during a SSL handshake @@ -4721,7 +4840,7 @@ Just as [[#ngx.print|ngx.print]] but also emit a trailing newline. '''syntax:''' ''ngx.log(log_level, ...)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Log arguments concatenated to error.log with the given logging level. @@ -4753,7 +4872,7 @@ Since v0.8.3 this function returns 1 on success, or re '''syntax:''' ''ngx.exit(status)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' When status >= 200 (i.e., ngx.HTTP_OK and above), it will interrupt the execution of the current request and return status code to Nginx. @@ -4834,7 +4953,7 @@ Since v0.8.3 this function returns 1 on success, or re '''syntax:''' ''ngx.sleep(seconds)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Sleeps for the specified seconds without blocking. One can specify time resolution up to 0.001 seconds (i.e., one millisecond). @@ -4848,7 +4967,7 @@ This method was introduced in the 0.5.0rc30 release. '''syntax:''' ''newstr = ngx.escape_uri(str, type?)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Since `v0.10.16`, this function accepts an optional type argument. It accepts the following values (defaults to `2`): @@ -4864,7 +4983,7 @@ alphabetic characters, digits, -, ., _, '''syntax:''' ''newstr = ngx.unescape_uri(str)'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Unescape str as an escaped URI component. @@ -4900,7 +5019,7 @@ gives the output '''syntax:''' ''str = ngx.encode_args(table)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_client_hello_by_lua*'' Encode the Lua table to a query args string according to the URI encoded rules. @@ -4952,7 +5071,7 @@ This method was first introduced in the v0.3.1rc27 release. '''syntax:''' ''table, err = ngx.decode_args(str, max_args?)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Decodes a URI encoded query-string into a Lua table. This is the inverse function of [[#ngx.encode_args|ngx.encode_args]]. @@ -4972,7 +5091,7 @@ This method was introduced in the v0.5.0rc29. '''syntax:''' ''newstr = ngx.encode_base64(str, no_padding?)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Encodes str to a base64 digest. @@ -4982,7 +5101,7 @@ Since the 0.9.16 release, an optional boolean-typed no_paddin '''syntax:''' ''newstr = ngx.decode_base64(str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Decodes the str argument as a base64 digest to the raw form. Returns nil if str is not well formed. @@ -4990,7 +5109,7 @@ Decodes the str argument as a base64 digest to the raw form. Return '''syntax:''' ''intval = ngx.crc32_short(str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Calculates the CRC-32 (Cyclic Redundancy Code) digest for the str argument. @@ -5004,7 +5123,7 @@ This API was first introduced in the v0.3.1rc8 release. '''syntax:''' ''intval = ngx.crc32_long(str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Calculates the CRC-32 (Cyclic Redundancy Code) digest for the str argument. @@ -5018,7 +5137,7 @@ This API was first introduced in the v0.3.1rc8 release. '''syntax:''' ''digest = ngx.hmac_sha1(secret_key, str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Computes the [https://en.wikipedia.org/wiki/HMAC HMAC-SHA1] digest of the argument str and turns the result using the secret key . @@ -5047,7 +5166,7 @@ This function was first introduced in the v0.3.1rc29 release. '''syntax:''' ''digest = ngx.md5(str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Returns the hexadecimal representation of the MD5 digest of the str argument. @@ -5071,7 +5190,7 @@ See [[#ngx.md5_bin|ngx.md5_bin]] if the raw binary MD5 digest is required. '''syntax:''' ''digest = ngx.md5_bin(str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Returns the binary form of the MD5 digest of the str argument. @@ -5081,7 +5200,7 @@ See [[#ngx.md5|ngx.md5]] if the hexadecimal form of the MD5 digest is required. '''syntax:''' ''digest = ngx.sha1_bin(str)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Returns the binary form of the SHA-1 digest of the str argument. @@ -5093,7 +5212,7 @@ This function was first introduced in the v0.5.0rc6. '''syntax:''' ''quoted_value = ngx.quote_sql_str(raw_value)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Returns a quoted SQL string literal according to the MySQL quoting rules. @@ -5101,7 +5220,7 @@ Returns a quoted SQL string literal according to the MySQL quoting rules. '''syntax:''' ''str = ngx.today()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns current date (in the format yyyy-mm-dd) from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -5111,7 +5230,7 @@ This is the local time. '''syntax:''' ''secs = ngx.time()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns the elapsed seconds from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -5121,7 +5240,7 @@ Updates of the Nginx time cache can be forced by calling [[#ngx.update_time|ngx. '''syntax:''' ''secs = ngx.now()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns a floating-point number for the elapsed time in seconds (including milliseconds as the decimal part) from the epoch for the current time stamp from the Nginx cached time (no syscall involved unlike Lua's date library). @@ -5133,7 +5252,7 @@ This API was first introduced in v0.3.1rc32. '''syntax:''' ''ngx.update_time()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Forcibly updates the Nginx current time cache. This call involves a syscall and thus has some overhead, so do not abuse it. @@ -5143,7 +5262,7 @@ This API was first introduced in v0.3.1rc32. '''syntax:''' ''str = ngx.localtime()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the Nginx cached time (no syscall involved unlike Lua's [https://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). @@ -5153,7 +5272,7 @@ This is the local time. '''syntax:''' ''str = ngx.utctime()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns the current time stamp (in the format yyyy-mm-dd hh:mm:ss) of the Nginx cached time (no syscall involved unlike Lua's [https://www.lua.org/manual/5.1/manual.html#pdf-os.date os.date] function). @@ -5163,7 +5282,7 @@ This is the UTC time. '''syntax:''' ''str = ngx.cookie_time(sec)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns a formatted string can be used as the cookie expiration time. The parameter sec is the time stamp in seconds (like those returned from [[#ngx.time|ngx.time]]). @@ -5176,7 +5295,7 @@ Returns a formatted string can be used as the cookie expiration time. The parame '''syntax:''' ''str = ngx.http_time(sec)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns a formated string can be used as the http header time (for example, being used in Last-Modified header). The parameter sec is the time stamp in seconds (like those returned from [[#ngx.time|ngx.time]]). @@ -5189,7 +5308,7 @@ Returns a formated string can be used as the http header time (for example, bein '''syntax:''' ''sec = ngx.parse_http_time(str)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Parse the http time string (as returned by [[#ngx.http_time|ngx.http_time]]) into seconds. Returns the seconds or nil if the input string is in bad forms. @@ -5212,7 +5331,7 @@ Returns true if the current request is an Nginx subrequest, or subject string using the Perl compatible regular expression regex with the optional options. @@ -5360,7 +5479,7 @@ This feature was introduced in the v0.2.1rc11 release. '''syntax:''' ''from, to, err = ngx.re.find(subject, regex, options?, ctx?, nth?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Similar to [[#ngx.re.match|ngx.re.match]] but only returns the beginning index (from) and end index (to) of the matched substring. The returned indexes are 1-based and can be fed directly into the [https://www.lua.org/manual/5.1/manual.html#pdf-string.sub string.sub] API function to obtain the matched substring. @@ -5410,7 +5529,7 @@ This API function was first introduced in the v0.9.2 release. '''syntax:''' ''iterator, err = ngx.re.gmatch(subject, regex, options?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Similar to [[#ngx.re.match|ngx.re.match]], but returns a Lua iterator instead, so as to let the user programmer iterate all the matches over the string argument with the PCRE regex. @@ -5484,7 +5603,7 @@ This feature was first introduced in the v0.2.1rc12 release. '''syntax:''' ''newstr, n, err = ngx.re.sub(subject, regex, replace, options?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Substitutes the first match of the Perl compatible regular expression regex on the subject argument string with the string or function argument replace. The optional options argument has exactly the same meaning as in [[#ngx.re.match|ngx.re.match]]. @@ -5545,7 +5664,7 @@ This feature was first introduced in the v0.2.1rc13 release. '''syntax:''' ''newstr, n, err = ngx.re.gsub(subject, regex, replace, options?)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Just like [[#ngx.re.sub|ngx.re.sub]], but does global substitution. @@ -5581,7 +5700,7 @@ This feature was first introduced in the v0.2.1rc15 release. '''syntax:''' ''dict = ngx.shared[name_var]'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Fetching the shm-based Lua dictionary object for the shared memory zone named DICT defined by the [[#lua_shared_dict|lua_shared_dict]] directive. @@ -5661,7 +5780,7 @@ This feature was first introduced in the v0.3.1rc22 release. '''syntax:''' ''value, flags = ngx.shared.DICT:get(key)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Retrieving the value in the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] for the key key. If the key does not exist or has expired, then nil will be returned. @@ -5695,7 +5814,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''value, flags, stale = ngx.shared.DICT:get_stale(key)'' -'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Similar to the [[#ngx.shared.DICT.get|get]] method but returns the value even if the key has already expired. @@ -5711,7 +5830,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''success, err, forcible = ngx.shared.DICT:set(key, value, exptime?, flags?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Unconditionally sets a key-value pair into the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns three values: @@ -5759,7 +5878,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''ok, err = ngx.shared.DICT:safe_set(key, value, exptime?, flags?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Similar to the [[#ngx.shared.DICT.set|set]] method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return nil and the string "no memory". @@ -5771,7 +5890,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''success, err, forcible = ngx.shared.DICT:add(key, value, exptime?, flags?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Just like the [[#ngx.shared.DICT.set|set]] method, but only stores the key-value pair into the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] if the key does ''not'' exist. @@ -5785,7 +5904,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''ok, err = ngx.shared.DICT:safe_add(key, value, exptime?, flags?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Similar to the [[#ngx.shared.DICT.add|add]] method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return nil and the string "no memory". @@ -5797,7 +5916,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''success, err, forcible = ngx.shared.DICT:replace(key, value, exptime?, flags?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Just like the [[#ngx.shared.DICT.set|set]] method, but only stores the key-value pair into the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] if the key ''does'' exist. @@ -5811,7 +5930,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''ngx.shared.DICT:delete(key)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Unconditionally removes the key-value pair from the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. @@ -5825,7 +5944,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''newval, err, forcible? = ngx.shared.DICT:incr(key, value, init?, init_ttl?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' '''optional requirement:''' resty.core.shdict or resty.core @@ -5876,7 +5995,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''length, err = ngx.shared.DICT:lpush(key, value)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Inserts the specified (numerical or string) value at the head of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the number of elements in the list after the push operation. @@ -5892,7 +6011,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''length, err = ngx.shared.DICT:rpush(key, value)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Similar to the [[#ngx.shared.DICT.lpush|lpush]] method, but inserts the specified (numerical or string) value at the tail of the list named key. @@ -5904,7 +6023,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''val, err = ngx.shared.DICT:lpop(key)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Removes and returns the first element of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. @@ -5918,7 +6037,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''val, err = ngx.shared.DICT:rpop(key)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Removes and returns the last element of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. @@ -5932,7 +6051,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''len, err = ngx.shared.DICT:llen(key)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Returns the number of elements in the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. @@ -5946,7 +6065,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''ttl, err = ngx.shared.DICT:ttl(key)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' '''requires:''' resty.core.shdict or resty.core @@ -5980,7 +6099,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''success, err = ngx.shared.DICT:expire(key, exptime)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' '''requires:''' resty.core.shdict or resty.core @@ -6016,7 +6135,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''ngx.shared.DICT:flush_all()'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Flushes out all the items in the dictionary. This method does not actually free up all the memory blocks in the dictionary but just marks all the existing items as expired. @@ -6028,7 +6147,7 @@ See also [[#ngx.shared.DICT.flush_expired|ngx.shared.DICT.flush_expired]] and [[ '''syntax:''' ''flushed = ngx.shared.DICT:flush_expired(max_count?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Flushes out the expired items in the dictionary, up to the maximal number specified by the optional max_count argument. When the max_count argument is given 0 or not given at all, then it means unlimited. Returns the number of items that have actually been flushed. @@ -6042,7 +6161,7 @@ See also [[#ngx.shared.DICT.flush_all|ngx.shared.DICT.flush_all]] and [[#ngx.sha '''syntax:''' ''keys = ngx.shared.DICT:get_keys(max_count?)'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Fetch a list of the keys from the dictionary, up to . @@ -6056,7 +6175,7 @@ This feature was first introduced in the v0.7.3 release. '''syntax:''' ''capacity_bytes = ngx.shared.DICT:capacity()'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' '''requires:''' resty.core.shdict or resty.core @@ -6084,7 +6203,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''free_page_bytes = ngx.shared.DICT:free_space()'' -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' '''requires:''' resty.core.shdict or resty.core @@ -6123,7 +6242,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''syntax:''' ''udpsock = ngx.socket.udp()'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Creates and returns a UDP or datagram-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: @@ -6145,7 +6264,7 @@ See also [[#ngx.socket.tcp|ngx.socket.tcp]]. '''syntax:''' ''ok, err = udpsock:setpeername("unix:/path/to/unix-domain.socket")'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Attempts to connect a UDP socket object to a remote server or to a datagram unix domain socket file. Because the datagram protocol is actually connection-less, this method does not really establish a "connection", but only just set the name of the remote peer for subsequent read/write operations. @@ -6202,7 +6321,7 @@ This method was first introduced in the v0.5.7 release. '''syntax:''' ''ok, err = udpsock:send(data)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Sends data on the current UDP or datagram unix domain socket object. @@ -6216,7 +6335,7 @@ This feature was first introduced in the v0.5.7 release. '''syntax:''' ''data, err = udpsock:receive(size?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Receives data from the UDP or datagram unix domain socket object with an optional receive buffer size argument, size. @@ -6248,7 +6367,7 @@ This feature was first introduced in the v0.5.7 release. '''syntax:''' ''ok, err = udpsock:close()'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Closes the current UDP or datagram unix domain socket. It returns the 1 in case of success and returns nil with a string describing the error otherwise. @@ -6260,7 +6379,7 @@ This feature was first introduced in the v0.5.7 release. '''syntax:''' ''udpsock:settimeout(time)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Set the timeout value in milliseconds for subsequent socket operations (like [[#udpsock:receive|receive]]). @@ -6279,7 +6398,7 @@ This API function was first added to the v0.10.1 release. '''syntax:''' ''tcpsock = ngx.socket.tcp()'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Creates and returns a TCP or stream-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: @@ -6325,7 +6444,7 @@ See also [[#ngx.socket.udp|ngx.socket.udp]]. '''syntax:''' ''ok, err = tcpsock:connect("unix:/path/to/unix-domain.socket", options_table?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Attempts to connect a TCP socket object to a remote server or to a stream unix domain socket file without blocking. @@ -6438,7 +6557,7 @@ This method was first introduced in the v0.5.0rc1 release. '''syntax:''' ''session, err = tcpsock:sslhandshake(reused_session?, server_name?, ssl_verify?, send_status_req?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Does SSL/TLS handshake on the currently established connection. @@ -6481,7 +6600,7 @@ This method was first introduced in the v0.9.11 release. '''syntax:''' ''bytes, err = tcpsock:send(data)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Sends data without blocking on the current TCP or Unix Domain Socket connection. @@ -6510,7 +6629,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''data, err, partial = tcpsock:receive(pattern?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Receives data from the connected socket according to the reading pattern or size. @@ -6549,7 +6668,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''data, err = tcpsock:receiveany(max)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Returns any data received by the connected socket, at most max bytes. @@ -6580,7 +6699,7 @@ This feature was first introduced in the v0.10.14 release. '''syntax:''' ''iterator = tcpsock:receiveuntil(pattern, options?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' This method returns an iterator Lua function that can be called to read the data stream until it sees the specified pattern or an error occurs. @@ -6673,7 +6792,7 @@ This method was first introduced in the v0.5.0rc1 release. '''syntax:''' ''ok, err = tcpsock:close()'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Closes the current TCP or stream unix domain socket. It returns the 1 in case of success and returns nil with a string describing the error otherwise. @@ -6687,7 +6806,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''tcpsock:settimeout(time)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Set the timeout value in milliseconds for subsequent socket operations ([[#tcpsock:connect|connect]], [[#tcpsock:receive|receive]], and iterators returned from [[#tcpsock:receiveuntil|receiveuntil]]). @@ -6701,7 +6820,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''tcpsock:settimeouts(connect_timeout, send_timeout, read_timeout)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Respectively sets the connect, send, and read timeout thresholds (in milliseconds) for subsequent socket operations ([[#tcpsock:connect|connect]], [[#tcpsock:send|send]], [[#tcpsock:receive|receive]], and iterators returned from [[#tcpsock:receiveuntil|receiveuntil]]). @@ -6718,7 +6837,7 @@ This feature was first introduced in the v0.10.7 release. '''syntax:''' ''ok, err = tcpsock:setoption(option, value?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' This function is added for [http://w3.impa.br/~diego/software/luasocket/tcp.html LuaSocket] API compatibility and does nothing for now. Its functionality is implemented v0.10.18. @@ -6812,7 +6931,7 @@ These options described above are supported in v0.10.18, and more o '''syntax:''' ''ok, err = tcpsock:setkeepalive(timeout?, size?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Puts the current socket's connection immediately into the cosocket built-in connection pool and keep it alive until other [[#tcpsock:connect|connect]] method calls request it or the associated maximal idle timeout is expired. @@ -6856,7 +6975,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''count, err = tcpsock:getreusedtimes()'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' This method returns the (successfully) reused times for the current connection. In case of error, it returns nil and a string describing the error. @@ -6891,7 +7010,7 @@ This feature was first introduced in the v0.5.0rc1 release. '''syntax:''' ''str = ngx.get_phase()'' -'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_by_lua*, init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Retrieves the current running phase name. Possible return values are @@ -6905,6 +7024,8 @@ Retrieves the current running phase name. Possible return values are : for the context of [[#ssl_session_fetch_by_lua_block|ssl_session_fetch_by_lua*]]. * ssl_session_store : for the context of [[#ssl_session_store_by_lua_block|ssl_session_store_by_lua*]]. +* ssl_client_hello +: for the context of [[#ssl_client_hello_by_lua_block|ssl_client_hello_by_lua*]]. * set : for the context of [[#set_by_lua|set_by_lua*]]. * rewrite @@ -6932,7 +7053,7 @@ This API was first introduced in the v0.5.10 release. '''syntax:''' ''co = ngx.thread.spawn(func, arg1, arg2, ...)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Spawns a new user "light thread" with the Lua function func as well as those optional arguments arg1, arg2, and etc. Returns a Lua thread (or Lua coroutine) object represents this "light thread". @@ -7066,7 +7187,7 @@ This API was first enabled in the v0.7.0 release. '''syntax:''' ''ok, res1, res2, ... = ngx.thread.wait(thread1, thread2, ...)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Waits on one or more child "light threads" and returns the results of the first "light thread" that terminates (either successfully or with an error). @@ -7165,7 +7286,7 @@ This API was first enabled in the v0.7.0 release. '''syntax:''' ''ok, err = ngx.thread.kill(thread)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' Kills a running "light thread" created by [[#ngx.thread.spawn|ngx.thread.spawn]]. Returns a true value when successful or nil and a string describing the error otherwise. @@ -7214,7 +7335,7 @@ See also [[#lua_check_client_abort|lua_check_client_abort]]. '''syntax:''' ''hdl, err = ngx.timer.at(delay, callback, user_arg1, user_arg2, ...)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Creates an Nginx timer with a user callback function as well as optional user arguments. @@ -7352,7 +7473,7 @@ This API was first introduced in the v0.8.0 release. '''syntax:''' ''hdl, err = ngx.timer.every(delay, callback, user_arg1, user_arg2, ...)'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Similar to the [[#ngx.timer.at|ngx.timer.at]] API function, but @@ -7372,7 +7493,7 @@ This API was first introduced in the v0.10.9 release. '''syntax:''' ''count = ngx.timer.running_count()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns the number of timers currently running. @@ -7382,7 +7503,7 @@ This directive was first introduced in the v0.9.20 release. '''syntax:''' ''count = ngx.timer.pending_count()'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' Returns the number of pending timers. @@ -7575,7 +7696,7 @@ This feature requires at least ngx_lua v0.10.0. '''syntax:''' ''res = ndk.set_var.DIRECTIVE_NAME'' -'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*'' +'''context:''' ''init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua*'' This mechanism allows calling other Nginx C modules' directives that are implemented by [https://github.com/simplresty/ngx_devel_kit Nginx Devel Kit] (NDK)'s set_var submodule's ndk_set_var_value. @@ -7613,7 +7734,7 @@ This feature requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel '''syntax:''' ''co = coroutine.create(f)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Creates a user Lua coroutines with a Lua function, and returns a coroutine object. @@ -7627,7 +7748,7 @@ This API was first introduced in the v0.6.0 release. '''syntax:''' ''ok, ... = coroutine.resume(co, ...)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Resumes the executation of a user Lua coroutine object previously yielded or just created. @@ -7641,7 +7762,7 @@ This API was first introduced in the v0.6.0 release. '''syntax:''' ''... = coroutine.yield(...)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Yields the execution of the current user Lua coroutine. @@ -7655,7 +7776,7 @@ This API was first introduced in the v0.6.0 release. '''syntax:''' ''co = coroutine.wrap(f)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Similar to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.wrap coroutine.wrap] API, but works in the context of the Lua coroutines created by ngx_lua. @@ -7667,7 +7788,7 @@ This API was first introduced in the v0.6.0 release. '''syntax:''' ''co = coroutine.running()'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Identical to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running coroutine.running] API. @@ -7679,7 +7800,7 @@ This API was first enabled in the v0.6.0 release. '''syntax:''' ''status = coroutine.status(co)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' Identical to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.status coroutine.status] API. diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index b3b6f59a0f..f75fa1a1ad 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -125,20 +125,21 @@ typedef struct { /* must be within 16 bit */ -#define NGX_HTTP_LUA_CONTEXT_SET 0x0001 -#define NGX_HTTP_LUA_CONTEXT_REWRITE 0x0002 -#define NGX_HTTP_LUA_CONTEXT_ACCESS 0x0004 -#define NGX_HTTP_LUA_CONTEXT_CONTENT 0x0008 -#define NGX_HTTP_LUA_CONTEXT_LOG 0x0010 -#define NGX_HTTP_LUA_CONTEXT_HEADER_FILTER 0x0020 -#define NGX_HTTP_LUA_CONTEXT_BODY_FILTER 0x0040 -#define NGX_HTTP_LUA_CONTEXT_TIMER 0x0080 -#define NGX_HTTP_LUA_CONTEXT_INIT_WORKER 0x0100 -#define NGX_HTTP_LUA_CONTEXT_BALANCER 0x0200 -#define NGX_HTTP_LUA_CONTEXT_SSL_CERT 0x0400 -#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE 0x0800 -#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x1000 -#define NGX_HTTP_LUA_CONTEXT_EXIT_WORKER 0x2000 +#define NGX_HTTP_LUA_CONTEXT_SET 0x0001 +#define NGX_HTTP_LUA_CONTEXT_REWRITE 0x0002 +#define NGX_HTTP_LUA_CONTEXT_ACCESS 0x0004 +#define NGX_HTTP_LUA_CONTEXT_CONTENT 0x0008 +#define NGX_HTTP_LUA_CONTEXT_LOG 0x0010 +#define NGX_HTTP_LUA_CONTEXT_HEADER_FILTER 0x0020 +#define NGX_HTTP_LUA_CONTEXT_BODY_FILTER 0x0040 +#define NGX_HTTP_LUA_CONTEXT_TIMER 0x0080 +#define NGX_HTTP_LUA_CONTEXT_INIT_WORKER 0x0100 +#define NGX_HTTP_LUA_CONTEXT_BALANCER 0x0200 +#define NGX_HTTP_LUA_CONTEXT_SSL_CERT 0x0400 +#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE 0x0800 +#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x1000 +#define NGX_HTTP_LUA_CONTEXT_EXIT_WORKER 0x2000 +#define NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO 0x4000 #define NGX_HTTP_LUA_FFI_NO_REQ_CTX -100 @@ -318,6 +319,11 @@ union ngx_http_lua_srv_conf_u { ngx_str_t ssl_sess_fetch_src; u_char *ssl_sess_fetch_src_key; int ssl_sess_fetch_src_ref; + + ngx_http_lua_srv_conf_handler_pt ssl_client_hello_handler; + ngx_str_t ssl_client_hello_src; + u_char *ssl_client_hello_src_key; + int ssl_client_hello_src_ref; } srv; #endif diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 9b51c4af37..bb18d8f5d9 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -370,6 +370,7 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, | NGX_HTTP_LUA_CONTEXT_TIMER | NGX_HTTP_LUA_CONTEXT_HEADER_FILTER | NGX_HTTP_LUA_CONTEXT_BALANCER + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO | NGX_HTTP_LUA_CONTEXT_SSL_CERT | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH, @@ -380,6 +381,7 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, } if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH)) { diff --git a/src/ngx_http_lua_ctx.c b/src/ngx_http_lua_ctx.c index dc84013493..d5431be724 100644 --- a/src/ngx_http_lua_ctx.c +++ b/src/ngx_http_lua_ctx.c @@ -88,6 +88,7 @@ ngx_http_lua_ffi_get_ctx_ref(ngx_http_request_t *r, int *in_ssl_phase, } *in_ssl_phase = ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE); *ssl_ctx_ref = LUA_NOREF; @@ -123,6 +124,7 @@ ngx_http_lua_ffi_set_ctx_ref(ngx_http_request_t *r, int ref) #if (NGX_HTTP_SSL) if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE)) { diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index baf8ad4a40..5a392ff648 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -26,6 +26,7 @@ #include "ngx_http_lua_probe.h" #include "ngx_http_lua_semaphore.h" #include "ngx_http_lua_balancer.h" +#include "ngx_http_lua_ssl_client_helloby.h" #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_ssl_session_storeby.h" #include "ngx_http_lua_ssl_session_fetchby.h" @@ -566,6 +567,20 @@ static ngx_command_t ngx_http_lua_cmds[] = { offsetof(ngx_http_lua_loc_conf_t, ssl_ciphers), NULL }, + { ngx_string("ssl_client_hello_by_lua_block"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS, + ngx_http_lua_ssl_client_hello_by_lua_block, + NGX_HTTP_SRV_CONF_OFFSET, + 0, + (void *) ngx_http_lua_ssl_client_hello_handler_inline }, + + { ngx_string("ssl_client_hello_by_lua_file"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_http_lua_ssl_client_hello_by_lua, + NGX_HTTP_SRV_CONF_OFFSET, + 0, + (void *) ngx_http_lua_ssl_client_hello_handler_file }, + { ngx_string("ssl_certificate_by_lua_block"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS, ngx_http_lua_ssl_cert_by_lua_block, @@ -1086,6 +1101,10 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) } /* set by ngx_pcalloc: + * lscf->srv.ssl_client_hello_handler = NULL; + * lscf->srv.ssl_client_hello_src = { 0, NULL }; + * lscf->srv.ssl_client_hello_src_key = NULL; + * * lscf->srv.ssl_cert_handler = NULL; * lscf->srv.ssl_cert_src = { 0, NULL }; * lscf->srv.ssl_cert_src_key = NULL; @@ -1104,6 +1123,7 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) */ #if (NGX_HTTP_SSL) + lscf->srv.ssl_client_hello_src_ref = LUA_REFNIL; lscf->srv.ssl_cert_src_ref = LUA_REFNIL; lscf->srv.ssl_sess_store_src_ref = LUA_REFNIL; lscf->srv.ssl_sess_fetch_src_ref = LUA_REFNIL; @@ -1126,6 +1146,45 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) dd("merge srv conf"); + if (conf->srv.ssl_client_hello_src.len == 0) { + conf->srv.ssl_client_hello_src = prev->srv.ssl_client_hello_src; + conf->srv.ssl_client_hello_src_ref = prev->srv.ssl_client_hello_src_ref; + conf->srv.ssl_client_hello_src_key = prev->srv.ssl_client_hello_src_key; + conf->srv.ssl_client_hello_handler = prev->srv.ssl_client_hello_handler; + } + + if (conf->srv.ssl_client_hello_src.len) { + sscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_ssl_module); + if (sscf == NULL || sscf->ssl.ctx == NULL) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no ssl configured for the server"); + + return NGX_CONF_ERROR; + } +#ifdef LIBRESSL_VERSION_NUMBER + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "LibreSSL does not support by ssl_client_hello_by_lua*"); + return NGX_CONF_ERROR; + +#else + +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + + SSL_CTX_set_client_hello_cb(sscf->ssl.ctx, + ngx_http_lua_ssl_client_hello_handler, + NULL); + +#else + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "OpenSSL too old to support " + "ssl_client_hello_by_lua*"); + return NGX_CONF_ERROR; + +#endif +#endif + } + if (conf->srv.ssl_cert_src.len == 0) { conf->srv.ssl_cert_src = prev->srv.ssl_cert_src; conf->srv.ssl_cert_src_ref = prev->srv.ssl_cert_src_ref; diff --git a/src/ngx_http_lua_ssl.h b/src/ngx_http_lua_ssl.h index f50ecc5194..46b563701f 100644 --- a/src/ngx_http_lua_ssl.h +++ b/src/ngx_http_lua_ssl.h @@ -25,6 +25,7 @@ typedef struct { ngx_str_t session_id; int exit_code; /* exit code for openssl's + set_client_hello_cb or set_cert_cb callback */ int ctx_ref; /* reference to anchor @@ -34,6 +35,7 @@ typedef struct { unsigned done:1; unsigned aborted:1; + unsigned entered_client_hello_handler:1; unsigned entered_cert_handler:1; unsigned entered_sess_fetch_handler:1; } ngx_http_lua_ssl_ctx_t; diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c new file mode 100644 index 0000000000..a65b6e88f3 --- /dev/null +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -0,0 +1,705 @@ +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#if (NGX_HTTP_SSL) + + +#include "ngx_http_lua_cache.h" +#include "ngx_http_lua_initworkerby.h" +#include "ngx_http_lua_util.h" +#include "ngx_http_ssl_module.h" +#include "ngx_http_lua_contentby.h" +#include "ngx_http_lua_ssl_client_helloby.h" +#include "ngx_http_lua_directive.h" +#include "ngx_http_lua_ssl.h" + + +static void ngx_http_lua_ssl_client_hello_done(void *data); +static void ngx_http_lua_ssl_client_hello_aborted(void *data); +static u_char *ngx_http_lua_log_ssl_client_hello_error(ngx_log_t *log, + u_char *buf, size_t len); +static ngx_int_t ngx_http_lua_ssl_client_hello_by_chunk(lua_State *L, + ngx_http_request_t *r); + + +ngx_int_t +ngx_http_lua_ssl_client_hello_handler_file(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L) +{ + ngx_int_t rc; + + rc = ngx_http_lua_cache_loadfile(r->connection->log, L, + lscf->srv.ssl_client_hello_src.data, + &lscf->srv.ssl_client_hello_src_ref, + lscf->srv.ssl_client_hello_src_key); + if (rc != NGX_OK) { + return rc; + } + + /* make sure we have a valid code chunk */ + ngx_http_lua_assert(lua_isfunction(L, -1)); + + return ngx_http_lua_ssl_client_hello_by_chunk(L, r); +} + + +ngx_int_t +ngx_http_lua_ssl_client_hello_handler_inline(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L) +{ + ngx_int_t rc; + + rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, + lscf->srv.ssl_client_hello_src.data, + lscf->srv.ssl_client_hello_src.len, + &lscf->srv.ssl_client_hello_src_ref, + lscf->srv.ssl_client_hello_src_key, + "=ssl_client_hello_by_lua"); + if (rc != NGX_OK) { + return rc; + } + + /* make sure we have a valid code chunk */ + ngx_http_lua_assert(lua_isfunction(L, -1)); + + return ngx_http_lua_ssl_client_hello_by_chunk(L, r); +} + + +char * +ngx_http_lua_ssl_client_hello_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ + char *rv; + ngx_conf_t save; + + save = *cf; + cf->handler = ngx_http_lua_ssl_client_hello_by_lua; + cf->handler_conf = conf; + + rv = ngx_http_lua_conf_lua_block_parse(cf, cmd); + + *cf = save; + + return rv; +} + + +char * +ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ +#ifndef SSL_ERROR_WANT_CLIENT_HELLO_CB + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "at least OpenSSL 1.1.1 required but found " + OPENSSL_VERSION_TEXT); + + return NGX_CONF_ERROR; + +#else + + u_char *cache_key = NULL; + u_char *name; + ngx_str_t *value; + ngx_http_lua_srv_conf_t *lscf = conf; + + /* must specify a concrete handler */ + if (cmd->post == NULL) { + return NGX_CONF_ERROR; + } + + if (lscf->srv.ssl_client_hello_handler) { + return "is duplicate"; + } + + if (ngx_http_lua_ssl_init(cf->log) != NGX_OK) { + return NGX_CONF_ERROR; + } + + value = cf->args->elts; + + lscf->srv.ssl_client_hello_handler = + (ngx_http_lua_srv_conf_handler_pt) cmd->post; + + if (cmd->post == ngx_http_lua_ssl_client_hello_handler_file) { + /* Lua code in an external file */ + + name = ngx_http_lua_rebase_path(cf->pool, value[1].data, + value[1].len); + if (name == NULL) { + return NGX_CONF_ERROR; + } + + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { + return NGX_CONF_ERROR; + } + + lscf->srv.ssl_client_hello_src.data = name; + lscf->srv.ssl_client_hello_src.len = ngx_strlen(name); + + } else { + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { + return NGX_CONF_ERROR; + } + + /* Don't eval nginx variables for inline lua code */ + lscf->srv.ssl_client_hello_src = value[1]; + + } + + lscf->srv.ssl_client_hello_src_key = cache_key; + + return NGX_CONF_OK; + +#endif /* NO SSL_ERROR_WANT_CLIENT_HELLO_CB */ +} + + +int +ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, + int *al, void *arg) +{ + lua_State *L; + ngx_int_t rc; + ngx_connection_t *c, *fc; + ngx_http_request_t *r = NULL; + ngx_pool_cleanup_t *cln; + ngx_http_connection_t *hc; + ngx_http_lua_srv_conf_t *lscf; + ngx_http_core_loc_conf_t *clcf; + ngx_http_lua_ssl_ctx_t *cctx; + ngx_http_core_srv_conf_t *cscf; + + c = ngx_ssl_get_connection(ssl_conn); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "ssl client hello: connection reusable: %ud", c->reusable); + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + + dd("ssl client hello handler, client-hello-ctx=%p", cctx); + + if (cctx && cctx->entered_client_hello_handler) { + /* not the first time */ + + if (cctx->done) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua_client_hello_by_lua: " + "client hello cb exit code: %d", + cctx->exit_code); + + dd("lua ssl client hello done, finally"); + return cctx->exit_code; + } + + return -1; + } + + dd("first time"); + +#if (nginx_version < 1017009) + ngx_reusable_connection(c, 0); +#endif + + hc = c->data; + + fc = ngx_http_lua_create_fake_connection(NULL); + if (fc == NULL) { + goto failed; + } + + fc->log->handler = ngx_http_lua_log_ssl_client_hello_error; + fc->log->data = fc; + + fc->addr_text = c->addr_text; + fc->listening = c->listening; + + r = ngx_http_lua_create_fake_request(fc); + if (r == NULL) { + goto failed; + } + + r->main_conf = hc->conf_ctx->main_conf; + r->srv_conf = hc->conf_ctx->srv_conf; + r->loc_conf = hc->conf_ctx->loc_conf; + + fc->log->file = c->log->file; + fc->log->log_level = c->log->log_level; + fc->ssl = c->ssl; + + clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); + + +#if nginx_version >= 1009000 + ngx_set_connection_log(fc, clcf->error_log); +#else + ngx_http_set_connection_log(fc, clcf->error_log); +#endif + + if (cctx == NULL) { + cctx = ngx_pcalloc(c->pool, sizeof(ngx_http_lua_ssl_ctx_t)); + if (cctx == NULL) { + goto failed; /* error */ + } + + cctx->ctx_ref = LUA_NOREF; + } + + cctx->exit_code = 1; /* successful by default */ + cctx->connection = c; + cctx->request = r; + cctx->entered_client_hello_handler = 1; + cctx->done = 0; + + dd("setting cctx"); + + if (SSL_set_ex_data(c->ssl->connection, ngx_http_lua_ssl_ctx_index, cctx) + == 0) + { + ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "SSL_set_ex_data() failed"); + goto failed; + } + + lscf = ngx_http_get_module_srv_conf(r, ngx_http_lua_module); + + /* TODO honor lua_code_cache off */ + L = ngx_http_lua_get_lua_vm(r, NULL); + + c->log->action = "loading SSL client hello by lua"; + + if (lscf->srv.ssl_client_hello_handler == NULL) { + cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module); + + ngx_log_error(NGX_LOG_ALERT, c->log, 0, + "no ssl_client_hello_by_lua* defined in " + "server %V", &cscf->server_name); + + goto failed; + } + + rc = lscf->srv.ssl_client_hello_handler(r, lscf, L); + + if (rc >= NGX_OK || rc == NGX_ERROR) { + cctx->done = 1; + + if (cctx->cleanup) { + *cctx->cleanup = NULL; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua_client_hello_by_lua: handler return value: %i, " + "client hello cb exit code: %d", rc, cctx->exit_code); + + c->log->action = "SSL handshaking"; + return cctx->exit_code; + } + + /* rc == NGX_DONE */ + + cln = ngx_pool_cleanup_add(fc->pool, 0); + if (cln == NULL) { + goto failed; + } + + cln->handler = ngx_http_lua_ssl_client_hello_done; + cln->data = cctx; + + if (cctx->cleanup == NULL) { + cln = ngx_pool_cleanup_add(c->pool, 0); + if (cln == NULL) { + goto failed; + } + + cln->data = cctx; + cctx->cleanup = &cln->handler; + } + + *cctx->cleanup = ngx_http_lua_ssl_client_hello_aborted; + + return -1; + +#if 1 +failed: + + if (r && r->pool) { + ngx_http_lua_free_fake_request(r); + } + + if (fc) { + ngx_http_lua_close_fake_connection(fc); + } + + return 0; +#endif +} + + +static void +ngx_http_lua_ssl_client_hello_done(void *data) +{ + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx = data; + + dd("lua ssl client hello done"); + + if (cctx->aborted) { + return; + } + + ngx_http_lua_assert(cctx->done == 0); + + cctx->done = 1; + + if (cctx->cleanup) { + *cctx->cleanup = NULL; + } + + c = cctx->connection; + + c->log->action = "SSL handshaking"; + + ngx_post_event(c->write, &ngx_posted_events); +} + + +static void +ngx_http_lua_ssl_client_hello_aborted(void *data) +{ + ngx_http_lua_ssl_ctx_t *cctx = data; + + dd("lua ssl client hello done"); + + if (cctx->done) { + /* completed successfully already */ + return; + } + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, cctx->connection->log, 0, + "lua_client_hello_by_lua: client hello cb aborted"); + + cctx->aborted = 1; + cctx->request->connection->ssl = NULL; + + ngx_http_lua_finalize_fake_request(cctx->request, NGX_ERROR); +} + + +static u_char * +ngx_http_lua_log_ssl_client_hello_error(ngx_log_t *log, + u_char *buf, size_t len) +{ + u_char *p; + ngx_connection_t *c; + + if (log->action) { + p = ngx_snprintf(buf, len, " while %s", log->action); + len -= p - buf; + buf = p; + } + + p = ngx_snprintf(buf, len, ", context: ssl_client_hello_by_lua*"); + len -= p - buf; + buf = p; + + c = log->data; + + if (c && c->addr_text.len) { + p = ngx_snprintf(buf, len, ", client: %V", &c->addr_text); + len -= p - buf; + buf = p; + } + + if (c && c->listening && c->listening->addr_text.len) { + p = ngx_snprintf(buf, len, ", server: %V", &c->listening->addr_text); + /* len -= p - buf; */ + buf = p; + } + + return buf; +} + + +static ngx_int_t +ngx_http_lua_ssl_client_hello_by_chunk(lua_State *L, ngx_http_request_t *r) +{ + int co_ref; + ngx_int_t rc; + lua_State *co; + ngx_http_lua_ctx_t *ctx; + ngx_http_cleanup_t *cln; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + if (ctx == NULL) { + ctx = ngx_http_lua_create_ctx(r); + if (ctx == NULL) { + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; + } + + } else { + dd("reset ctx"); + ngx_http_lua_reset_ctx(r, L, ctx); + } + + ctx->entered_content_phase = 1; + + /* {{{ new coroutine to handle request */ + co = ngx_http_lua_new_thread(r, L, &co_ref); + + if (co == NULL) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "lua: failed to create new coroutine to handle request"); + + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; + } + + /* move code closure to new coroutine */ + lua_xmove(L, co, 1); + +#ifndef OPENRESTY_LUAJIT + /* set closure's env table to new coroutine's globals table */ + ngx_http_lua_get_globals_table(co); + lua_setfenv(co, -2); +#endif + + /* save nginx request in coroutine globals table */ + ngx_http_lua_set_req(co, r); + + ctx->cur_co_ctx = &ctx->entry_co_ctx; + ctx->cur_co_ctx->co = co; + ctx->cur_co_ctx->co_ref = co_ref; +#ifdef NGX_LUA_USE_ASSERT + ctx->cur_co_ctx->co_top = 1; +#endif + + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + + /* register request cleanup hooks */ + if (ctx->cleanup == NULL) { + cln = ngx_http_cleanup_add(r, 0); + if (cln == NULL) { + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; + } + + cln->handler = ngx_http_lua_request_cleanup_handler; + cln->data = ctx; + ctx->cleanup = &cln->handler; + } + + ctx->context = NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO; + + rc = ngx_http_lua_run_thread(L, r, ctx, 0); + + if (rc == NGX_ERROR || rc >= NGX_OK) { + /* do nothing */ + + } else if (rc == NGX_AGAIN) { + rc = ngx_http_lua_content_run_posted_threads(L, r, ctx, 0); + + } else if (rc == NGX_DONE) { + rc = ngx_http_lua_content_run_posted_threads(L, r, ctx, 1); + + } else { + rc = NGX_OK; + } + + ngx_http_lua_finalize_request(r, rc); + return rc; +} + + +int +ngx_http_lua_ffi_ssl_get_client_hello_server_name(ngx_http_request_t *r, + const char **name, size_t *namelen, char **err) +{ + ngx_ssl_conn_t *ssl_conn; +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + const unsigned char *p; + size_t remaining, len; +#endif + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + remaining = 0; + + /* This code block is taken from OpenSSL's client_hello_select_server_ctx() + * */ + if (!SSL_client_hello_get0_ext(ssl_conn, TLSEXT_TYPE_server_name, &p, + &remaining)) + { + return NGX_DECLINED; + } + + if (remaining <= 2) { + *err = "Bad SSL Client Hello Extension"; + return NGX_ERROR; + } + + len = (*(p++) << 8); + len += *(p++); + if (len + 2 != remaining) { + *err = "Bad SSL Client Hello Extension"; + return NGX_ERROR; + } + + remaining = len; + if (remaining == 0 || *p++ != TLSEXT_NAMETYPE_host_name) { + *err = "Bad SSL Client Hello Extension"; + return NGX_ERROR; + } + + remaining--; + if (remaining <= 2) { + *err = "Bad SSL Client Hello Extension"; + return NGX_ERROR; + } + + len = (*(p++) << 8); + len += *(p++); + if (len + 2 > remaining) { + *err = "Bad SSL Client Hello Extension"; + return NGX_ERROR; + } + + remaining = len; + *name = (const char *) p; + *namelen = len; + + return NGX_OK; + +#else + *err = "OpenSSL too old to support this function"; + return NGX_ERROR; + +#endif + +#else + + *err = "no TLS extension support"; + return NGX_ERROR; +#endif +} + + +int +ngx_http_lua_ffi_ssl_get_client_hello_ext(ngx_http_request_t *r, + unsigned int type, const unsigned char **out, size_t *outlen, char **err) +{ + ngx_ssl_conn_t *ssl_conn; + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + if (SSL_client_hello_get0_ext(ssl_conn, type, out, outlen) == 0) { + return NGX_DECLINED; + } + + return NGX_OK; +#else + *err = "OpenSSL too old to support this function"; + return NGX_ERROR; +#endif + +} + + +int +ngx_http_lua_ffi_ssl_set_protocols(ngx_http_request_t *r, + int protocols, char **err) +{ + + ngx_ssl_conn_t *ssl_conn; + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + +#if OPENSSL_VERSION_NUMBER >= 0x009080dfL + /* only in 0.9.8m+ */ + SSL_clear_options(ssl_conn, + SSL_OP_NO_SSLv2|SSL_OP_NO_SSLv3|SSL_OP_NO_TLSv1); +#endif + + if (!(protocols & NGX_SSL_SSLv2)) { + SSL_set_options(ssl_conn, SSL_OP_NO_SSLv2); + } + + if (!(protocols & NGX_SSL_SSLv3)) { + SSL_set_options(ssl_conn, SSL_OP_NO_SSLv3); + } + + if (!(protocols & NGX_SSL_TLSv1)) { + SSL_set_options(ssl_conn, SSL_OP_NO_TLSv1); + } + +#ifdef SSL_OP_NO_TLSv1_1 + SSL_clear_options(ssl_conn, SSL_OP_NO_TLSv1_1); + if (!(protocols & NGX_SSL_TLSv1_1)) { + SSL_set_options(ssl_conn, SSL_OP_NO_TLSv1_1); + } +#endif + +#ifdef SSL_OP_NO_TLSv1_2 + SSL_clear_options(ssl_conn, SSL_OP_NO_TLSv1_2); + if (!(protocols & NGX_SSL_TLSv1_2)) { + SSL_set_options(ssl_conn, SSL_OP_NO_TLSv1_2); + } +#endif + +#ifdef SSL_OP_NO_TLSv1_3 + SSL_clear_options(ssl_conn, SSL_OP_NO_TLSv1_3); + if (!(protocols & NGX_SSL_TLSv1_3)) { + SSL_set_options(ssl_conn, SSL_OP_NO_TLSv1_3); + } +#endif + + return NGX_OK; +} + +#endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_ssl_client_helloby.h b/src/ngx_http_lua_ssl_client_helloby.h new file mode 100644 index 0000000000..682a7cf292 --- /dev/null +++ b/src/ngx_http_lua_ssl_client_helloby.h @@ -0,0 +1,35 @@ +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + +#ifndef _NGX_HTTP_LUA_SSL_CLIENT_HELLOBY_H_INCLUDED_ +#define _NGX_HTTP_LUA_SSL_CLIENT_HELLOBY_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +#if (NGX_HTTP_SSL) + +ngx_int_t ngx_http_lua_ssl_client_hello_handler_inline(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L); + +ngx_int_t ngx_http_lua_ssl_client_hello_handler_file(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L); + +char *ngx_http_lua_ssl_client_hello_by_lua_block(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf); + +char *ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); + +int ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, + int *al, void *arg); + + +#endif /* NGX_HTTP_SSL */ + + +#endif /* _NGX_HTTP_LUA_SSL_CLIENT_HELLOBY_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 1768e5e28e..8111dde762 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -33,11 +33,12 @@ #define NGX_HTTP_LUA_ESCAPE_HEADER_VALUE 8 #define NGX_HTTP_LUA_CONTEXT_YIELDABLE (NGX_HTTP_LUA_CONTEXT_REWRITE \ - | NGX_HTTP_LUA_CONTEXT_ACCESS \ - | NGX_HTTP_LUA_CONTEXT_CONTENT \ - | NGX_HTTP_LUA_CONTEXT_TIMER \ - | NGX_HTTP_LUA_CONTEXT_SSL_CERT \ - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH) + | NGX_HTTP_LUA_CONTEXT_ACCESS \ + | NGX_HTTP_LUA_CONTEXT_CONTENT \ + | NGX_HTTP_LUA_CONTEXT_TIMER \ + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO \ + | NGX_HTTP_LUA_CONTEXT_SSL_CERT \ + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH) /* key in Lua vm registry for all the "ngx.ctx" tables */ @@ -56,6 +57,8 @@ : (c) == NGX_HTTP_LUA_CONTEXT_INIT_WORKER ? "init_worker_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_EXIT_WORKER ? "exit_worker_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_BALANCER ? "balancer_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO ? \ + "ssl_client_hello_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CERT ? "ssl_certificate_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE ? \ "ssl_session_store_by_lua*" \ diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t new file mode 100644 index 0000000000..79787f63a0 --- /dev/null +++ b/t/166-ssl-client-hello.t @@ -0,0 +1,2567 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +repeat_each(3); + +# All these tests need to have new openssl +my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; +my $openssl_version = eval { `$NginxBinary -V 2>&1` }; + +if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { + plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +} else { + plan tests => repeat_each() * (blocks() * 6 + 6); +} + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); +$ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; + +#log_level 'warn'; +log_level 'debug'; + +no_long_string(); +#no_diff(); + +run_tests(); + +__DATA__ + +=== TEST 1: simple logging +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" + +--- no_error_log +[error] +[alert] +--- grep_error_log eval: qr/ssl_client_hello_by_lua:.*?,|\bssl client hello: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log_out eval +# Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) +# before call ssl callback function +$Test::Nginx::Util::NginxVersion >= 1.017009 ? +qr/reusable connection: 0 +ssl client hello: connection reusable: 0 +ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ +: qr /reusable connection: 1 +ssl client hello: connection reusable: 1 +reusable connection: 0 +ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ + + + +=== TEST 2: sleep +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + local begin = ngx.now() + ngx.sleep(0.1) + print("elapsed in ssl client hello by lua: ", ngx.now() - begin) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log eval +[ +'lua ssl server name: "test.com"', +qr/elapsed in ssl client hello by lua: 0.(?:09|1\d)\d+,/, +] + +--- no_error_log +[error] +[alert] + + + +=== TEST 3: timer +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + local function f() + print("my timer run!") + end + local ok, err = ngx.timer.at(0, f) + if not ok then + ngx.log(ngx.ERR, "failed to create timer: ", err) + return + end + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +my timer run! + +--- no_error_log +[error] +[alert] + + + +=== TEST 4: cosocket +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local bytes, err = sock:send("flush_all\r\n") + if not bytes then + ngx.log(ngx.ERR, "failed to send flush_all command: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + print("received memc reply: ", res) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +received memc reply: OK + +--- no_error_log +[error] +[alert] + + + +=== TEST 5: ngx.exit(0) - no yield +--- http_config + server { + listen 127.0.0.2:8080 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.exit(0) + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.2", 8080) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: boolean + +--- error_log +lua exit with code 0 + +--- no_error_log +should never reached here +[error] +[alert] +[emerg] + + + +=== TEST 6: ngx.exit(ngx.ERROR) - no yield +--- http_config + server { + listen 127.0.0.2:8080 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.exit(ngx.ERROR) + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.2", 8080) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +'lua_client_hello_by_lua: handler return value: -1, client hello cb exit code: 0', +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, +'lua exit with code -1', +] + +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 7: ngx.exit(0) - yield +--- http_config + server { + listen 127.0.0.2:8080 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.sleep(0.001) + ngx.exit(0) + + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.2", 8080) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: boolean + +--- error_log +lua exit with code 0 + +--- no_error_log +should never reached here +[error] +[alert] +[emerg] + + + +=== TEST 8: ngx.exit(ngx.ERROR) - yield +--- http_config + server { + listen 127.0.0.2:8080 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.sleep(0.001) + ngx.exit(ngx.ERROR) + + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.2", 8080) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +'lua_client_hello_by_lua: client hello cb exit code: 0', +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, +'lua exit with code -1', +] + +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 9: lua exception - no yield +--- http_config + server { + listen 127.0.0.2:8080 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + error("bad bad bad") + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.2", 8080) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +'runtime error: ssl_client_hello_by_lua:2: bad bad bad', +'lua_client_hello_by_lua: handler return value: 500, client hello cb exit code: 0', +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, +qr/context: ssl_client_hello_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, +] + +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 10: lua exception - yield +--- http_config + server { + listen 127.0.0.2:8080 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.sleep(0.001) + error("bad bad bad") + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.2", 8080) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +'runtime error: ssl_client_hello_by_lua:3: bad bad bad', +'lua_client_hello_by_lua: client hello cb exit code: 0', +qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, +] + +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 11: get phase +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block {print("get_phase: ", ngx.get_phase())} + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end + collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata + +--- error_log +lua ssl server name: "test.com" +get_phase: ssl_client_hello + +--- no_error_log +[error] +[alert] + + + +=== TEST 12: connection aborted prematurely +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.sleep(0.3) + print("ssl-client-hello-by-lua: after sleeping") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(150) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t + +--- response_body +connected: 1 +failed to do SSL handshake: timeout + +--- error_log +lua ssl server name: "test.com" +ssl-client-hello-by-lua: after sleeping + +--- no_error_log +[error] +[alert] +--- wait: 0.6 + + + +=== TEST 13: subrequests disabled +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block {ngx.location.capture("/foo")} + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +failed to do SSL handshake: handshake failed + +--- error_log eval +[ +'lua ssl server name: "test.com"', +'ssl_client_hello_by_lua:1: API disabled in the context of ssl_client_hello_by_lua*', +qr/\[info\] .*?callback failed/, +] + +--- no_error_log +[alert] + + + +=== TEST 14: simple logging (by_lua_file) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_file html/a.lua; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } + +--- user_files +>>> a.lua +print("ssl client hello by lua is running!") + +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +a.lua:1: ssl client hello by lua is running! + +--- no_error_log +[error] +[alert] + + + +=== TEST 15: coroutine API +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield + + local function f() + local cnt = 0 + for i = 1, 20 do + print("co yield: ", cnt) + cy() + cnt = cnt + 1 + end + end + + local c = cc(f) + for i = 1, 3 do + print("co resume, status: ", coroutine.status(c)) + cr(c) + end + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- grep_error_log eval: qr/co (?:yield: \d+|resume, status: \w+)/ +--- grep_error_log_out +co resume, status: suspended +co yield: 0 +co resume, status: suspended +co yield: 1 +co resume, status: suspended +co yield: 2 + +--- error_log +lua ssl server name: "test.com" + +--- no_error_log +[error] +[alert] + + + +=== TEST 16: simple user thread wait with yielding +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + local function f() + ngx.sleep(0.01) + print("uthread: hello in thread") + return "done" + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "uthread: failed to spawn thread: ", err) + return ngx.exit(ngx.ERROR) + end + + print("uthread: thread created: ", coroutine.status(t)) + + local ok, res = ngx.thread.wait(t) + if not ok then + print("uthread: failed to wait thread: ", res) + return + end + + print("uthread: ", res) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- no_error_log +[error] +[alert] +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out +uthread: thread created: running +uthread: hello in thread +uthread: done + + + +=== TEST 17: simple logging - use ssl_client_hello_by_lua* on the http {} level +GitHub openresty/lua-resty-core#42 +--- http_config + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +ssl_client_hello_by_lua:1: ssl client hello by lua is running! + +--- no_error_log +[error] +[alert] + + + +=== TEST 18: simple logging - use ssl_client_hello_by_lua* on the http {} level and server {} level +--- http_config + ssl_client_hello_by_lua_block { print("ssl client hello by lua on the http level is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + ssl_client_hello_by_lua_block { print("ssl client hello by lua on the server level is running!") } + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +ssl_client_hello_by_lua:1: ssl client hello by lua on the server level is running! + +--- no_error_log +[error] +[alert] + + + +=== TEST 19: use ssl_client_hello_by_lua* on the server {} level with non-ssl server +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- no_error_log +ssl_client_hello_by_lua:1: ssl client hello by lua is running! +[error] +[alert] + + + +=== TEST 20: use ssl_client_hello_by_lua* on the http {} level with non-ssl server +--- http_config + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- no_error_log +ssl_client_hello_by_lua:1: ssl client hello by lua is running! +[error] +[alert] + + + +=== TEST 21: listen two ports (one for ssl and one for non-ssl) in one server - connect ssl port +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx2.sock; + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +ssl_client_hello_by_lua:1: ssl client hello by lua is running! + +--- no_error_log +[error] +[alert] + + + +=== TEST 22: listen two ports (one for ssl and one for non-ssl) in one server - connect non-ssl port +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx2.sock; + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx2.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + + +--- no_error_log +ssl_client_hello_by_lua:1: ssl client hello by lua is running! +[error] +[alert] + + + +=== TEST 23: simple logging - use ssl_client_hello_by_lua* in multiple virtual servers +--- http_config + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + ssl_client_hello_by_lua_block { print("ssl client hello by lua in server1 is running!") } + server_name test.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + ssl_client_hello_by_lua_block { print("ssl client hello by lua in server2 is running!") } + server_name test2.com; + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo2") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test2.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 57 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 5 +received: Connection: close +received: +received: foo2 +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +ssl_client_hello_by_lua:1: ssl client hello by lua in server1 is running! + +--- no_error_log +ssl_client_hello_by_lua:1: ssl client hello by lua in server2 is running! +[error] +[alert] + + + +=== TEST 24: simple logging (syslog) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + error_log syslog:server=127.0.0.1:12345 debug; + + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log eval +[ +qr/\[error\] .*? send\(\) failed/, +'lua ssl server name: "test.com"', +] +--- no_error_log +[alert] +ssl_client_hello_by_lua:1: ssl client hello by lua is running! + + + +=== TEST 25: get raw_client_addr - IPv4 +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + server { + listen 127.0.0.1:12345 ssl; + server_name test.com; + + ssl_client_hello_by_lua_block { + local ssl = require "ngx.ssl" + local byte = string.byte + local addr, addrtype, err = ssl.raw_client_addr() + local ip = string.format("%d.%d.%d.%d", byte(addr, 1), byte(addr, 2), + byte(addr, 3), byte(addr, 4)) + print("client ip: ", ip) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", 12345) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +client ip: 127.0.0.1 + +--- no_error_log +[error] +[alert] + + + +=== TEST 26: get raw_client_addr - unix domain socket +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_client_hello_by_lua_block { + local ssl = require "ngx.ssl" + local addr, addrtyp, err = ssl.raw_client_addr() + print("client socket file: ", addr) + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +client socket file: + +--- no_error_log +[error] +[alert] + + + +=== TEST 27: ssl_client_hello_by_lua* can yield when reading early data +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_early_data on; + server_tokens off; + + ssl_client_hello_by_lua_block { + local begin = ngx.now() + ngx.sleep(0.1) + print("elapsed in ssl_client_hello_by_lua*: ", ngx.now() - begin) + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: boolean +--- grep_error_log eval +qr/elapsed in ssl_client_hello_by_lua\*: 0\.(?:09|1\d)\d+,/, +--- grep_error_log_out eval +[ +qr/elapsed in ssl_client_hello_by_lua\*: 0\.(?:09|1\d)\d+,/, +qr/elapsed in ssl_client_hello_by_lua\*: 0\.(?:09|1\d)\d+,/, +qr/elapsed in ssl_client_hello_by_lua\*: 0\.(?:09|1\d)\d+,/, +] +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 28: cosocket (UDP) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_tokens off; + + ssl_client_hello_by_lua_block { + local sock = ngx.socket.udp() + + sock:settimeout(1000) + + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = sock:send(req) + if not ok then + ngx.log(ngx.ERR, "failed to send flush_all to memc: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + ngx.log(ngx.INFO, "received memc reply of ", #res, " bytes") + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +--- no_error_log +[error] +[alert] +[emerg] +--- grep_error_log eval: qr/received memc reply of \d+ bytes/ +--- grep_error_log_out eval +[ +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +'received memc reply of 12 bytes +', +] + + + +=== TEST 29: uthread (kill) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + server_tokens off; + + ssl_client_hello_by_lua_block { + local function f() + ngx.log(ngx.INFO, "uthread: hello from f()") + ngx.sleep(1) + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return ngx.exit(ngx.ERROR) + end + + local ok, res = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.ERR, "failed to kill thread: ", res) + return + end + + ngx.log(ngx.INFO, "uthread: killed") + + local ok, err = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.INFO, "uthread: failed to kill: ", err) + end + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +--- no_error_log +[error] +[alert] +[emerg] +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out +uthread: hello from f() +uthread: killed +uthread: failed to kill: already waited or killed From 5425d6d8e6da003a0bb395d147d8f972c23bc439 Mon Sep 17 00:00:00 2001 From: Jianyong Chen Date: Sat, 25 Sep 2021 00:16:29 +0800 Subject: [PATCH 536/848] bugfix: we should ignore match limit in DFA mode. (#1924) lua_regex_match_limit takes effect globally for all regexes used in ngx.re api. However, the match_limit field of pcre_extra block is not supported in pcre_dfa_exec()(so does match_limit_recursion), because it is meaningless in DFA matching. And pcre_dfa_exec() will return -18(PCRE_ERROR_DFA_UMLIMIT) when the directive is used. Here we just ignore the directive in DFA mode. --- src/ngx_http_lua_regex.c | 5 ++++- t/120-re-find.t | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index e4f36bc44e..03d1a4d320 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -311,7 +311,10 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, #endif /* LUA_HAVE_PCRE_JIT */ - if (sd && lmcf && lmcf->regex_match_limit > 0) { + if (sd + && lmcf && lmcf->regex_match_limit > 0 + && !(flags & NGX_LUA_RE_MODE_DFA)) + { sd->flags |= PCRE_EXTRA_MATCH_LIMIT; sd->match_limit = lmcf->regex_match_limit; } diff --git a/t/120-re-find.t b/t/120-re-find.t index f80ca4f57d..ddf24a98e7 100644 --- a/t/120-re-find.t +++ b/t/120-re-find.t @@ -917,3 +917,35 @@ to: 4 pos: 5 --- no_error_log [error] + + + +=== TEST 32: ignore match limit in DFA mode +--- http_config + lua_regex_match_limit 1; +--- config + location /re { + content_by_lua_block { + local s = "This is no more" + local from, to, err = ngx.re.find(s, "<.*>", "d") + if from then + ngx.say("from: ", from) + ngx.say("to: ", to) + ngx.say("matched: ", string.sub(s, from, to)) + else + if err then + ngx.say("error: ", err) + return + end + ngx.say("not matched!") + end + } + } +--- request + GET /re +--- response_body +from: 9 +to: 56 +matched: +--- no_error_log +[error] From 5095da4a5479c10237cc77e9f470760105cff31d Mon Sep 17 00:00:00 2001 From: doujiang Date: Sat, 2 Oct 2021 22:57:42 +0800 Subject: [PATCH 537/848] change: do not need to create the Lua request ctx data table from C (#1926) since we already changed to add the ctx table to the registry table at Lua land in resty.core.ctx. related change: https://github.com/openresty/lua-resty-core/commit/2d667ec81497a7a2ec10bd6b7d1f27509637c928 Notice: people who use the raw ffi C api to build their own ctx related Lua library(just like resty.core.ctx) need to require resty.core.ctx first. Co-authored-by: doujiang24 --- src/ngx_http_lua_util.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 38e1b68384..56b5989fe2 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -781,10 +781,13 @@ ngx_http_lua_init_registry(lua_State *L, ngx_log_t *log) lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ - /* create the registry entry for the Lua request ctx data table */ - lua_pushliteral(L, ngx_http_lua_ctx_tables_key); - lua_createtable(L, 0, 32 /* nrec */); - lua_rawset(L, LUA_REGISTRYINDEX); + /* + * the the Lua request ctx data table will create in resty.core.ctx, + * just equivalent to the following code: + * lua_pushliteral(L, ngx_http_lua_ctx_tables_key); + * lua_createtable(L, 0, 0); + * lua_rawset(L, LUA_REGISTRYINDEX); + */ /* create the registry entry for the Lua socket connection pool table */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( From a3ec1565e23a52a9d7c2fb2a19762e666fd4a629 Mon Sep 17 00:00:00 2001 From: ngtee8 <90826180+ngtee8@users.noreply.github.com> Date: Sat, 2 Oct 2021 23:03:22 +0800 Subject: [PATCH 538/848] bugfix: should reset the value_len to 0 when reuse the expired list type key in shared dict. (#1928) --- src/ngx_http_lua_shdict.c | 2 +- t/145-shdict-list.t | 84 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 07effa88e2..8421ac1714 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -815,7 +815,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) "type matched, reusing it"); sd->expires = 0; - + sd->value_len = 0; /* free list nodes */ queue = ngx_http_lua_shdict_get_list_head(sd, key.len); diff --git a/t/145-shdict-list.t b/t/145-shdict-list.t index 9bb1592606..6a4759cd02 100644 --- a/t/145-shdict-list.t +++ b/t/145-shdict-list.t @@ -743,3 +743,87 @@ GET /test done --- no_error_log [error] + + + +=== TEST 20: push to an expired list +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + local len, err = dogs:lpush("cc", "1") --add another list to avoid key"aa" be cleaned (run ‘ngx_http_lua_shdict_expire(ctx, 1)’ may clean key ,ensure key'aa' not clean ,just expired)) + if not len then + ngx.say("push cc err: ", err) + end + local len, err = dogs:lpush("aa", "1") + if not len then + ngx.say("push1 err: ", err) + end + local succ, err = dogs:expire("aa", 0.2) + if not succ then + ngx.say("expire err: ",err) + end + ngx.sleep(0.3) -- list aa expired + local len, err = dogs:lpush("aa", "2") --push to an expired list may set as a new list + if not len then + ngx.say("push2 err: ", err) + end + local len, err = dogs:llen("aa") -- new list len is 1 + if not len then + ngx.say("llen err: ", err) + else + ngx.say("aa:len :", dogs:llen("aa")) + end + } + } + +--- request +GET /test +--- response_body +aa:len :1 +--- no_error_log +[error] + + + +=== TEST 21: push to an expired list then pop many time (more then list len ) +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + local len, err = dogs:lpush("cc", "1") --add another list to avoid key"aa" be cleaned (run ‘ngx_http_lua_shdict_expire(ctx, 1)’ may clean key ,ensure key'aa' not clean ,just expired)) + if not len then + ngx.say("push cc err: ", err) + end + local len, err = dogs:lpush("aa", "1") + if not len then + ngx.say("push1 err: ", err) + end + local succ, err = dogs:expire("aa", 0.2) + if not succ then + ngx.say("expire err: ",err) + end + ngx.sleep(0.3) -- list aa expired + local len, err = dogs:lpush("aa", "2") --push to an expired list may set as a new list + if not len then + ngx.say("push2 err: ", err) + end + local val, err = dogs:lpop("aa") + if not val then + ngx.say("llen err: ", err) + end + local val, err = dogs:lpop("aa") -- val == nil + ngx.say("aa list value: ", val) + } + } + +--- request +GET /test +--- response_body +aa list value: nil +--- no_error_log +[error] From 250f31d011aedc18e103bb2df08a4f303843048d Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 4 Oct 2021 12:02:47 +0800 Subject: [PATCH 539/848] tests: the certificate of the Let's Encrypt has expired. (#1929) --- t/cert/dst-ca.crt | 116 +++++++++++++++++++++++----------------------- 1 file changed, 57 insertions(+), 59 deletions(-) diff --git a/t/cert/dst-ca.crt b/t/cert/dst-ca.crt index 738121f63a..e693fb3457 100644 --- a/t/cert/dst-ca.crt +++ b/t/cert/dst-ca.crt @@ -1,63 +1,61 @@ -subject=/C=US/O=Digital Signature Trust Co./OU=TrustID Server/CN=TrustID Server CA A5 -issuer=/O=Digital Signature Trust Co./CN=DST Root CA X3 -----BEGIN CERTIFICATE----- -MIIGiTCCBXGgAwIBAgIRALCqqJD4YLUEJLy76IZcvf8wDQYJKoZIhvcNAQEFBQAw -PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD -Ew5EU1QgUm9vdCBDQSBYMzAeFw0wNzA4MjQxNzU0MTlaFw0xNDA4MjQxNDEzMzVa -MGsxCzAJBgNVBAYTAlVTMSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVz -dCBDby4xFzAVBgNVBAsTDlRydXN0SUQgU2VydmVyMR0wGwYDVQQDExRUcnVzdElE -IFNlcnZlciBDQSBBNTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOae -jyufM+mrKgTCpPm9ha5UBAORNLhAv3PFlmzA8n6lnEvBTJfeG5vUZYQrlZMMKmZl -F1D3mYRFAhT/qFbBsTMlF9XsDNF40NB21ABTquuEB3Ji8mmC1Z2cZjwa4VeKxGIq -kTdBypQgWekopNVvoOGLPbRXgYKW4UCc6pt7Q+We+eUuonp2XIRIOvmJQtXb/rNz -4B1ahu3QB4HnystmQ9dW8BySSGDhZ/kzIcSU8GuHcL58SHsei07gjFzXRhMkkbJH -MT8zS/hrjNsB8Z5kamI4QL7nLVNuCF1Sg9zgB1POdMPLEnm4uDx+BRLwVMGZMwza -MNfEATnWTvLazzIDarMCAwEAAaOCA1IwggNOMA8GA1UdEwEB/wQFMAMBAf8wDgYD -VR0PAQH/BAQDAgHGMEcGA1UdJQRAMD4GCCsGAQUFBwMBBggrBgEFBQcDAwYIKwYB -BQUHAwUGCCsGAQUFBwMGBggrBgEFBQcDBwYKKwYBBAGCNwoDATCCATIGA1UdIASC -ASkwggElMIIBIQYKYIZIAYb5LwAGAzCCAREwSgYIKwYBBQUHAgEWPmh0dHBzOi8v -c2VjdXJlLmlkZW50cnVzdC5jb20vY2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRl -eC5odG1sMIHCBggrBgEFBQcCAjCBtRqBslRoaXMgVHJ1c3RJRCBTZXJ2ZXIgQ2Vy -dGlmaWNhdGUgaGFzIGJlZW4gaXNzdWVkIGluIGFjY29yZGFuY2Ugd2l0aCBJZGVu -VHJ1c3QncyBUcnVzdElEIENlcnRpZmljYXRlIFBvbGljeSBmb3VuZCBhdCBodHRw -czovL3NlY3VyZS5pZGVudHJ1c3QuY29tLyBjZXJ0aWZpY2F0ZXMvcG9saWN5L3Rz -L2luZGV4Lmh0bWwwgbkGA1UdHwSBsTCBrjAuoCygKoYoaHR0cDovL2NybC5pZGVu -dHJ1c3QuY29tL0RTVFJPT1RDQVgzLmNybDB8oHqgeIZ2bGRhcDovL2xkYXAuaWRl -bnRydXN0LmNvbS9jbj1EU1QlMjBSb290JTIwQ0ElMjBYMyxvPURpZ2l0YWwlMjBT -aWduYXR1cmUlMjBUcnVzdCUyMENvLj9jZXJ0aWZpY2F0ZVJldm9jYXRpb25MaXN0 -O2JpbmFyeTCBrwYIKwYBBQUHAQEEgaIwgZ8wJQYIKwYBBQUHMAGGGWh0dHA6Ly9v -Y3NwLmlkZW50cnVzdC5jb20wdgYIKwYBBQUHMAKGamxkYXA6Ly9sZGFwLmlkZW50 -cnVzdC5jb20vY249RFNUJTIwUm9vdCUyMENBJTIwWDMsbz1EaWdpdGFsJTIwU2ln -bmF0dXJlJTIwVHJ1c3QlMjBDby4/Y0FDZXJ0aWZpY2F0ZTtiaW5hcnkwHwYDVR0j -BBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAwHQYDVR0OBBYEFLeoy2iu3EVD0tyJ -EamBZhn5+3XpMA0GCSqGSIb3DQEBBQUAA4IBAQAtreoVqLQx+G35L8byM/IvGqNa -MzbWYzGIidH3+hT8B2e0duWfZCDazgu272ErWL/T4HL5psUEOIHfzNxEWHA24XhY -zBFsYR64vaTRV7T+iYKv5Nxptidk26zOZ2/F+TKZvOfhPg5IqnH3FyXHEcRDWm1/ -QJsZ4mqgUFx38pc50i1ed6XUdvBObMnGJc1fGdfVqcNqbvHACNrw+WJE9PIeMKJQ -HC/NkE87xFP/Q1OW7hPmqQvmIIbm6BLZ7U0kxqNL3HmegxkJ2yJQntsX+5BooJqj -/uVpwzAEei7IjKsdtomxkHCDPFGEJYBldDLBH0ScSlngQZ6V5HNM473l8bWs +MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw +TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh +cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw +WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg +RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK +AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP +R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx +sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm +NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg +Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG +/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC +AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB +Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA +FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw +AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw +Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB +gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W +PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl +ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz +CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm +lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4 +avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2 +yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O +yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids +hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+ +HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv +MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX +nLRbwHOoq7hHwg== -----END CERTIFICATE----- - -subject=/O=Digital Signature Trust Co./CN=DST Root CA X3 -issuer=/O=Digital Signature Trust Co./CN=DST Root CA X3 -----BEGIN CERTIFICATE----- -MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/ -MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT -DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow -PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD -Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB -AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O -rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq -OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b -xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw -7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD -aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV -HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG -SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69 -ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr -AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz -R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5 -JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo -Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ +MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw +TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh +cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4 +WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu +ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY +MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc +h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+ +0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U +A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW +T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH +B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC +B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv +KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn +OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn +jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw +qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI +rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV +HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq +hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL +ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ +3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK +NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5 +ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur +TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC +jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc +oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq +4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA +mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d +emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc= -----END CERTIFICATE----- - From 1ea5b09f18765e0b4b140d2d63f57917f3bd0857 Mon Sep 17 00:00:00 2001 From: Jinhua Tan <312841925@qq.com> Date: Thu, 7 Oct 2021 21:39:41 +0800 Subject: [PATCH 540/848] feature: auto assign pr to project (#1772) --- .github/workflows/label.yml | 21 +++++++++++++++++++++ .mergify.yml | 8 ++++++++ 2 files changed, 29 insertions(+) create mode 100644 .github/workflows/label.yml diff --git a/.github/workflows/label.yml b/.github/workflows/label.yml new file mode 100644 index 0000000000..28ee659e13 --- /dev/null +++ b/.github/workflows/label.yml @@ -0,0 +1,21 @@ +name: Auto Assign to Project(s) +on: + issues: + types: [labeled] + pull_request: + types: [labeled] +env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} +jobs: + assign_to_project: + runs-on: ubuntu-latest + name: Assign to Project + steps: + - name: Assign pull requests with `could-be-merged` label to `Waiting review` + uses: srggrs/assign-one-project-github-action@1.2.0 + if: | + contains(github.event.issue.labels.*.name, 'could-be-merged') || + contains(github.event.pull_request.labels.*.name, 'could-be-merged') + with: + project: "https://github.com/orgs/openresty/projects/1" + column_name: "Waiting Review from agentzh" diff --git a/.mergify.yml b/.mergify.yml index c89be4fb54..f1a8a2e255 100644 --- a/.mergify.yml +++ b/.mergify.yml @@ -33,3 +33,11 @@ pull_request_rules: label: remove: - needs-test-cases + - name: add label could-be-merged + conditions: + - "#approved-reviews-by>=2" + - status-success=Travis CI - Pull Request + actions: + label: + add: + - could-be-merged From c4ca520322204eeffc2befe218e818a06a9733d1 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 13 Oct 2021 21:43:47 +0800 Subject: [PATCH 541/848] tests: ca-certificates of Let's Encrypt has expired. (#1937) --- t/cert/dst-ca.crt | 83 ++++++++++++++--------------------------------- 1 file changed, 24 insertions(+), 59 deletions(-) diff --git a/t/cert/dst-ca.crt b/t/cert/dst-ca.crt index e693fb3457..7b714c60a9 100644 --- a/t/cert/dst-ca.crt +++ b/t/cert/dst-ca.crt @@ -1,61 +1,26 @@ +# Comodo AAA Services root -----BEGIN CERTIFICATE----- -MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw -TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh -cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw -WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg -RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK -AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP -R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx -sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm -NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg -Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG -/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC -AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB -Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA -FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw -AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw -Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB -gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W -PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl -ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz -CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm -lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4 -avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2 -yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O -yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids -hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+ -HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv -MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX -nLRbwHOoq7hHwg== ------END CERTIFICATE----- ------BEGIN CERTIFICATE----- -MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw -TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh -cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4 -WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu -ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY -MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc -h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+ -0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U -A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW -T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH -B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC -B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv -KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn -OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn -jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw -qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI -rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV -HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq -hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL -ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ -3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK -NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5 -ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur -TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC -jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc -oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq -4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA -mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d -emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc= +MIIEMjCCAxqgAwIBAgIBATANBgkqhkiG9w0BAQUFADB7MQswCQYDVQQGEwJHQjEb +MBkGA1UECAwSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHDAdTYWxmb3JkMRow +GAYDVQQKDBFDb21vZG8gQ0EgTGltaXRlZDEhMB8GA1UEAwwYQUFBIENlcnRpZmlj +YXRlIFNlcnZpY2VzMB4XDTA0MDEwMTAwMDAwMFoXDTI4MTIzMTIzNTk1OVowezEL +MAkGA1UEBhMCR0IxGzAZBgNVBAgMEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UE +BwwHU2FsZm9yZDEaMBgGA1UECgwRQ29tb2RvIENBIExpbWl0ZWQxITAfBgNVBAMM +GEFBQSBDZXJ0aWZpY2F0ZSBTZXJ2aWNlczCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBAL5AnfRu4ep2hxxNRUSOvkbIgwadwSr+GB+O5AL686tdUIoWMQua +BtDFcCLNSS1UY8y2bmhGC1Pqy0wkwLxyTurxFa70VJoSCsN6sjNg4tqJVfMiWPPe +3M/vg4aijJRPn2jymJBGhCfHdr/jzDUsi14HZGWCwEiwqJH5YZ92IFCokcdmtet4 +YgNW8IoaE+oxox6gmf049vYnMlhvB/VruPsUK6+3qszWY19zjNoFmag4qMsXeDZR +rOme9Hg6jc8P2ULimAyrL58OAd7vn5lJ8S3frHRNG5i1R8XlKdH5kBjHYpy+g8cm +ez6KJcfA3Z3mNWgQIJ2P2N7Sw4ScDV7oL8kCAwEAAaOBwDCBvTAdBgNVHQ4EFgQU +oBEKIz6W8Qfs4q8p74Klf9AwpLQwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQF +MAMBAf8wewYDVR0fBHQwcjA4oDagNIYyaHR0cDovL2NybC5jb21vZG9jYS5jb20v +QUFBQ2VydGlmaWNhdGVTZXJ2aWNlcy5jcmwwNqA0oDKGMGh0dHA6Ly9jcmwuY29t +b2RvLm5ldC9BQUFDZXJ0aWZpY2F0ZVNlcnZpY2VzLmNybDANBgkqhkiG9w0BAQUF +AAOCAQEACFb8AvCb6P+k+tZ7xkSAzk/ExfYAWMymtrwUSWgEdujm7l3sAg9g1o1Q +GE8mTgHj5rCl7r+8dFRBv/38ErjHT1r0iWAFf2C3BUrz9vHCv8S5dIa2LX1rzNLz +Rt0vxuBqw8M0Ayx9lt1awg6nCpnBBYurDC/zXDrPbDdVCYfeU0BsWO/8tqtlbgT2 +G9w84FoVxp7Z8VlIMCFlA2zs6SFz7JsDoeA3raAVGI/6ugLOpyypEBMs1OUIJqsi +l2D4kF501KKaU73yqWjgom7C12yxow+ev+to51byrvLjKzg6CYG1a4XXvi3tPxq3 +smPi9WIsgtRqAEFQ8TmDn5XpNpaYbg== -----END CERTIFICATE----- From acd53645754ce42b436cecb0d7a10b547d41fef6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 13 Oct 2021 23:42:17 +0800 Subject: [PATCH 542/848] bugfix: nginx crash when resolve an not exist domain in thread create by ngx.thread.spawn.(#1931) FIX #1915 The resolve ctxes were dded to a link list. function ngx_resolver_process_a iterate through the link list when got the DNS reply. When processing the first resolve ctx, all the three ctxes were freed. So when ngx_resolver_process_a continues to process the second ctx, it will free the ctx again. Co-authored-by: doujiang24 --- src/ngx_http_lua_socket_tcp.c | 15 +-- src/ngx_http_lua_socket_udp.c | 3 +- t/014-bugs.t | 198 +++++++++++++++++++++++++++++++++- t/127-uthread-kill.t | 3 +- 4 files changed, 202 insertions(+), 17 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 55bd203d9c..b7c3bdd45d 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -147,8 +147,6 @@ static void ngx_http_lua_socket_free_pool(ngx_log_t *log, static int ngx_http_lua_socket_shutdown_pool(lua_State *L); static void ngx_http_lua_socket_shutdown_pool_helper( ngx_http_lua_socket_pool_t *spool); -static void - ngx_http_lua_socket_empty_resolve_handler(ngx_resolver_ctx_t *ctx); static int ngx_http_lua_socket_prepare_error_retvals(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L, ngx_uint_t ft_type); #if (NGX_HTTP_SSL) @@ -1151,13 +1149,6 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) } -static void -ngx_http_lua_socket_empty_resolve_handler(ngx_resolver_ctx_t *ctx) -{ - /* do nothing */ -} - - static void ngx_http_lua_socket_resolve_handler(ngx_resolver_ctx_t *ctx) { @@ -6084,10 +6075,8 @@ ngx_http_lua_tcp_resolve_cleanup(void *data) return; } - /* just to be safer */ - rctx->handler = ngx_http_lua_socket_empty_resolve_handler; - - ngx_resolve_name_done(rctx); + /* postpone free the rctx in the handler */ + rctx->handler = ngx_resolve_name_done; } diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index fd3e074270..a1714fa444 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -1584,7 +1584,8 @@ ngx_http_lua_udp_resolve_cleanup(void *data) return; } - ngx_resolve_name_done(rctx); + /* postpone free the rctx in the handler */ + rctx->handler = ngx_resolve_name_done; } diff --git a/t/014-bugs.t b/t/014-bugs.t index dad084ec4d..88519c0caa 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -8,7 +8,7 @@ log_level('debug'); repeat_each(3); -plan tests => repeat_each() * (blocks() * 2 + 30); +plan tests => repeat_each() * (blocks() * 2 + 32); our $HtmlDir = html_dir; #warn $html_dir; @@ -1019,3 +1019,199 @@ write timer set: 1 --- no_error_log [error] [alert] + + + +=== TEST 42: tcp: nginx crash when resolve an not exist domain in ngx.thread.spawn +https://github.com/openresty/lua-nginx-module/issues/1915 +--- config + resolver $TEST_NGINX_RESOLVER ipv6=off; + location = /t { + content_by_lua_block { + local function tcp(host, port) + local sock = ngx.socket.tcp() + local ok,err = sock:connect(host, port) + if not ok then + ngx.log(ngx.WARN, "failed: ", err) + sock:close() + return false + end + + sock:close() + return true + end + + local host = "www.notexistdomain.com" + local port = 80 + + local threads = {} + for i = 1, 3 do + threads[i] = ngx.thread.spawn(tcp, host, port) + end + + local ok, res = ngx.thread.wait(threads[1],threads[2],threads[3]) + if not ok then + ngx.say("failed to wait thread") + return + end + + ngx.say("res: ", res) + + for i = 1, 3 do + ngx.thread.kill(threads[i]) + end + } + } + +--- request +GET /t +--- response_body +res: false +--- error_log +www.notexistdomain.com could not be resolved + + + +=== TEST 43: domain exists with tcp socket +https://github.com/openresty/lua-nginx-module/issues/1915 +--- config + resolver $TEST_NGINX_RESOLVER ipv6=off; + location = /t { + content_by_lua_block { + local function tcp(host, port) + local sock = ngx.socket.tcp() + local ok,err = sock:connect(host, port) + if not ok then + ngx.log(ngx.WARN, "failed: ", err) + sock:close() + return false + end + + sock:close() + return true + end + + local host = "www.openresty.org" + local port = 80 + + local threads = {} + for i = 1, 3 do + threads[i] = ngx.thread.spawn(tcp, host, port) + end + + local ok, res = ngx.thread.wait(threads[1],threads[2],threads[3]) + if not ok then + ngx.say("failed to wait thread") + return + end + + ngx.say("res: ", res) + + for i = 1, 3 do + ngx.thread.kill(threads[i]) + end + } + } + +--- request +GET /t +--- response_body +res: true + + + +=== TEST 44: domain exists with udp socket +https://github.com/openresty/lua-nginx-module/issues/1915 +--- config + resolver $TEST_NGINX_RESOLVER ipv6=off; + location = /t { + content_by_lua_block { + local function udp(host, port) + local sock = ngx.socket.udp() + local ok,err = sock:setpeername(host, port) + if not ok then + ngx.log(ngx.WARN, "failed: ", err) + sock:close() + return false + end + + sock:close() + return true + end + + local host = "www.notexistdomain.com" + local port = 80 + + local threads = {} + for i = 1, 3 do + threads[i] = ngx.thread.spawn(udp, host, port) + end + + local ok, res = ngx.thread.wait(threads[1],threads[2],threads[3]) + if not ok then + ngx.say("failed to wait thread") + return + end + + ngx.say("res: ", res) + + for i = 1, 3 do + ngx.thread.kill(threads[i]) + end + } + } + +--- request +GET /t +--- response_body +res: false +--- error_log +www.notexistdomain.com could not be resolved + + + +=== TEST 45: udp: nginx crash when resolve an not exist domain in ngx.thread.spawn +https://github.com/openresty/lua-nginx-module/issues/1915 +--- config + resolver $TEST_NGINX_RESOLVER ipv6=off; + location = /t { + content_by_lua_block { + local function udp(host, port) + local sock = ngx.socket.udp() + local ok,err = sock:setpeername(host, port) + if not ok then + ngx.log(ngx.WARN, "failed: ", err) + sock:close() + return false + end + + sock:close() + return true + end + + local host = "www.openresty.org" + local port = 80 + + local threads = {} + for i = 1, 3 do + threads[i] = ngx.thread.spawn(udp, host, port) + end + + local ok, res = ngx.thread.wait(threads[1],threads[2],threads[3]) + if not ok then + ngx.say("failed to wait thread") + return + end + + ngx.say("res: ", res) + + for i = 1, 3 do + ngx.thread.kill(threads[i]) + end + } + } + +--- request +GET /t +--- response_body +res: true diff --git a/t/127-uthread-kill.t b/t/127-uthread-kill.t index 5542153cf2..32509c6f24 100644 --- a/t/127-uthread-kill.t +++ b/t/127-uthread-kill.t @@ -8,7 +8,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 5 + 2); +plan tests => repeat_each() * (blocks() * 5 + 1); $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= '11211'; @@ -184,7 +184,6 @@ killed [error] --- error_log lua tcp socket abort resolver -resolve name done: -2 From 7c60431a0ab64958bf01ca41425fdd3154ab9271 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 15 Oct 2021 17:22:13 +0800 Subject: [PATCH 543/848] tests: travis failed due to the cetificat of Let's Encrypt expired. (#1939) --- .travis.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index b4082f30a2..964ae2175b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,4 +1,4 @@ -dist: xenial +dist: bionic branches: only: @@ -59,6 +59,8 @@ services: - mysql before_install: + - sudo apt update + - sudo apt install --only-upgrade ca-certificates - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - sudo cpanm --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) @@ -136,4 +138,4 @@ script: - dig +short myip.opendns.com @resolver1.opendns.com || exit 0 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 - dig +short @$TEST_NGINX_RESOLVER agentzh.org || exit 0 - - prove -Itest-nginx/lib -r t + - prove -I. -Itest-nginx/lib -r t From 07e11f31678cd90acd50d5b21075773a66d33f3e Mon Sep 17 00:00:00 2001 From: nandsky Date: Wed, 13 Oct 2021 10:26:18 +0800 Subject: [PATCH 544/848] SSL: fixed potential leak on memory allocation errors. similar to https://hg.nginx.org/nginx/rev/8981dbb12254 If ngx_pool_cleanup_add() fails, we have to clean just created SSL context manually, thus appropriate call added. --- src/ngx_http_lua_module.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 5a392ff648..fbeba12a7f 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1485,6 +1485,7 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) cln = ngx_pool_cleanup_add(cf->pool, 0); if (cln == NULL) { + ngx_ssl_cleanup_ctx(llcf->ssl); return NGX_ERROR; } From c4a3e34fca455d5f9391cf4dd5e6f8c839e7910b Mon Sep 17 00:00:00 2001 From: balus Date: Sat, 16 Oct 2021 08:26:32 +0800 Subject: [PATCH 545/848] bugfix: we should use luaL_typename() with lua stack index. --- src/ngx_http_lua_cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index 30f29fc4a7..0b1b8ecda8 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -120,7 +120,7 @@ ngx_http_lua_cache_load_code(ngx_log_t *log, lua_State *L, dd("Value associated with given key in code cache table is not code " "chunk: stack top=%d, top value type=%s\n", - lua_gettop(L), lua_typename(L, -1)); + lua_gettop(L), luaL_typename(L, -1)); /* remove cache table and value from stack */ lua_pop(L, 2); /* sp-=2 */ From 421a8da81bb5e6f8eb3327f61a3ab906338afde0 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Sun, 17 Oct 2021 15:27:18 +0800 Subject: [PATCH 546/848] chore: fix possible null pointer dereference found by Coverity. fix #1941. ** CID 339181: Null pointer dereferences (REVERSE_INULL) /src/ngx_http_lua_timer.c: 326 in ngx_http_lua_ngx_timer_helper() ________________________________________________________________________________________________________ *** CID 339181: Null pointer dereferences (REVERSE_INULL) /src/ngx_http_lua_timer.c: 326 in ngx_http_lua_ngx_timer_helper() 320 321 } else { 322 tctx->client_addr_text.len = 0; 323 tctx->client_addr_text.data = NULL; 324 } 325 >>> CID 339181: Null pointer dereferences (REVERSE_INULL) >>> Null-checking "ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check. 326 if (ctx && ctx->vm_state) { 327 tctx->vm_state = ctx->vm_state; 328 tctx->vm_state->count++; 329 330 } else { 331 tctx->vm_state = NULL; --- src/ngx_http_lua_timer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index fc6a3ae9a3..84a9e3b0dc 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -173,6 +173,7 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) } ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + ngx_http_lua_assert(ctx != NULL); /* * Since nginx has been confirmed that all timers have been cleaned up when @@ -323,7 +324,7 @@ ngx_http_lua_ngx_timer_helper(lua_State *L, int every) tctx->client_addr_text.data = NULL; } - if (ctx && ctx->vm_state) { + if (ctx->vm_state) { tctx->vm_state = ctx->vm_state; tctx->vm_state->count++; From c4d5ae71e1256ecd08d7f96995461ddfc7d0e405 Mon Sep 17 00:00:00 2001 From: kingluo Date: Tue, 19 Oct 2021 00:03:25 +0800 Subject: [PATCH 547/848] feature: allowed some ngx APIs to be used in worker thread. --- README.markdown | 30 +- doc/HttpLuaModule.wiki | 30 +- src/ngx_http_lua_worker_thread.c | 34 +- t/166-worker-thread.t | 645 +++++++++++++++++++++++++++++-- 4 files changed, 698 insertions(+), 41 deletions(-) diff --git a/README.markdown b/README.markdown index 28717882b7..df4547c4e8 100644 --- a/README.markdown +++ b/README.markdown @@ -9124,7 +9124,35 @@ ngx.run_worker_thread Wrap the [nginx worker thread](http://nginx.org/en/docs/dev/development_guide.html#threads) to execute lua function. The caller coroutine would yield until the function returns. -Note that no ngx_lua API can be used in the `function_name` function of the `module` module since it is invoked in a separate thread. +Only the following ngx_lua APIs could be used in `function_name` function of the `module` module: + +* `ngx.encode_base64` +* `ngx.decode_base64` + +* `ngx.hmac_sha1` +* `ngx.encode_args` +* `ngx.decode_args` +* `ngx.quote_sql_str` + +* `ngx.re.match` +* `ngx.re.find` +* `ngx.re.gmatch` +* `ngx.re.sub` +* `ngx.re.gsub` + +* `ngx.crc32_short` +* `ngx.crc32_long` +* `ngx.hmac_sha1` +* `ngx.md5_bin` +* `ngx.md5` + +* `ngx.config.subsystem` +* `ngx.config.debug` +* `ngx.config.prefix` +* `ngx.config.nginx_version` +* `ngx.config.nginx_configure` +* `ngx.config.ngx_lua_version` + The first argument `threadpool` specifies the Nginx thread pool name defined by [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool). diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 2a647e879e..cc01147f63 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7820,7 +7820,35 @@ This API was first enabled in the v0.6.0 release. Wrap the [http://nginx.org/en/docs/dev/development_guide.html#threads nginx worker thread] to execute lua function. The caller coroutine would yield until the function returns. -Note that no ngx_lua API can be used in the `function_name` function of the `module` module since it is invoked in a separate thread. +Only the following ngx_lua APIs could be used in `function_name` function of the `module` module: + +* `ngx.encode_base64` +* `ngx.decode_base64` + +* `ngx.hmac_sha1` +* `ngx.encode_args` +* `ngx.decode_args` +* `ngx.quote_sql_str` + +* `ngx.re.match` +* `ngx.re.find` +* `ngx.re.gmatch` +* `ngx.re.sub` +* `ngx.re.gsub` + +* `ngx.crc32_short` +* `ngx.crc32_long` +* `ngx.hmac_sha1` +* `ngx.md5_bin` +* `ngx.md5` + +* `ngx.config.subsystem` +* `ngx.config.debug` +* `ngx.config.prefix` +* `ngx.config.nginx_version` +* `ngx.config.nginx_configure` +* `ngx.config.ngx_lua_version` + The first argument `threadpool` specifies the Nginx thread pool name defined by [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool). diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 080de53776..4da6c9f5da 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -14,6 +14,8 @@ #include "ngx_http_lua_worker_thread.h" #include "ngx_http_lua_util.h" +#include "ngx_http_lua_string.h" +#include "ngx_http_lua_config.h" #if (NGX_THREADS) @@ -127,8 +129,36 @@ ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) /* pop path, cpath and "package" table from L */ lua_pop(L, 3); - /* pop the "package" table */ - lua_pop(vm, 1); + /* inject API from C */ + lua_newtable(L); /* ngx.* */ + ngx_http_lua_inject_string_api(vm); + ngx_http_lua_inject_config_api(vm); + lua_setglobal(vm, "ngx"); + + /* inject API via ffi */ + lua_getglobal(vm, "require"); + lua_pushstring(vm, "resty.core.regex"); + if (lua_pcall(vm, 1, 0, 0) != 0) { + lua_close(vm); + ngx_free(ctx); + return NULL; + } + + lua_getglobal(vm, "require"); + lua_pushstring(vm, "resty.core.hash"); + if (lua_pcall(vm, 1, 0, 0) != 0) { + lua_close(vm); + ngx_free(ctx); + return NULL; + } + + lua_getglobal(vm, "require"); + lua_pushstring(vm, "resty.core.base64"); + if (lua_pcall(vm, 1, 0, 0) != 0) { + lua_close(vm); + ngx_free(ctx); + return NULL; + } } else { ctx = ctxpool->next; diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 4037615b46..7c52876720 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -250,35 +250,7 @@ true : hello , 200 -=== TEST 9: access ngx.* api ---- main_config - thread_pool testpool threads=100; ---- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" ---- config -location /hello { - default_type 'text/plain'; - - content_by_lua_block { - local ok, err = ngx.run_worker_thread("testpool", "hello", "hello") - ngx.say(ok, " : ", err) - } -} ---- user_files ->>> hello.lua -local function hello() - ngx.sleep(1) - return "ok" -end -return {hello=hello} ---- request -GET /hello ---- response_body_like -false : .*attempt to index global 'ngx' \(a nil value\) - - - -=== TEST 10: module not found +=== TEST 9: module not found --- main_config thread_pool testpool threads=100; --- http_config eval @@ -299,7 +271,7 @@ false : module 'hello' not found.* -=== TEST 11: the number of Lua VM exceeds the pool size +=== TEST 10 the number of Lua VM exceeds the pool size --- main_config thread_pool testpool threads=100; --- http_config eval: $::HttpConfig @@ -376,7 +348,7 @@ GET /t -=== TEST 12: kill uthread before worker thread callback +=== TEST 11 kill uthread before worker thread callback --- main_config thread_pool testpool threads=100; --- http_config eval: $::HttpConfig @@ -452,7 +424,7 @@ GET /t -=== TEST 13: exit before worker thread callback +=== TEST 12: exit before worker thread callback --- main_config thread_pool testpool threads=100; --- http_config eval @@ -484,7 +456,7 @@ GET /hello -=== TEST 14: unsupported argument type in nested table +=== TEST 13: unsupported argument type in nested table --- main_config thread_pool testpool threads=100; --- http_config eval @@ -513,7 +485,7 @@ false : unsupported argument type -=== TEST 15: return table with unsupported type +=== TEST 14: return table with unsupported type --- main_config thread_pool testpool threads=100; --- http_config eval @@ -546,7 +518,7 @@ false , unsupported return value -=== TEST 16: the type of module name is not string +=== TEST 15: the type of module name is not string --- main_config thread_pool testpool threads=100; --- http_config eval @@ -574,7 +546,7 @@ false : module name should be a string -=== TEST 17: the type of function name is not string +=== TEST 16: the type of function name is not string --- main_config thread_pool testpool threads=100; --- http_config eval @@ -602,7 +574,7 @@ false : function name should be a string -=== TEST 18: the type of thread pool name is not string +=== TEST 17: the type of thread pool name is not string --- main_config thread_pool testpool threads=100; --- http_config eval @@ -627,3 +599,602 @@ return {hello=hello} GET /hello --- response_body false : threadpool should be a string + + + +=== TEST 18: ngx.encode_base64 +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.encode_base64("hello") +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : aGVsbG8= + + + +=== TEST 19: ngx.config.subsystem +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.config.subsystem +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : http + + + +=== TEST 20: ngx.hmac_sha1 +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + local key = "thisisverysecretstuff" + local src = "some string we want to sign" + local digest = ngx.hmac_sha1(key, src) + return ngx.encode_base64(digest) +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : R/pvxzHC4NLtj7S+kXFg/NePTmk= + + + +=== TEST 21: ngx.encode_args +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.encode_args({foo = 3, ["b r"] = "hello world"}) +end +return {hello=hello} +--- request +GET /hello +--- response_body eval +qr/foo=3&b%20r=hello%20world|b%20r=hello%20world&foo=3/ + + + +=== TEST 22: ngx.decode_args +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ret = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", ret.a, " : ", ret.b) + } +} +--- user_files +>>> hello.lua +local function hello() + local args = "a=bar&b=foo" + args = ngx.decode_args(args) + return args +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : bar : foo + + + +=== TEST 23: ngx.quote_sql_str +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config + location /hello { + content_by_lua ' + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello", "a\\026b\\026") + ngx.say(ok, " : ", hello_or_err) + '; + } +--- user_files +>>> hello.lua +local function hello(str) + return ngx.quote_sql_str(str) +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 'a\Zb\Z' + + + +=== TEST 24: ngx.re.match +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, a, b = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", a, " : ", b) + } +} +--- user_files +>>> hello.lua +local function hello() + local m, err = ngx.re.match("hello, 1234", "([0-9])[0-9]+") + return m[0], m[1] +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 1234 : 1 + + + +=== TEST 25: ngx.re.find +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, a = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", a) + } +} +--- user_files +>>> hello.lua +local function hello() + local str = "hello, 1234" + local from, to = ngx.re.find(str, "([0-9])([0-9]+)", "jo", nil, 2) + if from then + return string.sub(str, from, to) + end +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 234 + + + +=== TEST 26: ngx.re.gmatch +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, ret = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok) + ngx.say(ret[1]) + ngx.say(ret[2]) + } +} +--- user_files +>>> hello.lua +local function hello() + local ret = {} + for m in ngx.re.gmatch("hello, world", "[a-z]+", "j") do + if m then + table.insert(ret, m[0]) + end + end + return ret +end +return {hello=hello} +--- request +GET /hello +--- response_body +true +hello +world + + + +=== TEST 27: ngx.re.sub +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, a, b = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok) + ngx.say(a) + ngx.say(b) + } +} +--- user_files +>>> hello.lua +local function hello() + local newstr, n = ngx.re.sub("hello, 1234", "[0-9]", "$$") + return newstr, n +end +return {hello=hello} +--- request +GET /hello +--- response_body +true +hello, $234 +1 + + + +=== TEST 28: ngx.re.gsub +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, a, b = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok) + ngx.say(a) + ngx.say(b) + } +} +--- user_files +>>> hello.lua +local function hello() + local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", "[$0,$1]", "i") + return newstr, n +end +return {hello=hello} +--- request +GET /hello +--- response_body +true +[hello,h], [world,w] +2 + + + +=== TEST 29: ngx.decode_base64 +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.decode_base64("aGVsbG8=") +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : hello + + + +=== TEST 30: ngx.crc32_short +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.crc32_short("hello, world") +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 4289425978 + + + +=== TEST 31: ngx.crc32_long +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.crc32_long("hello, world") +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 4289425978 + + + +=== TEST 32: ngx.md5_bin +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + local s = ngx.md5_bin(45) + s = string.gsub(s, ".", function (c) + return string.format("%02x", string.byte(c)) + end) + return s +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 6c8349cc7260ae62e3b1396831a8398f + + + +=== TEST 33: ngx.md5 +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.md5("hello") +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : 5d41402abc4b2a76b9719d911017c592 + + + +=== TEST 34: ngx.config.debug +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.config.debug +end +return {hello=hello} +--- request +GET /hello +--- response_body_like chop +^true : (?:true|false)$ + + + +=== TEST 35: ngx.config.prefix +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.config.prefix() +end +return {hello=hello} +--- request +GET /hello +--- response_body_like chop +^true : \/\S+$ + + + +=== TEST 36: ngx.config.nginx_version +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.config.nginx_version +end +return {hello=hello} +--- request +GET /hello +--- response_body_like chop +^true : \d+$ + + + +=== TEST 37: ngx.config.nginx_configure +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.config.nginx_configure() +end +return {hello=hello} +--- request +GET /hello +--- response_body_like chop +^\s*\-\-[^-]+ + + + +=== TEST 38: ngx.config.ngx_lua_version +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return ngx.config.ngx_lua_version +end +return {hello=hello} +--- request +GET /hello +--- response_body_like chop +^true : \d+$ From 3f1c6834ee018c057de3d96ca20c32fd87abe9c9 Mon Sep 17 00:00:00 2001 From: pengyanfeng Date: Tue, 19 Oct 2021 11:01:03 +0800 Subject: [PATCH 548/848] bugfix: fixed memcpy param overlap detected by asan. ==openresty==70603==ERROR: AddressSanitizer: memcpy-param-overlap: memory ranges [0x621000001500,0x621000002181) and [0x62100000187f, 0x621000002500) overlap #0 0x7f3db1899ffe (/lib64/libasan.so.5+0x99ffe) #1 0x9da926 (/usr/local/openresty-debug/nginx/sbin/nginx+0x9da926) #2 0x9dd1a1 (/usr/local/openresty-debug/nginx/sbin/nginx+0x9dd1a1) #3 0x4c89c6 (/usr/local/openresty-debug/nginx/sbin/nginx+0x4c89c6) #4 0x5d1e4e (/usr/local/openresty-debug/nginx/sbin/nginx+0x5d1e4e) #5 0x4c89c6 (/usr/local/openresty-debug/nginx/sbin/nginx+0x4c89c6) #6 0x5b8583 (/usr/local/openresty-debug/nginx/sbin/nginx+0x5b8583) #7 0x4c89c6 (/usr/local/openresty-debug/nginx/sbin/nginx+0x4c89c6) #8 0x4b4419 (/usr/local/openresty-debug/nginx/sbin/nginx+0x4b4419) #9 0x427f16 (/usr/local/openresty-debug/nginx/sbin/nginx+0x427f16) #10 0x7f3daff27554 (/lib64/libc.so.6+0x22554) #11 0x42d537 (/usr/local/openresty-debug/nginx/sbin/nginx+0x42d537) --- src/ngx_http_lua_directive.c | 2 +- t/014-bugs.t | 26 +++++++++++++++++++++++++- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 1ec641e079..bb3f5238fe 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1632,7 +1632,7 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, } if (len) { - ngx_memcpy(b->start, b->pos, len); + ngx_memmove(b->start, b->pos, len); } size = (ssize_t) (file_size - cf->conf_file->file.offset); diff --git a/t/014-bugs.t b/t/014-bugs.t index 88519c0caa..770e357225 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -8,7 +8,7 @@ log_level('debug'); repeat_each(3); -plan tests => repeat_each() * (blocks() * 2 + 32); +plan tests => repeat_each() * (blocks() * 2 + 33); our $HtmlDir = html_dir; #warn $html_dir; @@ -1215,3 +1215,27 @@ https://github.com/openresty/lua-nginx-module/issues/1915 GET /t --- response_body res: true + + + +=== TEST 46: nginx crash when parsing a word or a single configuration item that is too long +https://github.com/openresty/lua-nginx-module/issues/1938 +--- http_config + init_worker_by_lua ' + err_big_str = 'A NA' + '; +--- config + location /t { + content_by_lua ' + ngx.say("hello world") + '; + } +--- request +GET /t +--- response_body +res: true +--- no_error_log +[error] +--- must_die +--- error_log eval +qr/\[emerg\] \d+#\d+: unexpected "A" in/ From 5f47838e60077ea737970d9a9916830cb971486f Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 19 Oct 2021 21:10:11 +0800 Subject: [PATCH 549/848] doc: clearify the missing the closing long bracket may due to the lua code is too long. --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ src/ngx_http_lua_directive.c | 5 +++-- t/132-lua-blocks.t | 8 ++++---- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index df4547c4e8..5bb4637a44 100644 --- a/README.markdown +++ b/README.markdown @@ -908,6 +908,8 @@ As noted earlier, PCRE sequences presented within `*_by_lua_block {}` directives # evaluates to "1234" ``` +**NOTE** You are recommended to use `by_lua_file` when the Lua code is very long. + [Back to TOC](#table-of-contents) Mixing with SSI Not Supported diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index cc01147f63..a9470f06f6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -755,6 +755,8 @@ As noted earlier, PCRE sequences presented within *_by_lua_block {} # evaluates to "1234" +'''NOTE''' You are recommended to use `by_lua_file` when the Lua code is very long. + == Mixing with SSI Not Supported == Mixing SSI with ngx_lua in the same Nginx request is not supported at all. Just use ngx_lua exclusively. Everything you can do with SSI can be done atop ngx_lua anyway and it can be more efficient when using ngx_lua. diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index bb3f5238fe..831132f12d 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1537,7 +1537,7 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, for ( ;; ) { if (b->pos >= b->last - || (b->last - b->pos < (b->end - b->start) / 3 + || (b->last - b->pos < (b->end - b->start) / 2 && cf->conf_file->file.offset < file_size)) { @@ -1709,7 +1709,8 @@ ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, if (q == NULL) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "Lua code block missing the closing " - "long bracket \"%*s\"", + "long bracket \"%*s\", " + "the inlined Lua code may be too long", b->pos + ovec[1] - p, p); return NGX_ERROR; } diff --git a/t/132-lua-blocks.t b/t/132-lua-blocks.t index 76f4a6184d..f232cec035 100644 --- a/t/132-lua-blocks.t +++ b/t/132-lua-blocks.t @@ -312,7 +312,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]" in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ @@ -331,7 +331,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]==]" in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]==]", the inlined Lua code may be too long in .*?\bnginx.conf:41/ @@ -350,7 +350,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]" in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ @@ -369,7 +369,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]=]" in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]=]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ From aeaecd6525d94561f2824bb826b3661f9b48b0ea Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 19 Oct 2021 21:27:41 +0800 Subject: [PATCH 550/848] tests: added the missing test case which should be added in commit 5f47838e600. --- t/132-lua-blocks.t | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/t/132-lua-blocks.t b/t/132-lua-blocks.t index f232cec035..6bf8f944c5 100644 --- a/t/132-lua-blocks.t +++ b/t/132-lua-blocks.t @@ -606,3 +606,38 @@ GET /t --- response_body_like: } --- no_error_log [error] + + + +=== TEST 24: too long bracket +--- config + location = /t { + content_by_lua_block { + local foo = [[ +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + ]] + ngx.say(foo) + } + } +--- request +GET /t +--- response_body +hello, world +--- no_error_log +[error] +--- must_die +--- error_log eval +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ From a6e9e11f14a9040f35013a2efe3d753c24d078cf Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 19 Oct 2021 23:05:18 +0800 Subject: [PATCH 551/848] doc: fixed code style and test case style found by ngx-releng. --- .travis.yml | 2 +- src/ngx_http_lua_util.c | 5 ++--- src/ngx_http_lua_util.h | 5 ++--- t/166-worker-thread.t | 4 ++-- 4 files changed, 7 insertions(+), 9 deletions(-) diff --git a/.travis.yml b/.travis.yml index 964ae2175b..19ef3926cd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -100,7 +100,7 @@ before_script: script: - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH - ngx-releng > check.txt || true - - lines=`wc -l check.txt | awk '{print $1}'`; if [ $lines -gt 13 ]; then cat check.txt; exit 1; fi + - lines=`wc -l check.txt | awk '{print $1}'`; if [ $lines -gt 5 ]; then cat check.txt; exit 1; fi - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT - sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP - sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 56b5989fe2..64d713dc31 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -388,10 +388,9 @@ ngx_http_lua_new_thread(ngx_http_request_t *r, lua_State *L, int *ref) } #endif - } else { -#else - { + } else #endif + { base = lua_gettop(L); lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 8111dde762..5f1585c6f0 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -648,10 +648,9 @@ ngx_http_lua_new_cached_thread(lua_State *L, lua_State **out_co, lua_rawget(L, LUA_REGISTRYINDEX); lua_rawgeti(L, -1, co_ref); - } else { -#else - { + } else #endif + { lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( coroutines_key)); lua_rawget(L, LUA_REGISTRYINDEX); diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 7c52876720..9717e692c0 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -271,7 +271,7 @@ false : module 'hello' not found.* -=== TEST 10 the number of Lua VM exceeds the pool size +=== TEST 10: the number of Lua VM exceeds the pool size --- main_config thread_pool testpool threads=100; --- http_config eval: $::HttpConfig @@ -348,7 +348,7 @@ GET /t -=== TEST 11 kill uthread before worker thread callback +=== TEST 11: kill uthread before worker thread callback --- main_config thread_pool testpool threads=100; --- http_config eval: $::HttpConfig From ccc748eafbab96d0795dc92dca9374227e4b46f5 Mon Sep 17 00:00:00 2001 From: balus Date: Sat, 23 Oct 2021 23:38:25 +0800 Subject: [PATCH 552/848] bugfix: fix nginx crash caused by a bad format specifier. --- src/ngx_http_lua_subrequest.c | 2 +- t/020-subrequest.t | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index a798f9bdab..f6638c2991 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -734,7 +734,7 @@ ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, default: ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "unsupported HTTP method: %u", (unsigned) method); + "unsupported HTTP method: %ui", method); return NGX_ERROR; } diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 18befbd761..b3b6fb0129 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -3391,3 +3391,22 @@ method: POST, uri: /foo 0 --- no_error_log [error] + + + +=== TEST 81: bad HTTP method +--- config + location /other { } + + location /lua { + content_by_lua_block { + local res = ngx.location.capture("/other", + { method = 10240 }); + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +unsupported HTTP method: 10240 From 59d39ca2f0963695052c2593f957053f1a1779a2 Mon Sep 17 00:00:00 2001 From: Josh Soref <2119212+jsoref@users.noreply.github.com> Date: Mon, 25 Oct 2021 03:07:01 -0400 Subject: [PATCH 553/848] doc: fixed spelling errors in the docs and code. (#1947) --- README.markdown | 12 ++++++------ doc/HttpLuaModule.wiki | 12 ++++++------ src/ngx_http_lua_clfactory.c | 4 ++-- src/ngx_http_lua_common.h | 2 +- src/ngx_http_lua_headers.c | 4 ++-- src/ngx_http_lua_semaphore.c | 2 +- src/ngx_http_lua_socket_tcp.c | 4 ++-- src/ngx_http_lua_socket_udp.c | 2 +- src/ngx_http_lua_ssl.h | 2 +- src/ngx_http_lua_timer.c | 2 +- src/ngx_http_lua_uthread.c | 2 +- src/ngx_http_lua_util.c | 2 +- src/ngx_http_lua_util.h | 2 +- t/006-escape.t | 4 ++-- t/014-bugs.t | 8 ++++---- t/015-status.t | 2 +- t/016-resp-header.t | 4 ++-- t/023-rewrite/on-abort.t | 6 +++--- t/023-rewrite/socket-keepalive.t | 2 +- t/024-access/on-abort.t | 6 +++--- t/028-req-header.t | 12 ++++++------ t/044-req-body.t | 4 ++-- t/068-socket-keepalive.t | 2 +- t/087-udp-socket.t | 2 +- t/101-on-abort.t | 8 ++++---- t/114-config.t | 2 +- t/138-balancer.t | 2 +- t/lib/CRC32.lua | 2 +- t/lib/Memcached.lua | 2 +- util/build.sh | 2 +- 30 files changed, 61 insertions(+), 61 deletions(-) diff --git a/README.markdown b/README.markdown index 5bb4637a44..d8f28769d7 100644 --- a/README.markdown +++ b/README.markdown @@ -379,7 +379,7 @@ Build the source with this module: # current nginx build. # You can get usually those options using command nginx -V - # you can change the parallism number 2 below to fit the number of spare CPU cores in your + # you can change the parallelism number 2 below to fit the number of spare CPU cores in your # machine. make -j2 make install @@ -2654,7 +2654,7 @@ The [ngx.ssl.clienthello](https://github.com/openresty/lua-resty-core/blob/maste provided by the [lua-resty-core](https://github.com/openresty/lua-resty-core/#readme) library are particularly useful in this context. -Note that this handler runs in extremelly early stage of SSL handshake, before the SSL client hello extensions are parsed. +Note that this handler runs in extremely early stage of SSL handshake, before the SSL client hello extensions are parsed. So you can not use some Lua API like `ssl.server_name()` which is dependent on the later stage's processing. Also note that only the directive in default server is valid for several virtual servers with the same IP address and port. @@ -3363,7 +3363,7 @@ lua_transform_underscores_in_response_headers **context:** *http, server, location, location-if* -Controls whether to transform underscores (`_`) in the response header names specified in the [ngx.header.HEADER](#ngxheaderheader) API to hypens (`-`). +Controls whether to transform underscores (`_`) in the response header names specified in the [ngx.header.HEADER](#ngxheaderheader) API to hyphens (`-`). This directive was first introduced in the `v0.5.0rc32` release. @@ -3472,7 +3472,7 @@ Specifies the size limit of the Lua VM pool (default 100) that will be used in t Also, it is not allowed to create Lua VMs that exceeds the pool size limit. -The Lua VM in the VM pool is used to execute Lua code in seperate thread. +The Lua VM in the VM pool is used to execute Lua code in separate thread. The pool is global at Nginx worker level. And it is used to reuse Lua VMs between requests. @@ -5414,7 +5414,7 @@ ngx.req.init_body **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua** -Creates a new blank request body for the current request and inializes the buffer for later request body data writing via the [ngx.req.append_body](#ngxreqappend_body) and [ngx.req.finish_body](#ngxreqfinish_body) APIs. +Creates a new blank request body for the current request and initializes the buffer for later request body data writing via the [ngx.req.append_body](#ngxreqappend_body) and [ngx.req.finish_body](#ngxreqfinish_body) APIs. If the `buffer_size` argument is specified, then its value will be used for the size of the memory buffer for body writing with [ngx.req.append_body](#ngxreqappend_body). If the argument is omitted, then the value specified by the standard [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive will be used instead. @@ -9041,7 +9041,7 @@ coroutine.resume **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** -Resumes the executation of a user Lua coroutine object previously yielded or just created. +Resumes the execution of a user Lua coroutine object previously yielded or just created. Similar to the standard Lua [coroutine.resume](https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.resume) API, but works in the context of the Lua coroutines created by ngx_lua. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a9470f06f6..98f3c5879b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -309,7 +309,7 @@ Build the source with this module: # current nginx build. # You can get usually those options using command nginx -V - # you can change the parallism number 2 below to fit the number of spare CPU cores in your + # you can change the parallelism number 2 below to fit the number of spare CPU cores in your # machine. make -j2 make install @@ -2237,7 +2237,7 @@ The [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/clienth provided by the [https://github.com/openresty/lua-resty-core/#readme lua-resty-core] library are particularly useful in this context. -Note that this handler runs in extremelly early stage of SSL handshake, before the SSL client hello extensions are parsed. +Note that this handler runs in extremely early stage of SSL handshake, before the SSL client hello extensions are parsed. So you can not use some Lua API like ssl.server_name() which is dependent on the later stage's processing. Also note that only the directive in default server is valid for several virtual servers with the same IP address and port. @@ -2864,7 +2864,7 @@ This directive was first introduced in the v0.9.20 release. '''context:''' ''http, server, location, location-if'' -Controls whether to transform underscores (_) in the response header names specified in the [[#ngx.header.HEADER|ngx.header.HEADER]] API to hypens (-). +Controls whether to transform underscores (_) in the response header names specified in the [[#ngx.header.HEADER|ngx.header.HEADER]] API to hyphens (-). This directive was first introduced in the v0.5.0rc32 release. @@ -2957,7 +2957,7 @@ Specifies the size limit of the Lua VM pool (default 100) that will be used in t Also, it is not allowed to create Lua VMs that exceeds the pool size limit. -The Lua VM in the VM pool is used to execute Lua code in seperate thread. +The Lua VM in the VM pool is used to execute Lua code in separate thread. The pool is global at Nginx worker level. And it is used to reuse Lua VMs between requests. @@ -4549,7 +4549,7 @@ See also [[#ngx.req.set_body_data|ngx.req.set_body_data]]. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*'' -Creates a new blank request body for the current request and inializes the buffer for later request body data writing via the [[#ngx.req.append_body|ngx.req.append_body]] and [[#ngx.req.finish_body|ngx.req.finish_body]] APIs. +Creates a new blank request body for the current request and initializes the buffer for later request body data writing via the [[#ngx.req.append_body|ngx.req.append_body]] and [[#ngx.req.finish_body|ngx.req.finish_body]] APIs. If the buffer_size argument is specified, then its value will be used for the size of the memory buffer for body writing with [[#ngx.req.append_body|ngx.req.append_body]]. If the argument is omitted, then the value specified by the standard [[HttpCoreModule#client_body_buffer_size|client_body_buffer_size]] directive will be used instead. @@ -7752,7 +7752,7 @@ This API was first introduced in the v0.6.0 release. '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, init_by_lua*, ngx.timer.*, header_filter_by_lua*, body_filter_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' -Resumes the executation of a user Lua coroutine object previously yielded or just created. +Resumes the execution of a user Lua coroutine object previously yielded or just created. Similar to the standard Lua [https://www.lua.org/manual/5.1/manual.html#pdf-coroutine.resume coroutine.resume] API, but works in the context of the Lua coroutines created by ngx_lua. diff --git a/src/ngx_http_lua_clfactory.c b/src/ngx_http_lua_clfactory.c index e8a8a0e1f7..c13fd14dc1 100644 --- a/src/ngx_http_lua_clfactory.c +++ b/src/ngx_http_lua_clfactory.c @@ -126,7 +126,7 @@ * | at [p[i]] | * --------------------- * | Vector | Debug lineinfo vector - * | [lineinfo] | Empty vector here if dubug info is stripped + * | [lineinfo] | Empty vector here if debug info is stripped * --------------------- * | Int | Number of local variable in this function * | [sizelocvars] | 0 if debug info is stripped @@ -136,7 +136,7 @@ * | .varname] | | * --------------------- | * | Int | instruction counter | - * | [locvars[i]] | where lcoal var i start to be |-> repeat for i in + * | [locvars[i]] | where local var i start to be |-> repeat for i in * | .startpc] | referenced | [0..sizelocvars] * --------------------- | * | Int | instruction counter, where local | diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index f75fa1a1ad..ed88f0a2f5 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -273,7 +273,7 @@ struct ngx_http_lua_main_conf_s { ngx_http_lua_sema_mm_t *sema_mm; ngx_uint_t malloc_trim_cycle; /* a cycle is defined as the number - of reqeusts */ + of requests */ ngx_uint_t malloc_trim_req_count; #if (nginx_version >= 1011011) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index d8d5edc8b2..a376a0eac9 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -677,7 +677,7 @@ ngx_http_lua_ngx_req_header_set_helper(lua_State *L) p = (u_char *) luaL_checklstring(L, -1, &len); /* - * we also copy the trailling '\0' char here because nginx + * we also copy the trailing '\0' char here because nginx * header values must be null-terminated * */ @@ -705,7 +705,7 @@ ngx_http_lua_ngx_req_header_set_helper(lua_State *L) } else { /* - * we also copy the trailling '\0' char here because nginx + * we also copy the trailing '\0' char here because nginx * header values must be null-terminated * */ diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 8ca6efb345..6f45eeffcc 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -343,7 +343,7 @@ ngx_http_lua_ffi_sema_post(ngx_http_lua_sema_t *sem, int n) sem->resource_count += n; if (!ngx_queue_empty(&sem->wait_queue)) { - /* we need the extra paranthese around the first argument of + /* we need the extra parentheses around the first argument of * ngx_post_event() just to work around macro issues in nginx * cores older than nginx 1.7.12 (exclusive). */ diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index b7c3bdd45d..ace18a0f89 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -3112,7 +3112,7 @@ ngx_http_lua_socket_tcp_settimeout(lua_State *L) n = lua_gettop(L); if (n != 2) { - return luaL_error(L, "ngx.socket settimout: expecting 2 arguments " + return luaL_error(L, "ngx.socket settimeout: expecting 2 arguments " "(including the object) but seen %d", lua_gettop(L)); } @@ -3159,7 +3159,7 @@ ngx_http_lua_socket_tcp_settimeouts(lua_State *L) n = lua_gettop(L); if (n != 4) { - return luaL_error(L, "ngx.socket settimout: expecting 4 arguments " + return luaL_error(L, "ngx.socket settimeout: expecting 4 arguments " "(including the object) but seen %d", lua_gettop(L)); } diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index a1714fa444..4f970e6cca 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -1044,7 +1044,7 @@ ngx_http_lua_socket_udp_settimeout(lua_State *L) n = lua_gettop(L); if (n != 2) { - return luaL_error(L, "ngx.socket settimout: expecting at least 2 " + return luaL_error(L, "ngx.socket settimeout: expecting at least 2 " "arguments (including the object) but seen %d", lua_gettop(L)); } diff --git a/src/ngx_http_lua_ssl.h b/src/ngx_http_lua_ssl.h index 46b563701f..3d577c61fb 100644 --- a/src/ngx_http_lua_ssl.h +++ b/src/ngx_http_lua_ssl.h @@ -19,7 +19,7 @@ typedef struct { ngx_http_request_t *request; /* fake request */ ngx_pool_cleanup_pt *cleanup; - ngx_ssl_session_t *session; /* retrurn value for openssl's + ngx_ssl_session_t *session; /* return value for openssl's * session_get_cb */ ngx_str_t session_id; diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 84a9e3b0dc..e82e3406ce 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -875,7 +875,7 @@ ngx_http_lua_abort_pending_timers(ngx_event_t *ev) next = cur->parent; } else { - /* not reacheable */ + /* not reachable */ next = NULL; } diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index 5ed534c7f1..d30ffb4046 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -268,7 +268,7 @@ ngx_http_lua_uthread_kill(lua_State *L) return 1; } - /* not reacheable */ + /* not reachable */ } /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 64d713dc31..0d29c7bcb6 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4405,7 +4405,7 @@ ngx_http_lua_copy_escaped_header(ngx_http_request_t *r, escape = ngx_http_lua_escape_uri(NULL, data, len, type); if (escape > 0) { /* - * we allocate space for the trailling '\0' char here because nginx + * we allocate space for the trailing '\0' char here because nginx * header values must be null-terminated */ dst->data = ngx_palloc(r->pool, len + 2 * escape + 1); diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 5f1585c6f0..348a8666a4 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -165,7 +165,7 @@ void ngx_http_lua_reset_ctx(ngx_http_request_t *r, lua_State *L, void ngx_http_lua_generic_phase_post_read(ngx_http_request_t *r); -void ngx_http_lua_request_cleanup(ngx_http_lua_ctx_t *ctx, int foricible); +void ngx_http_lua_request_cleanup(ngx_http_lua_ctx_t *ctx, int forcible); void ngx_http_lua_request_cleanup_handler(void *data); diff --git a/t/006-escape.t b/t/006-escape.t index 266b23516c..28714ab893 100644 --- a/t/006-escape.t +++ b/t/006-escape.t @@ -315,8 +315,8 @@ GET /lua --- config location /lua { content_by_lua_block { - ngx.say(ngx.unescape_uri("%a")) -- first charactor is good - ngx.say(ngx.unescape_uri("%u")) -- first charactor is bad + ngx.say(ngx.unescape_uri("%a")) -- first character is good + ngx.say(ngx.unescape_uri("%u")) -- first character is bad ngx.say(ngx.unescape_uri("%")) ngx.say(ngx.unescape_uri("good%20job%")) } diff --git a/t/014-bugs.t b/t/014-bugs.t index 770e357225..8c93920bc7 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -1041,7 +1041,7 @@ https://github.com/openresty/lua-nginx-module/issues/1915 return true end - local host = "www.notexistdomain.com" + local host = "nonexistent.openresty.org" local port = 80 local threads = {} @@ -1068,7 +1068,7 @@ GET /t --- response_body res: false --- error_log -www.notexistdomain.com could not be resolved +nonexistent.openresty.org could not be resolved @@ -1139,7 +1139,7 @@ https://github.com/openresty/lua-nginx-module/issues/1915 return true end - local host = "www.notexistdomain.com" + local host = "nonexistent.openresty.org" local port = 80 local threads = {} @@ -1166,7 +1166,7 @@ GET /t --- response_body res: false --- error_log -www.notexistdomain.com could not be resolved +nonexistent.openresty.org could not be resolved diff --git a/t/015-status.t b/t/015-status.t index 666dae727c..a69c59377b 100644 --- a/t/015-status.t +++ b/t/015-status.t @@ -272,7 +272,7 @@ ngx.status = 502 -=== TEST 16: ngx.status assignmnt should clear r->err_status +=== TEST 16: ngx.status assignment should clear r->err_status --- config location = /t { return 502; diff --git a/t/016-resp-header.t b/t/016-resp-header.t index c7b5addee4..b558415b24 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -931,7 +931,7 @@ Hello local results = {} results.something = "hello" results.content_type = "anything" - results.somehing_else = "hi" + results.something_else = "hi" local arr = {} for k in pairs(results) do table.insert(arr, k) end @@ -948,8 +948,8 @@ Content-Type: text/my-plain --- response_body content_type: anything -somehing_else: hi something: hello +something_else: hi --- no_error_log [error] diff --git a/t/023-rewrite/on-abort.t b/t/023-rewrite/on-abort.t index 336b7ce4b5..59c5df1167 100644 --- a/t/023-rewrite/on-abort.t +++ b/t/023-rewrite/on-abort.t @@ -419,7 +419,7 @@ main handler done -=== TEST 9: regsiter on_abort callback but no client abortion +=== TEST 9: register on_abort callback but no client abortion --- config location /t { lua_check_client_abort on; @@ -560,7 +560,7 @@ on abort called -=== TEST 12: regsiter on_abort callback but no client abortion (uthread) +=== TEST 12: register on_abort callback but no client abortion (uthread) --- config location /t { lua_check_client_abort on; @@ -607,7 +607,7 @@ main handler done -=== TEST 13: regsiter on_abort callback multiple times +=== TEST 13: register on_abort callback multiple times --- config location /t { lua_check_client_abort on; diff --git a/t/023-rewrite/socket-keepalive.t b/t/023-rewrite/socket-keepalive.t index 8b532ead6b..c943e44616 100644 --- a/t/023-rewrite/socket-keepalive.t +++ b/t/023-rewrite/socket-keepalive.t @@ -903,7 +903,7 @@ lua tcp socket get keepalive peer: using connection -=== TEST 12: github issue #108: ngx.locaiton.capture + redis.set_keepalive +=== TEST 12: github issue #108: ngx.location.capture + redis.set_keepalive --- http_config eval qq{ lua_package_path "$::HtmlDir/?.lua;;"; diff --git a/t/024-access/on-abort.t b/t/024-access/on-abort.t index 5bb948bb68..b9532aea3f 100644 --- a/t/024-access/on-abort.t +++ b/t/024-access/on-abort.t @@ -417,7 +417,7 @@ main handler done -=== TEST 9: regsiter on_abort callback but no client abortion +=== TEST 9: register on_abort callback but no client abortion --- config location /t { lua_check_client_abort on; @@ -558,7 +558,7 @@ on abort called -=== TEST 12: regsiter on_abort callback but no client abortion (uthread) +=== TEST 12: register on_abort callback but no client abortion (uthread) --- config location /t { lua_check_client_abort on; @@ -603,7 +603,7 @@ main handler done -=== TEST 13: regsiter on_abort callback multiple times +=== TEST 13: register on_abort callback multiple times --- config location /t { lua_check_client_abort on; diff --git a/t/028-req-header.t b/t/028-req-header.t index c2c58eb52e..8c8ff83fd8 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -539,7 +539,7 @@ lua exceeding request header limit -=== TEST 21: execeeding custom max 102 header limit +=== TEST 21: exceeding custom max 102 header limit --- config location /lua { content_by_lua ' @@ -591,7 +591,7 @@ lua exceeding request header limit 103 > 102 -=== TEST 22: NOT execeeding custom max 102 header limit +=== TEST 22: NOT exceeding custom max 102 header limit --- config location /lua { content_by_lua ' @@ -1863,7 +1863,7 @@ ok -=== TEST 57: execeeding custom 3 header limit +=== TEST 57: exceeding custom 3 header limit --- config location /lua { content_by_lua ' @@ -1901,7 +1901,7 @@ lua exceeding request header limit 4 > 3 -=== TEST 58: NOT execeeding custom 3 header limit +=== TEST 58: NOT exceeding custom 3 header limit --- config location /lua { content_by_lua ' @@ -1937,7 +1937,7 @@ lua exceeding request header limit -=== TEST 59: execeeding custom 3 header limit (raw) +=== TEST 59: exceeding custom 3 header limit (raw) --- config location /lua { content_by_lua ' @@ -1975,7 +1975,7 @@ lua exceeding request header limit 4 > 3 -=== TEST 60: NOT execeeding custom 3 header limit (raw) +=== TEST 60: NOT exceeding custom 3 header limit (raw) --- config location /lua { content_by_lua ' diff --git a/t/044-req-body.t b/t/044-req-body.t index e645eeefe5..2ef7e1d29e 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -440,7 +440,7 @@ b.txt exists: yes -=== TEST 17: read buffered body to memoary and reset it to a new file (auto-clean) +=== TEST 17: read buffered body to memory and reset it to a new file (auto-clean) --- config client_body_in_file_only off; @@ -476,7 +476,7 @@ qr/500 Internal Server Error/] -=== TEST 18: read buffered body to memoary and reset it to a new file (no auto-clean) +=== TEST 18: read buffered body to memory and reset it to a new file (no auto-clean) --- config client_body_in_file_only off; diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index d67da3bd1e..cf8bc25acb 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -887,7 +887,7 @@ received response of 119 bytes -=== TEST 13: github issue #108: ngx.locaiton.capture + redis.set_keepalive +=== TEST 13: github issue #108: ngx.location.capture + redis.set_keepalive --- http_config eval qq{ lua_package_path "$::HtmlDir/?.lua;;"; diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index fc467d769a..80c5e36517 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -1171,7 +1171,7 @@ send: fd:\d+ 15 of 15/ -=== TEST 22: send tables of string framents (with numbers too) +=== TEST 22: send tables of string fragments (with numbers too) the maximum number of significant digits is 14 in lua --- config server_tokens off; diff --git a/t/101-on-abort.t b/t/101-on-abort.t index 182d12c546..ee1e41e3a8 100644 --- a/t/101-on-abort.t +++ b/t/101-on-abort.t @@ -408,7 +408,7 @@ main handler done -=== TEST 9: regsiter on_abort callback but no client abortion +=== TEST 9: register on_abort callback but no client abortion --- config location /t { lua_check_client_abort on; @@ -546,7 +546,7 @@ on abort called -=== TEST 12: regsiter on_abort callback but no client abortion (uthread) +=== TEST 12: register on_abort callback but no client abortion (uthread) --- config location /t { lua_check_client_abort on; @@ -591,7 +591,7 @@ main handler done -=== TEST 13: regsiter on_abort callback multiple times +=== TEST 13: register on_abort callback multiple times --- config location /t { lua_check_client_abort on; @@ -794,7 +794,7 @@ on abort called -=== TEST 18: regsiter on_abort callback but no client abortion (2 uthreads and 1 pending) +=== TEST 18: register on_abort callback but no client abortion (2 uthreads and 1 pending) --- config location /t { lua_check_client_abort on; diff --git a/t/114-config.t b/t/114-config.t index 9d6680dda8..46d98de2fb 100644 --- a/t/114-config.t +++ b/t/114-config.t @@ -33,7 +33,7 @@ GET /t -=== TEST 2: ngx.config.subystem +=== TEST 2: ngx.config.subsystem --- config location /t { content_by_lua ' diff --git a/t/138-balancer.t b/t/138-balancer.t index 2355b59303..dca1784673 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -435,7 +435,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ -=== TEST 15: test if execeed proxy_next_upstream_limit +=== TEST 15: test if exceed proxy_next_upstream_limit --- http_config lua_package_path "../lua-resty-core/lib/?.lua;;"; diff --git a/t/lib/CRC32.lua b/t/lib/CRC32.lua index fcf260cd4c..8ca0d5634e 100755 --- a/t/lib/CRC32.lua +++ b/t/lib/CRC32.lua @@ -156,7 +156,7 @@ end -- -- CRC32.lua -- --- A pure Lua implementation of a CRC32 hashing algorithm. Slower than using a C implemtation, +-- A pure Lua implementation of a CRC32 hashing algorithm. Slower than using a C implementation, -- but useful having no other dependencies. -- -- diff --git a/t/lib/Memcached.lua b/t/lib/Memcached.lua index e3288ac4fb..1f8c7f9578 100755 --- a/t/lib/Memcached.lua +++ b/t/lib/Memcached.lua @@ -541,7 +541,7 @@ end -- -- memcache:set_hash(hashfunc) -- Sets a custom hash function for key values. The default is a CRC32 hashing function. --- 'hashfunc' should be defined receiving a single string parameter and returing a single integer value. +-- 'hashfunc' should be defined receiving a single string parameter and returning a single integer value. -- -- memcache:set_encode(func) -- Sets a custom encode function for serialising table values. 'func' should be defined receiving a single diff --git a/util/build.sh b/util/build.sh index 0bf7bae0b4..b940e0447f 100755 --- a/util/build.sh +++ b/util/build.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # this script is for developers only. -# dependent on the ngx-build script from the nginx-devel-utils repostory: +# dependent on the ngx-build script from the nginx-devel-utils repository: # https://github.com/openresty/nginx-devel-utils/blob/master/ngx-build # the resulting nginx is located at ./work/nginx/sbin/nginx From 557e0097dc771e96f6bbf1422ce89a56d6c04c22 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Mon, 25 Oct 2021 15:31:24 +0800 Subject: [PATCH 554/848] tests: remove workflows/label.yml since it is not working properly. (#1950) --- .github/workflows/label.yml | 21 --------------------- 1 file changed, 21 deletions(-) delete mode 100644 .github/workflows/label.yml diff --git a/.github/workflows/label.yml b/.github/workflows/label.yml deleted file mode 100644 index 28ee659e13..0000000000 --- a/.github/workflows/label.yml +++ /dev/null @@ -1,21 +0,0 @@ -name: Auto Assign to Project(s) -on: - issues: - types: [labeled] - pull_request: - types: [labeled] -env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} -jobs: - assign_to_project: - runs-on: ubuntu-latest - name: Assign to Project - steps: - - name: Assign pull requests with `could-be-merged` label to `Waiting review` - uses: srggrs/assign-one-project-github-action@1.2.0 - if: | - contains(github.event.issue.labels.*.name, 'could-be-merged') || - contains(github.event.pull_request.labels.*.name, 'could-be-merged') - with: - project: "https://github.com/orgs/openresty/projects/1" - column_name: "Waiting Review from agentzh" From 617cb5dadc14dddb4796d4fff8821dae325f4229 Mon Sep 17 00:00:00 2001 From: Sharp Liu Date: Tue, 26 Oct 2021 17:53:54 +0800 Subject: [PATCH 555/848] style: removed extra space. (#1952) --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index ace18a0f89..26467fddbc 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2825,7 +2825,7 @@ ngx_http_lua_socket_tcp_send(lua_State *L) switch (type) { case LUA_TNUMBER: - b->last = ngx_http_lua_write_num(L, 2, b->last); + b->last = ngx_http_lua_write_num(L, 2, b->last); break; case LUA_TSTRING: From 73181ca90efab5d7bd26529adb1801097581b73f Mon Sep 17 00:00:00 2001 From: jiahao Date: Mon, 25 Oct 2021 22:53:33 +0800 Subject: [PATCH 556/848] bumped the API version to 0.10.21. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 43f90bd0b1..ff23042250 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10020 +#define ngx_http_lua_version 10021 typedef struct { From 0dffd12277696935a3c23d73cc3286b5bb743fe7 Mon Sep 17 00:00:00 2001 From: jiahao Date: Tue, 26 Oct 2021 16:24:00 +0800 Subject: [PATCH 557/848] travis-ci: bumped the NGINX core to 1.21.3. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 19ef3926cd..5a60f5e43a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,8 +50,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 jobs: - - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.19.9 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.3 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.21.3 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f services: - memcached From 913b5583bfed32230a78109b2fb7b1d4130c143d Mon Sep 17 00:00:00 2001 From: jiahao Date: Tue, 26 Oct 2021 23:32:25 +0800 Subject: [PATCH 558/848] tests: updated the test cases in t/020-subrequest.t to reflect the changes in nginx. http/1.0 requests with transfer-encoding is not permitted since nginx 1.21.1 --- t/020-subrequest.t | 122 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 117 insertions(+), 5 deletions(-) diff --git a/t/020-subrequest.t b/t/020-subrequest.t index b3b6fb0129..0cf6a1f760 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1161,7 +1161,7 @@ r%5B%5D=http%3A%2F%2Fajax.googleapis.com%3A80%2Fajax%2Flibs%2Fjquery%2F1.7.2%2Fj GET /main --- response_body status: 500 -body: +body: @@ -1182,7 +1182,7 @@ body: GET /main --- response_body status: 500 -body: +body: @@ -1487,7 +1487,7 @@ post subreq: rc=0, status=200 --- response_body status: 200 -body: +body: truncated: true --- error_log @@ -1609,7 +1609,7 @@ post subreq: rc=0, status=200 --- response_body status: 200 -body: +body: truncated: true --- error_log @@ -2082,7 +2082,7 @@ post subreq: rc=0, status=200 --- response_body status: 200 -body: +body: truncated: true --- error_log @@ -3000,6 +3000,8 @@ method: GET, uri: /foo, X: GET /bar HTTP/1.0 0 --- no_error_log [error] +--- skip_nginx +3: >= 1.21.1 @@ -3128,6 +3130,8 @@ method: POST, uri: /foo 0 --- no_error_log [error] +--- skip_nginx +3: >= 1.21.1 @@ -3259,6 +3263,8 @@ method: POST, uri: /foo 0 --- no_error_log [error] +--- skip_nginx +3: >= 1.21.1 @@ -3391,6 +3397,8 @@ method: POST, uri: /foo 0 --- no_error_log [error] +--- skip_nginx +3: >= 1.21.1 @@ -3410,3 +3418,107 @@ GET /lua --- error_code: 500 --- error_log unsupported HTTP method: 10240 + + + +=== TEST 82: bad requests with both Content-Length and Transfer-Encoding (nginx >= 1.21.1) +--- http_config + upstream backend { + server unix:$TEST_NGINX_HTML_DIR/nginx.sock; + keepalive 32; + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + content_by_lua_block { + ngx.say("method: ", ngx.var.request_method, + ", uri: ", ngx.var.uri, + ", X: ", ngx.var.http_x) + } + } + } +--- config + location /proxy { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://backend/foo; + } + + location /capture { + server_tokens off; + more_clear_headers Date; + + content_by_lua_block { + local res = ngx.location.capture("/proxy") + ngx.print(res.body) + } + } + + location /t { + content_by_lua_block { + local req = [[ +GET /capture HTTP/1.1 +Host: test.com +Content-Length: 37 +Transfer-Encoding: chunked + +0 + +GET /capture HTTP/1.1 +Host: test.com +X: GET /bar HTTP/1.0 + +]] + + local sock = ngx.socket.tcp() + sock:settimeout(1000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send req: ", err) + return + end + + ngx.say("req bytes: ", bytes) + + local n_resp = 0 + + local reader = sock:receiveuntil("\r\n") + while true do + local line, err = reader() + if line then + ngx.say(line) + if line == "0" then + n_resp = n_resp + 1 + end + + if n_resp >= 2 then + break + end + + else + ngx.say("err: ", err) + break + end + end + + sock:close() + } + } +--- request +GET /t +--- response_body_like +req bytes: 146 +HTTP/1.1 400 Bad Request +--- no_error_log +[error] +--- skip_nginx +3: < 1.21.1 From 560461e1f970154e14284e56e3227e95f650a300 Mon Sep 17 00:00:00 2001 From: jiahao Date: Tue, 26 Oct 2021 23:52:15 +0800 Subject: [PATCH 559/848] tests: updated the test cases in t/104-req-raw-header.t to reflect the changes in nginx. multi-line request headers are considered invalid since nginx 1.21.1. --- t/020-subrequest.t | 10 +++++----- t/104-req-raw-header.t | 29 +++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 0cf6a1f760..844b64794f 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1161,7 +1161,7 @@ r%5B%5D=http%3A%2F%2Fajax.googleapis.com%3A80%2Fajax%2Flibs%2Fjquery%2F1.7.2%2Fj GET /main --- response_body status: 500 -body: +body: @@ -1182,7 +1182,7 @@ body: GET /main --- response_body status: 500 -body: +body: @@ -1487,7 +1487,7 @@ post subreq: rc=0, status=200 --- response_body status: 200 -body: +body: truncated: true --- error_log @@ -1609,7 +1609,7 @@ post subreq: rc=0, status=200 --- response_body status: 200 -body: +body: truncated: true --- error_log @@ -2082,7 +2082,7 @@ post subreq: rc=0, status=200 --- response_body status: 200 -body: +body: truncated: true --- error_log diff --git a/t/104-req-raw-header.t b/t/104-req-raw-header.t index ac1f89e1b4..f147b2bcf9 100644 --- a/t/104-req-raw-header.t +++ b/t/104-req-raw-header.t @@ -331,6 +331,8 @@ Foo: bar baz\r } --- no_error_log [error] +--- skip_nginx +3: >= 1.21.1 @@ -362,6 +364,8 @@ Connection: close\r --- no_error_log [error] --- timeout: 5 +--- skip_nginx +3: >= 1.21.1 @@ -988,3 +992,28 @@ Connection: close --- no_error_log [error] --- timeout: 5 + + + +=== TEST 34: multi-line header is invalid (nginx >= 1.21.1) +--- config + location /t { + content_by_lua ' + ngx.print(ngx.req.raw_header()) + '; + } +--- raw_request eval +"GET /t HTTP/1.1\r +Host: localhost\r +Connection: close\r +Foo: bar baz\r + blah\r +\r +" +--- error_code: 400 +--- error_log +client sent invalid header line: "\x20..." while reading client request headers +--- no_error_log +[error] +--- skip_nginx +3: < 1.21.1 From 3f33dd862a3d8539b96dda09b624c89712cbe0c8 Mon Sep 17 00:00:00 2001 From: Odin Hultgren Van Der Horst Date: Thu, 28 Oct 2021 15:54:27 +0200 Subject: [PATCH 560/848] optimization: removed superfluous code from shdict_store. (#1956) avoid setting the key_len, key_data and value_len fields since they already have the same values when replacing the value data. --- src/ngx_http_lua_shdict.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 8421ac1714..d841eb95ee 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -1426,8 +1426,6 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, ngx_queue_remove(&sd->queue); ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); - sd->key_len = (u_short) key_len; - if (exptime > 0) { tp = ngx_timeofday(); sd->expires = (uint64_t) tp->sec * 1000 + tp->msec @@ -1439,14 +1437,11 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, sd->user_flags = user_flags; - sd->value_len = (uint32_t) str_value_len; - dd("setting value type to %d", value_type); sd->value_type = (uint8_t) value_type; - p = ngx_copy(sd->data, key, key_len); - ngx_memcpy(p, str_value_buf, str_value_len); + ngx_memcpy(sd->data + key_len, str_value_buf, str_value_len); ngx_shmtx_unlock(&ctx->shpool->mutex); From 931f62799cac6a657fa4a3ea524c43f3b0d55b20 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 2 Nov 2021 19:26:21 +0800 Subject: [PATCH 561/848] bugfix: ngx_http_lua_assert failure in ngx_http_lua_worker_thread.c. (#1960) --- src/ngx_http_lua_worker_thread.c | 3 ++- util/build.sh | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 4da6c9f5da..00c1d924b2 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -266,7 +266,8 @@ ngx_http_lua_worker_thread_handler(void *data, ngx_log_t *log) ngx_http_lua_worker_thread_ctx_t *ctx = data; lua_State *vm = ctx->ctx->vm; - ngx_http_lua_assert(lua_gettop(vm) == ctx->n_args); + /* function + args in the lua stack */ + ngx_http_lua_assert(lua_gettop(vm) == ctx->n_args + 1); ctx->rc = lua_pcall(vm, ctx->n_args, LUA_MULTRET, 0); } diff --git a/util/build.sh b/util/build.sh index b940e0447f..713b8482d3 100755 --- a/util/build.sh +++ b/util/build.sh @@ -28,7 +28,7 @@ time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ --with-ipv6 \ - --with-cc-opt="-I$PCRE_INC -I$OPENSSL_INC" \ + --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC" \ --with-http_v2_module \ --with-http_realip_module \ --with-http_ssl_module \ From 751667913ed5b229c1cb17cead3078d2955bc024 Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Tue, 2 Nov 2021 21:35:11 -0700 Subject: [PATCH 562/848] style: changed the comment a bit. --- src/ngx_http_lua_lex.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/ngx_http_lua_lex.c b/src/ngx_http_lua_lex.c index 45cf7540e7..c9064030c2 100644 --- a/src/ngx_http_lua_lex.c +++ b/src/ngx_http_lua_lex.c @@ -3,8 +3,7 @@ * * WARNING: DO NOT EVER EDIT THIS FILE!! * - * This file was automatically generated by the re.pl script of sregex's - * "dfa-multi-re" git branch. + * This file was automatically generated by the OpenResty Regex compiler. */ From e682133289b3f5f4007ded721fa5e4e374211b88 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 4 Nov 2021 09:31:21 +0800 Subject: [PATCH 563/848] tests: wrong pid path when running jobs greater than 1. (#1961) --- t/157-socket-keepalive-hup.t | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/t/157-socket-keepalive-hup.t b/t/157-socket-keepalive-hup.t index 357bb59682..d9436b3b73 100644 --- a/t/157-socket-keepalive-hup.t +++ b/t/157-socket-keepalive-hup.t @@ -32,7 +32,8 @@ __DATA__ set $port $TEST_NGINX_SERVER_PORT; content_by_lua_block { - local f, err = io.open("$TEST_NGINX_SERVER_ROOT/logs/nginx.pid", "r") + local pidfile = ngx.config.prefix() .. "/logs/nginx.pid" + local f, err = io.open(pidfile, "r") if not f then ngx.say("failed to open nginx.pid: ", err) return From 9e692e29cfe3c4b04399ec044e29a122a7849932 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 5 Nov 2021 22:46:39 +0800 Subject: [PATCH 564/848] travis-ci: updated ftp.pcre.org to sourceforge mirror. (#1965) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 5a60f5e43a..ff56f05f7b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -67,7 +67,7 @@ before_install: install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://ftp.pcre.org/pub/pcre/pcre-$PCRE_VER.tar.gz; fi + - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty From 9a93a3a0b57cc318790d702e761542cfa4cc2f5f Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 5 Nov 2021 23:26:39 +0800 Subject: [PATCH 565/848] travis-ci: bumped the NGINX core to 1.21.4. (#1964) --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index ff56f05f7b..eeda49dfba 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,8 +50,8 @@ env: - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 jobs: - - NGINX_VERSION=1.21.3 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.3 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f services: - memcached From 892ed514f7d2226f7b4c9fccb9d322b66bde0e0b Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Tue, 9 Nov 2021 18:32:21 +0800 Subject: [PATCH 566/848] tests: skip test cases for unsupported event types. (#1967) * skip test cases for t/082-body-filter-2.t when the event type is not 'poll'. * skip test cases for t/166-worker-thread.t when the event do not support worker thread. --- t/082-body-filter-2.t | 15 +++++++++++---- t/166-worker-thread.t | 12 +++++++++++- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t index 27baf400c2..eafe7cf26a 100644 --- a/t/082-body-filter-2.t +++ b/t/082-body-filter-2.t @@ -1,12 +1,19 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: +our $SkipReason; + BEGIN { - $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; - $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; - $ENV{MOCKEAGAIN}='w' + if ($ENV{TEST_NGINX_EVENT_TYPE} && $ENV{TEST_NGINX_EVENT_TYPE} ne 'poll') { + $SkipReason = "unavailable for the event type '$ENV{TEST_NGINX_EVENT_TYPE}'"; + + } else { + $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + $ENV{MOCKEAGAIN}='w' + } } -use Test::Nginx::Socket::Lua; +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); #worker_connections(1014); #master_process_enabled(1); diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 9717e692c0..8b874c55ad 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1,6 +1,16 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_EVENT_TYPE} + && $ENV{TEST_NGINX_EVENT_TYPE} !~ /^kqueue|epoll|eventport$/) + { + $SkipReason = "unavailable for the event type '$ENV{TEST_NGINX_EVENT_TYPE}'"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); #worker_connections(1014); #master_on(); From 91907a0f316e07b4bba1884cbb2f448e334310ee Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 12 Nov 2021 11:41:30 +0800 Subject: [PATCH 567/848] bugfix: forgot to clean up the cached thread contexts when worker process exits. (#1968) --- src/ngx_http_lua_exitworkerby.c | 8 ++++++++ src/ngx_http_lua_worker_thread.c | 14 ++++++++++++++ src/ngx_http_lua_worker_thread.h | 1 + 3 files changed, 23 insertions(+) diff --git a/src/ngx_http_lua_exitworkerby.c b/src/ngx_http_lua_exitworkerby.c index cd59dc4bed..f1329566bc 100644 --- a/src/ngx_http_lua_exitworkerby.c +++ b/src/ngx_http_lua_exitworkerby.c @@ -13,6 +13,10 @@ #include "ngx_http_lua_exitworkerby.h" #include "ngx_http_lua_util.h" +#if (NGX_THREADS) +#include "ngx_http_lua_worker_thread.h" +#endif + void ngx_http_lua_exit_worker(ngx_cycle_t *cycle) @@ -23,6 +27,10 @@ ngx_http_lua_exit_worker(ngx_cycle_t *cycle) ngx_http_lua_ctx_t *ctx; ngx_http_conf_ctx_t *conf_ctx; +#if (NGX_THREADS) + ngx_http_lua_thread_exit_process(); +#endif + lmcf = ngx_http_cycle_get_module_main_conf(cycle, ngx_http_lua_module); if (lmcf == NULL || lmcf->exit_worker_handler == NULL diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 00c1d924b2..5133fa6e76 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -45,6 +45,20 @@ static ngx_http_lua_task_ctx_t *ctxpool = &dummy_ctx; static ngx_uint_t worker_thread_vm_count; +void +ngx_http_lua_thread_exit_process(void) +{ + ngx_http_lua_task_ctx_t *ctx; + + while (ctxpool->next != NULL) { + ctx = ctxpool->next; + ctxpool->next = ctx->next; + lua_close(ctx->vm); + ngx_free(ctx); + } +} + + /* * Re-implement ngx_thread_task_alloc to avoid alloc from request pool * since the request may exit before worker thread finish. diff --git a/src/ngx_http_lua_worker_thread.h b/src/ngx_http_lua_worker_thread.h index d08fb903a7..2daa520a59 100644 --- a/src/ngx_http_lua_worker_thread.h +++ b/src/ngx_http_lua_worker_thread.h @@ -13,6 +13,7 @@ void ngx_http_lua_inject_worker_thread_api(ngx_log_t *log, lua_State *L); +void ngx_http_lua_thread_exit_process(void); #endif /* _NGX_HTTP_LUA_WORKER_THREAD_H_INCLUDED_ */ From 0b98a58adaced13af65160787c535d1d7aae54b1 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Mon, 15 Nov 2021 10:56:18 +0800 Subject: [PATCH 568/848] tests: 132-lua-blocks.t and 158-global-var.t: use RegExp instead of strict character match. (#1971) --- t/132-lua-blocks.t | 10 +++++----- t/158-global-var.t | 14 +++++++------- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/t/132-lua-blocks.t b/t/132-lua-blocks.t index 6bf8f944c5..2a1644e532 100644 --- a/t/132-lua-blocks.t +++ b/t/132-lua-blocks.t @@ -312,7 +312,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:\d+/ @@ -331,7 +331,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]==]", the inlined Lua code may be too long in .*?\bnginx.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]==]", the inlined Lua code may be too long in .*?\bnginx.conf:\d+/ @@ -350,7 +350,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:\d+/ @@ -369,7 +369,7 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]=]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]=]", the inlined Lua code may be too long in .*?\bnginx\.conf:\d+/ @@ -640,4 +640,4 @@ hello, world [error] --- must_die --- error_log eval -qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:41/ +qr/\[emerg\] .*? Lua code block missing the closing long bracket "]]", the inlined Lua code may be too long in .*?\bnginx\.conf:\d+/ diff --git a/t/158-global-var.t b/t/158-global-var.t index 258ccdbce0..def3b7418f 100644 --- a/t/158-global-var.t +++ b/t/158-global-var.t @@ -86,7 +86,7 @@ set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] +rewrite_by_lua\(nginx\.conf:\d+\):\d+: in main chunk, \n\z/, "old foo: 1\n"] @@ -109,7 +109,7 @@ rewrite_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] +access_by_lua\(nginx\.conf:\d+\):\d+: in main chunk, \n\z/, "old foo: 1\n"] @@ -132,7 +132,7 @@ access_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk, )/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -content_by_lua\(nginx\.conf:48\):3: in main chunk, \n\z/, "old foo: 1\n"] +content_by_lua\(nginx\.conf:\d+\):\d+: in main chunk, \n\z/, "old foo: 1\n"] @@ -179,10 +179,10 @@ header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|2)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk,)/ --- grep_error_log_out eval [qr/\[warn\] .*?writing a global Lua variable \('foo'\) -body_filter_by_lua:3: in main chunk, +body_filter_by_lua:3: in main chunk, old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] @@ -208,7 +208,7 @@ old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in main chunk)/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -log_by_lua\(nginx\.conf:50\):3: in main chunk\n\z/, "old foo: 1\n"] +log_by_lua\(nginx\.conf:\d+\):\d+: in main chunk\n\z/, "old foo: 1\n"] @@ -331,7 +331,7 @@ ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(.*?\):\d+: in\b)/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -content_by_lua\(nginx\.conf:56\):4: in\n\z/, "old foo: 1\n"] +content_by_lua\(nginx\.conf:\d+\):\d+: in\n\z/, "old foo: 1\n"] From ce8fd0268c6f18ff96b4f92acc9dc46fb7673af9 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Mon, 15 Nov 2021 10:58:03 +0800 Subject: [PATCH 569/848] tests: fixed test plan for 129-ssl-socket.t and 162-exit-worker.t. (#1972) --- t/129-ssl-socket.t | 2 +- t/162-exit-worker.t | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 3fecf3ee2f..c26f0cec53 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -6,7 +6,7 @@ use File::Basename; repeat_each(2); -plan tests => repeat_each() * blocks() * 7 - 6; +plan tests => repeat_each() * (blocks() * 7 - 3); $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index 5d92c4fe1a..3138617615 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -5,7 +5,8 @@ use Test::Nginx::Socket::Lua; master_on(); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 2) + 11; +# NB: the shutdown_error_log block is independent from repeat times +plan tests => repeat_each() * (blocks() * 2 + 1) + 13; #log_level("warn"); no_long_string(); From 3229945a2f3dbb13e1d9ff724e04a36dffb00231 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Tue, 16 Nov 2021 18:01:07 +0800 Subject: [PATCH 570/848] bugfix: prevent illegal memory access in ngx_http_lua_util.c. (#1974) found by core dump caused by running TEST 3 of t/072-conditional-get.t in mockeagain write mode. --- src/ngx_http_lua_util.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 0d29c7bcb6..bae59daf34 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -719,6 +719,10 @@ ngx_http_lua_output_filter(ngx_http_request_t *r, ngx_chain_t *in) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return rc; + } + ngx_chain_update_chains(r->pool, &ctx->free_bufs, &ctx->busy_bufs, &in, (ngx_buf_tag_t) &ngx_http_lua_module); @@ -1027,10 +1031,14 @@ ngx_http_lua_generic_phase_post_read(ngx_http_request_t *r) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - ctx->read_body_done = 1; - r->main->count--; + if (ctx == NULL) { + return; + } + + ctx->read_body_done = 1; + if (ctx->waiting_more_body) { ctx->waiting_more_body = 0; ngx_http_core_run_phases(r); From 94dd3ac9406f15369dd7f27c5dc4ce62b61a3784 Mon Sep 17 00:00:00 2001 From: Xiaobo Liu Date: Tue, 23 Nov 2021 22:21:08 +0800 Subject: [PATCH 571/848] style: updated the indentation of .travis.yml according to the yaml specification. (#1976) --- .travis.yml | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/.travis.yml b/.travis.yml index eeda49dfba..f4cc579b3f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -14,19 +14,19 @@ compiler: addons: apt: packages: - - axel - - cpanminus - - libtest-base-perl - - libtext-diff-perl - - liburi-perl - - libwww-perl - - libtest-longstring-perl - - liblist-moreutils-perl - - libgd-dev + - axel + - cpanminus + - libtest-base-perl + - libtext-diff-perl + - liburi-perl + - libwww-perl + - libtest-longstring-perl + - liblist-moreutils-perl + - libgd-dev cache: directories: - - download-cache + - download-cache env: global: @@ -54,9 +54,9 @@ env: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f services: - - memcached - - redis - - mysql + - memcached + - redis + - mysql before_install: - sudo apt update From 6302d47409e55b59838a9f450f3d713aae7e9eb5 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 10 Dec 2021 11:12:10 +0800 Subject: [PATCH 572/848] tests: NGINX_TEST_CHECK_LEAK detects memoryleak because nginx can not free resolver ctx quickly. This was introduce in commit acd53645754ce which is use to resovle double free of resolver ctx. This issue happens only when nginx can not get any reply from the dns server until timeout. --- t/127-uthread-kill.t | 1 + 1 file changed, 1 insertion(+) diff --git a/t/127-uthread-kill.t b/t/127-uthread-kill.t index 32509c6f24..11caee582f 100644 --- a/t/127-uthread-kill.t +++ b/t/127-uthread-kill.t @@ -139,6 +139,7 @@ lua clean up the timer for pending ngx.sleep === TEST 3: kill pending resolver --- config resolver 127.0.0.2:12345; + resolver_timeout 5ms; location /lua { content_by_lua ' local function f() From 94fa662bc3211f2557cbdb391e0c6feffcdde2d3 Mon Sep 17 00:00:00 2001 From: Xiaobo Liu Date: Fri, 17 Dec 2021 12:00:05 +0800 Subject: [PATCH 573/848] style: removed extra space. (#1981) --- src/ngx_http_lua_util.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 348a8666a4..9ad34b93ea 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -564,7 +564,7 @@ ngx_http_lua_free_thread(ngx_http_request_t *r, lua_State *L, int co_ref, { #ifdef HAVE_LUA_RESETTHREAD ngx_queue_t *q; - ngx_http_lua_thread_ref_t *tref ; + ngx_http_lua_thread_ref_t *tref; ngx_http_lua_ctx_t *ctx; ngx_log_debug2(NGX_LOG_DEBUG_HTTP, From 37ec9f75617544cfcb39201724374c66d9328094 Mon Sep 17 00:00:00 2001 From: kfang Date: Thu, 21 Nov 2019 14:33:03 +0800 Subject: [PATCH 574/848] doc: Readme: completed the Installation part, with add lua-resty-core and lua-resty-lrucache. --- README.markdown | 14 ++++++++++++++ doc/HttpLuaModule.wiki | 14 ++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/README.markdown b/README.markdown index d8f28769d7..39be3a034e 100644 --- a/README.markdown +++ b/README.markdown @@ -352,6 +352,8 @@ Alternatively, ngx_lua can be manually compiled into Nginx: 1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags) 1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags) 1. Download the latest supported version of Nginx [HERE](https://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) +1. Download the latest version of the lua-resty-core [HERE](https://lua-resty-core) +1. Download the latest version of the lua-resty-lrucache [HERE](https://github.com/openresty/lua-resty-lrucache) Build the source with this module: @@ -383,6 +385,18 @@ Build the source with this module: # machine. make -j2 make install + + # Note that this version of lug-nginx-module not allow to set `lua_load_resty_core off;` any more. + # So, you have to install `lua-resty-core` and `lua-resty-lrucache` manually as below. + + cd lua-resty-core + make install PREFIX=/opt/nginx + cd lua-resty-lrucache + make install PREFIX=/opt/nginx + + # add necessary `lua_package_path` directive to `nginx.conf`, in the http context + + lua_package_path "/opt/nginx/lib/lua/?.lua;;"; ``` [Back to TOC](#table-of-contents) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 98f3c5879b..6e72ab058a 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -283,6 +283,8 @@ Alternatively, ngx_lua can be manually compiled into Nginx: # Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE] # Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE] # Download the latest supported version of Nginx [https://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) +# Download the latest version of the lua-resty-core [HERE](https://lua-resty-core) +# Download the latest version of the lua-resty-lrucache [HERE](https://github.com/openresty/lua-resty-lrucache) Build the source with this module: @@ -313,6 +315,18 @@ Build the source with this module: # machine. make -j2 make install + + # Note that this version of lug-nginx-module not allow to set `lua_load_resty_core off;` any more. + # So, you have to install `lua-resty-core` and `lua-resty-lrucache` manually as below. + + cd lua-resty-core + make install PREFIX=/opt/nginx + cd lua-resty-lrucache + make install PREFIX=/opt/nginx + + # add necessary `lua_package_path` directive to `nginx.conf`, in the http context + + lua_package_path "/opt/nginx/lib/lua/?.lua;;"; == Building as a dynamic module == From 5483b9157cd8ca7d3d71d909922bd08437f34d08 Mon Sep 17 00:00:00 2001 From: Landon Manning Date: Sat, 25 Dec 2021 10:27:26 +0800 Subject: [PATCH 575/848] feature: added http const HTTP_NOT_IMPLEMENTED, fixes #1466. Co-authored-by: lijunlong --- README.markdown | 3 ++- doc/HttpLuaModule.wiki | 3 ++- src/ngx_http_lua_consts.c | 4 ++++ src/ngx_http_lua_util.c | 2 +- t/005-exit.t | 16 ++++++++++++++++ t/062-count.t | 8 ++++---- 6 files changed, 29 insertions(+), 7 deletions(-) diff --git a/README.markdown b/README.markdown index 39be3a034e..5f60b055c4 100644 --- a/README.markdown +++ b/README.markdown @@ -3875,7 +3875,8 @@ HTTP status constants value = ngx.HTTP_CLOSE (444) (first added in the v0.9.20 release) value = ngx.HTTP_ILLEGAL (451) (first added in the v0.9.20 release) value = ngx.HTTP_INTERNAL_SERVER_ERROR (500) - value = ngx.HTTP_METHOD_NOT_IMPLEMENTED (501) + value = ngx.HTTP_NOT_IMPLEMENTED (501) + value = ngx.HTTP_METHOD_NOT_IMPLEMENTED (501) (kept for compatibility) value = ngx.HTTP_BAD_GATEWAY (502) (first added in the v0.9.20 release) value = ngx.HTTP_SERVICE_UNAVAILABLE (503) value = ngx.HTTP_GATEWAY_TIMEOUT (504) (first added in the v0.3.1rc38 release) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 6e72ab058a..388777397c 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3178,7 +3178,8 @@ These constants are usually used in [[#ngx.location.capture|ngx.location.capture value = ngx.HTTP_CLOSE (444) (first added in the v0.9.20 release) value = ngx.HTTP_ILLEGAL (451) (first added in the v0.9.20 release) value = ngx.HTTP_INTERNAL_SERVER_ERROR (500) - value = ngx.HTTP_METHOD_NOT_IMPLEMENTED (501) + value = ngx.HTTP_NOT_IMPLEMENTED (501) + value = ngx.HTTP_METHOD_NOT_IMPLEMENTED (501) (kept for compatibility) value = ngx.HTTP_BAD_GATEWAY (502) (first added in the v0.9.20 release) value = ngx.HTTP_SERVICE_UNAVAILABLE (503) value = ngx.HTTP_GATEWAY_TIMEOUT (504) (first added in the v0.3.1rc38 release) diff --git a/src/ngx_http_lua_consts.c b/src/ngx_http_lua_consts.c index 47ba998576..f2dcf4a490 100644 --- a/src/ngx_http_lua_consts.c +++ b/src/ngx_http_lua_consts.c @@ -176,6 +176,10 @@ ngx_http_lua_inject_http_consts(lua_State *L) lua_pushinteger(L, NGX_HTTP_INTERNAL_SERVER_ERROR); lua_setfield(L, -2, "HTTP_INTERNAL_SERVER_ERROR"); + lua_pushinteger(L, NGX_HTTP_NOT_IMPLEMENTED); + lua_setfield(L, -2, "HTTP_NOT_IMPLEMENTED"); + + /* keep for backward compatibility */ lua_pushinteger(L, NGX_HTTP_NOT_IMPLEMENTED); lua_setfield(L, -2, "HTTP_METHOD_NOT_IMPLEMENTED"); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index bae59daf34..232a1c1315 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -827,7 +827,7 @@ static void ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log) { - lua_createtable(L, 0 /* narr */, 113 /* nrec */); /* ngx.* */ + lua_createtable(L, 0 /* narr */, 115 /* nrec */); /* ngx.* */ lua_pushcfunction(L, ngx_http_lua_get_raw_phase_context); lua_setfield(L, -2, "_phase_ctx"); diff --git a/t/005-exit.t b/t/005-exit.t index ef4e0f3031..de9f2c41cf 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -724,3 +724,19 @@ GET /t --- response_body --- no_error_log [error] + + + +=== TEST 25: 501 Method Not Implemented +--- config + location /lua { + content_by_lua ' + ngx.exit(ngx.HTTP_NOT_IMPLEMENTED) + '; + } +--- request +GET /lua +--- error_code: 501 +--- response_body_like: 501 (?:Method )?Not Implemented +--- no_error_log +[error] diff --git a/t/062-count.t b/t/062-count.t index 598441c513..f5ba12c91d 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -34,7 +34,7 @@ __DATA__ --- request GET /test --- response_body -ngx: 114 +ngx: 115 --- no_error_log [error] @@ -55,7 +55,7 @@ ngx: 114 --- request GET /test --- response_body -114 +115 --- no_error_log [error] @@ -83,7 +83,7 @@ GET /test --- request GET /test --- response_body -n = 114 +n = 115 --- no_error_log [error] @@ -305,7 +305,7 @@ GET /t --- response_body_like: 404 Not Found --- error_code: 404 --- error_log -ngx. entry count: 114 +ngx. entry count: 115 From e98f5c508ff782b49cafa7a970b6e454a4faeb77 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 25 Dec 2021 11:03:49 +0800 Subject: [PATCH 576/848] doc: Clarify ngx.req.set_header usage for proxies. `ngx.req.set_header` acts similar `proxy_set_header` for `proxy_pass` scenarios. Co-authored-by: Zaar Hai --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index 5f60b055c4..c3af842219 100644 --- a/README.markdown +++ b/README.markdown @@ -5222,6 +5222,8 @@ For security considerations, this method will automatically escape " ", """, "(" By default, all the subrequests subsequently initiated by [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) will inherit the new header. +It is not a Lua's equivalent of nginx `proxy_set_header` directive (same is true about [ngx.req.clear_header](#ngxreqclear_header)). `proxy_set_header` only affects the upstream request while `ngx.req.set_header` change the incoming request. Record the http headers in the access log file will show the difference. But you still can use it as an alternative of nginx `proxy_set_header` directive as long as you know the difference. + Here is an example of setting the `Content-Type` header: ```lua diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 388777397c..ad00ba2fa5 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4387,6 +4387,8 @@ For security considerations, this method will automatically escape " ", """, "(" By default, all the subrequests subsequently initiated by [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] will inherit the new header. +It is not a Lua's equivalent of nginx `proxy_set_header` directive (same is true about [ngx.req.clear_header](#ngxreqclear_header)). `proxy_set_header` only affects the upstream request while `ngx.req.set_header` change the incoming request. Record the http headers in the access log file will show the difference. But you still can use it as an alternative of nginx `proxy_set_header` directive as long as you know the difference. + Here is an example of setting the Content-Type header: From 0b6da3b6ce6ae9417b9d464d853ff1301cd0e9ed Mon Sep 17 00:00:00 2001 From: chronolaw Date: Sat, 28 Apr 2018 13:05:45 +0800 Subject: [PATCH 577/848] feature: added ffi-based function 'ngx_http_lua_ffi_req_is_internal'. --- src/ngx_http_lua_misc.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/ngx_http_lua_misc.c b/src/ngx_http_lua_misc.c index 2300aa3e4c..4e93f68f1e 100644 --- a/src/ngx_http_lua_misc.c +++ b/src/ngx_http_lua_misc.c @@ -99,6 +99,17 @@ ngx_http_lua_ffi_set_resp_status(ngx_http_request_t *r, int status) } +int +ngx_http_lua_ffi_req_is_internal(ngx_http_request_t *r) +{ + if (r->connection->fd == (ngx_socket_t) -1) { + return NGX_HTTP_LUA_FFI_BAD_CONTEXT; + } + + return r->internal; +} + + int ngx_http_lua_ffi_is_subrequest(ngx_http_request_t *r) { From b67b22af885fbe9e72609772d7071bd1bf04eb0d Mon Sep 17 00:00:00 2001 From: Thibault Charbonnier Date: Mon, 30 Jan 2017 19:04:21 -0800 Subject: [PATCH 578/848] feature: prevent calling ngx.exit() with invalid values. --- src/ngx_http_lua_control.c | 8 ++++ t/005-exit.t | 79 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index bb18d8f5d9..c79101cf06 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -358,6 +358,14 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, { ngx_http_lua_ctx_t *ctx; + if (status == NGX_AGAIN || status == NGX_DONE) { + *errlen = ngx_snprintf(err, *errlen, + "bad argument to 'ngx.exit': does not accept " + "NGX_AGAIN or NGX_DONE") + - err; + return NGX_ERROR; + } + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { *errlen = ngx_snprintf(err, *errlen, "no request ctx found") - err; diff --git a/t/005-exit.t b/t/005-exit.t index de9f2c41cf..eb123bd179 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -740,3 +740,82 @@ GET /lua --- response_body_like: 501 (?:Method )?Not Implemented --- no_error_log [error] + + + +=== TEST 26: accepts NGX_OK +--- config + location = /t { + content_by_lua_block { + ngx.exit(ngx.OK) + } + } +--- request +GET /t +--- response_body +--- no_error_log +[error] + + + +=== TEST 27: accepts NGX_ERROR +--- config + location = /t { + content_by_lua_block { + ngx.exit(ngx.ERROR) + } + } +--- request +GET /t +--- error_code: +--- response_body +--- no_error_log +[error] + + + +=== TEST 28: accepts NGX_DECLINED +--- config + location = /t { + content_by_lua_block { + ngx.exit(ngx.DECLINED) + } + } +--- request +GET /t +--- error_code: +--- response_body +--- no_error_log +[error] + + + +=== TEST 29: refuses NGX_AGAIN +--- config + location = /t { + content_by_lua_block { + ngx.exit(ngx.AGAIN) + } + } +--- request +GET /t +--- error_code: 500 +--- response_body_like: 500 Internal Server Error +--- error_log eval +qr/\[error\] .*? bad argument to 'ngx.exit': does not accept NGX_AGAIN or NGX_DONE/ + + + +=== TEST 30: refuses NGX_DONE +--- config + location = /t { + content_by_lua_block { + ngx.exit(ngx.DONE) + } + } +--- request +GET /t +--- error_code: 500 +--- response_body_like: 500 Internal Server Error +--- error_log eval +qr/\[error\] .*? bad argument to 'ngx.exit': does not accept NGX_AGAIN or NGX_DONE/ From a276476266efb2cd8fab35c2ecc6d703495a19bd Mon Sep 17 00:00:00 2001 From: "yang.yang" Date: Sun, 23 Apr 2017 10:33:32 +0800 Subject: [PATCH 579/848] change: use nil instead of false when lpush&rpush overflow. Co-authered-by: lijunlong --- src/ngx_http_lua_shdict.c | 2 +- t/145-shdict-list.t | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index d841eb95ee..26622d29f0 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -943,7 +943,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) ngx_shmtx_unlock(&ctx->shpool->mutex); - lua_pushboolean(L, 0); + lua_pushnil(L); lua_pushliteral(L, "no memory"); return 2; } diff --git a/t/145-shdict-list.t b/t/145-shdict-list.t index 6a4759cd02..5672adbc55 100644 --- a/t/145-shdict-list.t +++ b/t/145-shdict-list.t @@ -827,3 +827,27 @@ GET /test aa list value: nil --- no_error_log [error] + + + +=== TEST 22: lpush return nil +--- http_config + lua_shared_dict dogs 100k; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + for i = 1, 2920 + do + local len, err = dogs:lpush("foo", "bar") + end + local len, err = dogs:lpush("foo", "bar") + ngx.say(len) + } + } +--- request +GET /test +--- response_body +nil +--- no_error_log +[error] From d0e56ca14693911196f237cd60735c05b9cf3c4d Mon Sep 17 00:00:00 2001 From: spacewander Date: Sat, 12 May 2018 13:50:25 +0800 Subject: [PATCH 580/848] optimize: don't calculate hash when clearing the header. --- src/ngx_http_lua_headers_out.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 6e1879c50f..050f12b4ed 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -499,7 +499,13 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, return NGX_ERROR; } - hv.hash = ngx_hash_key_lc(key.data, key.len); + if (value.len > 0) { + hv.hash = ngx_hash_key_lc(key.data, key.len); + + } else { + hv.hash = 0; + } + hv.key = key; hv.offset = 0; From 85388f4ef68cfffc3c9ed4af927fdef5baaaaf5a Mon Sep 17 00:00:00 2001 From: tan jinhua <312841925@qq.com> Date: Wed, 25 Jul 2018 15:47:59 +0800 Subject: [PATCH 581/848] feature: check the number of parameters for ngx.thread.wait --- src/ngx_http_lua_uthread.c | 3 +++ t/098-uthread-wait.t | 19 +++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/src/ngx_http_lua_uthread.c b/src/ngx_http_lua_uthread.c index d30ffb4046..2208b2a819 100644 --- a/src/ngx_http_lua_uthread.c +++ b/src/ngx_http_lua_uthread.c @@ -125,6 +125,9 @@ ngx_http_lua_uthread_wait(lua_State *L) coctx = ctx->cur_co_ctx; nargs = lua_gettop(L); + if (nargs == 0) { + return luaL_error(L, "at least one coroutine should be specified"); + } for (i = 1; i <= nargs; i++) { sub_co = lua_tothread(L, i); diff --git a/t/098-uthread-wait.t b/t/098-uthread-wait.t index c2819ae886..e2818e008e 100644 --- a/t/098-uthread-wait.t +++ b/t/098-uthread-wait.t @@ -1321,3 +1321,22 @@ $s; --- no_error_log [error] [alert] + + + +=== TEST 23: no parameters for ngx.thread.wait +--- config + location /lua { + content_by_lua_block { + ngx.thread.wait() + ngx.say("ok") + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +at least one coroutine should be specified +--- no_error_log +[crit] From 2fce8ca380244e57657471346e956ea5ac7fa12e Mon Sep 17 00:00:00 2001 From: spacewander Date: Mon, 14 May 2018 22:35:31 +0800 Subject: [PATCH 582/848] optimize: don't calculate hash when clearing the request header. --- src/ngx_http_lua_headers_in.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 7626d1f949..a55d9cac7e 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -669,7 +669,13 @@ ngx_http_lua_set_input_header(ngx_http_request_t *r, ngx_str_t key, return NGX_ERROR; } - hv.hash = ngx_hash_key_lc(key.data, key.len); + if (value.len > 0) { + hv.hash = ngx_hash_key_lc(key.data, key.len); + + } else { + hv.hash = 0; + } + hv.key = key; hv.offset = 0; From de09fc6ccb80a49c66abfd025b0908f64d5bc16e Mon Sep 17 00:00:00 2001 From: detailyang Date: Fri, 3 Feb 2017 21:36:57 +0800 Subject: [PATCH 583/848] tests: removed duplicated c declaration in t/140-ssl-c-api.t. --- t/140-ssl-c-api.t | 2 -- 1 file changed, 2 deletions(-) diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 1c59135492..33f73ad50d 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -62,8 +62,6 @@ ffi.cdef[[ void ngx_http_lua_ffi_free_priv_key(void *cdata); - int ngx_http_lua_ffi_ssl_clear_certs(void *r, char **err); - int ngx_http_lua_ffi_ssl_verify_client(void *r, void *cdata, int depth, char **err); From 75c68b225a52db71768ebf51838bc6b79d820970 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 2 Jan 2022 10:11:42 +0800 Subject: [PATCH 584/848] bugfix: posted event handler was called after event memory was freed. (#1982) --- src/ngx_http_lua_semaphore.c | 9 ++++++- t/154-semaphore.t | 50 ++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 6f45eeffcc..435beaa29a 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -466,8 +466,11 @@ ngx_http_lua_sema_handler(ngx_event_t *ev) sem = ev->data; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "semaphore handler: wait queue: %sempty, resource count: %d", + ngx_queue_empty(&sem->wait_queue) ? "" : "not ", + sem->resource_count); while (!ngx_queue_empty(&sem->wait_queue) && sem->resource_count > 0) { - q = ngx_queue_head(&sem->wait_queue); ngx_queue_remove(q); @@ -566,6 +569,10 @@ ngx_http_lua_ffi_sema_gc(ngx_http_lua_sema_t *sem) "destroyed", sem); } + if (sem->sem_event.posted) { + ngx_delete_posted_event(&sem->sem_event); + } + ngx_http_lua_free_sema(sem); } diff --git a/t/154-semaphore.t b/t/154-semaphore.t index c7b00e03f6..5bb2b937af 100644 --- a/t/154-semaphore.t +++ b/t/154-semaphore.t @@ -118,3 +118,53 @@ hello, world --- no_shutdown_error_log semaphore gc wait queue is not empty --- SKIP + + + +=== TEST 3: exit before post_handler was called +If gc is called before the ngx_http_lua_sema_handler and free the sema memory +ngx_http_lua_sema_handler would use the freed memory. +--- config + location /up { + content_by_lua_block { + local semaphore = require "ngx.semaphore" + local sem = semaphore.new() + + local function sem_wait() + ngx.log(ngx.ERR, "ngx.sem wait start") + local ok, err = sem:wait(10) + if not ok then + ngx.log(ngx.ERR, "ngx.sem wait err: ", err) + else + ngx.log(ngx.ERR, "ngx.sem wait success") + end + end + local co = ngx.thread.spawn(sem_wait) + ngx.log(ngx.ERR, "ngx.sem post start") + sem:post() + ngx.log(ngx.ERR, "ngx.sem post end") + ngx.say("hello") + ngx.exit(200) + ngx.say("not reach here") + } + } + + location /t { + content_by_lua_block { + local res = ngx.location.capture("/up") + collectgarbage() + ngx.print(res.body) + } + } + +--- request +GET /t +--- response_body +hello +--- grep_error_log eval: qr/(ngx.sem .*?,|http close request|semaphore handler: wait queue: empty, resource count: 1|in lua gc, semaphore)/ +--- grep_error_log_out +ngx.sem wait start, +ngx.sem post start, +ngx.sem post end, +in lua gc, semaphore +http close request From f36fe2982b2dc748a842f2dda15a7490f91250b7 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 4 Jan 2022 13:14:34 +0800 Subject: [PATCH 585/848] feature: expose the 'Last-Modified' response header as ngx.header["Last-Modified"]. (#1983) Co-authored-by: Christian Becker --- src/ngx_http_lua_headers.c | 23 +++++++++++ t/016-resp-header.t | 85 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index a376a0eac9..3ebe7847cb 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -1068,6 +1068,7 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, { int found; u_char c, *p; + time_t last_modified; ngx_uint_t i; ngx_table_elt_t *h; ngx_list_part_t *part; @@ -1134,6 +1135,28 @@ ngx_http_lua_ffi_get_resp_header(ngx_http_request_t *r, break; + case 13: + if (ngx_strncasecmp(key_buf, (u_char *) "Last-Modified", 13) == 0) { + last_modified = r->headers_out.last_modified_time; + if (last_modified >= 0) { + p = ngx_palloc(r->pool, + sizeof("Mon, 28 Sep 1970 06:00:00 GMT")); + if (p == NULL) { + *errmsg = "no memory"; + return NGX_ERROR; + } + + values[0].data = p; + values[0].len = ngx_http_time(p, last_modified) - p; + + return 1; + } + + return 0; + } + + break; + default: break; } diff --git a/t/016-resp-header.t b/t/016-resp-header.t index b558415b24..65c5d75a8e 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -2141,3 +2141,88 @@ hi --- error_log my Content-Length: 8589934591 upstream prematurely closed connection while sending to client + + + +=== TEST 95: Expose the 'Last-Modified' response header as ngx.header["Last-Modified"] +--- config + location /a.txt { + header_filter_by_lua_block { + local last_modified = ngx.header["Last-Modified"] + if last_modified == nil then + ngx.log(ngx.ERR, "can not get lasted modified") + ngx.exit(500) + return + end + + local last_mod = ngx.parse_http_time(last_modified) + local age = ngx.time() - last_mod + ngx.header["Age"] = age + } + } +--- user_files +>>> a.txt +Foo +--- request +GET /a.txt +--- raw_response_headers_like chomp +Age: \d\r\n +--- no_error_log +[error] + + + +=== TEST 96: 'Last-Modified' from upstream +--- config + location /test/ { + proxy_pass http://127.0.0.1:$server_port/; + + header_filter_by_lua_block { + local last_modified = ngx.header["Last-Modified"] + if last_modified == nil then + ngx.log(ngx.ERR, "can not get lasted modified") + ngx.exit(500) + return + end + + local last_mod = ngx.parse_http_time(last_modified) + local age = ngx.time() - last_mod + ngx.header["Age"] = age + } + } + +--- user_files +>>> a.txt +Foo +--- request +GET /test/a.txt +--- raw_response_headers_like chomp +Age: \d\r\n +--- no_error_log +[error] + + + +=== TEST 97: 'Last-Modified' does not exist +--- config + location /test { + header_filter_by_lua_block { + local last_modified = ngx.header["Last-Modified"] + if last_modified == nil then + ngx.log(ngx.INFO, "Last-Modified is nil as expected") + return + end + + ngx.log(ngx.ERR, "Last-Modified expected to be nil, but got ", last_modified) + } + + content_by_lua_block { + ngx.say("Hello World") + } + } +--- request +GET /test +--- response_body +Hello World +--- no_error_log +[error] From 1637b35349be22e388b2673efad364b0c132d549 Mon Sep 17 00:00:00 2001 From: Xiaobo Liu Date: Tue, 4 Jan 2022 14:10:39 +0800 Subject: [PATCH 586/848] tests: travis.yml: ngx-releng depends on ack. (#1986) --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index f4cc579b3f..daae0704f7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -14,6 +14,7 @@ compiler: addons: apt: packages: + - ack - axel - cpanminus - libtest-base-perl From 320f030785e901e4310cb75607541240a771f5af Mon Sep 17 00:00:00 2001 From: qyihua Date: Sun, 2 Aug 2020 22:02:31 +0800 Subject: [PATCH 587/848] doc:move all the discouraged *_by_lua documentation to *_by_lua_block --- README.markdown | 561 +++++++++++++++++++++-------------------- doc/HttpLuaModule.wiki | 504 ++++++++++++++++++------------------ 2 files changed, 541 insertions(+), 524 deletions(-) diff --git a/README.markdown b/README.markdown index c3af842219..fcc0f5ee94 100644 --- a/README.markdown +++ b/README.markdown @@ -1457,9 +1457,33 @@ init_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [init_by_lua_block](#init_by_lua_block) directive instead. -Runs the Lua code specified by the argument `` on the global Lua VM level when the Nginx master process (if any) is loading the Nginx config file. +Similar to the [init_by_lua_block](#init_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + +```nginx + + init_by_lua ' + print("I need no extra escaping here, for example: \r\nblah") + ' +``` + +This directive was first introduced in the `v0.5.5` release. + +[Back to TOC](#directives) + +init_by_lua_block +----------------- + +**syntax:** *init_by_lua_block { lua-script }* -When Nginx receives the `HUP` signal and starts reloading the config file, the Lua VM will also be re-created and `init_by_lua` will run again on the new Lua VM. In case that the [lua_code_cache](#lua_code_cache) directive is turned off (default on), the `init_by_lua` handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. +**context:** *http* + +**phase:** *loading-config* + + +When Nginx receives the `HUP` signal and starts reloading the config file, the Lua VM will also be re-created and `init_by_lua_block` will run again on the new Lua VM. In case that the [lua_code_cache](#lua_code_cache) directive is turned off (default on), the `init_by_lua_block` handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating systems' copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: @@ -1500,11 +1524,11 @@ You can also initialize the [lua_shared_dict](#lua_shared_dict) shm storage at t } ``` -But note that, the [lua_shared_dict](#lua_shared_dict)'s shm storage will not be cleared through a config reload (via the `HUP` signal, for example). So if you do *not* want to re-initialize the shm storage in your `init_by_lua` code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your `init_by_lua` code. +But note that, the [lua_shared_dict](#lua_shared_dict)'s shm storage will not be cleared through a config reload (via the `HUP` signal, for example). So if you do *not* want to re-initialize the shm storage in your `init_by_lua_block` code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your `init_by_lua_block` code. Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the [Copy-on-write (COW)](https://en.wikipedia.org/wiki/Copy-on-write) feature provided by many operating systems among all the worker processes, thus saving a lot of memory. -Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [Lua Variable Scope](#lua-variable-scope) section for more details). The recommended way is to use proper [Lua module](https://www.lua.org/manual/5.1/manual.html#5.3) files (but do not use the standard Lua function [module()](https://www.lua.org/manual/5.1/manual.html#pdf-module) to define Lua modules because it pollutes the global namespace as well) and call [require()](https://www.lua.org/manual/5.1/manual.html#pdf-require) to load your own module files in `init_by_lua` or other contexts ([require()](https://www.lua.org/manual/5.1/manual.html#pdf-require) does cache the loaded Lua modules in the global `package.loaded` table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). +Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [Lua Variable Scope](#lua-variable-scope) section for more details). The recommended way is to use proper [Lua module](https://www.lua.org/manual/5.1/manual.html#5.3) files (but do not use the standard Lua function [module()](https://www.lua.org/manual/5.1/manual.html#pdf-module) to define Lua modules because it pollutes the global namespace as well) and call [require()](https://www.lua.org/manual/5.1/manual.html#pdf-require) to load your own module files in `init_by_lua_block` or other contexts ([require()](https://www.lua.org/manual/5.1/manual.html#pdf-require) does cache the loaded Lua modules in the global `package.loaded` table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). Only a small set of the [Nginx API for Lua](#nginx-api-for-lua) is supported in this context: @@ -1517,7 +1541,7 @@ Basically you can safely use Lua libraries that do blocking I/O in this very con You should be very careful about potential security vulnerabilities in your Lua code registered in this context because the Nginx master process is often run under the `root` account. -This directive was first introduced in the `v0.5.5` release. +This directive was first introduced in the `v0.9.17` release. See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: @@ -1526,33 +1550,6 @@ See also the following blog posts for more details on OpenResty and Nginx's shar [Back to TOC](#directives) -init_by_lua_block ------------------ - -**syntax:** *init_by_lua_block { lua-script }* - -**context:** *http* - -**phase:** *loading-config* - -Similar to the [init_by_lua](#init_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - -```nginx - - init_by_lua_block { - print("I need no extra escaping here, for example: \r\nblah") - } -``` - -This directive was first introduced in the `v0.9.17` release. - -[Back to TOC](#directives) - init_by_lua_file ---------------- @@ -1562,7 +1559,7 @@ init_by_lua_file **phase:** *loading-config* -Equivalent to [init_by_lua](#init_by_lua), except that the file specified by `` contains the Lua code or [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [init_by_lua_block](#init_by_lua_block), except that the file specified by `` contains the Lua code or [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -1581,13 +1578,40 @@ init_worker_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [init_worker_by_lua_block](#init_worker_by_lua_block) directive instead. -Runs the specified Lua code upon every Nginx worker process's startup when the master process is enabled. When the master process is disabled, this hook will just run after [init_by_lua*](#init_by_lua). +Similar to the [init_worker_by_lua_block](#init_worker_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). -This hook is often used to create per-worker reoccurring timers (via the [ngx.timer.at](#ngxtimerat) Lua API), either for backend health-check or other timed routine work. Below is an example, +For instance, ```nginx init_worker_by_lua ' + print("I need no extra escaping here, for example: \r\nblah") + '; +``` + +This directive was first introduced in the `v0.9.5` release. + +This hook no longer runs in the cache manager and cache loader processes since the `v0.10.12` release. + +[Back to TOC](#directives) + +init_worker_by_lua_block +------------------------ + +**syntax:** *init_worker_by_lua_block { lua-script }* + +**context:** *http* + +**phase:** *starting-worker* + +Runs the specified Lua code upon every Nginx worker process's startup when the master process is enabled. When the master process is disabled, this hook will just run after [init_by_lua*](#init_by_lua_block). + +This hook is often used to create per-worker reoccurring timers (via the [ngx.timer.at](#ngxtimerat) Lua API), either for backend health-check or other timed routine work. Below is an example, + +```nginx + + init_worker_by_lua_block { local delay = 3 -- in seconds local new_timer = ngx.timer.at local log = ngx.log @@ -1614,35 +1638,6 @@ This hook is often used to create per-worker reoccurring timers (via the [ngx.ti end -- other job in init_worker_by_lua - '; -``` - -This directive was first introduced in the `v0.9.5` release. - -This hook no longer runs in the cache manager and cache loader processes since the `v0.10.12` release. - -[Back to TOC](#directives) - -init_worker_by_lua_block ------------------------- - -**syntax:** *init_worker_by_lua_block { lua-script }* - -**context:** *http* - -**phase:** *starting-worker* - -Similar to the [init_worker_by_lua](#init_worker_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - -```nginx - - init_worker_by_lua_block { - print("I need no extra escaping here, for example: \r\nblah") } ``` @@ -1661,7 +1656,7 @@ init_worker_by_lua_file **phase:** *starting-worker* -Similar to [init_worker_by_lua](#init_worker_by_lua), but accepts the file path to a Lua source file or Lua bytecode file. +Similar to [init_worker_by_lua_block](#init_worker_by_lua_block), but accepts the file path to a Lua source file or Lua bytecode file. This directive was first introduced in the `v0.9.5` release. @@ -1680,7 +1675,7 @@ exit_worker_by_lua_block Runs the specified Lua code upon every Nginx worker process's exit when the master process is enabled. When the master process is disabled, this hook will run before the Nginx process exits. -This hook is often used to release resources allocated by each worker (e.g. resources allocated by [init_worker_by_lua*](#init_worker_by_lua)), or to prevent workers from exiting abnormally. +This hook is often used to release resources allocated by each worker (e.g. resources allocated by [init_worker_by_lua*](#init_worker_by_lua_block)), or to prevent workers from exiting abnormally. For example, @@ -1723,14 +1718,41 @@ set_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [set_by_lua_block](#set_by_lua_block) directive instead. -Executes code specified in `` with optional input arguments `$arg1 $arg2 ...`, and returns string output to `$res`. -The code in `` can make [API calls](#nginx-api-for-lua) and can retrieve input arguments from the `ngx.arg` table (index starts from `1` and increases sequentially). +Similar to the [set_by_lua_block](#set_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping), and +1. this directive support extra arguments after the Lua script. + +For example, + +```nginx + + set_by_lua $res ' return 32 + math.cos(32) '; + # $res now has the value "32.834223360507" or alike. +``` + +As from the `v0.5.0rc29` release, Nginx variable interpolation is disabled in the `` argument of this directive and therefore, the dollar sign character (`$`) can be used directly. + +This directive requires the [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) module. + +[Back to TOC](#directives) + +set_by_lua_block +---------------- + +**syntax:** *set_by_lua_block $res { lua-script }* + +**context:** *server, server if, location, location if* + +**phase:** *rewrite* + +Executes code specified inside a pair of curly braces (`{}`), and returns string output to `$res`. +The code inside a pair of curly braces (`{}`) can make [API calls](#nginx-api-for-lua) and can retrieve input arguments from the `ngx.arg` table (index starts from `1` and increases sequentially). This directive is designed to execute short, fast running code blocks as the Nginx event loop is blocked during code execution. Time consuming code sequences should therefore be avoided. This directive is implemented by injecting custom commands into the standard [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s command list. Because [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html) does not support nonblocking I/O in its commands, Lua APIs requiring yielding the current Lua "light thread" cannot work in this directive. -At least the following API functions are currently disabled within the context of `set_by_lua`: +At least the following API functions are currently disabled within the context of `set_by_lua_block`: * Output API functions (e.g., [ngx.say](#ngxsay) and [ngx.send_headers](#ngxsend_headers)) * Control API functions (e.g., [ngx.exit](#ngxexit)) @@ -1746,13 +1768,13 @@ a time. However, a workaround is possible using the [ngx.var.VARIABLE](#ngxvarva location /foo { set $diff ''; # we have to predefine the $diff variable here - set_by_lua $sum ' + set_by_lua_block $sum { local a = 32 local b = 56 ngx.var.diff = a - b -- write to $diff directly return a + b -- return the $sum value normally - '; + } echo "sum = $sum, diff = $diff"; } @@ -1763,42 +1785,14 @@ This directive can be freely mixed with all directives of the [ngx_http_rewrite_ ```nginx set $foo 32; - set_by_lua $bar 'return tonumber(ngx.var.foo) + 1'; + set_by_lua_block $bar { return tonumber(ngx.var.foo) + 1 } set $baz "bar: $bar"; # $baz == "bar: 33" ``` -As from the `v0.5.0rc29` release, Nginx variable interpolation is disabled in the `` argument of this directive and therefore, the dollar sign character (`$`) can be used directly. +No special escaping is required in the Lua code block. This directive requires the [ngx_devel_kit](https://github.com/simplresty/ngx_devel_kit) module. -[Back to TOC](#directives) - -set_by_lua_block ----------------- - -**syntax:** *set_by_lua_block $res { lua-script }* - -**context:** *server, server if, location, location if* - -**phase:** *rewrite* - -Similar to the [set_by_lua](#set_by_lua) directive except that - -1. this directive inlines the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping), and -1. this directive does not support extra arguments after the Lua script as in [set_by_lua](#set_by_lua). - -For example, - -```nginx - - set_by_lua_block $res { return 32 + math.cos(32) } - # $res now has the value "32.834223360507" or alike. -``` - -No special escaping is required in the Lua code block. - This directive was first introduced in the `v0.9.17` release. [Back to TOC](#directives) @@ -1812,7 +1806,7 @@ set_by_lua_file **phase:** *rewrite* -Equivalent to [set_by_lua](#set_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [set_by_lua_block](#set_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variable interpolation is supported in the `` argument string of this directive. But special care must be taken for injection attacks. @@ -1838,10 +1832,17 @@ content_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [content_by_lua_block](#content_by_lua_block) directive instead. -Acts as a "content handler" and executes Lua code string specified in `` for every request. -The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). +Similar to the [content_by_lua_block](#content_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). -Do not use this directive and other content handler directives in the same location. For example, this directive and the [proxy_pass](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive should not be used in the same location. +For instance, + +```nginx + + content_by_lua ' + ngx.say("I need no extra escaping here, for example: \r\nblah") + '; +``` [Back to TOC](#directives) @@ -1854,11 +1855,6 @@ content_by_lua_block **phase:** *content* -Similar to the [content_by_lua](#content_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - For instance, ```nginx @@ -1868,6 +1864,11 @@ For instance, } ``` +Acts as a "content handler" and executes Lua code string specified in `{ lua-script }` for every request. +The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). + +Do not use this directive and other content handler directives in the same location. For example, this directive and the [proxy_pass](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive should not be used in the same location. + This directive was first introduced in the `v0.9.17` release. [Back to TOC](#directives) @@ -1881,7 +1882,7 @@ content_by_lua_file **phase:** *content* -Equivalent to [content_by_lua](#content_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [content_by_lua_block](#content_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1919,7 +1920,30 @@ rewrite_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [rewrite_by_lua_block](#rewrite_by_lua_block) directive instead. -Acts as a rewrite phase handler and executes Lua code string specified in `` for every request. +Similar to the [rewrite_by_lua_block](#rewrite_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + +```nginx + + rewrite_by_lua ' + do_something("hello, world!\nhiya\n") + '; +``` + +[Back to TOC](#directives) + +rewrite_by_lua_block +-------------------- + +**syntax:** *rewrite_by_lua_block { lua-script }* + +**context:** *http, server, location, location if* + +**phase:** *rewrite tail* + +Acts as a rewrite phase handler and executes Lua code string specified in `{ lua-script }` for every request. The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). Note that this handler always runs *after* the standard [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html). So the following will work as expected: @@ -1929,12 +1953,14 @@ Note that this handler always runs *after* the standard [ngx_http_rewrite_module location /foo { set $a 12; # create and initialize $a set $b ""; # create and initialize $b - rewrite_by_lua 'ngx.var.b = tonumber(ngx.var.a) + 1'; + rewrite_by_lua_block { + ngx.var.b = tonumber(ngx.var.a) + 1 + } echo "res = $b"; } ``` -because `set $a 12` and `set $b ""` run *before* [rewrite_by_lua](#rewrite_by_lua). +because `set $a 12` and `set $b ""` run *before* [rewrite_by_lua_block](#rewrite_by_lua_block). On the other hand, the following will not work as expected: @@ -1943,7 +1969,9 @@ On the other hand, the following will not work as expected: ? location /foo { ? set $a 12; # create and initialize $a ? set $b ''; # create and initialize $b - ? rewrite_by_lua 'ngx.var.b = tonumber(ngx.var.a) + 1'; + ? rewrite_by_lua_block { + ? ngx.var.b = tonumber(ngx.var.a) + 1 + ? } ? if ($b = '13') { ? rewrite ^ /bar redirect; ? break; @@ -1953,7 +1981,7 @@ On the other hand, the following will not work as expected: ? } ``` -because `if` runs *before* [rewrite_by_lua](#rewrite_by_lua) even if it is placed after [rewrite_by_lua](#rewrite_by_lua) in the config. +because `if` runs *before* [rewrite_by_lua_block](#rewrite_by_lua_block) even if it is placed after [rewrite_by_lua_block](#rewrite_by_lua_block) in the config. The right way of doing this is as follows: @@ -1962,18 +1990,18 @@ The right way of doing this is as follows: location /foo { set $a 12; # create and initialize $a set $b ''; # create and initialize $b - rewrite_by_lua ' + rewrite_by_lua_block { ngx.var.b = tonumber(ngx.var.a) + 1 if tonumber(ngx.var.b) == 13 then return ngx.redirect("/bar") end - '; + } echo "res = $b"; } ``` -Note that the [ngx_eval](http://www.grid.net.ru/nginx/eval.en.html) module can be approximated by using [rewrite_by_lua](#rewrite_by_lua). For example, +Note that the [ngx_eval](http://www.grid.net.ru/nginx/eval.en.html) module can be approximated by using [rewrite_by_lua_block](#rewrite_by_lua_block). For example, ```nginx @@ -2000,62 +2028,39 @@ can be implemented in ngx_lua as: } location / { - rewrite_by_lua ' + rewrite_by_lua_block { local res = ngx.location.capture("/check-spam") if res.body == "spam" then return ngx.redirect("/terms-of-use.html") end - '; + } fastcgi_pass ...; } ``` -Just as any other rewrite phase handlers, [rewrite_by_lua](#rewrite_by_lua) also runs in subrequests. +Just as any other rewrite phase handlers, [rewrite_by_lua_block](#rewrite_by_lua_block) also runs in subrequests. -Note that when calling `ngx.exit(ngx.OK)` within a [rewrite_by_lua](#rewrite_by_lua) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [rewrite_by_lua](#rewrite_by_lua) handler, call [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. +Note that when calling `ngx.exit(ngx.OK)` within a [rewrite_by_lua_block](#rewrite_by_lua_block) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [rewrite_by_lua_block](#rewrite_by_lua_block) handler, call [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. -If the [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s [rewrite](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) directive is used to change the URI and initiate location re-lookups (internal redirections), then any [rewrite_by_lua](#rewrite_by_lua) or [rewrite_by_lua_file](#rewrite_by_lua_file) code sequences within the current location will not be executed. For example, +If the [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s [rewrite](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) directive is used to change the URI and initiate location re-lookups (internal redirections), then any [rewrite_by_lua_block](#rewrite_by_lua_block) or [rewrite_by_lua_file_block](#rewrite_by_lua_file_block) code sequences within the current location will not be executed. For example, ```nginx location /foo { rewrite ^ /bar; - rewrite_by_lua 'ngx.exit(503)'; + rewrite_by_lua_block { + ngx.exit(503) + } } location /bar { ... } ``` -Here the Lua code `ngx.exit(503)` will never run. This will be the case if `rewrite ^ /bar last` is used as this will similarly initiate an internal redirection. If the `break` modifier is used instead, there will be no internal redirection and the `rewrite_by_lua` code will be executed. +Here the Lua code `ngx.exit(503)` will never run. This will be the case if `rewrite ^ /bar last` is used as this will similarly initiate an internal redirection. If the `break` modifier is used instead, there will be no internal redirection and the `rewrite_by_lua_block` code will be executed. -The `rewrite_by_lua` code will always run at the end of the `rewrite` request-processing phase unless [rewrite_by_lua_no_postpone](#rewrite_by_lua_no_postpone) is turned on. - -[Back to TOC](#directives) - -rewrite_by_lua_block --------------------- - -**syntax:** *rewrite_by_lua_block { lua-script }* - -**context:** *http, server, location, location if* - -**phase:** *rewrite tail* - -Similar to the [rewrite_by_lua](#rewrite_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - -```nginx - - rewrite_by_lua_block { - do_something("hello, world!\nhiya\n") - } -``` +The `rewrite_by_lua_block` code will always run at the end of the `rewrite` request-processing phase unless [rewrite_by_lua_no_postpone](#rewrite_by_lua_no_postpone) is turned on. This directive was first introduced in the `v0.9.17` release. @@ -2070,7 +2075,7 @@ rewrite_by_lua_file **phase:** *rewrite tail* -Equivalent to [rewrite_by_lua](#rewrite_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [rewrite_by_lua_block](#rewrite_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -2095,7 +2100,30 @@ access_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [access_by_lua_block](#access_by_lua_block) directive instead. -Acts as an access phase handler and executes Lua code string specified in `` for every request. +Similar to the [access_by_lua_block](#access_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + +```nginx + + access_by_lua ' + do_something("hello, world!\nhiya\n") + '; +``` + +[Back to TOC](#directives) + +access_by_lua_block +------------------- + +**syntax:** *access_by_lua_block { lua-script }* + +**context:** *http, server, location, location if* + +**phase:** *access tail* + +Acts as an access phase handler and executes Lua code string specified in `{ = 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. +Note that when calling `ngx.exit(ngx.OK)` within a [access_by_lua_block](#access_by_lua_block) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [access_by_lua_block](#access_by_lua_block) handler, call [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. Starting from the `v0.9.20` release, you can use the [access_by_lua_no_postpone](#access_by_lua_no_postpone) directive to control when to run this handler inside the "access" request-processing phase of Nginx. -[Back to TOC](#directives) - -access_by_lua_block -------------------- - -**syntax:** *access_by_lua_block { lua-script }* - -**context:** *http, server, location, location if* - -**phase:** *access tail* - -Similar to the [access_by_lua](#access_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - -```nginx - - access_by_lua_block { - do_something("hello, world!\nhiya\n") - } -``` - This directive was first introduced in the `v0.9.17` release. [Back to TOC](#directives) @@ -2199,7 +2202,7 @@ access_by_lua_file **phase:** *access tail* -Equivalent to [access_by_lua](#access_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [access_by_lua_block](#access_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -2224,23 +2227,16 @@ header_filter_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [header_filter_by_lua_block](#header_filter_by_lua_block) directive instead. -Uses Lua code specified in `` to define an output header filter. - -Note that the following API functions are currently disabled within this context: - -* Output API functions (e.g., [ngx.say](#ngxsay) and [ngx.send_headers](#ngxsend_headers)) -* Control API functions (e.g., [ngx.redirect](#ngxredirect) and [ngx.exec](#ngxexec)) -* Subrequest API functions (e.g., [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi)) -* Cosocket API functions (e.g., [ngx.socket.tcp](#ngxsockettcp) and [ngx.req.socket](#ngxreqsocket)). +Similar to the [header_filter_by_lua_block](#header_filter_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). -Here is an example of overriding a response header (or adding one if absent) in our Lua header filter: +For instance, ```nginx - location / { - proxy_pass http://mybackend; - header_filter_by_lua 'ngx.header.Foo = "blah"'; - } + header_filter_by_lua ' + ngx.header["content-length"] = nil + '; ``` This directive was first introduced in the `v0.2.1rc20` release. @@ -2256,17 +2252,24 @@ header_filter_by_lua_block **phase:** *output-header-filter* -Similar to the [header_filter_by_lua](#header_filter_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). +Uses Lua code specified in `{ lua-script }` to define an output header filter. -For instance, +Note that the following API functions are currently disabled within this context: + +* Output API functions (e.g., [ngx.say](#ngxsay) and [ngx.send_headers](#ngxsend_headers)) +* Control API functions (e.g., [ngx.redirect](#ngxredirect) and [ngx.exec](#ngxexec)) +* Subrequest API functions (e.g., [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi)) +* Cosocket API functions (e.g., [ngx.socket.tcp](#ngxsockettcp) and [ngx.req.socket](#ngxreqsocket)). + +Here is an example of overriding a response header (or adding one if absent) in our Lua header filter: ```nginx - header_filter_by_lua_block { - ngx.header["content-length"] = nil + location / { + proxy_pass http://mybackend; + header_filter_by_lua_block { + ngx.header.Foo = "blah" + } } ``` @@ -2283,7 +2286,7 @@ header_filter_by_lua_file **phase:** *output-header-filter* -Equivalent to [header_filter_by_lua](#header_filter_by_lua), except that the file specified by `` contains the Lua code, or as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [header_filter_by_lua_block](#header_filter_by_lua_block), except that the file specified by `` contains the Lua code, or as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2302,7 +2305,32 @@ body_filter_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [body_filter_by_lua_block](#body_filter_by_lua_block) directive instead. -Uses Lua code specified in `` to define an output body filter. +Similar to the [body_filter_by_lua_block](#body_filter_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + +```nginx + + body_filter_by_lua ' + local data, eof = ngx.arg[1], ngx.arg[2] + '; +``` + +This directive was first introduced in the `v0.5.0rc32` release. + +[Back to TOC](#directives) + +body_filter_by_lua_block +------------------------ + +**syntax:** *body_filter_by_lua_block { lua-script-str }* + +**context:** *http, server, location, location if* + +**phase:** *output-body-filter* + +Uses Lua code specified in `{ lua-script }` to define an output body filter. The input data chunk is passed via [ngx.arg](#ngxarg)\[1\] (as a Lua string value) and the "eof" flag indicating the end of the response body data stream is passed via [ngx.arg](#ngxarg)\[2\] (as a Lua boolean value). @@ -2323,7 +2351,9 @@ The Lua code can pass its own modified version of the input data chunk to the do location / { proxy_pass http://mybackend; - body_filter_by_lua 'ngx.arg[1] = string.upper(ngx.arg[1])'; + body_filter_by_lua_block { + ngx.arg[1] = string.upper(ngx.arg[1]) + } } ``` @@ -2337,7 +2367,7 @@ Likewise, new "eof" flag can also be specified by setting a boolean value to [ng echo hello world; echo hiya globe; - body_filter_by_lua ' + body_filter_by_lua_block { local chunk = ngx.arg[1] if string.match(chunk, "hello") then ngx.arg[2] = true -- new eof @@ -2346,7 +2376,7 @@ Likewise, new "eof" flag can also be specified by setting a boolean value to [ng -- just throw away any remaining chunk data ngx.arg[1] = nil - '; + } } ``` @@ -2365,8 +2395,12 @@ When the Lua code may change the length of the response body, then it is require location /foo { # fastcgi_pass/proxy_pass/... - header_filter_by_lua_block { ngx.header.content_length = nil } - body_filter_by_lua 'ngx.arg[1] = string.len(ngx.arg[1]) .. "\\n"'; + header_filter_by_lua_block { + ngx.header.content_length = nil + } + body_filter_by_lua_block { + ngx.arg[1] = string.len(ngx.arg[1]) .. "\n" + } } ``` @@ -2379,33 +2413,6 @@ Note that the following API functions are currently disabled within this context Nginx output filters may be called multiple times for a single request because response body may be delivered in chunks. Thus, the Lua code specified by in this directive may also run multiple times in the lifetime of a single HTTP request. -This directive was first introduced in the `v0.5.0rc32` release. - -[Back to TOC](#directives) - -body_filter_by_lua_block ------------------------- - -**syntax:** *body_filter_by_lua_block { lua-script-str }* - -**context:** *http, server, location, location if* - -**phase:** *output-body-filter* - -Similar to the [body_filter_by_lua](#body_filter_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - -```nginx - - body_filter_by_lua_block { - local data, eof = ngx.arg[1], ngx.arg[2] - } -``` - This directive was first introduced in the `v0.9.17` release. [Back to TOC](#directives) @@ -2419,7 +2426,7 @@ body_filter_by_lua_file **phase:** *output-body-filter* -Equivalent to [body_filter_by_lua](#body_filter_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [body_filter_by_lua_block](#body_filter_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -2438,7 +2445,32 @@ log_by_lua **NOTE** Use of this directive is *discouraged* following the `v0.9.17` release. Use the [log_by_lua_block](#log_by_lua_block) directive instead. -Runs the Lua source code inlined as the `` at the `log` request processing phase. This does not replace the current access logs, but runs before. +Similar to the [log_by_lua_block](#log_by_lua_block) directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + +```nginx + + log_by_lua ' + print("I need no extra escaping here, for example: \r\nblah") + '; +``` + +This directive was first introduced in the `v0.5.0rc31` release. + +[Back to TOC](#directives) + +log_by_lua_block +---------------- + +**syntax:** *log_by_lua_block { lua-script }* + +**context:** *http, server, location, location if* + +**phase:** *log* + +Runs the Lua source code inlined as the `{ lua-script }` at the `log` request processing phase. This does not replace the current access logs, but runs before. Note that the following API functions are currently disabled within this context: @@ -2457,7 +2489,7 @@ Here is an example of gathering average data for [$upstream_response_time](http: location / { proxy_pass http://mybackend; - log_by_lua ' + log_by_lua_block { local log_dict = ngx.shared.log_dict local upstream_time = tonumber(ngx.var.upstream_response_time) @@ -2470,7 +2502,7 @@ Here is an example of gathering average data for [$upstream_response_time](http: log_dict:add("upstream_time-nb", 0) log_dict:incr("upstream_time-nb", 1) end - '; + } } location = /status { @@ -2490,33 +2522,6 @@ Here is an example of gathering average data for [$upstream_response_time](http: } ``` -This directive was first introduced in the `v0.5.0rc31` release. - -[Back to TOC](#directives) - -log_by_lua_block ----------------- - -**syntax:** *log_by_lua_block { lua-script }* - -**context:** *http, server, location, location if* - -**phase:** *log* - -Similar to the [log_by_lua](#log_by_lua) directive except that this directive inlines -the Lua source directly -inside a pair of curly braces (`{}`) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - -```nginx - - log_by_lua_block { - print("I need no extra escaping here, for example: \r\nblah") - } -``` - This directive was first introduced in the `v0.9.17` release. [Back to TOC](#directives) @@ -2530,7 +2535,7 @@ log_by_lua_file **phase:** *log* -Equivalent to [log_by_lua](#log_by_lua), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +Equivalent to [log_by_lua_block](#log_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. @@ -4419,7 +4424,9 @@ Lua tables can be used for both requests and responses when the number of subreq table.insert(reqs, { "/memcached" }) -- issue all the requests at once and wait until they all return - local resps = { ngx.location.capture_multi(reqs) } + local resps = { + ngx.location.capture_multi(reqs) + } -- loop over the responses table for i, resp in ipairs(resps) do @@ -6159,7 +6166,9 @@ For example, ```nginx location = /md5 { - content_by_lua_block { ngx.say(ngx.md5("hello")) } + content_by_lua_block { + ngx.say(ngx.md5("hello")) + } } ``` diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index ad00ba2fa5..9bfbc7a633 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1168,9 +1168,29 @@ As from the v0.5.0rc29 release, the special notation $prefix< '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#init_by_lua_block|init_by_lua_block]] directive instead. -Runs the Lua code specified by the argument on the global Lua VM level when the Nginx master process (if any) is loading the Nginx config file. +Similar to the [[#init_by_lua_block|init_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + + + init_by_lua ' + print("I need no extra escaping here, for example: \r\nblah") + ' + + +This directive was first introduced in the v0.5.5 release. + +== init_by_lua_block == + +'''syntax:''' ''init_by_lua_block { lua-script }'' + +'''context:''' ''http'' + +'''phase:''' ''loading-config'' -When Nginx receives the HUP signal and starts reloading the config file, the Lua VM will also be re-created and init_by_lua will run again on the new Lua VM. In case that the [[#lua_code_cache|lua_code_cache]] directive is turned off (default on), the init_by_lua handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. + +When Nginx receives the HUP signal and starts reloading the config file, the Lua VM will also be re-created and init_by_lua_block will run again on the new Lua VM. In case that the [[#lua_code_cache|lua_code_cache]] directive is turned off (default on), the init_by_lua_block handler will run upon every request because in this special mode a standalone Lua VM is always created for each request. Usually you can pre-load Lua modules at server start-up by means of this hook and take advantage of modern operating systems' copy-on-write (COW) optimization. Here is an example for pre-loading Lua modules: @@ -1209,11 +1229,11 @@ You can also initialize the [[#lua_shared_dict|lua_shared_dict]] shm storage at } -But note that, the [[#lua_shared_dict|lua_shared_dict]]'s shm storage will not be cleared through a config reload (via the HUP signal, for example). So if you do ''not'' want to re-initialize the shm storage in your init_by_lua code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your init_by_lua code. +But note that, the [[#lua_shared_dict|lua_shared_dict]]'s shm storage will not be cleared through a config reload (via the HUP signal, for example). So if you do ''not'' want to re-initialize the shm storage in your init_by_lua_block code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your init_by_lua_block code. Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the [https://en.wikipedia.org/wiki/Copy-on-write Copy-on-write (COW)] feature provided by many operating systems among all the worker processes, thus saving a lot of memory. -Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [[#Lua_Variable_Scope|Lua Variable Scope]] section for more details). The recommended way is to use proper [https://www.lua.org/manual/5.1/manual.html#5.3 Lua module] files (but do not use the standard Lua function [https://www.lua.org/manual/5.1/manual.html#pdf-module module()] to define Lua modules because it pollutes the global namespace as well) and call [https://www.lua.org/manual/5.1/manual.html#pdf-require require()] to load your own module files in init_by_lua or other contexts ([https://www.lua.org/manual/5.1/manual.html#pdf-require require()] does cache the loaded Lua modules in the global package.loaded table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). +Do *not* initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the [[#Lua_Variable_Scope|Lua Variable Scope]] section for more details). The recommended way is to use proper [https://www.lua.org/manual/5.1/manual.html#5.3 Lua module] files (but do not use the standard Lua function [https://www.lua.org/manual/5.1/manual.html#pdf-module module()] to define Lua modules because it pollutes the global namespace as well) and call [https://www.lua.org/manual/5.1/manual.html#pdf-require require()] to load your own module files in init_by_lua_block or other contexts ([https://www.lua.org/manual/5.1/manual.html#pdf-require require()] does cache the loaded Lua modules in the global package.loaded table in the Lua registry so your modules will only loaded once for the whole Lua VM instance). Only a small set of the [[#Nginx API for Lua|Nginx API for Lua]] is supported in this context: @@ -1226,36 +1246,13 @@ Basically you can safely use Lua libraries that do blocking I/O in this very con You should be very careful about potential security vulnerabilities in your Lua code registered in this context because the Nginx master process is often run under the root account. -This directive was first introduced in the v0.5.5 release. +This directive was first introduced in the v0.9.17 release. See also the following blog posts for more details on OpenResty and Nginx's shared memory zones: * [How OpenResty and Nginx Shared Memory Zones Consume RAM](https://blog.openresty.com/en/how-nginx-shm-consume-ram/?src=gh_ngxlua) * [Memory Fragmentation in OpenResty and Nginx's Shared Memory Zones](https://blog.openresty.com/en/nginx-shm-frag/?src=gh_ngxlua) -== init_by_lua_block == - -'''syntax:''' ''init_by_lua_block { lua-script }'' - -'''context:''' ''http'' - -'''phase:''' ''loading-config'' - -Similar to the [[#init_by_lua|init_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - - - init_by_lua_block { - print("I need no extra escaping here, for example: \r\nblah") - } - - -This directive was first introduced in the v0.9.17 release. - == init_by_lua_file == '''syntax:''' ''init_by_lua_file '' @@ -1264,7 +1261,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''loading-config'' -Equivalent to [[#init_by_lua|init_by_lua]], except that the file specified by contains the Lua code or [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#init_by_lua_block|init_by_lua_block]], except that the file specified by contains the Lua code or [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -1280,12 +1277,35 @@ This directive was first introduced in the v0.5.5 release. '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#init_worker_by_lua_block|init_worker_by_lua_block]] directive instead. -Runs the specified Lua code upon every Nginx worker process's startup when the master process is enabled. When the master process is disabled, this hook will just run after [[#init_by_lua|init_by_lua*]]. +Similar to the [[#init_worker_by_lua_block|init_worker_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). -This hook is often used to create per-worker reoccurring timers (via the [[#ngx.timer.at|ngx.timer.at]] Lua API), either for backend health-check or other timed routine work. Below is an example, +For instance, init_worker_by_lua ' + print("I need no extra escaping here, for example: \r\nblah") + '; + + +This directive was first introduced in the v0.9.5 release. + +This hook no longer runs in the cache manager and cache loader processes since the v0.10.12 release. + +== init_worker_by_lua_block == + +'''syntax:''' ''init_worker_by_lua_block { lua-script }'' + +'''context:''' ''http'' + +'''phase:''' ''starting-worker'' + +Runs the specified Lua code upon every Nginx worker process's startup when the master process is enabled. When the master process is disabled, this hook will just run after [[#init_by_lua_block|init_by_lua*]]. + +This hook is often used to create per-worker reoccurring timers (via the [[#ngx.timer.at|ngx.timer.at]] Lua API), either for backend health-check or other timed routine work. Below is an example, + + + init_worker_by_lua_block { local delay = 3 -- in seconds local new_timer = ngx.timer.at local log = ngx.log @@ -1312,31 +1332,6 @@ This hook is often used to create per-worker reoccurring timers (via the [[#ngx. end -- other job in init_worker_by_lua - '; - - -This directive was first introduced in the v0.9.5 release. - -This hook no longer runs in the cache manager and cache loader processes since the v0.10.12 release. - -== init_worker_by_lua_block == - -'''syntax:''' ''init_worker_by_lua_block { lua-script }'' - -'''context:''' ''http'' - -'''phase:''' ''starting-worker'' - -Similar to the [[#init_worker_by_lua|init_worker_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - - - init_worker_by_lua_block { - print("I need no extra escaping here, for example: \r\nblah") } @@ -1352,7 +1347,7 @@ This hook no longer runs in the cache manager and cache loader processes since t '''phase:''' ''starting-worker'' -Similar to [[#init_worker_by_lua|init_worker_by_lua]], but accepts the file path to a Lua source file or Lua bytecode file. +Similar to [[#init_worker_by_lua_block|init_worker_by_lua_block]], but accepts the file path to a Lua source file or Lua bytecode file. This directive was first introduced in the v0.9.5 release. @@ -1368,7 +1363,7 @@ This hook no longer runs in the cache manager and cache loader processes since t Runs the specified Lua code upon every Nginx worker process's exit when the master process is enabled. When the master process is disabled, this hook will run before the Nginx process exits. -This hook is often used to release resources allocated by each worker (e.g. resources allocated by [[#init_worker_by_lua|init_worker_by_lua*]]), or to prevent workers from exiting abnormally. +This hook is often used to release resources allocated by each worker (e.g. resources allocated by [[#init_worker_by_lua_block|init_worker_by_lua*]]), or to prevent workers from exiting abnormally. For example, @@ -1404,14 +1399,37 @@ This directive was first introduced in the v0.10.18 release. '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#set_by_lua_block|set_by_lua_block]] directive instead. -Executes code specified in with optional input arguments $arg1 $arg2 ..., and returns string output to $res. -The code in can make [[#Nginx API for Lua|API calls]] and can retrieve input arguments from the ngx.arg table (index starts from 1 and increases sequentially). +Similar to the [[#set_by_lua_block|set_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping), and +# this directive support extra arguments after the Lua script. + +For example, + + + set_by_lua $res ' return 32 + math.cos(32) '; + # $res now has the value "32.834223360507" or alike. + + +As from the v0.5.0rc29 release, Nginx variable interpolation is disabled in the argument of this directive and therefore, the dollar sign character ($) can be used directly. + +This directive requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] module. + +== set_by_lua_block == + +'''syntax:''' ''set_by_lua_block $res { lua-script }'' + +'''context:''' ''server, server if, location, location if'' + +'''phase:''' ''rewrite'' + +Executes code specified inside a pair of curly braces ({}), and returns string output to $res. +The code inside a pair of curly braces ({}) can make [[#Nginx API for Lua|API calls]] and can retrieve input arguments from the ngx.arg table (index starts from 1 and increases sequentially). This directive is designed to execute short, fast running code blocks as the Nginx event loop is blocked during code execution. Time consuming code sequences should therefore be avoided. This directive is implemented by injecting custom commands into the standard [[HttpRewriteModule]]'s command list. Because [[HttpRewriteModule]] does not support nonblocking I/O in its commands, Lua APIs requiring yielding the current Lua "light thread" cannot work in this directive. -At least the following API functions are currently disabled within the context of set_by_lua: +At least the following API functions are currently disabled within the context of set_by_lua_block: * Output API functions (e.g., [[#ngx.say|ngx.say]] and [[#ngx.send_headers|ngx.send_headers]]) * Control API functions (e.g., [[#ngx.exit|ngx.exit]]) @@ -1426,13 +1444,13 @@ a time. However, a workaround is possible using the [[#ngx.var.VARIABLE|ngx.var. location /foo { set $diff ''; # we have to predefine the $diff variable here - set_by_lua $sum ' + set_by_lua_block $sum { local a = 32 local b = 56 ngx.var.diff = a - b -- write to $diff directly return a + b -- return the $sum value normally - '; + } echo "sum = $sum, diff = $diff"; } @@ -1442,38 +1460,14 @@ This directive can be freely mixed with all directives of the [[HttpRewriteModul set $foo 32; - set_by_lua $bar 'return tonumber(ngx.var.foo) + 1'; + set_by_lua_block $bar { return tonumber(ngx.var.foo) + 1 } set $baz "bar: $bar"; # $baz == "bar: 33" -As from the v0.5.0rc29 release, Nginx variable interpolation is disabled in the argument of this directive and therefore, the dollar sign character ($) can be used directly. +No special escaping is required in the Lua code block. This directive requires the [https://github.com/simplresty/ngx_devel_kit ngx_devel_kit] module. -== set_by_lua_block == - -'''syntax:''' ''set_by_lua_block $res { lua-script }'' - -'''context:''' ''server, server if, location, location if'' - -'''phase:''' ''rewrite'' - -Similar to the [[#set_by_lua|set_by_lua]] directive except that - -# this directive inlines the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping), and -# this directive does not support extra arguments after the Lua script as in [[#set_by_lua|set_by_lua]]. - -For example, - - - set_by_lua_block $res { return 32 + math.cos(32) } - # $res now has the value "32.834223360507" or alike. - - -No special escaping is required in the Lua code block. - This directive was first introduced in the v0.9.17 release. == set_by_lua_file == @@ -1484,7 +1478,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''rewrite'' -Equivalent to [[#set_by_lua|set_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#set_by_lua_block|set_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variable interpolation is supported in the argument string of this directive. But special care must be taken for injection attacks. @@ -1507,10 +1501,16 @@ This directive requires the [https://github.com/simplresty/ngx_devel_kit ngx_dev '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#content_by_lua_block|content_by_lua_block]] directive instead. -Acts as a "content handler" and executes Lua code string specified in for every request. -The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). +Similar to the [[#content_by_lua_block|content_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). -Do not use this directive and other content handler directives in the same location. For example, this directive and the [[HttpProxyModule#proxy_pass|proxy_pass]] directive should not be used in the same location. +For instance, + + + content_by_lua ' + ngx.say("I need no extra escaping here, for example: \r\nblah") + '; + == content_by_lua_block == @@ -1520,11 +1520,6 @@ Do not use this directive and other content handler directives in the same locat '''phase:''' ''content'' -Similar to the [[#content_by_lua|content_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - For instance, @@ -1533,6 +1528,11 @@ For instance, } +Acts as a "content handler" and executes Lua code string specified in { lua-script } for every request. +The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). + +Do not use this directive and other content handler directives in the same location. For example, this directive and the [[HttpProxyModule#proxy_pass|proxy_pass]] directive should not be used in the same location. + This directive was first introduced in the v0.9.17 release. == content_by_lua_file == @@ -1543,7 +1543,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''content'' -Equivalent to [[#content_by_lua|content_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#content_by_lua_block|content_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1577,7 +1577,26 @@ But be very careful about malicious user inputs and always carefully validate or '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#rewrite_by_lua_block|rewrite_by_lua_block]] directive instead. -Acts as a rewrite phase handler and executes Lua code string specified in for every request. +Similar to the [[#rewrite_by_lua_block|rewrite_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + + + rewrite_by_lua ' + do_something("hello, world!\nhiya\n") + '; + + +== rewrite_by_lua_block == + +'''syntax:''' ''rewrite_by_lua_block { lua-script }'' + +'''context:''' ''http, server, location, location if'' + +'''phase:''' ''rewrite tail'' + +Acts as a rewrite phase handler and executes Lua code string specified in { lua-script } for every request. The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). Note that this handler always runs ''after'' the standard [[HttpRewriteModule]]. So the following will work as expected: @@ -1586,12 +1605,14 @@ Note that this handler always runs ''after'' the standard [[HttpRewriteModule]]. location /foo { set $a 12; # create and initialize $a set $b ""; # create and initialize $b - rewrite_by_lua 'ngx.var.b = tonumber(ngx.var.a) + 1'; + rewrite_by_lua_block { + ngx.var.b = tonumber(ngx.var.a) + 1 + } echo "res = $b"; } -because set $a 12 and set $b "" run ''before'' [[#rewrite_by_lua|rewrite_by_lua]]. +because set $a 12 and set $b "" run ''before'' [[#rewrite_by_lua_block|rewrite_by_lua_block]]. On the other hand, the following will not work as expected: @@ -1599,7 +1620,9 @@ On the other hand, the following will not work as expected: ? location /foo { ? set $a 12; # create and initialize $a ? set $b ''; # create and initialize $b - ? rewrite_by_lua 'ngx.var.b = tonumber(ngx.var.a) + 1'; + ? rewrite_by_lua_block { + ? ngx.var.b = tonumber(ngx.var.a) + 1 + ? } ? if ($b = '13') { ? rewrite ^ /bar redirect; ? break; @@ -1609,7 +1632,7 @@ On the other hand, the following will not work as expected: ? } -because if runs ''before'' [[#rewrite_by_lua|rewrite_by_lua]] even if it is placed after [[#rewrite_by_lua|rewrite_by_lua]] in the config. +because if runs ''before'' [[#rewrite_by_lua_block|rewrite_by_lua_block]] even if it is placed after [[#rewrite_by_lua_block|rewrite_by_lua_block]] in the config. The right way of doing this is as follows: @@ -1617,18 +1640,18 @@ The right way of doing this is as follows: location /foo { set $a 12; # create and initialize $a set $b ''; # create and initialize $b - rewrite_by_lua ' + rewrite_by_lua_block { ngx.var.b = tonumber(ngx.var.a) + 1 if tonumber(ngx.var.b) == 13 then return ngx.redirect("/bar") end - '; + } echo "res = $b"; } -Note that the [http://www.grid.net.ru/nginx/eval.en.html ngx_eval] module can be approximated by using [[#rewrite_by_lua|rewrite_by_lua]]. For example, +Note that the [http://www.grid.net.ru/nginx/eval.en.html ngx_eval] module can be approximated by using [[#rewrite_by_lua_block|rewrite_by_lua_block]]. For example, location / { @@ -1653,57 +1676,38 @@ can be implemented in ngx_lua as: } location / { - rewrite_by_lua ' + rewrite_by_lua_block { local res = ngx.location.capture("/check-spam") if res.body == "spam" then return ngx.redirect("/terms-of-use.html") end - '; + } fastcgi_pass ...; } -Just as any other rewrite phase handlers, [[#rewrite_by_lua|rewrite_by_lua]] also runs in subrequests. +Just as any other rewrite phase handlers, [[#rewrite_by_lua_block|rewrite_by_lua_block]] also runs in subrequests. -Note that when calling ngx.exit(ngx.OK) within a [[#rewrite_by_lua|rewrite_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#rewrite_by_lua|rewrite_by_lua]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. +Note that when calling ngx.exit(ngx.OK) within a [[#rewrite_by_lua_block|rewrite_by_lua_block]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#rewrite_by_lua_block|rewrite_by_lua_block]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. -If the [[HttpRewriteModule]]'s [[HttpRewriteModule#rewrite|rewrite]] directive is used to change the URI and initiate location re-lookups (internal redirections), then any [[#rewrite_by_lua|rewrite_by_lua]] or [[#rewrite_by_lua_file|rewrite_by_lua_file]] code sequences within the current location will not be executed. For example, +If the [[HttpRewriteModule]]'s [[HttpRewriteModule#rewrite|rewrite]] directive is used to change the URI and initiate location re-lookups (internal redirections), then any [[#rewrite_by_lua_block|rewrite_by_lua_block]] or [[#rewrite_by_lua_file_block|rewrite_by_lua_file_block]] code sequences within the current location will not be executed. For example, location /foo { rewrite ^ /bar; - rewrite_by_lua 'ngx.exit(503)'; + rewrite_by_lua_block { + ngx.exit(503) + } } location /bar { ... } -Here the Lua code ngx.exit(503) will never run. This will be the case if rewrite ^ /bar last is used as this will similarly initiate an internal redirection. If the break modifier is used instead, there will be no internal redirection and the rewrite_by_lua code will be executed. +Here the Lua code ngx.exit(503) will never run. This will be the case if rewrite ^ /bar last is used as this will similarly initiate an internal redirection. If the break modifier is used instead, there will be no internal redirection and the rewrite_by_lua_block code will be executed. -The rewrite_by_lua code will always run at the end of the rewrite request-processing phase unless [[#rewrite_by_lua_no_postpone|rewrite_by_lua_no_postpone]] is turned on. - -== rewrite_by_lua_block == - -'''syntax:''' ''rewrite_by_lua_block { lua-script }'' - -'''context:''' ''http, server, location, location if'' - -'''phase:''' ''rewrite tail'' - -Similar to the [[#rewrite_by_lua|rewrite_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - - - rewrite_by_lua_block { - do_something("hello, world!\nhiya\n") - } - +The rewrite_by_lua_block code will always run at the end of the rewrite request-processing phase unless [[#rewrite_by_lua_no_postpone|rewrite_by_lua_no_postpone]] is turned on. This directive was first introduced in the v0.9.17 release. @@ -1715,7 +1719,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''rewrite tail'' -Equivalent to [[#rewrite_by_lua|rewrite_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#rewrite_by_lua_block|rewrite_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1737,7 +1741,26 @@ Nginx variables are supported in the file path for dynamic dispatch just as in [ '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#access_by_lua_block|access_by_lua_block]] directive instead. -Acts as an access phase handler and executes Lua code string specified in for every request. +Similar to the [[#access_by_lua_block|access_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + + + access_by_lua ' + do_something("hello, world!\nhiya\n") + '; + + +== access_by_lua_block == + +'''syntax:''' ''access_by_lua_block { lua-script }'' + +'''context:''' ''http, server, location, location if'' + +'''phase:''' ''access tail'' + +Acts as an access phase handler and executes Lua code string specified in { for every request. The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). Note that this handler always runs ''after'' the standard [[HttpAccessModule]]. So the following will work as expected: @@ -1749,18 +1772,18 @@ Note that this handler always runs ''after'' the standard [[HttpAccessModule]]. allow 10.1.1.0/16; deny all; - access_by_lua ' + access_by_lua_block { local res = ngx.location.capture("/mysql", { ... }) ... - '; + } # proxy_pass/fastcgi_pass/... } -That is, if a client IP address is in the blacklist, it will be denied before the MySQL query for more complex authentication is executed by [[#access_by_lua|access_by_lua]]. +That is, if a client IP address is in the blacklist, it will be denied before the MySQL query for more complex authentication is executed by [[#access_by_lua_block|access_by_lua_block]]. -Note that the [http://mdounin.ru/hg/ngx_http_auth_request_module/ ngx_auth_request] module can be approximated by using [[#access_by_lua|access_by_lua]]: +Note that the [http://mdounin.ru/hg/ngx_http_auth_request_module/ ngx_auth_request] module can be approximated by using [[#access_by_lua_block|access_by_lua_block]]: location / { @@ -1774,7 +1797,7 @@ can be implemented in ngx_lua as: location / { - access_by_lua ' + access_by_lua_block { local res = ngx.location.capture("/auth") if res.status == ngx.HTTP_OK then @@ -1786,41 +1809,20 @@ can be implemented in ngx_lua as: end ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) - '; + } # proxy_pass/fastcgi_pass/postgres_pass/... } -As with other access phase handlers, [[#access_by_lua|access_by_lua]] will ''not'' run in subrequests. +As with other access phase handlers, [[#access_by_lua_block|access_by_lua_block]] will ''not'' run in subrequests. -Note that when calling ngx.exit(ngx.OK) within a [[#access_by_lua|access_by_lua]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#access_by_lua|access_by_lua]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. +Note that when calling ngx.exit(ngx.OK) within a [[#access_by_lua_block|access_by_lua_block]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#access_by_lua_block|access_by_lua_block]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. Starting from the v0.9.20 release, you can use the [[#access_by_lua_no_postpone|access_by_lua_no_postpone]] directive to control when to run this handler inside the "access" request-processing phase of Nginx. -== access_by_lua_block == - -'''syntax:''' ''access_by_lua_block { lua-script }'' - -'''context:''' ''http, server, location, location if'' - -'''phase:''' ''access tail'' - -Similar to the [[#access_by_lua|access_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - - - access_by_lua_block { - do_something("hello, world!\nhiya\n") - } - - This directive was first introduced in the v0.9.17 release. == access_by_lua_file == @@ -1831,7 +1833,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''access tail'' -Equivalent to [[#access_by_lua|access_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#access_by_lua_block|access_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. @@ -1853,22 +1855,15 @@ Nginx variables are supported in the file path for dynamic dispatch just as in [ '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#header_filter_by_lua_block|header_filter_by_lua_block]] directive instead. -Uses Lua code specified in to define an output header filter. - -Note that the following API functions are currently disabled within this context: - -* Output API functions (e.g., [[#ngx.say|ngx.say]] and [[#ngx.send_headers|ngx.send_headers]]) -* Control API functions (e.g., [[#ngx.redirect|ngx.redirect]] and [[#ngx.exec|ngx.exec]]) -* Subrequest API functions (e.g., [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]]) -* Cosocket API functions (e.g., [[#ngx.socket.tcp|ngx.socket.tcp]] and [[#ngx.req.socket|ngx.req.socket]]). +Similar to the [[#header_filter_by_lua_block|header_filter_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). -Here is an example of overriding a response header (or adding one if absent) in our Lua header filter: +For instance, - location / { - proxy_pass http://mybackend; - header_filter_by_lua 'ngx.header.Foo = "blah"'; - } + header_filter_by_lua ' + ngx.header["content-length"] = nil + '; This directive was first introduced in the v0.2.1rc20 release. @@ -1881,16 +1876,23 @@ This directive was first introduced in the v0.2.1rc20 release. '''phase:''' ''output-header-filter'' -Similar to the [[#header_filter_by_lua|header_filter_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). +Uses Lua code specified in { lua-script } to define an output header filter. -For instance, +Note that the following API functions are currently disabled within this context: + +* Output API functions (e.g., [[#ngx.say|ngx.say]] and [[#ngx.send_headers|ngx.send_headers]]) +* Control API functions (e.g., [[#ngx.redirect|ngx.redirect]] and [[#ngx.exec|ngx.exec]]) +* Subrequest API functions (e.g., [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]]) +* Cosocket API functions (e.g., [[#ngx.socket.tcp|ngx.socket.tcp]] and [[#ngx.req.socket|ngx.req.socket]]). + +Here is an example of overriding a response header (or adding one if absent) in our Lua header filter: - header_filter_by_lua_block { - ngx.header["content-length"] = nil + location / { + proxy_pass http://mybackend; + header_filter_by_lua_block { + ngx.header.Foo = "blah" + } } @@ -1904,7 +1906,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''output-header-filter'' -Equivalent to [[#header_filter_by_lua|header_filter_by_lua]], except that the file specified by contains the Lua code, or as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#header_filter_by_lua_block|header_filter_by_lua_block]], except that the file specified by contains the Lua code, or as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -1920,7 +1922,28 @@ This directive was first introduced in the v0.2.1rc20 release. '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#body_filter_by_lua_block|body_filter_by_lua_block]] directive instead. -Uses Lua code specified in to define an output body filter. +Similar to the [[#body_filter_by_lua_block|body_filter_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + + + body_filter_by_lua ' + local data, eof = ngx.arg[1], ngx.arg[2] + '; + + +This directive was first introduced in the v0.5.0rc32 release. + +== body_filter_by_lua_block == + +'''syntax:''' ''body_filter_by_lua_block { lua-script-str }'' + +'''context:''' ''http, server, location, location if'' + +'''phase:''' ''output-body-filter'' + +Uses Lua code specified in { lua-script } to define an output body filter. The input data chunk is passed via [[#ngx.arg|ngx.arg]][1] (as a Lua string value) and the "eof" flag indicating the end of the response body data stream is passed via [[#ngx.arg|ngx.arg]][2] (as a Lua boolean value). @@ -1939,7 +1962,9 @@ The Lua code can pass its own modified version of the input data chunk to the do location / { proxy_pass http://mybackend; - body_filter_by_lua 'ngx.arg[1] = string.upper(ngx.arg[1])'; + body_filter_by_lua_block { + ngx.arg[1] = string.upper(ngx.arg[1]) + } } @@ -1952,7 +1977,7 @@ Likewise, new "eof" flag can also be specified by setting a boolean value to [[# echo hello world; echo hiya globe; - body_filter_by_lua ' + body_filter_by_lua_block { local chunk = ngx.arg[1] if string.match(chunk, "hello") then ngx.arg[2] = true -- new eof @@ -1961,7 +1986,7 @@ Likewise, new "eof" flag can also be specified by setting a boolean value to [[# -- just throw away any remaining chunk data ngx.arg[1] = nil - '; + } } @@ -1979,8 +2004,12 @@ When the Lua code may change the length of the response body, then it is require location /foo { # fastcgi_pass/proxy_pass/... - header_filter_by_lua_block { ngx.header.content_length = nil } - body_filter_by_lua 'ngx.arg[1] = string.len(ngx.arg[1]) .. "\\n"'; + header_filter_by_lua_block { + ngx.header.content_length = nil + } + body_filter_by_lua_block { + ngx.arg[1] = string.len(ngx.arg[1]) .. "\n" + } } @@ -1993,29 +2022,6 @@ Note that the following API functions are currently disabled within this context Nginx output filters may be called multiple times for a single request because response body may be delivered in chunks. Thus, the Lua code specified by in this directive may also run multiple times in the lifetime of a single HTTP request. -This directive was first introduced in the v0.5.0rc32 release. - -== body_filter_by_lua_block == - -'''syntax:''' ''body_filter_by_lua_block { lua-script-str }'' - -'''context:''' ''http, server, location, location if'' - -'''phase:''' ''output-body-filter'' - -Similar to the [[#body_filter_by_lua|body_filter_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - - - body_filter_by_lua_block { - local data, eof = ngx.arg[1], ngx.arg[2] - } - - This directive was first introduced in the v0.9.17 release. == body_filter_by_lua_file == @@ -2026,7 +2032,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''output-body-filter'' -Equivalent to [[#body_filter_by_lua|body_filter_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#body_filter_by_lua_block|body_filter_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -2042,7 +2048,28 @@ This directive was first introduced in the v0.5.0rc32 release. '''NOTE''' Use of this directive is ''discouraged'' following the v0.9.17 release. Use the [[#log_by_lua_block|log_by_lua_block]] directive instead. -Runs the Lua source code inlined as the at the log request processing phase. This does not replace the current access logs, but runs before. +Similar to the [[#log_by_lua_block|log_by_lua_block]] directive, but accepts the Lua source directly in an Nginx string literal (which requires +special character escaping). + +For instance, + + + log_by_lua ' + print("I need no extra escaping here, for example: \r\nblah") + '; + + +This directive was first introduced in the v0.5.0rc31 release. + +== log_by_lua_block == + +'''syntax:''' ''log_by_lua_block { lua-script }'' + +'''context:''' ''http, server, location, location if'' + +'''phase:''' ''log'' + +Runs the Lua source code inlined as the { lua-script } at the log request processing phase. This does not replace the current access logs, but runs before. Note that the following API functions are currently disabled within this context: @@ -2060,7 +2087,7 @@ Here is an example of gathering average data for [[HttpUpstreamModule#$upstream_ location / { proxy_pass http://mybackend; - log_by_lua ' + log_by_lua_block { local log_dict = ngx.shared.log_dict local upstream_time = tonumber(ngx.var.upstream_response_time) @@ -2073,7 +2100,7 @@ Here is an example of gathering average data for [[HttpUpstreamModule#$upstream_ log_dict:add("upstream_time-nb", 0) log_dict:incr("upstream_time-nb", 1) end - '; + } } location = /status { @@ -2093,29 +2120,6 @@ Here is an example of gathering average data for [[HttpUpstreamModule#$upstream_ } -This directive was first introduced in the v0.5.0rc31 release. - -== log_by_lua_block == - -'''syntax:''' ''log_by_lua_block { lua-script }'' - -'''context:''' ''http, server, location, location if'' - -'''phase:''' ''log'' - -Similar to the [[#log_by_lua|log_by_lua]] directive except that this directive inlines -the Lua source directly -inside a pair of curly braces ({}) instead of in an Nginx string literal (which requires -special character escaping). - -For instance, - - - log_by_lua_block { - print("I need no extra escaping here, for example: \r\nblah") - } - - This directive was first introduced in the v0.9.17 release. == log_by_lua_file == @@ -2126,7 +2130,7 @@ This directive was first introduced in the v0.9.17 release. '''phase:''' ''log'' -Equivalent to [[#log_by_lua|log_by_lua]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. +Equivalent to [[#log_by_lua_block|log_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. @@ -3682,7 +3686,9 @@ Lua tables can be used for both requests and responses when the number of subreq table.insert(reqs, { "/memcached" }) -- issue all the requests at once and wait until they all return - local resps = { ngx.location.capture_multi(reqs) } + local resps = { + ngx.location.capture_multi(reqs) + } -- loop over the responses table for i, resp in ipairs(resps) do @@ -5193,7 +5199,9 @@ For example, location = /md5 { - content_by_lua_block { ngx.say(ngx.md5("hello")) } + content_by_lua_block { + ngx.say(ngx.md5("hello")) + } } From 30debf6740433729af881ca8553cf4ee0438abef Mon Sep 17 00:00:00 2001 From: "Yichun Zhang (agentzh)" Date: Sat, 8 Jan 2022 19:46:50 -0800 Subject: [PATCH 588/848] doc: mentioned OpenResty. --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index fcc0f5ee94..a3a631cb3e 100644 --- a/README.markdown +++ b/README.markdown @@ -14,6 +14,8 @@ then you are essentially using OpenResty. *This module is not distributed with the Nginx source.* See [the installation instructions](#installation). +This is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) + Table of Contents ================= diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 9bfbc7a633..0d7a6f73b1 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -8,6 +8,8 @@ then you are essentially using OpenResty. ''This module is not distributed with the Nginx source.'' See [[#Installation|the installation instructions]]. +This is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) + = Status = Production ready. From 7eb8ad02510fea37dc89731bf1a397c07ea4c326 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 10 Jan 2022 20:33:16 +0800 Subject: [PATCH 589/848] bugfix: ngx.pipe waits until timeout because child process forgot to close pipe after dup2. After the fork, the pipe will be copied by dup2, so there are two fd for the same open file description. eg: 0, 11 for the same pipe file, 1, 13 for another pipe file. When running a short life process, the fd will be closed automatically when the process exits. When running a daemon process, the daemon may close fd 0, fd 1 and redirect them to /dev/null, but left the fd 11 and fd 13 opened forever. when ngx.pipe reads from the pipe, it will wait for reading until the pipe is closed or timeout. Because the daemon won't close the pipe, so ngx.pipe will wait for the timeout. --- src/ngx_http_lua_pipe.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 9ea0ba1c41..93dc7d5bc5 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -773,6 +773,12 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } } + close(in[0]); + close(out[1]); + if (!merge_stderr) { + close(err[1]); + } + if (environ != NULL) { #if (NGX_HTTP_LUA_HAVE_EXECVPE) if (execvpe(file, (char * const *) argv, (char * const *) environ) From 5f02311a9f3680d8fd0ad2aa44af83dab8dce407 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 16 Jan 2022 21:21:59 +0800 Subject: [PATCH 590/848] feature: ssl/tls: support for passphrase protected key. (#1991) Co-authored-by: guanglinlv --- src/ngx_http_lua_ssl_certby.c | 4 +- t/140-ssl-c-api.t | 181 +++++++++++++++++++++++++++++++++- t/cert/test_passphrase.crt | 17 ++++ t/cert/test_passphrase.key | 18 ++++ 4 files changed, 214 insertions(+), 6 deletions(-) create mode 100644 t/cert/test_passphrase.crt create mode 100644 t/cert/test_passphrase.key diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index b561122b13..5be650d5c5 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1042,7 +1042,7 @@ ngx_http_lua_ffi_cert_pem_to_der(const u_char *pem, size_t pem_len, u_char *der, int ngx_http_lua_ffi_priv_key_pem_to_der(const u_char *pem, size_t pem_len, - u_char *der, char **err) + const u_char *passphrase, u_char *der, char **err) { int len; BIO *in; @@ -1055,7 +1055,7 @@ ngx_http_lua_ffi_priv_key_pem_to_der(const u_char *pem, size_t pem_len, return NGX_ERROR; } - pkey = PEM_read_bio_PrivateKey(in, NULL, NULL, NULL); + pkey = PEM_read_bio_PrivateKey(in, NULL, NULL, (void *) passphrase); if (pkey == NULL) { BIO_free(in); *err = "PEM_read_bio_PrivateKey() failed"; diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 33f73ad50d..2c4bcd2254 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -36,7 +36,8 @@ ffi.cdef[[ size_t pem_len, unsigned char *der, char **err); int ngx_http_lua_ffi_priv_key_pem_to_der(const unsigned char *pem, - size_t pem_len, unsigned char *der, char **err); + size_t pem_len, const unsigned char *passphrase, + unsigned char *der, char **err); int ngx_http_lua_ffi_ssl_set_der_certificate(void *r, const char *data, size_t len, char **err); @@ -130,7 +131,7 @@ __DATA__ out = ffi.new("char [?]", #pkey) - local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, out, errmsg) + local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, nil, out, errmsg) if rc < 1 then ngx.log(ngx.ERR, "failed to parse PEM priv key: ", ffi.string(errmsg[0])) @@ -284,7 +285,7 @@ lua ssl server name: "test.com" out = ffi.new("char [?]", #pkey) - local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, out, errmsg) + local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, nil, out, errmsg) if rc < 1 then ngx.log(ngx.ERR, "failed to parse PEM priv key: ", ffi.string(errmsg[0])) @@ -422,7 +423,7 @@ lua ssl server name: "test.com" out = ffi.new("char [?]", #pkey) - local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, out, errmsg) + local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, nil, out, errmsg) if rc < 1 then ngx.log(ngx.ERR, "failed to parse PEM priv key: ", ffi.string(errmsg[0])) @@ -1024,3 +1025,175 @@ client certificate subject: nil --- no_error_log [error] [alert] + + + +=== TEST 9: simple cert + private key with passphrase +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + local ffi = require "ffi" + + ffi.cdef[[ + int ngx_http_lua_ffi_cert_pem_to_der(const unsigned char *pem, + size_t pem_len, unsigned char *der, char **err); + + int ngx_http_lua_ffi_priv_key_pem_to_der(const unsigned char *pem, + size_t pem_len, const unsigned char *passphrase, + unsigned char *der, char **err); + + int ngx_http_lua_ffi_ssl_set_der_certificate(void *r, + const char *data, size_t len, char **err); + + int ngx_http_lua_ffi_ssl_set_der_private_key(void *r, + const char *data, size_t len, char **err); + + int ngx_http_lua_ffi_ssl_clear_certs(void *r, char **err); + ]] + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if not r then + ngx.log(ngx.ERR, "no request found") + return + end + + ffi.C.ngx_http_lua_ffi_ssl_clear_certs(r, errmsg) + + local f = assert(io.open("t/cert/test_passphrase.crt", "rb")) + local cert = f:read("*all") + f:close() + + local out = ffi.new("char [?]", #cert) + + local rc = ffi.C.ngx_http_lua_ffi_cert_pem_to_der(cert, #cert, out, errmsg) + if rc < 1 then + ngx.log(ngx.ERR, "failed to parse PEM cert: ", + ffi.string(errmsg[0])) + return + end + + local cert_der = ffi.string(out, rc) + + local rc = ffi.C.ngx_http_lua_ffi_ssl_set_der_certificate(r, cert_der, #cert_der, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to set DER cert: ", + ffi.string(errmsg[0])) + return + end + + f = assert(io.open("t/cert/test_passphrase.key", "rb")) + local pkey = f:read("*all") + f:close() + + local passphrase = "123456" + + out = ffi.new("char [?]", #pkey) + + local rc = ffi.C.ngx_http_lua_ffi_priv_key_pem_to_der(pkey, #pkey, passphrase, out, errmsg) + if rc < 1 then + ngx.log(ngx.ERR, "failed to parse PEM priv key: ", + ffi.string(errmsg[0])) + return + end + + local pkey_der = ffi.string(out, rc) + + local rc = ffi.C.ngx_http_lua_ffi_ssl_set_der_private_key(r, pkey_der, #pkey_der, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to set DER priv key: ", + ffi.string(errmsg[0])) + return + end + } + + ssl_certificate ../../cert/test2.crt; + ssl_certificate_key ../../cert/test2.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test_passphrase.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to recieve response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" + +--- no_error_log +[error] +[alert] diff --git a/t/cert/test_passphrase.crt b/t/cert/test_passphrase.crt new file mode 100644 index 0000000000..f4d516ba9e --- /dev/null +++ b/t/cert/test_passphrase.crt @@ -0,0 +1,17 @@ +-----BEGIN CERTIFICATE----- +MIICozCCAgwCCQDEutRdSs3vZjANBgkqhkiG9w0BAQUFADCBlDELMAkGA1UEBhMC +Q04xEjAQBgNVBAgMCUd1YW5nZG9uZzERMA8GA1UEBwwIU2hlblpoZW4xEjAQBgNV +BAoMCU9wZW5SZXN0eTESMBAGA1UECwwJT3BlblJlc3R5MREwDwYDVQQDDAh0ZXN0 +LmNvbTEjMCEGCSqGSIb3DQEJARYUZ3VhbmdsaW5sdkBnbWFpbC5jb20wIBcNMTYw +NDI4MTQ0MzI4WhgPMjE1MTAzMjcxNDQzMjhaMIGUMQswCQYDVQQGEwJDTjESMBAG +A1UECAwJR3Vhbmdkb25nMREwDwYDVQQHDAhTaGVuWmhlbjESMBAGA1UECgwJT3Bl +blJlc3R5MRIwEAYDVQQLDAlPcGVuUmVzdHkxETAPBgNVBAMMCHRlc3QuY29tMSMw +IQYJKoZIhvcNAQkBFhRndWFuZ2xpbmx2QGdtYWlsLmNvbTCBnzANBgkqhkiG9w0B +AQEFAAOBjQAwgYkCgYEA2KZ+HdH9R2tarxD8PKqu5EYq2BNGlFRg1xJmrw0XZBRM +UP/VPb+sIeioooz36uhiXfQjExlpBCA/0zNAN+HbFyqpPPTf1qLGrj/dqeE4MJaN +Bwzxiv3fZnENT65u2qbiFWIY+ATNHgA20d50nxNNjPTzLbkx/nYXL92r4kuAGk0C +AwEAATANBgkqhkiG9w0BAQUFAAOBgQCfMo0qbcs3kwl1tcNBO5hCcUUJRzyv041V +ff/nZ/JPIMo/LSZd12K82G/dLRN7uRT9nzqtm+JRkHALHWWWFKi6bdg1vcdOTWqC +08bCkJHQoXJQQLvvA6gNvnR+0b7L4CrCmrcyYgKDLXVGNP9Wv/PqSWWbxsmqngkA +Mvy6CVytFw== +-----END CERTIFICATE----- diff --git a/t/cert/test_passphrase.key b/t/cert/test_passphrase.key new file mode 100644 index 0000000000..b8fc97634a --- /dev/null +++ b/t/cert/test_passphrase.key @@ -0,0 +1,18 @@ +-----BEGIN RSA PRIVATE KEY----- +Proc-Type: 4,ENCRYPTED +DEK-Info: DES-EDE3-CBC,679ACC8E69ACAA92 + +Ssrjp3VU4somCNPiXkWqcudDnvnwbyj/Q0pS07at3lXKbhQSgI1Tzhg9Pm3BXXj5 +mkLdeGG5ocrj1Q9dhtmZgZeHHQIiynZBhjBu1Y+HPef8jXOWLrCOi8EKiWkJ2qG3 +V1KFM/95CcDt0mRLykUXEL3IpUst05SFb9XwiLokB7ypeu3NhgNUHjL6G+ubB4ri +TOUjCW4pEoNHjdC22IiqSncwCVhluYSGhr6ktHKehZMhYIXmL1wmSLdhTlsPXCQl +xvYILQ2vJcKIR1BkeYYPD/OQC6zCZlXIErzfgeZiz2+NTudKYpb9VmsQKsO+R8L7 +tZ/fNaR0vk8bbimMHgStAV4acVsC/7WxsqOjMJ8VTq1iqhYPl6N7kRdR3H3kSSOm +cN9T3SrOHDVaHbnWgToaOE4mKFjvFSLIOcWgus0iOHWXmY+SLG+Ndag3oVB6R9oB +cAHX19mq99+GhzA8IV4I0En2UCKQhnGPvkM+9mcCDxhRETlwncDjlMGOHpQ65J9r +eReVPIpnDkvHxPGTtsR3ZHTdWTZb+C0W2N3QIlJKrOzxFmfoj++yG3tMX42aDY0g +DVkrXgcKobiWN0AVrJNAwfG7uObKSCFYgz/0RRMCO4cjXRW99nxdjVDZhyc6R0Te +jzuF04okkOLNb25n2hP+yIULrn+6Nv/uHtFI0j0n3hOzcKh//dNbACSAKgkHni9g +JKDFJXgLJxf+Wc3So0DF9gYMKJJ+WbcdVT9gkC7RyQHlC90Pn7kNXzHr0ZawUsNI +ZxSkL4dMhYAfA4lUBJbOkwbSurv97LinOSRffpM0Nmf7VNw/Ue15eg== +-----END RSA PRIVATE KEY----- From a41fdcea494930166a0b72e2fcc9290769854bce Mon Sep 17 00:00:00 2001 From: James Callahan Date: Thu, 22 Mar 2018 15:02:28 +1100 Subject: [PATCH 591/848] feature: add api to fetch raw nginx ssl pointer of the downstream request. --- src/ngx_http_lua_ssl_certby.c | 11 +++ t/140-ssl-c-api.t | 122 ++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 5be650d5c5..ce7f6483af 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1479,4 +1479,15 @@ ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, } +ngx_ssl_conn_t * +ngx_http_lua_ffi_get_req_ssl_pointer(ngx_http_request_t *r) +{ + if (r->connection == NULL || r->connection->ssl == NULL) { + return NULL; + } + + return r->connection->ssl->connection; +} + + #endif /* NGX_HTTP_SSL */ diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 2c4bcd2254..1099340a29 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -59,6 +59,8 @@ ffi.cdef[[ int ngx_http_lua_ffi_set_priv_key(void *r, void *cdata, char **err); + void *ngx_http_lua_ffi_get_req_ssl_pointer(void *r); + void ngx_http_lua_ffi_free_cert(void *cdata); void ngx_http_lua_ffi_free_priv_key(void *cdata); @@ -1197,3 +1199,123 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] + + + +=== TEST 10: Raw SSL pointer +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + local ffi = require "ffi" + require "defines" + + local r = require "resty.core.base" .get_request() + if not r then + ngx.log(ngx.ERR, "no request found") + return + end + + local ssl = ffi.C.ngx_http_lua_ffi_get_req_ssl_pointer(r); + if ssl == nil then + ngx.log(ngx.ERR, "failed to retrieve SSL*") + return + end + + ffi.cdef[[ + const char *SSL_get_servername(const void *, const int); + ]] + local libssl = ffi.load "ssl" + local TLSEXT_NAMETYPE_host_name = 0 + local sni = ffi.string(libssl.SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name)) + ngx.log(ngx.INFO, "SNI is ", sni) + } + + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: userdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +SNI is test.com + +--- no_error_log +[error] +[alert] From fa5982c63986981a3d44bcf0dbfd5e728da7cdc6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 17 Jan 2022 11:11:30 +0800 Subject: [PATCH 592/848] tests: t/140-ssl-c-api.t failed because got wrong libssl.so. --- t/140-ssl-c-api.t | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 1099340a29..31a6e2f434 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -1229,10 +1229,14 @@ lua ssl server name: "test.com" ffi.cdef[[ const char *SSL_get_servername(const void *, const int); ]] - local libssl = ffi.load "ssl" local TLSEXT_NAMETYPE_host_name = 0 - local sni = ffi.string(libssl.SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name)) - ngx.log(ngx.INFO, "SNI is ", sni) + local sni = ffi.C.SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name) + if sni == nil then + ngx.log(ngx.ERR, "failed to get sni") + return + end + + ngx.log(ngx.INFO, "SNI is ", ffi.string(sni)) } ssl_certificate ../../cert/test.crt; From ce8405c73b8d19becb79cf73e8159aaa36356e07 Mon Sep 17 00:00:00 2001 From: jinhua luo Date: Wed, 19 Jan 2022 09:29:45 +0800 Subject: [PATCH 593/848] bugfix: ngx.run_worker_thread injected api into the wrong table.(#1996) --- src/ngx_http_lua_worker_thread.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 5133fa6e76..69c6da1d01 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -139,12 +139,13 @@ ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) lua_setfield(vm, -2, "path"); lua_pushlstring(vm, cpath, cpath_len); lua_setfield(vm, -2, "cpath"); + lua_pop(vm, 1); /* pop path, cpath and "package" table from L */ lua_pop(L, 3); /* inject API from C */ - lua_newtable(L); /* ngx.* */ + lua_newtable(vm); /* ngx.* */ ngx_http_lua_inject_string_api(vm); ngx_http_lua_inject_config_api(vm); lua_setglobal(vm, "ngx"); From 8b497f5a56df559c92df5b2de165a37697ac9d53 Mon Sep 17 00:00:00 2001 From: Tinglong Yang Date: Tue, 25 Jan 2022 22:36:19 +0800 Subject: [PATCH 594/848] change: delete useless code for get old_cpath. (#2000) --- src/ngx_http_lua_util.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 232a1c1315..b783cf850c 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -281,7 +281,6 @@ ngx_http_lua_new_state(lua_State *parent_vm, ngx_cycle_t *cycle, lua_pushliteral(L, LUA_DEFAULT_CPATH ";"); /* package default */ lua_getfield(L, -2, "cpath"); /* package default old */ - old_cpath = lua_tolstring(L, -1, &old_cpath_len); lua_concat(L, 2); /* package new */ lua_setfield(L, -2, "cpath"); /* package */ #endif From 8cf9c632d8ac453a74ce7aabf4e88d66f2fea6a6 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 26 Jan 2022 23:42:01 +0800 Subject: [PATCH 595/848] travis-ci: upgraded openssl to 1.1.1m, pcre to 8.45. (#2002) --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index daae0704f7..335ae71ee6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -37,7 +37,7 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_VER=8.44 + - PCRE_VER=8.45 - PCRE_PREFIX=/opt/pcre - PCRE_LIB=$PCRE_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include @@ -52,7 +52,7 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1l OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1m OPENSSL_PATCH_VER=1.1.1f services: - memcached From 2615bafa34f649923c6d80bc4b4393a2e62b87f3 Mon Sep 17 00:00:00 2001 From: Gordon McKinney Date: Tue, 8 Feb 2022 23:28:10 +0800 Subject: [PATCH 596/848] bugfix: segment fault when get header via ngx.req.raw_header with malformed requests. Co-authored-by: lijunlong@openresty.com --- src/ngx_http_lua_headers.c | 6 ++++++ t/104-req-raw-header.t | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 3ebe7847cb..54977dce8e 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -167,6 +167,12 @@ ngx_http_lua_ngx_req_raw_header(lua_State *L) size = 0; b = c->buffer; + if (mr->request_line.len == 0) { + /* return empty string on invalid request */ + lua_pushlstring(L, "", 0); + return 1; + } + if (mr->request_line.data[mr->request_line.len] == CR) { line_break_len = 2; diff --git a/t/104-req-raw-header.t b/t/104-req-raw-header.t index f147b2bcf9..aa66630485 100644 --- a/t/104-req-raw-header.t +++ b/t/104-req-raw-header.t @@ -1017,3 +1017,36 @@ client sent invalid header line: "\x20..." while reading client request headers [error] --- skip_nginx 3: < 1.21.1 + + + +=== TEST 35: bugfix: invalid http request +--- log_level: error +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + sock:send("\n") + sock:close() + + ngx.say("OK") + } + } + + log_by_lua_block { + local h = ngx.req.raw_header() + } + +--- request +GET /t +--- response_body +OK +--- no_error_log +[error] From 297e73cde6f08aa7f6bd530b9f0a301570ddf15d Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 9 Feb 2022 12:43:14 +0800 Subject: [PATCH 597/848] bugfix: the error message should use the first line rather than the last line of the code block when load lua code block failed. (#2005) --- src/ngx_http_lua_common.h | 2 + src/ngx_http_lua_directive.c | 15 +++++++- t/002-content.t | 71 +++++++++++++++++++++++++++++++++++- 3 files changed, 85 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index ed88f0a2f5..65a6415a89 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -276,6 +276,8 @@ struct ngx_http_lua_main_conf_s { of requests */ ngx_uint_t malloc_trim_req_count; + ngx_uint_t directive_line; + #if (nginx_version >= 1011011) /* the following 2 fields are only used by ngx.req.raw_headers() for now */ ngx_buf_t **busy_buf_ptrs; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 831132f12d..4366b4fff6 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1302,6 +1302,9 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, { u_char *p, *out; size_t len; + ngx_uint_t start_line; + + ngx_http_lua_main_conf_t *lmcf; len = sizeof("=(:)") - 1 + tag_len + cf->conf_file->file.name.len + NGX_INT64_LEN + 1; @@ -1328,10 +1331,14 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, found: + lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); + start_line = lmcf->directive_line > 0 + ? lmcf->directive_line : cf->conf_file->line; + p = ngx_snprintf(out, len, "=%*s(%*s:%d)%Z", tag_len, tag, cf->conf_file->file.name.data + cf->conf_file->file.name.len - p, - p, cf->conf_file->line); + p, start_line); *chunkname_len = p - out - 1; /* exclude the trailing '\0' byte */ @@ -1343,6 +1350,7 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, char * ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd) { + ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_block_parser_ctx_t ctx; int level = 1; @@ -1376,6 +1384,9 @@ ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd) ctx.token_len = 0; start_line = cf->conf_file->line; + lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); + lmcf->directive_line = start_line; + dd("init start line: %d", (int) start_line); ctx.start_line = start_line; @@ -1494,6 +1505,8 @@ ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd) done: + lmcf->directive_line = 0; + if (rc == NGX_ERROR) { return NGX_CONF_ERROR; } diff --git a/t/002-content.t b/t/002-content.t index d6a3600444..703ba84a26 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -10,7 +10,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 24); +plan tests => repeat_each() * (blocks() * 2 + 27); #no_diff(); #no_long_string(); @@ -849,4 +849,71 @@ GET /lua --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log eval -qr/failed to load inlined Lua code: / +qr/failed to load inlined Lua code: content_by_lua\(nginx.conf:40\)/ + + + +=== TEST 43: syntax error in content_by_lua_block +--- config + location /lua { + + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log eval +qr/failed to load inlined Lua code: content_by_lua\(nginx.conf:41\)/ + + + +=== TEST 44: syntax error in second content_by_lua_block +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /lua { + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log eval +qr/failed to load inlined Lua code: content_by_lua\(nginx.conf:46\)/ + + + +=== TEST 45: syntax error in thrid content_by_lua_block +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /bar { + content_by_lua_block { + 'for end'; + } + } + + location /lua { + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log eval +qr/failed to load inlined Lua code: content_by_lua\(nginx.conf:52\)/ From 907d1d9adac214abe93f8c7d5dea38909a868f0f Mon Sep 17 00:00:00 2001 From: Chrono Date: Fri, 11 Feb 2022 19:48:54 +0800 Subject: [PATCH 598/848] doc: modified the todo section. we now have exit_worker_by_lua (#2009) --- README.markdown | 1 - doc/HttpLuaModule.wiki | 37 ++++++++++++++++++------------------- 2 files changed, 18 insertions(+), 20 deletions(-) diff --git a/README.markdown b/README.markdown index a3a631cb3e..5a87d0c45d 100644 --- a/README.markdown +++ b/README.markdown @@ -974,7 +974,6 @@ TODO * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option * use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), [ngx.header.HEADER](#ngxheaderheader), and etc. -* add directives to run Lua codes when Nginx stops. * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add `stat` mode similar to [mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html). diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 0d7a6f73b1..5c79dda3d6 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -809,7 +809,6 @@ phases. * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [[#ngx.location.capture|ngx.location.capture]]'s extra_headers option * use ngx_hash_t to optimize the built-in header look-up process for [[#ngx.req.set_header|ngx.req.set_header]], [[#ngx.header.HEADER|ngx.header.HEADER]], and etc. -* add directives to run Lua codes when Nginx stops. * add ignore_resp_headers, ignore_resp_body, and ignore_resp options to [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add stat mode similar to [https://httpd.apache.org/docs/trunk/mod/mod_lua.html mod_lua]. @@ -1607,8 +1606,8 @@ Note that this handler always runs ''after'' the standard [[HttpRewriteModule]]. location /foo { set $a 12; # create and initialize $a set $b ""; # create and initialize $b - rewrite_by_lua_block { - ngx.var.b = tonumber(ngx.var.a) + 1 + rewrite_by_lua_block { + ngx.var.b = tonumber(ngx.var.a) + 1 } echo "res = $b"; } @@ -1622,8 +1621,8 @@ On the other hand, the following will not work as expected: ? location /foo { ? set $a 12; # create and initialize $a ? set $b ''; # create and initialize $b - ? rewrite_by_lua_block { - ? ngx.var.b = tonumber(ngx.var.a) + 1 + ? rewrite_by_lua_block { + ? ngx.var.b = tonumber(ngx.var.a) + 1 ? } ? if ($b = '13') { ? rewrite ^ /bar redirect; @@ -1698,8 +1697,8 @@ If the [[HttpRewriteModule]]'s [[HttpRewriteModule#rewrite|rewrite]] directive i location /foo { rewrite ^ /bar; - rewrite_by_lua_block { - ngx.exit(503) + rewrite_by_lua_block { + ngx.exit(503) } } location /bar { @@ -1892,8 +1891,8 @@ Here is an example of overriding a response header (or adding one if absent) in location / { proxy_pass http://mybackend; - header_filter_by_lua_block { - ngx.header.Foo = "blah" + header_filter_by_lua_block { + ngx.header.Foo = "blah" } } @@ -1964,8 +1963,8 @@ The Lua code can pass its own modified version of the input data chunk to the do location / { proxy_pass http://mybackend; - body_filter_by_lua_block { - ngx.arg[1] = string.upper(ngx.arg[1]) + body_filter_by_lua_block { + ngx.arg[1] = string.upper(ngx.arg[1]) } } @@ -2006,11 +2005,11 @@ When the Lua code may change the length of the response body, then it is require location /foo { # fastcgi_pass/proxy_pass/... - header_filter_by_lua_block { - ngx.header.content_length = nil + header_filter_by_lua_block { + ngx.header.content_length = nil } - body_filter_by_lua_block { - ngx.arg[1] = string.len(ngx.arg[1]) .. "\n" + body_filter_by_lua_block { + ngx.arg[1] = string.len(ngx.arg[1]) .. "\n" } } @@ -3688,8 +3687,8 @@ Lua tables can be used for both requests and responses when the number of subreq table.insert(reqs, { "/memcached" }) -- issue all the requests at once and wait until they all return - local resps = { - ngx.location.capture_multi(reqs) + local resps = { + ngx.location.capture_multi(reqs) } -- loop over the responses table @@ -5201,8 +5200,8 @@ For example, location = /md5 { - content_by_lua_block { - ngx.say(ngx.md5("hello")) + content_by_lua_block { + ngx.say(ngx.md5("hello")) } } From 70a50e66c4e43287d0bbd4ec9bd0485f53a7e110 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 14 Feb 2022 15:12:45 +0800 Subject: [PATCH 599/848] optimize: change lua chunkname to config filename and line number for (init/header_filter/body_filter)_by_lua_block and so on. (#2008) --- src/ngx_http_lua_balancer.c | 12 +- src/ngx_http_lua_bodyfilterby.c | 3 +- src/ngx_http_lua_common.h | 11 ++ src/ngx_http_lua_directive.c | 132 ++++++++++++++++--- src/ngx_http_lua_directive.h | 3 +- src/ngx_http_lua_exitworkerby.c | 10 +- src/ngx_http_lua_headerfilterby.c | 3 +- src/ngx_http_lua_initby.c | 11 +- src/ngx_http_lua_initworkerby.c | 10 +- src/ngx_http_lua_module.c | 12 ++ src/ngx_http_lua_ssl_certby.c | 12 +- src/ngx_http_lua_ssl_client_helloby.c | 13 +- src/ngx_http_lua_ssl_session_fetchby.c | 11 +- src/ngx_http_lua_ssl_session_storeby.c | 11 +- t/001-set.t | 2 +- t/002-content.t | 171 ++++++++++++++++++++++++- t/009-log.t | 8 +- t/025-codecache.t | 16 +-- t/041-header-filter.t | 52 +++++++- t/082-body-filter.t | 54 +++++++- t/086-init-by.t | 44 +++++++ t/091-coroutine.t | 6 +- t/124-init-worker.t | 50 ++++++++ t/138-balancer.t | 35 ++++- t/139-ssl-cert-by.t | 16 +-- t/142-ssl-session-store.t | 10 +- t/143-ssl-session-fetch.t | 38 +++--- t/158-global-var.t | 16 +-- t/162-exit-worker.t | 48 ++++++- 29 files changed, 722 insertions(+), 98 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index e4ac57a301..af4da73388 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -90,7 +90,7 @@ ngx_http_lua_balancer_handler_inline(ngx_http_request_t *r, lscf->balancer.src.len, &lscf->balancer.src_ref, lscf->balancer.src_key, - "=balancer_by_lua"); + (const char *) lscf->balancer.chunkname); if (rc != NGX_OK) { return rc; } @@ -125,6 +125,8 @@ char * ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; + u_char *chunkname; u_char *cache_key = NULL; u_char *name; ngx_str_t *value; @@ -172,8 +174,16 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "balancer_by_lua", + sizeof("balancer_by_lua") - 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + /* Don't eval nginx variables for inline lua code */ lscf->balancer.src = value[1]; + lscf->balancer.chunkname = chunkname; } lscf->balancer.src_key = cache_key; diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 902488911e..1aa01e552b 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -161,7 +161,8 @@ ngx_http_lua_body_filter_inline(ngx_http_request_t *r, ngx_chain_t *in) llcf->body_filter_src.value.len, &llcf->body_filter_src_ref, llcf->body_filter_src_key, - "=body_filter_by_lua"); + (const char *) + llcf->body_filter_chunkname); if (rc != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 65a6415a89..017e2a0434 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -31,6 +31,7 @@ typedef struct { size_t size; int ref; u_char *key; + u_char *chunkname; ngx_str_t script; } ngx_http_lua_set_var_data_t; #endif @@ -233,12 +234,15 @@ struct ngx_http_lua_main_conf_s { ngx_http_lua_main_conf_handler_pt init_handler; ngx_str_t init_src; + u_char *init_chunkname; ngx_http_lua_main_conf_handler_pt init_worker_handler; ngx_str_t init_worker_src; + u_char *init_worker_chunkname; ngx_http_lua_main_conf_handler_pt exit_worker_handler; ngx_str_t exit_worker_src; + u_char *exit_worker_chunkname; ngx_http_lua_balancer_peer_data_t *balancer_peer_data; /* neither yielding nor recursion is possible in @@ -310,21 +314,25 @@ union ngx_http_lua_srv_conf_u { ngx_http_lua_srv_conf_handler_pt ssl_cert_handler; ngx_str_t ssl_cert_src; u_char *ssl_cert_src_key; + u_char *ssl_cert_chunkname; int ssl_cert_src_ref; ngx_http_lua_srv_conf_handler_pt ssl_sess_store_handler; ngx_str_t ssl_sess_store_src; u_char *ssl_sess_store_src_key; + u_char *ssl_sess_store_chunkname; int ssl_sess_store_src_ref; ngx_http_lua_srv_conf_handler_pt ssl_sess_fetch_handler; ngx_str_t ssl_sess_fetch_src; u_char *ssl_sess_fetch_src_key; + u_char *ssl_sess_fetch_chunkname; int ssl_sess_fetch_src_ref; ngx_http_lua_srv_conf_handler_pt ssl_client_hello_handler; ngx_str_t ssl_client_hello_src; u_char *ssl_client_hello_src_key; + u_char *ssl_client_hello_chunkname; int ssl_client_hello_src_ref; } srv; #endif @@ -333,6 +341,7 @@ union ngx_http_lua_srv_conf_u { ngx_http_lua_srv_conf_handler_pt handler; ngx_str_t src; u_char *src_key; + u_char *chunkname; int src_ref; } balancer; }; @@ -403,6 +412,7 @@ typedef struct { inline script/script file path */ + u_char *header_filter_chunkname; u_char *header_filter_src_key; /* cached key for header_filter_src */ int header_filter_src_ref; @@ -410,6 +420,7 @@ typedef struct { ngx_http_complex_value_t body_filter_src; u_char *body_filter_src_key; + u_char *body_filter_chunkname; int body_filter_src_ref; ngx_msec_t keepalive_timeout; diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index 4366b4fff6..dda447fb7e 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -32,6 +32,9 @@ #include "ngx_http_lua_log.h" +#define LJ_CHUNKNAME_MAX_LEN 42 + + typedef struct ngx_http_lua_block_parser_ctx_s ngx_http_lua_block_parser_ctx_t; @@ -43,8 +46,6 @@ typedef struct ngx_http_lua_block_parser_ctx_s static ngx_int_t ngx_http_lua_set_by_lua_init(ngx_http_request_t *r); #endif -static u_char *ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, - size_t tag_len, size_t *chunkname_len); static ngx_int_t ngx_http_lua_conf_read_lua_token(ngx_conf_t *cf, ngx_http_lua_block_parser_ctx_t *ctx); static u_char *ngx_http_lua_strlstrn(u_char *s1, u_char *last, u_char *s2, @@ -280,6 +281,8 @@ ngx_http_lua_set_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, char * ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; + u_char *chunkname; u_char *cache_key; ngx_str_t *value; ngx_str_t target; @@ -312,7 +315,15 @@ ngx_http_lua_set_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "set_by_lua", + sizeof("set_by_lua") - 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + filter_data->key = cache_key; + filter_data->chunkname = chunkname; filter_data->ref = LUA_REFNIL; filter_data->script = value[2]; filter_data->size = filter.size; @@ -375,6 +386,7 @@ ngx_http_lua_set_by_lua_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) filter_data->key = cache_key; filter_data->ref = LUA_REFNIL; filter_data->size = filter.size; + filter_data->chunkname = NULL; ngx_str_null(&filter_data->script); @@ -404,7 +416,8 @@ ngx_http_lua_filter_set_by_lua_inline(ngx_http_request_t *r, ngx_str_t *val, filter_data->script.data, filter_data->script.len, &filter_data->ref, - filter_data->key, "=set_by_lua"); + filter_data->key, + (const char *) filter_data->chunkname); if (rc != NGX_OK) { return NGX_ERROR; } @@ -912,7 +925,8 @@ char * ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *cache_key = NULL; + size_t chunkname_len; + u_char *cache_key = NULL, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -947,8 +961,15 @@ ngx_http_lua_header_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "header_filter_by_lua", + sizeof("header_filter_by_lua") - 1, &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + /* Don't eval nginx variables for inline lua code */ llcf->header_filter_src.value = value[1]; + llcf->header_filter_chunkname = chunkname; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -1004,7 +1025,8 @@ char * ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { - u_char *cache_key = NULL; + size_t chunkname_len; + u_char *cache_key = NULL, *chunkname; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_loc_conf_t *llcf = conf; @@ -1039,8 +1061,16 @@ ngx_http_lua_body_filter_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "body_filter_by_lua", + sizeof("body_filter_by_lua") - 1, &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + + /* Don't eval nginx variables for inline lua code */ llcf->body_filter_src.value = value[1]; + llcf->body_filter_chunkname = chunkname; } else { ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); @@ -1100,6 +1130,8 @@ ngx_http_lua_init_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, u_char *name; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf = conf; + size_t chunkname_len; + u_char *chunkname; dd("enter"); @@ -1135,6 +1167,15 @@ ngx_http_lua_init_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } else { lmcf->init_src = value[1]; + + chunkname = ngx_http_lua_gen_chunk_name(cf, "init_by_lua", + sizeof("init_by_lua") - 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + + lmcf->init_chunkname = chunkname; } return NGX_CONF_OK; @@ -1167,6 +1208,8 @@ ngx_http_lua_init_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, u_char *name; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf = conf; + size_t chunkname_len; + u_char *chunkname; dd("enter"); @@ -1195,6 +1238,14 @@ ngx_http_lua_init_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } else { lmcf->init_worker_src = value[1]; + + chunkname = ngx_http_lua_gen_chunk_name(cf, "init_worker_by_lua", + sizeof("init_worker_by_lua") - 1, &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + + lmcf->init_worker_chunkname = chunkname; } return NGX_CONF_OK; @@ -1227,6 +1278,8 @@ ngx_http_lua_exit_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, u_char *name; ngx_str_t *value; ngx_http_lua_main_conf_t *lmcf = conf; + size_t chunkname_len; + u_char *chunkname; /* must specify a content handler */ if (cmd->post == NULL) { @@ -1253,6 +1306,15 @@ ngx_http_lua_exit_worker_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } else { lmcf->exit_worker_src = value[1]; + + chunkname = ngx_http_lua_gen_chunk_name(cf, "exit_worker_by_lua", + sizeof("exit_worker_by_lua")- 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + + lmcf->exit_worker_chunkname = chunkname; } return NGX_CONF_OK; @@ -1296,13 +1358,18 @@ ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) #endif -static u_char * +u_char * ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, size_t *chunkname_len) { u_char *p, *out; size_t len; ngx_uint_t start_line; + ngx_str_t *conf_prefix; + ngx_str_t *filename; + u_char *filename_end; + const char *pre_str = ""; + ngx_uint_t start_line_len; ngx_http_lua_main_conf_t *lmcf; @@ -1314,30 +1381,55 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, return NULL; } - if (cf->conf_file->file.name.len) { - p = cf->conf_file->file.name.data + cf->conf_file->file.name.len; - while (--p >= cf->conf_file->file.name.data) { - if (*p == '/' || *p == '\\') { - p++; + lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); + start_line = lmcf->directive_line > 0 + ? lmcf->directive_line : cf->conf_file->line; + p = ngx_snprintf(out, len, "%d", start_line); + start_line_len = p - out; + + filename = &cf->conf_file->file.name; + filename_end = filename->data + filename->len; + if (filename->len > 0) { + if (filename->len >= 11) { + p = filename_end - 11; + if ((*p == '/' || *p == '\\') + && ngx_memcmp(p, "/nginx.conf", 11) == 0) + { + p++; /* now p is nginx.conf */ goto found; } } - p++; + conf_prefix = &cf->cycle->conf_prefix; + p = filename->data + conf_prefix->len; + if ((conf_prefix->len < filename->len) + && ngx_memcmp(conf_prefix->data, + filename->data, conf_prefix->len) == 0) + { + /* files in conf_prefix directory, use the relative path */ + if (filename_end - p + start_line_len > LJ_CHUNKNAME_MAX_LEN) { + p = filename_end - LJ_CHUNKNAME_MAX_LEN + start_line_len + 3; + pre_str = "..."; + } - } else { - p = cf->conf_file->file.name.data; + goto found; + } } + p = filename->data; + + if (filename->len + start_line_len <= LJ_CHUNKNAME_MAX_LEN) { + goto found; + } + + p = filename_end - LJ_CHUNKNAME_MAX_LEN + start_line_len + 3; + pre_str = "..."; + found: - lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); - start_line = lmcf->directive_line > 0 - ? lmcf->directive_line : cf->conf_file->line; - p = ngx_snprintf(out, len, "=%*s(%*s:%d)%Z", - tag_len, tag, cf->conf_file->file.name.data - + cf->conf_file->file.name.len - p, + p = ngx_snprintf(out, len, "=%*s(%s%*s:%d)%Z", + tag_len, tag, pre_str, filename_end - p, p, start_line); *chunkname_len = p - out - 1; /* exclude the trailing '\0' byte */ diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index adfba12761..315c0a9f2d 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -81,7 +81,8 @@ char *ngx_http_lua_conf_lua_block_parse(ngx_conf_t *cf, ngx_command_t *cmd); char *ngx_http_lua_capture_error_log(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); - +u_char *ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, + size_t tag_len, size_t *chunkname_len); #endif /* _NGX_HTTP_LUA_DIRECTIVE_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_exitworkerby.c b/src/ngx_http_lua_exitworkerby.c index f1329566bc..65edc04e63 100644 --- a/src/ngx_http_lua_exitworkerby.c +++ b/src/ngx_http_lua_exitworkerby.c @@ -95,12 +95,20 @@ ngx_http_lua_exit_worker_by_inline(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, lua_State *L) { int status; + const char *chunkname; status = luaL_loadbuffer(L, (char *) lmcf->exit_worker_src.data, lmcf->exit_worker_src.len, "=exit_worker_by_lua") || ngx_http_lua_do_call(log, L); - return ngx_http_lua_report(log, L, status, "exit_worker_by_lua"); + if (lmcf->exit_worker_chunkname == NULL) { + chunkname = "exit_worker_by_lua"; + + } else { + chunkname = (const char *) lmcf->exit_worker_chunkname; + } + + return ngx_http_lua_report(log, L, status, chunkname); } diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index 4741c72267..f2fe165bd5 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -171,7 +171,8 @@ ngx_http_lua_header_filter_inline(ngx_http_request_t *r) llcf->header_filter_src.value.len, &llcf->header_filter_src_ref, llcf->header_filter_src_key, - "=header_filter_by_lua"); + (const char *) + llcf->header_filter_chunkname); if (rc != NGX_OK) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_initby.c b/src/ngx_http_lua_initby.c index e8941da6ca..bfd88c52ad 100644 --- a/src/ngx_http_lua_initby.c +++ b/src/ngx_http_lua_initby.c @@ -18,12 +18,21 @@ ngx_http_lua_init_by_inline(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, lua_State *L) { int status; + const char *chunkname; + status = luaL_loadbuffer(L, (char *) lmcf->init_src.data, lmcf->init_src.len, "=init_by_lua") || ngx_http_lua_do_call(log, L); - return ngx_http_lua_report(log, L, status, "init_by_lua"); + if (lmcf->init_chunkname == NULL) { + chunkname = "init_by_lua"; + + } else { + chunkname = (const char *) lmcf->init_chunkname; + } + + return ngx_http_lua_report(log, L, status, chunkname); } diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 94de796ade..e559c89613 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -317,12 +317,20 @@ ngx_http_lua_init_worker_by_inline(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, lua_State *L) { int status; + const char *chunkname; status = luaL_loadbuffer(L, (char *) lmcf->init_worker_src.data, lmcf->init_worker_src.len, "=init_worker_by_lua") || ngx_http_lua_do_call(log, L); - return ngx_http_lua_report(log, L, status, "init_worker_by_lua"); + if (lmcf->init_worker_chunkname == NULL) { + chunkname = "init_worker_by_lua"; + + } else { + chunkname = (const char *) lmcf->init_worker_chunkname; + } + + return ngx_http_lua_report(log, L, status, chunkname); } diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index fbeba12a7f..09fd0a91f8 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1103,22 +1103,27 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) /* set by ngx_pcalloc: * lscf->srv.ssl_client_hello_handler = NULL; * lscf->srv.ssl_client_hello_src = { 0, NULL }; + * lscf->srv.ssl_client_hello_chunkname = NULL; * lscf->srv.ssl_client_hello_src_key = NULL; * * lscf->srv.ssl_cert_handler = NULL; * lscf->srv.ssl_cert_src = { 0, NULL }; + * lscf->srv.ssl_cert_chunkname = NULL; * lscf->srv.ssl_cert_src_key = NULL; * * lscf->srv.ssl_session_store_handler = NULL; * lscf->srv.ssl_session_store_src = { 0, NULL }; + * lscf->srv.ssl_session_store_chunkname = NULL; * lscf->srv.ssl_session_store_src_key = NULL; * * lscf->srv.ssl_session_fetch_handler = NULL; * lscf->srv.ssl_session_fetch_src = { 0, NULL }; + * lscf->srv.ssl_session_fetch_chunkname = NULL; * lscf->srv.ssl_session_fetch_src_key = NULL; * * lscf->balancer.handler = NULL; * lscf->balancer.src = { 0, NULL }; + * lscf->balancer.chunkname = NULL; * lscf->balancer.src_key = NULL; */ @@ -1151,6 +1156,8 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) conf->srv.ssl_client_hello_src_ref = prev->srv.ssl_client_hello_src_ref; conf->srv.ssl_client_hello_src_key = prev->srv.ssl_client_hello_src_key; conf->srv.ssl_client_hello_handler = prev->srv.ssl_client_hello_handler; + conf->srv.ssl_client_hello_chunkname + = prev->srv.ssl_client_hello_chunkname; } if (conf->srv.ssl_client_hello_src.len) { @@ -1190,6 +1197,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) conf->srv.ssl_cert_src_ref = prev->srv.ssl_cert_src_ref; conf->srv.ssl_cert_src_key = prev->srv.ssl_cert_src_key; conf->srv.ssl_cert_handler = prev->srv.ssl_cert_handler; + conf->srv.ssl_cert_chunkname = prev->srv.ssl_cert_chunkname; } if (conf->srv.ssl_cert_src.len) { @@ -1229,6 +1237,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) conf->srv.ssl_sess_store_src_ref = prev->srv.ssl_sess_store_src_ref; conf->srv.ssl_sess_store_src_key = prev->srv.ssl_sess_store_src_key; conf->srv.ssl_sess_store_handler = prev->srv.ssl_sess_store_handler; + conf->srv.ssl_sess_store_chunkname = prev->srv.ssl_sess_store_chunkname; } if (conf->srv.ssl_sess_store_src.len) { @@ -1252,6 +1261,7 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) conf->srv.ssl_sess_fetch_src_ref = prev->srv.ssl_sess_fetch_src_ref; conf->srv.ssl_sess_fetch_src_key = prev->srv.ssl_sess_fetch_src_key; conf->srv.ssl_sess_fetch_handler = prev->srv.ssl_sess_fetch_handler; + conf->srv.ssl_sess_fetch_chunkname = prev->srv.ssl_sess_fetch_chunkname; } if (conf->srv.ssl_sess_fetch_src.len) { @@ -1393,6 +1403,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) conf->header_filter_handler = prev->header_filter_handler; conf->header_filter_src_ref = prev->header_filter_src_ref; conf->header_filter_src_key = prev->header_filter_src_key; + conf->header_filter_chunkname = prev->header_filter_chunkname; } if (conf->body_filter_src.value.len == 0) { @@ -1400,6 +1411,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) conf->body_filter_handler = prev->body_filter_handler; conf->body_filter_src_ref = prev->body_filter_src_ref; conf->body_filter_src_key = prev->body_filter_src_key; + conf->body_filter_chunkname = prev->body_filter_chunkname; } #if (NGX_HTTP_SSL) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index ce7f6483af..0499bfb355 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -70,7 +70,8 @@ ngx_http_lua_ssl_cert_handler_inline(ngx_http_request_t *r, lscf->srv.ssl_cert_src.len, &lscf->srv.ssl_cert_src_ref, lscf->srv.ssl_cert_src_key, - "=ssl_certificate_by_lua"); + (const char *) + lscf->srv.ssl_cert_chunkname); if (rc != NGX_OK) { return rc; } @@ -115,6 +116,8 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, #else + size_t chunkname_len; + u_char *chunkname; u_char *cache_key = NULL; u_char *name; ngx_str_t *value; @@ -163,8 +166,15 @@ ngx_http_lua_ssl_cert_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_certificate_by_lua", + sizeof("ssl_certificate_by_lua") - 1, &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + /* Don't eval nginx variables for inline lua code */ lscf->srv.ssl_cert_src = value[1]; + lscf->srv.ssl_cert_chunkname = chunkname; } lscf->srv.ssl_cert_src_key = cache_key; diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index a65b6e88f3..10500d89e3 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -61,7 +61,7 @@ ngx_http_lua_ssl_client_hello_handler_inline(ngx_http_request_t *r, lscf->srv.ssl_client_hello_src.len, &lscf->srv.ssl_client_hello_src_ref, lscf->srv.ssl_client_hello_src_key, - "=ssl_client_hello_by_lua"); + (const char *) lscf->srv.ssl_client_hello_chunkname); if (rc != NGX_OK) { return rc; } @@ -106,6 +106,8 @@ ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, #else + size_t chunkname_len; + u_char *chunkname; u_char *cache_key = NULL; u_char *name; ngx_str_t *value; @@ -154,9 +156,16 @@ ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_client_hello_by_lua", + sizeof("ssl_client_helloo_by_lua")- 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + /* Don't eval nginx variables for inline lua code */ lscf->srv.ssl_client_hello_src = value[1]; - + lscf->srv.ssl_client_hello_chunkname = chunkname; } lscf->srv.ssl_client_hello_src_key = cache_key; diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 6584e6ab58..66e9848550 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -66,7 +66,7 @@ ngx_http_lua_ssl_sess_fetch_handler_inline(ngx_http_request_t *r, lscf->srv.ssl_sess_fetch_src.len, &lscf->srv.ssl_sess_fetch_src_ref, lscf->srv.ssl_sess_fetch_src_key, - "=ssl_session_fetch_by_lua_block"); + (const char *) lscf->srv.ssl_sess_fetch_chunkname); if (rc != NGX_OK) { return rc; } @@ -102,6 +102,8 @@ char * ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; + u_char *chunkname; u_char *cache_key = NULL; u_char *name; ngx_str_t *value; @@ -153,8 +155,15 @@ ngx_http_lua_ssl_sess_fetch_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_session_fetch_by_lua", + sizeof("ssl_session_fetch_by_lua") - 1, &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + /* Don't eval nginx variables for inline lua code */ lscf->srv.ssl_sess_fetch_src = value[1]; + lscf->srv.ssl_sess_fetch_chunkname = chunkname; } lscf->srv.ssl_sess_fetch_src_key = cache_key; diff --git a/src/ngx_http_lua_ssl_session_storeby.c b/src/ngx_http_lua_ssl_session_storeby.c index 0654d93715..b25cbcc2c4 100644 --- a/src/ngx_http_lua_ssl_session_storeby.c +++ b/src/ngx_http_lua_ssl_session_storeby.c @@ -64,7 +64,7 @@ ngx_http_lua_ssl_sess_store_handler_inline(ngx_http_request_t *r, lscf->srv.ssl_sess_store_src.len, &lscf->srv.ssl_sess_store_src_ref, lscf->srv.ssl_sess_store_src_key, - "=ssl_session_store_by_lua_block"); + (const char *) lscf->srv.ssl_sess_store_chunkname); if (rc != NGX_OK) { return rc; } @@ -100,6 +100,8 @@ char * ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + size_t chunkname_len; + u_char *chunkname; u_char *cache_key = NULL; u_char *name; ngx_str_t *value; @@ -151,8 +153,15 @@ ngx_http_lua_ssl_sess_store_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_ERROR; } + chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_session_store_by_lua", + sizeof("ssl_session_store_by_lua") - 1, &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + /* Don't eval nginx variables for inline lua code */ lscf->srv.ssl_sess_store_src = value[1]; + lscf->srv.ssl_sess_store_chunkname = chunkname; } lscf->srv.ssl_sess_store_src_key = cache_key; diff --git a/t/001-set.t b/t/001-set.t index 62dec1a7f9..db18f0258c 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -573,7 +573,7 @@ GET /lua --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log -failed to run set_by_lua*: set_by_lua:1: Bad +failed to run set_by_lua*: set_by_lua(nginx.conf:40):1: Bad diff --git a/t/002-content.t b/t/002-content.t index 703ba84a26..661c2e39e3 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -10,7 +10,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); #repeat_each(1); -plan tests => repeat_each() * (blocks() * 2 + 27); +plan tests => repeat_each() * (blocks() * 2 + 32); #no_diff(); #no_long_string(); @@ -917,3 +917,172 @@ GET /lua --- error_code: 500 --- error_log eval qr/failed to load inlined Lua code: content_by_lua\(nginx.conf:52\)/ + + + +=== TEST 46: syntax error in included file +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /bar { + content_by_lua_block { + 'for end'; + } + } + + include ../html/lua.conf; +--- user_files +>>> lua.conf + location /lua { + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +failed to load inlined Lua code: content_by_lua(../html/lua.conf:2):2: unexpected symbol near ''for end'' + + + +=== TEST 47: syntax error with very long filename +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /bar { + content_by_lua_block { + 'for end'; + } + } + + include ../html/1234567890123456789012345678901234.conf; +--- user_files +>>> 1234567890123456789012345678901234.conf + location /lua { + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +failed to load inlined Lua code: content_by_lua(...234567890123456789012345678901234.conf:2) + + + +=== TEST 48: syntax error in /tmp/lua.conf +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /bar { + content_by_lua_block { + 'for end'; + } + } + + include /tmp/lua.conf; +--- user_files +>>> /tmp/lua.conf + location /lua { + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +failed to load inlined Lua code: content_by_lua(/tmp/lua.conf:2) + + + +=== TEST 49: syntax error in /tmp/12345678901234567890123456789012345.conf +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /bar { + content_by_lua_block { + 'for end'; + } + } + + include /tmp/12345678901234567890123456789012345.conf; + +--- user_files +>>> /tmp/12345678901234567890123456789012345.conf + location /lua { + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +failed to load inlined Lua code: content_by_lua(...345678901234567890123456789012345.conf:2) + + + +=== TEST 50: the error line number greater than 9 +--- config + location /foo { + content_by_lua_block { + 'for end'; + } + } + + location /bar { + content_by_lua_block { + 'for end'; + } + } + + include /tmp/12345678901234567890123456789012345.conf; + +--- user_files +>>> /tmp/12345678901234567890123456789012345.conf + location /lua { + + + + + + + + + + + + + content_by_lua_block { + 'for end'; + } + } +--- request +GET /lua +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log +failed to load inlined Lua code: content_by_lua(...45678901234567890123456789012345.conf:14) diff --git a/t/009-log.t b/t/009-log.t index c4597698ef..4446c71277 100644 --- a/t/009-log.t +++ b/t/009-log.t @@ -243,7 +243,7 @@ GET /log --- response_body 32 --- error_log eval -qr/\[error\] \S+: \S+ \[lua\] set_by_lua:2: HELLO,/ +qr/\[error\] \S+: \S+ \[lua\] set_by_lua\(nginx.conf:43\):2: HELLO,/ @@ -261,7 +261,7 @@ GET /log --- response_body 32 --- error_log eval -qr/\[error\] \S+: \S+ \[lua\] set_by_lua:2: truefalsenil,/ +qr/\[error\] \S+: \S+ \[lua\] set_by_lua\(nginx.conf:43\):2: truefalsenil,/ @@ -368,7 +368,7 @@ GET /log --- response_headers foo: 32 --- error_log eval -qr/\[notice\] .*? \[lua\] header_filter_by_lua:2: hello world/ +qr/\[notice\] .*? \[lua\] header_filter_by_lua\(nginx.conf:43\):2: hello world/ --- response_body hi @@ -390,7 +390,7 @@ foo: 32 --- response_body hi --- error_log eval -qr/\[error\] .*? \[lua\] header_filter_by_lua:2: howdy, lua!/ +qr/\[error\] .*? \[lua\] header_filter_by_lua\(nginx.conf:43\):2: howdy, lua!/ diff --git a/t/025-codecache.t b/t/025-codecache.t index cf62ff0182..ebe4f61b7b 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1862,17 +1862,17 @@ code cache hit (key='log_by_lua_nhli_8a9441d0a30531ba8bb34ab11c55cfc3', ref=12) "] --- error_log eval [ -qr/balancer_by_lua:\d+: hello/, -qr/ssl_session_fetch_by_lua_block:\d+: hello/, -qr/ssl_certificate_by_lua:\d+: hello/, -qr/ssl_session_store_by_lua_block:\d+: hello/, -qr/set_by_lua:\d+: hello/, +qr/balancer_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/ssl_session_fetch_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/ssl_certificate_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/ssl_session_store_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/set_by_lua\(nginx\.conf:\d+\):\d+: hello/, qr/rewrite_by_lua\(nginx\.conf:\d+\):\d+: hello/, qr/access_by_lua\(nginx\.conf:\d+\):\d+: hello/, qr/content_by_lua\(nginx\.conf:\d+\):\d+: hello/, -qr/header_filter_by_lua:\d+: hello/, -qr/body_filter_by_lua:\d+: hello/, -qr/log_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/header_filter_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/body_filter_by_lua\(nginx\.conf:\d+\):\d+: hello/, +qr/log_by_lua\(nginx.conf:\d+\):\d+: hello/, ] --- log_level: debug --- no_error_log diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 0adc699c6c..cfc6f53234 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -462,7 +462,7 @@ GET /lua GET /lua --- ignore_response --- error_log -failed to run header_filter_by_lua*: header_filter_by_lua:2: Something bad +failed to run header_filter_by_lua*: header_filter_by_lua(nginx.conf:47):2: Something bad --- no_error_log [alert] @@ -799,3 +799,53 @@ GET /t --- error_code: 302 --- no_error_log [error] + + + +=== TEST 42: syntax error in header_filter_by_lua_block +--- config + location /lua { + + header_filter_by_lua_block { + 'for end'; + } + content_by_lua_block { + ngx.say("Hello world") + } + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: header_filter_by_lua(nginx.conf:41):2: unexpected symbol near ''for end'' +--- no_error_log +no_such_error + + + +=== TEST 43: syntax error in second content_by_lua_block +--- config + location /foo { + header_filter_by_lua_block { + 'for end'; + } + content_by_lua_block { + ngx.say("Hello world") + } + } + + location /lua { + header_filter_by_lua_block { + 'for end'; + } + content_by_lua_block { + ngx.say("Hello world") + } + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: header_filter_by_lua(nginx.conf:49):2: unexpected symbol near ''for end'' +--- no_error_log +no_such_error diff --git a/t/082-body-filter.t b/t/082-body-filter.t index 5f765fa439..a69c78fa70 100644 --- a/t/082-body-filter.t +++ b/t/082-body-filter.t @@ -317,7 +317,7 @@ hiya globe GET /t --- ignore_response --- error_log -failed to run body_filter_by_lua*: body_filter_by_lua:4: something bad happened! +failed to run body_filter_by_lua*: body_filter_by_lua(nginx.conf:49):4: something bad happened! @@ -838,3 +838,55 @@ GET /lua --- ignore_response --- error_log API disabled in the context of body_filter_by_lua* + + + +=== TEST 27: syntax error in body_filter_by_lua_block +--- config + location /lua { + + body_filter_by_lua_block { + 'for end'; + } + content_by_lua_block { + ngx.say("Hello world") + } + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: body_filter_by_lua(nginx.conf:41):2: unexpected symbol near ''for end'' +--- no_error_log +no_such_error1 +no_such_error2 + + + +=== TEST 28: syntax error in second body_by_lua_block +--- config + location /foo { + body_filter_by_lua_block { + 'for end'; + } + content_by_lua_block { + ngx.say("Hello world") + } + } + + location /lua { + body_filter_by_lua_block { + 'for end'; + } + content_by_lua_block { + ngx.say("Hello world") + } + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: body_filter_by_lua(nginx.conf:49):2: unexpected symbol near ''for end'' +--- no_error_log +no_such_error1 +no_such_error2 diff --git a/t/086-init-by.t b/t/086-init-by.t index bea34a461a..8d9a8596b4 100644 --- a/t/086-init-by.t +++ b/t/086-init-by.t @@ -321,3 +321,47 @@ INIT 2: foo = 3 failed to init --- error_log [error] + + + +=== TEST 13: syntax error in init_by_lua_block +--- http_config + init_by_lua_block { + ngx.log(ngx.debug, "pass") + error("failed to init" + ngx.log(ngx.debug, "unreachable") + } +--- config + location /lua { + content_by_lua_block { + ngx.say("hello world") + } + } +--- must_die +--- error_log +=init_by_lua(nginx.conf:25) error: init_by_lua:4: ')' expected (to close '(' at line 3) near 'ngx' +--- no_error_log +no_such_error_log + + + +=== TEST 14: syntax error in init_by_lua_file +--- http_config + init_by_lua_file html/init.lua; +--- config + location /lua { + content_by_lua_block { + ngx.say("hello world") + } + } +--- user_files +>>> init.lua + ngx.log(ngx.debug, "pass") + error("failed to init" + ngx.log(ngx.debug, "unreachable") + +--- must_die +--- error_log eval +qr|init_by_lua_file error: .*lua-nginx-module/t/servroot/html/init.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| +--- no_error_log +no_such_error_log diff --git a/t/091-coroutine.t b/t/091-coroutine.t index b9b3d9d38d..74b7c682fe 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -1691,7 +1691,7 @@ GET /t --- error_log eval [ qr/\[notice\] .*? in wrapped coroutine/, - qr/\[error\] .*? failed to run header_filter_by_lua\*: header_filter_by_lua:\d+: header_filter_by_lua:\d+: something went wrong/, + qr/\[error\] .*? failed to run header_filter_by_lua\*: header_filter_by_lua\(nginx.conf:\d+\):\d+: header_filter_by_lua\(nginx.conf:\d+\):\d+: something went wrong/, "stack traceback:", "in function 'co'" ] @@ -1713,9 +1713,9 @@ GET /t --- config --- must_die ---- grep_error_log eval: qr/init_by_lua error: .*? something went wrong/ +--- grep_error_log eval: qr/init_by_lua\(nginx.conf:25\) error: .*? something went wrong/ --- grep_error_log_out -init_by_lua error: init_by_lua:7: init_by_lua:4: something went wrong +init_by_lua(nginx.conf:25) error: init_by_lua:7: init_by_lua:4: something went wrong diff --git a/t/124-init-worker.t b/t/124-init-worker.t index b7e17a07f1..d10ee17285 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -975,3 +975,53 @@ qr/lua close the global Lua VM ([0-9A-F]+)$/, --- no_error_log [error] start privileged agent process + + + +=== TEST 25: syntax error in init_worker_by_lua_block +--- http_config + init_worker_by_lua_block { + ngx.log(ngx.debug, "pass") + error("failed to init" + ngx.log(ngx.debug, "unreachable") + } +--- config + location /t { + content_by_lua_block { + ngx.say("hello world") + } + } +--- request + GET /t +--- response_body +hello world +--- error_log +=init_worker_by_lua(nginx.conf:25) error: init_worker_by_lua:4: ')' expected (to close '(' at line 3) near 'ngx' +--- no_error_log +no_such_error_log + + + +=== TEST 26: syntax error in init_worker_by_lua_file +--- http_config + init_worker_by_lua_file html/init.lua; +--- config + location /t { + content_by_lua_block { + ngx.say("hello world") + } + } +--- user_files +>>> init.lua + ngx.log(ngx.debug, "pass") + error("failed to init" + ngx.log(ngx.debug, "unreachable") + +--- request + GET /t +--- response_body +hello world +--- error_log eval +qr|init_worker_by_lua_file error: .*lua-nginx-module/t/servroot/html/init.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| +--- no_error_log +no_such_error_log diff --git a/t/138-balancer.t b/t/138-balancer.t index dca1784673..2894157e0e 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -38,7 +38,7 @@ __DATA__ --- error_code: 502 --- error_log eval [ -'[lua] balancer_by_lua:2: hello from balancer by lua! while connecting to upstream,', +'[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] --- no_error_log @@ -64,7 +64,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ --- response_body_like: 403 Forbidden --- error_code: 403 --- error_log -[lua] balancer_by_lua:2: hello from balancer by lua! while connecting to upstream, +[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream, --- no_error_log eval [ '[warn]', @@ -92,7 +92,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ --- error_code: 502 --- error_log eval [ -'[lua] balancer_by_lua:2: hello from balancer by lua! while connecting to upstream,', +'[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] --- no_error_log @@ -257,7 +257,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log eval -qr/\[error\] .*? failed to run balancer_by_lua\*: balancer_by_lua:2: API disabled in the context of balancer_by_lua\*/ +qr/\[error\] .*? failed to run balancer_by_lua\*: balancer_by_lua\(nginx\.conf:27\):2: API disabled in the context of balancer_by_lua\*/ @@ -278,7 +278,7 @@ qr/\[error\] .*? failed to run balancer_by_lua\*: balancer_by_lua:2: API disable --- response_body_like: 500 Internal Server Error --- error_code: 500 --- error_log eval -qr/\[error\] .*? failed to run balancer_by_lua\*: balancer_by_lua:2: API disabled in the context of balancer_by_lua\*/ +qr/\[error\] .*? failed to run balancer_by_lua\*: balancer_by_lua\(nginx\.conf:27\):2: API disabled in the context of balancer_by_lua\*/ @@ -427,7 +427,7 @@ ctx counter: nil --- error_code: 502 --- error_log eval [ -'[lua] balancer_by_lua:2: hello from balancer by lua! while connecting to upstream,', +'[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] --- no_error_log @@ -567,3 +567,26 @@ connect() failed (111: Connection refused) while connecting to upstream --- no_error_log upstream sent more data than specified in "Content-Length" header while reading upstream [alert] + + + +=== TEST 18: error in balancer_by_llua_block +--- http_config + upstream backend { + server 0.0.0.1; + balancer_by_lua_block { + ngx.say("hello" + } + } +--- config + location = /t { + proxy_pass http://backend; + } +--- request + GET /t +--- response_body_like: 500 Internal Server Error +--- error_code: 500 +--- error_log eval + "failed to load inlined Lua code: balancer_by_lua(nginx.conf:27):3: ')' expected (to close '(' at line 2) near ''", +--- no_error_log +[warn] diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index d7e20bafea..38296352f5 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -117,18 +117,18 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] ---- grep_error_log eval: qr/ssl_certificate_by_lua:.*?,|\bssl cert: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log eval: qr/ssl_certificate_by_lua\(nginx.conf:\d+\):.*?,|\bssl cert: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval # Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) # before call ssl callback function $Test::Nginx::Util::NginxVersion >= 1.017009 ? qr/reusable connection: 0 ssl cert: connection reusable: 0 -ssl_certificate_by_lua:1: ssl cert by lua is running!,/ +ssl_certificate_by_lua\(nginx.conf:28\):1: ssl cert by lua is running!,/ : qr /reusable connection: 1 ssl cert: connection reusable: 1 reusable connection: 0 -ssl_certificate_by_lua:1: ssl cert by lua is running!,/ +ssl_certificate_by_lua\(nginx.conf:28\):1: ssl cert by lua is running!,/ @@ -789,7 +789,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'runtime error: ssl_certificate_by_lua:2: bad bad bad', +'runtime error: ssl_certificate_by_lua(nginx.conf:28):2: bad bad bad', 'lua_certificate_by_lua: handler return value: 500, cert cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, qr/context: ssl_certificate_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, @@ -861,7 +861,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'runtime error: ssl_certificate_by_lua:3: bad bad bad', +'runtime error: ssl_certificate_by_lua(nginx.conf:28):3: bad bad bad', 'lua_certificate_by_lua: cert cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, ] @@ -1050,7 +1050,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua ssl server name: "test.com"', -'ssl_certificate_by_lua:1: API disabled in the context of ssl_certificate_by_lua*', +'ssl_certificate_by_lua(nginx.conf:28):1: API disabled in the context of ssl_certificate_by_lua*', qr/\[info\] .*?cert cb error/, ] @@ -1481,7 +1481,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_certificate_by_lua:1: ssl cert by lua is running! +ssl_certificate_by_lua(nginx.conf:25):1: ssl cert by lua is running! --- no_error_log [error] @@ -1681,7 +1681,7 @@ close: 1 nil --- error_log eval [ -'ssl_certificate_by_lua:1: ssl cert by lua is running!', +'ssl_certificate_by_lua(nginx.conf:29):1: ssl cert by lua is running!', 'lua ssl server name: "test.com"', ] --- no_error_log diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 14e90574b0..6769a336bc 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -86,11 +86,11 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] ---- grep_error_log eval: qr/ssl_session_store_by_lua_block:.*?,|\bssl session store: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log eval: qr/ssl_session_store_by_lua\(nginx.conf:\d+\):.*?,|\bssl session store: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval qr/^reusable connection: 0 ssl session store: connection reusable: 0 -ssl_session_store_by_lua_block:1: ssl session store by lua is running!, +ssl_session_store_by_lua\(nginx\.conf:25\):1: ssl session store by lua is running!, /m, @@ -536,7 +536,7 @@ ssl handshake: userdata close: 1 nil --- error_log -failed to run session_store_by_lua*: ssl_session_store_by_lua_block:2: bad bad bad +failed to run session_store_by_lua*: ssl_session_store_by_lua(nginx.conf:25):2: bad bad bad --- no_error_log should never reached here @@ -678,7 +678,7 @@ close: 1 nil [ 'lua ssl server name: "test.com"', qr/elapsed in ssl cert by lua: 0.(?:09|1[01])\d+,/, -'ssl_session_store_by_lua_block:1: ssl store session by lua is running!', +'ssl_session_store_by_lua(nginx.conf:25):1: ssl store session by lua is running!', ] --- no_error_log @@ -896,7 +896,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_session_store_by_lua_block:1: ssl session store by lua is running! +ssl_session_store_by_lua(nginx.conf:25):1: ssl session store by lua is running! --- no_error_log [error] diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 084e7d596c..0e90c23665 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -83,7 +83,7 @@ connected: 1 ssl handshake: userdata close: 1 nil ---- grep_error_log eval: qr/ssl_session_fetch_by_lua_block:.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log eval: qr/ssl_session_fetch_by_lua\(nginx\.conf:\d+\):.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval # Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) @@ -93,11 +93,11 @@ $Test::Nginx::Util::NginxVersion >= 1.017009 ? qr/\A(?:reusable connection: [01]\n)+\z/s, qr/^reusable connection: 0 ssl session fetch: connection reusable: 0 -ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, /m, qr/^reusable connection: 0 ssl session fetch: connection reusable: 0 -ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, /m, ] : @@ -106,12 +106,12 @@ qr/\A(?:reusable connection: [01]\n)+\z/s, qr/^reusable connection: 1 ssl session fetch: connection reusable: 1 reusable connection: 0 -ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, /m, qr/^reusable connection: 1 ssl session fetch: connection reusable: 1 reusable connection: 0 -ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!, +ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, /m, ] --- no_error_log @@ -694,14 +694,14 @@ ssl handshake: userdata close: 1 nil --- grep_error_log eval -qr/ssl_session_fetch_by_lua_block:2: bad bad bad/s +qr/ssl_session_fetch_by_lua\(nginx.conf:\d+\):2: bad bad bad/s --- grep_error_log_out eval [ '', -'ssl_session_fetch_by_lua_block:2: bad bad bad +'ssl_session_fetch_by_lua(nginx.conf:25):2: bad bad bad ', -'ssl_session_fetch_by_lua_block:2: bad bad bad +'ssl_session_fetch_by_lua(nginx.conf:25):2: bad bad bad ', ] @@ -777,15 +777,15 @@ ssl handshake: userdata close: 1 nil --- grep_error_log eval -qr/ssl_session_fetch_by_lua_block:3: bad bad bad|ssl_session_fetch_by_lua\*: sess get cb exit code: 0/s +qr/ssl_session_fetch_by_lua\(nginx.conf:\d+\):3: bad bad bad|ssl_session_fetch_by_lua\*: sess get cb exit code: 0/s --- grep_error_log_out eval [ '', -'ssl_session_fetch_by_lua_block:3: bad bad bad +'ssl_session_fetch_by_lua(nginx.conf:25):3: bad bad bad ssl_session_fetch_by_lua*: sess get cb exit code: 0 ', -'ssl_session_fetch_by_lua_block:3: bad bad bad +'ssl_session_fetch_by_lua(nginx.conf:25):3: bad bad bad ssl_session_fetch_by_lua*: sess get cb exit code: 0 ', @@ -1113,14 +1113,14 @@ ssl handshake: userdata close: 1 nil --- grep_error_log eval -qr/ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running!/s +qr/ssl_session_fetch_by_lua\(nginx.conf:\d+\):1: ssl fetch sess by lua is running!/s --- grep_error_log_out eval [ '', -'ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running! +'ssl_session_fetch_by_lua(nginx.conf:25):1: ssl fetch sess by lua is running! ', -'ssl_session_fetch_by_lua_block:1: ssl fetch sess by lua is running! +'ssl_session_fetch_by_lua(nginx.conf:25):1: ssl fetch sess by lua is running! ', ] @@ -1348,7 +1348,7 @@ GET /t connected: 1 ssl handshake: userdata close: 1 nil ---- grep_error_log eval: qr/ssl_session_fetch_by_lua_block:.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log eval: qr/ssl_session_fetch_by_lua\(nginx\.conf:\d+\):.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval # Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) # before call ssl callback function @@ -1357,11 +1357,11 @@ $Test::Nginx::Util::NginxVersion >= 1.017009 ? qr/\A(?:reusable connection: [01]\n)+\z/s, qr/^reusable connection: 0 ssl session fetch: connection reusable: 0 -ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, /m, qr/^reusable connection: 0 ssl session fetch: connection reusable: 0 -ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, /m, ] : @@ -1370,12 +1370,12 @@ qr/\A(?:reusable connection: [01]\n)+\z/s, qr/^reusable connection: 1 ssl session fetch: connection reusable: 1 reusable connection: 0 -ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, /m, qr/^reusable connection: 1 ssl session fetch: connection reusable: 1 reusable connection: 0 -ssl_session_fetch_by_lua_block:1: ssl_session_fetch_by_lua\* is running!, +ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, /m, ] --- no_error_log diff --git a/t/158-global-var.t b/t/158-global-var.t index def3b7418f..3f5db60fa2 100644 --- a/t/158-global-var.t +++ b/t/158-global-var.t @@ -60,10 +60,10 @@ __DATA__ --- response_body_like chomp \A[12]\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|set_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|set_by_lua\(nginx.conf:\d+\):\d+: in main chunk, )/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -set_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] +set_by_lua\(nginx.conf:40\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -154,10 +154,10 @@ content_by_lua\(nginx\.conf:\d+\):\d+: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|1)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk, )/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(nginx\.conf:\d+\):\d+: in main chunk, )/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] +header_filter_by_lua\(nginx.conf:43\):3: in main chunk, \n\z/, "old foo: 1\n"] @@ -179,10 +179,10 @@ header_filter_by_lua:3: in main chunk, \n\z/, "old foo: 1\n"] --- response_body_like chomp \A(?:nil|2)\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk,)/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(nginx\.conf:\d+\):\d+: in main chunk,)/ --- grep_error_log_out eval [qr/\[warn\] .*?writing a global Lua variable \('foo'\) -body_filter_by_lua:3: in main chunk, +body_filter_by_lua\(nginx.conf:43\):3: in main chunk, old foo: 1\n\z/, "old foo: 2\nold foo: 3\n"] @@ -297,10 +297,10 @@ log_by_lua\(nginx\.conf:\d+\):\d+: in main chunk\n\z/, "old foo: 1\n"] --- response_body_like chomp \A[12]done\n\z --- grep_error_log eval -qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua:\d+: in main chunk)/ +qr/(old foo: \d+|\[\w+\].*?writing a global Lua variable \('[^'\s]+'\)|\w+_by_lua\(nginx.conf:\d+\):\d+: in main chunk)/ --- grep_error_log_out eval [qr/\A\[warn\] .*?writing a global Lua variable \('foo'\) -ssl_certificate_by_lua:3: in main chunk\n\z/, "old foo: 1\n"] +ssl_certificate_by_lua\(nginx.conf:28\):3: in main chunk\n\z/, "old foo: 1\n"] diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index 3138617615..fcae47a4ee 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -6,7 +6,7 @@ master_on(); repeat_each(2); # NB: the shutdown_error_log block is independent from repeat times -plan tests => repeat_each() * (blocks() * 2 + 1) + 13; +plan tests => repeat_each() * (blocks() * 2 + 1) + 15; #log_level("warn"); no_long_string(); @@ -194,3 +194,49 @@ qr/cache loader process \d+ exited/, qr/cache manager process \d+ exited/, qr/hello from exit worker by lua, process type: worker/, ] + + + +=== TEST 8: syntax error in exit_worker_by_lua_block +--- http_config + exit_worker_by_lua_block { + ngx.log(ngx.debug, "pass") + error("failed to init" + ngx.log(ngx.debug, "unreachable") + } +--- config + location /t { + content_by_lua_block { + ngx.say("hello world") + } + } +--- request + GET /t +--- response_body +hello world +--- shutdown_error_log +=exit_worker_by_lua(nginx.conf:25) error: exit_worker_by_lua:4: ')' expected (to close '(' at line 3) near 'ngx' + + + +=== TEST 9: syntax error in exit_worker_by_lua_file +--- http_config + exit_worker_by_lua_file html/exit.lua; +--- config + location /t { + content_by_lua_block { + ngx.say("hello world") + } + } +--- user_files +>>> exit.lua + ngx.log(ngx.debug, "pass") + error("failed to init" + ngx.log(ngx.debug, "unreachable") + +--- request + GET /t +--- response_body +hello world +--- shutdown_error_log eval +qr|exit_worker_by_lua_file error: .*lua-nginx-module/t/servroot/html/exit.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| From 15b693b3381f6115fa8a7870e6cd313c608b6cdb Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 14 Feb 2022 17:48:42 +0800 Subject: [PATCH 600/848] bugfix: passing the wrong chunkname argument to luaL_loadbuffer. (#2010) The total length of the chunkname is 60. The format is "=tag(file:line)". So the space left for the tag, file and line is only 56 chars. --- src/ngx_http_lua_directive.c | 17 ++++++++++------- src/ngx_http_lua_exitworkerby.c | 12 ++++++------ src/ngx_http_lua_initby.c | 12 ++++++------ src/ngx_http_lua_initworkerby.c | 12 ++++++------ t/041-header-filter.t | 24 ++++++++++++++++++++++++ t/086-init-by.t | 2 +- t/089-phase.t | 2 +- t/091-coroutine.t | 4 ++-- t/124-init-worker.t | 2 +- t/162-exit-worker.t | 2 +- 10 files changed, 58 insertions(+), 31 deletions(-) diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index dda447fb7e..c8317edf3d 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -32,7 +32,10 @@ #include "ngx_http_lua_log.h" -#define LJ_CHUNKNAME_MAX_LEN 42 +/* the max length is 60, after deducting the fixed four characters "=(:)" + * only 56 left. + */ +#define LJ_CHUNKNAME_MAX_LEN 56 typedef struct ngx_http_lua_block_parser_ctx_s @@ -1369,7 +1372,7 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, ngx_str_t *filename; u_char *filename_end; const char *pre_str = ""; - ngx_uint_t start_line_len; + ngx_uint_t reserve_len; ngx_http_lua_main_conf_t *lmcf; @@ -1385,7 +1388,7 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, start_line = lmcf->directive_line > 0 ? lmcf->directive_line : cf->conf_file->line; p = ngx_snprintf(out, len, "%d", start_line); - start_line_len = p - out; + reserve_len = tag_len + p - out; filename = &cf->conf_file->file.name; filename_end = filename->data + filename->len; @@ -1407,8 +1410,8 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, filename->data, conf_prefix->len) == 0) { /* files in conf_prefix directory, use the relative path */ - if (filename_end - p + start_line_len > LJ_CHUNKNAME_MAX_LEN) { - p = filename_end - LJ_CHUNKNAME_MAX_LEN + start_line_len + 3; + if (filename_end - p + reserve_len > LJ_CHUNKNAME_MAX_LEN) { + p = filename_end - LJ_CHUNKNAME_MAX_LEN + reserve_len + 3; pre_str = "..."; } @@ -1418,11 +1421,11 @@ ngx_http_lua_gen_chunk_name(ngx_conf_t *cf, const char *tag, size_t tag_len, p = filename->data; - if (filename->len + start_line_len <= LJ_CHUNKNAME_MAX_LEN) { + if (filename->len + reserve_len <= LJ_CHUNKNAME_MAX_LEN) { goto found; } - p = filename_end - LJ_CHUNKNAME_MAX_LEN + start_line_len + 3; + p = filename_end - LJ_CHUNKNAME_MAX_LEN + reserve_len + 3; pre_str = "..."; found: diff --git a/src/ngx_http_lua_exitworkerby.c b/src/ngx_http_lua_exitworkerby.c index 65edc04e63..e8f65110ae 100644 --- a/src/ngx_http_lua_exitworkerby.c +++ b/src/ngx_http_lua_exitworkerby.c @@ -97,18 +97,18 @@ ngx_http_lua_exit_worker_by_inline(ngx_log_t *log, int status; const char *chunkname; - status = luaL_loadbuffer(L, (char *) lmcf->exit_worker_src.data, - lmcf->exit_worker_src.len, "=exit_worker_by_lua") - || ngx_http_lua_do_call(log, L); - if (lmcf->exit_worker_chunkname == NULL) { - chunkname = "exit_worker_by_lua"; + chunkname = "=exit_worker_by_lua"; } else { chunkname = (const char *) lmcf->exit_worker_chunkname; } - return ngx_http_lua_report(log, L, status, chunkname); + status = luaL_loadbuffer(L, (char *) lmcf->exit_worker_src.data, + lmcf->exit_worker_src.len, chunkname) + || ngx_http_lua_do_call(log, L); + + return ngx_http_lua_report(log, L, status, "exit_worker_by_lua"); } diff --git a/src/ngx_http_lua_initby.c b/src/ngx_http_lua_initby.c index bfd88c52ad..1f956e7ebd 100644 --- a/src/ngx_http_lua_initby.c +++ b/src/ngx_http_lua_initby.c @@ -21,18 +21,18 @@ ngx_http_lua_init_by_inline(ngx_log_t *log, ngx_http_lua_main_conf_t *lmcf, const char *chunkname; - status = luaL_loadbuffer(L, (char *) lmcf->init_src.data, - lmcf->init_src.len, "=init_by_lua") - || ngx_http_lua_do_call(log, L); - if (lmcf->init_chunkname == NULL) { - chunkname = "init_by_lua"; + chunkname = "=init_by_lua"; } else { chunkname = (const char *) lmcf->init_chunkname; } - return ngx_http_lua_report(log, L, status, chunkname); + status = luaL_loadbuffer(L, (char *) lmcf->init_src.data, + lmcf->init_src.len, chunkname) + || ngx_http_lua_do_call(log, L); + + return ngx_http_lua_report(log, L, status, "init_by_lua"); } diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index e559c89613..3235a4d30a 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -319,18 +319,18 @@ ngx_http_lua_init_worker_by_inline(ngx_log_t *log, int status; const char *chunkname; - status = luaL_loadbuffer(L, (char *) lmcf->init_worker_src.data, - lmcf->init_worker_src.len, "=init_worker_by_lua") - || ngx_http_lua_do_call(log, L); - if (lmcf->init_worker_chunkname == NULL) { - chunkname = "init_worker_by_lua"; + chunkname = "=init_worker_by_lua"; } else { chunkname = (const char *) lmcf->init_worker_chunkname; } - return ngx_http_lua_report(log, L, status, chunkname); + status = luaL_loadbuffer(L, (char *) lmcf->init_worker_src.data, + lmcf->init_worker_src.len, chunkname) + || ngx_http_lua_do_call(log, L); + + return ngx_http_lua_report(log, L, status, "init_worker_by_lua"); } diff --git a/t/041-header-filter.t b/t/041-header-filter.t index cfc6f53234..be390b3259 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -849,3 +849,27 @@ GET /lua failed to load inlined Lua code: header_filter_by_lua(nginx.conf:49):2: unexpected symbol near ''for end'' --- no_error_log no_such_error + + + +=== TEST 44: syntax error in /tmp/12345678901234567890123456789012345.conf +--- config + location /lua { + content_by_lua_block { + ngx.say("Hello world") + } + + include /tmp/12345678901234567890123456789012345.conf; + } +--- user_files +>>> /tmp/12345678901234567890123456789012345.conf + header_filter_by_lua_block { + 'for end'; + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: header_filter_by_lua(...901234567890123456789012345.conf:1):2: unexpected symbol near ''for end'' +--- no_error_log +[alert] diff --git a/t/086-init-by.t b/t/086-init-by.t index 8d9a8596b4..a998fc53d6 100644 --- a/t/086-init-by.t +++ b/t/086-init-by.t @@ -339,7 +339,7 @@ failed to init } --- must_die --- error_log -=init_by_lua(nginx.conf:25) error: init_by_lua:4: ')' expected (to close '(' at line 3) near 'ngx' +init_by_lua error: init_by_lua(nginx.conf:25):4: ')' expected (to close '(' at line 3) near 'ngx' --- no_error_log no_such_error_log diff --git a/t/089-phase.t b/t/089-phase.t index a7405f2d54..355b7ac33f 100644 --- a/t/089-phase.t +++ b/t/089-phase.t @@ -198,6 +198,6 @@ GET /lua ok --- shutdown_error_log eval [ -qr/exit_worker_by_lua:\d+: exit_worker/, +qr/exit_worker_by_lua\(nginx\.conf:\d+\):\d+: exit_worker/, qr/exiting now$/, ] diff --git a/t/091-coroutine.t b/t/091-coroutine.t index 74b7c682fe..fafc5a45cc 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -1713,9 +1713,9 @@ GET /t --- config --- must_die ---- grep_error_log eval: qr/init_by_lua\(nginx.conf:25\) error: .*? something went wrong/ +--- grep_error_log eval: qr/init_by_lua\(nginx.conf:25\).*? something went wrong/ --- grep_error_log_out -init_by_lua(nginx.conf:25) error: init_by_lua:7: init_by_lua:4: something went wrong +init_by_lua(nginx.conf:25):7: init_by_lua(nginx.conf:25):4: something went wrong diff --git a/t/124-init-worker.t b/t/124-init-worker.t index d10ee17285..d06ec9937b 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -996,7 +996,7 @@ start privileged agent process --- response_body hello world --- error_log -=init_worker_by_lua(nginx.conf:25) error: init_worker_by_lua:4: ')' expected (to close '(' at line 3) near 'ngx' +init_worker_by_lua error: init_worker_by_lua(nginx.conf:25):4: ')' expected (to close '(' at line 3) near 'ngx' --- no_error_log no_such_error_log diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index fcae47a4ee..60145ee737 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -215,7 +215,7 @@ qr/hello from exit worker by lua, process type: worker/, --- response_body hello world --- shutdown_error_log -=exit_worker_by_lua(nginx.conf:25) error: exit_worker_by_lua:4: ')' expected (to close '(' at line 3) near 'ngx' +exit_worker_by_lua error: exit_worker_by_lua(nginx.conf:25):4: ')' expected (to close '(' at line 3) near 'ngx' From cee06592f8bcdb7977b1f77d5cac4e4c24e8b36d Mon Sep 17 00:00:00 2001 From: Noah Jahn <36347998+noahjahn@users.noreply.github.com> Date: Thu, 17 Feb 2022 22:39:21 -0500 Subject: [PATCH 601/848] doc: fixed link to lua-resty-core --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 5a87d0c45d..f1e7826dac 100644 --- a/README.markdown +++ b/README.markdown @@ -354,7 +354,7 @@ Alternatively, ngx_lua can be manually compiled into Nginx: 1. Download the latest version of the ngx_devel_kit (NDK) module [HERE](https://github.com/simplresty/ngx_devel_kit/tags) 1. Download the latest version of ngx_lua [HERE](https://github.com/openresty/lua-nginx-module/tags) 1. Download the latest supported version of Nginx [HERE](https://nginx.org/) (See [Nginx Compatibility](#nginx-compatibility)) -1. Download the latest version of the lua-resty-core [HERE](https://lua-resty-core) +1. Download the latest version of the lua-resty-core [HERE](https://github.com/openresty/lua-resty-core) 1. Download the latest version of the lua-resty-lrucache [HERE](https://github.com/openresty/lua-resty-lrucache) Build the source with this module: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 5c79dda3d6..a0111a36e5 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -285,7 +285,7 @@ Alternatively, ngx_lua can be manually compiled into Nginx: # Download the latest version of the ngx_devel_kit (NDK) module [https://github.com/simplresty/ngx_devel_kit/tags HERE] # Download the latest version of ngx_lua [https://github.com/openresty/lua-nginx-module/tags HERE] # Download the latest supported version of Nginx [https://nginx.org/ HERE] (See [[#Nginx Compatibility|Nginx Compatibility]]) -# Download the latest version of the lua-resty-core [HERE](https://lua-resty-core) +# Download the latest version of the lua-resty-core [HERE](https://github.com/openresty/lua-resty-core) # Download the latest version of the lua-resty-lrucache [HERE](https://github.com/openresty/lua-resty-lrucache) Build the source with this module: From 77540f78c33f27d5b3b93a485d43bb423bc83372 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 18 Feb 2022 14:19:29 +0800 Subject: [PATCH 602/848] doc: change description for ssl_client_hello_by_lua. --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index f1e7826dac..6583c0a406 100644 --- a/README.markdown +++ b/README.markdown @@ -2738,7 +2738,7 @@ This directive requires OpenSSL 1.1.1 or greater. If you are using the [official pre-built packages](https://openresty.org/en/linux-packages.html) for -[OpenResty](https://openresty.org/) 1.19.9.2 or later, then everything should +[OpenResty](https://openresty.org/) 1.21.4.1 or later, then everything should work out of the box. If you are not using one of the [OpenSSL @@ -2749,7 +2749,7 @@ in order to use this directive: Similarly, if you are not using the Nginx core shipped with -[OpenResty](https://openresty.org) 1.19.9.2 or later, you will need to apply +[OpenResty](https://openresty.org) 1.21.4.1 or later, you will need to apply patches to the standard Nginx core: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a0111a36e5..3b94c211cf 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2319,7 +2319,7 @@ This directive requires OpenSSL 1.1.1 or greater. If you are using the [official pre-built packages](https://openresty.org/en/linux-packages.html) for -[OpenResty](https://openresty.org/) 1.19.9.2 or later, then everything should +[OpenResty](https://openresty.org/) 1.21.4.1 or later, then everything should work out of the box. If you are not using one of the [OpenSSL @@ -2330,7 +2330,7 @@ in order to use this directive: https://openresty.org/en/openssl-patches.html Similarly, if you are not using the Nginx core shipped with -[OpenResty](https://openresty.org) 1.19.9.2 or later, you will need to apply +[OpenResty](https://openresty.org) 1.21.4.1 or later, you will need to apply patches to the standard Nginx core: https://openresty.org/en/nginx-ssl-patches.html From 66294b9d0675c2393e7d4ee625d7bd2aefc6b9c0 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 24 Feb 2022 09:47:02 +0800 Subject: [PATCH 603/848] bugfix: ambiguous error message 'connection in dubious state' when connection is closed. (#2016) --- src/ngx_http_lua_socket_tcp.c | 10 ++++-- t/058-tcp-socket.t | 68 ++++++++++++++++++++++++++++++++++- t/071-idle-socket.t | 2 +- 3 files changed, 75 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 26467fddbc..ceaf4ff175 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5188,7 +5188,12 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) pc = &u->peer; c = pc->connection; - if (c == NULL || u->read_closed || u->write_closed) { + /* When the server closes the connection, + * epoll will return EPOLLRDHUP event and nginx will set pending_eof. + */ + if (c == NULL || u->read_closed || u->write_closed + || c->read->eof || c->read->pending_eof) + { lua_pushnil(L); lua_pushliteral(L, "closed"); return 2; @@ -5218,8 +5223,7 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) return 2; } - if (c->read->eof - || c->read->error + if (c->read->error || c->read->timedout || c->write->error || c->write->timedout) diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 593e49461f..6ffb32f584 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * 231; +plan tests => repeat_each() * (blocks() * 3 + 21); our $HtmlDir = html_dir; @@ -4367,3 +4367,69 @@ connect failed: missing the port number finish --- no_error_log [error] + + + +=== TEST 73: reset the buffer pos when keepalive +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + for i = 1, 10 + do + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local req = "GET /hi HTTP/1.1\r\nHost: localhost\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + local line, err, part = sock:receive() + if not line then + ngx.say("receive err: ", err) + return + end + + data, err = sock:receiveany(4096) + if not data then + ngx.say("receiveany er: ", err) + return + end + + ok, err = sock:setkeepalive(10000, 32) + if not ok then + ngx.say("reused times: ", i, ", setkeepalive err: ", err) + return + end + end + ngx.say("END") + } + } + + location /hi { + keepalive_requests 3; + content_by_lua_block { + ngx.say("Hello") + } + + more_clear_headers Date; + } + +--- request +GET /t +--- response_body +reused times: 3, setkeepalive err: closed +--- no_error_log +[error] +--- skip_eval: 3: $ENV{TEST_NGINX_EVENT_TYPE} && $ENV{TEST_NGINX_EVENT_TYPE} ne 'epoll' diff --git a/t/071-idle-socket.t b/t/071-idle-socket.t index c9002be8ea..49d45a1b54 100644 --- a/t/071-idle-socket.t +++ b/t/071-idle-socket.t @@ -367,7 +367,7 @@ Transfer-Encoding: chunked\r Connection: close\r \r 6\r -failed to set keepalive: (?:unread data in buffer|connection in dubious state) +failed to set keepalive: (?:unread data in buffer|closed|connection in dubious state) } --- no_error_log [error] From 7d585bd535bae9841ccb5a6856355b4b78bdc9f5 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Sun, 27 Feb 2022 20:58:41 +0800 Subject: [PATCH 604/848] bugfix: fixed size of the array when initialized in the init_worker_by* phase. (#2017) --- src/ngx_http_lua_initworkerby.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 3235a4d30a..449e604ace 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -118,14 +118,15 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) ngx_queue_init(&fake_cycle->reusable_connections_queue); if (ngx_array_init(&fake_cycle->listening, cycle->pool, - cycle->listening.nelts || 1, + cycle->listening.nelts ? cycle->listening.nelts : 1, sizeof(ngx_listening_t)) != NGX_OK) { goto failed; } - if (ngx_array_init(&fake_cycle->paths, cycle->pool, cycle->paths.nelts || 1, + if (ngx_array_init(&fake_cycle->paths, cycle->pool, + cycle->paths.nelts ? cycle->paths.nelts : 1, sizeof(ngx_path_t *)) != NGX_OK) { @@ -135,7 +136,8 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) part = &cycle->open_files.part; ofile = part->elts; - if (ngx_list_init(&fake_cycle->open_files, cycle->pool, part->nelts || 1, + if (ngx_list_init(&fake_cycle->open_files, cycle->pool, + part->nelts ? part->nelts : 1, sizeof(ngx_open_file_t)) != NGX_OK) { From d001dea7783c7128bbc9dc0f96f2a24badd9ec2c Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 2 Jan 2022 10:11:42 +0800 Subject: [PATCH 605/848] bugfix: posted event handler was called after event memory was freed. (#1982) --- src/ngx_http_lua_semaphore.c | 9 ++++++- t/154-semaphore.t | 50 ++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_semaphore.c b/src/ngx_http_lua_semaphore.c index 6f45eeffcc..435beaa29a 100644 --- a/src/ngx_http_lua_semaphore.c +++ b/src/ngx_http_lua_semaphore.c @@ -466,8 +466,11 @@ ngx_http_lua_sema_handler(ngx_event_t *ev) sem = ev->data; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "semaphore handler: wait queue: %sempty, resource count: %d", + ngx_queue_empty(&sem->wait_queue) ? "" : "not ", + sem->resource_count); while (!ngx_queue_empty(&sem->wait_queue) && sem->resource_count > 0) { - q = ngx_queue_head(&sem->wait_queue); ngx_queue_remove(q); @@ -566,6 +569,10 @@ ngx_http_lua_ffi_sema_gc(ngx_http_lua_sema_t *sem) "destroyed", sem); } + if (sem->sem_event.posted) { + ngx_delete_posted_event(&sem->sem_event); + } + ngx_http_lua_free_sema(sem); } diff --git a/t/154-semaphore.t b/t/154-semaphore.t index c7b00e03f6..5bb2b937af 100644 --- a/t/154-semaphore.t +++ b/t/154-semaphore.t @@ -118,3 +118,53 @@ hello, world --- no_shutdown_error_log semaphore gc wait queue is not empty --- SKIP + + + +=== TEST 3: exit before post_handler was called +If gc is called before the ngx_http_lua_sema_handler and free the sema memory +ngx_http_lua_sema_handler would use the freed memory. +--- config + location /up { + content_by_lua_block { + local semaphore = require "ngx.semaphore" + local sem = semaphore.new() + + local function sem_wait() + ngx.log(ngx.ERR, "ngx.sem wait start") + local ok, err = sem:wait(10) + if not ok then + ngx.log(ngx.ERR, "ngx.sem wait err: ", err) + else + ngx.log(ngx.ERR, "ngx.sem wait success") + end + end + local co = ngx.thread.spawn(sem_wait) + ngx.log(ngx.ERR, "ngx.sem post start") + sem:post() + ngx.log(ngx.ERR, "ngx.sem post end") + ngx.say("hello") + ngx.exit(200) + ngx.say("not reach here") + } + } + + location /t { + content_by_lua_block { + local res = ngx.location.capture("/up") + collectgarbage() + ngx.print(res.body) + } + } + +--- request +GET /t +--- response_body +hello +--- grep_error_log eval: qr/(ngx.sem .*?,|http close request|semaphore handler: wait queue: empty, resource count: 1|in lua gc, semaphore)/ +--- grep_error_log_out +ngx.sem wait start, +ngx.sem post start, +ngx.sem post end, +in lua gc, semaphore +http close request From 97d1b704d0d86b5370d57604a9e2e3f86e4a33ec Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 10 Jan 2022 20:33:16 +0800 Subject: [PATCH 606/848] bugfix: ngx.pipe waits until timeout because child process forgot to close pipe after dup2. After the fork, the pipe will be copied by dup2, so there are two fd for the same open file description. eg: 0, 11 for the same pipe file, 1, 13 for another pipe file. When running a short life process, the fd will be closed automatically when the process exits. When running a daemon process, the daemon may close fd 0, fd 1 and redirect them to /dev/null, but left the fd 11 and fd 13 opened forever. when ngx.pipe reads from the pipe, it will wait for reading until the pipe is closed or timeout. Because the daemon won't close the pipe, so ngx.pipe will wait for the timeout. --- src/ngx_http_lua_pipe.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 9ea0ba1c41..93dc7d5bc5 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -773,6 +773,12 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } } + close(in[0]); + close(out[1]); + if (!merge_stderr) { + close(err[1]); + } + if (environ != NULL) { #if (NGX_HTTP_LUA_HAVE_EXECVPE) if (execvpe(file, (char * const *) argv, (char * const *) environ) From 496d9853347e8eefa81a442e42d33b4b92f5c825 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 2 Mar 2022 17:23:00 +0800 Subject: [PATCH 607/848] tests: change the count of the record after adding ngx.get_raw_phase. (#2019) --- t/062-count.t | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/062-count.t b/t/062-count.t index f5ba12c91d..104126281a 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -34,7 +34,7 @@ __DATA__ --- request GET /test --- response_body -ngx: 115 +ngx: 116 --- no_error_log [error] @@ -55,7 +55,7 @@ ngx: 115 --- request GET /test --- response_body -115 +116 --- no_error_log [error] @@ -83,7 +83,7 @@ GET /test --- request GET /test --- response_body -n = 115 +n = 116 --- no_error_log [error] @@ -305,7 +305,7 @@ GET /t --- response_body_like: 404 Not Found --- error_code: 404 --- error_log -ngx. entry count: 115 +ngx. entry count: 116 From bd92edf3b82438455036af1408d2377954f28b8f Mon Sep 17 00:00:00 2001 From: Blaise Wang Date: Fri, 4 Mar 2022 09:04:19 +0800 Subject: [PATCH 608/848] doc: modify openssl patch requirements. (#2022) --- README.markdown | 18 ++---------------- doc/HttpLuaModule.wiki | 18 ++---------------- 2 files changed, 4 insertions(+), 32 deletions(-) diff --git a/README.markdown b/README.markdown index 6583c0a406..b603a8b06b 100644 --- a/README.markdown +++ b/README.markdown @@ -2741,14 +2741,7 @@ packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.21.4.1 or later, then everything should work out of the box. -If you are not using one of the [OpenSSL -packages](https://openresty.org/en/linux-packages.html) provided by -[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -in order to use this directive: - - - -Similarly, if you are not using the Nginx core shipped with +If you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.21.4.1 or later, you will need to apply patches to the standard Nginx core: @@ -2858,14 +2851,7 @@ packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.9.7.2 or later, then everything should work out of the box. -If you are not using one of the [OpenSSL -packages](https://openresty.org/en/linux-packages.html) provided by -[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -in order to use this directive: - - - -Similarly, if you are not using the Nginx core shipped with +If you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.9.7.2 or later, you will need to apply patches to the standard Nginx core: diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 3b94c211cf..b5bacd7772 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2322,14 +2322,7 @@ packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.21.4.1 or later, then everything should work out of the box. -If you are not using one of the [OpenSSL -packages](https://openresty.org/en/linux-packages.html) provided by -[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -in order to use this directive: - -https://openresty.org/en/openssl-patches.html - -Similarly, if you are not using the Nginx core shipped with +If you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.21.4.1 or later, you will need to apply patches to the standard Nginx core: @@ -2432,14 +2425,7 @@ packages](https://openresty.org/en/linux-packages.html) for [OpenResty](https://openresty.org/) 1.9.7.2 or later, then everything should work out of the box. -If you are not using one of the [OpenSSL -packages](https://openresty.org/en/linux-packages.html) provided by -[OpenResty](https://openresty.org), you will need to apply patches to OpenSSL -in order to use this directive: - -https://openresty.org/en/openssl-patches.html - -Similarly, if you are not using the Nginx core shipped with +If you are not using the Nginx core shipped with [OpenResty](https://openresty.org) 1.9.7.2 or later, you will need to apply patches to the standard Nginx core: From 974d5d163d86af6852c39143ac6076d0144f79de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Fri, 4 Mar 2022 17:18:50 +0800 Subject: [PATCH 609/848] feature: add FFI implementation for ngx.arg getter (#2021) --- src/ngx_http_lua_bodyfilterby.c | 78 ++++++++++++++++++++------------- src/ngx_http_lua_bodyfilterby.h | 1 - src/ngx_http_lua_setby.c | 14 +++--- src/ngx_http_lua_setby.h | 1 - src/ngx_http_lua_util.c | 33 -------------- 5 files changed, 53 insertions(+), 74 deletions(-) diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 1aa01e552b..632f5afea3 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -396,51 +396,48 @@ ngx_http_lua_body_filter_init(void) int -ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r) +ngx_http_lua_ffi_get_body_filter_param_eof(ngx_http_request_t *r) { - u_char *data, *p; - size_t size; ngx_chain_t *cl; - ngx_buf_t *b; - int idx; ngx_chain_t *in; ngx_http_lua_main_conf_t *lmcf; - idx = luaL_checkint(L, 2); + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + in = lmcf->body_filter_chain; - dd("index: %d", idx); + /* asking for the eof argument */ - if (idx != 1 && idx != 2) { - lua_pushnil(L); - return 1; + for (cl = in; cl; cl = cl->next) { + if (cl->buf->last_buf || cl->buf->last_in_chain) { + return 1; + } } - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - in = lmcf->body_filter_chain; + return 0; +} - if (idx == 2) { - /* asking for the eof argument */ - for (cl = in; cl; cl = cl->next) { - if (cl->buf->last_buf || cl->buf->last_in_chain) { - lua_pushboolean(L, 1); - return 1; - } - } +int +ngx_http_lua_ffi_get_body_filter_param_body(ngx_http_request_t *r, + u_char **data_p, size_t *len_p) +{ + size_t size; + ngx_chain_t *cl; + ngx_buf_t *b; + ngx_chain_t *in; - lua_pushboolean(L, 0); - return 1; - } + ngx_http_lua_main_conf_t *lmcf; - /* idx == 1 */ + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + in = lmcf->body_filter_chain; size = 0; if (in == NULL) { /* being a cleared chain on the Lua land */ - lua_pushliteral(L, ""); - return 1; + *len_p = 0; + return NGX_OK; } if (in->next == NULL) { @@ -448,8 +445,9 @@ ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r) dd("seen only single buffer"); b = in->buf; - lua_pushlstring(L, (char *) b->pos, b->last - b->pos); - return 1; + *data_p = b->pos; + *len_p = b->last - b->pos; + return NGX_OK; } dd("seen multiple buffers"); @@ -464,7 +462,26 @@ ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r) } } - data = (u_char *) lua_newuserdata(L, size); + /* the buf is need and is not allocated from Lua land yet, return with + * the actual size */ + *len_p = size; + return NGX_AGAIN; +} + + +int +ngx_http_lua_ffi_copy_body_filter_param_body(ngx_http_request_t *r, + u_char *data) +{ + u_char *p; + ngx_chain_t *cl; + ngx_buf_t *b; + ngx_chain_t *in; + + ngx_http_lua_main_conf_t *lmcf; + + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + in = lmcf->body_filter_chain; for (p = data, cl = in; cl; cl = cl->next) { b = cl->buf; @@ -475,8 +492,7 @@ ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r) } } - lua_pushlstring(L, (char *) data, size); - return 1; + return NGX_OK; } diff --git a/src/ngx_http_lua_bodyfilterby.h b/src/ngx_http_lua_bodyfilterby.h index b108202faa..359646f2b5 100644 --- a/src/ngx_http_lua_bodyfilterby.h +++ b/src/ngx_http_lua_bodyfilterby.h @@ -21,7 +21,6 @@ ngx_int_t ngx_http_lua_body_filter_inline(ngx_http_request_t *r, ngx_chain_t *in); ngx_int_t ngx_http_lua_body_filter_file(ngx_http_request_t *r, ngx_chain_t *in); -int ngx_http_lua_body_filter_param_get(lua_State *L, ngx_http_request_t *r); int ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx); diff --git a/src/ngx_http_lua_setby.c b/src/ngx_http_lua_setby.c index f00468c903..98e8d47f5a 100644 --- a/src/ngx_http_lua_setby.c +++ b/src/ngx_http_lua_setby.c @@ -123,16 +123,15 @@ ngx_http_lua_set_by_chunk(lua_State *L, ngx_http_request_t *r, ngx_str_t *val, } -int -ngx_http_lua_setby_param_get(lua_State *L, ngx_http_request_t *r) +void +ngx_http_lua_ffi_get_setby_param(ngx_http_request_t *r, int idx, + u_char **data_p, size_t *len_p) { - int idx; int n; ngx_http_variable_value_t *v; ngx_http_lua_main_conf_t *lmcf; - idx = luaL_checkint(L, 2); idx--; lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); @@ -144,13 +143,12 @@ ngx_http_lua_setby_param_get(lua_State *L, ngx_http_request_t *r) v = lmcf->setby_args; if (idx < 0 || idx > n - 1) { - lua_pushnil(L); + *len_p = 0; } else { - lua_pushlstring(L, (const char *) (v[idx].data), v[idx].len); + *data_p = v[idx].data; + *len_p = v[idx].len; } - - return 1; } diff --git a/src/ngx_http_lua_setby.h b/src/ngx_http_lua_setby.h index f43eef753e..8fd1e2c7b3 100644 --- a/src/ngx_http_lua_setby.h +++ b/src/ngx_http_lua_setby.h @@ -7,7 +7,6 @@ ngx_int_t ngx_http_lua_set_by_chunk(lua_State *L, ngx_http_request_t *r, ngx_str_t *val, ngx_http_variable_value_t *args, size_t nargs, ngx_str_t *script); -int ngx_http_lua_setby_param_get(lua_State *L, ngx_http_request_t *r); #endif /* _NGX_HTTP_LUA_SET_BY_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index b783cf850c..762958396b 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -119,7 +119,6 @@ static int ngx_http_lua_thread_traceback(lua_State *L, lua_State *co, static void ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log); static void ngx_http_lua_inject_arg_api(lua_State *L); -static int ngx_http_lua_param_get(lua_State *L); static int ngx_http_lua_param_set(lua_State *L); static ngx_int_t ngx_http_lua_output_filter(ngx_http_request_t *r, ngx_chain_t *in); @@ -3099,9 +3098,6 @@ ngx_http_lua_inject_arg_api(lua_State *L) lua_createtable(L, 0 /* narr */, 2 /* nrec */); /* the metatable */ - lua_pushcfunction(L, ngx_http_lua_param_get); - lua_setfield(L, -2, "__index"); - lua_pushcfunction(L, ngx_http_lua_param_set); lua_setfield(L, -2, "__newindex"); @@ -3113,35 +3109,6 @@ ngx_http_lua_inject_arg_api(lua_State *L) } -static int -ngx_http_lua_param_get(lua_State *L) -{ - ngx_http_lua_ctx_t *ctx; - ngx_http_request_t *r; - - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return 0; - } - - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { - return luaL_error(L, "ctx not found"); - } - - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_SET - | NGX_HTTP_LUA_CONTEXT_BODY_FILTER); - - if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SET)) { - return ngx_http_lua_setby_param_get(L, r); - } - - /* ctx->context & (NGX_HTTP_LUA_CONTEXT_BODY_FILTER) */ - - return ngx_http_lua_body_filter_param_get(L, r); -} - - static int ngx_http_lua_param_set(lua_State *L) { From d959374e027475c27a96c3dde38486b4506edda4 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 5 Mar 2022 17:03:37 +0800 Subject: [PATCH 610/848] optimize: use ngx_hash_t to optimize the built-in header look-up process for ngx.header.HEADER. --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- src/ngx_http_lua_common.h | 2 + src/ngx_http_lua_headers_out.c | 97 ++++++++++++++++++++-------------- src/ngx_http_lua_headers_out.h | 2 + src/ngx_http_lua_module.c | 10 ++++ src/ngx_http_lua_util.h | 37 +++++++++++++ 7 files changed, 110 insertions(+), 42 deletions(-) diff --git a/README.markdown b/README.markdown index b603a8b06b..f54e797eb4 100644 --- a/README.markdown +++ b/README.markdown @@ -973,7 +973,7 @@ TODO * cosocket: review and merge aviramc's [patch](https://github.com/openresty/lua-nginx-module/pull/290) for adding the `bsdrecv` method. * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option -* use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), [ngx.header.HEADER](#ngxheaderheader), and etc. +* use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), and etc. * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add `stat` mode similar to [mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html). diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index b5bacd7772..9a8dfc8b30 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -808,7 +808,7 @@ phases. * cosocket: review and merge aviramc's [https://github.com/openresty/lua-nginx-module/pull/290 patch] for adding the bsdrecv method. * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [[#ngx.location.capture|ngx.location.capture]]'s extra_headers option -* use ngx_hash_t to optimize the built-in header look-up process for [[#ngx.req.set_header|ngx.req.set_header]], [[#ngx.header.HEADER|ngx.header.HEADER]], and etc. +* use ngx_hash_t to optimize the built-in header look-up process for [[#ngx.req.set_header|ngx.req.set_header]], and etc. * add ignore_resp_headers, ignore_resp_body, and ignore_resp options to [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add stat mode similar to [https://httpd.apache.org/docs/trunk/mod/mod_lua.html mod_lua]. diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 017e2a0434..3a2480f041 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -214,6 +214,8 @@ struct ngx_http_lua_main_conf_s { ngx_int_t lua_thread_cache_max_entries; + ngx_hash_t builtin_headers_out; + #if (NGX_PCRE) ngx_int_t regex_cache_entries; ngx_int_t regex_cache_max_entries; diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 050f12b4ed..6e9f9c19ac 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -486,8 +486,9 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_str_t key, ngx_str_t value, unsigned override) { ngx_http_lua_header_val_t hv; - ngx_http_lua_set_header_t *handlers = ngx_http_lua_set_handlers; - ngx_uint_t i; + ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_set_header_t *lsh; + ngx_hash_t *hash; dd("set header value: %.*s", (int) value.len, value.data); @@ -499,53 +500,25 @@ ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, return NGX_ERROR; } - if (value.len > 0) { - hv.hash = ngx_hash_key_lc(key.data, key.len); - - } else { - hv.hash = 0; - } - + hv.hash = ngx_hash_key_lc(key.data, key.len); hv.key = key; hv.offset = 0; hv.no_override = !override; - hv.handler = NULL; - - for (i = 0; handlers[i].name.len; i++) { - if (hv.key.len != handlers[i].name.len - || ngx_strncasecmp(hv.key.data, handlers[i].name.data, - handlers[i].name.len) != 0) - { - dd("hv key comparison: %s <> %s", handlers[i].name.data, - hv.key.data); - - continue; - } - - dd("Matched handler: %s %s", handlers[i].name.data, hv.key.data); - - hv.offset = handlers[i].offset; - hv.handler = handlers[i].handler; - + hv.handler = ngx_http_set_header; + + lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); + hash = &lmcf->builtin_headers_out; + lsh = ngx_http_lua_hash_find_lc(hash, hv.hash, hv.key.data, hv.key.len); + if (lsh) { + dd("Matched handler: %s %s", lsh->name.data, hv.key.data); + hv.offset = lsh->offset; + hv.handler = lsh->handler; if (hv.handler == ngx_http_set_content_type_header) { ctx->mime_set = 1; } - - break; - } - - if (handlers[i].name.len == 0 && handlers[i].handler) { - hv.offset = handlers[i].offset; - hv.handler = handlers[i].handler; } -#if 1 - if (hv.handler == NULL) { - return NGX_ERROR; - } -#endif - return hv.handler(r, &hv, &value); } @@ -658,4 +631,48 @@ ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, return 1; } + +ngx_int_t +ngx_http_lua_init_builtin_headers_out(ngx_conf_t *cf, + ngx_http_lua_main_conf_t *lmcf) +{ + ngx_array_t headers; + ngx_hash_key_t *hk; + ngx_hash_init_t hash; + ngx_http_lua_set_header_t *handlers = ngx_http_lua_set_handlers; + ngx_uint_t count; + + count = sizeof(ngx_http_lua_set_handlers) + / sizeof(ngx_http_lua_set_header_t); + + if (ngx_array_init(&headers, cf->temp_pool, count, sizeof(ngx_hash_key_t)) + != NGX_OK) + { + return NGX_ERROR; + } + + while (handlers->name.data) { + hk = ngx_array_push(&headers); + if (hk == NULL) { + return NGX_ERROR; + } + + hk->key = handlers->name; + hk->key_hash = ngx_hash_key_lc(handlers->name.data, handlers->name.len); + hk->value = (void *) handlers; + + handlers++; + } + + hash.hash = &lmcf->builtin_headers_out; + hash.key = ngx_hash_key_lc; + hash.max_size = 512; + hash.bucket_size = ngx_align(64, ngx_cacheline_size); + hash.name = "builtin_headers_out_hash"; + hash.pool = cf->pool; + hash.temp_pool = NULL; + + return ngx_hash_init(&hash, headers.elts, headers.nelts); +} + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_headers_out.h b/src/ngx_http_lua_headers_out.h index 6ec1fe391d..944f57a462 100644 --- a/src/ngx_http_lua_headers_out.h +++ b/src/ngx_http_lua_headers_out.h @@ -16,6 +16,8 @@ ngx_int_t ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_str_t key, ngx_str_t value, unsigned override); int ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_str_t *key); +ngx_int_t ngx_http_lua_init_builtin_headers_out(ngx_conf_t *cf, + ngx_http_lua_main_conf_t *lmcf); #endif /* _NGX_HTTP_LUA_HEADERS_OUT_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 09fd0a91f8..bd4535af35 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -31,6 +31,7 @@ #include "ngx_http_lua_ssl_session_storeby.h" #include "ngx_http_lua_ssl_session_fetchby.h" #include "ngx_http_lua_headers.h" +#include "ngx_http_lua_headers_out.h" #include "ngx_http_lua_pipe.h" @@ -1086,6 +1087,15 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) lmcf->worker_thread_vm_pool_size = 100; } + if (ngx_http_lua_init_builtin_headers_out(cf, lmcf) != NGX_OK) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "init header out error"); + + return NGX_CONF_ERROR; + } + + dd("init built in headers out hash size: %ld", + lmcf->builtin_headers_out.size); + return NGX_CONF_OK; } diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 9ad34b93ea..52b1ff9356 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -685,6 +685,43 @@ ngx_http_lua_new_cached_thread(lua_State *L, lua_State **out_co, } +static ngx_inline void * +ngx_http_lua_hash_find_lc(ngx_hash_t *hash, ngx_uint_t key, u_char *name, + size_t len) +{ + ngx_uint_t i; + ngx_hash_elt_t *elt; + + elt = hash->buckets[key % hash->size]; + + if (elt == NULL) { + return NULL; + } + + while (elt->value) { + if (len != (size_t) elt->len) { + goto next; + } + + for (i = 0; i < len; i++) { + if (ngx_tolower(name[i]) != elt->name[i]) { + goto next; + } + } + + return elt->value; + + next: + + elt = (ngx_hash_elt_t *) ngx_align_ptr(&elt->name[0] + elt->len, + sizeof(void *)); + continue; + } + + return NULL; +} + + extern ngx_uint_t ngx_http_lua_location_hash; extern ngx_uint_t ngx_http_lua_content_length_hash; From 247086ed3892647ab5ecc4fd9946186bb76f15db Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Tue, 8 Mar 2022 08:40:57 +0800 Subject: [PATCH 611/848] doc: describe the second argument of get_uri_args (#2023) Signed-off-by: spacewander --- README.markdown | 5 +++-- doc/HttpLuaModule.wiki | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index f54e797eb4..4e0b979879 100644 --- a/README.markdown +++ b/README.markdown @@ -4903,11 +4903,12 @@ See also [ngx.req.set_uri](#ngxreqset_uri). ngx.req.get_uri_args -------------------- -**syntax:** *args, err = ngx.req.get_uri_args(max_args?)* +**syntax:** *args, err = ngx.req.get_uri_args(max_args?, tab?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** -Returns a Lua table holding all the current request URL query arguments. +Returns a Lua table holding all the current request URL query arguments. An optional `tab` argument +can be used to reuse the table returned by this method. ```nginx diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 9a8dfc8b30..0d69fd205b 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4101,11 +4101,12 @@ See also [[#ngx.req.set_uri|ngx.req.set_uri]]. == ngx.req.get_uri_args == -'''syntax:''' ''args, err = ngx.req.get_uri_args(max_args?)'' +'''syntax:''' ''args, err = ngx.req.get_uri_args(max_args?, tab?)'' '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua*'' -Returns a Lua table holding all the current request URL query arguments. +Returns a Lua table holding all the current request URL query arguments. An optional tab argument +can be used to reuse the table returned by this method. location = /test { From a7444abe8e50b49d91a922c958f904ba893e54e7 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 9 Mar 2022 16:17:15 +0800 Subject: [PATCH 612/848] tests: 160-disable-init-by-lua.t: prevent the test process from overwriting the original pid file. (#2024) --- t/160-disable-init-by-lua.t | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/t/160-disable-init-by-lua.t b/t/160-disable-init-by-lua.t index dd0098d0d5..e98c136dcf 100644 --- a/t/160-disable-init-by-lua.t +++ b/t/160-disable-init-by-lua.t @@ -12,6 +12,9 @@ my $http_config = <<_EOC_; function set_up_ngx_tmp_conf(conf) if conf == nil then conf = [[ + # to prevent the test process from overwriting the + # original pid file + pid logs/test_nginx.pid; events { worker_connections 64; } @@ -150,6 +153,7 @@ qr/\[error\] .*? init_by_lua:\d+: run init_by_lua/ location = /t { content_by_lua_block { local conf = [[ + pid logs/test_nginx.pid; events { worker_connections 64; } From 1fea11712274a26b9b3b5b3b44fb0791a312091d Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Thu, 10 Mar 2022 18:04:32 +0800 Subject: [PATCH 613/848] tests: use random port numbers instead of fixed port number to prevent conflict. (#2025) --- t/109-timer-hup.t | 4 ++-- t/138-balancer.t | 6 +++--- t/139-ssl-cert-by.t | 24 ++++++++++++------------ 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/t/109-timer-hup.t b/t/109-timer-hup.t index 2c197c2826..bff1b33466 100644 --- a/t/109-timer-hup.t +++ b/t/109-timer-hup.t @@ -336,7 +336,7 @@ lua found 100 pending timers lua_shared_dict test_dict 1m; server { - listen 12355; + listen $TEST_NGINX_RAND_PORT_1; location = /foo { echo 'foo'; } @@ -350,7 +350,7 @@ lua found 100 pending timers -- Connect the socket local sock = ngx.socket.tcp() - local ok,err = sock:connect("127.0.0.1", 12355) + local ok,err = sock:connect("127.0.0.1", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.log(ngx.ERR, err) end diff --git a/t/138-balancer.t b/t/138-balancer.t index 2894157e0e..d9c943388e 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -535,7 +535,7 @@ failed to set more tries: reduced tries due to limit lua_package_path "../lua-resty-core/lib/?.lua;;"; server { - listen 127.0.0.1:8888; + listen 127.0.0.1:$TEST_NGINX_RAND_PORT_1; location / { return 200 "it works"; @@ -543,8 +543,8 @@ failed to set more tries: reduced tries due to limit } upstream foo { - server 127.0.0.1:8888 max_fails=0; - server 127.0.0.1:8889 max_fails=0 weight=9999; + server 127.0.0.1:$TEST_NGINX_RAND_PORT_1 max_fails=0; + server 127.0.0.1:$TEST_NGINX_RAND_PORT_2 max_fails=0 weight=9999; balancer_by_lua_block { local bal = require "ngx.balancer" diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 38296352f5..502e0ac184 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -456,7 +456,7 @@ received memc reply: OK === TEST 5: ngx.exit(0) - no yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { ngx.exit(0) @@ -484,7 +484,7 @@ received memc reply: OK sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return @@ -523,7 +523,7 @@ should never reached here === TEST 6: ngx.exit(ngx.ERROR) - no yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { ngx.exit(ngx.ERROR) @@ -551,7 +551,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return @@ -593,7 +593,7 @@ should never reached here === TEST 7: ngx.exit(0) - yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { ngx.sleep(0.001) @@ -623,7 +623,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return @@ -662,7 +662,7 @@ should never reached here === TEST 8: ngx.exit(ngx.ERROR) - yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { ngx.sleep(0.001) @@ -692,7 +692,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return @@ -734,7 +734,7 @@ should never reached here === TEST 9: lua exception - no yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { error("bad bad bad") @@ -762,7 +762,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return @@ -805,7 +805,7 @@ should never reached here === TEST 10: lua exception - yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { ngx.sleep(0.001) @@ -834,7 +834,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return From 2b902657f370e392bd5066d2eafed7a3429af19e Mon Sep 17 00:00:00 2001 From: Datong Sun Date: Tue, 15 Mar 2022 22:12:30 +0800 Subject: [PATCH 614/848] cosocket: add function `tcpsock:setclientcert`, reimplemented `tcpsock:sslhandshake` with FFI This adds support for setting client certificate/private key that will be used later for mutual TLS handshake with a server. Also, the `tcpsock:sslhandshake` implementation has been rewritten to use FFI C API to be more performant and easier to maintain. Co-authored-by: Chrono Law --- .travis.yml | 2 +- README.markdown | 28 +- src/ngx_http_lua_socket_tcp.c | 420 +++++++++++++------------- src/ngx_http_lua_socket_tcp.h | 3 + t/062-count.t | 2 +- t/129-ssl-socket.t | 52 ++-- t/139-ssl-cert-by.t | 30 +- t/140-ssl-c-api.t | 14 +- t/142-ssl-session-store.t | 26 +- t/143-ssl-session-fetch.t | 38 +-- t/155-tls13.t | 2 +- t/162-socket-tls-handshake.t | 376 +++++++++++++++++++++++ t/166-ssl-client-hello.t | 34 +-- t/cert/mtls_ca.crt | 78 +++++ t/cert/mtls_ca.key | 27 ++ t/cert/mtls_cert_gen/.gitignore | 4 + t/cert/mtls_cert_gen/generate.sh | 23 ++ t/cert/mtls_cert_gen/mtls_ca.json | 18 ++ t/cert/mtls_cert_gen/mtls_client.json | 18 ++ t/cert/mtls_cert_gen/mtls_server.json | 17 ++ t/cert/mtls_cert_gen/profile.json | 27 ++ t/cert/mtls_client.crt | 87 ++++++ t/cert/mtls_client.key | 27 ++ t/cert/mtls_server.crt | 87 ++++++ t/cert/mtls_server.key | 27 ++ 25 files changed, 1158 insertions(+), 309 deletions(-) create mode 100644 t/162-socket-tls-handshake.t create mode 100644 t/cert/mtls_ca.crt create mode 100644 t/cert/mtls_ca.key create mode 100644 t/cert/mtls_cert_gen/.gitignore create mode 100755 t/cert/mtls_cert_gen/generate.sh create mode 100644 t/cert/mtls_cert_gen/mtls_ca.json create mode 100644 t/cert/mtls_cert_gen/mtls_client.json create mode 100644 t/cert/mtls_cert_gen/mtls_server.json create mode 100644 t/cert/mtls_cert_gen/profile.json create mode 100644 t/cert/mtls_client.crt create mode 100644 t/cert/mtls_client.key create mode 100644 t/cert/mtls_server.crt create mode 100644 t/cert/mtls_server.key diff --git a/.travis.yml b/.travis.yml index 335ae71ee6..dbf1ed6acf 100644 --- a/.travis.yml +++ b/.travis.yml @@ -88,7 +88,7 @@ install: - git clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core + - git clone https://github.com/dndx/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string diff --git a/README.markdown b/README.markdown index 4e0b979879..7f0ac24cb1 100644 --- a/README.markdown +++ b/README.markdown @@ -977,7 +977,6 @@ TODO * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add `stat` mode similar to [mod_lua](https://httpd.apache.org/docs/trunk/mod/mod_lua.html). -* cosocket: add client SSL certificate support. [Back to TOC](#table-of-contents) @@ -3594,6 +3593,7 @@ Nginx API for Lua * [ngx.socket.stream](#ngxsocketstream) * [ngx.socket.tcp](#ngxsockettcp) * [tcpsock:connect](#tcpsockconnect) +* [tcpsock:setclientcert](#tcpsocksetclientcert) * [tcpsock:sslhandshake](#tcpsocksslhandshake) * [tcpsock:send](#tcpsocksend) * [tcpsock:receive](#tcpsockreceive) @@ -7565,6 +7565,7 @@ ngx.socket.tcp Creates and returns a TCP or stream-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: * [connect](#tcpsockconnect) +* [setclientcert](#tcpsocksetclientcert) * [sslhandshake](#tcpsocksslhandshake) * [send](#tcpsocksend) * [receive](#tcpsockreceive) @@ -7724,6 +7725,31 @@ This method was first introduced in the `v0.5.0rc1` release. [Back to TOC](#nginx-api-for-lua) +tcpsock:setclientcert +-------------------- + +**syntax:** *ok, err = tcpsock:setclientcert(cert, pkey)* + +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** + +Set client certificate chain and corresponding private key to the TCP socket object. +The certificate chain and private key provided will be used later by the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. + +* `cert` specify a client certificate chain cdata object that will be used while handshaking with +remote server. These objects can be created using [ngx.ssl.parse\_pem\_cert](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_cert) +function provided by lua-resty-core. Note that specifying the `cert` option requires +corresponding `pkey` be provided too. See below. +* `pkey` specify a private key corresponds to the `cert` option above. +These objects can be created using [ngx.ssl.parse\_pem\_priv\_key](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_priv_key) +function provided by lua-resty-core. + +If both of `cert` and `pkey` are `nil`, this method will clear any existing client certificate and private key +that was previously set on the cosocket object. + +This method was first introduced in the `v0.10.22` release. + +[Back to TOC](#nginx-api-for-lua) + tcpsock:sslhandshake -------------------- diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index ceaf4ff175..72549ca41e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -22,7 +22,9 @@ static int ngx_http_lua_socket_tcp(lua_State *L); static int ngx_http_lua_socket_tcp_connect(lua_State *L); #if (NGX_HTTP_SSL) -static int ngx_http_lua_socket_tcp_sslhandshake(lua_State *L); +static void ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c); +static int ngx_http_lua_ssl_handshake_retval_handler(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); #endif static int ngx_http_lua_socket_tcp_receive(lua_State *L); static int ngx_http_lua_socket_tcp_receiveany(lua_State *L); @@ -149,12 +151,6 @@ static void ngx_http_lua_socket_shutdown_pool_helper( ngx_http_lua_socket_pool_t *spool); static int ngx_http_lua_socket_prepare_error_retvals(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L, ngx_uint_t ft_type); -#if (NGX_HTTP_SSL) -static int ngx_http_lua_ssl_handshake_retval_handler(ngx_http_request_t *r, - ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); -static void ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c); -static int ngx_http_lua_ssl_free_session(lua_State *L); -#endif static void ngx_http_lua_socket_tcp_close_connection(ngx_connection_t *c); @@ -164,6 +160,8 @@ enum { SOCKET_CONNECT_TIMEOUT_INDEX = 2, SOCKET_SEND_TIMEOUT_INDEX = 4, SOCKET_READ_TIMEOUT_INDEX = 5, + SOCKET_CLIENT_CERT_INDEX = 6 , + SOCKET_CLIENT_PKEY_INDEX = 7 , }; @@ -222,9 +220,6 @@ static char ngx_http_lua_upstream_udata_metatable_key; static char ngx_http_lua_downstream_udata_metatable_key; static char ngx_http_lua_pool_udata_metatable_key; static char ngx_http_lua_pattern_udata_metatable_key; -#if (NGX_HTTP_SSL) -static char ngx_http_lua_ssl_session_metatable_key; -#endif #define ngx_http_lua_tcp_socket_metatable_literal_key "__tcp_cosocket_mt" @@ -319,18 +314,11 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* {{{tcp object metatable */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( tcp_socket_metatable_key)); - lua_createtable(L, 0 /* narr */, 14 /* nrec */); + lua_createtable(L, 0 /* narr */, 15 /* nrec */); lua_pushcfunction(L, ngx_http_lua_socket_tcp_connect); lua_setfield(L, -2, "connect"); -#if (NGX_HTTP_SSL) - - lua_pushcfunction(L, ngx_http_lua_socket_tcp_sslhandshake); - lua_setfield(L, -2, "sslhandshake"); - -#endif - lua_pushcfunction(L, ngx_http_lua_socket_tcp_receive); lua_setfield(L, -2, "receive"); @@ -404,19 +392,6 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_setfield(L, -2, "__gc"); lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ - -#if (NGX_HTTP_SSL) - - /* {{{ssl session userdata metatable */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - ssl_session_metatable_key)); - lua_createtable(L, 0 /* narr */, 1 /* nrec */); /* metatable */ - lua_pushcfunction(L, ngx_http_lua_ssl_free_session); - lua_setfield(L, -2, "__gc"); - lua_rawset(L, LUA_REGISTRYINDEX); - /* }}} */ - -#endif } @@ -451,7 +426,7 @@ ngx_http_lua_socket_tcp(lua_State *L) ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); - lua_createtable(L, 5 /* narr */, 1 /* nrec */); + lua_createtable(L, 7 /* narr */, 1 /* nrec */); lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( tcp_socket_metatable_key)); lua_rawget(L, LUA_REGISTRYINDEX); @@ -1559,64 +1534,73 @@ ngx_http_lua_socket_conn_error_retval_handler(ngx_http_request_t *r, #if (NGX_HTTP_SSL) -static int -ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) +static const char * +ngx_http_lua_socket_tcp_check_busy(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, unsigned int ops) { - int n, top; - ngx_int_t rc; - ngx_str_t name = ngx_null_string; - ngx_connection_t *c; - ngx_ssl_session_t **psession; - ngx_http_request_t *r; - ngx_http_lua_ctx_t *ctx; - ngx_http_lua_co_ctx_t *coctx; - - ngx_http_lua_socket_tcp_upstream_t *u; - - /* Lua function arguments: self [,session] [,host] [,verify] - [,send_status_req] */ + if ((ops & SOCKET_OP_CONNECT) && u->conn_waiting) { + return "socket busy connecting"; + } - n = lua_gettop(L); - if (n < 1 || n > 5) { - return luaL_error(L, "ngx.socket sslhandshake: expecting 1 ~ 5 " - "arguments (including the object), but seen %d", n); + if ((ops & SOCKET_OP_READ) && u->read_waiting) { + return "socket busy reading"; } - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request found"); + if ((ops & SOCKET_OP_WRITE) + && (u->write_waiting + || (u->raw_downstream + && (r->connection->buffered & NGX_HTTP_LOWLEVEL_BUFFERED)))) + { + return "socket busy writing"; } - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket ssl handshake"); + return NULL; +} - luaL_checktype(L, 1, LUA_TTABLE); - lua_rawgeti(L, 1, SOCKET_CTX_INDEX); - u = lua_touserdata(L, -1); +int +ngx_http_lua_ffi_socket_tcp_sslhandshake(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, ngx_ssl_session_t *sess, + int enable_session_reuse, ngx_str_t *server_name, int verify, + int ocsp_status_req, STACK_OF(X509) *chain, EVP_PKEY *pkey, + const char **errmsg) +{ + ngx_int_t rc, i; + ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_co_ctx_t *coctx; + const char *busy_msg; + ngx_ssl_conn_t *ssl_conn; + X509 *x509; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket ssl handshake"); if (u == NULL || u->peer.connection == NULL || u->read_closed || u->write_closed) { - lua_pushnil(L); - lua_pushliteral(L, "closed"); - return 2; + *errmsg = "closed"; + return NGX_ERROR; } if (u->request != r) { - return luaL_error(L, "bad request"); + *errmsg = "bad request"; + return NGX_ERROR; } - ngx_http_lua_socket_check_busy_connecting(r, u, L); - ngx_http_lua_socket_check_busy_reading(r, u, L); - ngx_http_lua_socket_check_busy_writing(r, u, L); + busy_msg = ngx_http_lua_socket_tcp_check_busy(r, u, SOCKET_OP_CONNECT + | SOCKET_OP_READ + | SOCKET_OP_WRITE); + if (busy_msg != NULL) { + *errmsg = busy_msg; + return NGX_ERROR; + } if (u->raw_downstream || u->body_downstream) { - lua_pushnil(L); - lua_pushliteral(L, "not supported for downstream"); - return 2; + *errmsg = "not supported for downstream sockets"; + return NGX_ERROR; } c = u->peer.connection; @@ -1624,122 +1608,140 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) u->ssl_session_reuse = 1; if (c->ssl && c->ssl->handshaked) { - switch (lua_type(L, 2)) { - case LUA_TUSERDATA: - lua_pushvalue(L, 2); - break; + if (sess != NULL) { + return NGX_DONE; + } - case LUA_TBOOLEAN: - if (!lua_toboolean(L, 2)) { - /* avoid generating the ssl session */ - lua_pushboolean(L, 1); - break; - } - /* fall through */ + u->ssl_session_reuse = enable_session_reuse; - default: - ngx_http_lua_ssl_handshake_retval_handler(r, u, L); - break; - } + (void) ngx_http_lua_ssl_handshake_retval_handler(r, u, NULL); - return 1; + return NGX_OK; } if (ngx_ssl_create_connection(u->conf->ssl, c, NGX_SSL_BUFFER|NGX_SSL_CLIENT) != NGX_OK) { - lua_pushnil(L); - lua_pushliteral(L, "failed to create ssl connection"); - return 2; + *errmsg = "failed to create ssl connection"; + return NGX_ERROR; } + ssl_conn = c->ssl->connection; + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { - return luaL_error(L, "no ctx found"); + return NGX_HTTP_LUA_FFI_NO_REQ_CTX; } coctx = ctx->cur_co_ctx; c->sendfile = 0; - if (n >= 2) { - if (lua_type(L, 2) == LUA_TBOOLEAN) { - u->ssl_session_reuse = lua_toboolean(L, 2); + if (sess != NULL) { + if (ngx_ssl_set_session(c, sess) != NGX_OK) { + *errmsg = "ssl set session failed"; + return NGX_ERROR; + } - } else { - psession = lua_touserdata(L, 2); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua ssl set session: %p", sess); - if (psession != NULL && *psession != NULL) { - if (ngx_ssl_set_session(c, *psession) != NGX_OK) { - lua_pushnil(L); - lua_pushliteral(L, "lua ssl set session failed"); - return 2; - } + } else { + u->ssl_session_reuse = enable_session_reuse; + } - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua ssl set session: %p", *psession); - } + if (chain != NULL) { + ngx_http_lua_assert(pkey != NULL); /* ensured by resty.core */ + + if (sk_X509_num(chain) < 1) { + ERR_clear_error(); + *errmsg = "invalid client certificate chain"; + return NGX_ERROR; } - if (n >= 3) { - name.data = (u_char *) lua_tolstring(L, 3, &name.len); + x509 = sk_X509_value(chain, 0); + if (x509 == NULL) { + ERR_clear_error(); + *errmsg = "ssl fetch client certificate from chain failed"; + return NGX_ERROR; + } - if (name.data) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua ssl server name: \"%*s\"", name.len, - name.data); + if (SSL_use_certificate(ssl_conn, x509) == 0) { + ERR_clear_error(); + *errmsg = "ssl set client certificate failed"; + return NGX_ERROR; + } -#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + /* read rest of the chain */ - if (SSL_set_tlsext_host_name(c->ssl->connection, - (char *) name.data) - == 0) - { - lua_pushnil(L); - lua_pushliteral(L, "SSL_set_tlsext_host_name failed"); - return 2; - } + for (i = 1; i < sk_X509_num(chain); i++) { + x509 = sk_X509_value(chain, i); + if (x509 == NULL) { + ERR_clear_error(); + *errmsg = "ssl fetch client intermediate certificate from " + "chain failed"; + return NGX_ERROR; + } -#else + if (SSL_add1_chain_cert(ssl_conn, x509) == 0) { + ERR_clear_error(); + *errmsg = "ssl set client intermediate certificate failed"; + return NGX_ERROR; + } + } - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua socket SNI disabled because the current " - "version of OpenSSL lacks the support"); + if (SSL_use_PrivateKey(ssl_conn, pkey) == 0) { + ERR_clear_error(); + *errmsg = "ssl set client private key failed"; + return NGX_ERROR; + } + } + if (server_name != NULL && server_name->data != NULL) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua ssl server name: \"%V\"", server_name); + +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + if (SSL_set_tlsext_host_name(c->ssl->connection, + (char *) server_name->data) + == 0) + { + *errmsg = "SSL_set_tlsext_host_name failed"; + return NGX_ERROR; + } + +#else + *errmsg = "no TLS extension support"; + return NGX_ERROR; #endif - } + } - if (n >= 4) { - u->ssl_verify = lua_toboolean(L, 4); + u->ssl_verify = verify; - if (n >= 5) { - if (lua_toboolean(L, 5)) { + if (ocsp_status_req) { #ifdef NGX_HTTP_LUA_USE_OCSP - SSL_set_tlsext_status_type(c->ssl->connection, - TLSEXT_STATUSTYPE_ocsp); + SSL_set_tlsext_status_type(c->ssl->connection, + TLSEXT_STATUSTYPE_ocsp); + #else - return luaL_error(L, "no OCSP support"); + *errmsg = "no OCSP support"; + return NGX_ERROR; #endif - } - } - } - } } - dd("found sni name: %.*s %p", (int) name.len, name.data, name.data); - - if (name.len == 0) { + if (server_name->len == 0) { u->ssl_name.len = 0; } else { if (u->ssl_name.data) { /* buffer already allocated */ - if (u->ssl_name.len >= name.len) { + if (u->ssl_name.len >= server_name->len) { /* reuse it */ - ngx_memcpy(u->ssl_name.data, name.data, name.len); - u->ssl_name.len = name.len; + ngx_memcpy(u->ssl_name.data, server_name->data, + server_name->len); + u->ssl_name.len = server_name->len; } else { ngx_free(u->ssl_name.data); @@ -1750,17 +1752,15 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) new_ssl_name: - u->ssl_name.data = ngx_alloc(name.len, ngx_cycle->log); + u->ssl_name.data = ngx_alloc(server_name->len, ngx_cycle->log); if (u->ssl_name.data == NULL) { u->ssl_name.len = 0; - - lua_pushnil(L); - lua_pushliteral(L, "no memory"); - return 2; + *errmsg = "no memory"; + return NGX_ERROR; } - ngx_memcpy(u->ssl_name.data, name.data, name.len); - u->ssl_name.len = name.len; + ngx_memcpy(u->ssl_name.data, server_name->data, server_name->len); + u->ssl_name.len = server_name->len; } } @@ -1774,7 +1774,8 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) rc = ngx_ssl_handshake(c); - dd("ngx_ssl_handshake returned %d", (int) rc); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "ngx_ssl_handshake returned: %d", rc); if (rc == NGX_AGAIN) { if (c->write->timer_set) { @@ -1799,21 +1800,24 @@ ngx_http_lua_socket_tcp_sslhandshake(lua_State *L) r->write_event_handler = ngx_http_core_run_phases; } - return lua_yield(L, 0); + return NGX_AGAIN; } - top = lua_gettop(L); ngx_http_lua_ssl_handshake_handler(c); - return lua_gettop(L) - top; + + if (rc == NGX_ERROR) { + *errmsg = u->error_ret; + return NGX_ERROR; + } + + return NGX_OK; } static void ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) { - const char *err; int waiting; - lua_State *L; ngx_int_t rc; ngx_connection_t *dc; /* downstream connection */ ngx_http_request_t *r; @@ -1836,11 +1840,9 @@ ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) waiting = u->conn_waiting; dc = r->connection; - L = u->write_co_ctx->co; if (c->read->timedout) { - lua_pushnil(L); - lua_pushliteral(L, "timeout"); + u->error_ret = "timeout"; goto failed; } @@ -1849,19 +1851,18 @@ ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) } if (c->ssl->handshaked) { - if (u->ssl_verify) { rc = SSL_get_verify_result(c->ssl->connection); if (rc != X509_V_OK) { - lua_pushnil(L); - err = lua_pushfstring(L, "%d: %s", (int) rc, - X509_verify_cert_error_string(rc)); + u->error_ret = X509_verify_cert_error_string(rc); + u->openssl_error_code_ret = rc; llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (llcf->log_socket_errors) { ngx_log_error(NGX_LOG_ERR, dc->log, 0, "lua ssl " - "certificate verify error: (%s)", err); + "certificate verify error: (%d: %s)", + rc, u->error_ret); } goto failed; @@ -1872,8 +1873,7 @@ ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) if (u->ssl_name.len && ngx_ssl_check_host(c, &u->ssl_name) != NGX_OK) { - lua_pushnil(L); - lua_pushliteral(L, "certificate host mismatch"); + u->error_ret = "certificate host mismatch"; llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); if (llcf->log_socket_errors) { @@ -1892,7 +1892,7 @@ ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) ngx_http_lua_socket_handle_conn_success(r, u); } else { - (void) ngx_http_lua_ssl_handshake_retval_handler(r, u, L); + (void) ngx_http_lua_ssl_handshake_retval_handler(r, u, NULL); } if (waiting) { @@ -1902,60 +1902,83 @@ ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c) return; } - lua_pushnil(L); - lua_pushliteral(L, "handshake failed"); + u->error_ret = "handshake failed"; failed: if (waiting) { u->write_prepare_retvals = - ngx_http_lua_socket_conn_error_retval_handler; - ngx_http_lua_socket_handle_conn_error(r, u, - NGX_HTTP_LUA_SOCKET_FT_SSL); + ngx_http_lua_socket_conn_error_retval_handler; + ngx_http_lua_socket_handle_conn_error(r, u, NGX_HTTP_LUA_SOCKET_FT_SSL); ngx_http_run_posted_requests(dc); } else { - (void) ngx_http_lua_socket_conn_error_retval_handler(r, u, L); + u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_SSL; + + (void) ngx_http_lua_socket_conn_error_retval_handler(r, u, NULL); } } +int +ngx_http_lua_ffi_socket_tcp_get_sslhandshake_result(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, ngx_ssl_session_t **sess, + const char **errmsg, int *openssl_error_code) +{ + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua cosocket get SSL handshake result for upstream: %p", u); + + if (u->error_ret != NULL) { + *errmsg = u->error_ret; + *openssl_error_code = u->openssl_error_code_ret; + + return NGX_ERROR; + } + + *sess = u->ssl_session_ret; + + return NGX_OK; +} + + static int ngx_http_lua_ssl_handshake_retval_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) { ngx_connection_t *c; - ngx_ssl_session_t *ssl_session, **ud; + ngx_ssl_session_t *ssl_session; if (!u->ssl_session_reuse) { - lua_pushboolean(L, 1); - return 1; + return 0; } - ud = lua_newuserdata(L, sizeof(ngx_ssl_session_t *)); - c = u->peer.connection; ssl_session = ngx_ssl_get_session(c); if (ssl_session == NULL) { - *ud = NULL; + u->ssl_session_ret = NULL; } else { - *ud = ssl_session; + u->ssl_session_ret = ssl_session; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "lua ssl save session: %p", ssl_session); - - /* set up the __gc metamethod */ - lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( - ssl_session_metatable_key)); - lua_rawget(L, LUA_REGISTRYINDEX); - lua_setmetatable(L, -2); } - return 1; + return 0; } + +void +ngx_http_lua_ffi_ssl_free_session(ngx_ssl_session_t *sess) +{ + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua ssl free session: %p", sess); + + ngx_ssl_free_session(sess); +} + + #endif /* NGX_HTTP_SSL */ @@ -2008,12 +2031,14 @@ ngx_http_lua_socket_prepare_error_retvals(ngx_http_request_t *r, u_char errstr[NGX_MAX_ERROR_STR]; u_char *p; - if (ft_type & (NGX_HTTP_LUA_SOCKET_FT_RESOLVER - | NGX_HTTP_LUA_SOCKET_FT_SSL)) - { + if (ft_type & NGX_HTTP_LUA_SOCKET_FT_RESOLVER) { return 2; } + if (ft_type & NGX_HTTP_LUA_SOCKET_FT_SSL) { + return 0; + } + lua_pushnil(L); if (ft_type & NGX_HTTP_LUA_SOCKET_FT_TIMEOUT) { @@ -6105,27 +6130,6 @@ ngx_http_lua_coctx_cleanup(void *data) } -#if (NGX_HTTP_SSL) - -static int -ngx_http_lua_ssl_free_session(lua_State *L) -{ - ngx_ssl_session_t **psession; - - psession = lua_touserdata(L, 1); - if (psession && *psession != NULL) { - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua ssl free session: %p", *psession); - - ngx_ssl_free_session(*psession); - } - - return 0; -} - -#endif /* NGX_HTTP_SSL */ - - void ngx_http_lua_cleanup_conn_pools(lua_State *L) { diff --git a/src/ngx_http_lua_socket_tcp.h b/src/ngx_http_lua_socket_tcp.h index a0a5a5181a..ee9411bc8d 100644 --- a/src/ngx_http_lua_socket_tcp.h +++ b/src/ngx_http_lua_socket_tcp.h @@ -120,6 +120,9 @@ struct ngx_http_lua_socket_tcp_upstream_s { #if (NGX_HTTP_SSL) ngx_str_t ssl_name; + ngx_ssl_session_t *ssl_session_ret; + const char *error_ret; + int openssl_error_code_ret; #endif unsigned ft_type:16; diff --git a/t/062-count.t b/t/062-count.t index 104126281a..d977909bb6 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -459,7 +459,7 @@ worker: 4 --- request GET /test --- response_body -n = 14 +n = 15 --- no_error_log [error] diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index c26f0cec53..cbbd679468 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -106,7 +106,7 @@ __DATA__ GET /t --- response_body_like chop \Aconnected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 59 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil @@ -190,7 +190,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 53 bytes. received: HTTP/1.1 201 Created close: 1 nil @@ -271,7 +271,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 58 bytes. received: HTTP/1.1 302 Moved Temporarily close: 1 nil @@ -355,12 +355,12 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 200 OK close: 1 nil connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 200 OK close: 1 nil @@ -603,7 +603,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 80 bytes. received: HTTP/1.1 404 Not Found close: 1 nil @@ -688,7 +688,7 @@ $::DSTRootCertificate" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 58 bytes. received: HTTP/1.1 302 Moved Temporarily close: 1 nil @@ -1008,7 +1008,7 @@ $::DSTRootCertificate" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 58 bytes. received: HTTP/1.1 302 Moved Temporarily close: 1 nil @@ -1087,7 +1087,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 58 bytes. received: HTTP/1.1 302 Moved Temporarily close: 1 nil @@ -1179,7 +1179,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 53 bytes. received: HTTP/1.1 200 OK close: 1 nil @@ -1269,7 +1269,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 53 bytes. received: HTTP/1.1 200 OK close: 1 nil @@ -1418,13 +1418,13 @@ $::DSTRootCertificate" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata set keepalive: 1 nil connected: 1 -ssl handshake: userdata +ssl handshake: cdata set keepalive: 1 nil connected: 1 -ssl handshake: userdata +ssl handshake: cdata set keepalive: 1 nil --- log_level: debug @@ -1494,13 +1494,13 @@ $::DSTRootCertificate" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata set keepalive: 1 nil connected: 1 -ssl handshake: userdata +ssl handshake: cdata set keepalive: 1 nil connected: 1 -ssl handshake: userdata +ssl handshake: cdata set keepalive: 1 nil --- log_level: debug @@ -1637,7 +1637,7 @@ attempt to call method 'sslhandshake' (a nil value) GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1742,7 +1742,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2031,8 +2031,8 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata -ssl handshake: userdata +ssl handshake: cdata +ssl handshake: cdata sent http request: 58 bytes. received: HTTP/1.1 302 Moved Temporarily close: 1 nil @@ -2232,7 +2232,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- user_files eval ">>> test.key @@ -2405,7 +2405,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2529,7 +2529,7 @@ SSL reused session -=== TEST 31: handshake, too many arguments +=== TEST 31: handshake, too few arguments --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2633,7 +2633,7 @@ qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including t GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 53 bytes. received: HTTP/1.1 200 OK close: 1 nil @@ -2726,7 +2726,7 @@ SSL reused session GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 53 bytes. received: HTTP/1.1 200 OK close: 1 nil diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 502e0ac184..2180466907 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -100,7 +100,7 @@ __DATA__ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -209,7 +209,7 @@ ssl_certificate_by_lua\(nginx.conf:28\):1: ssl cert by lua is running!,/ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -314,7 +314,7 @@ qr/elapsed in ssl cert by lua: 0.(?:09|1\d)\d+,/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -432,7 +432,7 @@ my timer run! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -924,7 +924,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- error_log lua ssl server name: "test.com" @@ -1137,7 +1137,7 @@ print("ssl cert by lua is running!") GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1248,7 +1248,7 @@ a.lua:1: ssl cert by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1372,7 +1372,7 @@ lua ssl server name: "test.com" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1468,7 +1468,7 @@ GitHub openresty/lua-resty-core#42 GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1566,7 +1566,7 @@ github issue #723 GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1668,7 +1668,7 @@ ssl_certificate_by_lua:1: ssl cert by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 59 bytes. received: HTTP/1.1 200 OK received: Server: nginx @@ -1963,7 +1963,7 @@ qr/\[info\] .*? SSL_do_handshake\(\) failed\b/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2063,7 +2063,7 @@ client ip: 127.0.0.1 GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2221,7 +2221,7 @@ qr/elapsed in ssl_certificate_by_lua\*: 0\.(?:09|1\d)\d+,/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- no_error_log [error] [alert] @@ -2311,7 +2311,7 @@ ssl handshake: userdata GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- no_error_log [error] [alert] diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 31a6e2f434..5b2ae018fe 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -217,7 +217,7 @@ __DATA__ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -371,7 +371,7 @@ lua ssl server name: "test.com" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -499,7 +499,7 @@ lua ssl server name: "test.com" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -650,7 +650,7 @@ failed to parse PEM priv key: PEM_read_bio_PrivateKey() failed GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -800,7 +800,7 @@ lua ssl server name: "test.com" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1182,7 +1182,7 @@ client certificate subject: nil GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1306,7 +1306,7 @@ lua ssl server name: "test.com" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 6769a336bc..61598732be 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -77,7 +77,7 @@ __DATA__ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -152,7 +152,7 @@ ssl_session_store_by_lua\(nginx\.conf:25\):1: ssl session store by lua is runnin GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -227,7 +227,7 @@ API disabled in the context of ssl_session_store_by_lua* GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -319,7 +319,7 @@ my timer run! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -390,7 +390,7 @@ API disabled in the context of ssl_session_store_by_lua* GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -462,7 +462,7 @@ ngx.exit does not yield and the error code is eaten. GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -532,7 +532,7 @@ ssl_session_store_by_lua*: handler return value: 0, sess new cb exit code: 0 GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -600,7 +600,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -671,7 +671,7 @@ get_phase: ssl_session_store GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log eval @@ -744,7 +744,7 @@ print("ssl store session by lua is running!") GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -820,7 +820,7 @@ a.lua:1: ssl store session by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- no_error_log @@ -891,7 +891,7 @@ qr/\[emerg\] .*? "ssl_session_store_by_lua_block" directive is not allowed here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log @@ -955,7 +955,7 @@ ssl_session_store_by_lua(nginx.conf:25):1: ssl session store by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- error_log eval qr/ssl_session_store_by_lua\*: skipped since TLS version >= 1\.3 \(\d+\)/ diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 0e90c23665..8c5174dd94 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -80,7 +80,7 @@ __DATA__ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval: qr/ssl_session_fetch_by_lua\(nginx\.conf:\d+\):.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ @@ -180,7 +180,7 @@ ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -264,7 +264,7 @@ qr/elapsed in ssl fetch session by lua: 0.(?:09|1[01])\d+,/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -365,7 +365,7 @@ qr/my timer run!/s GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -446,7 +446,7 @@ qr/received memc reply: OK/s GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -527,7 +527,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -609,7 +609,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -690,7 +690,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -773,7 +773,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -853,7 +853,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -939,7 +939,7 @@ qr/get_phase: ssl_session_fetch/s GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -1025,7 +1025,7 @@ print("ssl fetch sess by lua is running!") GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -1109,7 +1109,7 @@ qr/\S+:\d+: ssl fetch sess by lua is running!/s GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval @@ -1285,7 +1285,7 @@ GET /t GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- no_error_log [warn] @@ -1346,7 +1346,7 @@ close: 1 nil GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval: qr/ssl_session_fetch_by_lua\(nginx\.conf:\d+\):.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval @@ -1444,7 +1444,7 @@ ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is run GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, @@ -1537,7 +1537,7 @@ qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval: qr/received memc reply of \d+ bytes/ --- grep_error_log_out eval @@ -1632,7 +1632,7 @@ close: 1 nil GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval: qr/uthread: [^.,]+/ --- grep_error_log_out eval @@ -1732,7 +1732,7 @@ uthread: failed to kill: already waited or killed GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata close: 1 nil --- grep_error_log eval: qr/uthread: [^.,]+/ --- grep_error_log_out eval diff --git a/t/155-tls13.t b/t/155-tls13.t index 9f7cf86d2e..4e684cd335 100644 --- a/t/155-tls13.t +++ b/t/155-tls13.t @@ -91,7 +91,7 @@ __DATA__ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- user_files eval ">>> test.key diff --git a/t/162-socket-tls-handshake.t b/t/162-socket-tls-handshake.t new file mode 100644 index 0000000000..c4076af042 --- /dev/null +++ b/t/162-socket-tls-handshake.t @@ -0,0 +1,376 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +repeat_each(2); + +plan tests => repeat_each() * 43; + +$ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; + +log_level 'debug'; + +no_long_string(); +#no_diff(); + +sub read_file { + my $infile = shift; + open my $in, $infile + or die "cannot open $infile for reading: $!"; + my $cert = do { local $/; <$in> }; + close $in; + $cert; +} + +our $MTLSCA = read_file("t/cert/mtls_ca.crt"); +our $MTLSClient = read_file("t/cert/mtls_client.crt"); +our $MTLSClientKey = read_file("t/cert/mtls_client.key"); +our $MTLSServer = read_file("t/cert/mtls_server.crt"); +our $MTLSServerKey = read_file("t/cert/mtls_server.key"); + +our $HtmlDir = html_dir; + +our $mtls_http_config = <<"_EOC_"; +server { + listen unix:$::HtmlDir/mtls.sock ssl; + + ssl_certificate $::HtmlDir/mtls_server.crt; + ssl_certificate_key $::HtmlDir/mtls_server.key; + ssl_client_certificate $::HtmlDir/mtls_ca.crt; + ssl_verify_client on; + server_tokens off; + + location / { + return 200 "hello, \$ssl_client_s_dn"; + } +} +_EOC_ + +our $mtls_user_files = <<"_EOC_"; +>>> mtls_server.key +$::MTLSServerKey +>>> mtls_server.crt +$::MTLSServer +>>> mtls_ca.crt +$::MTLSCA +>>> mtls_client.key +$::MTLSClientKey +>>> mtls_client.crt +$::MTLSClient +_EOC_ + +run_tests(); + +__DATA__ + +=== TEST 1: sanity: www.google.com +--- config + server_tokens off; + resolver $TEST_NGINX_RESOLVER ipv6=off; + + location /t { + content_by_lua_block { + -- avoid flushing google in "check leak" testing mode: + local counter = package.loaded.counter + if not counter then + counter = 1 + elseif counter >= 2 then + return ngx.exit(503) + else + counter = counter + 1 + end + + package.loaded.counter = counter + + do + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("www.google.com", 443) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake() + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + local line, err = sock:receive() + if not line then + ngx.say("failed to receive response status line: ", err) + return + end + + ngx.say("received: ", line) + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + + collectgarbage() + } + } +--- request +GET /t +--- response_body_like chop +\Aconnected: 1 +ssl handshake: cdata +sent http request: 59 bytes. +received: HTTP/1.1 (?:200 OK|302 Found) +close: 1 nil +\z +--- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ +--- grep_error_log_out eval +qr/^lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +$/ +--- no_error_log +lua ssl server name: +SSL reused session +[error] +[alert] +--- timeout: 5 + + + +=== TEST 2: mutual TLS handshake, upstream is not accessible without client certs +--- http_config eval: $::mtls_http_config +--- config eval +" + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = sock:connect('unix:$::HtmlDir/mtls.sock') + if not ok then + ngx.say('failed to connect: ', err) + end + + assert(sock:sslhandshake()) + + ngx.say('connected: ', ok) + + local req = 'GET /\\r\\n' + + local bytes, err = sock:send(req) + if not bytes then + ngx.say('failed to send request: ', err) + return + end + + ngx.say('request sent: ', bytes) + + ngx.say(sock:receive('*a')) + + assert(sock:close()) + } + } +" +--- user_files eval: $::mtls_user_files +--- request +GET /t +--- response_body_like: 400 No required SSL certificate was sent +--- no_error_log +[alert] +[error] +[crit] +[emerg] + + + +=== TEST 3: mutual TLS handshake, upstream is accessible with client certs +--- http_config eval: $::mtls_http_config +--- config eval +" + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local ok, err = sock:connect('unix:$::HtmlDir/mtls.sock') + if not ok then + ngx.say('failed to connect: ', err) + end + + local f = assert(io.open('$::HtmlDir/mtls_client.crt')) + local cert_data = f:read('*a') + f:close() + + f = assert(io.open('$::HtmlDir/mtls_client.key')) + local key_data = f:read('*a') + f:close() + + local ssl = require('ngx.ssl') + + local chain = assert(ssl.parse_pem_cert(cert_data)) + local priv = assert(ssl.parse_pem_priv_key(key_data)) + + sock:setclientcert(chain, priv) + + assert(sock:sslhandshake()) + + ngx.say('connected: ', ok) + + local req = 'GET /\\r\\n' + + local bytes, err = sock:send(req) + if not bytes then + ngx.say('failed to send request: ', err) + return + end + + ngx.say('request sent: ', bytes) + + ngx.say(sock:receive('*a')) + + assert(sock:close()) + } + } +" +--- user_files eval: $::mtls_user_files +--- request +GET /t +--- response_body +connected: 1 +request sent: 7 +hello, CN=foo@example.com,O=OpenResty,ST=California,C=US +--- no_error_log +[alert] +[error] +[crit] +[emerg] + + + +=== TEST 4: incorrect type of client cert +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + + local ok, err = sock:setclientcert("doesnt", "work") + if not ok then + ngx.say('failed to setclientcert: ', err) + return + end + + assert(sock:close()) + } + } +--- request +GET /t +--- response_body +failed to setclientcert: bad cert arg: cdata expected, got string +--- no_error_log +[alert] +[error] +[crit] +[emerg] + + + +=== TEST 5: incorrect type of client key +--- config eval +" + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + + local f = assert(io.open('$::HtmlDir/mtls_client.crt')) + local cert_data = f:read('*a') + f:close() + + local ssl = require('ngx.ssl') + + local chain = assert(ssl.parse_pem_cert(cert_data)) + + local ok, err = sock:setclientcert(chain, 'work') + if not ok then + ngx.say('failed to setclientcert: ', err) + return + end + + assert(sock:close()) + } + } +" +--- user_files eval: $::mtls_user_files +--- request +GET /t +--- response_body +failed to setclientcert: bad pkey arg: cdata expected, got string +--- no_error_log +[alert] +[error] +[crit] +[emerg] + + + +=== TEST 6: missing client cert +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + + local ok, err = sock:setclientcert(nil, "work") + if not ok then + ngx.say('failed to setclientcert: ', err) + return + end + + assert(sock:close()) + } + } +--- request +GET /t +--- response_body +failed to setclientcert: client certificate must be supplied with corresponding private key +--- no_error_log +[alert] +[error] +[crit] +[emerg] + + + +=== TEST 7: missing private key +--- config + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + + local ok, err = sock:setclientcert('doesnt', nil) + if not ok then + ngx.say('failed to setclientcert: ', err) + return + end + + assert(sock:close()) + } + } +--- request +GET /t +--- response_body +failed to setclientcert: client certificate must be supplied with corresponding private key +--- no_error_log +[alert] +[error] +[crit] +[emerg] + + + diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index 79787f63a0..da021300af 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -100,7 +100,7 @@ __DATA__ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -209,7 +209,7 @@ ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -314,7 +314,7 @@ qr/elapsed in ssl client hello by lua: 0.(?:09|1\d)\d+,/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -432,7 +432,7 @@ my timer run! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -924,7 +924,7 @@ should never reached here GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- error_log lua ssl server name: "test.com" @@ -1135,7 +1135,7 @@ print("ssl client hello by lua is running!") GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1246,7 +1246,7 @@ a.lua:1: ssl client hello by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1370,7 +1370,7 @@ lua ssl server name: "test.com" GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1466,7 +1466,7 @@ GitHub openresty/lua-resty-core#42 GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1561,7 +1561,7 @@ ssl_client_hello_by_lua:1: ssl client hello by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -1817,7 +1817,7 @@ ssl_client_hello_by_lua:1: ssl client hello by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2005,7 +2005,7 @@ ssl_client_hello_by_lua:1: ssl client hello by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 57 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2103,7 +2103,7 @@ ssl_client_hello_by_lua:1: ssl client hello by lua in server2 is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2208,7 +2208,7 @@ ssl_client_hello_by_lua:1: ssl client hello by lua is running! GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2308,7 +2308,7 @@ client ip: 127.0.0.1 GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata sent http request: 56 bytes. received: HTTP/1.1 201 Created received: Server: nginx @@ -2465,7 +2465,7 @@ qr/elapsed in ssl_client_hello_by_lua\*: 0\.(?:09|1\d)\d+,/, GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- no_error_log [error] [alert] @@ -2555,7 +2555,7 @@ ssl handshake: userdata GET /t --- response_body connected: 1 -ssl handshake: userdata +ssl handshake: cdata --- no_error_log [error] [alert] diff --git a/t/cert/mtls_ca.crt b/t/cert/mtls_ca.crt new file mode 100644 index 0000000000..1fe7e1f985 --- /dev/null +++ b/t/cert/mtls_ca.crt @@ -0,0 +1,78 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: + 32:ed:21:56:d8:4e:aa:03:89:a9:4a:a4:e2:85:2d:8a:3b:2b:89:22 + Signature Algorithm: sha256WithRSAEncryption + Issuer: C = US, ST = California, O = OpenResty, CN = OpenResty Testing Root CA + Validity + Not Before: Mar 13 15:49:00 2022 GMT + Not After : Mar 8 15:49:00 2042 GMT + Subject: C = US, ST = California, O = OpenResty, CN = OpenResty Testing Root CA + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + RSA Public-Key: (2048 bit) + Modulus: + 00:e6:37:d2:c6:17:36:c7:b2:7f:7d:cf:d0:62:87: + 99:d9:21:b8:de:ff:d8:e2:3a:1c:68:90:8f:ce:17: + 68:22:b0:60:30:cc:29:e8:34:ee:ff:b2:25:de:6e: + 1a:d4:df:10:19:11:4b:40:61:d3:a9:4d:80:ed:97: + 81:4e:c5:74:e8:4d:63:e3:5f:21:bc:5a:6e:22:a0: + 17:91:c1:cb:25:53:9b:9d:4e:e1:51:5b:f6:52:e7: + 0a:27:f6:16:c2:31:cb:6c:47:f4:89:51:15:cc:06: + be:31:3e:1c:ea:ee:81:9b:c4:97:96:fd:e5:1c:95: + 9e:c0:65:cd:a9:9a:cb:68:67:f2:62:a0:21:eb:5a: + c5:a1:92:ed:32:41:28:f9:47:34:eb:44:ae:d6:e7: + 76:71:11:98:c9:2e:ce:6c:7c:10:1b:c7:4c:c3:14: + 89:4e:d9:4c:d9:c7:43:e9:3c:29:ca:62:a9:91:b3: + 87:e7:d7:b4:18:ab:65:f9:6b:ed:82:ca:a1:36:35: + 18:05:cb:5c:24:26:13:13:f8:99:ac:99:be:9b:a6: + 73:df:0d:16:95:b1:dc:be:fe:7a:c2:b6:dc:c8:93: + cf:10:e0:29:03:0e:28:78:18:84:ee:14:92:ab:be: + 5a:a0:14:a2:4a:2f:d3:d0:b8:0e:00:d2:5a:cd:e4: + bd:a1 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Key Usage: critical + Certificate Sign, CRL Sign + X509v3 Basic Constraints: critical + CA:TRUE + X509v3 Subject Key Identifier: + F0:D7:4B:14:73:E1:67:00:6B:54:B4:19:20:76:12:9F:9D:8E:C8:09 + Signature Algorithm: sha256WithRSAEncryption + 6d:52:21:6d:6e:8c:e5:4a:28:07:65:6d:d8:7c:23:2e:c6:c1: + d0:ec:27:b3:b0:c3:d3:e8:fa:72:b9:de:32:4e:ff:97:8d:86: + a9:6d:b3:a9:b4:2d:77:ca:28:97:6a:3d:7b:a2:15:ed:34:dc: + 72:9f:6f:e7:01:0c:d3:28:6a:80:1b:50:09:fd:d7:2c:d8:92: + d5:10:c4:73:15:20:7d:99:dc:de:30:7b:3c:6e:e9:66:b2:0e: + 4e:1a:c1:51:57:6e:5b:b0:a9:f6:ff:0b:8f:07:67:31:40:5b: + 11:a9:06:d3:d3:76:c5:d2:56:95:9a:9e:4a:16:44:4b:32:e5: + af:dd:4b:4d:5d:57:b8:85:69:36:93:2a:c6:0c:8f:e1:42:35: + be:8e:f3:e7:35:d3:2c:3a:03:31:40:75:8e:e8:dd:57:35:20: + 5e:18:a9:76:ce:85:be:7e:3a:cf:6e:08:58:5b:47:d5:e9:c4: + ec:0e:e9:8e:3c:2d:5c:7b:59:20:5b:24:92:a0:e0:1e:a3:5a: + 67:d8:ff:7f:a5:82:f1:df:db:05:65:79:88:b1:3c:e6:01:d1: + 5a:c7:d2:6e:9a:e6:a2:da:4a:c7:19:78:d9:14:71:6e:1f:70: + f3:41:e5:b3:78:31:d5:22:0e:7c:1a:b2:43:d9:86:ff:53:ea: + 2b:ba:d2:27 +-----BEGIN CERTIFICATE----- +MIIDhDCCAmygAwIBAgIUMu0hVthOqgOJqUqk4oUtijsriSIwDQYJKoZIhvcNAQEL +BQAwWjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAoT +CU9wZW5SZXN0eTEiMCAGA1UEAxMZT3BlblJlc3R5IFRlc3RpbmcgUm9vdCBDQTAe +Fw0yMjAzMTMxNTQ5MDBaFw00MjAzMDgxNTQ5MDBaMFoxCzAJBgNVBAYTAlVTMRMw +EQYDVQQIEwpDYWxpZm9ybmlhMRIwEAYDVQQKEwlPcGVuUmVzdHkxIjAgBgNVBAMT +GU9wZW5SZXN0eSBUZXN0aW5nIFJvb3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB +DwAwggEKAoIBAQDmN9LGFzbHsn99z9Bih5nZIbje/9jiOhxokI/OF2gisGAwzCno +NO7/siXebhrU3xAZEUtAYdOpTYDtl4FOxXToTWPjXyG8Wm4ioBeRwcslU5udTuFR +W/ZS5won9hbCMctsR/SJURXMBr4xPhzq7oGbxJeW/eUclZ7AZc2pmstoZ/JioCHr +WsWhku0yQSj5RzTrRK7W53ZxEZjJLs5sfBAbx0zDFIlO2UzZx0PpPCnKYqmRs4fn +17QYq2X5a+2CyqE2NRgFy1wkJhMT+Jmsmb6bpnPfDRaVsdy+/nrCttzIk88Q4CkD +Dih4GITuFJKrvlqgFKJKL9PQuA4A0lrN5L2hAgMBAAGjQjBAMA4GA1UdDwEB/wQE +AwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBTw10sUc+FnAGtUtBkgdhKf +nY7ICTANBgkqhkiG9w0BAQsFAAOCAQEAbVIhbW6M5UooB2Vt2HwjLsbB0Owns7DD +0+j6crneMk7/l42GqW2zqbQtd8ool2o9e6IV7TTccp9v5wEM0yhqgBtQCf3XLNiS +1RDEcxUgfZnc3jB7PG7pZrIOThrBUVduW7Cp9v8LjwdnMUBbEakG09N2xdJWlZqe +ShZESzLlr91LTV1XuIVpNpMqxgyP4UI1vo7z5zXTLDoDMUB1jujdVzUgXhipds6F +vn46z24IWFtH1enE7A7pjjwtXHtZIFskkqDgHqNaZ9j/f6WC8d/bBWV5iLE85gHR +WsfSbprmotpKxxl42RRxbh9w80Hls3gx1SIOfBqyQ9mG/1PqK7rSJw== +-----END CERTIFICATE----- diff --git a/t/cert/mtls_ca.key b/t/cert/mtls_ca.key new file mode 100644 index 0000000000..d39b42f9d9 --- /dev/null +++ b/t/cert/mtls_ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEA5jfSxhc2x7J/fc/QYoeZ2SG43v/Y4jocaJCPzhdoIrBgMMwp +6DTu/7Il3m4a1N8QGRFLQGHTqU2A7ZeBTsV06E1j418hvFpuIqAXkcHLJVObnU7h +UVv2UucKJ/YWwjHLbEf0iVEVzAa+MT4c6u6Bm8SXlv3lHJWewGXNqZrLaGfyYqAh +61rFoZLtMkEo+Uc060Su1ud2cRGYyS7ObHwQG8dMwxSJTtlM2cdD6TwpymKpkbOH +59e0GKtl+WvtgsqhNjUYBctcJCYTE/iZrJm+m6Zz3w0WlbHcvv56wrbcyJPPEOAp +Aw4oeBiE7hSSq75aoBSiSi/T0LgOANJazeS9oQIDAQABAoIBAQDhH9+uNE8uUv/X +MNvvLfklWpOlBf25o+fZ3NuzRjJgEafOsCee2fyI8FWVwIfeeE8OpFm5GLDZk1+r +dwdM10xuSheO5Z1gyfF/TJwfvamA09SNrPArFkm3YhUNZNl2hykMtwSLL06oWEOu +dbXjit4VS9aNIbTlEe7O5/6Ih0W3zmr1yvUua2swmAZMx3GFA4kbjZZ9vDs27sdu +K+VY3DYRbq1HkiNFT0otfke5bObFBCG7Yp8JLyhYaIkGYFoBXuZ6JNY8EuU2+YyP +6r40tJ7StR1Q6eZJh9/1leaYGZLCh5oFyKpilTuxHbRbr5A28RJKjKvPsdDgTtQn +yHGg70FRAoGBAOhC3TQlFcT2WCCZHHql9JEEHnHVBWnL3Jg7VJuL1i6pEIz7qQkW +AtBEIY/nnTcVNfJ6eXznYtutYvvRSgQTUsBNRoj3s1z9wKOo4uw4LoIUXDEmHCr+ +49DiQyIO21SNMHA+dVxvGRDDjLI9Uc+Scb64QOodoX75HLRZG++24mtdAoGBAP2/ +gCjga2p8Jx9UnhIcrEIIGANyxEQeBdhF56Nt9CJy/Iwi3a6qQ/GkbeoDm5FhXnXo +xcBaHyv2lwi4uO/hONY8eRnYxAWMwAKMZe6VnU1hWI2Ytkh+OcMPMh7NIGQf6X1o +JZrBtnTms060TuuDjLeIlaubDR/xDrMWTMKjKbsVAoGAVLuYAZ8J6xpIGlRhbGlA +6OrMxJCHcgpahvsWKc0BLXKmRBjHmTX7fslsSRihZWgKj1SZH7U2fpgpxV6cFxKJ +nPhUJEHhoKo+bjZ92tnANdqBq7iQjCsDJ8Bz52fuIlGD+1795+PsDA6bNKdkQkrV +zlNf80kuEqmFDFJ5+6EHx00CgYAf+jkpbZa71aeMgDpnZ+uhaqm0DYuEVhBAgBa/ +9sRUbw86jc5IC7cCRcmAOzIosQ+ZZls9cV4KSUohVD4iJMzn2rkcM8AIPwOXjp/t +4DbxoHnrZjpaimW3Gjwju5AAbjEbl7tddFoNA2HHYlurvGlIW9MYzDJsOxGyKfZE +dRF2PQKBgQDUKNHgDYEjLJ99S5Fm5zN/64bKzzDtktGdqOxik5pBKcs/BvOdLM0i +eCjGz/3qrEoenFIBwF/IRz3ug90Zr8bWOu6DudReflAKI/N13dZ2gOTAfaX4ljJF +w0ohSi6xs+mu1GmtipGtNxHi/J3na2BeSnSRFSUg6Zd+oh8BZQKmNg== +-----END RSA PRIVATE KEY----- diff --git a/t/cert/mtls_cert_gen/.gitignore b/t/cert/mtls_cert_gen/.gitignore new file mode 100644 index 0000000000..f375caaefe --- /dev/null +++ b/t/cert/mtls_cert_gen/.gitignore @@ -0,0 +1,4 @@ +*.pem +*.csr +cfssl +cfssljson diff --git a/t/cert/mtls_cert_gen/generate.sh b/t/cert/mtls_cert_gen/generate.sh new file mode 100755 index 0000000000..46625fdd07 --- /dev/null +++ b/t/cert/mtls_cert_gen/generate.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +rm *.pem *.csr cfssl cfssljson + +wget -O cfssl https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl_1.6.1_linux_amd64 +wget -O cfssljson https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssljson_1.6.1_linux_amd64 +chmod +x cfssl cfssljson + +./cfssl gencert -initca -config profile.json mtls_ca.json | ./cfssljson -bare mtls_ca + +./cfssl gencert -ca mtls_ca.pem -ca-key mtls_ca-key.pem -config profile.json -profile=client mtls_client.json | ./cfssljson -bare mtls_client +./cfssl gencert -ca mtls_ca.pem -ca-key mtls_ca-key.pem -config profile.json -profile=server mtls_server.json | ./cfssljson -bare mtls_server + +openssl x509 -in mtls_ca.pem -text > ../mtls_ca.crt +mv mtls_ca-key.pem ../mtls_ca.key + +openssl x509 -in mtls_client.pem -text > ../mtls_client.crt +mv mtls_client-key.pem ../mtls_client.key + +openssl x509 -in mtls_server.pem -text > ../mtls_server.crt +mv mtls_server-key.pem ../mtls_server.key + +rm *.pem *.csr cfssl cfssljson diff --git a/t/cert/mtls_cert_gen/mtls_ca.json b/t/cert/mtls_cert_gen/mtls_ca.json new file mode 100644 index 0000000000..0a4a7ab139 --- /dev/null +++ b/t/cert/mtls_cert_gen/mtls_ca.json @@ -0,0 +1,18 @@ +{ + "CA": { + "expiry": "175200h", + "pathlen": 0 + }, + "CN": "OpenResty Testing Root CA", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "US", + "O": "OpenResty", + "ST": "California" + } + ] +} diff --git a/t/cert/mtls_cert_gen/mtls_client.json b/t/cert/mtls_cert_gen/mtls_client.json new file mode 100644 index 0000000000..4d59f47a5b --- /dev/null +++ b/t/cert/mtls_cert_gen/mtls_client.json @@ -0,0 +1,18 @@ +{ + "CN": "foo@example.com", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "US", + "O": "OpenResty", + "ST": "California" + } + ], + "hosts": [ + "foo@example.com", + "bar@example.com" + ] +} diff --git a/t/cert/mtls_cert_gen/mtls_server.json b/t/cert/mtls_cert_gen/mtls_server.json new file mode 100644 index 0000000000..655af54ef8 --- /dev/null +++ b/t/cert/mtls_cert_gen/mtls_server.json @@ -0,0 +1,17 @@ +{ + "CN": "example.com", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "US", + "O": "OpenResty", + "ST": "California" + } + ], + "hosts": [ + "example.com" + ] +} diff --git a/t/cert/mtls_cert_gen/profile.json b/t/cert/mtls_cert_gen/profile.json new file mode 100644 index 0000000000..d3b6ab16cb --- /dev/null +++ b/t/cert/mtls_cert_gen/profile.json @@ -0,0 +1,27 @@ +{ + "signing": { + "default": { + "expiry": "175200h" + }, + "profiles": { + "server": { + "usages": [ + "signing", + "digital signing", + "key encipherment", + "server auth" + ], + "expiry": "175199h" + }, + "client": { + "usages": [ + "signing", + "digital signature", + "key encipherment", + "client auth" + ], + "expiry": "175199h" + } + } + } +} diff --git a/t/cert/mtls_client.crt b/t/cert/mtls_client.crt new file mode 100644 index 0000000000..dd0efdf7f8 --- /dev/null +++ b/t/cert/mtls_client.crt @@ -0,0 +1,87 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: + 19:0a:a3:a8:9c:d4:0f:dc:c6:fa:23:7b:f8:fc:bd:f4:73:4e:7e:b1 + Signature Algorithm: sha256WithRSAEncryption + Issuer: C = US, ST = California, O = OpenResty, CN = OpenResty Testing Root CA + Validity + Not Before: Mar 13 15:49:00 2022 GMT + Not After : Mar 8 14:49:00 2042 GMT + Subject: C = US, ST = California, O = OpenResty, CN = foo@example.com + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + RSA Public-Key: (2048 bit) + Modulus: + 00:be:5b:09:4c:94:71:d3:82:54:4a:42:6a:76:aa: + 34:5d:28:d9:45:e6:44:9a:74:9f:a6:e6:78:49:9e: + c6:20:75:32:5f:92:3b:ec:6e:4b:7b:b0:75:1c:75: + 09:00:05:77:d6:59:ca:55:5b:13:b6:76:3a:c6:18: + dc:37:6a:20:93:e6:26:56:5d:0b:96:8c:01:f2:96: + 38:08:08:36:a2:64:12:21:a0:8d:48:cd:9a:26:78: + 92:29:b6:63:eb:14:d9:b6:e5:87:f7:d5:55:a4:cc: + 53:1c:a3:7c:b8:bd:ad:7c:a4:d4:86:1f:a7:1c:43: + c5:1a:b5:f1:03:bd:fe:19:98:1d:b7:13:2b:93:a2: + 2a:0e:21:7e:42:a9:bb:28:69:49:59:e7:89:0e:7d: + 5a:ce:fb:d4:0c:20:6a:e1:db:b2:6a:e5:a7:55:e0: + d0:58:4a:e2:08:78:82:b9:06:0c:65:f9:24:06:e6: + 8a:13:b2:9a:ef:1b:4a:b2:3a:b4:98:7f:dd:3c:0e: + 85:0b:a6:c6:47:2f:63:c2:73:52:41:db:7c:06:c3: + 2a:b5:2d:d1:e1:30:d5:c4:79:c9:b9:35:68:46:ad: + c4:45:57:ea:11:88:27:37:ed:ac:49:2d:c4:d6:c6: + a6:74:8d:d3:bc:e0:d9:69:25:0c:0c:b0:e3:b7:cb: + 8d:99 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Key Usage: critical + Digital Signature, Key Encipherment + X509v3 Extended Key Usage: + TLS Web Client Authentication + X509v3 Basic Constraints: critical + CA:FALSE + X509v3 Subject Key Identifier: + 22:70:5E:30:8C:4D:66:39:E7:60:C9:29:A2:ED:95:32:34:63:5C:C0 + X509v3 Authority Key Identifier: + keyid:F0:D7:4B:14:73:E1:67:00:6B:54:B4:19:20:76:12:9F:9D:8E:C8:09 + + X509v3 Subject Alternative Name: + email:foo@example.com, email:bar@example.com + Signature Algorithm: sha256WithRSAEncryption + 96:e7:2a:fc:2a:56:16:80:e2:d3:79:0c:46:db:c3:88:ab:d3: + ef:39:66:4b:a9:ab:6c:0e:30:08:07:7c:fc:03:6c:f7:dd:fb: + 3e:a8:c8:68:28:ab:4e:73:97:80:27:5d:c5:9d:52:00:aa:08: + 25:c8:f9:dc:df:64:73:a4:58:5b:bd:5f:1a:53:a4:33:a3:b1: + 45:38:2d:be:d7:f3:a4:c4:f4:7a:07:71:44:f1:a2:65:02:e4: + 71:84:01:b5:83:4b:de:83:b5:ad:ac:b9:3c:17:42:0c:9a:7d: + eb:7f:ab:26:dd:9b:3a:fd:95:37:55:cc:01:c3:3f:20:df:e5: + ed:49:51:7a:42:ea:f3:8a:3f:da:6e:c1:1a:11:b9:45:4d:6e: + c9:21:f4:e3:4f:31:72:5b:bb:01:92:b6:7f:f1:8a:9e:6c:d0: + 7f:96:d7:eb:29:09:53:38:26:41:00:f2:33:04:77:bd:a9:ee: + 60:9e:06:b7:7d:26:ae:1c:4f:56:bd:a5:b6:50:40:be:be:84: + 2a:54:21:59:47:7d:a5:1e:63:6d:28:36:4d:a6:e4:62:69:9b: + 9b:fa:2b:48:e8:64:d7:14:f4:62:a2:26:17:a5:05:58:4a:38: + d2:44:e7:33:90:b9:c1:8c:85:02:99:b8:03:1a:03:d2:cf:ac: + a5:6b:44:98 +-----BEGIN CERTIFICATE----- +MIID3DCCAsSgAwIBAgIUGQqjqJzUD9zG+iN7+Py99HNOfrEwDQYJKoZIhvcNAQEL +BQAwWjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAoT +CU9wZW5SZXN0eTEiMCAGA1UEAxMZT3BlblJlc3R5IFRlc3RpbmcgUm9vdCBDQTAe +Fw0yMjAzMTMxNTQ5MDBaFw00MjAzMDgxNDQ5MDBaMFAxCzAJBgNVBAYTAlVTMRMw +EQYDVQQIEwpDYWxpZm9ybmlhMRIwEAYDVQQKEwlPcGVuUmVzdHkxGDAWBgNVBAMM +D2Zvb0BleGFtcGxlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB +AL5bCUyUcdOCVEpCanaqNF0o2UXmRJp0n6bmeEmexiB1Ml+SO+xuS3uwdRx1CQAF +d9ZZylVbE7Z2OsYY3DdqIJPmJlZdC5aMAfKWOAgINqJkEiGgjUjNmiZ4kim2Y+sU +2bblh/fVVaTMUxyjfLi9rXyk1IYfpxxDxRq18QO9/hmYHbcTK5OiKg4hfkKpuyhp +SVnniQ59Ws771AwgauHbsmrlp1Xg0FhK4gh4grkGDGX5JAbmihOymu8bSrI6tJh/ +3TwOhQumxkcvY8JzUkHbfAbDKrUt0eEw1cR5ybk1aEatxEVX6hGIJzftrEktxNbG +pnSN07zg2WklDAyw47fLjZkCAwEAAaOBozCBoDAOBgNVHQ8BAf8EBAMCBaAwEwYD +VR0lBAwwCgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUInBeMIxN +ZjnnYMkpou2VMjRjXMAwHwYDVR0jBBgwFoAU8NdLFHPhZwBrVLQZIHYSn52OyAkw +KwYDVR0RBCQwIoEPZm9vQGV4YW1wbGUuY29tgQ9iYXJAZXhhbXBsZS5jb20wDQYJ +KoZIhvcNAQELBQADggEBAJbnKvwqVhaA4tN5DEbbw4ir0+85Zkupq2wOMAgHfPwD +bPfd+z6oyGgoq05zl4AnXcWdUgCqCCXI+dzfZHOkWFu9XxpTpDOjsUU4Lb7X86TE +9HoHcUTxomUC5HGEAbWDS96Dta2suTwXQgyafet/qybdmzr9lTdVzAHDPyDf5e1J +UXpC6vOKP9puwRoRuUVNbskh9ONPMXJbuwGStn/xip5s0H+W1+spCVM4JkEA8jME +d72p7mCeBrd9Jq4cT1a9pbZQQL6+hCpUIVlHfaUeY20oNk2m5GJpm5v6K0joZNcU +9GKiJhelBVhKONJE5zOQucGMhQKZuAMaA9LPrKVrRJg= +-----END CERTIFICATE----- diff --git a/t/cert/mtls_client.key b/t/cert/mtls_client.key new file mode 100644 index 0000000000..7d77e5506d --- /dev/null +++ b/t/cert/mtls_client.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAvlsJTJRx04JUSkJqdqo0XSjZReZEmnSfpuZ4SZ7GIHUyX5I7 +7G5Le7B1HHUJAAV31lnKVVsTtnY6xhjcN2ogk+YmVl0LlowB8pY4CAg2omQSIaCN +SM2aJniSKbZj6xTZtuWH99VVpMxTHKN8uL2tfKTUhh+nHEPFGrXxA73+GZgdtxMr +k6IqDiF+Qqm7KGlJWeeJDn1azvvUDCBq4duyauWnVeDQWEriCHiCuQYMZfkkBuaK +E7Ka7xtKsjq0mH/dPA6FC6bGRy9jwnNSQdt8BsMqtS3R4TDVxHnJuTVoRq3ERVfq +EYgnN+2sSS3E1samdI3TvODZaSUMDLDjt8uNmQIDAQABAoIBACqRsUKu78WdH7x7 +ndNrvMoYmH5JQI5KBmoMoFnWZ/haPSmiSkRVZgwDKi1y/tBCaMpGyjjMZVwolHw4 +kwbRdPeeQHSP2keQh974OQ+SxqUKPAPJI89kK1TvIcCySSYJQ6bjLcT+sGhqSSve +Y8XspR96vQxBh92KSknu5jcwBeMy/eG0mmszzP3y2R0BPztuZdE6dq/KxWQ/R4/P +JG9V1rNkIY+1JZvIICIH1Ehn4UKjiE+FJmyDbDlPKEi7W4CpRnShMLOF4cCFnQLW +RQds3Dj9GcVY+8Q/GLZF0ATjekIyEsKZEgrMAUF5ZSGRpjJQEHX7oseAiQGQxtHT +nj5b1AECgYEAwewXbbd1MqRQ6ohfsQ8j5HSMY6ahvUzs1dZUckr2jw8B98tfi/uj +a6Jq1KZe12+4dfwruRSaYdTsSVuvNiSJOxElY0C1p+lXdprFf7XfoQ6UNtg22jcH +9f8cftnlJoV5whh3YKjqnnnAWUQZ61FTNJ258/t+x0ZgpBJvqBoHwDUCgYEA+0qp +FZ5xS4FLJMc+Xf/hUeXo+04e4OD/se3atYqyuh1ghmQZfRRPOC110HG99H+rzq/x +xPMvRFahkAMyi+/3oIcBEuXvoQyqscIsAhkWD/e9t3Qc9OsWe1hlAgWKZxr6oR2U +KKR1FD7UVecOH+FKCKaL5UpEt4yEigc1NtSlTFUCgYBnV5agrIyzQSex5J0CMWxS +Od362PkGdXEc/8we4F4GnNvSnrm7Uo2jNXmy+zo9mtb1YT43sogXLK4C5e44bz4G +kTuYagqkgdBPb2lihpy3KprHo2+P2JXQfXRFEX9xiN37Fqi/hSUK8R0VNRqO8dbi +ik9nexXzwkiMBxsjvUN2JQKBgFy62FpZ9YTfWVNhEuqtGgCWzrqtwUdKwBBwrVyA +qiNz48Kz/ZPigrlATVF2J5qp4kSLOLRs6OxW65exFl39V2utZgALSbosanDeLk83 +4qRRz3h7KJRYjBtIKz3rvX7+va3mtF2rEmk+Jizs7pFlGWTH0Kf0GBeDiwVEU6bA +IZ9hAoGAQTjnRGMjvyhq0aPYP+mRFiMKSkcL1nyXizYInfAnbfbL/uEODH7D+iMf +kak+UgmeD9ce5d/APmZp3/FzYH/M8ivBgG+MnaI+MLVMhmQdLZyMtbSKKaDpiim7 +DdN1wCXYbur0HlO2t+wemMZPpQu7wybgEOLlIG7Yj/0OWDcal1c= +-----END RSA PRIVATE KEY----- diff --git a/t/cert/mtls_server.crt b/t/cert/mtls_server.crt new file mode 100644 index 0000000000..7cbc2a804b --- /dev/null +++ b/t/cert/mtls_server.crt @@ -0,0 +1,87 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: + 2f:d5:41:13:5a:ff:c7:1c:5b:ce:28:cd:a6:f7:a5:5a:0d:c0:e2:d2 + Signature Algorithm: sha256WithRSAEncryption + Issuer: C = US, ST = California, O = OpenResty, CN = OpenResty Testing Root CA + Validity + Not Before: Mar 13 15:49:00 2022 GMT + Not After : Mar 8 14:49:00 2042 GMT + Subject: C = US, ST = California, O = OpenResty, CN = example.com + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + RSA Public-Key: (2048 bit) + Modulus: + 00:d7:03:80:a7:42:7d:06:5a:7b:70:d8:11:96:dd: + 63:35:53:07:28:71:52:05:40:55:83:61:a7:14:ac: + cf:4b:9b:ab:b7:4e:9d:79:e9:13:3d:bc:c3:67:8f: + dd:88:d9:8b:c2:31:aa:b8:28:9e:13:70:db:76:b0: + 12:1c:f8:35:c6:2e:33:9c:b9:04:e3:47:e0:f9:e4: + 7f:a5:55:03:0c:2d:b2:54:17:29:12:dd:61:6e:5c: + 33:9f:e5:8f:8a:2b:41:53:dc:e1:98:49:63:df:e3: + 00:30:2d:1b:bb:f0:8f:cb:04:ec:c9:98:c4:09:5b: + b4:ba:a9:a0:0a:77:d2:42:76:7c:ac:64:c3:97:85: + 50:5d:7d:02:61:2a:00:93:d0:69:5e:87:22:f0:c1: + 1e:53:46:02:40:37:c9:55:77:99:7d:9d:3d:35:14: + 74:84:e3:73:ca:e7:4a:ab:33:98:26:aa:41:4b:b5: + e6:63:7c:a4:1e:25:6a:88:f4:56:d9:2c:63:dd:89: + 19:fa:25:41:44:95:87:40:a7:9b:4e:3a:91:29:32: + 79:66:05:f4:2f:68:2c:06:53:df:4d:60:be:ac:09: + 20:61:9c:6f:1a:a6:07:5a:e7:41:91:9d:36:77:38: + 18:3a:69:7b:67:29:9f:1d:e0:c2:d2:8f:16:5b:14: + e8:e1 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Key Usage: critical + Digital Signature, Key Encipherment + X509v3 Extended Key Usage: + TLS Web Server Authentication + X509v3 Basic Constraints: critical + CA:FALSE + X509v3 Subject Key Identifier: + 16:07:B5:C2:4C:B5:2D:4F:B8:E9:D6:FA:2F:3F:C0:1B:B6:4F:20:E6 + X509v3 Authority Key Identifier: + keyid:F0:D7:4B:14:73:E1:67:00:6B:54:B4:19:20:76:12:9F:9D:8E:C8:09 + + X509v3 Subject Alternative Name: + DNS:example.com + Signature Algorithm: sha256WithRSAEncryption + d9:c0:c0:d6:8b:44:04:26:b3:98:24:2c:12:82:6d:15:79:92: + 76:c9:77:94:c1:be:8f:8a:18:78:96:04:68:c9:0a:d1:84:c5: + de:cd:ba:b5:a2:3b:d4:0a:70:be:00:49:19:c0:6e:ca:e9:e5: + 8b:b6:e3:a2:39:0d:d8:ee:55:1a:08:73:39:19:d3:07:07:33: + 8c:d8:1b:0f:1b:73:0e:84:72:cf:e6:c1:a1:da:39:aa:c0:2e: + 3d:b9:a6:8f:ec:98:3a:07:58:34:c2:5e:4c:1a:6b:db:ce:51: + 92:25:1d:ba:78:4b:11:b6:f1:69:02:cb:ac:32:bb:80:f9:15: + 91:bf:4e:6a:ab:51:51:7c:7b:1a:72:80:96:eb:0c:fa:56:0e: + f2:87:3c:16:8a:04:aa:8a:9d:0c:d9:e0:c4:2a:20:42:5a:12: + 41:52:30:50:3d:85:f8:07:31:6b:af:a4:d2:44:38:69:ab:88: + 05:d4:5b:68:34:02:dc:99:5a:6c:b7:ea:fc:79:76:fe:68:29: + df:94:22:58:46:f2:40:cb:e1:92:17:d8:1e:3d:fa:a2:56:4f: + ac:3c:3d:ae:f7:90:12:ac:3b:6c:1e:1f:26:48:08:87:9a:0e: + 8d:9d:75:ef:86:1e:63:ac:e9:14:47:ad:3f:4f:10:57:2a:d1: + 95:ec:6f:24 +-----BEGIN CERTIFICATE----- +MIIDwzCCAqugAwIBAgIUL9VBE1r/xxxbzijNpvelWg3A4tIwDQYJKoZIhvcNAQEL +BQAwWjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAoT +CU9wZW5SZXN0eTEiMCAGA1UEAxMZT3BlblJlc3R5IFRlc3RpbmcgUm9vdCBDQTAe +Fw0yMjAzMTMxNTQ5MDBaFw00MjAzMDgxNDQ5MDBaMEwxCzAJBgNVBAYTAlVTMRMw +EQYDVQQIEwpDYWxpZm9ybmlhMRIwEAYDVQQKEwlPcGVuUmVzdHkxFDASBgNVBAMT +C2V4YW1wbGUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1wOA +p0J9Blp7cNgRlt1jNVMHKHFSBUBVg2GnFKzPS5urt06deekTPbzDZ4/diNmLwjGq +uCieE3DbdrASHPg1xi4znLkE40fg+eR/pVUDDC2yVBcpEt1hblwzn+WPiitBU9zh +mElj3+MAMC0bu/CPywTsyZjECVu0uqmgCnfSQnZ8rGTDl4VQXX0CYSoAk9BpXoci +8MEeU0YCQDfJVXeZfZ09NRR0hONzyudKqzOYJqpBS7XmY3ykHiVqiPRW2Sxj3YkZ ++iVBRJWHQKebTjqRKTJ5ZgX0L2gsBlPfTWC+rAkgYZxvGqYHWudBkZ02dzgYOml7 +ZymfHeDC0o8WWxTo4QIDAQABo4GOMIGLMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE +DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBQWB7XCTLUtT7jp +1vovP8Abtk8g5jAfBgNVHSMEGDAWgBTw10sUc+FnAGtUtBkgdhKfnY7ICTAWBgNV +HREEDzANggtleGFtcGxlLmNvbTANBgkqhkiG9w0BAQsFAAOCAQEA2cDA1otEBCaz +mCQsEoJtFXmSdsl3lMG+j4oYeJYEaMkK0YTF3s26taI71ApwvgBJGcBuyunli7bj +ojkN2O5VGghzORnTBwczjNgbDxtzDoRyz+bBodo5qsAuPbmmj+yYOgdYNMJeTBpr +285RkiUdunhLEbbxaQLLrDK7gPkVkb9OaqtRUXx7GnKAlusM+lYO8oc8FooEqoqd +DNngxCogQloSQVIwUD2F+Acxa6+k0kQ4aauIBdRbaDQC3JlabLfq/Hl2/mgp35Qi +WEbyQMvhkhfYHj36olZPrDw9rveQEqw7bB4fJkgIh5oOjZ1174YeY6zpFEetP08Q +VyrRlexvJA== +-----END CERTIFICATE----- diff --git a/t/cert/mtls_server.key b/t/cert/mtls_server.key new file mode 100644 index 0000000000..f5b85d18d8 --- /dev/null +++ b/t/cert/mtls_server.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEA1wOAp0J9Blp7cNgRlt1jNVMHKHFSBUBVg2GnFKzPS5urt06d +eekTPbzDZ4/diNmLwjGquCieE3DbdrASHPg1xi4znLkE40fg+eR/pVUDDC2yVBcp +Et1hblwzn+WPiitBU9zhmElj3+MAMC0bu/CPywTsyZjECVu0uqmgCnfSQnZ8rGTD +l4VQXX0CYSoAk9BpXoci8MEeU0YCQDfJVXeZfZ09NRR0hONzyudKqzOYJqpBS7Xm +Y3ykHiVqiPRW2Sxj3YkZ+iVBRJWHQKebTjqRKTJ5ZgX0L2gsBlPfTWC+rAkgYZxv +GqYHWudBkZ02dzgYOml7ZymfHeDC0o8WWxTo4QIDAQABAoIBAEnmZUiXnJsbbEPr +r5f3vYptYA9xa2xsoTeHz8JWZuUouwtE1PE6v6c/grXMh6rqgpObOH8VTseFyZhw +ibk1Ql48MPcTzG9FnDinZYvwvRxpdFpcn3xhZIRm4kN5xi0KEuj9CPireM1RmxXz +2w1scC+qIKxlejNxNpvVgzE136mBqEFKJzecP+yZuH/A86MQCgwqqa3jSz5ApNg+ +1aJE34cGFieDbAN+9sdqWA3OkRrHoy8EakUf4JEvwX1AwUN832mj+N/LfmcCGMeD +YhzybzlPBV2q2T1+pHIdNT99JVNPkgdTe1903EjnG5oSDGHt2i9MdnNkMsffDWNt +pJiqSHECgYEA2hL6l8Py4oa5AJ2WXriuHRJykAs90K0akftQt4i4lWCbeRhaGh7h +kPgpDS33RkE4SymVVr0c05abMCKabQBwbu4PNCqetCFtfmIQdQCTUbLbXjL8UuD2 +QnF7nbHiwyGBKRMU/F74oX3z7lXLgRtIiyyo5yYgIAQqpz3oJAaXNTUCgYEA/GhE +Ziez8FXVAg3XwwrE3SexRFKv1JqipYE4mr+ouzfpn9yn8mttxbOORiAAEBl3ZPhd +ZUBzLy19fdFZ8RJ0zPsqoZxsd09/XetaBU56C/g9u0fycj1L2elh9rQAlOW0Grus +l8jBh01TGtlg0xobK0zjwdGPcbYkp1IzIqyD9n0CgYEAicBvVyrJ5FnhxwfEkrTq +FycuAtt3Arg2DnzH8geFQaayzv2Y/OMA7Yg0tkSQ7GoKW0A7O31eFjIOeYuCLNSY +MRpjtDov4e0zsx/S8XWZmYP3mjtutBOyuyngQi655TTm18FcAkcjmy9qxOShFj7b +xj5BuzGUHWVEZDxwxUD8hvkCgYBnrcyqyZQ4HImqllUSYNIMpclC71QaWIqGwVWm ++yMsBAOLDvBNu6MTmnXOiEZ+VnecmgiDFr45ms35aI0xYQtpR6JzT/Wd7KG8ynfn +xhyL3iQ9UYhdNKB7mkoLNFUo1FHuyThUALq+AR0p4jDLheWzG5pSeuoZI2Ba+oDW +tVZfYQKBgC5phtERR5LKU5Wkzm+uY2j+Nzh4kuKkdLosB9pUW8VnrwFDLZ+r1CxG +L6CxOZ0AylCMIlrFeUXMa91kLDJYch0NUPHuGBkdIBDXi2kqN7GflTdV3Z8uev20 +uMjErA93yVOWHTR3Wo8WIHy5mdsNRQgGAPw1RVW7rnYIyXJW/mTs +-----END RSA PRIVATE KEY----- From 8df296c47992d33681cf639d2d251248d834dd2b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 15 Mar 2022 23:40:32 +0800 Subject: [PATCH 615/848] travis: fixe the uri of lua-resty-core. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index dbf1ed6acf..335ae71ee6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -88,7 +88,7 @@ install: - git clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - - git clone https://github.com/dndx/lua-resty-core.git ../lua-resty-core + - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string From 3d1431790f714fd5e6c8425b46224c2f0b7724a5 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 18 Mar 2022 08:11:44 +0800 Subject: [PATCH 616/848] travis: update openssl to 1.1.1n. (#2028) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 335ae71ee6..6e6ecee4f6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -52,7 +52,7 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1m OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1n OPENSSL_PATCH_VER=1.1.1f services: - memcached From c1d378824ce55ffab62fbf4d42ee3a7e49a79013 Mon Sep 17 00:00:00 2001 From: xiaobiaozhao <52393536+xiaobiaozhao@users.noreply.github.com> Date: Mon, 21 Mar 2022 15:45:22 +0800 Subject: [PATCH 617/848] feature: add `server_rewrite_by_lua*`. (#2004) Co-authored-by: lijunlong --- README.markdown | 93 ++++++ config | 2 + doc/HttpLuaModule.wiki | 82 +++++ src/ngx_http_lua_common.h | 12 +- src/ngx_http_lua_control.c | 3 + src/ngx_http_lua_directive.c | 106 ++++++ src/ngx_http_lua_directive.h | 4 + src/ngx_http_lua_module.c | 37 +++ src/ngx_http_lua_output.c | 4 + src/ngx_http_lua_req_body.c | 1 + src/ngx_http_lua_server_rewriteby.c | 339 +++++++++++++++++++ src/ngx_http_lua_server_rewriteby.h | 20 ++ src/ngx_http_lua_socket_tcp.c | 1 + src/ngx_http_lua_uri.c | 6 +- src/ngx_http_lua_util.c | 1 + src/ngx_http_lua_util.h | 2 + t/089-phase.t | 22 +- t/162-socket-tls-handshake.t | 3 - t/167-server-rewrite.t | 488 ++++++++++++++++++++++++++++ 19 files changed, 1219 insertions(+), 7 deletions(-) create mode 100644 src/ngx_http_lua_server_rewriteby.c create mode 100644 src/ngx_http_lua_server_rewriteby.h create mode 100644 t/167-server-rewrite.t diff --git a/README.markdown b/README.markdown index 7f0ac24cb1..98d92a6b08 100644 --- a/README.markdown +++ b/README.markdown @@ -1131,6 +1131,8 @@ Directives * [content_by_lua](#content_by_lua) * [content_by_lua_block](#content_by_lua_block) * [content_by_lua_file](#content_by_lua_file) +* [server_rewrite_by_lua_block](#server_rewrite_by_lua_block) +* [server_rewrite_by_lua_file](#server_rewrite_by_lua_file) * [rewrite_by_lua](#rewrite_by_lua) * [rewrite_by_lua_block](#rewrite_by_lua_block) * [rewrite_by_lua_file](#rewrite_by_lua_file) @@ -1909,6 +1911,97 @@ But be very careful about malicious user inputs and always carefully validate or [Back to TOC](#directives) +server_rewrite_by_lua_block +--------------------------- + +**syntax:** *server_rewrite_by_lua_block { lua-script }* + +**context:** *http, server* + +**phase:** *server rewrite* + +Acts as a server rewrite phase handler and executes Lua code string specified in `{ lua-script }` for every request. +The Lua code may make [API calls](#nginx-api-for-lua) and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). + +```nginx + + server { + ... + + server_rewrite_by_lua_block { + ngx.ctx.a = "server_rewrite_by_lua_block in http" + } + + location /lua { + content_by_lua_block { + ngx.say(ngx.ctx.a) + ngx.log(ngx.INFO, ngx.ctx.a) + } + } + } +``` + +Just as any other rewrite phase handlers, [server_rewrite_by_lua_block](#server_rewrite_by_lua_block) also runs in subrequests. + +```nginx + + server { + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "is_subrequest:", ngx.is_subrequest) + } + + location /lua { + content_by_lua_block { + local res = ngx.location.capture("/sub") + ngx.print(res.body) + } + } + + location /sub { + content_by_lua_block { + ngx.say("OK") + } + } + } +``` + +Note that when calling `ngx.exit(ngx.OK)` within a [server_rewrite_by_lua_block](#server_rewrite_by_lua_block) handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [server_rewrite_by_lua_block](#server_rewrite_by_lua_block) handler, call [ngx.exit](#ngxexit) with status >= 200 (`ngx.HTTP_OK`) and status < 300 (`ngx.HTTP_SPECIAL_RESPONSE`) for successful quits and `ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)` (or its friends) for failures. + + +```nginx + + server_rewrite_by_lua_block { + ngx.exit(503) + } + + location /bar { + ... + # never exec + } +``` + + +[Back to TOC](#directives) + +server_rewrite_by_lua_file +-------------------------- + +**syntax:** *server_rewrite_by_lua_file <path-to-lua-script-file>* + +**context:** *http, server* + +**phase:** *server rewrite* + +Equivalent to [server_rewrite_by_lua_block](#server_rewrite_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.10.22` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. + +Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. + +When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. + +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching [lua_code_cache](#lua_code_cache) `off` in `nginx.conf` to avoid reloading Nginx. + +[Back to TOC](#directives) + rewrite_by_lua -------------- diff --git a/config b/config index 14870a04e8..0e572c8bea 100644 --- a/config +++ b/config @@ -261,6 +261,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_util.c \ $ngx_addon_dir/src/ngx_http_lua_cache.c \ $ngx_addon_dir/src/ngx_http_lua_contentby.c \ + $ngx_addon_dir/src/ngx_http_lua_server_rewriteby.c \ $ngx_addon_dir/src/ngx_http_lua_rewriteby.c \ $ngx_addon_dir/src/ngx_http_lua_accessby.c \ $ngx_addon_dir/src/ngx_http_lua_setby.c \ @@ -325,6 +326,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_util.h \ $ngx_addon_dir/src/ngx_http_lua_cache.h \ $ngx_addon_dir/src/ngx_http_lua_contentby.h \ + $ngx_addon_dir/src/ngx_http_lua_server_rewriteby.c \ $ngx_addon_dir/src/ngx_http_lua_rewriteby.h \ $ngx_addon_dir/src/ngx_http_lua_accessby.h \ $ngx_addon_dir/src/ngx_http_lua_setby.h \ diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 0d69fd205b..9270d18da0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -1568,6 +1568,88 @@ Nginx variables are supported in the file path for dynamic dispatch, for example But be very careful about malicious user inputs and always carefully validate or filter out the user-supplied path components. +== server_rewrite_by_lua_block == + +'''syntax:''' ''server_rewrite_by_lua_block { lua-script }'' + +'''context:''' ''http, server'' + +'''phase:''' ''server rewrite'' + +Acts as a server rewrite phase handler and executes Lua code string specified in { lua-script } for every request. +The Lua code may make [[#Nginx API for Lua|API calls]] and is executed as a new spawned coroutine in an independent global environment (i.e. a sandbox). + + +server { + ... + + server_rewrite_by_lua_block { + ngx.ctx.a = "server_rewrite_by_lua_block in http" + } + + location /lua { + content_by_lua_block { + ngx.say(ngx.ctx.a) + ngx.log(ngx.INFO, ngx.ctx.a) + } + } +} + + +Just as any other rewrite phase handlers, [[#server_rewrite_by_lua_block|server_rewrite_by_lua_block]] also runs in subrequests. + + +server { + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "is_subrequest:", ngx.is_subrequest) + } + + location /lua { + content_by_lua_block { + local res = ngx.location.capture("/sub") + ngx.print(res.body) + } + } + + location /sub { + content_by_lua_block { + ngx.say("OK") + } + } +} + + +Note that when calling ngx.exit(ngx.OK) within a [[#server_rewrite_by_lua_block|server_rewrite_by_lua_block]] handler, the Nginx request processing control flow will still continue to the content handler. To terminate the current request from within a [[#server_rewrite_by_lua_block|server_rewrite_by_lua_block]] handler, call [[#ngx.exit|ngx.exit]] with status >= 200 (ngx.HTTP_OK) and status < 300 (ngx.HTTP_SPECIAL_RESPONSE) for successful quits and ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) (or its friends) for failures. + + + + server_rewrite_by_lua_block { + ngx.exit(503) + } + + location /bar { + ... + # never exec + } + + + +== server_rewrite_by_lua_file == + +'''syntax:''' ''server_rewrite_by_lua_file '' + +'''context:''' ''http, server'' + +'''phase:''' ''server rewrite'' + +Equivalent to [[#server_rewrite_by_lua_block|server_rewrite_by_lua_block]], except that the file specified by contains the Lua code, or, as from the v0.10.22 release, the [[#LuaJIT bytecode support|LuaJIT bytecode]] to be executed. + +Nginx variables can be used in the string to provide flexibility. This however carries some risks and is not ordinarily recommended. + +When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server. + +When the Lua code cache is turned on (by default), the user code is loaded once at the first request and cached and the Nginx config must be reloaded each time the Lua source file is modified. The Lua code cache can be temporarily disabled during development by switching [[#lua_code_cache|lua_code_cache]] off in nginx.conf to avoid reloading Nginx. + == rewrite_by_lua == '''syntax:''' ''rewrite_by_lua '' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 3a2480f041..f293c5b1d9 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -141,6 +141,7 @@ typedef struct { #define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x1000 #define NGX_HTTP_LUA_CONTEXT_EXIT_WORKER 0x2000 #define NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO 0x4000 +#define NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE 0x8000 #define NGX_HTTP_LUA_FFI_NO_REQ_CTX -100 @@ -307,6 +308,7 @@ struct ngx_http_lua_main_conf_s { unsigned requires_log:1; unsigned requires_shm:1; unsigned requires_capture_log:1; + unsigned requires_server_rewrite:1; }; @@ -336,6 +338,12 @@ union ngx_http_lua_srv_conf_u { u_char *ssl_client_hello_src_key; u_char *ssl_client_hello_chunkname; int ssl_client_hello_src_ref; + + ngx_http_lua_srv_conf_handler_pt server_rewrite_handler; + ngx_http_complex_value_t server_rewrite_src; + u_char *server_rewrite_src_key; + u_char *server_rewrite_chunkname; + int server_rewrite_src_ref; } srv; #endif @@ -378,6 +386,8 @@ typedef struct { ngx_http_output_body_filter_pt body_filter_handler; + + u_char *rewrite_chunkname; ngx_http_complex_value_t rewrite_src; /* rewrite_by_lua inline script/script @@ -631,7 +641,7 @@ typedef struct ngx_http_lua_ctx_s { response headers */ unsigned mime_set:1; /* whether the user has set Content-Type response header */ - + unsigned entered_server_rewrite_phase:1; unsigned entered_rewrite_phase:1; unsigned entered_access_phase:1; unsigned entered_content_phase:1; diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index c79101cf06..99460456c1 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -91,6 +91,7 @@ ngx_http_lua_ngx_exec(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); @@ -232,6 +233,7 @@ ngx_http_lua_ngx_redirect(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); @@ -373,6 +375,7 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, } if (ngx_http_lua_ffi_check_context(ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT | NGX_HTTP_LUA_CONTEXT_TIMER diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index c8317edf3d..f1623056b5 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -17,6 +17,7 @@ #include "ngx_http_lua_cache.h" #include "ngx_http_lua_contentby.h" #include "ngx_http_lua_accessby.h" +#include "ngx_http_lua_server_rewriteby.h" #include "ngx_http_lua_rewriteby.h" #include "ngx_http_lua_logby.h" #include "ngx_http_lua_headerfilterby.h" @@ -590,6 +591,111 @@ ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) } +char * +ngx_http_lua_server_rewrite_by_lua_block(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf) +{ + char *rv; + ngx_conf_t save; + save = *cf; + cf->handler = ngx_http_lua_server_rewrite_by_lua; + cf->handler_conf = conf; + + rv = ngx_http_lua_conf_lua_block_parse(cf, cmd); + + *cf = save; + + return rv; +} + + +char * +ngx_http_lua_server_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ + size_t chunkname_len; + u_char *cache_key = NULL, *chunkname; + ngx_str_t *value; + ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_srv_conf_t *lscf = conf; + + ngx_http_compile_complex_value_t ccv; + + dd("enter"); + + /* must specify a content handler */ + if (cmd->post == NULL) { + return NGX_CONF_ERROR; + } + + if (lscf->srv.server_rewrite_handler) { + return "is duplicate"; + } + + value = cf->args->elts; + + if (value[1].len == 0) { + /* Oops...Invalid location conf */ + ngx_conf_log_error(NGX_LOG_ERR, cf, 0, + "invalid location config: no runnable Lua code"); + + return NGX_CONF_ERROR; + } + + if (cmd->post == ngx_http_lua_server_rewrite_handler_inline) { + chunkname = + ngx_http_lua_gen_chunk_name(cf, "server_rewrite_by_lua", + sizeof("server_rewrite_by_lua") - 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + + cache_key = + ngx_http_lua_gen_chunk_cache_key(cf, "server_rewrite_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { + return NGX_CONF_ERROR; + } + + /* Don't eval nginx variables for inline lua code */ + lscf->srv.server_rewrite_src.value = value[1]; + lscf->srv.server_rewrite_chunkname = chunkname; + + } else { + ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); + ccv.cf = cf; + ccv.value = &value[1]; + ccv.complex_value = &lscf->srv.server_rewrite_src; + + if (ngx_http_compile_complex_value(&ccv) != NGX_OK) { + return NGX_CONF_ERROR; + } + + if (lscf->srv.server_rewrite_src.lengths == NULL) { + /* no variable found */ + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { + return NGX_CONF_ERROR; + } + } + } + + lscf->srv.server_rewrite_src_key = cache_key; + lscf->srv.server_rewrite_handler = + (ngx_http_lua_srv_conf_handler_pt) cmd->post; + + lmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_lua_module); + + lmcf->requires_server_rewrite = 1; + lmcf->requires_capture_filter = 1; + + return NGX_CONF_OK; +} + + char * ngx_http_lua_access_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) diff --git a/src/ngx_http_lua_directive.h b/src/ngx_http_lua_directive.h index 315c0a9f2d..4bec5e3094 100644 --- a/src/ngx_http_lua_directive.h +++ b/src/ngx_http_lua_directive.h @@ -25,6 +25,10 @@ char *ngx_http_lua_content_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_content_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +char *ngx_http_lua_server_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); +char *ngx_http_lua_server_rewrite_by_lua_block(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf); char *ngx_http_lua_rewrite_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); char *ngx_http_lua_rewrite_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index bd4535af35..243974d96d 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -14,6 +14,7 @@ #include "ngx_http_lua_directive.h" #include "ngx_http_lua_capturefilter.h" #include "ngx_http_lua_contentby.h" +#include "ngx_http_lua_server_rewriteby.h" #include "ngx_http_lua_rewriteby.h" #include "ngx_http_lua_accessby.h" #include "ngx_http_lua_logby.h" @@ -291,6 +292,22 @@ static ngx_command_t ngx_http_lua_cmds[] = { (void *) ngx_http_lua_filter_set_by_lua_file }, #endif + /* server_rewrite_by_lua_block { } */ + { ngx_string("server_rewrite_by_lua_block"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS, + ngx_http_lua_server_rewrite_by_lua_block, + NGX_HTTP_SRV_CONF_OFFSET, + 0, + (void *) ngx_http_lua_server_rewrite_handler_inline }, + + /* server_rewrite_by_lua_file filename; */ + { ngx_string("server_rewrite_by_lua_file"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_http_lua_server_rewrite_by_lua, + NGX_HTTP_SRV_CONF_OFFSET, + 0, + (void *) ngx_http_lua_server_rewrite_handler_file }, + /* rewrite_by_lua "" */ { ngx_string("rewrite_by_lua"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF @@ -753,6 +770,16 @@ ngx_http_lua_init(ngx_conf_t *cf) cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); + if (lmcf->requires_server_rewrite) { + h = ngx_array_push( + &cmcf->phases[NGX_HTTP_SERVER_REWRITE_PHASE].handlers); + if (h == NULL) { + return NGX_ERROR; + } + + *h = ngx_http_lua_server_rewrite_handler; + } + if (lmcf->requires_rewrite) { h = ngx_array_push(&cmcf->phases[NGX_HTTP_REWRITE_PHASE].handlers); if (h == NULL) { @@ -1291,6 +1318,16 @@ ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) } #endif /* NGX_HTTP_SSL */ + + if (conf->srv.server_rewrite_src.value.len == 0) { + conf->srv.server_rewrite_src = prev->srv.server_rewrite_src; + conf->srv.server_rewrite_src_ref = prev->srv.server_rewrite_src_ref; + conf->srv.server_rewrite_src_key = prev->srv.server_rewrite_src_key; + conf->srv.server_rewrite_handler = prev->srv.server_rewrite_handler; + conf->srv.server_rewrite_chunkname + = prev->srv.server_rewrite_chunkname; + } + return NGX_CONF_OK; } diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index 8d8c71d89e..b2a98d133a 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -63,6 +63,7 @@ ngx_http_lua_ngx_echo(lua_State *L, unsigned newline) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); @@ -499,6 +500,7 @@ ngx_http_lua_ngx_flush(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); @@ -653,6 +655,7 @@ ngx_http_lua_ngx_eof(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); @@ -715,6 +718,7 @@ ngx_http_lua_ngx_send_headers(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index d6e5640a0b..935fafc8a4 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -101,6 +101,7 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c new file mode 100644 index 0000000000..22bb69e8c8 --- /dev/null +++ b/src/ngx_http_lua_server_rewriteby.c @@ -0,0 +1,339 @@ + +/* + * Copyright (C) Xiaozhe Wang (chaoslawful) + * Copyright (C) Yichun Zhang (agentzh) + */ + + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + +#include +#include "ngx_http_lua_server_rewriteby.h" +#include "ngx_http_lua_util.h" +#include "ngx_http_lua_exception.h" +#include "ngx_http_lua_cache.h" + +static ngx_int_t ngx_http_lua_server_rewrite_by_chunk(lua_State *L, + ngx_http_request_t *r); + +ngx_int_t +ngx_http_lua_server_rewrite_handler(ngx_http_request_t *r) +{ + ngx_int_t rc; + lua_State *L; + ngx_http_lua_srv_conf_t *lscf; + ngx_http_lua_loc_conf_t *llcf; + ngx_http_lua_ctx_t *ctx; + + /* XXX we need to take into account ngx_rewrite's location dump */ + if (r->uri_changed) { + return NGX_DECLINED; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua server rewrite handler, uri:\"%V\" c:%ud", &r->uri, + r->main->count); + + lscf = ngx_http_get_module_srv_conf(r, ngx_http_lua_module); + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + L = ngx_http_lua_get_lua_vm(r, NULL); + + if (lscf->srv.server_rewrite_handler == NULL) { + dd("no rewrite handler found"); + return NGX_DECLINED; + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + dd("ctx = %p", ctx); + + if (ctx == NULL) { + ctx = ngx_http_lua_create_ctx(r); + if (ctx == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } + } + + dd("entered? %d", (int) ctx->entered_server_rewrite_phase); + + if (ctx->entered_server_rewrite_phase) { + dd("rewriteby: calling wev handler"); + rc = ctx->resume_handler(r); + dd("rewriteby: wev handler returns %d", (int) rc); + + if (rc == NGX_OK) { + rc = NGX_DECLINED; + } + + if (rc == NGX_DECLINED) { + if (r->header_sent) { + dd("header already sent"); + + /* response header was already generated in rewrite_by_lua*, + * so it is no longer safe to proceed to later phases + * which may generate responses again */ + + if (!ctx->eof) { + dd("eof not yet sent"); + + rc = ngx_http_lua_send_chain_link(r, ctx, NULL + /* indicate last_buf */); + if (rc == NGX_ERROR || rc > NGX_OK) { + return rc; + } + } + + return NGX_HTTP_OK; + } + + r->write_event_handler = ngx_http_core_run_phases; + ctx->entered_server_rewrite_phase = 0; + + return NGX_DECLINED; + } + + return rc; + } + + if (ctx->waiting_more_body) { + return NGX_DONE; + } + + /* TODO: lscf do not have force_read_body */ + if (llcf->force_read_body && !ctx->read_body_done) { + r->request_body_in_single_buf = 1; + r->request_body_in_persistent_file = 1; + r->request_body_in_clean_file = 1; + + rc = ngx_http_read_client_request_body(r, + ngx_http_lua_generic_phase_post_read); + + if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) { + return rc; + } + + if (rc == NGX_AGAIN) { + ctx->waiting_more_body = 1; + return NGX_DONE; + } + } + + dd("calling server rewrite handler"); + return lscf->srv.server_rewrite_handler(r, lscf, L); +} + + +ngx_int_t +ngx_http_lua_server_rewrite_handler_inline(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L) +{ + ngx_int_t rc; + + dd("server_rewrite by lua inline"); + + + /* load Lua inline script (w/ cache) sp = 1 */ + rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, + lscf->srv.server_rewrite_src.value.data, + lscf->srv.server_rewrite_src.value.len, + &lscf->srv.server_rewrite_src_ref, + lscf->srv.server_rewrite_src_key, + (const char *) + lscf->srv.server_rewrite_chunkname); + if (rc != NGX_OK) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } + + return ngx_http_lua_server_rewrite_by_chunk(L, r); +} + + +ngx_int_t +ngx_http_lua_server_rewrite_handler_file(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L) +{ + ngx_int_t rc; + u_char *script_path; + ngx_str_t eval_src; + + + if (ngx_http_complex_value(r, &lscf->srv.server_rewrite_src, + &eval_src) != NGX_OK) + { + return NGX_ERROR; + } + + script_path = ngx_http_lua_rebase_path(r->pool, eval_src.data, + eval_src.len); + + if (script_path == NULL) { + return NGX_ERROR; + } + + /* load Lua script file (w/ cache) sp = 1 */ + rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, + &lscf->srv.server_rewrite_src_ref, + lscf->srv.server_rewrite_src_key); + if (rc != NGX_OK) { + if (rc < NGX_HTTP_SPECIAL_RESPONSE) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } + + return rc; + } + + return ngx_http_lua_server_rewrite_by_chunk(L, r); +} + + +static ngx_int_t +ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) +{ + int co_ref; + lua_State *co; + ngx_int_t rc; + ngx_uint_t nreqs; + ngx_event_t *rev; + ngx_connection_t *c; + ngx_http_lua_ctx_t *ctx; + ngx_http_cleanup_t *cln; + + ngx_http_lua_loc_conf_t *llcf; + + /* {{{ new coroutine to handle request */ + co = ngx_http_lua_new_thread(r, L, &co_ref); + + if (co == NULL) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "lua: failed to create new coroutine to handle request"); + + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } + + /* move code closure to new coroutine */ + lua_xmove(L, co, 1); + +#ifndef OPENRESTY_LUAJIT + /* set closure's env table to new coroutine's globals table */ + ngx_http_lua_get_globals_table(co); + lua_setfenv(co, -2); +#endif + + /* save nginx request in coroutine globals table */ + ngx_http_lua_set_req(co, r); + + /* {{{ initialize request context */ + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + dd("ctx = %p", ctx); + + if (ctx == NULL) { + return NGX_ERROR; + } + + ngx_http_lua_reset_ctx(r, L, ctx); + + ctx->entered_server_rewrite_phase = 1; + + ctx->cur_co_ctx = &ctx->entry_co_ctx; + ctx->cur_co_ctx->co = co; + ctx->cur_co_ctx->co_ref = co_ref; +#ifdef NGX_LUA_USE_ASSERT + ctx->cur_co_ctx->co_top = 1; +#endif + + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + + /* }}} */ + + /* {{{ register request cleanup hooks */ + if (ctx->cleanup == NULL) { + cln = ngx_http_cleanup_add(r, 0); + if (cln == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } + + cln->handler = ngx_http_lua_request_cleanup_handler; + cln->data = ctx; + ctx->cleanup = &cln->handler; + } + /* }}} */ + + ctx->context = NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE; + + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + + if (llcf->check_client_abort) { + r->read_event_handler = ngx_http_lua_rd_check_broken_connection; + +#if (NGX_HTTP_V2) + if (!r->stream) { +#endif + + rev = r->connection->read; + + if (!rev->active) { + if (ngx_add_event(rev, NGX_READ_EVENT, 0) != NGX_OK) { + return NGX_ERROR; + } + } + +#if (NGX_HTTP_V2) + } +#endif + + } else { + r->read_event_handler = ngx_http_block_reading; + } + + c = r->connection; + nreqs = c->requests; + + rc = ngx_http_lua_run_thread(L, r, ctx, 0); + + if (rc == NGX_ERROR || rc > NGX_OK) { + return rc; + } + + if (rc == NGX_AGAIN) { + rc = ngx_http_lua_run_posted_threads(c, L, r, ctx, nreqs); + + } else if (rc == NGX_DONE) { + ngx_http_lua_finalize_request(r, NGX_DONE); + rc = ngx_http_lua_run_posted_threads(c, L, r, ctx, nreqs); + } + + if (rc == NGX_OK || rc == NGX_DECLINED) { + if (r->header_sent) { + dd("header already sent"); + + /* response header was already generated in rewrite_by_lua*, + * so it is no longer safe to proceed to later phases + * which may generate responses again */ + + if (!ctx->eof) { + dd("eof not yet sent"); + + rc = ngx_http_lua_send_chain_link(r, ctx, NULL + /* indicate last_buf */); + if (rc == NGX_ERROR || rc > NGX_OK) { + return rc; + } + } + + return NGX_HTTP_OK; + } + + r->write_event_handler = ngx_http_core_run_phases; + ctx->entered_server_rewrite_phase = 0; + + return NGX_DECLINED; + } + + return rc; +} + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_server_rewriteby.h b/src/ngx_http_lua_server_rewriteby.h new file mode 100644 index 0000000000..a35ebd2b72 --- /dev/null +++ b/src/ngx_http_lua_server_rewriteby.h @@ -0,0 +1,20 @@ + +/* + * Copyright (C) Xiaozhe Wang (chaoslawful) + * Copyright (C) Yichun Zhang (agentzh) + */ + +#ifndef _NGX_HTTP_LUA_SERVER_REWRITEBY_H_INCLUDED_ +#define _NGX_HTTP_LUA_SERVER_REWRITEBY_H_INCLUDED_ + +#include "ngx_http_lua_common.h" + +ngx_int_t ngx_http_lua_server_rewrite_handler(ngx_http_request_t *r); +ngx_int_t ngx_http_lua_server_rewrite_handler_inline(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L); +ngx_int_t ngx_http_lua_server_rewrite_handler_file(ngx_http_request_t *r, + ngx_http_lua_srv_conf_t *lscf, lua_State *L); + +#endif /* _NGX_HTTP_LUA_SERVER_REWRITEBY_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 72549ca41e..1f564ac129 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4931,6 +4931,7 @@ ngx_http_lua_req_socket(lua_State *L) } ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index 0f88846fef..c37dde9cb2 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -92,12 +92,14 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) return luaL_error(L, "no ctx found"); } - dd("rewrite: %d, access: %d, content: %d", + dd("server_rewrite: %d, rewrite: %d, access: %d, content: %d", + (int) ctx->entered_serverrewrite_phase, (int) ctx->entered_rewrite_phase, (int) ctx->entered_access_phase, (int) ctx->entered_content_phase); - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua set uri jump to \"%*s\"", len, p); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 762958396b..39ba0b21fb 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1003,6 +1003,7 @@ ngx_http_lua_reset_ctx(ngx_http_request_t *r, lua_State *L, ctx->entry_co_ctx.co_ref = LUA_NOREF; + ctx->entered_server_rewrite_phase = 0; ctx->entered_rewrite_phase = 0; ctx->entered_access_phase = 0; ctx->entered_content_phase = 0; diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 52b1ff9356..4c4da976c5 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -33,6 +33,7 @@ #define NGX_HTTP_LUA_ESCAPE_HEADER_VALUE 8 #define NGX_HTTP_LUA_CONTEXT_YIELDABLE (NGX_HTTP_LUA_CONTEXT_REWRITE \ + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE \ | NGX_HTTP_LUA_CONTEXT_ACCESS \ | NGX_HTTP_LUA_CONTEXT_CONTENT \ | NGX_HTTP_LUA_CONTEXT_TIMER \ @@ -48,6 +49,7 @@ #define ngx_http_lua_context_name(c) \ ((c) == NGX_HTTP_LUA_CONTEXT_SET ? "set_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_REWRITE ? "rewrite_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE ? "server_rewrite_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_ACCESS ? "access_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_CONTENT ? "content_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_LOG ? "log_by_lua*" \ diff --git a/t/089-phase.t b/t/089-phase.t index 355b7ac33f..028c400c73 100644 --- a/t/089-phase.t +++ b/t/089-phase.t @@ -8,7 +8,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 1) + 2; +plan tests => repeat_each() * (blocks() * 2 + 2) + 2; #no_diff(); #no_long_string(); @@ -201,3 +201,23 @@ ok qr/exit_worker_by_lua\(nginx\.conf:\d+\):\d+: exit_worker/, qr/exiting now$/, ] + + + +=== TEST 12: server_rewrite_by_lua_block in http +--- http_config + server_rewrite_by_lua_block { + ngx.ctx.phase = ngx.get_phase() + } +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.ctx.phase) + } + } +--- request +GET /lua +--- response_body +server_rewrite +--- no_error_log +[error] diff --git a/t/162-socket-tls-handshake.t b/t/162-socket-tls-handshake.t index c4076af042..4474895c53 100644 --- a/t/162-socket-tls-handshake.t +++ b/t/162-socket-tls-handshake.t @@ -371,6 +371,3 @@ failed to setclientcert: client certificate must be supplied with corresponding [error] [crit] [emerg] - - - diff --git a/t/167-server-rewrite.t b/t/167-server-rewrite.t new file mode 100644 index 0000000000..152c5ce35d --- /dev/null +++ b/t/167-server-rewrite.t @@ -0,0 +1,488 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; +use t::StapThread; + +our $GCScript = <<_EOC_; +$t::StapThread::GCScript + +F(ngx_http_lua_check_broken_connection) { + println("lua check broken conn") +} + +F(ngx_http_lua_request_cleanup) { + println("lua req cleanup") +} +_EOC_ + +our $StapScript = $t::StapThread::StapScript; + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3 + 10); + +#log_level("info"); +#no_long_string(); + +run_tests(); + +__DATA__ + +=== TEST 1: server_rewrite_by_lua_block in http +--- http_config + server_rewrite_by_lua_block { + ngx.ctx.a = "server_rewrite_by_lua_block in http" + } +--- config + location /lua { + content_by_lua_block { + ngx.say(ngx.ctx.a) + ngx.log(ngx.INFO, ngx.ctx.a) + } + } +--- request +GET /lua +--- response_body +server_rewrite_by_lua_block in http +--- error_log +server_rewrite_by_lua_block in http +--- no_error_log +[error] + + + +=== TEST 2: server_rewrite_by_lua_block in server +--- config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "server_rewrite_by_lua_block in server") + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +OK +--- error_log +server_rewrite_by_lua_block in server +--- no_error_log +[error] + + + +=== TEST 3: redirect +--- config + server_rewrite_by_lua_block { + ngx.redirect("/foo") + } +--- request +GET /lua +--- raw_response_headers_like: Location: /foo\r\n +--- response_body_like: 302 Found +--- error_code: 302 +--- no_error_log +[error] + + + +=== TEST 4: flush +--- config + server_rewrite_by_lua_block { + ngx.say("foo") + ngx.flush(true) + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +foo +--- no_error_log +[error] + + + +=== TEST 5: eof +--- config + server_rewrite_by_lua_block { + ngx.say("foo") + ngx.eof() + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +foo +--- no_error_log +[error] + + + +=== TEST 6: send_headers +--- config + server_rewrite_by_lua_block { + ngx.header["Foox"] = {"conx1", "conx2" } + ngx.header["Fooy"] = {"cony1", "cony2" } + ngx.send_headers() + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +--- response_headers +Foox: conx1, conx2 +Fooy: cony1, cony2 +--- no_error_log +[error] + + + +=== TEST 7: read_body +--- config + server_rewrite_by_lua_block { + ngx.req.read_body() + ngx.say(ngx.var.request_body) + } +--- request +POST /lua +hello, world +--- response_body +hello, world +--- no_error_log +[error] + + + +=== TEST 8: req_sock +--- config + server_rewrite_by_lua_block { + local sock = ngx.req.socket() + sock:receive(2) + sock:receive(2) + sock:receive(1) + ngx.sleep(1) + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +POST /lua +hello + +--- stap2 eval: $::StapScript +--- stap eval: $::GCScript +--- stap_out +lua check broken conn +lua check broken conn +lua req cleanup +delete thread 1 + +--- wait: 1 +--- timeout: 0.2 +--- abort +--- ignore_response +--- no_error_log +[error] + + + +=== TEST 9: rewrite args (not break cycle by default) +--- config + location /bar { + echo "bar: $uri?$args"; + } + server_rewrite_by_lua_block { + if ngx.var.uri ~= "/bar" then + ngx.req.set_uri_args("hello") + ngx.req.set_uri("/bar", true) + end + } + location /foo { + + echo "foo: $uri?$args"; + } +--- request + GET /foo?world +--- response_body +bar: /bar?hello + + + +=== TEST 10: server_rewrite_by_lua_block overwrite by server +--- http_config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "server_rewrite_by_lua_block in http") + } +--- config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "server_rewrite_by_lua_block in server") + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +OK +--- error_log +server_rewrite_by_lua_block in server +--- no_error_log +[error] + + + +=== TEST 11: sleep +--- config + server_rewrite_by_lua_block { + ngx.sleep(0.001) + ngx.log(ngx.INFO, "server_rewrite_by_lua_block in server") + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +OK +--- error_log +server_rewrite_by_lua_block in server +--- no_error_log +[error] + + + +=== TEST 12: ngx.exit(ngx.OK) +--- config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "ngx.exit") + ngx.exit(ngx.OK) + } + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +OK +--- error_log +ngx.exit +--- no_error_log +[error] + + + +=== TEST 13: ngx.exit(503) +--- config + server_rewrite_by_lua_block { + ngx.exit(503) + } + location /lua { + content_by_lua_block { + ngx.log(ngx.ERR, "content_by_lua") + ngx.say("OK") + } + } +--- request +GET /lua +--- error_code: 503 +--- no_error_log +[error] + + + +=== TEST 14: subrequests +--- config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "is_subrequest:", ngx.is_subrequest) + } + + location /lua { + content_by_lua_block { + local res = ngx.location.capture("/sub") + ngx.print(res.body) + } + } + + location /sub { + content_by_lua_block { + ngx.say("OK") + } + } + +--- request +GET /lua +--- response_body +OK +--- error_log +is_subrequest:false +is_subrequest:true +--- no_error_log +[error] + + + +=== TEST 15: rewrite by ngx_http_rewrite_module +--- config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "uri is ", ngx.var.uri) + } + + rewrite ^ /re; + + location /re { + content_by_lua_block { + ngx.say("RE") + } + } + + location /ok { + content_by_lua_block { + ngx.say("OK") + } + } + +--- request +GET /lua +--- response_body +RE +--- error_log +uri is /lua +--- no_error_log +[error] + + + +=== TEST 16: exec +--- config + server_rewrite_by_lua_block { + if ngx.var.uri ~= "/ok" then + ngx.exec("/ok") + end + ngx.log(ngx.INFO, "uri is ", ngx.var.uri) + } + + location /ok { + content_by_lua_block { + ngx.say("OK") + } + } + +--- request +GET /lua +--- response_body +OK +--- error_log +uri is /ok +--- no_error_log +[error] + + + +=== TEST 17: server_rewrite_by_lua and rewrite_by_lua +--- http_config + server_rewrite_by_lua_block { + ngx.log(ngx.INFO, "server_rewrite_by_lua_block in http") + } +--- config + location /lua { + rewrite_by_lua_block { + ngx.log(ngx.INFO, "rewrite_by_lua_block in location") + } + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- response_body +OK +--- error_log +server_rewrite_by_lua_block in http +rewrite_by_lua_block in location +--- no_error_log +[error] + + + +=== TEST 18: server_rewrite_by_lua_file +--- http_config + server_rewrite_by_lua_file 'html/foo.lua'; +--- config + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- user_files +>>> foo.lua +ngx.log(ngx.INFO, "rewrite_by_lua_file in server") +--- response_body +OK +--- error_log +rewrite_by_lua_file in server +--- no_error_log +[error] + + + +=== TEST 19: syntax error server_rewrite_by_lua_block in http +--- http_config + server_rewrite_by_lua_block { + 'for end'; + } +--- config + location /lua { + content_by_lua_block { + ngx.say("OK") + } + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: server_rewrite_by_lua(nginx.conf:25):2: unexpected symbol near ''for end'' +--- no_error_log +no_such_error + + + +=== TEST 20: syntax error server_rewrite_by_lua_block in server +--- config + server_rewrite_by_lua_block { + 'for end'; + } + location /lua { + content_by_lua_block { + ngx.say("Hello world") + } + } +--- request +GET /lua +--- ignore_response +--- error_log +failed to load inlined Lua code: server_rewrite_by_lua(nginx.conf:39):2: unexpected symbol near ''for end'' +--- no_error_log +no_such_error From 6c0cc5c45635a6fbaf9b6ebeda5232f4004f5cbc Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 21 Mar 2022 20:35:46 +0800 Subject: [PATCH 618/848] doc: did not generate README.markdown from doc/HttpLuaModule.wiki. (#2029) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 24 +++++++++++++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 98d92a6b08..7bcbedffd7 100644 --- a/README.markdown +++ b/README.markdown @@ -7819,7 +7819,7 @@ This method was first introduced in the `v0.5.0rc1` release. [Back to TOC](#nginx-api-for-lua) tcpsock:setclientcert --------------------- +--------------------- **syntax:** *ok, err = tcpsock:setclientcert(cert, pkey)* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 9270d18da0..6185b6d653 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -812,7 +812,6 @@ phases. * add ignore_resp_headers, ignore_resp_body, and ignore_resp options to [[#ngx.location.capture|ngx.location.capture]] and [[#ngx.location.capture_multi|ngx.location.capture_multi]] methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. * add stat mode similar to [https://httpd.apache.org/docs/trunk/mod/mod_lua.html mod_lua]. -* cosocket: add client SSL certificate support. = Changes = @@ -6500,6 +6499,7 @@ This API function was first added to the v0.10.1 release. Creates and returns a TCP or stream-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: * [[#tcpsock:connect|connect]] +* [[#tcpsock:setclientcert|setclientcert]] * [[#tcpsock:sslhandshake|sslhandshake]] * [[#tcpsock:send|send]] * [[#tcpsock:receive|receive]] @@ -6650,6 +6650,28 @@ The support for the options table argument was first introduced in the v0. This method was first introduced in the v0.5.0rc1 release. +== tcpsock:setclientcert == + +'''syntax:''' ''ok, err = tcpsock:setclientcert(cert, pkey)'' + +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua*'' + +Set client certificate chain and corresponding private key to the TCP socket object. +The certificate chain and private key provided will be used later by the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. + +* cert specify a client certificate chain cdata object that will be used while handshaking with +remote server. These objects can be created using [ngx.ssl.parse\_pem\_cert](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_cert) +function provided by lua-resty-core. Note that specifying the cert option requires +corresponding pkey be provided too. See below. +* pkey specify a private key corresponds to the cert option above. +These objects can be created using [ngx.ssl.parse\_pem\_priv\_key](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_priv_key) +function provided by lua-resty-core. + +If both of cert and pkey are nil, this method will clear any existing client certificate and private key +that was previously set on the cosocket object. + +This method was first introduced in the `v0.10.22` release. + == tcpsock:sslhandshake == '''syntax:''' ''session, err = tcpsock:sslhandshake(reused_session?, server_name?, ssl_verify?, send_status_req?)'' From f0c04dbf61b17d4742f90f1aeab94cbe4170c830 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 1 Apr 2022 23:04:02 +0800 Subject: [PATCH 619/848] bugfix: failed to compile when nginx https is disabled. (#2034) --- src/ngx_http_lua_common.h | 4 ++-- src/ngx_http_lua_module.c | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index f293c5b1d9..31b39fc709 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -313,8 +313,8 @@ struct ngx_http_lua_main_conf_s { union ngx_http_lua_srv_conf_u { -#if (NGX_HTTP_SSL) struct { +#if (NGX_HTTP_SSL) ngx_http_lua_srv_conf_handler_pt ssl_cert_handler; ngx_str_t ssl_cert_src; u_char *ssl_cert_src_key; @@ -338,6 +338,7 @@ union ngx_http_lua_srv_conf_u { u_char *ssl_client_hello_src_key; u_char *ssl_client_hello_chunkname; int ssl_client_hello_src_ref; +#endif ngx_http_lua_srv_conf_handler_pt server_rewrite_handler; ngx_http_complex_value_t server_rewrite_src; @@ -345,7 +346,6 @@ union ngx_http_lua_srv_conf_u { u_char *server_rewrite_chunkname; int server_rewrite_src_ref; } srv; -#endif struct { ngx_http_lua_srv_conf_handler_pt handler; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 243974d96d..16f442464c 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1180,10 +1180,11 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) static char * ngx_http_lua_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) { + ngx_http_lua_srv_conf_t *conf = child; + ngx_http_lua_srv_conf_t *prev = parent; + #if (NGX_HTTP_SSL) - ngx_http_lua_srv_conf_t *prev = parent; - ngx_http_lua_srv_conf_t *conf = child; ngx_http_ssl_srv_conf_t *sscf; dd("merge srv conf"); From bc2ecdae2cb88beba2d4fe190ec3560a4d3dec74 Mon Sep 17 00:00:00 2001 From: xiaobiaozhao <52393536+xiaobiaozhao@users.noreply.github.com> Date: Sat, 2 Apr 2022 08:47:09 +0800 Subject: [PATCH 620/848] ci: add util/build-without-ssl.sh. (#2035) --- .travis.yml | 2 ++ util/build-without-ssl.sh | 67 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100755 util/build-without-ssl.sh diff --git a/.travis.yml b/.travis.yml index 6e6ecee4f6..56ff18e272 100644 --- a/.travis.yml +++ b/.travis.yml @@ -130,6 +130,8 @@ script: - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - cd .. - export NGX_BUILD_CC=$CC + - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) + - rm -fr buildroot - sh util/build.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - nginx -V - ldd `which nginx`|grep -E 'luajit|ssl|pcre' diff --git a/util/build-without-ssl.sh b/util/build-without-ssl.sh new file mode 100755 index 0000000000..906be75706 --- /dev/null +++ b/util/build-without-ssl.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash + +# this script is for developers only. +# dependent on the ngx-build script from the nginx-devel-utils repository: +# https://github.com/openresty/nginx-devel-utils/blob/master/ngx-build +# the resulting nginx is located at ./work/nginx/sbin/nginx + +root=`pwd` +version=${1:-1.4.1} +home=~ +force=$2 + +# the ngx-build script is from https://github.com/agentzh/nginx-devel-utils + + #--add-module=$home/work/nginx_upload_module-2.2.0 \ + + #--without-pcre \ + #--without-http_rewrite_module \ + #--without-http_autoindex_module \ + #--with-cc=gcc46 \ + #--with-cc=clang \ + #--without-http_referer_module \ + #--with-http_spdy_module \ + +add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" + +time ngx-build $force $version \ + --with-threads \ + --with-pcre-jit \ + --with-ipv6 \ + --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC" \ + --with-http_v2_module \ + --with-http_realip_module \ + --add-module=$root/../ndk-nginx-module \ + --add-module=$root/../set-misc-nginx-module \ + --with-ld-opt="-L$PCRE_LIB -Wl,-rpath,$PCRE_LIB:$LIBDRIZZLE_LIB" \ + --without-mail_pop3_module \ + --without-mail_imap_module \ + --with-http_image_filter_module \ + --without-mail_smtp_module \ + --with-stream \ + --without-http_upstream_ip_hash_module \ + --without-http_memcached_module \ + --without-http_auth_basic_module \ + --without-http_userid_module \ + --with-http_auth_request_module \ + --add-module=$root/../echo-nginx-module \ + --add-module=$root/../memc-nginx-module \ + --add-module=$root/../srcache-nginx-module \ + --add-module=$root \ + --add-module=$root/../lua-upstream-nginx-module \ + --add-module=$root/../headers-more-nginx-module \ + --add-module=$root/../drizzle-nginx-module \ + --add-module=$root/../rds-json-nginx-module \ + --add-module=$root/../coolkit-nginx-module \ + --add-module=$root/../redis2-nginx-module \ + --add-module=$root/../stream-lua-nginx-module \ + --add-module=$root/t/data/fake-module \ + $add_fake_shm_module \ + --add-module=$root/t/data/fake-delayed-load-module \ + --with-http_gunzip_module \ + --with-http_dav_module \ + --with-select_module \ + --with-poll_module \ + $opts \ + --with-debug + From 653d6a36f46b077cb902d7ba40824c299cf9bbf4 Mon Sep 17 00:00:00 2001 From: Chrono Date: Wed, 6 Apr 2022 20:48:29 +0800 Subject: [PATCH 621/848] bugfix: Apple Silicon FFI ABI limitation workaround There is a known issue in LuaJIT (https://github.com/LuaJIT/LuaJIT/issues/205#issuecomment-236426398) that passing non 64-bit value after the 8th argument in FFI calls doesn't work on Apple ARM64 devices, there is no plan on addressing this issue anytime soon from LuaJIT side. This commit conditionally compiles proxy FFI functions that takes in pointer to struct in order to reduce the length of the argument list, getting around this issue. --- src/ngx_http_lua_headers.c | 12 +++++++++ src/ngx_http_lua_headers_out.h | 16 ++++++++++++ src/ngx_http_lua_shdict.c | 34 +++++++++++++++++++++++++ src/ngx_http_lua_shdict.h | 46 ++++++++++++++++++++++++++++++++++ 4 files changed, 108 insertions(+) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 54977dce8e..8098615dfc 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -1240,4 +1240,16 @@ ngx_http_lua_ngx_raw_header_cleanup(void *data) #endif +#if (NGX_DARWIN) +int +ngx_http_lua_ffi_set_resp_header_macos(ngx_http_lua_set_resp_header_params_t *p) +{ + return ngx_http_lua_ffi_set_resp_header(p->r, p->key_data, p->key_len, + p->is_nil, p->sval, p->sval_len, + p->mvals, p->mvals_len, + p->override, p->errmsg); +} +#endif + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_headers_out.h b/src/ngx_http_lua_headers_out.h index 944f57a462..eb3c794723 100644 --- a/src/ngx_http_lua_headers_out.h +++ b/src/ngx_http_lua_headers_out.h @@ -12,6 +12,22 @@ #include "ngx_http_lua_common.h" +#if (NGX_DARWIN) +typedef struct { + ngx_http_request_t *r; + const char *key_data; + size_t key_len; + int is_nil; + const char *sval; + size_t sval_len; + void *mvals; + size_t mvals_len; + int override; + char **errmsg; +} ngx_http_lua_set_resp_header_params_t; +#endif + + ngx_int_t ngx_http_lua_set_output_header(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_str_t key, ngx_str_t value, unsigned override); int ngx_http_lua_get_output_header(lua_State *L, ngx_http_request_t *r, diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 26622d29f0..27f3b100f3 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2092,4 +2092,38 @@ ngx_http_lua_ffi_shdict_free_space(ngx_shm_zone_t *zone) #endif +#if (NGX_DARWIN) +int +ngx_http_lua_ffi_shdict_get_macos(ngx_http_lua_shdict_get_params_t *p) +{ + return ngx_http_lua_ffi_shdict_get(p->zone, p->key, p->key_len, + p->value_type, p->str_value_buf, + p->str_value_len, p->num_value, + p->user_flags, p->get_stale, + p->is_stale, p->errmsg); +} + + +int +ngx_http_lua_ffi_shdict_store_macos(ngx_http_lua_shdict_store_params_t *p) +{ + return ngx_http_lua_ffi_shdict_store(p->zone, p->op, p->key, p->key_len, + p->value_type, p->str_value_buf, + p->str_value_len, p->num_value, + p->exptime, p->user_flags, + p->errmsg, p->forcible); +} + + +int +ngx_http_lua_ffi_shdict_incr_macos(ngx_http_lua_shdict_incr_params_t *p) +{ + return ngx_http_lua_ffi_shdict_incr(p->zone, p->key, p->key_len, + p->num_value, p->errmsg, + p->has_init, p->init, p->init_ttl, + p->forcible); +} +#endif + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_shdict.h b/src/ngx_http_lua_shdict.h index 90a0099f5d..67dd4d4eb0 100644 --- a/src/ngx_http_lua_shdict.h +++ b/src/ngx_http_lua_shdict.h @@ -55,6 +55,52 @@ typedef struct { } ngx_http_lua_shm_zone_ctx_t; +#if (NGX_DARWIN) +typedef struct { + void *zone; + const unsigned char *key; + size_t key_len; + int *value_type; + unsigned char **str_value_buf; + size_t *str_value_len; + double *num_value; + int *user_flags; + int get_stale; + int *is_stale; + char **errmsg; +} ngx_http_lua_shdict_get_params_t; + + +typedef struct { + void *zone; + int op; + const unsigned char *key; + size_t key_len; + int value_type; + const unsigned char *str_value_buf; + size_t str_value_len; + double num_value; + long exptime; + int user_flags; + char **errmsg; + int *forcible; +} ngx_http_lua_shdict_store_params_t; + + +typedef struct { + void *zone; + const unsigned char *key; + size_t key_len; + double *num_value; + char **errmsg; + int has_init; + double init; + long init_ttl; + int *forcible; +} ngx_http_lua_shdict_incr_params_t; +#endif + + ngx_int_t ngx_http_lua_shdict_init_zone(ngx_shm_zone_t *shm_zone, void *data); void ngx_http_lua_shdict_rbtree_insert_value(ngx_rbtree_node_t *temp, ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel); From 86e1a6446a3b33c41536734df2601a5c469330cf Mon Sep 17 00:00:00 2001 From: jizhuozhi Date: Sun, 8 May 2022 20:59:54 +0800 Subject: [PATCH 622/848] feature: content_by_lua_file return 503 if read err and file is existed #1992 (#1995) --- README.markdown | 2 ++ src/ngx_http_lua_cache.c | 8 +++++++- t/002-content.t | 12 ++++++++++++ 3 files changed, 21 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 7bcbedffd7..ab92fa06ca 100644 --- a/README.markdown +++ b/README.markdown @@ -1886,6 +1886,8 @@ content_by_lua_file Equivalent to [content_by_lua_block](#content_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. +If the file is not found, a `404 Not Found` status code will be returned, and a `503 Service Temporarily Unavailable` status code will be returned in case of errors in reading other files. + Nginx variables can be used in the `` string to provide flexibility. This however carries some risks and is not ordinarily recommended. When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. diff --git a/src/ngx_http_lua_cache.c b/src/ngx_http_lua_cache.c index 0b1b8ecda8..534424688d 100644 --- a/src/ngx_http_lua_cache.c +++ b/src/ngx_http_lua_cache.c @@ -307,7 +307,13 @@ ngx_http_lua_cache_loadfile(ngx_log_t *log, lua_State *L, break; case LUA_ERRFILE: - errcode = NGX_HTTP_NOT_FOUND; + if (errno == ENOENT) { + errcode = NGX_HTTP_NOT_FOUND; + + } else { + errcode = NGX_HTTP_SERVICE_UNAVAILABLE; + } + /* fall through */ default: diff --git a/t/002-content.t b/t/002-content.t index 661c2e39e3..54de40ebd5 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -1086,3 +1086,15 @@ GET /lua --- error_code: 500 --- error_log failed to load inlined Lua code: content_by_lua(...45678901234567890123456789012345.conf:14) + + + +=== TEST 51: Lua file permission denied +--- config + location /lua { + content_by_lua_file /etc/shadow; + } +--- request +GET /lua +--- response_body_like: 503 Service Temporarily Unavailable +--- error_code: 503 From 8a8e710cb9e388b8bfd16d9eee57504e90e3acef Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Fri, 13 May 2022 04:56:31 +0500 Subject: [PATCH 623/848] fix potential null pointer dereference found by Coverity (#2043) * fix potential null pointer dereference found by Coverity *** CID 352757: Null pointer dereferences (FORWARD_NULL) /src/ngx_http_lua_socket_tcp.c: 1733 in ngx_http_lua_ffi_socket_tcp_sslhandshake() 1727 #else 1728 *errmsg = "no OCSP support"; 1729 return NGX_ERROR; 1730 #endif 1731 } 1732 >>> CID 352757: Null pointer dereferences (FORWARD_NULL) >>> Dereferencing null pointer "server_name". 1733 if (server_name->len == 0) { 1734 u->ssl_name.len = 0; 1735 1736 } else { 1737 if (u->ssl_name.data) { 1738 /* buffer already allocated */ Co-authored-by: lijunlong --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 1f564ac129..35ff6250e1 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1730,7 +1730,7 @@ ngx_http_lua_ffi_socket_tcp_sslhandshake(ngx_http_request_t *r, #endif } - if (server_name->len == 0) { + if (server_name == NULL || server_name->len == 0) { u->ssl_name.len = 0; } else { From af130ae896aae23877a9ba76c8f57e269a3c4468 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Sat, 14 May 2022 17:57:02 +0500 Subject: [PATCH 624/848] fix dead code found by coverity *** CID 352755: Control flow issues (DEADCODE) /src/ngx_http_lua_socket_tcp.c: 1541 in ngx_http_lua_socket_tcp_check_busy() 1535 #if (NGX_HTTP_SSL) 1536 1537 static const char * 1538 ngx_http_lua_socket_tcp_check_busy(ngx_http_request_t *r, 1539 ngx_http_lua_socket_tcp_upstream_t *u, unsigned int ops) 1540 { >>> CID 352755: Control flow issues (DEADCODE) >>> Execution cannot reach the expression "u->conn_waiting" inside this statement: "if (ops & SOCKET_OP_CONNECT...". 1541 if ((ops & SOCKET_OP_CONNECT) && u->conn_waiting) { 1542 return "socket busy connecting"; 1543 } 1544 1545 if ((ops & SOCKET_OP_READ) && u->read_waiting) { 1546 return "socket busy reading"; --- src/ngx_http_lua_socket_tcp.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 35ff6250e1..d8f1d4d1c1 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -166,10 +166,10 @@ enum { enum { - SOCKET_OP_CONNECT, - SOCKET_OP_READ, - SOCKET_OP_WRITE, - SOCKET_OP_RESUME_CONN, + SOCKET_OP_CONNECT = 0x01, + SOCKET_OP_READ = 0x02, + SOCKET_OP_WRITE = 0x04, + SOCKET_OP_RESUME_CONN = 0x08, }; From 381e9b3a61a119df688a9b2452a18d710c385de9 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 20 May 2022 14:13:27 +0800 Subject: [PATCH 625/848] bugfix: wrong memory size for the pernding timers array. (#2048) --- src/ngx_http_lua_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index e82e3406ce..795bd4f5f3 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -821,7 +821,7 @@ ngx_http_lua_abort_pending_timers(ngx_event_t *ev) prev = NULL; events = ngx_pcalloc(ngx_cycle->pool, - lmcf->pending_timers * sizeof(ngx_event_t)); + lmcf->pending_timers * sizeof(ngx_event_t *)); if (events == NULL) { return; } From b91221bd6c234a20aab18e41905dcc7d64dea07e Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Wed, 25 May 2022 18:03:37 +0500 Subject: [PATCH 626/848] optimize: fix potential null pointer dereference found by coverity (#2051) deref_ptr: Directly dereferencing pointer ctx. 1143 *ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); CID 258020 (#1 of 1): Dereference before null check (REVERSE_INULL)check_after_deref: Null-checking ctx suggests that it may be null, but it has already been dereferenced on all paths leading to the check. 1144 if (ctx == NULL) { 1145 return NGX_HTTP_LUA_FFI_NO_REQ_CTX; 1146 } --- src/ngx_http_lua_pipe.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 93dc7d5bc5..d0936c6d9e 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -1141,7 +1141,7 @@ ngx_http_lua_pipe_get_lua_ctx(ngx_http_request_t *r, int rc; *ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { + if (*ctx == NULL) { return NGX_HTTP_LUA_FFI_NO_REQ_CTX; } From b6db2e741dc77fa7c5f6da12ef7d7f9c26a930a5 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 25 May 2022 21:04:25 +0800 Subject: [PATCH 627/848] optimize: add error log when closing the pipe failed. (#2050) --- src/ngx_http_lua_pipe.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index d0936c6d9e..1d9425b59f 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -773,10 +773,21 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, } } - close(in[0]); - close(out[1]); + if (close(in[0]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe failed to close the in[0]"); + } + + if (close(out[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe failed to close the out[1]"); + } + if (!merge_stderr) { - close(err[1]); + if (close(err[1]) == -1) { + ngx_log_error(NGX_LOG_EMERG, ngx_cycle->log, ngx_errno, + "lua pipe failed to close the err[1]"); + } } if (environ != NULL) { From 0ac2ab27976d0834a7baf6c418ee4bed4dcac9e6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 25 May 2022 21:09:00 +0800 Subject: [PATCH 628/848] bugfix: tctx.pool is double freed. (#2049) } else if (tctx.pool) { CID 251579 (#1 of 1): Use after free (USE_AFTER_FREE) 18. deref_arg: Calling ngx_destroy_pool dereferences freed pointer tctx.pool. 762 ngx_destroy_pool(tctx.pool); 763 } 764} --- src/ngx_http_lua_timer.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 795bd4f5f3..e75b4ad65d 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -565,6 +565,8 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) c = ngx_http_lua_create_fake_connection(tctx.pool); if (c == NULL) { errmsg = "could not create fake connection"; + /* tctx.pool is free in ngx_http_lua_create_fake_connection */ + tctx.pool = NULL; goto failed; } From e7660d75a3877faffcecef373ef66de214e6ad65 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 26 May 2022 09:03:07 +0800 Subject: [PATCH 629/848] doc: revised English grammar in ngx_http_lua_timer.c. (#2053) --- src/ngx_http_lua_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index e75b4ad65d..060cab543a 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -565,7 +565,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) c = ngx_http_lua_create_fake_connection(tctx.pool); if (c == NULL) { errmsg = "could not create fake connection"; - /* tctx.pool is free in ngx_http_lua_create_fake_connection */ + /* tctx.pool is freed in ngx_http_lua_create_fake_connection */ tctx.pool = NULL; goto failed; } From 2c60e8966da3d72d591d2cb12980b18c5b0de3cd Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 30 May 2022 11:28:58 +0800 Subject: [PATCH 630/848] optimize: destroy pipe proc when freeing the request. (#2052) * optimize: destroy pipe proc when freeing the request. The connections are cached until gc collect. check the number of the free via the following cmd: gdb --batch -q -p 3049 -ex "print ngx_cycle->free_connection_n" 2>/dev/null; The nginx config to reproduce the problem location / { content_by_lua_block { local ngx_pipe = require "ngx.pipe" local proc, err = ngx_pipe.spawn({"sh", "-c", "sleep 1; echo 100; sleep 5;"}) if not proc then ngx.say(err) return end proc:set_timeouts(nil, nil, nil, 2000) local data, err, partial = proc:stdout_read_line() proc:wait() ngx.say(data) } } --- src/ngx_http_lua_pipe.c | 30 +++++++++++++++++++++++++++++- src/ngx_http_lua_pipe.h | 2 ++ 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 1d9425b59f..c947d5599f 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -78,6 +78,8 @@ static void ngx_http_lua_pipe_proc_read_stdout_cleanup(void *data); static void ngx_http_lua_pipe_proc_read_stderr_cleanup(void *data); static void ngx_http_lua_pipe_proc_write_cleanup(void *data); static void ngx_http_lua_pipe_proc_wait_cleanup(void *data); +void ngx_http_lua_ffi_pipe_proc_destroy( + ngx_http_lua_ffi_pipe_proc_t *proc); static ngx_rbtree_t ngx_http_lua_pipe_rbtree; @@ -420,6 +422,9 @@ ngx_http_lua_pipe_sigchld_event_handler(ngx_event_t *ev) &ngx_posted_events); } + /* TODO: we should proactively close and free up the pipe after + * the user consume all the data in the pipe. + */ pipe_node->proc->pipe->dead = 1; if (WIFSIGNALED(status)) { @@ -562,7 +567,8 @@ ngx_http_lua_execvpe(const char *program, char * const argv[], int -ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, +ngx_http_lua_ffi_pipe_spawn(ngx_http_request_t *r, + ngx_http_lua_ffi_pipe_proc_t *proc, const char *file, const char **argv, int merge_stderr, size_t buffer_size, const char **environ, u_char *errbuf, size_t *errbuf_size) { @@ -582,6 +588,7 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, ngx_http_lua_pipe_node_t *pipe_node; struct sigaction sa; ngx_http_lua_pipe_signal_t *sig; + ngx_http_cleanup_t *cln; sigset_t set; pool_size = ngx_align(NGX_MIN_POOL_SIZE + buffer_size * 2, @@ -869,6 +876,21 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_lua_ffi_pipe_proc_t *proc, pp->stderr_fd = stderr_fd; } + if (pp->cleanup == NULL) { + cln = ngx_http_lua_cleanup_add(r, 0); + + if (cln == NULL) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no memory") + - errbuf; + goto close_in_out_err_fd; + } + + cln->handler = (ngx_http_cleanup_pt) ngx_http_lua_ffi_pipe_proc_destroy; + cln->data = proc; + pp->cleanup = &cln->handler; + pp->r = r; + } + node = (ngx_rbtree_node_t *) (pp + 1); node->key = pid; pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; @@ -1139,6 +1161,12 @@ ngx_http_lua_ffi_pipe_proc_destroy(ngx_http_lua_ffi_pipe_proc_t *proc) } } + if (pipe->cleanup != NULL) { + *pipe->cleanup = NULL; + ngx_http_lua_cleanup_free(pipe->r, pipe->cleanup); + pipe->cleanup = NULL; + } + ngx_http_lua_pipe_proc_finalize(proc); ngx_destroy_pool(pipe->pool); proc->pipe = NULL; diff --git a/src/ngx_http_lua_pipe.h b/src/ngx_http_lua_pipe.h index ecb86c8806..f1c9283526 100644 --- a/src/ngx_http_lua_pipe.h +++ b/src/ngx_http_lua_pipe.h @@ -57,6 +57,8 @@ struct ngx_http_lua_pipe_s { ngx_http_lua_pipe_ctx_t *stdout_ctx; ngx_http_lua_pipe_ctx_t *stderr_ctx; ngx_http_lua_pipe_retval_handler retval_handler; + ngx_http_cleanup_pt *cleanup; + ngx_http_request_t *r; size_t buffer_size; unsigned closed:1; unsigned dead:1; From 3fe8d9ddeb144b121e26cc2251b925e4f673f554 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 14 Jun 2022 18:15:12 +0800 Subject: [PATCH 631/848] change: can not receive udp datagram greater 8192. (#2057) --- src/ngx_http_lua_socket_udp.c | 2 +- t/087-udp-socket.t | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 4f970e6cca..cbb6621b47 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -24,7 +24,7 @@ #endif -#define UDP_MAX_DATAGRAM_SIZE 8192 +#define UDP_MAX_DATAGRAM_SIZE 65536 static int ngx_http_lua_socket_udp(lua_State *L); diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index 80c5e36517..6c26b93876 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -69,7 +69,7 @@ GET /t [error] --- log_level: debug --- error_log -lua udp socket receive buffer size: 8192 +lua udp socket receive buffer size: 65536 @@ -595,7 +595,7 @@ received a good response. [error] --- log_level: debug --- error_log -lua udp socket receive buffer size: 8192 +lua udp socket receive buffer size: 65536 --- no_check_leak @@ -662,7 +662,7 @@ received a good response. [error] --- log_level: debug --- error_log -lua udp socket receive buffer size: 8192 +lua udp socket receive buffer size: 65536 --- no_check_leak From 9a0a8968cded113ecd31913c4e0aefc195578ac0 Mon Sep 17 00:00:00 2001 From: kingluo Date: Sat, 18 Jun 2022 22:50:55 +0800 Subject: [PATCH 632/848] bugfix: did not wakeup coroutine when worker thread finished. --- README.markdown | 41 +++--------------- doc/HttpLuaModule.wiki | 33 +++------------ src/ngx_http_lua_common.h | 4 ++ src/ngx_http_lua_worker_thread.c | 73 ++++++++++++++++++++++---------- t/166-worker-thread.t | 38 +++++++++++++++++ 5 files changed, 105 insertions(+), 84 deletions(-) diff --git a/README.markdown b/README.markdown index ab92fa06ca..17563b5757 100644 --- a/README.markdown +++ b/README.markdown @@ -1,8 +1,3 @@ - - Name ==== @@ -9329,42 +9324,18 @@ Example1: do md5 calculation. default_type 'text/plain'; content_by_lua_block { - local ok, ret, md5_or_err = ngx.run_worker_thread("testpool", "calc_md5", "md5", ngx.var.arg_str) - if not ok then - ngx.say(ret) - return - end - if not ret then - ngx.say(md5_or_err) - return - end - ngx.say(md5_or_err) + local ok, md5_or_err = ngx.run_worker_thread("testpool", "md5", "md5") + ngx.say(ok, " : ", md5_or_err) } } ``` -`calc_md5.lua` +`md5.lua` ```lua - - local resty_md5 = require "resty.md5" - local resty_str = require "resty.string" - - local function md5(str) - local md5 = resty_md5:new() - if not md5 then - return false, "md5 new error" - end - - local ok = md5:update(str) - if not ok then - return false, "md5 update error" - end - - local digest = md5:final() - return true, resty_str.to_hex(digest) - end - return {md5=md5} +local function md5() + return ngx.md5("hello") +end ``` Example2: write logs into the log file. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 6185b6d653..371b9cb84f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -8005,39 +8005,18 @@ location /calc_md5 { default_type 'text/plain'; content_by_lua_block { - local ok, ret, md5_or_err = ngx.run_worker_thread("testpool", "calc_md5", "md5", ngx.var.arg_str) - if not ok then - ngx.say(ret) - return - end - if not ret then - ngx.say(md5_or_err) - return - end - ngx.say(md5_or_err) + local ok, md5_or_err = ngx.run_worker_thread("testpool", "md5", "md5") + ngx.say(ok, " : ", md5_or_err) } } + -calc_md5.lua +md5.lua -local resty_md5 = require "resty.md5" -local resty_str = require "resty.string" - -local function md5(str) - local md5 = resty_md5:new() - if not md5 then - return false, "md5 new error" - end - - local ok = md5:update(str) - if not ok then - return false, "md5 update error" - end - - local digest = md5:final() - return true, resty_str.to_hex(digest) +local function md5() + return ngx.md5("hello") end return {md5=md5} diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 31b39fc709..dafa729f8e 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -494,6 +494,10 @@ struct ngx_http_lua_co_ctx_s { uint8_t *sr_flags; + unsigned nresults_from_worker_thread; /* number of results + * from worker + * thread callback */ + unsigned nsubreqs; /* number of subrequests of the * current request */ diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 69c6da1d01..d3e9cbfd30 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -288,6 +288,50 @@ ngx_http_lua_worker_thread_handler(void *data, ngx_log_t *log) } +static ngx_int_t +ngx_http_lua_worker_thread_resume(ngx_http_request_t *r) +{ + lua_State *vm; + ngx_connection_t *c; + ngx_int_t rc; + ngx_uint_t nreqs; + ngx_http_lua_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return NGX_ERROR; + } + + ctx->resume_handler = ngx_http_lua_wev_handler; + + c = r->connection; + vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; + + rc = ngx_http_lua_run_thread(vm, r, ctx, + ctx->cur_co_ctx->nresults_from_worker_thread); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua run thread returned %d", rc); + + if (rc == NGX_AGAIN) { + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (rc == NGX_DONE) { + ngx_http_lua_finalize_request(r, NGX_DONE); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (ctx->entered_content_phase) { + ngx_http_lua_finalize_request(r, rc); + return NGX_DONE; + } + + return rc; +} + + /* executed in nginx event loop */ static void ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) @@ -300,7 +344,6 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) size_t len; const char *str; int i; - int rc; ngx_http_lua_ctx_t *ctx; lua_State *vm; int saved_top; @@ -350,6 +393,7 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) } ctx->cur_co_ctx = worker_thread_ctx->wait_co_ctx; + ctx->cur_co_ctx->nresults_from_worker_thread = nresults; ctx->cur_co_ctx->cleanup = NULL; ngx_http_lua_free_task_ctx(worker_thread_ctx->ctx); @@ -357,30 +401,15 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) /* resume the caller coroutine */ - vm = ngx_http_lua_get_lua_vm(r, ctx); - - rc = ngx_http_lua_run_thread(vm, r, ctx, nresults); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua run thread returned %d", rc); - - if (rc == NGX_AGAIN) { - ngx_http_lua_run_posted_threads(c, vm, r, ctx, c->requests); - return; - } + if (ctx->entered_content_phase) { + (void) ngx_http_lua_worker_thread_resume(r); - if (rc == NGX_DONE) { - ngx_http_lua_finalize_request(r, NGX_DONE); - ngx_http_lua_run_posted_threads(c, vm, r, ctx, c->requests); - return; + } else { + ctx->resume_handler = ngx_http_lua_worker_thread_resume; + ngx_http_core_run_phases(r); } - /* rc == NGX_ERROR || rc >= NGX_OK */ - - if (ctx->entered_content_phase) { - ngx_http_lua_finalize_request(r, rc); - return; - } + ngx_http_run_posted_requests(c); return; diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 8b874c55ad..3288f3a15b 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1208,3 +1208,41 @@ return {hello=hello} GET /hello --- response_body_like chop ^true : \d+$ + + + +=== TEST 39: write_log_file +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /write_log_file { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool", "write_log_file", "log", ngx.var.arg_str) + if not ok then + ngx.say(ok, " : ", err) + return + end + ngx.say(ok) + } +} +--- user_files +>>> write_log_file.lua +local function log(str) + local file, err = io.open("/tmp/tmp.log", "w") + if not file then + return false, err + end + file:write(str) + file:flush() + file:close() + return true +end +return {log=log} +--- request +GET /write_log_file?str=hello +--- response_body +true From c8597eda0c3181f15ff58dedf3fb41f4259146fb Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 23 Jun 2022 19:55:30 +0800 Subject: [PATCH 633/848] tests: change the running phase to cover the bug reported in google group. (#2064) bug: https://groups.google.com/g/openresty/c/qkS_9whMvXc --- t/166-worker-thread.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 3288f3a15b..2762acde54 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1220,7 +1220,7 @@ GET /hello location /write_log_file { default_type 'text/plain'; - content_by_lua_block { + access_by_lua_block { local ok, err = ngx.run_worker_thread("testpool", "write_log_file", "log", ngx.var.arg_str) if not ok then ngx.say(ok, " : ", err) From b6d167cf1a93c0c885c28db5a439f2404874cb26 Mon Sep 17 00:00:00 2001 From: Hiroaki Nakamura Date: Tue, 28 Jun 2022 14:37:24 +0900 Subject: [PATCH 634/848] bugfix: update handling of multiple headers changed in nginx 1.23.0 (#2063) --- src/ngx_http_lua_headers_in.c | 46 ++++++++++++++++++++++++ src/ngx_http_lua_headers_out.c | 64 ++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+) diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index a55d9cac7e..4405481408 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -152,9 +152,15 @@ static ngx_http_lua_set_header_t ngx_http_lua_set_handlers[] = { ngx_http_set_builtin_header }, #endif +#if defined(nginx_version) && nginx_version >= 1023000 + { ngx_string("Cookie"), + offsetof(ngx_http_headers_in_t, cookie), + ngx_http_set_builtin_multi_header }, +#else { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookies), ngx_http_set_builtin_multi_header }, +#endif { ngx_null_string, 0, ngx_http_set_header } }; @@ -580,6 +586,45 @@ static ngx_int_t ngx_http_set_builtin_multi_header(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, ngx_str_t *value) { +#if defined(nginx_version) && nginx_version >= 1023000 + ngx_table_elt_t **headers, **ph, *h; + int nelts; + + headers = (ngx_table_elt_t **) ((char *) &r->headers_in + hv->offset); + + if (!hv->no_override && *headers != NULL) { + nelts = 0; + for (h = *headers; h; h = h->next) { + nelts++; + } + + *headers = NULL; + + dd("clear multi-value headers: %d", nelts); + } + + if (ngx_http_set_header_helper(r, hv, value, &h) == NGX_ERROR) { + return NGX_ERROR; + } + + if (value->len == 0) { + return NGX_OK; + } + + dd("new multi-value header: %p", h); + + if (*headers) { + for (ph = headers; *ph; ph = &(*ph)->next) { /* void */ } + *ph = h; + + } else { + *headers = h; + } + + h->next = NULL; + + return NGX_OK; +#else ngx_array_t *headers; ngx_table_elt_t **v, *h; @@ -626,6 +671,7 @@ ngx_http_set_builtin_multi_header(ngx_http_request_t *r, *v = h; return NGX_OK; +#endif } diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 6e9f9c19ac..571723d9a6 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -311,6 +311,69 @@ static ngx_int_t ngx_http_set_builtin_multi_header(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, ngx_str_t *value) { +#if defined(nginx_version) && nginx_version >= 1023000 + ngx_table_elt_t **headers, *h, *ho, **ph; + + headers = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset); + + if (hv->no_override) { + for (h = *headers; h; h = h->next) { + if (!h->hash) { + h->value = *value; + h->hash = hv->hash; + return NGX_OK; + } + } + + goto create; + } + + /* override old values (if any) */ + + if (*headers) { + for (h = (*headers)->next; h; h = h->next) { + h->hash = 0; + h->value.len = 0; + } + + h = *headers; + + h->value = *value; + + if (value->len == 0) { + h->hash = 0; + + } else { + h->hash = hv->hash; + } + + return NGX_OK; + } + +create: + + for (ph = headers; *ph; ph = &(*ph)->next) { /* void */ } + + ho = ngx_list_push(&r->headers_out.headers); + if (ho == NULL) { + return NGX_ERROR; + } + + ho->value = *value; + + if (value->len == 0) { + ho->hash = 0; + + } else { + ho->hash = hv->hash; + } + + ho->key = hv->key; + ho->next = NULL; + *ph = ho; + + return NGX_OK; +#else ngx_array_t *pa; ngx_table_elt_t *ho, **ph; ngx_uint_t i; @@ -384,6 +447,7 @@ ngx_http_set_builtin_multi_header(ngx_http_request_t *r, *ph = ho; return NGX_OK; +#endif } From c55fceb5cf9dbafa179b6925a13bb6f47787b0ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Fri, 1 Jul 2022 10:02:31 +0800 Subject: [PATCH 635/848] ci: add a way to check PR title automatically (#2067) Signed-off-by: spacewander --- .github/workflows/semantic-pull-request.yml | 32 +++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 .github/workflows/semantic-pull-request.yml diff --git a/.github/workflows/semantic-pull-request.yml b/.github/workflows/semantic-pull-request.yml new file mode 100644 index 0000000000..2112f9e790 --- /dev/null +++ b/.github/workflows/semantic-pull-request.yml @@ -0,0 +1,32 @@ +name: "Lint PR" + +on: + pull_request_target: + types: + - opened + - edited + - synchronize + +jobs: + main: + name: Validate PR title + runs-on: ubuntu-latest + steps: + - uses: amannn/action-semantic-pull-request@v4 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + # Configure which types are allowed. + # Default: https://github.com/commitizen/conventional-commit-types + types: | + change + chore + ci + doc + feature + fix + optimize + refactor + revert + style + test From 74b8d4b4f969508a47cd0ef9e510eec367f1e9c1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Fri, 1 Jul 2022 13:10:50 +0800 Subject: [PATCH 636/848] chore: update the PR types (#2069) Signed-off-by: spacewander --- .github/workflows/semantic-pull-request.yml | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/.github/workflows/semantic-pull-request.yml b/.github/workflows/semantic-pull-request.yml index 2112f9e790..12b87cb3c1 100644 --- a/.github/workflows/semantic-pull-request.yml +++ b/.github/workflows/semantic-pull-request.yml @@ -19,14 +19,12 @@ jobs: # Configure which types are allowed. # Default: https://github.com/commitizen/conventional-commit-types types: | - change - chore - ci - doc - feature - fix - optimize - refactor - revert - style - test + bugfix # bug fixes + change # backward incompatible changes + doc # documentation changes including code comments + editor # code editor related configurations + feature # implementing a new feature + optimize # performance optimizations + refactor # code refactoring and other code rearrangement + style # coding style changes + tests # test suite changes From 1c43f1c4f2018a0f999fbf45afc8daf0352b104a Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Fri, 1 Jul 2022 13:11:15 +0800 Subject: [PATCH 637/848] ci: bumped OpenSSL to 1.1.1p. (#2070) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 56ff18e272..67263fa449 100644 --- a/.travis.yml +++ b/.travis.yml @@ -52,7 +52,7 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1n OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1p OPENSSL_PATCH_VER=1.1.1f services: - memcached From 1fd1e83c80612998647ee7ea933d3fdb2e9d9d9b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 3 Jul 2022 20:24:34 +0800 Subject: [PATCH 638/848] bugfix: doesn't set flags for Darwin arm64. (#2071) --- config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config b/config index 0e572c8bea..c4068c852b 100644 --- a/config +++ b/config @@ -94,7 +94,7 @@ END case "$NGX_PLATFORM" in Darwin:*) case "$NGX_MACHINE" in - amd64 | x86_64 | i386) + amd64 | arm64 | x86_64 | i386) echo "adding extra linking options needed by LuaJIT on $NGX_MACHINE" luajit_ld_opt="$luajit_ld_opt -pagezero_size 10000 -image_base 100000000" ngx_feature_libs="$ngx_feature_libs -pagezero_size 10000 -image_base 100000000" From 99b29cd169c01274c127f812482b956abcad2d6e Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 10 Jul 2022 20:26:25 +0800 Subject: [PATCH 639/848] doc: clarify the ngx.req.set_uri_args api when using Lua table as the 'arg' argument. (#2073) --- README.markdown | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 17563b5757..ed8bb658b1 100644 --- a/README.markdown +++ b/README.markdown @@ -4982,7 +4982,9 @@ Multi-value arguments are also supported: ngx.req.set_uri_args({ a = 3, b = {5, 6} }) ``` -which will result in a query string like `a=3&b=5&b=6`. +which will result in a query string like `a=3&b=5&b=6` or `b=5&b=6&a=3`. + +**Note that when using Lua table as the `arg` argument, the order of the arguments in the result query string which change from time to time. If you would like to get an ordered result, you need to use Lua string as the `arg` argument.** This interface was first introduced in the `v0.3.1rc13` release. From 44486a45f4de09122811a07c697a69513a60b24e Mon Sep 17 00:00:00 2001 From: jinhua luo Date: Tue, 12 Jul 2022 10:59:15 +0800 Subject: [PATCH 640/848] feature: add shdict APIs into worker thread (#2074) --- README.markdown | 1 + src/ngx_http_lua_worker_thread.c | 10 + t/166-worker-thread.t | 449 +++++++++++++++++++++++++++++++ 3 files changed, 460 insertions(+) diff --git a/README.markdown b/README.markdown index ed8bb658b1..14ad18c1eb 100644 --- a/README.markdown +++ b/README.markdown @@ -9287,6 +9287,7 @@ Only the following ngx_lua APIs could be used in `function_name` function of the * `ngx.config.nginx_configure` * `ngx.config.ngx_lua_version` +* `ngx.shared.DICT` The first argument `threadpool` specifies the Nginx thread pool name defined by [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool). diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index d3e9cbfd30..1512be4daf 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -16,6 +16,7 @@ #include "ngx_http_lua_util.h" #include "ngx_http_lua_string.h" #include "ngx_http_lua_config.h" +#include "ngx_http_lua_shdict.h" #if (NGX_THREADS) @@ -148,6 +149,7 @@ ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) lua_newtable(vm); /* ngx.* */ ngx_http_lua_inject_string_api(vm); ngx_http_lua_inject_config_api(vm); + ngx_http_lua_inject_shdict_api(lmcf, vm); lua_setglobal(vm, "ngx"); /* inject API via ffi */ @@ -175,6 +177,14 @@ ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) return NULL; } + lua_getglobal(vm, "require"); + lua_pushstring(vm, "resty.core.shdict"); + if (lua_pcall(vm, 1, 0, 0) != 0) { + lua_close(vm); + ngx_free(ctx); + return NULL; + } + } else { ctx = ctxpool->next; ctxpool->next = ctx->next; diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 2762acde54..2b7258df7c 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1246,3 +1246,452 @@ return {log=log} GET /write_log_file?str=hello --- response_body true + + + +=== TEST 40: shdict get, int value +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 10m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictget { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + local ok, err = ngx.run_worker_thread("testpool", "test_shdict", "dictget") + ngx.say(ok, ",", err) + } +} +--- user_files +>>> test_shdict.lua +local function dictget(str) + local dogs = ngx.shared.dogs + return dogs:get("Jim") +end +return {dictget=dictget} +--- request +GET /dictget +--- response_body +true,8 + + + +=== TEST 41: shdict set nil in main thread +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 10m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictget { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + local ok, err = ngx.run_worker_thread("testpool", "test_shdict", "dictget") + ngx.say(ok, ",", err) + dogs:set("Jim", nil) + local ok, err = ngx.run_worker_thread("testpool", "test_shdict", "dictget") + ngx.say(ok, ",", err) + } +} +--- user_files +>>> test_shdict.lua +local function dictget(str) + local dogs = ngx.shared.dogs + return dogs:get("Jim") +end +return {dictget=dictget} +--- request +GET /dictget +--- response_body +true,8 +true,nil + + + +=== TEST 42: shdict set nil in worker thread +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 10m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictsetnil { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + local ok, err = ngx.run_worker_thread("testpool", "test_shdict", "dictsetnil") + ngx.say(ok, ",", err) + ngx.say(ok, ",", dogs:get("Jim")) + } +} +--- user_files +>>> test_shdict.lua +local function dictsetnil(str) + local dogs = ngx.shared.dogs + return dogs:set("Jim", nil) +end +return {dictsetnil=dictsetnil} +--- request +GET /dictsetnil +--- response_body +true,true +true,nil + + + +=== TEST 43: shdict get_stale +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 10m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictget { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8, 1) + ngx.sleep(2) + local ok, err = ngx.run_worker_thread("testpool", "test_shdict", "dictget") + ngx.say(ok, ",", err) + } +} +--- user_files +>>> test_shdict.lua +local function dictget(str) + local dogs = ngx.shared.dogs + return dogs:get_stale("Jim") +end +return {dictget=dictget} +--- request +GET /dictget +--- response_body +true,8 + + + +=== TEST 44: shdict add failed +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 10m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictadd { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + local ok, err, err2 = ngx.run_worker_thread("testpool", "test_shdict", "dictadd") + ngx.say(ok, ",", err, ",", err2) + } +} +--- user_files +>>> test_shdict.lua +local function dictadd(str) + local dogs = ngx.shared.dogs + local success, err = dogs:add("Jim", "hello") + return success, err +end +return {dictadd=dictadd} +--- request +GET /dictadd +--- response_body +true,false,exists + + + +=== TEST 45: shdict force add +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictadd { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + local bigstr = string.rep("A", 1024*1024*3) + dogs:set("Jim", bigstr) + local ok, ret, err, forcible = ngx.run_worker_thread("testpool", "test_shdict", "dictadd") + ngx.say(ok, ",", ret, ",", forcible, ",", dogs:get("Jim")) + } +} +--- user_files +>>> test_shdict.lua +local function dictadd(str) + local dogs = ngx.shared.dogs + local bigstr = string.rep("A", 1024*1024*5) + local success, err, forcible = dogs:add("King", bigstr) + return success, err, forcible +end +return {dictadd=dictadd} +--- request +GET /dictadd +--- response_body +true,true,true,nil + + + +=== TEST 46: shdict replace +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictreplace { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + local bigstr = string.rep("A", 1024*1024*3) + dogs:set("Jim", bigstr) + local ok, ret, err = ngx.run_worker_thread("testpool", "test_shdict", "dictreplace") + ngx.say(ok, ",", ret, ",", err, ",", dogs:get("Jim")) + } +} +--- user_files +>>> test_shdict.lua +local function dictreplace(str) + local dogs = ngx.shared.dogs + local success, err = dogs:replace("Jim", 8) + return success, err +end +return {dictreplace=dictreplace} +--- request +GET /dictreplace +--- response_body +true,true,nil,8 + + + +=== TEST 47: shdict replace not found +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictreplace { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + local ok, ret, err = ngx.run_worker_thread("testpool", "test_shdict", "dictreplace") + ngx.say(ok, ",", ret, ",", err) + } +} +--- user_files +>>> test_shdict.lua +local function dictreplace(str) + local dogs = ngx.shared.dogs + local success, err = dogs:replace("Jim", 8) + return success, err +end +return {dictreplace=dictreplace} +--- request +GET /dictreplace +--- response_body +true,false,not found + + + +=== TEST 48: shdict incr +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictincr { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + local success, err = dogs:set("Jim", 8) + local ok, ret, err = ngx.run_worker_thread("testpool", "test_shdict", "dictincr") + ngx.say(ok, ",", ret, ",", err, ",", dogs:get("Jim")) + } +} +--- user_files +>>> test_shdict.lua +local function dictincr(str) + local dogs = ngx.shared.dogs + local success, err = dogs:incr("Jim", 1) + return success, err +end +return {dictincr=dictincr} +--- request +GET /dictincr +--- response_body +true,9,nil,9 + + + +=== TEST 49: shdict lpush lpop +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictlpush { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:lpush("Jim", 8) + dogs:lpush("Jim", 9) + local ok, val, len, err = ngx.run_worker_thread("testpool", "test_shdict", "dictlpush") + ngx.say(ok, ",", val, ",", len, ",", err, ",", dogs:lpop("Jim")) + } +} +--- user_files +>>> test_shdict.lua +local function dictlpush(str) + local dogs = ngx.shared.dogs + local val = dogs:lpop("Jim") + local len, err = dogs:lpush("Jim", 7) + return val, len, err +end +return {dictlpush=dictlpush} +--- request +GET /dictlpush +--- response_body +true,9,2,nil,7 + + + +=== TEST 50: shdict expire ttl +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictexpire { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + local ok, success, err = ngx.run_worker_thread("testpool", "test_shdict", "dictexpire") + ngx.say(ok, ",", success, ",", err, ",", dogs:ttl("Jim") <= 1) + } +} +--- user_files +>>> test_shdict.lua +local function dictexpire(str) + local dogs = ngx.shared.dogs + local success, err = dogs:expire("Jim", 1) + return success, err +end +return {dictexpire=dictexpire} +--- request +GET /dictexpire +--- response_body +true,true,nil,true + + + +=== TEST 51: shdict flush_all +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictexpire { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + dogs:set("King", 9) + local ok = ngx.run_worker_thread("testpool", "test_shdict", "dictexpire") + ngx.say(ok, ",", dogs:get("Jim"), ",", dogs:get("King")) + } +} +--- user_files +>>> test_shdict.lua +local function dictexpire(str) + local dogs = ngx.shared.dogs + dogs:flush_all() +end +return {dictexpire=dictexpire} +--- request +GET /dictexpire +--- response_body +true,nil,nil + + + +=== TEST 52: shdict get_keys +--- main_config + thread_pool testpool threads=100; +--- http_config eval +" + lua_shared_dict dogs 6m; + lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; +" +--- config +location /dictgetkeys { + default_type 'text/plain'; + + access_by_lua_block { + local dogs = ngx.shared.dogs + dogs:set("Jim", 8) + dogs:set("King", 9) + local ok, keys = ngx.run_worker_thread("testpool", "test_shdict", "dictgetkeys") + ngx.say(ok, ",", table.concat(keys, ":")) + } +} +--- user_files +>>> test_shdict.lua +local function dictgetkeys(str) + local dogs = ngx.shared.dogs + return dogs:get_keys() +end +return {dictgetkeys=dictgetkeys} +--- request +GET /dictgetkeys +--- response_body +true,Jim:King From 9b7bde2e7986d2150606e33e303c6bd1c40271a6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 13 Jul 2022 09:31:58 +0800 Subject: [PATCH 641/848] feature: added new API bind. Co-authored-by: doujiang24 Co-authored-by: lijunlong --- .travis.yml | 2 +- README.markdown | 38 ++++ doc/HttpLuaModule.wiki | 33 ++++ src/ngx_http_lua_socket_tcp.c | 71 ++++++- src/ngx_http_lua_util.c | 73 +++++++ src/ngx_http_lua_util.h | 2 + t/062-count.t | 2 +- t/168-tcp-socket-bind.t | 361 ++++++++++++++++++++++++++++++++++ 8 files changed, 579 insertions(+), 3 deletions(-) create mode 100644 t/168-tcp-socket-bind.t diff --git a/.travis.yml b/.travis.yml index 67263fa449..214420015e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -141,4 +141,4 @@ script: - dig +short myip.opendns.com @resolver1.opendns.com || exit 0 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 - dig +short @$TEST_NGINX_RESOLVER agentzh.org || exit 0 - - prove -I. -Itest-nginx/lib -r t + - prove -I. -Itest-nginx/lib -r t/ diff --git a/README.markdown b/README.markdown index 14ad18c1eb..6466a1edff 100644 --- a/README.markdown +++ b/README.markdown @@ -3682,6 +3682,7 @@ Nginx API for Lua * [udpsock:settimeout](#udpsocksettimeout) * [ngx.socket.stream](#ngxsocketstream) * [ngx.socket.tcp](#ngxsockettcp) +* [tcpsock:bind](#tcpsockbind) * [tcpsock:connect](#tcpsockconnect) * [tcpsock:setclientcert](#tcpsocksetclientcert) * [tcpsock:sslhandshake](#tcpsocksslhandshake) @@ -7656,6 +7657,7 @@ ngx.socket.tcp Creates and returns a TCP or stream-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: +* [bind](#tcpsockbind) * [connect](#tcpsockconnect) * [setclientcert](#tcpsocksetclientcert) * [sslhandshake](#tcpsocksslhandshake) @@ -7695,6 +7697,42 @@ See also [ngx.socket.udp](#ngxsocketudp). [Back to TOC](#nginx-api-for-lua) +tcpsock:bind +------------ +**syntax:** *ok, err = tcpsock:bind(address)* + +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*,ssl_session_fetch_by_lua*,ssl_client_hello_by_lua** + +Just like the standard [proxy_bind](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_bind) directive, this api makes the outgoing connection to a upstream server originate from the specified local IP address. + +Only IP addresses can be specified as the `address` argument. + +Here is an example for connecting to a TCP server from the specified local IP address: + +```nginx + + location /test { + content_by_lua_block { + local sock = ngx.socket.tcp() + -- assume "192.168.1.10" is the local ip address + local ok, err = sock:bind("192.168.1.10") + if not ok then + ngx.say("failed to bind") + return + end + local ok, err = sock:connect("192.168.1.67", 80) + if not ok then + ngx.say("failed to connect server: ", err) + return + end + ngx.say("successfully connected!") + sock:close() + } + } +``` + +[Back to TOC](#nginx-api-for-lua) + tcpsock:connect --------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 371b9cb84f..74e298df8f 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -6498,6 +6498,7 @@ This API function was first added to the v0.10.1 release. Creates and returns a TCP or stream-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: +* [[#tcpsock:bind|bind]] * [[#tcpsock:connect|connect]] * [[#tcpsock:setclientcert|setclientcert]] * [[#tcpsock:sslhandshake|sslhandshake]] @@ -6535,6 +6536,38 @@ This feature was first introduced in the v0.5.0rc1 release. See also [[#ngx.socket.udp|ngx.socket.udp]]. +== tcpsock:bind == +'''syntax:''' ''ok, err = tcpsock:bind(address)'' + +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*'' + +Just like the standard [[HttpProxyModule#proxy_bind|proxy_bind]] directive, this api makes the outgoing connection to a upstream server originate from the specified local IP address. + +Only IP addresses can be specified as the address argument. + +Here is an example for connecting to a TCP server from the specified local IP address: + + + location /test { + content_by_lua_block { + local sock = ngx.socket.tcp() + -- assume "192.168.1.10" is the local ip address + local ok, err = sock:bind("192.168.1.10") + if not ok then + ngx.say("failed to bind") + return + end + local ok, err = sock:connect("192.168.1.67", 80) + if not ok then + ngx.say("failed to connect server: ", err) + return + end + ngx.say("successfully connected!") + sock:close() + } + } + + == tcpsock:connect == '''syntax:''' ''ok, err = tcpsock:connect(host, port, options_table?)'' diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index d8f1d4d1c1..e40ecbb537 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -20,6 +20,7 @@ static int ngx_http_lua_socket_tcp(lua_State *L); +static int ngx_http_lua_socket_tcp_bind(lua_State *L); static int ngx_http_lua_socket_tcp_connect(lua_State *L); #if (NGX_HTTP_SSL) static void ngx_http_lua_ssl_handshake_handler(ngx_connection_t *c); @@ -162,6 +163,7 @@ enum { SOCKET_READ_TIMEOUT_INDEX = 5, SOCKET_CLIENT_CERT_INDEX = 6 , SOCKET_CLIENT_PKEY_INDEX = 7 , + SOCKET_BIND_INDEX = 8 /* only in upstream cosocket */ }; @@ -314,7 +316,10 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) /* {{{tcp object metatable */ lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( tcp_socket_metatable_key)); - lua_createtable(L, 0 /* narr */, 15 /* nrec */); + lua_createtable(L, 0 /* narr */, 16 /* nrec */); + + lua_pushcfunction(L, ngx_http_lua_socket_tcp_bind); + lua_setfield(L, -2, "bind"); lua_pushcfunction(L, ngx_http_lua_socket_tcp_connect); lua_setfield(L, -2, "connect"); @@ -827,6 +832,61 @@ ngx_http_lua_socket_tcp_connect_helper(lua_State *L, } +static int +ngx_http_lua_socket_tcp_bind(lua_State *L) +{ + ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + int n; + u_char *text; + size_t len; + ngx_addr_t *local; + + n = lua_gettop(L); + + if (n != 2) { + return luaL_error(L, "expecting 2 arguments, but got %d", + lua_gettop(L)); + } + + r = ngx_http_lua_get_req(L); + if (r == NULL) { + return luaL_error(L, "no request found"); + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return luaL_error(L, "no ctx found"); + } + + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE + | NGX_HTTP_LUA_CONTEXT_ACCESS + | NGX_HTTP_LUA_CONTEXT_CONTENT + | NGX_HTTP_LUA_CONTEXT_TIMER + | NGX_HTTP_LUA_CONTEXT_SSL_CERT); + + luaL_checktype(L, 1, LUA_TTABLE); + + text = (u_char *) luaL_checklstring(L, 2, &len); + + local = ngx_http_lua_parse_addr(L, text, len); + if (local == NULL) { + lua_pushnil(L); + lua_pushfstring(L, "bad address"); + return 2; + } + + /* TODO: we may reuse the userdata here */ + lua_rawseti(L, 1, SOCKET_BIND_INDEX); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket bind ip: %V", &local->name); + + lua_pushboolean(L, 1); + return 1; +} + + static int ngx_http_lua_socket_tcp_connect(lua_State *L) { @@ -838,6 +898,7 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) size_t len; ngx_http_lua_loc_conf_t *llcf; ngx_peer_connection_t *pc; + ngx_addr_t *local; int connect_timeout, send_timeout, read_timeout; unsigned custom_pool; int key_index; @@ -1068,6 +1129,14 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) dd("lua peer connection log: %p", pc->log); + lua_rawgeti(L, 1, SOCKET_BIND_INDEX); + local = lua_touserdata(L, -1); + lua_pop(L, 1); + + if (local) { + u->peer.local = local; + } + lua_rawgeti(L, 1, SOCKET_CONNECT_TIMEOUT_INDEX); lua_rawgeti(L, 1, SOCKET_SEND_TIMEOUT_INDEX); lua_rawgeti(L, 1, SOCKET_READ_TIMEOUT_INDEX); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 39ba0b21fb..8fd26561a7 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4396,4 +4396,77 @@ ngx_http_lua_copy_escaped_header(ngx_http_request_t *r, return NGX_OK; } + +ngx_addr_t * +ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len) +{ + ngx_addr_t *addr; + size_t socklen; + in_addr_t inaddr; + ngx_uint_t family; + struct sockaddr_in *sin; +#if (NGX_HAVE_INET6) + struct in6_addr inaddr6; + struct sockaddr_in6 *sin6; + + /* + * prevent MSVC8 warning: + * potentially uninitialized local variable 'inaddr6' used + */ + ngx_memzero(&inaddr6, sizeof(struct in6_addr)); +#endif + + inaddr = ngx_inet_addr(text, len); + + if (inaddr != INADDR_NONE) { + family = AF_INET; + socklen = sizeof(struct sockaddr_in); + +#if (NGX_HAVE_INET6) + + } else if (ngx_inet6_addr(text, len, inaddr6.s6_addr) == NGX_OK) { + family = AF_INET6; + socklen = sizeof(struct sockaddr_in6); +#endif + + } else { + return NULL; + } + + addr = lua_newuserdata(L, sizeof(ngx_addr_t) + socklen + len); + if (addr == NULL) { + luaL_error(L, "no memory"); + return NULL; + } + + addr->sockaddr = (struct sockaddr *) ((u_char *) addr + sizeof(ngx_addr_t)); + + ngx_memzero(addr->sockaddr, socklen); + + addr->sockaddr->sa_family = (u_char) family; + addr->socklen = socklen; + + switch (family) { + +#if (NGX_HAVE_INET6) + case AF_INET6: + sin6 = (struct sockaddr_in6 *) addr->sockaddr; + ngx_memcpy(sin6->sin6_addr.s6_addr, inaddr6.s6_addr, 16); + break; +#endif + + default: /* AF_INET */ + sin = (struct sockaddr_in *) addr->sockaddr; + sin->sin_addr.s_addr = inaddr; + break; + } + + addr->name.data = (u_char *) addr->sockaddr + socklen; + addr->name.len = len; + ngx_memcpy(addr->name.data, text, len); + + return addr; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 4c4da976c5..faea7a072c 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -262,6 +262,8 @@ void ngx_http_lua_cleanup_free(ngx_http_request_t *r, void ngx_http_lua_set_sa_restart(ngx_log_t *log); #endif +ngx_addr_t *ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len); + size_t ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size); diff --git a/t/062-count.t b/t/062-count.t index d977909bb6..d524da47d8 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -459,7 +459,7 @@ worker: 4 --- request GET /test --- response_body -n = 15 +n = 16 --- no_error_log [error] diff --git a/t/168-tcp-socket-bind.t b/t/168-tcp-socket-bind.t new file mode 100644 index 0000000000..6aca00c51e --- /dev/null +++ b/t/168-tcp-socket-bind.t @@ -0,0 +1,361 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +# more times than usual(2) for test case 6 +repeat_each(4); + +plan tests => repeat_each() * (blocks() * 3 + 7); + +our $HtmlDir = html_dir; + +# get ip address in the dev which is default route outgoing dev +my $dev = `ip route | awk '/default/ {printf "%s", \$5}'`; +my $local_ip = `ip route | grep $dev | grep -o "src .*" | head -n 1 | awk '{print \$2}'`; +chomp $local_ip; + +$ENV{TEST_NGINX_HTML_DIR} = $HtmlDir; +$ENV{TEST_NGINX_NOT_EXIST_IP} ||= '8.8.8.8'; +$ENV{TEST_NGINX_INVALID_IP} ||= '127.0.0.1:8899'; +$ENV{TEST_NGINX_SERVER_IP} ||= $local_ip; + +no_long_string(); +#no_diff(); + +#log_level 'warn'; +log_level 'debug'; + +no_shuffle(); + +run_tests(); + +__DATA__ + +=== TEST 1: upstream sockets bind 127.0.0.1 +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local ip = "127.0.0.1" + local port = ngx.var.port + + local sock = ngx.socket.tcp() + local ok, err = sock:bind(ip) + if not ok then + ngx.say("failed to bind", err) + return + end + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local bytes, err = sock:send("GET /foo HTTP/1.1\r\nHost: localhost\r\nConnection: keepalive\r\n\r\n") + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent") + + local reader = sock:receiveuntil("\r\n0\r\n\r\n") + local data, err = reader() + + if not data then + ngx.say("failed to receive response body: ", err) + return + end + + ngx.say("received response") + local remote_ip = string.match(data, "(bind: %d+%.%d+%.%d+%.%d+)") + ngx.say(remote_ip) + + ngx.say("done") + } + } + + location /foo { + echo bind: $remote_addr; + } +--- request +GET /t +--- response_body +connected: 1 +request sent +received response +bind: 127.0.0.1 +done +--- no_error_log +["[error]", +"bind(127.0.0.1) failed"] +--- error_log eval +"lua tcp socket bind ip: 127.0.0.1" + + + +=== TEST 2: upstream sockets bind server ip, not 127.0.0.1 +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local ip = "$TEST_NGINX_SERVER_IP" + local port = ngx.var.port + + local sock = ngx.socket.tcp() + local ok, err = sock:bind(ip) + if not ok then + ngx.say("failed to bind", err) + return + end + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local bytes, err = sock:send("GET /foo HTTP/1.1\r\nHost: localhost\r\nConnection: keepalive\r\n\r\n") + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent") + + local reader = sock:receiveuntil("\r\n0\r\n\r\n") + local data, err = reader() + + if not data then + ngx.say("failed to receive response body: ", err) + return + end + + ngx.say("received response") + local remote_ip = string.match(data, "(bind: %d+%.%d+%.%d+%.%d+)") + if remote_ip == "bind: $TEST_NGINX_SERVER_IP" then + ngx.say("ip matched") + end + + ngx.say("done") + } + } + + location /foo { + echo bind: $remote_addr; + } +--- request +GET /t +--- response_body +connected: 1 +request sent +received response +ip matched +done +--- no_error_log eval +["[error]", +"bind($ENV{TEST_NGINX_SERVER_IP}) failed"] +--- error_log eval +"lua tcp socket bind ip: $ENV{TEST_NGINX_SERVER_IP}" + + + +=== TEST 3: add setkeepalive +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local test = require "test" + local t1 = test.go() + local t2 = test.go() + ngx.say("t2 - t1: ", t2 - t1) + } + } +--- user_files +>>> test.lua +local _M = {} + +function _M.go() + local ip = "127.0.0.1" + local port = ngx.var.port + + local sock = ngx.socket.tcp() + local ok, err = sock:bind(ip) + if not ok then + ngx.say("failed to bind", err) + return + end + + ngx.say("bind: ", ip) + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local reused = sock:getreusedtimes() + + local ok, err = sock:setkeepalive() + if not ok then + ngx.say("failed to set reusable: ", err) + end + + return reused +end + +return _M +--- request +GET /t +--- response_body +bind: 127.0.0.1 +connected: 1 +bind: 127.0.0.1 +connected: 1 +t2 - t1: 1 +--- no_error_log +["[error]", +"bind(127.0.0.1) failed"] +--- error_log eval +"lua tcp socket bind ip: 127.0.0.1" + + + +=== TEST 4: upstream sockets bind not exist ip +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local ip = "$TEST_NGINX_NOT_EXIST_IP" + local port = ngx.var.port + + local sock = ngx.socket.tcp() + local ok, err = sock:bind(ip) + if not ok then + ngx.say("failed to bind", err) + return + end + + ngx.say("bind: ", ip) + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + } + } +--- request +GET /t +--- response_body +bind: 8.8.8.8 +failed to connect: cannot assign requested address +--- error_log eval +["bind(8.8.8.8) failed", +"lua tcp socket bind ip: 8.8.8.8"] + + + +=== TEST 5: upstream sockets bind invalid ip +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local ip = "$TEST_NGINX_INVALID_IP" + local port = ngx.var.port + + local sock = ngx.socket.tcp() + local ok, err = sock:bind(ip) + if not ok then + ngx.say("failed to bind: ", err) + return + end + + ngx.say("bind: ", ip) + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + } + } +--- request +GET /t +--- response_body +failed to bind: bad address +--- no_error_log +[error] + + + +=== TEST 6: tcpsock across request after bind +--- http_config + init_worker_by_lua_block { + -- this is not the recommend way, just for test + local function tcp() + local sock = ngx.socket.tcp() + + ---[[ + local ok, err = sock:bind("127.0.0.1") + if not ok then + ngx.log(ngx.ERR, "failed to bind") + end + --]] + + package.loaded.share_sock = sock + end + + local ok, err = ngx.timer.at(0, tcp) + if not ok then + ngx.log(ngx.ERR, "failed to create timer") + end + } +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local port = ngx.var.port + + -- make sure share_sock is created + ngx.sleep(0.002) + + local sock = package.loaded.share_sock + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + sock:close() + collectgarbage("collect") + } + } +--- request +GET /t +--- response_body +connected: 1 +--- no_error_log +[error] From 73f2b3f6fe13b4852ad4ae02a62d33f41879eaf9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=81=AA?= Date: Thu, 4 Aug 2022 23:54:55 +0800 Subject: [PATCH 642/848] bugfix: typo in the debug message. (#2081) --- src/ngx_http_lua_uri.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_uri.c b/src/ngx_http_lua_uri.c index c37dde9cb2..c4d65f2905 100644 --- a/src/ngx_http_lua_uri.c +++ b/src/ngx_http_lua_uri.c @@ -93,7 +93,7 @@ ngx_http_lua_ngx_req_set_uri(lua_State *L) } dd("server_rewrite: %d, rewrite: %d, access: %d, content: %d", - (int) ctx->entered_serverrewrite_phase, + (int) ctx->entered_server_rewrite_phase, (int) ctx->entered_rewrite_phase, (int) ctx->entered_access_phase, (int) ctx->entered_content_phase); From a7e67a596f707c916b009b17b3d22b925e6f2e11 Mon Sep 17 00:00:00 2001 From: levy001 Date: Sun, 7 Aug 2022 08:59:34 +0800 Subject: [PATCH 643/848] doc: add more explanation for premature in ngx.timer.at. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 6466a1edff..fd2ab039b0 100644 --- a/README.markdown +++ b/README.markdown @@ -8749,7 +8749,7 @@ be any Lua function, which will be invoked later in a background called automatically by the Nginx core with the arguments `premature`, `user_arg1`, `user_arg2`, and etc, where the `premature` argument takes a boolean value indicating whether it is a premature timer -expiration or not, and `user_arg1`, `user_arg2`, and etc, are +expiration or not(for the `0` delay timer it is always `false`), and `user_arg1`, `user_arg2`, and etc, are those (extra) user arguments specified when calling `ngx.timer.at` as the remaining arguments. From 5eb7101d3307f82226b83209806cf90a483fd236 Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Fri, 12 Aug 2022 09:23:35 +0800 Subject: [PATCH 644/848] doc: complete md5.lua example in run_worker_thread api (#2085) --- README.markdown | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index fd2ab039b0..cf3fb87b1a 100644 --- a/README.markdown +++ b/README.markdown @@ -9377,6 +9377,8 @@ Example1: do md5 calculation. local function md5() return ngx.md5("hello") end + +return { md5=md5, } ``` Example2: write logs into the log file. From eb6d6cbf00bc4891ad2d8af0d42ec722ce8bec30 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 12 Aug 2022 12:52:10 +0800 Subject: [PATCH 645/848] tests: build with DDEBUG=1. (#2084) --- .travis.yml | 1 + util/build-with-dd.sh | 57 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100755 util/build-with-dd.sh diff --git a/.travis.yml b/.travis.yml index 214420015e..9c579ab75d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -131,6 +131,7 @@ script: - cd .. - export NGX_BUILD_CC=$CC - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) + - sh util/build-with-dd.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - rm -fr buildroot - sh util/build.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - nginx -V diff --git a/util/build-with-dd.sh b/util/build-with-dd.sh new file mode 100755 index 0000000000..41ecc72685 --- /dev/null +++ b/util/build-with-dd.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash + +# this script is for developers only. +# dependent on the ngx-build script from the nginx-devel-utils repository: +# https://github.com/openresty/nginx-devel-utils/blob/master/ngx-build +# the resulting nginx is located at ./work/nginx/sbin/nginx + +root=`pwd` +version=${1:-1.4.1} +home=~ +force=$2 + +add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" + +time ngx-build $force $version \ + --with-threads \ + --with-pcre-jit \ + --with-ipv6 \ + --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC -DDDEBUG=1" \ + --with-http_v2_module \ + --with-http_realip_module \ + --with-http_ssl_module \ + --add-module=$root/../ndk-nginx-module \ + --add-module=$root/../set-misc-nginx-module \ + --with-ld-opt="-L$PCRE_LIB -L$OPENSSL_LIB -Wl,-rpath,$PCRE_LIB:$LIBDRIZZLE_LIB:$OPENSSL_LIB" \ + --without-mail_pop3_module \ + --without-mail_imap_module \ + --with-http_image_filter_module \ + --without-mail_smtp_module \ + --with-stream \ + --with-stream_ssl_module \ + --without-http_upstream_ip_hash_module \ + --without-http_memcached_module \ + --without-http_auth_basic_module \ + --without-http_userid_module \ + --with-http_auth_request_module \ + --add-module=$root/../echo-nginx-module \ + --add-module=$root/../memc-nginx-module \ + --add-module=$root/../srcache-nginx-module \ + --add-module=$root \ + --add-module=$root/../lua-upstream-nginx-module \ + --add-module=$root/../headers-more-nginx-module \ + --add-module=$root/../drizzle-nginx-module \ + --add-module=$root/../rds-json-nginx-module \ + --add-module=$root/../coolkit-nginx-module \ + --add-module=$root/../redis2-nginx-module \ + --add-module=$root/../stream-lua-nginx-module \ + --add-module=$root/t/data/fake-module \ + $add_fake_shm_module \ + --add-module=$root/t/data/fake-delayed-load-module \ + --with-http_gunzip_module \ + --with-http_dav_module \ + --with-select_module \ + --with-poll_module \ + $opts \ + --with-debug + From 3e2a7f459353a26d9475a54661c70a6e2c8cd27e Mon Sep 17 00:00:00 2001 From: fesily Date: Sun, 14 Aug 2022 11:53:45 +0800 Subject: [PATCH 646/848] bugfix: run_worker_thread arg is self-reference. Co-authored-by: lijunlong@openresty.com --- src/ngx_http_lua_worker_thread.c | 58 +++++++++++++++++++-------- t/166-worker-thread.t | 67 ++++++++++++++++++++++++++++++-- 2 files changed, 106 insertions(+), 19 deletions(-) diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 1512be4daf..ed7b089295 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -18,6 +18,10 @@ #include "ngx_http_lua_config.h" #include "ngx_http_lua_shdict.h" +#ifndef STRINGIFY +#define TOSTRING(x) #x +#define STRINGIFY(x) TOSTRING(x) +#endif #if (NGX_THREADS) @@ -25,6 +29,7 @@ #include #include +#define LUA_COPY_MAX_DEPTH 100 typedef struct ngx_http_lua_task_ctx_s { lua_State *vm; @@ -208,7 +213,7 @@ ngx_http_lua_free_task_ctx(ngx_http_lua_task_ctx_t *ctx) static int ngx_http_lua_xcopy(lua_State *from, lua_State *to, int idx, - const int allow_nil) + const int allow_nil, const int depth, const char **err) { size_t len = 0; const char *str; @@ -235,6 +240,13 @@ ngx_http_lua_xcopy(lua_State *from, lua_State *to, int idx, return LUA_TSTRING; case LUA_TTABLE: + if (depth >= LUA_COPY_MAX_DEPTH) { + *err = "suspicious circular references, " + "table depth exceed max depth: " + STRINGIFY(LUA_COPY_MAX_DEPTH); + return LUA_TNONE; + } + top_from = lua_gettop(from); top_to = lua_gettop(to); @@ -248,8 +260,9 @@ ngx_http_lua_xcopy(lua_State *from, lua_State *to, int idx, lua_pushnil(from); while (lua_next(from, idx) != 0) { - if (ngx_http_lua_xcopy(from, to, -2, 0) != LUA_TNONE - && ngx_http_lua_xcopy(from, to, -1, 0) != LUA_TNONE) + if (ngx_http_lua_xcopy(from, to, -2, 0, depth + 1, err) != LUA_TNONE + && ngx_http_lua_xcopy(from, to, -1, 0, + depth + 1, err) != LUA_TNONE) { lua_rawset(to, -3); @@ -269,16 +282,24 @@ ngx_http_lua_xcopy(lua_State *from, lua_State *to, int idx, lua_pushnil(to); return LUA_TNIL; } - /* fall through */ - - /* - * ignore unsupported values: - * LUA_TNONE - * LUA_TFUNCTION - * LUA_TUSERDATA - * LUA_TTHREAD - */ + + *err = "unsupported Lua type: LUA_TNIL"; + return LUA_TNONE; + + case LUA_TFUNCTION: + *err = "unsupported Lua type: LUA_TFUNCTION"; + return LUA_TNONE; + + case LUA_TUSERDATA: + *err = "unsupported Lua type: LUA_TUSERDATA"; + return LUA_TNONE; + + case LUA_TTHREAD: + *err = "unsupported Lua type: LUA_TTHREAD"; + return LUA_TNONE; + default: + *err = "unsupported Lua type"; return LUA_TNONE; } } @@ -357,6 +378,7 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) ngx_http_lua_ctx_t *ctx; lua_State *vm; int saved_top; + const char *err; worker_thread_ctx = ev->data; @@ -392,10 +414,12 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) lua_pushboolean(L, 1); nresults = lua_gettop(vm) + 1; for (i = 1; i < nresults; i++) { - if (ngx_http_lua_xcopy(vm, L, i, 1) == LUA_TNONE) { + err = NULL; + if (ngx_http_lua_xcopy(vm, L, i, 1, 1, &err) == LUA_TNONE) { lua_settop(L, saved_top); lua_pushboolean(L, 0); - lua_pushstring(L, "unsupported return value"); + lua_pushfstring(L, "%s in the return value", + err != NULL ? err : "unsupoorted Lua type"); nresults = 2; break; } @@ -557,9 +581,11 @@ ngx_http_lua_run_worker_thread(lua_State *L) /* copying passed arguments */ for (i = 4; i <= n_args; i++) { - if (ngx_http_lua_xcopy(L, vm, i, 1) == LUA_TNONE) { + err = NULL; + if (ngx_http_lua_xcopy(L, vm, i, 1, 1, &err) == LUA_TNONE) { lua_pushboolean(L, 0); - lua_pushstring(L, "unsupported argument type"); + lua_pushfstring(L, "%s in the argument", + err != NULL ? err : "unsupoorted Lua type"); ngx_http_lua_free_task_ctx(tctx); return 2; } diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 2b7258df7c..fd7fdd756e 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -229,7 +229,7 @@ return {hello=hello} --- request GET /hello --- response_body -false : unsupported argument type +false : unsupported Lua type: LUA_TFUNCTION in the argument @@ -491,7 +491,7 @@ return {hello=hello} --- request GET /hello --- response_body -false : unsupported argument type +false : unsupported Lua type: LUA_TFUNCTION in the argument @@ -524,7 +524,7 @@ return {hello=hello} --- request GET /hello --- response_body -false , unsupported return value +false , unsupported Lua type: LUA_TFUNCTION in the return value @@ -1695,3 +1695,64 @@ return {dictgetkeys=dictgetkeys} GET /dictgetkeys --- response_body true,Jim:King + + + +=== TEST 53: unsupported argument type in self-reference table +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local t = {} + t.a = t + local ok, ok_or_err = ngx.run_worker_thread("testpool", "hello", "hello", t) + ngx.say(ok, " , ", ok_or_err) + } +} +--- user_files +>>> hello.lua +local function hello(arg1) + return true +end +return {hello=hello} +--- request +GET /hello +--- response_body +false , suspicious circular references, table depth exceed max depth: 100 in the argument + + + +=== TEST 54: unsupported argument type in circular-reference table +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local t = {} + local s = {} + t.a = s + s.b = t + + local ok, ok_or_err = ngx.run_worker_thread("testpool", "hello", "hello", t) + ngx.say(ok, " , ", ok_or_err) + } +} +--- user_files +>>> hello.lua +local function hello(arg1) + return true +end +return {hello=hello} +--- request +GET /hello +--- response_body +false , suspicious circular references, table depth exceed max depth: 100 in the argument From edfa0f984ec60bd0658b80643c2fd253f3c5ad0b Mon Sep 17 00:00:00 2001 From: attenuation Date: Sun, 21 Aug 2022 21:59:28 +0800 Subject: [PATCH 647/848] feat: add ngx_http_lua_ffi_worker_pids to get all workers pid map --- README.markdown | 12 ++++++++++++ doc/HttpLuaModule.wiki | 7 +++++++ src/ngx_http_lua_worker.c | 31 +++++++++++++++++++++++++++++++ t/062-count.t | 2 +- t/122-worker.t | 32 +++++++++++++++++++++++++++++++- 5 files changed, 82 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index cf3fb87b1a..7a5ba6b295 100644 --- a/README.markdown +++ b/README.markdown @@ -3714,6 +3714,7 @@ Nginx API for Lua * [ngx.config.ngx_lua_version](#ngxconfigngx_lua_version) * [ngx.worker.exiting](#ngxworkerexiting) * [ngx.worker.pid](#ngxworkerpid) +* [ngx.worker.pids](#ngxworkerpids) * [ngx.worker.count](#ngxworkercount) * [ngx.worker.id](#ngxworkerid) * [ngx.semaphore](#ngxsemaphore) @@ -9025,6 +9026,17 @@ This API was first introduced in the `0.9.5` release. [Back to TOC](#nginx-api-for-lua) +ngx.worker.pids +-------------- + +**syntax:** *pids = ngx.worker.pids()* + +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, exit_worker_by_lua** + +This function returns a Lua table for all Nginx worker process ID (PID). Nginx uses channel to send the current worker PID to another worker in the worker process start or restart. So this API can get all current worker PID. + +[Back to TOC](#nginx-api-for-lua) + ngx.worker.count ---------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 74e298df8f..6b2bb99476 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7742,6 +7742,13 @@ This function returns a Lua number for the process ID (PID) of the current Nginx This API was first introduced in the 0.9.5 release. +== ngx.worker.pids == + +'''syntax:''' ''pid = ngx.worker.pids()'' + +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, exit_worker_by_lua*'' + +This function returns a Lua table for all Nginx worker process ID (PID). Nginx uses channel to send the current worker PID to another worker in the worker process start or restart. So this API can get all current worker PID. == ngx.worker.count == '''syntax:''' ''count = ngx.worker.count()'' diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 0ca2d414e3..52ec34a844 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -8,6 +8,7 @@ #define DDEBUG 0 #endif #include "ddebug.h" +#include #define NGX_PROCESS_PRIVILEGED_AGENT 99 @@ -20,6 +21,36 @@ ngx_http_lua_ffi_worker_pid(void) } +int +ngx_http_lua_ffi_worker_pids(int *pids, size_t *pids_len) +{ + ngx_int_t i, n; + + n = 0; + for (i = 0; i < NGX_MAX_PROCESSES; i++) { + if (i != ngx_process_slot && ngx_processes[i].pid == 0) { + break; + } + + if (i == ngx_process_slot && ngx_processes[i].pid == 0) { + pids[n++] = ngx_pid; + } + + if (ngx_processes[i].pid > 0) { + pids[n++] = ngx_processes[i].pid; + } + } + + if (n == 0) { + return NGX_ERROR; + } + + *pids_len = n; + + return NGX_OK; +} + + int ngx_http_lua_ffi_worker_id(void) { diff --git a/t/062-count.t b/t/062-count.t index d524da47d8..ad464ba2de 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -438,7 +438,7 @@ thread: 3 --- request GET /test --- response_body -worker: 4 +worker: 5 --- no_error_log [error] diff --git a/t/122-worker.t b/t/122-worker.t index b74c81fbcd..244c3142d3 100644 --- a/t/122-worker.t +++ b/t/122-worker.t @@ -9,7 +9,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3); +plan tests => repeat_each() * (blocks() * 4 - 4); #no_diff(); no_long_string(); @@ -79,3 +79,33 @@ worker pid: \d+ worker pid is correct\. --- no_error_log [error] + + + +=== TEST 4: content_by_lua + ngx.worker.pids +--- config + location /lua { + content_by_lua ' + local pids = ngx.worker.pids() + local pid = ngx.worker.pid() + ngx.say("worker pid: ", pid) + local count = ngx.worker.count() + if count ~= #pids then + ngx.say("worker pids is wrong.") + end + for i = 1, count do + if pids[i] == pid then + ngx.say("worker pid is correct.") + return + end + end + ngx.say("worker pid is wrong.") + '; + } +--- request +GET /lua +--- response_body_like +worker pid: \d+ +worker pid is correct\. +--- no_error_log +[error] From cff86dd7f677e3b856fb7ca1de90746b24eb6411 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sun, 21 Aug 2022 22:56:32 +0800 Subject: [PATCH 648/848] doc: fix the version in README (#2089) --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 7a5ba6b295..f752fc33e8 100644 --- a/README.markdown +++ b/README.markdown @@ -4113,7 +4113,7 @@ Because HTTP request is created after SSL handshake, the `ngx.ctx` created in [ssl_certificate_by_lua*](#ssl_certificate_by_lua), [ssl_session_store_by_lua*](#ssl_session_store_by_lua), [ssl_session_fetch_by_lua*](#ssl_session_fetch_by_lua) and [ssl_client_hello_by_lua*](#ssl_client_hello_by_lua) is not available in the following phases like [rewrite_by_lua*](#rewrite_by_lua). -Since `dev`, the `ngx.ctx` created during a SSL handshake +Since `v0.10.18`, the `ngx.ctx` created during a SSL handshake will be inherited by the requests which share the same TCP connection established by the handshake. Note that overwrite values in `ngx.ctx` in the http request phases (like `rewrite_by_lua*`) will only take affect in the current http request. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 6b2bb99476..34751144bd 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -3387,7 +3387,7 @@ Because HTTP request is created after SSL handshake, the ngx.ctx cr in [[#ssl_certificate_by_lua|ssl_certificate_by_lua*]], [[#ssl_session_store_by_lua|ssl_session_store_by_lua*]], [[#ssl_session_fetch_by_lua|ssl_session_fetch_by_lua*]] and [[#ssl_client_hello_by_lua|ssl_client_hello_by_lua*]] is not available in the following phases like [[#rewrite_by_lua|rewrite_by_lua*]]. -Since dev, the ngx.ctx created during a SSL handshake +Since v0.10.18, the ngx.ctx created during a SSL handshake will be inherited by the requests which share the same TCP connection established by the handshake. Note that overwrite values in ngx.ctx in the http request phases (like `rewrite_by_lua*`) will only take affect in the current http request. From 4add942393fbc487cd540a09f581a032f90be325 Mon Sep 17 00:00:00 2001 From: kingluo Date: Sat, 18 Jun 2022 22:50:55 +0800 Subject: [PATCH 649/848] bugfix: did not wakeup coroutine when worker thread finished. --- README.markdown | 41 +++--------------- doc/HttpLuaModule.wiki | 33 +++------------ src/ngx_http_lua_common.h | 4 ++ src/ngx_http_lua_worker_thread.c | 73 ++++++++++++++++++++++---------- t/166-worker-thread.t | 38 +++++++++++++++++ 5 files changed, 105 insertions(+), 84 deletions(-) diff --git a/README.markdown b/README.markdown index d8f28769d7..d6560185cf 100644 --- a/README.markdown +++ b/README.markdown @@ -1,8 +1,3 @@ - - Name ==== @@ -9194,42 +9189,18 @@ Example1: do md5 calculation. default_type 'text/plain'; content_by_lua_block { - local ok, ret, md5_or_err = ngx.run_worker_thread("testpool", "calc_md5", "md5", ngx.var.arg_str) - if not ok then - ngx.say(ret) - return - end - if not ret then - ngx.say(md5_or_err) - return - end - ngx.say(md5_or_err) + local ok, md5_or_err = ngx.run_worker_thread("testpool", "md5", "md5") + ngx.say(ok, " : ", md5_or_err) } } ``` -`calc_md5.lua` +`md5.lua` ```lua - - local resty_md5 = require "resty.md5" - local resty_str = require "resty.string" - - local function md5(str) - local md5 = resty_md5:new() - if not md5 then - return false, "md5 new error" - end - - local ok = md5:update(str) - if not ok then - return false, "md5 update error" - end - - local digest = md5:final() - return true, resty_str.to_hex(digest) - end - return {md5=md5} +local function md5() + return ngx.md5("hello") +end ``` Example2: write logs into the log file. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 98f3c5879b..7b187bc6f0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -7888,39 +7888,18 @@ location /calc_md5 { default_type 'text/plain'; content_by_lua_block { - local ok, ret, md5_or_err = ngx.run_worker_thread("testpool", "calc_md5", "md5", ngx.var.arg_str) - if not ok then - ngx.say(ret) - return - end - if not ret then - ngx.say(md5_or_err) - return - end - ngx.say(md5_or_err) + local ok, md5_or_err = ngx.run_worker_thread("testpool", "md5", "md5") + ngx.say(ok, " : ", md5_or_err) } } + -calc_md5.lua +md5.lua -local resty_md5 = require "resty.md5" -local resty_str = require "resty.string" - -local function md5(str) - local md5 = resty_md5:new() - if not md5 then - return false, "md5 new error" - end - - local ok = md5:update(str) - if not ok then - return false, "md5 update error" - end - - local digest = md5:final() - return true, resty_str.to_hex(digest) +local function md5() + return ngx.md5("hello") end return {md5=md5} diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index ed88f0a2f5..9f991d1aa0 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -469,6 +469,10 @@ struct ngx_http_lua_co_ctx_s { uint8_t *sr_flags; + unsigned nresults_from_worker_thread; /* number of results + * from worker + * thread callback */ + unsigned nsubreqs; /* number of subrequests of the * current request */ diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 5133fa6e76..5f785ffb22 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -287,6 +287,50 @@ ngx_http_lua_worker_thread_handler(void *data, ngx_log_t *log) } +static ngx_int_t +ngx_http_lua_worker_thread_resume(ngx_http_request_t *r) +{ + lua_State *vm; + ngx_connection_t *c; + ngx_int_t rc; + ngx_uint_t nreqs; + ngx_http_lua_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return NGX_ERROR; + } + + ctx->resume_handler = ngx_http_lua_wev_handler; + + c = r->connection; + vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; + + rc = ngx_http_lua_run_thread(vm, r, ctx, + ctx->cur_co_ctx->nresults_from_worker_thread); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua run thread returned %d", rc); + + if (rc == NGX_AGAIN) { + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (rc == NGX_DONE) { + ngx_http_lua_finalize_request(r, NGX_DONE); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (ctx->entered_content_phase) { + ngx_http_lua_finalize_request(r, rc); + return NGX_DONE; + } + + return rc; +} + + /* executed in nginx event loop */ static void ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) @@ -299,7 +343,6 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) size_t len; const char *str; int i; - int rc; ngx_http_lua_ctx_t *ctx; lua_State *vm; int saved_top; @@ -349,6 +392,7 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) } ctx->cur_co_ctx = worker_thread_ctx->wait_co_ctx; + ctx->cur_co_ctx->nresults_from_worker_thread = nresults; ctx->cur_co_ctx->cleanup = NULL; ngx_http_lua_free_task_ctx(worker_thread_ctx->ctx); @@ -356,30 +400,15 @@ ngx_http_lua_worker_thread_event_handler(ngx_event_t *ev) /* resume the caller coroutine */ - vm = ngx_http_lua_get_lua_vm(r, ctx); - - rc = ngx_http_lua_run_thread(vm, r, ctx, nresults); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua run thread returned %d", rc); - - if (rc == NGX_AGAIN) { - ngx_http_lua_run_posted_threads(c, vm, r, ctx, c->requests); - return; - } + if (ctx->entered_content_phase) { + (void) ngx_http_lua_worker_thread_resume(r); - if (rc == NGX_DONE) { - ngx_http_lua_finalize_request(r, NGX_DONE); - ngx_http_lua_run_posted_threads(c, vm, r, ctx, c->requests); - return; + } else { + ctx->resume_handler = ngx_http_lua_worker_thread_resume; + ngx_http_core_run_phases(r); } - /* rc == NGX_ERROR || rc >= NGX_OK */ - - if (ctx->entered_content_phase) { - ngx_http_lua_finalize_request(r, rc); - return; - } + ngx_http_run_posted_requests(c); return; diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 8b874c55ad..3288f3a15b 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1208,3 +1208,41 @@ return {hello=hello} GET /hello --- response_body_like chop ^true : \d+$ + + + +=== TEST 39: write_log_file +--- main_config + thread_pool testpool threads=100; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /write_log_file { + default_type 'text/plain'; + + content_by_lua_block { + local ok, err = ngx.run_worker_thread("testpool", "write_log_file", "log", ngx.var.arg_str) + if not ok then + ngx.say(ok, " : ", err) + return + end + ngx.say(ok) + } +} +--- user_files +>>> write_log_file.lua +local function log(str) + local file, err = io.open("/tmp/tmp.log", "w") + if not file then + return false, err + end + file:write(str) + file:flush() + file:close() + return true +end +return {log=log} +--- request +GET /write_log_file?str=hello +--- response_body +true From 8d9032298ef542aef058fa02940a6ecd9cf25423 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 15 Sep 2022 10:59:08 +0800 Subject: [PATCH 650/848] bumped the lua version to 0.10.22. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index ff23042250..a1ef1f7cb1 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10021 +#define ngx_http_lua_version 10022 typedef struct { From 4bbb57aa4a6995bd3efaf35c01e826de6f3cdf3a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 30 Sep 2022 14:00:43 +0800 Subject: [PATCH 651/848] doc: add release description for ngx.worker.pids(). (#2104) --- README.markdown | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index f752fc33e8..0c57cdc69f 100644 --- a/README.markdown +++ b/README.markdown @@ -9035,6 +9035,8 @@ ngx.worker.pids This function returns a Lua table for all Nginx worker process ID (PID). Nginx uses channel to send the current worker PID to another worker in the worker process start or restart. So this API can get all current worker PID. +This API was first introduced in the `0.10.23` release. + [Back to TOC](#nginx-api-for-lua) ngx.worker.count From a318d250f547c854ea2b091d0e06372ac0c00fd5 Mon Sep 17 00:00:00 2001 From: Wangchong Zhou Date: Thu, 13 Oct 2022 19:37:28 +0800 Subject: [PATCH 652/848] doc: add detail on explanation of a running timer (#2105) --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 0c57cdc69f..860ec0084a 100644 --- a/README.markdown +++ b/README.markdown @@ -3529,7 +3529,7 @@ lua_max_running_timers Controls the maximum number of "running timers" allowed. -Running timers are those timers whose user callback functions are still running. +Running timers are those timers whose user callback functions are still running or `lightthreads` spawned in callback functions are still running. When exceeding this limit, Nginx will stop running the callbacks of newly expired timers and log an error message "N lua_max_running_timers are not enough" where "N" is the current value of this directive. From 3207da152a1cead67ade7b3d4234681f2c44ba77 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Thu, 20 Oct 2022 23:43:26 +0800 Subject: [PATCH 653/848] chore: add assertion to make linter happy (#2107) Signed-off-by: spacewander --- src/ngx_http_lua_regex.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 03d1a4d320..5c17cfe09c 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -173,6 +173,8 @@ ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, ngx_http_lua_module); + ngx_http_lua_assert(lmcf != NULL); + if (size < NGX_LUA_RE_MIN_JIT_STACK_SIZE) { size = NGX_LUA_RE_MIN_JIT_STACK_SIZE; } @@ -270,6 +272,8 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, ngx_http_lua_module); + ngx_http_lua_assert(lmcf != NULL); + #if (LUA_HAVE_PCRE_JIT) if (flags & NGX_LUA_RE_MODE_JIT) { From b754786bb507ab76f8bfae2fc88aa59a959908a1 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 27 Oct 2022 11:59:40 +0800 Subject: [PATCH 654/848] feature: add new ffi api ngx_http_lua_ffi_msec. (#2108) --- src/ngx_http_lua_time.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/ngx_http_lua_time.c b/src/ngx_http_lua_time.c index 61324f37a2..72704b23a2 100644 --- a/src/ngx_http_lua_time.c +++ b/src/ngx_http_lua_time.c @@ -39,6 +39,13 @@ ngx_http_lua_ffi_time(void) } +long +ngx_http_lua_ffi_monotonic_msec(void) +{ + return (long) ngx_current_msec; +} + + void ngx_http_lua_ffi_update_time(void) { From eb977111bbf47779410e7e2ad267b95faa374fc1 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 27 Oct 2022 20:23:46 +0800 Subject: [PATCH 655/848] bugfix: add a timed recycling child process as a last resort. (#2112) --- src/ngx_http_lua_pipe.c | 148 ++++++++++++++++++++++++---------------- 1 file changed, 90 insertions(+), 58 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index c947d5599f..465d2bff69 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -78,12 +78,15 @@ static void ngx_http_lua_pipe_proc_read_stdout_cleanup(void *data); static void ngx_http_lua_pipe_proc_read_stderr_cleanup(void *data); static void ngx_http_lua_pipe_proc_write_cleanup(void *data); static void ngx_http_lua_pipe_proc_wait_cleanup(void *data); +static void ngx_http_lua_pipe_reap_pids(ngx_event_t *ev); +static void ngx_http_lua_pipe_reap_timer_handler(ngx_event_t *ev); void ngx_http_lua_ffi_pipe_proc_destroy( ngx_http_lua_ffi_pipe_proc_t *proc); static ngx_rbtree_t ngx_http_lua_pipe_rbtree; static ngx_rbtree_node_t ngx_http_lua_pipe_proc_sentinel; +static ngx_event_t ngx_reap_pid_event; #if (NGX_HTTP_LUA_HAVE_SIGNALFD) @@ -162,6 +165,15 @@ ngx_http_lua_pipe_add_signal_handler(ngx_cycle_t *cycle) struct sigaction sa; #endif + ngx_reap_pid_event.handler = ngx_http_lua_pipe_reap_timer_handler; + ngx_reap_pid_event.log = cycle->log; + ngx_reap_pid_event.data = cycle; + ngx_reap_pid_event.cancelable = 1; + + if (!ngx_reap_pid_event.timer_set) { + ngx_add_timer(&ngx_reap_pid_event, 1000); + } + #if (NGX_HTTP_LUA_HAVE_SIGNALFD) if (sigemptyset(&set) != 0) { ngx_log_error(NGX_LOG_ERR, cycle->log, ngx_errno, @@ -353,11 +365,7 @@ static void ngx_http_lua_pipe_sigchld_event_handler(ngx_event_t *ev) { int n; - int status; - ngx_pid_t pid; ngx_connection_t *c = ev->data; - ngx_rbtree_node_t *node; - ngx_http_lua_pipe_node_t *pipe_node; ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ngx_cycle->log, 0, "lua pipe reaping children"); @@ -379,75 +387,99 @@ ngx_http_lua_pipe_sigchld_event_handler(ngx_event_t *ev) break; } - for ( ;; ) { - pid = waitpid(-1, &status, WNOHANG); + ngx_http_lua_pipe_reap_pids(ev); + } +} - if (pid == 0) { - break; - } - if (pid < 0) { - if (ngx_errno != NGX_ECHILD) { - ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, - "lua pipe waitpid failed"); - } +static void +ngx_http_lua_pipe_reap_pids(ngx_event_t *ev) +{ + int status; + ngx_pid_t pid; + ngx_rbtree_node_t *node; + ngx_http_lua_pipe_node_t *pipe_node; - break; - } + for ( ;; ) { + pid = waitpid(-1, &status, WNOHANG); - /* This log is ported from Nginx's signal handler since we override - * or block it in this implementation. */ - ngx_log_error(NGX_LOG_NOTICE, ngx_cycle->log, 0, - "signal %d (SIGCHLD) received from %P", - SIGCHLD, pid); + if (pid == 0) { + break; + } - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua pipe SIGCHLD fd read pid:%P status:%d", pid, - status); - - node = ngx_http_lua_pipe_lookup_pid(pid); - if (node != NULL) { - pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; - if (pipe_node->wait_co_ctx != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "lua pipe resume process:%p waiting for %P", - pipe_node->proc, pid); - - /* - * We need the extra parentheses around the first argument - * of ngx_post_event() just to work around macro issues in - * nginx cores older than 1.7.12 (exclusive). - */ - ngx_post_event((&pipe_node->wait_co_ctx->sleep), - &ngx_posted_events); - } + if (pid < 0) { + if (ngx_errno != NGX_ECHILD) { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, ngx_errno, + "lua pipe waitpid failed"); + } - /* TODO: we should proactively close and free up the pipe after - * the user consume all the data in the pipe. - */ - pipe_node->proc->pipe->dead = 1; + break; + } - if (WIFSIGNALED(status)) { - pipe_node->status = WTERMSIG(status); - pipe_node->reason_code = REASON_SIGNAL_CODE; + /* This log is ported from Nginx's signal handler since we override + * or block it in this implementation. */ + ngx_log_error(NGX_LOG_NOTICE, ngx_cycle->log, 0, + "signal %d (SIGCHLD) received from %P", + SIGCHLD, pid); - } else if (WIFEXITED(status)) { - pipe_node->status = WEXITSTATUS(status); - pipe_node->reason_code = REASON_EXIT_CODE; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe SIGCHLD fd read pid:%P status:%d", pid, + status); + + node = ngx_http_lua_pipe_lookup_pid(pid); + if (node != NULL) { + pipe_node = (ngx_http_lua_pipe_node_t *) &node->color; + if (pipe_node->wait_co_ctx != NULL) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "lua pipe resume process:%p waiting for %P", + pipe_node->proc, pid); + + /* + * We need the extra parentheses around the first argument + * of ngx_post_event() just to work around macro issues in + * nginx cores older than 1.7.12 (exclusive). + */ + ngx_post_event((&pipe_node->wait_co_ctx->sleep), + &ngx_posted_events); + } - } else { - ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, - "lua pipe unknown exit status %d from " - "process %P", status, pid); - pipe_node->status = status; - pipe_node->reason_code = REASON_UNKNOWN_CODE; - } + /* TODO: we should proactively close and free up the pipe after + * the user consume all the data in the pipe. + */ + pipe_node->proc->pipe->dead = 1; + + if (WIFSIGNALED(status)) { + pipe_node->status = WTERMSIG(status); + pipe_node->reason_code = REASON_SIGNAL_CODE; + + } else if (WIFEXITED(status)) { + pipe_node->status = WEXITSTATUS(status); + pipe_node->reason_code = REASON_EXIT_CODE; + + } else { + ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, + "lua pipe unknown exit status %d from " + "process %P", status, pid); + pipe_node->status = status; + pipe_node->reason_code = REASON_UNKNOWN_CODE; } } } } +static void +ngx_http_lua_pipe_reap_timer_handler(ngx_event_t *ev) +{ + ngx_http_lua_pipe_reap_pids(ev); + + if (!ngx_exiting) { + ngx_add_timer(&ngx_reap_pid_event, 1000); + ngx_reap_pid_event.timedout = 0; + } +} + + static ssize_t ngx_http_lua_pipe_fd_read(ngx_connection_t *c, u_char *buf, size_t size) { From a31e9cf3377f2bfe0ddd18ccbf0b62b0baa47f30 Mon Sep 17 00:00:00 2001 From: jinhua luo Date: Tue, 1 Nov 2022 23:07:16 +0800 Subject: [PATCH 656/848] bugfix: avoid buf double free in ngx.print if lua body filter enabled (#2115) Fix #2110 Co-authored-by: lijunlong --- src/ngx_http_lua_bodyfilterby.c | 7 +++++-- t/185-ngx-buf-double-free.t | 25 +++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 2 deletions(-) create mode 100644 t/185-ngx-buf-double-free.t diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 632f5afea3..a8a382ac44 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -299,7 +299,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) out = NULL; ngx_chain_update_chains(r->pool, &ctx->free_bufs, &ctx->filter_busy_bufs, &out, - (ngx_buf_tag_t) &ngx_http_lua_module); + (ngx_buf_tag_t) &ngx_http_lua_body_filter); if (rc != NGX_OK && ctx->filter_busy_bufs != NULL && (r->connection->buffered @@ -378,7 +378,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) ngx_chain_update_chains(r->pool, &ctx->free_bufs, &ctx->filter_busy_bufs, &out, - (ngx_buf_tag_t) &ngx_http_lua_module); + (ngx_buf_tag_t) &ngx_http_lua_body_filter); return rc; } @@ -657,6 +657,7 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, return luaL_error(L, "no memory"); } + cl->buf->tag = (ngx_buf_tag_t) &ngx_http_lua_body_filter; if (type == LUA_TTABLE) { cl->buf->last = ngx_http_lua_copy_str_in_table(L, 3, cl->buf->last); @@ -674,6 +675,8 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, if (cl == NULL) { return luaL_error(L, "no memory"); } + + cl->buf->tag = (ngx_buf_tag_t) &ngx_http_lua_body_filter; } if (last) { diff --git a/t/185-ngx-buf-double-free.t b/t/185-ngx-buf-double-free.t new file mode 100644 index 0000000000..699083db27 --- /dev/null +++ b/t/185-ngx-buf-double-free.t @@ -0,0 +1,25 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket 'no_plan'; + +repeat_each(2); + +run_tests(); + +__DATA__ + +=== TEST 1: one buf was linked to multiple ngx_chain_t nodes +--- config + location /t { + content_by_lua_block { + local str = string.rep(".", 1300) + ngx.print(str) + ngx.flush() + ngx.print("small chunk") + ngx.flush() + } + body_filter_by_lua_block {local dummy=1} + } +--- request +GET /t +--- response_body_like: small chunk From 0499e844305807b1126f5aaf1080bdf3ca9bbd87 Mon Sep 17 00:00:00 2001 From: Marcelo Parada Date: Wed, 2 Nov 2022 16:14:11 +0100 Subject: [PATCH 657/848] doc: add link to pre-defined Nginx variables (#2116) Closes Issue #2111 --- README.markdown | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 860ec0084a..d85c6b6628 100644 --- a/README.markdown +++ b/README.markdown @@ -3838,7 +3838,8 @@ For example: } ``` -That is, Nginx variables cannot be created on-the-fly. +That is, Nginx variables cannot be created on-the-fly. Here is a list of pre-defined +[Nginx variables](http://nginx.org/en/docs/varindex.html). Some special Nginx variables like `$args` and `$limit_rate` can be assigned a value, many others are not, like `$query_string`, `$arg_PARAMETER`, and `$http_NAME`. From 56347d174dc3ca296fc436477a250f28f8bda84f Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 3 Nov 2022 09:15:01 +0800 Subject: [PATCH 658/848] tests: upgrade openssl to 1.1.1s. (#2117) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 9c579ab75d..b651b525cd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -52,7 +52,7 @@ env: - TEST_NGINX_SLEEP=0.006 jobs: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1p OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f services: - memcached From 6e3b1c4fde47039f0ce2ee75fc678dab27944667 Mon Sep 17 00:00:00 2001 From: jinhua luo Date: Sun, 6 Nov 2022 09:39:13 +0800 Subject: [PATCH 659/848] bugfix: define busy_bufs per cosocket (#2119) Fix #2099 --- src/ngx_http_lua_socket_tcp.c | 2 +- src/ngx_http_lua_socket_tcp.h | 2 + t/186-cosocket-busy-bufs.t | 81 +++++++++++++++++++++++++++++++++++ 3 files changed, 84 insertions(+), 1 deletion(-) create mode 100644 t/186-cosocket-busy-bufs.t diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index e40ecbb537..d880377e4b 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -3457,7 +3457,7 @@ ngx_http_lua_socket_send(ngx_http_request_t *r, ngx_chain_update_chains(r->pool, - &ctx->free_bufs, &ctx->busy_bufs, + &ctx->free_bufs, &u->busy_bufs, &u->request_bufs, (ngx_buf_tag_t) &ngx_http_lua_module); diff --git a/src/ngx_http_lua_socket_tcp.h b/src/ngx_http_lua_socket_tcp.h index ee9411bc8d..0cc6641538 100644 --- a/src/ngx_http_lua_socket_tcp.h +++ b/src/ngx_http_lua_socket_tcp.h @@ -125,6 +125,8 @@ struct ngx_http_lua_socket_tcp_upstream_s { int openssl_error_code_ret; #endif + ngx_chain_t *busy_bufs; + unsigned ft_type:16; unsigned no_close:1; unsigned conn_waiting:1; diff --git a/t/186-cosocket-busy-bufs.t b/t/186-cosocket-busy-bufs.t new file mode 100644 index 0000000000..03afe58fc0 --- /dev/null +++ b/t/186-cosocket-busy-bufs.t @@ -0,0 +1,81 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket 'no_plan'; +use Test::Nginx::Socket::Lua::Stream; + +log_level('warn'); +repeat_each(2); +run_tests(); + +__DATA__ + +=== TEST 1: ngx.say and cosocket +--- stream_server_config + content_by_lua_block { + local sock = assert(ngx.req.socket(true)) + sock:settimeout(1000) + while true do + local data = sock:receive(5) + if not data then + return + end + ngx.print(data) + ngx.flush(true) + end + } +--- config + location /test { + content_by_lua_block { + ngx.say("hello") + --ngx.flush(true) + + local sock = ngx.socket.tcp() + local ok, err = sock:connect("127.0.0.1", 1985) + assert(ok) + + local last_duration = 0 + local cnt = 0 + local t1, t2 + local err_cnt = 0 + local ERR_THRESHOLD_MS = 100 + + for i = 1,100000 do + if cnt == 0 then + ngx.update_time() + t1 = ngx.now() + end + + cnt = cnt + 1 + + local sent = sock:send("hello") + local data = sock:receive(5) + assert(data=="hello") + + if cnt == 1000 then + cnt = 0 + ngx.update_time() + t2 = ngx.now() + local duration = (t2 - t1) * 1000 + if last_duration > 0 and (duration - last_duration) > ERR_THRESHOLD_MS then + if err_cnt >= 3 then + ngx.log(ngx.ERR, + "more than ", err_cnt, " times, duration larger than ", + ERR_THRESHOLD_MS, " ms, ", + "last_duration: ", math.floor(duration), " ms") + return ngx.exit(500) + end + err_cnt = err_cnt + 1 + end + last_duration = duration + end + end + + sock:close() + ngx.exit(200) + } + } +--- no_error_log +[error] +--- timeout: 30 +--- request +GET /test From 4045ea8eb72c792de8465dfa6c9bbc638ff96bb8 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 9 Dec 2022 20:29:45 +0800 Subject: [PATCH 660/848] tests: test failed when running in randmonize mode. For example: TEST_NGINX_RANDOMIZE=1 prove -j8 -r -I. t/ --- t/086-init-by.t | 2 +- t/124-init-worker.t | 7 ++++++- t/162-exit-worker.t | 2 +- 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/t/086-init-by.t b/t/086-init-by.t index a998fc53d6..04f8160eb1 100644 --- a/t/086-init-by.t +++ b/t/086-init-by.t @@ -362,6 +362,6 @@ no_such_error_log --- must_die --- error_log eval -qr|init_by_lua_file error: .*lua-nginx-module/t/servroot/html/init.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| +qr|init_by_lua_file error: .*?/t/servroot\w*?/html/init.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| --- no_error_log no_such_error_log diff --git a/t/124-init-worker.t b/t/124-init-worker.t index d06ec9937b..57df4a5b3b 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -842,6 +842,11 @@ lua close the global Lua VM \3 in the cache helper process \d+ lua close the global Lua VM \3 lua close the global Lua VM \3 in the cache helper process \d+ )(?:lua close the global Lua VM [0-9A-F]+ +|lua close the global Lua VM ([0-9A-F]+) +lua close the global Lua VM \4 in the cache helper process \d+ +lua close the global Lua VM \4 in the cache helper process \d+ +lua close the global Lua VM \4 +lua close the global Lua VM \4 )*\z/ --- no_error_log [error] @@ -1022,6 +1027,6 @@ no_such_error_log --- response_body hello world --- error_log eval -qr|init_worker_by_lua_file error: .*lua-nginx-module/t/servroot/html/init.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| +qr|init_worker_by_lua_file error: .*?t/servroot\w*/html/init.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| --- no_error_log no_such_error_log diff --git a/t/162-exit-worker.t b/t/162-exit-worker.t index 60145ee737..7aff2a619b 100644 --- a/t/162-exit-worker.t +++ b/t/162-exit-worker.t @@ -239,4 +239,4 @@ exit_worker_by_lua error: exit_worker_by_lua(nginx.conf:25):4: ')' expected (to --- response_body hello world --- shutdown_error_log eval -qr|exit_worker_by_lua_file error: .*lua-nginx-module/t/servroot/html/exit.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| +qr|exit_worker_by_lua_file error: .*?t/servroot\w*/html/exit.lua:3: '\)' expected \(to close '\(' at line 2\) near 'ngx'| From 3f4005fa879fc2ed329275c47e238bff1f8b890e Mon Sep 17 00:00:00 2001 From: ZongRun <1181591811hzr@gmail.com> Date: Mon, 19 Dec 2022 19:15:38 +0800 Subject: [PATCH 661/848] bugfix: fix receiveuntil rest bytes count. Co-authored-by: timzrhuang --- src/ngx_http_lua_socket_tcp.c | 10 +++-- t/066-socket-receiveuntil.t | 81 ++++++++++++++++++++++++++++++++-- t/084-inclusive-receiveuntil.t | 5 +-- 3 files changed, 85 insertions(+), 11 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index d880377e4b..4ee1103103 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4864,11 +4864,13 @@ ngx_http_lua_socket_read_until(void *data, ssize_t bytes) /* matched */ - dd("adding pending data: %.*s", (int) (old_state + 1 - state), + int pending_bytes = old_state + 1 - state; + + dd("adding pending data: %.*s", (int) pending_bytes, (char *) pat); rc = ngx_http_lua_socket_add_pending_data(r, u, b->pos, i, pat, - old_state + 1 - state, + pending_bytes, old_state); if (rc != NGX_OK) { @@ -4879,14 +4881,14 @@ ngx_http_lua_socket_read_until(void *data, ssize_t bytes) i++; if (u->length) { - if (u->rest <= (size_t) state) { + if (u->rest <= (size_t) pending_bytes) { u->rest = 0; cp->state = state; b->pos += i; return NGX_OK; } else { - u->rest -= state; + u->rest -= pending_bytes; } } diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 89d2abf373..1182a536c1 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -1027,7 +1027,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc") - for i = 1, 7 do + for i = 1, 6 do local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -1055,7 +1055,6 @@ read: hell read: o, w read: orld read: -- -read: failed to read a line: nil [nil] failed to read a line: closed [ ] @@ -1104,7 +1103,7 @@ close: 1 nil local reader = sock:receiveuntil("--abc") - for i = 1, 7 do + for i = 1, 6 do local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -1132,7 +1131,6 @@ read: hell read: o, w read: orld read: -- -read: failed to read a line: nil [nil] failed to read a line: closed [ ] @@ -1329,3 +1327,78 @@ this exposed a memory leak in receiveuntil ok --- no_error_log [error] + + + +=== TEST 20: add pending bytes +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + lua_socket_buffer_size 1; + + content_by_lua ' + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\\r\\n\\r\\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + local reader = sock:receiveuntil("--abc") + + for i = 1, 4 do + local line, err, part = reader(2) + if line then + ngx.say("read: ", line) + + else + ngx.say("failed to read a line: ", err, " [", part, "]") + end + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + '; + } + + location /foo { + echo -- -----abc; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: -- +read: - +failed to read a line: nil [nil] +failed to read a line: closed [ +] +close: 1 nil +} +--- no_error_log +[error] diff --git a/t/084-inclusive-receiveuntil.t b/t/084-inclusive-receiveuntil.t index 66be893d5c..eb5aa1ecdf 100644 --- a/t/084-inclusive-receiveuntil.t +++ b/t/084-inclusive-receiveuntil.t @@ -619,7 +619,7 @@ bad "inclusive" option value type: string local reader = sock:receiveuntil("--abc", { inclusive = true }) - for i = 1, 7 do + for i = 1, 6 do local line, err, part = reader(4) if line then ngx.say("read: ", line) @@ -646,8 +646,7 @@ request sent: 57 read: hell read: o, w read: orld -read: -- -read: --abc +read: ----abc failed to read a line: nil [nil] failed to read a line: closed [ ] From f488965b89238e0bba11c13fdb9b11b4a0f20d67 Mon Sep 17 00:00:00 2001 From: "Yu.Zhu" Date: Wed, 28 Dec 2022 20:12:00 +0800 Subject: [PATCH 662/848] feature: added HTTP 3.0 support to ngx.req.http_version(). --- src/ngx_http_lua_headers.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 8098615dfc..9d03f4ed40 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -84,6 +84,12 @@ ngx_http_lua_ngx_req_http_version(lua_State *L) break; #endif +#ifdef NGX_HTTP_VERSION_30 + case NGX_HTTP_VERSION_30: + lua_pushnumber(L, 3.0); + break; +#endif + default: lua_pushnil(L); break; From 81bc96401edadc4a306152eb0ecfd3e68c4ce429 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 13 Jan 2023 17:25:24 +0800 Subject: [PATCH 663/848] tests: fixed some test cases in t/129-ssl-socket.t t/162-socket-tls-handshake.t t/186-cosocket-busy-bufs.t. --- t/129-ssl-socket.t | 10 +++++----- t/162-socket-tls-handshake.t | 10 +++++----- t/186-cosocket-busy-bufs.t | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index cbbd679468..81ca51ca41 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -39,7 +39,7 @@ run_tests(); __DATA__ -=== TEST 1: www.google.com +=== TEST 1: www.bing.com --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -48,7 +48,7 @@ __DATA__ set $port $TEST_NGINX_MEMCACHED_PORT; content_by_lua ' - -- avoid flushing google in "check leak" testing mode: + -- avoid flushing bing in "check leak" testing mode: local counter = package.loaded.counter if not counter then counter = 1 @@ -62,7 +62,7 @@ __DATA__ do local sock = ngx.socket.tcp() sock:settimeout(2000) - local ok, err = sock:connect("www.google.com", 443) + local ok, err = sock:connect("www.bing.com", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -78,7 +78,7 @@ __DATA__ ngx.say("ssl handshake: ", type(sess)) - local req = "GET / HTTP/1.1\\r\\nHost: www.google.com\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: www.bing.com\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -107,7 +107,7 @@ GET /t --- response_body_like chop \Aconnected: 1 ssl handshake: cdata -sent http request: 59 bytes. +sent http request: 57 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil \z diff --git a/t/162-socket-tls-handshake.t b/t/162-socket-tls-handshake.t index 4474895c53..80fb2f9825 100644 --- a/t/162-socket-tls-handshake.t +++ b/t/162-socket-tls-handshake.t @@ -63,14 +63,14 @@ run_tests(); __DATA__ -=== TEST 1: sanity: www.google.com +=== TEST 1: sanity: www.bing.com --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; location /t { content_by_lua_block { - -- avoid flushing google in "check leak" testing mode: + -- avoid flushing bing in "check leak" testing mode: local counter = package.loaded.counter if not counter then counter = 1 @@ -86,7 +86,7 @@ __DATA__ local sock = ngx.socket.tcp() sock:settimeout(2000) - local ok, err = sock:connect("www.google.com", 443) + local ok, err = sock:connect("www.bing.com", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -102,7 +102,7 @@ __DATA__ ngx.say("ssl handshake: ", type(sess)) - local req = "GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n" + local req = "GET / HTTP/1.1\r\nHost: www.bing.com\r\nConnection: close\r\n\r\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -131,7 +131,7 @@ GET /t --- response_body_like chop \Aconnected: 1 ssl handshake: cdata -sent http request: 59 bytes. +sent http request: 57 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil \z diff --git a/t/186-cosocket-busy-bufs.t b/t/186-cosocket-busy-bufs.t index 03afe58fc0..e50f62d9ee 100644 --- a/t/186-cosocket-busy-bufs.t +++ b/t/186-cosocket-busy-bufs.t @@ -30,7 +30,7 @@ __DATA__ --ngx.flush(true) local sock = ngx.socket.tcp() - local ok, err = sock:connect("127.0.0.1", 1985) + local ok, err = sock:connect("127.0.0.1", ngx.var.server_port + 1) assert(ok) local last_duration = 0 From 47f0a367626eefe912482fd931637d2e3b0d0374 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 13 Jan 2023 17:49:22 +0800 Subject: [PATCH 664/848] bugfix: cosocket did not exit when worker_shutdown_timeout handler is called. --- .travis.yml | 1 + src/ngx_http_lua_socket_tcp.c | 8 ++++++ t/014-bugs.t | 46 ++++++++++++++++++++++++++++++++++- util/nc_server.py | 25 +++++++++++++++++++ 4 files changed, 79 insertions(+), 1 deletion(-) create mode 100644 util/nc_server.py diff --git a/.travis.yml b/.travis.yml index b651b525cd..54f2ab5b17 100644 --- a/.travis.yml +++ b/.travis.yml @@ -135,6 +135,7 @@ script: - rm -fr buildroot - sh util/build.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - nginx -V + - python3 ./util/nc_server.py & - ldd `which nginx`|grep -E 'luajit|ssl|pcre' - export LD_PRELOAD=$PWD/mockeagain/mockeagain.so - export LD_LIBRARY_PATH=$PWD/mockeagain:$LD_LIBRARY_PATH diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 4ee1103103..89fa133d28 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2535,6 +2535,14 @@ ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, "lua tcp socket read data: wait:%d", (int) u->read_waiting); + /* ngx_shutdown_timer_handler will set c->close and c->error on timeout + * when worker_shutdown_timeout is configured. + * The rev->ready is false at that time, so we need to set u->eof. + */ + if (c->close && c->error) { + u->eof = 1; + } + b = &u->buffer; read = 0; diff --git a/t/014-bugs.t b/t/014-bugs.t index 8c93920bc7..bfc86c9e9a 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -8,7 +8,8 @@ log_level('debug'); repeat_each(3); -plan tests => repeat_each() * (blocks() * 2 + 33); +# NB: the shutdown_error_log block is independent from repeat times +plan tests => repeat_each() * (blocks() * 2 + 33) + 1; our $HtmlDir = html_dir; #warn $html_dir; @@ -1239,3 +1240,46 @@ res: true --- must_die --- error_log eval qr/\[emerg\] \d+#\d+: unexpected "A" in/ + + + +=== TEST 47: cosocket does not exit on worker_shutdown_timeout +--- main_config +worker_shutdown_timeout 1; +--- config +location /t { + content_by_lua_block { + local function thread_func() + local sock = ngx.socket.tcp() + local ok, err = sock:connect("127.0.0.1", 65110) + local bytes, err = sock:send("hello") + if bytes ~= 5 then + sock:close() + return ngx.exit(500) + end + + local data, err = sock:receive(20) + local line, err, partial = sock:receive() + if not line then + ngx.log(ngx.ERR, "failed to read a line: ", err) + return + end + + ngx.log(ngx.ERR, "successfully read a line: ", line) + end + + local function timer_func() + ngx.thread.spawn(thread_func) + end + + ngx.timer.at(1, timer_func) + ngx.say("Hello world") + } +} +--- request + GET /t +--- response_body +Hello world +--- shutdown_error_log eval +qr|failed to read a line: closed| +--- timeout: 1.2 diff --git a/util/nc_server.py b/util/nc_server.py new file mode 100644 index 0000000000..48c6f4bf15 --- /dev/null +++ b/util/nc_server.py @@ -0,0 +1,25 @@ +import select, socket +server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +server.setblocking(0) +server.bind(('localhost', 65110)) +server.listen(5) +inputs = [server] + +while inputs: + outputs = [] + readable, writable, exceptional = select.select( + inputs, outputs, inputs) + for s in readable: + if s is server: + connection, client_address = s.accept() + connection.setblocking(0) + inputs.append(connection) + else: + data = s.recv(1024) + if not data: + inputs.remove(s) + s.close() + + for s in exceptional: + inputs.remove(s) + s.close() From 1a485992435771f354c09330c2706d53c8d9cbf4 Mon Sep 17 00:00:00 2001 From: jiahao Date: Thu, 19 Jan 2023 12:40:11 +0800 Subject: [PATCH 665/848] bumped the lua version to 0.10.23. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index a1ef1f7cb1..664698a727 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10022 +#define ngx_http_lua_version 10023 typedef struct { From 992bfc352a25cd7da25f86dc203f0dd6a3cd7926 Mon Sep 17 00:00:00 2001 From: ZongRun <1181591811hzr@gmail.com> Date: Fri, 27 Jan 2023 14:03:46 +0800 Subject: [PATCH 666/848] style: use forward declaration. Co-authored-by: timzrhuang --- src/ngx_http_lua_socket_tcp.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 89fa133d28..dcd27dc174 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -4744,6 +4744,7 @@ ngx_http_lua_socket_read_until(void *data, ssize_t bytes) u_char c; u_char *pat; size_t pat_len; + size_t pending_len; int i; int state; int old_state = 0; /* just to make old @@ -4872,13 +4873,12 @@ ngx_http_lua_socket_read_until(void *data, ssize_t bytes) /* matched */ - int pending_bytes = old_state + 1 - state; + pending_len = old_state + 1 - state; - dd("adding pending data: %.*s", (int) pending_bytes, - (char *) pat); + dd("adding pending data: %.*s", (int) pending_len, (char *) pat); rc = ngx_http_lua_socket_add_pending_data(r, u, b->pos, i, pat, - pending_bytes, + pending_len, old_state); if (rc != NGX_OK) { @@ -4889,14 +4889,14 @@ ngx_http_lua_socket_read_until(void *data, ssize_t bytes) i++; if (u->length) { - if (u->rest <= (size_t) pending_bytes) { + if (u->rest <= pending_len) { u->rest = 0; cp->state = state; b->pos += i; return NGX_OK; } else { - u->rest -= pending_bytes; + u->rest -= pending_len; } } From a6ee03abaaf1dd08e12a0292f60618d15623e0c7 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 6 Feb 2023 11:14:24 +0800 Subject: [PATCH 667/848] bugfix: cleanup of http request will not be callled for the request created by ngx_http_lua_create_fake_request. --- src/ngx_http_lua_pipe.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_pipe.c b/src/ngx_http_lua_pipe.c index 465d2bff69..8c0884bcc2 100644 --- a/src/ngx_http_lua_pipe.c +++ b/src/ngx_http_lua_pipe.c @@ -620,7 +620,7 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_request_t *r, ngx_http_lua_pipe_node_t *pipe_node; struct sigaction sa; ngx_http_lua_pipe_signal_t *sig; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; sigset_t set; pool_size = ngx_align(NGX_MIN_POOL_SIZE + buffer_size * 2, @@ -909,7 +909,7 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_request_t *r, } if (pp->cleanup == NULL) { - cln = ngx_http_lua_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, "no memory") @@ -917,7 +917,7 @@ ngx_http_lua_ffi_pipe_spawn(ngx_http_request_t *r, goto close_in_out_err_fd; } - cln->handler = (ngx_http_cleanup_pt) ngx_http_lua_ffi_pipe_proc_destroy; + cln->handler = (ngx_pool_cleanup_pt) ngx_http_lua_ffi_pipe_proc_destroy; cln->data = proc; pp->cleanup = &cln->handler; pp->r = r; From 5e05fa3adb0d2492ecaaf2cb76498e23765aa6ab Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 6 Feb 2023 15:15:47 +0800 Subject: [PATCH 668/848] tests: cross requests cosocket can only be use one time. tests: cross requests cosocket can only be use one time. cross request cosocket can not be reuese after connect(). --- t/168-tcp-socket-bind.t | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/t/168-tcp-socket-bind.t b/t/168-tcp-socket-bind.t index 6aca00c51e..a2aa50b4e7 100644 --- a/t/168-tcp-socket-bind.t +++ b/t/168-tcp-socket-bind.t @@ -314,12 +314,10 @@ failed to bind: bad address local function tcp() local sock = ngx.socket.tcp() - ---[[ local ok, err = sock:bind("127.0.0.1") if not ok then ngx.log(ngx.ERR, "failed to bind") end - --]] package.loaded.share_sock = sock end @@ -340,17 +338,25 @@ failed to bind: bad address ngx.sleep(0.002) local sock = package.loaded.share_sock - - local ok, err = sock:connect("127.0.0.1", port) - if not ok then - ngx.say("failed to connect: ", err) - return + if sock ~= nil then + package.loaded.share_sock = nil + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + sock:close() + collectgarbage("collect") + else + -- the sock from package.loaded.share_sock is just + -- for the first request after worker init + -- add following code to keep the same result for other request + ngx.say("connected: ", 1) end - - ngx.say("connected: ", ok) - - sock:close() - collectgarbage("collect") } } --- request From 6dbc4e1bd799c254f258687566fd899c322388c7 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 22 Feb 2023 15:20:51 +0800 Subject: [PATCH 669/848] tests: use the specified perl to execute cpanm. (#2157) --- .travis.yml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 54f2ab5b17..a3dc4d2ffd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -64,8 +64,7 @@ before_install: - sudo apt install --only-upgrade ca-certificates - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - - sudo cpanm --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) - + - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi @@ -143,4 +142,4 @@ script: - dig +short myip.opendns.com @resolver1.opendns.com || exit 0 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 - dig +short @$TEST_NGINX_RESOLVER agentzh.org || exit 0 - - prove -I. -Itest-nginx/lib -r t/ + - /usr/bin/env perl $(command -v prove) -I. -Itest-nginx/lib -r t/ From 14be603805c4273852c4076293261416b18b6392 Mon Sep 17 00:00:00 2001 From: Chrono Date: Tue, 28 Feb 2023 16:08:25 +0800 Subject: [PATCH 670/848] doc: setoption is already available now. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index d85c6b6628..a8044cfe7c 100644 --- a/README.markdown +++ b/README.markdown @@ -8200,7 +8200,7 @@ tcpsock:setoption **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** -This function is added for [LuaSocket](http://w3.impa.br/~diego/software/luasocket/tcp.html) API compatibility and does nothing for now. Its functionality is implemented `v0.10.18`. +This function is added for [LuaSocket](http://w3.impa.br/~diego/software/luasocket/tcp.html) API compatibility, its functionality is implemented `v0.10.18`. This feature was first introduced in the `v0.5.0rc1` release. From 6f7a9a3305ae2fa60e6c8472bb18d2a510fa9f58 Mon Sep 17 00:00:00 2001 From: Xiaobo Liu Date: Mon, 27 Dec 2021 08:43:18 +0800 Subject: [PATCH 671/848] feature: add four new apis to manipulate the coctx. ngx_http_lua_co_ctx_t *ngx_http_lua_get_cur_co_ctx(ngx_http_request_t *r); void ngx_http_lua_set_cur_co_ctx(ngx_http_request_t *r, ngx_http_lua_co_ctx_t *coctx); lua_State *ngx_http_lua_get_co_ctx_vm(ngx_http_lua_co_ctx_t *coctx); void ngx_http_lua_co_ctx_resume_helper(ngx_http_lua_co_ctx_t *coctx, int nrets); --- src/api/ngx_http_lua_api.h | 12 ++++ src/ngx_http_lua_api.c | 116 +++++++++++++++++++++++++++++++++++++ src/ngx_http_lua_common.h | 1 + 3 files changed, 129 insertions(+) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 664698a727..ebe971054a 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -22,6 +22,9 @@ #define ngx_http_lua_version 10023 +typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; + + typedef struct { uint8_t type; @@ -56,6 +59,15 @@ ngx_shm_zone_t *ngx_http_lua_find_zone(u_char *name_data, size_t name_len); ngx_shm_zone_t *ngx_http_lua_shared_memory_add(ngx_conf_t *cf, ngx_str_t *name, size_t size, void *tag); +ngx_http_lua_co_ctx_t *ngx_http_lua_get_cur_co_ctx(ngx_http_request_t *r); + +void ngx_http_lua_set_cur_co_ctx(ngx_http_request_t *r, + ngx_http_lua_co_ctx_t *coctx); + +lua_State *ngx_http_lua_get_co_ctx_vm(ngx_http_lua_co_ctx_t *coctx); + +void ngx_http_lua_co_ctx_resume_helper(ngx_http_lua_co_ctx_t *coctx, int nrets); + #endif /* _NGX_HTTP_LUA_API_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index b69d627dce..0f643c77d6 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -213,4 +213,120 @@ ngx_http_lua_shared_memory_init(ngx_shm_zone_t *shm_zone, void *data) return NGX_OK; } + +ngx_http_lua_co_ctx_t * +ngx_http_lua_get_cur_co_ctx(ngx_http_request_t *r) +{ + ngx_http_lua_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + return ctx->cur_co_ctx; +} + + +void +ngx_http_lua_set_cur_co_ctx(ngx_http_request_t *r, ngx_http_lua_co_ctx_t *coctx) +{ + ngx_http_lua_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + coctx->data = r; + + ctx->cur_co_ctx = coctx; +} + + +lua_State * +ngx_http_lua_get_co_ctx_vm(ngx_http_lua_co_ctx_t *coctx) +{ + return coctx->co; +} + + +static ngx_int_t +ngx_http_lua_co_ctx_resume(ngx_http_request_t *r) +{ + lua_State *vm; + ngx_connection_t *c; + ngx_int_t rc; + ngx_uint_t nreqs; + ngx_http_lua_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return NGX_ERROR; + } + + ctx->resume_handler = ngx_http_lua_wev_handler; + + c = r->connection; + vm = ngx_http_lua_get_lua_vm(r, ctx); + nreqs = c->requests; + + rc = ngx_http_lua_run_thread(vm, r, ctx, ctx->cur_co_ctx->nrets); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua run thread returned %d", rc); + + if (rc == NGX_AGAIN) { + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (rc == NGX_DONE) { + ngx_http_lua_finalize_request(r, NGX_DONE); + return ngx_http_lua_run_posted_threads(c, vm, r, ctx, nreqs); + } + + if (ctx->entered_content_phase) { + ngx_http_lua_finalize_request(r, rc); + return NGX_DONE; + } + + return rc; +} + + +void +ngx_http_lua_co_ctx_resume_helper(ngx_http_lua_co_ctx_t *coctx, int nrets) +{ + ngx_connection_t *c; + ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + ngx_http_log_ctx_t *log_ctx; + + r = coctx->data; + c = r->connection; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + if (ctx == NULL) { + return; + } + + if (c->fd != (ngx_socket_t) -1) { /* not a fake connection */ + log_ctx = c->log->data; + log_ctx->current_request = r; + } + + coctx->nrets = nrets; + coctx->cleanup = NULL; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua coctx resume handler: \"%V?%V\"", &r->uri, &r->args); + + ctx->cur_co_ctx = coctx; + + if (ctx->entered_content_phase) { + (void) ngx_http_lua_co_ctx_resume(r); + + } else { + ctx->resume_handler = ngx_http_lua_co_ctx_resume; + ngx_http_core_run_phases(r); + } + + ngx_http_run_posted_requests(c); +} + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index dafa729f8e..d977ee8e3f 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -497,6 +497,7 @@ struct ngx_http_lua_co_ctx_s { unsigned nresults_from_worker_thread; /* number of results * from worker * thread callback */ + unsigned nrets; /* ngx_http_lua_run_thread nrets arg. */ unsigned nsubreqs; /* number of subrequests of the * current request */ From 644b98faa508c4fd6d5870dece10b4bbb8579bd0 Mon Sep 17 00:00:00 2001 From: Xiaobo Liu Date: Fri, 25 Feb 2022 09:21:35 +0800 Subject: [PATCH 672/848] feature: introduced ngx_http_lua_get_lua_http10_buffering. --- src/api/ngx_http_lua_api.h | 2 ++ src/ngx_http_lua_api.c | 12 ++++++++++++ 2 files changed, 14 insertions(+) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index ebe971054a..ba8772f770 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -68,6 +68,8 @@ lua_State *ngx_http_lua_get_co_ctx_vm(ngx_http_lua_co_ctx_t *coctx); void ngx_http_lua_co_ctx_resume_helper(ngx_http_lua_co_ctx_t *coctx, int nrets); +int ngx_http_lua_get_lua_http10_buffering(ngx_http_request_t *r); + #endif /* _NGX_HTTP_LUA_API_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index 0f643c77d6..0d3ec9c988 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -329,4 +329,16 @@ ngx_http_lua_co_ctx_resume_helper(ngx_http_lua_co_ctx_t *coctx, int nrets) ngx_http_run_posted_requests(c); } + +int +ngx_http_lua_get_lua_http10_buffering(ngx_http_request_t *r) +{ + ngx_http_lua_loc_conf_t *llcf; + + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + + return llcf->http10_buffering; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From ce30e2fb0aa86f9c29d42f906192627a373af6e6 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Tue, 7 Mar 2023 15:19:27 +0800 Subject: [PATCH 673/848] Revert "bugfix: doesn't set flags for Darwin arm64. (#2071)" (#2163) This reverts commit 1fd1e83c80612998647ee7ea933d3fdb2e9d9d9b. --- config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config b/config index c4068c852b..0e572c8bea 100644 --- a/config +++ b/config @@ -94,7 +94,7 @@ END case "$NGX_PLATFORM" in Darwin:*) case "$NGX_MACHINE" in - amd64 | arm64 | x86_64 | i386) + amd64 | x86_64 | i386) echo "adding extra linking options needed by LuaJIT on $NGX_MACHINE" luajit_ld_opt="$luajit_ld_opt -pagezero_size 10000 -image_base 100000000" ngx_feature_libs="$ngx_feature_libs -pagezero_size 10000 -image_base 100000000" From 74786af9540841a2a6c11e7352a28c6c198fb05b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 9 Mar 2023 09:03:01 +0800 Subject: [PATCH 674/848] bugfix: windows does not support ngx_http_lua_ffi_worker_pids. --- .travis.yml | 1 + src/ngx_http_lua_socket_tcp.c | 6 +++--- src/ngx_http_lua_worker.c | 10 ++++++++-- 3 files changed, 12 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index a3dc4d2ffd..a547a1d60e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -65,6 +65,7 @@ before_install: - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) + - pyenv global 2.7 install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index dcd27dc174..6d5c3d48c5 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -6483,7 +6483,7 @@ ngx_http_lua_ffi_socket_tcp_getoption(ngx_http_lua_socket_tcp_upstream_t *u, fd = u->peer.connection->fd; - if (fd == (ngx_socket_t) -1) { + if (fd == (int) -1) { *errlen = ngx_snprintf(err, *errlen, "invalid socket fd") - err; return NGX_ERROR; } @@ -6540,7 +6540,7 @@ ngx_http_lua_ffi_socket_tcp_setoption(ngx_http_lua_socket_tcp_upstream_t *u, fd = u->peer.connection->fd; - if (fd == (ngx_socket_t) -1) { + if (fd == (int) -1) { *errlen = ngx_snprintf(err, *errlen, "invalid socket fd") - err; return NGX_ERROR; } @@ -6601,7 +6601,7 @@ ngx_http_lua_ffi_socket_tcp_hack_fd(ngx_http_lua_socket_tcp_upstream_t *u, } rc = u->peer.connection->fd; - if (rc == (ngx_socket_t) -1) { + if (rc == (int) -1) { *errlen = ngx_snprintf(err, *errlen, "invalid socket fd") - err; return -1; } diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 52ec34a844..6870495a1b 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -8,7 +8,10 @@ #define DDEBUG 0 #endif #include "ddebug.h" + +#if !(NGX_WIN32) #include +#endif #define NGX_PROCESS_PRIVILEGED_AGENT 99 @@ -21,13 +24,15 @@ ngx_http_lua_ffi_worker_pid(void) } +#if !(NGX_WIN32) int ngx_http_lua_ffi_worker_pids(int *pids, size_t *pids_len) { - ngx_int_t i, n; + size_t n; + ngx_int_t i; n = 0; - for (i = 0; i < NGX_MAX_PROCESSES; i++) { + for (i = 0; n < *pids_len && i < NGX_MAX_PROCESSES; i++) { if (i != ngx_process_slot && ngx_processes[i].pid == 0) { break; } @@ -49,6 +54,7 @@ ngx_http_lua_ffi_worker_pids(int *pids, size_t *pids_len) return NGX_OK; } +#endif int From 1953052a026eeec5dae9af96cc189d8725d72a5e Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 9 Mar 2023 16:32:17 +0800 Subject: [PATCH 675/848] bugfix: fixed warning of signed and unsigned. Signed-off-by: lijunlong --- src/ngx_http_lua_headers.c | 6 ++++-- src/ngx_http_lua_shdict.c | 11 +++++++---- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index 9d03f4ed40..a2ce6e1452 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -1250,8 +1250,10 @@ ngx_http_lua_ngx_raw_header_cleanup(void *data) int ngx_http_lua_ffi_set_resp_header_macos(ngx_http_lua_set_resp_header_params_t *p) { - return ngx_http_lua_ffi_set_resp_header(p->r, p->key_data, p->key_len, - p->is_nil, p->sval, p->sval_len, + return ngx_http_lua_ffi_set_resp_header(p->r, (const u_char *) p->key_data, + p->key_len, p->is_nil, + (const u_char *) p->sval, + p->sval_len, p->mvals, p->mvals_len, p->override, p->errmsg); } diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 27f3b100f3..2e0b044065 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2096,7 +2096,8 @@ ngx_http_lua_ffi_shdict_free_space(ngx_shm_zone_t *zone) int ngx_http_lua_ffi_shdict_get_macos(ngx_http_lua_shdict_get_params_t *p) { - return ngx_http_lua_ffi_shdict_get(p->zone, p->key, p->key_len, + return ngx_http_lua_ffi_shdict_get(p->zone, + (u_char *) p->key, p->key_len, p->value_type, p->str_value_buf, p->str_value_len, p->num_value, p->user_flags, p->get_stale, @@ -2107,8 +2108,10 @@ ngx_http_lua_ffi_shdict_get_macos(ngx_http_lua_shdict_get_params_t *p) int ngx_http_lua_ffi_shdict_store_macos(ngx_http_lua_shdict_store_params_t *p) { - return ngx_http_lua_ffi_shdict_store(p->zone, p->op, p->key, p->key_len, - p->value_type, p->str_value_buf, + return ngx_http_lua_ffi_shdict_store(p->zone, p->op, + (u_char *) p->key, p->key_len, + p->value_type, + (u_char *) p->str_value_buf, p->str_value_len, p->num_value, p->exptime, p->user_flags, p->errmsg, p->forcible); @@ -2118,7 +2121,7 @@ ngx_http_lua_ffi_shdict_store_macos(ngx_http_lua_shdict_store_params_t *p) int ngx_http_lua_ffi_shdict_incr_macos(ngx_http_lua_shdict_incr_params_t *p) { - return ngx_http_lua_ffi_shdict_incr(p->zone, p->key, p->key_len, + return ngx_http_lua_ffi_shdict_incr(p->zone, (u_char *) p->key, p->key_len, p->num_value, p->errmsg, p->has_init, p->init, p->init_ttl, p->forcible); From 9bee7e7890987d2641e2e16c4685d99d2b769948 Mon Sep 17 00:00:00 2001 From: "user.tax" Date: Sat, 11 Mar 2023 00:11:27 +0800 Subject: [PATCH 676/848] bugfix : comparison of integer expressions of different signedness. Signed-off-by: lijunlong --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 6d5c3d48c5..9391569529 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -1744,7 +1744,7 @@ ngx_http_lua_ffi_socket_tcp_sslhandshake(ngx_http_request_t *r, /* read rest of the chain */ - for (i = 1; i < sk_X509_num(chain); i++) { + for (i = 1; i < (ngx_int_t) sk_X509_num(chain); i++) { x509 = sk_X509_value(chain, i); if (x509 == NULL) { ERR_clear_error(); From ec6df3249a554ec67200857cb9de58814ca48dc5 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 20 Mar 2023 17:39:53 +0800 Subject: [PATCH 677/848] bugfix: windows also need to support ngx_http_lua_ffi_worker_pids. --- README.markdown | 2 +- src/ngx_http_lua_worker.c | 5 ++-- t/122-worker-2.t | 49 +++++++++++++++++++++++++++++++++ t/122-worker-3.t | 58 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 111 insertions(+), 3 deletions(-) create mode 100644 t/122-worker-2.t create mode 100644 t/122-worker-3.t diff --git a/README.markdown b/README.markdown index a8044cfe7c..c0d0c079d4 100644 --- a/README.markdown +++ b/README.markdown @@ -9034,7 +9034,7 @@ ngx.worker.pids **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, exit_worker_by_lua** -This function returns a Lua table for all Nginx worker process ID (PID). Nginx uses channel to send the current worker PID to another worker in the worker process start or restart. So this API can get all current worker PID. +This function returns a Lua table for all Nginx worker process IDs (PIDs). Nginx uses channel to send the current worker PID to another worker in the worker process start or restart. So this API can get all current worker PIDs. Windows does not have this API. This API was first introduced in the `0.10.23` release. diff --git a/src/ngx_http_lua_worker.c b/src/ngx_http_lua_worker.c index 6870495a1b..5fdf0b4760 100644 --- a/src/ngx_http_lua_worker.c +++ b/src/ngx_http_lua_worker.c @@ -37,11 +37,12 @@ ngx_http_lua_ffi_worker_pids(int *pids, size_t *pids_len) break; } - if (i == ngx_process_slot && ngx_processes[i].pid == 0) { + /* The current process */ + if (i == ngx_process_slot) { pids[n++] = ngx_pid; } - if (ngx_processes[i].pid > 0) { + if (ngx_processes[i].channel[0] > 0 && ngx_processes[i].pid > 0) { pids[n++] = ngx_processes[i].pid; } } diff --git a/t/122-worker-2.t b/t/122-worker-2.t new file mode 100644 index 0000000000..c4ad2aea2a --- /dev/null +++ b/t/122-worker-2.t @@ -0,0 +1,49 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +master_on(); +workers(4); +#log_level('warn'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3); + +#no_diff(); +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: get worker pids with multiple worker +--- config + location /lua { + content_by_lua_block { + local pids, err = ngx.worker.pids() + if err ~= nil then + return + end + local pid = ngx.worker.pid() + ngx.say("worker pid: ", pid) + local count = ngx.worker.count() + ngx.say("worker count: ", count) + ngx.say("worker pids count: ", #pids) + for i = 1, count do + if pids[i] == pid then + ngx.say("worker pid is correct.") + return + end + end + } + } +--- request +GET /lua +--- response_body_like +worker pid: \d+ +worker count: 4 +worker pids count: 4 +worker pid is correct\. +--- no_error_log +[error] diff --git a/t/122-worker-3.t b/t/122-worker-3.t new file mode 100644 index 0000000000..050486e532 --- /dev/null +++ b/t/122-worker-3.t @@ -0,0 +1,58 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_CHECK_LEAK}) { + $SkipReason = "unavailable for the hup tests"; + + } else { + $ENV{TEST_NGINX_USE_HUP} = 1; + undef $ENV{TEST_NGINX_USE_STAP}; + } +} + +use Test::Nginx::Socket::Lua 'no_plan'; + +#worker_connections(1014); +master_on(); +workers(4); +#log_level('warn'); + +repeat_each(2); + +#no_diff(); +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: get worker pids with multiple worker +--- config + location /lua { + content_by_lua_block { + local pids, err = ngx.worker.pids() + if err ~= nil then + return + end + local pid = ngx.worker.pid() + ngx.say("worker pid: ", pid) + local count = ngx.worker.count() + ngx.say("worker count: ", count) + ngx.say("worker pids count: ", #pids) + for i = 1, count do + if pids[i] == pid then + ngx.say("worker pid is correct.") + return + end + end + } + } +--- request +GET /lua +--- response_body_like +worker pid: \d+ +worker count: 4 +worker pids count: 4 +worker pid is correct\. +--- no_error_log +[error] From 68acad14e4a8f42e31d4a4bb5ed44d6f5b55fc1c Mon Sep 17 00:00:00 2001 From: jiahao Date: Tue, 21 Mar 2023 21:51:04 +0800 Subject: [PATCH 678/848] bumped the lua version to 0.10.24. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index ba8772f770..018184990d 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10023 +#define ngx_http_lua_version 10024 typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; From 39941d4c9a8f94191084800471f6433c757a90df Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 3 Apr 2023 08:44:36 +0800 Subject: [PATCH 679/848] tests: add two more test cases for body_filter_by_lua*. --- .travis.yml | 4 +- t/000--init.t | 2 +- t/082-body-filter-2.t | 154 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 158 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index a547a1d60e..1a051b7120 100644 --- a/.travis.yml +++ b/.travis.yml @@ -91,12 +91,13 @@ install: - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql + - git clone https://github.com/spacewander/lua-resty-rsa.git ../lua-resty-rsa - git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 before_script: - - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;' + - mysql -uroot -e "create database ngx_test; CREATE USER 'ngx_test'@'%' IDENTIFIED BY 'ngx_test'; grant all on ngx_test.* to 'ngx_test'@'%'; flush privileges;" script: - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH @@ -106,6 +107,7 @@ script: - sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP - sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP - sudo ip route add prohibit 0.0.0.1/32 + - sudo sysctl -w kernel.pid_max=10000 - cd luajit2/ - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1) - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1) diff --git a/t/000--init.t b/t/000--init.t index 0dd08fe188..0016e14486 100644 --- a/t/000--init.t +++ b/t/000--init.t @@ -11,7 +11,7 @@ $ENV{TEST_NGINX_MYSQL_PORT} ||= 3306; our $http_config = <<'_EOC_'; # lua-resty-string is required for lua-resty-mysql - lua_package_path "../lua-resty-mysql/lib/?.lua;../lua-resty-string/lib/?.lua;;"; + lua_package_path "../lua-resty-rsa/lib/?.lua;../lua-resty-mysql/lib/?.lua;../lua-resty-string/lib/?.lua;;"; _EOC_ no_shuffle(); diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t index eafe7cf26a..99ed447f56 100644 --- a/t/082-body-filter-2.t +++ b/t/082-body-filter-2.t @@ -60,3 +60,157 @@ waiting body filter busy buffer to be sent lua say response has busy bufs --- no_error_log [error] + + + +=== TEST 2: arg[1] not change and say long string +--- config + location /t { + header_filter_by_lua_block {ngx.header.content_length = nil} + body_filter_by_lua_block { + local function anyting_not_change_arg1() + return + end + anyting_not_change_arg1() + } + content_by_lua_block { + for i = 1, 100 do + ngx.say("12345678901234567890") + end + } + } +--- request +GET /t +--- response_body eval +("12345678901234567890\n" x 100) +--- no_error_log +[error] +[alert] +[crit] + + + +=== TEST 3: arg[1] not change and chunked_transfer_encoding off +--- config + location /t { + header_filter_by_lua_block {ngx.header.content_length = nil} + body_filter_by_lua_block { + local function anyting_not_change_arg1() + return + end + anyting_not_change_arg1() + } + chunked_transfer_encoding off; + content_by_lua_block { + for i = 1, 100 do + ngx.say("12345678901234567890123456789012345678901234567890".."_"..tostring(i/3)) + end + } + } +--- request +GET /t +--- response_body +12345678901234567890123456789012345678901234567890_0.33333333333333 +12345678901234567890123456789012345678901234567890_0.66666666666667 +12345678901234567890123456789012345678901234567890_1 +12345678901234567890123456789012345678901234567890_1.3333333333333 +12345678901234567890123456789012345678901234567890_1.6666666666667 +12345678901234567890123456789012345678901234567890_2 +12345678901234567890123456789012345678901234567890_2.3333333333333 +12345678901234567890123456789012345678901234567890_2.6666666666667 +12345678901234567890123456789012345678901234567890_3 +12345678901234567890123456789012345678901234567890_3.3333333333333 +12345678901234567890123456789012345678901234567890_3.6666666666667 +12345678901234567890123456789012345678901234567890_4 +12345678901234567890123456789012345678901234567890_4.3333333333333 +12345678901234567890123456789012345678901234567890_4.6666666666667 +12345678901234567890123456789012345678901234567890_5 +12345678901234567890123456789012345678901234567890_5.3333333333333 +12345678901234567890123456789012345678901234567890_5.6666666666667 +12345678901234567890123456789012345678901234567890_6 +12345678901234567890123456789012345678901234567890_6.3333333333333 +12345678901234567890123456789012345678901234567890_6.6666666666667 +12345678901234567890123456789012345678901234567890_7 +12345678901234567890123456789012345678901234567890_7.3333333333333 +12345678901234567890123456789012345678901234567890_7.6666666666667 +12345678901234567890123456789012345678901234567890_8 +12345678901234567890123456789012345678901234567890_8.3333333333333 +12345678901234567890123456789012345678901234567890_8.6666666666667 +12345678901234567890123456789012345678901234567890_9 +12345678901234567890123456789012345678901234567890_9.3333333333333 +12345678901234567890123456789012345678901234567890_9.6666666666667 +12345678901234567890123456789012345678901234567890_10 +12345678901234567890123456789012345678901234567890_10.333333333333 +12345678901234567890123456789012345678901234567890_10.666666666667 +12345678901234567890123456789012345678901234567890_11 +12345678901234567890123456789012345678901234567890_11.333333333333 +12345678901234567890123456789012345678901234567890_11.666666666667 +12345678901234567890123456789012345678901234567890_12 +12345678901234567890123456789012345678901234567890_12.333333333333 +12345678901234567890123456789012345678901234567890_12.666666666667 +12345678901234567890123456789012345678901234567890_13 +12345678901234567890123456789012345678901234567890_13.333333333333 +12345678901234567890123456789012345678901234567890_13.666666666667 +12345678901234567890123456789012345678901234567890_14 +12345678901234567890123456789012345678901234567890_14.333333333333 +12345678901234567890123456789012345678901234567890_14.666666666667 +12345678901234567890123456789012345678901234567890_15 +12345678901234567890123456789012345678901234567890_15.333333333333 +12345678901234567890123456789012345678901234567890_15.666666666667 +12345678901234567890123456789012345678901234567890_16 +12345678901234567890123456789012345678901234567890_16.333333333333 +12345678901234567890123456789012345678901234567890_16.666666666667 +12345678901234567890123456789012345678901234567890_17 +12345678901234567890123456789012345678901234567890_17.333333333333 +12345678901234567890123456789012345678901234567890_17.666666666667 +12345678901234567890123456789012345678901234567890_18 +12345678901234567890123456789012345678901234567890_18.333333333333 +12345678901234567890123456789012345678901234567890_18.666666666667 +12345678901234567890123456789012345678901234567890_19 +12345678901234567890123456789012345678901234567890_19.333333333333 +12345678901234567890123456789012345678901234567890_19.666666666667 +12345678901234567890123456789012345678901234567890_20 +12345678901234567890123456789012345678901234567890_20.333333333333 +12345678901234567890123456789012345678901234567890_20.666666666667 +12345678901234567890123456789012345678901234567890_21 +12345678901234567890123456789012345678901234567890_21.333333333333 +12345678901234567890123456789012345678901234567890_21.666666666667 +12345678901234567890123456789012345678901234567890_22 +12345678901234567890123456789012345678901234567890_22.333333333333 +12345678901234567890123456789012345678901234567890_22.666666666667 +12345678901234567890123456789012345678901234567890_23 +12345678901234567890123456789012345678901234567890_23.333333333333 +12345678901234567890123456789012345678901234567890_23.666666666667 +12345678901234567890123456789012345678901234567890_24 +12345678901234567890123456789012345678901234567890_24.333333333333 +12345678901234567890123456789012345678901234567890_24.666666666667 +12345678901234567890123456789012345678901234567890_25 +12345678901234567890123456789012345678901234567890_25.333333333333 +12345678901234567890123456789012345678901234567890_25.666666666667 +12345678901234567890123456789012345678901234567890_26 +12345678901234567890123456789012345678901234567890_26.333333333333 +12345678901234567890123456789012345678901234567890_26.666666666667 +12345678901234567890123456789012345678901234567890_27 +12345678901234567890123456789012345678901234567890_27.333333333333 +12345678901234567890123456789012345678901234567890_27.666666666667 +12345678901234567890123456789012345678901234567890_28 +12345678901234567890123456789012345678901234567890_28.333333333333 +12345678901234567890123456789012345678901234567890_28.666666666667 +12345678901234567890123456789012345678901234567890_29 +12345678901234567890123456789012345678901234567890_29.333333333333 +12345678901234567890123456789012345678901234567890_29.666666666667 +12345678901234567890123456789012345678901234567890_30 +12345678901234567890123456789012345678901234567890_30.333333333333 +12345678901234567890123456789012345678901234567890_30.666666666667 +12345678901234567890123456789012345678901234567890_31 +12345678901234567890123456789012345678901234567890_31.333333333333 +12345678901234567890123456789012345678901234567890_31.666666666667 +12345678901234567890123456789012345678901234567890_32 +12345678901234567890123456789012345678901234567890_32.333333333333 +12345678901234567890123456789012345678901234567890_32.666666666667 +12345678901234567890123456789012345678901234567890_33 +12345678901234567890123456789012345678901234567890_33.333333333333 +--- no_error_log +[error] +[alert] +[crit] From 16de4bea620b6281f948b0d65341e31b2c90d597 Mon Sep 17 00:00:00 2001 From: A compound of Fe and O Date: Fri, 7 Apr 2023 01:47:27 +0100 Subject: [PATCH 680/848] bugfix: Fix handling of new list elements. Signed-off by: lijunlong --- .travis.yml | 1 + src/ngx_http_lua_control.c | 3 +++ src/ngx_http_lua_headers_in.c | 3 +++ src/ngx_http_lua_headers_out.c | 3 +++ src/ngx_http_lua_subrequest.c | 3 +++ 5 files changed, 13 insertions(+) diff --git a/.travis.yml b/.travis.yml index 1a051b7120..64d27d8529 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,6 +50,7 @@ env: - LD_LIBRARY_PATH=$LUAJIT_LIB:$LD_LIBRARY_PATH - DRIZZLE_VER=2011.07.21 - TEST_NGINX_SLEEP=0.006 + - MALLOC_PERTURB_=9 jobs: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index 99460456c1..d7e427385d 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -280,6 +280,9 @@ ngx_http_lua_ngx_redirect(lua_State *L) h->value.len = len; h->value.data = uri; +#if defined(nginx_version) && nginx_version >= 1023000 + h->next = NULL; +#endif ngx_str_set(&h->key, "Location"); r->headers_out.status = rc; diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 4405481408..960e2f85f6 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -280,6 +280,9 @@ ngx_http_set_header_helper(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, h->key = hv->key; h->value = *value; +#if defined(nginx_version) && nginx_version >= 1023000 + h->next = NULL; +#endif h->lowcase_key = ngx_pnalloc(r->pool, h->key.len); if (h->lowcase_key == NULL) { diff --git a/src/ngx_http_lua_headers_out.c b/src/ngx_http_lua_headers_out.c index 571723d9a6..c51146a3fc 100644 --- a/src/ngx_http_lua_headers_out.c +++ b/src/ngx_http_lua_headers_out.c @@ -229,6 +229,9 @@ ngx_http_set_header_helper(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, h->key = hv->key; h->value = *value; +#if defined(nginx_version) && nginx_version >= 1023000 + h->next = NULL; +#endif h->lowcase_key = ngx_pnalloc(r->pool, h->key.len); if (h->lowcase_key == NULL) { diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index f6638c2991..c09207417e 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1667,6 +1667,9 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, clh->hash = ngx_http_lua_content_length_hash; clh->key = ngx_http_lua_content_length_header_key; +#if defined(nginx_version) && nginx_version >= 1023000 + clh->next = NULL; +#endif clh->lowcase_key = ngx_pnalloc(sr->pool, clh->key.len); if (clh->lowcase_key == NULL) { return NGX_ERROR; From bf8c91d1116fe767ed4f0b1d9b0bbc4055faf31f Mon Sep 17 00:00:00 2001 From: willmafh Date: Wed, 10 May 2023 22:30:39 +0800 Subject: [PATCH 681/848] doc: fixed typo in error messages. --- src/ngx_http_lua_socket_tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 9391569529..119994d010 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -3261,7 +3261,7 @@ ngx_http_lua_socket_tcp_settimeouts(lua_State *L) n = lua_gettop(L); if (n != 4) { - return luaL_error(L, "ngx.socket settimeout: expecting 4 arguments " + return luaL_error(L, "ngx.socket settimeouts: expecting 4 arguments " "(including the object) but seen %d", lua_gettop(L)); } From 0090f3faf66e6870b3974e983ae4b47459c96fa3 Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Thu, 11 May 2023 19:27:13 +0800 Subject: [PATCH 682/848] bugfix: disable http2 in body read due to http2 stream processing bug. --- README.markdown | 4 ++++ src/ngx_http_lua_accessby.c | 5 +++++ src/ngx_http_lua_contentby.c | 5 +++++ src/ngx_http_lua_req_body.c | 14 ++++++++++++++ src/ngx_http_lua_rewriteby.c | 5 +++++ src/ngx_http_lua_server_rewriteby.c | 7 ++++++- t/023-rewrite/request_body.t | 20 ++++++++++++++++++++ t/024-access/request_body.t | 20 ++++++++++++++++++++ t/044-req-body.t | 19 ++++++++++++++++++- 9 files changed, 97 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index c0d0c079d4..448fce86f5 100644 --- a/README.markdown +++ b/README.markdown @@ -2721,6 +2721,8 @@ lua_need_request_body **phase:** *depends on usage* +Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. + Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code. To read the request body data within the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable, @@ -5384,6 +5386,8 @@ Reads the client request body synchronously without blocking the Nginx event loo local args = ngx.req.get_post_args() ``` +Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. + If the request body is already read previously by turning on [lua_need_request_body](#lua_need_request_body) or by using other modules, then this function does not run and returns immediately. If the request body has already been explicitly discarded, either by the [ngx.req.discard_body](#ngxreqdiscard_body) function or other modules, this function does not run and returns immediately. diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 58c251443e..8814b924bb 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -136,7 +136,12 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) return NGX_DONE; } +/* http2 read body may break http2 stream process */ +#if (NGX_HTTP_V2) + if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { +#else if (llcf->force_read_body && !ctx->read_body_done) { +#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 76e6a074da..65067cd6a1 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -195,7 +195,12 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) return rc; } +/* http2 read body may break http2 stream process */ +#if (NGX_HTTP_V2) + if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { +#else if (llcf->force_read_body && !ctx->read_body_done) { +#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 935fafc8a4..3c452b3051 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -85,6 +85,13 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) return luaL_error(L, "request object not found"); } +/* http2 read body may break http2 stream process */ +#if (NGX_HTTP_V2) + if (r->main->stream) { + return luaL_error(L, "http2 requests are not supported yet"); + } +#endif + r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; @@ -312,6 +319,13 @@ ngx_http_lua_ngx_req_get_body_file(lua_State *L) return luaL_error(L, "request object not found"); } +/* http2 read body may break http2 stream process */ +#if (NGX_HTTP_V2) + if (r->main->stream) { + return luaL_error(L, "http2 requests are not supported yet"); + } +#endif + ngx_http_lua_check_fake_request(L, r); if (r->request_body == NULL || r->request_body->temp_file == NULL) { diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index d1eabeccdd..2e3065364e 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -140,7 +140,12 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) return NGX_DONE; } +/* http2 read body may break http2 stream process */ +#if (NGX_HTTP_V2) + if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { +#else if (llcf->force_read_body && !ctx->read_body_done) { +#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c index 22bb69e8c8..cc6192d925 100644 --- a/src/ngx_http_lua_server_rewriteby.c +++ b/src/ngx_http_lua_server_rewriteby.c @@ -102,8 +102,13 @@ ngx_http_lua_server_rewrite_handler(ngx_http_request_t *r) return NGX_DONE; } - /* TODO: lscf do not have force_read_body */ +/* TODO: lscf do not have force_read_body + * http2 read body may break http2 stream process */ +#if (NGX_HTTP_V2) + if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { +#else if (llcf->force_read_body && !ctx->read_body_done) { +#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/t/023-rewrite/request_body.t b/t/023-rewrite/request_body.t index b867d3a82c..f9af17236f 100644 --- a/t/023-rewrite/request_body.t +++ b/t/023-rewrite/request_body.t @@ -170,3 +170,23 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug + + + +=== TEST 9: test HTTP2 reading request body was disabled +--- config + location /echo_body { + lua_need_request_body on; + rewrite_by_lua_block { + ngx.print(ngx.var.request_body or "nil") + } + content_by_lua 'ngx.exit(ngx.OK)'; + } +--- http2 +--- request eval +"POST /echo_body +hello\x00\x01\x02 +world\x03\x04\xff" +--- response_body eval +"nil" +--- no_error_log diff --git a/t/024-access/request_body.t b/t/024-access/request_body.t index fa03195272..4970e37aef 100644 --- a/t/024-access/request_body.t +++ b/t/024-access/request_body.t @@ -170,3 +170,23 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug + + + +=== TEST 9: test HTTP2 reading request body was disabled +--- config + location /echo_body { + lua_need_request_body on; + access_by_lua_block { + ngx.print(ngx.var.request_body or "nil") + } + content_by_lua 'ngx.exit(ngx.OK)'; + } +--- http2 +--- request eval +"POST /echo_body +hello\x00\x01\x02 +world\x03\x04\xff" +--- response_body eval +"nil" +--- no_error_log diff --git a/t/044-req-body.t b/t/044-req-body.t index 2ef7e1d29e..d8a62c7dbe 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -7,7 +7,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 52 ); +plan tests => repeat_each() * (blocks() * 4 + 50 ); #no_diff(); no_long_string(); @@ -1744,3 +1744,20 @@ content length: 5 --- no_error_log [error] [alert] + + + +=== TEST 53: HTTP2 read buffered body was discarded +--- config + location = /test { + content_by_lua_block { + local err = pcall(ngx.req.read_body()) + ngx.say(err) + } + } +--- http2 +--- request +POST /test +hello, world +--- error_code: 500 +--- error_log: http2 requests are not supported yet From c91fb13a8c168ce39a8dd434fc324c3b5f2cf985 Mon Sep 17 00:00:00 2001 From: willmafh Date: Thu, 11 May 2023 19:31:18 +0800 Subject: [PATCH 683/848] doc: fix typo in the comment. --- src/ngx_http_lua_clfactory.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_clfactory.c b/src/ngx_http_lua_clfactory.c index c13fd14dc1..9eab164661 100644 --- a/src/ngx_http_lua_clfactory.c +++ b/src/ngx_http_lua_clfactory.c @@ -91,7 +91,7 @@ * | Int | At which line this function is defined * | [linedefined] | * --------------------- - * | Int | At while line this function definition ended + * | Int | At which line this function definition ended * | [lastlinedefined] | * --------------------- * | Char | Number of upvalues referenced by this function @@ -128,7 +128,7 @@ * | Vector | Debug lineinfo vector * | [lineinfo] | Empty vector here if debug info is stripped * --------------------- - * | Int | Number of local variable in this function + * | Int | Number of local variables in this function * | [sizelocvars] | 0 if debug info is stripped * --------------------- * | String | ------------------------------------ From 4bf1b0b8d7614ba094a3e3d3619f1f52a83b03f7 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 14 Apr 2023 14:03:29 +0800 Subject: [PATCH 684/848] change: lua-ssl-protocols: Disabled SSLv3 and enable TLSv1.3 by default. --- README.markdown | 3 ++- src/ngx_http_lua_module.c | 4 ++-- t/025-codecache.t | 1 + t/129-ssl-socket.t | 2 ++ t/143-ssl-session-fetch.t | 19 +++++++++++++++++++ 5 files changed, 26 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index 448fce86f5..19fea19e64 100644 --- a/README.markdown +++ b/README.markdown @@ -3323,13 +3323,14 @@ lua_ssl_protocols **syntax:** *lua_ssl_protocols \[SSLv2\] \[SSLv3\] \[TLSv1\] [TLSv1.1] [TLSv1.2] [TLSv1.3]* -**default:** *lua_ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2* +**default:** *lua_ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3* **context:** *http, server, location* Enables the specified protocols for requests to a SSL/TLS server in the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. The support for the `TLSv1.3` parameter requires version `v0.10.12` *and* OpenSSL 1.1.1. +From version v0.10.25, the default value change from `SSLV3 TLSv1 TLSv1.1 TLSv1.2` to `TLSv1 TLSv1.1 TLSv1.2 TLSv1.3`. This directive was first introduced in the `v0.9.11` release. diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 16f442464c..8ea93e2a0e 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1465,9 +1465,9 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) #if (NGX_HTTP_SSL) ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, - (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3 + (NGX_CONF_BITMASK_SET |NGX_SSL_TLSv1|NGX_SSL_TLSv1_1 - |NGX_SSL_TLSv1_2)); + |NGX_SSL_TLSv1_2|NGX_SSL_TLSv1_3)); ngx_conf_merge_str_value(conf->ssl_ciphers, prev->ssl_ciphers, "DEFAULT"); diff --git a/t/025-codecache.t b/t/025-codecache.t index ebe4f61b7b..2a3eeb3b08 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -1720,6 +1720,7 @@ grep me: b } --- config lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location = /proxy { proxy_pass http://backend; diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 81ca51ca41..7e48d59bca 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1034,6 +1034,8 @@ SSL reused session --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; + location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 8c5174dd94..74f2e6e1bb 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -41,6 +41,7 @@ __DATA__ server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -141,6 +142,7 @@ ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -225,6 +227,7 @@ qr/elapsed in ssl fetch session by lua: 0.(?:09|1[01])\d+,/, server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -326,6 +329,7 @@ qr/my timer run!/s server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -407,6 +411,7 @@ qr/received memc reply: OK/s resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -488,6 +493,7 @@ should never reached here resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -570,6 +576,7 @@ should never reached here resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -650,6 +657,7 @@ should never reached here server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; lua_ssl_verify_depth 3; location /t { @@ -733,6 +741,7 @@ should never reached here server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; lua_ssl_verify_depth 3; location /t { @@ -814,6 +823,7 @@ should never reached here server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -900,6 +910,7 @@ qr/get_phase: ssl_session_fetch/s server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -982,6 +993,7 @@ ssl store session by lua is running! --- config server_tokens off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1070,6 +1082,7 @@ qr/\S+:\d+: ssl fetch sess by lua is running!/s server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1170,6 +1183,7 @@ qr/ssl_session_fetch_by_lua\(nginx.conf:\d+\):1: ssl fetch sess by lua is runnin --- config server_tokens off; lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { content_by_lua_block { @@ -1249,6 +1263,7 @@ GET /t --- config server_tokens off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { content_by_lua_block { @@ -1406,6 +1421,7 @@ ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is run --- config server_tokens off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -1501,6 +1517,7 @@ qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, --- config server_tokens off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { content_by_lua_block { @@ -1596,6 +1613,7 @@ close: 1 nil --- config server_tokens off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { content_by_lua_block { @@ -1696,6 +1714,7 @@ uthread: failed to kill: already waited or killed --- config server_tokens off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; location /t { content_by_lua_block { From 0c3853e124e6b06c47c3ad501087f74bf70f5c43 Mon Sep 17 00:00:00 2001 From: ZongRun <1181591811hzr@gmail.com> Date: Mon, 22 May 2023 23:22:26 +0800 Subject: [PATCH 685/848] bugfix: mixed recv api call causes unexpected result. Co-authored-by: timzrhuang --- src/ngx_http_lua_socket_tcp.c | 113 ++++++- t/066-socket-receiveuntil.t | 591 ++++++++++++++++++++++++++++++++++ 2 files changed, 697 insertions(+), 7 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 119994d010..e8a70a19b3 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -75,6 +75,8 @@ static void ngx_http_lua_socket_dummy_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); static int ngx_http_lua_socket_tcp_receive_helper(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); +static void ngx_http_lua_socket_tcp_read_prepare(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, void *data, lua_State *L); static ngx_int_t ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u); static int ngx_http_lua_socket_tcp_receive_retval_handler(ngx_http_request_t *r, @@ -2163,8 +2165,6 @@ ngx_http_lua_socket_tcp_receive_helper(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx; - u->input_filter_ctx = u; - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (u->bufs_in == NULL) { @@ -2193,6 +2193,8 @@ ngx_http_lua_socket_tcp_receive_helper(ngx_http_request_t *r, u->read_waiting = 0; u->read_co_ctx = NULL; + ngx_http_lua_socket_tcp_read_prepare(r, u, u, L); + rc = ngx_http_lua_socket_tcp_read(r, u); if (rc == NGX_ERROR) { @@ -2514,6 +2516,87 @@ ngx_http_lua_socket_read_any(void *data, ssize_t bytes) } +static void +ngx_http_lua_socket_tcp_read_prepare(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, void *data, lua_State *L) +{ + ngx_http_lua_ctx_t *ctx; + ngx_chain_t *new_cl; + ngx_buf_t *b; + off_t size; + + ngx_http_lua_socket_compiled_pattern_t *cp; + + /* input_filter_ctx doesn't change, no need recovering */ + if (u->input_filter_ctx == data) { + return; + } + + /* last input_filter_ctx is null or upstream, no data pending */ + if (u->input_filter_ctx == NULL || u->input_filter_ctx == u) { + u->input_filter_ctx = data; + return; + } + + /* compiled pattern may be with data pending */ + + cp = u->input_filter_ctx; + u->input_filter_ctx = data; + + cp->upstream = NULL; + + /* no data pending */ + if (cp->state <= 0) { + return; + } + + b = &u->buffer; + + if (b->pos - b->start >= cp->state) { + dd("pending data in one buffer"); + + b->pos -= cp->state; + + u->buf_in->buf->pos = b->pos; + u->buf_in->buf->last = b->pos; + + /* reset dfa state for future matching */ + cp->state = 0; + return; + } + + dd("pending data in multiple buffers"); + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + size = ngx_buf_size(b); + + new_cl = + ngx_http_lua_chain_get_free_buf(r->connection->log, r->pool, + &ctx->free_recv_bufs, + cp->state + size); + + if (new_cl == NULL) { + luaL_error(L, "no memory"); + return; + } + + ngx_memcpy(b, new_cl->buf, sizeof(ngx_buf_t)); + + b->last = ngx_copy(b->last, cp->pattern.data, cp->state); + b->last = ngx_copy(b->last, u->buf_in->buf->pos, size); + + u->buf_in->next = ctx->free_recv_bufs; + ctx->free_recv_bufs = u->buf_in; + + u->bufs_in = new_cl; + u->buf_in = new_cl; + + /* reset dfa state for future matching */ + cp->state = 0; +} + + static ngx_int_t ngx_http_lua_socket_tcp_read(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u) @@ -4243,6 +4326,11 @@ ngx_http_lua_socket_tcp_finalize(ngx_http_request_t *r, ngx_http_lua_socket_tcp_finalize_read_part(r, u); ngx_http_lua_socket_tcp_finalize_write_part(r, u); + if (u->input_filter_ctx != NULL && u->input_filter_ctx != u) { + ((ngx_http_lua_socket_compiled_pattern_t *) + u->input_filter_ctx)->upstream = NULL; + } + if (u->raw_downstream || u->body_downstream) { u->peer.connection = NULL; return; @@ -4559,8 +4647,6 @@ ngx_http_lua_socket_receiveuntil_iterator(lua_State *L) (u_char *) lua_tolstring(L, lua_upvalueindex(2), &cp->pattern.len); - u->input_filter_ctx = cp; - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (u->bufs_in == NULL) { @@ -4587,6 +4673,8 @@ ngx_http_lua_socket_receiveuntil_iterator(lua_State *L) u->read_waiting = 0; u->read_co_ctx = NULL; + ngx_http_lua_socket_tcp_read_prepare(r, u, cp, L); + rc = ngx_http_lua_socket_tcp_read(r, u); if (rc == NGX_ERROR) { @@ -4915,13 +5003,24 @@ ngx_http_lua_socket_cleanup_compiled_pattern(lua_State *L) { ngx_http_lua_socket_compiled_pattern_t *cp; - ngx_http_lua_dfa_edge_t *edge, *p; - unsigned i; + ngx_http_lua_socket_tcp_upstream_t *u; + ngx_http_lua_dfa_edge_t *edge, *p; + unsigned i; dd("cleanup compiled pattern"); cp = lua_touserdata(L, 1); - if (cp == NULL || cp->recovering == NULL) { + if (cp == NULL) { + return 0; + } + + u = cp->upstream; + if (u != NULL) { + ngx_http_lua_socket_tcp_read_prepare(u->request, u, NULL, L); + u->input_filter_ctx = NULL; + } + + if (cp->recovering == NULL) { return 0; } diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 1182a536c1..7fac6bfd0f 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -1402,3 +1402,594 @@ close: 1 nil } --- no_error_log [error] + + + +=== TEST 21: ambiguous boundary patterns (--abc), mixed by other reading calls consume boundary +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\r\n\r\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + local reader = sock:receiveuntil("--abc") + + for i = 1, 5 do + local line, err, part = reader(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + break + + else + ngx.say("read: ", line) + end + + local data, err, part = sock:receive(1) + if not data then + ngx.say("failed to read a byte: ", err, " [", part, "]") + break + + else + ngx.say("read one byte: ", data) + end + end + + local line, err, part = reader(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + echo -- ----abc----abc-; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: -- +read one byte: - +read: -a +read one byte: b +read: c- +read one byte: - +read: +read one byte: - +failed to read a line: nil [nil] +failed to read a line: closed [ +] +close: 1 nil +} +--- no_error_log +[error] + + + +=== TEST 22: ambiguous boundary patterns (--abc), mixed by other reading calls (including receiveuntil) consume boundary +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\r\n\r\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + local reader1 = sock:receiveuntil("--abc") + local reader2 = sock:receiveuntil("-ab") + + local line, err, part = reader1(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local data, err, part = sock:receive(1) + if not data then + ngx.say("failed to read a byte: ", err, " [", part, "]") + + else + ngx.say("read one byte: ", data) + end + + local line, err, part = reader1(1) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local line, err, part = reader2(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local line, err, part = reader1() + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local line, err, part = reader1() + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + echo -- ------abd----abc; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: -- +read one byte: - +read: - +read: - +read: d-- +failed to read a line: closed [ +] +close: 1 nil +} +--- no_error_log +[error] + + + +=== TEST 23: ambiguous boundary patterns (--abc), mixed by other reading calls consume boundary, small buffer +--- config + lua_socket_buffer_size 3; + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\r\n\r\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + local reader = sock:receiveuntil("--abc") + + for i = 1, 5 do + local line, err, part = reader(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + break + + else + ngx.say("read: ", line) + end + + local data, err, part = sock:receive(1) + if not data then + ngx.say("failed to read a byte: ", err, " [", part, "]") + break + + else + ngx.say("read one byte: ", data) + end + end + + local line, err, part = reader(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + echo -- ----abc----abc-; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: -- +read one byte: - +read: -a +read one byte: b +read: c- +read one byte: - +read: +read one byte: - +failed to read a line: nil [nil] +failed to read a line: closed [ +] +close: 1 nil +} +--- no_error_log +[error] + + + +=== TEST 24: ambiguous boundary patterns (--abc), mixed by other reading calls (including receiveuntil) consume boundary, small buffer +--- config + lua_socket_buffer_size 3; + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\r\n\r\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + local reader1 = sock:receiveuntil("--abc") + local reader2 = sock:receiveuntil("-ab") + + local line, err, part = reader1(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local data, err, part = sock:receive(1) + if not data then + ngx.say("failed to read a byte: ", err, " [", part, "]") + + else + ngx.say("read one byte: ", data) + end + + local line, err, part = reader1(1) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local line, err, part = reader2(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local line, err, part = reader1() + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + local line, err, part = reader1() + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + echo -- ------abd----abc; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: -- +read one byte: - +read: - +read: - +read: d-- +failed to read a line: closed [ +] +close: 1 nil +} +--- no_error_log +[error] + + + +=== TEST 25: ambiguous boundary patterns (ab1ab2), ends half way +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\r\n\r\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + if true then + local reader = sock:receiveuntil("ab1ab2") + + local line, err, part = reader(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + end + + collectgarbage("collect") + + local data, err, part = sock:receive(3) + if not data then + ngx.say("failed to read three bytes: ", err, " [", part, "]") + + else + ngx.say("read three bytes: ", data) + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + echo -- ab1ab1; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: ab1 +read three bytes: ab1 +close: 1 nil +} +--- no_error_log +[error] + + + +=== TEST 26: ambiguous boundary patterns (ab1ab2), ends half way, small buffer +--- config + lua_socket_buffer_size 3; + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + -- collectgarbage("collect") + + local sock = ngx.socket.tcp() + local port = ngx.var.port + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + ngx.say("request sent: ", bytes) + + local read_headers = sock:receiveuntil("\r\n\r\n") + local headers, err, part = read_headers() + if not headers then + ngx.say("failed to read headers: ", err, " [", part, "]") + end + + if true then + local reader = sock:receiveuntil("ab1ab2") + + local line, err, part = reader(2) + if not line then + ngx.say("failed to read a line: ", err, " [", part, "]") + + else + ngx.say("read: ", line) + end + end + + collectgarbage("collect") + + local data, err, part = sock:receive(3) + if not data then + ngx.say("failed to read three bytes: ", err, " [", part, "]") + + else + ngx.say("read three bytes: ", data) + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + echo -- ab1ab1; + more_clear_headers Date; + } +--- request +GET /t + +--- response_body eval +qq{connected: 1 +request sent: 57 +read: ab1 +read three bytes: ab1 +close: 1 nil +} +--- no_error_log +[error] From c9a0808c89219f74e4d20cef44a322a1bbe72df3 Mon Sep 17 00:00:00 2001 From: willmafh Date: Mon, 22 May 2023 23:24:47 +0800 Subject: [PATCH 686/848] bugfix: when value type is SHDICT_TNUMBER, then it should get the number field instead of the bool field. --- src/ngx_http_lua_shdict.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 2e0b044065..31bdcdb6a5 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -654,7 +654,7 @@ ngx_http_lua_shared_dict_get(ngx_shm_zone_t *zone, u_char *key_data, return NGX_ERROR; } - ngx_memcpy(&value->value.b, data, len); + ngx_memcpy(&value->value.n, data, len); break; case SHDICT_TBOOLEAN: From 64631297fbfe21a0d260b538285b8f53b067e1a5 Mon Sep 17 00:00:00 2001 From: mafh Date: Wed, 31 May 2023 13:40:22 +0800 Subject: [PATCH 687/848] bugfix: ssl_client_hello_by_lua generating chunk cache key and chunk name --- src/ngx_http_lua_ssl_client_helloby.c | 8 ++++--- t/166-ssl-client-hello.t | 30 +++++++++++++-------------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 10500d89e3..1e68e07f4c 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -150,14 +150,16 @@ ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_client_hello_src.len = ngx_strlen(name); } else { - cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, - value[1].len); + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, + "ssl_client_hello_by_lua", + value[1].data, + value[1].len); if (cache_key == NULL) { return NGX_CONF_ERROR; } chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_client_hello_by_lua", - sizeof("ssl_client_helloo_by_lua")- 1, + sizeof("ssl_client_hello_by_lua")- 1, &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index da021300af..850a0d6550 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -117,18 +117,18 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] ---- grep_error_log eval: qr/ssl_client_hello_by_lua:.*?,|\bssl client hello: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log eval: qr/ssl_client_hello_by_lua\(.*?,|\bssl client hello: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval # Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) # before call ssl callback function $Test::Nginx::Util::NginxVersion >= 1.017009 ? qr/reusable connection: 0 ssl client hello: connection reusable: 0 -ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ +ssl_client_hello_by_lua\(nginx.conf:\d+\):1: ssl client hello by lua is running!,/ : qr /reusable connection: 1 ssl client hello: connection reusable: 1 reusable connection: 0 -ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ +ssl_client_hello_by_lua\(nginx.conf:\d+\):1: ssl client hello by lua is running!,/ @@ -789,7 +789,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'runtime error: ssl_client_hello_by_lua:2: bad bad bad', +'runtime error: ssl_client_hello_by_lua(nginx.conf:28):2: bad bad bad', 'lua_client_hello_by_lua: handler return value: 500, client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, qr/context: ssl_client_hello_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, @@ -861,7 +861,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'runtime error: ssl_client_hello_by_lua:3: bad bad bad', +'runtime error: ssl_client_hello_by_lua(nginx.conf:28):3: bad bad bad', 'lua_client_hello_by_lua: client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, ] @@ -1048,7 +1048,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua ssl server name: "test.com"', -'ssl_client_hello_by_lua:1: API disabled in the context of ssl_client_hello_by_lua*', +'ssl_client_hello_by_lua(nginx.conf:28):1: API disabled in the context of ssl_client_hello_by_lua*', qr/\[info\] .*?callback failed/, ] @@ -1479,7 +1479,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl_client_hello_by_lua(nginx.conf:25):1: ssl client hello by lua is running! --- no_error_log [error] @@ -1574,7 +1574,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua on the server level is running! +ssl_client_hello_by_lua(nginx.conf:31):1: ssl client hello by lua on the server level is running! --- no_error_log [error] @@ -1657,7 +1657,7 @@ received: foo close: 1 nil --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! [error] [alert] @@ -1738,7 +1738,7 @@ received: foo close: 1 nil --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! [error] [alert] @@ -1830,7 +1830,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl_client_hello_by_lua(nginx.conf:28):1: ssl client hello by lua is running! --- no_error_log [error] @@ -1915,7 +1915,7 @@ close: 1 nil --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! [error] [alert] @@ -2018,10 +2018,10 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua in server1 is running! +ssl_client_hello_by_lua(nginx.conf:29):1: ssl client hello by lua in server1 is running! --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua in server2 is running! +ssl client hello by lua in server2 is running! [error] [alert] @@ -2121,7 +2121,7 @@ qr/\[error\] .*? send\(\) failed/, ] --- no_error_log [alert] -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! From 28aced45f6ab5dc801ad77e368f38af2e26f15b6 Mon Sep 17 00:00:00 2001 From: willmafh Date: Sun, 11 Jun 2023 23:44:25 +0800 Subject: [PATCH 688/848] bugfix: ngx_http_lua_socket_tcp_bind is also allowed in ssl_session_fetch_by and ssl_client_hello_by. --- src/ngx_http_lua_socket_tcp.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index e8a70a19b3..5be85dcd46 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -865,7 +865,9 @@ ngx_http_lua_socket_tcp_bind(lua_State *L) | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT); + | NGX_HTTP_LUA_CONTEXT_SSL_CERT + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO); luaL_checktype(L, 1, LUA_TTABLE); From 992cae739d39eb8d4f270f2a68ee6d8a1d09b5f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=BF=9E=E5=BF=97=E9=9B=B7?= Date: Sun, 11 Jun 2023 23:48:43 +0800 Subject: [PATCH 689/848] doc: fixed typo. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 19fea19e64..695e6b0a4b 100644 --- a/README.markdown +++ b/README.markdown @@ -9358,7 +9358,7 @@ The second argument `module_name` specifies the lua module name to execute in th The third argument `func_name` specifies the function field in the module table as the second argument. -The type of `arg`s must be one of type below: +The type of `args` must be one of type below: * boolean * number From 592a91c1ab4079c0942f1d2b2e804d3c6737be72 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20Pracha=C5=99?= Date: Sun, 11 Jun 2023 17:58:45 +0200 Subject: [PATCH 690/848] optimize: Optimized use of SSL contexts. Analogous to https://hg.nginx.org/nginx/rev/9d98d524bd02 --- src/ngx_http_lua_module.c | 60 +++++++++++++++++++++++++++++++++++---- 1 file changed, 55 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 8ea93e2a0e..869cc91a82 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -48,6 +48,8 @@ static char *ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, static ngx_int_t ngx_http_lua_init(ngx_conf_t *cf); static char *ngx_http_lua_lowat_check(ngx_conf_t *cf, void *post, void *data); #if (NGX_HTTP_SSL) +static ngx_int_t ngx_http_lua_merge_ssl(ngx_conf_t *cf, + ngx_http_lua_loc_conf_t *conf, ngx_http_lua_loc_conf_t *prev); static ngx_int_t ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf); #if (nginx_version >= 1019004) @@ -1464,6 +1466,10 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) #if (NGX_HTTP_SSL) + if (ngx_http_lua_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, (NGX_CONF_BITMASK_SET |NGX_SSL_TLSv1|NGX_SSL_TLSv1_1 @@ -1528,16 +1534,60 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) #if (NGX_HTTP_SSL) static ngx_int_t -ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) +ngx_http_lua_merge_ssl(ngx_conf_t *cf, + ngx_http_lua_loc_conf_t *conf, ngx_http_lua_loc_conf_t *prev) { - ngx_pool_cleanup_t *cln; + ngx_uint_t preserve; - llcf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (llcf->ssl == NULL) { + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL +#if (nginx_version >= 1019004) + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR +#endif + ) + { + if (prev->ssl) { + conf->ssl = prev->ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->ssl == NULL) { return NGX_ERROR; } - llcf->ssl->log = cf->log; + conf->ssl->log = cf->log; + + /* + * special handling to preserve conf->ssl_* in the "http" section + * to inherit it to all servers + */ + + if (preserve) { + prev->ssl = conf->ssl; + } + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) +{ + ngx_pool_cleanup_t *cln; + + if (llcf->ssl->ctx) { + return NGX_OK; + } if (ngx_ssl_create(llcf->ssl, llcf->ssl_protocols, NULL) != NGX_OK) { return NGX_ERROR; From bba3db08e784c5b9a40c015fc8d9b5240cc26525 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 19 Jun 2023 09:27:51 +0800 Subject: [PATCH 691/848] bugfix: used after free when encountering invalid http IF-Match header. --- src/ngx_http_lua_accessby.c | 6 ++-- src/ngx_http_lua_bodyfilterby.c | 4 +-- src/ngx_http_lua_common.h | 2 +- src/ngx_http_lua_contentby.c | 4 +-- src/ngx_http_lua_directive.c | 4 +-- src/ngx_http_lua_headerfilterby.c | 4 +-- src/ngx_http_lua_rewriteby.c | 6 ++-- src/ngx_http_lua_server_rewriteby.c | 4 +-- src/ngx_http_lua_socket_udp.c | 4 +-- src/ngx_http_lua_ssl_certby.c | 4 +-- src/ngx_http_lua_ssl_client_helloby.c | 4 +-- src/ngx_http_lua_ssl_session_fetchby.c | 4 +-- src/ngx_http_lua_timer.c | 4 +-- t/014-bugs.t | 49 ++++++++++++++++++++++++++ 14 files changed, 76 insertions(+), 27 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 8814b924bb..fa6810d9ed 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -245,7 +245,7 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -296,9 +296,9 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) /* }}} */ - /* {{{ register request cleanup hooks */ + /* {{{ register nginx pool cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index a8a382ac44..78e3b5c2d6 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -234,7 +234,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) ngx_http_lua_ctx_t *ctx; ngx_int_t rc; uint16_t old_context; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_chain_t *out; ngx_chain_t *cl, *ln; ngx_http_lua_main_conf_t *lmcf; @@ -314,7 +314,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) } if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index d977ee8e3f..843504549a 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -587,7 +587,7 @@ typedef struct ngx_http_lua_ctx_s { ngx_chain_t *filter_in_bufs; /* for the body filter */ ngx_chain_t *filter_busy_bufs; /* for the body filter */ - ngx_http_cleanup_pt *cleanup; + ngx_pool_cleanup_pt *cleanup; ngx_http_cleanup_t *free_cleanup; /* free list of cleanup records */ diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 65067cd6a1..d1c3bc9fd9 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -29,7 +29,7 @@ ngx_http_lua_content_by_chunk(lua_State *L, ngx_http_request_t *r) lua_State *co; ngx_event_t *rev; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -83,7 +83,7 @@ ngx_http_lua_content_by_chunk(lua_State *L, ngx_http_request_t *r) /* {{{ register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index f1623056b5..f42aae9d51 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1436,7 +1436,7 @@ ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) { lua_State *L; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { @@ -1451,7 +1451,7 @@ ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) } if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index f2fe165bd5..71553558bb 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -231,7 +231,7 @@ ngx_http_lua_header_filter(ngx_http_request_t *r) ngx_http_lua_loc_conf_t *llcf; ngx_http_lua_ctx_t *ctx; ngx_int_t rc; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; uint16_t old_context; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, @@ -260,7 +260,7 @@ ngx_http_lua_header_filter(ngx_http_request_t *r) } if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 2e3065364e..c56bba5d7f 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -246,7 +246,7 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -296,9 +296,9 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) /* }}} */ - /* {{{ register request cleanup hooks */ + /* {{{ register nginx pool cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c index cc6192d925..997262eeb7 100644 --- a/src/ngx_http_lua_server_rewriteby.c +++ b/src/ngx_http_lua_server_rewriteby.c @@ -204,7 +204,7 @@ ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -256,7 +256,7 @@ ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) /* {{{ register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index cbb6621b47..10285f6d0a 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -591,7 +591,7 @@ ngx_http_lua_socket_resolve_retval_handler(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx; ngx_connection_t *c; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_upstream_resolved_t *ur; ngx_int_t rc; ngx_http_lua_udp_connection_t *uc; @@ -625,7 +625,7 @@ ngx_http_lua_socket_resolve_retval_handler(ngx_http_request_t *r, } if (u->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_ERROR; lua_pushnil(L); diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 0499bfb355..b8e70ddefa 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -453,7 +453,7 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_int_t rc; lua_State *co; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -507,7 +507,7 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) /* register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { rc = NGX_ERROR; ngx_http_lua_finalize_request(r, rc); diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 1e68e07f4c..a1dac61887 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -449,7 +449,7 @@ ngx_http_lua_ssl_client_hello_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_int_t rc; lua_State *co; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -503,7 +503,7 @@ ngx_http_lua_ssl_client_hello_by_chunk(lua_State *L, ngx_http_request_t *r) /* register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { rc = NGX_ERROR; ngx_http_lua_finalize_request(r, rc); diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 66e9848550..ebce63ce28 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -477,7 +477,7 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_int_t rc; lua_State *co; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -531,7 +531,7 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) /* register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { rc = NGX_ERROR; ngx_http_lua_finalize_request(r, rc); diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 060cab543a..11d29349a5 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -519,7 +519,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) ngx_connection_t *c = NULL; ngx_http_request_t *r = NULL; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_pool_cleanup_t *pcln; ngx_http_lua_timer_ctx_t tctx; @@ -620,7 +620,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) L = ngx_http_lua_get_lua_vm(r, ctx); - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { errmsg = "could not add request cleanup"; goto failed; diff --git a/t/014-bugs.t b/t/014-bugs.t index bfc86c9e9a..877aecfdb5 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -1283,3 +1283,52 @@ Hello world --- shutdown_error_log eval qr|failed to read a line: closed| --- timeout: 1.2 + + + +=== TEST 48: nginx crashes when encountering an illegal http if header +crash with ngx.send_headers() +--- main_config +--- config +error_page 412 /my_error_handler_412; + +location /t { + rewrite_by_lua_block { + ngx.send_headers() + -- ngx.print() -- this also triggers the bug + } +} +location = /my_error_handler_412 { + return 412 "hello"; +} +--- request + GET /t +--- more_headers +If-Match: 1 +--- error_code: 412 +--- response_body eval +qr/\Ahello\z/ + + + +=== TEST 49: nginx crashes when encountering an illegal http if header +crash with ngx.print() +--- main_config +--- config +error_page 412 /my_error_handler_412; + +location /t { + rewrite_by_lua_block { + ngx.print() + } +} +location = /my_error_handler_412 { + return 412 "hello"; +} +--- request + GET /t +--- more_headers +If-Match: 1 +--- error_code: 412 +--- response_body eval +qr/\Ahello\z/ From 6e9948dcdac5618da47223bec1f328e417ab0fcb Mon Sep 17 00:00:00 2001 From: mafh Date: Wed, 31 May 2023 13:40:22 +0800 Subject: [PATCH 692/848] bugfix: ssl_client_hello_by_lua generating chunk cache key and chunk name --- src/ngx_http_lua_ssl_client_helloby.c | 8 ++++--- t/166-ssl-client-hello.t | 30 +++++++++++++-------------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 10500d89e3..1e68e07f4c 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -150,14 +150,16 @@ ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->srv.ssl_client_hello_src.len = ngx_strlen(name); } else { - cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, - value[1].len); + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, + "ssl_client_hello_by_lua", + value[1].data, + value[1].len); if (cache_key == NULL) { return NGX_CONF_ERROR; } chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_client_hello_by_lua", - sizeof("ssl_client_helloo_by_lua")- 1, + sizeof("ssl_client_hello_by_lua")- 1, &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index da021300af..850a0d6550 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -117,18 +117,18 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] ---- grep_error_log eval: qr/ssl_client_hello_by_lua:.*?,|\bssl client hello: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log eval: qr/ssl_client_hello_by_lua\(.*?,|\bssl client hello: connection reusable: \d+|\breusable connection: \d+/ --- grep_error_log_out eval # Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) # before call ssl callback function $Test::Nginx::Util::NginxVersion >= 1.017009 ? qr/reusable connection: 0 ssl client hello: connection reusable: 0 -ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ +ssl_client_hello_by_lua\(nginx.conf:\d+\):1: ssl client hello by lua is running!,/ : qr /reusable connection: 1 ssl client hello: connection reusable: 1 reusable connection: 0 -ssl_client_hello_by_lua:1: ssl client hello by lua is running!,/ +ssl_client_hello_by_lua\(nginx.conf:\d+\):1: ssl client hello by lua is running!,/ @@ -789,7 +789,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'runtime error: ssl_client_hello_by_lua:2: bad bad bad', +'runtime error: ssl_client_hello_by_lua(nginx.conf:28):2: bad bad bad', 'lua_client_hello_by_lua: handler return value: 500, client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, qr/context: ssl_client_hello_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, @@ -861,7 +861,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'runtime error: ssl_client_hello_by_lua:3: bad bad bad', +'runtime error: ssl_client_hello_by_lua(nginx.conf:28):3: bad bad bad', 'lua_client_hello_by_lua: client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, ] @@ -1048,7 +1048,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua ssl server name: "test.com"', -'ssl_client_hello_by_lua:1: API disabled in the context of ssl_client_hello_by_lua*', +'ssl_client_hello_by_lua(nginx.conf:28):1: API disabled in the context of ssl_client_hello_by_lua*', qr/\[info\] .*?callback failed/, ] @@ -1479,7 +1479,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl_client_hello_by_lua(nginx.conf:25):1: ssl client hello by lua is running! --- no_error_log [error] @@ -1574,7 +1574,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua on the server level is running! +ssl_client_hello_by_lua(nginx.conf:31):1: ssl client hello by lua on the server level is running! --- no_error_log [error] @@ -1657,7 +1657,7 @@ received: foo close: 1 nil --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! [error] [alert] @@ -1738,7 +1738,7 @@ received: foo close: 1 nil --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! [error] [alert] @@ -1830,7 +1830,7 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl_client_hello_by_lua(nginx.conf:28):1: ssl client hello by lua is running! --- no_error_log [error] @@ -1915,7 +1915,7 @@ close: 1 nil --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! [error] [alert] @@ -2018,10 +2018,10 @@ close: 1 nil --- error_log lua ssl server name: "test.com" -ssl_client_hello_by_lua:1: ssl client hello by lua in server1 is running! +ssl_client_hello_by_lua(nginx.conf:29):1: ssl client hello by lua in server1 is running! --- no_error_log -ssl_client_hello_by_lua:1: ssl client hello by lua in server2 is running! +ssl client hello by lua in server2 is running! [error] [alert] @@ -2121,7 +2121,7 @@ qr/\[error\] .*? send\(\) failed/, ] --- no_error_log [alert] -ssl_client_hello_by_lua:1: ssl client hello by lua is running! +ssl client hello by lua is running! From 6d0a2c184703604d6245288ca9cf50addb95b178 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 19 Jun 2023 09:27:51 +0800 Subject: [PATCH 693/848] bugfix: used after free when encountering invalid http IF-Match header. --- src/ngx_http_lua_accessby.c | 6 ++-- src/ngx_http_lua_bodyfilterby.c | 4 +-- src/ngx_http_lua_common.h | 2 +- src/ngx_http_lua_contentby.c | 4 +-- src/ngx_http_lua_directive.c | 4 +-- src/ngx_http_lua_headerfilterby.c | 4 +-- src/ngx_http_lua_rewriteby.c | 6 ++-- src/ngx_http_lua_server_rewriteby.c | 4 +-- src/ngx_http_lua_socket_udp.c | 4 +-- src/ngx_http_lua_ssl_certby.c | 4 +-- src/ngx_http_lua_ssl_client_helloby.c | 4 +-- src/ngx_http_lua_ssl_session_fetchby.c | 4 +-- src/ngx_http_lua_timer.c | 4 +-- t/014-bugs.t | 49 ++++++++++++++++++++++++++ 14 files changed, 76 insertions(+), 27 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 58c251443e..d40eab123e 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -240,7 +240,7 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -291,9 +291,9 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) /* }}} */ - /* {{{ register request cleanup hooks */ + /* {{{ register nginx pool cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index a8a382ac44..78e3b5c2d6 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -234,7 +234,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) ngx_http_lua_ctx_t *ctx; ngx_int_t rc; uint16_t old_context; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_chain_t *out; ngx_chain_t *cl, *ln; ngx_http_lua_main_conf_t *lmcf; @@ -314,7 +314,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) } if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index d977ee8e3f..843504549a 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -587,7 +587,7 @@ typedef struct ngx_http_lua_ctx_s { ngx_chain_t *filter_in_bufs; /* for the body filter */ ngx_chain_t *filter_busy_bufs; /* for the body filter */ - ngx_http_cleanup_pt *cleanup; + ngx_pool_cleanup_pt *cleanup; ngx_http_cleanup_t *free_cleanup; /* free list of cleanup records */ diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 76e6a074da..5e2ae55209 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -29,7 +29,7 @@ ngx_http_lua_content_by_chunk(lua_State *L, ngx_http_request_t *r) lua_State *co; ngx_event_t *rev; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -83,7 +83,7 @@ ngx_http_lua_content_by_chunk(lua_State *L, ngx_http_request_t *r) /* {{{ register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_directive.c b/src/ngx_http_lua_directive.c index f1623056b5..f42aae9d51 100644 --- a/src/ngx_http_lua_directive.c +++ b/src/ngx_http_lua_directive.c @@ -1436,7 +1436,7 @@ ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) { lua_State *L; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { @@ -1451,7 +1451,7 @@ ngx_http_lua_set_by_lua_init(ngx_http_request_t *r) } if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index f2fe165bd5..71553558bb 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -231,7 +231,7 @@ ngx_http_lua_header_filter(ngx_http_request_t *r) ngx_http_lua_loc_conf_t *llcf; ngx_http_lua_ctx_t *ctx; ngx_int_t rc; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; uint16_t old_context; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, @@ -260,7 +260,7 @@ ngx_http_lua_header_filter(ngx_http_request_t *r) } if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_ERROR; } diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index d1eabeccdd..4109f288e3 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -241,7 +241,7 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -291,9 +291,9 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) /* }}} */ - /* {{{ register request cleanup hooks */ + /* {{{ register nginx pool cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c index 22bb69e8c8..be860693f4 100644 --- a/src/ngx_http_lua_server_rewriteby.c +++ b/src/ngx_http_lua_server_rewriteby.c @@ -199,7 +199,7 @@ ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_event_t *rev; ngx_connection_t *c; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_lua_loc_conf_t *llcf; @@ -251,7 +251,7 @@ ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) /* {{{ register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index cbb6621b47..10285f6d0a 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -591,7 +591,7 @@ ngx_http_lua_socket_resolve_retval_handler(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx; ngx_http_lua_co_ctx_t *coctx; ngx_connection_t *c; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_http_upstream_resolved_t *ur; ngx_int_t rc; ngx_http_lua_udp_connection_t *uc; @@ -625,7 +625,7 @@ ngx_http_lua_socket_resolve_retval_handler(ngx_http_request_t *r, } if (u->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { u->ft_type |= NGX_HTTP_LUA_SOCKET_FT_ERROR; lua_pushnil(L); diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 0499bfb355..b8e70ddefa 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -453,7 +453,7 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_int_t rc; lua_State *co; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -507,7 +507,7 @@ ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) /* register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { rc = NGX_ERROR; ngx_http_lua_finalize_request(r, rc); diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 1e68e07f4c..a1dac61887 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -449,7 +449,7 @@ ngx_http_lua_ssl_client_hello_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_int_t rc; lua_State *co; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -503,7 +503,7 @@ ngx_http_lua_ssl_client_hello_by_chunk(lua_State *L, ngx_http_request_t *r) /* register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { rc = NGX_ERROR; ngx_http_lua_finalize_request(r, rc); diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index 66e9848550..ebce63ce28 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -477,7 +477,7 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) ngx_int_t rc; lua_State *co; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); @@ -531,7 +531,7 @@ ngx_http_lua_ssl_sess_fetch_by_chunk(lua_State *L, ngx_http_request_t *r) /* register request cleanup hooks */ if (ctx->cleanup == NULL) { - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { rc = NGX_ERROR; ngx_http_lua_finalize_request(r, rc); diff --git a/src/ngx_http_lua_timer.c b/src/ngx_http_lua_timer.c index 060cab543a..11d29349a5 100644 --- a/src/ngx_http_lua_timer.c +++ b/src/ngx_http_lua_timer.c @@ -519,7 +519,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) ngx_connection_t *c = NULL; ngx_http_request_t *r = NULL; ngx_http_lua_ctx_t *ctx; - ngx_http_cleanup_t *cln; + ngx_pool_cleanup_t *cln; ngx_pool_cleanup_t *pcln; ngx_http_lua_timer_ctx_t tctx; @@ -620,7 +620,7 @@ ngx_http_lua_timer_handler(ngx_event_t *ev) L = ngx_http_lua_get_lua_vm(r, ctx); - cln = ngx_http_cleanup_add(r, 0); + cln = ngx_pool_cleanup_add(r->pool, 0); if (cln == NULL) { errmsg = "could not add request cleanup"; goto failed; diff --git a/t/014-bugs.t b/t/014-bugs.t index bfc86c9e9a..877aecfdb5 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -1283,3 +1283,52 @@ Hello world --- shutdown_error_log eval qr|failed to read a line: closed| --- timeout: 1.2 + + + +=== TEST 48: nginx crashes when encountering an illegal http if header +crash with ngx.send_headers() +--- main_config +--- config +error_page 412 /my_error_handler_412; + +location /t { + rewrite_by_lua_block { + ngx.send_headers() + -- ngx.print() -- this also triggers the bug + } +} +location = /my_error_handler_412 { + return 412 "hello"; +} +--- request + GET /t +--- more_headers +If-Match: 1 +--- error_code: 412 +--- response_body eval +qr/\Ahello\z/ + + + +=== TEST 49: nginx crashes when encountering an illegal http if header +crash with ngx.print() +--- main_config +--- config +error_page 412 /my_error_handler_412; + +location /t { + rewrite_by_lua_block { + ngx.print() + } +} +location = /my_error_handler_412 { + return 412 "hello"; +} +--- request + GET /t +--- more_headers +If-Match: 1 +--- error_code: 412 +--- response_body eval +qr/\Ahello\z/ From c47084b5d719ce507d2419d8660f39544a9d1fea Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 19 Jun 2023 14:28:57 +0800 Subject: [PATCH 694/848] bumped version to 0.10.25. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 018184990d..640e0c396b 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10024 +#define ngx_http_lua_version 10025 typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; From b4c0450bbe36dba64e4adb0ff3bc8b773202b98a Mon Sep 17 00:00:00 2001 From: zhangjie <49422942+ZJfans@users.noreply.github.com> Date: Mon, 26 Jun 2023 18:29:48 +0800 Subject: [PATCH 695/848] feature: Support lua_ssl_certificate and lua_ssl_certificate_key. Signed-off-by: lijunlong --- doc/HttpLuaModule.wiki | 32 ++++++++ src/ngx_http_lua_common.h | 2 + src/ngx_http_lua_module.c | 46 +++++++++++ t/187-ssl-two-verification.t | 143 +++++++++++++++++++++++++++++++++++ 4 files changed, 223 insertions(+) create mode 100644 t/187-ssl-two-verification.t diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 34751144bd..118d60bbb0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2845,6 +2845,38 @@ The support for the TLSv1.3 parameter requires version v0.10. This directive was first introduced in the v0.9.11 release. +== lua_ssl_certificate == + +'''syntax:''' ''lua_ssl_certificate '' + +'''default:''' ''no'' + +'''context:''' ''http, server, location'' + +Specifies the file path to the SSL/TLS certificate in PEM format used for the [[#tcpsock:sslhandshake|tcpsock:sslhandshake]] method. + +This directive allows you to specify the SSL/TLS certificate that will be presented to server during the SSL/TLS handshake process. + +This directive was first introduced in the v0.10.26 release. + +See also [[#lua_ssl_certificate_key|lua_ssl_certificate_key]] and [[#lua_ssl_verify_depth|lua_ssl_verify_depth]]. + +== lua_ssl_certificate_key == + +'''syntax:''' ''lua_ssl_certificate_key '' + +'''default:''' ''no'' + +'''context:''' ''http, server, location'' + +Specifies the file path to the private key associated with the SSL/TLS certificate used in the [[#tcpsock:sslhandshake|tcpsock:sslhandshake]] method. + +This directive allows you to specify the private key file corresponding to the SSL/TLS certificate specified by lua_ssl_certificate. The private key should be in PEM format and must match the certificate. + +This directive was first introduced in the v0.10.26 release. + +See also [[#lua_ssl_certificate|lua_ssl_certificate]] and [[#lua_ssl_verify_depth|lua_ssl_verify_depth]]. + == lua_ssl_trusted_certificate == '''syntax:''' ''lua_ssl_trusted_certificate '' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 843504549a..7d9e10cea1 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -360,6 +360,8 @@ union ngx_http_lua_srv_conf_u { typedef struct { #if (NGX_HTTP_SSL) ngx_ssl_t *ssl; /* shared by SSL cosockets */ + ngx_array_t *ssl_certificates; + ngx_array_t *ssl_certificate_keys; ngx_uint_t ssl_protocols; ngx_str_t ssl_ciphers; ngx_uint_t ssl_verify_depth; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 869cc91a82..4d47e83e20 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -650,6 +650,20 @@ static ngx_command_t ngx_http_lua_cmds[] = { offsetof(ngx_http_lua_loc_conf_t, ssl_verify_depth), NULL }, + { ngx_string("lua_ssl_certificate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_array_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_lua_loc_conf_t, ssl_certificates), + NULL }, + + { ngx_string("lua_ssl_certificate_key"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_array_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_lua_loc_conf_t, ssl_certificate_keys), + NULL }, + { ngx_string("lua_ssl_trusted_certificate"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, ngx_conf_set_str_slot, @@ -1401,6 +1415,8 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) #if (NGX_HTTP_SSL) conf->ssl_verify_depth = NGX_CONF_UNSET_UINT; + conf->ssl_certificates = NGX_CONF_UNSET_PTR; + conf->ssl_certificate_keys = NGX_CONF_UNSET_PTR; #if (nginx_version >= 1019004) conf->ssl_conf_commands = NGX_CONF_UNSET_PTR; #endif @@ -1480,6 +1496,10 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) ngx_conf_merge_uint_value(conf->ssl_verify_depth, prev->ssl_verify_depth, 1); + ngx_conf_merge_ptr_value(conf->ssl_certificates, + prev->ssl_certificates, NULL); + ngx_conf_merge_ptr_value(conf->ssl_certificate_keys, + prev->ssl_certificate_keys, NULL); ngx_conf_merge_str_value(conf->ssl_trusted_certificate, prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); @@ -1542,6 +1562,8 @@ ngx_http_lua_merge_ssl(ngx_conf_t *cf, if (conf->ssl_protocols == 0 && conf->ssl_ciphers.data == NULL && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_certificates == NGX_CONF_UNSET_PTR + && conf->ssl_certificate_keys == NGX_CONF_UNSET_PTR && conf->ssl_trusted_certificate.data == NULL && conf->ssl_crl.data == NULL #if (nginx_version >= 1019004) @@ -1589,6 +1611,20 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) return NGX_OK; } + if (llcf->ssl_certificates) { + if (llcf->ssl_certificate_keys == NULL + || llcf->ssl_certificate_keys->nelts + < llcf->ssl_certificates->nelts) + { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no \"lua_ssl_certificate_key\" is defined " + "for certificate \"%V\"", + ((ngx_str_t *) llcf->ssl_certificates->elts) + + llcf->ssl_certificates->nelts - 1); + return NGX_ERROR; + } + } + if (ngx_ssl_create(llcf->ssl, llcf->ssl_protocols, NULL) != NGX_OK) { return NGX_ERROR; } @@ -1612,6 +1648,16 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) return NGX_ERROR; } + if (llcf->ssl_certificates + && ngx_ssl_certificates(cf, llcf->ssl, + llcf->ssl_certificates, + llcf->ssl_certificate_keys, + NULL) + != NGX_OK) + { + return NGX_ERROR; + } + if (llcf->ssl_trusted_certificate.len && ngx_ssl_trusted_certificate(cf, llcf->ssl, &llcf->ssl_trusted_certificate, diff --git a/t/187-ssl-two-verification.t b/t/187-ssl-two-verification.t new file mode 100644 index 0000000000..527df7cbde --- /dev/null +++ b/t/187-ssl-two-verification.t @@ -0,0 +1,143 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +repeat_each(3); + +# All these tests need to have new openssl +my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; +my $openssl_version = eval { `$NginxBinary -V 2>&1` }; + +if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { + plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +} else { + plan tests => repeat_each() * (blocks() * 7); +} + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); +$ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; + +#log_level 'warn'; +log_level 'debug'; + +no_long_string(); +#no_diff(); + +run_tests(); + +__DATA__ + +=== TEST 1: simple logging +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + #listen 127.0.0.1:4433 ssl; + server_name test.com; + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + #ssl_trusted_certificate ../../cert/test.crt; + ssl_client_certificate ../../cert/test.crt; + ssl_verify_client on; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + log_by_lua_block { + ngx.log(ngx.INFO, "ssl_client_s_dn: ", ngx.var.ssl_client_s_dn) + } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_certificate ../../cert/test.crt; + lua_ssl_certificate_key ../../cert/test.key; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + -- local ok, err = sock:connect("127.0.0.1", 4433) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: cdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" +ssl_client_s_dn: emailAddress=agentzh@gmail.com,CN=test.com,OU=OpenResty,O=OpenResty,L=San Francisco,ST=California,C=US + +--- no_error_log +[error] +[alert] +--- grep_error_log eval: qr/ssl_client_hello_by_lua\(.*?,|\bssl client hello: connection reusable: \d+|\breusable connection: \d+/ +--- grep_error_log_out eval +# Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) +# before call ssl callback function +$Test::Nginx::Util::NginxVersion >= 1.017009 ? +qr/reusable connection: 0 +ssl client hello: connection reusable: 0 +ssl_client_hello_by_lua\(nginx.conf:\d+\):1: ssl client hello by lua is running!,/ +: qr /reusable connection: 1 +ssl client hello: connection reusable: 1 +reusable connection: 0 +ssl_client_hello_by_lua\(nginx.conf:\d+\):1: ssl client hello by lua is running!,/ From 05da2493165845886a3d94515fd7651abf6d6705 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 20 Jul 2023 23:40:50 +0800 Subject: [PATCH 696/848] feature: allow get empty key for ngx.req.get_uri(). --- README.markdown | 11 +++++++- src/ngx_http_lua_args.c | 6 ++--- t/030-uri-args.t | 59 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 72 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 695e6b0a4b..96d8716b6a 100644 --- a/README.markdown +++ b/README.markdown @@ -5001,7 +5001,7 @@ See also [ngx.req.set_uri](#ngxreqset_uri). ngx.req.get_uri_args -------------------- -**syntax:** *args, err = ngx.req.get_uri_args(max_args?, tab?)* +**syntax:** *args, err = ngx.req.get_uri_args(max_args?, tab?, allow_empty_key?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** @@ -5102,6 +5102,15 @@ This argument can be set to zero to remove the limit and to process all request Removing the `max_args` cap is strongly discouraged. +If the key of the argument is empty, the argument will be skipped by default. +For example, calling `ngx.req.get_uri_args()` against the HTTP request `https://example.com/q?a=b&=d` +will only get a=b. The optional `allow_empty_key` function argument can be used to override this limit: + + +```lua + local args, err = ngx.req.get_uri_args(nil, nil, true) +``` + [Back to TOC](#nginx-api-for-lua) ngx.req.get_post_args diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index 1280846545..b878602252 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -461,7 +461,7 @@ ngx_http_lua_ffi_req_get_uri_args_count(ngx_http_request_t *r, int max, int ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, - ngx_http_lua_ffi_table_elt_t *out, int count) + ngx_http_lua_ffi_table_elt_t *out, int count, int allow_empty_key) { int i, parsing_value = 0; u_char *last, *p, *q; @@ -511,7 +511,7 @@ ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, /* end of the current pair's value */ parsing_value = 0; - if (out[i].key.len) { + if (out[i].key.len > 0 || allow_empty_key) { out[i].value.data = q; out[i].value.len = (int) (dst - q); i++; @@ -553,7 +553,7 @@ ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, dd("pushing key or value %.*s", (int) (dst - q), q); if (parsing_value) { - if (out[i].key.len) { + if (out[i].key.len > 0 || allow_empty_key) { out[i].value.data = q; out[i].value.len = (int) (dst - q); i++; diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 0633476c8d..334ca617a0 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1767,3 +1767,62 @@ request_uri: /foo%20bar uri: /foo bar --- no_error_log [error] + + + +=== TEST 67: empty key in the end +--- config + location /lua { + content_by_lua_block { + local args, err = ngx.req.get_uri_args(nil, nil, true) + + if err then + ngx.say("err: ", err) + end + + local keys = {} + for key, val in pairs(args) do + table.insert(keys, key) + end + + table.sort(keys) + for i, key in ipairs(keys) do + ngx.say(key, " = ", args[key]) + end + } + } +--- request +GET /lua?ka=va&=vb +--- response_body + = vb +ka = va + + + +=== TEST 68: empty key in the middle +--- config + location /lua { + content_by_lua_block { + local args, err = ngx.req.get_uri_args(nil, nil, true) + + if err then + ngx.say("err: ", err) + end + + local keys = {} + for key, val in pairs(args) do + table.insert(keys, key) + end + + table.sort(keys) + for i, key in ipairs(keys) do + ngx.say(key, " = ", args[key]) + end + } + } +--- request +GET /lua?ka=va&=vb&kc=vc +--- response_body + = vb +ka = va +kc = vc From 4ca27c48f5239d310b2f6838673a4f1f87328327 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 21 Jul 2023 11:30:16 +0800 Subject: [PATCH 697/848] Revert "feature: allow get empty key for ngx.req.get_uri()." This reverts commit 05da2493165845886a3d94515fd7651abf6d6705. --- README.markdown | 11 +------- src/ngx_http_lua_args.c | 6 ++--- t/030-uri-args.t | 59 ----------------------------------------- 3 files changed, 4 insertions(+), 72 deletions(-) diff --git a/README.markdown b/README.markdown index 96d8716b6a..695e6b0a4b 100644 --- a/README.markdown +++ b/README.markdown @@ -5001,7 +5001,7 @@ See also [ngx.req.set_uri](#ngxreqset_uri). ngx.req.get_uri_args -------------------- -**syntax:** *args, err = ngx.req.get_uri_args(max_args?, tab?, allow_empty_key?)* +**syntax:** *args, err = ngx.req.get_uri_args(max_args?, tab?)* **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, balancer_by_lua** @@ -5102,15 +5102,6 @@ This argument can be set to zero to remove the limit and to process all request Removing the `max_args` cap is strongly discouraged. -If the key of the argument is empty, the argument will be skipped by default. -For example, calling `ngx.req.get_uri_args()` against the HTTP request `https://example.com/q?a=b&=d` -will only get a=b. The optional `allow_empty_key` function argument can be used to override this limit: - - -```lua - local args, err = ngx.req.get_uri_args(nil, nil, true) -``` - [Back to TOC](#nginx-api-for-lua) ngx.req.get_post_args diff --git a/src/ngx_http_lua_args.c b/src/ngx_http_lua_args.c index b878602252..1280846545 100644 --- a/src/ngx_http_lua_args.c +++ b/src/ngx_http_lua_args.c @@ -461,7 +461,7 @@ ngx_http_lua_ffi_req_get_uri_args_count(ngx_http_request_t *r, int max, int ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, - ngx_http_lua_ffi_table_elt_t *out, int count, int allow_empty_key) + ngx_http_lua_ffi_table_elt_t *out, int count) { int i, parsing_value = 0; u_char *last, *p, *q; @@ -511,7 +511,7 @@ ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, /* end of the current pair's value */ parsing_value = 0; - if (out[i].key.len > 0 || allow_empty_key) { + if (out[i].key.len) { out[i].value.data = q; out[i].value.len = (int) (dst - q); i++; @@ -553,7 +553,7 @@ ngx_http_lua_ffi_req_get_uri_args(ngx_http_request_t *r, u_char *buf, dd("pushing key or value %.*s", (int) (dst - q), q); if (parsing_value) { - if (out[i].key.len > 0 || allow_empty_key) { + if (out[i].key.len) { out[i].value.data = q; out[i].value.len = (int) (dst - q); i++; diff --git a/t/030-uri-args.t b/t/030-uri-args.t index 334ca617a0..0633476c8d 100644 --- a/t/030-uri-args.t +++ b/t/030-uri-args.t @@ -1767,62 +1767,3 @@ request_uri: /foo%20bar uri: /foo bar --- no_error_log [error] - - - -=== TEST 67: empty key in the end ---- config - location /lua { - content_by_lua_block { - local args, err = ngx.req.get_uri_args(nil, nil, true) - - if err then - ngx.say("err: ", err) - end - - local keys = {} - for key, val in pairs(args) do - table.insert(keys, key) - end - - table.sort(keys) - for i, key in ipairs(keys) do - ngx.say(key, " = ", args[key]) - end - } - } ---- request -GET /lua?ka=va&=vb ---- response_body - = vb -ka = va - - - -=== TEST 68: empty key in the middle ---- config - location /lua { - content_by_lua_block { - local args, err = ngx.req.get_uri_args(nil, nil, true) - - if err then - ngx.say("err: ", err) - end - - local keys = {} - for key, val in pairs(args) do - table.insert(keys, key) - end - - table.sort(keys) - for i, key in ipairs(keys) do - ngx.say(key, " = ", args[key]) - end - } - } ---- request -GET /lua?ka=va&=vb&kc=vc ---- response_body - = vb -ka = va -kc = vc From caad24b355e8359edea089c2df7a6306410254f1 Mon Sep 17 00:00:00 2001 From: zhangjie <49422942+ZJfans@users.noreply.github.com> Date: Sat, 29 Jul 2023 16:29:32 +0800 Subject: [PATCH 698/848] doc: modified lua_ssl_certificate, lua_ssl_certificate_key and lua_ssl_trusted_certificate. --- README.markdown | 44 ++++++++++++++++++++++++++++++++++++++++-- doc/HttpLuaModule.wiki | 8 ++++---- 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index 695e6b0a4b..daba4c8218 100644 --- a/README.markdown +++ b/README.markdown @@ -1166,6 +1166,8 @@ Directives * [lua_ssl_ciphers](#lua_ssl_ciphers) * [lua_ssl_crl](#lua_ssl_crl) * [lua_ssl_protocols](#lua_ssl_protocols) +* [lua_ssl_certificate](#lua_ssl_certificate) +* [lua_ssl_certificate_key](#lua_ssl_certificate_key) * [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate) * [lua_ssl_verify_depth](#lua_ssl_verify_depth) * [lua_ssl_conf_command](#lua_ssl_conf_command) @@ -3336,12 +3338,50 @@ This directive was first introduced in the `v0.9.11` release. [Back to TOC](#directives) +lua_ssl_certificate +------------------- + +**syntax:** *lua_ssl_certificate <file>* + +**default:** *none* + +**context:** *http, server, location* + +Specifies the file path to the SSL/TLS certificate in PEM format used for the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. + +This directive allows you to specify the SSL/TLS certificate that will be presented to server during the SSL/TLS handshake process. + +This directive was first introduced in the `v0.10.26` release. + +See also [lua_ssl_certificate_key](#lua_ssl_certificate_key) and [lua_ssl_verify_depth](#lua_ssl_verify_depth). + +[Back to TOC](#directives) + +lua_ssl_certificate_key +----------------------- + +**syntax:** *lua_ssl_certificate_key <file>* + +**default:** *none* + +**context:** *http, server, location* + +Specifies the file path to the private key associated with the SSL/TLS certificate used in the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. + +This directive allows you to specify the private key file corresponding to the SSL/TLS certificate specified by lua_ssl_certificate. The private key should be in PEM format and must match the certificate. + +This directive was first introduced in the `v0.10.26` release. + +See also [lua_ssl_certificate](#lua_ssl_certificate) and [lua_ssl_verify_depth](#lua_ssl_verify_depth). + +[Back to TOC](#directives) + lua_ssl_trusted_certificate --------------------------- **syntax:** *lua_ssl_trusted_certificate <file>* -**default:** *no* +**default:** *none* **context:** *http, server, location* @@ -3366,7 +3406,7 @@ Sets the verification depth in the server certificates chain. This directive was first introduced in the `v0.9.11` release. -See also [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate). +See also [lua_ssl_certificate](#lua_ssl_certificate), [lua_ssl_certificate_key](#lua_ssl_certificate_key) and [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate). [Back to TOC](#directives) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 118d60bbb0..8d003bd602 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2849,7 +2849,7 @@ This directive was first introduced in the v0.9.11 release. '''syntax:''' ''lua_ssl_certificate '' -'''default:''' ''no'' +'''default:''' ''none'' '''context:''' ''http, server, location'' @@ -2865,7 +2865,7 @@ See also [[#lua_ssl_certificate_key|lua_ssl_certificate_key]] and [[#lua_ssl_ver '''syntax:''' ''lua_ssl_certificate_key '' -'''default:''' ''no'' +'''default:''' ''none'' '''context:''' ''http, server, location'' @@ -2881,7 +2881,7 @@ See also [[#lua_ssl_certificate|lua_ssl_certificate]] and [[#lua_ssl_verify_dept '''syntax:''' ''lua_ssl_trusted_certificate '' -'''default:''' ''no'' +'''default:''' ''none'' '''context:''' ''http, server, location'' @@ -2903,7 +2903,7 @@ Sets the verification depth in the server certificates chain. This directive was first introduced in the v0.9.11 release. -See also [[#lua_ssl_trusted_certificate|lua_ssl_trusted_certificate]]. +See also [[#lua_ssl_certificate|lua_ssl_certificate]], [[#lua_ssl_certificate_key|lua_ssl_certificate_key]] and [[#lua_ssl_trusted_certificate|lua_ssl_trusted_certificate]]. == lua_ssl_conf_command == From f39e7e50fe871f27111218af06d4074059cd0e86 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 31 Jul 2023 22:11:09 +0800 Subject: [PATCH 699/848] change: remove ngx.re from ngx.run_worker_thread. lmcf->jit_stack shared by ngx.re cause nginx crash when multiple ngx.re instances are executed in parallel. --- README.markdown | 6 - src/ngx_http_lua_worker_thread.c | 9 -- t/166-worker-thread.t | 212 ++++--------------------------- 3 files changed, 26 insertions(+), 201 deletions(-) diff --git a/README.markdown b/README.markdown index daba4c8218..ac04585c5a 100644 --- a/README.markdown +++ b/README.markdown @@ -9366,12 +9366,6 @@ Only the following ngx_lua APIs could be used in `function_name` function of the * `ngx.decode_args` * `ngx.quote_sql_str` -* `ngx.re.match` -* `ngx.re.find` -* `ngx.re.gmatch` -* `ngx.re.sub` -* `ngx.re.gsub` - * `ngx.crc32_short` * `ngx.crc32_long` * `ngx.hmac_sha1` diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index ed7b089295..9de59def75 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -157,15 +157,6 @@ ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) ngx_http_lua_inject_shdict_api(lmcf, vm); lua_setglobal(vm, "ngx"); - /* inject API via ffi */ - lua_getglobal(vm, "require"); - lua_pushstring(vm, "resty.core.regex"); - if (lua_pcall(vm, 1, 0, 0) != 0) { - lua_close(vm); - ngx_free(ctx); - return NULL; - } - lua_getglobal(vm, "require"); lua_pushstring(vm, "resty.core.hash"); if (lua_pcall(vm, 1, 0, 0) != 0) { diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index fd7fdd756e..c8ae62965f 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -777,167 +777,7 @@ true : 'a\Zb\Z' -=== TEST 24: ngx.re.match ---- main_config - thread_pool testpool threads=100; ---- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" ---- config -location /hello { - default_type 'text/plain'; - - content_by_lua_block { - local ok, a, b = ngx.run_worker_thread("testpool", "hello", "hello") - ngx.say(ok, " : ", a, " : ", b) - } -} ---- user_files ->>> hello.lua -local function hello() - local m, err = ngx.re.match("hello, 1234", "([0-9])[0-9]+") - return m[0], m[1] -end -return {hello=hello} ---- request -GET /hello ---- response_body -true : 1234 : 1 - - - -=== TEST 25: ngx.re.find ---- main_config - thread_pool testpool threads=100; ---- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" ---- config -location /hello { - default_type 'text/plain'; - - content_by_lua_block { - local ok, a = ngx.run_worker_thread("testpool", "hello", "hello") - ngx.say(ok, " : ", a) - } -} ---- user_files ->>> hello.lua -local function hello() - local str = "hello, 1234" - local from, to = ngx.re.find(str, "([0-9])([0-9]+)", "jo", nil, 2) - if from then - return string.sub(str, from, to) - end -end -return {hello=hello} ---- request -GET /hello ---- response_body -true : 234 - - - -=== TEST 26: ngx.re.gmatch ---- main_config - thread_pool testpool threads=100; ---- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" ---- config -location /hello { - default_type 'text/plain'; - - content_by_lua_block { - local ok, ret = ngx.run_worker_thread("testpool", "hello", "hello") - ngx.say(ok) - ngx.say(ret[1]) - ngx.say(ret[2]) - } -} ---- user_files ->>> hello.lua -local function hello() - local ret = {} - for m in ngx.re.gmatch("hello, world", "[a-z]+", "j") do - if m then - table.insert(ret, m[0]) - end - end - return ret -end -return {hello=hello} ---- request -GET /hello ---- response_body -true -hello -world - - - -=== TEST 27: ngx.re.sub ---- main_config - thread_pool testpool threads=100; ---- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" ---- config -location /hello { - default_type 'text/plain'; - - content_by_lua_block { - local ok, a, b = ngx.run_worker_thread("testpool", "hello", "hello") - ngx.say(ok) - ngx.say(a) - ngx.say(b) - } -} ---- user_files ->>> hello.lua -local function hello() - local newstr, n = ngx.re.sub("hello, 1234", "[0-9]", "$$") - return newstr, n -end -return {hello=hello} ---- request -GET /hello ---- response_body -true -hello, $234 -1 - - - -=== TEST 28: ngx.re.gsub ---- main_config - thread_pool testpool threads=100; ---- http_config eval - "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" ---- config -location /hello { - default_type 'text/plain'; - - content_by_lua_block { - local ok, a, b = ngx.run_worker_thread("testpool", "hello", "hello") - ngx.say(ok) - ngx.say(a) - ngx.say(b) - } -} ---- user_files ->>> hello.lua -local function hello() - local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", "[$0,$1]", "i") - return newstr, n -end -return {hello=hello} ---- request -GET /hello ---- response_body -true -[hello,h], [world,w] -2 - - - -=== TEST 29: ngx.decode_base64 +=== TEST 24: ngx.decode_base64 --- main_config thread_pool testpool threads=100; --- http_config eval @@ -964,7 +804,7 @@ true : hello -=== TEST 30: ngx.crc32_short +=== TEST 25: ngx.crc32_short --- main_config thread_pool testpool threads=100; --- http_config eval @@ -991,7 +831,7 @@ true : 4289425978 -=== TEST 31: ngx.crc32_long +=== TEST 26: ngx.crc32_long --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1018,7 +858,7 @@ true : 4289425978 -=== TEST 32: ngx.md5_bin +=== TEST 27: ngx.md5_bin --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1049,7 +889,7 @@ true : 6c8349cc7260ae62e3b1396831a8398f -=== TEST 33: ngx.md5 +=== TEST 28: ngx.md5 --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1076,7 +916,7 @@ true : 5d41402abc4b2a76b9719d911017c592 -=== TEST 34: ngx.config.debug +=== TEST 29: ngx.config.debug --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1103,7 +943,7 @@ GET /hello -=== TEST 35: ngx.config.prefix +=== TEST 30: ngx.config.prefix --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1130,7 +970,7 @@ GET /hello -=== TEST 36: ngx.config.nginx_version +=== TEST 31: ngx.config.nginx_version --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1157,7 +997,7 @@ GET /hello -=== TEST 37: ngx.config.nginx_configure +=== TEST 32: ngx.config.nginx_configure --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1184,7 +1024,7 @@ GET /hello -=== TEST 38: ngx.config.ngx_lua_version +=== TEST 33: ngx.config.ngx_lua_version --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1211,7 +1051,7 @@ GET /hello -=== TEST 39: write_log_file +=== TEST 34: write_log_file --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1249,7 +1089,7 @@ true -=== TEST 40: shdict get, int value +=== TEST 35: shdict get, int value --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1282,7 +1122,7 @@ true,8 -=== TEST 41: shdict set nil in main thread +=== TEST 36: shdict set nil in main thread --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1319,7 +1159,7 @@ true,nil -=== TEST 42: shdict set nil in worker thread +=== TEST 37: shdict set nil in worker thread --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1354,7 +1194,7 @@ true,nil -=== TEST 43: shdict get_stale +=== TEST 38: shdict get_stale --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1388,7 +1228,7 @@ true,8 -=== TEST 44: shdict add failed +=== TEST 39: shdict add failed --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1422,7 +1262,7 @@ true,false,exists -=== TEST 45: shdict force add +=== TEST 40: shdict force add --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1458,7 +1298,7 @@ true,true,true,nil -=== TEST 46: shdict replace +=== TEST 41: shdict replace --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1493,7 +1333,7 @@ true,true,nil,8 -=== TEST 47: shdict replace not found +=== TEST 42: shdict replace not found --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1526,7 +1366,7 @@ true,false,not found -=== TEST 48: shdict incr +=== TEST 43: shdict incr --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1560,7 +1400,7 @@ true,9,nil,9 -=== TEST 49: shdict lpush lpop +=== TEST 44: shdict lpush lpop --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1596,7 +1436,7 @@ true,9,2,nil,7 -=== TEST 50: shdict expire ttl +=== TEST 45: shdict expire ttl --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1630,7 +1470,7 @@ true,true,nil,true -=== TEST 51: shdict flush_all +=== TEST 46: shdict flush_all --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1664,7 +1504,7 @@ true,nil,nil -=== TEST 52: shdict get_keys +=== TEST 47: shdict get_keys --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1698,7 +1538,7 @@ true,Jim:King -=== TEST 53: unsupported argument type in self-reference table +=== TEST 48: unsupported argument type in self-reference table --- main_config thread_pool testpool threads=100; --- http_config eval @@ -1727,7 +1567,7 @@ false , suspicious circular references, table depth exceed max depth: 100 in the -=== TEST 54: unsupported argument type in circular-reference table +=== TEST 49: unsupported argument type in circular-reference table --- main_config thread_pool testpool threads=100; --- http_config eval From e69fd3de281f31804857aa6dc0b8e79055716138 Mon Sep 17 00:00:00 2001 From: swananan Date: Sun, 6 Aug 2023 08:02:34 +0800 Subject: [PATCH 700/848] feature: upgrade nginx core to 1.25.1. --- .travis.yml | 2 ++ t/028-req-header.t | 10 +++++++-- t/129-ssl-socket.t | 8 +++---- t/142-ssl-session-store.t | 12 +++++++++++ t/162-socket-tls-handshake.t | 8 +++---- util/build-with-dd.sh | 14 ++++++++++++ util/build-without-ssl.sh | 7 ++++++ util/build.sh | 14 ++++++++++++ util/ver-ge | 41 ++++++++++++++++++++++++++++++++++++ 9 files changed, 106 insertions(+), 10 deletions(-) create mode 100755 util/ver-ge diff --git a/.travis.yml b/.travis.yml index 64d27d8529..19f560a4fe 100644 --- a/.travis.yml +++ b/.travis.yml @@ -54,6 +54,8 @@ env: jobs: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f services: - memcached diff --git a/t/028-req-header.t b/t/028-req-header.t index 8c8ff83fd8..f4c393977d 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -273,8 +273,14 @@ Content-Type: } --- request GET /bar ---- response_body -Foo: a +--- response_body eval +# Since nginx version 1.23.0, nginx combines same $http_* variable together +# wtf +$Test::Nginx::Util::NginxVersion >= 1.023000 ? + +"Foo: a, b\n" +: +"Foo: a\n" diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 7e48d59bca..85d3a80985 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -39,7 +39,7 @@ run_tests(); __DATA__ -=== TEST 1: www.bing.com +=== TEST 1: www.google.com --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -62,7 +62,7 @@ __DATA__ do local sock = ngx.socket.tcp() sock:settimeout(2000) - local ok, err = sock:connect("www.bing.com", 443) + local ok, err = sock:connect("www.google.com", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -78,7 +78,7 @@ __DATA__ ngx.say("ssl handshake: ", type(sess)) - local req = "GET / HTTP/1.1\\r\\nHost: www.bing.com\\r\\nConnection: close\\r\\n\\r\\n" + local req = "GET / HTTP/1.1\\r\\nHost: www.google.com\\r\\nConnection: close\\r\\n\\r\\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -107,7 +107,7 @@ GET /t --- response_body_like chop \Aconnected: 1 ssl handshake: cdata -sent http request: 57 bytes. +sent http request: 59 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil \z diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 61598732be..ead2a167e5 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -40,6 +40,7 @@ __DATA__ server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -115,6 +116,7 @@ ssl_session_store_by_lua\(nginx\.conf:25\):1: ssl session store by lua is runnin server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -190,6 +192,7 @@ API disabled in the context of ssl_session_store_by_lua* server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -282,6 +285,7 @@ my timer run! server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -351,6 +355,7 @@ API disabled in the context of ssl_session_store_by_lua* resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -423,6 +428,7 @@ ngx.exit does not yield and the error code is eaten. resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -496,6 +502,7 @@ ssl_session_store_by_lua*: handler return value: 0, sess new cb exit code: 0 resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -564,6 +571,7 @@ should never reached here resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -634,6 +642,7 @@ get_phase: ssl_session_store server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -703,6 +712,7 @@ qr/elapsed in ssl cert by lua: 0.(?:09|1[01])\d+,/, server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -784,6 +794,7 @@ a.lua:1: ssl store session by lua is running! resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; lua_ssl_verify_depth 3; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -854,6 +865,7 @@ qr/\[emerg\] .*? "ssl_session_store_by_lua_block" directive is not allowed here server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + lua_ssl_protocols TLSv1.2; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; diff --git a/t/162-socket-tls-handshake.t b/t/162-socket-tls-handshake.t index 80fb2f9825..19ad67a749 100644 --- a/t/162-socket-tls-handshake.t +++ b/t/162-socket-tls-handshake.t @@ -63,7 +63,7 @@ run_tests(); __DATA__ -=== TEST 1: sanity: www.bing.com +=== TEST 1: sanity: www.google.com --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -86,7 +86,7 @@ __DATA__ local sock = ngx.socket.tcp() sock:settimeout(2000) - local ok, err = sock:connect("www.bing.com", 443) + local ok, err = sock:connect("www.google.com", 443) if not ok then ngx.say("failed to connect: ", err) return @@ -102,7 +102,7 @@ __DATA__ ngx.say("ssl handshake: ", type(sess)) - local req = "GET / HTTP/1.1\r\nHost: www.bing.com\r\nConnection: close\r\n\r\n" + local req = "GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n" local bytes, err = sock:send(req) if not bytes then ngx.say("failed to send http request: ", err) @@ -131,7 +131,7 @@ GET /t --- response_body_like chop \Aconnected: 1 ssl handshake: cdata -sent http request: 57 bytes. +sent http request: 59 bytes. received: HTTP/1.1 (?:200 OK|302 Found) close: 1 nil \z diff --git a/util/build-with-dd.sh b/util/build-with-dd.sh index 41ecc72685..db318ce391 100755 --- a/util/build-with-dd.sh +++ b/util/build-with-dd.sh @@ -12,12 +12,26 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" +add_http3_module=--with-http_v3_module +answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +if [ "$OPENSSL_VER" = "1.1.0l" ] || [ "$answer" = "N" ]; then + add_http3_module="" +fi + +disable_pcre2=--without-pcre2 +answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +if [ "$answer" = "N" ]; then + disable_pcre2="" +fi + time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ + $disable_pcre2 \ --with-ipv6 \ --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC -DDDEBUG=1" \ --with-http_v2_module \ + $add_http3_module \ --with-http_realip_module \ --with-http_ssl_module \ --add-module=$root/../ndk-nginx-module \ diff --git a/util/build-without-ssl.sh b/util/build-without-ssl.sh index 906be75706..97097daac0 100755 --- a/util/build-without-ssl.sh +++ b/util/build-without-ssl.sh @@ -24,9 +24,16 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" +disable_pcre2=--without-pcre2 +answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +if [ "$answer" = "N" ]; then + disable_pcre2="" +fi + time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ + $disable_pcre2 \ --with-ipv6 \ --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC" \ --with-http_v2_module \ diff --git a/util/build.sh b/util/build.sh index 713b8482d3..3a786a186f 100755 --- a/util/build.sh +++ b/util/build.sh @@ -24,12 +24,26 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" +add_http3_module=--with-http_v3_module +answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +if [ "$OPENSSL_VER" = "1.1.0l" ] || [ "$answer" = "N" ]; then + add_http3_module="" +fi + +disable_pcre2=--without-pcre2 +answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +if [ "$answer" = "N" ]; then + disable_pcre2="" +fi + time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ + $disable_pcre2 \ --with-ipv6 \ --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC" \ --with-http_v2_module \ + $add_http3_module \ --with-http_realip_module \ --with-http_ssl_module \ --add-module=$root/../ndk-nginx-module \ diff --git a/util/ver-ge b/util/ver-ge new file mode 100755 index 0000000000..8d7a401e94 --- /dev/null +++ b/util/ver-ge @@ -0,0 +1,41 @@ +#!/usr/bin/env perl + +use strict; +use warnings; + +sub usage { + die "Usage: $0 \n"; +} + +my $a = shift or usage(); +my $b = shift or usage(); + +my @as = split /\./, $a; +my @bs = split /\./, $b; + +my $n = @as > @bs ? scalar(@as) : scalar(@bs); + +for (my $i = 0; $i < $n; $i++) { + my $x = $as[$i]; + my $y = $bs[$i]; + + if (!defined $x) { + $x = 0; + } + + if (!defined $y) { + $y = 0; + } + + if ($x > $y) { + print "Y\n"; + exit; + + } elsif ($x < $y) { + print "N\n"; + exit; + } +} + +print "Y\n"; + From 25b7959b29e73069823bbdeb466b49aa12194bb9 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 7 Aug 2023 21:48:50 +0800 Subject: [PATCH 701/848] bugfix: fixed compilation error: variable 'nelts' set but not used. --- src/ngx_http_lua_headers_in.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_headers_in.c b/src/ngx_http_lua_headers_in.c index 960e2f85f6..e756700b06 100644 --- a/src/ngx_http_lua_headers_in.c +++ b/src/ngx_http_lua_headers_in.c @@ -591,19 +591,21 @@ ngx_http_set_builtin_multi_header(ngx_http_request_t *r, { #if defined(nginx_version) && nginx_version >= 1023000 ngx_table_elt_t **headers, **ph, *h; - int nelts; headers = (ngx_table_elt_t **) ((char *) &r->headers_in + hv->offset); if (!hv->no_override && *headers != NULL) { - nelts = 0; +#if defined(DDEBUG) && (DDEBUG) + int nelts = 0; + for (h = *headers; h; h = h->next) { nelts++; } - *headers = NULL; - dd("clear multi-value headers: %d", nelts); +#endif + + *headers = NULL; } if (ngx_http_set_header_helper(r, hv, value, &h) == NGX_ERROR) { From e2d0505955de189d55187c4bf24c400fa4215329 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 8 Aug 2023 16:02:11 +0800 Subject: [PATCH 702/848] doc: revise the description regarding Nginx compatibility. --- README.markdown | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.markdown b/README.markdown index ac04585c5a..021251dfce 100644 --- a/README.markdown +++ b/README.markdown @@ -309,6 +309,8 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.25.x (last tested: 1.25.1) +* 1.21.x (last tested: 1.21.4) * 1.19.x (last tested: 1.19.3) * 1.17.x (last tested: 1.17.8) * 1.15.x (last tested: 1.15.8) From c89469e920713d17d703a5f3736c9335edac22bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sat, 19 Aug 2023 22:26:55 +0800 Subject: [PATCH 703/848] doc: update release date Signed-off-by: spacewander --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 021251dfce..29945d4997 100644 --- a/README.markdown +++ b/README.markdown @@ -65,8 +65,8 @@ Version ======= This document describes ngx_lua -[v0.10.19](https://github.com/openresty/lua-nginx-module/tags), which was released -on 3 Nov, 2020. +[v0.10.25](https://github.com/openresty/lua-nginx-module/tags), which was released +on 19 June 2023. Videos ====== diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 8d003bd602..678e60b47c 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -17,8 +17,8 @@ Production ready. = Version = This document describes ngx_lua -[https://github.com/openresty/lua-nginx-module/tags v0.10.19], which was released -on 3 Nov, 2020. +[https://github.com/openresty/lua-nginx-module/tags v0.10.25], which was released +on 19 June 2023. = Videos = From cb83e33e26579788be0fe0282fa5976de6dfa6e3 Mon Sep 17 00:00:00 2001 From: swananan Date: Sat, 23 Sep 2023 08:20:12 +0800 Subject: [PATCH 704/848] feature: support pcre2 Signed-off-by: lijunlong --- .travis.yml | 17 +- src/ngx_http_lua_common.h | 10 +- src/ngx_http_lua_module.c | 14 + src/ngx_http_lua_pcrefix.c | 88 +++++- src/ngx_http_lua_pcrefix.h | 9 +- src/ngx_http_lua_regex.c | 539 +++++++++++++++++++++++++++++++------ t/028-req-header.t | 1 - t/034-match.t | 32 ++- t/035-gmatch.t | 24 +- t/036-sub.t | 24 +- t/037-gsub.t | 17 +- t/038-match-o.t | 7 +- t/047-match-jit.t | 48 +++- t/049-gmatch-jit.t | 67 ++++- t/050-gmatch-dfa.t | 7 +- t/051-sub-jit.t | 44 ++- t/052-sub-dfa.t | 14 +- t/053-gsub-jit.t | 44 ++- t/054-gsub-dfa.t | 14 +- t/120-re-find.t | 25 +- t/cert/test.crt | 35 +-- t/cert/test.key | 43 +-- util/build-with-dd.sh | 6 +- util/build-without-ssl.sh | 6 +- util/build.sh | 6 +- valgrind.suppress | 70 +++++ 26 files changed, 998 insertions(+), 213 deletions(-) diff --git a/.travis.yml b/.travis.yml index 19f560a4fe..b29ab4cc2a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -38,9 +38,13 @@ env: - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - PCRE_VER=8.45 + - PCRE2_VER=10.37 - PCRE_PREFIX=/opt/pcre + - PCRE2_PREFIX=/opt/pcre2 - PCRE_LIB=$PCRE_PREFIX/lib + - PCRE2_LIB=$PCRE2_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include + - PCRE2_INC=$PCRE2_PREFIX/include - OPENSSL_PREFIX=/opt/ssl - OPENSSL_LIB=$OPENSSL_PREFIX/lib - OPENSSL_INC=$OPENSSL_PREFIX/include @@ -55,7 +59,7 @@ env: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y services: - memcached @@ -71,7 +75,8 @@ before_install: - pyenv global 2.7 install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - - if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi + - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi + - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty @@ -121,12 +126,8 @@ script: - sudo make install-libdrizzle-1.0 > build.log 2>&1 || (cat build.log && exit 1) - cd ../mockeagain/ && make CC=$CC -j$JOBS && cd .. - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. - - tar zxf download-cache/pcre-$PCRE_VER.tar.gz - - cd pcre-$PCRE_VER/ - - ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1) - - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - - sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1) - - cd .. + - if [ "$USE_PCRE2" != "Y" ]; then tar zxf download-cache/pcre-$PCRE_VER.tar.gz; cd pcre-$PCRE_VER/; ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + - if [ "$USE_PCRE2" = "Y" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - cd openssl-$OPENSSL_VER/ - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 7d9e10cea1..541fa881db 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -63,6 +63,10 @@ typedef struct { # endif #endif +#if (NGX_PCRE2) +# define LUA_HAVE_PCRE_JIT 1 +#endif + #if (nginx_version < 1006000) # error at least nginx 1.6.0 is required but found an older version @@ -217,11 +221,13 @@ struct ngx_http_lua_main_conf_s { ngx_hash_t builtin_headers_out; -#if (NGX_PCRE) +#if (NGX_PCRE || NGX_PCRE2) ngx_int_t regex_cache_entries; ngx_int_t regex_cache_max_entries; ngx_int_t regex_match_limit; -# if (LUA_HAVE_PCRE_JIT) +#if (NGX_PCRE2) + pcre2_jit_stack *jit_stack; +#elif (LUA_HAVE_PCRE_JIT) pcre_jit_stack *jit_stack; # endif #endif diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 4d47e83e20..6984fb087d 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -59,6 +59,9 @@ static char *ngx_http_lua_ssl_conf_command_check(ngx_conf_t *cf, void *post, #endif static char *ngx_http_lua_malloc_trim(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +#if (NGX_PCRE2) +extern void ngx_http_lua_regex_cleanup(void *data); +#endif static ngx_conf_post_t ngx_http_lua_lowat_post = @@ -855,6 +858,17 @@ ngx_http_lua_init(ngx_conf_t *cf) cln->data = lmcf; cln->handler = ngx_http_lua_sema_mm_cleanup; +#if (NGX_PCRE2) + /* add the cleanup of pcre2 regex */ + cln = ngx_pool_cleanup_add(cf->pool, 0); + if (cln == NULL) { + return NGX_ERROR; + } + + cln->data = lmcf; + cln->handler = ngx_http_lua_regex_cleanup; +#endif + #ifdef HAVE_NGX_LUA_PIPE ngx_http_lua_pipe_init(); #endif diff --git a/src/ngx_http_lua_pcrefix.c b/src/ngx_http_lua_pcrefix.c index 562847a639..76e745e6db 100644 --- a/src/ngx_http_lua_pcrefix.c +++ b/src/ngx_http_lua_pcrefix.c @@ -14,19 +14,65 @@ #include "ngx_http_lua_pcrefix.h" #include "stdio.h" -#if (NGX_PCRE) +#if (NGX_PCRE || NGX_PCRE2) static ngx_pool_t *ngx_http_lua_pcre_pool = NULL; + +#if (NGX_PCRE2) +static ngx_uint_t ngx_regex_direct_alloc; +#else static void *(*old_pcre_malloc)(size_t); static void (*old_pcre_free)(void *ptr); +#endif /* XXX: work-around to nginx regex subsystem, must init a memory pool * to use PCRE functions. As PCRE still has memory-leaking problems, * and nginx overwrote pcre_malloc/free hooks with its own static * functions, so nobody else can reuse nginx regex subsystem... */ -static void * +#if (NGX_PCRE2) + +void * +ngx_http_lua_pcre_malloc(size_t size, void *data) +{ + dd("lua pcre pool is %p", ngx_http_lua_pcre_pool); + + if (ngx_http_lua_pcre_pool) { + return ngx_palloc(ngx_http_lua_pcre_pool, size); + } + + if (ngx_regex_direct_alloc) { + return ngx_alloc(size, ngx_cycle->log); + } + + fprintf(stderr, "error: lua pcre malloc failed due to empty pcre pool"); + + return NULL; +} + + +void +ngx_http_lua_pcre_free(void *ptr, void *data) +{ + dd("lua pcre pool is %p", ngx_http_lua_pcre_pool); + + if (ngx_http_lua_pcre_pool) { + ngx_pfree(ngx_http_lua_pcre_pool, ptr); + return; + } + + if (ngx_regex_direct_alloc) { + ngx_free(ptr); + return; + } + + fprintf(stderr, "error: lua pcre free failed due to empty pcre pool"); +} + +#else + +void * ngx_http_lua_pcre_malloc(size_t size) { dd("lua pcre pool is %p", ngx_http_lua_pcre_pool); @@ -54,6 +100,41 @@ ngx_http_lua_pcre_free(void *ptr) fprintf(stderr, "error: lua pcre free failed due to empty pcre pool"); } +#endif + + +#if (NGX_PCRE2) + +ngx_pool_t * +ngx_http_lua_pcre_malloc_init(ngx_pool_t *pool) +{ + ngx_pool_t *old_pool; + + dd("lua pcre pool was %p", ngx_http_lua_pcre_pool); + + ngx_regex_direct_alloc = (pool == NULL) ? 1 : 0; + + old_pool = ngx_http_lua_pcre_pool; + ngx_http_lua_pcre_pool = pool; + + dd("lua pcre pool is %p", ngx_http_lua_pcre_pool); + + return old_pool; +} + + +void +ngx_http_lua_pcre_malloc_done(ngx_pool_t *old_pool) +{ + dd("lua pcre pool was %p", ngx_http_lua_pcre_pool); + + ngx_http_lua_pcre_pool = old_pool; + ngx_regex_direct_alloc = 0; + + dd("lua pcre pool is %p", ngx_http_lua_pcre_pool); +} + +#else ngx_pool_t * ngx_http_lua_pcre_malloc_init(ngx_pool_t *pool) @@ -101,6 +182,7 @@ ngx_http_lua_pcre_malloc_done(ngx_pool_t *old_pool) } } -#endif /* NGX_PCRE */ +#endif +#endif /* NGX_PCRE || NGX_PCRE2 */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_pcrefix.h b/src/ngx_http_lua_pcrefix.h index 80f29f9e75..ef09636313 100644 --- a/src/ngx_http_lua_pcrefix.h +++ b/src/ngx_http_lua_pcrefix.h @@ -12,9 +12,16 @@ #include "ngx_http_lua_common.h" -#if (NGX_PCRE) +#if (NGX_PCRE || NGX_PCRE2) + ngx_pool_t *ngx_http_lua_pcre_malloc_init(ngx_pool_t *pool); void ngx_http_lua_pcre_malloc_done(ngx_pool_t *old_pool); + +#if NGX_PCRE2 +void *ngx_http_lua_pcre_malloc(size_t size, void *data); +void ngx_http_lua_pcre_free(void *ptr, void *data); +#endif + #endif diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 5c17cfe09c..7317451f65 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -9,21 +9,31 @@ #endif #include "ddebug.h" - -#if (NGX_PCRE) +#if (NGX_PCRE || NGX_PCRE2) #include "ngx_http_lua_pcrefix.h" #include "ngx_http_lua_script.h" #include "ngx_http_lua_util.h" -#if (PCRE_MAJOR >= 6) +#if (PCRE_MAJOR >= 6 || NGX_PCRE2) # define LUA_HAVE_PCRE_DFA 1 #else # define LUA_HAVE_PCRE_DFA 0 #endif +#if (NGX_PCRE2) +static pcre2_compile_context *ngx_regex_compile_context; +static pcre2_match_context *ngx_regex_match_context; +static pcre2_match_data *ngx_regex_match_data; +static ngx_uint_t ngx_regex_match_data_size = 0; + +#define PCRE2_VERSION_SIZE 64 +static char ngx_pcre2_version[PCRE2_VERSION_SIZE]; +#endif + + #define NGX_LUA_RE_MODE_DFA (1<<1) #define NGX_LUA_RE_MODE_JIT (1<<2) #define NGX_LUA_RE_NO_UTF8_CHECK (1<<4) @@ -42,8 +52,17 @@ typedef struct { int ncaptures; int *captures; +#if (NGX_PCRE2) + pcre2_code *regex; + /* + * pcre2 doesn't use pcre_extra any more, + * just for keeping same memory layout in the lua ffi cdef + */ + void *regex_sd; +#else pcre *regex; pcre_extra *regex_sd; +#endif ngx_http_lua_complex_value_t *replace; @@ -57,7 +76,11 @@ typedef struct { ngx_pool_t *pool; ngx_int_t options; +#if (NGX_PCRE2) + pcre2_code *regex; +#else pcre *regex; +#endif int captures; ngx_str_t err; } ngx_http_lua_regex_compile_t; @@ -65,8 +88,12 @@ typedef struct { typedef struct { ngx_http_request_t *request; +#if (NGX_PCRE2) + pcre2_code *regex; +#else pcre *regex; pcre_extra *regex_sd; +#endif int ncaptures; int *captures; int captures_len; @@ -74,8 +101,6 @@ typedef struct { } ngx_http_lua_regex_ctx_t; -static void ngx_http_lua_regex_free_study_data(ngx_pool_t *pool, - pcre_extra *sd); static ngx_int_t ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc); @@ -91,21 +116,155 @@ static ngx_int_t ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc); static void -ngx_http_lua_regex_free_study_data(ngx_pool_t *pool, pcre_extra *sd) +ngx_http_lua_regex_free_study_data(ngx_pool_t *pool, ngx_http_lua_regex_t *re) { - ngx_pool_t *old_pool; + ngx_pool_t *old_pool; - old_pool = ngx_http_lua_pcre_malloc_init(pool); +#if (NGX_PCRE2) + if (re && re->regex) { + old_pool = ngx_http_lua_pcre_malloc_init(pool); + + pcre2_code_free(re->regex); + + ngx_http_lua_pcre_malloc_done(old_pool); + re->regex = NULL; + } +#else + if (re && re->regex_sd) { + old_pool = ngx_http_lua_pcre_malloc_init(pool); #if LUA_HAVE_PCRE_JIT - pcre_free_study(sd); + pcre_free_study(re->regex_sd); #else - pcre_free(sd); + pcre_free(re->regex_sd); #endif + ngx_http_lua_pcre_malloc_done(old_pool); + + re->regex_sd = NULL; + } +#endif +} + + +#if (NGX_PCRE2) +static ngx_int_t +ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc) +{ + int n, errcode; + char *p; + size_t erroff; + u_char errstr[128]; + pcre2_code *re; + ngx_pool_t *old_pool; + pcre2_general_context *gctx; + pcre2_compile_context *cctx; + + ngx_http_lua_main_conf_t *lmcf; + + if (ngx_regex_compile_context == NULL) { + /* + * Allocate a compile context if not yet allocated. This uses + * direct allocations from heap, so the result can be cached + * even at runtime. + */ + + old_pool = ngx_http_lua_pcre_malloc_init(NULL); + + gctx = pcre2_general_context_create(ngx_http_lua_pcre_malloc, + ngx_http_lua_pcre_free, + NULL); + if (gctx == NULL) { + ngx_http_lua_pcre_malloc_done(old_pool); + goto nomem; + } + + cctx = pcre2_compile_context_create(gctx); + if (cctx == NULL) { + pcre2_general_context_free(gctx); + ngx_http_lua_pcre_malloc_done(old_pool); + goto nomem; + } + + ngx_regex_compile_context = cctx; + + ngx_regex_match_context = pcre2_match_context_create(gctx); + if (ngx_regex_match_context == NULL) { + pcre2_general_context_free(gctx); + ngx_http_lua_pcre_malloc_done(old_pool); + goto nomem; + } + + lmcf = ngx_http_cycle_get_module_main_conf(ngx_cycle, + ngx_http_lua_module); + if (lmcf && lmcf->regex_match_limit > 0) { + pcre2_set_match_limit(ngx_regex_match_context, + lmcf->regex_match_limit); + } + + pcre2_general_context_free(gctx); + ngx_http_lua_pcre_malloc_done(old_pool); + } + + old_pool = ngx_http_lua_pcre_malloc_init(rc->pool); + + re = pcre2_compile(rc->pattern.data, + rc->pattern.len, rc->options, + &errcode, &erroff, ngx_regex_compile_context); ngx_http_lua_pcre_malloc_done(old_pool); + + if (re == NULL) { + pcre2_get_error_message(errcode, errstr, 128); + + if ((size_t) erroff == rc->pattern.len) { + rc->err.len = ngx_snprintf(rc->err.data, rc->err.len, + "pcre2_compile() failed: %s in \"%V\"", + errstr, &rc->pattern) + - rc->err.data; + + } else { + rc->err.len = ngx_snprintf(rc->err.data, rc->err.len, + "pcre2_compile() failed: %s in " + "\"%V\" at \"%s\"", errstr, &rc->pattern, + rc->pattern.data + erroff) + - rc->err.data; + } + + return NGX_ERROR; + } + + rc->regex = re; + + n = pcre2_pattern_info(re, PCRE2_INFO_CAPTURECOUNT, &rc->captures); + if (n < 0) { + p = "pcre2_pattern_info(\"%V\", PCRE_INFO_CAPTURECOUNT) failed: %d"; + goto failed; + } + +#if (NGX_DEBUG) + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "pcre2_compile: pattern[%V], options 0x%08Xd, ncaptures %d", + &rc->pattern, rc->options, rc->captures); +#endif + + return NGX_OK; + +failed: + + rc->err.len = ngx_snprintf(rc->err.data, rc->err.len, p, &rc->pattern, n) + - rc->err.data; + return NGX_ERROR; + +nomem: + + rc->err.len = ngx_snprintf(rc->err.data, rc->err.len, + "regex \"%V\" compilation failed: no memory", + &rc->pattern) + - rc->err.data; + return NGX_ERROR; } +#else static ngx_int_t ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc) @@ -159,13 +318,14 @@ ngx_http_lua_regex_compile(ngx_http_lua_regex_compile_t *rc) - rc->err.data; return NGX_OK; } +#endif ngx_int_t ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, size_t *errstr_size) { -#if LUA_HAVE_PCRE_JIT +#if (NGX_PCRE2 || LUA_HAVE_PCRE_JIT) ngx_http_lua_main_conf_t *lmcf; ngx_pool_t *pool, *old_pool; @@ -186,15 +346,24 @@ ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, if (lmcf->jit_stack) { old_pool = ngx_http_lua_pcre_malloc_init(pool); +#if (NGX_PCRE2) + pcre2_jit_stack_free(lmcf->jit_stack); +#else pcre_jit_stack_free(lmcf->jit_stack); +#endif ngx_http_lua_pcre_malloc_done(old_pool); } old_pool = ngx_http_lua_pcre_malloc_init(pool); +#if (NGX_PCRE2) + lmcf->jit_stack = pcre2_jit_stack_create(NGX_LUA_RE_MIN_JIT_STACK_SIZE, + size, NULL); +#else lmcf->jit_stack = pcre_jit_stack_alloc(NGX_LUA_RE_MIN_JIT_STACK_SIZE, size); +#endif ngx_http_lua_pcre_malloc_done(old_pool); @@ -214,8 +383,148 @@ ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, - errstr; return NGX_ERROR; +#endif +} + + +#if (NGX_PCRE2) +static void +ngx_http_lua_regex_jit_compile(ngx_http_lua_regex_t *re, int flags, + ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, + ngx_http_lua_regex_compile_t *re_comp) +{ + ngx_int_t ret; + ngx_pool_t *old_pool; + + if (flags & NGX_LUA_RE_MODE_JIT) { + old_pool = ngx_http_lua_pcre_malloc_init(pool); + ret = pcre2_jit_compile(re_comp->regex, PCRE2_JIT_COMPLETE); + + if (ret != 0) { + ngx_log_error(NGX_LOG_INFO, ngx_cycle->log, 0, + "pcre2_jit_compile() failed: %d in \"%V\", " + "ignored", + ret, &re_comp->pattern); + +#if (NGX_DEBUG) + + } else { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "pcre2 JIT compiled successfully"); +# endif /* !(NGX_DEBUG) */ + } + + ngx_http_lua_pcre_malloc_done(old_pool); + + } + + if (lmcf && lmcf->jit_stack) { + pcre2_jit_stack_assign(ngx_regex_match_context, NULL, + lmcf->jit_stack); + } + + return; +} + +#else + +static void +ngx_http_lua_regex_jit_compile(ngx_http_lua_regex_t *re, int flags, + ngx_pool_t *pool, ngx_http_lua_main_conf_t *lmcf, + ngx_http_lua_regex_compile_t *re_comp) +{ + const char *msg; + pcre_extra *sd = NULL; + ngx_pool_t *old_pool; + + +#if (LUA_HAVE_PCRE_JIT) + if (flags & NGX_LUA_RE_MODE_JIT) { + old_pool = ngx_http_lua_pcre_malloc_init(pool); + sd = pcre_study(re_comp->regex, PCRE_STUDY_JIT_COMPILE, &msg); + ngx_http_lua_pcre_malloc_done(old_pool); + +# if (NGX_DEBUG) + if (msg != NULL) { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "pcre study failed with PCRE_STUDY_JIT_COMPILE: " + "%s (%p)", msg, sd); + } + + if (sd != NULL) { + int jitted; + + old_pool = ngx_http_lua_pcre_malloc_init(pool); + + pcre_fullinfo(re_comp->regex, sd, PCRE_INFO_JIT, &jitted); + + ngx_http_lua_pcre_malloc_done(old_pool); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "pcre JIT compiling result: %d", jitted); + } +# endif /* !(NGX_DEBUG) */ + + } else { + old_pool = ngx_http_lua_pcre_malloc_init(pool); + sd = pcre_study(re_comp->regex, 0, &msg); + ngx_http_lua_pcre_malloc_done(old_pool); + } + + if (sd && lmcf && lmcf->jit_stack) { + pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); + } + + if (sd + && lmcf && lmcf->regex_match_limit > 0 + && !(flags & NGX_LUA_RE_MODE_DFA)) + { + sd->flags |= PCRE_EXTRA_MATCH_LIMIT; + sd->match_limit = lmcf->regex_match_limit; + } + #endif /* LUA_HAVE_PCRE_JIT */ + + re->regex_sd = sd; } +#endif + + +#if (NGX_PCRE2) +void +ngx_http_lua_regex_cleanup(void *data) +{ + ngx_pool_t *old_pool; + ngx_http_lua_main_conf_t *lmcf; + + lmcf = data; + + if (ngx_regex_compile_context) { + old_pool = ngx_http_lua_pcre_malloc_init(NULL); + pcre2_compile_context_free(ngx_regex_compile_context); + ngx_regex_compile_context = NULL; + ngx_http_lua_pcre_malloc_done(old_pool); + } + + if (lmcf && lmcf->jit_stack) { + old_pool = ngx_http_lua_pcre_malloc_init(NULL); + + pcre2_jit_stack_free(lmcf->jit_stack); + lmcf->jit_stack = NULL; + + ngx_http_lua_pcre_malloc_done(old_pool); + } + + if (ngx_regex_match_data) { + old_pool = ngx_http_lua_pcre_malloc_init(NULL); + pcre2_match_data_free(ngx_regex_match_data); + ngx_regex_match_data = NULL; + ngx_regex_match_data_size = 0; + ngx_http_lua_pcre_malloc_done(old_pool); + } + +} +#endif ngx_http_lua_regex_t * @@ -228,8 +537,7 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, ngx_int_t rc; const char *msg; ngx_pool_t *pool, *old_pool; - pcre_extra *sd = NULL; - ngx_http_lua_regex_t *re; + ngx_http_lua_regex_t *re = NULL; ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_regex_compile_t re_comp; @@ -251,6 +559,8 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, } re->pool = pool; + re->regex = NULL; + re->regex_sd = NULL; re_comp.options = pcre_opts; re_comp.pattern.data = (u_char *) pat; @@ -274,54 +584,7 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, ngx_http_lua_assert(lmcf != NULL); -#if (LUA_HAVE_PCRE_JIT) - - if (flags & NGX_LUA_RE_MODE_JIT) { - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - sd = pcre_study(re_comp.regex, PCRE_STUDY_JIT_COMPILE, &msg); - ngx_http_lua_pcre_malloc_done(old_pool); - -# if (NGX_DEBUG) - if (msg != NULL) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "pcre study failed with PCRE_STUDY_JIT_COMPILE: " - "%s (%p)", msg, sd); - } - - if (sd != NULL) { - int jitted; - - old_pool = ngx_http_lua_pcre_malloc_init(pool); - - pcre_fullinfo(re_comp.regex, sd, PCRE_INFO_JIT, &jitted); - - ngx_http_lua_pcre_malloc_done(old_pool); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "pcre JIT compiling result: %d", jitted); - } -# endif /* !(NGX_DEBUG) */ - - } else { - old_pool = ngx_http_lua_pcre_malloc_init(pool); - sd = pcre_study(re_comp.regex, 0, &msg); - ngx_http_lua_pcre_malloc_done(old_pool); - } - - if (sd && lmcf->jit_stack) { - pcre_assign_jit_stack(sd, NULL, lmcf->jit_stack); - } - -#endif /* LUA_HAVE_PCRE_JIT */ - - if (sd - && lmcf && lmcf->regex_match_limit > 0 - && !(flags & NGX_LUA_RE_MODE_DFA)) - { - sd->flags |= PCRE_EXTRA_MATCH_LIMIT; - sd->match_limit = lmcf->regex_match_limit; - } + ngx_http_lua_regex_jit_compile(re, flags, pool, lmcf, &re_comp); if (flags & NGX_LUA_RE_MODE_DFA) { ovecsize = 2; @@ -339,6 +602,31 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, goto error; } +#if (NGX_PCRE2) + if (pcre2_pattern_info(re_comp.regex, PCRE2_INFO_NAMECOUNT, + &re->name_count) < 0) + { + msg = "cannot acquire named subpattern count"; + goto error; + } + + if (re->name_count > 0) { + if (pcre2_pattern_info(re_comp.regex, PCRE2_INFO_NAMEENTRYSIZE, + &re->name_entry_size) != 0) + { + msg = "cannot acquire named subpattern entry size"; + goto error; + } + + if (pcre2_pattern_info(re_comp.regex, PCRE2_INFO_NAMETABLE, + &re->name_table) != 0) + { + msg = "cannot acquire named subpattern table"; + goto error; + } + } + +#else if (pcre_fullinfo(re_comp.regex, NULL, PCRE_INFO_NAMECOUNT, &re->name_count) != 0) { @@ -361,9 +649,9 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, goto error; } } +#endif re->regex = re_comp.regex; - re->regex_sd = sd; re->ncaptures = re_comp.captures; re->captures = cap; re->replace = NULL; @@ -379,9 +667,7 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, p = ngx_snprintf(errstr, errstr_size - 1, "%s", msg); *p = '\0'; - if (sd) { - ngx_http_lua_regex_free_study_data(pool, sd); - } + ngx_http_lua_regex_free_study_data(pool, re); if (pool) { ngx_destroy_pool(pool); @@ -391,6 +677,103 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, } +#if (NGX_PCRE2) +int +ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, + const u_char *s, size_t len, int pos) +{ + int rc, exec_opts = 0; + size_t *ov; + ngx_uint_t ovecsize, n, i; + ngx_pool_t *old_pool; + + if (flags & NGX_LUA_RE_MODE_DFA) { + ovecsize = 2; + re->ncaptures = 0; + + } else { + ovecsize = (re->ncaptures + 1) * 3; + } + + old_pool = ngx_http_lua_pcre_malloc_init(NULL); + + if (ngx_regex_match_data == NULL + || ovecsize > ngx_regex_match_data_size) + { + /* + * Allocate a match data if not yet allocated or smaller than + * needed. + */ + + if (ngx_regex_match_data) { + pcre2_match_data_free(ngx_regex_match_data); + } + + ngx_regex_match_data_size = ovecsize; + ngx_regex_match_data = pcre2_match_data_create(ovecsize / 3, NULL); + + if (ngx_regex_match_data == NULL) { + rc = PCRE2_ERROR_NOMEMORY; + goto failed; + } + } + + if (flags & NGX_LUA_RE_NO_UTF8_CHECK) { + exec_opts = PCRE2_NO_UTF_CHECK; + + } else { + exec_opts = 0; + } + + if (flags & NGX_LUA_RE_MODE_DFA) { + int ws[NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT]; + rc = pcre2_dfa_match(re->regex, s, len, pos, exec_opts, + ngx_regex_match_data, ngx_regex_match_context, + ws, sizeof(ws) / sizeof(ws[0])); + + + } else { + rc = pcre2_match(re->regex, s, len, pos, exec_opts, + ngx_regex_match_data, ngx_regex_match_context); + } + + if (rc < 0) { +#if (NGX_DEBUG) + ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "pcre2_match failed: flags 0x%05Xd, options 0x%08Xd, " + "rc %d, ovecsize %ui", flags, exec_opts, rc, ovecsize); +#endif + + goto failed; + } + + n = pcre2_get_ovector_count(ngx_regex_match_data); + ov = pcre2_get_ovector_pointer(ngx_regex_match_data); + +#if (NGX_DEBUG) + ngx_log_debug5(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "pcre2_match: flags 0x%05Xd, options 0x%08Xd, rc %d, " + "n %ui, ovecsize %ui", flags, exec_opts, rc, n, ovecsize); +#endif + + if (!(flags & NGX_LUA_RE_MODE_DFA) && n > ovecsize / 3) { + n = ovecsize / 3; + } + + for (i = 0; i < n; i++) { + re->captures[i * 2] = ov[i * 2]; + re->captures[i * 2 + 1] = ov[i * 2 + 1]; + } + +failed: + + ngx_http_lua_pcre_malloc_done(old_pool); + + return rc; +} + +#else + int ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, const u_char *s, size_t len, int pos) @@ -427,7 +810,8 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, int ws[NGX_LUA_RE_DFA_MODE_WORKSPACE_COUNT]; rc = ngx_http_lua_regex_dfa_exec(re->regex, sd, &subj, (int) pos, cap, ovecsize, ws, - sizeof(ws)/sizeof(ws[0]), exec_opts); + sizeof(ws) / sizeof(ws[0]), + exec_opts); #else @@ -443,28 +827,19 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, return rc; } +#endif + void ngx_http_lua_ffi_destroy_regex(ngx_http_lua_regex_t *re) { - ngx_pool_t *old_pool; - dd("destroy regex called"); if (re == NULL || re->pool == NULL) { return; } - if (re->regex_sd) { - old_pool = ngx_http_lua_pcre_malloc_init(re->pool); -#if LUA_HAVE_PCRE_JIT - pcre_free_study(re->regex_sd); -#else - pcre_free(re->regex_sd); -#endif - ngx_http_lua_pcre_malloc_done(old_pool); - re->regex_sd = NULL; - } + ngx_http_lua_regex_free_study_data(re->pool, re); ngx_destroy_pool(re->pool); } @@ -592,11 +967,17 @@ ngx_http_lua_ffi_max_regex_cache_size(void) const char * ngx_http_lua_ffi_pcre_version(void) { +#if (NGX_PCRE2) + pcre2_config(PCRE2_CONFIG_VERSION, ngx_pcre2_version); + + return ngx_pcre2_version; +#else return pcre_version(); +#endif } -#endif /* NGX_PCRE */ +#endif /* NGX_PCRE || NGX_PCRE2 */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/028-req-header.t b/t/028-req-header.t index f4c393977d..18b991052b 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -275,7 +275,6 @@ Content-Type: GET /bar --- response_body eval # Since nginx version 1.23.0, nginx combines same $http_* variable together -# wtf $Test::Nginx::Util::NginxVersion >= 1.023000 ? "Foo: a, b\n" diff --git a/t/034-match.t b/t/034-match.t index 1bf45bce1a..fd4d9439e2 100644 --- a/t/034-match.t +++ b/t/034-match.t @@ -361,8 +361,11 @@ he } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -648,8 +651,12 @@ regex: (?:>[\w\s]*) } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "([0-9]+" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile\(\) failed: missing closing parenthesis in \"\([0-9]+\"\n" +: +"error: pcre_compile\(\) failed: missing \) in \"\([0-9]+\"\n" + --- no_error_log [error] @@ -939,8 +946,11 @@ nil } --- request GET /t ---- response_body_like chop -^error: pcre_exec\(\) failed: -10$ +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre_exec\(\) failed: -4\n" +: +"error: pcre_exec\(\) failed: -10\n" --- no_error_log [error] @@ -1050,8 +1060,14 @@ end --- request GET /re ---- response_body -error: pcre_exec() failed: -8 +--- response_body eval +# lua_regex_match_limit uses pcre_extra->match_limit in the PCRE, +# but PCRE2 replaces this with pcre2_set_match_limit interface, +# which has different effects. +$Test::Nginx::Util::PcreVersion == 2 ? +"failed to match\n" +: +"error: pcre_exec() failed: -8\n" diff --git a/t/035-gmatch.t b/t/035-gmatch.t index 0dfeaeaf39..0a4efd2b25 100644 --- a/t/035-gmatch.t +++ b/t/035-gmatch.t @@ -698,8 +698,11 @@ not matched! } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -735,8 +738,11 @@ error: pcre_compile() failed: missing ) in "(abc" } --- request GET /t ---- response_body_like chop -error: pcre_exec\(\) failed: -10 +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre_exec\(\) failed: -4\n" +: +"error: pcre_exec\(\) failed: -10\n" --- no_error_log [error] @@ -854,8 +860,14 @@ end --- request GET /re ---- response_body -error: pcre_exec() failed: -8 +--- response_body eval +# lua_regex_match_limit uses pcre_extra->match_limit in the PCRE, +# but PCRE2 replaces this with pcre2_set_match_limit interface, +# which has different effects. +$Test::Nginx::Util::PcreVersion == 2 ? +"failed to match\n" +: +"error: pcre_exec() failed: -8\n" diff --git a/t/036-sub.t b/t/036-sub.t index 3e88eba5a0..0657ebb2ca 100644 --- a/t/036-sub.t +++ b/t/036-sub.t @@ -480,8 +480,11 @@ a [b c] [b] [c] [] [] d } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -506,8 +509,11 @@ error: pcre_compile() failed: missing ) in "(abc" } --- request GET /t ---- response_body_like chop -error: pcre_exec\(\) failed: -10 +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre_exec\(\) failed: -4\n" +: +"error: pcre_exec\(\) failed: -10\n" --- no_error_log [error] @@ -610,8 +616,14 @@ ngx.say("sub: ", cnt) --- request GET /re ---- response_body -error: pcre_exec() failed: -8 +--- response_body eval +# lua_regex_match_limit uses pcre_extra->match_limit in the PCRE, +# but PCRE2 replaces this with pcre2_set_match_limit interface, +# which has different effects. +$Test::Nginx::Util::PcreVersion == 2 ? +"sub: 0\n" +: +"error: pcre_exec() failed: -8\n" diff --git a/t/037-gsub.t b/t/037-gsub.t index 41f86ac013..a6fe579bd4 100644 --- a/t/037-gsub.t +++ b/t/037-gsub.t @@ -423,8 +423,11 @@ n: 1 } --- request GET /t ---- response_body_like chop -error: pcre_exec\(\) failed: -10 +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre_exec\(\) failed: -4\n" +: +"error: pcre_exec\(\) failed: -10\n" --- no_error_log [error] @@ -531,8 +534,14 @@ ngx.say("gsub: ", cnt) --- request GET /re ---- response_body -error: pcre_exec() failed: -8 +--- response_body eval +# lua_regex_match_limit uses pcre_extra->match_limit in the PCRE, +# but PCRE2 replaces this with pcre2_set_match_limit interface, +# which has different effects. +$Test::Nginx::Util::PcreVersion == 2 ? +"gsub: 0\n" +: +"error: pcre_exec() failed: -8\n" diff --git a/t/038-match-o.t b/t/038-match-o.t index f6ae0103fe..cc80244e82 100644 --- a/t/038-match-o.t +++ b/t/038-match-o.t @@ -336,8 +336,11 @@ he } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/047-match-jit.t b/t/047-match-jit.t index f53a4083a2..9b40a889dc 100644 --- a/t/047-match-jit.t +++ b/t/047-match-jit.t @@ -32,8 +32,11 @@ __DATA__ GET /re --- response_body 1234 ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -53,8 +56,11 @@ pcre JIT compiling result: 1 GET /re --- response_body not matched! ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -76,9 +82,15 @@ pcre JIT compiling result: 1 1234 --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -101,9 +113,15 @@ qr/pcre JIT compiling result: \d+/ not matched! --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -128,8 +146,11 @@ qr/pcre JIT compiling result: \d+/ } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -170,8 +191,15 @@ end --- request GET /re ---- response_body -error: pcre_exec() failed: -8 +--- response_body eval +# lua_regex_match_limit uses pcre_extra->match_limit in the PCRE, +# but PCRE2 replaces this with pcre2_set_match_limit interface, +# which has different effects. +$Test::Nginx::Util::PcreVersion == 2 ? +# PCRE2_ERROR_MATCHLIMIT (-47) +"error: pcre_exec() failed: -47\n" +: +"error: pcre_exec() failed: -8\n" diff --git a/t/049-gmatch-jit.t b/t/049-gmatch-jit.t index 5134d526b5..b76df35e98 100644 --- a/t/049-gmatch-jit.t +++ b/t/049-gmatch-jit.t @@ -34,8 +34,11 @@ __DATA__ --- response_body hello world ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -60,8 +63,11 @@ pcre JIT compiling result: 1 nil nil nil ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -77,8 +83,11 @@ pcre JIT compiling result: 1 GET /re --- response_body done ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -99,8 +108,11 @@ pcre JIT compiling result: 1 GET /re --- response_body hello ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -124,9 +136,15 @@ hello world --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -154,9 +172,15 @@ nil nil --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -175,9 +199,15 @@ qr/pcre JIT compiling result: \d+/ done --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -201,9 +231,15 @@ qr/pcre JIT compiling result: \d+/ hello --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -222,7 +258,10 @@ qr/pcre JIT compiling result: \d+/ } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/050-gmatch-dfa.t b/t/050-gmatch-dfa.t index d2ac2bc666..4c5f820e49 100644 --- a/t/050-gmatch-dfa.t +++ b/t/050-gmatch-dfa.t @@ -214,8 +214,11 @@ hello } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/051-sub-jit.t b/t/051-sub-jit.t index c8a49c0516..7aec7ccb79 100644 --- a/t/051-sub-jit.t +++ b/t/051-sub-jit.t @@ -32,8 +32,11 @@ __DATA__ GET /re --- response_body hello, world 5678: 1 ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -53,8 +56,11 @@ pcre JIT compiling result: 1 GET /re --- response_body hello, world: 0 ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -76,9 +82,15 @@ pcre JIT compiling result: 1 hello, world 5678: 1 --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -101,9 +113,15 @@ qr/pcre JIT compiling result: \d+/ hello, world: 0 --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -122,8 +140,11 @@ qr/pcre JIT compiling result: \d+/ } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -143,7 +164,10 @@ error: pcre_compile() failed: missing ) in "(abc" } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/052-sub-dfa.t b/t/052-sub-dfa.t index 254913b3e0..2329c77aa3 100644 --- a/t/052-sub-dfa.t +++ b/t/052-sub-dfa.t @@ -107,8 +107,11 @@ hello, world: 0 } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -129,8 +132,11 @@ error: pcre_compile() failed: missing ) in "(abc" } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/053-gsub-jit.t b/t/053-gsub-jit.t index 3efc0a24a5..c7c87ee29c 100644 --- a/t/053-gsub-jit.t +++ b/t/053-gsub-jit.t @@ -32,8 +32,11 @@ __DATA__ GET /re --- response_body hello, world world: 2 ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -53,8 +56,11 @@ pcre JIT compiling result: 1 GET /re --- response_body hello, world: 0 ---- error_log -pcre JIT compiling result: 1 +--- error_log eval +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully\n" +: +"pcre JIT compiling result: 1\n" @@ -76,9 +82,15 @@ pcre JIT compiling result: 1 hello, world world: 2 --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -101,9 +113,15 @@ qr/pcre JIT compiling result: \d+/ hello, world: 0 --- grep_error_log eval -qr/pcre JIT compiling result: \d+/ +$Test::Nginx::Util::PcreVersion == 2 ? +"pcre2 JIT compiled successfully" +: +"pcre JIT compiling result: 1" --- grep_error_log_out eval +$Test::Nginx::Util::PcreVersion == 2 ? +["pcre2 JIT compiled successfully\n", ""] +: ["pcre JIT compiling result: 1\n", ""] @@ -122,8 +140,11 @@ qr/pcre JIT compiling result: \d+/ } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -143,7 +164,10 @@ error: pcre_compile() failed: missing ) in "(abc" } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/054-gsub-dfa.t b/t/054-gsub-dfa.t index 937aa1c421..bd0825d4ba 100644 --- a/t/054-gsub-dfa.t +++ b/t/054-gsub-dfa.t @@ -107,8 +107,11 @@ hello, world: 0 } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" @@ -126,8 +129,11 @@ error: pcre_compile() failed: missing ) in "(abc" } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] diff --git a/t/120-re-find.t b/t/120-re-find.t index ddf24a98e7..43ccece1ab 100644 --- a/t/120-re-find.t +++ b/t/120-re-find.t @@ -354,8 +354,11 @@ matched: he } --- request GET /re ---- response_body -error: pcre_compile() failed: missing ) in "(abc" +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre2_compile() failed: missing closing parenthesis in \"(abc\"\n" +: +"error: pcre_compile() failed: missing ) in \"(abc\"\n" --- no_error_log [error] @@ -562,8 +565,11 @@ matched: hello, 1234 } --- request GET /t ---- response_body_like chop -^error: pcre_exec\(\) failed: -10$ +--- response_body eval +$Test::Nginx::Util::PcreVersion == 2 ? +"error: pcre_exec\(\) failed: -4\n" +: +"error: pcre_exec\(\) failed: -10\n" --- no_error_log [error] @@ -587,6 +593,7 @@ GET /t '; } --- stap +# TODO: PCRE2 use different option values from PCRE probe process("$LIBPCRE_PATH").function("pcre_compile") { printf("compile opts: %x\n", $options) } @@ -645,8 +652,14 @@ end --- request GET /re ---- response_body -error: pcre_exec() failed: -8 +--- response_body eval +# lua_regex_match_limit uses pcre_extra->match_limit in the PCRE, +# but PCRE2 replaces this with pcre2_set_match_limit interface, +# which has different effects. +$Test::Nginx::Util::PcreVersion == 2 ? +"failed to match.\n" +: +"error: pcre_exec() failed: -8\n" --- no_error_log [error] diff --git a/t/cert/test.crt b/t/cert/test.crt index a69a01141a..9fed1bc3b4 100644 --- a/t/cert/test.crt +++ b/t/cert/test.crt @@ -1,17 +1,22 @@ -----BEGIN CERTIFICATE----- -MIICqTCCAhICCQClDm1WkreW4jANBgkqhkiG9w0BAQUFADCBlzELMAkGA1UEBhMC -VVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28x -EjAQBgNVBAoMCU9wZW5SZXN0eTESMBAGA1UECwwJT3BlblJlc3R5MREwDwYDVQQD -DAh0ZXN0LmNvbTEgMB4GCSqGSIb3DQEJARYRYWdlbnR6aEBnbWFpbC5jb20wIBcN -MTQwNzIxMDMyMzQ3WhgPMjE1MTA2MTMwMzIzNDdaMIGXMQswCQYDVQQGEwJVUzET -MBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzESMBAG -A1UECgwJT3BlblJlc3R5MRIwEAYDVQQLDAlPcGVuUmVzdHkxETAPBgNVBAMMCHRl -c3QuY29tMSAwHgYJKoZIhvcNAQkBFhFhZ2VudHpoQGdtYWlsLmNvbTCBnzANBgkq -hkiG9w0BAQEFAAOBjQAwgYkCgYEA6P18zUvtmaKQK2xePy8ZbFwSyTLw+jW6t9eZ -aiTec8X3ibN9WemrxHzkTRikxP3cAQoITRuZiQvF4Q7DO6wMkz/b0zwfgX5uedGq -047AJP6n/mwlDOjGSNomBLoXQzo7tVe60ikEm3ZyDUqnJPJMt3hImO5XSop4MPMu -Za9WhFcCAwEAATANBgkqhkiG9w0BAQUFAAOBgQA4OBb9bOyWB1//93nSXX1mdENZ -IQeyTK0Dd6My76lnZxnZ4hTWrvvd0b17KLDU6JnS2N5ee3ATVkojPidRLWLIhnh5 -0eXrcKalbO2Ce6nShoFvQCQKXN2Txmq2vO/Mud2bHAWwJALg+qi1Iih/gVYB9sct -FLg8zFOzRlYiU+6Mmw== +MIIDtzCCAp8CFCJnLifDCaXjYb2ARKBBhs+aAgYOMA0GCSqGSIb3DQEBCwUAMIGX +MQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2Fu +IEZyYW5jaXNjbzESMBAGA1UECgwJT3BlblJlc3R5MRIwEAYDVQQLDAlPcGVuUmVz +dHkxETAPBgNVBAMMCHRlc3QuY29tMSAwHgYJKoZIhvcNAQkBFhFhZ2VudHpoQGdt +YWlsLmNvbTAeFw0yMzA5MDUwNDE5MjhaFw0zMzA5MDIwNDE5MjhaMIGXMQswCQYD +VQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5j +aXNjbzESMBAGA1UECgwJT3BlblJlc3R5MRIwEAYDVQQLDAlPcGVuUmVzdHkxETAP +BgNVBAMMCHRlc3QuY29tMSAwHgYJKoZIhvcNAQkBFhFhZ2VudHpoQGdtYWlsLmNv +bTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJzRMFoLDuYOwJ8szrS4 +nOibtiimiXZJGx3I/RcFZxaH4nL/WcEb1fwftMQxx73IBJnqnDYkDOzUmzItPMn0 +t2WrNYesC5GqLNRm87m6PVt010tZvq/WxTn6+9qruiGm1PhFxzLQfrClpEeOshlG +UeoQjPOMrhCmofDM2NQo3D4wIQT0kCJxIPq6wCZt22/Yqz1EmR0UnF/R3ZtiB8O+ +SQGcsUKy4se3919xq+ZkzBdMxLneO5sofUiDC9MgRfiU960tbHPGX9I9P+kLK89S +yajPEYaRUkSBFjV5kdDK3+L6XckdMbY2pvwhAnVXSmd13Bf2V9XisUrX2Mr4YlnS +sy0CAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAVPY/z6Mvjg5EGHzU8bXyuXqxrx8Q +GBwf3PY25aDF6ofRrTCzMdIhthv8eRtGwHinkpgaK34D7hI/dPB7aswQTzED5c+l +S2au5OzzCj454oXdhSRA5Rt0mu/+pxmQ+iNk+7XJxgTN0mk1dYQqodyZ+vC4NIYb +javMlU4zDm4JPtwDs0Mz/d7gf14MU60jppF2vl6AYFHKYBLMHBmqxjy6H9YHjRjQ +oe4TNpn0zxJAPu5LqMkfB2+eLOe6ced7DcLLbbeVJ4Xtqj6Y5KsAyVojWQxrk4vW +3WO/953pHofO5F2ricS/rsf+5ivTmfiP8mQYTtp7k3T11sIZ4DOmtNwO4A== -----END CERTIFICATE----- diff --git a/t/cert/test.key b/t/cert/test.key index 6c13527117..4c8c905b2f 100644 --- a/t/cert/test.key +++ b/t/cert/test.key @@ -1,15 +1,28 @@ ------BEGIN RSA PRIVATE KEY----- -MIICXgIBAAKBgQDo/XzNS+2ZopArbF4/LxlsXBLJMvD6Nbq315lqJN5zxfeJs31Z -6avEfORNGKTE/dwBCghNG5mJC8XhDsM7rAyTP9vTPB+Bfm550arTjsAk/qf+bCUM -6MZI2iYEuhdDOju1V7rSKQSbdnINSqck8ky3eEiY7ldKingw8y5lr1aEVwIDAQAB -AoGBANgB66sKMga2SKN5nQdHS3LDCkevCutu1OWM5ZcbB4Kej5kC57xsf+tzPtab -emeIVGhCPOAALqB4YcT+QtMX967oM1MjcFbtH7si5oq6UYyp3i0G9Si6jIoVHz3+ -8yOUaqwKbK+bRX8VS0YsHZmBsPK5ryN50iUwsU08nemoA94BAkEA9GS9Q5OPeFkM -tFxsIQ1f2FSsZAuN/1cpZgJqY+YaAN7MSPGTWyfd7nWG/Zgk3GO9/2ihh4gww+7B -To09GkmW4QJBAPQOHC2V+t2TA98+6Lj6+TYwcGEkhOENfVpH25mQ+kXgF/1Bd6rA -nosT1bdAY+SnmWXbSw6Kv5C20Em+bEX8WjcCQCSRRjhsRdVODbaW9Z7kb2jhEoJN -sEt6cTlQNzcHYPCsZYisjM3g4zYg47fiIfHQAsfKkhDDcfh/KvFj9LaQOEECQQCH -eBWYEDpSJ7rsfqT7mQQgWj7nDThdG/nK1TxGP71McBmg0Gg2dfkLRhVJRQqt74Is -kc9V4Rp4n6F6baL4Lh19AkEA6pZZer0kg3Kv9hjhaITIKUYdfIp9vYnDRWbQlBmR -atV8V9u9q2ETZvqfHpN+9Lu6NYR4yXIEIRf1bnIZ/mr9eQ== ------END RSA PRIVATE KEY----- +-----BEGIN PRIVATE KEY----- +MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCc0TBaCw7mDsCf +LM60uJzom7Yopol2SRsdyP0XBWcWh+Jy/1nBG9X8H7TEMce9yASZ6pw2JAzs1Jsy +LTzJ9LdlqzWHrAuRqizUZvO5uj1bdNdLWb6v1sU5+vvaq7ohptT4Rccy0H6wpaRH +jrIZRlHqEIzzjK4QpqHwzNjUKNw+MCEE9JAicSD6usAmbdtv2Ks9RJkdFJxf0d2b +YgfDvkkBnLFCsuLHt/dfcavmZMwXTMS53jubKH1IgwvTIEX4lPetLWxzxl/SPT/p +CyvPUsmozxGGkVJEgRY1eZHQyt/i+l3JHTG2Nqb8IQJ1V0pnddwX9lfV4rFK19jK ++GJZ0rMtAgMBAAECggEABjaOkcllis1o/yrVZMPPabLpAHV6tZ5MuKfNiUOMSPr+ +HfF1OFQL7MxCdfyFQ1prqOp/9nAut+puMgp99wAfDQ7qanNGq7vgQKkfPSD+dy4V +rUquELBJH6nh9SZqfpSqKaJgHlNe6vehHuRYikJRkrJwVzegGjuekm3B+y6Zl/gc +e0p5Ha3MTLTFjocwYzgTjJlxD40wlbjpuVnmzKjo8AKNv1F1azMaqBmt1VfPiDn0 +Xyq4SPEsWKnEAl2kZdaIBR6zIx7Z3zNUwkfb32QwNoSyo8wS7lCgf2GVS7r1Eul6 +iiCE/Gd7w10alW4Pu96shVqkvKn7ROF2nBP9xOSPwQKBgQDCuD6mlNpA07iOX364 +aAzIAYookceVA0I9L/fbOQW7RgpvYpM8lxr31TQ3fBDkXSgjzMMYjnk4kz+xN+BB +WFdjb4raUBtrvip8Q8QZ53DVQK/LodHh0XhipbOxZrDm+6o5nQD0fTqHCBIHSVFF +tXX2Y90t1cxWMMleRhfNEuzkQQKBgQDOK0rs7mf04Xhc4ZIRIxOtNFnthGp4Kqp7 +SD8VQpbPOLV8iqZEtXIy/hvoTpfQW30c1931KgDQ3Pv5MZYpI7PLqrqkj4tGCQ91 +DJ03GWkSXcMwlPmJRbvgWIeCLgShU5PLxmQu3mH2DP+uGFUBq5/6miDDVjF9z6vb +BwYlG66j7QKBgA0n/bOrowN2SqXz9c/n19U7pWYQU3fR/Iu9zfVV6Pk6RkI4WtJh +M0VDdn+5Njr3wFqK3zOtjKsx57/FkrVXjq/9PVh6yR+CfcRfn8RQSuNdt4L+r/ud +95BSuc1mrtUsc9for8PVIjs1ZGJxpbgcBphbLvqF04SPT0u7WKhWewMBAoGAcJO/ +RAUiitsbaExcADORKQDvIf0uThOuJ8dZevhzdQ/YOftTsy0JAMM05fMUfteWR8uw +DZE0BNjGVlo3TpuKL+o4JGele0azRAzxRAcCEt9UGBEg+U40utpclD8glB8ZEypv +xg/0mfCbJKtwr4rRvnuu7DsCp1pg0ybQui6VfDkCgYBXHwcrZwmv7kgr4pUG6oZj +fzjFenQFqibvb2h7QESyCW13O885GxU13DKv4zg1yi6EqPIopz16qCiUNCvWr5Us +6sI74wEVI3MzmzG0Htgl29q5yWpeY+7libC/fbZYG8GFgdINq58ko9be1u/8644S +t2hoKM9/vrVFh9p9qGzckg== +-----END PRIVATE KEY----- diff --git a/util/build-with-dd.sh b/util/build-with-dd.sh index db318ce391..a56a89119b 100755 --- a/util/build-with-dd.sh +++ b/util/build-with-dd.sh @@ -20,9 +20,13 @@ fi disable_pcre2=--without-pcre2 answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$answer" = "N" ]; then +if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then disable_pcre2="" fi +if [ "$USE_PCRE2" = "Y" ]; then + PCRE_INC=$PCRE2_INC + PCRE_LIB=$PCRE2_LIB +fi time ngx-build $force $version \ --with-threads \ diff --git a/util/build-without-ssl.sh b/util/build-without-ssl.sh index 97097daac0..c52d5873f6 100755 --- a/util/build-without-ssl.sh +++ b/util/build-without-ssl.sh @@ -26,9 +26,13 @@ add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" disable_pcre2=--without-pcre2 answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$answer" = "N" ]; then +if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then disable_pcre2="" fi +if [ "$USE_PCRE2" = "Y" ]; then + PCRE_INC=$PCRE2_INC + PCRE_LIB=$PCRE2_LIB +fi time ngx-build $force $version \ --with-threads \ diff --git a/util/build.sh b/util/build.sh index 3a786a186f..41896f2c7e 100755 --- a/util/build.sh +++ b/util/build.sh @@ -32,9 +32,13 @@ fi disable_pcre2=--without-pcre2 answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$answer" = "N" ]; then +if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then disable_pcre2="" fi +if [ "$USE_PCRE2" = "Y" ]; then + PCRE_INC=$PCRE2_INC + PCRE_LIB=$PCRE2_LIB +fi time ngx-build $force $version \ --with-threads \ diff --git a/valgrind.suppress b/valgrind.suppress index 8761fb350e..743fb67d77 100644 --- a/valgrind.suppress +++ b/valgrind.suppress @@ -234,3 +234,73 @@ fun:ngx_pass_open_channel fun:ngx_start_privileged_agent_processes } +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_regex_malloc + fun:pcre2_compile_context_create_8 + fun:ngx_regex_compile + fun:ngx_http_regex_compile + fun:ngx_http_core_regex_location + fun:ngx_http_core_location + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_core_server + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_block + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_init_cycle + fun:main +} +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_regex_malloc + fun:pcre2_compile_context_create_8 + fun:ngx_regex_compile + fun:ngx_http_regex_compile + fun:ngx_http_rewrite + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_core_location + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_core_server + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_block + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_init_cycle + fun:main +} +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:ngx_alloc + fun:ngx_regex_malloc + fun:pcre2_compile_context_create_8 + fun:ngx_regex_compile + fun:ngx_http_regex_compile + fun:ngx_http_rewrite + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_core_server + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_http_block + fun:ngx_conf_handler + fun:ngx_conf_parse + fun:ngx_init_cycle + fun:main +} From f29e8487b298acd30ee30aa58b7ed65ec80b3bc5 Mon Sep 17 00:00:00 2001 From: willmafh Date: Sat, 23 Sep 2023 20:41:56 +0800 Subject: [PATCH 705/848] doc: update TODO list since tcpsock:bind is already implemented. --- README.markdown | 1 - 1 file changed, 1 deletion(-) diff --git a/README.markdown b/README.markdown index 29945d4997..aaf53b1459 100644 --- a/README.markdown +++ b/README.markdown @@ -966,7 +966,6 @@ TODO * cosocket: implement LuaSocket's unconnected UDP API. * cosocket: add support in the context of [init_by_lua*](#init_by_lua). -* cosocket: implement the `bind()` method for stream-typed cosockets. * cosocket: review and merge aviramc's [patch](https://github.com/openresty/lua-nginx-module/pull/290) for adding the `bsdrecv` method. * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. * review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option From 2448743b6862d63c2005d75544d9071b1210d67a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 6 Oct 2023 14:32:53 +0800 Subject: [PATCH 706/848] tests: update t/cert/test.crl. --- t/cert/test.crl | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/t/cert/test.crl b/t/cert/test.crl index 098fd54bf9..7ce2f6dc7c 100644 --- a/t/cert/test.crl +++ b/t/cert/test.crl @@ -1,11 +1,14 @@ -----BEGIN X509 CRL----- -MIIBjzCB+QIBATANBgkqhkiG9w0BAQUFADCBlzELMAkGA1UEBhMCVVMxEzARBgNV -BAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28xEjAQBgNVBAoM -CU9wZW5SZXN0eTESMBAGA1UECwwJT3BlblJlc3R5MREwDwYDVQQDDAh0ZXN0LmNv -bTEgMB4GCSqGSIb3DQEJARYRYWdlbnR6aEBnbWFpbC5jb20XDTE0MDcyMTIxNDEy -MloXDTE0MDgyMDIxNDEyMlowHDAaAgkApQ5tVpK3luIXDTE0MDcyMTIxNDEwMlqg -DzANMAsGA1UdFAQEAgIQATANBgkqhkiG9w0BAQUFAAOBgQBDZ6UY0Qg7qDoLrXXl -gJElFilZ7LiKPqjE3+Rfx7XkgdbPxjGCr77TfMm+smdvawk7WHv1AOvRH7kGrgGT -kGJZwqJ4vKa/NpEWJIMAZ1Gq9BIH/Ig6ffmPk+S9ozcVHKJDW7x4nMuotyj1hILN -EePv78DZCYMZgf8WwMElNgz6Hw== +MIICGzCCAQMCAQEwDQYJKoZIhvcNAQELBQAwgZcxCzAJBgNVBAYTAlVTMRMwEQYD +VQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMRIwEAYDVQQK +DAlPcGVuUmVzdHkxEjAQBgNVBAsMCU9wZW5SZXN0eTERMA8GA1UEAwwIdGVzdC5j +b20xIDAeBgkqhkiG9w0BCQEWEWFnZW50emhAZ21haWwuY29tFw0yMzEwMDYwNjMw +MTlaFw0yMzExMDUwNjMwMTlaMCcwJQIUImcuJ8MJpeNhvYBEoEGGz5oCBg4XDTIz +MTAwNjA2MjkyNVqgDjAMMAoGA1UdFAQDAgECMA0GCSqGSIb3DQEBCwUAA4IBAQBw +bqXCiJyDh804u7n4g1QrHpuiP9tdqCyxmvRVx34U51THGsLhkSufOzlaT2okTG0s +Oj0KiThRcwfjUpSBu1Lg81XtImEpZ1E97Doq/7QfH6P5D7co5U9MCdtzfNuWWKFD +likVmARV4aAl0VF0+Rqk+yvmh8yN1p9gLzM0AFoALVv2doU2oLid/DVuzxfMfRqC +Kr6OxEV/5umm1K72m8DaVtGjpX+M/wdLoTeX5r1bFp5q0QMPPZb8ciW+/9ovIG+I +EzJwrxMRMbi5DEnFgnkNHghUUDUlqZVVgdIIxR3un5gKrsmg7ZHrevF/iiJtKHAj +vgmsV9XKo4AUjcVtXrqx -----END X509 CRL----- From 8e9c33a9efcde74f30f53993b208f8c3d6b37a3c Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 6 Oct 2023 14:41:24 +0800 Subject: [PATCH 707/848] tests: update t/cert/test.crl. --- t/cert/test.crl | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/t/cert/test.crl b/t/cert/test.crl index 7ce2f6dc7c..0abfa77ac2 100644 --- a/t/cert/test.crl +++ b/t/cert/test.crl @@ -2,13 +2,13 @@ MIICGzCCAQMCAQEwDQYJKoZIhvcNAQELBQAwgZcxCzAJBgNVBAYTAlVTMRMwEQYD VQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMRIwEAYDVQQK DAlPcGVuUmVzdHkxEjAQBgNVBAsMCU9wZW5SZXN0eTERMA8GA1UEAwwIdGVzdC5j -b20xIDAeBgkqhkiG9w0BCQEWEWFnZW50emhAZ21haWwuY29tFw0yMzEwMDYwNjMw -MTlaFw0yMzExMDUwNjMwMTlaMCcwJQIUImcuJ8MJpeNhvYBEoEGGz5oCBg4XDTIz -MTAwNjA2MjkyNVqgDjAMMAoGA1UdFAQDAgECMA0GCSqGSIb3DQEBCwUAA4IBAQBw -bqXCiJyDh804u7n4g1QrHpuiP9tdqCyxmvRVx34U51THGsLhkSufOzlaT2okTG0s -Oj0KiThRcwfjUpSBu1Lg81XtImEpZ1E97Doq/7QfH6P5D7co5U9MCdtzfNuWWKFD -likVmARV4aAl0VF0+Rqk+yvmh8yN1p9gLzM0AFoALVv2doU2oLid/DVuzxfMfRqC -Kr6OxEV/5umm1K72m8DaVtGjpX+M/wdLoTeX5r1bFp5q0QMPPZb8ciW+/9ovIG+I -EzJwrxMRMbi5DEnFgnkNHghUUDUlqZVVgdIIxR3un5gKrsmg7ZHrevF/iiJtKHAj -vgmsV9XKo4AUjcVtXrqx +b20xIDAeBgkqhkiG9w0BCQEWEWFnZW50emhAZ21haWwuY29tFw0yMzEwMDYwNjM4 +MzlaFw0zMTEyMjMwNjM4MzlaMCcwJQIUImcuJ8MJpeNhvYBEoEGGz5oCBg4XDTIz +MTAwNjA2MjkyNVqgDjAMMAoGA1UdFAQDAgEEMA0GCSqGSIb3DQEBCwUAA4IBAQBj +FciorrAuXxn1ULW0XJ7PElwTxZtBrb838EHYvkZ5OdT5tZcucYR6XTZpfT1Up/Px +rC9EZ1/aZ0wSQfYEQuctafyVCJoPN71IV9IWpPTWm8JyEvnE1W3SgHJujItanyZ3 +aMDihljxV9eKyEQnZPQZkaOjAKhj8d2/XZLQ3uIrjWy+/OxcaBQK4a8bDoSM3GZT +J6YCD2UBVYKSiROMZZAj3m1thLAGm1RM7A6vjEcH7rPyoxhok5SdxXH5ERY94/ro +McjNCv6zV8/Ue5/+ajz5pu/48T901mlIicHry8uImJvlnqAXlH8ReJ+hiSfGhbZ5 +WYNf0wN81NXwTxPIb+v2 -----END X509 CRL----- From 03ec7f1d71b53aef91e7f42cc7ea675ca5ccd5c0 Mon Sep 17 00:00:00 2001 From: Christian Marangi Date: Tue, 17 Oct 2023 17:35:45 +0200 Subject: [PATCH 708/848] bugfix: don't include pcre.h with PCRE2 used. pcre.h is a PCRE header and is not exposed by PCRE2 library causing compilation error as the header is not found. Don't include pcre.h if nginx is compiled with PCRE2 support enabled. Fixes: cb83e33e2657 ("feature: support pcre2") Signed-off-by: Christian Marangi --- src/ngx_http_lua_common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 541fa881db..4bbfd8c780 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -54,7 +54,7 @@ typedef struct { #endif -#if (NGX_PCRE) +#if defined(NGX_PCRE) && !defined(NGX_PCRE2) #include # if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) # define LUA_HAVE_PCRE_JIT 1 From 309edb6832903b88644eff94f278e75bcf188a2b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 9 Nov 2021 12:43:23 +0800 Subject: [PATCH 709/848] tests: fixed test for http3. --- .travis.yml | 15 +- src/ngx_http_lua_socket_tcp.c | 6 + t/001-set.t | 12 +- t/005-exit.t | 5 +- t/014-bugs.t | 23 ++- t/015-status.t | 1 + t/016-resp-header.t | 82 ++++++-- t/020-subrequest.t | 4 + t/022-redirect.t | 11 +- t/023-rewrite/client-abort.t | 6 +- t/023-rewrite/exit.t | 3 + t/023-rewrite/on-abort.t | 6 +- t/023-rewrite/redirect.t | 11 +- t/023-rewrite/req-socket.t | 10 +- t/023-rewrite/sleep.t | 3 + t/023-rewrite/socket-keepalive.t | 8 + t/023-rewrite/tcp-socket.t | 6 + t/023-rewrite/uthread-exit.t | 2 + t/023-rewrite/uthread-spawn.t | 1 + t/024-access/client-abort.t | 10 +- t/024-access/exit.t | 3 + t/024-access/on-abort.t | 6 +- t/024-access/redirect.t | 11 +- t/024-access/sleep.t | 3 + t/024-access/uthread-exit.t | 2 + t/024-access/uthread-spawn.t | 1 + t/025-codecache.t | 3 + t/026-mysql.t | 1 + t/028-req-header.t | 315 ++++++++++++++++++++++++++----- t/033-ctx.t | 2 + t/041-header-filter.t | 50 ++++- t/043-shdict.t | 4 + t/044-req-body.t | 44 ++++- t/045-ngx-var.t | 18 +- t/056-flush.t | 1 + t/057-flush-timeout.t | 6 +- t/058-tcp-socket.t | 6 + t/062-count.t | 4 + t/065-tcp-socket-timeout.t | 1 + t/066-socket-receiveuntil.t | 2 + t/067-req-socket.t | 10 +- t/068-socket-keepalive.t | 13 ++ t/075-logby.t | 12 +- t/077-sleep.t | 3 + t/082-body-filter-2.t | 12 +- t/082-body-filter.t | 7 + t/083-bad-sock-self.t | 2 + t/084-inclusive-receiveuntil.t | 4 + t/087-udp-socket.t | 9 +- t/088-req-method.t | 4 + t/091-coroutine.t | 6 + t/093-uthread-spawn.t | 2 + t/094-uthread-exit.t | 10 + t/095-uthread-exec.t | 2 + t/096-uthread-redirect.t | 2 + t/100-client-abort.t | 8 +- t/101-on-abort.t | 8 +- t/103-req-http-ver.t | 11 +- t/104-req-raw-header.t | 11 +- t/105-pressure.t | 4 +- t/106-timer.t | 1 + t/109-timer-hup.t | 4 + t/116-raw-req-socket.t | 10 +- t/129-ssl-socket.t | 7 + t/131-duplex-req-socket.t | 8 +- t/151-initby-hup.t | 4 +- t/152-timer-every.t | 1 + t/153-semaphore-hup.t | 2 + t/157-socket-keepalive-hup.t | 4 + t/162-static-module-location.t | 16 +- t/163-signal.t | 2 + t/166-ssl-client-hello.t | 94 +++++++-- t/166-worker-thread.t | 2 + t/cert/http3/http3.crt | 29 +++ t/cert/http3/http3.key | 52 +++++ 75 files changed, 954 insertions(+), 130 deletions(-) create mode 100644 t/cert/http3/http3.crt create mode 100644 t/cert/http3/http3.key diff --git a/.travis.yml b/.travis.yml index b29ab4cc2a..f70f825574 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,4 +1,4 @@ -dist: bionic +dist: focal branches: only: @@ -24,6 +24,14 @@ addons: - libtest-longstring-perl - liblist-moreutils-perl - libgd-dev + - time + - cmake + - curl + - libunwind-dev + - libpsl0 + - wget + - libldap-2.4-2 + - libbrotli1 cache: directories: @@ -60,6 +68,7 @@ env: - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y + - NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y services: - memcached @@ -67,8 +76,6 @@ services: - mysql before_install: - - sudo apt update - - sudo apt install --only-upgrade ca-certificates - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) @@ -145,6 +152,8 @@ script: - ldd `which nginx`|grep -E 'luajit|ssl|pcre' - export LD_PRELOAD=$PWD/mockeagain/mockeagain.so - export LD_LIBRARY_PATH=$PWD/mockeagain:$LD_LIBRARY_PATH + - export TEST_NGINX_HTTP3_CRT=$PWD/t/cert/http3/http3.crt + - export TEST_NGINX_HTTP3_KEY=$PWD/t/cert/http3/http3.key - export TEST_NGINX_RESOLVER=8.8.4.4 - dig +short myip.opendns.com @resolver1.opendns.com || exit 0 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 5be85dcd46..88fe923eb1 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5099,6 +5099,12 @@ ngx_http_lua_req_socket(lua_State *L) } #endif +#if (NGX_HTTP_V3) + if (r->http_version == NGX_HTTP_VERSION_30) { + return luaL_error(L, "http v3 not supported yet"); + } +#endif + if (!raw && r->headers_in.chunked) { lua_pushnil(L); lua_pushliteral(L, "chunked request bodies not supported yet"); diff --git a/t/001-set.t b/t/001-set.t index db18f0258c..abbee72121 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -404,8 +404,16 @@ API disabled in the context of set_by_lua* GET /lua --- response_body_like: 500 Internal Server Error --- error_code: 500 ---- error_log -API disabled in the context of set_by_lua* +--- error_log eval +my $err_log; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $err_log = "http v3 not supported yet"; +} else { + $err_log = "API disabled in the context of set_by_lua*"; +} + +$err_log; diff --git a/t/005-exit.t b/t/005-exit.t index eb123bd179..e057fe1c69 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -65,7 +65,7 @@ GET /lua --- error_log attempt to set status 404 via ngx.exit after sending out the response status 200 --- no_error_log -alert +[alert] --- response_body hi @@ -123,6 +123,7 @@ GET /api?user=agentz === TEST 6: working with ngx_auth_request (simplest form, w/o ngx_memc) +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -195,6 +196,7 @@ Logged in 56 === TEST 7: working with ngx_auth_request (simplest form) +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -267,6 +269,7 @@ Logged in 56 === TEST 8: working with ngx_auth_request +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} --- http_config eval " lua_package_cpath '$::LuaCpath'; diff --git a/t/014-bugs.t b/t/014-bugs.t index 877aecfdb5..1f140be51d 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -194,6 +194,14 @@ Hi" === TEST 8: github issue 37: header bug https://github.com/chaoslawful/lua-nginx-module/issues/37 + +https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2 + Just as in HTTP/1.x, header field names are strings of ASCII + characters that are compared in a case-insensitive fashion. However, + header field names MUST be converted to lowercase prior to their + encoding in HTTP/2. A request or response containing uppercase + header field names MUST be treated as malformed + --- config location /sub { content_by_lua ' @@ -220,8 +228,17 @@ https://github.com/chaoslawful/lua-nginx-module/issues/37 --- request GET /lua --- raw_response_headers_like eval -".*Set-Cookie: TestCookie1=foo\r +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = ".*set-cookie: TestCookie1=foo\r +set-cookie: TestCookie2=bar.*" +} else { + $headers = ".*Set-Cookie: TestCookie1=foo\r Set-Cookie: TestCookie2=bar.*" +} + +$headers; @@ -770,6 +787,10 @@ eof found in body stream === TEST 34: testing a segfault when using ngx_poll_module + ngx_resolver See more details here: http://mailman.nginx.org/pipermail/nginx-devel/2013-January/003275.html + +http3 may cache the dns result. +so need to skip for http3 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} --- config location /t { set $myserver nginx.org; diff --git a/t/015-status.t b/t/015-status.t index a69c59377b..43faa204a7 100644 --- a/t/015-status.t +++ b/t/015-status.t @@ -233,6 +233,7 @@ GET /t --- error_code: 101 --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 65c5d75a8e..60c63929ae 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -65,6 +65,7 @@ GET /read Content-Length: 3 --- response_body chop Hel +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -113,8 +114,16 @@ Hello } --- request GET /read ---- raw_response_headers_like chomp -X-Foo: a\r\n.*?X-Foo: bc\r\n +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = qr/x-foo: a\r\n.*?x-foo: bc\r\n/ +} else { + $headers = qr/X-Foo: a\r\n.*?X-Foo: bc\r\n/ +} + +$headers; --- response_body Hello @@ -184,8 +193,16 @@ Hello } --- request GET /read ---- raw_response_headers_like chomp -X-Foo: a\r\n.*?X-Foo: abc\r\n +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = "x-foo: a\r\n.*?x-foo: abc\r\n" +} else { + $headers = "X-Foo: a\r\n.*?X-Foo: abc\r\n" +} + +$headers; --- response_body Hello @@ -203,8 +220,17 @@ Hello --- request GET /lua --- raw_response_headers_like eval -".*Foo: a\r -Foo: b.*" +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = ".*foo: a\r +foo: b.*"; +} else { + $headers = ".*Foo: a\r +Foo: b.*"; +} + +$headers; --- response_body @@ -222,8 +248,17 @@ Foo: b.*" --- request GET /lua --- raw_response_headers_like eval -".*Foo: a\r -Foo: b.*" +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = ".*foo: a\r +foo: b.*"; +} else { + $headers = ".*Foo: a\r +Foo: b.*"; +} + +$headers; --- response_body @@ -278,7 +313,7 @@ hello --- error_log attempt to set ngx.header.HEADER after sending out response headers --- no_error_log eval -["alert", "warn"] +["[alert]", "[warn]"] @@ -1071,8 +1106,8 @@ GET /t --- more_headers Foo: bar Bah: baz ---- response_headers -Location: http://localhost:$ServerPort/t/ +--- response_headers_like +Location: https?://localhost:\d+/t/ --- response_body_like: 301 Moved Permanently --- error_code: 301 --- no_error_log @@ -1094,8 +1129,8 @@ GET /t Foo: bar Bah: baz --- response_body_like: 301 Moved Permanently ---- response_headers -Location: http://localhost:$ServerPort/t/ +--- response_headers_like +Location: https?://localhost:\d+/t/ --- error_code: 301 --- no_error_log [error] @@ -1115,8 +1150,8 @@ GET /t --- more_headers Foo: bar Bah: baz ---- response_headers -Location: http://localhost:$ServerPort/t/ +--- response_headers_like +Location: https?://localhost:\d+/t/ Foo: /t/ --- response_body_like: 301 Moved Permanently --- error_code: 301 @@ -1139,8 +1174,8 @@ GET /t --- more_headers Foo: bar Bah: baz ---- response_headers -Location: http://localhost:$ServerPort/t/ +--- response_headers_like +Location: https?://localhost:\d+/t/ Foo: /t/ --- response_body_like: 301 Moved Permanently --- error_code: 301 @@ -1159,8 +1194,16 @@ Foo: /t/ } --- request GET /lua ---- raw_response_headers_like chop -cache-Control: private +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = "cache-control: private" +} else { + $headers = "cache-Control: private" +} + +$headers; --- response_body Cache-Control: private @@ -1509,6 +1552,7 @@ hi --- error_log my Transfer-Encoding: chunked my transfer-encoding: chunked +--- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 844b64794f..f629ca8754 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1227,6 +1227,8 @@ F(ngx_http_finalize_request) { --- error_code --- no_error_log [error] +--- curl_error +curl: (52) Empty reply from server @@ -2622,6 +2624,7 @@ pr: Host: localhost --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -2655,6 +2658,7 @@ pr: Host: localhost --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/022-redirect.t b/t/022-redirect.t index ef80174ca5..197c76e1d7 100644 --- a/t/022-redirect.t +++ b/t/022-redirect.t @@ -122,7 +122,16 @@ GET /read } --- request GET /read ---- raw_response_headers_like: Location: /echo\r\n +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = "location: /echo\r\n" +} else { + $headers = "Location: /echo\r\n" +} + +$headers; --- response_body_like: 302 Found --- error_code: 302 diff --git a/t/023-rewrite/client-abort.t b/t/023-rewrite/client-abort.t index 0d6f2b39b3..963717b012 100644 --- a/t/023-rewrite/client-abort.t +++ b/t/023-rewrite/client-abort.t @@ -19,7 +19,11 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 3 - 1); +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + plan(skip_all => "HTTP3 does not support client abort"); +} else { + plan tests => repeat_each() * (blocks() * 3 - 1); +} $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= '11211'; diff --git a/t/023-rewrite/exit.t b/t/023-rewrite/exit.t index 39ea5cbd62..9add80441c 100644 --- a/t/023-rewrite/exit.t +++ b/t/023-rewrite/exit.t @@ -187,6 +187,7 @@ ngx.var.uid = res[1].uid; GET /api?uid=32 --- response_body Logged in 56 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} @@ -258,6 +259,7 @@ ngx.var.uid = res[1].uid; GET /api?uid=32 --- response_body Logged in 56 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} @@ -342,6 +344,7 @@ GET /api?uid=32 Logged in 56 --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/023-rewrite/on-abort.t b/t/023-rewrite/on-abort.t index 59c5df1167..d0bae78ee8 100644 --- a/t/023-rewrite/on-abort.t +++ b/t/023-rewrite/on-abort.t @@ -19,7 +19,11 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 15); +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + plan(skip_all => "HTTP3 does not support on_abort"); +} else { + plan tests => repeat_each() * (blocks() * 4 + 15); +} $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= '11211'; diff --git a/t/023-rewrite/redirect.t b/t/023-rewrite/redirect.t index 8fa9aa2e79..e387ee8fb7 100644 --- a/t/023-rewrite/redirect.t +++ b/t/023-rewrite/redirect.t @@ -119,7 +119,16 @@ GET /read } --- request GET /read ---- raw_response_headers_like: Location: /foo\r\n +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = "location: /foo\r\n" +} else { + $headers = "Location: /foo\r\n" +} + +$headers; --- response_body_like: 302 Found --- error_code: 302 diff --git a/t/023-rewrite/req-socket.t b/t/023-rewrite/req-socket.t index 9292385265..f4dd6f4e18 100644 --- a/t/023-rewrite/req-socket.t +++ b/t/023-rewrite/req-socket.t @@ -1,6 +1,14 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support ngx.req.socket"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); repeat_each(2); diff --git a/t/023-rewrite/sleep.t b/t/023-rewrite/sleep.t index 8d4c2da2ea..0135d5e440 100644 --- a/t/023-rewrite/sleep.t +++ b/t/023-rewrite/sleep.t @@ -110,6 +110,7 @@ bad argument #1 to 'sleep' === TEST 5: sleep 0.5 - multi-times +--- quic_max_idle_timeout: 1.0 --- config location /test { rewrite_by_lua ' @@ -135,6 +136,7 @@ lua sleep timer expired: "/test?" === TEST 6: sleep 0.5 - interleaved by ngx.say() - ended by ngx.sleep +--- quic_max_idle_timeout: 2.05 --- config location /test { rewrite_by_lua ' @@ -163,6 +165,7 @@ lua sleep timer expired: "/test?" === TEST 7: sleep 0.5 - interleaved by ngx.say() - not ended by ngx.sleep +--- quic_max_idle_timeout: 0.85 --- config location /test { rewrite_by_lua ' diff --git a/t/023-rewrite/socket-keepalive.t b/t/023-rewrite/socket-keepalive.t index c943e44616..5316af6be5 100644 --- a/t/023-rewrite/socket-keepalive.t +++ b/t/023-rewrite/socket-keepalive.t @@ -176,6 +176,7 @@ received: OK === TEST 3: upstream sockets close prematurely +--- quic_max_idle_timeout: 1.1 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -253,6 +254,7 @@ done === TEST 4: http keepalive +--- quic_max_idle_timeout: 1.1 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -330,6 +332,7 @@ done === TEST 5: lua_socket_keepalive_timeout +--- quic_max_idle_timeout: 1.1 --- config server_tokens off; location /t { @@ -409,6 +412,7 @@ qr/lua tcp socket connection pool size: 30\b/] === TEST 6: lua_socket_pool_size +--- quic_max_idle_timeout: 1.1 --- config server_tokens off; location /t { @@ -489,6 +493,7 @@ qr/lua tcp socket connection pool size: 1\b/] === TEST 7: "lua_socket_keepalive_timeout 0" means unlimited +--- quic_max_idle_timeout: 1.1 --- config server_tokens off; location /t { @@ -571,6 +576,7 @@ lua tcp socket keepalive timeout: unlimited === TEST 8: setkeepalive(timeout) overrides lua_socket_keepalive_timeout +--- quic_max_idle_timeout: 1.1 --- config server_tokens off; location /t { @@ -650,6 +656,7 @@ qr/lua tcp socket connection pool size: 30\b/] === TEST 9: sock:setkeepalive(timeout, size) overrides lua_socket_pool_size +--- quic_max_idle_timeout: 1.1 --- config server_tokens off; location /t { @@ -730,6 +737,7 @@ qr/lua tcp socket connection pool size: 25\b/] === TEST 10: sock:keepalive_timeout(0) means unlimited +--- quic_max_idle_timeout: 1.1 --- config server_tokens off; location /t { diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index 5258487eba..b4bf7a6b78 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -1654,6 +1654,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type nil found) +--- curl_error +curl: (52) Empty reply from server @@ -1715,6 +1717,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type boolean found) +--- curl_error +curl: (52) Empty reply from server @@ -1776,6 +1780,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type userdata found) +--- curl_error +curl: (52) Empty reply from server diff --git a/t/023-rewrite/uthread-exit.t b/t/023-rewrite/uthread-exit.t index e25f7f8b0c..87f850ce8b 100644 --- a/t/023-rewrite/uthread-exit.t +++ b/t/023-rewrite/uthread-exit.t @@ -987,6 +987,7 @@ hello in thread after --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -1072,6 +1073,7 @@ hello in thread after --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/023-rewrite/uthread-spawn.t b/t/023-rewrite/uthread-spawn.t index dccef87dac..62d837cff7 100644 --- a/t/023-rewrite/uthread-spawn.t +++ b/t/023-rewrite/uthread-spawn.t @@ -1120,6 +1120,7 @@ body: hello world)$ --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/024-access/client-abort.t b/t/024-access/client-abort.t index 83bf0a4df1..99f1b810fd 100644 --- a/t/024-access/client-abort.t +++ b/t/024-access/client-abort.t @@ -1,6 +1,14 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support ngx.req.socket and lua_check_client_abort"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); use t::StapThread; our $GCScript = <<_EOC_; diff --git a/t/024-access/exit.t b/t/024-access/exit.t index d6d66b8cfe..b77778213a 100644 --- a/t/024-access/exit.t +++ b/t/024-access/exit.t @@ -177,6 +177,7 @@ GET /api?uid=32 --- response_body Logged in 56 --- timeout: 3 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} @@ -243,6 +244,7 @@ ngx.var.uid = res[1].uid; GET /api?uid=32 --- response_body Logged in 56 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} @@ -320,6 +322,7 @@ ngx.var.uid = res[1].uid; GET /api?uid=32 --- response_body Logged in 56 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/024-access/on-abort.t b/t/024-access/on-abort.t index b9532aea3f..4d58779742 100644 --- a/t/024-access/on-abort.t +++ b/t/024-access/on-abort.t @@ -19,7 +19,11 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 15); +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + plan(skip_all => "HTTP3 does not support on_abort"); +} else { + plan tests => repeat_each() * (blocks() * 4 + 15); +} $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= '11211'; diff --git a/t/024-access/redirect.t b/t/024-access/redirect.t index b45fac718b..84b64c5f0b 100644 --- a/t/024-access/redirect.t +++ b/t/024-access/redirect.t @@ -119,6 +119,15 @@ GET /read } --- request GET /read ---- raw_response_headers_like: Location: /foo\r\n +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = "location: /foo\r\n" +} else { + $headers = "Location: /foo\r\n" +} + +$headers; --- response_body_like: 302 Found --- error_code: 302 diff --git a/t/024-access/sleep.t b/t/024-access/sleep.t index fc1fc024b5..c7aa0b65a4 100644 --- a/t/024-access/sleep.t +++ b/t/024-access/sleep.t @@ -110,6 +110,7 @@ bad argument #1 to 'sleep' === TEST 5: sleep 0.5 - multi-times +--- quic_max_idle_timeout: 1.0 --- config location /test { access_by_lua ' @@ -135,6 +136,7 @@ lua sleep timer expired: "/test?" === TEST 6: sleep 0.5 - interleaved by ngx.say() - ended by ngx.sleep +--- quic_max_idle_timeout: 2.2 --- config location /test { access_by_lua ' @@ -163,6 +165,7 @@ lua sleep timer expired: "/test?" === TEST 7: sleep 0.5 - interleaved by ngx.say() - not ended by ngx.sleep +--- quic_max_idle_timeout: 0.85 --- config location /test { access_by_lua ' diff --git a/t/024-access/uthread-exit.t b/t/024-access/uthread-exit.t index 27572370d5..bd165ae708 100644 --- a/t/024-access/uthread-exit.t +++ b/t/024-access/uthread-exit.t @@ -880,6 +880,7 @@ after === TEST 11: exit in user thread (entry thread is still pending on reqsock:receive) +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} --- config location /lua { access_by_lua ' @@ -973,6 +974,7 @@ after === TEST 12: exit in user thread (entry thread is still pending on ngx.req.read_body) +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} --- config location /lua { client_body_timeout 12000ms; diff --git a/t/024-access/uthread-spawn.t b/t/024-access/uthread-spawn.t index bc92ccd32c..b31edbcb1e 100644 --- a/t/024-access/uthread-spawn.t +++ b/t/024-access/uthread-spawn.t @@ -1120,3 +1120,4 @@ body: hello world)$ --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/025-codecache.t b/t/025-codecache.t index 2a3eeb3b08..5be94d8ce7 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -966,6 +966,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, === TEST 27: GC issue with the on_abort thread object +curl: (52) Empty reply from server --- config lua_code_cache off; location = /t { @@ -991,6 +992,8 @@ decrementing the reference count for Lua VM: 3 qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, "lua close the global Lua VM", ] +--- curl_error eval +qr/curl: \(\d+\) Empty reply from server/ diff --git a/t/026-mysql.t b/t/026-mysql.t index 569f96fbc9..02e14b9382 100644 --- a/t/026-mysql.t +++ b/t/026-mysql.t @@ -69,6 +69,7 @@ kill status = 200 kill body = \{"errcode":0\}$ --- error_log eval qr{upstream timed out \(\d+: Connection timed out\) while sending query to drizzle upstream} +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/028-req-header.t b/t/028-req-header.t index 18b991052b..b9c5e97fe0 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -59,8 +59,13 @@ lua exceeding request header limit for k, v in pairs(headers) do h[k] = v end - ngx.say("Foo: ", h["Foo"] or "nil") - ngx.say("Bar: ", h["Bar"] or "nil") + if (ngx.req.http_version() == 3) then + ngx.say("Foo: ", h["foo"] or "nil") + ngx.say("Bar: ", h["bar"] or "nil") + else + ngx.say("Foo: ", h["Foo"] or "nil") + ngx.say("Bar: ", h["Bar"] or "nil") + end '; } --- request @@ -127,6 +132,7 @@ Foo: --- response_body eval "a" x 2048 --- timeout: 15 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} @@ -145,6 +151,7 @@ Foo: --- response_body eval "a" x 2048 --- timeout: 15 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} @@ -485,6 +492,7 @@ for my $k (@k) { --- timeout: 4 --- error_log lua exceeding request header limit 101 > 100 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -523,20 +531,30 @@ while ($i <= 98) { $s --- response_body eval my @k; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + push @k, "host: localhost\n"; +} my $i = 1; while ($i <= 98) { push @k, "x-$i"; $i++; } -push @k, "connection: close\n"; -push @k, "host: localhost\n"; + +my $found_headers = "found 99 headers\n"; +if (!defined $ENV{TEST_NGINX_USE_HTTP3}) { + push @k, "connection: close\n"; + push @k, "host: localhost\n"; + $found_headers = "found 100 headers\n"; +} @k = sort @k; for my $k (@k) { if ($k =~ /\d+/) { $k .= ": $&\n"; } } -CORE::join("", @k) . "found 100 headers\n"; + +CORE::join("", @k) . $found_headers; --- timeout: 4 --- no_error_log [error] @@ -576,6 +594,9 @@ while ($i <= 101) { $s --- response_body eval my @k; +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + push @k, "host: localhost\n"; +} my $i = 1; while ($i <= 100) { push @k, "x-$i"; @@ -593,6 +614,7 @@ for my $k (@k) { --- timeout: 4 --- error_log lua exceeding request header limit 103 > 102 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -628,13 +650,20 @@ while ($i <= 100) { $s --- response_body eval my @k; +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + push @k, "host: localhost\n"; +} my $i = 1; while ($i <= 100) { push @k, "x-$i"; $i++; } -push @k, "connection: close\n"; -push @k, "host: localhost\n"; + +if (!defined($ENV{TEST_NGINX_USE_HTTP3})) { + push @k, "connection: close\n"; + push @k, "host: localhost\n"; +} + @k = sort @k; for my $k (@k) { if ($k =~ /\d+/) { @@ -681,13 +710,18 @@ while ($i <= 105) { $s --- response_body eval my @k; +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + push @k, "host: localhost\n"; +} my $i = 1; while ($i <= 105) { push @k, "x-$i"; $i++; } -push @k, "connection: close\n"; -push @k, "host: localhost\n"; +if (!defined($ENV{TEST_NGINX_USE_HTTP3})) { + push @k, "connection: close\n"; + push @k, "host: localhost\n"; +} @k = sort @k; for my $k (@k) { if ($k =~ /\d+/) { @@ -830,10 +864,21 @@ hello world Content-Type: application/ocsp-request Test-Header: 1 --- response_body_like eval -qr/Connection: close\r +my $body; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $body = qr/Connection: close\r +test-header: 1\r +\r +$/; +} else { + $body = qr/Connection: close\r Test-Header: 1\r \r -$/ +$/; +} + +$body; --- no_error_log [error] @@ -881,7 +926,36 @@ Foo20: foo20 Foo21: foo21 Foo22: foo22 --- response_body_like eval -qr/Bah: bah\r +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = qr/bah: bah\r +test-header: 1\r +foo1: foo1\r +foo2: foo2\r +foo3: foo3\r +foo4: foo4\r +foo5: foo5\r +foo6: foo6\r +foo7: foo7\r +foo8: foo8\r +foo9: foo9\r +foo10: foo10\r +foo11: foo11\r +foo12: foo12\r +foo13: foo13\r +foo14: foo14\r +foo15: foo15\r +foo16: foo16\r +foo17: foo17\r +foo18: foo18\r +foo19: foo19\r +foo20: foo20\r +foo21: foo21\r +foo22: foo22\r +/; +} else { + $headers = qr/Bah: bah\r Test-Header: 1\r Foo1: foo1\r Foo2: foo2\r @@ -905,7 +979,10 @@ Foo19: foo19\r Foo20: foo20\r Foo21: foo21\r Foo22: foo22\r -/ +/; +} + +$headers; @@ -936,11 +1013,21 @@ GET /t --- more_headers My-Foo: bar Bar: baz ---- response_body -Bar: baz +--- response_body eval +my $body; +if ($ENV{TEST_NGINX_USE_HTTP3}) { + $body = "bar: baz +host: localhost +my-foo: bar +"; +} else { + $body = "Bar: baz Connection: close Host: localhost My-Foo: bar +"; +} +$body; @@ -1010,29 +1097,23 @@ for my $i ('a' .. 'r') { } $s --- response_body eval -"GET /back HTTP/1.0\r +my $s = "GET /back HTTP/1.0\r Host: foo\r -Connection: close\r -User-Agent: curl\r -A: a\r -B: b\r -C: c\r -D: d\r -E: e\r -F: f\r -G: g\r -H: h\r -I: i\r -J: j\r -K: k\r -L: l\r -M: m\r -N: n\r -O: o\r -P: p\r -Q: q\r -\r -" +Connection: close\r\n"; + +if (defined ($ENV{TEST_NGINX_USE_HTTP3})) { + $s .= "user-agent: curl\r\n"; + for my $i ('a' .. 'q') { + $s .= $i . ": " . "$i\r\n" + } +} else { + $s .= "User-Agent: curl\r\n"; + for my $i ('a' .. 'q') { + $s .= uc($i) . ": " . "$i\r\n" + } +} + +$s . "\r\n"; @@ -1063,7 +1144,55 @@ for my $i ('a' .. 'r') { } $s --- response_body eval -"GET /back HTTP/1.0\r +my $body; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $body = "GET /back HTTP/1.0\r +Host: foo\r +Connection: close\r +user-agent: curl\r +a: a\r +b: b\r +c: c\r +d: d\r +e: e\r +f: f\r +g: g\r +h: h\r +i: i\r +j: j\r +k: k\r +l: l\r +m: m\r +n: n\r +o: o\r +p: p\r +q: q\r +foo-1: 1\r +foo-2: 2\r +foo-3: 3\r +foo-4: 4\r +foo-5: 5\r +foo-6: 6\r +foo-7: 7\r +foo-8: 8\r +foo-9: 9\r +foo-10: 10\r +foo-11: 11\r +foo-12: 12\r +foo-13: 13\r +foo-14: 14\r +foo-15: 15\r +foo-16: 16\r +foo-17: 17\r +foo-18: 18\r +foo-19: 19\r +foo-20: 20\r +foo-21: 21\r +\r +"; +} else { + $body = "GET /back HTTP/1.0\r Host: foo\r Connection: close\r User-Agent: curl\r @@ -1106,7 +1235,10 @@ foo-19: 19\r foo-20: 20\r foo-21: 21\r \r -" +"; +} + +$body; @@ -1135,7 +1267,34 @@ for my $i ('a' .. 'r') { } $s --- response_body eval -"GET /back HTTP/1.0\r +my $body; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $body = "GET /back HTTP/1.0\r +Host: foo\r +Connection: close\r +user-agent: curl\r +bah: bah\r +a: a\r +b: b\r +c: c\r +d: d\r +e: e\r +f: f\r +g: g\r +h: h\r +i: i\r +j: j\r +k: k\r +l: l\r +m: m\r +n: n\r +o: o\r +p: p\r +\r +" +} else { +$body = "GET /back HTTP/1.0\r Host: foo\r Connection: close\r User-Agent: curl\r @@ -1158,6 +1317,9 @@ O: o\r P: p\r \r " +} + +$body; @@ -1189,7 +1351,55 @@ for my $i ('a' .. 'r') { } $s --- response_body eval -"GET /back HTTP/1.0\r +my $body; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $body = "GET /back HTTP/1.0\r +Host: foo\r +Connection: close\r +user-agent: curl\r +bah: bah\r +a: a\r +b: b\r +c: c\r +d: d\r +e: e\r +f: f\r +g: g\r +h: h\r +i: i\r +j: j\r +k: k\r +l: l\r +m: m\r +n: n\r +o: o\r +p: p\r +foo-1: 1\r +foo-2: 2\r +foo-3: 3\r +foo-4: 4\r +foo-5: 5\r +foo-6: 6\r +foo-7: 7\r +foo-8: 8\r +foo-9: 9\r +foo-10: 10\r +foo-11: 11\r +foo-12: 12\r +foo-13: 13\r +foo-14: 14\r +foo-15: 15\r +foo-16: 16\r +foo-17: 17\r +foo-18: 18\r +foo-19: 19\r +foo-20: 20\r +foo-21: 21\r +\r +"; +} else { + $body = "GET /back HTTP/1.0\r Host: foo\r Connection: close\r User-Agent: curl\r @@ -1233,6 +1443,9 @@ foo-20: 20\r foo-21: 21\r \r " +} + +$body; @@ -1264,11 +1477,23 @@ GET /t --- more_headers My-Foo: bar Bar: baz ---- response_body -Bar: baz +--- response_body eval +my $body; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $body="bar: baz +host: localhost +my-foo: bar +"; +} else { + $body="Bar: baz Connection: close Host: localhost My-Foo: bar +"; +} + +$body; --- no_error_log [error] @@ -1903,6 +2128,7 @@ found 3 headers. lua exceeding request header limit 4 > 3 --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -1939,6 +2165,7 @@ found 3 headers. --- no_error_log lua exceeding request header limit [error] +--- skip_eval: 4: $ENV{TEST_NGINX_USE_HTTP3} @@ -1977,6 +2204,7 @@ found 3 headers. lua exceeding request header limit 4 > 3 --- no_error_log [error] +--- skip_eval: 4: $ENV{TEST_NGINX_USE_HTTP3} @@ -2013,6 +2241,7 @@ found 3 headers. --- no_error_log lua exceeding request header limit [error] +--- skip_eval: 4: $ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/033-ctx.t b/t/033-ctx.t index 8fc50aa7a0..1f879191b3 100644 --- a/t/033-ctx.t +++ b/t/033-ctx.t @@ -278,6 +278,8 @@ GET /t [error] --- error_log ngx.ctx = 32 +--- curl_error +curl: (52) Empty reply from server diff --git a/t/041-header-filter.t b/t/041-header-filter.t index be390b3259..a6ee25c8b2 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -124,6 +124,8 @@ Hi GET /read --- error_code --- response_body +--- curl_error +curl: (52) Empty reply from server @@ -465,6 +467,8 @@ GET /lua failed to run header_filter_by_lua*: header_filter_by_lua(nginx.conf:47):2: Something bad --- no_error_log [alert] +--- curl_error +curl: (56) Failure when receiving data from the peer @@ -487,6 +491,8 @@ GET /lua failed to run header_filter_by_lua*: unknown reason --- no_error_log [alert] +--- curl_error +curl: (56) Failure when receiving data from the peer @@ -501,6 +507,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -515,6 +523,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -529,6 +539,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -543,6 +555,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -557,6 +571,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -575,6 +591,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -593,6 +611,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -607,6 +627,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -621,6 +643,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -635,6 +659,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -667,6 +693,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -679,8 +707,18 @@ API disabled in the context of header_filter_by_lua* --- request GET /lua --- ignore_response ---- error_log -API disabled in the context of header_filter_by_lua* +--- error_log eval +my $err_log; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $err_log = "http v3 not supported yet"; +} else { + $err_log = "API disabled in the context of header_filter_by_lua*"; +} + +$err_log; +--- curl_error +curl: (52) Empty reply from server @@ -695,6 +733,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -709,6 +749,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -756,6 +798,8 @@ stack traceback: in function 'error' in function 'bar' in function 'foo' +--- curl_error +curl: (52) Empty reply from server @@ -774,6 +818,8 @@ GET /lua?a=1&b=2 --- ignore_response --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ +--- curl_error +curl: (52) Empty reply from server diff --git a/t/043-shdict.t b/t/043-shdict.t index b0528e5cc7..4fcdcb33b6 100644 --- a/t/043-shdict.t +++ b/t/043-shdict.t @@ -1184,6 +1184,7 @@ nil nil === TEST 45: flush_expires +--- quic_max_idle_timeout: 1.6 --- http_config lua_shared_dict dogs 1m; --- config @@ -1210,6 +1211,7 @@ GET /t === TEST 46: flush_expires with number +--- quic_max_idle_timeout: 1.6 --- http_config lua_shared_dict dogs 1m; --- config @@ -1337,6 +1339,7 @@ GET /t === TEST 51: list all keys in a shdict with expires +--- quic_max_idle_timeout: 1.6 --- http_config lua_shared_dict dogs 1m; --- config @@ -1426,6 +1429,7 @@ GET /t === TEST 55: list all keys in a shdict with all keys expired +--- quic_max_idle_timeout: 1.6 --- http_config lua_shared_dict dogs 1m; --- config diff --git a/t/044-req-body.t b/t/044-req-body.t index d8a62c7dbe..0c378c44aa 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -351,9 +351,18 @@ hello, world --- user_files >>> a.txt Will you change this world? ---- raw_response_headers_like -X-Old: \S+/client_body_temp/\d+\r -.*?X-New: \S+/html/a\.txt\r +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = qr#x-old: \S+/client_body_temp/\d+\r +.*?x-new: \S+/html/a\.txt\r#; +} else { + $headers = qr#X-Old: \S+/client_body_temp/\d+\r +.*?X-New: \S+/html/a\.txt\r#; +} + +$headers; --- response_body Will you change this world? --- no_error_log @@ -390,9 +399,18 @@ hello, world! --- user_files >>> a.txt Will you change this world? ---- raw_response_headers_like -X-Old: \S+/client_body_temp/\d+\r -.*?X-New: \S+/html/a\.txt\r +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = qr#x-old: \S+/client_body_temp/\d+\r +.*?x-new: \S+/html/a\.txt\r#; +} else { + $headers = qr#X-Old: \S+/client_body_temp/\d+\r +.*?X-New: \S+/html/a\.txt\r#; +} + +$headers; --- response_body Will you change this world? --- no_error_log @@ -1243,6 +1261,7 @@ body: hello, my dear friend! [alert] --- no_error_log a client request body is buffered to a temporary file +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -1384,6 +1403,7 @@ failed to get req socket: request body already exists [alert] --- no_error_log a client request body is buffered to a temporary file +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -1480,8 +1500,16 @@ probe syscall.fcntl { --- stap_out_unlike fcntl\(O_DIRECT\) ---- raw_response_headers_like -.*?X-New: \S+/html/a\.txt\r +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = qr#.*?x-new: \S+/html/a\.txt\r#; +} else { + $headers = qr#.*?X-New: \S+/html/a\.txt\r#; +} + +$headers; --- response_body Will you change this world? --- no_error_log diff --git a/t/045-ngx-var.t b/t/045-ngx-var.t index 81fcef6458..f0f9f415c6 100644 --- a/t/045-ngx-var.t +++ b/t/045-ngx-var.t @@ -170,10 +170,22 @@ invalid referer: 1 } --- request GET /t ---- raw_response_headers_like -Proxy-Host: 127.0.0.1\:\d+\r +--- raw_response_headers_like eval +my $headers; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $headers = +qr/proxy-host: 127.0.0.1\:\d+\r +proxy-port: \d+\r +proxy-add-x-forwarded-for: 127.0.0.1\r/; +} else { + $headers = +qr/Proxy-Host: 127.0.0.1\:\d+\r Proxy-Port: \d+\r -Proxy-Add-X-Forwarded-For: 127.0.0.1\r +Proxy-Add-X-Forwarded-For: 127.0.0.1\r/; +} + +$headers; --- response_body hello --- no_error_log diff --git a/t/056-flush.t b/t/056-flush.t index bdd33d4f9a..aa409ad51c 100644 --- a/t/056-flush.t +++ b/t/056-flush.t @@ -491,6 +491,7 @@ GET /test === TEST 17: limit_rate +--- quic_max_idle_timeout: 2 --- config location /test { limit_rate 150; diff --git a/t/057-flush-timeout.t b/t/057-flush-timeout.t index 1f8cfaadf9..4ef104f707 100644 --- a/t/057-flush-timeout.t +++ b/t/057-flush-timeout.t @@ -19,9 +19,13 @@ BEGIN { $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; $ENV{MOCKEAGAIN_WRITE_TIMEOUT_PATTERN} = 'hello, world'; $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "HTTP3 does not support mockeagain"; + } } -use Test::Nginx::Socket::Lua; +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); use t::StapThread; our $GCScript = $t::StapThread::GCScript; diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 6ffb32f584..14995bf9fe 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -1619,6 +1619,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type nil found) +--- curl_error +curl: (52) Empty reply from server @@ -1678,6 +1680,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type boolean found) +--- curl_error +curl: (52) Empty reply from server @@ -1737,6 +1741,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type userdata found) +--- curl_error +curl: (52) Empty reply from server diff --git a/t/062-count.t b/t/062-count.t index ad464ba2de..07605f95b9 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -262,6 +262,7 @@ hello world n = 6 --- no_error_log [error] +--- skip_eval: 3: $ENV{TEST_NGINX_USE_HTTP3} @@ -515,6 +516,7 @@ GET /test n = 7 --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -554,6 +556,7 @@ narr = 2 nrec = 3 --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -589,3 +592,4 @@ narr = 2 nrec = 3 --- no_error_log [error] +--- skip_eval: 3: $ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index 092094a3c9..e06dfb1fc8 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -873,6 +873,7 @@ quitting request now --- no_error_log lua tcp socket write timed out [alert] +--- skip_eval: 4: $ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 7fac6bfd0f..2d38fb5c37 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -1327,6 +1327,7 @@ this exposed a memory leak in receiveuntil ok --- no_error_log [error] +<<<<<<< HEAD @@ -1993,3 +1994,4 @@ close: 1 nil } --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/067-req-socket.t b/t/067-req-socket.t index c5b2631e5e..fbaa0b3f41 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -1,6 +1,14 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support ngx.req.socket"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); repeat_each(2); diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index cf8bc25acb..fe8dab152f 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -168,6 +168,7 @@ received: OK === TEST 3: upstream sockets close prematurely +--- quic_max_idle_timeout: 1.2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -243,6 +244,7 @@ done === TEST 4: http keepalive +--- quic_max_idle_timeout: 1.2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -318,6 +320,7 @@ done === TEST 5: lua_socket_keepalive_timeout +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -395,6 +398,7 @@ qr/lua tcp socket connection pool size: 30\b/] === TEST 6: lua_socket_pool_size +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -473,6 +477,7 @@ qr/lua tcp socket connection pool size: 1\b/] === TEST 7: "lua_socket_keepalive_timeout 0" means unlimited +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -548,6 +553,7 @@ qr/lua tcp socket connection pool size: 30\b/] === TEST 8: setkeepalive(timeout) overrides lua_socket_keepalive_timeout +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -625,6 +631,7 @@ qr/lua tcp socket connection pool size: 30\b/] === TEST 9: sock:setkeepalive(timeout, size) overrides lua_socket_pool_size +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -731,6 +738,7 @@ bad argument #3 to '?' (bad "pool_size" option value: 0) === TEST 11: sock:keepalive_timeout(0) means unlimited +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -1508,6 +1516,7 @@ done === TEST 25: setkeepalive() with explicit nil args +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { @@ -1919,6 +1928,7 @@ too many waiting connect operations === TEST 32: conn queuing: once 'pool_size' is reached and pool has 'backlog' +--- quic_max_idle_timeout: 1.2 --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -2536,6 +2546,8 @@ GET /t --- abort --- no_error_log [error] +--- curl_error eval +qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received/ @@ -2643,6 +2655,7 @@ lua tcp socket connect timed out, when connecting to === TEST 46: conn queuing: resume connect operation if resumed connect failed (could not be resolved) +--- quic_max_idle_timeout: 1.2 --- config resolver 127.0.0.2:12345 ipv6=off; resolver_timeout 1s; diff --git a/t/075-logby.t b/t/075-logby.t index 8eece698ec..de662d0fb4 100644 --- a/t/075-logby.t +++ b/t/075-logby.t @@ -457,8 +457,16 @@ API disabled in the context of log_by_lua* GET /lua --- response_body ok ---- error_log -API disabled in the context of log_by_lua* +--- error_log eval +my $err_log; + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $err_log = "http v3 not supported yet"; +} else { + $err_log = "API disabled in the context of log_by_lua*"; +} + +$err_log; diff --git a/t/077-sleep.t b/t/077-sleep.t index 96f04bd98e..4867b9ca7f 100644 --- a/t/077-sleep.t +++ b/t/077-sleep.t @@ -105,6 +105,7 @@ bad argument #1 to 'sleep' === TEST 5: sleep 0.33 - multi-times in content +--- quic_max_idle_timeout: 1.1 --- config location /test { content_by_lua ' @@ -129,6 +130,7 @@ lua sleep timer expired: "/test?" === TEST 6: sleep 0.5 - interleaved by ngx.say() - ended by ngx.sleep +--- quic_max_idle_timeout: 2.2 --- config location /test { content_by_lua ' @@ -156,6 +158,7 @@ lua sleep timer expired: "/test?" === TEST 7: sleep 0.5 - interleaved by ngx.say() - not ended by ngx.sleep +--- quic_max_idle_timeout: 0.9 --- config location /test { content_by_lua ' diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t index 99ed447f56..d19f46f660 100644 --- a/t/082-body-filter-2.t +++ b/t/082-body-filter-2.t @@ -3,13 +3,11 @@ our $SkipReason; BEGIN { - if ($ENV{TEST_NGINX_EVENT_TYPE} && $ENV{TEST_NGINX_EVENT_TYPE} ne 'poll') { - $SkipReason = "unavailable for the event type '$ENV{TEST_NGINX_EVENT_TYPE}'"; - - } else { - $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; - $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; - $ENV{MOCKEAGAIN}='w' + $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + $ENV{MOCKEAGAIN}='w'; + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 has bug about the hup reload"; } } diff --git a/t/082-body-filter.t b/t/082-body-filter.t index a69c78fa70..54e7414cb8 100644 --- a/t/082-body-filter.t +++ b/t/082-body-filter.t @@ -482,6 +482,8 @@ stack traceback: in function 'error' in function 'bar' in function 'foo' +--- curl_error +curl: (52) Empty reply from server @@ -524,6 +526,8 @@ GET /lua?a=1&b=2 --- ignore_response --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ +--- curl_error +curl: (52) Empty reply from server @@ -838,6 +842,7 @@ GET /lua --- ignore_response --- error_log API disabled in the context of body_filter_by_lua* +<<<<<<< HEAD @@ -890,3 +895,5 @@ failed to load inlined Lua code: body_filter_by_lua(nginx.conf:49):2: unexpected --- no_error_log no_such_error1 no_such_error2 +--- curl_error +curl: (52) Empty reply from server diff --git a/t/083-bad-sock-self.t b/t/083-bad-sock-self.t index 7a76752d9f..206de8b990 100644 --- a/t/083-bad-sock-self.t +++ b/t/083-bad-sock-self.t @@ -33,6 +33,7 @@ __DATA__ --- error_code: 500 --- error_log bad argument #1 to 'receive' (table expected, got string) +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -51,6 +52,7 @@ bad argument #1 to 'receive' (table expected, got string) --- error_code: 500 --- error_log bad argument #1 to 'receiveuntil' (table expected, got number) +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/084-inclusive-receiveuntil.t b/t/084-inclusive-receiveuntil.t index eb5aa1ecdf..95d898501c 100644 --- a/t/084-inclusive-receiveuntil.t +++ b/t/084-inclusive-receiveuntil.t @@ -511,6 +511,8 @@ bad "inclusive" option value type: string --- no_error_log [alert] [warn] +--- curl_error +curl: (52) Empty reply from server @@ -578,6 +580,8 @@ bad "inclusive" option value type: string --- no_error_log [alert] [warn] +--- curl_error +curl: (52) Empty reply from server diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index 6c26b93876..ffe6b13e92 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -143,12 +143,15 @@ GET /t === TEST 3: access a TCP interface +test-nginx use the same port for tcp(http) and udp(http3) +so need to change to a port that is not listen by any app. +default port range: +net.ipv4.ip_local_port_range = 32768 60999 +choose a port greater than 61000 should be less race. --- config server_tokens off; location /t { - #set $port 5000; - set $port $TEST_NGINX_SERVER_PORT; - #set $port 1234; + set $port 65432; content_by_lua ' local socket = ngx.socket diff --git a/t/088-req-method.t b/t/088-req-method.t index 198b3b3492..de3421662c 100644 --- a/t/088-req-method.t +++ b/t/088-req-method.t @@ -209,6 +209,7 @@ main: GET === TEST 10: set HEAD to GET +XXX: does http3 do not support set HEAD to GET?? --- config location /t { rewrite_by_lua ' @@ -223,10 +224,12 @@ main: GET method: GET --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} === TEST 11: set GET to WebDAV methods +XXX: does http3 do not support change HEAD method? --- config location /t { content_by_lua ' @@ -262,3 +265,4 @@ method: PATCH method: TRACE --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/091-coroutine.t b/t/091-coroutine.t index fafc5a45cc..e8761045b8 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -159,6 +159,8 @@ cc3: 2 === TEST 3: basic coroutine and cosocket +access the public network is unstable, need a bigger timeout +--- quic_max_idle_timeout: 4 --- config resolver $TEST_NGINX_RESOLVER ipv6=off; location /lua { @@ -760,6 +762,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* +--- curl_error +curl: (52) Empty reply from server @@ -1695,6 +1699,8 @@ GET /t "stack traceback:", "in function 'co'" ] +--- curl_error +curl: (52) Empty reply from server diff --git a/t/093-uthread-spawn.t b/t/093-uthread-spawn.t index ab6db2ff26..d905c100c5 100644 --- a/t/093-uthread-spawn.t +++ b/t/093-uthread-spawn.t @@ -1064,6 +1064,8 @@ body: hello world)$ === TEST 23: simple user thread with ngx.req.socket() +ngx.req.socket() does not support in http3 +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} --- config location /lua { content_by_lua ' diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index 58d8d0b7db..39b1f22b55 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -966,6 +966,7 @@ hello in thread after --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -1051,6 +1052,7 @@ hello in thread after --- no_error_log [error] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -1399,6 +1401,8 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] +--- curl_error +curl: (52) Empty reply from server @@ -1482,6 +1486,8 @@ free request [alert] [error] [warn] +--- curl_error +curl: (52) Empty reply from server @@ -1565,6 +1571,8 @@ free request [alert] [error] [warn] +--- curl_error +curl: (52) Empty reply from server @@ -1648,3 +1656,5 @@ free request [alert] [error] [warn] +--- curl_error +curl: (52) Empty reply from server diff --git a/t/095-uthread-exec.t b/t/095-uthread-exec.t index 7bf37c0e8d..6e18c844ea 100644 --- a/t/095-uthread-exec.t +++ b/t/095-uthread-exec.t @@ -423,3 +423,5 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] +--- curl_error +curl: (52) Empty reply from server diff --git a/t/096-uthread-redirect.t b/t/096-uthread-redirect.t index 95bff01f3a..2da84bd7aa 100644 --- a/t/096-uthread-redirect.t +++ b/t/096-uthread-redirect.t @@ -277,3 +277,5 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] +--- curl_error +curl: (52) Empty reply from server diff --git a/t/100-client-abort.t b/t/100-client-abort.t index f5713874e7..fed01176c7 100644 --- a/t/100-client-abort.t +++ b/t/100-client-abort.t @@ -1,6 +1,12 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +BEGIN { +if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "client abort detect does not support in http3" + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); use t::StapThread; our $GCScript = <<_EOC_; diff --git a/t/101-on-abort.t b/t/101-on-abort.t index ee1e41e3a8..f10027a47a 100644 --- a/t/101-on-abort.t +++ b/t/101-on-abort.t @@ -1,6 +1,12 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +BEGIN { +if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "client abort detect does not support in http3" + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); use t::StapThread; our $GCScript = <<_EOC_; diff --git a/t/103-req-http-ver.t b/t/103-req-http-ver.t index e6de2388c5..ebc65c9d38 100644 --- a/t/103-req-http-ver.t +++ b/t/103-req-http-ver.t @@ -26,8 +26,15 @@ __DATA__ } --- request GET /t ---- response_body -1.1 +--- response_body eval +my $body; +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + $body="3\n"; +} else { + $body="1.1\n"; +} + +$body; --- no_error_log [error] diff --git a/t/104-req-raw-header.t b/t/104-req-raw-header.t index aa66630485..459d190ac7 100644 --- a/t/104-req-raw-header.t +++ b/t/104-req-raw-header.t @@ -1,5 +1,14 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; + +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support ngx.req.raw_header()"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); #worker_connections(1014); #master_on(); diff --git a/t/105-pressure.t b/t/105-pressure.t index 9d1ba1ff1a..2fc130d09b 100644 --- a/t/105-pressure.t +++ b/t/105-pressure.t @@ -2,7 +2,7 @@ use Test::Nginx::Socket::Lua; -#worker_connections(1014); +worker_connections(1014); #master_on(); #log_level('debug'); @@ -28,6 +28,8 @@ run_tests(); __DATA__ === TEST 1: memory issue in the "args" string option for ngx.location.capture +the default worker_connections is 64, HTTP3 will keep the connection when curl +request finished. So need to change the worker_connection. --- config location /test1 { content_by_lua ' diff --git a/t/106-timer.t b/t/106-timer.t index acd6e4c800..d455ea2add 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -1332,6 +1332,7 @@ API disabled === TEST 19: exit in user thread (entry thread is still pending on ngx.sleep) +--- quic_max_idle_timeout: 1.3 --- config location /t { content_by_lua ' diff --git a/t/109-timer-hup.t b/t/109-timer-hup.t index bff1b33466..551dd486c7 100644 --- a/t/109-timer-hup.t +++ b/t/109-timer-hup.t @@ -6,6 +6,10 @@ BEGIN { if ($ENV{TEST_NGINX_CHECK_LEAK}) { $SkipReason = "unavailable for the hup tests"; + } elsif (defined $ENV{TEST_NGINX_USE_HTTP3}) { + #os.execute("kill -HUP " .. pid) + $SkipReason = "send HUP relaod signal by self make two workers with same id"; + } else { $ENV{TEST_NGINX_USE_HUP} = 1; undef $ENV{TEST_NGINX_USE_STAP}; diff --git a/t/116-raw-req-socket.t b/t/116-raw-req-socket.t index 4dfb92664e..6704a92084 100644 --- a/t/116-raw-req-socket.t +++ b/t/116-raw-req-socket.t @@ -1,6 +1,14 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +our $SkipReason; + +BEGIN { + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support ngx.req.socket(true)"; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); repeat_each(2); diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 85d3a80985..f89a538fd0 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -40,6 +40,8 @@ run_tests(); __DATA__ === TEST 1: www.google.com +access the public network is unstable, need a bigger timeout value. +--- quic_max_idle_timeout: 3 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -293,6 +295,8 @@ SSL reused session === TEST 4: ssl session reuse +access the public network is unstable, need a bigger timeout value. +--- quic_max_idle_timeout: 3 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1564,6 +1568,7 @@ attempt to call method 'sslhandshake' (a nil value) --- no_error_log [alert] --- timeout: 3 +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -2563,6 +2568,8 @@ qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including t --- no_error_log [alert] --- timeout: 10 +--- curl_error +curl: (52) Empty reply from server diff --git a/t/131-duplex-req-socket.t b/t/131-duplex-req-socket.t index 5d698b9344..f88649b8d8 100644 --- a/t/131-duplex-req-socket.t +++ b/t/131-duplex-req-socket.t @@ -1,5 +1,7 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: +our $SkipReason; + BEGIN { if (!defined $ENV{LD_PRELOAD}) { $ENV{LD_PRELOAD} = ''; @@ -18,9 +20,13 @@ BEGIN { $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; $ENV{MOCKEAGAIN_WRITE_TIMEOUT_PATTERN} = 'slow'; + + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support ngx.req.socket()"; + } } -use Test::Nginx::Socket::Lua; +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); log_level('debug'); diff --git a/t/151-initby-hup.t b/t/151-initby-hup.t index f2788670db..a37db9fe43 100644 --- a/t/151-initby-hup.t +++ b/t/151-initby-hup.t @@ -12,7 +12,7 @@ BEGIN { } } -use Test::Nginx::Socket::Lua 'no_plan'; +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); #worker_connections(1014); #master_on(); @@ -21,7 +21,7 @@ use Test::Nginx::Socket::Lua 'no_plan'; repeat_each(1); -#plan tests => repeat_each() * (blocks() * 3 + 3); +plan tests => repeat_each() * (blocks() * 3); #no_diff(); #no_long_string(); diff --git a/t/152-timer-every.t b/t/152-timer-every.t index 7b42d2d724..374e80e04b 100644 --- a/t/152-timer-every.t +++ b/t/152-timer-every.t @@ -224,6 +224,7 @@ registered timer === TEST 6: memory leak check +--- quic_max_idle_timeout: 8 --- config location /t { content_by_lua_block { diff --git a/t/153-semaphore-hup.t b/t/153-semaphore-hup.t index 5c271a48e7..496db6862e 100644 --- a/t/153-semaphore-hup.t +++ b/t/153-semaphore-hup.t @@ -104,6 +104,7 @@ run_tests(); __DATA__ === TEST 1: timer + reload +--- quic_max_idle_timeout: 1.1 --- config location /test { content_by_lua_block { @@ -129,6 +130,7 @@ created semaphore object === TEST 2: timer + reload (lua code cache off) +--- quic_max_idle_timeout: 1.1 --- http_config lua_code_cache off; --- config diff --git a/t/157-socket-keepalive-hup.t b/t/157-socket-keepalive-hup.t index d9436b3b73..4ebf637a33 100644 --- a/t/157-socket-keepalive-hup.t +++ b/t/157-socket-keepalive-hup.t @@ -6,6 +6,10 @@ BEGIN { if ($ENV{TEST_NGINX_CHECK_LEAK}) { $SkipReason = "unavailable for the hup tests"; + } elsif (defined $ENV{TEST_NGINX_USE_HTTP3}) { + #os.execute("kill -HUP " .. pid) + $SkipReason = "send HUP relaod signal by self make two workers with same id"; + } else { $ENV{TEST_NGINX_USE_HUP} = 1; undef $ENV{TEST_NGINX_USE_STAP}; diff --git a/t/162-static-module-location.t b/t/162-static-module-location.t index fde93c71ba..9f09dbe631 100644 --- a/t/162-static-module-location.t +++ b/t/162-static-module-location.t @@ -40,9 +40,11 @@ Location: %00%0A%0Dset-cookie:1234567/ Hello, world --- request GET /t/中文 +--- more_headers +host: localhost --- error_code: 301 --- response_headers_like -Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ +Location: https?:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ --- response_body_like .*301 Moved Permanently.* @@ -56,9 +58,11 @@ Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ Hello, world --- request GET /t/中文?q=name +--- more_headers +host: localhost --- error_code: 301 --- response_headers_like -Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name +Location: https?:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name --- response_body_like .*301 Moved Permanently.* @@ -72,9 +76,11 @@ Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name Hello, world --- request GET /t/%E4%B8%AD%E6%96%87 +--- more_headers +host: localhost --- error_code: 301 --- response_headers_like -Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ +Location: https?:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ --- response_body_like .*301 Moved Permanently.* @@ -88,8 +94,10 @@ Location: http:\/\/localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/ Hello, world --- request GET /t/%E4%B8%AD%E6%96%87?q=name +--- more_headers +host: localhost --- error_code: 301 --- response_headers_like -Location: http://localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name +Location: https?://localhost:\d+\/t\/%E4%B8%AD%E6%96%87\/\?q=name --- response_body_like .*301 Moved Permanently.* diff --git a/t/163-signal.t b/t/163-signal.t index 5a6b880b09..2b85e12482 100644 --- a/t/163-signal.t +++ b/t/163-signal.t @@ -38,6 +38,8 @@ GET /t qr/\[notice\] \d+#\d+: exit$/ --- no_error_log eval qr/\[notice\] \d+#\d+: reconfiguring/ +--- curl_error eval +qr/curl: \(28\) Operation timed out after 3\d+ milliseconds with 0 bytes received/ diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index 850a0d6550..445434289a 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -456,7 +456,7 @@ received memc reply: OK === TEST 5: ngx.exit(0) - no yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; ssl_client_hello_by_lua_block { ngx.exit(0) @@ -484,7 +484,7 @@ received memc reply: OK sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -523,7 +523,7 @@ should never reached here === TEST 6: ngx.exit(ngx.ERROR) - no yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; ssl_client_hello_by_lua_block { ngx.exit(ngx.ERROR) @@ -551,7 +551,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -593,7 +593,7 @@ should never reached here === TEST 7: ngx.exit(0) - yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; ssl_client_hello_by_lua_block { ngx.sleep(0.001) @@ -623,7 +623,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -662,7 +662,7 @@ should never reached here === TEST 8: ngx.exit(ngx.ERROR) - yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; ssl_client_hello_by_lua_block { ngx.sleep(0.001) @@ -692,7 +692,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -805,7 +805,7 @@ should never reached here === TEST 10: lua exception - yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; server_name test.com; ssl_client_hello_by_lua_block { ngx.sleep(0.001) @@ -834,7 +834,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") if not ok then ngx.say("failed to connect: ", err) return @@ -1741,6 +1741,7 @@ close: 1 nil ssl client hello by lua is running! [error] [alert] +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -2130,7 +2131,7 @@ ssl client hello by lua is running! lua_package_path "../lua-resty-core/lib/?.lua;;"; server { - listen 127.0.0.1:12345 ssl; + listen 127.0.0.1:12346 ssl; server_name test.com; ssl_client_hello_by_lua_block { @@ -2162,7 +2163,7 @@ ssl client hello by lua is running! sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.1", 12345) + local ok, err = sock:connect("127.0.0.1", 12346) if not ok then ngx.say("failed to connect: ", err) return @@ -2565,3 +2566,72 @@ ssl handshake: cdata uthread: hello from f() uthread: killed uthread: failed to kill: already waited or killed + + + +=== TEST 30: ngx.exit(ngx.OK) - no yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_client_hello_by_lua_block { + ngx.exit(ngx.OK) + ngx.log(ngx.ERR, "should never reached here...") + } + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block {ngx.status = 201 ngx.say("foo") ngx.exit(201)} + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(false, nil, true, false) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + end -- do + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: boolean + +--- error_log eval +[ +'lua_client_hello_by_lua: handler return value: 0, client hello cb exit code: 1', +qr/\[debug\] .*? SSL_do_handshake: 1/, +'lua exit with code 0', +] +--- no_error_log +should never reached here +[alert] +[emerg] diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index c8ae62965f..afe7f3c410 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -282,6 +282,7 @@ false : module 'hello' not found.* === TEST 10: the number of Lua VM exceeds the pool size +--- quic_max_idle_timeout: 5 --- main_config thread_pool testpool threads=100; --- http_config eval: $::HttpConfig @@ -359,6 +360,7 @@ GET /t === TEST 11: kill uthread before worker thread callback +--- quic_max_idle_timeout: 10 --- main_config thread_pool testpool threads=100; --- http_config eval: $::HttpConfig diff --git a/t/cert/http3/http3.crt b/t/cert/http3/http3.crt new file mode 100644 index 0000000000..8683f41adf --- /dev/null +++ b/t/cert/http3/http3.crt @@ -0,0 +1,29 @@ +-----BEGIN CERTIFICATE----- +MIIFCTCCAvGgAwIBAgIUHLeNm7JwH368JWXBYJ1Dv+xcL6kwDQYJKoZIhvcNAQEL +BQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTIxMTEwNTA2NTQxNVoXDTQxMDEw +NDA2NTQxNVowFDESMBAGA1UEAwwJbG9jYWxob3N0MIICIjANBgkqhkiG9w0BAQEF +AAOCAg8AMIICCgKCAgEAx5/M18tKWKfacgldf9gBguTdLA3JWiblRWM/38fSZGKL +Mcb1vLErY/qQyLYxoLKBht0FUZAEZ08y/iheYFZT7H082b8bNvwY+v6bScOQOvcX +hkTWNlSQORDH4qIFxVXq0soXga+0ukSZ2RQRcCUWeKUaZwhGCYNIj04/FB9Lef95 +Ku7LkNauTBmRGIwXgQWhfnoPM21o9D9i54R9L2RHU7fGeTGhiS0nCe1nPPB7KBgO +s9rBHMXV9qHxCNMWWiVNsMX049S7aD9yvRrV7NAHssVQdaRR234IPJxb7BYSUPi8 +5U+8l43Ornd+cY/R/sXDQluFidlnZvHT+akdy0ObWDK0lMUweDWvFVLY3ZvMvsGR +rQE8RR5/fy83y7w//0734EBk4ttEzlilmjA8UvnumOpK0UYaW8LkikvxB+48e89s +NOEKZf3Mfw/fDRz0tO+cr4cIgUPQ4ru6KNVnGH/ZiD97AVMPXJO9nPOUIRVH9aXM +wC74CSt5idWOwpTKy+sLg7anM235NvZ9bTiS+V5CTzBlqL/wKzEI0Injds3kBc1a +Y8Lk/cIdNhuwlN01fYluA8XyB3DWoeQYbySEoHC5ksvaFLBM3yPomWrmyM5lJrj5 +QbWa46b0bONJQ2qmrQa5KREVvDthHQvKELsabHX1qbCShSxoG45aclqpmKy2AjcC +AwEAAaNTMFEwHQYDVR0OBBYEFA2T9Sgv31hCl3INL5MB++NrMu0iMB8GA1UdIwQY +MBaAFA2T9Sgv31hCl3INL5MB++NrMu0iMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZI +hvcNAQELBQADggIBAJdxtUq4T/sh5Ww7pMOB4+JF5zsxEBpZiFMzT2hYZ+6a2mYh +UTy7ff1lNfNl0BfslaOD404Qt0SSgI2TByLs838/sKpVk+QePcw6kl6IFIPERUBO +uxq/+QXqsDHOXb6m4qpEBmrknUa65dThdUw5r4sHo0XD2l/y2/zEZtd4e3ZGAWts +spdjPBk4UHPtG3p29eY/8ehw+zDXuWG4nlJYjn+6PRnLhvpgt2/E/Wr9PeYcv3IC +UgavxXVtk9fnclg8BuKlnZYki7txn8+F9Rh03CVVZ9R16Q/aVALI2iTs8T4gCt7Z +eexfLGfBkLBkLvpL7wpzxNsvOC2b5bJZCDUTprLVmdpmMQu75qLg1mOfubMo983H +8G91V4XOokRoCRub/SLKA16/gpEwnE2aDsVMUVSxwpRu2Rjw4GpzbCNAHUzmblrh +zYMSAsEuTcsZEAdZQrzmhGc1Yg5Q88V4o+qyywzkgd86O65QUozhnkCs+eS9ikMV +cPLXoW5SDIsrrcoTR6bH5MdDjS7ILKUUC5+x0qo6EhK94Fx49TkRBNIYk3o0fG7j +o/0YvozXjqTRnodYegL4LKoGZyfL4qbuh3t8ZGQ/Z0ECmvjcmJzPyObIiMe2InT3 +GRY+ypPTyeiumjHFFVO0zx+DAv+HFPtq1XaygWvxKY1DTP6FNN0BzQdzAgKm +-----END CERTIFICATE----- diff --git a/t/cert/http3/http3.key b/t/cert/http3/http3.key new file mode 100644 index 0000000000..5825540f58 --- /dev/null +++ b/t/cert/http3/http3.key @@ -0,0 +1,52 @@ +-----BEGIN PRIVATE KEY----- +MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQDHn8zXy0pYp9py +CV1/2AGC5N0sDclaJuVFYz/fx9JkYosxxvW8sStj+pDItjGgsoGG3QVRkARnTzL+ +KF5gVlPsfTzZvxs2/Bj6/ptJw5A69xeGRNY2VJA5EMfiogXFVerSyheBr7S6RJnZ +FBFwJRZ4pRpnCEYJg0iPTj8UH0t5/3kq7suQ1q5MGZEYjBeBBaF+eg8zbWj0P2Ln +hH0vZEdTt8Z5MaGJLScJ7Wc88HsoGA6z2sEcxdX2ofEI0xZaJU2wxfTj1LtoP3K9 +GtXs0AeyxVB1pFHbfgg8nFvsFhJQ+LzlT7yXjc6ud35xj9H+xcNCW4WJ2Wdm8dP5 +qR3LQ5tYMrSUxTB4Na8VUtjdm8y+wZGtATxFHn9/LzfLvD//TvfgQGTi20TOWKWa +MDxS+e6Y6krRRhpbwuSKS/EH7jx7z2w04Qpl/cx/D98NHPS075yvhwiBQ9Diu7oo +1WcYf9mIP3sBUw9ck72c85QhFUf1pczALvgJK3mJ1Y7ClMrL6wuDtqczbfk29n1t +OJL5XkJPMGWov/ArMQjQieN2zeQFzVpjwuT9wh02G7CU3TV9iW4DxfIHcNah5Bhv +JISgcLmSy9oUsEzfI+iZaubIzmUmuPlBtZrjpvRs40lDaqatBrkpERW8O2EdC8oQ +uxpsdfWpsJKFLGgbjlpyWqmYrLYCNwIDAQABAoICAENb1qESRbn4matVIamb15a1 +ZzQQStsSuNZbERiPspyQ6+sV+aF8HuoTiHtRjxlsYmyBc+P7tqCthsVgFchoGNV5 +xOispaA+HKfE9d1EEgzzh4qU+7tFeYzn7qq4hT37KcuKybfG9DLOJyOqs9+lhBmd +jHUrw4Y+OGOywXImxS8bV2V3QlVTO2kOT3l6/AtbPQ0SXsK5rmqMYPFCMYOmULMd +FembJ6jEBaJB604Sz1vOElf5/qOY1gPszQpvP+GXKMn3YhTmmX4puqu4vGq2H4Lh +Na8cjUqFEn5xPEtDf1a3N/Ygm8B/5zfTtmTXZMKVNLfVbg//vfZsr1xVBmqqG2Zk +PUxZ9pyRn25zAYIgIC6DBgmM32S5Zl0axeoWIHdgOTAOwqLLF/ZFozdVSN+skJhl +q1ndTw+MmVHi19HDUt8YfIICn8g+Dw9NRdX+VjXrRMBweCL41JzHzP6YKo73VJty +WIVt/LwH2sxdGH0QeP6+Dajel3/ouXAcS31ZpeWZ0QUfa5I0DOn50mvZSuaC0m7Z +baHBGqawq9EKdjbU5WN6popOCWCx23BZDorKmPGBQ4x54LxfUeet0HAYjsvzUZ0q +a/AC6pMMHUjG4i5dPhOAhjxkpwfhLjIBTC8VrelGfap50Mlst5InCjClUsSKg43w +tsRg0HQflxYsPXUGtBqxAoIBAQD29Q6gPNsd7JMzmVzK7fy5YaKq7aI/vOU/tp7U +LQqhBfsy70U3UNsclsQQ/1RG2cgpFVctlpE8ITpFqSlUDRtlvrK9NaIPEYBV84Xn +bWPqFjGiRowuXiCR8axbPnsEquO2+VHNuVD/PsCocvbpqtpbfEGmcuyWiO4uFMyW +beOcRxEsn3U7ABZGDIhxYjpmkiVMu1pRjj+ddqtNMia398K86c3ETRg+zrHnpzio +kuhxv6TjtnXT1ibI0yKzdQ5sKpa3DYUN/wJgeFVHsSuCZjCzufcMdjCjG3kTTljT +FrMbweISSTJNs6KsbCqzmc6dmpwuU1ySX07F9sgdJ/7lBCXTAoIBAQDO7wy9Zr8C +Ridte1EaMP04OSfqc2Jha/U2te47R1VDyHKzpFHvHwWHqFng6J181M9iubUsh/63 +d12uyjZoToMgT7AJ7PMwbEKJbMxpIjqF7Dp1aM11TjRq0mQbZ3wkeuYdK7jYc92j +X4hF+tJpCYo/0lMA0UFYhMyUPbi33xqlJKLghVncBYmKHluf3k0g73eRg4MGlYHG +tZ5qVdgwFTztpBb4ySJMOYmjb/OoYPdfoLC8P4ZoHiax4MU30R8b/oGCmYzwOsMC +dNz9dZgeCFo3+lrzMUnIh3y+YcGo2JJ9ZW1LxGvEIU0aEQ7mSPV8k7m3QiH2MiPQ +P0t17w1hQP+NAoIBAHJch1pi9CGGZaB2e8cpsGf0s8yt4P3dLthzbFfbR9nLmEk9 +DnOQSPeTRdaNNuzce1mzHTzqRfVvebm6nX3j1/Uk+0atqI+Lzj9/V1oViThk8LUy +MEZkpnaPUP6sD3HY5TzddilrkPuyhqs7GeaZjSbigtBe1frcDFhgn2FmIApFysk8 +SqB46NelhCXllB/du9ItzKSJ2CHGS4ujFtUIsjCjoPsvrHOhajdZc950sZnDYstk +umnP+QP06lPqeDRVAJhidWRG3EXqU6uwevKW+iSwkJw/u0Q9O7NaC74s++J1xYgs +R1Q+RK3OJXQoXMsVRxAY4HyUEDmSj5cY52wMoKsCggEAQMXu9PJOY8XV3Z02G76t +5IVviyGm79u9G+0Cryd69watcLHEu9a4AmieCZqGgWaTq9F5doDzKDaC6o19TlUV +Em4fKlwzGzsn8KBPs7D1JKp2+f1eIpPiMHW+xB02bKzTjtn6uDY8cEEdBNqoNhy4 +W5XYSW82xyB6cQSI53U8f+jh2umi4Q4SqVsrTvVkqySKBtBlmQ//WVXMSniofRSI +x9IPJry+saFpBfGrEU+Y3yQLbkFsLvcRIai70ubwl/CoVVr/FMsv83rlGalPfkcb +Bl6lTW5mLBDM6ULsPY/c+sde2NKY8QGDgt9IDKlVvjL3dPeMbeXv8+V8F2RGieSw +mQKCAQEA1n4t4CLKZMWZ4oonBEp8u9rjK23ObCSxANsfoIw94v2zHTFqLraFg9aH +s+DIc6M/XWX+Ie55v1QuYt8LrMtc9/rtOJdISybAtnheSqjQ+IkTEvBab/8pqhYg +Jhv/RxmBnCLwiIzRGVpxv9/5bbKXq1JKgdQKO3RBG53lcFFMO1O5srhKqj2KgAHv +XCQxBmtj83e6gp6hvUOU4YC2aKyL9QqNVndGzPJikEguyvJPUFw6RoB0StVQnzLY +UOgIH+8VTjzL90nQ/JitVT8uGdw1Ge5xJfCXDe/PEYpDsY65DKPrIMWFeQs6T/Mb +nvNeBReQOuYYpcCc2GM96RMUz3iyoA== +-----END PRIVATE KEY----- From 7c8a9d53e74f5192b54cce577d685b81487c1a60 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 26 Aug 2023 08:58:57 +0800 Subject: [PATCH 710/848] test3: more http3 tests fixes. --- .travis.yml | 5 +++-- t/023-rewrite/socket-keepalive.t | 4 ++-- t/068-socket-keepalive.t | 2 +- t/163-exit-worker-hup.t | 3 +++ 4 files changed, 9 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index f70f825574..f8558c1ea0 100644 --- a/.travis.yml +++ b/.travis.yml @@ -26,7 +26,6 @@ addons: - libgd-dev - time - cmake - - curl - libunwind-dev - libpsl0 - wget @@ -85,6 +84,7 @@ install: - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + - wget https://github.com/zhuizhuhaomeng/curl-http3/releases/download/v1.0.0/curl-h3.tar.gz - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx @@ -115,7 +115,8 @@ before_script: - mysql -uroot -e "create database ngx_test; CREATE USER 'ngx_test'@'%' IDENTIFIED BY 'ngx_test'; grant all on ngx_test.* to 'ngx_test'@'%'; flush privileges;" script: - - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH + - sudo tar -C / -xf curl-h3.tar.gz + - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:/opt/curl-h3/bin:$PATH - ngx-releng > check.txt || true - lines=`wc -l check.txt | awk '{print $1}'`; if [ $lines -gt 5 ]; then cat check.txt; exit 1; fi - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT diff --git a/t/023-rewrite/socket-keepalive.t b/t/023-rewrite/socket-keepalive.t index 5316af6be5..5884c02b8e 100644 --- a/t/023-rewrite/socket-keepalive.t +++ b/t/023-rewrite/socket-keepalive.t @@ -176,7 +176,7 @@ received: OK === TEST 3: upstream sockets close prematurely ---- quic_max_idle_timeout: 1.1 +--- no_http3 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -254,7 +254,7 @@ done === TEST 4: http keepalive ---- quic_max_idle_timeout: 1.1 +--- no_http3 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index fe8dab152f..5ca94e4b72 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -168,7 +168,7 @@ received: OK === TEST 3: upstream sockets close prematurely ---- quic_max_idle_timeout: 1.2 +--- no_http3 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config diff --git a/t/163-exit-worker-hup.t b/t/163-exit-worker-hup.t index a54d81db14..cd2e5ce649 100644 --- a/t/163-exit-worker-hup.t +++ b/t/163-exit-worker-hup.t @@ -6,6 +6,9 @@ BEGIN { if ($ENV{TEST_NGINX_CHECK_LEAK}) { $SkipReason = "unavailable for the hup tests"; + } elsif ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support hub reload"; + } else { $ENV{TEST_NGINX_USE_HUP} = 1; undef $ENV{TEST_NGINX_USE_STAP}; From c27df5c6ba842c3edc764cc1480b94d0203621a9 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 3 Dec 2021 23:05:59 +0800 Subject: [PATCH 711/848] tests: fixed test cases for http2 when running with TEST_NGINX_USE_HTTP2=1. (#62) --- .travis.yml | 9 +- t/014-bugs.t | 3 + t/015-status.t | 1 + t/016-resp-header.t | 14 +-- t/023-rewrite/client-abort.t | 2 + t/023-rewrite/on-abort.t | 2 + t/023-rewrite/redirect.t | 2 +- t/023-rewrite/tcp-socket-timeout.t | 1 + t/023-rewrite/tcp-socket.t | 22 +++++ t/024-access/client-abort.t | 2 + t/024-access/on-abort.t | 2 + t/024-access/redirect.t | 2 +- t/028-req-header.t | 42 ++++++--- t/033-ctx.t | 4 +- t/041-header-filter.t | 76 ++++++++-------- t/056-flush.t | 13 ++- t/057-flush-timeout.t | 1 + t/058-tcp-socket.t | 54 +++++++++++ t/065-tcp-socket-timeout.t | 1 + t/066-socket-receiveuntil.t | 17 ++++ t/067-req-socket.t | 2 + t/082-body-filter-2.t | 18 +++- t/084-inclusive-receiveuntil.t | 10 ++ t/088-req-method.t | 3 + t/091-coroutine.t | 8 +- t/100-client-abort.t | 6 +- t/101-on-abort.t | 6 +- t/103-req-http-ver.t | 2 + t/106-timer.t | 1 + t/108-timer-safe.t | 1 + t/132-lua-blocks.t | 1 + t/138-balancer-upstream-bind.t | 141 +++++++++++++++++++++++++++++ t/138-balancer.t | 1 + t/147-tcp-socket-timeouts.t | 1 + t/156-slow-network.t | 1 + t/163-signal.t | 2 +- t/166-worker-thread.t | 2 + 37 files changed, 394 insertions(+), 82 deletions(-) create mode 100644 t/138-balancer-upstream-bind.t diff --git a/.travis.yml b/.travis.yml index f8558c1ea0..48d92855c0 100644 --- a/.travis.yml +++ b/.travis.yml @@ -63,11 +63,12 @@ env: - TEST_NGINX_SLEEP=0.006 - MALLOC_PERTURB_=9 jobs: - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f - - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d + #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f + #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y - NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y + #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s TEST_NGINX_USE_HTTP2=1 services: - memcached @@ -80,10 +81,10 @@ before_install: - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) - pyenv global 2.7 install: - - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - wget https://github.com/zhuizhuhaomeng/curl-http3/releases/download/v1.0.0/curl-h3.tar.gz - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty diff --git a/t/014-bugs.t b/t/014-bugs.t index 1f140be51d..f4e596640e 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -731,6 +731,7 @@ Content-Type: application/json; charset=utf-8 === TEST 32: hang on upstream_next (from kindy) +--- no_http2 --- no_check_leak --- http_config upstream xx { @@ -807,6 +808,8 @@ so need to skip for http3 [alert] --- error_log eval qr/(?:send|recv)\(\) failed \(\d+: Connection refused\) while resolving/ +--- curl_error eval +qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received/ diff --git a/t/015-status.t b/t/015-status.t index 43faa204a7..aa816c08d6 100644 --- a/t/015-status.t +++ b/t/015-status.t @@ -234,6 +234,7 @@ GET /t --- no_error_log [error] --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 60c63929ae..ac0c52eeef 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -65,7 +65,7 @@ GET /read Content-Length: 3 --- response_body chop Hel ---- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- skip_eval: 3:defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2}) @@ -117,7 +117,7 @@ GET /read --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = qr/x-foo: a\r\n.*?x-foo: bc\r\n/ } else { $headers = qr/X-Foo: a\r\n.*?X-Foo: bc\r\n/ @@ -196,7 +196,7 @@ GET /read --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = "x-foo: a\r\n.*?x-foo: abc\r\n" } else { $headers = "X-Foo: a\r\n.*?X-Foo: abc\r\n" @@ -222,7 +222,7 @@ Hello --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = ".*foo: a\r foo: b.*"; } else { @@ -250,7 +250,7 @@ $headers; --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = ".*foo: a\r foo: b.*"; } else { @@ -1197,7 +1197,7 @@ Foo: /t/ --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = "cache-control: private" } else { $headers = "cache-Control: private" @@ -1552,7 +1552,7 @@ hi --- error_log my Transfer-Encoding: chunked my transfer-encoding: chunked ---- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} +--- skip_eval: 6:defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2}) diff --git a/t/023-rewrite/client-abort.t b/t/023-rewrite/client-abort.t index 963717b012..61ada3a1b0 100644 --- a/t/023-rewrite/client-abort.t +++ b/t/023-rewrite/client-abort.t @@ -21,6 +21,8 @@ repeat_each(2); if (defined $ENV{TEST_NGINX_USE_HTTP3}) { plan(skip_all => "HTTP3 does not support client abort"); +} elsif (defined $ENV{TEST_NGINX_USE_HTTP2}) { + plan(skip_all => "HTTP2 does not support client abort"); } else { plan tests => repeat_each() * (blocks() * 3 - 1); } diff --git a/t/023-rewrite/on-abort.t b/t/023-rewrite/on-abort.t index d0bae78ee8..0535732a0b 100644 --- a/t/023-rewrite/on-abort.t +++ b/t/023-rewrite/on-abort.t @@ -21,6 +21,8 @@ repeat_each(2); if (defined $ENV{TEST_NGINX_USE_HTTP3}) { plan(skip_all => "HTTP3 does not support on_abort"); +} elsif (defined $ENV{TEST_NGINX_USE_HTTP2}) { + plan(skip_all => "HTTP2 does not support on_abort"); } else { plan tests => repeat_each() * (blocks() * 4 + 15); } diff --git a/t/023-rewrite/redirect.t b/t/023-rewrite/redirect.t index e387ee8fb7..2b3dccb0c0 100644 --- a/t/023-rewrite/redirect.t +++ b/t/023-rewrite/redirect.t @@ -122,7 +122,7 @@ GET /read --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = "location: /foo\r\n" } else { $headers = "Location: /foo\r\n" diff --git a/t/023-rewrite/tcp-socket-timeout.t b/t/023-rewrite/tcp-socket-timeout.t index e713bb5755..9be8081321 100644 --- a/t/023-rewrite/tcp-socket-timeout.t +++ b/t/023-rewrite/tcp-socket-timeout.t @@ -16,6 +16,7 @@ BEGIN { $ENV{MOCKEAGAIN} = 'w'; } + delete($ENV{TEST_NGINX_USE_HTTP2}); $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; $ENV{MOCKEAGAIN_WRITE_TIMEOUT_PATTERN} = 'get helloworld'; } diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index b4bf7a6b78..9ca66c64bb 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -86,6 +86,7 @@ failed to receive a line: closed [] close: 1 nil --- no_error_log [error] +--- no_http2 @@ -155,6 +156,7 @@ failed to receive a line: closed [foo] closed --- no_error_log [error] +--- no_http2 @@ -197,6 +199,7 @@ connected: nil failed to send request: closed --- error_log attempt to send data on a closed socket: +--- no_http2 @@ -521,6 +524,7 @@ failed to receive a line: closed close: 1 nil --- no_error_log [error] +--- no_http2 @@ -591,6 +595,7 @@ close: 1 nil " --- no_error_log [error] +--- no_http2 @@ -672,6 +677,7 @@ close: 1 nil " --- no_error_log [error] +--- no_http2 @@ -749,6 +755,7 @@ close: 1 nil " --- no_error_log [error] +--- no_http2 @@ -827,6 +834,7 @@ close: 1 nil " --- no_error_log [error] +--- no_http2 @@ -898,6 +906,7 @@ failed to receive a line: closed [] close: 1 nil --- no_error_log [error] +--- no_http2 @@ -967,6 +976,7 @@ failed to receive a line: closed [] close: 1 nil --- no_error_log [error] +--- no_http2 @@ -1077,6 +1087,7 @@ close: 1 nil " --- no_error_log [error] +--- no_http2 @@ -1521,6 +1532,7 @@ GET /t 2: close: 1 nil --- no_error_log [error] +--- no_http2 @@ -1593,6 +1605,7 @@ failed to receive a line: closed [] close: 1 nil --- no_error_log [error] +--- no_http2 @@ -1656,6 +1669,7 @@ GET /t bad argument #1 to 'send' (bad data type nil found) --- curl_error curl: (52) Empty reply from server +--- no_http2 @@ -1719,6 +1733,7 @@ GET /t bad argument #1 to 'send' (bad data type boolean found) --- curl_error curl: (52) Empty reply from server +--- no_http2 @@ -1782,6 +1797,7 @@ GET /t bad argument #1 to 'send' (bad data type userdata found) --- curl_error curl: (52) Empty reply from server +--- no_http2 @@ -1851,6 +1867,7 @@ subrequest: 200, OK\r " --- no_error_log [error] +--- no_http2 @@ -1922,6 +1939,7 @@ close: 1 nil --- no_error_log [error] --- SKIP +--- no_http2 @@ -1982,6 +2000,7 @@ receive(0): [] close: 1 nil --- no_error_log [error] +--- no_http2 @@ -2042,6 +2061,7 @@ send(""): 0 close: 1 nil --- no_error_log [error] +--- no_http2 @@ -2098,6 +2118,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):7: bad request/ --- no_error_log [alert] +--- no_http2 @@ -2157,6 +2178,7 @@ qr/runtime error: rewrite_by_lua\(nginx\.conf:\d+\):14: bad request/ --- no_error_log [alert] +--- no_http2 diff --git a/t/024-access/client-abort.t b/t/024-access/client-abort.t index 99f1b810fd..e4abb4ebd9 100644 --- a/t/024-access/client-abort.t +++ b/t/024-access/client-abort.t @@ -5,6 +5,8 @@ our $SkipReason; BEGIN { if ($ENV{TEST_NGINX_USE_HTTP3}) { $SkipReason = "http3 does not support ngx.req.socket and lua_check_client_abort"; + } elsif ($ENV{TEST_NGINX_USE_HTTP2}) { + $SkipReason = "http2 does not support ngx.req.socket and lua_check_client_abort"; } } diff --git a/t/024-access/on-abort.t b/t/024-access/on-abort.t index 4d58779742..70637ba862 100644 --- a/t/024-access/on-abort.t +++ b/t/024-access/on-abort.t @@ -21,6 +21,8 @@ repeat_each(2); if (defined $ENV{TEST_NGINX_USE_HTTP3}) { plan(skip_all => "HTTP3 does not support on_abort"); +} elsif (defined $ENV{TEST_NGINX_USE_HTTP2}) { + plan(skip_all => "HTTP2 does not support on_abort"); } else { plan tests => repeat_each() * (blocks() * 4 + 15); } diff --git a/t/024-access/redirect.t b/t/024-access/redirect.t index 84b64c5f0b..e8609af060 100644 --- a/t/024-access/redirect.t +++ b/t/024-access/redirect.t @@ -122,7 +122,7 @@ GET /read --- raw_response_headers_like eval my $headers; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { $headers = "location: /foo\r\n" } else { $headers = "Location: /foo\r\n" diff --git a/t/028-req-header.t b/t/028-req-header.t index b9c5e97fe0..21da3fc1ad 100644 --- a/t/028-req-header.t +++ b/t/028-req-header.t @@ -59,7 +59,7 @@ lua exceeding request header limit for k, v in pairs(headers) do h[k] = v end - if (ngx.req.http_version() == 3) then + if (ngx.req.http_version() == 3 or ngx.req.http_version() == 2) then ngx.say("Foo: ", h["foo"] or "nil") ngx.say("Bar: ", h["bar"] or "nil") else @@ -133,6 +133,7 @@ Foo: "a" x 2048 --- timeout: 15 --- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -152,6 +153,7 @@ Foo: "a" x 2048 --- timeout: 15 --- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -493,6 +495,7 @@ for my $k (@k) { --- error_log lua exceeding request header limit 101 > 100 --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -532,7 +535,7 @@ $s --- response_body eval my @k; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { push @k, "host: localhost\n"; } my $i = 1; @@ -542,7 +545,7 @@ while ($i <= 98) { } my $found_headers = "found 99 headers\n"; -if (!defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (!defined($ENV{TEST_NGINX_USE_HTTP3}) && !defined($ENV{TEST_NGINX_USE_HTTP2})) { push @k, "connection: close\n"; push @k, "host: localhost\n"; $found_headers = "found 100 headers\n"; @@ -615,6 +618,7 @@ for my $k (@k) { --- error_log lua exceeding request header limit 103 > 102 --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -650,7 +654,7 @@ while ($i <= 100) { $s --- response_body eval my @k; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { push @k, "host: localhost\n"; } my $i = 1; @@ -659,7 +663,7 @@ while ($i <= 100) { $i++; } -if (!defined($ENV{TEST_NGINX_USE_HTTP3})) { +if (!defined($ENV{TEST_NGINX_USE_HTTP3}) && !defined($ENV{TEST_NGINX_USE_HTTP2})) { push @k, "connection: close\n"; push @k, "host: localhost\n"; } @@ -710,7 +714,7 @@ while ($i <= 105) { $s --- response_body eval my @k; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3}) || defined($ENV{TEST_NGINX_USE_HTTP2})) { push @k, "host: localhost\n"; } my $i = 1; @@ -718,7 +722,7 @@ while ($i <= 105) { push @k, "x-$i"; $i++; } -if (!defined($ENV{TEST_NGINX_USE_HTTP3})) { +if (!defined($ENV{TEST_NGINX_USE_HTTP3}) && !defined($ENV{TEST_NGINX_USE_HTTP2})) { push @k, "connection: close\n"; push @k, "host: localhost\n"; } @@ -1015,7 +1019,7 @@ My-Foo: bar Bar: baz --- response_body eval my $body; -if ($ENV{TEST_NGINX_USE_HTTP3}) { +if ($ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_HTTP2}) { $body = "bar: baz host: localhost my-foo: bar @@ -1101,7 +1105,7 @@ my $s = "GET /back HTTP/1.0\r Host: foo\r Connection: close\r\n"; -if (defined ($ENV{TEST_NGINX_USE_HTTP3})) { +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { $s .= "user-agent: curl\r\n"; for my $i ('a' .. 'q') { $s .= $i . ": " . "$i\r\n" @@ -1269,7 +1273,7 @@ $s --- response_body eval my $body; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if ($ENV{TEST_NGINX_USE_HTTP3}) { $body = "GET /back HTTP/1.0\r Host: foo\r Connection: close\r @@ -1480,7 +1484,7 @@ Bar: baz --- response_body eval my $body; -if (defined $ENV{TEST_NGINX_USE_HTTP3}) { +if (defined($ENV{TEST_NGINX_USE_HTTP3})|| defined($ENV{TEST_NGINX_USE_HTTP2})) { $body="bar: baz host: localhost my-foo: bar @@ -2129,6 +2133,7 @@ lua exceeding request header limit 4 > 3 --- no_error_log [error] --- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -2166,6 +2171,7 @@ found 3 headers. lua exceeding request header limit [error] --- skip_eval: 4: $ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -2205,6 +2211,7 @@ lua exceeding request header limit 4 > 3 --- no_error_log [error] --- skip_eval: 4: $ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 @@ -2235,8 +2242,17 @@ while ($i <= 1) { $i++; } $s ---- response_body -found 3 headers. +--- response_body eval +my $body; +if (!defined $ENV{TEST_NGINX_USE_HTTP2}) { + $body = "found 3 headers. +"; +} else { + $body = "found 2 headers. +"; +} + +$body; --- timeout: 4 --- no_error_log lua exceeding request header limit diff --git a/t/033-ctx.t b/t/033-ctx.t index 1f879191b3..6133cee2bb 100644 --- a/t/033-ctx.t +++ b/t/033-ctx.t @@ -278,8 +278,8 @@ GET /t [error] --- error_log ngx.ctx = 32 ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ diff --git a/t/041-header-filter.t b/t/041-header-filter.t index a6ee25c8b2..d36d3e025c 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -124,8 +124,8 @@ Hi GET /read --- error_code --- response_body ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -467,8 +467,8 @@ GET /lua failed to run header_filter_by_lua*: header_filter_by_lua(nginx.conf:47):2: Something bad --- no_error_log [alert] ---- curl_error -curl: (56) Failure when receiving data from the peer +--- curl_error eval +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server/ @@ -491,8 +491,8 @@ GET /lua failed to run header_filter_by_lua*: unknown reason --- no_error_log [alert] ---- curl_error -curl: (56) Failure when receiving data from the peer +--- curl_error eval +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server/ @@ -507,8 +507,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -523,8 +523,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -539,8 +539,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -555,8 +555,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -571,8 +571,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -591,8 +591,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -611,8 +611,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -627,8 +627,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -643,8 +643,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -659,8 +659,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -693,8 +693,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -717,8 +717,8 @@ if (defined $ENV{TEST_NGINX_USE_HTTP3}) { } $err_log; ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -733,8 +733,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -749,8 +749,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -798,8 +798,8 @@ stack traceback: in function 'error' in function 'bar' in function 'foo' ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -818,8 +818,8 @@ GET /lua?a=1&b=2 --- ignore_response --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ diff --git a/t/056-flush.t b/t/056-flush.t index aa409ad51c..4376b18930 100644 --- a/t/056-flush.t +++ b/t/056-flush.t @@ -513,10 +513,19 @@ GET /test --- response_body eval "a" x 200 --- error_log eval -[ +my @errlog; +if (defined $ENV{TEST_NGINX_USE_HTTP2}) { + @errlog = [ +qr/lua writes elapsed 0\.[7-9]\d+ sec/, +qr/lua flush requires waiting: buffered 0x[0-9a-f]+, delayed:1/, +]; +} else { + @errlog = [ qr/lua writes elapsed [12](?:\.\d+)? sec/, qr/lua flush requires waiting: buffered 0x[0-9a-f]+, delayed:1/, -] +]; +} +@errlog; --- no_error_log [error] diff --git a/t/057-flush-timeout.t b/t/057-flush-timeout.t index 4ef104f707..8f0b7790a0 100644 --- a/t/057-flush-timeout.t +++ b/t/057-flush-timeout.t @@ -19,6 +19,7 @@ BEGIN { $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; $ENV{MOCKEAGAIN_WRITE_TIMEOUT_PATTERN} = 'hello, world'; $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + delete($ENV{TEST_NGINX_USE_HTTP2}); if ($ENV{TEST_NGINX_USE_HTTP3}) { $SkipReason = "HTTP3 does not support mockeagain"; diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 14995bf9fe..5602140fe1 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -21,6 +21,7 @@ run_tests(); __DATA__ === TEST 1: sanity +--- no_http2 --- config server_tokens off; location /t { @@ -90,6 +91,7 @@ close: 1 nil === TEST 2: no trailing newline +--- no_http2 --- config server_tokens off; location /t { @@ -257,6 +259,7 @@ second line received: (?:Date|Server): .*? === TEST 5: connection refused (tcp) +--- no_http2 --- config location /test { content_by_lua ' @@ -327,6 +330,7 @@ lua tcp socket connect timed out, when connecting to 127.0.0.2:12345 === TEST 7: not closed manually +--- no_http2 --- config server_tokens off; location /t { @@ -441,6 +445,7 @@ attempt to send data on a closed socket === TEST 10: explicit *l pattern for receive +--- no_http2 --- config server_tokens off; location /t { @@ -509,6 +514,7 @@ close: 1 nil === TEST 11: *a pattern for receive +--- no_http2 --- config server_tokens off; location /t { @@ -577,6 +583,7 @@ close: 1 nil === TEST 12: mixing *a and *l patterns for receive +--- no_http2 --- config server_tokens off; location /t { @@ -656,6 +663,7 @@ close: 1 nil === TEST 13: receive by chunks +--- no_http2 --- timeout: 5 --- config server_tokens off; @@ -731,6 +739,7 @@ close: 1 nil === TEST 14: receive by chunks (very small buffer) +--- no_http2 --- timeout: 5 --- config server_tokens off; @@ -807,6 +816,7 @@ close: 1 nil === TEST 15: line reading (very small buffer) +--- no_http2 --- config server_tokens off; lua_socket_buffer_size 1; @@ -876,6 +886,7 @@ close: 1 nil === TEST 16: ngx.socket.connect (working) +--- no_http2 --- config server_tokens off; location /t { @@ -982,6 +993,7 @@ qr/connect\(\) failed \(\d+: Connection refused\)/ === TEST 18: receive by chunks (stringified size) +--- no_http2 --- config server_tokens off; location /t { @@ -1056,6 +1068,7 @@ close: 1 nil === TEST 19: cannot survive across request boundary (send) +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -1115,6 +1128,7 @@ received: OK|failed to send request: closed)\$" === TEST 20: cannot survive across request boundary (receive) +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -1190,6 +1204,7 @@ received: OK|failed to receive a line: closed \[nil\])$/ === TEST 21: cannot survive across request boundary (close) +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -1259,6 +1274,7 @@ received: OK|failed to close: closed)$/ === TEST 22: cannot survive across request boundary (connect) +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -1335,6 +1351,7 @@ lua reuse socket upstream ctx === TEST 23: connect again immediately +--- no_http2 --- config server_tokens off; location /t { @@ -1403,6 +1420,7 @@ close: 1 nil === TEST 24: two sockets mix together +--- no_http2 --- config server_tokens off; location /t { @@ -1494,6 +1512,7 @@ GET /t === TEST 25: send tables of string fragments (with integers too) +--- no_http2 --- config server_tokens off; location /t { @@ -1564,6 +1583,7 @@ close: 1 nil === TEST 26: send tables of string fragments (bad type "nil") +--- no_http2 --- config server_tokens off; location /t { @@ -1625,6 +1645,7 @@ curl: (52) Empty reply from server === TEST 27: send tables of string fragments (bad type "boolean") +--- no_http2 --- config server_tokens off; location /t { @@ -1686,6 +1707,7 @@ curl: (52) Empty reply from server === TEST 28: send tables of string fragments (bad type ngx.null) +--- no_http2 --- config server_tokens off; location /t { @@ -1747,6 +1769,7 @@ curl: (52) Empty reply from server === TEST 29: cosocket before location capture (tcpsock:send did not clear u->waiting) +--- no_http2 --- config server_tokens off; location /t { @@ -1814,6 +1837,7 @@ subrequest: 200, OK\r === TEST 30: CR in a line +--- no_http2 --- config server_tokens off; location /t { @@ -1883,6 +1907,7 @@ close: nil closed === TEST 31: receive(0) +--- no_http2 --- config server_tokens off; location /t { @@ -1941,6 +1966,7 @@ close: 1 nil === TEST 32: send("") +--- no_http2 --- config server_tokens off; location /t { @@ -2182,6 +2208,7 @@ lua tcp socket read timed out === TEST 37: successful reread after a read time out happen (receive -> receive) +--- no_http2 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2259,6 +2286,7 @@ lua tcp socket read timed out === TEST 38: successful reread after a read time out happen (receive -> receiveuntil) +--- no_http2 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2339,6 +2367,7 @@ lua tcp socket read timed out === TEST 39: successful reread after a read time out happen (receiveuntil -> receiveuntil) +--- no_http2 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2421,6 +2450,7 @@ lua tcp socket read timed out === TEST 40: successful reread after a read time out happen (receiveuntil -> receive) +--- no_http2 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -2501,6 +2531,7 @@ lua tcp socket read timed out === TEST 41: receive(0) +--- no_http2 --- config server_tokens off; location /t { @@ -2548,6 +2579,7 @@ close: 1 nil === TEST 42: empty options table +--- no_http2 --- config server_tokens off; location /t { @@ -2586,6 +2618,7 @@ close: 1 nil === TEST 43: u->coctx left over bug +--- no_http2 --- config server_tokens off; location = /t { @@ -2677,6 +2710,7 @@ lua clean up the timer for pending ngx.sleep === TEST 44: bad request tries to connect +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -2731,6 +2765,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):7: bad request/ === TEST 45: bad request tries to receive +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -2788,6 +2823,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 46: bad request tries to send +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -2845,6 +2881,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 47: bad request tries to close +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -2902,6 +2939,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 48: bad request tries to set keepalive +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -2959,6 +2997,7 @@ qr/runtime error: content_by_lua\(nginx\.conf:\d+\):14: bad request/ === TEST 49: bad request tries to receiveuntil +--- no_http2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" --- config @@ -3090,6 +3129,7 @@ could not cancel === TEST 52: tcp_nodelay on +--- no_http2 --- config tcp_nodelay on; server_tokens off; @@ -3163,6 +3203,7 @@ lua socket tcp_nodelay === TEST 53: tcp_nodelay off +--- no_http2 --- config tcp_nodelay off; server_tokens off; @@ -3354,6 +3395,7 @@ lua tcp socket connect timeout: 100 === TEST 56: reuse cleanup +--- no_http2 --- config server_tokens off; location /t { @@ -3418,6 +3460,7 @@ lua http cleanup reuse === TEST 57: reuse cleanup in ngx.timer (fake_request) +--- no_http2 --- config server_tokens off; location /t { @@ -3501,6 +3544,7 @@ lua http cleanup reuse === TEST 58: free cleanup in ngx.timer (without sock:close) +--- no_http2 --- config server_tokens off; location /t { @@ -3582,6 +3626,7 @@ total_send_bytes: 114 === TEST 59: reuse cleanup in subrequest +--- no_http2 --- config server_tokens off; location /t { @@ -3650,6 +3695,7 @@ lua http cleanup reuse === TEST 60: setkeepalive on socket already shutdown +--- no_http2 --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -3688,6 +3734,7 @@ failed to setkeepalive: closed === TEST 61: options_table is nil +--- no_http2 --- config location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -3794,6 +3841,7 @@ failed to connect: bad port number: 65536 === TEST 64: send boolean and nil +--- no_http2 --- config location /t { set $port $TEST_NGINX_SERVER_PORT; @@ -3856,6 +3904,7 @@ received: truefalsenil === TEST 65: receiveany method in cosocket +--- no_http2 --- config server_tokens off; location = /t { @@ -3989,6 +4038,7 @@ GET /t === TEST 67: receiveany with limited, max <= 0 +--- no_http2 --- config location = /t { set $port $TEST_NGINX_SERVER_PORT; @@ -4025,6 +4075,7 @@ GET /t === TEST 68: receiveany with limited, max is larger than data +--- no_http2 --- config server_tokens off; location = /t { @@ -4094,6 +4145,7 @@ lua tcp socket calling receiveany() method to read at most 128 bytes === TEST 69: receiveany with limited, max is smaller than data +--- no_http2 --- config server_tokens off; location = /t { @@ -4168,6 +4220,7 @@ lua tcp socket calling receiveany() method to read at most 7 bytes === TEST 70: send tables of string fragments (with floating point number too) +--- no_http2 --- config server_tokens off; location /t { @@ -4242,6 +4295,7 @@ close: 1 nil === TEST 71: send numbers the maximum number of significant digits is 14 in lua +--- no_http2 --- config server_tokens off; location /t { diff --git a/t/065-tcp-socket-timeout.t b/t/065-tcp-socket-timeout.t index e06dfb1fc8..14563f70c6 100644 --- a/t/065-tcp-socket-timeout.t +++ b/t/065-tcp-socket-timeout.t @@ -16,6 +16,7 @@ BEGIN { $ENV{MOCKEAGAIN} = 'w'; } + delete($ENV{TEST_NGINX_USE_HTTP2}); $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; $ENV{MOCKEAGAIN_WRITE_TIMEOUT_PATTERN} = 'get helloworld'; } diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 2d38fb5c37..9d9f2804b4 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -71,6 +71,7 @@ close: 1 nil === TEST 2: http read lines +--- no_http2 --- config server_tokens off; location /t { @@ -140,6 +141,7 @@ close: 1 nil === TEST 3: http read all the headers in a single run +--- no_http2 --- config server_tokens off; location /t { @@ -208,6 +210,7 @@ close: 1 nil === TEST 4: ambiguous boundary patterns (abcabd) +--- no_http2 --- config server_tokens off; location /t { @@ -279,6 +282,7 @@ close: 1 nil === TEST 5: ambiguous boundary patterns (aa) +--- no_http2 --- config server_tokens off; location /t { @@ -350,6 +354,7 @@ close: 1 nil === TEST 6: ambiguous boundary patterns (aaa) +--- no_http2 --- config server_tokens off; location /t { @@ -421,6 +426,7 @@ close: 1 nil === TEST 7: ambiguous boundary patterns (aaaaad) +--- no_http2 --- config server_tokens off; location /t { @@ -492,6 +498,7 @@ close: 1 nil === TEST 8: ambiguous boundary patterns (aaaaad), small buffer, 2 bytes +--- no_http2 --- config server_tokens off; lua_socket_buffer_size 2; @@ -564,6 +571,7 @@ close: 1 nil === TEST 9: ambiguous boundary patterns (aaaaad), small buffer, 1 byte +--- no_http2 --- config server_tokens off; lua_socket_buffer_size 1; @@ -636,6 +644,7 @@ close: 1 nil === TEST 10: ambiguous boundary patterns (abcabdabcabe) +--- no_http2 --- config server_tokens off; location /t { @@ -707,6 +716,7 @@ close: 1 nil === TEST 11: ambiguous boundary patterns (abcabdabcabe 2) +--- no_http2 --- config server_tokens off; location /t { @@ -778,6 +788,7 @@ close: 1 nil === TEST 12: ambiguous boundary patterns (abcabdabcabe 3) +--- no_http2 --- config server_tokens off; location /t { @@ -849,6 +860,7 @@ close: 1 nil === TEST 13: ambiguous boundary patterns (abcabdabcabe 4) +--- no_http2 --- config server_tokens off; location /t { @@ -920,6 +932,7 @@ close: 1 nil === TEST 14: ambiguous boundary patterns (--abc) +--- no_http2 --- config server_tokens off; location /t { @@ -991,6 +1004,7 @@ close: 1 nil === TEST 15: ambiguous boundary patterns (--abc) +--- no_http2 --- config server_tokens off; location /t { @@ -1066,6 +1080,7 @@ close: 1 nil === TEST 16: ambiguous boundary patterns (--abc), small buffer +--- no_http2 --- config server_tokens off; location /t { @@ -1142,6 +1157,7 @@ close: 1 nil === TEST 17: ambiguous boundary patterns (--abc), small buffer, mixed by other reading calls +--- no_http2 --- config server_tokens off; location /t { @@ -1230,6 +1246,7 @@ close: 1 nil === TEST 18: ambiguous boundary patterns (abcabd), small buffer +--- no_http2 --- config server_tokens off; lua_socket_buffer_size 3; diff --git a/t/067-req-socket.t b/t/067-req-socket.t index fbaa0b3f41..9aff58b647 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -5,6 +5,8 @@ our $SkipReason; BEGIN { if ($ENV{TEST_NGINX_USE_HTTP3}) { $SkipReason = "http3 does not support ngx.req.socket"; + } elsif ($ENV{TEST_NGINX_USE_HTTP2}) { + $SkipReason = "http2 does not support ngx.req.socket"; } } diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t index d19f46f660..ee4fe56ece 100644 --- a/t/082-body-filter-2.t +++ b/t/082-body-filter-2.t @@ -3,11 +3,19 @@ our $SkipReason; BEGIN { - $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; - $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; - $ENV{MOCKEAGAIN}='w'; - if ($ENV{TEST_NGINX_USE_HTTP3}) { - $SkipReason = "http3 has bug about the hup reload"; + if ($ENV{TEST_NGINX_EVENT_TYPE} && $ENV{TEST_NGINX_EVENT_TYPE} ne 'poll') { + $SkipReason = "unavailable for the event type '$ENV{TEST_NGINX_EVENT_TYPE}'"; + + } elsif ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "http3 does not support mockeagain"; + + } elsif ($ENV{TEST_NGINX_USE_HTTP2}) { + $SkipReason = "http2 does not support mockeagain"; + + } else { + $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + $ENV{MOCKEAGAIN}='w' } } diff --git a/t/084-inclusive-receiveuntil.t b/t/084-inclusive-receiveuntil.t index 95d898501c..5921044b49 100644 --- a/t/084-inclusive-receiveuntil.t +++ b/t/084-inclusive-receiveuntil.t @@ -19,6 +19,7 @@ run_tests(); __DATA__ === TEST 1: ambiguous boundary patterns (abcabd) - inclusive mode +--- no_http2 --- config server_tokens off; location /t { @@ -91,6 +92,7 @@ close: 1 nil === TEST 2: ambiguous boundary patterns (abcabdabcabe 4) - inclusive mode +--- no_http2 --- config server_tokens off; location /t { @@ -162,6 +164,7 @@ close: 1 nil === TEST 3: ambiguous boundary patterns (abcabd) - inclusive mode - small buffers +--- no_http2 --- config server_tokens off; lua_socket_buffer_size 1; @@ -235,6 +238,7 @@ close: 1 nil === TEST 4: inclusive option value nil +--- no_http2 --- config server_tokens off; location /t { @@ -306,6 +310,7 @@ close: 1 nil === TEST 5: inclusive option value false +--- no_http2 --- config server_tokens off; location /t { @@ -377,6 +382,7 @@ close: 1 nil === TEST 6: inclusive option value true (aa) +--- no_http2 --- config server_tokens off; location /t { @@ -448,6 +454,7 @@ close: 1 nil === TEST 7: bad inclusive option value type +--- no_http2 --- config server_tokens off; location /t { @@ -517,6 +524,7 @@ curl: (52) Empty reply from server === TEST 8: bad option table +--- no_http2 --- config server_tokens off; location /t { @@ -586,6 +594,7 @@ curl: (52) Empty reply from server === TEST 9: ambiguous boundary patterns (--abc), small buffer +--- no_http2 --- config server_tokens off; location /t { @@ -662,6 +671,7 @@ close: 1 nil === TEST 10: ambiguous boundary patterns (--abc), small buffer, mixed by other reading calls +--- no_http2 --- config server_tokens off; location /t { diff --git a/t/088-req-method.t b/t/088-req-method.t index de3421662c..9e1499fb86 100644 --- a/t/088-req-method.t +++ b/t/088-req-method.t @@ -164,6 +164,7 @@ PUT === TEST 8: set GET to HEAD +--- no_http2 --- config location /t { rewrite_by_lua ' @@ -210,6 +211,7 @@ main: GET === TEST 10: set HEAD to GET XXX: does http3 do not support set HEAD to GET?? +--- no_http2 --- config location /t { rewrite_by_lua ' @@ -230,6 +232,7 @@ method: GET === TEST 11: set GET to WebDAV methods XXX: does http3 do not support change HEAD method? +--- no_http2 --- config location /t { content_by_lua ' diff --git a/t/091-coroutine.t b/t/091-coroutine.t index e8761045b8..2c10d54209 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -762,8 +762,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of header_filter_by_lua* ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ @@ -1699,8 +1699,8 @@ GET /t "stack traceback:", "in function 'co'" ] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ diff --git a/t/100-client-abort.t b/t/100-client-abort.t index fed01176c7..39d3244b16 100644 --- a/t/100-client-abort.t +++ b/t/100-client-abort.t @@ -1,8 +1,10 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: BEGIN { -if ($ENV{TEST_NGINX_USE_HTTP3}) { - $SkipReason = "client abort detect does not support in http3" + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "client abort detect does not support in http3"; + } elsif ($ENV{TEST_NGINX_USE_HTTP2}) { + $SkipReason = "client abort detect does not support in http2"; } } diff --git a/t/101-on-abort.t b/t/101-on-abort.t index f10027a47a..784f244e04 100644 --- a/t/101-on-abort.t +++ b/t/101-on-abort.t @@ -1,8 +1,10 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: BEGIN { -if ($ENV{TEST_NGINX_USE_HTTP3}) { - $SkipReason = "client abort detect does not support in http3" + if ($ENV{TEST_NGINX_USE_HTTP3}) { + $SkipReason = "client abort detect does not support in http3"; + } elsif ($ENV{TEST_NGINX_USE_HTTP2}) { + $SkipReason = "client abort detect does not support in http2"; } } diff --git a/t/103-req-http-ver.t b/t/103-req-http-ver.t index ebc65c9d38..73ecccae28 100644 --- a/t/103-req-http-ver.t +++ b/t/103-req-http-ver.t @@ -30,6 +30,8 @@ GET /t my $body; if (defined $ENV{TEST_NGINX_USE_HTTP3}) { $body="3\n"; +} elsif (defined $ENV{TEST_NGINX_USE_HTTP2}) { + $body="2\n"; } else { $body="1.1\n"; } diff --git a/t/106-timer.t b/t/106-timer.t index d455ea2add..a9b4aed7bd 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -228,6 +228,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:1[4-9]|2[0-6]?) === TEST 5: tcp cosocket in timer handler (short connections) +--- no_http2 --- config server_tokens off; diff --git a/t/108-timer-safe.t b/t/108-timer-safe.t index 387f203c3d..a23634cd3b 100644 --- a/t/108-timer-safe.t +++ b/t/108-timer-safe.t @@ -124,6 +124,7 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: 0\.(?:6[4-9]|7[0-6])/ === TEST 3: tcp cosocket in timer handler (short connections) +--- no_http2 --- config server_tokens off; location = /t { diff --git a/t/132-lua-blocks.t b/t/132-lua-blocks.t index 2a1644e532..35638a8489 100644 --- a/t/132-lua-blocks.t +++ b/t/132-lua-blocks.t @@ -166,6 +166,7 @@ ok === TEST 8: content_by_lua_block (cosockets) +--- no_http2 --- config server_tokens off; location = /t { diff --git a/t/138-balancer-upstream-bind.t b/t/138-balancer-upstream-bind.t new file mode 100644 index 0000000000..dc2c9665f3 --- /dev/null +++ b/t/138-balancer-upstream-bind.t @@ -0,0 +1,141 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_on(); +#workers(2); +#log_level('warn'); + +repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3); + +#no_diff(); +no_long_string(); +run_tests(); + +__DATA__ + +=== TEST 1: bind to empty +--- no_http2 +--- http_config + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; + + upstream backend { + server 127.0.0.1:$TEST_NGINX_SERVER_PORT; + } +--- config + set $proxy_local_addr ""; + proxy_bind $proxy_local_addr; + + location = /t { + proxy_pass http://backend/back; + } + + location = /back { + echo ok; + } + +--- request + GET /t +--- response_body +ok +--- no_error_log +[cirt] + + + +=== TEST 2: bind to 127.0.0.1 +--- no_http2 +--- http_config + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; + + upstream backend { + server 127.0.0.1:$TEST_NGINX_SERVER_PORT; + } +--- config + set $proxy_local_addr ""; + proxy_bind $proxy_local_addr; + + location = /t { + access_by_lua_block { + ngx.var.proxy_local_addr="127.0.0.1" + } + proxy_pass http://backend/back; + } + + location = /back { + echo ok; + } + +--- request + GET /t +--- response_body +ok +--- no_error_log +[cirt] + + + +=== TEST 3: bind to 127.0.0.10 +--- no_http2 +--- http_config + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; + + upstream backend { + server 127.0.0.1:$TEST_NGINX_SERVER_PORT; + } +--- config + set $proxy_local_addr ""; + proxy_bind $proxy_local_addr; + + location = /t { + access_by_lua_block { + ngx.var.proxy_local_addr="127.0.0.10" + } + proxy_pass http://backend/back; + } + + location = /back { + echo ok; + } + +--- request + GET /t +--- response_body +ok +--- no_error_log +[cirt] + + + +=== TEST 4: bind to not exist addr 100.100.100.100 +--- no_http2 +--- http_config + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; + + upstream backend { + server 127.0.0.1:$TEST_NGINX_SERVER_PORT; + } +--- config + set $proxy_local_addr ""; + proxy_bind $proxy_local_addr; + + location = /t { + access_by_lua_block { + ngx.var.proxy_local_addr="100.100.100.100" + } + proxy_pass http://backend/back; + } + + location = /back { + echo ok; + } + +--- request + GET /t +--- response_body +ok +--- error_log +bind(100.100.100.100) failed (99: Cannot assign requested address) diff --git a/t/138-balancer.t b/t/138-balancer.t index d9c943388e..8ccdee76c4 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -309,6 +309,7 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ === TEST 12: code cache off +--- no_http2 --- http_config lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; diff --git a/t/147-tcp-socket-timeouts.t b/t/147-tcp-socket-timeouts.t index 8566c0ba6f..ceddb61dcc 100644 --- a/t/147-tcp-socket-timeouts.t +++ b/t/147-tcp-socket-timeouts.t @@ -17,6 +17,7 @@ BEGIN { } $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + delete($ENV{TEST_NGINX_USE_HTTP2}); $ENV{MOCKEAGAIN_WRITE_TIMEOUT_PATTERN} = 'slowdata'; } diff --git a/t/156-slow-network.t b/t/156-slow-network.t index 2d80506085..d5897fb33e 100644 --- a/t/156-slow-network.t +++ b/t/156-slow-network.t @@ -15,6 +15,7 @@ BEGIN { } $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + delete($ENV{TEST_NGINX_USE_HTTP2}); } use Test::Nginx::Socket::Lua; diff --git a/t/163-signal.t b/t/163-signal.t index 2b85e12482..ec3a418d90 100644 --- a/t/163-signal.t +++ b/t/163-signal.t @@ -39,7 +39,7 @@ qr/\[notice\] \d+#\d+: exit$/ --- no_error_log eval qr/\[notice\] \d+#\d+: reconfiguring/ --- curl_error eval -qr/curl: \(28\) Operation timed out after 3\d+ milliseconds with 0 bytes received/ +qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received|curl: \(56\) Recv failure: Connection reset by peer/ diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index afe7f3c410..15e9d762eb 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -282,6 +282,7 @@ false : module 'hello' not found.* === TEST 10: the number of Lua VM exceeds the pool size +--- no_http2 --- quic_max_idle_timeout: 5 --- main_config thread_pool testpool threads=100; @@ -360,6 +361,7 @@ GET /t === TEST 11: kill uthread before worker thread callback +--- no_http2 --- quic_max_idle_timeout: 10 --- main_config thread_pool testpool threads=100; From 028852c1564d9f1ea5e395509c44b2825b77dfb7 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 2 Sep 2023 19:12:32 +0800 Subject: [PATCH 712/848] enable http3 test. --- .travis.yml | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/.travis.yml b/.travis.yml index 48d92855c0..5c366f0b96 100644 --- a/.travis.yml +++ b/.travis.yml @@ -27,9 +27,7 @@ addons: - time - cmake - libunwind-dev - - libpsl0 - wget - - libldap-2.4-2 - libbrotli1 cache: @@ -66,7 +64,7 @@ env: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y + #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y - NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s TEST_NGINX_USE_HTTP2=1 @@ -81,11 +79,13 @@ before_install: - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) - pyenv global 2.7 install: + - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - - wget https://github.com/zhuizhuhaomeng/curl-http3/releases/download/v1.0.0/curl-h3.tar.gz + - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz + - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx @@ -116,7 +116,7 @@ before_script: - mysql -uroot -e "create database ngx_test; CREATE USER 'ngx_test'@'%' IDENTIFIED BY 'ngx_test'; grant all on ngx_test.* to 'ngx_test'@'%'; flush privileges;" script: - - sudo tar -C / -xf curl-h3.tar.gz + - sudo tar -C / -xf curl-h3-x64-focal.tar.gz - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:/opt/curl-h3/bin:$PATH - ngx-releng > check.txt || true - lines=`wc -l check.txt | awk '{print $1}'`; if [ $lines -gt 5 ]; then cat check.txt; exit 1; fi @@ -137,13 +137,8 @@ script: - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. - if [ "$USE_PCRE2" != "Y" ]; then tar zxf download-cache/pcre-$PCRE_VER.tar.gz; cd pcre-$PCRE_VER/; ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - if [ "$USE_PCRE2" = "Y" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - - tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz - - cd openssl-$OPENSSL_VER/ - - patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch - - ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1) - - make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1) - - sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1) - - cd .. + - if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + - if [ -n "$BORINGSSL" ]; then sudo mkdir -p /opt/ssl && sudo tar -C /opt/ssl -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi - export NGX_BUILD_CC=$CC - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - sh util/build-with-dd.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) From 488ba271109fb19ce5bfb901ec2fc638a9e1de9c Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 18 Oct 2023 19:32:30 +0800 Subject: [PATCH 713/848] bugfix: failed to compile with clang. ``` src/ngx_http_lua_worker_thread.c:454:57: error: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion] worker_thread_ctx->is_abort = 1; ^ ~ ``` --- src/ngx_http_lua_worker_thread.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 9de59def75..3820d6377c 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -42,7 +42,7 @@ typedef struct { ngx_http_lua_co_ctx_t *wait_co_ctx; int n_args; int rc; - int is_abort:1; + ngx_uint_t is_abort:1; } ngx_http_lua_worker_thread_ctx_t; From 5ae767d504179fe1d2ca848fb65c321e22b082b3 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 23 Oct 2023 16:32:15 +0800 Subject: [PATCH 714/848] tests: build drizzle with python3. --- .travis.yml | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index 5c366f0b96..2fd8a03509 100644 --- a/.travis.yml +++ b/.travis.yml @@ -62,9 +62,9 @@ env: - MALLOC_PERTURB_=9 jobs: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y + - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y - NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s TEST_NGINX_USE_HTTP2=1 @@ -77,9 +77,8 @@ before_install: - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) - - pyenv global 2.7 install: - - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi + - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi @@ -113,7 +112,7 @@ install: - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 before_script: - - mysql -uroot -e "create database ngx_test; CREATE USER 'ngx_test'@'%' IDENTIFIED BY 'ngx_test'; grant all on ngx_test.* to 'ngx_test'@'%'; flush privileges;" + - mysql -uroot -e "create database ngx_test; CREATE USER 'ngx_test'@'%' IDENTIFIED WITH mysql_native_password BY 'ngx_test'; grant all on ngx_test.* to 'ngx_test'@'%'; flush privileges;" script: - sudo tar -C / -xf curl-h3-x64-focal.tar.gz From 51984aea047a3aaefb1c382e077a23d7f1952db5 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 25 Oct 2023 18:09:15 +0800 Subject: [PATCH 715/848] tests: check if the version number has been specified before downloading OpenSSL. (#2240) --- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 2fd8a03509..60e1a053d9 100644 --- a/.travis.yml +++ b/.travis.yml @@ -77,11 +77,12 @@ before_install: - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) + install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - - if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz From 25c9643cf42b71046fda31ec3b491a4ab9fe4868 Mon Sep 17 00:00:00 2001 From: Michael McQuade Date: Wed, 25 Oct 2023 05:13:22 -0500 Subject: [PATCH 716/848] docs: remove duplicated sentence. --- README.markdown | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index aaf53b1459..3aa9f592d8 100644 --- a/README.markdown +++ b/README.markdown @@ -4,13 +4,11 @@ Name ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers. This module is a core component of [OpenResty](https://openresty.org). If you are using this module, -then you are essentially using OpenResty. +then you are essentially using OpenResty :) *This module is not distributed with the Nginx source.* See [the installation instructions](#installation). -This is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :) - Table of Contents ================= From 87f41a9545c43e94a1d582e1db4eb65a7124a903 Mon Sep 17 00:00:00 2001 From: menglei Date: Thu, 26 Oct 2023 10:05:05 +0800 Subject: [PATCH 717/848] bugfix: fix some failed test case. --- t/066-socket-receiveuntil.t | 1 - t/082-body-filter.t | 1 - t/138-balancer-upstream-bind.t | 5 +++-- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 9d9f2804b4..3b1f280c29 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -1344,7 +1344,6 @@ this exposed a memory leak in receiveuntil ok --- no_error_log [error] -<<<<<<< HEAD diff --git a/t/082-body-filter.t b/t/082-body-filter.t index 54e7414cb8..49a1d76397 100644 --- a/t/082-body-filter.t +++ b/t/082-body-filter.t @@ -842,7 +842,6 @@ GET /lua --- ignore_response --- error_log API disabled in the context of body_filter_by_lua* -<<<<<<< HEAD diff --git a/t/138-balancer-upstream-bind.t b/t/138-balancer-upstream-bind.t index dc2c9665f3..442f37c8f3 100644 --- a/t/138-balancer-upstream-bind.t +++ b/t/138-balancer-upstream-bind.t @@ -135,7 +135,8 @@ ok --- request GET /t ---- response_body -ok +--- response_body_like chomp +500 Internal Server Error +--- error_code: 500 --- error_log bind(100.100.100.100) failed (99: Cannot assign requested address) From 7661f9b48a89ecc803ebd59a7b465e9c53c26ab8 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 26 Oct 2023 10:14:50 +0800 Subject: [PATCH 718/848] tests: t/166-ssl-client-hello.t: fixed test plan. --- t/166-ssl-client-hello.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index 445434289a..102cd2e303 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -11,7 +11,7 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); } else { - plan tests => repeat_each() * (blocks() * 6 + 6); + plan tests => repeat_each() * (blocks() * 6 + 8); } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); From db80075e0a318d719ff105808f2e69c07c5b4fb6 Mon Sep 17 00:00:00 2001 From: willmafh Date: Thu, 26 Oct 2023 21:34:11 +0800 Subject: [PATCH 719/848] doc: more accurate error messages. --- src/ngx_http_lua_socket_tcp.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 88fe923eb1..505614c47f 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -2413,7 +2413,7 @@ ngx_http_lua_socket_tcp_receive(lua_State *L) case LUA_TNUMBER: bytes = lua_tointeger(L, 2); if (bytes < 0) { - return luaL_argerror(L, 2, "bad pattern argument"); + return luaL_argerror(L, 2, "bad number argument"); } #if 1 @@ -2430,7 +2430,7 @@ ngx_http_lua_socket_tcp_receive(lua_State *L) break; default: - return luaL_argerror(L, 2, "bad pattern argument"); + return luaL_argerror(L, 2, "bad argument"); break; } @@ -4586,7 +4586,7 @@ ngx_http_lua_socket_receiveuntil_iterator(lua_State *L) n = lua_gettop(L); if (n > 1) { - return luaL_error(L, "expecting 0 or 1 arguments, " + return luaL_error(L, "expecting 0 or 1 argument, " "but seen %d", n); } @@ -5076,7 +5076,7 @@ ngx_http_lua_req_socket(lua_State *L) lua_pop(L, 1); } else { - return luaL_error(L, "expecting zero arguments, but got %d", + return luaL_error(L, "expecting 0 or 1 argument, but got %d", lua_gettop(L)); } From 86bea01b244938e0ab6eda780906c06fa62c3b5c Mon Sep 17 00:00:00 2001 From: swananan Date: Sun, 5 Nov 2023 20:52:20 +0800 Subject: [PATCH 720/848] changes: modify read body api limitation for HTTP/2 or HTTP/3 requests. --- README.markdown | 4 ++-- src/ngx_http_lua_accessby.c | 25 +++++++++++++++++++++---- src/ngx_http_lua_contentby.c | 25 +++++++++++++++++++++---- src/ngx_http_lua_req_body.c | 28 ++++++++++++++++++++++++---- t/023-rewrite/request_body.t | 2 ++ t/024-access/request_body.t | 2 ++ t/044-req-body.t | 4 +++- 7 files changed, 75 insertions(+), 15 deletions(-) diff --git a/README.markdown b/README.markdown index 3aa9f592d8..b6ccecf8a8 100644 --- a/README.markdown +++ b/README.markdown @@ -2722,7 +2722,7 @@ lua_need_request_body **phase:** *depends on usage* -Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. +Due to the stream processing feature of HTTP/2 or HTTP/3, this configuration could potentially block the entire request. Therefore, this configuration is effective only when HTTP/2 or HTTP/3 requests send content-length header. For requests with versions lower than HTTP/2, this configuration can still be used without any problems. Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code. @@ -5426,7 +5426,7 @@ Reads the client request body synchronously without blocking the Nginx event loo local args = ngx.req.get_post_args() ``` -Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. +Due to the stream processing feature of HTTP/2 or HTTP/3, this api could potentially block the entire request. Therefore, this api is effective only when HTTP/2 or HTTP/3 requests send content-length header. For requests with versions lower than HTTP/2, this api can still be used without any problems. If the request body is already read previously by turning on [lua_need_request_body](#lua_need_request_body) or by using other modules, then this function does not run and returns immediately. diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index fa6810d9ed..1669f161d6 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -136,11 +136,26 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) return NGX_DONE; } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { -#else if (llcf->force_read_body && !ctx->read_body_done) { + +#if (NGX_HTTP_V2) + if (r->main->stream && r->headers_in.content_length_n < 0) { + ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, + "disable lua_need_request_body, since " + "http2 read_body may break http2 stream process"); + goto done; + } +#endif + +#if (NGX_HTTP_V3) + if (r->http_version == NGX_HTTP_VERSION_30 + && r->headers_in.content_length_n < 0) + { + ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, + "disable lua_need_request_body, since " + "http2 read_body may break http2 stream process"); + goto done; + } #endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; @@ -159,6 +174,8 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) } } +done: + dd("calling access handler"); return llcf->access_handler(r); } diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index d1c3bc9fd9..38aebc3bb2 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -195,11 +195,26 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) return rc; } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { -#else if (llcf->force_read_body && !ctx->read_body_done) { + +#if (NGX_HTTP_V2) + if (r->main->stream && r->headers_in.content_length_n < 0) { + ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, + "disable lua_need_request_body, since " + "http2 read_body may break http2 stream process"); + goto done; + } +#endif + +#if (NGX_HTTP_V3) + if (r->http_version == NGX_HTTP_VERSION_30 + && r->headers_in.content_length_n < 0) + { + ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, + "disable lua_need_request_body, since " + "http2 read_body may break http2 stream process"); + goto done; + } #endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; @@ -219,6 +234,8 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) } } +done: + dd("setting entered"); ctx->entered_content_phase = 1; diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 3c452b3051..4dd105e144 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -87,8 +87,18 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) /* http2 read body may break http2 stream process */ #if (NGX_HTTP_V2) - if (r->main->stream) { - return luaL_error(L, "http2 requests are not supported yet"); + if (r->main->stream && r->headers_in.content_length_n < 0) { + return luaL_error(L, "http2 requests are not supported" + " without content-length header"); + } +#endif + +#if (NGX_HTTP_V3) + if (r->http_version == NGX_HTTP_VERSION_30 + && r->headers_in.content_length_n < 0) + { + return luaL_error(L, "http3 requests are not supported" + " without content-length header"); } #endif @@ -321,8 +331,18 @@ ngx_http_lua_ngx_req_get_body_file(lua_State *L) /* http2 read body may break http2 stream process */ #if (NGX_HTTP_V2) - if (r->main->stream) { - return luaL_error(L, "http2 requests are not supported yet"); + if (r->main->stream && r->headers_in.content_length_n < 0) { + return luaL_error(L, "http2 requests are not supported" + " without content-length header"); + } +#endif + +#if (NGX_HTTP_V3) + if (r->http_version == NGX_HTTP_VERSION_30 + && r->headers_in.content_length_n < 0) + { + return luaL_error(L, "http3 requests are not supported" + " without content-length header"); } #endif diff --git a/t/023-rewrite/request_body.t b/t/023-rewrite/request_body.t index f9af17236f..a721eba31e 100644 --- a/t/023-rewrite/request_body.t +++ b/t/023-rewrite/request_body.t @@ -187,6 +187,8 @@ http finalize request: 500, "/echo_body?" a:1, c:0 "POST /echo_body hello\x00\x01\x02 world\x03\x04\xff" +--- more_headers +Content-Length: --- response_body eval "nil" --- no_error_log diff --git a/t/024-access/request_body.t b/t/024-access/request_body.t index 4970e37aef..9086bf9e36 100644 --- a/t/024-access/request_body.t +++ b/t/024-access/request_body.t @@ -187,6 +187,8 @@ http finalize request: 500, "/echo_body?" a:1, c:0 "POST /echo_body hello\x00\x01\x02 world\x03\x04\xff" +--- more_headers +Content-Length: --- response_body eval "nil" --- no_error_log diff --git a/t/044-req-body.t b/t/044-req-body.t index 0c378c44aa..29cb914257 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -1787,5 +1787,7 @@ content length: 5 --- request POST /test hello, world +--- more_headers +Content-Length: --- error_code: 500 ---- error_log: http2 requests are not supported yet +--- error_log: http2 requests are not supported without content-length header From 187ad5c0509b152b4bceff388f38b02ae9395d01 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 9 Nov 2023 23:05:47 +0800 Subject: [PATCH 721/848] bugfix: failed to build when building without http2&http3 modules. --- src/ngx_http_lua_accessby.c | 4 ++++ src/ngx_http_lua_contentby.c | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 1669f161d6..2bf40aaa87 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -174,8 +174,12 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) } } +#if defined(NGX_HTTP_V3) || defined(NGX_HTTP_V2) + done: +#endif + dd("calling access handler"); return llcf->access_handler(r); } diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 38aebc3bb2..2014d52d8d 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -234,8 +234,12 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) } } +#if defined(NGX_HTTP_V3) || defined(NGX_HTTP_V2) + done: +#endif + dd("setting entered"); ctx->entered_content_phase = 1; From 339b08a4be9f98d3b88da5a88a478d359db01023 Mon Sep 17 00:00:00 2001 From: menglei Date: Sun, 12 Nov 2023 22:10:11 +0800 Subject: [PATCH 722/848] doc: make some debug message and comments accurate. --- src/ngx_http_lua_module.c | 16 ++++++++-------- src/ngx_http_lua_ssl_client_helloby.c | 2 +- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 6984fb087d..fb10bf933e 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1178,15 +1178,15 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) * lscf->srv.ssl_cert_chunkname = NULL; * lscf->srv.ssl_cert_src_key = NULL; * - * lscf->srv.ssl_session_store_handler = NULL; - * lscf->srv.ssl_session_store_src = { 0, NULL }; - * lscf->srv.ssl_session_store_chunkname = NULL; - * lscf->srv.ssl_session_store_src_key = NULL; + * lscf->srv.ssl_sess_store_handler = NULL; + * lscf->srv.ssl_sess_store_src = { 0, NULL }; + * lscf->srv.ssl_sess_store_chunkname = NULL; + * lscf->srv.ssl_sess_store_src_key = NULL; * - * lscf->srv.ssl_session_fetch_handler = NULL; - * lscf->srv.ssl_session_fetch_src = { 0, NULL }; - * lscf->srv.ssl_session_fetch_chunkname = NULL; - * lscf->srv.ssl_session_fetch_src_key = NULL; + * lscf->srv.ssl_sess_fetch_handler = NULL; + * lscf->srv.ssl_sess_fetch_src = { 0, NULL }; + * lscf->srv.ssl_sess_fetch_chunkname = NULL; + * lscf->srv.ssl_sess_fetch_src_key = NULL; * * lscf->balancer.handler = NULL; * lscf->balancer.src = { 0, NULL }; diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index a1dac61887..03ac430ef2 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -390,7 +390,7 @@ ngx_http_lua_ssl_client_hello_aborted(void *data) { ngx_http_lua_ssl_ctx_t *cctx = data; - dd("lua ssl client hello done"); + dd("lua ssl client hello aborted"); if (cctx->done) { /* completed successfully already */ From 577dfebba4f05796662f2be3b14a884a922a87ba Mon Sep 17 00:00:00 2001 From: masterlvng Date: Tue, 14 Nov 2023 21:17:45 +0800 Subject: [PATCH 723/848] feature: add max_bytes argument for get_body_data() function. --- README.markdown | 4 +++- doc/HttpLuaModule.wiki | 4 +++- src/ngx_http_lua_req_body.c | 32 ++++++++++++++++++++++++++------ t/010-request_body.t | 17 +++++++++++++++++ 4 files changed, 49 insertions(+), 8 deletions(-) diff --git a/README.markdown b/README.markdown index b6ccecf8a8..dce83887fa 100644 --- a/README.markdown +++ b/README.markdown @@ -5467,12 +5467,14 @@ See also [ngx.req.read_body](#ngxreqread_body). ngx.req.get_body_data --------------------- -**syntax:** *data = ngx.req.get_body_data()* +**syntax:** *data = ngx.req.get_body_data(max_bytes?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, log_by_lua** Retrieves in-memory request body data. It returns a Lua string rather than a Lua table holding all the parsed query arguments. Use the [ngx.req.get_post_args](#ngxreqget_post_args) function instead if a Lua table is required. +The optional `max_bytes` argument can be used when you don't need the entire body. + This function returns `nil` if 1. the request body has not been read, diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 678e60b47c..551f804992 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4587,12 +4587,14 @@ See also [[#ngx.req.read_body|ngx.req.read_body]]. == ngx.req.get_body_data == -'''syntax:''' ''data = ngx.req.get_body_data()'' +'''syntax:''' ''data = ngx.req.get_body_data(max_bytes?)'' '''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, log_by_lua*'' Retrieves in-memory request body data. It returns a Lua string rather than a Lua table holding all the parsed query arguments. Use the [[#ngx.req.get_post_args|ngx.req.get_post_args]] function instead if a Lua table is required. +The optional max_bytes function argument can be used when you don't need the entire body. + This function returns nil if # the request body has not been read, diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 4dd105e144..61ab999045 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -246,15 +246,21 @@ ngx_http_lua_ngx_req_get_body_data(lua_State *L) { ngx_http_request_t *r; int n; - size_t len; + size_t len, max; + size_t size, rest; ngx_chain_t *cl; u_char *p; u_char *buf; n = lua_gettop(L); - if (n != 0) { - return luaL_error(L, "expecting 0 arguments but seen %d", n); + if (n != 0 && n != 1) { + return luaL_error(L, "expecting 0 or 1 arguments but seen %d", n); + } + + max = 0; + if (n == 1) { + max = (size_t) luaL_checknumber(L, 1); } r = ngx_http_lua_get_req(L); @@ -282,6 +288,7 @@ ngx_http_lua_ngx_req_get_body_data(lua_State *L) return 1; } + len = (max > 0 && len > max) ? max : len; lua_pushlstring(L, (char *) cl->buf->pos, len); return 1; } @@ -292,7 +299,13 @@ ngx_http_lua_ngx_req_get_body_data(lua_State *L) for (; cl; cl = cl->next) { dd("body chunk len: %d", (int) ngx_buf_size(cl->buf)); - len += cl->buf->last - cl->buf->pos; + size = cl->buf->last - cl->buf->pos; + if (max > 0 && (len + size > max)) { + len = max; + break; + } + + len += size; } if (len == 0) { @@ -303,8 +316,15 @@ ngx_http_lua_ngx_req_get_body_data(lua_State *L) buf = (u_char *) lua_newuserdata(L, len); p = buf; - for (cl = r->request_body->bufs; cl; cl = cl->next) { - p = ngx_copy(p, cl->buf->pos, cl->buf->last - cl->buf->pos); + rest = len; + for (cl = r->request_body->bufs; cl != NULL && rest > 0; cl = cl->next) { + size = ngx_buf_size(cl->buf); + if (size > rest) { /* reach limit*/ + size = rest; + } + + p = ngx_copy(p, cl->buf->pos, size); + rest -= size; } lua_pushlstring(L, (char *) buf, len); diff --git a/t/010-request_body.t b/t/010-request_body.t index e669d9480e..5e4d96d0e9 100644 --- a/t/010-request_body.t +++ b/t/010-request_body.t @@ -270,3 +270,20 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug + + + +=== TEST 13: test reading the first n bytes of request body +--- config + location /echo_body { + lua_need_request_body on; + content_by_lua_block { + local data = ngx.req.get_body_data(1) + ngx.say(data) + } + } +--- request +POST /echo_body +hello +--- response_body +h From 3a0b823d9fe5e9eb9cf3b41afd5497192ae57110 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 14 Nov 2023 21:23:09 +0800 Subject: [PATCH 724/848] feature: make HTTP3 authority as http_host header. --- src/ngx_http_lua_headers.c | 73 ++++++++++++++++++++++++++++++++ src/ngx_http_lua_subrequest.c | 13 ++++++ src/ngx_http_lua_variable.c | 12 ++++++ t/001-set.t | 4 +- t/005-exit.t | 4 ++ t/010-request_body.t | 4 +- t/014-bugs.t | 12 +++++- t/016-resp-header.t | 8 ++-- t/020-subrequest.t | 5 ++- t/023-rewrite/req-body.t | 1 + t/023-rewrite/request_body.t | 1 + t/023-rewrite/socket-keepalive.t | 2 +- t/023-rewrite/tcp-socket.t | 12 +++--- t/024-access/req-body.t | 1 + t/024-access/request_body.t | 1 + t/025-codecache.t | 3 +- t/031-post-args.t | 1 + t/033-ctx.t | 2 +- t/041-header-filter.t | 48 ++++++++++++--------- t/043-shdict.t | 4 +- t/044-req-body.t | 17 +++++--- t/058-tcp-socket.t | 13 +++--- t/066-socket-receiveuntil.t | 1 + t/075-logby.t | 1 + t/082-body-filter.t | 16 ++++--- t/084-inclusive-receiveuntil.t | 8 ++-- t/091-coroutine.t | 4 +- t/094-uthread-exit.t | 16 +++---- t/095-uthread-exec.t | 4 +- t/096-uthread-redirect.t | 4 +- t/124-init-worker.t | 1 + t/129-ssl-socket.t | 12 +++++- t/139-ssl-cert-by.t | 10 ++--- t/142-ssl-session-store.t | 1 + t/143-ssl-session-fetch.t | 17 ++++++++ t/163-signal.t | 2 +- t/166-ssl-client-hello.t | 2 + t/166-worker-thread.t | 2 + t/167-server-rewrite.t | 4 +- t/186-cosocket-busy-bufs.t | 12 +++++- t/187-ssl-two-verification.t | 2 + 41 files changed, 271 insertions(+), 89 deletions(-) diff --git a/src/ngx_http_lua_headers.c b/src/ngx_http_lua_headers.c index a2ce6e1452..85836a1289 100644 --- a/src/ngx_http_lua_headers.c +++ b/src/ngx_http_lua_headers.c @@ -782,6 +782,11 @@ ngx_http_lua_ffi_req_get_headers_count(ngx_http_request_t *r, int max, { int count; ngx_list_part_t *part; +#if (NGX_HTTP_V3) + int has_host = 0; + ngx_uint_t i; + ngx_table_elt_t *header; +#endif if (r->connection->fd == (ngx_socket_t) -1) { return NGX_HTTP_LUA_FFI_BAD_CONTEXT; @@ -794,11 +799,54 @@ ngx_http_lua_ffi_req_get_headers_count(ngx_http_request_t *r, int max, } part = &r->headers_in.headers.part; + +#if (NGX_HTTP_V3) + count = 0; + header = part->elts; + + if (r->http_version == NGX_HTTP_VERSION_30 + && r->headers_in.server.data != NULL) + { + has_host = 1; + count++; + } + + if (has_host == 1) { + for (i = 0; /* void */; i++) { + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + if (header[i].key.len == 4 + && ngx_strncasecmp(header[i].key.data, + (u_char *) "host", 4) == 0) + { + continue; + } + + count++; + } + + } else { + count = part->nelts; + while (part->next != NULL) { + part = part->next; + count += part->nelts; + } + } +#else count = part->nelts; while (part->next != NULL) { part = part->next; count += part->nelts; } +#endif if (max > 0 && count > max) { *truncated = 1; @@ -821,12 +869,29 @@ ngx_http_lua_ffi_req_get_headers(ngx_http_request_t *r, ngx_uint_t i; ngx_list_part_t *part; ngx_table_elt_t *header; +#if (NGX_HTTP_V3) + int has_host = 0; +#endif if (count <= 0) { return NGX_OK; } n = 0; + +#if (NGX_HTTP_V3) + if (r->http_version == NGX_HTTP_VERSION_30 + && r->headers_in.server.data != NULL) + { + out[n].key.data = (u_char *) "host"; + out[n].key.len = sizeof("host") - 1; + out[n].value.len = r->headers_in.server.len; + out[n].value.data = r->headers_in.server.data; + has_host = 1; + ++n; + } +#endif + part = &r->headers_in.headers.part; header = part->elts; @@ -842,6 +907,14 @@ ngx_http_lua_ffi_req_get_headers(ngx_http_request_t *r, i = 0; } +#if (NGX_HTTP_V3) + if (has_host == 1 && header[i].key.len == 4 + && ngx_strncasecmp(header[i].key.data, (u_char *) "host", 4) == 0) + { + continue; + } +#endif + if (raw) { out[n].key.data = header[i].key.data; out[n].key.len = (int) header[i].key.len; diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index c09207417e..f4db9aaf6c 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -52,6 +52,8 @@ ngx_str_t ngx_http_lua_patch_method = ngx_str_t ngx_http_lua_trace_method = ngx_http_lua_method_name("TRACE"); +ngx_str_t host_header = ngx_string("host"); + static ngx_str_t ngx_http_lua_content_length_header_key = ngx_string("Content-Length"); @@ -1699,6 +1701,17 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, part = &pr->headers_in.headers.part; header = part->elts; +#if (NGX_HTTP_V3) + if (pr->headers_in.server.data != NULL) { + if (ngx_http_lua_set_input_header(sr, host_header, + pr->headers_in.server, 0) + == NGX_ERROR) + { + return NGX_ERROR; + } + } +#endif + for (i = 0; /* void */; i++) { if (i >= part->nelts) { diff --git a/src/ngx_http_lua_variable.c b/src/ngx_http_lua_variable.c index 573629ad21..2c6c6233bd 100644 --- a/src/ngx_http_lua_variable.c +++ b/src/ngx_http_lua_variable.c @@ -70,6 +70,18 @@ ngx_http_lua_ffi_var_get(ngx_http_request_t *r, u_char *name_data, } #endif +#if (NGX_HTTP_V3) + if (name_len == 9 + && r->http_version == NGX_HTTP_VERSION_30 + && ngx_strncasecmp(name_data, (u_char *) "http_host", 9) == 0 + && r->headers_in.server.data != NULL) + { + *value = r->headers_in.server.data; + *value_len = r->headers_in.server.len; + return NGX_OK; + } +#endif + hash = ngx_hash_strlow(lowcase_buf, name_data, name_len); name.data = lowcase_buf; diff --git a/t/001-set.t b/t/001-set.t index abbee72121..009159e484 100644 --- a/t/001-set.t +++ b/t/001-set.t @@ -389,8 +389,8 @@ GET /lua GET /lua --- response_body_like: 500 Internal Server Error --- error_code: 500 ---- error_log -API disabled in the context of set_by_lua* +--- error_log eval +qr/(?:API disabled in the context of set_by_lua\*|http3 requests are not supported without content-length header)/ms diff --git a/t/005-exit.t b/t/005-exit.t index e057fe1c69..0783c69295 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -774,6 +774,8 @@ GET /t --- response_body --- no_error_log [error] +--- curl_error +curl: (95) HTTP/3 stream 0 reset by server @@ -790,6 +792,8 @@ GET /t --- response_body --- no_error_log [error] +--- curl_error +curl: (95) HTTP/3 stream 0 reset by server diff --git a/t/010-request_body.t b/t/010-request_body.t index 5e4d96d0e9..c47f9fc13d 100644 --- a/t/010-request_body.t +++ b/t/010-request_body.t @@ -270,6 +270,7 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -281,9 +282,10 @@ http finalize request: 500, "/echo_body?" a:1, c:0 local data = ngx.req.get_body_data(1) ngx.say(data) } - } + } --- request POST /echo_body hello --- response_body h +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/014-bugs.t b/t/014-bugs.t index f4e596640e..ff85ca3558 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -885,6 +885,7 @@ GET /t --- no_error_log [error] --- timeout: 10 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -1305,8 +1306,17 @@ location /t { --- response_body Hello world --- shutdown_error_log eval -qr|failed to read a line: closed| +my $expr; + +if ($ENV{TEST_NGINX_USE_HTTP3}) { + $expr = qr|lua close the global Lua VM| +} else { + $expr = qr|failed to read a line: closed| +} + +$expr; --- timeout: 1.2 +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/016-resp-header.t b/t/016-resp-header.t index ac0c52eeef..6cf699d88e 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -2209,8 +2209,8 @@ upstream prematurely closed connection while sending to client Foo --- request GET /a.txt ---- raw_response_headers_like chomp -Age: \d\r\n +--- raw_response_headers_like eval +qr/^(a|A)ge: \d\r\n/ms --- no_error_log [error] @@ -2240,8 +2240,8 @@ Age: \d\r\n Foo --- request GET /test/a.txt ---- raw_response_headers_like chomp -Age: \d\r\n +--- raw_response_headers_like eval +qr/^(a|A)ge: \d\r\n/ms --- no_error_log [error] diff --git a/t/020-subrequest.t b/t/020-subrequest.t index f629ca8754..d8aa34d7e8 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1227,8 +1227,8 @@ F(ngx_http_finalize_request) { --- error_code --- no_error_log [error] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr{(\Qcurl: (52) Empty reply from server\E|\Qcurl: (95) HTTP/3 stream 0 reset by server\E)}ms @@ -2340,6 +2340,7 @@ hello world nil --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/023-rewrite/req-body.t b/t/023-rewrite/req-body.t index 13bdcb2511..0c1857384e 100644 --- a/t/023-rewrite/req-body.t +++ b/t/023-rewrite/req-body.t @@ -125,6 +125,7 @@ Expect: 100-Continue [alert] [error] http finalize request: 500, "/test?" a:1, c:0 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/023-rewrite/request_body.t b/t/023-rewrite/request_body.t index a721eba31e..32c02e151f 100644 --- a/t/023-rewrite/request_body.t +++ b/t/023-rewrite/request_body.t @@ -170,6 +170,7 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/023-rewrite/socket-keepalive.t b/t/023-rewrite/socket-keepalive.t index 5884c02b8e..9ce8d5e290 100644 --- a/t/023-rewrite/socket-keepalive.t +++ b/t/023-rewrite/socket-keepalive.t @@ -493,7 +493,7 @@ qr/lua tcp socket connection pool size: 1\b/] === TEST 7: "lua_socket_keepalive_timeout 0" means unlimited ---- quic_max_idle_timeout: 1.1 +--- quic_max_idle_timeout: 1.2 --- config server_tokens off; location /t { diff --git a/t/023-rewrite/tcp-socket.t b/t/023-rewrite/tcp-socket.t index 9ca66c64bb..966365f906 100644 --- a/t/023-rewrite/tcp-socket.t +++ b/t/023-rewrite/tcp-socket.t @@ -1667,8 +1667,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type nil found) ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# --- no_http2 @@ -1731,8 +1731,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type boolean found) ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# --- no_http2 @@ -1795,8 +1795,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type userdata found) ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# --- no_http2 diff --git a/t/024-access/req-body.t b/t/024-access/req-body.t index 70db85c1c2..48caeb9001 100644 --- a/t/024-access/req-body.t +++ b/t/024-access/req-body.t @@ -122,6 +122,7 @@ Expect: 100-Continue [alert] [error] http finalize request: 500, "/test?" a:1, c:0 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/024-access/request_body.t b/t/024-access/request_body.t index 9086bf9e36..0aa12c8b55 100644 --- a/t/024-access/request_body.t +++ b/t/024-access/request_body.t @@ -170,6 +170,7 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/025-codecache.t b/t/025-codecache.t index 5be94d8ce7..cd56cf57aa 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -993,7 +993,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, "lua close the global Lua VM", ] --- curl_error eval -qr/curl: \(\d+\) Empty reply from server/ +qr/curl: \(\d+\) Empty reply from server|curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received/ @@ -1881,3 +1881,4 @@ qr/log_by_lua\(nginx.conf:\d+\):\d+: hello/, --- log_level: debug --- no_error_log [error] +--- skip_eval: 14:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/031-post-args.t b/t/031-post-args.t index 4659f059fc..78805d3723 100644 --- a/t/031-post-args.t +++ b/t/031-post-args.t @@ -108,6 +108,7 @@ a=3&b=4&c --- request POST /lua --- response_body +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/033-ctx.t b/t/033-ctx.t index 6133cee2bb..782a0fab6e 100644 --- a/t/033-ctx.t +++ b/t/033-ctx.t @@ -279,7 +279,7 @@ GET /t --- error_log ngx.ctx = 32 --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ diff --git a/t/041-header-filter.t b/t/041-header-filter.t index d36d3e025c..23fdac02cd 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -125,7 +125,7 @@ GET /read --- error_code --- response_body --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -468,7 +468,7 @@ failed to run header_filter_by_lua*: header_filter_by_lua(nginx.conf:47):2: Some --- no_error_log [alert] --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -492,7 +492,7 @@ failed to run header_filter_by_lua*: unknown reason --- no_error_log [alert] --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -508,7 +508,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -524,7 +524,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -540,7 +540,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -556,7 +556,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -572,7 +572,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -592,7 +592,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -612,7 +612,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -628,7 +628,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -644,7 +644,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -660,7 +660,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -691,10 +691,10 @@ uri: /blah --- request GET /lua --- ignore_response ---- error_log -API disabled in the context of header_filter_by_lua* +--- error_log eval +qr/API disabled in the context of header_filter_by_lua\*|http3 requests are not supported without content-length header/ms --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -718,7 +718,7 @@ if (defined $ENV{TEST_NGINX_USE_HTTP3}) { $err_log; --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -734,7 +734,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -750,7 +750,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -799,7 +799,7 @@ in function 'error' in function 'bar' in function 'foo' --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -819,7 +819,7 @@ GET /lua?a=1&b=2 --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -866,6 +866,8 @@ GET /lua failed to load inlined Lua code: header_filter_by_lua(nginx.conf:41):2: unexpected symbol near ''for end'' --- no_error_log no_such_error +--- curl_error eval +qr/curl: \(56\) Failure when receiving data from the peer/ @@ -895,6 +897,8 @@ GET /lua failed to load inlined Lua code: header_filter_by_lua(nginx.conf:49):2: unexpected symbol near ''for end'' --- no_error_log no_such_error +--- curl_error eval +qr/curl: \(56\) Failure when receiving data from the peer/ @@ -919,3 +923,5 @@ GET /lua failed to load inlined Lua code: header_filter_by_lua(...901234567890123456789012345.conf:1):2: unexpected symbol near ''for end'' --- no_error_log [alert] +--- curl_error eval +qr/curl: \(56\) Failure when receiving data from the peer/ diff --git a/t/043-shdict.t b/t/043-shdict.t index 4fcdcb33b6..649683b9fe 100644 --- a/t/043-shdict.t +++ b/t/043-shdict.t @@ -768,7 +768,7 @@ foo = 10502 dogs:set("bar", 32, 0.001) dogs:set("baz", 32, 0.001) dogs:set("foo", 32, 0.001) - ngx.location.capture("/sleep/0.002") + ngx.location.capture("/sleep/0.003") local res, err, forcible = dogs:add("foo", 10502) ngx.say("add: ", res, " ", err, " ", forcible) ngx.say("foo = ", dogs:get("foo")) @@ -797,7 +797,7 @@ foo = 10502 dogs:set("bar", 32, 0.001) dogs:set("baz", 32, 0.001) dogs:set("foo", "hi", 0.001) - ngx.location.capture("/sleep/0.002") + ngx.location.capture("/sleep/0.003") local res, err, forcible = dogs:add("foo", "hello") ngx.say("add: ", res, " ", err, " ", forcible) ngx.say("foo = ", dogs:get("foo")) diff --git a/t/044-req-body.t b/t/044-req-body.t index 29cb914257..f4509e1497 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -7,7 +7,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 50 ); +plan tests => repeat_each() * (blocks() * 4 + 56); #no_diff(); no_long_string(); @@ -916,6 +916,7 @@ body: hell --- no_error_log [error] [alert] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -977,6 +978,7 @@ body file: hello [alert] --- error_log a client request body is buffered to a temporary file +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -1003,9 +1005,9 @@ a client request body is buffered to a temporary file --- error_code: 500 --- error_log eval qr/lua entry thread aborted: runtime error: content_by_lua\(nginx\.conf:\d+\):2: request body not read yet/ - --- no_error_log [alert] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} @@ -1035,8 +1037,8 @@ body: hell --- no_error_log [error] [alert] ---- no_error_log a client request body is buffered to a temporary file +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -1259,9 +1261,8 @@ body: hello, my dear friend! --- no_error_log [error] [alert] ---- no_error_log a client request body is buffered to a temporary file ---- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -1401,9 +1402,8 @@ failed to get req socket: request body already exists --- no_error_log [error] [alert] ---- no_error_log a client request body is buffered to a temporary file ---- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -1425,6 +1425,7 @@ Expect: 100-Continue [alert] [error] http finalize request: 500, "/test?" a:1, c:0 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -1515,6 +1516,7 @@ Will you change this world? --- no_error_log [error] [alert] +--- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} @@ -1772,6 +1774,7 @@ content length: 5 --- no_error_log [error] [alert] +--- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index 5602140fe1..b5cf58ec5b 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -255,6 +255,7 @@ second line received: (?:Date|Server): .*? --- no_error_log [error] --- timeout: 10 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} @@ -1639,8 +1640,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type nil found) ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# @@ -1701,8 +1702,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type boolean found) ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# @@ -1763,8 +1764,8 @@ GET /t --- ignore_response --- error_log bad argument #1 to 'send' (bad data type userdata found) ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# diff --git a/t/066-socket-receiveuntil.t b/t/066-socket-receiveuntil.t index 3b1f280c29..43744a3721 100644 --- a/t/066-socket-receiveuntil.t +++ b/t/066-socket-receiveuntil.t @@ -1344,6 +1344,7 @@ this exposed a memory leak in receiveuntil ok --- no_error_log [error] +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/075-logby.t b/t/075-logby.t index de662d0fb4..3e9743584a 100644 --- a/t/075-logby.t +++ b/t/075-logby.t @@ -444,6 +444,7 @@ GET /lua ok --- error_log API disabled in the context of log_by_lua* +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/082-body-filter.t b/t/082-body-filter.t index 49a1d76397..4033bac8ce 100644 --- a/t/082-body-filter.t +++ b/t/082-body-filter.t @@ -482,8 +482,8 @@ stack traceback: in function 'error' in function 'bar' in function 'foo' ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# @@ -526,8 +526,8 @@ GET /lua?a=1&b=2 --- ignore_response --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# @@ -842,6 +842,8 @@ GET /lua --- ignore_response --- error_log API disabled in the context of body_filter_by_lua* +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# @@ -864,6 +866,8 @@ failed to load inlined Lua code: body_filter_by_lua(nginx.conf:41):2: unexpected --- no_error_log no_such_error1 no_such_error2 +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# @@ -894,5 +898,5 @@ failed to load inlined Lua code: body_filter_by_lua(nginx.conf:49):2: unexpected --- no_error_log no_such_error1 no_such_error2 ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# diff --git a/t/084-inclusive-receiveuntil.t b/t/084-inclusive-receiveuntil.t index 5921044b49..60f8363f92 100644 --- a/t/084-inclusive-receiveuntil.t +++ b/t/084-inclusive-receiveuntil.t @@ -518,8 +518,8 @@ bad "inclusive" option value type: string --- no_error_log [alert] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# @@ -588,8 +588,8 @@ bad "inclusive" option value type: string --- no_error_log [alert] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server# diff --git a/t/091-coroutine.t b/t/091-coroutine.t index 2c10d54209..bfbdb38937 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -763,7 +763,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -1700,7 +1700,7 @@ GET /t "in function 'co'" ] --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index 39b1f22b55..0194e44b31 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -1401,8 +1401,8 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# @@ -1486,8 +1486,8 @@ free request [alert] [error] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# @@ -1571,8 +1571,8 @@ free request [alert] [error] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# @@ -1656,5 +1656,5 @@ free request [alert] [error] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# diff --git a/t/095-uthread-exec.t b/t/095-uthread-exec.t index 6e18c844ea..4cd121da1e 100644 --- a/t/095-uthread-exec.t +++ b/t/095-uthread-exec.t @@ -423,5 +423,5 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# diff --git a/t/096-uthread-redirect.t b/t/096-uthread-redirect.t index 2da84bd7aa..62909b944c 100644 --- a/t/096-uthread-redirect.t +++ b/t/096-uthread-redirect.t @@ -277,5 +277,5 @@ attempt to abort with pending subrequests --- no_error_log [alert] [warn] ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 57df4a5b3b..07c2658f06 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -529,6 +529,7 @@ second line received: (?:Date|Server): .*? --- no_error_log [error] --- timeout: 10 +--- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index f89a538fd0..329e7145d6 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -14,6 +14,12 @@ $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_SERVER_SSL_PORT} ||= 12345; $ENV{TEST_NGINX_CERT_DIR} ||= dirname(realpath(abs_path(__FILE__))); +my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; +my $openssl_version = eval { `$NginxBinary -V 2>&1` }; +if ($openssl_version =~ m/BoringSSL/) { + $ENV{TEST_NGINX_USE_BORINGSSL} = 1; +} + #log_level 'warn'; log_level 'debug'; @@ -2568,8 +2574,8 @@ qr/\[error\] .* ngx.socket sslhandshake: expecting 1 ~ 5 arguments \(including t --- no_error_log [alert] --- timeout: 10 ---- curl_error -curl: (52) Empty reply from server +--- curl_error eval +qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# @@ -2667,6 +2673,7 @@ SSL reused session === TEST 33: explicit cipher configuration - TLSv1.3 --- skip_openssl: 8: < 1.1.1 --- skip_nginx: 8: < 1.19.4 +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -2760,6 +2767,7 @@ SSL reused session === TEST 34: explicit cipher configuration not in the default list - TLSv1.3 --- skip_openssl: 8: < 1.1.1 --- skip_nginx: 8: < 1.19.4 +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 2180466907..3fd6413307 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -579,7 +579,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua_certificate_by_lua: handler return value: -1, cert cb exit code: 0', -qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, 'lua exit with code -1', ] @@ -720,7 +720,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'lua_certificate_by_lua: cert cb exit code: 0', -qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, 'lua exit with code -1', ] @@ -791,7 +791,7 @@ failed to do SSL handshake: handshake failed [ 'runtime error: ssl_certificate_by_lua(nginx.conf:28):2: bad bad bad', 'lua_certificate_by_lua: handler return value: 500, cert cb exit code: 0', -qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, qr/context: ssl_certificate_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, ] @@ -863,7 +863,7 @@ failed to do SSL handshake: handshake failed [ 'runtime error: ssl_certificate_by_lua(nginx.conf:28):3: bad bad bad', 'lua_certificate_by_lua: cert cb exit code: 0', -qr/\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error/, +qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, ] --- no_error_log @@ -1051,7 +1051,7 @@ failed to do SSL handshake: handshake failed [ 'lua ssl server name: "test.com"', 'ssl_certificate_by_lua(nginx.conf:28):1: API disabled in the context of ssl_certificate_by_lua*', -qr/\[info\] .*?cert cb error/, +qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, ] --- no_error_log diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index ead2a167e5..11deb83207 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -975,3 +975,4 @@ qr/ssl_session_store_by_lua\*: skipped since TLS version >= 1\.3 \(\d+\)/ [error] [alert] [emerg] +--- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 74f2e6e1bb..2f988ded9c 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -957,6 +957,21 @@ close: 1 nil qr/ssl ((fetch|store) session|cert) by lua is running!/s --- grep_error_log_out eval +if ($ENV{TEST_NGINX_USE_HTTP3}) { +[ +'ssl cert by lua is running! +ssl store session by lua is running! +', +'ssl cert by lua is running! +ssl fetch session by lua is running! +ssl store session by lua is running! +', +'ssl cert by lua is running! +ssl fetch session by lua is running! +ssl store session by lua is running! +', +] +} else { [ 'ssl cert by lua is running! ssl store session by lua is running! @@ -970,6 +985,7 @@ ssl cert by lua is running! ssl store session by lua is running! ', ] +} --- no_error_log [error] @@ -1397,6 +1413,7 @@ ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is run [error] [alert] [emerg] +--- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/163-signal.t b/t/163-signal.t index ec3a418d90..15f41e2200 100644 --- a/t/163-signal.t +++ b/t/163-signal.t @@ -39,7 +39,7 @@ qr/\[notice\] \d+#\d+: exit$/ --- no_error_log eval qr/\[notice\] \d+#\d+: reconfiguring/ --- curl_error eval -qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received|curl: \(56\) Recv failure: Connection reset by peer/ +qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received|curl: \(56\) Recv failure: Connection reset by peer|curl: \(55\) sendmsg\(\) returned -1 \(errno 111\)/ diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index 102cd2e303..a9d8ac8d2a 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -10,6 +10,8 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +} elsif ($openssl_version =~ m/running with BoringSSL/) { + plan(skip_all => "does not support BoringSSL"); } else { plan tests => repeat_each() * (blocks() * 6 + 8); } diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 15e9d762eb..925a060949 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1199,6 +1199,8 @@ true,nil === TEST 38: shdict get_stale +For http3: curl: (55) ngtcp2_conn_handle_expiry returned error: ERR_IDLE_CLOSE +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} --- main_config thread_pool testpool threads=100; --- http_config eval diff --git a/t/167-server-rewrite.t b/t/167-server-rewrite.t index 152c5ce35d..6aea288ef8 100644 --- a/t/167-server-rewrite.t +++ b/t/167-server-rewrite.t @@ -79,7 +79,8 @@ server_rewrite_by_lua_block in server } --- request GET /lua ---- raw_response_headers_like: Location: /foo\r\n +--- raw_response_headers_like eval +qr{[Ll]ocation: /foo\r\n} --- response_body_like: 302 Found --- error_code: 302 --- no_error_log @@ -198,6 +199,7 @@ delete thread 1 --- ignore_response --- no_error_log [error] +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/186-cosocket-busy-bufs.t b/t/186-cosocket-busy-bufs.t index e50f62d9ee..7439443761 100644 --- a/t/186-cosocket-busy-bufs.t +++ b/t/186-cosocket-busy-bufs.t @@ -1,10 +1,20 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket 'no_plan'; + +use Test::Nginx::Socket; use Test::Nginx::Socket::Lua::Stream; log_level('warn'); repeat_each(2); + +if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + plan(skip_all => "HTTP3 does not support client abort"); +} elsif (defined $ENV{TEST_NGINX_USE_HTTP2}) { + plan(skip_all => "HTTP2 does not support client abort"); +} else { + plan tests => repeat_each() * (blocks() * 2); +} + run_tests(); __DATA__ diff --git a/t/187-ssl-two-verification.t b/t/187-ssl-two-verification.t index 527df7cbde..312847252c 100644 --- a/t/187-ssl-two-verification.t +++ b/t/187-ssl-two-verification.t @@ -10,6 +10,8 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +} elsif ($openssl_version =~ m/running with BoringSSL/) { + plan(skip_all => "does not support BoringSSL"); } else { plan tests => repeat_each() * (blocks() * 7); } From 0c3675ff4d5ecbfdcdeeb661a4bf0f5a8e10e51c Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 21 Nov 2023 22:55:32 +0800 Subject: [PATCH 725/848] tests: adjust some test cases for HTTP3. --- t/102-req-start-time.t | 4 ++-- t/129-ssl-socket.t | 11 ++++++++++- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/t/102-req-start-time.t b/t/102-req-start-time.t index 3b041afda7..d54b19941d 100644 --- a/t/102-req-start-time.t +++ b/t/102-req-start-time.t @@ -59,7 +59,7 @@ GET /start --- request GET /req_time --- response_body_like chop -^(?:0\.[12]|0\.099)\d* +^(?:0\.[12]|0\.099|0\.098)\d* true$ --- no_error_log [error] @@ -88,7 +88,7 @@ true$ --- request GET /req_time --- response_body_like chomp -^(?:0\.[12]|0\.099)\d* +^(?:0\.[12]|0\.099|0\.098)\d* 0\.\d+ true$ --- no_error_log diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 329e7145d6..3aea20bc08 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1,6 +1,15 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: -use Test::Nginx::Socket::Lua; +our $SkipReason; +BEGIN { + if (defined $ENV{TEST_NGINX_USE_HTTP3}) { + # FIXME: we still need to enable this test file for HTTP3. + $SkipReason = "the test cases are very unstable, skip for now."; + } +} + +use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); + use Cwd qw(abs_path realpath); use File::Basename; From 75ab35d3c49df4f63310609ac9817c510b81583c Mon Sep 17 00:00:00 2001 From: oowl Date: Tue, 21 Nov 2023 22:56:32 +0800 Subject: [PATCH 726/848] doc: add http2 and http3 limitation doc comment for ngx.req.socket API. --- README.markdown | 2 ++ doc/HttpLuaModule.wiki | 2 ++ 2 files changed, 4 insertions(+) diff --git a/README.markdown b/README.markdown index dce83887fa..d6ec8c94be 100644 --- a/README.markdown +++ b/README.markdown @@ -5643,6 +5643,8 @@ Returns a read-only cosocket object that wraps the downstream connection. Only [ In case of error, `nil` will be returned as well as a string describing the error. +Due to the streaming nature of HTTP2 and HTTP3, this API cannot be used when the downstream connection is HTTP2 and HTTP3. + The socket object returned by this method is usually used to read the current request's body in a streaming fashion. Do not turn on the [lua_need_request_body](#lua_need_request_body) directive, and do not mix this call with [ngx.req.read_body](#ngxreqread_body) and [ngx.req.discard_body](#ngxreqdiscard_body). If any request body data has been pre-read into the Nginx core request header buffer, the resulting cosocket object will take care of this to avoid potential data loss resulting from such pre-reading. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 551f804992..305626c767 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4741,6 +4741,8 @@ Returns a read-only cosocket object that wraps the downstream connection. Only [ In case of error, nil will be returned as well as a string describing the error. +Due to the streaming nature of HTTP2 and HTTP3, this API cannot be used when the downstream connection is HTTP2 and HTTP3. + The socket object returned by this method is usually used to read the current request's body in a streaming fashion. Do not turn on the [[#lua_need_request_body|lua_need_request_body]] directive, and do not mix this call with [[#ngx.req.read_body|ngx.req.read_body]] and [[#ngx.req.discard_body|ngx.req.discard_body]]. If any request body data has been pre-read into the Nginx core request header buffer, the resulting cosocket object will take care of this to avoid potential data loss resulting from such pre-reading. From 2d32c7c8d6d7270163201f0790489451675ec259 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 23 Nov 2023 21:58:23 +0800 Subject: [PATCH 727/848] bumped version of lua-nginx-module to 10026. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 640e0c396b..193c44e3a8 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10025 +#define ngx_http_lua_version 10026 typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; From ff96d75a49dd4bc9f3009332973f2b2cfc6326a3 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 28 Nov 2023 10:16:27 +0800 Subject: [PATCH 728/848] tests: update openssl to 1.1.1w. --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 60e1a053d9..d6742ed52b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -62,11 +62,11 @@ env: - MALLOC_PERTURB_=9 jobs: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y + - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f + - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y - NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y - #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s TEST_NGINX_USE_HTTP2=1 + #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 services: - memcached From 0d054a57f453d822a4800bf42cc1161adebe8e30 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 20 Dec 2023 10:43:40 +0800 Subject: [PATCH 729/848] tests: t/128-duplex-tcp-socket: replacing constant port 7658 with a random port. (#2268) --- t/058-tcp-socket.t | 7 +++++-- t/128-duplex-tcp-socket.t | 8 ++++---- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index b5cf58ec5b..ef2b05f0d0 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -3998,10 +3998,13 @@ lua tcp socket read any --- config server_tokens off; location = /t { + set $port $TEST_NGINX_RAND_PORT_1; + content_by_lua_block { local sock = ngx.socket.tcp() + local port = ngx.var.port sock:settimeout(500) - assert(sock:connect("127.0.0.1", 7658)) + assert(sock:connect("127.0.0.1", port)) while true do local data, err = sock:receiveany(1024) @@ -4028,7 +4031,7 @@ lua tcp socket read any --- request GET /t ---- tcp_listen: 7658 +--- tcp_listen: $TEST_NGINX_RAND_PORT_1 --- tcp_shutdown: 1 --- tcp_query eval: "send data after read side closed" --- tcp_query_len: 32 diff --git a/t/128-duplex-tcp-socket.t b/t/128-duplex-tcp-socket.t index 22b9f036dd..511cc20bfc 100644 --- a/t/128-duplex-tcp-socket.t +++ b/t/128-duplex-tcp-socket.t @@ -193,7 +193,7 @@ close: 1 nil$ lua_socket_log_errors off; location /t { #set $port 5000; - set $port 7658; + set $port $TEST_NGINX_RAND_PORT_1; content_by_lua ' local sock = ngx.socket.tcp() @@ -258,7 +258,7 @@ received: OK! close: (?:nil socket busy writing|1 nil failed to send request: closed)$ ---- tcp_listen: 7658 +--- tcp_listen: $TEST_NGINX_RAND_PORT_1 --- tcp_shutdown: 0 --- tcp_reply: OK! --- tcp_no_close: 1 @@ -273,7 +273,7 @@ failed to send request: closed)$ lua_socket_log_errors off; location /t { #set $port 5000; - set $port 7658; + set $port $TEST_NGINX_RAND_PORT_1; content_by_lua ' local sock = ngx.socket.tcp() @@ -349,7 +349,7 @@ F(ngx_http_lua_socket_tcp_finalize_write_part) { print_ubacktrace() } --- stap_out2 ---- tcp_listen: 7658 +--- tcp_listen: $TEST_NGINX_RAND_PORT_1 --- tcp_shutdown: 1 --- tcp_query eval: "flush_all\r\n" --- tcp_query_len: 11 From 8c7f9bf46a43ecf378c446b0fd2c06bc8c848f57 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 20 Dec 2023 10:44:07 +0800 Subject: [PATCH 730/848] tests: t/129-ssl-socket.t: use the same IP to ensure that the ssl session can be reused. (#2267) --- t/129-ssl-socket.t | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 3aea20bc08..cc3f243c07 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -15,6 +15,8 @@ use File::Basename; repeat_each(2); +sub resolve($$); + plan tests => repeat_each() * (blocks() * 7 - 3); $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -22,6 +24,7 @@ $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_SERVER_SSL_PORT} ||= 12345; $ENV{TEST_NGINX_CERT_DIR} ||= dirname(realpath(abs_path(__FILE__))); +$ENV{TEST_NGINX_OPENRESTY_ORG_IP} ||= resolve("openresty.org", $ENV{TEST_NGINX_RESOLVER}); my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; @@ -44,6 +47,19 @@ sub read_file { $cert; } +sub resolve ($$) { + my ($domain, $resolver) = @_; + my $ips = qx/dig \@$resolver +short $domain/; + + my $exit_code = $? >> 8; + if (!$ips || $exit_code != 0) { + die "failed to resolve '$domain' using '$resolver' as resolver"; + } + + my ($ip) = split /\n/, $ips; + return $ip; +} + our $DSTRootCertificate = read_file("t/cert/dst-ca.crt"); our $EquifaxRootCertificate = read_file("t/cert/equifax.crt"); our $TestCertificate = read_file("t/cert/test.crt"); @@ -1397,6 +1413,7 @@ SSL reused session location /t { #set $port 5000; set $port $TEST_NGINX_MEMCACHED_PORT; + set $openresty_org_ip $TEST_NGINX_OPENRESTY_ORG_IP; content_by_lua ' local sock = ngx.socket.tcp() @@ -1406,7 +1423,8 @@ SSL reused session local session for i = 1, 3 do - local ok, err = sock:connect("openresty.org", 443) + -- Use the same IP to ensure that the connection can be reused + local ok, err = sock:connect(ngx.var.openresty_org_ip, 443) if not ok then ngx.say("failed to connect: ", err) return @@ -1473,6 +1491,7 @@ SSL reused session lua_ssl_verify_depth 2; location /t { #set $port 5000; + set $openresty_org_ip $TEST_NGINX_OPENRESTY_ORG_IP; set $port $TEST_NGINX_MEMCACHED_PORT; content_by_lua ' @@ -1482,7 +1501,8 @@ SSL reused session do for i = 1, 3 do - local ok, err = sock:connect("openresty.org", 443) + -- Use the same IP to ensure that the connection can be reused + local ok, err = sock:connect(ngx.var.openresty_org_ip, 443) if not ok then ngx.say("failed to connect: ", err) return From f249fa9b7ed877e7ae88bcc1f1074bc8b26cc97b Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 20 Dec 2023 10:46:17 +0800 Subject: [PATCH 731/848] bugfix: fixed memory leak in the case of connection in dubious. (#2270) Can be reproduced in valgrind mode in t/129-ssl-socket.t: ==55579== 13 bytes in 1 blocks are definitely lost in loss record 3 of 19 ==55579== at 0x483A809: malloc (vg_replace_malloc.c:307) ==55579== by 0x44F522: ngx_alloc (ngx_alloc.c:22) ==55579== by 0x549DDD: ngx_http_lua_ffi_socket_tcp_sslhandshake (ngx_http_lua_socket_tcp.c:1828) ==55579== by 0x4897FEA: lj_vm_ffi_call (in /opt/luajit21/lib/libluajit-5.1.so.2.1.1700206165) ==55579== by 0x48F7980: lj_ccall_func (lj_ccall.c:1402) ==55579== by 0x4910E74: lj_cf_ffi_meta___call (lib_ffi.c:230) ==55579== by 0x4895B7D: lj_BC_FUNCC (in /opt/luajit21/lib/libluajit-5.1.so.2.1.1700206165) ==55579== by 0x5360E6: ngx_http_lua_run_thread (ngx_http_lua_util.c:1184) ==55579== by 0x54203E: ngx_http_lua_socket_tcp_resume_helper (ngx_http_lua_socket_tcp.c:6207) ==55579== by 0x542123: ngx_http_lua_socket_tcp_conn_resume (ngx_http_lua_socket_tcp.c:6113) ==55579== by 0x5393B1: ngx_http_lua_content_wev_handler (ngx_http_lua_contentby.c:152) ==55579== by 0x54014B: ngx_http_lua_socket_handle_conn_success (ngx_http_lua_socket_tcp.c:3638) ==55579== by 0x54504E: ngx_http_lua_socket_connected_handler (ngx_http_lua_socket_tcp.c:3906) ==55579== by 0x5405A7: ngx_http_lua_socket_tcp_handler (ngx_http_lua_socket_tcp.c:3426) ==55579== by 0x4567C0: ngx_epoll_process_events (ngx_epoll_module.c:930) ==55579== by 0x44AAB3: ngx_process_events_and_timers (ngx_event.c:258) ==55579== by 0x454623: ngx_worker_process_cycle (ngx_process_cycle.c:793) ==55579== by 0x45295B: ngx_spawn_process (ngx_process.c:199) ==55579== by 0x45340E: ngx_start_worker_processes (ngx_process_cycle.c:382) ==55579== by 0x4551F1: ngx_master_process_cycle (ngx_process_cycle.c:241) ==55579== by 0x429DD6: main (nginx.c:387) --- src/ngx_http_lua_socket_tcp.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 505614c47f..230679fa7d 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5601,6 +5601,7 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) if (c->read->ready) { rc = ngx_http_lua_socket_keepalive_close_handler(c->read); if (rc != NGX_OK) { + ngx_http_lua_socket_tcp_finalize(r, u); lua_pushnil(L); lua_pushliteral(L, "connection in dubious state"); return 2; From e8da8781c9a014d2eac3425f63516cd3f2a0047a Mon Sep 17 00:00:00 2001 From: lynch Date: Fri, 22 Dec 2023 11:44:55 +0800 Subject: [PATCH 732/848] optimize: Make the use of NGX_PCRE2 clearer. --- src/ngx_http_lua_common.h | 27 ++++++++++++++++----------- src/ngx_http_lua_pcrefix.c | 4 ++-- src/ngx_http_lua_pcrefix.h | 2 +- src/ngx_http_lua_regex.c | 6 +++--- 4 files changed, 22 insertions(+), 17 deletions(-) diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 4bbfd8c780..4c946297f6 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -54,17 +54,19 @@ typedef struct { #endif -#if defined(NGX_PCRE) && !defined(NGX_PCRE2) -#include -# if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) +#if (NGX_PCRE) +# if (NGX_PCRE2) # define LUA_HAVE_PCRE_JIT 1 # else -# define LUA_HAVE_PCRE_JIT 0 -# endif -#endif -#if (NGX_PCRE2) -# define LUA_HAVE_PCRE_JIT 1 +#include + +# if (PCRE_MAJOR > 8) || (PCRE_MAJOR == 8 && PCRE_MINOR >= 21) +# define LUA_HAVE_PCRE_JIT 1 +# else +# define LUA_HAVE_PCRE_JIT 0 +# endif +# endif #endif @@ -221,15 +223,18 @@ struct ngx_http_lua_main_conf_s { ngx_hash_t builtin_headers_out; -#if (NGX_PCRE || NGX_PCRE2) +#if (NGX_PCRE) ngx_int_t regex_cache_entries; ngx_int_t regex_cache_max_entries; ngx_int_t regex_match_limit; +#endif + +#if (LUA_HAVE_PCRE_JIT) #if (NGX_PCRE2) pcre2_jit_stack *jit_stack; -#elif (LUA_HAVE_PCRE_JIT) +#else pcre_jit_stack *jit_stack; -# endif +#endif #endif ngx_array_t *shm_zones; /* of ngx_shm_zone_t* */ diff --git a/src/ngx_http_lua_pcrefix.c b/src/ngx_http_lua_pcrefix.c index 76e745e6db..5693b5495c 100644 --- a/src/ngx_http_lua_pcrefix.c +++ b/src/ngx_http_lua_pcrefix.c @@ -14,7 +14,7 @@ #include "ngx_http_lua_pcrefix.h" #include "stdio.h" -#if (NGX_PCRE || NGX_PCRE2) +#if (NGX_PCRE) static ngx_pool_t *ngx_http_lua_pcre_pool = NULL; @@ -183,6 +183,6 @@ ngx_http_lua_pcre_malloc_done(ngx_pool_t *old_pool) } #endif -#endif /* NGX_PCRE || NGX_PCRE2 */ +#endif /* NGX_PCRE */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_pcrefix.h b/src/ngx_http_lua_pcrefix.h index ef09636313..e1097b72eb 100644 --- a/src/ngx_http_lua_pcrefix.h +++ b/src/ngx_http_lua_pcrefix.h @@ -12,7 +12,7 @@ #include "ngx_http_lua_common.h" -#if (NGX_PCRE || NGX_PCRE2) +#if (NGX_PCRE) ngx_pool_t *ngx_http_lua_pcre_malloc_init(ngx_pool_t *pool); void ngx_http_lua_pcre_malloc_done(ngx_pool_t *old_pool); diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 7317451f65..1b52fa233d 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -9,7 +9,7 @@ #endif #include "ddebug.h" -#if (NGX_PCRE || NGX_PCRE2) +#if (NGX_PCRE) #include "ngx_http_lua_pcrefix.h" #include "ngx_http_lua_script.h" @@ -325,7 +325,7 @@ ngx_int_t ngx_http_lua_ffi_set_jit_stack_size(int size, u_char *errstr, size_t *errstr_size) { -#if (NGX_PCRE2 || LUA_HAVE_PCRE_JIT) +#if (LUA_HAVE_PCRE_JIT) ngx_http_lua_main_conf_t *lmcf; ngx_pool_t *pool, *old_pool; @@ -977,7 +977,7 @@ ngx_http_lua_ffi_pcre_version(void) } -#endif /* NGX_PCRE || NGX_PCRE2 */ +#endif /* NGX_PCRE */ /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From bf5d9d3b5c2801034e15e579704d4ef8f2cd6d38 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 22 Dec 2023 17:03:25 +0800 Subject: [PATCH 733/848] change: don't move the expired item to the lru queue head when lookup. --- src/ngx_http_lua_shdict.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 31bdcdb6a5..c63eb3c107 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -198,9 +198,6 @@ ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, rc = ngx_memn2cmp(kdata, sd->data, klen, (size_t) sd->key_len); if (rc == 0) { - ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); - *sdp = sd; dd("node expires: %lld", (long long) sd->expires); @@ -219,6 +216,9 @@ ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, } } + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); + return NGX_OK; } From 766b7a8add2327bef6bc3b5af15b111161745586 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Mon, 25 Dec 2023 11:19:39 +0800 Subject: [PATCH 734/848] tests: use random ports instead of constant ports to prevent conflicts during testing. (#2274) --- t/014-bugs.t | 4 +-- t/020-subrequest.t | 56 ++++++++++++++++++++-------------------- t/045-ngx-var.t | 12 ++++----- t/093-uthread-spawn.t | 4 +-- t/106-timer.t | 4 +-- t/124-init-worker.t | 4 +-- t/129-ssl-socket.t | 4 +-- t/139-ssl-cert-by.t | 4 +-- t/163-exit-worker-hup.t | 4 +-- t/166-ssl-client-hello.t | 8 +++--- 10 files changed, 51 insertions(+), 53 deletions(-) diff --git a/t/014-bugs.t b/t/014-bugs.t index ff85ca3558..1b79aa4c59 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -893,7 +893,7 @@ GET /t --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;'; server { - listen 12354; + listen \$TEST_NGINX_RAND_PORT_1; location = /t { echo 'args: \$args'; @@ -903,7 +903,7 @@ GET /t --- config location = /t { set $args "foo=1&bar=2"; - proxy_pass http://127.0.0.1:12354; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_1; } --- request diff --git a/t/020-subrequest.t b/t/020-subrequest.t index d8aa34d7e8..c731f1e61c 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1240,7 +1240,7 @@ qr{(\Qcurl: (52) Empty reply from server\E|\Qcurl: (95) HTTP/3 stream 0 reset by set $memc_key 'foo'; #set $memc_exptime 300; - memc_pass 127.0.0.1:19112; #$TEST_NGINX_MEMCACHED_PORT; + memc_pass 127.0.0.1:$TEST_NGINX_RAND_PORT_1; #$TEST_NGINX_MEMCACHED_PORT; } location /main { @@ -1253,7 +1253,7 @@ qr{(\Qcurl: (52) Empty reply from server\E|\Qcurl: (95) HTTP/3 stream 0 reset by } --- request GET /main ---- tcp_listen: 19112 +--- tcp_listen: $TEST_NGINX_RAND_PORT_1 --- tcp_query_len: 9 --- tcp_reply eval "VALUE foo 0 1024\r\nhello world" @@ -1310,7 +1310,7 @@ upstream prematurely closed connection set $memc_key 'foo'; #set $memc_exptime 300; - memc_pass 127.0.0.1:19112; #$TEST_NGINX_MEMCACHED_PORT; + memc_pass 127.0.0.1:$TEST_NGINX_RAND_PORT_1; #$TEST_NGINX_MEMCACHED_PORT; } location /main { @@ -1323,7 +1323,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: 19112 +--- tcp_listen: $TEST_NGINX_RAND_PORT_1 --- tcp_no_close --- tcp_reply eval "VALUE foo 0 1024\r\nhello world" @@ -1383,7 +1383,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1396,7 +1396,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\nContent-Length: 1024\r\n\r\nhello world" @@ -1443,7 +1443,7 @@ upstream prematurely closed connection proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1456,7 +1456,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_no_close --- tcp_reply eval "HTTP/1.0 200 OK\r\nContent-Length: 1024\r\n\r\nhello world" @@ -1505,7 +1505,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1518,7 +1518,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1565,7 +1565,7 @@ truncated: false proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1578,7 +1578,7 @@ truncated: false } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_no_close --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1628,7 +1628,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering off; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1641,7 +1641,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1688,7 +1688,7 @@ truncated: false proxy_read_timeout 500ms; proxy_buffering off; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1701,7 +1701,7 @@ truncated: false } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_no_close --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1914,7 +1914,7 @@ a client request body is buffered to a temporary file #proxy_read_timeout 100ms; proxy_http_version 1.1; proxy_buffering on; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1927,7 +1927,7 @@ a client request body is buffered to a temporary file } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nb\r\nhello world\r" @@ -1977,7 +1977,7 @@ upstream prematurely closed connection #proxy_read_timeout 100ms; proxy_http_version 1.1; proxy_buffering off; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -1990,7 +1990,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nb\r\nhello world\r" @@ -2038,7 +2038,7 @@ upstream prematurely closed connection proxy_read_timeout 100ms; proxy_buffering on; proxy_http_version 1.1; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -2051,7 +2051,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_no_close --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nb\r\nhello world\r" @@ -2100,7 +2100,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering on; proxy_http_version 1.1; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -2113,7 +2113,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_no_close --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n" @@ -2158,7 +2158,7 @@ truncated: false #proxy_read_timeout 100ms; proxy_buffering off; proxy_http_version 1.1; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -2171,7 +2171,7 @@ truncated: false } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_no_close --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n" @@ -2217,7 +2217,7 @@ truncated: false #proxy_read_timeout 100ms; proxy_buffering off; - proxy_pass http://127.0.0.1:19113; + proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; } location /main { @@ -2230,7 +2230,7 @@ truncated: false } --- request GET /main ---- tcp_listen: 19113 +--- tcp_listen: $TEST_NGINX_RAND_PORT_2 --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\nContent-Length: 1024\r\n\r\nhello world" diff --git a/t/045-ngx-var.t b/t/045-ngx-var.t index f0f9f415c6..6fe5590d99 100644 --- a/t/045-ngx-var.t +++ b/t/045-ngx-var.t @@ -260,19 +260,19 @@ variable "query_string" not changeable } server { # this is the real entry point - listen 8091; + listen $TEST_NGINX_RAND_PORT_1; location / { content_by_lua_block{ - ngx.print("this is backend peer 8091") + ngx.print("this is backend peer $TEST_NGINX_RAND_PORT_1") } } } server { # this is the real entry point - listen 8092; + listen $TEST_NGINX_RAND_PORT_2; location / { content_by_lua_block{ - ngx.print("this is backend peer 8092") + ngx.print("this is backend peer $TEST_NGINX_RAND_PORT_2") } } } @@ -287,6 +287,6 @@ variable "query_string" not changeable proxy_pass http://balancer; } --- pipelined_requests eval -["GET /balancer?port=8091", "GET /balancer?port=8092"] +["GET /balancer?port=\$TEST_NGINX_RAND_PORT_1", "GET /balancer?port=\$TEST_NGINX_RAND_PORT_2"] --- response_body eval -["this is backend peer 8091", "this is backend peer 8092"] +["this is backend peer \$TEST_NGINX_RAND_PORT_1", "this is backend peer \$TEST_NGINX_RAND_PORT_2"] diff --git a/t/093-uthread-spawn.t b/t/093-uthread-spawn.t index d905c100c5..99750d3c0b 100644 --- a/t/093-uthread-spawn.t +++ b/t/093-uthread-spawn.t @@ -962,7 +962,7 @@ received: OK content_by_lua ' local function f() local sock = ngx.socket.udp() - local ok, err = sock:setpeername("127.0.0.1", 12345) + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_RAND_PORT_1) local bytes, err = sock:send("blah") if not bytes then ngx.say("failed to send query: ", err) @@ -1002,7 +1002,7 @@ delete thread 2 delete thread 1 )$ ---- udp_listen: 12345 +--- udp_listen: $TEST_NGINX_RAND_PORT_1 --- udp_query: blah --- udp_reply: hello udp --- response_body_like chop diff --git a/t/106-timer.t b/t/106-timer.t index a9b4aed7bd..513e1e5b45 100644 --- a/t/106-timer.t +++ b/t/106-timer.t @@ -2210,8 +2210,8 @@ qr/\[lua\] content_by_lua\(nginx\.conf:\d+\):\d+: elapsed: .*?, context: ngx\.ti '; } --- log_level: error ---- error_log_file: syslog:server=127.0.0.1:12345 ---- udp_listen: 12345 +--- error_log_file: syslog:server=127.0.0.1:$TEST_NGINX_RAND_PORT_1 +--- udp_listen: $TEST_NGINX_RAND_PORT_1 --- udp_query eval: qr/Bad bad bad/ --- udp_reply: hello --- wait: 0.1 diff --git a/t/124-init-worker.t b/t/124-init-worker.t index 07c2658f06..c68d74bb18 100644 --- a/t/124-init-worker.t +++ b/t/124-init-worker.t @@ -730,8 +730,8 @@ ok '; } --- log_level: error ---- error_log_file: syslog:server=127.0.0.1:12345 ---- udp_listen: 12345 +--- error_log_file: syslog:server=127.0.0.1:$TEST_NGINX_RAND_PORT_1 +--- udp_listen: $TEST_NGINX_RAND_PORT_1 --- udp_query eval: qr/Bad bad bad/ --- udp_reply: hello --- wait: 0.1 diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index cc3f243c07..ccfa19fffb 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -17,7 +17,7 @@ repeat_each(2); sub resolve($$); -plan tests => repeat_each() * (blocks() * 7 - 3); +plan tests => repeat_each() * (blocks() * 7 - 4); $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; @@ -2141,8 +2141,6 @@ failed to do SSL handshake: timeout --- log_level: debug --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out ---- error_log -lua ssl server name: "openresty.org" --- no_error_log SSL reused session [error] diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 3fd6413307..abf447856d 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -1885,7 +1885,7 @@ qr/\[info\] .*? SSL_do_handshake\(\) failed\b/, lua_package_path "../lua-resty-core/lib/?.lua;;"; server { - listen 127.0.0.1:12345 ssl; + listen 127.0.0.1:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_certificate_by_lua_block { @@ -1917,7 +1917,7 @@ qr/\[info\] .*? SSL_do_handshake\(\) failed\b/, sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.1", 12345) + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return diff --git a/t/163-exit-worker-hup.t b/t/163-exit-worker-hup.t index cd2e5ce649..136efdc7ff 100644 --- a/t/163-exit-worker-hup.t +++ b/t/163-exit-worker-hup.t @@ -57,7 +57,7 @@ log from exit_worker_by_lua_block } server { - listen 12345; + listen $TEST_NGINX_RAND_PORT_1; location = /t { echo 'hello world'; @@ -68,7 +68,7 @@ log from exit_worker_by_lua_block content_by_lua_block { ngx.timer.at(0, function () local sock = ngx.socket.tcp() - sock:connect("127.0.0.1", 12345) + sock:connect("127.0.0.1", $TEST_NGINX_RAND_PORT_1) local reader = sock:receiveuntil("unknow") ngx.log(ngx.NOTICE, "reading to block the exiting") reader() diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index a9d8ac8d2a..a356b6eeaa 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -736,7 +736,7 @@ should never reached here === TEST 9: lua exception - no yield --- http_config server { - listen 127.0.0.2:8080 ssl; + listen 127.0.0.2:$TEST_NGINX_RAND_PORT_2 ssl; server_name test.com; ssl_client_hello_by_lua_block { error("bad bad bad") @@ -764,7 +764,7 @@ should never reached here sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.2", 8080) + local ok, err = sock:connect("127.0.0.2", $TEST_NGINX_RAND_PORT_2) if not ok then ngx.say("failed to connect: ", err) return @@ -2133,7 +2133,7 @@ ssl client hello by lua is running! lua_package_path "../lua-resty-core/lib/?.lua;;"; server { - listen 127.0.0.1:12346 ssl; + listen 127.0.0.1:$TEST_NGINX_RAND_PORT_1 ssl; server_name test.com; ssl_client_hello_by_lua_block { @@ -2165,7 +2165,7 @@ ssl client hello by lua is running! sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.1", 12346) + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.say("failed to connect: ", err) return From 0e769b76432df91e5f10aa56a56858e8a190faf7 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 27 Dec 2023 18:23:53 +0800 Subject: [PATCH 735/848] tests: 082-body-filter-2.t: skip this test if mockeagain is not loaded correctly. (#2275) --- t/082-body-filter-2.t | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t index ee4fe56ece..ffccad28ab 100644 --- a/t/082-body-filter-2.t +++ b/t/082-body-filter-2.t @@ -13,9 +13,14 @@ BEGIN { $SkipReason = "http2 does not support mockeagain"; } else { - $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; - $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; - $ENV{MOCKEAGAIN}='w' + if ($ENV{LD_PRELOAD} && $ENV{LD_PRELOAD} =~ /\bmockeagain\.so\b/) { + $ENV{TEST_NGINX_POSTPONE_OUTPUT} = 1; + $ENV{TEST_NGINX_EVENT_TYPE} = 'poll'; + $ENV{MOCKEAGAIN}='w' + } else { + $SkipReason = "'mockeagain.so' does not appear to be preloaded " + . "with 'LD_PRELOAD'"; + } } } From 7598ff389ef5a1a3e8949c48a6e13292fa9adc9e Mon Sep 17 00:00:00 2001 From: Brian Rak Date: Sat, 20 Jan 2024 14:11:31 +0800 Subject: [PATCH 736/848] feature: add ngx_http_lua_ffi_parse_der_cert and ngx_http_lua_ffi_parse_der_key functions. --- .travis.yml | 4 +- README.markdown | 4 +- src/ngx_http_lua_ssl_certby.c | 109 ++++++++++++++++++++++++ t/140-ssl-c-api.t | 156 ++++++++++++++++++++++++++++++++++ t/cert/test_der.crt | Bin 0 -> 955 bytes t/cert/test_der.key | Bin 0 -> 1216 bytes 6 files changed, 269 insertions(+), 4 deletions(-) create mode 100644 t/cert/test_der.crt create mode 100644 t/cert/test_der.key diff --git a/.travis.yml b/.travis.yml index d6742ed52b..d3b973c80d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -43,7 +43,7 @@ env: - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - PCRE_VER=8.45 - - PCRE2_VER=10.37 + - PCRE2_VER=10.42 - PCRE_PREFIX=/opt/pcre - PCRE2_PREFIX=/opt/pcre2 - PCRE_LIB=$PCRE_PREFIX/lib @@ -81,7 +81,7 @@ before_install: install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi + - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz diff --git a/README.markdown b/README.markdown index d6ec8c94be..56fb0565bf 100644 --- a/README.markdown +++ b/README.markdown @@ -7916,11 +7916,11 @@ Set client certificate chain and corresponding private key to the TCP socket obj The certificate chain and private key provided will be used later by the [tcpsock:sslhandshake](#tcpsocksslhandshake) method. * `cert` specify a client certificate chain cdata object that will be used while handshaking with -remote server. These objects can be created using [ngx.ssl.parse\_pem\_cert](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_cert) +remote server. These objects can be created using [ngx.ssl.parse\_pem\_cert](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_cert) or [ngx.ssl.parse\_der\_cert](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_der_cert) function provided by lua-resty-core. Note that specifying the `cert` option requires corresponding `pkey` be provided too. See below. * `pkey` specify a private key corresponds to the `cert` option above. -These objects can be created using [ngx.ssl.parse\_pem\_priv\_key](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_priv_key) +These objects can be created using [ngx.ssl.parse\_pem\_priv\_key](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_pem_priv_key) or [ngx.ssl.parse\_der\_priv\_key](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#parse_der_priv_key) function provided by lua-resty-core. If both of `cert` and `pkey` are `nil`, this method will clear any existing client certificate and private key diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index b8e70ddefa..d775f2b311 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1172,6 +1172,81 @@ ngx_http_lua_ffi_parse_pem_cert(const u_char *pem, size_t pem_len, } +void * +ngx_http_lua_ffi_parse_der_cert(const char *data, size_t len, + char **err) +{ + BIO *bio; + X509 *x509; + STACK_OF(X509) *chain; + + if (data == NULL || len == 0) { + *err = "invalid argument"; + ERR_clear_error(); + return NULL; + } + + bio = BIO_new_mem_buf((char *) data, len); + if (bio == NULL) { + *err = "BIO_new_mem_buf() failed"; + ERR_clear_error(); + return NULL; + } + + x509 = d2i_X509_bio(bio, NULL); + if (x509 == NULL) { + *err = "d2i_X509_bio() failed"; + BIO_free(bio); + ERR_clear_error(); + return NULL; + } + + chain = sk_X509_new_null(); + if (chain == NULL) { + *err = "sk_X509_new_null() failed"; + X509_free(x509); + BIO_free(bio); + ERR_clear_error(); + return NULL; + } + + if (sk_X509_push(chain, x509) == 0) { + *err = "sk_X509_push() failed"; + sk_X509_free(chain); + X509_free(x509); + BIO_free(bio); + ERR_clear_error(); + return NULL; + } + + /* read rest of the chain */ + + while (!BIO_eof(bio)) { + x509 = d2i_X509_bio(bio, NULL); + if (x509 == NULL) { + *err = "d2i_X509_bio() failed in rest of chain"; + sk_X509_pop_free(chain, X509_free); + BIO_free(bio); + ERR_clear_error(); + return NULL; + } + + if (sk_X509_push(chain, x509) == 0) { + *err = "sk_X509_push() failed in rest of chain"; + sk_X509_pop_free(chain, X509_free); + X509_free(x509); + BIO_free(bio); + ERR_clear_error(); + return NULL; + } + } + + BIO_free(bio); + + return chain; +} + + void ngx_http_lua_ffi_free_cert(void *cdata) { @@ -1209,6 +1284,40 @@ ngx_http_lua_ffi_parse_pem_priv_key(const u_char *pem, size_t pem_len, } +void * +ngx_http_lua_ffi_parse_der_priv_key(const char *data, size_t len, + char **err) +{ + BIO *bio = NULL; + EVP_PKEY *pkey = NULL; + + if (data == NULL || len == 0) { + *err = "invalid argument"; + ERR_clear_error(); + return NULL; + } + + bio = BIO_new_mem_buf((char *) data, len); + if (bio == NULL) { + *err = "BIO_new_mem_buf() failed"; + ERR_clear_error(); + return NULL; + } + + pkey = d2i_PrivateKey_bio(bio, NULL); + if (pkey == NULL) { + *err = "d2i_PrivateKey_bio() failed"; + BIO_free(bio); + ERR_clear_error(); + return NULL; + } + + BIO_free(bio); + + return pkey; +} + + void ngx_http_lua_ffi_free_priv_key(void *cdata) { diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 5b2ae018fe..6b5ff2f8ca 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -53,6 +53,12 @@ ffi.cdef[[ void *ngx_http_lua_ffi_parse_pem_priv_key(const unsigned char *pem, size_t pem_len, char **err); + void *ngx_http_lua_ffi_parse_der_cert(const char *data, size_t len, + char **err); + + void *ngx_http_lua_ffi_parse_der_priv_key(const char *data, size_t len, + char **err); + int ngx_http_lua_ffi_set_cert(void *r, void *cdata, char **err); @@ -1323,3 +1329,153 @@ SNI is test.com --- no_error_log [error] [alert] + + + +=== TEST 11: DER cert + private key cdata +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + local ffi = require "ffi" + require "defines" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + ffi.C.ngx_http_lua_ffi_ssl_clear_certs(r, errmsg) + + local f = assert(io.open("t/cert/test_der.crt", "rb")) + local cert_data = f:read("*all") + f:close() + + local cert = ffi.C.ngx_http_lua_ffi_parse_der_cert(cert_data, #cert_data, errmsg) + if not cert then + ngx.log(ngx.ERR, "failed to parse DER cert: ", + ffi.string(errmsg[0])) + return + end + + local rc = ffi.C.ngx_http_lua_ffi_set_cert(r, cert, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to set cdata cert: ", + ffi.string(errmsg[0])) + return + end + + ffi.C.ngx_http_lua_ffi_free_cert(cert) + + f = assert(io.open("t/cert/test_der.key", "rb")) + local pkey_data = f:read("*all") + f:close() + + local pkey = ffi.C.ngx_http_lua_ffi_parse_der_priv_key(pkey_data, #pkey_data, errmsg) + if pkey == nil then + ngx.log(ngx.ERR, "failed to parse DER priv key: ", + ffi.string(errmsg[0])) + return + end + + local rc = ffi.C.ngx_http_lua_ffi_set_priv_key(r, pkey, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to set cdata priv key: ", + ffi.string(errmsg[0])) + return + end + + ffi.C.ngx_http_lua_ffi_free_priv_key(pkey) + } + + ssl_certificate ../../cert/test2.crt; + ssl_certificate_key ../../cert/test2.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: cdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" + +--- no_error_log +[error] +[alert] diff --git a/t/cert/test_der.crt b/t/cert/test_der.crt new file mode 100644 index 0000000000000000000000000000000000000000..273b98638f7853e003f7d68d084b30f143a748a1 GIT binary patch literal 955 zcmXqLV%~1h#5A8tL@8ZQ{V?a!$BBCzToyRCou9?T#%I9G#;Mij(e|B}k&&B~!Ju)v zA-4f18*?ZNn=n&ou%WPlAc(`k!{wZqlbM!Zl$V)kC}to65@hG$4NlBca4Sm8OU^7# z&Nmb?5CDmD@o@SVq~-;s7ME1Q#JG{g1P%B>>X>;rN`PwglJj#76%6Fy4&h`J6HH7` z%`2(Oa7fQh%*+9)Fpv}HH8M7^G%z(VF|;(Yh!W>D262s`+(GQ}CPpRXU}j`xU~XdM zXE11DDrw9vORURY5N-PiK}$3qShn!PnHyI$?${Y9vV57KRH*Q4 z*8#QM+xa(E+q%q@6`2!%@$T#-_QU%;8Ru+t+Vtr7_V4k9tDmKu5%)Q=^Pcr=jarXp z?#l|UKc;+NtD94NEdG+M{Y!4`^Fb$9oEL1H8069@W?DJ%!m0a@e#M@YHQZ*l?2jT- zX}DK<=^gQJ;a4AR^tyiI)Q_adOPh6>m>C%ukwXZW5P>1Y$Pn_){`_M7K0X(Tnkyf- ze%e{JalO2NgpB;1Z)Q&y9DUV(ajn5-!%K?Wr2kY(yB(-lK52%Ob{+FOA^VaK)mdi* z{0*6(o?q&nw(iNB&s=tvk6Q1xsyIBAE}8ZI-*U+bzm!vcZ#{X8KO}cmWnW=upe0iL9cbGRj8~?rcpgxW#c&+lXiDmoZ8WI9eB?z67kz94mW|#am z_Fjn#3*QNw&HQp+$ieQN_llG9?D_L_p6@DrUd?;xbnf=4>aA~A+0A&en&D)WawJdo z_YVok#YsJO1yP_T`yB`ah*e_}!|WT=MnWA;|~E%eLI%djJ5* CVRZTc literal 0 HcmV?d00001 diff --git a/t/cert/test_der.key b/t/cert/test_der.key new file mode 100644 index 0000000000000000000000000000000000000000..14e9fa5821954099040a8490a1f344be65228cc9 GIT binary patch literal 1216 zcmV;x1V8&Qf&{z*0RS)!1_>&LNQUrrZ9p8q5=T`0)hbn0G!bJW_cjII!-q43Pu)F|9OFd+o=kRov)`ntd-ZQF0yt35=S9Tc2j(cPP32gANe0i3Zy zvf{_L_g`_V=48wlOvJg~JDVtdNP`R0AVv6;_pL2#bH-oNJwNFSE6-BNsLv6Gky1o~ z7BzX1(8}N9`d!H#F}60Q{2>B$S4wAf+!yv&)#9;A*VxMVVp-C&Edm1p009Dm0RRRz zj*-b_ip^;MD%E7e&uOyh0Cjq`o=mu>&51*dNc#RA@pU*<3+&87b^L`xT5G84fA(-L zx9V;(3Vrtg9}N!bYI8=byWl{nA3Y%ccP;Lm>i3}@ zGkHG?gfdIm| zKBkn~K-0L6Uw*h~49EeBD3QmN142D7_uDxIyG9CcVv{_V8u!&SH+&G}T__{W!x)Zv zIFmoIH{d~7S7UFA+E5#7zAAh}#2M#s)j+Syq0!;dcw(iqv1YL5`>Hve0Q7x2hX@h} zNl`_$b@pT3E!E6cFv(s<7tIpvqb62@iD)oi`b2 z5?#YEl=+E8yWm)df-VH1Q!e_6+~77*zqQpZc00!&WhU&#T)91|=J=qwNBL zfDI@8v#X;6c1or5_0Q+m)jOqT5L0*2{ENNK^;PKkIz~b`TGC-NMMHLfxi&iYz*>sm zGp&rPG3UR07! zC&yWO_;Ym+*g5-Cvn>e#!!+gd6n@v1N6WAcku(I@##UN4PMeA^>Nq54rF%xRL=5pn z2Lck`R2UH;`A#&t+FX=BAe0{&6Dn`U5A>Pvny|0;gJF(69|tRE39s%*E8>*~>V{)~IK_H&1*#_RZ)kf!M6wBOcRSBH8x=L&GOyz}HOel8 zsPZVMJ$k4plr$^Wuazw7!aL&u6(e&qn=!N=*d^QAxyfo?WA5dNu)lq_SR28Gfzl1D epCqH!-q!E@>y8q)XlN+Uf4;RvhuVFpY}}FtKTk3M literal 0 HcmV?d00001 From e0d19f787e74ce3ffbb9b9c5abad02fd6ea81f41 Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Wed, 28 Feb 2024 11:57:06 +0800 Subject: [PATCH 737/848] Revert "changes: modify read body api limitation for HTTP/2 or HTTP/3 requests." This reverts commit 86bea01b244938e0ab6eda780906c06fa62c3b5c. --- README.markdown | 4 ++-- src/ngx_http_lua_accessby.c | 29 ++++------------------------- src/ngx_http_lua_contentby.c | 29 ++++------------------------- src/ngx_http_lua_req_body.c | 28 ++++------------------------ t/023-rewrite/request_body.t | 2 -- t/024-access/request_body.t | 2 -- t/044-req-body.t | 4 +--- 7 files changed, 15 insertions(+), 83 deletions(-) diff --git a/README.markdown b/README.markdown index 56fb0565bf..d59a1b5818 100644 --- a/README.markdown +++ b/README.markdown @@ -2722,7 +2722,7 @@ lua_need_request_body **phase:** *depends on usage* -Due to the stream processing feature of HTTP/2 or HTTP/3, this configuration could potentially block the entire request. Therefore, this configuration is effective only when HTTP/2 or HTTP/3 requests send content-length header. For requests with versions lower than HTTP/2, this configuration can still be used without any problems. +Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code. @@ -5426,7 +5426,7 @@ Reads the client request body synchronously without blocking the Nginx event loo local args = ngx.req.get_post_args() ``` -Due to the stream processing feature of HTTP/2 or HTTP/3, this api could potentially block the entire request. Therefore, this api is effective only when HTTP/2 or HTTP/3 requests send content-length header. For requests with versions lower than HTTP/2, this api can still be used without any problems. +Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. If the request body is already read previously by turning on [lua_need_request_body](#lua_need_request_body) or by using other modules, then this function does not run and returns immediately. diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 2bf40aaa87..fa6810d9ed 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -136,26 +136,11 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) return NGX_DONE; } - if (llcf->force_read_body && !ctx->read_body_done) { - +/* http2 read body may break http2 stream process */ #if (NGX_HTTP_V2) - if (r->main->stream && r->headers_in.content_length_n < 0) { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "disable lua_need_request_body, since " - "http2 read_body may break http2 stream process"); - goto done; - } -#endif - -#if (NGX_HTTP_V3) - if (r->http_version == NGX_HTTP_VERSION_30 - && r->headers_in.content_length_n < 0) - { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "disable lua_need_request_body, since " - "http2 read_body may break http2 stream process"); - goto done; - } + if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { +#else + if (llcf->force_read_body && !ctx->read_body_done) { #endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; @@ -174,12 +159,6 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) } } -#if defined(NGX_HTTP_V3) || defined(NGX_HTTP_V2) - -done: - -#endif - dd("calling access handler"); return llcf->access_handler(r); } diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 2014d52d8d..d1c3bc9fd9 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -195,26 +195,11 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) return rc; } - if (llcf->force_read_body && !ctx->read_body_done) { - +/* http2 read body may break http2 stream process */ #if (NGX_HTTP_V2) - if (r->main->stream && r->headers_in.content_length_n < 0) { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "disable lua_need_request_body, since " - "http2 read_body may break http2 stream process"); - goto done; - } -#endif - -#if (NGX_HTTP_V3) - if (r->http_version == NGX_HTTP_VERSION_30 - && r->headers_in.content_length_n < 0) - { - ngx_log_error(NGX_LOG_WARN, r->connection->log, 0, - "disable lua_need_request_body, since " - "http2 read_body may break http2 stream process"); - goto done; - } + if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { +#else + if (llcf->force_read_body && !ctx->read_body_done) { #endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; @@ -234,12 +219,6 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) } } -#if defined(NGX_HTTP_V3) || defined(NGX_HTTP_V2) - -done: - -#endif - dd("setting entered"); ctx->entered_content_phase = 1; diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 61ab999045..32c75796d9 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -87,18 +87,8 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) /* http2 read body may break http2 stream process */ #if (NGX_HTTP_V2) - if (r->main->stream && r->headers_in.content_length_n < 0) { - return luaL_error(L, "http2 requests are not supported" - " without content-length header"); - } -#endif - -#if (NGX_HTTP_V3) - if (r->http_version == NGX_HTTP_VERSION_30 - && r->headers_in.content_length_n < 0) - { - return luaL_error(L, "http3 requests are not supported" - " without content-length header"); + if (r->main->stream) { + return luaL_error(L, "http2 requests are not supported yet"); } #endif @@ -351,18 +341,8 @@ ngx_http_lua_ngx_req_get_body_file(lua_State *L) /* http2 read body may break http2 stream process */ #if (NGX_HTTP_V2) - if (r->main->stream && r->headers_in.content_length_n < 0) { - return luaL_error(L, "http2 requests are not supported" - " without content-length header"); - } -#endif - -#if (NGX_HTTP_V3) - if (r->http_version == NGX_HTTP_VERSION_30 - && r->headers_in.content_length_n < 0) - { - return luaL_error(L, "http3 requests are not supported" - " without content-length header"); + if (r->main->stream) { + return luaL_error(L, "http2 requests are not supported yet"); } #endif diff --git a/t/023-rewrite/request_body.t b/t/023-rewrite/request_body.t index 32c02e151f..7383a1cebc 100644 --- a/t/023-rewrite/request_body.t +++ b/t/023-rewrite/request_body.t @@ -188,8 +188,6 @@ http finalize request: 500, "/echo_body?" a:1, c:0 "POST /echo_body hello\x00\x01\x02 world\x03\x04\xff" ---- more_headers -Content-Length: --- response_body eval "nil" --- no_error_log diff --git a/t/024-access/request_body.t b/t/024-access/request_body.t index 0aa12c8b55..3a3fbd1486 100644 --- a/t/024-access/request_body.t +++ b/t/024-access/request_body.t @@ -188,8 +188,6 @@ http finalize request: 500, "/echo_body?" a:1, c:0 "POST /echo_body hello\x00\x01\x02 world\x03\x04\xff" ---- more_headers -Content-Length: --- response_body eval "nil" --- no_error_log diff --git a/t/044-req-body.t b/t/044-req-body.t index f4509e1497..31b4b59586 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -1790,7 +1790,5 @@ content length: 5 --- request POST /test hello, world ---- more_headers -Content-Length: --- error_code: 500 ---- error_log: http2 requests are not supported without content-length header +--- error_log: http2 requests are not supported yet From 6e29c1a96e641ea3e3498b1524baeaa28d3ab58c Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Wed, 28 Feb 2024 11:59:10 +0800 Subject: [PATCH 738/848] Revert "bugfix: disable http2 in body read due to http2 stream processing bug." This reverts commit 0090f3faf66e6870b3974e983ae4b47459c96fa3. --- README.markdown | 4 ---- src/ngx_http_lua_accessby.c | 5 ----- src/ngx_http_lua_contentby.c | 5 ----- src/ngx_http_lua_req_body.c | 14 -------------- src/ngx_http_lua_rewriteby.c | 5 ----- src/ngx_http_lua_server_rewriteby.c | 7 +------ t/023-rewrite/request_body.t | 21 --------------------- t/024-access/request_body.t | 21 --------------------- t/044-req-body.t | 20 +------------------- 9 files changed, 2 insertions(+), 100 deletions(-) diff --git a/README.markdown b/README.markdown index d59a1b5818..11c7b64f8e 100644 --- a/README.markdown +++ b/README.markdown @@ -2722,8 +2722,6 @@ lua_need_request_body **phase:** *depends on usage* -Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. - Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code. To read the request body data within the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable, @@ -5426,8 +5424,6 @@ Reads the client request body synchronously without blocking the Nginx event loo local args = ngx.req.get_post_args() ``` -Due to the stream processing feature of HTTP2, it does not support HTTP2 connection. - If the request body is already read previously by turning on [lua_need_request_body](#lua_need_request_body) or by using other modules, then this function does not run and returns immediately. If the request body has already been explicitly discarded, either by the [ngx.req.discard_body](#ngxreqdiscard_body) function or other modules, this function does not run and returns immediately. diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index fa6810d9ed..d40eab123e 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -136,12 +136,7 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) return NGX_DONE; } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { -#else if (llcf->force_read_body && !ctx->read_body_done) { -#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index d1c3bc9fd9..5e2ae55209 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -195,12 +195,7 @@ ngx_http_lua_content_handler(ngx_http_request_t *r) return rc; } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { -#else if (llcf->force_read_body && !ctx->read_body_done) { -#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/src/ngx_http_lua_req_body.c b/src/ngx_http_lua_req_body.c index 32c75796d9..5d69735cde 100644 --- a/src/ngx_http_lua_req_body.c +++ b/src/ngx_http_lua_req_body.c @@ -85,13 +85,6 @@ ngx_http_lua_ngx_req_read_body(lua_State *L) return luaL_error(L, "request object not found"); } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (r->main->stream) { - return luaL_error(L, "http2 requests are not supported yet"); - } -#endif - r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; @@ -339,13 +332,6 @@ ngx_http_lua_ngx_req_get_body_file(lua_State *L) return luaL_error(L, "request object not found"); } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (r->main->stream) { - return luaL_error(L, "http2 requests are not supported yet"); - } -#endif - ngx_http_lua_check_fake_request(L, r); if (r->request_body == NULL || r->request_body->temp_file == NULL) { diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index c56bba5d7f..4109f288e3 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -140,12 +140,7 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) return NGX_DONE; } -/* http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { -#else if (llcf->force_read_body && !ctx->read_body_done) { -#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c index 997262eeb7..be860693f4 100644 --- a/src/ngx_http_lua_server_rewriteby.c +++ b/src/ngx_http_lua_server_rewriteby.c @@ -102,13 +102,8 @@ ngx_http_lua_server_rewrite_handler(ngx_http_request_t *r) return NGX_DONE; } -/* TODO: lscf do not have force_read_body - * http2 read body may break http2 stream process */ -#if (NGX_HTTP_V2) - if (llcf->force_read_body && !ctx->read_body_done && !r->main->stream) { -#else + /* TODO: lscf do not have force_read_body */ if (llcf->force_read_body && !ctx->read_body_done) { -#endif r->request_body_in_single_buf = 1; r->request_body_in_persistent_file = 1; r->request_body_in_clean_file = 1; diff --git a/t/023-rewrite/request_body.t b/t/023-rewrite/request_body.t index 7383a1cebc..b867d3a82c 100644 --- a/t/023-rewrite/request_body.t +++ b/t/023-rewrite/request_body.t @@ -170,24 +170,3 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug ---- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} - - - -=== TEST 9: test HTTP2 reading request body was disabled ---- config - location /echo_body { - lua_need_request_body on; - rewrite_by_lua_block { - ngx.print(ngx.var.request_body or "nil") - } - content_by_lua 'ngx.exit(ngx.OK)'; - } ---- http2 ---- request eval -"POST /echo_body -hello\x00\x01\x02 -world\x03\x04\xff" ---- response_body eval -"nil" ---- no_error_log diff --git a/t/024-access/request_body.t b/t/024-access/request_body.t index 3a3fbd1486..fa03195272 100644 --- a/t/024-access/request_body.t +++ b/t/024-access/request_body.t @@ -170,24 +170,3 @@ Expect: 100-Continue http finalize request: 500, "/echo_body?" a:1, c:2 http finalize request: 500, "/echo_body?" a:1, c:0 --- log_level: debug ---- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} - - - -=== TEST 9: test HTTP2 reading request body was disabled ---- config - location /echo_body { - lua_need_request_body on; - access_by_lua_block { - ngx.print(ngx.var.request_body or "nil") - } - content_by_lua 'ngx.exit(ngx.OK)'; - } ---- http2 ---- request eval -"POST /echo_body -hello\x00\x01\x02 -world\x03\x04\xff" ---- response_body eval -"nil" ---- no_error_log diff --git a/t/044-req-body.t b/t/044-req-body.t index 31b4b59586..e2b9374a05 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -7,7 +7,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 56); +plan tests => repeat_each() * (blocks() * 4 + 52 ); #no_diff(); no_long_string(); @@ -1774,21 +1774,3 @@ content length: 5 --- no_error_log [error] [alert] ---- skip_eval: 4:$ENV{TEST_NGINX_USE_HTTP3} - - - -=== TEST 53: HTTP2 read buffered body was discarded ---- config - location = /test { - content_by_lua_block { - local err = pcall(ngx.req.read_body()) - ngx.say(err) - } - } ---- http2 ---- request -POST /test -hello, world ---- error_code: 500 ---- error_log: http2 requests are not supported yet From 8dec675832574bdaf33e59258545b38633821137 Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Wed, 28 Feb 2024 12:27:58 +0800 Subject: [PATCH 739/848] docs: docs about ngx.read_body() API limitations in stream HTTP2 and HTTP3 processing case --- README.markdown | 2 +- doc/HttpLuaModule.wiki | 3 +-- t/044-req-body.t | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 11c7b64f8e..960c3a19fa 100644 --- a/README.markdown +++ b/README.markdown @@ -5639,7 +5639,7 @@ Returns a read-only cosocket object that wraps the downstream connection. Only [ In case of error, `nil` will be returned as well as a string describing the error. -Due to the streaming nature of HTTP2 and HTTP3, this API cannot be used when the downstream connection is HTTP2 and HTTP3. +**Note:** This method will block while waiting for client request body to be fully received. Block time depends on the [client_body_timeout](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout) directive and maximum body size specified by the [client_max_body_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. If read timeout occurs or client body size exceeds the defined limit, this function will not return and `408 Request Time-out` or `413 Request Entity Too Large` response will be returned to the client instead. The socket object returned by this method is usually used to read the current request's body in a streaming fashion. Do not turn on the [lua_need_request_body](#lua_need_request_body) directive, and do not mix this call with [ngx.req.read_body](#ngxreqread_body) and [ngx.req.discard_body](#ngxreqdiscard_body). diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 305626c767..0db9dd52b0 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4741,8 +4741,7 @@ Returns a read-only cosocket object that wraps the downstream connection. Only [ In case of error, nil will be returned as well as a string describing the error. -Due to the streaming nature of HTTP2 and HTTP3, this API cannot be used when the downstream connection is HTTP2 and HTTP3. - +'''Note:''' This method will block while waiting for client request body to be fully received. Block time depends on the [http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout client_body_timeout] directive and maximum body size specified by the [http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size client_max_body_size] directive. If read timeout occurs or client body size exceeds the defined limit, this function will not return and 408 Request Time-out or 413 Request Entity Too Large response will be returned to the client instead. The socket object returned by this method is usually used to read the current request's body in a streaming fashion. Do not turn on the [[#lua_need_request_body|lua_need_request_body]] directive, and do not mix this call with [[#ngx.req.read_body|ngx.req.read_body]] and [[#ngx.req.discard_body|ngx.req.discard_body]]. If any request body data has been pre-read into the Nginx core request header buffer, the resulting cosocket object will take care of this to avoid potential data loss resulting from such pre-reading. diff --git a/t/044-req-body.t b/t/044-req-body.t index e2b9374a05..d72b1eb632 100644 --- a/t/044-req-body.t +++ b/t/044-req-body.t @@ -7,7 +7,7 @@ log_level('warn'); repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 52 ); +plan tests => repeat_each() * (blocks() * 4 + 58); #no_diff(); no_long_string(); From 1c77f025faf0413b17bfc3bdcda284ac031286a4 Mon Sep 17 00:00:00 2001 From: Zhongwei Yao Date: Mon, 4 Mar 2024 22:45:01 +0800 Subject: [PATCH 740/848] bugfix: fix memory corruption in consecutive regex calls. This memory corruption will cause nginx crash and can reproduce under "--with-no-pool-patch" build by running t/048-match-dfa.t TEST 9. test log: ok 1 - t/048-match-dfa.t TEST 9: multiple match calls with captures and DFA. - status code ok ok 2 - t/048-match-dfa.t TEST 9: multiple match calls with captures and DFA. - response_body - response is expected (repeated req 0, req 0) t/048-match-dfa.t TEST 9: multiple match calls with captures and DFA. - Can't connect to 127.0.0.1:1984: Connection refused Retry connecting after 0.675 sec t/048-match-dfa.t TEST 9: multiple match calls with captures and DFA. - Can't connect to 127.0.0.1:1984: Connection refused Retry connecting after 0.825 sec error.log [notice] 1683757#0: using the "epoll" event method [notice] 1683757#0: openresty/1.25.3.1 (no pool) [notice] 1683757#0: built by gcc 13.2.1 20231011 (Red Hat 13.2.1-4) (GCC) [notice] 1683757#0: OS: Linux 6.7.4-100.fc38.x86_64 [notice] 1683757#0: getrlimit(RLIMIT_NOFILE): 1024:524288 free(): invalid next size (fast) Co-author-by: lijunlong --- src/ngx_http_lua_regex.c | 10 +++++----- t/048-match-dfa.t | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 1b52fa233d..30c1650653 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -688,11 +688,11 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, ngx_pool_t *old_pool; if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 2; + ovecsize = 1; re->ncaptures = 0; } else { - ovecsize = (re->ncaptures + 1) * 3; + ovecsize = re->ncaptures + 1; } old_pool = ngx_http_lua_pcre_malloc_init(NULL); @@ -710,7 +710,7 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, } ngx_regex_match_data_size = ovecsize; - ngx_regex_match_data = pcre2_match_data_create(ovecsize / 3, NULL); + ngx_regex_match_data = pcre2_match_data_create(ovecsize, NULL); if (ngx_regex_match_data == NULL) { rc = PCRE2_ERROR_NOMEMORY; @@ -756,8 +756,8 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, "n %ui, ovecsize %ui", flags, exec_opts, rc, n, ovecsize); #endif - if (!(flags & NGX_LUA_RE_MODE_DFA) && n > ovecsize / 3) { - n = ovecsize / 3; + if (n > ovecsize) { + n = ovecsize; } for (i = 0; i < n; i++) { diff --git a/t/048-match-dfa.t b/t/048-match-dfa.t index edf3662d4f..023231b5c0 100644 --- a/t/048-match-dfa.t +++ b/t/048-match-dfa.t @@ -207,3 +207,41 @@ exec opts: 0 你 --- no_error_log [error] + + + +=== TEST 9: matched with do +--- config + location /re { + content_by_lua ' + local m = ngx.re.match("hello", "(h)(e)(l)", "jo") + if m then + ngx.say(m[0]) + ngx.say(m[1]) + ngx.say(m[2]) + ngx.say(m[3]) + else + ngx.say("not matched!") + end + local m = ngx.re.match("horld", "(h)(e)?(l)?", "jo") + if m then + ngx.say(m[0]) + ngx.say(m[1]) + ngx.say(m[2]) + ngx.say(m[3]) + else + ngx.say("not matched!") + end + '; + } +--- request + GET /re +--- response_body +hel +h +e +l +h +h +false +false From 54e5cb09d3685c0d805644482cb28be43786ebf6 Mon Sep 17 00:00:00 2001 From: Ruidong-X Date: Wed, 6 Mar 2024 12:15:06 +0800 Subject: [PATCH 741/848] feature: add ngx_http_lua_ffi_ssl_client_random. --- src/ngx_http_lua_ssl_certby.c | 23 ++++++ t/140-ssl-c-api.t | 144 ++++++++++++++++++++++++++++++++++ 2 files changed, 167 insertions(+) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index d775f2b311..996c3d62a3 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1609,4 +1609,27 @@ ngx_http_lua_ffi_get_req_ssl_pointer(ngx_http_request_t *r) } +int +ngx_http_lua_ffi_ssl_client_random(ngx_http_request_t *r, + unsigned char *out, size_t *outlen, char **err) +{ + ngx_ssl_conn_t *ssl_conn; + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + *outlen = SSL_get_client_random(ssl_conn, out, *outlen); + + return NGX_OK; +} + + #endif /* NGX_HTTP_SSL */ diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 6b5ff2f8ca..001d26ae31 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -74,6 +74,9 @@ ffi.cdef[[ int ngx_http_lua_ffi_ssl_verify_client(void *r, void *cdata, int depth, char **err); + int ngx_http_lua_ffi_ssl_client_random(ngx_http_request_t *r, + unsigned char *out, size_t *outlen, char **err); + ]] _EOC_ } @@ -1479,3 +1482,144 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] + + + +=== TEST 12: client random +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + local ffi = require "ffi" + require "defines" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + -- test client random length + local out = ffi.new("unsigned char[?]", 0) + local sizep = ffi.new("size_t[1]", 0) + + local rc = ffi.C.ngx_http_lua_ffi_ssl_client_random(r, out, sizep, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to get client random length: ", + ffi.string(errmsg[0])) + return + end + + if tonumber(sizep[0]) ~= 32 then + ngx.log(ngx.ERR, "client random length does not equal 32") + return + end + + -- test client random value + out = ffi.new("unsigned char[?]", 50) + sizep = ffi.new("size_t[1]", 50) + + rc = ffi.C.ngx_http_lua_ffi_ssl_client_random(r, out, sizep, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to get client random: ", + ffi.string(errmsg[0])) + return + end + + local init_v = "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0" + if ffi.string(out, sizep[0]) == init_v then + ngx.log(ngx.ERR, "maybe the client random value is incorrect") + return + end + } + + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { ngx.status = 201 ngx.say("foo") ngx.exit(201) } + more_clear_headers Date; + } + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + + location /t { + content_by_lua_block { + do + local sock = ngx.socket.tcp() + + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local sess, err = sock:sslhandshake(nil, "test.com", true) + if not sess then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(sess)) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.say("failed to receive response status line: ", err) + break + end + + ngx.say("received: ", line) + end + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } + +--- request +GET /t +--- response_body +connected: 1 +ssl handshake: cdata +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: nginx +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil + +--- error_log +lua ssl server name: "test.com" + +--- no_error_log +[error] +[alert] From 9e59105fd9fd49aa3a34f8023afa1826c7a57f95 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 7 Mar 2024 10:31:25 +0800 Subject: [PATCH 742/848] tests: fixed typo. --- t/138-balancer.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/138-balancer.t b/t/138-balancer.t index 8ccdee76c4..679744a9e1 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -571,7 +571,7 @@ upstream sent more data than specified in "Content-Length" header while reading -=== TEST 18: error in balancer_by_llua_block +=== TEST 18: error in balancer_by_lua_block --- http_config upstream backend { server 0.0.0.1; From 6df2b6f01c2f75cafae122a5832f194c71280a15 Mon Sep 17 00:00:00 2001 From: Zhongwei Yao Date: Thu, 7 Mar 2024 23:59:06 -0800 Subject: [PATCH 743/848] bugfix: correct offset vector memory allocation size for PCRE2. --- src/ngx_http_lua_regex.c | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 30c1650653..646b483e1f 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -591,7 +591,11 @@ ngx_http_lua_ffi_compile_regex(const unsigned char *pat, size_t pat_len, re_comp.captures = 0; } else { +#if (NGX_PCRE2) + ovecsize = (re_comp.captures + 1) * 2; +#else ovecsize = (re_comp.captures + 1) * 3; +#endif } dd("allocating cap with size: %d", (int) ovecsize); @@ -684,21 +688,21 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, { int rc, exec_opts = 0; size_t *ov; - ngx_uint_t ovecsize, n, i; + ngx_uint_t ovecpair, n, i; ngx_pool_t *old_pool; if (flags & NGX_LUA_RE_MODE_DFA) { - ovecsize = 1; + ovecpair = 1; re->ncaptures = 0; } else { - ovecsize = re->ncaptures + 1; + ovecpair = re->ncaptures + 1; } old_pool = ngx_http_lua_pcre_malloc_init(NULL); if (ngx_regex_match_data == NULL - || ovecsize > ngx_regex_match_data_size) + || ovecpair > ngx_regex_match_data_size) { /* * Allocate a match data if not yet allocated or smaller than @@ -709,8 +713,8 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, pcre2_match_data_free(ngx_regex_match_data); } - ngx_regex_match_data_size = ovecsize; - ngx_regex_match_data = pcre2_match_data_create(ovecsize, NULL); + ngx_regex_match_data_size = ovecpair; + ngx_regex_match_data = pcre2_match_data_create(ovecpair, NULL); if (ngx_regex_match_data == NULL) { rc = PCRE2_ERROR_NOMEMORY; @@ -741,7 +745,7 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, #if (NGX_DEBUG) ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "pcre2_match failed: flags 0x%05Xd, options 0x%08Xd, " - "rc %d, ovecsize %ui", flags, exec_opts, rc, ovecsize); + "rc %d, ovecpair %ui", flags, exec_opts, rc, ovecpair); #endif goto failed; @@ -753,11 +757,11 @@ ngx_http_lua_ffi_exec_regex(ngx_http_lua_regex_t *re, int flags, #if (NGX_DEBUG) ngx_log_debug5(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, "pcre2_match: flags 0x%05Xd, options 0x%08Xd, rc %d, " - "n %ui, ovecsize %ui", flags, exec_opts, rc, n, ovecsize); + "n %ui, ovecpair %ui", flags, exec_opts, rc, n, ovecpair); #endif - if (n > ovecsize) { - n = ovecsize; + if (n > ovecpair) { + n = ovecpair; } for (i = 0; i < n; i++) { From 1654cc6a4da6806aa4688625db494fa0438d9841 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 8 Mar 2024 16:10:54 +0800 Subject: [PATCH 744/848] feature: add FFI function for balancer.disable_ssl. This is a rewrite PR https://github.com/openresty/lua-nginx-module/pull/1687 from vislee --- src/ngx_http_lua_balancer.c | 36 +++++++++++++++++++++++++++++ t/138-balancer.t | 46 +++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index af4da73388..146545bbd8 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -809,4 +809,40 @@ ngx_http_lua_ffi_balancer_recreate_request(ngx_http_request_t *r, } +int +ngx_http_lua_ffi_balancer_disable_ssl(ngx_http_request_t *r, char **err) +{ + ngx_http_lua_ctx_t *ctx; + ngx_http_upstream_t *u; + + if (r == NULL) { + *err = "no request found"; + return NGX_ERROR; + } + + u = r->upstream; + + if (u == NULL) { + *err = "no upstream found"; + return NGX_ERROR; + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + *err = "no ctx found"; + return NGX_ERROR; + } + + if ((ctx->context & NGX_HTTP_LUA_CONTEXT_BALANCER) == 0) { + *err = "API disabled in the current context"; + return NGX_ERROR; + } + + u->ssl = 0; + u->schema.len = sizeof("http://") - 1; + + return NGX_OK; +} + + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/138-balancer.t b/t/138-balancer.t index 679744a9e1..515168883e 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -591,3 +591,49 @@ upstream sent more data than specified in "Content-Length" header while reading "failed to load inlined Lua code: balancer_by_lua(nginx.conf:27):3: ')' expected (to close '(' at line 2) near ''", --- no_error_log [warn] + + + +=== TEST 19: disable ssl +--- http_config + lua_package_path "$TEST_NGINX_SERVER_ROOT/html/?.lua;;"; + + upstream backend { + server 127.0.0.1:$TEST_NGINX_SERVER_PORT; + balancer_by_lua_block { + local ffi = require "ffi" + local C = ffi.C +ffi.cdef[[ +int +ngx_http_lua_ffi_balancer_disable_ssl(ngx_http_request_t *r, char **err); +]] + local errmsg = ffi.new("char *[1]") + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + local rc = C.ngx_http_lua_ffi_balancer_disable_ssl(r, errmsg) + if rc < 0 then + ngx.log(ngx.ERR, "failed to disable ssl: ", ffi.string(errmsg[0])) + return + end + } + } +--- config + location = /t { + proxy_pass https://backend/back; + } + + location = /back { + echo ok; + } + +--- request + GET /t +--- response_body +ok +--- no_error_log +[error] +[cirt] From 6394debe28079d8516dfd58305244bd480c82bc6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 8 Mar 2024 18:59:51 +0800 Subject: [PATCH 745/848] optimize: allow to reenable the tls for the upstream. --- src/ngx_http_lua_balancer.c | 13 ++++++++++--- t/138-balancer.t | 4 ++-- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 146545bbd8..a8192db038 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -810,7 +810,8 @@ ngx_http_lua_ffi_balancer_recreate_request(ngx_http_request_t *r, int -ngx_http_lua_ffi_balancer_disable_ssl(ngx_http_request_t *r, char **err) +ngx_http_lua_ffi_balancer_set_upstream_tls(ngx_http_request_t *r, int on, + char **err) { ngx_http_lua_ctx_t *ctx; ngx_http_upstream_t *u; @@ -838,8 +839,14 @@ ngx_http_lua_ffi_balancer_disable_ssl(ngx_http_request_t *r, char **err) return NGX_ERROR; } - u->ssl = 0; - u->schema.len = sizeof("http://") - 1; + if (on == 0) { + u->ssl = 0; + u->schema.len = sizeof("http://") - 1; + + } else { + u->ssl = 1; + u->schema.len = sizeof("https://") - 1; + } return NGX_OK; } diff --git a/t/138-balancer.t b/t/138-balancer.t index 515168883e..5ea94df96d 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -605,7 +605,7 @@ upstream sent more data than specified in "Content-Length" header while reading local C = ffi.C ffi.cdef[[ int -ngx_http_lua_ffi_balancer_disable_ssl(ngx_http_request_t *r, char **err); +ngx_http_lua_ffi_balancer_set_upstream_tls(ngx_http_request_t *r, int on, char **err); ]] local errmsg = ffi.new("char *[1]") local r = require "resty.core.base" .get_request() @@ -614,7 +614,7 @@ ngx_http_lua_ffi_balancer_disable_ssl(ngx_http_request_t *r, char **err); return end - local rc = C.ngx_http_lua_ffi_balancer_disable_ssl(r, errmsg) + local rc = C.ngx_http_lua_ffi_balancer_set_upstream_tls(r, 0, errmsg) if rc < 0 then ngx.log(ngx.ERR, "failed to disable ssl: ", ffi.string(errmsg[0])) return From e5248aa8203d3e0075822a577c1cdd19f5f1f831 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 9 Mar 2024 12:30:14 +0800 Subject: [PATCH 746/848] bugfix: fixed HTTP HEAD request smuggling issue. --- src/ngx_http_lua_util.c | 6 ++++ t/020-subrequest.t | 80 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 8fd26561a7..727ca3da39 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -599,6 +599,12 @@ ngx_http_lua_send_chain_link(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, if (r->header_only) { ctx->eof = 1; + if (!r->request_body && r == r->main) { + if (ngx_http_discard_request_body(r) != NGX_OK) { + return NGX_ERROR; + } + } + if (ctx->buffering) { return ngx_http_lua_send_http10_headers(r, ctx); } diff --git a/t/020-subrequest.t b/t/020-subrequest.t index c731f1e61c..59b9f61a34 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -3527,3 +3527,83 @@ HTTP/1.1 400 Bad Request [error] --- skip_nginx 3: < 1.21.1 + + + +=== TEST 83: avoid request smuggling of HEAD req +--- config + location /capture { + server_tokens off; + more_clear_headers Date; + + content_by_lua_block { + ngx.say("Hello") + } + } + + location /t { + content_by_lua_block { + local req = [[ +HEAD /capture HTTP/1.1 +Host: test.com +Content-Length: 63 + +GET /capture HTTP/1.1 +Host: test.com +X: GET /bar HTTP/1.0 + +]] + + local sock = ngx.socket.tcp() + sock:settimeout(1000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send req: ", err) + return + end + + ngx.say("req bytes: ", bytes) + + local n_resp = 0 + + local reader = sock:receiveuntil("\r\n") + while true do + local line, err = reader() + if line then + ngx.say(line) + if line == "0" then + n_resp = n_resp + 1 + end + + if n_resp >= 2 then + break + end + + else + ngx.say("err: ", err) + break + end + end + + sock:close() + } + } +--- request +GET /t +--- response_body +req bytes: 117 +HTTP/1.1 200 OK +Server: nginx +Content-Type: text/plain +Connection: keep-alive + +err: timeout +--- error_log +lua tcp socket read timed out From ca942b698404ba3012ecbc220d75442d74074ff5 Mon Sep 17 00:00:00 2001 From: syz Date: Sun, 10 Mar 2024 18:03:43 +0800 Subject: [PATCH 747/848] feature: add udp cosocket bind api. Co-authored-by: lijunlong --- README.markdown | 32 ++++++++++ src/ngx_http_lua_socket_tcp.c | 8 +-- src/ngx_http_lua_socket_udp.c | 81 +++++++++++++++++++++++- src/ngx_http_lua_socket_udp.h | 4 +- t/062-count.t | 2 +- t/087-udp-socket.t | 112 +++++++++++++++++++++++++++++++++- 6 files changed, 226 insertions(+), 13 deletions(-) diff --git a/README.markdown b/README.markdown index 960c3a19fa..dc0380159e 100644 --- a/README.markdown +++ b/README.markdown @@ -3715,6 +3715,7 @@ Nginx API for Lua * [ngx.shared.DICT.capacity](#ngxshareddictcapacity) * [ngx.shared.DICT.free_space](#ngxshareddictfree_space) * [ngx.socket.udp](#ngxsocketudp) +* [udpsock:bind](#udpsockbind) * [udpsock:setpeername](#udpsocksetpeername) * [udpsock:send](#udpsocksend) * [udpsock:receive](#udpsockreceive) @@ -7522,6 +7523,7 @@ ngx.socket.udp Creates and returns a UDP or datagram-oriented unix domain socket object (also known as one type of the "cosocket" objects). The following methods are supported on this object: +* [bind](#udpsockbind) * [setpeername](#udpsocksetpeername) * [send](#udpsocksend) * [receive](#udpsockreceive) @@ -7536,6 +7538,36 @@ See also [ngx.socket.tcp](#ngxsockettcp). [Back to TOC](#nginx-api-for-lua) +udpsock:bind +------------ +**syntax:** *ok, err = udpsock:bind(address)* + +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*,ssl_session_fetch_by_lua*,ssl_client_hello_by_lua** + +Just like the standard [proxy_bind](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_bind) directive, this api makes the outgoing connection to a upstream server originate from the specified local IP address. + +Only IP addresses can be specified as the `address` argument. + +Here is an example for connecting to a TCP server from the specified local IP address: + +```nginx + + location /test { + content_by_lua_block { + local sock = ngx.socket.udp() + -- assume "192.168.1.10" is the local ip address + local ok, err = sock:bind("192.168.1.10") + if not ok then + ngx.say("failed to bind: ", err) + return + end + sock:close() + } + } +``` + +[Back to TOC](#nginx-api-for-lua) + udpsock:setpeername ------------------- diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 230679fa7d..0aa7109758 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -861,13 +861,7 @@ ngx_http_lua_socket_tcp_bind(lua_State *L) return luaL_error(L, "no ctx found"); } - ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_REWRITE - | NGX_HTTP_LUA_CONTEXT_ACCESS - | NGX_HTTP_LUA_CONTEXT_CONTENT - | NGX_HTTP_LUA_CONTEXT_TIMER - | NGX_HTTP_LUA_CONTEXT_SSL_CERT - | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH - | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO); + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); luaL_checktype(L, 1, LUA_TTABLE); diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 10285f6d0a..e2ba790fc5 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -54,16 +54,19 @@ static void ngx_http_lua_socket_udp_read_handler(ngx_http_request_t *r, ngx_http_lua_socket_udp_upstream_t *u); static void ngx_http_lua_socket_udp_handle_success(ngx_http_request_t *r, ngx_http_lua_socket_udp_upstream_t *u); -static ngx_int_t ngx_http_lua_udp_connect(ngx_http_lua_udp_connection_t *uc); +static ngx_int_t ngx_http_lua_udp_connect(ngx_http_lua_udp_connection_t *uc, + ngx_addr_t *local); static int ngx_http_lua_socket_udp_close(lua_State *L); static ngx_int_t ngx_http_lua_socket_udp_resume(ngx_http_request_t *r); static void ngx_http_lua_udp_resolve_cleanup(void *data); static void ngx_http_lua_udp_socket_cleanup(void *data); +static int ngx_http_lua_socket_udp_bind(lua_State *L); enum { SOCKET_CTX_INDEX = 1, SOCKET_TIMEOUT_INDEX = 2, + SOCKET_BIND_INDEX = 3, }; @@ -100,6 +103,9 @@ ngx_http_lua_inject_socket_udp_api(ngx_log_t *log, lua_State *L) lua_pushcfunction(L, ngx_http_lua_socket_udp_close); lua_setfield(L, -2, "close"); /* ngx socket mt */ + lua_pushcfunction(L, ngx_http_lua_socket_udp_bind); + lua_setfield(L, -2, "bind"); + lua_pushvalue(L, -1); lua_setfield(L, -2, "__index"); lua_rawset(L, LUA_REGISTRYINDEX); @@ -159,6 +165,7 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; ngx_str_t host; + ngx_addr_t *local; int port; ngx_resolver_ctx_t *rctx, temp; ngx_http_core_loc_conf_t *clcf; @@ -291,6 +298,13 @@ ngx_http_lua_socket_udp_setpeername(lua_State *L) u->read_timeout = u->conf->read_timeout; } + lua_rawgeti(L, 1, SOCKET_BIND_INDEX); + local = lua_touserdata(L, -1); + lua_pop(L, 1); + if (local != NULL) { + u->local = local; + } + ngx_memzero(&url, sizeof(ngx_url_t)); url.url.len = host.len; @@ -618,7 +632,7 @@ ngx_http_lua_socket_resolve_retval_handler(ngx_http_request_t *r, return 2; } - rc = ngx_http_lua_udp_connect(uc); + rc = ngx_http_lua_udp_connect(uc, u->local); if (rc != NGX_OK) { u->socket_errno = ngx_socket_errno; @@ -721,6 +735,56 @@ ngx_http_lua_socket_error_retval_handler(ngx_http_request_t *r, } +static int +ngx_http_lua_socket_udp_bind(lua_State *L) +{ + ngx_http_request_t *r; + ngx_http_lua_ctx_t *ctx; + int n; + u_char *text; + size_t len; + ngx_addr_t *local; + + n = lua_gettop(L); + + if (n != 2) { + return luaL_error(L, "expecting 2 arguments, but got %d", + lua_gettop(L)); + } + + r = ngx_http_lua_get_req(L); + if (r == NULL) { + return luaL_error(L, "no request found"); + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + return luaL_error(L, "no ctx found"); + } + + ngx_http_lua_check_context(L, ctx, NGX_HTTP_LUA_CONTEXT_YIELDABLE); + + luaL_checktype(L, 1, LUA_TTABLE); + + text = (u_char *) luaL_checklstring(L, 2, &len); + + local = ngx_http_lua_parse_addr(L, text, len); + if (local == NULL) { + lua_pushnil(L); + lua_pushfstring(L, "bad address"); + return 2; + } + + lua_rawseti(L, 1, SOCKET_BIND_INDEX); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua udp socket bind ip: %V", &local->name); + + lua_pushinteger(L, 1); + return 1; +} + + static int ngx_http_lua_socket_udp_send(lua_State *L) { @@ -1340,7 +1404,7 @@ ngx_http_lua_socket_udp_handle_success(ngx_http_request_t *r, static ngx_int_t -ngx_http_lua_udp_connect(ngx_http_lua_udp_connection_t *uc) +ngx_http_lua_udp_connect(ngx_http_lua_udp_connection_t *uc, ngx_addr_t *local) { int rc; ngx_int_t event; @@ -1413,8 +1477,19 @@ ngx_http_lua_udp_connect(ngx_http_lua_udp_connection_t *uc) return NGX_ERROR; } } + #endif + if (local != NULL) { + fprintf(stderr, "=== have local address\n"); + if (bind(s, local->sockaddr, local->socklen) == -1) { + ngx_log_error(NGX_LOG_CRIT, &uc->log, ngx_socket_errno, + "bind(%V) failed", &local->name); + + return NGX_ERROR; + } + } + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, &uc->log, 0, "connect to %V, fd:%d #%d", &uc->server, s, c->number); diff --git a/src/ngx_http_lua_socket_udp.h b/src/ngx_http_lua_socket_udp.h index 83333461db..61245b3647 100644 --- a/src/ngx_http_lua_socket_udp.h +++ b/src/ngx_http_lua_socket_udp.h @@ -42,6 +42,8 @@ struct ngx_http_lua_socket_udp_upstream_s { ngx_http_request_t *request; ngx_http_lua_udp_connection_t udp_connection; + ngx_addr_t *local; + ngx_msec_t read_timeout; ngx_http_upstream_resolved_t *resolved; @@ -53,7 +55,7 @@ struct ngx_http_lua_socket_udp_upstream_s { ngx_http_lua_co_ctx_t *co_ctx; - unsigned waiting; /* :1 */ + unsigned waiting:1; /* :1 */ }; diff --git a/t/062-count.t b/t/062-count.t index 07605f95b9..957590292b 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -481,7 +481,7 @@ n = 16 --- request GET /test --- response_body -n = 6 +n = 7 --- no_error_log [error] diff --git a/t/087-udp-socket.t b/t/087-udp-socket.t index ffe6b13e92..d1bc654365 100644 --- a/t/087-udp-socket.t +++ b/t/087-udp-socket.t @@ -4,7 +4,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (3 * blocks() + 15); +plan tests => repeat_each() * (3 * blocks() + 16); our $HtmlDir = html_dir; @@ -1222,3 +1222,113 @@ qr/send: fd:\d+ 22 of 22 send: fd:\d+ 16 of 16 send: fd:\d+ 22 of 22/ --- log_level: debug + + + +=== TEST 23: udp bind +--- config + server_tokens off; + location /t { + #set $port 5000; + set $port $TEST_NGINX_MEMCACHED_PORT; + #set $port 1234; + + content_by_lua_block { + local socket = ngx.socket + local udp = socket.udp() + + local port = ngx.var.port + udp:settimeout(1000) -- 1 sec + + local ok, err = udp:bind("127.0.0.10") + if not ok then + ngx.say("failed to bind: ", err) + return + end + + local ok, err = udp:setpeername("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected") + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = udp:send(req) + if not ok then + ngx.say("failed to send: ", err) + return + end + + local data, err = udp:receive() + if not data then + ngx.say("failed to receive data: ", err) + return + end + ngx.print("received ", #data, " bytes: ", data) + } + } +--- request +GET /t +--- response_body eval +"connected\nreceived 12 bytes: \x{00}\x{01}\x{00}\x{00}\x{00}\x{01}\x{00}\x{00}OK\x{0d}\x{0a}" +--- no_error_log +[error] +--- log_level: debug +--- error_log +lua udp socket receive buffer size: 65536 + + + +=== TEST 24: udp bind failed +--- config + server_tokens off; + location /t { + #set $port 5000; + set $port $TEST_NGINX_MEMCACHED_PORT; + #set $port 1234; + + content_by_lua_block { + local socket = ngx.socket + local udp = socket.udp() + + local port = ngx.var.port + udp:settimeout(1000) -- 1 sec + + local ok, err = udp:bind("127.0.0.1000") + if not ok then + ngx.say("failed to bind: ", err) + return + end + + local ok, err = udp:setpeername("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected") + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = udp:send(req) + if not ok then + ngx.say("failed to send: ", err) + return + end + + local data, err = udp:receive() + if not data then + ngx.say("failed to receive data: ", err) + return + end + ngx.print("received ", #data, " bytes: ", data) + } + } +--- request +GET /t +--- response_body +failed to bind: bad address +--- no_error_log +[error] +--- log_level: debug From 0189eb14f694a7c1f57a4f6efeec90fa0298f368 Mon Sep 17 00:00:00 2001 From: bas-vk Date: Sun, 10 Mar 2024 13:53:59 +0100 Subject: [PATCH 748/848] feature: add suport for deriving key from tls master secret. Co-authored-by: Bas van Kervel Co-authored-by: lijunlong --- config | 2 + src/ngx_http_lua_ssl_export_keying_material.c | 113 ++++++++++++++++++ src/ngx_http_lua_ssl_export_keying_material.h | 24 ++++ 3 files changed, 139 insertions(+) create mode 100644 src/ngx_http_lua_ssl_export_keying_material.c create mode 100644 src/ngx_http_lua_ssl_export_keying_material.h diff --git a/config b/config index 0e572c8bea..0266e308a5 100644 --- a/config +++ b/config @@ -289,6 +289,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_worker.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_client_helloby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_certby.c \ + $ngx_addon_dir/src/ngx_http_lua_ssl_export_keying_material.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_ocsp.c \ $ngx_addon_dir/src/ngx_http_lua_lex.c \ $ngx_addon_dir/src/ngx_http_lua_balancer.c \ @@ -354,6 +355,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_certby.h \ $ngx_addon_dir/src/ngx_http_lua_lex.h \ $ngx_addon_dir/src/ngx_http_lua_balancer.h \ + $ngx_addon_dir/src/ngx_http_lua_ssl_export_keying_material.h \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_storeby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl.h \ diff --git a/src/ngx_http_lua_ssl_export_keying_material.c b/src/ngx_http_lua_ssl_export_keying_material.c new file mode 100644 index 0000000000..45c791bb2b --- /dev/null +++ b/src/ngx_http_lua_ssl_export_keying_material.c @@ -0,0 +1,113 @@ + +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + + +#ifndef DDEBUG +#define DDEBUG 0 +#endif + + +#include "ddebug.h" + +#if (NGX_HTTP_SSL) + +#include + +#include "ngx_http_lua_cache.h" +#include "ngx_http_lua_initworkerby.h" +#include "ngx_http_lua_util.h" +#include "ngx_http_ssl_module.h" +#include "ngx_http_lua_contentby.h" +#include "ngx_http_lua_ssl_session_fetchby.h" +#include "ngx_http_lua_ssl.h" +#include "ngx_http_lua_directive.h" +#include "ngx_http_lua_ssl_export_keying_material.h" + + +ngx_int_t +ngx_http_lua_ffi_ssl_export_keying_material(ngx_http_request_t *r, + u_char *out, size_t out_size, const char *label, size_t llen, + const u_char *context, size_t ctxlen, int use_ctx, char **err) +{ +#if OPENSSL_VERSION_NUMBER < 0x10101000L + *err = "OpenSSL too old"; + return NGX_ERROR; +#else + ngx_connection_t *c; + ngx_ssl_conn_t *ssl_conn; + int rc; + + c = r->connection; + if (c == NULL || c->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = c->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl connection"; + return NGX_ERROR; + } + + rc = SSL_export_keying_material(ssl_conn, out, out_size, label, llen, + context, ctxlen, use_ctx); + if (rc == 1) { + return NGX_OK; + } + + ngx_ssl_error(NGX_LOG_INFO, c->log, 0, + "SSL_export_keying_material rc: %d, error: %s", + rc, ERR_error_string(ERR_get_error(), NULL)); + + *err = "SSL_export_keying_material() failed"; + + return NGX_ERROR; +#endif +} + + +ngx_int_t +ngx_http_lua_ffi_ssl_export_keying_material_early(ngx_http_request_t *r, + u_char *out, size_t out_size, const char *label, size_t llen, + const u_char *context, size_t ctxlen, char **err) +{ +#if OPENSSL_VERSION_NUMBER < 0x10101000L + *err = "OpenSSL too old"; + return NGX_ERROR; +#else + int rc; + ngx_ssl_conn_t *ssl_conn; + ngx_connection_t *c; + + c = r->connection; + if (c == NULL || c->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = c->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl connection"; + return NGX_ERROR; + } + + rc = SSL_export_keying_material_early(ssl_conn, out, out_size, + label, llen, context, ctxlen); + + if (rc == 1) { + return NGX_OK; + } + + ngx_ssl_error(NGX_LOG_INFO, c->log, 0, + "SSL_export_keying_material_early rc: %d, error: %s", + rc, ERR_error_string(ERR_get_error(), NULL)); + + *err = "SSL_export_keying_material_early() failed"; + + return NGX_ERROR; +#endif +} + +#endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_ssl_export_keying_material.h b/src/ngx_http_lua_ssl_export_keying_material.h new file mode 100644 index 0000000000..7a54bcda35 --- /dev/null +++ b/src/ngx_http_lua_ssl_export_keying_material.h @@ -0,0 +1,24 @@ + +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + + +#ifndef _NGX_HTTP_LUA_SSL_EXPORT_KEYING_MATERIAL_H_INCLUDED_ +#define _NGX_HTTP_LUA_SSL_EXPORT_KEYING_MATERIAL_H_INCLUDED_ + +#include "ngx_http_lua_common.h" + +#if (NGX_HTTP_SSL) +ngx_int_t ngx_http_lua_ffi_ssl_export_keying_material(ngx_http_request_t *r, + u_char *out, size_t out_size, const char *label, size_t llen, + const u_char *ctx, size_t ctxlen, int use_ctx, char **err); + +ngx_int_t ngx_http_lua_ffi_ssl_export_keying_material_early( + ngx_http_request_t *r, u_char *out, size_t out_size, const char *label, + size_t llen, const u_char *ctx, size_t ctxlen, char **err); +#endif + +#endif /* _NGX_HTTP_LUA_SSL_EXPORT_KEYING_MATERIAL_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ From 76e3d67a0498bd44d7e0c1c4eb60b60629ff1260 Mon Sep 17 00:00:00 2001 From: Elvin Efendi Date: Sun, 10 Mar 2024 10:36:00 -0400 Subject: [PATCH 749/848] feature: validate and expose nextUpdate field in OCSP response. --- src/ngx_http_lua_ssl_ocsp.c | 49 +++++++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_ssl_ocsp.c b/src/ngx_http_lua_ssl_ocsp.c index d1805b23dd..c9f24cd281 100644 --- a/src/ngx_http_lua_ssl_ocsp.c +++ b/src/ngx_http_lua_ssl_ocsp.c @@ -19,8 +19,9 @@ #ifdef NGX_HTTP_LUA_USE_OCSP static int ngx_http_lua_ssl_empty_status_callback(ngx_ssl_conn_t *ssl_conn, void *data); -#endif +static long ngx_http_lua_ssl_stapling_time(ASN1_GENERALIZEDTIME *asn1time); +#endif int ngx_http_lua_ffi_ssl_get_ocsp_responder_from_der_chain( @@ -262,7 +263,7 @@ ngx_http_lua_ffi_ssl_create_ocsp_request(const char *chain_data, int ngx_http_lua_ffi_ssl_validate_ocsp_response(const u_char *resp, size_t resp_len, const char *chain_data, size_t chain_len, - u_char *errbuf, size_t *errbuf_size) + u_char *errbuf, size_t *errbuf_size, long *valid) { #ifndef NGX_HTTP_LUA_USE_OCSP @@ -383,6 +384,16 @@ ngx_http_lua_ffi_ssl_validate_ocsp_response(const u_char *resp, goto error; } + if (nextupdate) { + *valid = ngx_http_lua_ssl_stapling_time(nextupdate); + if (*valid == NGX_ERROR) { + *errbuf_size = ngx_snprintf(errbuf, *errbuf_size, + "invalid nextUpdate time " + "in certificate status") - errbuf; + goto error; + } + } + sk_X509_free(chain); X509_free(cert); X509_free(issuer); @@ -437,6 +448,40 @@ ngx_http_lua_ssl_empty_status_callback(ngx_ssl_conn_t *ssl_conn, void *data) { return SSL_TLSEXT_ERR_OK; } + + +static long +ngx_http_lua_ssl_stapling_time(ASN1_GENERALIZEDTIME *asn1time) +{ + BIO *bio; + char *value; + size_t len; + time_t time; + + /* + * OpenSSL doesn't provide a way to convert ASN1_GENERALIZEDTIME + * into long. To do this, we use ASN1_GENERALIZEDTIME_print(), + * which uses the "MMM DD HH:MM:SS YYYY [GMT]" format (e.g., + * "Feb 3 00:55:52 2015 GMT"), and parse the result. + */ + + bio = BIO_new(BIO_s_mem()); + if (bio == NULL) { + return NGX_ERROR; + } + + /* fake weekday prepended to match C asctime() format */ + + BIO_write(bio, "Tue ", sizeof("Tue ") - 1); + ASN1_GENERALIZEDTIME_print(bio, asn1time); + len = BIO_get_mem_data(bio, &value); + + time = ngx_parse_http_time((u_char *) value, len); + + BIO_free(bio); + + return time; +} #endif From f725c60ea0e580b046e51e9bc8ee4cdec5eb6b40 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 10 Mar 2024 22:56:57 +0800 Subject: [PATCH 750/848] bugfix: Fixing compatibility issues with BoringSSL. --- src/ngx_http_lua_ssl_export_keying_material.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_ssl_export_keying_material.c b/src/ngx_http_lua_ssl_export_keying_material.c index 45c791bb2b..ec64c049a9 100644 --- a/src/ngx_http_lua_ssl_export_keying_material.c +++ b/src/ngx_http_lua_ssl_export_keying_material.c @@ -31,7 +31,13 @@ ngx_http_lua_ffi_ssl_export_keying_material(ngx_http_request_t *r, u_char *out, size_t out_size, const char *label, size_t llen, const u_char *context, size_t ctxlen, int use_ctx, char **err) { -#if OPENSSL_VERSION_NUMBER < 0x10101000L +#if defined(OPENSSL_IS_BORINGSSL) || OPENSSL_VERSION_NUMBER < 0x10101000L + *err = "BoringSSL does not support SSL_export_keying_material"; + return NGX_ERROR; +#elif defined(LIBRESSL_VERSION_NUMBER) + *err = "LibreSSL does not support SSL_export_keying_material"; + return NGX_ERROR; +#elif OPENSSL_VERSION_NUMBER < 0x10101000L *err = "OpenSSL too old"; return NGX_ERROR; #else @@ -73,7 +79,13 @@ ngx_http_lua_ffi_ssl_export_keying_material_early(ngx_http_request_t *r, u_char *out, size_t out_size, const char *label, size_t llen, const u_char *context, size_t ctxlen, char **err) { -#if OPENSSL_VERSION_NUMBER < 0x10101000L +#if defined(OPENSSL_IS_BORINGSSL) || OPENSSL_VERSION_NUMBER < 0x10101000L + *err = "BoringSSL does not support SSL_export_keying_material"; + return NGX_ERROR; +#elif defined(LIBRESSL_VERSION_NUMBER) + *err = "LibreSSL does not support SSL_export_keying_material"; + return NGX_ERROR; +#elif OPENSSL_VERSION_NUMBER < 0x10101000L *err = "OpenSSL too old"; return NGX_ERROR; #else From e2067ddd2b2897d3c6fa6f91ce4e8169fe8c97c6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 20 Mar 2024 12:02:38 +0800 Subject: [PATCH 751/848] bugfix: wrong arguments of setkeepalive() result in the compromise of data integrity. ==338736== Invalid read of size 8 ==338736== at 0x209890: ngx_http_lua_socket_tcp_handler (ngx_http_lua_socket_tcp.c:3341) ==338736== by 0x16CB21: ngx_epoll_process_events (ngx_epoll_module.c:1001) ==338736== by 0x160213: ngx_process_events_and_timers (ngx_event.c:262) ==338736== by 0x16B772: ngx_single_process_cycle (ngx_process_cycle.c:338) ==338736== by 0x13E8B7: main (nginx.c:394) ==338736== Address 0x68c8678 is 8 bytes inside a block of size 1,488 free'd ==338736== at 0x48472AC: free (vg_replace_malloc.c:974) ==338736== by 0x14035D: ngx_destroy_pool (ngx_palloc.c:76) ==338736== by 0x18694E: ngx_http_free_request (ngx_http_request.c:3799) ==338736== by 0x186AE0: ngx_http_close_request (ngx_http_request.c:3708) ==338736== by 0x187A6A: ngx_http_finalize_connection (ngx_http_request.c:2812) ==338736== by 0x1887C7: ngx_http_finalize_request (ngx_http_request.c:2685) ==338736== by 0x1883CC: ngx_http_finalize_request (ngx_http_request.c:2571) ==338736== by 0x2010B2: ngx_http_lua_finalize_request (ngx_http_lua_util.c:3706) ==338736== by 0x20B6A1: ngx_http_lua_socket_tcp_resume_helper (ngx_http_lua_socket_tcp.c:6132) ==338736== by 0x20BA75: ngx_http_lua_socket_tcp_read_resume (ngx_http_lua_socket_tcp.c:6030) ==338736== by 0x20356B: ngx_http_lua_content_wev_handler (ngx_http_lua_contentby.c:152) ==338736== by 0x20CA9F: ngx_http_lua_socket_handle_read_success (ngx_http_lua_socket_tcp.c:3602) ==338736== by 0x20CA9F: ngx_http_lua_socket_tcp_read (ngx_http_lua_socket_tcp.c:2607) ==338736== by 0x20D289: ngx_http_lua_socket_read_handler (ngx_http_lua_socket_tcp.c:3405) ==338736== by 0x20991D: ngx_http_lua_socket_tcp_handler (ngx_http_lua_socket_tcp.c:3356) ==338736== by 0x16C970: ngx_epoll_process_events (ngx_epoll_module.c:968) ==338736== by 0x160213: ngx_process_events_and_timers (ngx_event.c:262) ==338736== by 0x16B772: ngx_single_process_cycle (ngx_process_cycle.c:338) ==338736== by 0x13E8B7: main (nginx.c:394) ==338736== Block was alloc'd at ==338736== at 0x484482F: malloc (vg_replace_malloc.c:431) ==338736== by 0x165448: ngx_alloc (ngx_alloc.c:22) ==338736== by 0x1401B2: ngx_malloc (ngx_palloc.c:137) ==338736== by 0x1403EC: ngx_palloc (ngx_palloc.c:120) ==338736== by 0x140503: ngx_pcalloc (ngx_palloc.c:215) ==338736== by 0x185BC9: ngx_http_alloc_request (ngx_http_request.c:580) ==338736== by 0x186356: ngx_http_create_request (ngx_http_request.c:536) ==338736== by 0x189F2A: ngx_http_wait_request_handler (ngx_http_request.c:518) ==338736== by 0x16C970: ngx_epoll_process_events (ngx_epoll_module.c:968) ==338736== by 0x160213: ngx_process_events_and_timers (ngx_event.c:262) ==338736== by 0x16B772: ngx_single_process_cycle (ngx_process_cycle.c:338) ==338736== by 0x13E8B7: main (nginx.c:394) ==338736== --- src/ngx_http_lua_socket_tcp.c | 50 ++++++----- t/068-socket-keepalive.t | 160 ++++++++++++++++++++++++++++++++++ 2 files changed, 188 insertions(+), 22 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 0aa7109758..214e78329e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5379,6 +5379,34 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) luaL_checktype(L, 1, LUA_TTABLE); + r = ngx_http_lua_get_req(L); + if (r == NULL) { + return luaL_error(L, "no request found"); + } + + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + + /* luaL_checkinteger will throw error if the argument is not a number. + * e.g.: bad argument \#2 to '?' (number expected, got string) + * + * We should check the argument in advance; otherwise, + * throwing an exception in the middle can compromise data integrity. + * e.g.: set pc->connection to NULL without following cleanup. + */ + if (n >= 2 && !lua_isnil(L, 2)) { + timeout = (ngx_msec_t) luaL_checkinteger(L, 2); + + } else { + timeout = llcf->keepalive_timeout; + } + + if (n >= 3 && !lua_isnil(L, 3)) { + pool_size = luaL_checkinteger(L, 3); + + } else { + pool_size = llcf->pool_size; + } + lua_rawgeti(L, 1, SOCKET_CTX_INDEX); u = lua_touserdata(L, -1); lua_pop(L, 1); @@ -5405,11 +5433,6 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) return 2; } - r = ngx_http_lua_get_req(L); - if (r == NULL) { - return luaL_error(L, "no request found"); - } - if (u->request != r) { return luaL_error(L, "bad request"); } @@ -5480,18 +5503,8 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) /* stack: obj timeout? size? pools cache_key */ - llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); - if (spool == NULL) { /* create a new socket pool for the current peer key */ - - if (n >= 3 && !lua_isnil(L, 3)) { - pool_size = luaL_checkinteger(L, 3); - - } else { - pool_size = llcf->pool_size; - } - if (pool_size <= 0) { msg = lua_pushfstring(L, "bad \"pool_size\" option value: %d", pool_size); @@ -5555,13 +5568,6 @@ ngx_http_lua_socket_tcp_setkeepalive(lua_State *L) ngx_del_timer(c->write); } - if (n >= 2 && !lua_isnil(L, 2)) { - timeout = (ngx_msec_t) luaL_checkinteger(L, 2); - - } else { - timeout = llcf->keepalive_timeout; - } - #if (NGX_DEBUG) if (timeout == 0) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index 5ca94e4b72..1660a3a361 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -3029,3 +3029,163 @@ connected: 1, reused: 0 --- error_log lua tcp socket keepalive create connection pool for key "A" lua tcp socket keepalive create connection pool for key "B" + + + +=== TEST 54: wrong first argument for setkeepalive +--- quic_max_idle_timeout: 1.2 +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config + server_tokens off; + location /t { + keepalive_timeout 60s; + + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local port = ngx.var.port + + local sock = ngx.socket.tcp() + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.1\r\nHost: localhost\r\nConnection: keepalive\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent: ", bytes) + + local reader = sock:receiveuntil("\r\n0\r\n\r\n") + local data, err = reader() + + if not data then + ngx.say("failed to receive response body: ", err) + return + end + + ngx.say("received response of ", #data, " bytes") + + ok, err = sock:setkeepalive() + if not ok then + ngx.say("failed to set reusable: ", err) + end + + ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ok, err = sock:setkeepalive("not a number", "not a number") + if not ok then + ngx.say("failed to set reusable: ", err) + end + } + } + + location /foo { + echo foo; + } + + location /sleep { + echo_sleep 1; + } +--- request +GET /t +--- error_code: +--- response_body +--- error_log eval +qr/\Qbad argument #1 to 'setkeepalive' (number expected, got string)\E/ +--- no_error_log +[crit] +--- timeout: 4 + + + +=== TEST 55: wrong second argument for setkeepalive +--- quic_max_idle_timeout: 1.2 +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config + server_tokens off; + location /t { + keepalive_timeout 60s; + + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local port = ngx.var.port + + local sock = ngx.socket.tcp() + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.1\r\nHost: localhost\r\nConnection: keepalive\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent: ", bytes) + + local reader = sock:receiveuntil("\r\n0\r\n\r\n") + local data, err = reader() + + if not data then + ngx.say("failed to receive response body: ", err) + return + end + + ngx.say("received response of ", #data, " bytes") + + ok, err = sock:setkeepalive() + if not ok then + ngx.say("failed to set reusable: ", err) + end + + ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ok, err = sock:setkeepalive(10, "not a number") + if not ok then + ngx.say("failed to set reusable: ", err) + end + } + } + + location /foo { + echo foo; + } + + location /sleep { + echo_sleep 1; + } +--- request +GET /t +--- error_code: +--- response_body +--- error_log eval +qr/\Qbad argument #2 to 'setkeepalive' (number expected, got string)\E/ +--- no_error_log +[crit] +--- timeout: 4 From 6c00bd4765ec5f7bf090a2c6424d11845fc4ab72 Mon Sep 17 00:00:00 2001 From: Liu Wei <375636559@qq.com> Date: Thu, 11 Apr 2024 20:54:19 +0800 Subject: [PATCH 752/848] bugfix: the connection won't be closed normally when set arg[1] = "" before arg[2] = true. --- src/ngx_http_lua_bodyfilterby.c | 18 ++++++++++++-- t/082-body-filter-2.t | 44 +++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 78e3b5c2d6..c0484c8de0 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -532,9 +532,23 @@ ngx_http_lua_body_filter_param_set(lua_State *L, ngx_http_request_t *r, if (last) { ctx->seen_last_in_filter = 1; - /* the "in" chain cannot be NULL and we set the "last_buf" or - * "last_in_chain" flag in the last buf of "in" */ + /* the "in" chain cannot be NULL except that we set arg[1] = "" + * before arg[2] = true + */ + if (in == NULL) { + in = ngx_http_lua_chain_get_free_buf(r->connection->log, + r->pool, + &ctx->free_bufs, 0); + if (in == NULL) { + return luaL_error(L, "no memory"); + } + + in->buf->tag = (ngx_buf_tag_t) &ngx_http_lua_body_filter; + lmcf->body_filter_chain = in; + } + /* we set the "last_buf" or "last_in_chain" flag + * in the last buf of "in" */ for (cl = in; cl; cl = cl->next) { if (cl->next == NULL) { if (r == r->main) { diff --git a/t/082-body-filter-2.t b/t/082-body-filter-2.t index ffccad28ab..3c9b9797d5 100644 --- a/t/082-body-filter-2.t +++ b/t/082-body-filter-2.t @@ -225,3 +225,47 @@ GET /t [error] [alert] [crit] + + + +=== TEST 4: set resp body nil with ngx.arg[1] first +--- config + location /t { + content_by_lua_block { + ngx.say("Hello World!") + } + + body_filter_by_lua_block { + ngx.arg[1] = "" + ngx.arg[2] = true + } + } +--- request +GET /t +--- response_body +--- no_error_log +[error] +[alert] +[crit] + + + +=== TEST 5: set resp body nil with ngx.arg[2] first +--- config + location /t { + content_by_lua_block { + ngx.say("Hello World!") + } + + body_filter_by_lua_block { + ngx.arg[2] = true + ngx.arg[1] = "" + } + } +--- request +GET /t +--- response_body +--- no_error_log +[error] +[alert] +[crit] From c1d309284d26e7ba44c155758a85ef793505b2b2 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 27 May 2024 11:40:03 +0800 Subject: [PATCH 753/848] bugfix: worker thread Lua VM may take lots of memory. --- README.markdown | 4 +- src/ngx_http_lua_module.c | 2 +- src/ngx_http_lua_worker_thread.c | 12 +++++- t/166-worker-thread.t | 71 ++++++++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index dc0380159e..51607e7f36 100644 --- a/README.markdown +++ b/README.markdown @@ -3599,7 +3599,7 @@ lua_worker_thread_vm_pool_size **syntax:** *lua_worker_thread_vm_pool_size <size>* -**default:** *lua_worker_thread_vm_pool_size 100* +**default:** *lua_worker_thread_vm_pool_size 10* **context:** *http* @@ -3611,6 +3611,8 @@ The Lua VM in the VM pool is used to execute Lua code in separate thread. The pool is global at Nginx worker level. And it is used to reuse Lua VMs between requests. +**Warning:** Each worker thread uses a separate Lua VM and caches the Lua VM for reuse in subsequent operations. Configuring too many worker threads can result in consuming a lot of memory. + [Back to TOC](#directives) Nginx API for Lua diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index fb10bf933e..f64ab89315 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1141,7 +1141,7 @@ ngx_http_lua_init_main_conf(ngx_conf_t *cf, void *conf) #endif if (lmcf->worker_thread_vm_pool_size == NGX_CONF_UNSET_UINT) { - lmcf->worker_thread_vm_pool_size = 100; + lmcf->worker_thread_vm_pool_size = 10; } if (ngx_http_lua_init_builtin_headers_out(cf, lmcf) != NGX_OK) { diff --git a/src/ngx_http_lua_worker_thread.c b/src/ngx_http_lua_worker_thread.c index 3820d6377c..ad747e8104 100644 --- a/src/ngx_http_lua_worker_thread.c +++ b/src/ngx_http_lua_worker_thread.c @@ -194,11 +194,21 @@ ngx_http_lua_get_task_ctx(lua_State *L, ngx_http_request_t *r) static void ngx_http_lua_free_task_ctx(ngx_http_lua_task_ctx_t *ctx) { + lua_State *vm; + ctx->next = ctxpool->next; ctxpool->next = ctx; /* clean Lua stack */ - lua_settop(ctx->vm, 0); + vm = ctx->vm; + + /* call collectgarbage("collect") */ + lua_settop(vm, 0); + lua_getglobal(vm, "collectgarbage"); + lua_pushstring(vm, "collect"); + lua_pcall(vm, 1, 1, 0); + + lua_settop(vm, 0); } diff --git a/t/166-worker-thread.t b/t/166-worker-thread.t index 925a060949..1cbeec5eba 100644 --- a/t/166-worker-thread.t +++ b/t/166-worker-thread.t @@ -1602,3 +1602,74 @@ return {hello=hello} GET /hello --- response_body false , suspicious circular references, table depth exceed max depth: 100 in the argument + + + +=== TEST 50: call run_worker_thread twice +--- main_config + thread_pool testpool threads=1; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + + ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", hello_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return "hello" +end +return {hello=hello} +--- request +GET /hello +--- response_body +true : hello +true : hello + + + +=== TEST 51: big object +--- main_config + thread_pool testpool threads=1; +--- http_config eval + "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" +--- config +location /hello { + default_type 'text/plain'; + + content_by_lua_block { + local ok, hello_or_err = ngx.run_worker_thread("testpool", "hello", "hello") + ngx.say(ok, " : ", #hello_or_err) + + local ok, gcsize_or_err = ngx.run_worker_thread("testpool", "hello", "gcsize") + ngx.say(ok, " : ", gcsize_or_err) + } +} +--- user_files +>>> hello.lua +local function hello() + return string.rep("helloworld", 1000000) +end + +local function gcsize() + return collectgarbage("count") +end + +return { + hello = hello, + gcsize = gcsize +} +--- request +GET /hello +--- response_body eval +qr/\Atrue : 10000000 +true : \d{3,4}\.\d+ +\z/ms From 5bf876104cabbc02aeb59c79ddfebb0a52cff6d6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=BE=90=E7=91=9E=E4=B8=9C?= Date: Wed, 29 May 2024 22:07:23 +0800 Subject: [PATCH 754/848] doc: update doc for 'ngx.req.http_version'. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 51607e7f36..ba1e5bfb5f 100644 --- a/README.markdown +++ b/README.markdown @@ -4789,7 +4789,7 @@ ngx.req.http_version Returns the HTTP version number for the current request as a Lua number. -Current possible values are 2.0, 1.0, 1.1, and 0.9. Returns `nil` for unrecognized values. +Current possible values are 3.0, 2.0, 1.0, 1.1, and 0.9. Returns `nil` for unrecognized values. This method was first introduced in the `v0.7.17` release. From b5d1688ae722538ba4dc8a7ec08820a08abfb93d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ji=C5=99=C3=AD=20Setni=C4=8Dka?= Date: Wed, 5 Jun 2024 08:14:59 +0200 Subject: [PATCH 755/848] bugfix: fix config test for signalfd with gcc 11. Without the initialization, the feature test ends with: error: 'set' may be used uninitialized [-Werror=maybe-uninitialized] --- config | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/config b/config index 0266e308a5..24ebd126d6 100644 --- a/config +++ b/config @@ -462,7 +462,8 @@ ngx_feature_libs= ngx_feature_name="NGX_HTTP_LUA_HAVE_SIGNALFD" ngx_feature_run=no ngx_feature_incs="#include " -ngx_feature_test="sigset_t set; signalfd(-1, &set, SFD_NONBLOCK|SFD_CLOEXEC);" +ngx_feature_test="sigset_t set = { 0 }; + signalfd(-1, &set, SFD_NONBLOCK|SFD_CLOEXEC);" SAVED_CC_TEST_FLAGS="$CC_TEST_FLAGS" CC_TEST_FLAGS="-Werror -Wall $CC_TEST_FLAGS" From 892b7ee5e84e7e24e5976de45e6fdfdbc08233e2 Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Fri, 28 Jun 2024 08:46:45 +0800 Subject: [PATCH 756/848] bugfix: lua-nginx-module context was clear when ngx.send_header() trigger filter_finalize case. --- src/ngx_http_lua_util.c | 4 ++++ t/002-content.t | 22 ++++++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 727ca3da39..f1e0cd08f5 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -549,6 +549,10 @@ ngx_http_lua_send_header_if_needed(ngx_http_request_t *r, if (!ctx->buffering) { dd("sending headers"); rc = ngx_http_send_header(r); + if (r->filter_finalize) { + ngx_http_set_ctx(r, ctx, ngx_http_lua_module); + } + ctx->header_sent = 1; return rc; } diff --git a/t/002-content.t b/t/002-content.t index 54de40ebd5..eb9d587f89 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -1098,3 +1098,25 @@ failed to load inlined Lua code: content_by_lua(...45678901234567890123456789012 GET /lua --- response_body_like: 503 Service Temporarily Unavailable --- error_code: 503 + + + +=== TEST 52: send_header trigger filter finalize does not clear the ctx +--- config + location /lua { + content_by_lua_block { + ngx.header["Last-Modified"] = ngx.http_time(ngx.time()) + ngx.send_headers() + local phase = ngx.get_phase() + } + header_filter_by_lua_block { + ngx.header["X-Hello-World"] = "Hello World" + } + } +--- request +GET /lua +--- more_headers +If-Unmodified-Since: Wed, 01 Jan 2020 07:28:00 GMT +--- error_code: 412 +--- no_error_log +unknown phase: 0 From 4f8b94375983a47ecce4810a203e56b5ef4ed6c5 Mon Sep 17 00:00:00 2001 From: fesily Date: Mon, 1 Jul 2024 14:03:12 +0800 Subject: [PATCH 757/848] bugfix:main thread access free fake request. --- src/ngx_http_lua_initworkerby.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ngx_http_lua_initworkerby.c b/src/ngx_http_lua_initworkerby.c index 449e604ace..edb68df08a 100644 --- a/src/ngx_http_lua_initworkerby.c +++ b/src/ngx_http_lua_initworkerby.c @@ -297,6 +297,8 @@ ngx_http_lua_init_worker(ngx_cycle_t *cycle) (void) lmcf->init_worker_handler(cycle->log, lmcf, lmcf->lua); + ngx_http_lua_set_req(lmcf->lua, NULL); + ngx_destroy_pool(c->pool); return NGX_OK; From 7b6fa21abc41b86f0bd59b377fa354b8556d9f54 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 1 Jul 2024 14:43:39 +0800 Subject: [PATCH 758/848] feature: implemented keepalive pooling in 'balancer_by_lua*'. Co-authored-by: Thibault Charbonnier --- README.markdown | 24 ++ doc/HttpLuaModule.wiki | 20 + src/ngx_http_lua_balancer.c | 736 +++++++++++++++++++++++++++++++----- src/ngx_http_lua_balancer.h | 2 + src/ngx_http_lua_common.h | 19 +- src/ngx_http_lua_module.c | 13 +- 6 files changed, 709 insertions(+), 105 deletions(-) diff --git a/README.markdown b/README.markdown index ba1e5bfb5f..4957e50059 100644 --- a/README.markdown +++ b/README.markdown @@ -1144,6 +1144,7 @@ Directives * [log_by_lua_file](#log_by_lua_file) * [balancer_by_lua_block](#balancer_by_lua_block) * [balancer_by_lua_file](#balancer_by_lua_file) +* [balancer_keepalive](#balancer_keepalive) * [lua_need_request_body](#lua_need_request_body) * [ssl_client_hello_by_lua_block](#ssl_client_hello_by_lua_block) * [ssl_client_hello_by_lua_file](#ssl_client_hello_by_lua_file) @@ -2711,6 +2712,29 @@ This directive was first introduced in the `v0.10.0` release. [Back to TOC](#directives) +balancer_keepalive +------------------ + +**syntax:** *balancer_keepalive <total-connections>* + +**context:** *upstream* + +**phase:** *loading-config* + +The `total-connections` parameter sets the maximum number of idle +keepalive connections to upstream servers that are preserved in the cache of +each worker process. When this number is exceeded, the least recently used +connections are closed. + +It should be particularly noted that the keepalive directive does not limit the +total number of connections to upstream servers that an nginx worker process +can open. The connections parameter should be set to a number small enough to +let upstream servers process new incoming connections as well. + +This directive was first introduced in the `v0.10.21` release. + +[Back to TOC](#directives) + lua_need_request_body --------------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 0db9dd52b0..89cf1ca707 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2288,6 +2288,26 @@ When a relative path like foo/bar.lua is given, they will be turned This directive was first introduced in the v0.10.0 release. +== balancer_keepalive == + +'''syntax:''' ''balancer_keepalive '' + +'''context:''' ''upstream'' + +'''phase:''' ''loading-config'' + +The total-connections parameter sets the maximum number of idle +keepalive connections to upstream servers that are preserved in the cache of +each worker process. When this number is exceeded, the least recently used +connections are closed. + +It should be particularly noted that the keepalive directive does not limit the +total number of connections to upstream servers that an nginx worker process +can open. The connections parameter should be set to a number small enough to +let upstream servers process new incoming connections as well. + +This directive was first introduced in the v0.10.21 release. + == lua_need_request_body == '''syntax:''' ''lua_need_request_body '' diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index a8192db038..9bb770c8e6 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -15,10 +15,34 @@ #include "ngx_http_lua_util.h" #include "ngx_http_lua_directive.h" +#define NGX_BALANCER_DEF_HOST_LEN 32 +typedef struct { + ngx_queue_t queue; + ngx_queue_t hnode; + ngx_uint_t hash; + ngx_http_lua_srv_conf_t *lscf; + ngx_connection_t *connection; + socklen_t socklen; + ngx_sockaddr_t sockaddr; + ngx_str_t host; + /* try to avoid allocating memory from the connection pool */ + u_char host_data[NGX_BALANCER_DEF_HOST_LEN]; +} ngx_http_lua_balancer_ka_item_t; /*balancer keepalive item*/ + struct ngx_http_lua_balancer_peer_data_s { - /* the round robin data must be first */ - ngx_http_upstream_rr_peer_data_t rrp; + ngx_uint_t keepalive_requests; + ngx_msec_t keepalive_timeout; + + void *data; + + ngx_event_get_peer_pt original_get_peer; + ngx_event_free_peer_pt original_free_peer; + +#if (NGX_HTTP_SSL) + ngx_event_set_peer_session_pt original_set_session; + ngx_event_save_peer_session_pt original_save_session; +#endif ngx_http_lua_srv_conf_t *conf; ngx_http_request_t *request; @@ -29,14 +53,16 @@ struct ngx_http_lua_balancer_peer_data_s { struct sockaddr *sockaddr; socklen_t socklen; - ngx_str_t *host; - in_port_t port; + ngx_str_t host; + ngx_str_t *addr_text; int last_peer_state; #if !(HAVE_NGX_UPSTREAM_TIMEOUT_FIELDS) - unsigned cloned_upstream_conf; /* :1 */ + unsigned cloned_upstream_conf:1; #endif + + unsigned keepalive:1; }; @@ -45,6 +71,9 @@ static ngx_int_t ngx_http_lua_balancer_set_session(ngx_peer_connection_t *pc, void *data); static void ngx_http_lua_balancer_save_session(ngx_peer_connection_t *pc, void *data); +static ngx_int_t +ngx_http_lua_upstream_get_ssl_name(ngx_http_request_t *r, + ngx_http_upstream_t *u); #endif static ngx_int_t ngx_http_lua_balancer_init(ngx_conf_t *cf, ngx_http_upstream_srv_conf_t *us); @@ -56,6 +85,16 @@ static ngx_int_t ngx_http_lua_balancer_by_chunk(lua_State *L, ngx_http_request_t *r); static void ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, ngx_uint_t state); +static void ngx_http_lua_balancer_close(ngx_connection_t *c); +static void ngx_http_lua_balancer_dummy_handler(ngx_event_t *ev); +static void ngx_http_lua_balancer_close_handler(ngx_event_t *ev); +static ngx_connection_t *ngx_http_lua_balancer_get_cached_item( + ngx_http_lua_srv_conf_t *lscf, ngx_peer_connection_t *pc, ngx_str_t *name); +static ngx_uint_t ngx_http_lua_balancer_calc_hash(ngx_str_t *name, + struct sockaddr *sockaddr, socklen_t socklen); + + +static struct sockaddr *ngx_http_lua_balancer_default_server_sockaddr; ngx_int_t @@ -131,8 +170,10 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, u_char *name; ngx_str_t *value; ngx_http_lua_srv_conf_t *lscf = conf; + ngx_url_t url; ngx_http_upstream_srv_conf_t *uscf; + ngx_http_upstream_server_t *us; dd("enter"); @@ -188,11 +229,42 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, lscf->balancer.src_key = cache_key; + /* balancer setup */ + uscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_upstream_module); + if (uscf->servers->nelts == 0) { + us = ngx_array_push(uscf->servers); + if (us == NULL) { + return NGX_CONF_ERROR; + } + + ngx_memzero(us, sizeof(ngx_http_upstream_server_t)); + ngx_memzero(&url, sizeof(ngx_url_t)); + + ngx_str_set(&url.url, "0.0.0.1"); + url.default_port = 80; + + if (ngx_parse_url(cf->pool, &url) != NGX_OK) { + return NGX_CONF_ERROR; + } + + us->name = url.url; + us->addrs = url.addrs; + us->naddrs = url.naddrs; + + ngx_http_lua_balancer_default_server_sockaddr = us->addrs[0].sockaddr; + } + if (uscf->peer.init_upstream) { ngx_conf_log_error(NGX_LOG_WARN, cf, 0, "load balancing method redefined"); + + lscf->balancer.original_init_upstream = uscf->peer.init_upstream; + + } else { + lscf->balancer.original_init_upstream = + ngx_http_upstream_init_round_robin; } uscf->peer.init_upstream = ngx_http_lua_balancer_init; @@ -208,16 +280,58 @@ ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, static ngx_int_t -ngx_http_lua_balancer_init(ngx_conf_t *cf, - ngx_http_upstream_srv_conf_t *us) +ngx_http_lua_balancer_init(ngx_conf_t *cf, ngx_http_upstream_srv_conf_t *us) { - if (ngx_http_upstream_init_round_robin(cf, us) != NGX_OK) { + ngx_uint_t i; + ngx_uint_t bucket_cnt; + ngx_queue_t *buckets; + ngx_http_lua_srv_conf_t *lscf; + ngx_http_lua_balancer_ka_item_t *cached; + + lscf = ngx_http_conf_upstream_srv_conf(us, ngx_http_lua_module); + + ngx_conf_init_uint_value(lscf->balancer.max_cached, 32); + + if (lscf->balancer.original_init_upstream(cf, us) != NGX_OK) { return NGX_ERROR; } - /* this callback is called upon individual requests */ + lscf->balancer.original_init_peer = us->peer.init; + us->peer.init = ngx_http_lua_balancer_init_peer; + /* allocate cache items and add to free queue */ + + cached = ngx_pcalloc(cf->pool, + sizeof(ngx_http_lua_balancer_ka_item_t) + * lscf->balancer.max_cached); + if (cached == NULL) { + return NGX_ERROR; + } + + ngx_queue_init(&lscf->balancer.cache); + ngx_queue_init(&lscf->balancer.free); + + for (i = 0; i < lscf->balancer.max_cached; i++) { + ngx_queue_insert_head(&lscf->balancer.free, &cached[i].queue); + cached[i].lscf = lscf; + } + + bucket_cnt = lscf->balancer.max_cached / 2; + bucket_cnt = bucket_cnt > 0 ? bucket_cnt : 1; + buckets = ngx_pcalloc(cf->pool, sizeof(ngx_queue_t) * bucket_cnt); + + if (buckets == NULL) { + return NGX_ERROR; + } + + for (i = 0; i < bucket_cnt; i++) { + ngx_queue_init(&buckets[i]); + } + + lscf->balancer.buckets = buckets; + lscf->balancer.bucket_cnt = bucket_cnt; + return NGX_OK; } @@ -226,33 +340,38 @@ static ngx_int_t ngx_http_lua_balancer_init_peer(ngx_http_request_t *r, ngx_http_upstream_srv_conf_t *us) { - ngx_http_lua_srv_conf_t *bcf; + ngx_http_lua_srv_conf_t *lscf; ngx_http_lua_balancer_peer_data_t *bp; - bp = ngx_pcalloc(r->pool, sizeof(ngx_http_lua_balancer_peer_data_t)); - if (bp == NULL) { + lscf = ngx_http_conf_upstream_srv_conf(us, ngx_http_lua_module); + + if (lscf->balancer.original_init_peer(r, us) != NGX_OK) { return NGX_ERROR; } - r->upstream->peer.data = &bp->rrp; - - if (ngx_http_upstream_init_round_robin_peer(r, us) != NGX_OK) { + bp = ngx_pcalloc(r->pool, sizeof(ngx_http_lua_balancer_peer_data_t)); + if (bp == NULL) { return NGX_ERROR; } + bp->conf = lscf; + bp->request = r; + bp->data = r->upstream->peer.data; + bp->original_get_peer = r->upstream->peer.get; + bp->original_free_peer = r->upstream->peer.free; + + r->upstream->peer.data = bp; r->upstream->peer.get = ngx_http_lua_balancer_get_peer; r->upstream->peer.free = ngx_http_lua_balancer_free_peer; #if (NGX_HTTP_SSL) + bp->original_set_session = r->upstream->peer.set_session; + bp->original_save_session = r->upstream->peer.save_session; + r->upstream->peer.set_session = ngx_http_lua_balancer_set_session; r->upstream->peer.save_session = ngx_http_lua_balancer_save_session; #endif - bcf = ngx_http_conf_upstream_srv_conf(us, ngx_http_lua_module); - - bp->conf = bcf; - bp->request = r; - return NGX_OK; } @@ -260,25 +379,26 @@ ngx_http_lua_balancer_init_peer(ngx_http_request_t *r, static ngx_int_t ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) { + void *pdata; lua_State *L; ngx_int_t rc; + ngx_connection_t *c; ngx_http_request_t *r; + ngx_http_upstream_t *u; ngx_http_lua_ctx_t *ctx; ngx_http_lua_srv_conf_t *lscf; - ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_balancer_peer_data_t *bp = data; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "lua balancer peer, tries: %ui", pc->tries); - - lscf = bp->conf; + "lua balancer: get peer, tries: %ui", pc->tries); r = bp->request; + u = r->upstream; + lscf = bp->conf; ngx_http_lua_assert(lscf->balancer.handler && r); ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - if (ctx == NULL) { ctx = ngx_http_lua_create_ctx(r); if (ctx == NULL) { @@ -299,18 +419,18 @@ ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) bp->sockaddr = NULL; bp->socklen = 0; bp->more_tries = 0; + bp->keepalive_requests = 0; + bp->keepalive_timeout = 0; + bp->keepalive = 0; bp->total_tries++; - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - /* balancer_by_lua does not support yielding and - * there cannot be any conflicts among concurrent requests, - * thus it is safe to store the peer data in the main conf. - */ - lmcf->balancer_peer_data = bp; + pdata = r->upstream->peer.data; + r->upstream->peer.data = bp; rc = lscf->balancer.handler(r, lscf, L); + r->upstream->peer.data = pdata; + if (rc == NGX_ERROR) { return NGX_ERROR; } @@ -335,22 +455,53 @@ ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) if (bp->sockaddr && bp->socklen) { pc->sockaddr = bp->sockaddr; pc->socklen = bp->socklen; + pc->name = bp->addr_text; pc->cached = 0; pc->connection = NULL; - pc->name = bp->host; - - bp->rrp.peers->single = 0; if (bp->more_tries) { r->upstream->peer.tries += bp->more_tries; } - dd("tries: %d", (int) r->upstream->peer.tries); + if (bp->keepalive) { +#if (NGX_HTTP_SSL) + if (bp->host.len == 0 && u->ssl) { + ngx_http_lua_upstream_get_ssl_name(r, u); + bp->host = u->ssl_name; + } +#endif + + c = ngx_http_lua_balancer_get_cached_item(lscf, pc, &bp->host); + + if (c) { + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "lua balancer: keepalive reusing connection %p," + " host: %V, name: %V", + c, bp->addr_text, &bp->host); + return NGX_DONE; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "lua balancer: keepalive no free connection, " + "host: %V, name: %v", bp->addr_text, &bp->host); + } return NGX_OK; } - return ngx_http_upstream_get_round_robin_peer(pc, &bp->rrp); + rc = bp->original_get_peer(pc, bp->data); + if (rc == NGX_ERROR) { + return rc; + } + + if (pc->sockaddr == ngx_http_lua_balancer_default_server_sockaddr) { + ngx_log_error(NGX_LOG_ERR, pc->log, 0, + "lua balancer: no peer set"); + + return NGX_ERROR; + } + + return rc; } @@ -413,10 +564,20 @@ static void ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, ngx_uint_t state) { - ngx_http_lua_balancer_peer_data_t *bp = data; + ngx_uint_t hash; + ngx_str_t *host; + ngx_queue_t *q; + ngx_connection_t *c; + ngx_http_upstream_t *u; + ngx_http_lua_balancer_ka_item_t *item; + ngx_http_lua_balancer_peer_data_t *bp = data; + ngx_http_lua_srv_conf_t *lscf = bp->conf; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "lua balancer free peer, tries: %ui", pc->tries); + "lua balancer: free peer, tries: %ui", pc->tries); + + u = bp->request->upstream; + c = pc->connection; if (bp->sockaddr && bp->socklen) { bp->last_peer_state = (int) state; @@ -425,12 +586,206 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, pc->tries--; } + if (bp->keepalive) { + if (state & NGX_PEER_FAILED + || c == NULL + || c->read->eof + || c->read->error + || c->read->timedout + || c->write->error + || c->write->timedout) + { + goto invalid; + } + + if (bp->keepalive_requests + && c->requests >= bp->keepalive_requests) + { + goto invalid; + } + + if (!u->keepalive) { + goto invalid; + } + + if (!u->request_body_sent) { + goto invalid; + } + + if (ngx_terminate || ngx_exiting) { + goto invalid; + } + + if (ngx_handle_read_event(c->read, 0) != NGX_OK) { + goto invalid; + } + + if (ngx_queue_empty(&lscf->balancer.free)) { + q = ngx_queue_last(&lscf->balancer.cache); + + item = ngx_queue_data(q, ngx_http_lua_balancer_ka_item_t, + queue); + ngx_queue_remove(q); + ngx_queue_remove(&item->hnode); + + ngx_http_lua_balancer_close(item->connection); + + } else { + q = ngx_queue_head(&lscf->balancer.free); + ngx_queue_remove(q); + + item = ngx_queue_data(q, ngx_http_lua_balancer_ka_item_t, + queue); + } + + host = &bp->host; + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "lua balancer: keepalive saving connection %p, " + "host: %V, name: %V", + c, bp->addr_text, host); + + ngx_queue_insert_head(&lscf->balancer.cache, q); + hash = ngx_http_lua_balancer_calc_hash(host, + bp->sockaddr, bp->socklen); + item->hash = hash; + hash %= lscf->balancer.bucket_cnt; + ngx_queue_insert_head(&lscf->balancer.buckets[hash], &item->hnode); + item->connection = c; + pc->connection = NULL; + + c->read->delayed = 0; + ngx_add_timer(c->read, bp->keepalive_timeout); + + if (c->write->timer_set) { + ngx_del_timer(c->write); + } + + c->write->handler = ngx_http_lua_balancer_dummy_handler; + c->read->handler = ngx_http_lua_balancer_close_handler; + + c->data = item; + c->idle = 1; + c->log = ngx_cycle->log; + c->read->log = ngx_cycle->log; + c->write->log = ngx_cycle->log; + c->pool->log = ngx_cycle->log; + + item->socklen = pc->socklen; + ngx_memcpy(&item->sockaddr, pc->sockaddr, pc->socklen); + if (host->data && host->len) { + if (host->len <= sizeof(item->host_data)) { + ngx_memcpy(item->host_data, host->data, host->len); + item->host.data = item->host_data; + item->host.len = host->len; + + } else { + item->host.data = ngx_pstrdup(c->pool, bp->addr_text); + if (item->host.data == NULL) { + ngx_http_lua_balancer_close(c); + + ngx_queue_remove(&item->queue); + ngx_queue_remove(&item->hnode); + ngx_queue_insert_head(&item->lscf->balancer.free, + &item->queue); + return; + } + + item->host.len = bp->addr_text->len; + } + + } else { + ngx_str_null(&item->host); + } + + if (c->read->ready) { + ngx_http_lua_balancer_close_handler(c->read); + } + + return; + +invalid: + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "lua balancer: keepalive not saving connection %p", + c); + } + return; } - /* fallback */ + bp->original_free_peer(pc, bp->data, state); +} + + +static void +ngx_http_lua_balancer_close(ngx_connection_t *c) +{ +#if (NGX_HTTP_SSL) + if (c->ssl) { + c->ssl->no_wait_shutdown = 1; + c->ssl->no_send_shutdown = 1; + + if (ngx_ssl_shutdown(c) == NGX_AGAIN) { + c->ssl->handler = ngx_http_lua_balancer_close; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua balancer: keepalive shutdown " + "connection %p failed", c); + return; + } + } +#endif + + ngx_destroy_pool(c->pool); + ngx_close_connection(c); - ngx_http_upstream_free_round_robin_peer(pc, data, state); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua balancer: keepalive closing connection %p", c); +} + + +static void +ngx_http_lua_balancer_dummy_handler(ngx_event_t *ev) +{ + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ev->log, 0, + "lua balancer: dummy handler"); +} + + +static void +ngx_http_lua_balancer_close_handler(ngx_event_t *ev) +{ + ngx_http_lua_balancer_ka_item_t *item; + + int n; + char buf[1]; + ngx_connection_t *c; + + c = ev->data; + if (c->close || c->read->timedout) { + goto close; + } + + n = recv(c->fd, buf, 1, MSG_PEEK); + + if (n == -1 && ngx_socket_errno == NGX_EAGAIN) { + ev->ready = 0; + + if (ngx_handle_read_event(c->read, 0) != NGX_OK) { + goto close; + } + + return; + } + +close: + + item = c->data; + c->log = ev->log; + + ngx_http_lua_balancer_close(c); + + ngx_queue_remove(&item->queue); + ngx_queue_remove(&item->hnode); + ngx_queue_insert_head(&item->lscf->balancer.free, &item->queue); } @@ -446,7 +801,7 @@ ngx_http_lua_balancer_set_session(ngx_peer_connection_t *pc, void *data) return NGX_OK; } - return ngx_http_upstream_set_round_robin_peer_session(pc, &bp->rrp); + return bp->original_set_session(pc, bp->data); } @@ -460,8 +815,7 @@ ngx_http_lua_balancer_save_session(ngx_peer_connection_t *pc, void *data) return; } - ngx_http_upstream_save_round_robin_peer_session(pc, &bp->rrp); - return; + bp->original_save_session(pc, bp->data); } #endif @@ -469,13 +823,14 @@ ngx_http_lua_balancer_save_session(ngx_peer_connection_t *pc, void *data) int ngx_http_lua_ffi_balancer_set_current_peer(ngx_http_request_t *r, - const u_char *addr, size_t addr_len, int port, char **err) + const u_char *addr, size_t addr_len, int port, + const u_char *host, size_t host_len, + char **err) { ngx_url_t url; ngx_http_lua_ctx_t *ctx; ngx_http_upstream_t *u; - ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_balancer_peer_data_t *bp; if (r == NULL) { @@ -501,18 +856,6 @@ ngx_http_lua_ffi_balancer_set_current_peer(ngx_http_request_t *r, return NGX_ERROR; } - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - /* we cannot read r->upstream->peer.data here directly because - * it could be overridden by other modules like - * ngx_http_upstream_keepalive_module. - */ - bp = lmcf->balancer_peer_data; - if (bp == NULL) { - *err = "no upstream peer data found"; - return NGX_ERROR; - } - ngx_memzero(&url, sizeof(ngx_url_t)); url.url.data = ngx_palloc(r->pool, addr_len); @@ -536,16 +879,84 @@ ngx_http_lua_ffi_balancer_set_current_peer(ngx_http_request_t *r, return NGX_ERROR; } + bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; + if (url.addrs && url.addrs[0].sockaddr) { bp->sockaddr = url.addrs[0].sockaddr; bp->socklen = url.addrs[0].socklen; - bp->host = &url.addrs[0].name; + bp->addr_text = &url.addrs[0].name; } else { *err = "no host allowed"; return NGX_ERROR; } + if (host && host_len) { + bp->host.data = ngx_palloc(r->pool, host_len); + if (bp->host.data == NULL) { + *err = "no memory"; + return NGX_ERROR; + } + + ngx_memcpy(bp->host.data, host, host_len); + bp->host.len = host_len; + +#if (NGX_HTTP_SSL) + if (u->ssl) { + u->ssl_name = bp->host; + } +#endif + + } else { + ngx_str_null(&bp->host); + } + + return NGX_OK; +} + + +int +ngx_http_lua_ffi_balancer_enable_keepalive(ngx_http_request_t *r, + unsigned long timeout, unsigned int max_requests, char **err) +{ + ngx_http_upstream_t *u; + ngx_http_lua_ctx_t *ctx; + ngx_http_lua_balancer_peer_data_t *bp; + + if (r == NULL) { + *err = "no request found"; + return NGX_ERROR; + } + + u = r->upstream; + + if (u == NULL) { + *err = "no upstream found"; + return NGX_ERROR; + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + *err = "no ctx found"; + return NGX_ERROR; + } + + if ((ctx->context & NGX_HTTP_LUA_CONTEXT_BALANCER) == 0) { + *err = "API disabled in the current context"; + return NGX_ERROR; + } + + bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; + + if (!(bp->sockaddr && bp->socklen)) { + *err = "no current peer set"; + return NGX_ERROR; + } + + bp->keepalive_timeout = (ngx_msec_t) timeout; + bp->keepalive_requests = (ngx_uint_t) max_requests; + bp->keepalive = 1; + return NGX_OK; } @@ -555,14 +966,13 @@ ngx_http_lua_ffi_balancer_set_timeouts(ngx_http_request_t *r, long connect_timeout, long send_timeout, long read_timeout, char **err) { - ngx_http_lua_ctx_t *ctx; - ngx_http_upstream_t *u; + ngx_http_lua_ctx_t *ctx; + ngx_http_upstream_t *u; #if !(HAVE_NGX_UPSTREAM_TIMEOUT_FIELDS) ngx_http_upstream_conf_t *ucf; -#endif - ngx_http_lua_main_conf_t *lmcf; ngx_http_lua_balancer_peer_data_t *bp; +#endif if (r == NULL) { *err = "no request found"; @@ -587,15 +997,9 @@ ngx_http_lua_ffi_balancer_set_timeouts(ngx_http_request_t *r, return NGX_ERROR; } - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - bp = lmcf->balancer_peer_data; - if (bp == NULL) { - *err = "no upstream peer data found"; - return NGX_ERROR; - } - #if !(HAVE_NGX_UPSTREAM_TIMEOUT_FIELDS) + bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; + if (!bp->cloned_upstream_conf) { /* we clone the upstream conf for the current request so that * we do not affect other requests at all. */ @@ -650,12 +1054,10 @@ ngx_http_lua_ffi_balancer_set_more_tries(ngx_http_request_t *r, int count, char **err) { #if (nginx_version >= 1007005) - ngx_uint_t max_tries, total; + ngx_uint_t max_tries, total; #endif - ngx_http_lua_ctx_t *ctx; - ngx_http_upstream_t *u; - - ngx_http_lua_main_conf_t *lmcf; + ngx_http_lua_ctx_t *ctx; + ngx_http_upstream_t *u; ngx_http_lua_balancer_peer_data_t *bp; if (r == NULL) { @@ -681,13 +1083,7 @@ ngx_http_lua_ffi_balancer_set_more_tries(ngx_http_request_t *r, return NGX_ERROR; } - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - bp = lmcf->balancer_peer_data; - if (bp == NULL) { - *err = "no upstream peer data found"; - return NGX_ERROR; - } + bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; #if (nginx_version >= 1007005) max_tries = r->upstream->conf->next_upstream_tries; @@ -713,12 +1109,10 @@ int ngx_http_lua_ffi_balancer_get_last_failure(ngx_http_request_t *r, int *status, char **err) { - ngx_http_lua_ctx_t *ctx; - ngx_http_upstream_t *u; - ngx_http_upstream_state_t *state; - + ngx_http_lua_ctx_t *ctx; + ngx_http_upstream_t *u; + ngx_http_upstream_state_t *state; ngx_http_lua_balancer_peer_data_t *bp; - ngx_http_lua_main_conf_t *lmcf; if (r == NULL) { *err = "no request found"; @@ -743,13 +1137,7 @@ ngx_http_lua_ffi_balancer_get_last_failure(ngx_http_request_t *r, return NGX_ERROR; } - lmcf = ngx_http_get_module_main_conf(r, ngx_http_lua_module); - - bp = lmcf->balancer_peer_data; - if (bp == NULL) { - *err = "no upstream peer data found"; - return NGX_ERROR; - } + bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; if (r->upstream_states && r->upstream_states->nelts > 1) { state = r->upstream_states->elts; @@ -852,4 +1240,162 @@ ngx_http_lua_ffi_balancer_set_upstream_tls(ngx_http_request_t *r, int on, } +char * +ngx_http_lua_balancer_keepalive(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_int_t n; + ngx_str_t *value; + +#if 0 + ngx_http_upstream_srv_conf_t *uscf; +#endif + ngx_http_lua_srv_conf_t *lscf = conf; + + if (lscf->balancer.max_cached != NGX_CONF_UNSET_UINT) { + return "is duplicate"; + } + + /* read options */ + + value = cf->args->elts; + + n = ngx_atoi(value[1].data, value[1].len); + + if (n == NGX_ERROR || n == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid value \"%V\" in \"%V\" directive", + &value[1], &cmd->name); + return NGX_CONF_ERROR; + } + + lscf->balancer.max_cached = n; + + return NGX_CONF_OK; +} + + +#if (NGX_HTTP_SSL) +static ngx_int_t +ngx_http_lua_upstream_get_ssl_name(ngx_http_request_t *r, + ngx_http_upstream_t *u) +{ + u_char *p, *last; + ngx_str_t name; + + if (u->conf->ssl_name) { + if (ngx_http_complex_value(r, u->conf->ssl_name, &name) != NGX_OK) { + return NGX_ERROR; + } + + } else { + name = u->ssl_name; + } + + if (name.len == 0) { + goto done; + } + + /* + * ssl name here may contain port, notably if derived from $proxy_host + * or $http_host; we have to strip it. eg: www.example.com:443 + */ + + p = name.data; + last = name.data + name.len; + + if (*p == '[') { + p = ngx_strlchr(p, last, ']'); + + if (p == NULL) { + p = name.data; + } + } + + p = ngx_strlchr(p, last, ':'); + + if (p != NULL) { + name.len = p - name.data; + } + +done: + + u->ssl_name = name; + + return NGX_OK; +} +#endif + + +static ngx_uint_t +ngx_http_lua_balancer_calc_hash(ngx_str_t *name, + struct sockaddr *sockaddr, socklen_t socklen) +{ + ngx_uint_t hash; + + hash = ngx_hash_key_lc(name->data, name->len); + hash ^= ngx_hash_key((u_char *) sockaddr, socklen); + + return hash; +} + + +static ngx_connection_t * +ngx_http_lua_balancer_get_cached_item(ngx_http_lua_srv_conf_t *lscf, + ngx_peer_connection_t *pc, ngx_str_t *name) +{ + ngx_uint_t hash; + ngx_queue_t *q; + ngx_queue_t *head; + ngx_connection_t *c; + struct sockaddr *sockaddr; + socklen_t socklen; + ngx_http_lua_balancer_ka_item_t *item; + + sockaddr = pc->sockaddr; + socklen = pc->socklen; + + hash = ngx_http_lua_balancer_calc_hash(name, sockaddr, socklen); + head = &lscf->balancer.buckets[hash % lscf->balancer.bucket_cnt]; + + c = NULL; + for (q = ngx_queue_head(head); + q != ngx_queue_sentinel(head); + q = ngx_queue_next(q)) + { + item = ngx_queue_data(q, ngx_http_lua_balancer_ka_item_t, hnode); + if (item->hash != hash) { + continue; + } + + if (name->len == item->host.len + && ngx_memn2cmp((u_char *) &item->sockaddr, + (u_char *) sockaddr, + item->socklen, socklen) == 0 + && ngx_strncasecmp(name->data, + item->host.data, name->len) == 0) + { + c = item->connection; + ngx_queue_remove(q); + ngx_queue_remove(&item->queue); + ngx_queue_insert_head(&lscf->balancer.free, &item->queue); + c->idle = 0; + c->sent = 0; + c->log = pc->log; + c->read->log = pc->log; + c->write->log = pc->log; + c->pool->log = pc->log; + + if (c->read->timer_set) { + ngx_del_timer(c->read); + } + + pc->cached = 1; + pc->connection = c; + return c; + } + } + + return NULL; +} + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_balancer.h b/src/ngx_http_lua_balancer.h index bb49dc02ca..34db2fe82f 100644 --- a/src/ngx_http_lua_balancer.h +++ b/src/ngx_http_lua_balancer.h @@ -23,5 +23,7 @@ char *ngx_http_lua_balancer_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, char *ngx_http_lua_balancer_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +char *ngx_http_lua_balancer_keepalive(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); #endif /* _NGX_HTTP_LUA_BALANCER_H_INCLUDED_ */ diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 4c946297f6..cc2d36a386 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -167,7 +167,7 @@ typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; typedef struct ngx_http_lua_sema_mm_s ngx_http_lua_sema_mm_t; -typedef union ngx_http_lua_srv_conf_u ngx_http_lua_srv_conf_t; +typedef struct ngx_http_lua_srv_conf_s ngx_http_lua_srv_conf_t; typedef struct ngx_http_lua_main_conf_s ngx_http_lua_main_conf_t; @@ -258,13 +258,6 @@ struct ngx_http_lua_main_conf_s { ngx_str_t exit_worker_src; u_char *exit_worker_chunkname; - ngx_http_lua_balancer_peer_data_t *balancer_peer_data; - /* neither yielding nor recursion is possible in - * balancer_by_lua*, so there cannot be any races among - * concurrent requests and it is safe to store the peer - * data pointer in the main conf. - */ - ngx_chain_t *body_filter_chain; /* neither yielding nor recursion is possible in * body_filter_by_lua*, so there cannot be any races among @@ -323,7 +316,7 @@ struct ngx_http_lua_main_conf_s { }; -union ngx_http_lua_srv_conf_u { +struct ngx_http_lua_srv_conf_s { struct { #if (NGX_HTTP_SSL) ngx_http_lua_srv_conf_handler_pt ssl_cert_handler; @@ -359,6 +352,14 @@ union ngx_http_lua_srv_conf_u { } srv; struct { + ngx_uint_t max_cached; + ngx_queue_t cache; + ngx_queue_t free; + ngx_queue_t *buckets; + ngx_uint_t bucket_cnt; + ngx_http_upstream_init_pt original_init_upstream; + ngx_http_upstream_init_peer_pt original_init_peer; + ngx_http_lua_srv_conf_handler_pt handler; ngx_str_t src; u_char *src_key; diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index f64ab89315..14ed235838 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -33,7 +33,9 @@ #include "ngx_http_lua_ssl_session_fetchby.h" #include "ngx_http_lua_headers.h" #include "ngx_http_lua_headers_out.h" +#if !(NGX_WIN32) #include "ngx_http_lua_pipe.h" +#endif static void *ngx_http_lua_create_main_conf(ngx_conf_t *cf); @@ -494,6 +496,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, (void *) ngx_http_lua_balancer_handler_file }, + { ngx_string("balancer_keepalive"), + NGX_HTTP_UPS_CONF|NGX_CONF_TAKE1, + ngx_http_lua_balancer_keepalive, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_lua_srv_conf_t, balancer.max_cached), + NULL }, + { ngx_string("lua_socket_keepalive_timeout"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF |NGX_HTTP_LIF_CONF|NGX_CONF_TAKE1, @@ -1188,6 +1197,8 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) * lscf->srv.ssl_sess_fetch_chunkname = NULL; * lscf->srv.ssl_sess_fetch_src_key = NULL; * + * lscf->balancer.original_init_upstream = NULL; + * lscf->balancer.original_init_peer = NULL; * lscf->balancer.handler = NULL; * lscf->balancer.src = { 0, NULL }; * lscf->balancer.chunkname = NULL; @@ -1202,7 +1213,7 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) #endif lscf->balancer.src_ref = LUA_REFNIL; - + lscf->balancer.max_cached = NGX_CONF_UNSET_UINT; return lscf; } From 6738c3a3b0d62b98ff35acf7f7c4a191207da918 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 23 Nov 2022 14:20:26 +0800 Subject: [PATCH 759/848] change: should match the local address when get connection from the keepalive pool. --- src/ngx_http_lua_balancer.c | 98 +++++++++++++++++++++++++++++++++++-- 1 file changed, 93 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 9bb770c8e6..79f0e7a9c7 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -24,6 +24,7 @@ typedef struct { ngx_connection_t *connection; socklen_t socklen; ngx_sockaddr_t sockaddr; + ngx_sockaddr_t local_sockaddr; ngx_str_t host; /* try to avoid allocating memory from the connection pool */ u_char host_data[NGX_BALANCER_DEF_HOST_LEN]; @@ -52,6 +53,7 @@ struct ngx_http_lua_balancer_peer_data_s { struct sockaddr *sockaddr; socklen_t socklen; + ngx_addr_t *local; ngx_str_t host; ngx_str_t *addr_text; @@ -91,7 +93,7 @@ static void ngx_http_lua_balancer_close_handler(ngx_event_t *ev); static ngx_connection_t *ngx_http_lua_balancer_get_cached_item( ngx_http_lua_srv_conf_t *lscf, ngx_peer_connection_t *pc, ngx_str_t *name); static ngx_uint_t ngx_http_lua_balancer_calc_hash(ngx_str_t *name, - struct sockaddr *sockaddr, socklen_t socklen); + struct sockaddr *sockaddr, socklen_t socklen, ngx_addr_t *local); static struct sockaddr *ngx_http_lua_balancer_default_server_sockaddr; @@ -452,6 +454,10 @@ ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) } } + if (bp->local != NULL) { + pc->local = bp->local; + } + if (bp->sockaddr && bp->socklen) { pc->sockaddr = bp->sockaddr; pc->socklen = bp->socklen; @@ -646,7 +652,8 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, ngx_queue_insert_head(&lscf->balancer.cache, q); hash = ngx_http_lua_balancer_calc_hash(host, - bp->sockaddr, bp->socklen); + bp->sockaddr, bp->socklen, + bp->local); item->hash = hash; hash %= lscf->balancer.bucket_cnt; ngx_queue_insert_head(&lscf->balancer.buckets[hash], &item->hnode); @@ -672,6 +679,15 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, item->socklen = pc->socklen; ngx_memcpy(&item->sockaddr, pc->sockaddr, pc->socklen); + if (pc->local) { + ngx_memcpy(&item->local_sockaddr, + pc->local->sockaddr, pc->local->socklen); + + } else { + ngx_memzero(&item->local_sockaddr, + sizeof(item->local_sockaddr)); + } + if (host->data && host->len) { if (host->len <= sizeof(item->host_data)) { ngx_memcpy(item->host_data, host->data, host->len); @@ -704,6 +720,7 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, return; invalid: + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, "lua balancer: keepalive not saving connection %p", c); @@ -915,6 +932,68 @@ ngx_http_lua_ffi_balancer_set_current_peer(ngx_http_request_t *r, } +int +ngx_http_lua_ffi_balancer_bind_to_local_addr(ngx_http_request_t *r, + const u_char *addr, size_t addr_len, + u_char *errbuf, size_t *errbuf_size) +{ + u_char *p; + ngx_http_lua_ctx_t *ctx; + ngx_http_upstream_t *u; + ngx_int_t rc; + + ngx_http_lua_balancer_peer_data_t *bp; + + if (r == NULL) { + p = ngx_snprintf(errbuf, *errbuf_size, "no request found"); + *errbuf_size = p - errbuf; + return NGX_ERROR; + } + + u = r->upstream; + + if (u == NULL) { + p = ngx_snprintf(errbuf, *errbuf_size, "no upstream found"); + *errbuf_size = p - errbuf; + return NGX_ERROR; + } + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + if (ctx == NULL) { + p = ngx_snprintf(errbuf, *errbuf_size, "no ctx found"); + *errbuf_size = p - errbuf; + return NGX_ERROR; + } + + if ((ctx->context & NGX_HTTP_LUA_CONTEXT_BALANCER) == 0) { + p = ngx_snprintf(errbuf, *errbuf_size, + "API disabled in the current context"); + *errbuf_size = p - errbuf; + return NGX_ERROR; + } + + bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; + + if (bp->local == NULL) { + bp->local = ngx_palloc(r->pool, sizeof(ngx_addr_t)); + if (bp->local == NULL) { + p = ngx_snprintf(errbuf, *errbuf_size, "no memory"); + *errbuf_size = p - errbuf; + return NGX_ERROR; + } + } + + rc = ngx_parse_addr_port(r->pool, bp->local, (u_char *) addr, addr_len); + if (rc == NGX_ERROR) { + p = ngx_snprintf(errbuf, *errbuf_size, "invalid addr %s", addr); + *errbuf_size = p - errbuf; + return NGX_ERROR; + } + + return NGX_OK; +} + + int ngx_http_lua_ffi_balancer_enable_keepalive(ngx_http_request_t *r, unsigned long timeout, unsigned int max_requests, char **err) @@ -1328,12 +1407,15 @@ ngx_http_lua_upstream_get_ssl_name(ngx_http_request_t *r, static ngx_uint_t ngx_http_lua_balancer_calc_hash(ngx_str_t *name, - struct sockaddr *sockaddr, socklen_t socklen) + struct sockaddr *sockaddr, socklen_t socklen, ngx_addr_t *local) { ngx_uint_t hash; hash = ngx_hash_key_lc(name->data, name->len); hash ^= ngx_hash_key((u_char *) sockaddr, socklen); + if (local != NULL) { + hash ^= ngx_hash_key((u_char *) local->sockaddr, local->socklen); + } return hash; } @@ -1349,12 +1431,14 @@ ngx_http_lua_balancer_get_cached_item(ngx_http_lua_srv_conf_t *lscf, ngx_connection_t *c; struct sockaddr *sockaddr; socklen_t socklen; + ngx_addr_t *local; ngx_http_lua_balancer_ka_item_t *item; sockaddr = pc->sockaddr; socklen = pc->socklen; + local = pc->local; - hash = ngx_http_lua_balancer_calc_hash(name, sockaddr, socklen); + hash = ngx_http_lua_balancer_calc_hash(name, sockaddr, socklen, pc->local); head = &lscf->balancer.buckets[hash % lscf->balancer.bucket_cnt]; c = NULL; @@ -1372,7 +1456,11 @@ ngx_http_lua_balancer_get_cached_item(ngx_http_lua_srv_conf_t *lscf, (u_char *) sockaddr, item->socklen, socklen) == 0 && ngx_strncasecmp(name->data, - item->host.data, name->len) == 0) + item->host.data, name->len) == 0 + && (local == NULL + || ngx_memn2cmp((u_char *) &item->local_sockaddr, + (u_char *) local->sockaddr, + socklen, local->socklen) == 0)) { c = item->connection; ngx_queue_remove(q); From 6477a7b46d78e9ad841f2f3b2e1d37161b748669 Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 3 Jul 2024 22:40:36 +0800 Subject: [PATCH 760/848] bugfix: fixed compilation errors when building without SSL. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit lua-nginx-module/src/ngx_http_lua_balancer.c: In function ‘ngx_http_lua_balancer_get_peer’: lua-nginx-module/src/ngx_http_lua_balancer.c:389:41: error: variable ‘u’ set but not used [-Werror=unused-but-set-variable] 389 | ngx_http_upstream_t *u; | --- src/ngx_http_lua_balancer.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 79f0e7a9c7..3469c36ad1 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -386,7 +386,9 @@ ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) ngx_int_t rc; ngx_connection_t *c; ngx_http_request_t *r; +#if (NGX_HTTP_SSL) ngx_http_upstream_t *u; +#endif ngx_http_lua_ctx_t *ctx; ngx_http_lua_srv_conf_t *lscf; ngx_http_lua_balancer_peer_data_t *bp = data; @@ -395,7 +397,9 @@ ngx_http_lua_balancer_get_peer(ngx_peer_connection_t *pc, void *data) "lua balancer: get peer, tries: %ui", pc->tries); r = bp->request; +#if (NGX_HTTP_SSL) u = r->upstream; +#endif lscf = bp->conf; ngx_http_lua_assert(lscf->balancer.handler && r); From 39d165ca411887604aca03869e00051fc718896d Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 7 Jul 2024 11:39:10 +0800 Subject: [PATCH 761/848] bugfix: undefined symbol SSL_client_hello_get0_ext when linking against libressl. --- src/ngx_http_lua_ssl_client_helloby.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 03ac430ef2..b808f98ed7 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -541,6 +541,10 @@ int ngx_http_lua_ffi_ssl_get_client_hello_server_name(ngx_http_request_t *r, const char **name, size_t *namelen, char **err) { +#ifdef LIBRESSL_VERSION_NUMBER + *err = "LibreSSL does not support by ssl_client_hello_by_lua*"; + return NGX_ERROR; +#else ngx_ssl_conn_t *ssl_conn; #ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB const unsigned char *p; @@ -619,6 +623,7 @@ ngx_http_lua_ffi_ssl_get_client_hello_server_name(ngx_http_request_t *r, *err = "no TLS extension support"; return NGX_ERROR; #endif +#endif /* LIBRESSL_VERSION_NUMBER */ } @@ -626,6 +631,10 @@ int ngx_http_lua_ffi_ssl_get_client_hello_ext(ngx_http_request_t *r, unsigned int type, const unsigned char **out, size_t *outlen, char **err) { +#ifdef LIBRESSL_VERSION_NUMBER + *err = "LibreSSL does not support by ssl_client_hello_by_lua*"; + return NGX_ERROR; +#else ngx_ssl_conn_t *ssl_conn; if (r->connection == NULL || r->connection->ssl == NULL) { @@ -649,7 +658,7 @@ ngx_http_lua_ffi_ssl_get_client_hello_ext(ngx_http_request_t *r, *err = "OpenSSL too old to support this function"; return NGX_ERROR; #endif - +#endif /* LIBRESSL_VERSION_NUMBER */ } From 0a1c704c2eb30aa3f76f2fdd7a584514050be814 Mon Sep 17 00:00:00 2001 From: Tinglong Yang Date: Sun, 7 Jul 2024 12:22:30 +0800 Subject: [PATCH 762/848] feature: support ngx.location.capture and ngx.location.capture_multi with `headers` option. --- README.markdown | 30 ++++++++++++++- src/ngx_http_lua_subrequest.c | 66 +++++++++++++++++++++++++++----- t/027-multi-capture.t | 72 +++++++++++++++++++++++++++++++++++ 3 files changed, 158 insertions(+), 10 deletions(-) diff --git a/README.markdown b/README.markdown index 4957e50059..af9555f80b 100644 --- a/README.markdown +++ b/README.markdown @@ -966,7 +966,6 @@ TODO * cosocket: add support in the context of [init_by_lua*](#init_by_lua). * cosocket: review and merge aviramc's [patch](https://github.com/openresty/lua-nginx-module/pull/290) for adding the `bsdrecv` method. * cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools. -* review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option * use `ngx_hash_t` to optimize the built-in header look-up process for [ngx.req.set_header](#ngxreqset_header), and etc. * add `ignore_resp_headers`, `ignore_resp_body`, and `ignore_resp` options to [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) methods, to allow micro performance tuning on the user side. * add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks. @@ -4305,6 +4304,8 @@ argument, which supports the options: specify the subrequest's request body (string value only). * `args` specify the subrequest's URI query arguments (both string value and Lua tables are accepted) +* `headers` + specify the subrequest's request headers (Lua table only). this headers will override the original headers of the subrequest. * `ctx` specify a Lua table to be the [ngx.ctx](#ngxctx) table for the subrequest. It can be the current request's [ngx.ctx](#ngxctx) table, which effectively makes the parent and its subrequest to share exactly the same context table. This option was first introduced in the `v0.3.1rc25` release. * `vars` @@ -4456,6 +4457,33 @@ Accessing `/lua` will yield the output dog = hello cat = 32 +The `headers` option can be used to specify the request headers for the subrequest. The value of this option should be a Lua table where the keys are the header names and the values are the header values. For example, + +```lua + +location /foo { + content_by_lua_block { + ngx.print(ngx.var.http_x_test) + } +} + +location /lua { + content_by_lua_block { + local res = ngx.location.capture("/foo", { + headers = { + ["X-Test"] = "aa", + } + }) + ngx.print(res.body) + } +} +``` + +Accessing `/lua` will yield the output + + + aa + The `ctx` option can be used to specify a custom Lua table to serve as the [ngx.ctx](#ngxctx) table for the subrequest. diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index f4db9aaf6c..2ccd271a90 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -62,10 +62,10 @@ static ngx_str_t ngx_http_lua_content_length_header_key = static ngx_int_t ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, int forward_body, ngx_http_request_body_t *body, unsigned vars_action, - ngx_array_t *extra_vars); + ngx_array_t *extra_vars, ngx_array_t *extra_headers); static int ngx_http_lua_ngx_location_capture(lua_State *L); static int ngx_http_lua_ngx_location_capture_multi(lua_State *L); -static void ngx_http_lua_process_vars_option(ngx_http_request_t *r, +static void ngx_http_lua_process_keyval_option(ngx_http_request_t *r, lua_State *L, int table, ngx_array_t **varsp); static ngx_int_t ngx_http_lua_subrequest_add_extra_vars(ngx_http_request_t *r, ngx_array_t *extra_vars); @@ -79,7 +79,7 @@ static void ngx_http_lua_cancel_subreq(ngx_http_request_t *r); static ngx_int_t ngx_http_post_request_to_head(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_copy_in_file_request_body(ngx_http_request_t *r); static ngx_int_t ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, - ngx_http_request_t *pr, int pr_not_chunked); + ngx_http_request_t *pr, int pr_not_chunked, ngx_array_t *extra_headers); enum { @@ -127,6 +127,7 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) ngx_http_lua_ctx_t *sr_ctx; ngx_http_lua_ctx_t *ctx; ngx_array_t *extra_vars; + ngx_array_t *extra_headers; ngx_str_t uri; ngx_str_t args; ngx_str_t extra_args; @@ -224,6 +225,7 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) coctx->pending_subreqs = 0; extra_vars = NULL; + extra_headers = NULL; for (index = 0; index < nsubreqs; index++) { coctx->pending_subreqs++; @@ -263,6 +265,11 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) extra_vars->nelts = 0; } + if (extra_headers != NULL) { + /* flush out existing elements in the array */ + extra_headers->nelts = 0; + } + vars_action = 0; custom_ctx = 0; @@ -318,7 +325,7 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) switch (lua_type(L, -1)) { case LUA_TTABLE: - ngx_http_lua_process_vars_option(r, L, -1, &extra_vars); + ngx_http_lua_process_keyval_option(r, L, -1, &extra_vars); dd("post process vars top: %d", lua_gettop(L)); break; @@ -335,6 +342,29 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) dd("queries query uri opts: %d", lua_gettop(L)); + /* check the headers option */ + + lua_getfield(L, 4, "headers"); + + switch (lua_type(L, -1)) { + case LUA_TTABLE: + ngx_http_lua_process_keyval_option(r, L, -1, &extra_headers); + + dd("post process vars top: %d", lua_gettop(L)); + break; + + case LUA_TNIL: + /* do nothing */ + break; + + default: + return luaL_error(L, "Bad headers option value"); + } + + lua_pop(L, 1); /* pop the headers */ + + dd("queries query uri opts: %d", lua_gettop(L)); + /* check the share_all_vars option */ lua_getfield(L, 4, "share_all_vars"); @@ -595,7 +625,8 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) ngx_http_set_ctx(sr, sr_ctx, ngx_http_lua_module); rc = ngx_http_lua_adjust_subrequest(sr, method, always_forward_body, - body, vars_action, extra_vars); + body, vars_action, extra_vars, + extra_headers); if (rc != NGX_OK) { ngx_http_lua_cancel_subreq(sr); @@ -631,7 +662,7 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) static ngx_int_t ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, int always_forward_body, ngx_http_request_body_t *body, - unsigned vars_action, ngx_array_t *extra_vars) + unsigned vars_action, ngx_array_t *extra_vars, ngx_array_t *extra_headers) { ngx_http_request_t *r; ngx_http_core_main_conf_t *cmcf; @@ -667,7 +698,9 @@ ngx_http_lua_adjust_subrequest(ngx_http_request_t *sr, ngx_uint_t method, } } - if (ngx_http_lua_copy_request_headers(sr, r, pr_not_chunked) != NGX_OK) { + if (ngx_http_lua_copy_request_headers(sr, r, pr_not_chunked, extra_headers) + != NGX_OK) + { return NGX_ERROR; } @@ -882,7 +915,7 @@ ngx_http_lua_subrequest_add_extra_vars(ngx_http_request_t *sr, static void -ngx_http_lua_process_vars_option(ngx_http_request_t *r, lua_State *L, +ngx_http_lua_process_keyval_option(ngx_http_request_t *r, lua_State *L, int table, ngx_array_t **varsp) { ngx_array_t *vars; @@ -1635,10 +1668,11 @@ ngx_http_lua_copy_in_file_request_body(ngx_http_request_t *r) static ngx_int_t ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, - ngx_http_request_t *pr, int pr_not_chunked) + ngx_http_request_t *pr, int pr_not_chunked, ngx_array_t *extra_headers) { ngx_table_elt_t *clh, *header; ngx_list_part_t *part; + ngx_keyval_t *header_keyval; ngx_chain_t *in; ngx_uint_t i; u_char *p; @@ -1742,6 +1776,20 @@ ngx_http_lua_copy_request_headers(ngx_http_request_t *sr, } } + if (extra_headers && extra_headers->nelts > 0) { + + header_keyval = extra_headers->elts; + + for (i = 0; i < extra_headers->nelts; i++, header_keyval++) { + + if (ngx_http_lua_set_input_header(sr, header_keyval->key, + header_keyval->value, 1) == NGX_ERROR) + { + return NGX_ERROR; + } + } + } + dd("after: parent req headers count: %d", (int) pr->headers_in.headers.part.nelts); diff --git a/t/027-multi-capture.t b/t/027-multi-capture.t index 9299561687..b3cdbaff09 100644 --- a/t/027-multi-capture.t +++ b/t/027-multi-capture.t @@ -752,3 +752,75 @@ proxy_cache_path conf/cache levels=1:2 keys_zone=STATIC:10m inactive=10m max_siz GET /foo --- response_body ok + + + +=== TEST 14: capture multi with headers +--- config + location /foo { + content_by_lua_block { + local res1, res2, res3 = ngx.location.capture_multi{ + {"/test", { headers = { ["X-Test-Header"] = "aa"} } }, + {"/test", { headers = { ["X-Test-Header"] = "bb"} } }, + {"/test"}, + } + ngx.say("res1.status = " .. res1.status) + ngx.say("res1.body = " .. res1.body) + ngx.say("res2.status = " .. res2.status) + ngx.say("res2.body = " .. res2.body) + ngx.say("res3.status = " .. res3.status) + ngx.say("res3.body = " .. res3.body) + } + } + + location = /test { + content_by_lua_block { + ngx.print(ngx.var.http_x_test_header) + } + } +--- request + GET /foo +--- response_body +res1.status = 200 +res1.body = aa +res2.status = 200 +res2.body = bb +res3.status = 200 +res3.body = nil + + + +=== TEST 15: capture multi with headers override +--- config + location /foo { + content_by_lua_block { + local res1, res2, res3 = ngx.location.capture_multi{ + {"/test", { headers = { ["X-Test-Header"] = "aa"} } }, + {"/test", { headers = { ["X-Test-Header"] = "bb"} } }, + {"/test"}, + } + ngx.say("res1.status = " .. res1.status) + ngx.say("res1.body = " .. res1.body) + ngx.say("res2.status = " .. res2.status) + ngx.say("res2.body = " .. res2.body) + ngx.say("res3.status = " .. res3.status) + ngx.say("res3.body = " .. res3.body) + } + } + + location = /test { + content_by_lua_block { + ngx.print(ngx.var.http_x_test_header) + } + } +--- request + GET /foo +--- more_headers +X-Test-Header: cc +--- response_body +res1.status = 200 +res1.body = aa +res2.status = 200 +res2.body = bb +res3.status = 200 +res3.body = cc From 94f55f7a4d493f627d545e890592c49a27c4a3bc Mon Sep 17 00:00:00 2001 From: kurt Date: Tue, 9 Jul 2024 12:10:08 +0800 Subject: [PATCH 763/848] bugfix: respect max retry after using balancer pool. In the balancer phase, when obtaining a connection from the upstream connection pool, the `cached` attribute of the peer connection is set to 1(`pc->cached = 1;`), indicating that the connection is obtained from the cache. If an error occurs during the use of this connection, such as "upstream prematurely closed connection" the system will increase the `tries` attribute of the peer connection by executing `u->peer.tries++`. This PR restores tries by callbacks to the balancer when `u->peer.tries++` is unexpectedly set. Signed-off-by: tzssangglass --- src/ngx_http_lua_balancer.c | 15 ++++ t/188-balancer_keepalive_pool_max_retry.t | 89 +++++++++++++++++++++++ 2 files changed, 104 insertions(+) create mode 100644 t/188-balancer_keepalive_pool_max_retry.t diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 3469c36ad1..620f71eea0 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -87,6 +87,8 @@ static ngx_int_t ngx_http_lua_balancer_by_chunk(lua_State *L, ngx_http_request_t *r); static void ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, ngx_uint_t state); +static void ngx_http_lua_balancer_notify_peer(ngx_peer_connection_t *pc, + void *data, ngx_uint_t type); static void ngx_http_lua_balancer_close(ngx_connection_t *c); static void ngx_http_lua_balancer_dummy_handler(ngx_event_t *ev); static void ngx_http_lua_balancer_close_handler(ngx_event_t *ev); @@ -365,6 +367,7 @@ ngx_http_lua_balancer_init_peer(ngx_http_request_t *r, r->upstream->peer.data = bp; r->upstream->peer.get = ngx_http_lua_balancer_get_peer; r->upstream->peer.free = ngx_http_lua_balancer_free_peer; + r->upstream->peer.notify = ngx_http_lua_balancer_notify_peer; #if (NGX_HTTP_SSL) bp->original_set_session = r->upstream->peer.set_session; @@ -737,6 +740,18 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, } +static void +ngx_http_lua_balancer_notify_peer(ngx_peer_connection_t *pc, void *data, + ngx_uint_t type) +{ +#ifdef NGX_HTTP_UPSTREAM_NOTIFY_CACHED_CONNECTION_ERROR + if (type == NGX_HTTP_UPSTREAM_NOTIFY_CACHED_CONNECTION_ERROR) { + pc->tries--; + } +#endif +} + + static void ngx_http_lua_balancer_close(ngx_connection_t *c) { diff --git a/t/188-balancer_keepalive_pool_max_retry.t b/t/188-balancer_keepalive_pool_max_retry.t new file mode 100644 index 0000000000..456ea794c9 --- /dev/null +++ b/t/188-balancer_keepalive_pool_max_retry.t @@ -0,0 +1,89 @@ +# vim:set ft= ts=4 sw=4 et: + +use Test::Nginx::Socket::Lua; +use Cwd qw(cwd); + +log_level('info'); +repeat_each(1); + +plan tests => repeat_each() * (blocks() * 6); + +my $pwd = cwd(); + +no_long_string(); + +check_accum_error_log(); +run_tests(); + +__DATA__ + +=== TEST 1: sanity +--- http_config + lua_shared_dict request_counter 1m; + upstream my_upstream { + server 127.0.0.1; + balancer_by_lua_block { + local balancer = require "ngx.balancer" + + if not ngx.ctx.tries then + ngx.ctx.tries = 0 + end + + ngx.ctx.tries = ngx.ctx.tries + 1 + ngx.log(ngx.INFO, "tries ", ngx.ctx.tries) + + if ngx.ctx.tries == 1 then + balancer.set_more_tries(5) + end + + local host = "127.0.0.1" + local port = 8090; + + local ok, err = balancer.set_current_peer(host, port) + if not ok then + ngx.log(ngx.ERR, "failed to set the current peer: ", err) + return ngx.exit(500) + end + + balancer.set_timeouts(60000, 60000, 60000) + + local ok, err = balancer.enable_keepalive(60, 100) + if not ok then + ngx.log(ngx.ERR, "failed to enable keepalive: ", err) + return ngx.exit(500) + end + } + } + + server { + listen 0.0.0.0:8090; + location /hello { + content_by_lua_block{ + local request_counter = ngx.shared.request_counter + local first_request = request_counter:get("first_request") + if first_request == nil then + request_counter:set("first_request", "yes") + ngx.print("hello") + else + ngx.exit(ngx.HTTP_CLOSE) + end + } + } + } +--- config + location = /t { + proxy_pass http://my_upstream; + proxy_set_header Connection "keep-alive"; + + rewrite_by_lua_block { + ngx.req.set_uri("/hello") + } + } +--- pipelined_requests eval +["GET /t HTTP/1.1" , "GET /t HTTP/1.1"] +--- response_body eval +["hello", qr/502/] +--- error_code eval +[200, 502] +--- no_error_log eval +qr/tries 7/ From 45c63cda4170728ec7561c102849eb7ea28e0720 Mon Sep 17 00:00:00 2001 From: xiangwei <1031205858@qq.com> Date: Tue, 9 Jul 2024 12:10:49 +0800 Subject: [PATCH 764/848] feature: add ssl trusted certificate. --- src/ngx_http_lua_ssl_certby.c | 84 ++++++++++------ t/140-ssl-c-api.t | 180 +++++++++++++++++++++++++++++++--- 2 files changed, 220 insertions(+), 44 deletions(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 996c3d62a3..0901f06eab 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1468,8 +1468,8 @@ ngx_http_lua_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) int -ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, - int depth, char **err) +ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *client_certs, + void *trusted_certs, int depth, char **err) { #ifdef LIBRESSL_VERSION_NUMBER @@ -1481,7 +1481,8 @@ ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, ngx_http_lua_ctx_t *ctx; ngx_ssl_conn_t *ssl_conn; ngx_http_ssl_srv_conf_t *sscf; - STACK_OF(X509) *chain = ca_certs; + STACK_OF(X509) *client_chain = client_certs; + STACK_OF(X509) *trusted_chain = trusted_certs; STACK_OF(X509_NAME) *name_chain = NULL; X509 *x509 = NULL; X509_NAME *subject = NULL; @@ -1535,54 +1536,75 @@ ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *ca_certs, /* set CA chain */ - if (chain != NULL) { + if (client_chain != NULL || trusted_chain != NULL) { + ca_store = X509_STORE_new(); if (ca_store == NULL) { *err = "X509_STORE_new() failed"; return NGX_ERROR; } - /* construct name chain */ + if (client_chain != NULL) { - name_chain = sk_X509_NAME_new_null(); - if (name_chain == NULL) { - *err = "sk_X509_NAME_new_null() failed"; - goto failed; - } - - for (i = 0; i < sk_X509_num(chain); i++) { - x509 = sk_X509_value(chain, i); - if (x509 == NULL) { - *err = "sk_X509_value() failed"; + /* construct name chain */ + name_chain = sk_X509_NAME_new_null(); + if (name_chain == NULL) { + *err = "sk_X509_NAME_new_null() failed"; goto failed; } - /* add subject to name chain, which will be sent to client */ - subject = X509_NAME_dup(X509_get_subject_name(x509)); - if (subject == NULL) { - *err = "X509_get_subject_name() failed"; - goto failed; + for (i = 0; i < sk_X509_num(client_chain); i++) { + x509 = sk_X509_value(client_chain, i); + if (x509 == NULL) { + *err = "sk_X509_value() failed"; + goto failed; + } + + /* add subject to name chain, which will be sent to client */ + subject = X509_NAME_dup(X509_get_subject_name(x509)); + if (subject == NULL) { + *err = "X509_get_subject_name() failed"; + goto failed; + } + + if (!sk_X509_NAME_push(name_chain, subject)) { + *err = "sk_X509_NAME_push() failed"; + X509_NAME_free(subject); + goto failed; + } + + /* add to trusted CA store */ + if (X509_STORE_add_cert(ca_store, x509) == 0) { + *err = "X509_STORE_add_cert() failed"; + goto failed; + } } - if (!sk_X509_NAME_push(name_chain, subject)) { - *err = "sk_X509_NAME_push() failed"; - X509_NAME_free(subject); - goto failed; - } + /* clean subject name list, and set it for send to client */ + SSL_set_client_CA_list(ssl_conn, name_chain); + } - /* add to trusted CA store */ - if (X509_STORE_add_cert(ca_store, x509) == 0) { - *err = "X509_STORE_add_cert() failed"; - goto failed; + if (trusted_chain != NULL) { + for (i = 0; i < sk_X509_num(trusted_chain); i++) { + x509 = sk_X509_value(trusted_chain, i); + if (x509 == NULL) { + *err = "sk_X509_value() failed"; + goto failed; + } + + /* add to trusted CA store */ + if (X509_STORE_add_cert(ca_store, x509) == 0) { + *err = "X509_STORE_add_cert() failed"; + goto failed; + } } } + /* clean ca_store, and store new ca_store */ if (SSL_set0_verify_cert_store(ssl_conn, ca_store) == 0) { *err = "SSL_set0_verify_cert_store() failed"; goto failed; } - - SSL_set_client_CA_list(ssl_conn, name_chain); } return NGX_OK; diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 001d26ae31..4c81b4f05a 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -12,7 +12,7 @@ if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); } else { - plan tests => repeat_each() * (blocks() * 5 + 1); + plan tests => repeat_each() * (blocks() * 5 - 1); } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -72,7 +72,7 @@ ffi.cdef[[ void ngx_http_lua_ffi_free_priv_key(void *cdata); int ngx_http_lua_ffi_ssl_verify_client(void *r, void *cdata, - int depth, char **err); + void *cdata, int depth, char **err); int ngx_http_lua_ffi_ssl_client_random(ngx_http_request_t *r, unsigned char *out, size_t *outlen, char **err); @@ -853,21 +853,21 @@ lua ssl server name: "test.com" local cert_data = f:read("*all") f:close() - local cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) - if not cert then - ngx.log(ngx.ERR, "failed to parse PEM cert: ", + local client_cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not client_cert then + ngx.log(ngx.ERR, "failed to parse PEM client cert: ", ffi.string(errmsg[0])) return end - local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, cert, 1, errmsg) + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, client_cert, nil, 1, errmsg) if rc ~= 0 then ngx.log(ngx.ERR, "failed to verify client: ", ffi.string(errmsg[0])) return end - ffi.C.ngx_http_lua_ffi_free_cert(cert) + ffi.C.ngx_http_lua_ffi_free_cert(client_cert) } ssl_certificate ../../cert/test2.crt; @@ -924,7 +924,7 @@ client certificate subject: emailAddress=agentzh@gmail.com,CN=test.com return end - local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, nil, -1, errmsg) + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, nil, nil, -1, errmsg) if rc ~= 0 then ngx.log(ngx.ERR, "failed to verify client: ", ffi.string(errmsg[0])) @@ -990,21 +990,21 @@ client certificate subject: emailAddress=agentzh@gmail.com,CN=test.com local cert_data = f:read("*all") f:close() - local cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) - if not cert then - ngx.log(ngx.ERR, "failed to parse PEM cert: ", + local client_cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not client_cert then + ngx.log(ngx.ERR, "failed to parse PEM client cert: ", ffi.string(errmsg[0])) return end - local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, cert, 1, errmsg) + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, client_cert, nil, 1, errmsg) if rc ~= 0 then ngx.log(ngx.ERR, "failed to verify client: ", ffi.string(errmsg[0])) return end - ffi.C.ngx_http_lua_ffi_free_cert(cert) + ffi.C.ngx_http_lua_ffi_free_cert(client_cert) } ssl_certificate ../../cert/test2.crt; @@ -1623,3 +1623,157 @@ lua ssl server name: "test.com" --- no_error_log [error] [alert] + + + +=== TEST 13: verify client, but server don't trust root ca +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name example.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + require "defines" + local ffi = require "ffi" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + local f = assert(io.open("t/cert/mtls_server.crt", "rb")) + local cert_data = f:read("*all") + f:close() + + local client_cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not client_cert then + ngx.log(ngx.ERR, "failed to parse PEM client cert: ", + ffi.string(errmsg[0])) + return + end + + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, client_cert, nil, 2, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to verify client: ", + ffi.string(errmsg[0])) + return + end + + ffi.C.ngx_http_lua_ffi_free_cert(client_cert) + } + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + content_by_lua_block { + ngx.say(ngx.var.ssl_client_verify) + } + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_session_reuse off; + } + +--- request +GET /t +--- response_body +FAILED:unable to verify the first certificate + +--- no_error_log +[error] +[alert] + + + +=== TEST 14: verify client and server trust root ca +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name example.com; + + ssl_certificate_by_lua_block { + collectgarbage() + + require "defines" + local ffi = require "ffi" + + local errmsg = ffi.new("char *[1]") + + local r = require "resty.core.base" .get_request() + if r == nil then + ngx.log(ngx.ERR, "no request found") + return + end + + local f = assert(io.open("t/cert/mtls_server.crt", "rb")) + local cert_data = f:read("*all") + f:close() + + local client_cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not client_cert then + ngx.log(ngx.ERR, "failed to parse PEM client cert: ", + ffi.string(errmsg[0])) + return + end + + local f = assert(io.open("t/cert/mtls_ca.crt", "rb")) + local cert_data = f:read("*all") + f:close() + + local trusted_cert = ffi.C.ngx_http_lua_ffi_parse_pem_cert(cert_data, #cert_data, errmsg) + if not trusted_cert then + ngx.log(ngx.ERR, "failed to parse PEM trusted cert: ", + ffi.string(errmsg[0])) + return + end + + local rc = ffi.C.ngx_http_lua_ffi_ssl_verify_client(r, cert, trusted_cert, 2, errmsg) + if rc ~= 0 then + ngx.log(ngx.ERR, "failed to verify client: ", + ffi.string(errmsg[0])) + return + end + + ffi.C.ngx_http_lua_ffi_free_cert(client_cert) + ffi.C.ngx_http_lua_ffi_free_cert(trusted_cert) + } + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + content_by_lua_block { + ngx.say(ngx.var.ssl_client_verify) + } + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_session_reuse off; + } + +--- request +GET /t +--- response_body +SUCCESS + +--- no_error_log +[error] +[alert] From 8670e53ea7da7d445b3c1907d2c406b1077a1dcd Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Fri, 12 Jul 2024 21:11:05 +0800 Subject: [PATCH 765/848] bugfix: let `balancer.recreate_request` API work for body data changed case. --- README.markdown | 8 +++++-- doc/HttpLuaModule.wiki | 8 +++++-- src/ngx_http_lua_balancer.c | 2 +- t/138-balancer.t | 42 ++++++++++++++++++++++++++++++++++++- 4 files changed, 54 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index af9555f80b..e14d302bfb 100644 --- a/README.markdown +++ b/README.markdown @@ -5563,6 +5563,8 @@ If the request body has been read into memory, try calling the [ngx.req.get_body To force in-file request bodies, try turning on [client_body_in_file_only](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_file_only). +Note that this function is also work for balancer phase but it needs to call [balancer.recreate_request](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md#recreate_request) to make the change take effect after set the request body data or headers. + This function was first introduced in the `v0.3.1rc17` release. See also [ngx.req.get_body_data](#ngxreqget_body_data). @@ -5574,7 +5576,7 @@ ngx.req.set_body_data **syntax:** *ngx.req.set_body_data(data)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, balancer_by_lua*,* Set the current request's request body using the in-memory data specified by the `data` argument. @@ -5582,6 +5584,8 @@ If the request body has not been read yet, call [ngx.req.read_body](#ngxreqread_ Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. +Note that this function is also work for balancer phase but it needs to call [balancer.recreate_request](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md#recreate_request) to make the change take effect after set the request body data or headers. + This function was first introduced in the `v0.3.1rc18` release. See also [ngx.req.set_body_file](#ngxreqset_body_file). @@ -5593,7 +5597,7 @@ ngx.req.set_body_file **syntax:** *ngx.req.set_body_file(file_name, auto_clean?)* -**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua** +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, balancer_by_lua*,* Set the current request's request body using the in-file data specified by the `file_name` argument. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 89cf1ca707..a0e6d28fac 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -4657,7 +4657,7 @@ See also [[#ngx.req.get_body_data|ngx.req.get_body_data]]. '''syntax:''' ''ngx.req.set_body_data(data)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, balancer_by_lua*'' Set the current request's request body using the in-memory data specified by the data argument. @@ -4665,6 +4665,8 @@ If the request body has not been read yet, call [[#ngx.req.read_body|ngx.req.rea Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. +Note that this function is also work for balancer phase but it needs to call [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md#recreate_request balancer.recreate_request] to make the change take effect after set the request body data or headers. + This function was first introduced in the v0.3.1rc18 release. See also [[#ngx.req.set_body_file|ngx.req.set_body_file]]. @@ -4673,7 +4675,7 @@ See also [[#ngx.req.set_body_file|ngx.req.set_body_file]]. '''syntax:''' ''ngx.req.set_body_file(file_name, auto_clean?)'' -'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*'' +'''context:''' ''rewrite_by_lua*, access_by_lua*, content_by_lua*, balancer_by_lua*'' Set the current request's request body using the in-file data specified by the file_name argument. @@ -4685,6 +4687,8 @@ Please ensure that the file specified by the file_name argument exi Whether the previous request body has been read into memory or buffered into a disk file, it will be freed or the disk file will be cleaned up immediately, respectively. +Note that this function is also work for balancer phase but it needs to call [https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md#recreate_request balancer.recreate_request] to make the change take effect after set the request body data or headers. + This function was first introduced in the v0.3.1rc18 release. See also [[#ngx.req.set_body_data|ngx.req.set_body_data]]. diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 620f71eea0..5401a1ac88 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -1288,7 +1288,7 @@ ngx_http_lua_ffi_balancer_recreate_request(ngx_http_request_t *r, /* u->request_bufs already contains a valid request buffer * remove it from chain first */ - u->request_bufs = u->request_bufs->next; + u->request_bufs = r->request_body->bufs; } return u->create_request(r); diff --git a/t/138-balancer.t b/t/138-balancer.t index 5ea94df96d..df63a03cda 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -12,7 +12,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 9); +plan tests => repeat_each() * (blocks() * 4 + 7); #no_diff(); no_long_string(); @@ -637,3 +637,43 @@ ok --- no_error_log [error] [cirt] + + +=== TEST 20: recreate_request refresh body buffer when ngx.req.set_body_data is used in balancer phase +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; + + server { + listen 127.0.0.1:$TEST_NGINX_RAND_PORT_1; + + location / { + content_by_lua_block { + ngx.req.read_body() + local body = ngx.req.get_body_data() + ngx.log(ngx.ERR, "body: ", body) + ngx.say(body) + } + } + } + + upstream foo { + server 127.0.0.1:$TEST_NGINX_RAND_PORT_1 max_fails=0; + + balancer_by_lua_block { + local bal = require "ngx.balancer" + ngx.req.set_body_data("hello world") + assert(bal.recreate_request()) + } + } + +--- config + location = /t { + proxy_http_version 1.1; + proxy_set_header Connection ""; + proxy_pass http://foo; + } +--- request +GET /t +--- error_code: 200 +--- response_body +hello world From 6f311f82c3eac71bc6cfb537b303b5fe4f424653 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 12 Jul 2024 21:12:48 +0800 Subject: [PATCH 766/848] tests: update ngixn to 1.25.3. --- .travis.yml | 4 ++-- t/138-balancer.t | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index d3b973c80d..e6321c453d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -64,8 +64,8 @@ env: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f - - NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y - - NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y + - NGINX_VERSION=1.25.3 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y + - NGINX_VERSION=1.25.3 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 services: diff --git a/t/138-balancer.t b/t/138-balancer.t index df63a03cda..41be75fcdc 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -639,6 +639,7 @@ ok [cirt] + === TEST 20: recreate_request refresh body buffer when ngx.req.set_body_data is used in balancer phase --- http_config lua_package_path "../lua-resty-core/lib/?.lua;;"; From ed8cb8fe302734a4d66c83348369785c3d57470c Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 16 Jul 2024 14:27:17 +0800 Subject: [PATCH 767/848] bugfix: treat shdict entries with ttl equal to 0 as expired. --- src/ngx_http_lua_shdict.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index c63eb3c107..146974d954 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -210,7 +210,7 @@ ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, dd("time to live: %lld", (long long) ms); - if (ms < 0) { + if (ms <= 0) { dd("node already expired"); return NGX_DONE; } From 676872125514dbe02e2cf0b4c522e87e6e5db3d4 Mon Sep 17 00:00:00 2001 From: leslie Date: Thu, 25 Jul 2024 10:45:12 +0800 Subject: [PATCH 768/848] doc: update lua-cjson ref link. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index e14d302bfb..29a81c1ca4 100644 --- a/README.markdown +++ b/README.markdown @@ -1011,7 +1011,7 @@ The order in which these modules are added during configuration is important bec filtering chain determines the final output, for example. The correct adding order is shown above. * 3rd-party Lua libraries: - * [lua-cjson](http://www.kyne.com.au/~mark/software/lua-cjson.php) + * [lua-cjson](https://www.kyne.au/~mark/software/lua-cjson.php) * Applications: * mysql: create database 'ngx_test', grant all privileges to user 'ngx_test', password is 'ngx_test' From 29fe7a504f843e270049d16277ca117f681d20ba Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Wed, 31 Jul 2024 08:38:06 +0800 Subject: [PATCH 769/848] dev: util/build.sh: fixed command line argument validation and environment variable usage. --- util/build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/util/build.sh b/util/build.sh index 41896f2c7e..fdf5c4d53d 100755 --- a/util/build.sh +++ b/util/build.sh @@ -25,13 +25,13 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" add_http3_module=--with-http_v3_module -answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +answer=`$root/util/ver-ge "$version" 1.25.1` if [ "$OPENSSL_VER" = "1.1.0l" ] || [ "$answer" = "N" ]; then add_http3_module="" fi disable_pcre2=--without-pcre2 -answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` +answer=`$root/util/ver-ge "$version" 1.25.1` if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then disable_pcre2="" fi From d535753f5421c66715bf71f3df65b11e255407e2 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 1 Aug 2024 16:34:36 +0800 Subject: [PATCH 770/848] bugfix: nginx crashed when binding local address failed from lua. * bugfix: nginx crashed when binding local address failed from lua. 0 0x00007f10b0b16c59 in __memcpy_ssse3_back () from /lib64/libc.so.6 1 0x000000000042e2ba in ngx_sprintf_str (buf=, last=last@entry=0x7fff82779250 "P\251#\002", src=, len=192, hexadecimal=hexadecimal@entry=0) at src/core/ngx_string.c:586 2 0x000000000042e824 in ngx_vslprintf (buf=, buf@entry=0x7fff82778279 "bind() failed (\347(G\300\a", last=last@entry=0x7fff82779250 "P\251#\002", fmt=0x59c7fc "V) failed", fmt@entry=0x59c7f6 "bind(%V) failed", args=args@entry=0x7fff82779258) at src/core/ngx_string.c:255 3 0x000000000042a243 in ngx_log_error_core (level=level@entry=3, log=log@entry=0x2263360, err=98, fmt=fmt@entry=0x59c7f6 "bind(%V) failed") at src/core/ngx_log.c:137 4 0x000000000044c4fc in ngx_event_connect_peer (pc=pc@entry=0x223a950) at src/event/ngx_event_connect.c:169 5 0x000000000049af8c in ngx_http_upstream_connect (r=r@entry=0x22923b0, u=u@entry=0x223a940) at src/http/ngx_http_upstream.c:1562 6 0x000000000049c410 in ngx_http_upstream_init_request (r=r@entry=0x22923b0) at src/http/ngx_http_upstream.c:826 7 0x000000000049ee79 in ngx_http_upstream_init (r=0x22923b0) at src/http/ngx_http_upstream.c:554 8 0x00000000004906ad in ngx_http_read_client_request_body (r=r@entry=0x22923b0, post_handler=0x49ed31 ) at src/http/ngx_http_request_body.c:47 9 0x00000000004dba84 in ngx_http_proxy_handler (r=0x22923b0) at src/http/modules/ngx_http_proxy_module.c:1023 10 0x00000000004822f4 in ngx_http_core_content_phase (r=0x22923b0, ph=) at src/http/ngx_http_core_module.c:1271 11 0x000000000047cb13 in ngx_http_core_run_phases (r=0x22923b0) at src/http/ngx_http_core_module.c:885 12 0x000000000047cc1e in ngx_http_handler (r=) at src/http/ngx_http_core_module.c:868 13 0x0000000000485f0e in ngx_http_run_posted_requests (c=c@entry=0x227c8f0) at src/http/ngx_http_request.c:2470 14 0x0000000000488b8b in ngx_http_process_request_headers (rev=rev@entry=0x22204f0) at src/http/ngx_http_request.c:1552 15 0x0000000000488e83 in ngx_http_process_request_line (rev=rev@entry=0x22204f0) at src/http/ngx_http_request.c:1196 16 0x0000000000489b82 in ngx_http_keepalive_handler (rev=0x22204f0) at src/http/ngx_http_request.c:3441 17 0x00000000004556e1 in ngx_epoll_process_events (cycle=0x2213ce0, timer=, flags=) at src/event/modules/ngx_epoll_module.c:901 18 0x000000000044a286 in ngx_process_events_and_timers (cycle=cycle@entry=0x2213ce0) at src/event/ngx_event.c:258 19 0x00000000004546a1 in ngx_single_process_cycle (cycle=cycle@entry=0x2213ce0) at src/os/unix/ngx_process_cycle.c:323 20 0x0000000000429793 in main (argc=, argv=) at src/core/nginx.c:384 --- src/ngx_http_lua_balancer.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index 5401a1ac88..ae0f1380b5 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -994,7 +994,7 @@ ngx_http_lua_ffi_balancer_bind_to_local_addr(ngx_http_request_t *r, bp = (ngx_http_lua_balancer_peer_data_t *) u->peer.data; if (bp->local == NULL) { - bp->local = ngx_palloc(r->pool, sizeof(ngx_addr_t)); + bp->local = ngx_palloc(r->pool, sizeof(ngx_addr_t) + addr_len); if (bp->local == NULL) { p = ngx_snprintf(errbuf, *errbuf_size, "no memory"); *errbuf_size = p - errbuf; @@ -1009,6 +1009,10 @@ ngx_http_lua_ffi_balancer_bind_to_local_addr(ngx_http_request_t *r, return NGX_ERROR; } + bp->local->name.len = addr_len; + bp->local->name.data = (u_char *) (bp->local + 1); + ngx_memcpy(bp->local->name.data, addr, addr_len); + return NGX_OK; } From 0b5507a25553e1d4271535d2efce29ad67c60bfe Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 1 Aug 2024 13:54:53 +0800 Subject: [PATCH 771/848] bugfix: added initialization. --- src/ngx_http_lua_ssl_ocsp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_ocsp.c b/src/ngx_http_lua_ssl_ocsp.c index c9f24cd281..73e1b9c7f5 100644 --- a/src/ngx_http_lua_ssl_ocsp.c +++ b/src/ngx_http_lua_ssl_ocsp.c @@ -280,7 +280,7 @@ ngx_http_lua_ffi_ssl_validate_ocsp_response(const u_char *resp, OCSP_RESPONSE *ocsp = NULL; OCSP_BASICRESP *basic = NULL; STACK_OF(X509) *chain = NULL; - ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + ASN1_GENERALIZEDTIME *thisupdate = NULL, *nextupdate = NULL; ocsp = d2i_OCSP_RESPONSE(NULL, &resp, resp_len); if (ocsp == NULL) { From 5777a36a93fe7520d958dd512a14d553e50b807b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 1 Aug 2024 18:51:09 +0800 Subject: [PATCH 772/848] bugfix: failed to build on the old nginx version. --- src/ngx_http_lua_module.c | 6 +++++- src/ngx_http_lua_ssl_client_helloby.c | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 14ed235838..63367f46b3 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1514,7 +1514,11 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, (NGX_CONF_BITMASK_SET |NGX_SSL_TLSv1|NGX_SSL_TLSv1_1 - |NGX_SSL_TLSv1_2|NGX_SSL_TLSv1_3)); + |NGX_SSL_TLSv1_2 +#ifdef NGX_SSL_TLSv1_3 + |NGX_SSL_TLSv1_3 +#endif + )); ngx_conf_merge_str_value(conf->ssl_ciphers, prev->ssl_ciphers, "DEFAULT"); diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index b808f98ed7..9800f7d41f 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -712,7 +712,7 @@ ngx_http_lua_ffi_ssl_set_protocols(ngx_http_request_t *r, } #endif -#ifdef SSL_OP_NO_TLSv1_3 +#if defined(NGX_SSL_TLSv1_3) && defined( SSL_OP_NO_TLSv1_3) SSL_clear_options(ssl_conn, SSL_OP_NO_TLSv1_3); if (!(protocols & NGX_SSL_TLSv1_3)) { SSL_set_options(ssl_conn, SSL_OP_NO_TLSv1_3); From bf4bdcd5b299fc5680eb7d6fd594d4a49983de21 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 6 Aug 2024 12:36:58 +0800 Subject: [PATCH 773/848] bugfix: fixed keepalive error in cosocket. This bug was found in t/129-ssl-socket.t Test 18 when TEST_NGINX_EVENT_TYPE=poll enabled. --- src/ngx_http_lua_socket_tcp.c | 10 +++++ t/129-ssl-socket.t | 76 ++++++++++++++++++++++++++++++++--- 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 214e78329e..5010dfa6ed 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5747,6 +5747,16 @@ ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev) "lua tcp socket keepalive close handler check stale events"); n = recv(c->fd, buf, 1, MSG_PEEK); +#if (NGX_HTTP_SSL) + /* ignore ssl protocol data like change cipher spec */ + if (n == 1 && c->ssl != NULL) { + n = c->recv(c, (unsigned char *) buf, 1); + if (n == NGX_AGAIN) { + n = -1; + ngx_socket_errno = NGX_EAGAIN; + } + } +#endif if (n == -1 && ngx_socket_errno == NGX_EAGAIN) { /* stale event */ diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index ccfa19fffb..ca8d5a49e6 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1484,6 +1484,72 @@ SSL reused session === TEST 18: openresty.org: passing SSL verify: keepalive (no reusing the ssl session) +The session returned by SSL_get1_session maybe different. +After function tls_process_new_session_ticket, the session saved in SSL->session +will be replace by a new one. + +ngx_ssl_session_t * +ngx_ssl_get_session(ngx_connection_t *c) +{ +#ifdef TLS1_3_VERSION + if (c->ssl->session) { + SSL_SESSION_up_ref(c->ssl->session); + return c->ssl->session; + } +#endif + + return SSL_get1_session(c->ssl->connection); +} + +SSL_SESSION *SSL_get1_session(SSL *ssl) +/* variant of SSL_get_session: caller really gets something */ +{ + SSL_SESSION *sess; + /* + * Need to lock this all up rather than just use CRYPTO_add so that + * somebody doesn't free ssl->session between when we check it's non-null + * and when we up the reference count. + */ + CRYPTO_THREAD_read_lock(ssl->lock); + sess = ssl->session; + if (sess) + SSL_SESSION_up_ref(sess); + CRYPTO_THREAD_unlock(ssl->lock); + return sess; +} + +#0 tls_process_new_session_ticket (s=0x7e6ea0, pkt=0x7fffffffc820) at ssl/statem/statem_clnt.c:2650 +#1 0x00007ffff7af50fd in read_state_machine (s=0x7e6ea0) at ssl/statem/statem.c:636 +#2 state_machine (s=0x7e6ea0, server=0) at ssl/statem/statem.c:434 +#3 0x00007ffff7aca6b3 in ssl3_read_bytes (s=, type=23, recvd_type=0x0, buf=0x7fffffffc9d7 "\027\320\355t", len=1, + peek=0, readbytes=0x7fffffffc978) at ssl/record/rec_layer_s3.c:1677 +#4 0x00007ffff7ad2250 in ssl3_read_internal (readbytes=0x7fffffffc978, peek=0, len=1, buf=0x7fffffffc9d7, s=0x7e6ea0) + at ssl/s3_lib.c:4477 +#5 ssl3_read (s=0x7e6ea0, buf=0x7fffffffc9d7, len=1, readbytes=0x7fffffffc978) at ssl/s3_lib.c:4500 +#6 0x00007ffff7ade695 in SSL_read (s=, buf=buf@entry=0x7fffffffc9d7, num=num@entry=1) at ssl/ssl_lib.c:1799 +#7 0x000000000045a965 in ngx_ssl_recv (c=0x72c3b0, buf=0x7fffffffc9d7 "\027\320\355t", size=1) + at src/event/ngx_event_openssl.c:2337 +#8 0x0000000000533b17 in ngx_http_lua_socket_keepalive_close_handler (ev=0x7e2f20) + at /var/code/openresty/lua-nginx-module/src/ngx_http_lua_socket_tcp.c:5753 +#9 0x000000000052cf40 in ngx_http_lua_socket_tcp_setkeepalive (L=0x74edd0) + at /var/code/openresty/lua-nginx-module/src/ngx_http_lua_socket_tcp.c:5602 +#10 0x00007ffff7f0fabe in lj_BC_FUNCC () + from /tmp/undodb.72729.1722915526.2470007.80d50d088e818fd4/debuggee-1-zwqz8svp/symbol-files/opt/luajit-sysm/lib/libluajit-5.1.so.2 +#11 0x000000000051f2b2 in ngx_http_lua_run_thread (L=L@entry=0x767670, r=r@entry=0x7edf80, ctx=ctx@entry=0x750e40, nrets=0) + at /var/code/openresty/lua-nginx-module/src/ngx_http_lua_util.c:1194 +#12 0x0000000000524347 in ngx_http_lua_content_by_chunk (L=0x767670, r=0x7edf80) + at /var/code/openresty/lua-nginx-module/src/ngx_http_lua_contentby.c:124 +#13 0x000000000047c663 in ngx_http_core_content_phase (r=0x7edf80, ph=0x7b4470) at src/http/ngx_http_core_module.c:1271 +#14 0x000000000047b80d in ngx_http_core_run_phases (r=0x7edf80) at src/http/ngx_http_core_module.c:885 +#15 ngx_http_handler (r=r@entry=0x7edf80) at src/http/ngx_http_core_module.c:868 +#16 0x00000000004854ad in ngx_http_process_request (r=r@entry=0x7edf80) at src/http/ngx_http_request.c:2140 +#17 0x00000000004868e8 in ngx_http_process_request_headers (rev=rev@entry=0x7e2f80) at src/http/ngx_http_request.c:1529 +#18 0x0000000000486468 in ngx_http_process_request_line (rev=0x7e2f80) at src/http/ngx_http_request.c:1196 +#19 0x000000000044b338 in ngx_event_process_posted (cycle=cycle@entry=0x721690, posted=0x62f250 ) + at src/event/ngx_event_posted.c:35 +#20 0x000000000044a522 in ngx_process_events_and_timers (cycle=cycle@entry=0x721690) at src/event/ngx_event.c:273 +#21 0x0000000000453819 in ngx_single_process_cycle (cycle=cycle@entry=0x721690) at src/os/unix/ngx_process_cycle.c:323 +#22 0x0000000000429dee in main (argc=argc@entry=5, argv=argv@entry=0x7fffffffd1a8) at src/core/nginx.c:384 --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; @@ -1548,11 +1614,11 @@ set keepalive: 1 nil --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out eval qr/^lua ssl save session: ([0-9A-F]+) -lua ssl save session: \1 -lua ssl save session: \1 -lua ssl free session: \1 -lua ssl free session: \1 -lua ssl free session: \1 +lua ssl save session: ([0-9A-F]+) +lua ssl save session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) +lua ssl free session: ([0-9A-F]+) $/ --- error_log From b3c6aebcc79528c69794441d55cf228c9fd64af0 Mon Sep 17 00:00:00 2001 From: jiahao Date: Thu, 30 May 2024 11:43:06 +0800 Subject: [PATCH 774/848] travis: bumped the NGINX core to 1.27.0. --- .travis.yml | 6 +++--- t/014-bugs.t | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.travis.yml b/.travis.yml index e6321c453d..24902f8168 100644 --- a/.travis.yml +++ b/.travis.yml @@ -64,8 +64,8 @@ env: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f - - NGINX_VERSION=1.25.3 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y - - NGINX_VERSION=1.25.3 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y + - NGINX_VERSION=1.27.0 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 + - NGINX_VERSION=1.27.0 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y TEST_NGINX_QUIC_IDLE_TIMEOUT=3 #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 services: @@ -87,7 +87,7 @@ install: - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git - - git clone https://github.com/openresty/openresty.git ../openresty + - git clone -b bump-1.27.0 https://github.com/xiaocang/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - git clone https://github.com/openresty/openresty-devel-utils.git - git clone https://github.com/openresty/mockeagain.git diff --git a/t/014-bugs.t b/t/014-bugs.t index 1b79aa4c59..d34f42e23d 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -828,7 +828,7 @@ qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received rewrite ^/myproxy/(.*) /$1 break; resolver_timeout 3s; #resolver 172.16.0.23; # AWS DNS resolver address is the same in all regions - 172.16.0.23 - resolver 8.8.8.8; + resolver $TEST_NGINX_RESOLVER; proxy_read_timeout 1s; proxy_send_timeout 1s; proxy_connect_timeout 1s; From 0ea5e94a80fe92213891942b16f0e4db20d48407 Mon Sep 17 00:00:00 2001 From: jiahao Date: Tue, 4 Jun 2024 18:44:26 +0800 Subject: [PATCH 775/848] tests: added curl_error for http/3. --- t/068-socket-keepalive.t | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index 1660a3a361..626b441678 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -3109,6 +3109,8 @@ qr/\Qbad argument #1 to 'setkeepalive' (number expected, got string)\E/ --- no_error_log [crit] --- timeout: 4 +--- curl_error eval +qr{HTTP/3 stream 0 reset by server} @@ -3189,3 +3191,5 @@ qr/\Qbad argument #2 to 'setkeepalive' (number expected, got string)\E/ --- no_error_log [crit] --- timeout: 4 +--- curl_error eval +qr{HTTP/3 stream 0 reset by server} From f1cbe4ae945b2005c38147e14431c766a2146eec Mon Sep 17 00:00:00 2001 From: jiahao Date: Wed, 3 Jul 2024 18:10:49 +0800 Subject: [PATCH 776/848] tests: t/020-subrequest.t: replace the random port with an unused five-digit port. --- .travis.yml | 2 +- t/020-subrequest.t | 59 +++++++++++++++++++++++++++------------------- 2 files changed, 36 insertions(+), 25 deletions(-) diff --git a/.travis.yml b/.travis.yml index 24902f8168..6033ee185f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -87,7 +87,7 @@ install: - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git - - git clone -b bump-1.27.0 https://github.com/xiaocang/openresty.git ../openresty + - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - git clone https://github.com/openresty/openresty-devel-utils.git - git clone https://github.com/openresty/mockeagain.git diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 59b9f61a34..37914be061 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1,6 +1,7 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: use Test::Nginx::Socket::Lua; +use Test::Nginx::Util 'is_tcp_port_used'; #master_on(); #workers(1); @@ -16,6 +17,16 @@ plan tests => repeat_each() * (blocks() * 3 + 23); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); +# NB: tcp_listen_port needs to be greater than 10000, +# because the test cases expect it to be a 5-digit number +my $tcp_listen_port = 19113; +while (++$tcp_listen_port < 65535) { + if (!is_tcp_port_used $tcp_listen_port) { + last; + } +} +$ENV{TEST_NGINX_TCP_LISTEN_PORT} = $tcp_listen_port; + #no_diff(); no_long_string(); #no_shuffle(); @@ -1383,7 +1394,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1396,7 +1407,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\nContent-Length: 1024\r\n\r\nhello world" @@ -1443,7 +1454,7 @@ upstream prematurely closed connection proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1456,7 +1467,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_no_close --- tcp_reply eval "HTTP/1.0 200 OK\r\nContent-Length: 1024\r\n\r\nhello world" @@ -1505,7 +1516,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1518,7 +1529,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1565,7 +1576,7 @@ truncated: false proxy_read_timeout 100ms; proxy_buffering on; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1578,7 +1589,7 @@ truncated: false } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_no_close --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1628,7 +1639,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering off; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1641,7 +1652,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1688,7 +1699,7 @@ truncated: false proxy_read_timeout 500ms; proxy_buffering off; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1701,7 +1712,7 @@ truncated: false } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_no_close --- tcp_reply eval "HTTP/1.0 200 OK\r\n\r\nhello world" @@ -1914,7 +1925,7 @@ a client request body is buffered to a temporary file #proxy_read_timeout 100ms; proxy_http_version 1.1; proxy_buffering on; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1927,7 +1938,7 @@ a client request body is buffered to a temporary file } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nb\r\nhello world\r" @@ -1977,7 +1988,7 @@ upstream prematurely closed connection #proxy_read_timeout 100ms; proxy_http_version 1.1; proxy_buffering off; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -1990,7 +2001,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nb\r\nhello world\r" @@ -2038,7 +2049,7 @@ upstream prematurely closed connection proxy_read_timeout 100ms; proxy_buffering on; proxy_http_version 1.1; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -2051,7 +2062,7 @@ upstream prematurely closed connection } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_no_close --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nb\r\nhello world\r" @@ -2100,7 +2111,7 @@ upstream timed out #proxy_read_timeout 100ms; proxy_buffering on; proxy_http_version 1.1; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -2113,7 +2124,7 @@ upstream timed out } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_no_close --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n" @@ -2158,7 +2169,7 @@ truncated: false #proxy_read_timeout 100ms; proxy_buffering off; proxy_http_version 1.1; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -2171,7 +2182,7 @@ truncated: false } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_no_close --- tcp_reply eval "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n" @@ -2217,7 +2228,7 @@ truncated: false #proxy_read_timeout 100ms; proxy_buffering off; - proxy_pass http://127.0.0.1:$TEST_NGINX_RAND_PORT_2; + proxy_pass http://127.0.0.1:$TEST_NGINX_TCP_LISTEN_PORT; } location /main { @@ -2230,7 +2241,7 @@ truncated: false } --- request GET /main ---- tcp_listen: $TEST_NGINX_RAND_PORT_2 +--- tcp_listen: $TEST_NGINX_TCP_LISTEN_PORT --- tcp_query_len: 65 --- tcp_reply eval "HTTP/1.0 200 OK\r\nContent-Length: 1024\r\n\r\nhello world" From 2d8fcbe6435b0bf4b0ae91dd84002cce9fe20c8c Mon Sep 17 00:00:00 2001 From: jiahao Date: Thu, 1 Aug 2024 23:12:49 +0800 Subject: [PATCH 777/848] tests: skip t/163-signal.t in check leak mode. --- t/163-signal.t | 3 +++ 1 file changed, 3 insertions(+) diff --git a/t/163-signal.t b/t/163-signal.t index 15f41e2200..0ce8fa2613 100644 --- a/t/163-signal.t +++ b/t/163-signal.t @@ -5,6 +5,9 @@ our $SkipReason; BEGIN { if ($ENV{TEST_NGINX_USE_HUP}) { $SkipReason = "unavailable under hup test mode"; + + } elsif ($ENV{TEST_NGINX_CHECK_LEAK}) { + $SkipReason = "unavailable under check leak test mode"; } } From 0cc05a6090886880d4e04ab6a5cdebbfa7da2ccc Mon Sep 17 00:00:00 2001 From: jiahao Date: Fri, 2 Aug 2024 16:37:00 +0800 Subject: [PATCH 778/848] tests: t/188-*.t: use random port instead of 8090 port. --- t/188-balancer_keepalive_pool_max_retry.t | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/t/188-balancer_keepalive_pool_max_retry.t b/t/188-balancer_keepalive_pool_max_retry.t index 456ea794c9..679ee680f8 100644 --- a/t/188-balancer_keepalive_pool_max_retry.t +++ b/t/188-balancer_keepalive_pool_max_retry.t @@ -35,9 +35,9 @@ __DATA__ if ngx.ctx.tries == 1 then balancer.set_more_tries(5) end - + local host = "127.0.0.1" - local port = 8090; + local port = $TEST_NGINX_RAND_PORT_1; local ok, err = balancer.set_current_peer(host, port) if not ok then @@ -56,9 +56,9 @@ __DATA__ } server { - listen 0.0.0.0:8090; + listen 127.0.0.1:$TEST_NGINX_RAND_PORT_1; location /hello { - content_by_lua_block{ + content_by_lua_block{ local request_counter = ngx.shared.request_counter local first_request = request_counter:get("first_request") if first_request == nil then @@ -74,7 +74,7 @@ __DATA__ location = /t { proxy_pass http://my_upstream; proxy_set_header Connection "keep-alive"; - + rewrite_by_lua_block { ngx.req.set_uri("/hello") } From f44c18859050be5af314bae8f05660478af5e31d Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Thu, 8 Aug 2024 23:41:21 +0800 Subject: [PATCH 779/848] bumped version of lua-nginx-module to 10027. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 193c44e3a8..4b374f56ee 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10026 +#define ngx_http_lua_version 10027 typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; From d6cf113d6b80946b9d440fd1d43d0795edcbfb63 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 17 Aug 2024 18:29:38 +0800 Subject: [PATCH 780/848] feature: update nginx to v1.27.1. --- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 6033ee185f..177d6488ca 100644 --- a/.travis.yml +++ b/.travis.yml @@ -64,8 +64,8 @@ env: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f - - NGINX_VERSION=1.27.0 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 - - NGINX_VERSION=1.27.0 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y TEST_NGINX_QUIC_IDLE_TIMEOUT=3 + - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 + - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y TEST_NGINX_QUIC_IDLE_TIMEOUT=3 #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 services: From aad278bc8c37d55023cf2e6c3b3040c01bf0083a Mon Sep 17 00:00:00 2001 From: Johnny Wang Date: Sat, 17 Aug 2024 20:23:35 +0800 Subject: [PATCH 781/848] tests: bumped the NGINX core to 1.27.1. From ea09d92adf835e30cab8d79343c7b8266e888128 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 20 Aug 2024 16:13:59 +0800 Subject: [PATCH 782/848] bugfix: failed to build on windows. --- src/ngx_http_lua_socket_tcp.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 5010dfa6ed..a0e041b017 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5725,6 +5725,7 @@ ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev) ngx_http_lua_socket_pool_t *spool; int n; + int err; char buf[1]; ngx_connection_t *c; @@ -5747,18 +5748,19 @@ ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev) "lua tcp socket keepalive close handler check stale events"); n = recv(c->fd, buf, 1, MSG_PEEK); + err = ngx_socket_errno; #if (NGX_HTTP_SSL) /* ignore ssl protocol data like change cipher spec */ if (n == 1 && c->ssl != NULL) { n = c->recv(c, (unsigned char *) buf, 1); if (n == NGX_AGAIN) { n = -1; - ngx_socket_errno = NGX_EAGAIN; + err = NGX_EAGAIN; } } #endif - if (n == -1 && ngx_socket_errno == NGX_EAGAIN) { + if (n == -1 && err == NGX_EAGAIN) { /* stale event */ if (ngx_handle_read_event(c->read, 0) != NGX_OK) { From 55f55efbd159f7376aba6e388983b96c20b5698c Mon Sep 17 00:00:00 2001 From: Josh Soref <2119212+jsoref@users.noreply.github.com> Date: Fri, 23 Aug 2024 20:45:17 -0400 Subject: [PATCH 783/848] doc: improve grammar. --- README.markdown | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.markdown b/README.markdown index 29a81c1ca4..4d09757015 100644 --- a/README.markdown +++ b/README.markdown @@ -269,7 +269,7 @@ memory use. Request contexts are segregated using lightweight Lua coroutines. Loaded Lua modules persist in the Nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads. -This module is plugged into Nginx's "http" subsystem so it can only speaks +This module is plugged into Nginx's "http" subsystem so it can only speak downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, etc...). If you want to do generic TCP communications with the downstream clients, then you should use the @@ -283,7 +283,7 @@ Typical Uses Just to name a few: -* Mashup'ing and processing outputs of various Nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua, +* Mashup'ing and processing outputs of various Nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, etc.) in Lua, * doing arbitrarily complex access control and security checks in Lua before requests actually reach the upstream backends, * manipulating response headers in an arbitrary way (by Lua) * fetching backend information from external storage backends (like redis, memcached, mysql, postgresql) and use that information to choose which upstream backend to access on-the-fly, @@ -337,7 +337,7 @@ It is discouraged to build this module with Nginx yourself since it is tricky to set up exactly right. Note that Nginx, LuaJIT, and OpenSSL official releases have various limitations -and long standing bugs that can cause some of this module's features to be +and long-standing bugs that can cause some of this module's features to be disabled, not work properly, or run slower. Official OpenResty releases are recommended because they bundle [OpenResty's optimized LuaJIT 2.1 fork](https://github.com/openresty/luajit2) and [Nginx/OpenSSL @@ -421,7 +421,7 @@ While building this module either via OpenResty or with the Nginx core, you can * `NGX_LUA_USE_ASSERT` When defined, will enable assertions in the ngx_lua C code base. Recommended for debugging or testing builds. It can introduce some (small) runtime overhead when enabled. This macro was first introduced in the `v0.9.10` release. * `NGX_LUA_ABORT_AT_PANIC` - When the LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the `v0.9.8` release. + When the LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually results in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the `v0.9.8` release. To enable one or more of these macros, just pass extra C compiler options to the `./configure` script of either Nginx or OpenResty. For instance, @@ -4149,7 +4149,7 @@ Then `GET /main` will give the output Here, modification of the `ngx.ctx.blah` entry in the subrequest does not affect the one in the parent request. This is because they have two separate versions of `ngx.ctx.blah`. -Internal redirects (triggered by nginx configuration directives like `error_page`, `try_files`, `index` and etc) will destroy the original request `ngx.ctx` data (if any) and the new request will have an empty `ngx.ctx` table. For instance, +Internal redirects (triggered by nginx configuration directives like `error_page`, `try_files`, `index`, etc.) will destroy the original request `ngx.ctx` data (if any) and the new request will have an empty `ngx.ctx` table. For instance, ```nginx @@ -8996,7 +8996,7 @@ this context. You must notice that each timer will be based on a fake request (this fake request is also based on a fake connection). Because Nginx's memory release is based on the connection closure, if you run a lot of APIs that apply for memory resources in a timer, such as [tcpsock:connect](#tcpsockconnect), will cause the accumulation of memory resources. So it is recommended to create a new timer after running several times to release memory resources. -You can pass most of the standard Lua values (nils, booleans, numbers, strings, tables, closures, file handles, and etc) into the timer callback, either explicitly as user arguments or implicitly as upvalues for the callback closure. There are several exceptions, however: you *cannot* pass any thread objects returned by [coroutine.create](#coroutinecreate) and [ngx.thread.spawn](#ngxthreadspawn) or any cosocket objects returned by [ngx.socket.tcp](#ngxsockettcp), [ngx.socket.udp](#ngxsocketudp), and [ngx.req.socket](#ngxreqsocket) because these objects' lifetime is bound to the request context creating them while the timer callback is detached from the creating request's context (by design) and runs in its own (fake) request context. If you try to share the thread or cosocket objects across the boundary of the creating request, then you will get the "no co ctx found" error (for threads) or "bad request" (for cosockets). It is fine, however, to create all these objects inside your timer callback. +You can pass most of the standard Lua values (nils, booleans, numbers, strings, tables, closures, file handles, etc.) into the timer callback, either explicitly as user arguments or implicitly as upvalues for the callback closure. There are several exceptions, however: you *cannot* pass any thread objects returned by [coroutine.create](#coroutinecreate) and [ngx.thread.spawn](#ngxthreadspawn) or any cosocket objects returned by [ngx.socket.tcp](#ngxsockettcp), [ngx.socket.udp](#ngxsocketudp), and [ngx.req.socket](#ngxreqsocket) because these objects' lifetime is bound to the request context creating them while the timer callback is detached from the creating request's context (by design) and runs in its own (fake) request context. If you try to share the thread or cosocket objects across the boundary of the creating request, then you will get the "no co ctx found" error (for threads) or "bad request" (for cosockets). It is fine, however, to create all these objects inside your timer callback. Please note that the timer Lua handler has its own copy of the `ngx.ctx` magic table. It won't share the same `ngx.ctx` with the Lua handler creating the timer. @@ -9489,7 +9489,7 @@ The type of `args` must be one of type below: * nil * table (the table may be recursive, and contains members of types above.) -The `ok` is in boolean type, which indicate the C land error (failed to get thread from thread pool, pcall the module function failed, .etc). If `ok` is `false`, the `res1` is the error string. +The `ok` is in boolean type, which indicate the C land error (failed to get thread from thread pool, pcall the module function failed, etc.). If `ok` is `false`, the `res1` is the error string. The return values (res1, ...) are returned by invocation of the module function. Normally, the `res1` should be in boolean type, so that the caller could inspect the error. From 8ec4f0b5bde37532c4725dcf88579823a8e5f7a3 Mon Sep 17 00:00:00 2001 From: xiaobiaozhao <52393536+xiaobiaozhao@users.noreply.github.com> Date: Fri, 30 Aug 2024 10:27:50 +0800 Subject: [PATCH 784/848] doc: Redraw directives png (#2353) --- README.markdown | 2 +- .../lua_nginx_modules_directives.drawio.png | Bin 0 -> 145387 bytes 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 doc/images/lua_nginx_modules_directives.drawio.png diff --git a/README.markdown b/README.markdown index 4d09757015..25d1383f36 100644 --- a/README.markdown +++ b/README.markdown @@ -1184,7 +1184,7 @@ Directives The basic building blocks of scripting Nginx with Lua are directives. Directives are used to specify when the user Lua code is run and how the result will be used. Below is a diagram showing the order in which directives are executed. -![Lua Nginx Modules Directives](https://cloud.githubusercontent.com/assets/2137369/15272097/77d1c09e-1a37-11e6-97ef-d9767035fc3e.png) +![Lua Nginx Modules Directives](./doc/images/lua_nginx_modules_directives.drawio.png) [Back to TOC](#table-of-contents) diff --git a/doc/images/lua_nginx_modules_directives.drawio.png b/doc/images/lua_nginx_modules_directives.drawio.png new file mode 100644 index 0000000000000000000000000000000000000000..41112147f2fb1320c9ce132cdb334a1ef94109f7 GIT binary patch literal 145387 zcmeEu1z1(vx;7w6h=`P+NG_0;4(VnRQi^~eUD6#Q2!cq%0x1zx5JW&qNh#@8lrCwI zl8}ynEM@J@c6-jf|2ZDdz585T&*NHSjycB|-+0H@-waT`By$4$3^oc1$_YqTQWXW| z2!w)iR0i`HXt|wCgSdd|peiGd@}iAw1_edQz)?!W(aP1t+|n3@mQ&)}H(E|9@ZHE< z-_%~;ik5@T+=!M_l9q!*mCHm)-AU1f?~1AO6+12?2U{5x&suLr>!Yl1YRtlp zX#M&SD_0qNeOog{8zW;&(9g*ATQ43i{%l=;mZ@WbANo^S35P8yibUbK5^Q8roP}8yh^6ft9zj5`q`P&;?9}NEYZ{XtjHf3jhOQ(ayeQR)V zbVIJz-p0w=$QVH(Hd;<`7c+B5V>MfSLqvBMu)m(9!vn4$Y4-b+4_4u9Z0~4%fSIpNNKhHuSQ$Ioy8)^o zuks&|^8irX2R~g7Z4TE#o7v%w;ov=J)jvRl>2J>l$UdMl_+sVybsK)XC0yM9VjAij z^O+d_h=x3de8vVQ{~8*yaQ-)m$oCUS$bE4Be;y2NtQ`;d20H?N-@(#=-N?uUf#yHK zlAVp0lOI6)XTXw^=YTW^uw>!*GfBCTWc~v#kzoC&VENN&`p2VfVq(H!_(ND4@fh%M z|D>bMfuzp?4Y_`Bwt11{I!r@eBv1HfXb4vERmA-&T$<<`@#_DOrUqO*+-wLW{{(jV zJubPBI6cHKxsg~sOjAzeX8cn${psxT@1J!O6Jwsgfm2?712zN~_!rRhH~9URG({5s zpQ0((FUBdi3HbLT{>k(84_q+;yK+d}bAJ8%*U*}s{Rd3phXDQ=vWn|K%lmme;XA&{ zZDeBfL)pa1!^zJH^st|S*YEi%JMw5B!s`$DDkqXL{4;pvI8gBpX_>}GK*c^#I<0N2 z5hD1IT50TR?x=y#Kv@x}L;TV_P&g6aB@uccf@EKRASA`tAJ#xgJaiM{C*rQ(uKj)+ z;*uotzHd6g)dSs;^Y78y!N$q{t0D}JALq9t=%{aRYJ8x3ADl|W(!jXj`rlAv?Tsz< z9nGB&cj93CXXtw{mqQRWw>EdwHE`3lbkb-1Hqk@0|EDy3q;?AIMI&=yb{snNyvF)G z#y^De&(vJG_z$cr#0G-3u^`RFLr~`6IFRIr#Tim<_-82nXU6-L(k?dk7RL7fAp}Pn zQNM!TNE`bQdVfCW_y;+_fnYey0Ss=^5Xl9l7Y;yJ)1H&WyOC7Apb zn$|zaFAo5C7?ViR{p&$_FquOt(|;DH1nz}zoby+pk_&iO5ut;>!6|L*9nEY^ZLIYz zrGB>x{+F2pNF*OdH|N2!e?7X9vpEEBeM3WI2Z#Ru6t7Vq}oHGQT(doh%vlKZkt{^ey$R4ga$8k{4-oA$Q;hHpstB zxkHAd{>Fj$Um0OzKRD!nIBf@x?tg<^{c~=JgM+26p(P>+rfUY$_H6#5R?GJRH+G~I z^DAU4$AKdCmjd<=B1i21w$Ks#4`N3MV1sl_a}#qz{l9GR=zCmqAOpm|1lNDxY4pF7 zVgDfN#D3s)_%|@@AHe@8VAHe3Ba6%G9%Bl8D407{O2cbWX=!Qf!~`6-xx4iX&1vVf1q)Y#hC9w^dA z;JX3Bt3%5v{ileyy_t=bffFc9`v0xD`nP9fa&z*4Jl(JU?gQTk-wzZHc5Y5&ufw4& zB+K zI{0Ra=!ble0kXkL-vI=Dz%91+Hm3GKn*us@cLs)@V9CG;JE$wYxFpf4)_B^AXC5(X`t*#L3|hvNT=C90|#Vk%OL@er2jYK z|4cZ1(;EH{4qrvxfp7W{qeZ0Z12}yT2X+o*z1A-Q#GexlkbrQo)wlk+{Nb{K8hD_k zu`{9y`WqAa8=T{GY`#a#Mc(&dXlDto(ml zkI-+Bryj}854N72<41Y{5)VIJ4E))#`LEl1)o*$Bh`8C;L_6e220&BzcEXj-^c{@< z+>C7bkLD5Im#Fe`AGrO036J>O+T+jX5fCWMG6%YvJE#D&0g~sNfcP6I`9Dk`gd)!j?BXO6)c12SE>~XHm1le{~ZMX zCryxvFn=~SzLzHK?EDAriC=<^13q=g`c!bz{~v1UBkbYBFHZRhmOmHL_lV@%Nd4{* z@sYmgpU$`coON`?7I;vM5%0D_aEsslK>tGwtbf4<@P6ld;roRfz$jKx{GjUJ#KegI>zlp)=&7XT;QW~{2Kqj`jt|LY4sn&Am0brEtKVPA27dhE#dgT- zN8~H>kiUO>ogSAlr6dXpB?=@dcG>m%+(TnMLV^Cfi=*~@MWx`+!PbSv8pFjUkhpM^ zWnn?%sHbHMleoYOIBZ7_j-EEZ$x8XHaqG+%`e#$YG;e2G@4Im4;vj4Fk zxXqK`qXNFnF|2TuKYVQI0t~+!>oz)H%`!&4;Ba1o%>LY(OakD_3qzfs-$5&k3C(fbMP?a44w&c*mS zZdxeP-0y_%E9r?-zwlm!CF%Sy;b3S$9|)=kKAoI;br?e!7#W{%WI3g?kJdI zYKYjus&3rrK5dwNm2BwJc+Z^CIO)268-rzYAbjKJF8;*-6FDk2A+IJM?{P~aDE_*PH z&?6W-NB%$spl;hD}?45hj#MWMW7Qez*)AdX8fxQEr`;FD@*g}9i|4NaQ;eL6UbhyMlB zQH$k|t~r~F^}!QSuu`4X9_2@1MreKkex7)bV0QFocTo^zws(Cb4v)XujB!2^XDKa= zFyaV*j~!Xdb+>z*5N^d84wPZAEV7f);2ur!Amej@mgCAh@i(+(Rr8;3auIRfp+zk_ z6`LOjPbo~eQ6>@v%QAgizN7}9YkrV*^bS}i$!8qr=iw=J_gD6&UwJvu)6wq?Eb`MW zFHu!4o%*meQZUJ83deDtzfx8j1?%KYQTK>~y;`m^Lj%hk{fx+cTF9GYi zkxr)mJfstF33R{8=T`Onk%WY{g7qd~E<6$$&4CP1i9&|3fE3hWdfnb2;w^kjxfWv_gbdft=~v%fkJ%r}V#;Z|MW;PbaP(%7^lbrSPKKh z$M7*hu(?P^e+))gy5&e_IeGMRwV>SGr)9Txv0ZF*#NvYu2yiMY=DqeOLsrrRBRF)_ zJ@$66UIo!ry8AX~KE(UGeNb_L=hU^IHwg*2cm7bx)AqwgGH26hj_NiurME z9xkmYWKZv@$ju5%cch|%1brkvemZ)AIcdsg$dFjS`MI0x+6F55V1-{~Vre+FBC#Vm z*NIcq(&7~b_uo17kOj|91?Qi~rG%PRNiI6EP|Jjjn67LWWchB-vFG8o3!M#z86SH& zFGT*%uuf{<`O3Q&?dUN1hftz`p`%NUVk{2P?91hRR-47_X9)twQnUC1d9!u({Q6y2 z&rY#6Ul*JY;0{WA(mj22`OC7~SkBEA4*qLV$)XHg^IYgu;=VN=-C}%;P9(XbbdueV z^L_3FPi;}>I8o3B91&*$3w^cx*qsY;)@wFe5ZndFo0bxmF6qt8lW6*G|3*2d%zEY1 zBu-+>G9Ow_nM%Vovk!I)ev*)y;nuV@y>2o;W2a4+mfxuUr=~GikA31}1=)A6C!eOi zxu=SG?qYD#3OhzY)|;O4R^?}oPK5NDL>Z?q)nuG*|MK3&eHC~k%9^A=r%HM zhx2QO)4__W)WnMCg3q5^eB`tUJyYW&Pkq4`DjSjfSgg?~Hh+owNiAOU{LZA8?KWf4 zs@2`vQfnHI8rdSJdBKa_Oq5bnAKnS|l-T3a72h0f7z&By%ON6OD5BzZ`naLhvt2B@nfHJ5xAAs_scRc<_nkWr z$63E-Fp6T@((6b{o-aK*lU!=n-+jf|we?Lvk^3I{MVth0cd0SMeYp zVrd&MToi}%us#SK$uDROtA!V99W`j|erH%K(JV*$v1Er&y(C1U=<}&}na%WGR1ex7 zPJ2qW!4xTau{gUDBPYV<`_T)bTkOWQkl-!Hlk%oGwL`7sm=q%yHaA6_`k$EOlW!?L zx^uH&>a%$pjGqs~a?ccT5;1>t5|tMeZbuS)-WBjV7E#?hw#{a)cf6&~^=Pw6Naae} z$+yp^BpV&(-(VHFkHGrYlR}Tx66ihIW|s`aKcUzRs}FWMjmPb7=~YZIP<&?Ly}!Y7 zU?hjg$;ClY|9Ji!+ioG>6K8REwPGY*_)O#&43Vso*H4C8nN0K}>m7UBb^^-mCI;*r&j_AnAESG-QLY$HX@2H|S>MAkntNko zN)@`>p$U&+=AW9<_w^VMa*_KckYjX;UB+r)u@6Q&uS|%;m#12$XPdLl2V?zb20II%5TMj+O*#+*|F%Hketha&=w5=pLraFk!MC?-o1Tq1eT1)#~Pp zNxEELQZ_sWBxi*zSrPNa;OrsYx5+A0t1dcW=#Z+uZE4`f&L#~VtAO{r!B3RxuWQT( z%wg8rLU*X<8J}3Tsy};|nU?*anNQeb@0s4i;jlaPZK`lbC*w{}Q+lYc{jtzZaQ4ny z6%a9_MWyJCaDHe6i=AvRb4F2{QXJlG$@cO%KRG3g?m6MGS984x*E+!5;)4FffQswh zII0#VgqGvE1klSWg+yv?--@?}!5FHhEi8lcweS7Su5uq?>{`61)wb&?Fzm8d?ztOyS87ae zj#{ZEGzmSqw6m^v?55E}@yZ45irY&I=WW+|tQDraB@?$?qNW?*X|RH<_>fZ$>(nZN ztDbAau{BUkvh~7%m@Ks_wscib+`97u%|>yR^IgU5UYs zU`|CZqJ^t-aN zc#GRkDUDI58{bc0iu(%IcAf7K=h1AMUH|SE<5RlMF4^!r?}W@tMu32(7F(BcsU= zW~`9Z)y=AdTf*!3=Dk3+wLeNDbX?lY_PqHAaMo6a}bG{{FOJ zEzxfcH2r}|#!pz0dge4dirSb_v9?7I6172pp;PuI4pv5cWo#EP9YJqr+$6Y>=|@Y} zX>gyIsp(5jKNzh%uwAr%Q76%#{ur|r0K31KRgD3ao>r{YQ z@NqCBq`oh4^TjBj#6RBP?koyV(O8O%+fTH{9^$H~?iNOGs5UJR8mrHb*U(tH>EOz8=*t`tNk*Z8d7F)gj>OjP88bWq>0|3KvjfP7u#5J3}ysz z&Tt=fI`hhi`~>3o<@n~8-UW&*hvlbUJNz65u(gUypI$8ML8|?!ol7%gGM6*nlQ6}U zjO-2eKhw?^Zo2wBVB|>2XlA$Fc$+TCE&ouNar$?#*RPm}s|`n(2R0V5JB(YXO?}T4 z&iUGoyct`WQtHX+JNKZAwSJTHd_+TqmjIa7kkl@B-Cv|=HXqIT+@0wL9aFR z_op+f`gyb2oi%LchRYPo&yU{dEyoCqX100Z70~a|U-?cxwT$^8XYefWy zgq`%UgunFgi>z>Z$ljVzMd0BinxA?U0e%YKCT>r8i!D4#d(Ng8?gYIlm_;~u~$xeArnf8^FDIbYi;x!VP z#+wL}HrAV166492{|-g8WZ->HGyXSToqc>InbDqDqUBjpn`S7H{2-fQhwP@YXMff4 zSYbQ#=nHZF7ef->y;i)s8Q1)|ydk0#B|3BPj=diT#Qtf>3l`7k;Ta|9%vEw#5Zcc*>Sy|C1Z@(ypr?$+dW&Q)g(F&)})1$OY-| z?%mH9N!=rEe~9g3t>e#^X1I>3A!u`kF8B;(`{orm=;W_{?k2TOl|e&NqiLP#n_yb{ zD=z})?%=(!C$e}n{5hcE0afCTwyfHuWieZk3kGReLv@)z;oEXI#YVmP&@oJ+i!aCN zLYFwBcVPFl&H~ArWgNT&7kvwCSD(5N(+J>?K)66p8DODiw7{%-Zegc)m{dMWoX_pC ziU$CG+>cUB_(H66)IP3TIQszyYp%A{`dWy6Z*OC=|M*P<6O-P$JNCp-3te0&e1b9A zyt!@C*rY;EwTnfF_yg`L3djEb~()DNt_2+2fcd`ApU93lTG%qj+ z`?n2A1PG=-^$3u7eLk9o$GrE{5&>P+ooyFozH6$S5>%F*H@KtZF=%$@-xsK#l!L2t z))zsQ96;@EcFhHjPtlKnY-sfXPdzX-q~JV@B0=K}?+wg`+m_4YF9{uctT-ji=k7}J z=_-RsG|WynBG&4Ze|S*E-qtnNvz!pTD7(JXm|p$Dwa@SE%icu}Di=%d4x7LgnsYXkakkW4e?LV>O0cN(fNdx3>22}L zE)=D7PvtW(XrYWa!vUBcDG&|DuQlS)Whrfe)bSpxQ58m?V}zbAr&YJ^zo>)>jY+RY z%76BSwd$5DV#E0o1RU!hUgI4KcRy)7B=j}yAC(c*T6+`SCv}s257R|X}fK7z86ko^V;h)1MU6i-PP|Tw#S7OH;>0u3Wl*vhMYxXf(&>V6t@)kid_6#p7*>I!wfYa zJ2nho-RhJlY)iacccD!_iQ$1;x^Df#k#}pu<;#0Cxtxi7H433UaqW&9ypb?rUOTy1 zK2mHVmJpV+>T;~&$7**qQzu-X=o-W&_Xh_oufzp)jm$*KsL~xJu%oyd_3Ry++nio+ zW-LPxDO0G43^Og1pKkaOrgsX!K&nV17Qi4J0R!|}1Gs1otdUDCZ=#aF*slPOxG;{O z6>MEW?-$0?+!qQ7nip%@EK_ynYwjLPJvWP+SbhfxU&9X{7nX;}-%~B{#^*cIJdq5+ zZ9h`_uEg#N16}7VS5M2(HeWiB4P%`AHByyHY0cL34bAB!v+q@>NF(^DwF1DmUx!g zO!ZQB^^(%Y4d4%Dh$+8+A)!%l#pshS-otVoExE^B=M0Wzw1?Z-j~!(X;&z)0PwL~I zr<~sX!X$lihJiQhb6NGr>|6`G9X0sNSnfInv}gk0RcQ-01wP)LI278M_F#=sFLWPT zNDOsmem(@Ol}hYM%2%J)m3xX<*ULP-r`Zh~vnp><=mHZgw)WKely4ZGiYLtIV^_#N z9;doxP4KzyCpiku^9lkT0j7BT{QOkXEj`}Pr}_(RTx}P5u9tgw!yq-;5%zNik=VJk zg|QFucgtL+=mIC?7(|2}Rzf^@E{pnyKdnDWL8(8QQejm>*+4y4tf9{)u^ubr?4hJdWg}~KR4p`7r^c^>*3r$O6&mDG$6SOAI&)iChwCk z)2jovFrIObo3`YS00eTRd$vFy;!t(4>;iYQ% zj;AZ8<44TZQZJn^q$-@GS|o@UJmbH%7|xvez^h7kP@{YgyP`yr1?kj*{H#o zr}D{HfxYC@7HZlD_M@?_vX=~t?;iLj_OX{4YdQghRHm>B?AJk5;aJlrOBtXB3lwC= zpeI+2pJ`!bZTBRu@NW0-Mc{M14#aEglApNTWV7 zVlLW7{psGEr(mAZ_=axyZ|T4!cvkA3L84ATg#0O3LWyb!{BWc+?Q^tPJrBUNhH737 zCThZ84xW&vLr*n8%8xZK>{r7qR8wwG{ zn?qO=ydO%;?!i-nxXvUKM8OQljQP&|FV^a&4=}on?Bk|BhmvNNnOJo81@;oFM zgg52DG>HI}GW56$mDE{V_Zv{j(r@Mj^g^s3JS%?>B1l^0A!Q+%3eHBjgK6kAm#`{GQ zLZ_Cklmtu4{mzPQuY&;66!IbOtb}}7Z`xXfMP31HXFCNAgf9e(CEAG|?5o65J}{F= zQHS#(uN%JkvR>3R?ckw1GywHIIlf>w@wJ#|mBEVqM@`2NyXO5+ag-U}s#+NcRpJ4R z+?CDmya+*gXXMn4EI_|&1SY$%k9=s}afp6n%hl14=tg zlJULJ9fHpmJOTN#$0!Krgt0UL$s6S;PpF=tZKMq7lHg?k)1=-OMMEXna6)t=V+UmG z0_P}?7&81I0dYttI=z#ZyZnq-2AK;0XYFXX6kbVzwUG!G+(BWN3j)`oKaBevlS8Yb z){6(-RCCytjqoYGbpdDMgMDYR_wr%&+`k=*AfHY|K0E$fAwZS|HOo#!EH*ImelOr9 zjnR%@uB@-Uy1w-Z@1_LW8Crye9U|S9^96+WQ|dGOUMNMu`g4t2)e#b%L;2j*^RRTC zd9(vxGW8RxDdVGzu4aq0*<3D4r!$lpb8n@kHX|xdBw;u z`R5*wC{D;-{(0Rt@Ikqv&65d#6(ZVifnn~`)@Ejk{aZws4%=fszb#PE0~74p4MFB% zkZ;lV>gtz0RruxhaIGa3_=hHpWUSj@p4Bvny}rV7pC+ePGZaw&7`nQZKwBK_tgZ2V zvGX8b`~l#AT$#+ivl%}p7OVMi;=jUHE4x_ zSL{|($Vn@frIo#{qf`=pYYyxq*}H5Zv@BoR*G>Gi?s*Ni5&SJj?^<{YM&XOG)}X0- zXP9se>f}GmJWCSstf?hP%+sFSk=kMQ+WtiQ+I0ud4xd(P&NpCRV=5$Q>hr{V8JOIC zTo72?Fx>UZWV?+PT}PZyFz5CnJOx`SzuzOsaK-R0Vb6`5?_QpAb=)T|!AV@OsVECw zsC#z3b)K_Wk0>4cYNhUDFd|jdLcJU z1W4}PxKr}qG#UV!IVyz<80A9ka4lDDwZ|Tw4V36Fwri>D)D)( zQ~rZx|A}`7_1+fQYdz(bRA<)7f*DyZ?&P*bOXYnWdHBFqGyiPF6UptYJrVVfO5VH| zk_?xM^fAv03&yXWVU*WAb4|IwVBJj|-bCE?X;wcSTO&B6VqUn!g#hS((mpVT-TC$Y z?rhujE~1pi*yA#5R&^kh$!y*r2@Cw#);X-$Ty5}WyoKJ3zcVhrA_ zs#p2f8?L9oIL28&%iJxeg<6Dnwe~5PKBwKryP)E-anGx2w?#*@siViIub`}*0Die7 zJRCMg*5A!q6J#2uT#duds`*KYe+0am%E&KzJ0LR6YCO1e~=eO`)8Ni%nt1p6i_t`HyN$7))CS)PTO5AulHtXsH_9Oydgh4Cn)RDcQjuOYIyfPHCnQ%j?3U)1 zj;va$5I?TIE3rP}5CN0AIEGV4Av9P5a-)h;n1=M_;GlxpT zmkT}_rkCDW>|cs{-if){d?i|!^{T?zFhc>}v5EZivJZrK?Fm^zo?N-n+_oK^GHo}y zdwGAd*5PBZRkcmHeGI+7{Ru5u(rXv(K&($N$SikMmVxPNMe&Y3|0^3)vf;8TiPs)^ z5b>QP2^q`~@(i5cF0|1QaCx4fxV^Yz4uc}e{g^r>Crsw?sEuR=y;hai*yVy_Ws{gh z3Q9^x9u)FoM`Z`vpNQXPc2NC@2fzG8l=!}Sb??sNro2gL<=A#12cK7cFfehcw+ZFt4ttQ+-^v=x!@8~@>6w>obAIs#0P{=up z)1z|%yCnP;L35@yQ}zu}`D~wFPLJ4BNR_(0&=t=hXMJ{!ncXCl-;Zwze{BB2)n``) zu_mow-)m5L^4Q!D=h@(E0acTRP$@NkO8q=8vmrOWq1T??rMxdFwSDhx2-tH;1`)L@ zkEz%W*?lB5g%cK$bNKFq7>ga(A)X*3+mW(hmE$l0AKqdyF&q%gZXl%rUYU#0?O_jM z+M1UmlItZis#y7oUAC2)=LPA)-oUYFT8t&njZX9SOEjs9HqTGu%!W8Qr0o$aZQgA& zd;3b{hUs!~m|V^J$l$fj#qk-=!Yz3eXOMwbjDI(bVgy86TY^LV^s90`Iik8svI zOC)SSOAGT%I1dX@_22l_A6qxqDD5H2?|OJ7A;EUP>&nZFxZ^84I(2JK$8+Bq9-Fi( zf$fuw=nB^=sI7Q)7=9e@Y)Ht!^!?n;lt`j@@ubZ1hq!l7_4Ql&Jv6(9Qxq73bv;lZ z)s*$Zaw2iKwYHCg%jpjn5=q`)O%k7@fPZ)-iQm%edP@I|Z*4CDMtI@0i0kesgWzOv z-4`ZPHiJpx6*k_c4Nj*X7cUp`6j<+=dp!@D640Pr)tM0yFl;V~$_O}2($Ke_c%aDj zn9z-==oUI_L3-oy?wfGn4quK<#ORnK2wZiOrwujBoWBHBMr7}%;iITI#8ue4QbeXfo zmKSv60;kz)7mcs2ksFD_1IND1Nj zb#hK->k3!bpti;4&v1lN^!T_DxEnSFMxs(*%rg~=ZY2$o`_ebI`fkwH9@{%5on!Pv zgbuyN^Aa+GvoWoezO??WUskO&iQNLPNG^ZSWP2^o!aU%1&2!Sp731#iHMKnf`wy`k zb)Qes=!eP!!9caJySYhHxIq~@-|t(>%bGk!nUuRQOa)6feu|^C z@T|Ys1sJ^sxyFs!`L8kAPx(;e=5h}0ePOcQ9o5vnXjdcn;N{%Fk=L$U^rDblG%LXt zZzl7srrQj=xF0S}%rSMPv)9Ub^@gx(6cbvQp$t}$h<+BEbq$p6p>yG7w$N%)sdfM= z6D%F1>)3ap3|E{Toyyas`MgnO^>TXB2PEZGI@LlofiaS66()PrB%EFsu2*=3LPxIY zb=5KP+39f((5+UDPn<^^r%8FFSsc}HEfb$(f-w+}N_=#>JOuPOo|-KbCgYNAXjFJH zw(!eEabFx-Q`<8!ay~<2(PRDVHYB}Ii6<9}SjO<=mr`-9?yw)neRMV7pyBFk>@zRV z;)H3^*nVobgp-gb7QM2z7bvi=Bf9L^n`6CXG*IS7dhS>YCR>|n*urC{`@yjVTg*@D zLodvlt5BV|cp~l&NT<=;Oi(Fun_x~5=c6+J)bgzBnf&c9S@>|r6^?ip(mQoeBMsp7 zb~R8e4liqxbQql!yWzZ>c9|`j;Z5$EhT>90_MfG$00qdBs|=rcaDsCIZELsBWgP5i z02B2cHUzU~KFqA;;2Jzv3qZ8g>%D@CGOf95a|vIbVPt|I{sVr&Vz1LNWua6#a7Tkh zXB=xSi8|`&%7CY~qj*k^+1FL~uC;hPZA|PJd{0OM$%Qi$)iB50vzVC?u;&b19Ii;d zH0i5a;C@Mmeo+O#dEUcjFUO(QK;hFnb$+NRi8DF-LRG4h3!Qn#@GMcjOk-S%iH^?O zA|r7HZRa=4kP~)YWrN$zrL9iRB3%2r;4d*Boy-JM!_30cGuj((c6oF?+-yzDc7S$V z-W^XqO(r`1R{w4BC03x!oiQXn=L51<%v6MFKfNOLrQZKg-@ZZ7K3(&u>MYCA>! zwDl`?mwrXf3cSZp-ybvluv5lY?!~F?&Rt1gSmwI9!!EsZ?Z~c9H;r3-rbdM$2guI3 zlWx&wIQS%>$Yq72@5?$1R57)UPv>TTvpP15!yBGmT0g3Em`d*I&OY1@C|SFeBHr znmRb4;`HM6vYLh$+!GY1kG1#UQ?~C9CZr56(J$Zn+@lE`O8ebLV}EiqwL%@K60!*w z2bKCQPruM0P+Wc*L1sD!#64EzBa8b6_747)vhbo|XkZ-{iXEc$SZY6FGNFa+gN|H; z23@yj3e)Sg#W$68S0<*sBT^D-n;u!EYv=9wp1=s_vO31k*_rOs9_kUvb51Q74vy?Mq<@k9sSgPU~q0wm@N90#-big6rx>pAa|jEd*Dfc2k?^JF8Q4nGW<||?^Y(MLN$iq!)S(eN zHTGpEN!W~Oh?Qn8^yAeJs~A2Sysc>;d6)Zm-Pm3hli8q4e8p0LL!wA=*jgtCl~vbH z(X^9t*ZZOs`l!s&mHHEQj#)8hM`-p!s5lF^S9x#mWb15HIA1Le@&3pZiuTlEw6e<> zksy{R^32MuI9P@3>2t2?l3IdynX`O^89i^6$J(=(95(o98wEEuPSOIGMU?Cv6Q6JH z#>-4_qdr%$IlilH8xHYIz44Q40Y@1UzJDR$Q&J;*K zyj1p)CvRBkEN|>W7{2jx)N{QdNNGTT+p_W4O5F*qSg7ef%bgKtg8Wr;<>A*AqhtOq z_KgX;O6T!Q+iRg%YGtxoX`Bm*12Qc|5FU^A2Ic(Z`q~8NYmX%oyV3I!&)D?6EY_?0 zue#cV( z*B4${UX_iuX&Wy00{Jbxr*`)jrLc)O2qlZ(K7P}XRl(BgKh#7aa?POnL0N$*p z9gjWOqF@thH7JVca|}?YmNUFvh-2RRJP!xGT46d)?aKk{mZo#63R;A1uwh;y1@$@H z^UeVTGHJQ*?e?atD5yEmuV}rjcQ2fY6-r6l(Vn{<^Gc6>PIHVZrXl27;re-KOW?+8 z(kB}oYJW`+k1(8ErtAJHFP0Oj-!Hh+TJThQhh#hrk8^G#Xr2k3U+s01Z!yY>{j{uR zX%Muyq?0hQtC}i-*-jxaYH#CdfA{F7Czc?Q$1Z&iq7B2S+Q=%B68Xb<9 z5W=n?H(0ql6-FT{$gDoFZP+^Y?%}n2gtUy*l3V@lH)@aA*5bMun6lA%aOK!svv?_pBS@eU!eQK=s>PcgIXrzle*w-L;c6*mm|a4#pP#L z_}sPxP2n${mp-_?j)FZr-_tah4AUf~&&WTA$^yaX>2oG4@$leLv}zZ1U3#ajs0g-K zL^f9}9#d{UtDO1}cRVVkQaeYZ&Kfw|oF%e??$SdgvdQchfk#<7G3%W?6-4?zZFl$* z`c7Ns8yAzwYQ3?~bDKfHBdGmW!@m>Z5xm=hN2CJBi7>h42#2SH3HBwQ7KeAF$y;ok zojS4hFky4;RgtRS=Bt&{z-e7Hlp9hBoI6JoiR1ygLeUN)0k|hG%c}Nq317=v0pTcH z9PgkQRy(>z_UYc|A_za=nIHK9KYgHsmKSY4rLrzLRCHHGgq=hU4SN(C)*QnC#h2Kg zq12Frqrxp~S%GUAGxADr^lf6=nV~NkuIpR(NrQV7S@i?CFi%tYYzpP}54^`1EoSwi zQQz?`@G}eV^MgH=jccMTGvO~F}z(QIEdgwCFjR<@Mt=n44tk#Sm#U+ z!7G8)sOfh11({*@{RXcKeC|Z`>z4JvUJNgltlzKf*P!mxr?|*TuP~jmM;wLH-|7V#~xdf zkz6eOYAMl>RXd||)zOxfv!~nHpIb>thLRHybS=s>ynW18Y$~wnRv56t_C)cP>wJFz zLNhQ=dfbz55*Njx;yE-Dgh|Q~^#z_Hs|cn|M+mRE102yR=;Q;BVf=wY{a-*hGJvKU z9~>ijv)oJxEHh2G=g}CRo8&L2MO54H;}#%%d~WwzXH7>1e>&AvEgTX*>=V~UH%NAk~v9ZI|Gul+%`>Pu$mXwy}( z+#61(s3m1n8e)$ES1jk9&lY3HVnD*9&7eKktN@x+#s~5G<)bMiSdEREnK-yMIv04o z@Zhio5qyDZmL!g@*B>rjW^HQgb2)ZJtv1{$;Nj|C=$YL#^q1$@A#2LVg+Tdfacwjz z+=zPcNO+>o^l~`G=H@Hew~notW*&mYN>E4tt1GjJTw0+!Yat+)R(Hsfg#;bUuAJ9; zgQzMA#sDM<$zXYIR9w11*jV)g-e)mF>6y~99cd>iY?pk~wA*5MWXKbQi!fLp#y7Y7 zUl{a_yHnB_a_)@deAuU~{aWBVvo&bv*>o%CAUefyd0Nj%s#VPowuE4Ag5L1^c^8E- zr9r79V4!@2zlX?ftI$Godgm8Z@cm;oFBaSQzpj@hJ2IISF27VGBx@kyKic_eB%5Ak zMA$rKXP|Y3W6egxk6+?uV|x&=&K+cMN84V>H>f_MKi0ZlEE0%uPzO5w5LlRtqy^Ud zAkZ|?*k>s#8U*Kh;&jW^y8m{`L-)t(+vJ`dy=8sp2ucckG~nG<9fF!B9x>59`E^HH zw?k?~LaxODYa%INJFbSxB<%S`7s=fhyFkqmDY(hZw&;Fd*oM~wJ}|ir+#dwAPXNK$45c9X;%%!Fjz)>YyB#5eVb- zs1sidAH&j>rD;kha*f22(Dp34} zsPUD9IJPkL_7Smb@cI88S{&{Qg5?JvQDrQ$xoDWU{Z@_u z!nl9&N#@Xf}6 z6L4sy|3emHyk*&o%mw(%Eb9Ljg5(IY|3hH*ZAbjDRyB6TN2W>6V3GJ&K4Vk~d9S$^ zh3*I#aEhMK!Ew9&29bo3p@U*;k5=wArDXq+&$e>WJ7XIaobPgF4lewhaF9VFuD3>i z3ChDTE)YZ8qN0SQJGJhVk7HK4e14@F-Ee2DYOUhr`{;8iu$ig6ro?a*z%_8u4tT&L zHc%kXVlmFE1d9`Q+8Q}F*)2jBzfpV=b$Q9oJT#1Aj4HN*f-@SN^Ku2mfU#ge1ET^F zP(O6D&e9uwRT8|SVCBYUi|&@9?%w7~y>GlZb%4k%x~PUKIx>@N!J0Ujkd8+0>1srR z8XtmXAvnp`rEzD(D~({1p23<`)8Ye3-Xyrvi7D1|6j?QlR6 zU<(RnQ3oVLBm`My>C&X2h^nwL zKm7>eH3uh=MO3*S$7!LFW|#yGY7hvGFiV?k6VO8irY*#>qDtSgHzbsF40L5h9S)K= zT8%?cBX92`=pBlCc(W08{DhZINpPZgbN4fV_W^L0no%N%5&4IS7%a_khzlRBS68Z9 zB1I9K*xucC0il!-JSqwP@pQ%Eid)r%i^F7H zhxrn2phk)^9}EBODS$GfKoi~%YX(zF_Vr;nV=UgXmi@;WoV7s$76 zt?BKw@c+1btFS7&?f+X46$D8UkcLHfhm>?9ASs<9BHb-5jnYV~ARW@3(gI31(%lXJ zdE@i!$G!J&d-k5Z*X4=pV%=-jJ?9*s@f~BZ&9YiPO9kk>T6Ef)LNaM!?zGOuvysd4 zS6u%@+5C8U0U{Pyi>RT+xhKsL|?9MKCwh7gM!h#`x z>+qn%q4=(+blX1Tn-HzG7 ziu!qFy*GACmT-sZl@5PE(kk2QfnY$goar7&+Zy0E$CGe>oSzFW2J^ik34TJ;F# zF|v*AM_W%awHUs?nz}bNrQ2WP%=0d92q6bYp)pp0STUoco?4}n)TKPv>pznQFnOOG#fQ(UcJaM)};TZ?NK_S*EjI-QSq50Kxy z%l7uSI3t%GvATRcQf@&pVH*DU4ZFrnrRmM6%xFT=q_HsaAwa)LU;BueS~+tCu*@x`gVG=mv8;TI%>)>eiD< zcq=_0GXtTYh(Ra$KAEuAH;L%l0lfo<4Hp;}L+&6T4i@#E#G4p;g-}7X-}Rm8dcUN& zYs7MOF@FE^%GR!?lRJo>Mp;wBsnw2|*|`{x+v2rM{I^9opo*VP0rB zDNHZ8jSg0mf!^7RWgZt&n?)wSh_vQoygs1NVNTV?{z;<*M?pK(UPYn2z%MNFDetIU zo76yMug!O@IK$~;H@mWae5kv<2Fbm@aN`C{IudpM^o!8WU36=LVUce*E2^%mG# zr<@{b13wf~jZ|`VxG{ulC|wD}w;mWra#&h_H|tlHHryaXVs={9H0o9S+IQ*;i$CVC z@|0OL>h<5VIVXHnaj&H9up#7!PKEmh{OOmx5YAevnS%NZJ-fcPs9HS|b<<8obR7u? za??guRwIT`wfecqUy^`2dFUX5q9TjJKEA|FYM(gaylXuJ(d3?`%}sP$K|26t6nPFPK(d~(r}15FJ!kn=Xv7Nn!TSwbc4F%zGQBr)^g5DV z5s3xht$Ful=6yUA)e| zP}(|YPmcD3?Mnt({C>qUhOqfN<@#kZXtK{1{9L&TW_s3QCEt@iO6IQ%@2`DG4$@!% zkw&|vb%F=s7hVss^G9ePAl^>HSWVLF2$s!#8;J2u{8%Ers=&De(lcmG-nfYouP)4K zuYPBJwDkTg`MF;@_G+5W+9e_(Wk|}FZoS>z=IZZFP4CMu^4!lJaskN;#e_Ficq)tX8 zye%uyE4*N2YZ|`QntG^Wq0h0*X>T>V8%{RRJ){SBWnm8OAv_0LBuhAjI;ffwy`;fpwu0^Q3!ibdCigaGSsf4I zva9W0d%LXYo5+q+Vk6ra-wISTe$qZAqFW3T?4ezZq*&>C&q{>|(wK}*Q{8}nxzVy`pJo`^_BiMN{xS#LeE@sIk`jpM}D z_35iu|D?I$=1}70;n~_S zlNDnMroASbP#Wu(qUX|{ZkPLTz5G<^98tXw|0X5oN$#knbB6XRBdoAU2EVB&V zoNd1H(=g;EmyNTs+T_3sm4P+1EKp(4g@wa@(4Z5NPabwMUG;QQw#VJN6G5V(c}1}@ zLL^JvPo%7x9J!<`;EsE9T(x1~LgB)qT<&URk-JqkD!V>Wt-GbsqN`v3+e^`444h_O zwYRU8%T&!h#|29bu_@7BdNVfTkhq66sqxxnoD$yg55u)EdTALKykzN}@wh4q9sAa+BcZvY-iYwBnimQXYDARkGNtf^O z^kVDId561i@HP%6g+h2fInM0ba6x^&paI6_R1N(H{oa_wtMS6o@XJlja`cjRyO>d# zWKKHGLmROP-;Vy&=L9rT22vyMNEuixHYnVjj_%H%vA3PPJ?mL`gqj=x`3UF0his@1?+4wC6-!1xg2kVnt)3W$tSj9tzuK!rqXu62 zW!@JwWD>!%^GO~*&p=L+zp``OA=P?q-B&~;8BbbHe|5ehQks(hhrMPmx)F2Xwu!fE zdLHN+C3_Ae%CA`_U(B!X@E7+2-Njt*0W$8E2fS(cMw_pPoKc9UL{J#Lvy8!b=O`|V zBNGUhC|KE9#Yt&y2lk&XC1lkA!TWz*_n>4PkKL+*c@;kGT<+q9Z%()9MyAhfJum9M z*x7R5SdoluecQ@PVGrJ?woW{wa7(i zB?#*!PwgG{$kezTA`%PMUFEqUQc=OAF6t-mTzu{uaA=Jx|Ah96+$if?AQ4?%$lhg7 zm*>kXDB|mMn=`%jP56^E;Y8=aCH^njaSt-%bo^9m1`?OE4jFou5CZ-~g$uWn z^P=v=k;y*Cs}@FIfY?R%xsgb@L7ooTO}nN0WJakKr{$48Z?=AUWL>HXi<$g^_*}{A z8fHO^$q!gkz`^O$gfYfg8mx22>Y_L{<=#&w4UyjUk+qTfRze`CI?3Zn(^z;>UgC`V z*y8H=^4>_bHHNU?U7s)!_hid`-~tUiy5Jo!>J0w5($ehcd`xN89Sq`)AA_qj#K02& zVqil0Aj+WFSpJG!n87lgzN+E)B6*zcp3vIf4sQjvrLLezZ&Nfnqp+y10{t#Ol~MJf z`)SU_sY0?qUp&mO;4Mlh0zKsHc^6063DVi!h?k4-3PS`Ph@yFqO4Ocx&>qC2OKynFDa#T z%-iuzvOx)$Z!$rnX?zqNs1c(GNXvo1n%!R+SbbH_eC43FXMR-Xb;*)4^_l_uA**-s>@Tv-@EYz=*p<(yFlwVv{;mU~~OHEX3=4AMrzC@$if0 z^<1|}h>76|h=(2$$1?qy6@csg{&$K)KLmD(nXw;qAZvxz5H2&)F%bS`&MZTG62gX% z-T#jM7BG76v#{SpukZ+sGV)?C3BG+jNu$4y5acE#FTGT@+uyy9pMnq-UYpWnl(b{J zcdF70`;++&(&=4Qh!we+P?g(IQia`2JSUo{1Mw*R#?Xeh`RdagZYnH04!cbheiak_ zI=zQ3+S`S4y+k+UkH!PVH;?I>YsaswDWxX6VC0ZxZbkXuY-Gas7wG1G)8O;y9ZaIg zv>FP4gQD8*nY{lid`gcQtTy01XmdvGwYB2-aC1qSc)A+IVEAaej%vF@=p;_}cN2tV zp5iUWZZ_AD%W9}XQ^#Ar);X0yjBid(HYwmpG=_}uR$#b1ATPN9v328YCdjZ2VLWY?|K1w9!Q{|=SDBn9Az4* zGO{ig=rbXHFun&iO@Y@a8>}lZk^jQcoL~tRW2mdjKy11ZB(k`BoA23sX{lmtjiKw$ zRzb(IW3j}q;Wb!k|Dr_Da6w_lp$)~F&^RI9hLcueo*7+zDjk*gwOHqR>>$-?Z4xsX z^dE?Lf&22`?A$99jL1ubm>*F9qzf^F|Gb9lQ~d<;DvHG%K%aTJfWE9Q_+ng>cEF}s zpe^YyTK*D*2O5zV=Egwi{58mSdLPB9j5j*Uh^{y{7W#}pLhra`rFdtG@Y%(EQK6{P z={;_Lxuh@#8Pi9MR%NnKe{X}}P$lM-zuq5c_~O7d{$3(8p*!!1WRz6Qc@*p|K~PHi zd4U0T_Y)^xX!t=;9JC1&kp1(uhPw!E$dOVpzTEMsEQT`{gOxXtZ#QP&AJi+(XmFOj)zZ-%!Nk)a7@8W+2vVzZ~F}L{> z_dng>3Erm?K2L(a`4adcjPUatKSxL1r+4Fvhdz5pi9Rw4>m{RkFMb<5E7cZE`YkNk ze?}}(hz4ReQ7Z_&N$utY&yZ}eMz~~8Y|((VFZeS+jxH~- zB9yEMK!5*sY|vpLK8IomRsDNbufGDZLuBvK3?RHBdN+EVYL`34%M&4^KPyjteiJU3q zHS{fr-a%KdzuYx?a zCAWP&|Mwmb{$wQLS4`qPxRPs6<^KVZzZo7NU)cHclfvD@!d0R8?4Mz}3?`=T`b|Xp zYZ+W0kz+q-j>((0ZX~&hbq)V)+}WA~`6`-lr2WO=|8vxZ1R<=QA3xoG ztHn`huo3#2Uioyub&lG7TMc7{|HIM*v52~pH}LR~aJ?F!oU}JHNGX zbU^D`I9PWIII4%O+tAuU{d?_PV=Mcvr5MjyPN}GsXQaAklH|_Exotn43MZY^i$VIZ zKO7n_qI*7kVk}~UrSiZNC5GEjO4(`{Ho#^Cp~q+V{B9M5vhq|~`<$3n4OL>K;6IGW zxJvZ6s%vcCWoQTO`QNDi>6fRdx=}`}Nh>mp*ExDqm@Wv=%5c$g%7FZ6Ww{FC>c4M@ z*R z#H76UnEY1$&B^oP z&F)qEl!lN>QLX#2sbjD=oPne3#*n#Quy&Rn*BkH2H=bD4`0G^;YpoBSZGRqlGwI!Q zw9)93N?^~|t7Jwja%0U@s)peiGj*hMZ-N*T!)VYYX1Ys#0 z>@P8b-!TEo%PX!d7!_I$Q)}` z((A3ll1!ng{)qNwbu9CKLg`k(fXeDzzqanN=cVty>gcz65xM&`ig@=nTxWalS+)e> z5N8i$s$I-{9xl*HmAYk4Kkkg(q5M(Ny@r~wGIRyBTPY(xV~?1Sc(+klILhClqb1)jaa-DKwzGmsDWcl1AnRmc&h_V8Whh6XC zLBMc0t$z7$x_u$$BM&p)o8;%l8-K zK9RyPqUiT@>yDesG~s#fgeVJJ2RW}mvv z3*}oJ56hZQ7X~BCt<26#OazoZ7K?`Kmq5`2e4eWT&->H$v&RuLrA+Ug5oK5{En=%@ zc6hzvNNhN&?~3Co<8mv|7VRJK%n+l(`Eb|%V%6D+-cE&`YhS6RpAIXzRQ^YWKJ96)Ba19JmOAyK^A7Tz zJJ0wRXPVEBo{v-o^q;t$?B2RbU`|{AINF>;F);)5-95VO=UO++oc@WBAgPVK)}9lfOtBNz z>2G!A1C}+%NQ&iRT=!{Iu(58BWf=5)%$h2EPj+w4rdoXC)F4;c_FSco@&&tHgHpME zVRxJuPg|O*&V7bGF4{59W`Er>q&(XHi})YH$qTUs!rD?x92X&5JUWHY*+W_zK~jz@U>77 z?wY0%;4Nuk=I5uZ^(6f@!yC0il~W!2nJ`wWqZ6zddpF{?-f{iXf-cb1yr}`Hq2*58`eGJF3hih=<{v&i z-QXRO3liBMuXZ+Lf4SzX6uoNjW1?ug+7o#)pvPY(;9-_hIY+0OWe80(FPU~5ePM^< zjSQDj{U42If%sfRL{s&CllV9pVoPbX#%@`Y> ze#GHcUoxL4aGNo^&huo-Ct|>Pdq021|A^qdwqmicMaEtdKhUt9YBf~cs*XN+sat)c zYr(fL^7~wv&nXB@X;ivxW-a_n&L+?6zd-gcjOd5z^b^hk+ccw{0=E z1Z!OH1=Qt>FJV1$Ajq(O*rjZA?l!IX$1QM|Vt36Jj@z^-{<#V7iTN$WH1*gkG( zDIROUi8HMX9XsRuyUJ?B61;Tt(U~#Ptm%8HE7Xcb^3*z@^Q{--!y@jKv<$(HlTS|H z7q7-C>0=fMFm$D+)xI4FIIFSVcC|vD^zUw(k>^mXbf9~eGle@SVW(su7e!A|mTS|@ ze)LxGr|DAFt!;H?IHnXq@m0GaC%ZnJ98`gZcC@W;i(B1s)eEdOAPgn*@u48W+K46Q z=b^J5Tao5UtW)Y(EKX~JEH}G@bL$GbEY$Ff@FKk^HUal@uiJ<2-@fSU8xt%{QIRu| zN*SKAw-wix10y*CX8#cPsgx$4J^J~DU}K{hSx66+>N|09hQ=Cz-{1zeJ%87@_#;Ok#a=pB!{Z=V%l1;PY9fuE_ET=UJ-nC(g@OrpIu zQn6LL!Y!T$meVz&Z@z<1w>S0}eN5BM;-ALV?BWj4u}c?1uP_E#dS{gp-O&pOpX=&_GiSP5apeyoRyNi-iiH*}o&?dU{ z;Q2@|o@v;6!;^A`X;nwuwfW8yL)_s{EHUeqq1bEzbV^jB%G-&{69gR$^!nqaY@kV~ zEGg#8G8mA~Gm1qFCmFMW4c=o;~+8Z}I$Qdy}>(XBC*%otk}~Fha*qa0;nsC^$JF2c?ZU!z?~ZpWNn)I zwar;f8a-M2c2i&HT^m676ogc>zH1|qSC2JIlOAN_lVSUrzQ0$ zuM11AY53n46k0{mb%&1y!=-5(R=^hmlaEmvb?&ilurumXM$vaq<}oEL{Ty*Szbl*u zo4~L@tuIB-ZCZT$fT!T8$!CuB8`Xz5w>NuaWUbA7WK2=%0>*w)2PTu_keYR}3+}H8 zINs-EN_oB6j5D*@ljC&0jmnFfpY+rF#yO5Z@`QCZLeY zryPy13G19rml^%-^Syz8g$^4QT#6UTV#`>%$#eN>uYb4`#g#Mp6Sb5gX6ao0sgbT75;t2;+^0! zzK$+@-Aqj1$;=Vy5;4)6P0Z)JH@Ye`U!_!NhQ)VN*$+ z_C75m@N(}#FZUPla=!?QPvfWPK#Ab_ivp+L@`2HOt`|7-Z0fbQZEDerY`fdqIXrYL zUGV*Bj{5PMe$u6^ zp?;E1n_K**N8l?>&CqDPZLEgW98Ww@P(kMe>+k0T_E+F`2pAq0J6yYk=NaLHtHZlE zYY%QWJ7y5jd;ze_kFTYIz?|kc4Ev>hp|MO(dTbwq?gv+-Y_shu@;*oqjgcS%ZCJwT z(nG7l$uAnl|6ZLZmKxw?*et+5QrCDH6T-_>YVr$R^|LIjASUX<(D-;SwS5EEbshC2 zB7E$?dWZFoyJjx|&ptavuO9*#p*NjcL2=ammEtxpsRnLv2lmbJ`Ta(AxIS8GeUr;! zMjQpB0>iJooa>KMMz2w#7)34hE)(%x=2iX)DCuLTv+xvTgM!UI{Js4~Zz=Gxz2BgN ze>(VP5>pf+)Pa67ea)#(?P(rIVLW1@JkWJkVa9 zDMi5q2cUynh_7 zrz*-eLu$*}5iu^I7Wg-zu@ zbz%n)L-ZYlwqcnEJCB*+0kN|J8_!*wklAr03K}=(VlIf-MZ^oemc8}5+0}JWX7Mjn zE{X?r4+x#3{zkX#067mJ(pO(&aV)@6WxYQANCUZVzZUCWKLvuub^n6KQWXFiU$-0L zc@XS==xtnn4OQX6+#qe2@`8L+*cyk|xOx}h>W+VKHPZ0k;|iMh4PbUD^1}6NW-JTH zCM#UBYJgHyDOs1tH7FMVO+EP+DF1DRfnR$F9&CmXGkHy>kw7ab8wOXOfGo5|qt}T3 zD?s!x-G3o^m=dfhPwK2AupE)IDZzux}tZX%d1Y=x^xHu1DbnjKa@4RT` z7&|+B*DI)d4G5qWSTkn!)D%*5#p~CUoGh#%F1sYU@y_8gE^T0n)WEFaCuWz`m+R^r zf^+aoU!iBB-@0Z8VBuXaP^b%$V2$=Gn;1UnBaveLs-zxe6hXdI@cgP)Js$Ti8r3ho zKa^S0SH}0Bp7`GMju1AApdgj^kIljj>WqvaZ(iGwE3m?i*M?v%@N+=Qdyy_#3mPdC z_#XXhzTAy~@*{JaFh(VFnxfU=xm4w54I*+pG4Yp>hWm`Tu`mN$u6K!`UyPg-8ec8I$p5*XRB_VW z6AiwNaC=wRW`;M=YuX1dCmf95P7rjgNrN>S-O$|+ig2KI68L}|pTLU$B>sUKD&MAv z@HKKa9IJ#c3{-@$)m)H;(0B*5Kd@eluxfyLWnV~lkQaeKf=-$gwAJCW_IPhPoX4D6 zYB!kz@bWAwaOre0NgxI}771AHkicm}%ILbsQle~M(XZ}Bk=c;kVaxY9S-Ui*kWw>M zm8bj++>KHd@V{@JUvhMn=KfM@zdpEDYO{q%g9*koN&Nk&ORS=(6MvrTZxDCGlgJ@7 z$n7!o>gZ9gIzA5gnFF4sLLoKWK>Nn#h`eqTnR$ zxTC}EB1Vzq-(eVP9*_gzn3uP}hW6j*8De@bUY+4sHqFXOwjqO5d@`Wh*w4bDOB({jZ1l?uWcg^cst>dPr zI3_etWg1Ms(lZ(MwA5J`-A;0roy`mL4X2qh3Kg=w{WhTWwnAvrMMcuR) z+vMt!SkPeF0qxyYHR)VSFwVywyg99!4fp|d%?pR5!S;RO;``a((&ulyEl01nYr`Uc zuc>!1wN}sX@zwrdcJfh;`x#~l)xN}*{;NP7!CB%1!|#XeS_RLJl);rr6S^{0kN?N3 zqQa!c+|sOcu+xyrq+6HjP;@`8WD_><%rvy{U+0!gTf25dQ+U(-pk;$$&JFp?a65(!OUBPlhsH$i=pj3=CMz*L@D+OIdA}2X6Ipi zyRpTUYkt;Pk_+o_O)xTor6zzi_Mg`Jz_hU+tmA5Tq8|I1YG0eDTFsN18U|io1yLwp z@-@a5`3Gb7EdK<2%(p#Vird|etN16q7voJT^7ksLZiiRC*d^^VUz0;6BdC#9%U8pS zRq&}VvVzD}b4PF|J}@SUGYNG*jp6oq{wSs!Hw+XIBh&3KUS3}R>{e%ZLB@CB>IM7} zv)x7RU!yW^xZFzt&1#7~FVrA)KL+I_{ z#glB+k`zXi{yvb{U>gKXPHfkC85rke7}f`k)H?GJP3?(MnW`-~O`FFuTNrVLBvoP{ zVN#-&^vRg=v(y9-@}W+)a)vuQ0a^IN^0l1O$yF8{;;DL(<{^ITi&Y*c`lQUSre#`w z-%{-reln`Vg9M0eHzb~Ya9_~udGh5I-3iXYFrp(-I!r9Vh~)|9F(_1;IqhY%N%;U^B@Rw4`m=H=O!6N2^`!`AJe!}(`caD2shW*;bPH<-D7ix7s|j)hfeJw(ILFltN)Z^0!VAQ9Ea`fty`bs>eY8fDLur+Tg(_~P2KY_MRM#SH@&S+`uCuHwV zXt@7^;#O7(scw~kLGrY?vs=%#AYazJwqOIftO!yB>Le2O77SK_ z;O{H_nMo8NYo1^2Gu4@Gnp$go^S5;p8iLo>WtS-HRoqaKV1aS2M zj^A8cYG!`yxqAq>p9Z!W59l49mPr3-&9=V<6y&I&5{9Ur9?5!+sl**qH83)_8FEJS zF3adD7Hl!dT}Sl~8qYf2E;gr|@+p++75(1M zy%Kr9V?X#Awrno|j2j>?q6Rz+6WVUC! z1)WD4NGGaQ>4V~K(z$*zDq@L58IhbKCcX#1@pCoC6t=@Et5uSZDAp}qOX1i&#pX*P zvZH&AMXLG4s5Bbcj4t7txQkA0bCR$0gBGy|^sCyXaue7zD|6*q7<5tuJcLSiW8w;* z6U!4#c~gaNrU!SSESTcQ%C&xx`PA)2Rxq^`&VJ+7=q>b58 zm>4FX053;yht?uLSr9cA^~bFBJ_|sNs*58Hb!b(b^tmA^e8vwm+_ix_B2FN*=;f9L zeF_gjy32@}UIdIGnHTZ$GG%6l9plk3gID`&BhM= z>G1G3zh#!*q6AZs(_I;wAMBhjFO4*iHVvcby?K?}1Tr!)1F63jpFJYhS-DVi*@e~O zv^FlU`=G+`Fm{0_OK&z(%zB+4c8U1R0-KvkpchQ_-Z$ z1u-LB`7VvU`ZvI*7tMgIGe`v6WV4w0QPH!#VQBD73I|wmSugS4q<@b{W0Z>Z>`ypZ zqVEq&IJFy>lMf=u>|GYRu1ea`Sjxf)SIZq<%k{$RhnX%hGj|`mglGkFMziVduD%5c zKsm%6a_%h!91jJ{2YfP{CA@YBuyL;N`~}xFevy^G_s~~dK4w?VV%tF`m59ToMfH?_L7eXrt6V{-hP05 zeWD8tCNoM*}HFClSlV)_Vh=|GK)>*?$nGE=}B-`pQc)I%l)H4^jMrX z9-q)`5p^s=3AI5UV1R#^opHJM2JqBZAYnGLiae-sJ|O5L78KH4dl`sFZemrRQt#$JU7Hw)#RR9Mu6{WL zjt9ibuCWp-v>i|hQY=9@3}=YZ;mEyqXSTVTcrKiX;bCscRR!JbC$z1ZqnQ{b{x>Pq zW?Azsi>OFgjvgO({?-Ces4UK2&GJvxxnt9_wK;g75qhN8_oWETwXz9j%D&fO;T7>D zvXz)Sc^k0I{Y?%}t;Y=>9)5dwlpH1%dE20l^n`D(IsgnH>8pujbbNIvxC97qg4oxN zU8kseA(*c~rP@mYRr8RlfbKE>qL+nfUsfdTJHex2MDoFp9b`sr_;znYXDQIa0ns-c9vh{8 zGG_^%)PC>*ojEy$ZHci6ob}ES%g_WY<4a5THxMR$`zyg;Vbb1i2Gwg{cSq#fPByyA zirTw(_r8`T*Y@gyd}9Cin@D51nH0J4h+_T&-D)Q`lU9}RpCl$Uj`>NsT;)E~`xjbf9*zs>o}>c!?inDd+(V0u3VrisD9 zb6sZqPjkMwtmCsNb)T}sYcCcejg;3d=e|d$`?c1gf8;5wnx+d5CW}+OA@eIQkS&$V z18O#)SRS)zWk}jiH4j_~9oN$KT_A0L$7pCGAUpKkvXU=3F$;gPVs1FGbGq97eKgEj z;cZ9=Ko@!xQXH%sXsIx>zBuv-uVfI)Af5PF5>potIE?Gkl}e&TOBJwGh=D$MDz^?& z$dCyU$-^s#0CEZ-{p&Rn{}ufF+w*56@INu^>=!FagVYh{Cm5<=I+}-3!Wo6i;_4A z#UA_@lm?w8i}3=Pt#TwW5^=(=oejA#sVV@XqbU9Lq3;NBVQ~`kx-u}TNWObp?eyUp0HY_V@?c@k@Jsy% zbS7Mv?!YG@C-5y)yVaf$@x zjlU2r=AFg<@pKsbfffa;gXYw~0pR@1K+--%DIt7tcw1B?e5a%6fVuI-A*2%kK5eo3 z0BtOb^m~tln*x#l7~a*7(<8&3HbX#LDodECa-o9vNF_i5`p-35g^*OLW-4qL-^1Rx z*=6Ct&!oD}YK8KTH@(F#mv=vt`%&WtzEAd<#$S~^ceZ-$?N%Ysw0C6K8Z-X~&#?)| z)!hSiSo3^D2>%1%{&&&BJn_94q0UD;STkJU{NvBI{Rmz$lTJcL66J;899Jm(-(LJE z?JwyQ72f!2R`X$q6BR~1VQ<0A|8IxJsv3i^R60yCitI@g;QC_V>~)6DUa++O4o z@YtNlk#`mAb;{Ns0doJpl+eX_Pt-33zvQ@;ZBQ^Ngscz3=NxH=e=}`L<}V_izf;O( zPT+C0)o?aI^DbLUVXua0qVhqaT3>5sYqjj+IjCU#o&H}sb(N!|(P?5hINh1=We+dQ zjkw?iJjK^5cAg$#VNiU@JNsi-KF8ASA9R&^5c$GjX3x1Eq9%j4_zDL+^!7 zWk*#616K|Y=;ByDaxuQYz!|l{d2N_d^fmr!%_oW@!E<}iL~h;ltdZ>x3#z{B0ZNgz z52?Dyig@yod8}+(vRv2BPV=Lu=HDKKjvYQOecoT_(D4pEWjsUi8I{%{um3Fn`i%AR z<}i{kKXKp1vm)bzm*rx>&t6#CS=3`^xf5(+p_qNJW!ee7IAd20Pwac{SaSbg~aQ=@xy8fkgaO ztPr)2q_w!s9hz{Y?D3d_l4t0sSHu)rO;EC@ED$rMB-4$u0et3O-K3I262J?p!l*5D zig%*L=hyF^pg)rt4J66h>dp4te7Bu?kVV z4CC0e!p9U@0sdAR=jOIvyjL(cX@`bHW1p>y~IR90oC5*8B`8j^9LVG3F zo&NL^HRv>+1E0&s2MwBC^fVAIWRknQIYzNpKL3r#+Z0(j%Lya}Hv$ zEvLgI3!_o5!CWohgu4nSX8AWM{4JHWGV%Gti;y-er(qXa8?%67PcG3F+Jhqf(Wflyn*6BYw9m|x`pXz(d#3{*Z`r?@t{@_jI zG99I{WZo*|J+1dFrp+1?r(3N4cG8ilaUseSReOd9KhB-6MQAj-a(;c^Y1L#7ONk7u>HhdP1}<+1K-I0I-e;O+)& z(dl%{wYzb(I|NFfdIG%~gi7<03B!8xg@n95b8k*myzUSoC~^uwI~`LKZk)sJFKs6A zJKB_!nHBKlg9_AqT%cz$STun!`qotKeOzr}sIq*v`G@J9ZGW_bR3Qd;oHy&%ne-XN}e1s?UKV>{;&kNR<+ z%7?pbZP_%yTOzddc<|lv;Y*D$;MX!}OL!|j+}=O=II7fO+M-1!0pG6?u>!6Yp)rRb{-2*;O4WP*ezdY3UW(i5h>FgcsH z^#fDu_Tf90y0t<>e11FKTLP3Q&wa=~d`?$%HC-7lDtfkd2x?b~tzyymP?^Qp~ZP!`BD+llzGzv~V*PtPCn`Nv`Qr2w_#`n_BDP++Uyx%)Ay3Hr&u1H_iAUq#7 zt?56(4SQd8T0tgN(tgmAB!?4yNQHLc(>e(1_VPR1q2(-wG&OE)c50#*X!ca4Dn7a% z+8bz2$%Yg(1190#ZyUm|Z;y9>zdafUOY~==M@`hrL?IRQ)R;Zl7Y`tVdwBbCR6e~ssN6B zgW(kS!d2Ig=WqJ?@%5&La)47U5`I2^OGQ0XDmiYmR8y9U&) zx8cfi)IrhkgaYUi*QSUEgg$q3;e3 zCo7(MIvr6~anZguKHDUfpQyl{D7yV7Ixpx8GF58*8tqT|C!Nb~E&Nh)U72k^RUSvN z+dehz9$(BTovt%sytGcq3n?*LB{|mO3rZ$mY9^BtmvtzTIF;qm8sP(YR{Ae}`0=W%J<|TTo$3?L?RuRCf+p)_H~z=H+$6iEuo4vw4Zk zaknQ>1rl6#`wto(E;o*zp};?xpL@iU8%XLjh>0~-1p=Q|a%8-DUWCJp_Zj}jb- z!d~1XeD|jBFAe&_Rr5n85@wR_V=&kHZGot9B@47=uHB6w6-~eUh#O2H`ay>4kVH{2 z5X4t=j6Ln}$|m|0Nt+jq`D&1rCb|LyH*h|@%DiNVWRZFG-E#5{2xjya3C+c=wh~Mh zbPgOftkz%j53X>(C}x!~9sU8zz;Ubg;RQY7DoEct#vUpB=~=O~7Md@zNbSbA7}{Fo zjz7!SPq>2#YpL0(eZ6V+kh5?3^w@Fhwk+|Ymw;g`nPXb9I)|$-nb}RM0u=(iqc$;h z?rbJZkWE_ypNZ6EA6I;dyzGXhqQE^$hNBj+UCm5X5%_nHq)!lYq?=o~o$kW{UjkjF zq_LSFCx83wM};S-Ib5rV)eaZ+7cZpCJeoB$-D%oxf;#R%r9^65oWlLqo2d+Rj`&aobT*^TyiaM2rz2qOY4*qlrx$RqOGh z+5DuxqY{-6D;HE0j7q!5epuyph$w!Pm9vdrFo(U`)ad+%OzP%%wAvz@^$ANwsFm;1 z)II4(r2O*ps${mk!Xu?GSvRM5A5+C@bna|T)wJA?<*cwjpXzo zj_WRvCbWXv);ktoBjc{KSeuZ*59SwqATQUQhmMFQb+{yjC!txJAx9^}HB2*z20qqeJMD{IOcdC0C|Eg={ zKa}}>*xL&RaoZ0GzfJ-pkJI*MC?#(GOe1=z&}5s*+&NX90H5e1@$dBC)A!JcY2O5+ zdwDr+FWv_Vck$)j*tdsotO6|ts>0B_Kh)5;cBMOcUBABR>hxsjF~3W@W{q{Z`yOh7 znMqUqToZ2P#zc9&cNtf$wr&d(Qv{R5veFC?)lfz z>A)1SR;l+Vn`z>%QegQou+iBl*e>&GuEZU>~DN*re~l5KE@=$3Rk8 zhBVb0j?d|uHWw|}0eUo3G>?X#1)Ob$_BriMZ=oAXh&}9m-qZISH52V+1BigaWPfJ9 zOMtuUmMV2%De6Yn&nQ)i1)8(@Yt2JO?$LItz>M0EXDF_#g2P=dC9^yKzr?p)^xz8b zv%P>~{WCOQKJchy>v3E@kx1|P>1y|y6KMSr82#zY{zh=tj~SDdh4zv*a4p&;X)~3O zhhOV=6#n^IFDys_6*5oUH|(8wX}Rxy=bV3c-fR~8{j~wV?!CVw(?9S>6dirGy_Sep zNsFF~RB$TzHw5|*MsGpc+!$w1?wLZul0UJQkbQfCjt??s{*C^8u029bQFU5d`d$xa zvCSR>BQ)av3sj&9zhqT-|Cv=W?o3P|=6W+6^`02j&DDj8Yn7$RmvsCR<^S``{a2$O zc%q;&FuNPi{3V&X76aH@`A6%PO=D-p zW%`Kg^G+*`9FJhTb_R3WANb+R7U&!Q$@VF^l2d-ur)WH8rk^;MooZ#Z@WH?P<0_@W zT9qEoA=N_ZJ0kLKO6oWsN&_IwCLflTiOslxHFqf~fKP(|Hrez%Yi=J~D=r%=LK1a% zbz>OtiS2Lj<{S98x3Y8@lhjl~YX%6$(e1;rG7)H?o2g+vL=U7lLTejD{%L;Er>BR+ zp6hfxQE^(qe(8>~La+TFg+73M|DP-LW2u+=$4)j~w|f_RBWV=ge9LCGngbnNgYfQ3 zY%j15@Bb)s9JJlk9W8Z5x1On&I3MLvD!1}TU31?Ut((d046k8h?n#r$#`Q>_FzM8t zKK?v{vD0E^_<2cl6@;Z(N*WBihl(so}<_wR z(AOIlr3X?WGIM#)&ei0b3>LsQD8~tZ&seiQ9HhTl8>+seoZ%Fab@xrdC76s1zspG1 z9#AwhL0=TD*LP!Qz70nw<@5P8V0rK9``#?A^!RP9M3SL!_9yeYGAdqy_Eu~2%(v;y zlnNLU`CUUr3~3qNcQ@w!YbyA_mZBX852n9&3MPMT61^(~4%DlH|B$23s@YLQW%L>20CHqYEM%bfmA=&Lg|%KVS?)2b|`&m*PS&bH6$ zW-)Jg(nGzxyfpJ{n4?|H>(^Y-mAE}Ecb9tcsDp6o4G#7+i|p~gnsguz=y%BIjvSEI zwo5STCs2@Pgb|`zAB~4^sp1aFT0~%h0emg{MZq7Nj}usS^z;j40cgzveqk{SKy&{w zSr95@VZxSA(>~Z_CuE5ZaeWT6p$9e7&L^C`TfS{8VCOtH0FE7StjNF= zu-i-l?k^6vWg-8>$Av((3kOP{DEZLNesZ_vo3r7|KdL+U5)PZ~szsx>IZc14#pcex zto8|UYlnKBII8UcPoC7T-BbOMxcS)Yr7Z1SNYC7ZPtx5>Pgk}&$Gr*oJs40H+CnTC z=cj9^gm7}S6_+-8JM4=W-9ru_(8IDC;QoVOxm?;A7C=@L#t;|>NV}d|IK=C0eywb< z^yJz;DUa_{q)fC+h;w9%$jZ z9gXhIxOxoE^BGZRx9j>Se|)4mr)iz7)2#8@X)B6!spDyBAgoXx2h(gk8BbC^5x1OO z$FM0eFgc*@?CYSW4{xPdTjSyEh9VTU>F7-n@rn$L!fuRu2Ye!bg&%k^A{DYdXMP^d zk3X0x?LAv!Qe#jRJ@WMK^}ndHoxca8mBnJ0a3 z3K9WlU%+ya?>0$_gtO~c}`0Y78}Pvv0oc7dhkZ3ddq`kbnSQ+>MoRnw&2%Ttko z@@HA9fF$q1;Z)ZKmu23I39*tQX|=$QQH%k1FIJV;MTo|Wh~s&SR6`vTO0J2~DrlVR z^1F)jdyALX+rDq@H_biuj4b8YT5XV>g9%Zn4)8k~EO=4BTB?U3X8eg4GjN2lRKWHGHE1uCdWSVv?J29E&11@!{`90lXKx_nHbI z4eFlC<$I}Yri4y;t@o{P729512Rf068aDYDo83>=Z3%HGYVlX^*(d6X zkJ5?4zThLt@IGY7KV|;w*aZ{x|0-MizsQ*;3WG}m0N?0OgqW+t8>aC^havJuEFU=* zwE`9gO^4PcuRV*1XAwi~{ALP&yfpGBhDAMIQ}r#0GC$ofQ=_Wi*r~z`?5eER-(FIr z*N~f~^$rJ12=*^70tWJ~hZ68sD>t$bf|(8mogP6WR8Ckgg+lpRM^qYJq#5Wt2Hy{m zKw{i%Gj=LCTKpfw;iCDtxBovd&HcYq3vk-(m3>y+YQ|+r1)je{Dg?$(M7_DCYjcCo zd{R@jQWiaX(b3^b4PHbu1ztRR8e$Q4fKy@|Cf&Ux6oFAGJndtJhE#Pk2@JUT;t?fF zo%Lo>(T$sHo)YU@y_1|LNi>9?L2L+R=L3kqR}%>ut0m?yW5RO|mbS0S!0!KSg+^4P zgQ3=jJH5<6#n9Vzyi*vJCS;O16cGja(IxpYlpY$v@Dw!~@f@_bSPJXj62|g}fqya& z1`ZBL&g5WkK(gK)k?9HVaO&aI31j$5wG|TJRjux zjv*H5C{gn+Jvo@D$N5t0Fr8e8DyVyI1Am@T;vO(_qMN9v0&m+?kAEaYN?Qw8d{V+SO06hU1uUO867VOq7rE-sy%L?&fO`mwmB4zUgJKo|^#Eqm_+m5I*&?y7 zQvk;G8m}MZIy=^(7O)aR*lAviMF>Nvs=*J$lDz6K3bXujB^FDAzZeWmudyZ;fUl_2 z>Z6N-s2AIf#K6)tRcgqf z1PV^#SBSO1zQ)#B`)3~yzkDQa@Iepo?7NJjQ#-aYDu-6613JL?8OYw_APEipmTfo* zY(c=f2)aRqgd&9tN5OxDVk4$iNu1OD#ahFAFmsR;)v>S6MWq{LaYUWe+Jy z;gj?bUbZ>o-TrO9eWt-)sSxm?Is2-*gIVl2c4vtVpJn0|2^1UQ;?ucWq{k3FG67gQ z?FD$?{r%yqIAUYM({kn8uH8n&V6G5W7 z!R>Y}G3c>?in%d7)s%kM<3dxWXQP1aiL4B4NJ&-zgBrTLD~%r?4)ME1?fsyTU0cm) zb(-@0^oc%2PEdEVpBt(VEys%Cc&Vor6t=#aRrLxyZwviZ1BMOj&GmrFA3w_klc7pV zuf_#@RfGZKF9_z~q)7$lk@R-xu?no+_wlk99aJS#s&osLu zCJG|oVYLdr_8f?!*@Sj=L-oW?6t?;i8H$ob+%<&GV`GT4N6eQq1aO13(X1vX_{#yA zv47a1!ED9`m7Q7l7nl)2?Iv~DhWG7osqj-90icZmT8tJM7>x)rMt^h6@Y5bi+irk!wr(IB2q~Af;hsO%$g6n=jD*nD)@KL8@ z!&=<^lB2-s!~l9U49L?7B8>Eb?dsJzoFUf}WdmnAZhi0Xys$ zbN*3jx|D%fcR{!diqr@C_OrQ;vogd5?2;g4}=j&mW^gO94thXTlD@ZSQL4izs1JgESe<9zB(sE)`azI3tHP4$_o944P;6g!f*t@qJ za(q%$1>j+Jq2!~!ciT+dLJBA!JCge1WHaIFoRNN{dK6#0$n}gOr(hN(KgIWWY*y}m zh>tDQY=J^Xr8uE?FhUt22b_M^zn^|)Y^)T}x3c(MX&e5Fb~D>c+>X=t9@{z85O!8* zvcACSCvo<_lZj?1uE@vD9U|qjVjjc!;m*f?ixA)a3b?pnx$7zs$r&7#o(@hfxF^_m-yO$!sK`FO=K#ootfqGLgA1=Fa*&45T6DZUZ@=cAS6U+3t9=h&qvz2(s) zj^CW3Xa(a7UB{w^=K4n_>bQnF!7AVha|UAnyug(n02&X{#inYax?{k^Zuo;^5nF!UHvySw~Gef_TW7Gnh#B8py*0Vg{?igdV61b#Q zGJJOqji0OWa%8y1R|8T@G{p71{<_2_-o|E4KpkJ6)%Laa$DVjkG&*ZDaM@X2?numG zeLpG6O?i=Opt&2y1xu60eO9h`y3RO*9#?t&DCn(}u<*6VYF}Twe_=)G6#U$(-|g@+ zUFgJYh{d@9Q`4jc1iOKkQQ?dW)Clxv?ZcBk52GJpyt`ePV;*uel6y~Km0RX*uDH)* zr=wG3emuGAyGc6)e2#vW{h9esCicmI(%I&D%L|*<6}&rIo04Hu(>)IfLS%QT z4J?Q35$DVHeZ+J%f>%aRQy12&FSLgZIx>Qar{@UGw3Wx+^Y&uiIp1phC&9$c$oPEs zwQ_QUlZn98=I4>@U^z9R*>CGLRF?T;Kd9_^W)ybZ{Nx+|a^Yg={ksxAEl^AW)=Xe` z^yGD{D{tU`te)T4Zf_FN+Jo{%m*2;IF9)732&SyBeV}+pJ2@ptCKaE2lYkf$>tv9!=XDF}w9kE$#{DqviNKGlu7teTCgb7t+HB(RPMvwI%T z?6qZbJ3H~=$*+gCFox;RFXlF@=iv|)u6-J9+urZ9Eq9tn`jq3`^-Yc;U7lA0hA?Gs zP7Y|?*A_o>iyfA5;v4TblmMskWh9>qu609)nHM~g@5vFC=SDwA@XAjD*?kZV%agPx z3UjOIc3&g`qMhaxGbQbP_qRZAC#2zERnU@1-u4XDtC*{~%@!SRqa3+uxFr0O68)S_ z!M;1qG)U;1XuON6CT=;*f&^3v-an-Z4Y#61Vk-AhM?qJ$fvpOH`9k%8jkbl(V%sw| zwgrYaG5H2a#)oC6g6?+c3CnX0IU)Go_bu+AzVWs|+a)Qv2JGOC?{Cgbt=L}g{vail zv3dl2=}E~UIKk}`EE_-u)tzBRBWEk*^N`To>gNJ~{76(&y!<<>wQbX4wC6t7cMfWJ zUy!Yie?ydV_}tJuG-6*M7_!A<7Ys@m;?toV3bU4iG>u5Iv%YII-<2ceQksvf&cyf-03yZRG zIWAbvdmK2Q`_UY%#wQ%e?RUo6I`$tpa*Z%%olMqMpNN(9iUrzfDVK=2@_U}+$hT%l zU6?+}?{vil3Mtqu@NGYV5YyBL+W%qOL5CSC%|;&!3oweYE^oHFKBPW z{(LVk9HKWhdrscLXa>Gy)>Y@Tt6XEZd)!@Q2Bi;_!@KWaTf2MK@9*8&vaO13K2hh1 z7c6Yz+VE|YJPRe?Y>XNI!D2mUN;3W(6Z^QRu{fR9YP-u}_o9TzgSY_n?VGD_6_PZp zgJlyGts>U?1#M2bo%J1@C&WG1;^C|OG)C0--ma$OZV*fUg<{P2qg#}cq2|($GH*!) zQ~9h$$p5EXFj`@$cmVeIfcky)0t)C9XKhDg3FMRcr?o60C% zRbKAtfeuWNeQU13Z)W~&!Sim5a!lM_XJ?3rB1aC$h*0 zjsXpk#HBOOmmlokt~SRPU?`A|o7Qufb74%VZEy2sh$Wm_h2del(^#Mu_MR_P7&YpF zbm1=U46E$r!F&&jMkXG8x3MyH9#Fz{9&Ws3bk=Vejo4gfL8WFutRR%kYpD~iUcV8Q z#-fzL;u#nP;h;`~WKS`@^?^ZrWJmO5V1?-%#$erSFKq9Yv5rUAJ9s;5o*tKUVjBjT zTPv!}(p4V(BnbKHxM8)iZ(r`RhkzWJF2OxBqhI+YyyKw3vvYGpNCSgU$3Q#`^CWU+ zqSD>F%_Qt5uTJlC)^P5UFaAfJ2kT>eqZRI;!A*tm2P4y|`FXtQoukpCD)$|tFFL0t zw^N%BcBcgRU96ChBQr$aNJnhJLo!M|Si6pz0+JA|`LT#L#>#AOC>z%MmbrerwsZE` z6zZm$%?jGF_;9_Pm~0&~^xC!y!@Cgx%VZ}qFMJo)X*jcz)opOA`GMwWi8CSDMHN21 z=;3Xr`5gk#(VngFeBaYedWHIlu1Uj&fZgi^h-$7CNiRGT{yo5^RO+-CK+jT?cZD^; zRUrGK6fz&Lk63mYHRIRKuoeIZ^!P#z=#^Oj0GA@dN*F zTHA=n<8!u6lNBy#Xii`1yN>)Y=)dyS_LMWLUh|~=dci{75@_GO77yk*1GqU}^>wWg z>T!N8yf1gQk~tmwH%#VIF?O%mtQ7j?->#TvmGS?#OgH=u)9)@=%*W^^hAk zC0->!I9`A_eBS~!JQ*0yeG6}T=x}s@6u7syuE!1yULP)Gj4M`e+ITTibR>9GKJh(l zrEW`zyh$Pc10>eQXC`?Sk`2EY?X`RVkcOh*MaUv5!L&cLqs66#LppORVMZr&0~b}~ zLi_VG+?5+IB4(TRu-_*oVbjt>(aN%{lkXwtMW#Ii^?R>XbStnj)uvm3q_2^l_we#P zmfe`m*c?p;P-ScgnF>kIXz7iz2yjsVKcG|{$qfP0o_k~Alq8-<4ElBWyK>VGKs&P;uH0v$?z>CMU{Obu^$uCK z+SXPrbSCK2wnQT=CU=;#o@=xPyz1stFLRSA}6Wl9X7 zvchi}a8Lwjua0Ql7e)g`*Jmw9c^&t#s9U1$kRaU!?zgmOB3J)XsA|T}!6d$?tJg^4 zeX~`QY9#6{5vZFIO#eF#KizWU+nv>4zx=c~;TWJQDxLg;!Jy+0W{Eg}9;jPa^PnhD znRx!OlMrI_x?uK1)`4m2Fdbq8246Uc%h>QDiuQI+Cv zcQYWSPadlR?2KNd=V3%{Fm##-K$CfKU_>nKRFG7_i^)dL06VPt9yqu@Y4|TlA%s#@E5K$nZ2Q2)CI>RvEw@;|fy|C?tQzt>BmUEv5ft?mlr zy=zFGxsl&2CZM~7Wf;UJ`!L03Dlk940Ukj`Rps}2*AANQO&yhrpnD4+JnB0$qGy4A zZHYLjjJ$5}lz==F_&~40mp0*V8>EfFeFb5=$Tx?BI-=3X!CH$4dl|76!5tA7F<@IL z;pR|C0&7(g{*Rfj{KwUvT=WpSQ^$BwtWE;37%Sk5toPj?PgnRNa~qXECc$!uDAogy zatgThY&iJ+hix7(O!v~pAmCG);8TKd*LVR9*uwsGrvv*4%k4L5arO5v2`jKE@qjaD zmSjEY2j~!{ryOr369u6aHj%+nlD=1LiB%?z5>%rG!SDr3I4wi-s&75SJk;h2cnZ0X z1GGBTaQ-pp&hG;qjCgt*u}|IrKOpE0Pzb;g+AC@a@U&jn5D`{r@`LIn;31$;n|0vi z=XvLuJVX!015}peM$%JwyU*@ofq(I?TE{#(^7YAtwxP zq*#IxXTc1ZjUBQDd?}e34LN|u2(1VR6Ja=#D8>;#Ny>lTn80iEBj~2QZnfk1XycFr zkQ;}XEC&nvQ^RLj(OYQLEtCTiPZdFhmhttsH+5MCRE{P}V+r;9ZTT<;Odwl4@Q?>& zR>nC$la6ON%%~MN)cY8|Ll=c_MEkh#m`vvYGIbh>`_py71*K-GVCit&dw`h1Sx-4QxyhV~@)^K0-GcIWlg&l0a?2Lpx^ zH=r%x_O8xIkW!8$FS5Pm@~#yM_DRd%mhWXHrb^Qd4eSUZMi@gBu;ZHbEY0R^0>bqq z%hNn#{9omyF>MPnd_O^4tVvIU+>~nO7|(>1NG6bS$bw3n?U`sfw#a)8Tj5 zoDW;c2E4UaroIR?DFr1!q@MP3D`0?1K&9Z})e~@+kgb-g!!K+^89574jq@OOJ^mCOV2>Dg@W z;4;^hTg%ZM!MLs)_rMi7rLw{FJc^>&WeJscFTq@+B1ypgwU}$+-x{;dG!d`8i0K8oy9 zl}%u@vatR+?zs{td-Qsxk95SC*x9{LH+>-ql2T%@S#m@US8ZMM$2|Cgie#zJml&R( ztz;d1ZRme1z;E&7;|7n@>tEZz_Kd?aJZM{BM~__v_F7^V3#7WfqbsPRp@+~*EFZvT z{Xra>(`lCMTJ*v0){XYKhu20pnIzi>$(|2>q}|4)H{{oRzSHWg$@!A$+;!gD!1x7$ z!ERI9bxjs~mT79~*T5n(6XYgwW6u7dN_l^N6O}p1&N!`Eu}76R)WRKIGKZ1or8!( zaeG%fm&B~srcEDi@Z)@!O{=drmI`Nq${g}Lzu~1;)gN9~^(W-JU*78aL2^c+c`}yR zwsB8n`(f;4UoQC@RGl(AymjSN3dx*eU+yhj7X+lRQ$XGdNQPp-<6+)|l#t)I-&^bc zL22|qsT^_^SE+6ec_LSOvRGRo#S3^`6WSvP!!c!>>dmi^j$=jgbvUV6Y(G6*q4VCz zcx#(y*DoD>WGm#6t4g5mx%f(yn(@|m&-x)#VMzNbn_^3D$K1ovG65f6|62ZLFxp zd_BgYlnM-}2x?QZ@6i$>mOo8w8cTp!J0wxrHzxeG;g7Z^vw z{rOMNbvGE*EPAislyxIFGGnzY*brcKY_k`42s5kSGgT!V+Qt)WC>2UER`1qdeq}XP ztG^YTLhxPh!X%Tz06dj94XI!9IafA|<(57~8ioaY>|&{GORT1I6aJWtXn2a`fqi#q zCXWg-C5z}hKFluiSzA2E*c$A_;I>)CS#|;oRvR)8)Tps-1Dqh({R&((iq4t0K5)xp z=GK!h)rEfFm_JPnY+O9+IB@3@bh;>Yn0g7m7)6n6qVC+{1*`kv0|LR9=#6LNC?F10 zv`>;W#3_juJvd>i$Ee*a^lo6XP2B+u2T_*-%FN(maN0AoDIL?hXwO0a4%y;FPNHAi zXKSI?tJrOyd$hKM!oHCtt0%8wLqJrd*CZf3t<9vX_)PC1(_5lcJTf@uXSLL>Y-g!P zf~^r>UlNJLnkMUx8dG>Po6V@M(U?8Yu#R#!_8uf)?}9|v^@xN|0m?Rq&?IsLo=~HO zam5x5K#x-+Q%pveZ|E~f-*F3>UvGGBAAL%x`U`ko*e;v~-ONu~`9rdOSj*zFIs!H0 zL*bUdy9k~xpNXw5I^@6Q#FI|+S%#Q}l<6%rO2iCa(6$e|-N2-h@(nLgxEo0DmXOT^ zAC}RMvD-g>ExLoqm-t<0vhe}(CPMXbf+ zp6S`L2q#ugfW|l5y-shKz|O#YTC->r^T^MyOP%)Svw$MZjZ1tRU7 z&E1$GZJuWnm9BoCFLfl##al21-4G8C`6cUbN4SwcCl)2sR;0La=Cz+kUCxg*3V89M zgH{BTQctgbM+%g>(CIcF#9yee#?ANqs{5~{<%;pgkpA^!<3PweMM z@@Ze5^7~+)aHR?dF!*B&v993z4s0CRnkYXY-T;e=vPt9&KE0T3UH#L2!_%^q-AP;a zQCb$-2o!a+A5M~%ER+>|9~2p;k(VTI z(M|Q-IMS140WB3p1SU^{VJ!8X`yG}=ACUqW-ckClIO2Y3Cemb zV~0VA52U1F^DU$hHYl_0I*$HCNXCjStjfv(3ta z=A+d}6k8(FZWu5_@;5l`-i>7Am9+|&^QTn-w9Eq2`6@8kg_whok`QL(I%GMya0wfMUa0!GClp>RYe^29yB9uIvMC1MKGLPHW-NJio?EH>qcNA$AFug z5z+9q8>&|_3@=kN-#j_IT}~ZP1lfv)&0+l9SpC6W&w0f|L~9JHC%+9gmIW5g6_1zXX@tpvt5S{_JEPzM;QS<(NBux zfJ=XO)p=MSenpUf``cX!pghSC>$?D%9t8c1jJSBSZ_oo+y%!PwScNWtyy;`C-=VtR z1W=O}YbG9G>QMklRYvV^lO3Q+d6D%T@ZdLie&N_+8d@5_%~jxS$m@od0b2j)3r1%c z=*0O zk9XC195&OtNiD^oZ1A03UKCZ)8jH~p$YhKl* z&{az72rx8Cuy`QCf_BW2KX;IU-3*$3qxu2DbDdWe_6qwppg#=oTaYF6WUy|F!O=lg zrmLP;kYcjx62HFP=ExW##V1gcEBmlN>06*|BOokk1d2WUKArf42Ai6`<-_d4FA$H4hy)-blRkjn`X9tDj?q z!+Sa*J~IIjrGnKnGT_dMv>YUigW))s8T+u~!g@dUYgCsxzKSEhphn8QL3>IPd}PQ6 z&eUScWp^|%!oN?JOZ4w?FWsq5=(Te1DqYwSoCa_^obhrNLZ883g_oZ|XmNfyFMuEA ziW_3K^PZ?ARm;AO>qS{IfB=Z)+$P>#J31_VIM8&mH~A_l4Ec_u$}gRj%%9g{W-4>C z6TSg!=yRv<_>0M+&iUnZ%7b9Dd7p@oPy)dTZPtJ5kMrnKpEav~B&*su53j!{s95Yl zw?U&c0$d^>)9C8DbCe&I7G9u&2XJWL;hq3GX!QNCIRyo2mAb&2XM3UMna@|4w_fNX z%ubEGm~~+>6lm8hj7xyDJtf`aqX31+dzC9Ix4o)^2hh_=QX!{ibRCay0}dhdf@oFn zjMdg%*WhIa9^#e1Rc)c{^rI)OkswEvYosl~6u;w~&fk%sm1%oes#S-eL|Lj~NRCPj zbvBguf#UwY6$wz`(B^;*13MY%9Snv;LNVZ!1e+zYcj#sEf%MTub}H9dlYmWt?>3WW zSy%#uuXqcC2dv7Lt*>0~VMY&lu)|UzvA4NU?*hb5tl?{QHVX9PqsP)JN}eM{36Kf& zQyuEg!PWsS;(74g{1 z8=ibM>QDX#eZrvnaKOZ}+;RJ57|})a>FySlc9jiY?AN)vaR8i|Hb00do%Vkk7co)c ze&cTIvznfZ>sp|@FCDRruo*e1x zV4x7QUqd+Ubw@pq7sSz7^N>+{I`t1NfK(CQ#7uuk+u>L7n|6c`@0ci()qlervpQ>a zo@G$Nk{>=03)Zyuu6I4?kpCUmuWRPXlck!yf5=gptJK7;6MxZtHhx%gP=BxK3Acqm z1@yBAX+Z9e*3^+MXL#@%>%#xw`TPLS_fEH!tP<+PdfxSsmLCh!SiyHO`HsdIgVhGd zZkg+Dc8_tlg(MhE74$R@gaUDQX!YjEXSPO~g)kq87(Ap)zyysw8s1S? zHT7swZM82TAIm>Bj91m$!-UPM3o`bYwD_2X3>$_Wn3-#FphrfNH)PBeSf5Th>?W0W zfUy(XFPFHRf?`(gbMda|1-@dc^Fm72Q%Y)@`+Oh_Or>_Ag{qf67Uk!oB+>Rk>nZhd zYz|^aQJ2oq81`IrSZ2$c#&$o|> z=sySIMVP`6uP+;Y3JZrsR4pu~gP>gk(97hN2X&3Aotv@6T* z7HU%24KQ`~o~V)!Xf#jlf!0YmKb64PVa>cFgBzK)cEM1d@Zx6V2>LWfP|noXkkTog z@&Q3AKClJVHWV~j zmmFkrTZ`M9vx0{3=g~iygYs?=chgIEmjH}?MfSO*9<-R{PfVbk{* zq2YWw4`pV5S8Lh;bNeWinn@U?rgq;x07`t&k+2?ZE!JKz}X2sq&2N6 z?X4%3WDd6`KX};>VqPtDCe~7kVAU%|k+{?5_NDB~ENl*}r>;Wisn_4PfYWtZ;g6U2 zOTjEIefSNhgN{YEO<=~2z6~vPyCt+2^L+E@drMI>tN`O9=kjy6wVlK3WaCwab}ycV z9o;p!;X~!hso!`k`}{FQ5GFB|gZ4v9A%e`%wVGG^94xj!)xCGNZj7*eG{KhBamPb) z%xMV5tTNb5Ldw^(@gx~4BNS-LWt$!^cMf@he=a%9RN_{2h?b^sU^yWiBs0X~v1IK8 zkuxJ1NOPkB$T+T+EpB6yD$?}!(2Uic1>asm+hiH<{iM%D7UR?9#L+S#%6HK2DQ`an zF?V#lTnwS$rA&`b{11K6<7zOo9Xpt>k9jPo*=7$p+WBUI{#TF@5VQJh8!tTgE^?|; zW;d@iTI`1R*&|L*x}mn^EYGBk&5o@68UWz+wyO()LY)+d{A;`2pRgpm+vCtFV_LY# z*A!O@8JryZ9F|Nl&7OT^QXn7k79o9MFzm}%3A3{zZrn8s3}Eb>+|0cT!frX>G(@DI zL-IwZ`qoIyTj@A@+DGTCw}_+VYKo0%F$J{`h3Dqa6Ns;t`TOQO*Mcm|?rafE!@GB5 z3y6Vm^dbi`G(YzB#L4NcLdeF-cd;cic~)C-eDkLqP28P;@|Xsc2aDc3=OMq#tLwnK zCt5GPC}1rx5Sa#98+8o+8**#+SgyF2H(Cm)zxt>EfwLNK-3;Le;K&+HypLw;~@UiG3rmw%Q*g zLtRzF{@Jg~>6FAj8Nv#ZmTLol7GG>I+wC#Nu zPCg3K3r#U7RE+pNk8fXSR}n1UBDS8U6(I?G(`fzS6Ap)@d2yN49V?wG4(5O_j2x2P zY~MHM*NC-wT$#FdLW!7fVNomM-wT85>0eN2opbFwovf3dSNxBn9>W@wN}R1!!^;A( zHq-KErAX_z7%d$@Yd=~iC&P7R&p;f(Rf2A3TbK82oJjUeUL#!hr>ghWz0ls%yNaa! zfYEjx+)v`Wwb5s5Rqaac+aUxT0b}4vWpi{%4sBu6|3#GS&+-hLAg==B*)$d_pe0(| z8Y%yT?`8BGL>X4+Ix5ufb37loe;)Eb49S*MT*Zoz=LF&7vP!?uM5Di<@zymeg2I?@ zlGC;|Ep1!Zl6y?HUpRBy1Osu!%~0=Kpi#L>0WjYEL7aD4j+-aOezq<^e`%mks=b?$ zK0~Vj_#}8?{53Mo1N4&aWk=_vqNdjrzmEyjaRF?6=D?TDU+@ggp@@CmqiO7-_+-k} zk3;{U>HoHB{;PXOi-Yg!aFL@pStVjIS-Up6Ne2)jJ6QB?xq*lKc(rXIT{fvEDj$AR z7H+;BFXR}VkQSa|phgGINzv5}qCm$vOM$fy zPZW_%n`Wn8^3`?n2lb1@=jy;c41aN;)0+m`&0pX(Pc)|N7Flkisg}Rg7U=wIUIU2a zKi4(R2!ku}HsX1SgBJ`E79u4Lm!Dw8Buqs?_-bOE38-7-O7%_5WmE>U-{N+tNvqVR znGQPMR&_`6;O!AfvMFdUt#Y5Cq;5gM7yr=-%C(GYI3oVL8{iZ|4rz&gqu9W@lg;h6 zj4_~5y|)L~$kXFZA&_Xn8j5_y293|R@n0S=xEh|{gda9o`pBr*Q0+~5~| z;SbQZi36R)$Ks;kxgZ!2I57Y+5wC#rssAHw|BnaYKOTVpZ_N_GVHW)KIZ5w7%i{fK z0Gdgi1mZ%jAwQRbbAD;RZ4vP?cd#+O{xoS9wR<_o2<6m5k%O7r%vNlqVP52Gmas?y zz+C%1*d*kSenXUaSn+8mb^Xo1Lh4)ghaVQa>*m3-F>91=8#h}Ts02hi&8m!NFiA8)5{ zEP7_f?p(+K9|ta;|Bo#Ge+*B6ApXB5iw9W!eLH}bYs0aB`I6tJhrgN;pn49zs<2f=lX-p6a%{;rA)g+qpZVw!w<-XJ%-nC5yCM~r zZ0Xue@>Th5jUsoj{nUCnoRLLI9ci8N5>Cqx!= zu*2d}0~}|6DOh>0@xbUIkSrG+RNXmtKdY>%KX6A55Bnl(>t3#lj&^tylV?TyOELkf zkv}sxrqQTZ?pnkQo-1{Dws^0IEOpk+HkAFH8d&FT)x-ZrXI0;U8c8xxIfX(PigLFi zoLm;pI2nVKH{&ek1b>*((zfRM>>CJ2j%JtcMzcd;;W$>+KD97eWrbue5_^ALuyNTC z^y_G!auxOmK#0@-@!-k_WD-Kt~2+-{O{m z1z6Q}bpb|4jpbRWpgy(0G!==NFlhB_s=E`m0BiWa$NZYo4;(kAPS9u(P8 zzq#EmZLS|rEy(_xy2vxC15yQ1U+_M_R)untKR^s*MdzvcsKC@B3LVH~VBc2^zZU>Q z`W7)zTrAh_STiHEajL+yKTUVUGJ~;qyn4opwv4)=biqxN;dRrh5u5wWto1R|TD<5E8G zMQ3VlgllnA%oH`Q1QSSaCDZ8EduHfHbUIr0biV^N#b4f8*d%P-=NnyVFZLH1s;K$& z5=Z(U)U@7=WN{ikRvOs(5vP$A{(y|;{i9t=pV#E7l}v7nnOfcL&d3E-Nj>1H1y|Ak-0sW zZ=>KahRr7+Q33Uzh=&keBM`MWp3-|q-2aI`xHG_zCbJO`{nW3H%EREa|0CPuGhPH! zwr5;uN!(f3ShR}XDBs^B@y5sKHk@v;cytT$JD*+)dWI@>puZC`s48`!H#BO!$o4#X zYcJ`ipZ?0Q4-oSzPQBa`8xg#Xf1m$Z=#o#QpCiDGXBWF1zgo)HrR-=`tpO`|ia3v^ z-la3*{?*sUmnfe4yR7hi%5%eU?+a*Zg8XGW5j-DukL!G6ytR8czdX%D%K`M2Xt}0f z<*Jmk4b`cgLki!6=e_}Lp_9#t%t>yZJzuGcJ&s6Riaz_od$%}$L?sw1qptiscn!$Z zQ_hQ)y7DjVfQ_H{r<9#o!1i~g9CP`qjV6EaKP2tMwzj`-y01lhpu3+H3yyFx)pS|v z(5ZZ6K=D*1lr;&fPenRlc~)a-XR30G=T5Zv51OjU26pmZx@gpjCT6L9ehCE9jlH;m zEEN{PMy63e(8@#4yoxiXbU7_zd$Qwh%)>NxIO@*0aMAdZnQ^))mkLARnK|llI)%q7 z8!8CSX-at-7tY+N!CmUexb3F?eq}|#tJT&TWzhf?gH?Yl=!Xqcx}4U*r!M@X=&B%UP4wwHOHBg zzH(kd{(%Rvp+?#421lda{=d~@cws3z(yZAh^gAD7;C zS4T|VEK9DgU{GclIT4mKkTIh!_h2}qxCbqV@DVR8wXR4#JCD1hegBFQ&Cx?sZA4q~ zt^`pJL7C_c%Qvw%Plr^&!h3xf088Dd|DL76b`w1KLzL)4!Rm+J$LES1+tc%Xcdi?Z z{lc{K4aPOwiN?m99yq4F?9ThK0hG2D;S=bApTg7nKv$fJ*}Ibdht1j{waCF=8cMiM z#<300`Cm1#`%(nDK6Ls{PGA&Z-V_>NHLwVgC+Nb%ewV~Deem*$H(_e2K7hT8T~BVE z#|+|+zrYu6mnsLzBv^eFyncJzi~<)`4&;DTqqiQAlLSS>pKQU>UL^lT%kk{dA?ohYzAl3N|jyX{VsBN2u-7hZMfNcZG(Y>mZ+E+ggxWwv&rA?B_PMdD$? z|CY^nr@J^^gi<^eR3sviT<$_WkQ{O8Adr)-Dlpl>=|vrCop8OrxSNd2a>tJOl% zCxU?kDS&wQ{~zAoI;_gBTOU6UI-lpsiV=R!aMDM64Bq`M>qB&8PJNGd2P-Q6H1 z-5}ka>o@V+`|S67U-xhC>pJH;|F88t&zy72F~=O^zV8vJSM!$RCrkY6_n*D|Y%kK9x6Lna82Rn0}EC<<@|IxKeoR_GEAw;&0qO~vmIb-{ig(!f0 z6E0OVXEG1OJ_d_*snOFKCcsmFXjvM2OW)jU zlkwnXO$sxW6r?dub|Grmf;0z(Pu{2QkbrFvJoWoz@3eHBq_H-gj(jUv)1~r8Y*!tB z?_kA?sA>=URxt+=s*<5Ub+^)ChwtM11?HPPUf%w+A@T!Jfvx|2)_HfgyWCCHqG77) zYz%96R-#aSW$$c3*!Q|3rnbmnM~-WU54{R4t}Q)m)OLM~T`Gu#JUgF9^0@ugN4GeU zbI&N&J=X(d+G_6BF3G9T;(sNmb6RNiIX-SIe5lioBR#-<{EmfdQ-EZ>^w;Il=oqc7G6ARto`%#yR4XYw9SMvq7x)R+S#sk(P8ojx1-aLEpdFlaoq7&SrPOrf; z^r8(~N~E?YUr_8B$_<+?p#7<#|#IO^*B5 zxvsQts=wBs3fKzYzdS56hui)uNCvsQTUp4B1f--$*h3bYIOKFll~e-xt)k(%y{o`H zlkj`%E9{?XB}XSSg10zgp&KtY6~8^4`6VEzLMtWF+Nq4*Re8Fr*cwcZwLmL_TJ%Zc z^&DXX6DX7a7Jg|4bdCPLzVOS2io%7yd=2o~ir%&&H#f&UCVK}BCTSy2w3r*pOMf`FtKF(vuK)qS8UBNa+ay3q1rwHnnxGCJIg^@Kt|{J)nxk)(l| zu+6l#Fo@hGl0v4Wfa11GXSfu{6rW#SabF$o-pf)GJL!HK znh|GvmHjRb>s+PbRfUQF!$1`_0sO&Uj4PdU!~FTOyy`qo#WgzEq%sCq#->*}pTgZ?$9 z#0-L{nQ~(@^X_6riw`Hx8A$Q6A)9hsj((kTQP8U;z7`yjbSvCtcmBY8LdGAm@dj-n zUr0+Lxqi4Y0cd;D)vjKL`S5d+bWF4ZC_%9`8dshQsc?VKe_{bRT9>L+zzB5sOCFS< zeBZX7;UOugM5*kh@BjJB-evWrkcT^^Ylym7^k?>~(WK*Jy7}n~^52Ry8r6FOrUf7u zF#p#ZqNWVc8R`Uut{NaGbhGjJ>+HM6d5((;`nT%&(d!=x3a3JS8CcU_L?$yPvN0z* zkp>WlRJ>LQ&%rJsfQ|KN6atbwZEFZAUhY`<`=lh&)az4(V*8)_g0(g~z2h9Y&*c2= ztB9zfzZ3M97w*YlciH-u7oK`=HfM3$OwRP9vV0$^j zia{-;3ZM2v{}M%%^7&BXrs)~*$||kML^t__FHe#KxtMSldy*5L826K8x;d;Z-?x~y zeW*KH-`*#9%5HIiI$i0AHNlZ1RhpQMu3YVYjaiMEpE;{ zs<2+HF1QVc!I+Cy26^eTR=(}YfnsJp-N8>5IDYd7d(JGo1E0|NVk5MI*(*XE|x-)gWz!{Lv0N{7!hmnuFqh|TPVNXUJ0As4m3@AY*$OjO-kcJa*idC6-h zu0pkLKD|*kYUqRzC(?DmQIV%A&Vb%PQX5zP=00970QPz8+c&ublaT&55Rn_*deDHq z5$kAoWt_?D+>5tkpAov>9XI8?lD$`i+t5j=DS#*;<+;UH!eYLoFjF`x8Lnl|V!Y@~QnR@9sfy!x0_^D{6Ar*8&3OG%NL z_Epd??A2dzwE`Ck7V^`O{l`8 zoBU=jl#;U`ObusMuO&b#J>#z!`=XB!6yhLa+Pq!1y6|5q~~4D_qoyp<1m0GsO_Owc1p8Zy@EeQ@tQGa;dX>VJ{Hx zUKbiKkCesevQ_)(uCTj-`;}6y(pTaSeKp~q6w{bwoHyo6KDD3LU?%udtHu#8S;Y?y zM|_rld_lT9Gb@OfO?u2BU}i}Y4LGx9gVt9x=*iy^`Af6dcie){r61Ee?G!nEK2yB= zeS^O%UUrm2&!DwXS5AGr$3?I8S5GR1<%LpH=+x=pc88hFD%7D0*QK@jhRzp^vOd>syitMMh**G$Ff+hRTW|si$TJ*o=5Z^$Uo*?UaCEN??aB9CA zoeJ&4@{@~UjGsZqr&Pq_P|KB2R-haw`4Ym*Sx<0!8U1;RJfnqHB!|aR%=}|&*(7%8 zQoiisaT2d53<7CAi7XH2wK~&;kvLI&aAhv1c0BOwOl6O;#mta|=o)bPgHA;0GrPlk z9eXFARjSkXuT4XEGF?yi+u3C`!Wy$TDX4Ko?Rggl!y}VhMWguu(lQ( z6iB!iSf=?XWWQV(Dx0cba&6_{$-r)o;V?zeX>cSo4dy+SWXt;QnQib^H=oO(j!-=& zL{(0u{i6iTXYzBC6Oa2`Mcls%e`Hk@{mDQlt9GZ#j1?GW_umY2VrR+q_yU7xSHK^& z&y%+@Fnk~g1Ap=~%}NfoR(a+X^sQW4LkhWjoZlSX6jyNgang_b_(`X-z$ra(o6u_) z=7}p;YVESuz<}Z!&Imr4vi>{|={ivsU{~^g0Yymmt;EXf=oGa9dbv7N>}}Q|*3{;L zn5n4W>l?~H2UCcS`fblL+s?&Q!n@&{gHa}dX@&Ls-&8j{s4Rn}?#>c&K*9*wrks|_ zy+7|nhKAn_b_;tKmJ08}^zkYr4Rv6`Bc0n7wJn~L-EQ4C2wUX|RV~R+hohiC<>O{g zb(wA(%_mFFUsaU?!6c-*r*A+akcs!NskJFuC=b!1p~2z}@gPbt9&`0R4i_~~Tpa^3 zd`HCn!Qw1+F~9FxQDA{`Hl^saEN!(Ee&S77o$LUKWH4*!+RsjTn5A=W5X#MHc99_lT@9;g@ryz)t{ZI!8;inKcpcY{IWCQz z{vqPX_7k#@pF0PF>b+lHm-Pn{vHJ|3%WGu?;aG1z54tRTd7Aq1O3$b<+Vfk)6R&%Q zZ>ggeQ(5+t>8WEf@S?ds1q@!}Qr@>v&i&q`VDXw-BzXKVHxZaHofEl$@0DIwM_d($ zVb6}?=P`Xh_Ss((b=l=2&aE^AuxNU6x-J|R{s$=IPk+~L>$Y5UIsrggqW=+BwKfa20{?1vw>*NHZ8V&vg7$#Q+G$QHu5v(b+8ScCwit`j?liwh=Ty>jb3^OADDp}0l!7ElFfeYsSP3~S zB3^!upwTCi{KqFa`p3kw?oQSHq$|yG+d^tZ+8SOSrvp3#Mnq-ztil%yY(>lPBu!)` zkI7M)Y-`jr53*+;A5$0!n*DmTkxMl=7S%HUQj?vJkb_yCR)kmFA*77a zUh?>XO}dX(7lBkPnS#wQ)kBY3UT8^aZLXP(NPm1iIDS%Xb-B>Cob@)e(d{Ddw~ z*XCqK-eB22y!*`Z>5|B&w2ioq^nQ$w;Tu7HL>6p=IEkVqzSK-u6qo7BxyldFrk-w< zGn8`>p3=?KpJMmbJlk_Chz`V=ipScX>^vJ!t7MZ847cP&->-Lx|5h0;Kbzu1aw^YyGV-DOT#9+Cz62Sh+kYW*N|iqrzOSKAY?2H{fH8w%si3Y03R z&Exh_pq$6Qgs^774}xFRe}gcD#X>B_Z4DlP#+HEeO@{%X;0(BDtCN83jdxxnZwCCU z{9pa+f9W3rWla5sX1WXfEy9;&dhagVrT-W8jYWoYDrh;x9W{(L&8H<7)mGTUQ~25~ z>VHlR{C_N%S2RkuzLlNia@N`r?#0K;Yoru6ZOtIK??3Kyc_Hc?O1{+EZ+*+4LAuM4zt^mCg)`{nN3#YN^ySlNw@0^2 znv*({qwUv;`d47tbI+qhc|QUrC$+4E0K{gITWkW!U%3M|nvq#T6GNZCy}{*ZnOGg} zc*l81SYl%t#n!k5xU2H~C_H;E@VCq$J0aX3aks@T%fDV__`b@i08KHVO6Hh}>9q~6 z$D}7*H~zjV0A!}yfNAM{0kI05 z_)Kuyp5!QOvH(p<;TagD@MgDGq&G1jIv78{c2M~1%?2|B_Pza*>JiVb^~myM7?Wnh zb`dr7eccHQiJy+vaJtmY&baMfG!8CpqG0No&}r1yvBEhjwc?!iu?$1U`7_rB_-vWY z9VZH*&_%}@500yOJ(3JX--F2l3?&6o@a}7~ZsJI9t=o4;V(_v4^;`_Vhju5>VH~W= zh1UQ-JvjMFMi}_>&^LJ1XCBi^?$)Pi_d<7`HJPQ@F8vh#xI04@^Aum|eO?MSor;u2 z!Z4C@rj*RrK-p!$Tcc}kLFzWB>J~#uq7Dcq2tYF z|E9U9K$Pee%~U(axxG>2$@%riACsb3oJMA6g`(+iW=hUh`#pMJRbjY=C%3%ETg7;f zTUa*iDcL>(jmUAaZ+N(_N+*6T-}a+c-PEu{C|v0HV}H~d2c6>Ddl0P|fTK$0Du_EX zg4z-HkDsLin^o(&j3Q}vGWrCWxU1MA*&GZ}dwfW@p}`Wix6ndtu#6+Qa&b5)QeKT5@Y(J7w3J-v$0eIp>Y@W^rTLU?PNCuJF)Z^*cPQag2? z2?lN4IFwB{IsUGClizG#Cw9B}wVfy>>o93Q zp7IXtgaUvj38~wsXw9*tMb{{?HeDyUw851R54p5qD zbi5@0>II9Kf#U9yn5_FEJQj?V0kiitW+!d=^2_~(k`#{Y{uoD zE9l*jpy$cn{9*z~(Xi*7eS0%S8rxqlaY z6JyRv*C+U_*T=C!Jz86z%J-O;b0He-_7T&WQXI<+??xE!F1G%{AS{@bXm%+MHDTJ9 zM$iwvL&x-0?c?fGg#KHFyp~kJR0P1WqNv{*&>Ap^Wk_dGq!{vT)P@|a4=zu@1hha$ zvcJL00AJi)0{Ca?|K2|{(lPbMd}L7g5At>x63n~jR%nZf|F5p$R?YkWM|!prTc_c% z>hk%mAX=cTxc<&89T1qVe>iJb}r zoAItZF5v?JgOt0SRryB_wjdVX=UjOUj&04B>2y%C*5F8e@}C@j3ONMU--w;%|5vl# zguBM=0K*y1^ccQVoEDV@${~JyR+||?O<@Cfz-X_*$Bc~pqx}6ZGOg3QV0c{|B#Grd zJ(P$4!hB4e_QtT-PX8n5!B%;D7$zS4byxn5O>UJk8ym1EjG%#0+hvaXQknLU<<67E zFlW~Kw|pMBd$)r!<_B;6cTMNt|NE(k%Jje~QbIo17BWwAOVr!=qaHx8h9yC|>JdK# z5K5Z>w(0D@kue0o7w^is&n_{>O|aL507Z*VbvN#}IH=uU55?c|!++z@?y=vFkNZ#f zMQ`zcX=Um#f{pS1w`v^WpBCw_Qkmdkhd+T^aozImRrs6qrUKRHTgA#?v?hoDtMn!! z6}IMY=}p|;C3V^!s0&wGGJ>O^jz~MG! z@GV3$LY2JxqapJUfYidh-vzz@`sM$Xl8Zn`RSj)J{UsBAq6h{2S#Lxiz=%~T<@-ks z`!^t|u}aXo=%Ur!NMYWeBQ;X)_dmcpb%A6!10hw>Q8WaNpbaHg3TlVo0JdBkq955e zfqjxf92D1*T#)$*FFHNLk3RmWprxXNmP7l#NaB`p9Rr{)|K+uB_3UpM*F601=$1P< zxnYpN`@DB{?<~y$>Iw-wlf||e*pIuytx+oe$tV>X8^~5p6hgC53_N1nJ-cCOg#-W* zhw$6tIQ}p)Ldd*c*iFZ*mba{Jt=CmZ4IM;G!sZkK|9dAE^gg4Nw0tB1+e_3orX7Om znVA`CK1645G|RCyRra<dfxk&( zaT{x-m?k(GE!zQ~m$YA{>{bY!xEvrTioU$*K$m%ayt-(Xw@Gyg){b3q>v7dBBYJP00aj~kIQ8(F-R@r+v zT5MlIWgI8}%Skz_OYoVFFSZFUuy;hT&3Tr9E&XF=29w1C$+8vv5%`B`6y=M@lz))S z=BRBR?M*_S8c4pheIA=}-e`CTr2;KGp-G>_{`nonZ{@{=thwH4-w~R@rL{hBkh7$h zU9fh^IvJMz*(T_@!xNTP)3q9tv0iCd3Z=T$*Wxh*l>L8x%13Y0w(7=bwvi0l0$Bv4 zP5u=qUl*UGc@wcaoZkc;RxmytwAd=Hf?!kp9?l=1;RlA}@zGakVwg6@eCX}aL1z+^ z5%h||s%7c1i@zL`;DXWZ`a+Wb@Bso0RLgIVznr>UGGW@0%7rx>h(fi7&gF~_<*3|m zS5pQs$^WsM4)To<+4pISR2n}WW2lqs0;j>qjp;3g4PX$O*Eih5sfx>)N zU;3Z^sDvEfqukH3K3~wO?8nT(_|#pnYV%0rxX+H%LC#EX_0ZXUl!DC#57F7gtMwz4m$ z0reLy_BEcp*tCf+WI(?u5Cxh4rI!Hqq8_zuRq1n)OcFK|)*1ESS*)jCy5H1Cc(??Ast(>;cm z9!#&i{rN>yWQN}W-!N4QAr{Pq8rUt}8Z{s*dNy!o*ud1E! z8p?KkzqGhfnYTO0�~zw(j_LJTWw4TPd4s%O?_cy!+E)HuO8im*>@Qfkx7ghx?1| zYIWbS(q0Mk1y0mF#!ramMY;Z)Hz0Ceq%AbzlbaT(W3Ag4)M@>zHj}h+x*H zS4b@-W4nj&&AUTZh4?B&4D%bm$f(t&?J4KCmu}>hxeD*Ww=Df6_ZrF5q42#i&HNq{U<;zN z?_B^p__y6qR%sXs9fCH5bn2C3hFK+s6do997@Ux8_|ln|FZPB>yTEVm*~R6(A1^k{ z+}(R;_8cRHr$89BFB=wjyix~Q+r+&N>{}^_(JYf%TZ=@aq6=LR!K=?%SHli z^nH=oZ&Utmr<4l=GThiFPJ;Viqaw0O1qqo~EhapWSIn-1VmSRXRnB4ZX0i&20S)gk zI)?29_x6W7+#uq0UZYZVEVy%+TGz`((-_=t*7I7FC zAq5Ayg*SsO=t6=Jt1it1eHr=FSvVTvu__z#bAkq%;UN)C@O3=Poz!z@h}&Y;5gT6~ zpB>PT)xWwOvSXGQANZIbw2J4CB&g>uH8`y-IG>jnyn*&!7(Ko~`}K^-dord$r&50@ z>^c}R!5G$?#*ACa>E&>ZRam&w(kpHLBctceo18BjSa0@Jt8f`#@})KN2tfs$E`;0_ zrMZ7CN{{Va?{v-Az`Vr!TsBb1|Sa1+ZmLb{iXF5KuB&0&zJ#srAzB`#(3qn7}yy&uDUL6`+GjgtIF#A9yTK@*k5 zxmrl|Y&~(j)b3HRM~YsMR;?YLkn4#=BxJZ^GPnG(R4Q3Q%&D|crW~w_>`i~?)CLR1 z3le-GFKNA!qx~gW$bd4cG}*Yu&v|_NT;EaRVZl;_~tZ#=z2)fbJ zovho(6+3LJJ6tVk^lI0O^NVH-u+Tv>n@BP8K7lbo{xl}PkwED|1Ni z3-sAH!H=*acc*tlPJ{!P^X*WexXhE+(?xZ5%u12X=TF?aUaojhzYvU7Le*02S!`d+ znm6GO4(%l#YNm6=4P;vpwZ1{E|A`ZRsIjwnrRLGg67&JTb&=kHGE+e95o4*WfwAng z|05B+IYK5P8t7w^7cHWeNsMWWcSq_)n zG}b?XIllezOOH%f6isiX>;12k%K%uG;0@ONc?_>7tAp$vqyNIXCS5ySnDvj7Ro{?I}}kF&vR_vjsZYI^OfC)SuoU2ECyoR%+V zc5)wVQ>B(+cZ}V)W15Q%(e|t^H}iv6mz7kKzhh)veme@{q(f>RjyG~>W1Uv{!;-!e z{?8FvbsTG1f>wubHn=6UBKsX6&}3}D=*F6Otv+qVYt+nQbaddSDXv1s?Rk?{^0hu* z3pc6lX7WOB+Urcy^%+YrN1nCpiW6_~mGgT$$7t=n0t%h``{xf0zFYOLMO?IU%z_4` z8r^}CLQ~%jE}p#6M`Gv1pDJGYb3M6@6T3j4$-f0% zc}`m^`FBGj1y< zH7~3RI^+9kN8AeM5GAe%Psk1>7Cnk1g3ri)b;9nl8~V=ct!B*3Wn z=Y0naIevL<(zoJlC*PvU&gdn-H#~9X_1mSBB5K}8W+bbNWrCP-*?a!_E@-@(Pjs~W z5V`1X-4-bR{_T8z(?LTGV%+O>u|Tgw@zbyf3=4mXFm@U@Yfr(?TJwBo(2{C37& z?tH->A*~Xq2I3DXAnCVV?JYSZ|M_Xf$jBdO3DP^238ce@5aT%d9O9K0%^Tg%*VYCk zJ7MP2V|*?PU#yIRC{=KOaNe%Q1lX2;Ta7JEB81In4QaFug{J2?1Aml{l~?Y0<3xs0b^`wRNu_qr4wrpTg$2_9#Gw- zy^VRJ)c#@eJz@9pzMWGy1f*Syu6x}-{% zNy4^nwc-u;Hwc|d*Jx}7W!tYBEJ(t3=lmN2`cXz_QSQMd>@=K5On;7Ucy7(rKj<+I zZ9Z2G*xx=CP1n!Y1?uD4pIzdtr(0Gwo%a>Xm*t>Zi^o2Q^?;x;Wc(b&P75q*zuUIy zqp8BslV0ZT#zVuVRUv*FFR`-ZjN$m3x*}da7w{fGQ}gTjMdKNZ(S;e(CvVQ?_Ov`% zUM~NbJh9T`AqTSVl4DJ!lE?NLG|(^4YfK6AFDTD`ag(U^`a9KmM8G$9*$o}AYF%y~ zraVwUU%=}A`{n$nsbJ1mP3Hq)1d-OT=V85R zMXu+IOzsaH8zY62BDgigUs#S4oHso7^(Gv6;;KnVHeE~rRn3wr^WT16FE3f^K^kb) z=PA~f7g}Vu!?)siEVHd#V0X^}zGRYoGUJP`WtTZniGQ}YPn(0jnRmrvSS|#VbO<@# z3nTu<-UCg6n>D1n)1jAlK_&J+X`Et5CX(}l*Q!7m<#<>x z|6!)(0E%2J4PB~GHKo-hYno^f-EaW4_Kw7Ol_!o&|CFz<;F$i(bg}#u!d#5{#lg+Y z6x$@JvS91BBhXrDDhPc#AKQ*rU}cKEci>tHJGz>{6Y^LN&nHw`*u=#j{CbEDL?)0< zt@I=h`z2#gzHP?!zt(BsSrc~R$EClibw8nNFdVObSD>Oqrmf!Beg(b9?%0=r9jl*K zgliq&gDc%-%0{_WNqoC&TK;*f5_r|oyTbU*gaq;|7V+!|B@l;p$Sc6n8UZdi_T#gq z7ui$=G3ZI$QP~#2o_9)Fx#f(GF?H0I@SIXvSnm!000|=hjI3ir{KiAN$1e z)u$`!dYe<>Wd3SGk4be&poH_hNYnLdc$zc_m!X5Lp}eV!R;T0FzyV61Yb}%Cl2%2^bSiCFh@lpw$5=<)-B5g< z2{myz|7KCSjfXH)*A~7f7e}nNQ6bK>`||BeCN?vkbw3MNRKxx=k-hF{yyKS+Sfjde z(WW#O0rLX-V%pi{5(UQLo8OKadeSF89V~HH`qDT#COZ1o$O`lljozD!xq&iG%({+m60;I?dp9j^y=mi=*yYG3lf0Nh&Kc=6j+lbj$F_P}(<7P=ii5fdwSZ{G#8~Myd&nkMif@?im^v8$J zKw?_Z04r%F0-;t^V{n?xHTn!gW54a541K&@Lv1K@Zvwq^Z?P{UJgP-f}oC$Arl&^Kg zyATtSeep%92YD)vU?zg5PvcuN!@+ktr!A&00qQzqI4H z`KzMFk944kbMA~#&vcA8IeLzXpUd*oobP1*D~7%iHI}XiS?_XfSZaRC6}5j*Z8HfOclTI+4LSapSZs1eSeqS zO#kF3X*d7w?0tjvBYpYn!h5>SqciuczSR8~F!A_&qD20auG#5o_&dVQmC08-(ocNq zedPH^HN%ZspC*M(ZIB>}JHeBE)l8zI&IeT0yr=CMby(m6j=!15z2mJuisli?dnmS8 zd$VoMxcJsnFm;QH-}4tos)$$6%F@)*N%m4nZfPy#y)Jp;CzAkUm`QP^Ft}K5i68gG z|84eT8v0@AShEnZcZDy+^W&u*;?=$G?fp7tzIyo+ z%{RyTwuv)!YSKid`y*6>FW#}EXs_44&VYU<6}YA@r|q55nRjZ-iKc~z zn0H4RH+-mpjJoh(rGc@+OGg6Ug0oM0HD>W8zl0VU&|h*b{w%ZaKI9u?O>R27dCYFx zX1cN}K~T+G7}$16vD7(>5zn4M+U_5j<)IyGxY$@8UwjVueu3FSaU)x+$wd5cDkp25 zk9bHG8Do{)-6t}XvvsN)1F%D8mK1(6ATicN=@qWN@sgcE4%Q7H;%h#csdvJGJAVG4 zrR@V*|IyD)bl`jsJ+T~vxMa2M*9vEm$uRCw3K0a#Df6Jrxhd;z+7z|a&h@I(k<}|= za6-0YmAzy<-zOf&97q$E$%RWi9a^R)j}}OBBFVX`G29li(B>oryrDMJxmQHf*&=3K z>j-moq=V*Ld2FR*$)J-4Xi_D6E$a4x)i4%r6Ck&DIc9ynH2KY~bAtj6?4#D{iRm-7 znrGZ%T0=0WGe>2R3%dL7u_kn8#GgTKu^ji%TE_KhP<5~3phEsc_Om4UbLDpfF9>FI zxhw63A&LvZ{@*Sa<_pR+xm@pn9>M-Na4rd$QNi0E)ocg|x*3vnG-u9RkFpfgFdAYN z^X1-aCmwYNPjQD-8Fcoh*eGs)n0sW@T*U7dBTbZe6@k%Ez7-nGowzWvKu$FNCG21} zD&QvmNjiQzn`m}28{~^Dnl;PxC344!WwYPiZ|t@!#H}Ck1&ySw;o2I5IqLdcn?hvL z-8Vkxc$Wsl2c|^qyWSkCBMlyoM)Ad154%P=3p4_&%%!d^D zevp)7=|fgp8jCc|Z0pw@lfVk9Rp1-Lmd=QZH&uE>K%D7|3ZO;_^T;Xb%fzuS?2& zJ$P`oGUn919ijNhSlzf(>aypB8TLZ(sv)3G{Ts|e4bi|Q_*uh_9^GU-ZuLD zi%6mqahS99T!#3>ccRi|#_3_iKFFvF#FTL8O;@NF{7jr*LoRGC=$hzwcb9r}%yjUc znx|!eYJju99L(j=CjoM+?DmSpBit`2N`Ki$@>62Hbq@zinlLUeYl?Uzhgy->6e{4R zA$}3!G&qT;&@6pjtD^f=XDsLR;WE<;zym~o0r_%1ml=Jk-cOv(Lbk5~CmyeGbgkSL zbUwKA!0B_g9E?vY>D|pfZK|lp=h}kj+se-Hv1C6P;7gS8W;g{V6na{EHhnV} zuOLF*55nV39l|`6lpBgU?1;VY~*r2Q~r)e z9S92!hCFtAus3n2QR#NdirelP&>fxmS%k;QRWDF=-K__`y^1A_GOu6X*_o@yBs}Hn zmbatcev&V`b)F%-IFW?7$Wu4()bQ^7p$Rz0ZMrD;_OFqI+z#%1%UkIHUFETvjV-dL z4;jMuwz@pBPI|MD1MQ@~S6SFx$CmHni8R4~a#y;#Fdg5oeOJ&+-xf-#w{Uv0O$w32 z9?{Jv58q`T&}|DMWb;d6WKhRdSV$E3Y#sUiS&`SpI3<&+8?`?&$#gi_XpBQACZ|-? zAM?DF>)=}}%+ob)gh2)kxnxkU?h(-@ooE+wwX{M7V5;bkH+xiIUrkIZIGLbMlBC}d zK$#z^F>Wp9Dq1I?Ga6{X`qWf-I!3gzjq+-6doG+(Bz?G0`nGk9QO_lYQ$Z41p+eP< z+Re4M6O>vvg?b_#d9Inob|-S6#}7Gi>95i+a|3#a^Rsn=(X`wmM(XX@dMrV-TM`(* zI1kcx$WZWK>6o{nqoQec-in40hJ&>5ZwUI z$Q-`PrrQLceFX__NuXuM$#Y6vuVyMp_*L={q&Sx4geqFiznU=xa$(=P+`pZfjw+y4 za6l%3vO-C;dZ$XFja5s?8o>Jr9c}v$e(X4mE3`?o%W9KV$iuX!=IDZ*Z=^3)L2Ny^ z^+u?-wqP$luvpyU3;yU67z}^8(d_FRI>t{5$_KJ!+Xl#z*pljgce+z9>T^}+a*m31 zwQKvUbC!~kd8x-Lp-~IMEUs+}+AE{JRZj*DeGwAto&mI2lFd=zGpGu~vtO`QYyMIF zJhg{WEUr1W0cYU>?8wOc)w^C;L9W^xXW*l`immtTXU*3wdVa7$KD^=~^?A+++vgm6OF$W}({MaM=Ofr2wJ*`4ol% z;Q&m-e6dDAFX^ZGaz~p6)M*i7F+f(Er?GSlT!`HwLv?LIJ!u(~lQCswWpPU7+2-2X z3l0nCBPYozzRW~(=P2ecBy4#X`l=(1;DyJr-LqMp^#~+}bVp{2LJy9$Ig(nnAg%@F z$>PwG52-7wK=IrpTMH25j6XZR!l8qv9D%z2!sj?fiP)V7tu1v7@5O433%k+g8;WyLi~NF2fTxmmO^Ki;$RIN2ix>_H1$lSr~CZE$Iw25s5@T6f&YioKo7I$8F( z>=zH|`i561S!CB=Ev4Jt@I}-1alVG5wj?PovcRsZ0yKa=9_h{>joZ*6$l%r^?(8s; zU(1CpY;jo%GsGW%MLF@5Rp91C(d==e9EQvoiu;2RT>KSWY6Xol82VzUmb0i-^PSRN)omgxqjxC%V1 zCTYq%&LV zhB{hC=z~hT=ipi3G{B5n^ri=$Qr`oc4akhlL{`S_069b(M@1A3jj{Bdc4XbvrV-bB z4W8Emcfu)zeIg(x(V5RVp>2~Y;AG^H9|GOI?s}U3@o}6|I3*67(mKhtHwdf+1va)Q zv2?!9H=v&3aPAr}>5F7M9q?VU-wSDKLP$Bu9cm9X9i6?QeRB1M^?07?19X!v2t z3jAW5@)?w)kqyA8{_wnZMtUvzG##LsH$mNTI{{aCSCp*F!5L0T zb%&30s!Ml+kPk8=?C6!fJm|E*0~MGD{d^mxKY%5H{C!DM_-|Jh(iWbEy^jY8jO9yx z(4#Cc#-~)AT)_=R1tMM1U`(Mft^}A-AJTJ^RWNa}*n?z$TM63CJK*zDoAJ9;DiObJ zVmnD2kk@0PyN8YqP||1#t&VSY#un$v4ON6%qBPk?8aoD>3eWP&C`ODxl}Bo9Fc1^1gHCxAz=Ccgc!e}CS{ z?T2wCpur%+O0fv{=H|b}^9MiU4;FYD8TTCW#pbbg%bq&}!@RQ*H83qh00U`(95@M* z)3@DK1W>YjDYwp4ra0Lf8VmD-(G8#H6s}o%UGwZYMvw(1WFml33Q6L`p~JFB%=`?{ z8R)^2*uj&|bHS$sMMM7fph$+-tWSfYf)nT*F{x#Db%mP`HQv8frKh&eS4GOlm5$7k zP*eNZ`Mh7Qaln}vJSXVS&nXV`#-_9VDo7sK+NBf02Sv?Kb9A{p;hsh68N_m3YyzO9 zO_H`!8VEK7s)i0-I}aHfuRVNtMJXhCz0nrQ1wESHWfp*R3^?DXV-kQo3I<9=|J?4h znd35IokL|NKrQ#zQ;2*bLXaPemP;*~yRQaYs#q_Tau|!B@f&t&80wA1%EX1pw~lnW z_~O^RsddO=+mEE5n66o0SlMt{rOXYE0%S+BsOm6inDPe$9H9pBH_rA&DW6`a$~Z%; zA6ebI8$Y`&%p1l`MCeeszs}6gNgV^DR<`J3M@YzhJh;@Xjw$Wd>d#ju>|SkmA@W3o zXzFBA1r`+Xd|5apCi0oRemA^k7{6F%0#0{D=R54q*|TIVfgAx|&)B)aQMBTouq%dI zEtL8oeW3HfHiG4xG~;zxK{iDC+;rFrL~5A!-~{5hXrV zqkAQWXe_l|yt%dOJrTS2v$SZhd%8l(H$C7EKkK@0) za`AJ;7}1bWkjfq z=o-G`uKk48YR1)vq>p~SqvZTYrIKwH+cdG^#h7$vVjES~*70VATm+R>rRYE^U70@r zCHWKpcQ&qSY(D~28ay>>3OeZhzz4_xbjtvC4)6xS9w_lfL3T!Z{mRlB5GJpQ&%wku z*Y}yRmKxYeZ491bWfxM4dP=Yq5{x%gqY0_zHj4VHzdnK81J}>`1J8h8D?bhk)1LI2 za0;AgWn67_l(@2II4c_Utx+rtgrW}ZwyJ}(3HB7ME$*#d2H#7Rf~p{aj;dgCBoO+= zsN`VsNselD9^ZU?WWrt0)G5Ss>w9bt<-ArFQtab)s8>(xY+dyzKPJ@QLvk-PX!4tj z6tP4du@JdmRqB?mwxr^6u!UbMq*JupiGM^^O`=P|^rYJQv5+f4`W3vMHg3>h3w?xtzy$YlSJg0* zkHZxX`1WD9D`{-q;we#qLC~Dg?ZJ>(A~W!kZHt8J4w}41MYPu&=sT*5}OD z7QTR%r5HuVDf?44!9c6nbk+5RwTPPj@-G;XXmnp_@v!s4Sq|%(=sQ}1LVnECXlR6x zDS`r}E}=AB$rDd*j2lgM5TLqL@5Za^i(Gb8yJqq+o_#)sp5Oo3Y>;IBl;?i^xyrH3a)u22 zN{KuRLOkySu(tn=wTS@|+J_Vg7){jsQo@~uA3@r0U({jKby>hUo@W2pJ;DLT%1U&5 zrbL2cDvK{2#4?WQ;KsJl$q1Q82YBl*>BCQ9*&K~_;z^yhpq$qVk+6)23Y@^d~sNG2XS?`$YiAV+{7@@Dn1x} z1Uecr58)m^2RC)%O5j;imI1f!KfdW9Abk!0#~;$tDCxSw+{;iEp+mvnr2wi4Me_kG z$b&Id{od}kw*8(8+P(81A2hHl^gsR{>;*d`TmOQRa3{#T^9~5V{ueJWsz=JhITd&V z7x5?#D9Cd?5p)L1cmKndE6L538rPzEy?5rhTKEU^=n~%*wt(453}E(2q!1-Qw#RqT zY-dmCMC@XUxj?5VY7|5$jF7Ht9@o$sdMM}xSeM?+;6nV@O^#U|k)pfU@EYKmR9s+2 zuuSqx_m~6yHu(bMSo*uTPf)>dOWqM>3vLDYBA>mg1&uhd-MI4sQwge?tr-o85tBK5 z5)C;=Ohk(WQ6^tP*!EM8=mc77*_@@}6xsv)eRRP3ki-Rhc*K8;R4|+TZbQf^xUVgg zYb!=WYCe^T0QB@bCmDe#7$i+`*aDVqVn@)WqQWAJ&@T?`lf|lkQ0rvK3h|0n__iBk zfzx|%|12)Aq`ysPpuYlXwpZ`^OrkP^8?~}NBkz0Q+OQ&s!rKFg!~eeX+U8-M`GY9)ke6>&<}T!2sjO|kwB zV1E21)a>@s`m=N;0nj)4pMrodd>ZhNx^Az+e}#~jkI3-crWU5g4=li(t=`t~WoN-) zm;YWR);6O}yuDCc>bUZNbv^{ph;`aiaH~E5fBEa(u0*(Op^x5{pv$S?<86~o3xvg| z^tOI8Zv{D^9F{|M9zo&73v21l;IFWM{#D;3iZ8@SP*sKs7=p;%;j-cHOcl`B<|s%M zS&(A2Yv`!4#VzLHpBKtnVX5Ee8Dw4uv9~aVUTXtis0AWtp|^35T2&%U0m)>ejF|Q| z4FD$Mub7Du=(ls;41H^I<=h7{fF?q9cL^uZEapd%@Glma(`~SfVg9!Xe)k1`F7SX< z>DYw>7fgmdc9GEle?N_>C;_mtCtykb3zky;0``*4MPKuQnd8>U%Fv_DF9@WKEDCN_c0fdJC)hH>3{GP`Oqv)x<_xkZDFyK+=l} z_^_nkoKXi@6S|7y?6^g#V*knnrG6_j0qIaW24@hTgvHTkgLl9F=|KnHT`N-rS+Ccy zNY_OL#{||d_8CKQywGS(TfQYnoZ%Rs}Tj@`eulx_v-*izCUAYJd=p7WgFIUb+*@P7P%@zU#ZuQk`4V~)JXJtCrCS|ii^DOAP6 zNibtp{#p>|I5coE0o{pQvCSME4QOUXCo z=U>Sku;e`1=&wQ1{VP&5iJS{VZE122Hf)?4n=A4#gHrlUBJ>GGs!hZ_v@)}>Mh2v& zOG|^9U>w6PdZdZM6NOAB z*W&qe1$_W(MS6&ktG{|u9>CPDR&3)Ww_@G1lpek&dK6Z-%kBO|fZ|(`hz#_>=4b#o zcW1N&Qy}YEca$tM!R`Ld&Z`~e+KLx-|Bx>0fgKCTko3@YC(@E&?T>+A*+===4Jd%g zxBtwMC4pf)Jm>$&k(og9Ew=z>xo7jI;-DsdliTt$hwm7M-fE5Wsrl4aU)TKG z+Lt0EnTr||j*d3q5g-oxo9F_DC=!A|2>;Q?8=KA6C;AP(paUihUC@-{I^Zr|kG)^7 zH12fu;0m)YGKq|3{=UnlO}+P;1ho8^)>`$EEKK2e)A0dakBCmDwi+?C;AxYc-I4rJ z+}js@|lxP+;}(ke6++AFLjp7?k8^_{5l#*tOei*Q57m6&KsA&q}F>IWs_@rqGD zNVq!qF~8_>{-hg4RpXe(`m|_xch1Kx2vLDpv?bv1>@@W}Kc=u>q>qZ&KRG1apj6PV z2>;u6D-_)QP$Pr^d+3Io!wtT3dzJs$j@xcIHgFW#zwJPsaLv(CZmaKmLN3%%-P(W1>m`1_ubs52RkN~AVQ17cS6q1CLx-}#^W-*NpSombl@~UzjS$()2MW%v z!_GF3>ZqL!nE7WDB9*$_)>kH+-&ZfmkKwfunW$CNauG=8a1SI6_V2A9^TnfvXKz@0 zgiHI{${KB~*FNE}C7VNT5X+~uNCz7LddyLsE@nyRMoGZ%Eqgb@SCbWvvf?S*T5NK3#C{=>;e5 z+wv7nx6|9(uetQzKF%McoZlnZ%HJzyrf1=QO*lkp)bl!emBX`1r4&>zpK!PeT~Y|i zxi`9<-nY_kMx%r`OV2M5eAliQ2`1*`F^$*)WkwGYcBf6=4r+J&t?3#Nx%?&KnD;QC z!;@-)(|E-;*OEWL00uiqu?qze<^jd{Gyi1_S@Y@m0Iyi+G!d=LSH)Klydy?Z+Mn>C zh}nXBEkK@sFVner=euKy=;I6>b(TZ{UadPOg8hH^5%LRrfo62s7 zOE`coH_}F-aaTRMJa4wLkjCtTT()G|E^~CLU5}lS*YS6X(a@To#Hav<*NJH!mTx)8 zXcDQuK)C| zb4chNqUVp)Z6+OOO5mOXMMgyl$PhF%Xu;9`%0TTJ{pVfGR*2nC^Ie8B#hwGx$DX1ZlnEn`=cNGTDd~Fjtige2>oadtmiHL8oOh2 zJGoBlEDcw#muqJ8kU(~$W7kpZg$M7hW?GNAW+|V=0W7BTY>m2NCSRK;=yxx%<@6s}?r*Mez;x3HK?Gz^n-y$43&H#9#)$f+*YPK_A+tj3zp6V|S zhWDi}(90wcsg9*eHv)uC{V|$K?m$+9eq%7ktHG(j*nFYSd13&g<8_yfNmUxnXy=>B zlX`QSoAPs6u{afvK7`gYm+H<6e*)~}UfmFb8J2ELfAB&+w- z5j0f5Hdjw1C!wN0QF=E2;>>1n;4Ngsk=Bu(4*Duy%7h6@;ydQf4b0M>ASd*}RrU8b7SRHjLO*3pn=hwc#W_kXF1)2(J%(F3Sv*EkCv}#H9*+C2RB@~OK%GPFs<@)O zt-eNgXN}g!&9pm0zt7EZ6J}(0B$L}-s`Uj$+;q#Yu%5xYFo9Tvd59`dJ8@_TKVa=@ zZa}WAC0DeQWmkQ$*>(Tc)R%j&C#P*__0Ex3KYt{J$DZS4_~-_)Jsdoj^t-EapWO7k zofGOTTVvFIP6Z`_m(tOf^5@3tH@o4Eu_gy|gn4l|I=KRHC(+iaVxV?GCDG?a-mHOy z8{7eqNhVS1qm(B^KI0%^ImnJ~m3k8)W_gSdrcE-%ZWJoNv!P>eY~SE-P03+H3m`X$ zDN^E6>IXA+ZQmn!3`huxcW)Cz?;R@4eQHjUe@HX+RV?tRG+8jh+q&>w->1Ou@@WJ& zrZlAk*A=4K9BqDB>!RzvVtIIY4mW6RZ>n)WAOl+;MmKVEe%HU=e9RBice7Q4SPGzAKs-BmN z-Aj6-)KzJa62&Dci+~Ps7`2~1P`oc%R;LRH(2XW$F5IrRS?jn?DXt=Pi-n{|bJgh@ zyv}ZB-aR$6dm|w0LHdC*FF5<_qHYC>#rYifc{7t*x01w#=H)ANhRZqeT)r6Nb`110 zh~z^VmC?ONNsjwe*DcXmrj37kI>y$f|8|x`@dpdteE?=M80tS|IR2t6krbePQ7YGj z6}0Xf=%krWG&}PzFv!2I*4Ej{qm8&FK@D*T7F`Ccve!^`A znfj3$+MxgDWA}Er-M3X6I+q+W{d8UU8o*u?g!g$EEQ#sYphJ-!mm15S^Lpjdm^Er& z1_K)Wq{d?!_b#qk$f&9T9|~n3`XxN>ATZw zFjCHRY&)fx)d$nwp9$2QzXTzrF2pa~juAM;>b=S^vP9$?YE6cmASd-4k@(R1GrI|U z4FU5Em1HnJ%zY4J@giq7Wr4^i%Rl$e_ogNVGYkTfq;Hzc-x$HsIzs^Vkwaqc?TUDv z21s94e?Qcv8qOMcP1IW5AUWFnhCOv*lGX>g2_0>eVi&LwM>RhDggnv}>1MZcQ)fmP zN6;lX>M-Q;T#HHYRX=~8&=JJuV87X!ZsxGQ#+~Y|HDt+_x@-+5T-~wnFUdWu(_2d!FDOR1?Z0#M);3B-+ z=P?l-)G)X#?-4p0^fX_dS|J`r!TC;DrKT>^JFJ&G^lx?D~xn@vD0CV{t_m_)e+SJ)4aK zxA`vyHTKJ70teMLFsk1HE)@ST9@ZtKzeb=` z`XxhT$p@D=8{t0b`-kMB?(x8JECk^j6OdE_S#Oi)_DQ|rS3B$tZihrl6eo+lC#uIe zS-el3C^D@0yiR7&SrU0}n?>wMSLvWA7aIsJT^67Xl1)c~kCG2`lp08lb z1%N7kuD5?Ekh3X)r(DoRw>iKM9w#|+=SCFdEe3EHF+lUdF7W6;A!)qN|*a z2MVI)2M6uGJ_c>&YkAWRq$=)Nn#kEQYiXY{Htf!|E-aT=)yS3$xLURq`s|$GrmwZ$ zQ<^D}J%qb(bNF;A%+xsD6@IGGnl>wS&yf4Syxap_tuOc1u17j)eJUEE+MH|ET&J1x|&uoSK|=$E}Fb8*{9LDrb^?1dHW-wa)A%Pr58uT%W1}%u?cJl+Pce?MUj=UmuJhwObN=NvcefRR*PEkcVJ`5tzzXTl z&0Y$KwvIoYU;=*Z3L*cipC0hyn^Wt@XC7ETh?{S@-->HVdif?#dkVvdkm6&N^S<4m zVS(p!ClD5x_}fxTFt0>(WU1%I1)2ca4<6%gZmo1)DRUmRD$-i@U)vZ&bo8ACN0>L8 zx9y0oNR@K*Meii3v~)#+Jrnk}6ukz*OzEJTaHFZyec5vMDyKT0Z|H$UH>ChMXy#qJ&;MtA9~PaJ%>CI3}M!x^r-zA^}Un_yMvAW2Tu;h&W*&NuUU0jQ#DoWK7V|Y*<_U*!;;dxmTv!jXNi8mBjMC}|FxbQ->_T8>%8jcx#{MQ*=}^Q z6C?%aYHHlH-b^6?O&O9fV4OXMv#ol5UNhKc zVpG)DU$k(naD5cSnnK1zxO2(#5)^vg6XLn_8-hfVtoNI%eSF6WCRYM|C5kEK+GL(B zA^CjUuDX+nCR4V{O_96BmiS_Q6pPB0@&q%r?!+l1u`M&?VlYpR_eR$JGoIv}(ym>B zBSD`_cGOHbw{V2LRe6R>)JieESrK@heJ`8?wspe5tpVf z;(-swzqF3-GuWMf<@rHU|McV}$kjGZp@nmW@um8HC+JexHy^Q7$m?n>g|qkcOq7qb)0NRG3{dSKcYWDzcTB^sDI%o*mrU)`SEyO-p-zHBR zO|3uIeMT1*U;g4B*ImBQ1g%Ie| zj;5a65QqZ#7`g{9Yv;@7g4lu@EU&jlkn{;yBZUHGGdwCBh?*B}<1#4t64TZjP8Vn; zlr7&aNfnG}O|dkZ>a+yi3nesb{?#WSz6JMucD=TlD{J^B9& z3jorR>-er6EyRMm^Rt3KEVWT|U$F(4omvrnY9hI(X8Q`iB>%g95D92EurOKN8QqsM z8Y#~wYCh)z4KV)GHuNX{Jy?r|C9@={X%=k0JHb% z%>7mOoOH*-R!;Fl8GiOOZn6C8Ps%C{B9nKTnwXvviIo@;)(KKZ1F z{=E^zq2NAe1n%F|RZN4aoELSC)LvZh}&W=0&kM)msFNDtD6N6(6OAknY-s-6HzX zO~|C0tC>2Gul*pK2KPDVQhU}T>oUs=&*cWcj(hK{cb*56^HO|^I3wJ+8-W%2hKBCx zQQ~VI{kWEbNm{$JQbBe75IO2|UMt@j6RJ@vR|2bg!QM;C!zv)9apK0lehT`iNq z$aE?{MMei#cMO2&wxrF1YRhZtCjzXicUy4@EDVo2C5fufD~D4>ioM&xppDog3EjeP z5n?eBZOT;+{xZ{+L+9Iq4!tGgsvtrBhWsCuAYy256qBW~E33J^>-CMu?S%c1I7oc)_PZ4#0Za?oI{nn64>Qb8g+Qpm4V&zd`HBf`Ie~6u=A8AmG9~#=FIM> zKWoz&|M1&_CVLt{(8t-G)YSd-@3ZHmGq#{GT0K9B^r_F8T3WGZ%dWBCOZ=56MZXr* zT(Bh{b|}RnQ|M`ljqab0IxahBNI=Cebk6s7n;jM0?=qPdO<*q@c0`1^?Jig*O7DA< z2_X{Qpu(Wz@=L;&Nw>2o`4-2wOtHk}gHF&aZNN^ynE;5)AYK>h6k&aW+K;ZLHSLX$ z67A|!X!UW72i5szj!JZeLV$z+wR>=ssOl@@LZ{rzqn$a!#o_NqPzSjb3V3V*zV$AJ zh!ixms)y5(@1KE`drlwj7craI_VA3N8ODvnaJggz#wJ;PJq(Dh--%-oXSIPvu11(l zP%*=cai9HOtz?k~OnBT03e$`b=Frh<2&YS9sslYVY2zO}OrTztz1vIH-#WI1>F)jM zOZb`{1%<1vUeyM>bDQ_$Ne*!wY?vCAKAWDBsL#f?*m3a+Tk_)dtKAs0trx=Ckmrv7 zY)X%h285)1Sh(*4TP~(GH&5kewU?&c|)k z`o#7w%p29J8{-2T?c-p^0UMwK_*aS$z7TwjGSS*{s_k zGlNyul7Mcve^#Oa4bq>WEU;XVr62sHe)u^8d3&w_)U9asi^cxu54@)YQi=6w*7aeZ zuByCvSH8)^cru-*z398L)#aQRGs55M^n zvhX@TMFIg?_$vQD;Jd-j0ph{$FA5?#K&77^>tCnYr@#Iaas!I}Uk}tM5w&&&iht-A zfXT^XToH_$nZeFUM8AUW5D~k2`3k{!b}B z2LR)rUaDvY0Rg%OR#uvON(6Wq|Ak-x=YF1D0syf;!2o(n2OMBb)V~lBS&9}X2*f^s z(1DdTq^=v?Mi5#3g0O(5eA#jANa{zIh! z9#yb2(@KAQ^GioC#c++`3LJ*RKYWy*mk{*!OhTL!5S#PT5$8T3L{FML%>yih|JUPv z^K^d&=EmOmLWej6S*C>_EkR%0y)b3bQ4xG6>aXIbYT4vej7?Y8OTLq@`GzACFy#`3 zy-WxptdPke5R=?Nu(%Z|O0~^_bA^)pT_J7aHM^x$Wb(r&S{dz*?R=pxPhRIHY=Pw*jwo9jHvV^^1N}z$5Do?mAhTKl@Ygi$4+5H{Mmq;@VyFeGhBb-Xe=a)_@{?4!#D|JoN5A zMd#0t8h#4_l7|I89XB`_vBI+7uJG9J<;gHS7IFt02v_${0#wRh1Q|hqN(-2myMR)A zISRI3BQr)I8c;OKC4EHv*NsNN5;?~S2KKysfeeaJ6mn8D*E7M5dmdx(yCVXN=DP{@ zqlo??w45a63xFvz=}h70Aav;tpdbB_N?jF+{Dl^9e;qsl04Smq#PVGo0O0SB@s($`pr_Lkb1YAzEw|11_V^dkg)(WVSpezB*y#g0S3POiSJu@_J@d> zB1EiBc=+jGm~Dq^PC%$gaD#t>`$*{h)uSJB(STF1hDrWC2lgkmEiM{0_9Oxx#P=8+ z=EtAHt@E3$2EvUd!G%Bp-a(Kq2mh3@f2NcFAlaq$eT@7O(N%|wiVn;t!XUhaEj_uM z9|P+3*IR%9hdDgNeL{yWLHm;sL+f`R4JdsRh5%SFBBRv((Yt`xNVHNM;=Ajmhy z5bhhw{>95L8Of~gM2A|aWp}Tj4vPy1;D2r-Ja_L#B3vS<{DjIs(QeQgdI6m8+0oB# zm>EDLA)@Smk|qD#D~rb`^+P_3i5Bx0YU^`KlBr9?Jr{`;8WaGZyL_wb-8aNWzy5DF zy5MTRy7$)C9Q5mD!;amSBatT}C&)@f_MUlHCkYj&RDbL?h~7jIjvi-Zxs21C#eTPrM>^na+B@eC)kyo!!mwgKy#Zy_ILbHt3Ap%IJKEgA4p& z9{+zp!G6+4Ph}g4!7SW;jNVM#mIZo`*f$U`Inwt$P-8K<0qhy(cL;!!KwE|j0J5BU z__PE-({|2Ri=Dx*)bL@(E~4a=?F?8Es7ZGXqb`J4=YU{LfA}1{g zX(@X>`=KgRR2*978~v)a`yff_5NP!Hzq7R;$ zIFpi6v`M-@p-eseiswFdnuwRgj2pUIIH|IasORy!FL<<4EDGzY{vK`^;S%FD0BffL zQ>!oFcr1aiQv(zK*3&L)2~k;Tz4#9{XXSlM^)nr!)e_3S;j82-*uL?;RGGQamSZiE zv+1rL(rn+sB2P9&pm4Pw$titAYu#CtH<>5w&9dpo)aMo5%Sm2UtQ(**H;TiqhopY! zKt56`tdivJ3m`|<6`+Uo)T=$J2-oi)L(;tLeLY7?htlL8UeKXpG*y5n zv;3BN_0_#N-qU=&VLb61tB0LdVD0b^CrXSuOFnsXdBQe&!jS+a_;Hk#`<_<}u815K z_1iE(i#LNWCrJleJ6ILof0?L?9<8~)75$N$V4Y{fkdu-cDy%);0-LQaCK2_hE=CZ7 zffOt;K8woTlv5?CVPm#ebKRB~5t*R9u#X_((ZEl$^lN<9mhTCP>>}Zhj%uzSnm2mLbaH^Z4#`%Kgmbd0z1;N`7 z0RzjeNKi&HIy;p-KE4Ey9rj50)PW?B3cHJPN_xyKKj}-?#~EVWED1trnJZh5z!uQF zt~MDZLxB0Dk9MduHYdy{t+TX5ME41t2T8_J^MyIz*-!sOV}MNj{8{MeRb*quwbBLi z>F@rr3)vmB#o73-)rK9jOxvwt_4RmUp%9smst! zttJ9Fu7^pfM$5IlY-t^UN$0-tDM@=Uv0%&PAjMJ3J4Nm?&mnTFCnBV!`i&2_OlOU! z!KC2%t*gr*l5oNK^}v`XBDQ1!Y9`0rs0F+(Ew+>U;xnh?g(fJIO<(j69_Pp!jg%;b zN!m<`L(6+mB2rEfR=)xW<_kV%$#e+Q`~zzvS=e(`%a{;!=sk7DB!yERv0R{J2%&xu zrr~sh`b$2?FS?iWO@=#O4el363caQhflpbL-x-?}>2aZ7HM$4RSMV%+i*sCsm)x2< zObt#bTYUCIRN+(3xZ4u522OE-sWfcqxpINSjUv~?kbMR$!v$Q|_K+m1PnR0~ltFZy zuV_4)V0mKzO*o{*_fROcy3|mj`m|fS2&s46+*fksW^WmWZ(gdXNDiB5Ko^#c(X*iS zW3wO4-CFOsUR1M05|w$2jlLqcU;l1X^pp=rt_t2VP`I+>x^P~ADfrH5S;{xZ_**yI z#O4FagQo{lA0bQ}wubsk?0PsSp_;K}@mqD_=AVr|x(}Ks^VQs-HMPA{S?ajpA{g{^ zFB!1K%`fjWn8RJmt=L7bcA7QKM~zsf#=65<_%YJuHx%3a$JmWaaEJC=8t`?pVS-QGc6UQwdL4va3|ifTy8xN zC)`~hb!>1yAzQ0hY99$oC7LZncOYQ#|L8OJu%4v${j)xTb9{!9qA@3HHT*@wdb#Tu zMa{(3owMyU758R3#dR`jl67(v2 zhALT&iEm)?2~O;0ZwV> zR#1=UaM;$Ev>EBGJ$9XX$TIZ;Gh8FA14vst=jd3sw{$S% zT<1D7p8J1aKf$Zq;?$}lOoLw@-D0ha>}V=x>(+b6MI#mb(!ZY9aJWUo&ZRk$QQ4bo z>&5L&Yvbu7Wtf$NuTgFhB(Rm_9VGDjk9T4!Fh3e&N;sgyp&c4a%#O-3Jh^(W;rePr zS`D^Ns1%yd93i8OtseRmWzlFeLwKTTTi@y_|5*}8q<%TpXQTGc2VLnCjXhc#kK1=O z8(o5vsF=psQUiJhgO-QR8JY&)ft|LZ&woS5|0rbF^u2f*Aa5YUOeTx$Qg}H$ z$!D*eYhpWj_%Yvqb-I8oe#axAcJal+MSXSNTqX0=csJAHB`&-2JFi>N@^)PzK-Vz! z<1in>n(zgcQlvpT%*N=w$*2xXnji^#sxWLQOt{A5Mjt3uV<~&zc4#oQeBZYS*uG8W z3a+oT$-+{zKUSAt}{{j(ruMY5R)-;uGt6Ays%0!1uZ>u;M+9(Yl)rSKyKQ9CYkKEgZY zku3l6Hi}={`}X!@l)f3tZtD(8fKPN8y+kW-c9zTeIv5_d%J+!|>bp<=DF;bb!zO^n z#L}oBDhvO)a<&1(7uIBKy!aEDVYtc2NBdd+nK~DWHnIYv!&8i465)9HxfId`{v6{P zkNsOxF2&)2)1b;AnpV-}6t_~fDx!GlIQ}XdaBiaC9GG|sjQ%tEp__T1ix?udrv0v~ zj(b%%?=uK{F0#E9k&%5oEuy;AiIG8BHYw3jXZ0-xo+ilyr@U1tt-7R2 z0$tyV=r|C<&6GkG6Hh7W*Upp;8D&qzrd`?T49msh$gouSk+vRM`L?{xc@n>- zw!h;Cx7q#ev{y1w5V}OZVBIlA1DP{fM$JO*0-klhcmmFME`3dm8tEcBcVD;G(Z%?0 z+RiTY)Oak?OdYxY;B6}HViv)&(8r;>vuiX60URAOl) zg)ok=!-ZVH2xZdoIuGuG-8*LPJ0k63AsX#vkfI5{rkF)Y^ZZ51FMsLcP+(;fg>9-^x%;aSf{Yfhm`B&?U99=CD`h6A^TH!*sElEYSY+yQ2K& ziX&-K4JNC0)3BZ6TYsE=m&Nm*izi(9Z_e4;cHHeZ*0lhS(|r9!@r5grkgc9rQWK{i zvhWibmF!MNwH$iW+m_6>)n|p?3|#k8(EvQ*;R62J{XAV19RV+FRo%g_3rtw@#1kK@ z^{MURNcg^HM8#1!-*?>)r!pEO9mgDJPi_Bz(Pg6DnzJ&o9E``Ce$Mi(-elB@ucSF3 zp{&DY#+TB2QN-O2^~Q93*NAlLF`YMVyHYX|)9__z=bcO-CNKE5%hX-es!T06;Z?YR zPfk*h8s_5SMh0k+w0hf&!FW~S38vo~{@x^7xp!E-+I}|=TWX1$2#5NGd=9ZgU^$PL z7)4L-W;BR2bX9a6;jk+^)RwhvtSDg%yO2qQ5}||ftsT?6?AdOT!Pu>oqMrwc)0R(2 zn$ItfPEodJYfarT;v~#ZE?17BOJ|`-_56LdqF0SCB8bWsy7xOUo>0Q;L-=-aX-$jx z;K@77TY*V$6un~1>}@KE%Ay$E1^82scHr~NBSFV>1`le}{g&-gxh2ZYM?Dg)7;MNr zuMS1wJ6>I+)JVg|=jAxT$w}3`slr|_(-!GrWUPTAyF(0EJ2a-7X*hHiP5_j4X9wOa z`8q)q#qv^LitxDJw5{({$ZhjBGnA+HW0`6*2pM0Dfs7-TSvE)^(COq@gXrGc!ukP- z?rST|%AoK~u4hPC+j?C%uDfJks-<&^PNEOTbu`rBY~0O*u7Hk zshBJv{l^njU_VNMuJiu(NGc1064K2^0?0LK1~ihgqOyv#9xRS9oYZu`G1EAHD^x>K zc_ET4;Ftgi{W&y%g$*Bo@Mrh|? znOXndipSD8zZjKfoJfr%*;S@xZc;rSbN;_rtl`+TrcL{;vo29N zW|rargXbNCdNa;P-v~5-lND%`+c2wm1)QukGYVq~S(T{Z&5Y|HUovd<0eTB6n=C}` zTD`dK?ah<--Y_IU$G_uX&RP~W%4V_2_!y}OsBW^~t8O}(#7=sz3RRGB{EFk|c0|Gq zZ58j(NC&Luw@`!vRhgi--aBz;L)K`lH~C^AMk`(BnRqaKQ7Fmf(dDy?BX5z=4(T$h zea10bC0W?#l5kTpQ1w;P_`SkgK+JADM9%pVLU8E+6-^$IJZ1n*u)i1;FhuPuDH0BB zY42i^VTcwYfvD{H{GE9!y?N57l|VfGJWQ2+Tw)WC>IP&N2}kNSUm1@zAU6NW!8&SQrA<})nr z!!Ju%FH+7MWrx`!GqG2;bk_1HqRc=QARz)MP$k)a-T1zcNzkAyc%$PA^cP6RBf%hH zWLr}{YE@THi#Ca9a}|+Xe%}9!-`xMr<`pv6ybGa^=ARKTF^Zk1 zB5#60yig@jJ^IB=_k~2iY)Bd0VRiAPHXsNI1NecjmY_Z&Yzb!7{ab&?AhqVxmxZ=> zAkYZVTS?sD57KRTxGeO+GnJ3eWWeOrmxED|chL_Yk3uw}x#<1VWvIw-o`H9l|I0-{ zyqn#3M?L0=D%s@19C7SE?i0Whv=!^e%7Kc1pZ!$~Y1*G!d zNj;NId1Arvo*SJJSPCX23BA`RV6 z_=&4Fqw8d-85GndX>pHaca78E}7t~yVE#|*9b;fx0z_=w)Er^x!>ow&wO z5t#AG;`okev$yr|_||db)wpbQj((N(2IXvuVMvydPu&-nvNRsJ*15wgX6SY8n(iKo)-~-{7>P7ES=-cI(zYF zqyhm)X&xT#i+Bw<9EMBRc#F`s&%ubE;O=PJpip>{<`538M)Kou+M2}=V-5bKky}V} z*vpn|8&#B`)VNj98G%IG@-Ll3ssNkwIer;q#=f{PN-C3lpQ7AYJ}6+D4_w#HQ^%lhtvS(-qQBVby=d@0AiE(j_VGiX*?p5; z8?MqsJkkiEj!+al`;Ak{A4EK)XVvvI&aYTxl36}DTwRZQRm5$p{Nl&nW$-P2pVnk< z>1Y;{7t6a|KTfaNva>s#)w6W6Pu#NIi44Z2OLP!5W%MnEKZ4$}9UuJ82O2I?i+bGq8RE!}hE*!&KV!ln713c^+17dJDc!$Qk0$Zt>Qs^2tS zRt$i|F~_$Jh3TJfqOe_^a;=2H4z2XiHCHByIBok@M0>>#Y4S({V|qnYa<3FlBvs8K zwxoydE|*O$iio)Td|Yw`CiuICFH?rBoq%bePx1u7D7#QDrkD)x z4-R||dt^LY6YP^m=F@1Ir!JMU6eY6_VU;J$v>N@ZZtl+`4bCciTZ(2v0EtN0((yUh ztL*(DTg-!Slcph$8I6z62Z^%!Vlh!&ilqyFXniU; z?#bj+^DiZo8RtXeAA)&$#qi&0*&q* zl0AVn3)>VTCA-cXhk#xrZvF@^;;X2QR(mHbh+^u^xb-S@J1;kPFNwH@-TOw%1`U9Z zotaSnVe_t{L(_`Vi)RC{wN>sH(dR5~+cTA9bCjYqn(3mu)osrtUK62%7TS-Kixe_Z zeEfBoDHrbj-?VK-vinJMkOtJgu`=BsORJ)Q1_V0 zMgC+5BU$%z-BAib##O-ZLHY`+Ollqgto7=h$}VViydw zJlD;AaJP7Z%67K!X$jvHcX1#Z4$V8~q4|mT)}>b3g(}bGiKobS7cXyj`RLvsdd_My zR~tDq>D?i^mGtg$sulG%=Ody0S$al5B|uF{f9Oy_`{<{Pu?_B%#x)bXe*DF_vsm4II${?)I6U^SYeIvI-s{e^-y6%9`ljU7B zwlI3Ad%P^8fVj;Jjl}n^JLGJ{aKeM%WO=t_;b@ zkb9T{UrPmwafD4buPu7p&NXVsiXWcSDd%>Gs(f6>mwVMZGEQSzLUa9)4gv*3LeH@& zUd**u&ZpdWz0jIG>=xnerqctoO_Fpkj|Z8DjZjPnyj$*moi+8}0T7g-`fR%G=$(AEU#T z87ZHM*ih(2;zRMM4aOdn(;OTqmq=WfCmF^JyvKWVA)>W_Jh3A{DW}7eKK?x)o?Kx& zD}>)2$1UMSuLS$#v>_#<(&IO|Y2zw*RLn*MuO_~`QxizUVL0CzSFsA{6mnQZ1E4pw zX+4^7^TrDzm&&cTX!wSR1F1*G)2+jGp42SgGVBNm+aOd*x6I5)ct4@5k=EWkH0^|g zLo4wxrkwb`L62UR{E_y%$o^#h^$2^fM1$6Eay5?o4+OsS`V*W{8VqDWLERdu?sh6> z&}rb~jUAhA{qkE5Ui5wTTXwoEffnU4s22Fak#{a~(%m{H9jb$1e9?ncN(Egs1~5w&bEa!ve~GbT@(4qMtkEPCizG`+b;i5KD;F47~qg* zs(W7y7P?GWtvNYfW?P8C!;%&OuW7=cq!3hJN-rQtwxjNKmvHw<0UyYXJ%*p-RNEmvNx6c=9B;f-dZ*o-etFt7&|&Y?mst|(7SOy?!bCQP&TIY5^|A62yP>|HaKma5NOwKYMq>7BONxUMCu9eC}lhz}0)aB^3Ua&7A+W<7Kt4O_NmN5{88 zZQG>EwU4C#EWXjGWXZTWUEP^dKf7UccPpA8lfY4Wu`_!l7@LDe#jHPcU7_woiwC|Z zNVvE@MdF{MbX~?@G%B3oBt<5n_z^gl|}n|D&|RE z6eMuxM6&NN^V%4-FP-dFquj#)^dK?a|Hd*p#q&E73K@{O$sc_WV}*6vf*Gh9vaz1*-hat0otdP2?@e= z6*KY~)>p*lkGXIpBP)yWiI!xnFGg&1kM>JNUv8pkEv{ zY)Za*T+L}-gvhG+8;0Ds2|*R{c8g3^2+g=s(}c*ST=$<=|9Ey}%h7YfD0Ng6h8&`r z5z;0Dh?j1O3#f_JJ}p+Rm=qNeo`Q>D!p}vv1f3G&AQgJXm|X0@+XbX#e~T8SqO8@9kIg?u3I|z{THYK$>jX*6!wN)5|axkv1eQxemJmL z(eWMA6uo!*dlOA0eP&o4(^rkHeP)q3+%M#RkS)Hx-MrGV59+f8mI^a<>lNsa66Gg9 zdwqGpbO$k8F$3DTG$g70*|en+Qzlx&>iQk0!ZgCqhV7kETH5}`FD{Qo<4!qCypLEVs>xt+Ot^F#xX1uzV?;DFO zw95rZb>JCnF5HkwrkQm`H~|aoz>CiG`oNN>){JLSy?!*?6adizjLI{ z7oPN{T!6+!6F(?WsX3lbxz557pG}i6t)bCwahS;mu016gj)MV(`QRk(?0Q}75odI~ z4wu29Z9N-d5c0A^-4SgNUod{&9KwFO+hC7l3%@jT?~AhaEK8Rtcs;o^)FFizxdRuQ zL5AY=3^jCU%vm#=w6AucM)PTroFAPA7QIK|YO2|dp;FXI8<1y#W3;{|X1+snf!(2h zjzu}Qc(QmFoWxi=GqC`GLk_uuTKRj9m*e&y&rWY`A7fDnw#llvHATIODFapn68iVt z9Zx6(y#ZwkkWUiu>&s_VINiBxn}}7|GooIl9wKL*Uz#u5i(VHiE;84d=Q-qv1+Dhi`7@bZ zC2&_{JFOW@p5uSlD-)4a+}>3A!O66BP-%XKZ)2()L{+WM*0V+gPBt<#9%Xt_c5(yd z5|^U%;a`-g6M_bvi}8T%%%RMz@ImLOA-Ta1s_l(sQc8&s4_T3xi(oet+kBBrzE4yy zM=m8VjF=OCG|oJG?vw~jYI(2&&%jw<{+}x(1bV-nMYGm@Oj-yPh7(LDaJh?lb3)EP z2ZqPK*x}kB1iL^+9*S2!Zf|eD7{gt-jHSCDS|cGhe>Qh-AYF3(A}{mP`QC-IpmSED z>0ueD3vx+eVvm*+-3V|{gVz1sALTbS&RdK zGHQ9@aKPrWlXC0qbe1%HL&VTIk=r^1_ReGDTmKh$nvUYnmFI7USJC?p#>aLGdk<#M zQ3N7TMm@`BeP8PLfMP9hK+Oe?0_6az^52WrA8)-p7K1Um61+|KPjTLUMuko8|I?wM z+kx8>xVJ>0gAAVpJi{mSkKp-lCQl^KOPYVD+U733(Q_ck*aWKH9w6(9zl#qj#`FyX z!@%r1u%vKN9u|~Do)#=Csn{AM4EsJSep}q>Rtjllh;R|}{{|f8tb+KfZztE=Eq7^d z3ZEGH3|VxUbWzwuR&Sl%5@-qVkLNZ`SuM`CZfDMsOy1FUFWb2Y_U-ZMI%CiN{zPh&){Zm7oHA|iemFM<9iFsMinN2 z?cOH0%p2Jps0D!7=r3XEN`?cLw!GUv+omtSn{m~Xj534is(!o)MA}ff^*=L=i@TK2 z+tp>w0G7}rv)LxhnicT1@BNm9NeGT7rOf7bh1G4XYUiHL_?{U_A~ zANcGZ-@b05UFFbWI40+(7a)8bhK^|zDUr%IzO?dkg+;d`OGdfM;R!BnkguZm!RL<= z@siQp?w2m3*J(uT=2$&y6#+qpFFz}EqZw%A2N;fuX#oxw9R1~&YLYM%-lxMH4UQV= z&fNf3*VU&!jb7mHfq(iRi|xDujMI2BFd&%JX=x>$gmY^*Jl!pXD&a<~CnruJ{R4Uh zIUjREa{%$fe2trx$13FfE}})G%gg!-h0mN85_js!Cn``Kv6BSn8M^D&T)>%Pb*d=Y za8z6nekV`$s#qM=4#c1-dhQw2(b0!d(LlfY9ni?t?VH{gfsO&yoc_+(!rPO_rPs@e z$F*(oOK+jm^y|1#v932|?Q2 zs z@A6M%Rc59OnW9$R!{cG-hfq3<87LQ^2*Y`S88;~g+YgW^#O=Dxm> zjH_JT_K3-!eYBlV{fCvd-ZQ_jKO3JyV@rQ%+hE7bxo73usw0WOEA0ktDP}z%jEmKqUD4 z^tL4g^EzE(9+mtluN2L>a~Rf3)3VLCF-uih$eaF_hFbt;Koa6@qNe6PSP0S zmr~U}`v$u+XV3NJ!pTLV9`!1p0kQq>-G=doqjzZE^Zz0ZBn<)5z|Vw7`hYi}#Q{Dy zJN42;;?F=-*iQ{LjxTT+l*L#dLA$ChL1o1bd~GbaD0};nNrz=Jk8qjcSZe*0miEG2 z-s(_I-p&)L1>QZr2=3#D{QhJgNwmiVw)%8eAwN{AT5U&p_pUYzotHUhH8f^cpX)G) z%V{o=XdV{6n~+bJjvgy9^2B=rXDgmrbP|WVfaByWLF4FpTqe?pf|B7jFB24`rf0UpI13{9O^ zN^$jce90KlR| zB^HVhoJT>tJO7N}zZ3-kGX|V5wcCI(LjiQ7Xl&0i@aO^a{qSifn0v;DfMJ)W@3%;W zumM2l4hP0@yt}b*-Kd<;|GU)x*jCnvO}^MMv`5?1ld~PO*^tWWYTdB{?XepBVh6!w zFlDr!!NVIM7XK06ek1em+khv>-TAe_FQEb|Xp%5S3&=J7ge%i)o1pNQv9&tW%L9*>WY^*Y>xvUvyEPzU1D;S#WR|Ig6yr~*z_ zs89y;xek8A#Wepw9^gtAI9CCJllTEQTT%_Ku#HSFP`M4h&0kM{jfC8$p~mnLg`(YF z#Q@6LyGXA9wE(KiP85Lj{T>{6g@BTr|M{_}xThK%{;wd+_+Tw@gWp~b`hU|>khvF+ z3VwYs1kMwK$3YdKoBjJx%?4*G1D;p|xw{*jR$v|us0#A1?>3m=TePcNKRxu0v=kAB zKMem)A2AH+yIaF3{DHqex(jZ1-VaWqgiCw>vjyuYKpVo@6sUK@8cu%QA@+ZFhr5^v zFMd4<7I>0R;B`PGk+ee(9ho;624y&3w-B4~ra`Jrvs6g4LhWnjve}=4e(ErDj_{Agc&LSxI_s9OM`oH{GMe3Uvlz}TC zmmLFg*~*c`?xB!;1ZLEK8Ct(I44(D-oevwBkP!+8i+A+9LeHL` zIWZHNJ1gIu=oW4W{k{bFU{^4V51(MOQ&Q7QirVyplq{iP*#rcrWwTeQpwR*@SJwv{ zqcqObA+r~*x%g+Nni6W{jjH4?Mqk2mY!q}Q+;O!4tzhAQpcU9u@^P*uBmv6@q)@km zOp*V)vurnj6E3IZ+EDg7JU1wmNZ7t!^XHBMV)sO8f6yWTPdXU>`<@!^T*}0kM=OzX z&k~KTtqq*i{ucrEe|uifitZ5;`$SpbLgaJvP3vfcDiV^QHa`YN!~aN{%~E35|3t!t zRF-<%m26j>P{`aK7`y6R;e&$mPWwJFr6-4QmHx zwKhOqW4DLc0izaKX!e&$piP74TPk&UE^AI(dNt+-5I0xEp!%VXT7qry%9SU1!WXHD?|BM>5shl%edM zV*SmFpoQVyQGc&;q%+jtoX+;gY8|v)vIDT<;39}@qdvG^psL&AFT6AUYs&Q z4CTMXE7z?s*VGE=v|R3u)qLRRce-@b(KGt2P<`#Uud4)vO$Z^n4s+uXzH}OS_kkUn z_xmJY3XSjSDgI|GM~D5+hacY{1D{b)MmM9st``!tHHp#mzDj5KyMKlC7A5eGLtd%W z#bFV`wR*sPdwd_i#bz`^x!CH#%=O6~P!c>EkpSdSt&9{OEE_%b>{_5-<+ZsBOP3_n zx<0PR^%zWKlZ>G!!P zkkfZo!Pd|qm;aYA7LqK7cH?4^nF3*7BzfR+Rl;7< z#-U~3FveHsx4v9nU-257$Ato5NqcT+*pGOFwj?_aC7X||QLi6DFISpy(xB!0mQEp+ zxpUaS2l%t<7Kq4{ta74tDC#vVHEzM>wL=CI$ioM15+$o_3K>Eum#@BXd~OSRsHJAh zMHsX(i6M5K<1U<$t$kD)N&;7s{8Dy<9>hbE}7i=stKDvWg zURx6k5>5oogOWGlYRlopDlQz-Z_%rN+q9|G417HM%7 z^RK5q%|D>!V=b{{Vw|=onNElnJ$9m&FMmaDKR6H6&~tqH_~Jd&0U@OaI;oSdE_X%n z;RdhIZdy$m4lr;}=gFDMe0mfYi!7t);T4HYNqPHeKBRh=P*9U-cI_o?oiXz&CA z>sV+%kGTOM@Q)6J0w~@9K=BX89^9Bzhxmn4qMUvmmdFof&x-{azcs37+94Y%#5Z6H zn?Gk8T;?{Jt?=SvNuhF%PP9Ak!%K7I5X$q=lh3x7i;}2_+<*VA%WK8eWL5%-)^QZL z@)hj&zU`qI8c8Wic8u@x0TYLLpR~MgW@vJ9vSbSX+>MUw{C8xCTrz2QJcnmQL~pL3 zCOJ6j-IW+GxsvyW5Om%6Zr%3wL=}el*yaYimd^?|!RUV@>53X5mxtlgL_?_z&kj@E zF?*hXVjZ$)mI~V6eydDH2O<&jU-WPPP_UI=9VNBo((qczeY~p7i=BAF-yHAi4N({U zB2?U0s;v!%Eauq@CGob2%6a>ZEP)T79ZDLu;)A%%#}QPt&u1ur8P0wj5&vZOta8iB z|AeEj*d`0Z-iKJZElxZsx&m_8LM;nTX;piSQ1YpkzqGT}zqGSohja9~l@YYLn$^i@ zRiJ3Rmo|8DASXNg=iaz*oci0Fe2rK>d-fu|uAh&Et=ESd{qbn~GG$1t)`#xKvFpLk zlDJIC0`Tc6Q$D^eH%Yn#rGjq;2!HEUU&wsluf1~Sciq4BTM25SEq_rTmT#wbsZv{s zwD6Oi(07_E)z16ejCB0&!E_#lPIEgbEQ9d&DvB5{`k~-Jq)SGNKs~_T*a|f*f6UBL zL9wj&wc;*BatL5nitVB=5#pHASLm@|o3U~emeebH2=O@pfMfgg_@uRQbm&JL< z7vAeU`MSyQr@XX^z4XtM>!giU<;R5A)8>Q%;=utfxm{Fp;b_V{rcbYlt1;q=2@Z?7 zFGDr?B9zL_gh5G_zAvz+B~0&4DT5YG3O9^#^n5SXKn*i(|2#L>@M+ zhFjpTclQ?C+JZ^AmQJ>}x9L`-%{%gG zg-BmC@E%^VgmrX+c*z#mZJ=O9V^3~hx!%fV)6+t?dRT~Bzf9+$_XK6}I>~7Dxpu}6 zV6ISOK7H?E#SR#PFoI=gsTdKFsD2D_3il*!)bY?*4gcy(Rp*D1x#Ota>nl}h3;uxq z=j)W9)~CS&EAA*u()Tc|DEYkCMnlHOq0{;KJ6A5hA?v~G`XcjRP5h5jAZX%9E>n#} z9?Ls}nKB{~G&1kVR(exHkGCefCU?E~hV#@(pb<3QW3p+F0q(?M62gj#h+h3e zl>azB?(UE4iUyOtlzQVc zd(at?{Jbvoc(~}3;+5>z>8x4X3*7R_f8fsVQa~RB0VTM$$Dji6SH;7-;D~Z=3Be;A z-;eT1=pqe@EhnKLxXOfKRe#ukW6y8yVyAV0tgzcZvce!_$|P{M>@RmIWJnP(YSbwI zEEMv*qg?zRGwW5hPx5m|VZTW31&fhi#G& zkI=EF6bk!R2qtQ~9Q8o|+%dgQ>6g{>Mb2)24vXK(&eyABpz~C7MMx=iruJ%Qvf>`) zBN_1&b#wuH$Hn<8RX^IFIA0 zo8u+;9WV987khxaK$NcAJcvO4d@T$rj&wx=C*87kXO~^)Wk~*RsC1fNoIkOJR^+#B zV+P*ivB0-_uzui@_8(BOdlI5==u21yma!Z-Bs#WI{v&^6Yy-t5dE$qrnEg8Ua=CA3 z%l2e@+7tYnLgWLB0^OTD(oDG&=E;|^x~OzgSHH(qZb~9?@=LZfzn6J^a$fuBC=fCK+v&_*^Fq-K z@8J5%QLWkr6Ic}qPm~xg^M*C=ogW!(junbrI~MA+;PBcQSE476-LEm5|6VlFyoioq zUakHMFdiQp4Xc|83Yxv3739C>GF?2Ee@jj)o0JCG`)&-kTuPvdM)*hm$}v2>0rzx! znm1iCT(}c9hQq3vHXx zi{Z9C#vwpYh*IbPq^r-KJW1Bxfk7-BE>Fz?%Mp*)cDs4H+IBh5R;R6cd%Bv+VyXST zw4>h&gaP^w5w2~2_@5$Nv^}vP+B*p#+dlSHcsML7U4Fi}yCV`ZX@+T9s|dT)-&|}9 z#ZTK|)S-aJ3Di4r)#O@-#320;{tY?DQODg$1}<)L7XNtKfBVjK#O><5IFbwYv8IIkQ2c4kP2U zb5kuhIa_4md@C8tn!3}QEZ`af+NV8ft=X2(SOVDXPFNA2(y|h`p&iBvD%0%^KaQb; zmR0{KT8`u~$gu&0tRrkN=WQ%slbmC)I|2q*d}5Omwk zMiC#3uW$0n;-4DoOjzF=1CLHE6Sv((Hd^P(SXA9c?1%U*5Sm-inaXX$_(mdqK)& zG8HyVzh?q5U-}V;ClN|^banFF@mTBT`if4y>X2(Xl#~be;WI1NtqD{E;JbrME7vx? zRZ*Rkg2`}MgX6DOVpeGjs?NS4oMDY;1Oe6V&kYoJjU)_*M4{IHWcwkf$&??sen(>B zbI1F_UW}mJDF5ekDh9EImV%dGWW|QFUMYN)r?^}*LVW!f7vOX;bgaty@n^9>AK;{c*CaC?3KfOPOroy4*=n z?T`-(OGknZ#vY%z<59Wq3eL?>8v)N7Us3M^Hn569KzRgs^6w$c>UzPt#_8w|eFUxe zXD@T-Hrx3D%o@A12OSYKq9q0b3j+i<)iGyDlIfE7?k)S0@79;^ASR`(YLar9&K`{Z z+^EpBHM`2BCF6Gpzz3M0Ic17K{6}j(`*XxMdDG&FDLNFo=aFS-X;Y?PK;?r4LjVg4 zYd^ZYC6MriVMhdQsFMbiH249>cQE={Z8nwZ`9KVhu&@M+jtV_4B6Y~o#%LDTH&>m3{d{4JUJECLIN{D^RJ)y8J7^v3GL%TZe=J%;k8 zkt7th4`veSGD#R54~|c_$DX9W*i#S3y^;tby7#a;^ei*wJ91H)WwpcC^EZ2@bw-2s zNbh^pz4xf9+BwVk5?@!PZqL^4y{V*BUg4XGPK`XrVu3-L^$r2&3YRMRc@gP-=zg`i z2-yEEABd%i2g3-yf+P`?4Hxz@PmL!>z54a*G%tw%N@%ly*iE$Rmgwx~T)R7KUs7CP zsYUNPJSxoV+v`rpOAH5>Eai_aP@(@)s)MvT-hU|7iNs)1guQ`0fg|mOn)T*{31!L_#5Pb9ItZ(=Ds?xmg^G*Jjf#jM3b9ya-vn+9ryG7vsTGNfQa?9>Vf9|1_y6 zhW%qcG5{06zqDxJ8NqQ-PzC@BEp)JW$0~ zj-8i!_8woa4Ip{%>o+BO8x>6HJ!aRF^zP8Dy0~<&9e)_1&rM;Sm2Q?eM6z&P>v~7J zk@+NT|MLx!Pg&2JqrLgX7FfBJ4%(?TK zde>e*WbktS>+5ewUZUOVCHG6SqFM(ko5CO>D&w=~1bBP;9haxnV&?Q>`q)e{g`xyd zrIfPD9=DZRiT1>7pUs^gjmzB5$7m@~#kZ*=rkAICx|P$m3(Y_%>g4Q)N!Dlw|`-!}qh zt-$rhBA{RbJg6tZ4r`~ka-6G^!an<_RYC9TWPhY!a)0PT{mmei6sMQhKTWiM>RR@m z2bIYvuggBA1%4jOg|S*b|exzt3YWIj|+$MW0V@r%~xNM~@d`@Tr&RbrwW ziJJba*RJDV4~igy15dC2+j*y5KjgBG_M74sv)Ocs#Or|0XlA!4qp~jjFdu9KUk!EV zGA}YV6MhHm{qRpZ-j+HmcO=esd_$8&l6Clbk2t=nlgTCX$4thd$5lCNQ*l;qeSy&D z%5&2EtBY|_e_@$Rzjqq_pjgo364<16AX71KN!Au`UC}FEtu7nvAhH1kewKGUjH+(v zg*d$J$X3c_w^BIszU)*-idKIsj!qLP@5f;Yoi4VwDNuG|f?|{Liv?ndWYDRE+?UDf zjF}e?z5nW3QYmIYKbzskdFJriVB!c44M@?sn4M?*s$J)LBNU(%Nhk2gx-;Dm zpI%4)i`Ux8f?g5dzvdw5P2dm^qN|D<5DY&UU~ogGU&4figI56R{P&leHR$o2rv9v2 z@LxbgQusgqhzo#0T;}IM-tWNut+6dc|ftcBo?0^E>)cP99u8qi4CwX)!7 zHk$9h9}6Dzxj1q_}_H-ZU5-#H=7!c$>ys zgz~RG1e3LZ1T@p_%y_tAfSKgK!s@TZ#RkB*?;4cQf32Rsmz&?;acq$ImsiPc8aX61 ztnJt-F>F4beRJ3ji5Gc7`p0MfY5^i(A<$vRGw=83ZGxY_1E*_xjQJ-||M>>DJkk0dVZV}v#!Z!Z2BeL#s&3|#l)o;M|u z-yMTj;bh7p++`wCVRfoo>xn@V=t{upSbTzzhq969JDz=9414!wR7*Ai={w-4Dcbcu zYLf#bxkuoAfK~A5A^0R#2_bT-Ee-Uw<03L<7gKr1OMJ853Gmi~bK zCwG88zaSC+ZfpveDVIDyF`y01p!n*C-8UMQw<|IpJ+g5)zA`>ztEWxrMW)Kb-nemL zw0REB^<9ryX=M=S!Q1FBSHhlvoYYtMlEPddv=HV4nV!Ev8pYIbl36M&UWHs;#MtzqM>JtBJen*OkSiJt~H! zb7afcXkP?7qrURYK@P^Shg|PngOm2Pu4&#N3!T_*l-J4(Jg}jI(+uFbvP5L>l@YU+o zMm-&E7~!k#6!u#)8X5+`{koH-Xrj0UIj^Uf_5F3U{L1XcB8pNc9(G7J7KO@sP^XiL z`t#6lP1SRodspqds#qnP&k*Bo+V#0MOb7FH!Ety0ZR;Qy(%->~L2q2~gaJ&@V0g{n zV#$zRIbCHBXBVTdrp5$3dAV4qB74qbCHaF`PXH0yC3e-bFBU57iRbS+0Cr6mVqq1qSpWmamcn2yd zqiASo02Cb6M2$c2Thl}N`d6m!df$*vs>kSb4;b2A%#z?g)>K7P^FqDXmkTwF%e|rE>01p7t$Am!=zyqMD@hk$R@O{VA!w0pJE*fmE zh&Cn`(ZiqBygTPieN%xZ_w$nZaRmx4L^i}kL?5MnG?XK5{Q?ftbJh(N!p8*(Kt0D% z8LO$m2vr4hOp2iMewk+<{?*x_uYn>b6~oX6e;`BDJ^Bni-RXXeky3iEnv1E;3n7=) z`iz1xfdf&F++HVk@eUAgGi7AFAIu}#lt-s#7!s>|(2c_@J9SH<)+p^sbT`@Ew&Aw< zviB63*tUXc$G7Ek!*AcQ&T+44VVEo5MD9zq;lF%Uj)@kN=L1|T=58jc4+pKYHRSAn&}X&$Jo;Qcjhi`p5?!7N|H1!1CI)TgGJ#*!GtpE;ph5e}rZO8d-1A3W-JT_rWk0vr9OoXo|o!DBDz-Y8&3 zA8=_4@^Yw{NQM#iNO-60*ne3%{7lZBLq*iFNLd{cEWbVymqF(J4 z+m6c9Z5lc5#Ix4ZIEACv`Az3CC}aO*?iM!jd17to=bMu(*0@r{DghF`b>cD>dWbz) zh5G_^z0gz?oiy2y?46`1Z6XD2rkIJfP3D*KIEf!ybzbgr;4>ELMF;i8EM zX3<7VqY6bpODE8hTzD>ofVKohxCeV?=o&+N5ncAu2wcYKMW>+}RG)Dlqp7@oW_UiG2AoJHRmM!_W55ml7vG8)_1$U9HR@+AwT|euuNc#r zq|rYseqX&ETV_^#sWNoWb@4-%jbDG-k)y75Umk9>op;y0%pX|yU>;6y8XKMS`DB9b z_>Tt6K32*K&~z};I@_V0FL${YK)^zO`>nX|LTS%T+dIj(9Hy|%Y;IYgxI22GuhSC! zvch$Tvqo*p+#n{DbPHF4B~6n}D^jnn0_Dm2yY~KKQfTL>?JZt8_o>GXVUdpc4^?Fm z(B4r#5>`ZEx(92%Ijf)jDwLdZ;&Ni15cs6+xaqyvr_GeH@U8coVnc>Qh~BT13i>Qi zDXFj)cj}Y1y=M)-zE<$vS-0PRQSV=h&gf$!I6EH@7FpPvsaI?!FwfaTirIQzJIa>% zhU%%wH&4^f$TkB@iPK!$=A+mHR?D$R8K!Erw+<+_=ImFo5cy&jM~)1nOU$E2n+986 zhu9GZ#bJRzYFBDKd0V;>Lfop}U*U~)9-PR~1q#&PBNLExb9jzPRF?16qA~w9GnpNd zVQQ*eAW-vWTz5SB<-p>0u2C$vAYKC2lz6{R3#-wloWWo!ZCQ@Gn_-BC!H^nlOHyPN zUm;CLLG6S4j?~@|!izmK?11oTKWJT4tEW}aC*S;1x zB&R+PFW;&Eac;|5GEcmAf~=fo2yDPo;CjBoS*~k9XWlZmgdW^2*=!oGfhLASyF` z>#=)#Qme8LrR7Dm$Hip_xCNrt-nagZNeoRLi*#98qc4VUN=8rGU0fN9*?h*Qf(-_W zJaaa-bCs2uW+*SpO(&n_SQiPdSr8OjsN$pDXN=Ugr!-?&cfR^b&FE_#*`xEct49=@ zMf3W$ehMmM><2%oSHo;h1AJM!? z!9ab?LP8<~1}gOQi);xU4Z!YHq2Y z2Q{ZYlB?+E@-~{&i)2v8+L@_sLaS!uu9_upk&1-Pw>r(d2a^DIl8Z^)Ni8Txm2@W0 z#_r^p38QU%?rgBFC`o`gNFwyQUaRBByx%hdhTiPy0%V$Q4c{Vy$GWT7QS+FE!g`uxwup>|(s3nwYDY?B0zm#q3Nkka)yuqq2sZg#+EW zUD-vW$J6y3r-EwTT4wHXyDmJ6{_!AuNwW+w-pIOfCO4bSc8kJEwH%sU*~?!_B_YQS zan2&{1uIFUh#LizC-1f@h=)$vARbPX4YixO*MEN2vXoVF^Rsm&<++PnTPUu4<=c;6 z@#z_=cMGo^qfaH1nQUq<`j!eT0Gg8L@zy7T;N{4{>9zXh!O0V)thY~^Fz>M+v{c*L zneE)ysC7UnUALAm5OlY?!bwwPxjuV5y{NMoFo6lP;lQV0oZrS&%2C2ZM$d+re%j^R zqi9{@Xb=w7mCx27aTRd%;4RmA`LW=Wf#!rupbHJZv|xv6=jQ^caHXSq`y*WJomuY> z%MUEN_P>OZe-d3QqEjl6KewsNpVQ5kGpQ8_O$u0&1nai3t~r>9$Ed6jz0fB{sbC(e z2zsTDFg-1*9GOy|uALk;V^_*Cl5%}&oEiKqm?J9FGqt)l{PeWfYKz_t@zn4W)MpF- ziWh&${kwNy!w~=44#{$+%j0h3({`Uk?(VR|a3#KAM0|O61Rn@>^&?{}WV&5FBvHu^ zXGyU?Bl~ZL5B8Qo{@);CeNel4mvN8ct?iIVA@1z%^=ao$=u7=HZrGt7|9~5o#{1k(;N9fKyxNkO4`lgN^4()_c{vc7 z&P7?1L9k!K0wdsQ{3Qtte-@1()xVzhBZX{eI!^b+{Kgb^xB>>WauF{gH~kj!0N9M| zC4ahwyq^xXBEb(he?&OzexVCfMPk;dY6b#ci^4HkH2FtSw?uGFQNa&#XntWspWIQQ zQ=cdVO!^u&@<>mdt)7Y~CSSVgfj43!5yAEw#oiOrhmj+TN@l6%H(rFL z_#yIsh2Zxsp%dke@5d96YjerO+tbEDqmBV34qae)kaRw;Yr zr=3t)5TRLz0>8RNo00niZgd z-pfbz3v2d7#BaBEHy6+1nMEygL;UBK6pS}CdH65EkSJ@=_f6;M8hym~lr*W@)@L47 zWb^KrCli^fe6cLk53>ScHDl7vFOwo-V#r-w%x#*zH{Ta{0{7Y)S@|4xQIAMQefPZj zSI08a@Ug~85PLb2SEob>Pw=*>`dDxRrxIB|B?#?H4%G6V%&|5hfXuZB_7&MFv>rv4 zl*c_P@U>9Tv*E0l=nl6J`%L?8*cmMy+d1(cYcQT=X;eEr#CmNhpKbDS=%YN!q*K}A zaMznv-LIkUO(>tW8aZ z&A@<-3z3a5&HUh8=0_qgp7$%Xrq-!D93Msm14o$V3ny;>9C_b2G~%_>I$Jnw^mxIv zCC$(hGbow$0n=nzXsllWKCSPotXHv!&;<{N$Dt;^)XFm>ya23?;k< zO=iu4Ywe)fL8JcFzU^vr#UcYc2`;|D-b`;x3=u*fCa^YH(UhGTN0cU2TfcS@Dvvin zP9nD|7|^8-a(F^R0JTVffaZQX_k*O#?l+B5fG%z@2b&rjYgw|DHH- z_>0s_x#TqUKEmI?j^3&sB@_$z+P>yRA39vOjq~v>pFXr|)c%g$C;N?`u{y>yB8J`1 zIKodM_po;aB3FFGy*u4TE5-EBs>LmX-&yVz_nMg2k^l9mznm;(>5((SviV?ZAD4uC&VcmZW3Ow}KNmTODRiQ%-K(Lh*!(3Mt)K z)2U=ZT8XYb3Ou!m_#ALDC#1#PG~FMBbIA%zmM(s4qsfQm3noG}t?$m9^& zWDWX2wAovSh~93M_7$qo+J~fNQ&n9arfqtns!bnNY5O!`l#H%EN+C@BX4v&UR#z|7 z(H56=#+Y=$v{iRtwPT=x#SR@U`>WQ-lFGgl?VC0wgzz1#zoy}@W_(l&<|0HvY2|(u z&dWJYlUcXaTHn-{qp}tgiA{cq^-h|_k%ot*ch1Suda9aVJUEO3S8vu4co{?P7fT0vn!I2DwtEz~WpG|tKC>^ni+ zG5wMn>a$vGUfxh7q1v)%*Vh}S1|ud+nkk%+hr#?F1A0;-(l-|>EBf{!yNqfTI18pT z;n9?*r2MX^)1?mWyVO{Fv9UIE;yqczUwx|gY*Q!q+m=iMaI#GMNN)R*EF=tMoRa-9 ziNKFVIHz(H9onsGl_8S(t}L6U&u=l4F%t>c@i}0x25;3(MvUSzl~^~QM{V>Mo60eo z#jzydJM!GGo!IC|X(7CsOS;hebT5rs)Qfg zdZ}fRm~wl#m0sqNycgsiCsq=Ac$zf5L`R4kn&-3n%lL{ zAV)0RIw_!aC&VRuoJ~h^&!no0brzm<~ z-ivLoVgt8aX^^DFA>Enw__O@Hu+z(bh2wusr#RHhFJTbzV3&sw6~sSLmvy}8-sIwO-U>xerBh5ttr@XzbU9C4qW>7c0j2khq=fj7=1x{&-L9hX zMq%@olh($2xhJX>`%VvS(ZAJ#%Dd?zLL63gyh-`xtN&b7(GTEKVy715SNdTRO=F{0 zTcBGb`GGYTyM0wsZW`t}<~A+)O)NjXcon`;f`_Mv_OlO)nH10*wgHpfT0V)KBV408 zQG-A}67PYGdZ7%452#bPZbby0jRsy6vMJn54_lPl?ndMFeKL1WxXLW^lS1{Y|gIU2|4bt6V0&U6s^Ygt%AF>m6^vo5VtUZ;9L58;B(NSveeste5p7T~o(cn-Zt0ZfbE?88 z$=ssk%|oIs;r)^@I}IecdxF{7ZzL|rC$#0}G%9&KQ9T8YH(`7a<&=j_m9RvcxC2c`w)ybg+dm7^36-(dY-dVf9doRK$#7~+5}(p-po$g$}; z9;mOoQl?%&WJHn8XMV&j70F29%5S?)N)sHP$Ux`U|L9=$hkS{F!l}uj_twx*8vmQm z`JZehoQdxrlTI>?9$I}Il(~brH(^-(rEy&Q?nsVu!ju@>K4s?Nt6YsEPrdi}Fww6D zQ#plGifW{kQ{Ad;a(Q@CSWwxrj>$k*=@g9`JA{?W7t1573#(7$$**T7-ziU3&C1C- z<6XVrnU+nY^#~~)H)W@z$71u~a6W!ox#vODg4;-|w|btUpy?M(J5%FwyKP+7V$xh8 zbfG|I(54RTT>rh{0Gn8B^vf@6HYow;fS5E@>5L*rCgASCWKE1!*G%!h&CA<>i#`2+!ODEW5U|whI6pR{rlGl8Mt7#o7L6W`tMi%`YR6;*jo?2aXWhX z$H;O+l%j3`_B%12!QlAxM-N!M1>4{+`L{N}miUifM@oUy Date: Fri, 6 Sep 2024 13:05:16 +0800 Subject: [PATCH 785/848] bugfix: `setkeepalive` failure on TLSv1.3 When TLSv1.3 is used, the server may send a NewSessionTicket message after the handshake. While this message is ssl-layer data, `tcpsock:sslhandshake` does not consume it. In the implementation of `setkeepalive`, `recv` is used to confirm the connection is still open and there is no unread data in the buffer. But it treats the NewSessionTicket message as application layer data and then `setkeepalive` fails with this error `connection in dubious state`. In fact we don't need to peek here, because if the application data is read successfully then the connection is going to be closed anyway. Therefore, `c->recv` can be used instead which will consume the ssl-layer data implicitly. --- src/ngx_http_lua_socket_tcp.c | 19 +++------- t/058-tcp-socket.t | 66 +++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+), 15 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index a0e041b017..998881d1ca 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -5725,8 +5725,7 @@ ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev) ngx_http_lua_socket_pool_t *spool; int n; - int err; - char buf[1]; + unsigned char buf[1]; ngx_connection_t *c; c = ev->data; @@ -5747,20 +5746,10 @@ ngx_http_lua_socket_keepalive_close_handler(ngx_event_t *ev) ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ev->log, 0, "lua tcp socket keepalive close handler check stale events"); - n = recv(c->fd, buf, 1, MSG_PEEK); - err = ngx_socket_errno; -#if (NGX_HTTP_SSL) - /* ignore ssl protocol data like change cipher spec */ - if (n == 1 && c->ssl != NULL) { - n = c->recv(c, (unsigned char *) buf, 1); - if (n == NGX_AGAIN) { - n = -1; - err = NGX_EAGAIN; - } - } -#endif + /* consume the possible ssl-layer data implicitly */ + n = c->recv(c, buf, 1); - if (n == -1 && err == NGX_EAGAIN) { + if (n == NGX_AGAIN) { /* stale event */ if (ngx_handle_read_event(c->read, 0) != NGX_OK) { diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index ef2b05f0d0..db5cb60e84 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -1,6 +1,7 @@ # vim:set ft= ts=4 sw=4 et fdm=marker: use Test::Nginx::Socket::Lua; +use Test::Nginx::Socket::Lua::Stream; repeat_each(2); @@ -10,6 +11,7 @@ our $HtmlDir = html_dir; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); #log_level 'warn'; log_level 'debug'; @@ -4497,3 +4499,67 @@ reused times: 3, setkeepalive err: closed --- no_error_log [error] --- skip_eval: 3: $ENV{TEST_NGINX_EVENT_TYPE} && $ENV{TEST_NGINX_EVENT_TYPE} ne 'epoll' + + + +=== TEST 74: setkeepalive with TLSv1.3 +--- skip_openssl: 3: < 1.1.1 +--- stream_server_config + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_protocols TLSv1.3; + + content_by_lua_block { + local sock = assert(ngx.req.socket(true)) + local data + while true do + data = assert(sock:receive()) + assert(data == "hello") + end + } +--- config + location /test { + lua_ssl_protocols TLSv1.3; + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local ok, err = sock:sslhandshake(false, nil, false) + if not ok then + ngx.say("failed to sslhandshake: ", err) + return + end + + local ok, err = sock:send("hello\n") + if not ok then + ngx.say("failed to send: ", err) + return + end + + -- sleep a while to make sure the NewSessionTicket message has arrived + ngx.sleep(1) + + local ok, err = sock:setkeepalive() + if not ok then + ngx.say("failed to setkeepalive: ", err) + else + ngx.say("setkeepalive: ", ok) + end + } + } +--- request +GET /test +--- response_body +connected: 1 +setkeepalive: 1 +--- no_error_log +[error] From dfeaab4fdac09e1056873c920c42d70f99342777 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 15 Sep 2024 09:18:17 +0800 Subject: [PATCH 786/848] optimize: removed unreachable code in ngx_http_lua_send_http10_headers(). --- src/ngx_http_lua_util.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index f1e0cd08f5..5d871241db 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -764,10 +764,6 @@ ngx_http_lua_send_http10_headers(ngx_http_request_t *r, } r->headers_out.content_length_n = size; - - if (r->headers_out.content_length) { - r->headers_out.content_length->hash = 0; - } } send: From b190a9e3629b408d6c5da9e403041cc6200f54de Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 18 Sep 2024 18:01:06 +0800 Subject: [PATCH 787/848] tests: add support for openssl3.0. --- .travis.yml | 10 ++-- t/129-ssl-socket.t | 24 +++++----- t/140-ssl-c-api.t | 4 +- t/143-ssl-session-fetch.t | 99 ++------------------------------------- 4 files changed, 23 insertions(+), 114 deletions(-) diff --git a/.travis.yml b/.travis.yml index 177d6488ca..c37d3751c2 100644 --- a/.travis.yml +++ b/.travis.yml @@ -63,9 +63,10 @@ env: jobs: #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - - NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f - - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 - - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y TEST_NGINX_QUIC_IDLE_TIMEOUT=3 + #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f + #- NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 + #- NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y TEST_NGINX_QUIC_IDLE_TIMEOUT=3 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.12 USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 services: @@ -82,8 +83,7 @@ install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index ca8d5a49e6..611e72ab2f 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1155,7 +1155,7 @@ SSL reused session server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols TLSv1; + ssl_protocols TLSv1.2; location / { content_by_lua_block { @@ -1165,7 +1165,7 @@ SSL reused session } --- config server_tokens off; - lua_ssl_ciphers ECDHE-RSA-AES256-SHA; + lua_ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384; location /t { content_by_lua ' @@ -1229,7 +1229,7 @@ lua ssl free session: ([0-9A-F]+) $/ --- error_log eval ['lua ssl server name: "test.com"', -qr/SSL: TLSv\d(?:\.\d)?, cipher: "ECDHE-RSA-AES256-SHA (SSLv3|TLSv1)/] +qr/SSL: TLSv\d(?:\.\d)?, cipher: "ECDHE-RSA-AES256-GCM-SHA384 (SSLv3|TLSv1\.2)/] --- no_error_log SSL reused session [error] @@ -1245,7 +1245,7 @@ SSL reused session server_name test.com; ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols TLSv1; + ssl_protocols TLSv1.2; location / { content_by_lua_block { @@ -1255,7 +1255,7 @@ SSL reused session } --- config server_tokens off; - lua_ssl_protocols TLSv1; + lua_ssl_protocols TLSv1.2; location /t { content_by_lua ' @@ -1319,7 +1319,7 @@ lua ssl free session: ([0-9A-F]+) $/ --- error_log eval ['lua ssl server name: "test.com"', -qr/SSL: TLSv1, cipher: "ECDHE-RSA-AES256-SHA (SSLv3|TLSv1)/] +qr/SSL: TLSv1\.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1\.2/] --- no_error_log SSL reused session [error] @@ -2614,10 +2614,10 @@ SSL reused session --- request GET /t ---- response_body -connected: 1 -failed to do SSL handshake: 18: self signed certificate - +--- response_body eval +qr/connected: 1 +failed to do SSL handshake: 18: self[- ]signed certificate +/ms --- user_files eval ">>> test.key $::TestCertificateKey @@ -2626,8 +2626,8 @@ $::TestCertificate" --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out ---- error_log -lua ssl certificate verify error: (18: self signed certificate) +--- error_log eval +qr/lua ssl certificate verify error: \(18: self[- ]signed certificate\)/ --- no_error_log SSL reused session [alert] diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 4c81b4f05a..81d8375bb5 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -954,8 +954,8 @@ client certificate subject: emailAddress=agentzh@gmail.com,CN=test.com --- request GET /t ---- response_body -FAILED:self signed certificate +--- response_body eval +qr/FAILED:self[- ]signed certificate/ --- error_log client certificate subject: emailAddress=agentzh@gmail.com,CN=test.com diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 2f988ded9c..b8ca095aa0 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -1326,98 +1326,7 @@ close: 1 nil -=== TEST 16: ssl_session_fetch_by_lua* always runs when using SSLv3 (SSLv3 does not support session tickets) ---- http_config - ssl_session_fetch_by_lua_block { print("ssl_session_fetch_by_lua* is running!") } - server { - listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; - server_name test.com; - ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; - ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; - ssl_protocols SSLv3; - server_tokens off; - } ---- config - server_tokens off; - lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; - lua_ssl_protocols SSLv3; - - location /t { - content_by_lua_block { - do - local sock = ngx.socket.tcp() - - sock:settimeout(5000) - - local ok, err = sock:connect("unix:$TEST_NGINX_HTML_DIR/nginx.sock") - if not ok then - ngx.say("failed to connect: ", err) - return - end - - ngx.say("connected: ", ok) - - local sess, err = sock:sslhandshake(package.loaded.session, "test.com", true) - if not sess then - ngx.say("failed to do SSL handshake: ", err) - return - end - - ngx.say("ssl handshake: ", type(sess)) - - package.loaded.session = sess - - local ok, err = sock:close() - ngx.say("close: ", ok, " ", err) - end -- do - -- collectgarbage() - } - } ---- request -GET /t ---- response_body -connected: 1 -ssl handshake: cdata -close: 1 nil ---- grep_error_log eval: qr/ssl_session_fetch_by_lua\(nginx\.conf:\d+\):.*?,|\bssl session fetch: connection reusable: \d+|\breusable connection: \d+/ ---- grep_error_log_out eval -# Since nginx version 1.17.9, nginx call ngx_reusable_connection(c, 0) -# before call ssl callback function -$Test::Nginx::Util::NginxVersion >= 1.017009 ? -[ -qr/\A(?:reusable connection: [01]\n)+\z/s, -qr/^reusable connection: 0 -ssl session fetch: connection reusable: 0 -ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, -/m, -qr/^reusable connection: 0 -ssl session fetch: connection reusable: 0 -ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, -/m, -] -: -[ -qr/\A(?:reusable connection: [01]\n)+\z/s, -qr/^reusable connection: 1 -ssl session fetch: connection reusable: 1 -reusable connection: 0 -ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, -/m, -qr/^reusable connection: 1 -ssl session fetch: connection reusable: 1 -reusable connection: 0 -ssl_session_fetch_by_lua\(nginx\.conf:\d+\):1: ssl_session_fetch_by_lua\* is running!, -/m, -] ---- no_error_log -[error] -[alert] -[emerg] ---- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} - - - -=== TEST 17: ssl_session_fetch_by_lua* can yield when reading early data +=== TEST 16: ssl_session_fetch_by_lua* can yield when reading early data --- skip_openssl: 6: < 1.1.1 --- http_config ssl_session_fetch_by_lua_block { @@ -1494,7 +1403,7 @@ qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, -=== TEST 18: cosocket (UDP) +=== TEST 17: cosocket (UDP) --- http_config ssl_session_fetch_by_lua_block { local sock = ngx.socket.udp() @@ -1589,7 +1498,7 @@ close: 1 nil -=== TEST 19: uthread (kill) +=== TEST 18: uthread (kill) --- http_config ssl_session_fetch_by_lua_block { local function f() @@ -1689,7 +1598,7 @@ uthread: failed to kill: already waited or killed -=== TEST 20: uthread (wait) +=== TEST 19: uthread (wait) --- http_config ssl_session_fetch_by_lua_block { local function f() From 24cafc638c3596f81cdba370371f024486ff0bee Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 19 Sep 2024 07:27:14 +0800 Subject: [PATCH 788/848] tests: fixed CI. --- .travis.yml | 29 ++++++++++++++++------------- util/build-without-ssl.sh | 4 ++-- 2 files changed, 18 insertions(+), 15 deletions(-) diff --git a/.travis.yml b/.travis.yml index c37d3751c2..d73f3fbdc3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -42,8 +42,6 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_VER=8.45 - - PCRE2_VER=10.42 - PCRE_PREFIX=/opt/pcre - PCRE2_PREFIX=/opt/pcre2 - PCRE_LIB=$PCRE_PREFIX/lib @@ -61,13 +59,12 @@ env: - TEST_NGINX_SLEEP=0.006 - MALLOC_PERTURB_=9 jobs: - #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d - #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f - #- NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 - #- NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y TEST_NGINX_QUIC_IDLE_TIMEOUT=3 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.12 USE_PCRE2=Y TEST_NGINX_TIMEOUT=5 #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 + - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f TEST_NGINX_TIMEOUT=5 PCRE_VER=8.45 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.42 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.42 + - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.42 services: - memcached @@ -81,9 +78,12 @@ before_install: install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - - if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi - - if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + #- if [ -n "$PCRE_VER" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi + #- if [ -n "$PCRE2_VER" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi + #- if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + - if [ -n "$OPENSSL_VER" ]; then wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v1.0.0/openssl-${OPENSSL_VER}-x64-focal.tar.gz; fi + - if [ -n "$PCRE_VER" ]; then wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v1.0.0/pcre-${PCRE_VER}-x64-focal.tar.gz; fi + - if [ -n "$PCRE2_VER" ]; then wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v1.0.0/pcre2-${PCRE2_VER}-x64-focal.tar.gz; fi - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git @@ -135,10 +135,13 @@ script: - sudo make install-libdrizzle-1.0 > build.log 2>&1 || (cat build.log && exit 1) - cd ../mockeagain/ && make CC=$CC -j$JOBS && cd .. - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. - - if [ "$USE_PCRE2" != "Y" ]; then tar zxf download-cache/pcre-$PCRE_VER.tar.gz; cd pcre-$PCRE_VER/; ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - - if [ "$USE_PCRE2" = "Y" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - - if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + #- if [ -n "PCRE_VER" ]; then tar zxf download-cache/pcre-$PCRE_VER.tar.gz; cd pcre-$PCRE_VER/; ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + #- if [ -n "$PCRE2_VER" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + #- if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX --libdir=lib -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - if [ -n "$BORINGSSL" ]; then sudo mkdir -p /opt/ssl && sudo tar -C /opt/ssl -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi + - if [ -n "$OPENSSL_VER" ]; then sudo mkdir -p /opt/ssl && sudo tar -C /opt/ssl -xf openssl-$OPENSSL_VER-x64-focal.tar.gz --strip-components=2; fi + - if [ -n "$PCRE_VER" ]; then sudo mkdir -p $PCRE_PREFIX && sudo tar -C $PCRE_PREFIX -xf pcre-$PCRE_VER-x64-focal.tar.gz --strip-components=2; fi + - if [ -n "$PCRE2_VER" ]; then sudo mkdir -p $PCRE2_PREFIX && sudo tar -C $PCRE2_PREFIX -xf pcre2-$PCRE2_VER-x64-focal.tar.gz --strip-components=2; fi - export NGX_BUILD_CC=$CC - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - sh util/build-with-dd.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) diff --git a/util/build-without-ssl.sh b/util/build-without-ssl.sh index c52d5873f6..2a998e3c79 100755 --- a/util/build-without-ssl.sh +++ b/util/build-without-ssl.sh @@ -26,10 +26,10 @@ add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" disable_pcre2=--without-pcre2 answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then +if [ "$answer" = "N" ] || [ -n "$PCRE2_VER" ]; then disable_pcre2="" fi -if [ "$USE_PCRE2" = "Y" ]; then +if [ -n "$PCRE2_VER" ]; then PCRE_INC=$PCRE2_INC PCRE_LIB=$PCRE2_LIB fi From 34167407fa2c912c8d3f701503a54443f7439e06 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 19 Sep 2024 09:36:56 +0800 Subject: [PATCH 789/848] feature: add ngx.resp.set_status(status, reason). --- src/ngx_http_lua_misc.c | 32 +++++++++++++++++++++++- t/015-status.t | 54 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 84 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_misc.c b/src/ngx_http_lua_misc.c index 4e93f68f1e..2fed53f7ad 100644 --- a/src/ngx_http_lua_misc.c +++ b/src/ngx_http_lua_misc.c @@ -64,8 +64,11 @@ ngx_http_lua_ffi_get_resp_status(ngx_http_request_t *r) int -ngx_http_lua_ffi_set_resp_status(ngx_http_request_t *r, int status) +ngx_http_lua_ffi_set_resp_status_and_reason(ngx_http_request_t *r, int status, + const char *reason, size_t reason_len) { + u_char *buf; + if (r->connection->fd == (ngx_socket_t) -1) { return NGX_HTTP_LUA_FFI_BAD_CONTEXT; } @@ -77,6 +80,14 @@ ngx_http_lua_ffi_set_resp_status(ngx_http_request_t *r, int status) return NGX_DECLINED; } + /* per RFC-7230 sec 3.1.2, the status line must be 3 digits, it also makes + * buffer size calculation easier */ + if (status < 100 || status > 999) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "invalid HTTP status code %d", status); + return NGX_DECLINED; + } + r->headers_out.status = status; if (r->err_status) { @@ -91,6 +102,18 @@ ngx_http_lua_ffi_set_resp_status(ngx_http_request_t *r, int status) ngx_str_set(&r->headers_out.status_line, "101 Switching Protocols"); + } else if (reason != NULL && reason_len > 0) { + reason_len += 5; /* "ddd \0" */ + buf = ngx_palloc(r->pool, reason_len); + if (buf == NULL) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "no memory"); + return NGX_DECLINED; + } + + ngx_snprintf(buf, reason_len, "%d %s", status, reason); + r->headers_out.status_line.len = reason_len; + r->headers_out.status_line.data = buf; + } else { r->headers_out.status_line.len = 0; } @@ -99,6 +122,13 @@ ngx_http_lua_ffi_set_resp_status(ngx_http_request_t *r, int status) } +int +ngx_http_lua_ffi_set_resp_status(ngx_http_request_t *r, int status) +{ + return ngx_http_lua_ffi_set_resp_status_and_reason(r, status, NULL, 0); +} + + int ngx_http_lua_ffi_req_is_internal(ngx_http_request_t *r) { diff --git a/t/015-status.t b/t/015-status.t index aa816c08d6..c768aebcea 100644 --- a/t/015-status.t +++ b/t/015-status.t @@ -9,7 +9,7 @@ log_level('warn'); #repeat_each(120); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 9); +plan tests => repeat_each() * (blocks() * 2 + 10); #no_diff(); #no_long_string(); @@ -293,3 +293,55 @@ ngx.status: 654 --- no_error_log [error] --- error_code: 654 + + + +=== TEST 17: set status and reason +--- config +location = /upstream { + content_by_lua_block { + local resp = require "ngx.resp" + resp.set_status(500, "user defined reason") + ngx.say("set_status_and_reason") + } +} + +location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.server_port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local req = "GET /upstream HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + local found = false + while true do + local line, err, part = sock:receive() + if line then + if ngx.re.find(line, "HTTP/1.1 500 user defined reason") then + ngx.say("match") + end + else + break + end + end + + sock:close() + } +} +--- request +GET /t +--- response_body +match +--- no_error_log +[error] From de1b896999c95bcbfc2b60c4a2247092d9c7fe40 Mon Sep 17 00:00:00 2001 From: spacewander Date: Tue, 7 Feb 2017 21:04:18 +0800 Subject: [PATCH 790/848] style: remove duplicate check in ngx.send_headers --- src/ngx_http_lua_output.c | 16 +++++++--------- src/ngx_http_lua_util.c | 9 +++------ 2 files changed, 10 insertions(+), 15 deletions(-) diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index b2a98d133a..2648e4e952 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -722,16 +722,14 @@ ngx_http_lua_ngx_send_headers(lua_State *L) | NGX_HTTP_LUA_CONTEXT_ACCESS | NGX_HTTP_LUA_CONTEXT_CONTENT); - if (!r->header_sent && !ctx->header_sent) { - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua send headers"); + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua send headers"); - rc = ngx_http_lua_send_header_if_needed(r, ctx); - if (rc == NGX_ERROR || rc > NGX_OK) { - lua_pushnil(L); - lua_pushliteral(L, "nginx output filter error"); - return 2; - } + rc = ngx_http_lua_send_header_if_needed(r, ctx); + if (rc == NGX_ERROR || rc > NGX_OK) { + lua_pushnil(L); + lua_pushliteral(L, "nginx output filter error"); + return 2; } lua_pushinteger(L, 1); diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 5d871241db..4e6204c79d 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -535,13 +535,10 @@ ngx_http_lua_send_header_if_needed(ngx_http_request_t *r, r->headers_out.status = NGX_HTTP_OK; } - if (!ctx->mime_set - && ngx_http_lua_set_content_type(r, ctx) != NGX_OK) - { - return NGX_ERROR; - } - if (!ctx->headers_set) { + if (ngx_http_lua_set_content_type(r) != NGX_OK) { + return NGX_ERROR; + } ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); } From 408819422433ece1904e4851e2c859dcd4cbb823 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 19 Sep 2024 10:57:28 +0800 Subject: [PATCH 791/848] style: fixed coding style. --- src/ngx_http_lua_output.c | 2 +- src/ngx_http_lua_util.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_output.c b/src/ngx_http_lua_output.c index 2648e4e952..8681947263 100644 --- a/src/ngx_http_lua_output.c +++ b/src/ngx_http_lua_output.c @@ -723,7 +723,7 @@ ngx_http_lua_ngx_send_headers(lua_State *L) | NGX_HTTP_LUA_CONTEXT_CONTENT); ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua send headers"); + "lua send headers"); rc = ngx_http_lua_send_header_if_needed(r, ctx); if (rc == NGX_ERROR || rc > NGX_OK) { diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 4e6204c79d..4e9a8946ef 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -539,6 +539,7 @@ ngx_http_lua_send_header_if_needed(ngx_http_request_t *r, if (ngx_http_lua_set_content_type(r) != NGX_OK) { return NGX_ERROR; } + ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); } From a634f1028af51ee3c70e94ba8f5176d7f781b3dd Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 21 Sep 2024 15:25:36 +0800 Subject: [PATCH 792/848] bugfix: build failure introduced by commit de1b896999c95. --- src/ngx_http_lua_util.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 4e9a8946ef..4386bc67d6 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -536,7 +536,7 @@ ngx_http_lua_send_header_if_needed(ngx_http_request_t *r, } if (!ctx->headers_set) { - if (ngx_http_lua_set_content_type(r) != NGX_OK) { + if (ngx_http_lua_set_content_type(r, ctx) != NGX_OK) { return NGX_ERROR; } From a7eebb4793668eaab015647749469f25a3d37038 Mon Sep 17 00:00:00 2001 From: WenMing Date: Thu, 26 Jul 2018 20:02:22 +0800 Subject: [PATCH 793/848] feature: implemented ngx_http_lua_ffi_decode_base64mime. --- README.markdown | 20 +++++++++- doc/HttpLuaModule.wiki | 16 +++++++- src/ngx_http_lua_string.c | 20 ++++++++++ src/ngx_http_lua_util.c | 80 +++++++++++++++++++++++++++++++++++++++ src/ngx_http_lua_util.h | 1 + 5 files changed, 135 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 25d1383f36..2b2d2b251d 100644 --- a/README.markdown +++ b/README.markdown @@ -3695,6 +3695,7 @@ Nginx API for Lua * [ngx.decode_args](#ngxdecode_args) * [ngx.encode_base64](#ngxencode_base64) * [ngx.decode_base64](#ngxdecode_base64) +* [ngx.decode_base64mime](#ngxdecode_base64mime) * [ngx.crc32_short](#ngxcrc32_short) * [ngx.crc32_long](#ngxcrc32_long) * [ngx.hmac_sha1](#ngxhmac_sha1) @@ -6267,7 +6268,24 @@ ngx.decode_base64 **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** -Decodes the `str` argument as a base64 digest to the raw form. Returns `nil` if `str` is not well formed. +Decodes the `str` argument as a base64 digest to the raw form. +The `str` should be standard 'base64' encoding for RFC 3548 or RFC 4648, and will returns `nil` if is not well formed or any characters not in the base encoding alphabet. + +[Back to TOC](#nginx-api-for-lua) + +ngx.decode_base64mime +--------------------- +**syntax:** *newstr = ngx.decode_base64mime(str)* + +**context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** + +**requires:** `resty.core.base64` or `resty.core` + +Decodes the `str` argument as a base64 digest to the raw form. +The `str` follows base64 transfer encoding for MIME (RFC 2045), and will discard characters outside the base encoding alphabet. +Returns `nil` if `str` is not well formed. + + '''Note:''' This method requires the resty.core.base64 or resty.core modules from the [lua-resty-core](https://github.com/openresty/lua-resty-core) library. [Back to TOC](#nginx-api-for-lua) diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index a0e6d28fac..733f11dc66 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5256,7 +5256,21 @@ Since the 0.9.16 release, an optional boolean-typed no_paddin '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua*'' -Decodes the str argument as a base64 digest to the raw form. Returns nil if str is not well formed. +Decodes the str argument as a base64 digest to the raw form. +The str should be standard 'base64' encoding for RFC 3548 or RFC 4648, and will returns nil if is not well formed or any characters not in the base encoding alphabet. + +== ngx.decode_base64mime == +'''syntax:''' ''newstr = ngx.decode_base64mime(str)'' + +'''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' + +'''requires:''' resty.core.base64 or resty.core + +Decodes the str argument as a base64 digest to the raw form. +The str follows base64 transfer encoding for MIME (RFC 2045), and will discard characters outside the base encoding alphabet. +Returns nil if str is not well formed. + + '''Note:''' This method requires the resty.core.base64 or resty.core modules from the [https://github.com/openresty/lua-resty-core lua-resty-core] library. == ngx.crc32_short == diff --git a/src/ngx_http_lua_string.c b/src/ngx_http_lua_string.c index 4c755f67f3..3028483609 100644 --- a/src/ngx_http_lua_string.c +++ b/src/ngx_http_lua_string.c @@ -424,6 +424,26 @@ ngx_http_lua_ffi_decode_base64(const u_char *src, size_t slen, u_char *dst, } +int +ngx_http_lua_ffi_decode_base64mime(const u_char *src, size_t slen, u_char *dst, + size_t *dlen) +{ + ngx_int_t rc; + ngx_str_t in, out; + + in.data = (u_char *) src; + in.len = slen; + + out.data = dst; + + rc = ngx_http_lua_decode_base64mime(&out, &in); + + *dlen = out.len; + + return rc == NGX_OK; +} + + size_t ngx_http_lua_ffi_unescape_uri(const u_char *src, size_t len, u_char *dst) { diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 4386bc67d6..e3d46db788 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4401,6 +4401,86 @@ ngx_http_lua_copy_escaped_header(ngx_http_request_t *r, } +ngx_int_t +ngx_http_lua_decode_base64mime(ngx_str_t *dst, ngx_str_t *src) +{ + size_t i; + u_char *d, *s, ch; + size_t data_len = 0; + u_char buf[4]; + size_t buf_len = 0; + static u_char basis[] = { + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 62, 77, 77, 77, 63, + 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 77, 77, 77, 77, 77, 77, + 77, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, + 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 77, 77, 77, 77, 77, + 77, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, + 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 77, 77, 77, 77, 77, + + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77 + }; + + for (i = 0; i < src->len; i++) { + ch = src->data[i]; + if (ch == '=') { + break; + } + + if (basis[ch] == 77) { + continue; + } + + data_len++; + } + + if (data_len % 4 == 1) { + return NGX_ERROR; + } + + s = src->data; + d = dst->data; + + for (i = 0; i < src->len; i++) { + if (s[i] == '=') { + break; + } + + if (basis[s[i]] == 77) { + continue; + } + + buf[buf_len++] = s[i]; + if (buf_len == 4) { + *d++ = (u_char) (basis[buf[0]] << 2 | basis[buf[1]] >> 4); + *d++ = (u_char) (basis[buf[1]] << 4 | basis[buf[2]] >> 2); + *d++ = (u_char) (basis[buf[2]] << 6 | basis[buf[3]]); + buf_len = 0; + } + } + + if (buf_len > 1) { + *d++ = (u_char) (basis[buf[0]] << 2 | basis[buf[1]] >> 4); + } + + if (buf_len > 2) { + *d++ = (u_char) (basis[buf[1]] << 4 | basis[buf[2]] >> 2); + } + + dst->len = d - dst->data; + + return NGX_OK; +} + + ngx_addr_t * ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len) { diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index faea7a072c..85c6e614d6 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -261,6 +261,7 @@ void ngx_http_lua_cleanup_free(ngx_http_request_t *r, #if (NGX_HTTP_LUA_HAVE_SA_RESTART) void ngx_http_lua_set_sa_restart(ngx_log_t *log); #endif +ngx_int_t ngx_http_lua_decode_base64mime(ngx_str_t *dst, ngx_str_t *src); ngx_addr_t *ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len); From 0d9f578070388f58d8e74f10897a2e05fd919e1a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 21 Sep 2024 17:24:51 +0800 Subject: [PATCH 794/848] tests: fixed t/062-count.t --- t/062-count.t | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/062-count.t b/t/062-count.t index 957590292b..d0eabb1f2d 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -34,7 +34,7 @@ __DATA__ --- request GET /test --- response_body -ngx: 116 +ngx: 117 --- no_error_log [error] @@ -55,7 +55,7 @@ ngx: 116 --- request GET /test --- response_body -116 +117 --- no_error_log [error] @@ -83,7 +83,7 @@ GET /test --- request GET /test --- response_body -n = 116 +n = 117 --- no_error_log [error] @@ -306,7 +306,7 @@ GET /t --- response_body_like: 404 Not Found --- error_code: 404 --- error_log -ngx. entry count: 116 +ngx. entry count: 117 From 2e1718e15d8450c8c5ef6bb2a4a29d9332a7ca9a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 21 Sep 2024 17:33:03 +0800 Subject: [PATCH 795/848] bugfix: missing the content-type header introduced by commit de1b896999c9. --- src/ngx_http_lua_util.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index e3d46db788..c73bddc609 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -535,11 +535,13 @@ ngx_http_lua_send_header_if_needed(ngx_http_request_t *r, r->headers_out.status = NGX_HTTP_OK; } - if (!ctx->headers_set) { - if (ngx_http_lua_set_content_type(r, ctx) != NGX_OK) { - return NGX_ERROR; - } + if (!ctx->mime_set + && ngx_http_lua_set_content_type(r, ctx) != NGX_OK) + { + return NGX_ERROR; + } + if (!ctx->headers_set) { ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); } From e1ff117183c25a20a35c59c536a882820296a5cf Mon Sep 17 00:00:00 2001 From: n-bes Date: Wed, 25 Sep 2024 03:29:15 +0300 Subject: [PATCH 796/848] fix: removed null terminator. --- src/ngx_http_lua_misc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_misc.c b/src/ngx_http_lua_misc.c index 2fed53f7ad..8b74167161 100644 --- a/src/ngx_http_lua_misc.c +++ b/src/ngx_http_lua_misc.c @@ -103,7 +103,7 @@ ngx_http_lua_ffi_set_resp_status_and_reason(ngx_http_request_t *r, int status, ngx_str_set(&r->headers_out.status_line, "101 Switching Protocols"); } else if (reason != NULL && reason_len > 0) { - reason_len += 5; /* "ddd \0" */ + reason_len += 4; /* "ddd " */ buf = ngx_palloc(r->pool, reason_len); if (buf == NULL) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "no memory"); From e6219b05283ed26c7ee1d58777bf252cc614819d Mon Sep 17 00:00:00 2001 From: Thijs Schreijer Date: Thu, 3 Oct 2024 13:53:25 +0200 Subject: [PATCH 797/848] doc: clarify the backlog option (#2367) --- README.markdown | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.markdown b/README.markdown index 2b2d2b251d..d85a1f0753 100644 --- a/README.markdown +++ b/README.markdown @@ -7989,14 +7989,14 @@ An optional Lua table can be specified as the last argument to this method to sp * `backlog` if specified, this module will limit the total number of opened connections for this pool. No more connections than `pool_size` can be opened - for this pool at any time. If the connection pool is full, subsequent - connect operations will be queued into a queue equal to this option's - value (the "backlog" queue). + for this pool at any time. If `pool_size` number of connections are in use, + subsequent connect operations will be queued into a queue equal to this + option's value (the "backlog" queue). If the number of queued connect operations is equal to `backlog`, subsequent connect operations will fail and return `nil` plus the error string `"too many waiting connect operations"`. - The queued connect operations will be resumed once the number of connections - in the pool is less than `pool_size`. + The queued connect operations will be resumed once the number of active + connections becomes less than `pool_size`. The queued connect operation will abort once they have been queued for more than `connect_timeout`, controlled by [settimeouts](#tcpsocksettimeouts), and will return `nil` plus From 9a871ddad3f7919740c4d026d0f6c21e7f7a1a0f Mon Sep 17 00:00:00 2001 From: Thijs Schreijer Date: Tue, 15 Oct 2024 13:03:50 +0200 Subject: [PATCH 798/848] doc: clarify base64 for url encoding and padding. --- README.markdown | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.markdown b/README.markdown index d85a1f0753..35c0b83e58 100644 --- a/README.markdown +++ b/README.markdown @@ -6255,7 +6255,7 @@ ngx.encode_base64 **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** -Encodes `str` to a base64 digest. +Encodes `str` to a base64 digest. For base64url encoding use [`base64.encode_base64url`](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/base64.md#encode_base64url). Since the `0.9.16` release, an optional boolean-typed `no_padding` argument can be specified to control whether the base64 padding should be appended to the resulting digest (default to `false`, i.e., with padding enabled). @@ -6268,8 +6268,9 @@ ngx.decode_base64 **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, ssl_client_hello_by_lua** -Decodes the `str` argument as a base64 digest to the raw form. -The `str` should be standard 'base64' encoding for RFC 3548 or RFC 4648, and will returns `nil` if is not well formed or any characters not in the base encoding alphabet. +Decodes the `str` argument as a base64 digest to the raw form. For base64url decoding use [`base64.decode_base64url`](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/base64.md#decode_base64url). + +The `str` should be standard 'base64' encoding for RFC 3548 or RFC 4648, and will returns `nil` if is not well formed or any characters not in the base encoding alphabet. Padding may be omitted from the input. [Back to TOC](#nginx-api-for-lua) From 501ebacd731a3a27bf76c2514b3e45cdca380605 Mon Sep 17 00:00:00 2001 From: willmafh Date: Sun, 27 Oct 2024 08:51:17 +0800 Subject: [PATCH 799/848] chore: more accurate error message. --- src/ngx_http_lua_ssl_certby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 0901f06eab..72a651bdbf 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1433,7 +1433,7 @@ ngx_http_lua_ffi_set_priv_key(ngx_http_request_t *r, pkey = cdata; if (pkey == NULL) { - *err = "invalid private key failed"; + *err = "invalid private key"; goto failed; } From 950b1fb2010c8847a1e0bc7f079741ec47a08f17 Mon Sep 17 00:00:00 2001 From: Jun Ouyang Date: Wed, 6 Nov 2024 10:12:11 +0800 Subject: [PATCH 800/848] bugfix: remove http2 hardcode limitation in `ngx.location` subrequest API. --- .travis.yml | 2 ++ src/ngx_http_lua_subrequest.c | 6 ----- t/005-exit.t | 5 ++++ t/014-bugs.t | 2 +- t/016-resp-header.t | 1 + t/020-subrequest.t | 1 + t/023-rewrite/exit.t | 3 +++ t/024-access/exit.t | 3 +++ t/026-mysql.t | 1 + t/033-ctx.t | 2 +- t/041-header-filter.t | 46 ++++++++++++++++++----------------- t/056-flush.t | 3 +-- t/068-socket-keepalive.t | 3 +++ t/091-coroutine.t | 5 ++-- t/094-uthread-exit.t | 8 +++--- t/095-uthread-exec.t | 1 + t/096-uthread-redirect.t | 1 + t/138-balancer.t | 22 ++--------------- t/143-ssl-session-fetch.t | 3 +-- t/163-signal.t | 2 +- 20 files changed, 60 insertions(+), 60 deletions(-) diff --git a/.travis.yml b/.travis.yml index d73f3fbdc3..4eb8d3e5ce 100644 --- a/.travis.yml +++ b/.travis.yml @@ -63,6 +63,8 @@ env: #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f TEST_NGINX_TIMEOUT=5 PCRE_VER=8.45 - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.42 + - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f TEST_NGINX_TIMEOUT=5 PCRE_VER=8.45 TEST_NGINX_USE_HTTP2=1 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.42 TEST_NGINX_USE_HTTP2=1 - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.42 - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.42 diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index 2ccd271a90..edb24e9c0d 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -173,12 +173,6 @@ ngx_http_lua_ngx_location_capture_multi(lua_State *L) return luaL_error(L, "no request object found"); } -#if (NGX_HTTP_V2) - if (r->main->stream) { - return luaL_error(L, "http2 requests not supported yet"); - } -#endif - ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx == NULL) { return luaL_error(L, "no ctx found"); diff --git a/t/005-exit.t b/t/005-exit.t index 0783c69295..bbaeda517e 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -124,6 +124,7 @@ GET /api?user=agentz === TEST 6: working with ngx_auth_request (simplest form, w/o ngx_memc) --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -197,6 +198,7 @@ Logged in 56 === TEST 7: working with ngx_auth_request (simplest form) --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -269,6 +271,7 @@ Logged in 56 === TEST 8: working with ngx_auth_request +--- no_http2 --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} --- http_config eval " @@ -762,6 +765,7 @@ GET /t === TEST 27: accepts NGX_ERROR +--- no_http2 --- config location = /t { content_by_lua_block { @@ -780,6 +784,7 @@ curl: (95) HTTP/3 stream 0 reset by server === TEST 28: accepts NGX_DECLINED +--- no_http2 --- config location = /t { content_by_lua_block { diff --git a/t/014-bugs.t b/t/014-bugs.t index d34f42e23d..303187929f 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -201,7 +201,7 @@ https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2 header field names MUST be converted to lowercase prior to their encoding in HTTP/2. A request or response containing uppercase header field names MUST be treated as malformed - +--- no_http2 --- config location /sub { content_by_lua ' diff --git a/t/016-resp-header.t b/t/016-resp-header.t index 6cf699d88e..b30090812c 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -298,6 +298,7 @@ Fooy: cony1, cony2 === TEST 15: set header after ngx.print +--- no_http2 --- config location /lua { default_type "text/plain"; diff --git a/t/020-subrequest.t b/t/020-subrequest.t index 37914be061..e91f3f6253 100644 --- a/t/020-subrequest.t +++ b/t/020-subrequest.t @@ -1198,6 +1198,7 @@ body: === TEST 43: subrequests with an output body filter returning NGX_ERROR +--- no_http2 --- config location /sub { echo hello world; diff --git a/t/023-rewrite/exit.t b/t/023-rewrite/exit.t index 9add80441c..7f01717eb6 100644 --- a/t/023-rewrite/exit.t +++ b/t/023-rewrite/exit.t @@ -120,6 +120,7 @@ GET /api?user=agentz === TEST 6: working with ngx_auth_request (simplest form, w/o ngx_memc) +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -192,6 +193,7 @@ Logged in 56 === TEST 7: working with ngx_auth_request (simplest form) +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -264,6 +266,7 @@ Logged in 56 === TEST 8: working with ngx_auth_request +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; diff --git a/t/024-access/exit.t b/t/024-access/exit.t index b77778213a..661ee2d8bc 100644 --- a/t/024-access/exit.t +++ b/t/024-access/exit.t @@ -114,6 +114,7 @@ GET /api?user=agentz === TEST 6: working with ngx_auth_request (simplest form, w/o ngx_memc) +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -182,6 +183,7 @@ Logged in 56 === TEST 7: working with ngx_auth_request (simplest form) +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; @@ -249,6 +251,7 @@ Logged in 56 === TEST 8: working with ngx_auth_request +--- no_http2 --- http_config eval " lua_package_cpath '$::LuaCpath'; diff --git a/t/026-mysql.t b/t/026-mysql.t index 02e14b9382..e7ab170600 100644 --- a/t/026-mysql.t +++ b/t/026-mysql.t @@ -16,6 +16,7 @@ run_tests(); __DATA__ === TEST 1: when mysql query timed out, kill that query by Lua +--- no_http2 --- http_config upstream backend { drizzle_server 127.0.0.1:$TEST_NGINX_MYSQL_PORT protocol=mysql diff --git a/t/033-ctx.t b/t/033-ctx.t index 782a0fab6e..77bd15f8c1 100644 --- a/t/033-ctx.t +++ b/t/033-ctx.t @@ -279,7 +279,7 @@ GET /t --- error_log ngx.ctx = 32 --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 23fdac02cd..84bb39fba2 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -125,7 +125,7 @@ GET /read --- error_code --- response_body --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -449,6 +449,7 @@ GET /lua === TEST 21: lua error (string) +--- no_http2 --- config location /lua { set $foo ''; @@ -468,11 +469,12 @@ failed to run header_filter_by_lua*: header_filter_by_lua(nginx.conf:47):2: Some --- no_error_log [alert] --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(28\) Remote peer returned unexpected data|curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server/ === TEST 22: lua error (nil) +--- no_http2 --- config location /lua { set $foo ''; @@ -492,7 +494,7 @@ failed to run header_filter_by_lua*: unknown reason --- no_error_log [alert] --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(28\) Remote peer returned unexpected data|curl: \(52\) Empty reply from server|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -508,7 +510,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -524,7 +526,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -540,7 +542,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -556,7 +558,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -572,7 +574,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -592,7 +594,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -612,7 +614,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -628,7 +630,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -644,7 +646,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -660,7 +662,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -694,7 +696,7 @@ GET /lua --- error_log eval qr/API disabled in the context of header_filter_by_lua\*|http3 requests are not supported without content-length header/ms --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -718,7 +720,7 @@ if (defined $ENV{TEST_NGINX_USE_HTTP3}) { $err_log; --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -734,7 +736,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -750,7 +752,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -799,7 +801,7 @@ in function 'error' in function 'bar' in function 'foo' --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -819,7 +821,7 @@ GET /lua?a=1&b=2 --- error_log eval qr/failed to load external Lua file ".*?test2\.lua": cannot open .*? No such file or directory/ --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -867,7 +869,7 @@ failed to load inlined Lua code: header_filter_by_lua(nginx.conf:41):2: unexpect --- no_error_log no_such_error --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly: INTERNAL_ERROR \(err 2\)/ @@ -898,7 +900,7 @@ failed to load inlined Lua code: header_filter_by_lua(nginx.conf:49):2: unexpect --- no_error_log no_such_error --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly: INTERNAL_ERROR \(err 2\)/ @@ -924,4 +926,4 @@ failed to load inlined Lua code: header_filter_by_lua(...90123456789012345678901 --- no_error_log [alert] --- curl_error eval -qr/curl: \(56\) Failure when receiving data from the peer/ +qr/curl: \(56\) Failure when receiving data from the peer|curl: \(56\) Failure when receiving data from the peer|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly: INTERNAL_ERROR \(err 2\)/ diff --git a/t/056-flush.t b/t/056-flush.t index 4376b18930..d2b107c754 100644 --- a/t/056-flush.t +++ b/t/056-flush.t @@ -516,7 +516,7 @@ GET /test my @errlog; if (defined $ENV{TEST_NGINX_USE_HTTP2}) { @errlog = [ -qr/lua writes elapsed 0\.[7-9]\d+ sec/, +qr/lua writes elapsed (?:0\.[7-9]\d+|[12]\.\d+) sec/, qr/lua flush requires waiting: buffered 0x[0-9a-f]+, delayed:1/, ]; } else { @@ -526,7 +526,6 @@ qr/lua flush requires waiting: buffered 0x[0-9a-f]+, delayed:1/, ]; } @errlog; - --- no_error_log [error] --- timeout: 4 diff --git a/t/068-socket-keepalive.t b/t/068-socket-keepalive.t index 626b441678..423d391799 100644 --- a/t/068-socket-keepalive.t +++ b/t/068-socket-keepalive.t @@ -1384,6 +1384,7 @@ bad argument #3 to 'connect' (bad "pool" option type: boolean) === TEST 23: clear the redis store +--- no_http2 --- config location /t { redis2_query flushall; @@ -3033,6 +3034,7 @@ lua tcp socket keepalive create connection pool for key "B" === TEST 54: wrong first argument for setkeepalive +--- no_http2 --- quic_max_idle_timeout: 1.2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" @@ -3115,6 +3117,7 @@ qr{HTTP/3 stream 0 reset by server} === TEST 55: wrong second argument for setkeepalive +--- no_http2 --- quic_max_idle_timeout: 1.2 --- http_config eval "lua_package_path '$::HtmlDir/?.lua;./?.lua;;';" diff --git a/t/091-coroutine.t b/t/091-coroutine.t index bfbdb38937..16930ed3c0 100644 --- a/t/091-coroutine.t +++ b/t/091-coroutine.t @@ -763,7 +763,7 @@ GET /lua --- error_log API disabled in the context of header_filter_by_lua* --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ @@ -1423,6 +1423,7 @@ GET /t === TEST 35: coroutine.wrap runtime errors do not log errors +--- no_http2 --- config location = /t { content_by_lua_block { @@ -1700,7 +1701,7 @@ GET /t "in function 'co'" ] --- curl_error eval -qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 0 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ +qr/curl: \(52\) Empty reply from server|curl: \(92\) HTTP\/2 stream 1 was not closed cleanly|curl: \(95\) HTTP\/3 stream 0 reset by server/ diff --git a/t/094-uthread-exit.t b/t/094-uthread-exit.t index 0194e44b31..375d8d6023 100644 --- a/t/094-uthread-exit.t +++ b/t/094-uthread-exit.t @@ -8,7 +8,7 @@ our $StapScript = $t::StapThread::StapScript; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4); +plan tests => repeat_each() * (blocks() * 4 - 2); $ENV{TEST_NGINX_RESOLVER} ||= '8.8.8.8'; $ENV{TEST_NGINX_MEMCACHED_PORT} ||= '11211'; @@ -1135,7 +1135,6 @@ free request attempt to abort with pending subrequests --- no_error_log [alert] -[warn] @@ -1313,6 +1312,7 @@ attempt to abort with pending subrequests === TEST 16: exit in entry thread (user thread is still pending on ngx.location.capture_multi), without pending output +--- no_http2 --- config location /lua { client_body_timeout 12000ms; @@ -1407,6 +1407,7 @@ qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by se === TEST 17: exit(444) in user thread (entry thread is still pending on ngx.location.capture), with pending output +--- no_http2 --- config location /lua { client_body_timeout 12000ms; @@ -1492,6 +1493,7 @@ qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by se === TEST 18: exit(408) in user thread (entry thread is still pending on ngx.location.capture), with pending output +--- no_http2 --- config location /lua { client_body_timeout 12000ms; @@ -1655,6 +1657,6 @@ free request --- no_error_log [alert] [error] -[warn] + --- curl_error eval qr#curl: \(52\) Empty reply from server|curl: \(95\) HTTP/3 stream 0 reset by server# diff --git a/t/095-uthread-exec.t b/t/095-uthread-exec.t index 4cd121da1e..9ef6356090 100644 --- a/t/095-uthread-exec.t +++ b/t/095-uthread-exec.t @@ -344,6 +344,7 @@ attempt to abort with pending subrequests === TEST 6: exec in entry thread (user thread is still pending on ngx.location.capture), without pending output +--- no_http2 --- config location /lua { client_body_timeout 12000ms; diff --git a/t/096-uthread-redirect.t b/t/096-uthread-redirect.t index 62909b944c..5df5ecfb09 100644 --- a/t/096-uthread-redirect.t +++ b/t/096-uthread-redirect.t @@ -190,6 +190,7 @@ free request === TEST 3: ngx.redirect() in entry thread (user thread is still pending on ngx.location.capture_multi), without pending output +--- no_http2 --- config location /lua { client_body_timeout 12000ms; diff --git a/t/138-balancer.t b/t/138-balancer.t index 41be75fcdc..f48c7fa41d 100644 --- a/t/138-balancer.t +++ b/t/138-balancer.t @@ -12,7 +12,7 @@ use Test::Nginx::Socket::Lua; repeat_each(2); -plan tests => repeat_each() * (blocks() * 4 + 7); +plan tests => repeat_each() * (blocks() * 4 - 3); #no_diff(); no_long_string(); @@ -41,8 +41,6 @@ __DATA__ '[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] ---- no_error_log -[warn] @@ -67,7 +65,6 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ [lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream, --- no_error_log eval [ -'[warn]', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] @@ -95,8 +92,6 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ '[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] ---- no_error_log -[warn] @@ -125,8 +120,6 @@ qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\ "2: variable foo = 33", qr/\[crit\] .* connect\(\) .*? failed/, ] ---- no_error_log -[warn] @@ -153,8 +146,6 @@ Foo: bar "header foo: bar", qr/\[crit\] .* connect\(\) .*? failed/, ] ---- no_error_log -[warn] @@ -180,12 +171,11 @@ Foo: bar ["arg foo: bar", qr/\[crit\] .* connect\(\) .*? failed/, ] ---- no_error_log -[warn] === TEST 7: ngx.req.get_method() works +--- no_http2 --- http_config upstream backend { server 0.0.0.1; @@ -208,8 +198,6 @@ Foo: bar "method: GET", qr/\[crit\] .* connect\(\) .*? failed/, ] ---- no_error_log -[warn] @@ -235,8 +223,6 @@ print("hello from balancer by lua!") '[lua] a.lua:1: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] ---- no_error_log -[warn] @@ -431,8 +417,6 @@ ctx counter: nil '[lua] balancer_by_lua(nginx.conf:27):2: hello from balancer by lua! while connecting to upstream,', qr{\[crit\] .*? connect\(\) to 0\.0\.0\.1:80 failed .*?, upstream: "http://0\.0\.0\.1:80/t"}, ] ---- no_error_log -[warn] @@ -589,8 +573,6 @@ upstream sent more data than specified in "Content-Length" header while reading --- error_code: 500 --- error_log eval "failed to load inlined Lua code: balancer_by_lua(nginx.conf:27):3: ')' expected (to close '(' at line 2) near ''", ---- no_error_log -[warn] diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index b8ca095aa0..8e09a52d64 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -7,7 +7,7 @@ use File::Basename; repeat_each(3); -plan tests => repeat_each() * (blocks() * 6); +plan tests => repeat_each() * (blocks() * 6) - 3; $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -1319,7 +1319,6 @@ connected: 1 ssl handshake: cdata close: 1 nil --- no_error_log -[warn] [error] [alert] [emerg] diff --git a/t/163-signal.t b/t/163-signal.t index 0ce8fa2613..a0c2ee8082 100644 --- a/t/163-signal.t +++ b/t/163-signal.t @@ -42,7 +42,7 @@ qr/\[notice\] \d+#\d+: exit$/ --- no_error_log eval qr/\[notice\] \d+#\d+: reconfiguring/ --- curl_error eval -qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received|curl: \(56\) Recv failure: Connection reset by peer|curl: \(55\) sendmsg\(\) returned -1 \(errno 111\)/ +qr/curl: \(28\) Operation timed out after \d+ milliseconds with 0 bytes received|curl: \(56\) Recv failure: Connection reset by peer|curl: \(56\) Failure when receiving data from the peer|curl: \(55\) sendmsg\(\) returned -1 \(errno 111\)/ From 5bd471a4bce6c02dd68b1ca72fbabb09fb2041d8 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 17 Jan 2025 13:24:35 +0800 Subject: [PATCH 801/848] bumped version of lua-nginx-module to 10028. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 4b374f56ee..021044eb87 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10027 +#define ngx_http_lua_version 10028 typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; From 004922e1cf95eabde001203e2010365ff5d3e70d Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 13 Feb 2025 17:35:03 +0800 Subject: [PATCH 802/848] feature: Export three functions for manipulating ngx_http_lua_co_ctx_t structures. --- src/api/ngx_http_lua_api.h | 8 ++++++++ src/ngx_http_lua_api.c | 25 ++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index 021044eb87..fa59095c6a 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -20,6 +20,7 @@ #define ngx_http_lua_version 10028 +#define NGX_HTTP_LUA_EXPORT_CO_CTX_CLEANUP 1 typedef struct ngx_http_lua_co_ctx_s ngx_http_lua_co_ctx_t; @@ -66,6 +67,13 @@ void ngx_http_lua_set_cur_co_ctx(ngx_http_request_t *r, lua_State *ngx_http_lua_get_co_ctx_vm(ngx_http_lua_co_ctx_t *coctx); + +void *ngx_http_lua_get_co_ctx_data(ngx_http_lua_co_ctx_t *coctx); +void ngx_http_lua_set_co_ctx_cleanup(ngx_http_lua_co_ctx_t *coctx, + ngx_http_cleanup_pt cleanup, void *data); +void ngx_http_lua_cleanup_co_ctx_pending_operation( + ngx_http_lua_co_ctx_t *coctx); + void ngx_http_lua_co_ctx_resume_helper(ngx_http_lua_co_ctx_t *coctx, int nrets); int ngx_http_lua_get_lua_http10_buffering(ngx_http_request_t *r); diff --git a/src/ngx_http_lua_api.c b/src/ngx_http_lua_api.c index 0d3ec9c988..2478f0da80 100644 --- a/src/ngx_http_lua_api.c +++ b/src/ngx_http_lua_api.c @@ -221,7 +221,7 @@ ngx_http_lua_get_cur_co_ctx(ngx_http_request_t *r) ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); - return ctx->cur_co_ctx; + return ctx == NULL ? NULL : ctx->cur_co_ctx; } @@ -245,6 +245,29 @@ ngx_http_lua_get_co_ctx_vm(ngx_http_lua_co_ctx_t *coctx) } +void * +ngx_http_lua_get_co_ctx_data(ngx_http_lua_co_ctx_t *coctx) +{ + return coctx ? coctx->data : NULL; +} + + +void +ngx_http_lua_set_co_ctx_cleanup(ngx_http_lua_co_ctx_t *coctx, + ngx_http_cleanup_pt cleanup, void *data) +{ + coctx->cleanup = cleanup; + coctx->data = data; +} + + +void +ngx_http_lua_cleanup_co_ctx_pending_operation(ngx_http_lua_co_ctx_t *coctx) +{ + ngx_http_lua_cleanup_pending_operation(coctx); +} + + static ngx_int_t ngx_http_lua_co_ctx_resume(ngx_http_request_t *r) { From e8f65dc53d6a4f20c6e9eb4fad3b8f2eae3cca59 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 15 Feb 2025 18:13:21 +0800 Subject: [PATCH 803/848] bugfix: didn't flush send buffer after lua phase(access/rewrite/server_rewrite) done. --- src/ngx_http_lua_accessby.c | 5 +- src/ngx_http_lua_rewriteby.c | 4 +- src/ngx_http_lua_server_rewriteby.c | 4 +- t/015-status.t | 186 +++++++++++++++++++++++++++- 4 files changed, 192 insertions(+), 7 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index d40eab123e..b8c26605cf 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -95,6 +95,7 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) dd("entered? %d", (int) ctx->entered_access_phase); + if (ctx->entered_access_phase) { dd("calling wev handler"); rc = ctx->resume_handler(r); @@ -105,7 +106,7 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) } if (rc == NGX_OK) { - if (r->header_sent) { + if (r->header_sent || r->headers_out.status!= 0) { dd("header already sent"); /* response header was already generated in access_by_lua*, @@ -369,7 +370,7 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) #if 1 if (rc == NGX_OK) { - if (r->header_sent) { + if (r->header_sent || r->headers_out.status != 0) { dd("header already sent"); /* response header was already generated in access_by_lua*, diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 4109f288e3..091d765e41 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -107,7 +107,7 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) } if (rc == NGX_DECLINED) { - if (r->header_sent) { + if (r->header_sent || r->headers_out.status != 0) { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, @@ -349,7 +349,7 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) } if (rc == NGX_OK || rc == NGX_DECLINED) { - if (r->header_sent) { + if (r->header_sent || r->headers_out.status != 0) { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c index be860693f4..ed37f03efa 100644 --- a/src/ngx_http_lua_server_rewriteby.c +++ b/src/ngx_http_lua_server_rewriteby.c @@ -69,7 +69,7 @@ ngx_http_lua_server_rewrite_handler(ngx_http_request_t *r) } if (rc == NGX_DECLINED) { - if (r->header_sent) { + if (r->header_sent || r->headers_out.status != 0) { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, @@ -307,7 +307,7 @@ ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) } if (rc == NGX_OK || rc == NGX_DECLINED) { - if (r->header_sent) { + if (r->header_sent || r->headers_out.status != 0) { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, diff --git a/t/015-status.t b/t/015-status.t index c768aebcea..d3f6a3b4cf 100644 --- a/t/015-status.t +++ b/t/015-status.t @@ -9,7 +9,7 @@ log_level('warn'); #repeat_each(120); repeat_each(2); -plan tests => repeat_each() * (blocks() * 2 + 10); +plan tests => repeat_each() * (blocks() * 3 + 2); #no_diff(); #no_long_string(); @@ -29,6 +29,8 @@ __DATA__ GET /nil --- response_body nil +--- no_error_log +[error] @@ -43,6 +45,8 @@ nil GET /nil --- response_body not nil +--- no_error_log +[error] @@ -57,6 +61,8 @@ not nil GET /nil --- response_body 0 +--- no_error_log +[error] @@ -73,6 +79,8 @@ GET /nil --- response_body blah 200 +--- no_error_log +[error] @@ -89,6 +97,8 @@ GET /201 --- response_body created --- error_code: 201 +--- no_error_log +[error] @@ -105,6 +115,8 @@ GET /201 --- response_body created --- error_code: 201 +--- no_error_log +[error] @@ -121,6 +133,8 @@ GET /201 --- response_body created --- error_code: 201 +--- no_error_log +[error] @@ -136,6 +150,8 @@ created GET /201 --- response_body_like: 500 Internal Server Error --- error_code: 500 +--- no_error_log +[crit] @@ -248,6 +264,8 @@ GET /t --- http09 --- response_body status = 9 +--- no_error_log +[error] @@ -345,3 +363,169 @@ GET /t match --- no_error_log [error] + + + +=== TEST 18: set ngx.status in server_rewrite_by_lua_block +don't proxy_pass to upstream +--- config + server_rewrite_by_lua_block { + if ngx.var.uri == "/t" then + ngx.status = 403 + ngx.say("Hello World") + end + } + + location /t { + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/u; + } + + location /u { + content_by_lua_block { + ngx.say("From upstream") + } + } +--- request +GET /t HTTP/1.0 +--- response_body +Hello World +--- error_code: 403 +--- no_error_log +[error] + + + +=== TEST 19: set ngx.status in rewrite_by_lua_block +don't proxy_pass to upstream +--- config + location /t { + rewrite_by_lua_block { + ngx.status = 403 + ngx.say("Hello World") + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/u; + } + + location /u { + content_by_lua_block { + ngx.say("From upstream") + } + } +--- request +GET /t HTTP/1.0 +--- response_body +Hello World +--- error_code: 403 +--- no_error_log +[error] + + + +=== TEST 20: set ngx.status in access_by_lua_block +don't proxy_pass to upstream +--- config + location /t { + access_by_lua_block { + ngx.status = 403 + ngx.say("Hello World") + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/u; + } + + location /u { + content_by_lua_block { + ngx.say("From upstream") + } + } +--- request +GET /t HTTP/1.0 +--- response_body +Hello World +--- error_code: 403 +--- no_error_log +[error] + + + +=== TEST 21: set ngx.status in server_rewrite_by_lua_block with sleep +don't proxy_pass to upstream +--- config + server_rewrite_by_lua_block { + if ngx.var.uri == "/t" then + ngx.sleep(0.001) + ngx.status = 403 + ngx.say("Hello World") + end + } + + location /t { + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/u; + } + + location /u { + content_by_lua_block { + ngx.say("From upstream") + } + } +--- request +GET /t HTTP/1.0 +--- response_body +Hello World +--- error_code: 403 +--- no_error_log +[error] + + + +=== TEST 22: set ngx.status in rewrite_by_lua_block with sleep +don't proxy_pass to upstream +--- config + location /t { + rewrite_by_lua_block { + ngx.sleep(0.001) + ngx.status = 403 + ngx.say("Hello World") + } + + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/u; + } + + location /u { + content_by_lua_block { + ngx.say("From upstream") + } + } +--- request +GET /t HTTP/1.0 +--- response_body +Hello World +--- error_code: 403 +--- no_error_log +[error] + + + +=== TEST 23: set ngx.status in access_by_lua_block +don't proxy_pass to upstream +--- config + location /t { + access_by_lua_block { + ngx.sleep(0.001) + ngx.status = 403 + ngx.say("Hello World") + } + proxy_pass http://127.0.0.1:$TEST_NGINX_SERVER_PORT/u; + } + + location /u { + content_by_lua_block { + ngx.say("From upstream") + } + } +--- request +GET /t HTTP/1.0 +--- response_body +Hello World +--- error_code: 403 +--- no_error_log +[error] From 53588e986bad6a0edec465da1e42f14a918d5500 Mon Sep 17 00:00:00 2001 From: willmafh Date: Sun, 2 Mar 2025 13:34:47 +0800 Subject: [PATCH 804/848] style: add missing space. --- src/ngx_http_lua_accessby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index b8c26605cf..206f32ec1e 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -106,7 +106,7 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) } if (rc == NGX_OK) { - if (r->header_sent || r->headers_out.status!= 0) { + if (r->header_sent || r->headers_out.status != 0) { dd("header already sent"); /* response header was already generated in access_by_lua*, From ee3dd989fda7c6a70d0a61250685a902a3f79f92 Mon Sep 17 00:00:00 2001 From: willmafh Date: Sun, 2 Mar 2025 15:13:05 +0800 Subject: [PATCH 805/848] bugfix: unnecessary to do error check. --- src/ngx_http_lua_socket_tcp.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 998881d1ca..5f2255e7eb 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -128,7 +128,7 @@ static int ngx_http_lua_socket_tcp_conn_op_resume_retval_handler( ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); static int ngx_http_lua_socket_tcp_upstream_destroy(lua_State *L); static int ngx_http_lua_socket_downstream_destroy(lua_State *L); -static ngx_int_t ngx_http_lua_socket_push_input_data(ngx_http_request_t *r, +static void ngx_http_lua_socket_push_input_data(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L); static ngx_int_t ngx_http_lua_socket_add_pending_data(ngx_http_request_t *r, @@ -3153,7 +3153,6 @@ ngx_http_lua_socket_tcp_receive_retval_handler(ngx_http_request_t *r, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) { int n; - ngx_int_t rc; ngx_http_lua_ctx_t *ctx; ngx_event_t *ev; @@ -3200,12 +3199,7 @@ ngx_http_lua_socket_tcp_receive_retval_handler(ngx_http_request_t *r, dd("u->bufs_in: %p", u->bufs_in); if (u->bufs_in) { - rc = ngx_http_lua_socket_push_input_data(r, ctx, u, L); - if (rc == NGX_ERROR) { - lua_pushnil(L); - lua_pushliteral(L, "no memory"); - return 2; - } + ngx_http_lua_socket_push_input_data(r, ctx, u, L); (void) ngx_http_lua_socket_read_error_retval_handler(r, u, L); @@ -3219,12 +3213,7 @@ ngx_http_lua_socket_tcp_receive_retval_handler(ngx_http_request_t *r, return n + 1; } - rc = ngx_http_lua_socket_push_input_data(r, ctx, u, L); - if (rc == NGX_ERROR) { - lua_pushnil(L); - lua_pushliteral(L, "no memory"); - return 2; - } + ngx_http_lua_socket_push_input_data(r, ctx, u, L); return 1; } @@ -5908,7 +5897,7 @@ ngx_http_lua_socket_downstream_destroy(lua_State *L) } -static ngx_int_t +static void ngx_http_lua_socket_push_input_data(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx, ngx_http_lua_socket_tcp_upstream_t *u, lua_State *L) @@ -5980,8 +5969,6 @@ ngx_http_lua_socket_push_input_data(ngx_http_request_t *r, u->buf_in->buf->last = u->buffer.pos; u->buf_in->buf->pos = u->buffer.pos; } - - return NGX_OK; } From cb0b22d3868bb3e060aa23bd640592eaf09ad016 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 4 Mar 2025 10:03:22 +0800 Subject: [PATCH 806/848] tests: update SSL certificates. --- t/129-ssl-socket.t | 2 +- t/cert/gen-test-passphrase.sh | 23 +++++++++++++++++ t/cert/gen-test2.sh | 3 +++ t/cert/test2.crt | 36 ++++++++++++++++---------- t/cert/test2.key | 43 ++++++++++++++++++++----------- t/cert/test_passphrase.crt | 37 ++++++++++++++++----------- t/cert/test_passphrase.key | 48 ++++++++++++++++++++++------------- 7 files changed, 129 insertions(+), 63 deletions(-) create mode 100755 t/cert/gen-test-passphrase.sh create mode 100755 t/cert/gen-test2.sh diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 611e72ab2f..8a05306ab0 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -1963,7 +1963,7 @@ $::TestCertificate" --- grep_error_log eval: qr/lua ssl (?:set|save|free) session: [0-9A-F]+/ --- grep_error_log_out --- error_log eval -qr/SSL_do_handshake\(\) failed .*?(unknown protocol|wrong version number)/ +qr/SSL_do_handshake\(\) failed .*?(unknown protocol|wrong version number|routines::record layer failure)/ --- no_error_log lua ssl server name: SSL reused session diff --git a/t/cert/gen-test-passphrase.sh b/t/cert/gen-test-passphrase.sh new file mode 100755 index 0000000000..45e6806a39 --- /dev/null +++ b/t/cert/gen-test-passphrase.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# Set variables +SUBJECT="/C=CN/ST=Guangdong/L=ShenZhen/O=OpenResty/OU=OpenResty/CN=test.com/emailAddress=guanglinlv@gmail.com" +DAYS=49000 # Approximately 134 years +KEY_FILE="test_passphrase.key" +CERT_FILE="test_passphrase.crt" +PASSWORD="123456" + +# Generate a new 2048-bit RSA private key, encrypted with the password +openssl genrsa -aes256 -passout pass:$PASSWORD -out $KEY_FILE 2048 + +# Generate a new self-signed certificate +openssl req -x509 -new -nodes -key $KEY_FILE -sha256 -days $DAYS \ + -out $CERT_FILE -subj "$SUBJECT" -passin pass:$PASSWORD + +# Display information about the new certificate +openssl x509 -in $CERT_FILE -text -noout + +echo "New 2048-bit certificate generated successfully!" +echo "Private key (encrypted): $KEY_FILE" +echo "Certificate: $CERT_FILE" + diff --git a/t/cert/gen-test2.sh b/t/cert/gen-test2.sh new file mode 100755 index 0000000000..ed7a9de398 --- /dev/null +++ b/t/cert/gen-test2.sh @@ -0,0 +1,3 @@ +openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 \ + -subj "/C=US/ST=California/L=San Francisco/O=OpenResty/CN=test2.com/emailAddress=openresty@gmail.com" \ + -keyout test2.key -out test2.crt diff --git a/t/cert/test2.crt b/t/cert/test2.crt index edc3b0df3b..6ba864c3fe 100644 --- a/t/cert/test2.crt +++ b/t/cert/test2.crt @@ -1,16 +1,24 @@ -----BEGIN CERTIFICATE----- -MIIChzCCAfACCQDjCkJpJUtZmjANBgkqhkiG9w0BAQUFADCBhjELMAkGA1UEBhMC -VVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28x -EjAQBgNVBAoMCU9wZW5SZXN0eTESMBAGA1UEAwwJdGVzdDIuY29tMSIwIAYJKoZI -hvcNAQkBFhNvcGVucmVzdHlAZ21haWwuY29tMCAXDTE0MDkxMzAwMTgxMFoYDzIx -MTQwODIwMDAxODEwWjCBhjELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3Ju -aWExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28xEjAQBgNVBAoMCU9wZW5SZXN0eTES -MBAGA1UEAwwJdGVzdDIuY29tMSIwIAYJKoZIhvcNAQkBFhNvcGVucmVzdHlAZ21h -aWwuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDy+OVI2u5NBOeB2Cyz -Gnwy9b7Ao4CSi05XtUxh2IoVdzYZz6c4PFb9C1ad52LDdRStiQT5A7+RKLj6Kr7f -JrKFziJxMy4g4Kdn9G659vE7CWu/UAVjRUtc+mTBAEfjdbumizmHLG7DmnNhGl3R -NGiVNLsUInSMGfUlJRzZJXhI4QIDAQABMA0GCSqGSIb3DQEBBQUAA4GBAEMmRvyN -N7uE24Tc6TR19JadNHK8g3YGktRoXWiqd/y0HY4NRPgvnK/nX7CY/wXa1j+uDO8K -e6/Ldm5RZrjtvfHJmTSAu8zkqTJz8bqRDH7kzL5Ni2Ky2x8r9dtB0ImpOiSlwvZN -snMvbrxEdwBiqlC9prV2f9aG+ACo1KnPL0j6 +MIID7zCCAtegAwIBAgIUS1f/CoGJaRaJvBgrXEXeTSm7kM4wDQYJKoZIhvcNAQEL +BQAwgYYxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQH +DA1TYW4gRnJhbmNpc2NvMRIwEAYDVQQKDAlPcGVuUmVzdHkxEjAQBgNVBAMMCXRl +c3QyLmNvbTEiMCAGCSqGSIb3DQEJARYTb3BlbnJlc3R5QGdtYWlsLmNvbTAeFw0y +NDAzMjQxMDI1NDhaFw0zNDAzMjIxMDI1NDhaMIGGMQswCQYDVQQGEwJVUzETMBEG +A1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzESMBAGA1UE +CgwJT3BlblJlc3R5MRIwEAYDVQQDDAl0ZXN0Mi5jb20xIjAgBgkqhkiG9w0BCQEW +E29wZW5yZXN0eUBnbWFpbC5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK +AoIBAQChXBMRP5X9Y0l5NogyIZatou1YDT2BDE9tza6dYKUdraJlGo1T4DZf7LoD +mKFvIpVqLedppZ9z9SrlrsZ8Be6OUbw3kxSyUPDvpauI4XXBR2ZuAHIzTy/Uare1 +1iZSjbT0KATiWviMWCWWblyBU5QPMFJJOmcjRDAZocfIUs4CzglRaDorl5qUFJHr +mROaIsag/10nnClTwaGwj1pHmFitKRHTIrYICYd5wGgstte9mDFz15s9a3f/Dwhc +/7neEHYaryRVhR627aDGVHHpcMRWr+FC8QmZTTkiFmO80eeaiyGGTS0AEW51EPI9 +Nr142PEaQETpjQaQnfNhZGnwTZHTAgMBAAGjUzBRMB0GA1UdDgQWBBQb73HlJHF7 +B4a1PWRkb3aovk7EtDAfBgNVHSMEGDAWgBQb73HlJHF7B4a1PWRkb3aovk7EtDAP +BgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCQJcLh7+NzAukTG4hV +nVd572U4sCnlXwTN4lnRIkSGbQxOWwbuI7p05iAT9HJrFLm+rV0EOzxyl/fYkPgR +DKYfj7sGPp4Lo0NyQ3xhoVypqDDmv7Wm9CLy2mVTJ07khW5tV8GHg9mFwgb/XBTQ +c+RnNalIapqdlkQbxms5HLzKzTVRuDt+wvqcAsiKFwKDAL9DFrp36TP1t023w+vd +Vb6Ms+l+xFFcxMCxFEOkSPc7szwSce9Q+Hk2C6JdCU30gJPyEBUMl6vD+0l0IFG0 +ZEw1BhBp4d92xHiYgdnDEH1RdLPI0q1r7LJL+5ic+VFWycgbC0KRHCarduYgvMYy +GbRG -----END CERTIFICATE----- diff --git a/t/cert/test2.key b/t/cert/test2.key index 82ce6ce91d..3bd546da0e 100644 --- a/t/cert/test2.key +++ b/t/cert/test2.key @@ -1,15 +1,28 @@ ------BEGIN RSA PRIVATE KEY----- -MIICXAIBAAKBgQDy+OVI2u5NBOeB2CyzGnwy9b7Ao4CSi05XtUxh2IoVdzYZz6c4 -PFb9C1ad52LDdRStiQT5A7+RKLj6Kr7fJrKFziJxMy4g4Kdn9G659vE7CWu/UAVj -RUtc+mTBAEfjdbumizmHLG7DmnNhGl3RNGiVNLsUInSMGfUlJRzZJXhI4QIDAQAB -AoGAEqBB83PVENJvbOTFiHVfUAjGtr3R/Wnwd4jOcjHHZB3fZ9sjVoxJntxfp3s1 -dwZir2rxlqVS6i3VAFiGiVTOGo2Vvzhw2J7f58twCECmnLb2f863AkGEYe4dAndD -GHGD0WI0CBMD1sT18YCj561o0Wol5deWH0gM9pr2N3HkeIECQQD6hUKFlFhrpaHP -WNJsl6BxgE6pB5kxLcMcpIQ7P+kHUvtyvCJl5QZJqPrpPGjRsAI5Ph92rpsp/zDp -/IZNWGVjAkEA+Ele31Rt+XbV32MrLKZgBDBk+Pzss5LTn9fZ5v1k/7hrMk2VVWvk -AD6n5QiGe/g59woANpPb1T9l956SBf0d6wJABTXOS17pc9uvANP1FGMW6CVl/Wf2 -DKrJ+weE5IKQwyE7r4gwIvRfbBrClSU3fNzvPueG2f4JphbzmnoxBNzIxwJAYivY -mGNwzHehXx99/byXMHDWK+EN0n8WsBgP75Z3rekEcbJdfpYXY8Via1vwmOnwOW65 -4NqbzHix37PSNw37GwJBALxaGNpREO2Tk+oWOvsD2QyviMVae3mXAJHc6nLVdKDM -q0YvDT6VdeNYYFTkAuzJacsVXOpn6AnUMFj0OBedMhc= ------END RSA PRIVATE KEY----- +-----BEGIN PRIVATE KEY----- +MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQChXBMRP5X9Y0l5 +NogyIZatou1YDT2BDE9tza6dYKUdraJlGo1T4DZf7LoDmKFvIpVqLedppZ9z9Srl +rsZ8Be6OUbw3kxSyUPDvpauI4XXBR2ZuAHIzTy/Uare11iZSjbT0KATiWviMWCWW +blyBU5QPMFJJOmcjRDAZocfIUs4CzglRaDorl5qUFJHrmROaIsag/10nnClTwaGw +j1pHmFitKRHTIrYICYd5wGgstte9mDFz15s9a3f/Dwhc/7neEHYaryRVhR627aDG +VHHpcMRWr+FC8QmZTTkiFmO80eeaiyGGTS0AEW51EPI9Nr142PEaQETpjQaQnfNh +ZGnwTZHTAgMBAAECggEAAhemuvqx8A25YQQsgvM9jXXckbTmPxtQ7QcVmb6BlZ9v +834DJtB0VA0b97pNyZpPXrJGsG+F4E6QZlcRsCpcMWXs72Q4Xfw/tj6AjsSsWWZw +H8K2Dbqb+lYB7i6J463TvWDGzhqeOM+seRq9l1hlRfHQogN1spbf8CJ8Fo8VqHf3 +OyId5db0nRUFYV9ciPvJSnk8xi+0aJ2LWVAkCaXQkntbI3529x8KPQm9bGUjdMgo +A+HXAKZTZL7GtGZoSU4jjyymGLM4hPtzXBExbkX7s7HOP1AeWGom4reOC6XNScQq +ysfs0wVzJAVSl0vFbQfUT6GZ2ZsaTdkT+JgXrYwp5QKBgQDIzY1O/Hj2koVgfKmS +2+1JVehX8+yjgz3rSNVUauISLoMaYN4hipsKb7Vi6aDFl+Db296Qql7kIzzd/nFk +UwdBjShUi/xbewGcrYEXbdcXSIxt9ius76eJ9INfa4EmLrZoiX/3Am1u+CcdtcKm +jkqy1o9zoIPbD6lc242zGFK2nQKBgQDNtukEZop/PngCX0yDGWLxisrvFYSrHxnW +x9/Fw0B56qmyG2FpFg5WDlErenHXh+KSlQ3pV1sWK/+3pVzgGP2ZK+nv3ThEQJmg +IbI32dABmUG9+HckxgOuAzfqGW6Y4m5I8YpODjn26V6GeS+xQj0bolr4lZ00QSEG +R5BxxLLHLwKBgQDAzYzWwFgs+HaUNF5lokFt3syeVplqTsOPPHmI+q9iocJD+6qN +Lha7qJLTDFvQHYpL0AsdgFhoWOVCieK6X3ZiHHUS6O4/sBXWKEoBAvg5ZPFhS7KX +8+w768iQQBrbXJLMQOnbdDs7B44XWsQxRAK5QIawkPjJx3norO6bfck9iQKBgA05 +Ze9ffCXD6UkVwUBoQGEQsA0AkB/EBxA0lyEKdTmhKTmG4xMzVMaZcwRdgbX+SUVt +CZDnibZ6K50tpzPMx3iyRv6hdP2GPZn6sI9AlEuWA5DnyRj3FcN49344FlEDtV1B +EcgEyBskU2xHnBZEENOW80wpqgpy6WGS9ikqIOgHAoGAeAUetHWIdZZ+Qt6SS1s2 +R17YAiZe88QFAo3uK/dyApLgY7wvbM0n7Cor/3tbpzpzAujUb/qNqTdTXsZIeRfq +bI767g4/pNl7+jhDUKmDI001YdfHvcRSwjzyjshuSinJY9rydXLUIEWtkTG3f9v8 +d+yLXD2Dz9gF6l2Msj+C3To= +-----END PRIVATE KEY----- diff --git a/t/cert/test_passphrase.crt b/t/cert/test_passphrase.crt index f4d516ba9e..78d57577b0 100644 --- a/t/cert/test_passphrase.crt +++ b/t/cert/test_passphrase.crt @@ -1,17 +1,24 @@ -----BEGIN CERTIFICATE----- -MIICozCCAgwCCQDEutRdSs3vZjANBgkqhkiG9w0BAQUFADCBlDELMAkGA1UEBhMC -Q04xEjAQBgNVBAgMCUd1YW5nZG9uZzERMA8GA1UEBwwIU2hlblpoZW4xEjAQBgNV -BAoMCU9wZW5SZXN0eTESMBAGA1UECwwJT3BlblJlc3R5MREwDwYDVQQDDAh0ZXN0 -LmNvbTEjMCEGCSqGSIb3DQEJARYUZ3VhbmdsaW5sdkBnbWFpbC5jb20wIBcNMTYw -NDI4MTQ0MzI4WhgPMjE1MTAzMjcxNDQzMjhaMIGUMQswCQYDVQQGEwJDTjESMBAG -A1UECAwJR3Vhbmdkb25nMREwDwYDVQQHDAhTaGVuWmhlbjESMBAGA1UECgwJT3Bl -blJlc3R5MRIwEAYDVQQLDAlPcGVuUmVzdHkxETAPBgNVBAMMCHRlc3QuY29tMSMw -IQYJKoZIhvcNAQkBFhRndWFuZ2xpbmx2QGdtYWlsLmNvbTCBnzANBgkqhkiG9w0B -AQEFAAOBjQAwgYkCgYEA2KZ+HdH9R2tarxD8PKqu5EYq2BNGlFRg1xJmrw0XZBRM -UP/VPb+sIeioooz36uhiXfQjExlpBCA/0zNAN+HbFyqpPPTf1qLGrj/dqeE4MJaN -Bwzxiv3fZnENT65u2qbiFWIY+ATNHgA20d50nxNNjPTzLbkx/nYXL92r4kuAGk0C -AwEAATANBgkqhkiG9w0BAQUFAAOBgQCfMo0qbcs3kwl1tcNBO5hCcUUJRzyv041V -ff/nZ/JPIMo/LSZd12K82G/dLRN7uRT9nzqtm+JRkHALHWWWFKi6bdg1vcdOTWqC -08bCkJHQoXJQQLvvA6gNvnR+0b7L4CrCmrcyYgKDLXVGNP9Wv/PqSWWbxsmqngkA -Mvy6CVytFw== +MIIEDTCCAvWgAwIBAgIUZn0PL6eNqoKWughHd0SFO4aE/UswDQYJKoZIhvcNAQEL +BQAwgZQxCzAJBgNVBAYTAkNOMRIwEAYDVQQIDAlHdWFuZ2RvbmcxETAPBgNVBAcM +CFNoZW5aaGVuMRIwEAYDVQQKDAlPcGVuUmVzdHkxEjAQBgNVBAsMCU9wZW5SZXN0 +eTERMA8GA1UEAwwIdGVzdC5jb20xIzAhBgkqhkiG9w0BCQEWFGd1YW5nbGlubHZA +Z21haWwuY29tMCAXDTI1MDMwNDAxNDkwNFoYDzIxNTkwNTAxMDE0OTA0WjCBlDEL +MAkGA1UEBhMCQ04xEjAQBgNVBAgMCUd1YW5nZG9uZzERMA8GA1UEBwwIU2hlblpo +ZW4xEjAQBgNVBAoMCU9wZW5SZXN0eTESMBAGA1UECwwJT3BlblJlc3R5MREwDwYD +VQQDDAh0ZXN0LmNvbTEjMCEGCSqGSIb3DQEJARYUZ3VhbmdsaW5sdkBnbWFpbC5j +b20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCmsvo9t9/lhfavK6S8 +7lqP8lfpV1gZE4ahyRCQI0vgQitlhGjEzvBc7qTtkunQpSWkuHSczOR1WuKzO+PG +ubH0gjeyvEE2X/cipQ1kjiO9Mv8w2qOm9BkC9Wrob01X4mOhGZUgthfwRs0QFSUh +tzvjg+aApWyIlpmgE/T8bnCwtPS7BuzwEpTt5a51q8JKOr4AMTy64mrMFQRqtcIk +GYL9CnoxAjYT6NAcEqoapSLDm1xJ7w2RDL1mqkAdXYUQeNPwMivrtb+rCDD4p99A +TZ8rOuKQzDSxZ3BdCmR5WUXp9z+NeQE5SBu+vkOYMTvEN+e9ApRRnZTO8UHeRB0+ +6vJZAgMBAAGjUzBRMB0GA1UdDgQWBBSmnXGC+KOqNzyGXoujr3TMe3pepDAfBgNV +HSMEGDAWgBSmnXGC+KOqNzyGXoujr3TMe3pepDAPBgNVHRMBAf8EBTADAQH/MA0G +CSqGSIb3DQEBCwUAA4IBAQAokEqnbJNDUICvbsRQBFrDBZXjZaEPu69GrTg9OceB +N9Qu4CKRTL4WsV6uSbdj+Ox/r21UtCI1590oBpC3RiJ+WnGmpdUGVytGIODt/vJ7 +/R03EBy0o9e7XdlR4GfKMC7FVBsP/kkeacR2fswHYXtQrNlyuLhZW04HTDdYNNqb +LNCnsmwQ4WR4ek9PdcpeFfMY1Up6hTYZ441K8b6xSzMV0/LjzfmHXWoT3U1rQrfM +esm9t/R3sOvB/wBWQaRe0BbLYpkS7pcq8NQsWvSUFNl5TzrGtUf8KTyvEEdcUcZf +im48Ea7WYpE3RYs0laLBk/D3sz7vsK5wfiD0R/ASLE+s -----END CERTIFICATE----- diff --git a/t/cert/test_passphrase.key b/t/cert/test_passphrase.key index b8fc97634a..2cabedfb57 100644 --- a/t/cert/test_passphrase.key +++ b/t/cert/test_passphrase.key @@ -1,18 +1,30 @@ ------BEGIN RSA PRIVATE KEY----- -Proc-Type: 4,ENCRYPTED -DEK-Info: DES-EDE3-CBC,679ACC8E69ACAA92 - -Ssrjp3VU4somCNPiXkWqcudDnvnwbyj/Q0pS07at3lXKbhQSgI1Tzhg9Pm3BXXj5 -mkLdeGG5ocrj1Q9dhtmZgZeHHQIiynZBhjBu1Y+HPef8jXOWLrCOi8EKiWkJ2qG3 -V1KFM/95CcDt0mRLykUXEL3IpUst05SFb9XwiLokB7ypeu3NhgNUHjL6G+ubB4ri -TOUjCW4pEoNHjdC22IiqSncwCVhluYSGhr6ktHKehZMhYIXmL1wmSLdhTlsPXCQl -xvYILQ2vJcKIR1BkeYYPD/OQC6zCZlXIErzfgeZiz2+NTudKYpb9VmsQKsO+R8L7 -tZ/fNaR0vk8bbimMHgStAV4acVsC/7WxsqOjMJ8VTq1iqhYPl6N7kRdR3H3kSSOm -cN9T3SrOHDVaHbnWgToaOE4mKFjvFSLIOcWgus0iOHWXmY+SLG+Ndag3oVB6R9oB -cAHX19mq99+GhzA8IV4I0En2UCKQhnGPvkM+9mcCDxhRETlwncDjlMGOHpQ65J9r -eReVPIpnDkvHxPGTtsR3ZHTdWTZb+C0W2N3QIlJKrOzxFmfoj++yG3tMX42aDY0g -DVkrXgcKobiWN0AVrJNAwfG7uObKSCFYgz/0RRMCO4cjXRW99nxdjVDZhyc6R0Te -jzuF04okkOLNb25n2hP+yIULrn+6Nv/uHtFI0j0n3hOzcKh//dNbACSAKgkHni9g -JKDFJXgLJxf+Wc3So0DF9gYMKJJ+WbcdVT9gkC7RyQHlC90Pn7kNXzHr0ZawUsNI -ZxSkL4dMhYAfA4lUBJbOkwbSurv97LinOSRffpM0Nmf7VNw/Ue15eg== ------END RSA PRIVATE KEY----- +-----BEGIN ENCRYPTED PRIVATE KEY----- +MIIFNTBfBgkqhkiG9w0BBQ0wUjAxBgkqhkiG9w0BBQwwJAQQ2v8WxwirSc2M7jaZ +mrWsPgICCAAwDAYIKoZIhvcNAgkFADAdBglghkgBZQMEASoEEAlDrk7LLHI/jJqN +QJ6z8MYEggTQtV86xHehGFn/Qkn5qSZicJ/sDLXQ+m4dFXgw9KMnm0zLG8kzEmb7 +sobUJ4CnwmpBv/GmpPxgy5v1fuvc8HXRJWr7xwv1FoD4lwVOWVFv/qTaII4TfcC6 +3vKqO7MbRaksyI/NmmepGKTGhbzW6XlEdE1TCG7tO02I3puSN0MbJACTpop53TG9 +g+b8HXegPXkLJ2p+ECjUqPwA+h6jZi2pIAztMHQpIizokomrLJ4+2ol3czJpA5nn +n8JbuYt1XWekq+PBi4l0+lAbk/RiLLK1PCN/Njp7iBBAtNtz9+yJd2Ev7wNDDxKk +dW73s8uTGde/n6hR4sl/akK8HLaw9tDcWsaHbIMl8zn3WZXksVUfb6CZVOU+BjBa +yZu3CnP1JTda16HgdZ5dGSNPNbrUeLLIRzM/OxK0yTtmkexOVIgSW0hGE1BXxFXk +v+mnzrjFrHNRQFhoCFciPqv+WfXsecSASeN0aLWP/RP6CLz6QXWQaK68Dn92v/WP +xc1NKk8l54wNCLqGaSXadUoTqEdHQH3GV5EkXP7MTb+fYi9/Y/S8wcXJtjUi3ABe +W+XCOh4zuuQPG7C3i6l/jLIL1ogQNYldad7MoM8eBU8u+TF+NqiJjTLCB1Quycqn +sN5oVS1VZfsx8XCFsvUAZgxzhjeXViBfwxKSKyT0b1drQZaY58LnyzjmzGkOpRfO +uzGzWPvHRxkGb9071XaRxD8YDLIngHCw3F2USto88efzmvBfB8ZIxk90S9TZZDeJ +rqIKpNXsfM4nCR07DW48n3IoiHLCtRFinfpboS5/47ZN4XBE7Q/QNJrWa2P07aWR +HdOBdD/N3DnM5bbSz7m+5TA+xC4/hEWLXt7Au+vnnhVyDdU80a3+3JY7y4vScHLS +Fn2grTk5F9pqieNhwawpXUCYNGFtJOY8LOQpUZdMpy6n6N5qzSjZ+vgE1Nwr0G/S ++n8ORUcJzF3s8h+u+SN8OC9W6chnftSIWKiqGsU4oM0FFkr2nOrfNNKqAQo2OU3W ++5lJovyqAcxtj+4qQsBrYzH7vRzdkigXhb/R7tRDnBRWBpWlMmnUhkaWG8EI0JLs +gjKNOUFCT1HgTSfonRxScDE5IAg3JxieNz9ppDCphvc0ii3wwdS7uwstKGZCC+Wr +lQsiV0Wa4WlWVMfLCONI13+zeOli4BmwepgkjJc+YogvLodblcHN14kTTCDvhDMp +KZglwFT+3zkxH9ArnO7/a8WQCWZV6bJ29HFUfAioE8Sie04WD4zgYEfU02r9A+vq +wl/w5UK6eXOO6Hz2OrfYh2dZXiCOLp+DhftKq6MKIIAcqFuAzrPvzxd7Uup/fxyX +n2Ub4YKmUV89RzJ2ipZRjiBcQo+9h4O9dIh4+2I0m7NPALqirutCFBe2cHu/Fwho +SzacLI21OzWLKbyZ2D6ueH4NrhnOZ10bo/lKbc5d3JzuGv6KW1NmGo32qiKeKNzG +Q52+sm0/TLiLxy2RlqJ1p7R9NLLZTY8bZTD125n2RtNaJWT564vseY8SaiUW62r8 +ksU5aPINa7Qkf4xgsmkSJymFs/9AjXfSCS/aZ13PQMUuI5A4fY9IZjSoiUlSx1bX ++2B91azW4yfHcAJrluIjZ13LvBU08ooPKPm2TazDi89/8GjBMpHUQN4= +-----END ENCRYPTED PRIVATE KEY----- From 8f3987f6bf48c8788b5c88c676086849d59342f9 Mon Sep 17 00:00:00 2001 From: kurt Date: Fri, 7 Mar 2025 19:24:50 +0800 Subject: [PATCH 807/848] feature: add function to bypass HTTP conditional request checks (#2401) I hereby granted the copyright of the changes in this pull request to the authors of this lua-nginx-module project. releated: https://github.com/openresty/lua-nginx-module/issues/2397 refer: https://github.com/nginx/nginx/commit/d9887ee2ae9069843eed67d5b5ea625a7faeedb1 Signed-off-by: tzssangglass * refactor: modify bypass_if_checks function to return void Signed-off-by: tzssangglass --------- Signed-off-by: tzssangglass --- src/ngx_http_lua_util.c | 6 ++++++ t/014-bugs.t | 30 ++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index c73bddc609..61f56cb7c0 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4555,4 +4555,10 @@ ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len) } +void +ngx_http_lua_ffi_bypass_if_checks(ngx_http_request_t *r) +{ + r->disable_not_modified = 1; +} + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/t/014-bugs.t b/t/014-bugs.t index 303187929f..9d57afe134 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -1366,3 +1366,33 @@ If-Match: 1 --- error_code: 412 --- response_body eval qr/\Ahello\z/ + + + +=== TEST 50: nginx crashes when encountering an illegal http if header +crash with ngx.print() +--- main_config +--- config +error_page 412 /my_error_handler_412; + +location /t { + access_by_lua_block { + local ngx_resp = require "ngx.resp" + ngx_resp.bypass_if_checks() + ngx.print("hello") + ngx.exit(200) + } +} +location = /my_error_handler_412 { + content_by_lua_block { + ngx.sleep(0.002) + ngx.header["Content-Type"] = "text/plain" + } +} +--- request + GET /t +--- more_headers +If-Match: 1 +--- error_code: 200 +--- response_body eval +qr/\Ahello\z/ From 18ce5fbd58171a5faec966a5f7099ce930c812c2 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 21 Mar 2025 12:34:43 +0800 Subject: [PATCH 808/848] bugfix: didn't use right hostname when the length of hostname is greater than 32. --- src/ngx_http_lua_balancer.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_balancer.c b/src/ngx_http_lua_balancer.c index ae0f1380b5..aa26a4a07a 100644 --- a/src/ngx_http_lua_balancer.c +++ b/src/ngx_http_lua_balancer.c @@ -702,7 +702,7 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, item->host.len = host->len; } else { - item->host.data = ngx_pstrdup(c->pool, bp->addr_text); + item->host.data = ngx_pstrdup(c->pool, host); if (item->host.data == NULL) { ngx_http_lua_balancer_close(c); @@ -713,7 +713,7 @@ ngx_http_lua_balancer_free_peer(ngx_peer_connection_t *pc, void *data, return; } - item->host.len = bp->addr_text->len; + item->host.len = host->len; } } else { From bdba93c3543a083f737c20fc067f4ad539d1942a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 24 Mar 2025 23:37:16 +0800 Subject: [PATCH 809/848] bugfix: the modifications in this PR are to supplement the overlooked changes in the commit e8f65dc53. --- src/ngx_http_lua_accessby.c | 7 +++++-- src/ngx_http_lua_rewriteby.c | 8 ++++++-- src/ngx_http_lua_server_rewriteby.c | 8 ++++++-- 3 files changed, 17 insertions(+), 6 deletions(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 206f32ec1e..31ee82fd93 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -106,7 +106,9 @@ ngx_http_lua_access_handler(ngx_http_request_t *r) } if (rc == NGX_OK) { - if (r->header_sent || r->headers_out.status != 0) { + if (r->header_sent + || (r->headers_out.status != 0 && ctx->out != NULL)) + { dd("header already sent"); /* response header was already generated in access_by_lua*, @@ -370,7 +372,8 @@ ngx_http_lua_access_by_chunk(lua_State *L, ngx_http_request_t *r) #if 1 if (rc == NGX_OK) { - if (r->header_sent || r->headers_out.status != 0) { + if (r->header_sent || (r->headers_out.status != 0 && ctx->out != NULL)) + { dd("header already sent"); /* response header was already generated in access_by_lua*, diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 091d765e41..2c1cc04c7a 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -107,7 +107,9 @@ ngx_http_lua_rewrite_handler(ngx_http_request_t *r) } if (rc == NGX_DECLINED) { - if (r->header_sent || r->headers_out.status != 0) { + if (r->header_sent + || (r->headers_out.status != 0 && ctx->out != NULL)) + { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, @@ -349,7 +351,9 @@ ngx_http_lua_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) } if (rc == NGX_OK || rc == NGX_DECLINED) { - if (r->header_sent || r->headers_out.status != 0) { + if (r->header_sent + || (r->headers_out.status != 0 && ctx->out != NULL)) + { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, diff --git a/src/ngx_http_lua_server_rewriteby.c b/src/ngx_http_lua_server_rewriteby.c index ed37f03efa..d012a8e341 100644 --- a/src/ngx_http_lua_server_rewriteby.c +++ b/src/ngx_http_lua_server_rewriteby.c @@ -69,7 +69,9 @@ ngx_http_lua_server_rewrite_handler(ngx_http_request_t *r) } if (rc == NGX_DECLINED) { - if (r->header_sent || r->headers_out.status != 0) { + if (r->header_sent + || (r->headers_out.status != 0 && ctx->out != NULL)) + { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, @@ -307,7 +309,9 @@ ngx_http_lua_server_rewrite_by_chunk(lua_State *L, ngx_http_request_t *r) } if (rc == NGX_OK || rc == NGX_DECLINED) { - if (r->header_sent || r->headers_out.status != 0) { + if (r->header_sent + || (r->headers_out.status != 0 && ctx->out != NULL)) + { dd("header already sent"); /* response header was already generated in rewrite_by_lua*, From 0547c254f4166aa0c8df569020046b7c99eed587 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 4 May 2025 11:41:37 +0800 Subject: [PATCH 810/848] feature: add sock:getfd(). --- README.markdown | 16 +++++++ src/ngx_http_lua_socket_tcp.c | 38 +++++++++++++++ src/ngx_http_lua_socket_udp.c | 2 +- t/058-tcp-socket.t | 76 ++++++++++++++++++++++++++++++ t/062-count.t | 6 +-- t/067-req-socket.t | 28 +++++++++++ t/116-raw-req-socket.t | 88 ++++++++++++++++++++++++++++++++++- 7 files changed, 249 insertions(+), 5 deletions(-) diff --git a/README.markdown b/README.markdown index 35c0b83e58..fa8f2c92ba 100644 --- a/README.markdown +++ b/README.markdown @@ -3751,6 +3751,7 @@ Nginx API for Lua * [ngx.socket.tcp](#ngxsockettcp) * [tcpsock:bind](#tcpsockbind) * [tcpsock:connect](#tcpsockconnect) +* [tcpsock:getfd](#getfd) * [tcpsock:setclientcert](#tcpsocksetclientcert) * [tcpsock:sslhandshake](#tcpsocksslhandshake) * [tcpsock:send](#tcpsocksend) @@ -8010,6 +8011,21 @@ This method was first introduced in the `v0.5.0rc1` release. [Back to TOC](#nginx-api-for-lua) + +tcpsock:getfd +-------------------- + +**syntax:** *fd, err = tcpsock:getfd()* + +**context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** + +Get the file describer of the current tcp socket. + +This method was first introduced in the `v0.10.29` release. + +[Back to TOC](#nginx-api-for-lua) + + tcpsock:setclientcert --------------------- diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 5f2255e7eb..b943c8ece4 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -227,6 +227,10 @@ static char ngx_http_lua_pattern_udata_metatable_key; #define ngx_http_lua_tcp_socket_metatable_literal_key "__tcp_cosocket_mt" +#define ngx_http_lua_tcp_req_socket_metatable_literal_key \ + "__tcp_req_cosocket_mt" +#define ngx_http_lua_tcp_raw_req_socket_metatable_literal_key \ + "__tcp_raw_req_cosocket_mt" void @@ -283,6 +287,12 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_pushvalue(L, -1); lua_setfield(L, -2, "__index"); + lua_rawset(L, LUA_REGISTRYINDEX); + + lua_pushliteral(L, ngx_http_lua_tcp_req_socket_metatable_literal_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + req_socket_metatable_key)); + lua_rawget(L, LUA_REGISTRYINDEX); lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ @@ -312,6 +322,12 @@ ngx_http_lua_inject_socket_tcp_api(ngx_log_t *log, lua_State *L) lua_pushvalue(L, -1); lua_setfield(L, -2, "__index"); + lua_rawset(L, LUA_REGISTRYINDEX); + + lua_pushliteral(L, ngx_http_lua_tcp_raw_req_socket_metatable_literal_key); + lua_pushlightuserdata(L, ngx_http_lua_lightudata_mask( + raw_req_socket_metatable_key)); + lua_rawget(L, LUA_REGISTRYINDEX); lua_rawset(L, LUA_REGISTRYINDEX); /* }}} */ @@ -6684,6 +6700,28 @@ ngx_http_lua_ffi_socket_tcp_setoption(ngx_http_lua_socket_tcp_upstream_t *u, } +int +ngx_http_lua_ffi_socket_tcp_getfd(ngx_http_request_t *r, + ngx_http_lua_socket_tcp_upstream_t *u, const char **errmsg) +{ + int fd; + + *errmsg = NULL; + + if (u == NULL || u->peer.connection == NULL) { + *errmsg = "closed"; + return -1; + } + + fd = u->peer.connection->fd; + if (fd == -1) { + *errmsg = "faked connection"; + } + + return fd; +} + + /* just hack the fd for testing bad case, it will also return the original fd */ int ngx_http_lua_ffi_socket_tcp_hack_fd(ngx_http_lua_socket_tcp_upstream_t *u, diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index e2ba790fc5..3297cbbeca 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -1207,7 +1207,7 @@ ngx_http_lua_socket_udp_read(ngx_http_request_t *r, ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "lua udp socket read data: waiting: %d", (int) u->waiting); - n = ngx_udp_recv(u->udp_connection.connection, + n = ngx_udp_recv(c, ngx_http_lua_socket_udp_buffer, u->recv_buf_size); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index db5cb60e84..e17d038038 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -4563,3 +4563,79 @@ connected: 1 setkeepalive: 1 --- no_error_log [error] + + + +=== TEST 75: getfd() +--- no_http2 +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; +--- config + server_tokens off; + location /t { + #set $port 5000; + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local s, err = sock:getfd() + ngx.say("fd: ", s) + ngx.say("connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: localhost\r\nConnection: close\r\n\r\n" + -- req = "OK" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + ngx.say("request sent: ", bytes) + + while true do + local line, err, part = sock:receive() + if line then + ngx.say("received: ", line) + + else + ngx.say("failed to receive a line: ", err, " [", part, "]") + break + end + end + + ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + } + } + + location /foo { + content_by_lua 'ngx.say("foo")'; + more_clear_headers Date; + } + +--- request +GET /t +--- response_body eval +qr/fd: \d+ +connected: 1 +request sent: 57 +received: HTTP\/1.1 200 OK +received: Server: nginx +received: Content-Type: text\/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +failed to receive a line: closed \[\] +close: 1 nil +/ms +--- no_error_log +[error] diff --git a/t/062-count.t b/t/062-count.t index d0eabb1f2d..3fda3b9547 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -259,7 +259,7 @@ n = 10 POST /test hello world --- response_body -n = 6 +n = 7 --- no_error_log [error] --- skip_eval: 3: $ENV{TEST_NGINX_USE_HTTP3} @@ -460,7 +460,7 @@ worker: 5 --- request GET /test --- response_body -n = 16 +n = 17 --- no_error_log [error] @@ -513,7 +513,7 @@ n = 7 --- request GET /test --- response_body -n = 7 +n = 8 --- no_error_log [error] --- skip_eval: 3:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/t/067-req-socket.t b/t/067-req-socket.t index 9aff58b647..54d59841f9 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -1176,3 +1176,31 @@ GET /t received: received: abc --- no_error_log [error] + + + +=== TEST 19: getfd +--- http_config + lua_package_path "../lua-resty-core/lib/?.lua;;"; +--- config + location /t { + content_by_lua_block { + local sock, err = ngx.req.socket() + if sock then + ngx.say("got the request socket") + else + ngx.say("failed to get the request socket: ", err) + end + + ngx.say(sock:getfd()) + } + } +--- request +POST /t +hello world +--- response_body eval +qr/\Agot the request socket +\d+ +\z/ms +--- no_error_log +[error] diff --git a/t/116-raw-req-socket.t b/t/116-raw-req-socket.t index 6704a92084..94ef077c41 100644 --- a/t/116-raw-req-socket.t +++ b/t/116-raw-req-socket.t @@ -12,7 +12,7 @@ use Test::Nginx::Socket::Lua $SkipReason ? (skip_all => $SkipReason) : (); repeat_each(2); -plan tests => repeat_each() * 43; +plan tests => repeat_each() * 46; our $HtmlDir = html_dir; @@ -977,3 +977,89 @@ GET /t msg: 1: received: hello --- no_error_log [error] + + + +=== TEST 17: getfd() +--- config + server_tokens off; + location = /t { + #set $port 5000; + set $port $TEST_NGINX_SERVER_PORT; + + content_by_lua_block { + local sock = ngx.socket.tcp() + local port = ngx.var.port + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local req = "GET /mysock HTTP/1.1\r\nUpgrade: mysock\r\nHost: localhost\r\nConnection: close\r\n\r\nhello" + -- req = "OK" + + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + local reader = sock:receiveuntil("\r\n\r\n") + local data, err, partial = reader() + if not data then + ngx.say("no response header found") + return + end + + local msg, err = sock:receive() + if not msg then + ngx.say("failed to receive: ", err) + return + end + + ngx.say("msg: ", msg) + + ok, err = sock:close() + if not ok then + ngx.say("failed to close socket: ", err) + return + end + } + } + + location = /mysock { + content_by_lua_block { + ngx.status = 101 + ngx.send_headers() + ngx.flush(true) + ngx.req.read_body() + local sock, err = ngx.req.socket(true) + if not sock then + ngx.log(ngx.ERR, "server: failed to get raw req socket: ", err) + return + end + + local s = sock:getfd() + local data, err = sock:receive(5) + if not data then + ngx.log(ngx.ERR, "server: failed to receive: ", err) + return + end + + local bytes, err = sock:send("fd: " .. tostring(s) .. " 1: received: " .. data .. "\n") + if not bytes then + ngx.log(ngx.ERR, "server: failed to send: ", err) + return + end + } + more_clear_headers Date; + } + +--- request +GET /t +--- response_body eval +qr/\Amsg: fd: \d+ 1: received: hello +/ms +--- no_error_log +[error] From 02b5e4919fef3d457b8ee749afdefc73390ed985 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sun, 4 May 2025 12:25:49 +0800 Subject: [PATCH 811/848] style: fixed coding style. --- src/ngx_http_lua_socket_tcp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index b943c8ece4..db3fea507c 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -227,9 +227,9 @@ static char ngx_http_lua_pattern_udata_metatable_key; #define ngx_http_lua_tcp_socket_metatable_literal_key "__tcp_cosocket_mt" -#define ngx_http_lua_tcp_req_socket_metatable_literal_key \ +#define ngx_http_lua_tcp_req_socket_metatable_literal_key \ "__tcp_req_cosocket_mt" -#define ngx_http_lua_tcp_raw_req_socket_metatable_literal_key \ +#define ngx_http_lua_tcp_raw_req_socket_metatable_literal_key \ "__tcp_raw_req_cosocket_mt" From bf94a7a054497f395f47040ebdc8371216b36fef Mon Sep 17 00:00:00 2001 From: willmafh Date: Mon, 5 May 2025 23:24:11 +0800 Subject: [PATCH 812/848] chore: code style consistency. --- src/ngx_http_lua_module.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 63367f46b3..0f23c2d769 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -1030,6 +1030,13 @@ ngx_http_lua_create_main_conf(ngx_conf_t *cf) * lmcf->shm_zones = NULL; * lmcf->init_handler = NULL; * lmcf->init_src = { 0, NULL }; + * lmcf->init_chunkname = NULL; + * lmcf->init_worker_handler = NULL; + * lmcf->init_worker_src = { 0, NULL }; + * lmcf->init_worker_chunkname = NULL; + * lmcf->exit_worker_handler = NULL; + * lmcf->exit_worker_src = { 0, NULL }; + * lmcf->exit_worker_chunkname = NULL; * lmcf->shm_zones_inited = 0; * lmcf->shdict_zones = NULL; * lmcf->preload_hooks = NULL; @@ -1197,6 +1204,11 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) * lscf->srv.ssl_sess_fetch_chunkname = NULL; * lscf->srv.ssl_sess_fetch_src_key = NULL; * + * lscf->srv.server_rewrite_handler = NULL; + * lscf->srv.server_rewrite_src = { 0, NULL }; + * lscf->srv.server_rewrite_chunkname = NULL; + * lscf->srv.server_rewrite_src_key = NULL; + * * lscf->balancer.original_init_upstream = NULL; * lscf->balancer.original_init_peer = NULL; * lscf->balancer.handler = NULL; @@ -1212,6 +1224,7 @@ ngx_http_lua_create_srv_conf(ngx_conf_t *cf) lscf->srv.ssl_sess_fetch_src_ref = LUA_REFNIL; #endif + lscf->srv.server_rewrite_src_ref = LUA_REFNIL; lscf->balancer.src_ref = LUA_REFNIL; lscf->balancer.max_cached = NGX_CONF_UNSET_UINT; return lscf; @@ -1387,25 +1400,33 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) /* set by ngx_pcalloc: * conf->access_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->access_src_key = NULL + * conf->access_handler = NULL; + * conf->access_chunkname = NULL; + * * conf->rewrite_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->rewrite_src_key = NULL; * conf->rewrite_handler = NULL; + * conf->rewrite_chunkname = NULL; * * conf->content_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->content_src_key = NULL; * conf->content_handler = NULL; + * conf->content_chunkname = NULL; * * conf->log_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->log_src_key = NULL; * conf->log_handler = NULL; + * conf->log_chunkname = NULL; * * conf->header_filter_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->header_filter_src_key = NULL; * conf->header_filter_handler = NULL; + * conf->header_filter_chunkname = NULL; * * conf->body_filter_src = {{ 0, NULL }, NULL, NULL, NULL}; * conf->body_filter_src_key = NULL; * conf->body_filter_handler = NULL; + * conf->body_filter_chunkname = NULL; * * conf->ssl = 0; * conf->ssl_protocols = 0; From 058126221710f79e2aa68ac6e42b7a19602a5b48 Mon Sep 17 00:00:00 2001 From: Zeping Bai Date: Mon, 5 May 2025 23:31:09 +0800 Subject: [PATCH 813/848] bugfix: ensure context is restorable on fd writable events. --- src/ngx_http_lua_bodyfilterby.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index c0484c8de0..179a501a59 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -368,6 +368,7 @@ ngx_http_lua_body_filter(ngx_http_request_t *r, ngx_chain_t *in) } } else { + ctx->context = old_context; out = NULL; } From d1d5b7375db0b4ca14269157bbf5194cda741eec Mon Sep 17 00:00:00 2001 From: Gabriel Clima Date: Tue, 6 May 2025 14:31:51 +0300 Subject: [PATCH 814/848] feature: added ngx_http_lua_ffi_ssl_get_client_hello_ext_present(). --- src/ngx_http_lua_ssl_client_helloby.c | 45 +++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 9800f7d41f..26c84345d9 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -662,6 +662,51 @@ ngx_http_lua_ffi_ssl_get_client_hello_ext(ngx_http_request_t *r, } +int +ngx_http_lua_ffi_ssl_get_client_hello_ext_present(ngx_http_request_t *r, + int **extensions, size_t *extensions_len, char **err) +{ + ngx_ssl_conn_t *ssl_conn; + int got_extensions; + size_t ext_len; + int *ext_out; + /* OPENSSL will allocate memory for us and make the ext_out point to it */ + + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + got_extensions = SSL_client_hello_get1_extensions_present(ssl_conn, + &ext_out, &ext_len); + if (!got_extensions || !ext_out || !ext_len) { + *err = "failed SSL_client_hello_get1_extensions_present()"; + return NGX_DECLINED; + } + + *extensions = ngx_palloc(r->connection->pool, sizeof(int) * ext_len); + if (*extensions != NULL) { + ngx_memcpy(*extensions, ext_out, sizeof(int) * ext_len); + *extensions_len = ext_len; + } + + OPENSSL_free(ext_out); + return NGX_OK; +#else + *err = "OpenSSL too old to support this function"; + return NGX_ERROR; +#endif +} + + int ngx_http_lua_ffi_ssl_set_protocols(ngx_http_request_t *r, int protocols, char **err) From b328ac17e20c562ec707c0e719b1c76394621643 Mon Sep 17 00:00:00 2001 From: ElvaLiu Date: Sun, 1 Dec 2024 18:23:11 -0800 Subject: [PATCH 815/848] fix: support tcp binding ip:port or ip of ipv4 or ipv6 --- README.markdown | 5 +-- src/ngx_http_lua_socket_tcp.c | 35 +++++++++++++++++--- t/168-tcp-socket-bind.t | 61 +++++++++++++++++++++++++++++++++++ 3 files changed, 95 insertions(+), 6 deletions(-) diff --git a/README.markdown b/README.markdown index fa8f2c92ba..117a9e484b 100644 --- a/README.markdown +++ b/README.markdown @@ -7855,13 +7855,14 @@ See also [ngx.socket.udp](#ngxsocketudp). tcpsock:bind ------------ -**syntax:** *ok, err = tcpsock:bind(address)* +**syntax:** *ok, err = tcpsock:bind(address, port?)* **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*,ssl_session_fetch_by_lua*,ssl_client_hello_by_lua** Just like the standard [proxy_bind](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_bind) directive, this api makes the outgoing connection to a upstream server originate from the specified local IP address. -Only IP addresses can be specified as the `address` argument. +IP addresses can be specified as the `address` argument. +The optional `port` argument is usually used in the transparent proxy. Here is an example for connecting to a TCP server from the specified local IP address: diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index db3fea507c..9b8c224624 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -862,8 +862,9 @@ ngx_http_lua_socket_tcp_bind(lua_State *L) n = lua_gettop(L); - if (n != 2) { - return luaL_error(L, "expecting 2 arguments, but got %d", + /* Correct the parameter check and allow 2 or 3 parameters */ + if (n != 2 && n != 3) { + return luaL_error(L, "expecting 2 or 3 arguments, but got %d", lua_gettop(L)); } @@ -881,6 +882,26 @@ ngx_http_lua_socket_tcp_bind(lua_State *L) luaL_checktype(L, 1, LUA_TTABLE); + port = 0; + /* handle case: host:port */ + /* Hit the following parameter combination: + * sock:bind("127.0.0.1", port) */ + if (n == 3) { + if (!lua_isnumber(L, 3)) { + lua_pushnil(L); + lua_pushfstring(L, "expecting port to be a" + "number but got type: %s", luaL_typename(L, 3)); + return 2; + } + + port = (int) lua_tointeger(L, 3); + if (port < 0 || port > 65535) { + lua_pushnil(L); + lua_pushfstring(L, "bad port number: %d", port); + return 2; + } + } + text = (u_char *) luaL_checklstring(L, 2, &len); local = ngx_http_lua_parse_addr(L, text, len); @@ -890,9 +911,11 @@ ngx_http_lua_socket_tcp_bind(lua_State *L) return 2; } + if (port > 0) { + ngx_inet_set_port(local->sockaddr, (in_port_t) port); + } /* TODO: we may reuse the userdata here */ lua_rawseti(L, 1, SOCKET_BIND_INDEX); - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "lua tcp socket bind ip: %V", &local->name); @@ -1145,6 +1168,10 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) lua_rawgeti(L, 1, SOCKET_BIND_INDEX); local = lua_touserdata(L, -1); + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket sock:connect ip: %V", &local->name); + lua_pop(L, 1); if (local) { @@ -3435,7 +3462,7 @@ ngx_http_lua_socket_tcp_handler(ngx_event_t *ev) static ngx_int_t ngx_http_lua_socket_tcp_get_peer(ngx_peer_connection_t *pc, void *data) { - /* empty */ + pc->type = SOCK_STREAM; return NGX_OK; } diff --git a/t/168-tcp-socket-bind.t b/t/168-tcp-socket-bind.t index a2aa50b4e7..123b525fe3 100644 --- a/t/168-tcp-socket-bind.t +++ b/t/168-tcp-socket-bind.t @@ -365,3 +365,64 @@ GET /t connected: 1 --- no_error_log [error] + + + +=== TEST 7: upstream sockets bind with ip port +--- config + server_tokens off; + location /t { + set $port $TEST_NGINX_SERVER_PORT; + content_by_lua_block { + local ip = "127.0.0.1" + local port = ngx.var.port + + local sock = ngx.socket.tcp() + + local ok, err = sock:bind(ip, 12345) + if not ok then + ngx.say("failed to bind", err) + return + end + + local ok, err = sock:connect("127.0.0.1", port) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + local ok, err = sock:setoption("reuseaddr", 1) + if not ok then + ngx.say("setoption reuseaddr failed: ", err) + end + + ngx.say("connected: ", ok) + + local bytes, err = sock:send("GET /foo HTTP/1.1\r\nHost: localhost\r\nConnection: keepalive\r\n\r\n") + if not bytes then + ngx.say("failed to send request: ", err) + return + end + + local reader = sock:receiveuntil("\r\n0\r\n\r\n") + local data, err = reader() + + if not data then + ngx.say("failed to receive response body: ", err) + return + end + sock:close() + ngx.say(data) + + } + } + + location /foo { + echo bind: $remote_addr:$remote_port; + } +--- request +GET /t +--- response_body eval +qr/bind:\s127\.0\.0\.1:12345|failed\s+to\s+connect:\s+address\s+already\s+in\s+use/ +--- error_log eval +"lua tcp socket bind ip: 127.0.0.1" From edd1b6a5998f6fe63cde72cef45c1d89e814b0e4 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 7 May 2025 23:33:46 +0800 Subject: [PATCH 816/848] bugfix: fixed typo. --- src/ngx_http_lua_socket_tcp.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_socket_tcp.c b/src/ngx_http_lua_socket_tcp.c index 9b8c224624..85cf49069e 100644 --- a/src/ngx_http_lua_socket_tcp.c +++ b/src/ngx_http_lua_socket_tcp.c @@ -853,6 +853,7 @@ ngx_http_lua_socket_tcp_connect_helper(lua_State *L, static int ngx_http_lua_socket_tcp_bind(lua_State *L) { + int port; ngx_http_request_t *r; ngx_http_lua_ctx_t *ctx; int n; @@ -1169,8 +1170,10 @@ ngx_http_lua_socket_tcp_connect(lua_State *L) lua_rawgeti(L, 1, SOCKET_BIND_INDEX); local = lua_touserdata(L, -1); - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "lua tcp socket sock:connect ip: %V", &local->name); + if (local != NULL) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "lua tcp socket sock:connect ip: %V", &local->name); + } lua_pop(L, 1); From 1f4d846f02cd3980d1f64fbc3e5359fd0e7716f6 Mon Sep 17 00:00:00 2001 From: Gabriel Clima Date: Sat, 24 May 2025 15:29:14 +0300 Subject: [PATCH 817/848] feature: ngx_http_lua_ffi_ssl_get_client_hello_ciphers(). Co-authored-by: lijunlong --- src/ngx_http_lua_ssl_client_helloby.c | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 26c84345d9..b92fc46b5d 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -707,6 +707,55 @@ ngx_http_lua_ffi_ssl_get_client_hello_ext_present(ngx_http_request_t *r, } +int ngx_http_lua_ffi_ssl_get_client_hello_ciphers(ngx_http_request_t *r, + unsigned short *ciphers, size_t ciphers_size, char **err) +{ + int i; + size_t ciphers_cnt; + size_t ciphersuites_bytes; + ngx_ssl_conn_t *ssl_conn; + const unsigned char *ciphers_raw; + + if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + ciphersuites_bytes = SSL_client_hello_get0_ciphers(ssl_conn, &ciphers_raw); + + if (ciphersuites_bytes == 0) { + *err = "failed SSL_client_hello_get0_ciphers()"; + return NGX_DECLINED; + } + + if (ciphersuites_bytes % 2 != 0) { + *err = "SSL_client_hello_get0_ciphers() odd ciphersuites_bytes"; + return NGX_DECLINED; + } + + ciphers_cnt = ciphersuites_bytes / 2; + ciphers_cnt = ciphers_cnt > ciphers_size ? ciphers_size : ciphers_cnt; + + for (i = 0 ; i < (int) ciphers_cnt ; i++) { + ciphers[i] = (ciphers_raw[i * 2] << 8) | ciphers_raw[i * 2 + 1]; + } + + return ciphers_cnt; +#else + *err = "OpenSSL too old to support this function"; + return NGX_ERROR; +#endif +} + + int ngx_http_lua_ffi_ssl_set_protocols(ngx_http_request_t *r, int protocols, char **err) From 9688812a4eba47c1f43892c998e50b988d740f5d Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 9 Jun 2025 08:19:02 +0800 Subject: [PATCH 818/848] tests: bumped openssl to v3.4.1. --- .travis.yml | 40 +++++++++++++++++---------------------- t/143-ssl-session-fetch.t | 10 +++++++++- util/build-with-dd.sh | 19 +------------------ util/build-without-ssl.sh | 24 +---------------------- util/build.sh | 9 --------- 5 files changed, 28 insertions(+), 74 deletions(-) diff --git a/.travis.yml b/.travis.yml index 4eb8d3e5ce..8eb5abd914 100644 --- a/.travis.yml +++ b/.travis.yml @@ -29,6 +29,10 @@ addons: - libunwind-dev - wget - libbrotli1 + - lsb-release + - wget + - gnupg + - ca-certificates cache: directories: @@ -42,13 +46,10 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_PREFIX=/opt/pcre - - PCRE2_PREFIX=/opt/pcre2 + - PCRE_PREFIX=/usr/local/openresty/pcre2 - PCRE_LIB=$PCRE_PREFIX/lib - - PCRE2_LIB=$PCRE2_PREFIX/lib - PCRE_INC=$PCRE_PREFIX/include - - PCRE2_INC=$PCRE2_PREFIX/include - - OPENSSL_PREFIX=/opt/ssl + - OPENSSL_PREFIX=/usr/local/openresty/openssl3 - OPENSSL_LIB=$OPENSSL_PREFIX/lib - OPENSSL_INC=$OPENSSL_PREFIX/include - LIBDRIZZLE_PREFIX=/opt/drizzle @@ -59,14 +60,10 @@ env: - TEST_NGINX_SLEEP=0.006 - MALLOC_PERTURB_=9 jobs: - #- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f - #- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1w TEST_NGINX_USE_HTTP2=1 - - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f TEST_NGINX_TIMEOUT=5 PCRE_VER=8.45 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.42 - - NGINX_VERSION=1.27.1 OPENSSL_VER=1.1.1w OPENSSL_PATCH_VER=1.1.1f TEST_NGINX_TIMEOUT=5 PCRE_VER=8.45 TEST_NGINX_USE_HTTP2=1 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.42 TEST_NGINX_USE_HTTP2=1 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.0.15 OPENSSL_PATCH_VER=3.0.15 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.42 - - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.42 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.4.1 OPENSSL_PATCH_VER=3.4.1 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.4.1 OPENSSL_PATCH_VER=3.4.1 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 TEST_NGINX_USE_HTTP2=1 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.4.1 OPENSSL_PATCH_VER=3.4.1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 + - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 services: - memcached @@ -77,15 +74,16 @@ before_install: - '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)' - '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)' - /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1) + - wget -O - https://openresty.org/package/pubkey.gpg | sudo apt-key add - + - echo "deb http://openresty.org/package/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/openresty.list + - sudo apt-get update + - sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends openresty-pcre2 openresty-openssl3 openresty-pcre2-dev openresty-openssl3-dev + install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - #- if [ -n "$PCRE_VER" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi #- if [ -n "$PCRE2_VER" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi #- if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - - if [ -n "$OPENSSL_VER" ]; then wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v1.0.0/openssl-${OPENSSL_VER}-x64-focal.tar.gz; fi - - if [ -n "$PCRE_VER" ]; then wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v1.0.0/pcre-${PCRE_VER}-x64-focal.tar.gz; fi - - if [ -n "$PCRE2_VER" ]; then wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v1.0.0/pcre2-${PCRE2_VER}-x64-focal.tar.gz; fi - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git @@ -137,13 +135,9 @@ script: - sudo make install-libdrizzle-1.0 > build.log 2>&1 || (cat build.log && exit 1) - cd ../mockeagain/ && make CC=$CC -j$JOBS && cd .. - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. - #- if [ -n "PCRE_VER" ]; then tar zxf download-cache/pcre-$PCRE_VER.tar.gz; cd pcre-$PCRE_VER/; ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi #- if [ -n "$PCRE2_VER" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi #- if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX --libdir=lib -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - - if [ -n "$BORINGSSL" ]; then sudo mkdir -p /opt/ssl && sudo tar -C /opt/ssl -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi - - if [ -n "$OPENSSL_VER" ]; then sudo mkdir -p /opt/ssl && sudo tar -C /opt/ssl -xf openssl-$OPENSSL_VER-x64-focal.tar.gz --strip-components=2; fi - - if [ -n "$PCRE_VER" ]; then sudo mkdir -p $PCRE_PREFIX && sudo tar -C $PCRE_PREFIX -xf pcre-$PCRE_VER-x64-focal.tar.gz --strip-components=2; fi - - if [ -n "$PCRE2_VER" ]; then sudo mkdir -p $PCRE2_PREFIX && sudo tar -C $PCRE2_PREFIX -xf pcre2-$PCRE2_VER-x64-focal.tar.gz --strip-components=2; fi + - if [ -n "$BORINGSSL" ]; then sudo rm -fr /usr/local/openresty/openssl3/ && sudo tar -C /usr/local/openresty/openssl3 -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi - export NGX_BUILD_CC=$CC - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - sh util/build-with-dd.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) @@ -160,4 +154,4 @@ script: - dig +short myip.opendns.com @resolver1.opendns.com || exit 0 - dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0 - dig +short @$TEST_NGINX_RESOLVER agentzh.org || exit 0 - - /usr/bin/env perl $(command -v prove) -I. -Itest-nginx/lib -r t/ + - /usr/bin/env perl $(command -v prove) -I. -Itest-nginx/inc -Itest-nginx/lib -r t/ diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 8e09a52d64..8c7c156ede 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -142,7 +142,7 @@ ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; - lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2; + lua_ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; @@ -199,6 +199,7 @@ qr/elapsed in ssl fetch session by lua: 0.(?:09|1[01])\d+,/, [error] [alert] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -388,6 +389,7 @@ qr/received memc reply: OK/s [alert] [error] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -635,6 +637,7 @@ qr/ssl_session_fetch_by_lua\*: sess get cb exit code: 0/s should never reached here [alert] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -804,6 +807,7 @@ ssl_session_fetch_by_lua*: sess get cb exit code: 0 should never reached here [alert] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -991,6 +995,7 @@ ssl store session by lua is running! [error] [alert] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -1399,6 +1404,7 @@ qr/elapsed in ssl_session_fetch_by_lua\*: 0\.(?:09|1[01])\d+,/, [error] [alert] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -1494,6 +1500,7 @@ close: 1 nil [alert] [error] [emerg] +--- skip_openssl: 6: > 1.1.1w @@ -1695,3 +1702,4 @@ uthread: failed to kill: already waited or killed [alert] [error] [emerg] +--- skip_openssl: 6: > 1.1.1w diff --git a/util/build-with-dd.sh b/util/build-with-dd.sh index a56a89119b..92ca6da328 100755 --- a/util/build-with-dd.sh +++ b/util/build-with-dd.sh @@ -12,30 +12,13 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" -add_http3_module=--with-http_v3_module -answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$OPENSSL_VER" = "1.1.0l" ] || [ "$answer" = "N" ]; then - add_http3_module="" -fi - -disable_pcre2=--without-pcre2 -answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then - disable_pcre2="" -fi -if [ "$USE_PCRE2" = "Y" ]; then - PCRE_INC=$PCRE2_INC - PCRE_LIB=$PCRE2_LIB -fi - time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ - $disable_pcre2 \ --with-ipv6 \ --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC -DDDEBUG=1" \ --with-http_v2_module \ - $add_http3_module \ + --with-http_v3_module \ --with-http_realip_module \ --with-http_ssl_module \ --add-module=$root/../ndk-nginx-module \ diff --git a/util/build-without-ssl.sh b/util/build-without-ssl.sh index 2a998e3c79..0bab08c8b6 100755 --- a/util/build-without-ssl.sh +++ b/util/build-without-ssl.sh @@ -10,34 +10,12 @@ version=${1:-1.4.1} home=~ force=$2 -# the ngx-build script is from https://github.com/agentzh/nginx-devel-utils - - #--add-module=$home/work/nginx_upload_module-2.2.0 \ - - #--without-pcre \ - #--without-http_rewrite_module \ - #--without-http_autoindex_module \ - #--with-cc=gcc46 \ - #--with-cc=clang \ - #--without-http_referer_module \ - #--with-http_spdy_module \ - add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" -disable_pcre2=--without-pcre2 -answer=`$root/util/ver-ge "$NGINX_VERSION" 1.25.1` -if [ "$answer" = "N" ] || [ -n "$PCRE2_VER" ]; then - disable_pcre2="" -fi -if [ -n "$PCRE2_VER" ]; then - PCRE_INC=$PCRE2_INC - PCRE_LIB=$PCRE2_LIB -fi - +rm -fr buildroot time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ - $disable_pcre2 \ --with-ipv6 \ --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC" \ --with-http_v2_module \ diff --git a/util/build.sh b/util/build.sh index fdf5c4d53d..a901c03cd3 100755 --- a/util/build.sh +++ b/util/build.sh @@ -30,20 +30,11 @@ if [ "$OPENSSL_VER" = "1.1.0l" ] || [ "$answer" = "N" ]; then add_http3_module="" fi -disable_pcre2=--without-pcre2 answer=`$root/util/ver-ge "$version" 1.25.1` -if [ "$answer" = "N" ] || [ "$USE_PCRE2" = "Y" ]; then - disable_pcre2="" -fi -if [ "$USE_PCRE2" = "Y" ]; then - PCRE_INC=$PCRE2_INC - PCRE_LIB=$PCRE2_LIB -fi time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ - $disable_pcre2 \ --with-ipv6 \ --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC" \ --with-http_v2_module \ From 4e40c1314f316e5ba4bf538213a2312daa950a8a Mon Sep 17 00:00:00 2001 From: lijunlong Date: Sat, 21 Jun 2025 17:14:40 +0800 Subject: [PATCH 819/848] tests: update openssl to 3.5.0. --- .travis.yml | 12 +++--- t/014-bugs.t | 12 +++--- util/build-with-dd.sh | 4 +- util/build-without-ssl.sh | 4 +- util/build.sh | 82 ++++++++++++++++++--------------------- 5 files changed, 54 insertions(+), 60 deletions(-) diff --git a/.travis.yml b/.travis.yml index 8eb5abd914..b65a9d528f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -46,9 +46,9 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE_PREFIX=/usr/local/openresty/pcre2 - - PCRE_LIB=$PCRE_PREFIX/lib - - PCRE_INC=$PCRE_PREFIX/include + - PCRE2_PREFIX=/usr/local/openresty/pcre2 + - PCRE2_LIB=$PCRE2_PREFIX/lib + - PCRE2_INC=$PCRE2_PREFIX/include - OPENSSL_PREFIX=/usr/local/openresty/openssl3 - OPENSSL_LIB=$OPENSSL_PREFIX/lib - OPENSSL_INC=$OPENSSL_PREFIX/include @@ -60,9 +60,9 @@ env: - TEST_NGINX_SLEEP=0.006 - MALLOC_PERTURB_=9 jobs: - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.4.1 OPENSSL_PATCH_VER=3.4.1 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.4.1 OPENSSL_PATCH_VER=3.4.1 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 TEST_NGINX_USE_HTTP2=1 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.4.1 OPENSSL_PATCH_VER=3.4.1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.5.0 OPENSSL_PATCH_VER=3.5.0 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.5.0 OPENSSL_PATCH_VER=3.5.0 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 TEST_NGINX_USE_HTTP2=1 + - NGINX_VERSION=1.27.1 OPENSSL_VER=3.5.0 OPENSSL_PATCH_VER=3.5.0 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 services: diff --git a/t/014-bugs.t b/t/014-bugs.t index 9d57afe134..a89efec3c1 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -1274,9 +1274,9 @@ worker_shutdown_timeout 1; --- config location /t { content_by_lua_block { - local function thread_func() + local function thread_func(port) local sock = ngx.socket.tcp() - local ok, err = sock:connect("127.0.0.1", 65110) + local ok, err = sock:connect("127.0.0.1", port) local bytes, err = sock:send("hello") if bytes ~= 5 then sock:close() @@ -1293,11 +1293,11 @@ location /t { ngx.log(ngx.ERR, "successfully read a line: ", line) end - local function timer_func() - ngx.thread.spawn(thread_func) + local function timer_func(port) + ngx.thread.spawn(thread_func, port) end - ngx.timer.at(1, timer_func) + ngx.timer.at(1, timer_func, ngx.var.server_port) ngx.say("Hello world") } } @@ -1311,7 +1311,7 @@ my $expr; if ($ENV{TEST_NGINX_USE_HTTP3}) { $expr = qr|lua close the global Lua VM| } else { - $expr = qr|failed to read a line: closed| + $expr = qr/failed to read a line: closed|attempt to send data on a closed socket/ } $expr; diff --git a/util/build-with-dd.sh b/util/build-with-dd.sh index 92ca6da328..4f4e785f53 100755 --- a/util/build-with-dd.sh +++ b/util/build-with-dd.sh @@ -16,14 +16,14 @@ time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ --with-ipv6 \ - --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC -DDDEBUG=1" \ + --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE2_INC -I$OPENSSL_INC -DDDEBUG=1" \ --with-http_v2_module \ --with-http_v3_module \ --with-http_realip_module \ --with-http_ssl_module \ --add-module=$root/../ndk-nginx-module \ --add-module=$root/../set-misc-nginx-module \ - --with-ld-opt="-L$PCRE_LIB -L$OPENSSL_LIB -Wl,-rpath,$PCRE_LIB:$LIBDRIZZLE_LIB:$OPENSSL_LIB" \ + --with-ld-opt="-L$PCRE2_LIB -L$OPENSSL_LIB -Wl,-rpath,$PCRE2_LIB:$LIBDRIZZLE_LIB:$OPENSSL_LIB" \ --without-mail_pop3_module \ --without-mail_imap_module \ --with-http_image_filter_module \ diff --git a/util/build-without-ssl.sh b/util/build-without-ssl.sh index 0bab08c8b6..022d9617e2 100755 --- a/util/build-without-ssl.sh +++ b/util/build-without-ssl.sh @@ -17,12 +17,12 @@ time ngx-build $force $version \ --with-threads \ --with-pcre-jit \ --with-ipv6 \ - --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC" \ + --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE2_INC" \ --with-http_v2_module \ --with-http_realip_module \ --add-module=$root/../ndk-nginx-module \ --add-module=$root/../set-misc-nginx-module \ - --with-ld-opt="-L$PCRE_LIB -Wl,-rpath,$PCRE_LIB:$LIBDRIZZLE_LIB" \ + --with-ld-opt="-L$PCRE2_LIB -Wl,-rpath,$PCRE2_LIB:$LIBDRIZZLE_LIB" \ --without-mail_pop3_module \ --without-mail_imap_module \ --with-http_image_filter_module \ diff --git a/util/build.sh b/util/build.sh index a901c03cd3..9dda855223 100755 --- a/util/build.sh +++ b/util/build.sh @@ -24,53 +24,47 @@ force=$2 add_fake_shm_module="--add-module=$root/t/data/fake-shm-module" -add_http3_module=--with-http_v3_module -answer=`$root/util/ver-ge "$version" 1.25.1` -if [ "$OPENSSL_VER" = "1.1.0l" ] || [ "$answer" = "N" ]; then - add_http3_module="" -fi - answer=`$root/util/ver-ge "$version" 1.25.1` time ngx-build $force $version \ - --with-threads \ - --with-pcre-jit \ - --with-ipv6 \ - --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE_INC -I$OPENSSL_INC" \ - --with-http_v2_module \ - $add_http3_module \ - --with-http_realip_module \ - --with-http_ssl_module \ - --add-module=$root/../ndk-nginx-module \ - --add-module=$root/../set-misc-nginx-module \ - --with-ld-opt="-L$PCRE_LIB -L$OPENSSL_LIB -Wl,-rpath,$PCRE_LIB:$LIBDRIZZLE_LIB:$OPENSSL_LIB" \ - --without-mail_pop3_module \ - --without-mail_imap_module \ - --with-http_image_filter_module \ - --without-mail_smtp_module \ - --with-stream \ - --with-stream_ssl_module \ - --without-http_upstream_ip_hash_module \ - --without-http_memcached_module \ - --without-http_auth_basic_module \ - --without-http_userid_module \ - --with-http_auth_request_module \ - --add-module=$root/../echo-nginx-module \ - --add-module=$root/../memc-nginx-module \ - --add-module=$root/../srcache-nginx-module \ - --add-module=$root \ - --add-module=$root/../lua-upstream-nginx-module \ - --add-module=$root/../headers-more-nginx-module \ - --add-module=$root/../drizzle-nginx-module \ - --add-module=$root/../rds-json-nginx-module \ - --add-module=$root/../coolkit-nginx-module \ - --add-module=$root/../redis2-nginx-module \ - --add-module=$root/../stream-lua-nginx-module \ - --add-module=$root/t/data/fake-module \ - $add_fake_shm_module \ - --add-module=$root/t/data/fake-delayed-load-module \ - --with-http_gunzip_module \ - --with-http_dav_module \ + --with-threads \ + --with-pcre-jit \ + --with-ipv6 \ + --with-cc-opt="-DNGX_LUA_USE_ASSERT -I$PCRE2_INC -I$OPENSSL_INC" \ + --with-http_v2_module \ + --with-http_v3_module \ + --with-http_realip_module \ + --with-http_ssl_module \ + --add-module=$root/../ndk-nginx-module \ + --add-module=$root/../set-misc-nginx-module \ + --with-ld-opt="-L$PCRE2_LIB -L$OPENSSL_LIB -Wl,-rpath,$PCRE2_LIB:$LIBDRIZZLE_LIB:$OPENSSL_LIB" \ + --without-mail_pop3_module \ + --without-mail_imap_module \ + --with-http_image_filter_module \ + --without-mail_smtp_module \ + --with-stream \ + --with-stream_ssl_module \ + --without-http_upstream_ip_hash_module \ + --without-http_memcached_module \ + --without-http_auth_basic_module \ + --without-http_userid_module \ + --with-http_auth_request_module \ + --add-module=$root/../echo-nginx-module \ + --add-module=$root/../memc-nginx-module \ + --add-module=$root/../srcache-nginx-module \ + --add-module=$root \ + --add-module=$root/../lua-upstream-nginx-module \ + --add-module=$root/../headers-more-nginx-module \ + --add-module=$root/../drizzle-nginx-module \ + --add-module=$root/../rds-json-nginx-module \ + --add-module=$root/../coolkit-nginx-module \ + --add-module=$root/../redis2-nginx-module \ + --add-module=$root/../stream-lua-nginx-module \ + --add-module=$root/t/data/fake-module \ + $add_fake_shm_module \ + --add-module=$root/t/data/fake-delayed-load-module \ + --with-http_gunzip_module \ + --with-http_dav_module \ --with-select_module \ --with-poll_module \ $opts \ From 1793b3b88af4de1128db4ff7d30fe3e47a9e5459 Mon Sep 17 00:00:00 2001 From: swananan Date: Tue, 1 Jul 2025 11:48:33 +0800 Subject: [PATCH 820/848] bugfix: improve HTTP/3 SSL Lua callback yield handling. --- .travis.yml | 2 +- src/ngx_http_lua_ssl_certby.c | 12 +- src/ngx_http_lua_ssl_client_helloby.c | 17 +- src/ngx_http_lua_util.c | 79 ++- src/ngx_http_lua_util.h | 4 + t/139-ssl-cert-by.t | 448 +++++++++++++- t/143-ssl-session-fetch.t | 9 +- t/166-ssl-client-hello.t | 822 +++++++++++++++++++++++++- 8 files changed, 1371 insertions(+), 22 deletions(-) diff --git a/.travis.yml b/.travis.yml index b65a9d528f..f48c9a75bc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -137,7 +137,7 @@ script: - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. #- if [ -n "$PCRE2_VER" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi #- if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX --libdir=lib -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - - if [ -n "$BORINGSSL" ]; then sudo rm -fr /usr/local/openresty/openssl3/ && sudo tar -C /usr/local/openresty/openssl3 -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi + - if [ -n "$BORINGSSL" ]; then sudo rm -fr /usr/local/openresty/openssl3/ && sudo mkdir -p /usr/local/openresty/openssl3 && sudo tar -C /usr/local/openresty/openssl3 -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi - export NGX_BUILD_CC=$CC - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) - sh util/build-with-dd.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 72a651bdbf..0c13af91c4 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -346,7 +346,6 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) return -1; -#if 1 failed: if (r && r->pool) { @@ -358,15 +357,14 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) } return 0; -#endif } static void ngx_http_lua_ssl_cert_done(void *data) { - ngx_connection_t *c; - ngx_http_lua_ssl_ctx_t *cctx = data; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx = data; dd("lua ssl cert done"); @@ -387,6 +385,12 @@ ngx_http_lua_ssl_cert_done(void *data) c->log->action = "SSL handshaking"; ngx_post_event(c->write, &ngx_posted_events); + +#if (NGX_HTTP_V3) && OPENSSL_VERSION_NUMBER >= 0x1000205fL +# if (NGX_QUIC_OPENSSL_COMPAT) + ngx_http_lua_resume_quic_ssl_handshake(c); +# endif +#endif } diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index b92fc46b5d..b4377f2f1b 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -341,7 +341,6 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, return -1; -#if 1 failed: if (r && r->pool) { @@ -353,15 +352,14 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, } return 0; -#endif } static void ngx_http_lua_ssl_client_hello_done(void *data) { - ngx_connection_t *c; - ngx_http_lua_ssl_ctx_t *cctx = data; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx = data; dd("lua ssl client hello done"); @@ -382,6 +380,12 @@ ngx_http_lua_ssl_client_hello_done(void *data) c->log->action = "SSL handshaking"; ngx_post_event(c->write, &ngx_posted_events); + +#if (NGX_HTTP_V3) && defined(SSL_ERROR_WANT_CLIENT_HELLO_CB) +# if (NGX_QUIC_OPENSSL_COMPAT) + ngx_http_lua_resume_quic_ssl_handshake(c); +# endif +#endif } @@ -666,6 +670,7 @@ int ngx_http_lua_ffi_ssl_get_client_hello_ext_present(ngx_http_request_t *r, int **extensions, size_t *extensions_len, char **err) { +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB ngx_ssl_conn_t *ssl_conn; int got_extensions; size_t ext_len; @@ -684,7 +689,6 @@ ngx_http_lua_ffi_ssl_get_client_hello_ext_present(ngx_http_request_t *r, return NGX_ERROR; } -#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB got_extensions = SSL_client_hello_get1_extensions_present(ssl_conn, &ext_out, &ext_len); if (!got_extensions || !ext_out || !ext_len) { @@ -710,6 +714,7 @@ ngx_http_lua_ffi_ssl_get_client_hello_ext_present(ngx_http_request_t *r, int ngx_http_lua_ffi_ssl_get_client_hello_ciphers(ngx_http_request_t *r, unsigned short *ciphers, size_t ciphers_size, char **err) { +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB int i; size_t ciphers_cnt; size_t ciphersuites_bytes; @@ -727,8 +732,6 @@ int ngx_http_lua_ffi_ssl_get_client_hello_ciphers(ngx_http_request_t *r, return NGX_ERROR; } - -#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB ciphersuites_bytes = SSL_client_hello_get0_ciphers(ssl_conn, &ciphers_raw); if (ciphersuites_bytes == 0) { diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 61f56cb7c0..d47c691000 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -49,6 +49,10 @@ #if (NGX_THREADS) #include "ngx_http_lua_worker_thread.h" #endif +#if (NGX_HTTP_V3) +#include +#include +#endif #if 1 @@ -3826,6 +3830,18 @@ ngx_http_lua_close_fake_connection(ngx_connection_t *c) c->read->closed = 1; c->write->closed = 1; + /* When destroying the pool, the registered clean callbacks will be + * executed. If the ngx_connection_t is freed before these callbacks are + * run, and a new ngx_connection_t is created within a clean callback, + * it is possible for the freed ngx_connection_t to be reused again. + * If this reused ngx_connection_t is destroyed again within the clean + * callback logic, it may result in other clean callbacks holding a + * ngx_connection_t that has already been destroyed. + */ + if (pool) { + ngx_destroy_pool(pool); + } + /* we temporarily use a valid fd (0) to make ngx_free_connection happy */ c->fd = 0; @@ -3841,10 +3857,6 @@ ngx_http_lua_close_fake_connection(ngx_connection_t *c) if (ngx_cycle->files) { ngx_cycle->files[0] = saved_c; } - - if (pool) { - ngx_destroy_pool(pool); - } } @@ -4561,4 +4573,63 @@ ngx_http_lua_ffi_bypass_if_checks(ngx_http_request_t *r) r->disable_not_modified = 1; } + +#if (NGX_HTTP_V3) +void +ngx_http_lua_resume_quic_ssl_handshake(ngx_connection_t *c) +{ + ngx_int_t rc, sslerr; + ngx_ssl_conn_t *ssl_conn; + + if (c == NULL || c->ssl == NULL || c->ssl->connection == NULL) { + return; + } + + if (!c->udp || c->read == NULL + || c->read->handler != ngx_quic_input_handler) + { + return; + } + + ssl_conn = c->ssl->connection; + + /* Openresty ssl lua scripts are triggered by registering callbacks into + * openssl. If a lua script calls a yield api during execution, openssl's + * SSL_do_handshake will return a specific error code. The application + * should not treat these codes as fatal. The lua script will resume on + * yield-related events until it finishes. After completion, + * SSL_do_handshake should be called again to advance openssl's state + * machine. + * + * Note that nginx quic and openresty ssl lua scripts are independent. When + * openresty ssl lua runs, the client is waiting for a server response, so + * nginx quic does not affect the execution of the lua script. After the lua + * script finishes, SSL_do_handshake is called again, and nginx quic's + * registered callback continues the handshake. + */ + rc = SSL_do_handshake(ssl_conn); + sslerr = SSL_get_error(ssl_conn, rc); + + if (rc <= 0 && sslerr != SSL_ERROR_WANT_READ +#if OPENSSL_VERSION_NUMBER >= 0x10002000L + && sslerr != SSL_ERROR_WANT_X509_LOOKUP +#endif +#ifdef SSL_ERROR_WANT_CLIENT_HELLO_CB + && sslerr != SSL_ERROR_WANT_CLIENT_HELLO_CB +#endif + ) { + /* If a fatal error occurs or lua script exits with error during quic + * handshake, the quic connection will be closed immediately. + */ + ngx_quic_close_connection(c, NGX_ERROR); + + } else { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, + "lua resuming quic ssl handshake: rc %d, err %d, will " + "continue driving handshake or next lua script phase", + rc, sslerr); + } +} +#endif + /* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index 85c6e614d6..d76508868c 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -267,6 +267,10 @@ ngx_addr_t *ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len); size_t ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size); +#if (NGX_HTTP_V3) +void ngx_http_lua_resume_quic_ssl_handshake(ngx_connection_t *c); +#endif + static ngx_inline void ngx_http_lua_init_ctx(ngx_http_request_t *r, ngx_http_lua_ctx_t *ctx) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index abf447856d..c5b1665046 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -7,6 +7,9 @@ repeat_each(3); # All these tests need to have new openssl my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; +if ($openssl_version =~ m/BoringSSL/) { + $ENV{TEST_NGINX_USE_BORINGSSL} = 1; +} if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); @@ -16,6 +19,7 @@ if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; +$ENV{TEST_NGINX_QUIC_IDLE_TIMEOUT} ||= 0.6; #log_level 'warn'; log_level 'debug'; @@ -2075,7 +2079,7 @@ received: foo close: 1 nil --- error_log -client socket file: +client socket file: --- no_error_log [error] @@ -2321,3 +2325,445 @@ ssl handshake: cdata uthread: hello from f() uthread: killed uthread: failed to kill: already waited or killed + + + +=== TEST 27: ssl_certificate_by_lua* with TCP cosocket (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: TCP cosocket test start") + + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local bytes, err = sock:send("flush_all\r\n") + if not bytes then + ngx.log(ngx.ERR, "failed to send flush_all command: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + print("ssl certificate: received TCP memc reply: ", res) + sock:close() + print("ssl certificate: TCP cosocket test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: TCP cosocket test start|ssl certificate: received TCP memc reply: OK|ssl certificate: TCP cosocket test done|test completed)/ +--- grep_error_log_out +ssl certificate: TCP cosocket test start +ssl certificate: received TCP memc reply: OK +ssl certificate: TCP cosocket test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 28: ssl_certificate_by_lua* with UDP cosocket (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: UDP cosocket test start") + + local sock = ngx.socket.udp() + sock:settimeout(1000) + + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = sock:send(req) + if not ok then + ngx.log(ngx.ERR, "failed to send flush_all to memc: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + print("ssl certificate: received UDP memc reply of ", #res, " bytes") + sock:close() + print("ssl certificate: UDP cosocket test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: UDP cosocket test start|ssl certificate: received UDP memc reply of \d+ bytes|ssl certificate: UDP cosocket test done|test completed)/ +--- grep_error_log_out eval +[ +qr/ssl certificate: UDP cosocket test start/, +qr/ssl certificate: received UDP memc reply of 12 bytes/, +qr/ssl certificate: UDP cosocket test done/, +qr/test completed/, +] + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 29: ssl_certificate_by_lua* with ngx.timer (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: timer test start") + + local function timer_handler() + print("ssl certificate: timer executed") + end + + local ok, err = ngx.timer.at(0, timer_handler) + if not ok then + ngx.log(ngx.ERR, "failed to create timer: ", err) + return + end + + print("ssl certificate: timer created") + print("ssl certificate: timer test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: timer test start|ssl certificate: timer created|ssl certificate: timer test done|ssl certificate: timer executed|test completed)/ +--- grep_error_log_out +ssl certificate: timer test start +ssl certificate: timer created +ssl certificate: timer test done +ssl certificate: timer executed +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 30: ssl_certificate_by_lua* with user threads (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: uthread test start") + + local function worker() + ngx.sleep(0.01) + print("ssl certificate: uthread worker executed") + return "worker_result" + end + + local t, err = ngx.thread.spawn(worker) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return + end + + print("ssl certificate: uthread spawned") + + local ok, res = ngx.thread.wait(t) + if not ok then + ngx.log(ngx.ERR, "failed to wait thread: ", res) + return + end + + print("ssl certificate: uthread result: ", res) + print("ssl certificate: uthread test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: uthread test start|ssl certificate: uthread spawned|ssl certificate: uthread worker executed|ssl certificate: uthread result: worker_result|ssl certificate: uthread test done|test completed)/ +--- grep_error_log_out +ssl certificate: uthread test start +ssl certificate: uthread spawned +ssl certificate: uthread worker executed +ssl certificate: uthread result: worker_result +ssl certificate: uthread test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 31: ssl_certificate_by_lua* with coroutines (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: coroutine test start") + + local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield + + local function coro_func() + local cnt = 0 + for i = 1, 3 do + print("ssl certificate: coro yield: ", cnt) + cy() + cnt = cnt + 1 + end + return "coro_done" + end + + local c = cc(coro_func) + for i = 1, 4 do + print("ssl certificate: coro resume, status: ", coroutine.status(c)) + local ok, res = cr(c) + if not ok then + print("ssl certificate: coro error: ", res) + break + end + if coroutine.status(c) == "dead" then + print("ssl certificate: coro result: ", res) + break + end + end + + print("ssl certificate: coroutine test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: coroutine test start|ssl certificate: coro resume, status: \w+|ssl certificate: coro yield: \d+|ssl certificate: coro result: coro_done|ssl certificate: coroutine test done|test completed)/ +--- grep_error_log_out +ssl certificate: coroutine test start +ssl certificate: coro resume, status: suspended +ssl certificate: coro yield: 0 +ssl certificate: coro resume, status: suspended +ssl certificate: coro yield: 1 +ssl certificate: coro resume, status: suspended +ssl certificate: coro yield: 2 +ssl certificate: coro resume, status: suspended +ssl certificate: coro result: coro_done +ssl certificate: coroutine test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 32: ssl_certificate_by_lua* without yield API (simple logic) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: simple test start") + + -- Simple calculations without yield + local sum = 0 + for i = 1, 10 do + sum = sum + i + end + + print("ssl certificate: calculated sum: ", sum) + + -- String operations + local str = "hello" + str = str .. " world" + print("ssl certificate: concatenated string: ", str) + + -- Table operations + local t = {a = 1, b = 2, c = 3} + local count = 0 + for k, v in pairs(t) do + count = count + v + end + print("ssl certificate: table sum: ", count) + + print("ssl certificate: simple test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: simple test start|ssl certificate: calculated sum: 55|ssl certificate: concatenated string: hello world|ssl certificate: table sum: 6|ssl certificate: simple test done|test completed)/ +--- grep_error_log_out +ssl certificate: simple test start +ssl certificate: calculated sum: 55 +ssl certificate: concatenated string: hello world +ssl certificate: table sum: 6 +ssl certificate: simple test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 33: ssl_certificate_by_lua* with multiple network operations (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} || $ENV{TEST_NGINX_USE_BORINGSSL} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_certificate_by_lua_block { + print("ssl certificate: multiple network test start") + + -- First TCP operation + local sock1 = ngx.socket.tcp() + sock1:settimeout(2000) + local ok, err = sock1:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if ok then + local bytes, err = sock1:send("version\r\n") + if bytes then + local res, err = sock1:receive() + if res then + print("ssl certificate: TCP1 version: ", res) + end + end + sock1:close() + end + + ngx.sleep(0.01) -- Small delay + + -- Second UDP operation + local sock2 = ngx.socket.udp() + sock2:settimeout(1000) + local ok, err = sock2:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if ok then + local req = "\0\1\0\0\0\1\0\0version\r\n" + local ok, err = sock2:send(req) + if ok then + local res, err = sock2:receive() + if res then + print("ssl certificate: UDP version reply length: ", #res) + end + end + sock2:close() + end + + print("ssl certificate: multiple network test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl certificate: multiple network test start|ssl certificate: TCP1 version: VERSION|ssl certificate: UDP version reply length: \d+|ssl certificate: multiple network test done|test completed)/ +--- grep_error_log_out eval +[ +qr/ssl certificate: multiple network test start/, +qr/ssl certificate: TCP1 version: VERSION/, +qr/ssl certificate: UDP version reply length: \d+/, +qr/ssl certificate: multiple network test done/, +qr/test completed/, +] + +--- no_error_log +[error] +[alert] +[emerg] diff --git a/t/143-ssl-session-fetch.t b/t/143-ssl-session-fetch.t index 8c7c156ede..9cc1870a87 100644 --- a/t/143-ssl-session-fetch.t +++ b/t/143-ssl-session-fetch.t @@ -9,6 +9,12 @@ repeat_each(3); plan tests => repeat_each() * (blocks() * 6) - 3; +my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; +my $openssl_version = eval { `$NginxBinary -V 2>&1` }; +if ($openssl_version =~ m/BoringSSL/) { + $ENV{TEST_NGINX_USE_BORINGSSL} = 1; +} + $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; @@ -138,11 +144,12 @@ ssl_session_fetch_by_lua\(nginx\.conf:25\):1: ssl fetch sess by lua is running!, server_tokens off; } +--- skip_eval: 6:$ENV{TEST_NGINX_USE_BORINGSSL} --- config server_tokens off; resolver $TEST_NGINX_RESOLVER ipv6=off; lua_ssl_trusted_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; - lua_ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; + lua_ssl_protocols TLSv1 TLSv1.1 TLSV1.2 TLSv1.3; location /t { set $port $TEST_NGINX_MEMCACHED_PORT; diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index a356b6eeaa..680011d091 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -12,12 +12,15 @@ if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); } elsif ($openssl_version =~ m/running with BoringSSL/) { plan(skip_all => "does not support BoringSSL"); +} elsif ($ENV{TEST_NGINX_USE_HTTP3}) { + plan tests => repeat_each() * (blocks() * 6 + 6); } else { - plan tests => repeat_each() * (blocks() * 6 + 8); + plan tests => repeat_each() * (blocks() * 6 + 10); } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; +$ENV{TEST_NGINX_QUIC_IDLE_TIMEOUT} ||= 0.6; #log_level 'warn'; log_level 'debug'; @@ -1667,11 +1670,11 @@ ssl client hello by lua is running! === TEST 20: use ssl_client_hello_by_lua* on the http {} level with non-ssl server --- http_config - ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } ssl_certificate ../../cert/test.crt; ssl_certificate_key ../../cert/test.key; server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + ssl_client_hello_by_lua_block { print("ssl client hello by lua is running!") } server_name test.com; server_tokens off; location /foo { @@ -1743,7 +1746,6 @@ close: 1 nil ssl client hello by lua is running! [error] [alert] ---- skip_eval: 5:$ENV{TEST_NGINX_USE_HTTP3} @@ -2323,7 +2325,7 @@ received: foo close: 1 nil --- error_log -client socket file: +client socket file: --- no_error_log [error] @@ -2637,3 +2639,815 @@ qr/\[debug\] .*? SSL_do_handshake: 1/, should never reached here [alert] [emerg] + + + +=== TEST 31: ssl_client_hello_by_lua* can yield when reading early data +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello by lua is running!") + ngx.sleep(0.1) + print("ssl client hello by lua is done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello by lua is running!|ssl client hello by lua is done|test completed)/ +--- grep_error_log_out +ssl client hello by lua is running! +ssl client hello by lua is done +test completed +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 32: ssl_client_hello_by_lua* with TCP cosocket (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: TCP cosocket test start") + + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local bytes, err = sock:send("flush_all\r\n") + if not bytes then + ngx.log(ngx.ERR, "failed to send flush_all command: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + print("ssl client hello: received TCP memc reply: ", res) + sock:close() + print("ssl client hello: TCP cosocket test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: TCP cosocket test start|ssl client hello: received TCP memc reply: OK|ssl client hello: TCP cosocket test done|test completed)/ +--- grep_error_log_out +ssl client hello: TCP cosocket test start +ssl client hello: received TCP memc reply: OK +ssl client hello: TCP cosocket test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 33: ssl_client_hello_by_lua* with UDP cosocket (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: UDP cosocket test start") + + local sock = ngx.socket.udp() + sock:settimeout(1000) + + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "failed to connect to memc: ", err) + return + end + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = sock:send(req) + if not ok then + ngx.log(ngx.ERR, "failed to send flush_all to memc: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "failed to receive memc reply: ", err) + return + end + + print("ssl client hello: received UDP memc reply of ", #res, " bytes") + sock:close() + print("ssl client hello: UDP cosocket test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: UDP cosocket test start|ssl client hello: received UDP memc reply of \d+ bytes|ssl client hello: UDP cosocket test done|test completed)/ +--- grep_error_log_out +ssl client hello: UDP cosocket test start +ssl client hello: received UDP memc reply of 12 bytes +ssl client hello: UDP cosocket test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 34: ssl_client_hello_by_lua* with ngx.timer (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: timer test start") + + local function timer_handler() + print("ssl client hello: timer executed") + end + + local ok, err = ngx.timer.at(0, timer_handler) + if not ok then + ngx.log(ngx.ERR, "failed to create timer: ", err) + return + end + + print("ssl client hello: timer created") + print("ssl client hello: timer test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: timer test start|ssl client hello: timer created|ssl client hello: timer test done|ssl client hello: timer executed|test completed)/ +--- grep_error_log_out +ssl client hello: timer test start +ssl client hello: timer created +ssl client hello: timer test done +ssl client hello: timer executed +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 35: ssl_client_hello_by_lua* with user threads (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: uthread test start") + + local function worker() + ngx.sleep(0.01) + print("ssl client hello: uthread worker executed") + return "worker_result" + end + + local t, err = ngx.thread.spawn(worker) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return + end + + print("ssl client hello: uthread spawned") + + local ok, res = ngx.thread.wait(t) + if not ok then + ngx.log(ngx.ERR, "failed to wait thread: ", res) + return + end + + print("ssl client hello: uthread result: ", res) + print("ssl client hello: uthread test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: uthread test start|ssl client hello: uthread spawned|ssl client hello: uthread worker executed|ssl client hello: uthread result: worker_result|ssl client hello: uthread test done|test completed)/ +--- grep_error_log_out +ssl client hello: uthread test start +ssl client hello: uthread spawned +ssl client hello: uthread worker executed +ssl client hello: uthread result: worker_result +ssl client hello: uthread test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 36: ssl_client_hello_by_lua* with coroutines (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: coroutine test start") + + local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield + + local function coro_func() + local cnt = 0 + for i = 1, 3 do + print("ssl client hello: coro yield: ", cnt) + cy() + cnt = cnt + 1 + end + return "coro_done" + end + + local c = cc(coro_func) + for i = 1, 4 do + print("ssl client hello: coro resume, status: ", coroutine.status(c)) + local ok, res = cr(c) + if not ok then + print("ssl client hello: coro error: ", res) + break + end + if coroutine.status(c) == "dead" then + print("ssl client hello: coro result: ", res) + break + end + end + + print("ssl client hello: coroutine test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: coroutine test start|ssl client hello: coro resume, status: \w+|ssl client hello: coro yield: \d+|ssl client hello: coro result: coro_done|ssl client hello: coroutine test done|test completed)/ +--- grep_error_log_out +ssl client hello: coroutine test start +ssl client hello: coro resume, status: suspended +ssl client hello: coro yield: 0 +ssl client hello: coro resume, status: suspended +ssl client hello: coro yield: 1 +ssl client hello: coro resume, status: suspended +ssl client hello: coro yield: 2 +ssl client hello: coro resume, status: suspended +ssl client hello: coro result: coro_done +ssl client hello: coroutine test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 37: ssl_client_hello_by_lua* without yield API (simple logic) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: simple test start") + + -- Simple calculations without yield + local sum = 0 + for i = 1, 10 do + sum = sum + i + end + + print("ssl client hello: calculated sum: ", sum) + + -- String operations + local str = "hello" + str = str .. " world" + print("ssl client hello: concatenated string: ", str) + + -- Table operations + local t = {a = 1, b = 2, c = 3} + local count = 0 + for k, v in pairs(t) do + count = count + v + end + print("ssl client hello: table sum: ", count) + + print("ssl client hello: simple test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: simple test start|ssl client hello: calculated sum: 55|ssl client hello: concatenated string: hello world|ssl client hello: table sum: 6|ssl client hello: simple test done|test completed)/ +--- grep_error_log_out +ssl client hello: simple test start +ssl client hello: calculated sum: 55 +ssl client hello: concatenated string: hello world +ssl client hello: table sum: 6 +ssl client hello: simple test done +test completed + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 38: ssl_client_hello_by_lua* with multiple network operations (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("ssl client hello: multiple network test start") + + -- First TCP operation + local sock1 = ngx.socket.tcp() + sock1:settimeout(2000) + local ok, err = sock1:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if ok then + local bytes, err = sock1:send("version\r\n") + if bytes then + local res, err = sock1:receive() + if res then + print("ssl client hello: TCP1 version: ", res) + end + end + sock1:close() + end + + ngx.sleep(0.01) -- Small delay + + -- Second UDP operation + local sock2 = ngx.socket.udp() + sock2:settimeout(1000) + local ok, err = sock2:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if ok then + local req = "\0\1\0\0\0\1\0\0version\r\n" + local ok, err = sock2:send(req) + if ok then + local res, err = sock2:receive() + if res then + print("ssl client hello: UDP version reply length: ", #res) + end + end + sock2:close() + end + + print("ssl client hello: multiple network test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(ssl client hello: multiple network test start|ssl client hello: TCP1 version: VERSION|ssl client hello: UDP version reply length: \d+|ssl client hello: multiple network test done|test completed)/ +--- grep_error_log_out eval +[ +qr/ssl client hello: multiple network test start/, +qr/ssl client hello: TCP1 version: VERSION/, +qr/ssl client hello: UDP version reply length: \d+/, +qr/ssl client hello: multiple network test done/, +qr/test completed/, +] + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 39: ssl_cert_by_lua* and ssl_client_hello_by_lua* with sleep (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- http_config + ssl_certificate_by_lua_block { + print("cert by: starting with sleep") + local begin = ngx.now() + ngx.sleep(0.05) + print("cert by: slept for ", ngx.now() - begin, " seconds") + } + +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("client hello: starting with sleep") + local begin = ngx.now() + ngx.sleep(0.1) + print("client hello: slept for ", ngx.now() - begin, " seconds") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(client hello: starting with sleep|client hello: slept for 0\.\d+|cert by: starting with sleep|cert by: slept for 0\.\d+|test completed)/ +--- grep_error_log_out eval +[ +qr/client hello: starting with sleep/, +qr/client hello: slept for 0\.(?:09|1\d)\d+/, +qr/cert by: starting with sleep/, +qr/cert by: slept for 0\.0[4-6]\d+/, +qr/test completed/, +] + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 40: ssl_cert_by_lua* and ssl_client_hello_by_lua* with cosocket (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- http_config + ssl_certificate_by_lua_block { + print("cert by: cosocket test start") + + local sock = ngx.socket.udp() + sock:settimeout(1000) + + local ok, err = sock:setpeername("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "cert by: failed to connect to memc: ", err) + return + end + + local req = "\0\1\0\0\0\1\0\0flush_all\r\n" + local ok, err = sock:send(req) + if not ok then + ngx.log(ngx.ERR, "cert by: failed to send flush_all to memc: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "cert by: failed to receive memc reply: ", err) + return + end + + print("cert by: received UDP memc reply of ", #res, " bytes") + sock:close() + print("cert by: cosocket test done") + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("client hello: cosocket test start") + + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "client hello: failed to connect to memc: ", err) + return + end + + local bytes, err = sock:send("version\r\n") + if not bytes then + ngx.log(ngx.ERR, "client hello: failed to send version command: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "client hello: failed to receive memc reply: ", err) + return + end + + print("client hello: received memc reply: ", res) + sock:close() + print("client hello: cosocket test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(client hello: cosocket test start|client hello: received memc reply: VERSION|client hello: cosocket test done|cert by: cosocket test start|cert by: received UDP memc reply of \d+ bytes|cert by: cosocket test done|test completed)/ +--- grep_error_log_out eval +[ +qr/client hello: cosocket test start/, +qr/client hello: received memc reply: VERSION/, +qr/client hello: cosocket test done/, +qr/cert by: cosocket test start/, +qr/cert by: received UDP memc reply of \d+ bytes/, +qr/cert by: cosocket test done/, +qr/test completed/, +] + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 41: ssl_cert_by_lua* and ssl_client_hello_by_lua* with timer and uthread (yield API) +--- skip_eval: 6:!$ENV{TEST_NGINX_USE_HTTP3} +--- http_config + ssl_certificate_by_lua_block { + print("cert by: coroutine test start") + + local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield + + local function coro_func() + local cnt = 0 + for i = 1, 2 do + print("cert by: coro yield: ", cnt) + cy() + cnt = cnt + 1 + end + return "cert_by_coro_done" + end + + local c = cc(coro_func) + for i = 1, 3 do + print("cert by: coro resume, status: ", coroutine.status(c)) + local ok, res = cr(c) + if not ok then + print("cert by: coro error: ", res) + break + end + if coroutine.status(c) == "dead" then + print("cert by: coro result: ", res) + break + end + end + + -- Small sleep to allow timer to execute + ngx.sleep(0.01) + print("cert by: coroutine test done") + } + +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("client hello: timer and uthread test start") + + -- Timer test + local function timer_handler() + print("client hello: timer executed") + end + + local ok, err = ngx.timer.at(0, timer_handler) + if not ok then + ngx.log(ngx.ERR, "client hello: failed to create timer: ", err) + return + end + + print("client hello: timer created") + + -- User thread test + local function worker() + ngx.sleep(0.01) + print("client hello: uthread worker executed") + return "client_hello_result" + end + + local t, err = ngx.thread.spawn(worker) + if not t then + ngx.log(ngx.ERR, "client hello: failed to spawn thread: ", err) + return + end + + print("client hello: uthread spawned") + + local ok, res = ngx.thread.wait(t) + if not ok then + ngx.log(ngx.ERR, "client hello: failed to wait thread: ", res) + return + end + + print("client hello: uthread result: ", res) + print("client hello: timer and uthread test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- response_body +test completed + +--- grep_error_log eval: qr/(client hello: timer and uthread test start|client hello: timer created|client hello: uthread spawned|client hello: uthread worker executed|client hello: uthread result: client_hello_result|client hello: timer and uthread test done|cert by: coroutine test start|cert by: coro resume, status: \w+|cert by: coro yield: \d+|cert by: coro result: cert_by_coro_done|cert by: coroutine test done|client hello: timer executed|test completed)/ +--- grep_error_log_out eval +[ +qr/client hello: timer and uthread test start/, +qr/client hello: timer created/, +qr/client hello: uthread spawned/, +qr/client hello: uthread worker executed/, +qr/client hello: uthread result: client_hello_result/, +qr/client hello: timer and uthread test done/, +qr/cert by: coroutine test start/, +qr/cert by: coro resume, status: suspended/, +qr/cert by: coro yield: 0/, +qr/cert by: coro resume, status: suspended/, +qr/cert by: coro yield: 1/, +qr/cert by: coro resume, status: suspended/, +qr/cert by: coro result: cert_by_coro_done/, +qr/cert by: coroutine test done/, +qr/client hello: timer executed/, +qr/test completed/, +] + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 42: ssl_cert_by_lua* with cosocket (yield API) and ssl_client_hello_by_lua* failure +--- skip_eval: 8:!$ENV{TEST_NGINX_USE_HTTP3} +--- http_config + ssl_certificate_by_lua_block { + print("cert by: test start") + ngx.exit(500) + } +--- config + server_tokens off; + lua_ssl_trusted_certificate ../../cert/test.crt; + lua_ssl_verify_depth 3; + + ssl_client_hello_by_lua_block { + print("client hello: cosocket test start") + + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_MEMCACHED_PORT) + if not ok then + ngx.log(ngx.ERR, "client hello: failed to connect to memc: ", err) + return + end + + local bytes, err = sock:send("version\r\n") + if not bytes then + ngx.log(ngx.ERR, "client hello: failed to send version command: ", err) + return + end + + local res, err = sock:receive() + if not res then + ngx.log(ngx.ERR, "client hello: failed to receive memc reply: ", err) + return + end + + print("client hello: received memc reply: ", res) + sock:close() + print("client hello: cosocket test done") + } + + location /t { + content_by_lua_block { + print("test completed") + ngx.say("test completed") + } + } +--- request +GET /t +--- ignore_response +--- curl_error eval +qr/Connection time/ +--- grep_error_log eval: qr/(client hello: cosocket test start|client hello: received memc reply: VERSION|client hello: cosocket test done|cert by: test start)/ +--- grep_error_log_out eval +[ +qr/client hello: cosocket test start/, +qr/client hello: received memc reply: VERSION/, +qr/client hello: cosocket test done/, +qr/cert by: test start/, +] + +--- no_error_log +[error] +[alert] +[emerg] From 25ebf2072de6065dc1e2f5bfe97801fa9a9adeb6 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 30 Jun 2025 23:37:16 +0800 Subject: [PATCH 821/848] tests: add utility script run-ci.sh --- README.markdown | 19 ++++- util/run-ci.sh | 193 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 211 insertions(+), 1 deletion(-) create mode 100755 util/run-ci.sh diff --git a/README.markdown b/README.markdown index 117a9e484b..2db20f3e05 100644 --- a/README.markdown +++ b/README.markdown @@ -45,6 +45,7 @@ Table of Contents * [Missing data on short circuited requests](#missing-data-on-short-circuited-requests) * [TODO](#todo) * [Changes](#changes) +* [Build And Test](#build-and-test) * [Test Suite](#test-suite) * [Copyright and License](#copyright-and-license) * [See Also](#see-also) @@ -982,6 +983,23 @@ The changes made in every release of this module are listed in the change logs o [Back to TOC](#table-of-contents) +Build And Test +============== + +This module uses `.travis.yml` as the CI configuration. +You can always check `.travis.yml` for the latest CI configuration. + +For developers, you need to run tests locally. You can use `util/run-ci.sh` +to easily set up the environment and execute the test suite. + +To run the Test from the beginning: + +```shell +git clone https://github.com/openresty/lua-nginx-module.git +cd lua-nginx-module +bash util/run-ci.sh +``` + Test Suite ========== @@ -1026,7 +1044,6 @@ To run the whole test suite in the default testing mode: export PATH=/path/to/your/nginx/sbin:$PATH prove -I/path/to/test-nginx/lib -r t - To run specific test files: cd /path/to/lua-nginx-module diff --git a/util/run-ci.sh b/util/run-ci.sh new file mode 100755 index 0000000000..b5e5407c17 --- /dev/null +++ b/util/run-ci.sh @@ -0,0 +1,193 @@ +#!/bin/bash + +#export CC=clang +export CC=gcc +export NGX_BUILD_CC=$CC + +mkdir -p download-cache + +export JOBS=$(nproc) +export NGX_BUILD_JOBS=$JOBS +export VALGRIND_INC=/usr/include/valgrind/ + +export LUAJIT_PREFIX=/opt/luajit21 +export LUAJIT_LIB=$LUAJIT_PREFIX/lib +export LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 +export LUA_INCLUDE_DIR=$LUAJIT_INC + +export PCRE2_VER=10.45 +export PCRE2_PREFIX=/opt/pcre2 +export PCRE2_LIB=$PCRE2_PREFIX/lib +export PCRE2_INC=$PCRE2_PREFIX/include + +export OPENSSL_VER=3.5.0 +export OPENSSL_PATCH_VER=3.5.0 +export OPENSSL_PREFIX=/opt/ssl3 +export OPENSSL_LIB=$OPENSSL_PREFIX/lib +export OPENSSL_INC=$OPENSSL_PREFIX/include + +export LIBDRIZZLE_PREFIX=/opt/drizzle +export LIBDRIZZLE_INC=$LIBDRIZZLE_PREFIX/include/libdrizzle-1.0 +export LIBDRIZZLE_LIB=$LIBDRIZZLE_PREFIX/lib +export DRIZZLE_VER=2011.07.21 + +#export TEST_NGINX_SLEEP=0.006 +export NGINX_VERSION=1.27.1 +#export NGX_BUILD_ASAN=1 + +export PATH=/opt/bin:$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH + +if [ ! -f /opt/bin/curl ]; then + wget https://github.com/stunnel/static-curl/releases/download/8.14.1/curl-linux-x86_64-glibc-8.14.1.tar.xz + tar -xf curl-linux-x86_64-glibc-8.14.1.tar.xz + tar -xf curl-linux-x86_64-glibc-8.14.1.tar.xz + sudo mkdir -p /opt/bin + sudo mv curl /opt/bin/ +fi + +function git_download() +{ + dir=${!#} + if [ ! -d $dir ]; then + git clone $@ + fi +} + +function download_deps() +{ + if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then + wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz + fi + + if [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then + wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz + fi + + if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then + wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz + fi + + git_download clone https://github.com/openresty/test-nginx.git + git_download clone https://github.com/openresty/openresty.git ../openresty + git_download clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx + git_download clone https://github.com/openresty/openresty-devel-utils.git + git_download clone https://github.com/openresty/mockeagain.git + git_download clone https://github.com/openresty/lua-cjson.git lua-cjson + git_download clone https://github.com/openresty/lua-upstream-nginx-module.git ../lua-upstream-nginx-module + git_download clone https://github.com/openresty/echo-nginx-module.git ../echo-nginx-module + git_download clone https://github.com/openresty/nginx-eval-module.git ../nginx-eval-module + git_download clone https://github.com/simpl/ngx_devel_kit.git ../ndk-nginx-module + git_download clone https://github.com/FRiCKLE/ngx_coolkit.git ../coolkit-nginx-module + git_download clone https://github.com/openresty/headers-more-nginx-module.git ../headers-more-nginx-module + git_download clone https://github.com/openresty/drizzle-nginx-module.git ../drizzle-nginx-module + git_download clone https://github.com/openresty/set-misc-nginx-module.git ../set-misc-nginx-module + git_download clone https://github.com/openresty/memc-nginx-module.git ../memc-nginx-module + git_download clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module + git_download clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module + git_download clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module + git_download clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core + git_download clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache + git_download clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql + git_download clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string + git_download clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module + git_download clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 +} + +function make_deps() +{ + if [ "$TEST_NGINX_BUILD_DEPS" = "n" ]; then + return + fi + + cd luajit2/ + make clean + make -j"$JOBS" CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS="-DLUAJIT_USE_VALGRIND -I$VALGRIND_INC -DLUAJIT_USE_SYSMALLOC -DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2" > build.log 2>&1 || (cat build.log && exit 1) + sudo make install PREFIX=$LUAJIT_PREFIX + cd .. + + tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz + cd openssl-$OPENSSL_VER/ + patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch > ssl.log + ./config -DOPENSSL_TLS_SECURITY_LEVEL=1 shared enable-ssl3 enable-ssl3-method -g -O2 --prefix=$OPENSSL_PREFIX --libdir=lib -DPURIFY >> ssl.log + make -j$JOBS >> ssl.log + sudo make PATH=$PATH install_sw >> ssl.log + cd .. + + tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; + cd pcre2-$PCRE2_VER/; + ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf + sudo PATH=$PATH make install + cd .. + + tar xzf download-cache/drizzle7-$DRIZZLE_VER.tar.gz && cd drizzle7-$DRIZZLE_VER + ./configure --prefix=$LIBDRIZZLE_PREFIX --without-server + make libdrizzle-1.0 -j$JOBS + sudo make install-libdrizzle-1.0 + cd .. + + cd mockeagain/ && make CC=$CC -j$JOBS + cd .. + + cd lua-cjson/ && make -j$JOBS && sudo make install + cd .. +} + +function make_ngx() +{ + if [ "$TEST_NGINX_FRESH_BUILD" = "y" ]; then + rm -fr buildroot + fi + + sh util/build.sh $NGINX_VERSION 2>&1 | tee build.log +} + +download_deps +make_deps +make_ngx + +find t -name "*.t" | xargs reindex >/dev/null 2>&1 + +#nginx -V + +export LD_PRELOAD=$PWD/mockeagain/mockeagain.so +export LD_LIBRARY_PATH=$PWD/mockeagain:$LD_LIBRARY_PATH + +#export ASAN_OPTIONS=detect_leaks=0,log_path=/tmp/asan/asan,log_exe_name=true +#export LD_PRELOAD=/lib64/libasan.so.5:$PWD/mockeagain/mockeagain.so + +export LD_LIBRARY_PATH=$LUAJIT_LIB:$OPENSSL_LIB:$LD_LIBRARY_PATH +export TEST_NGINX_RESOLVER=8.8.4.4 + +#export TEST_NGINX_NO_CLEAN=1 +#export TEST_NGINX_CHECK_LEAK=1 +#export TEST_NGINX_CHECK_LEAK_COUNT=100 +#export TEST_NGINX_TIMEOUT=5 + +#export TEST_NGINX_USE_VALGRIND=1 +#export TEST_NGINX_VALGRIND_EXIT_ON_FIRST_ERR=1 +#export TEST_NGINX_USE_HTTP2=1 + +export MALLOC_PERTURB_=33 +export TEST_NGINX_HTTP3_CRT=$PWD/t/cert/http3/http3.crt +export TEST_NGINX_HTTP3_KEY=$PWD/t/cert/http3/http3.key +#export TEST_NGINX_USE_HTTP3=1 + +#export TEST_NGINX_VERBOSE=1 + +#export TEST_NGINX_EVENT_TYPE=poll +#export TEST_NGINX_POSTPONE_OUTPUT=1 +#export MOCKEAGAIN=r +#export MOCKEAGAIN=w +#export MOCKEAGAIN=rw +#export MOCKEAGAIN_VERBOSE=1 + +ldd `which nginx`|grep -E 'luajit|ssl|pcre' +which nginx +nginx -V + +#export TEST_NGINX_INIT_BY_LUA="debug.sethook(function () collectgarbage() end, 'l') jit.off() package.path = '/usr/share/lua/5.1/?.lua;$PWD/../lua-resty-core/lib/?.lua;$PWD/../lua-resty-lrucache/lib/?.lua;' .. (package.path or '') require 'resty.core' require('resty.core.base').set_string_buf_size(1) require('resty.core.regex').set_buf_grow_ratio(1)" + + +# comment out TEST_NGINX_RANDOMIZE when debugging one test case +export TEST_NGINX_RANDOMIZE=1 +prove -j$JOBS -I. -Itest-nginx/inc -Itest-nginx/lib -r t/ From 9aed90f4707bfdc2df443aec5d6bbcd3489def12 Mon Sep 17 00:00:00 2001 From: willmafh Date: Tue, 1 Jul 2025 21:40:19 +0800 Subject: [PATCH 822/848] feature: add lua_ssl_key_log directive. --- README.markdown | 14 +++++ doc/HttpLuaModule.wiki | 10 ++++ src/ngx_http_lua_common.h | 1 + src/ngx_http_lua_module.c | 117 ++++++++++++++++++++++++++++++++++++++ src/ngx_http_lua_ssl.c | 12 ++++ src/ngx_http_lua_ssl.h | 8 +++ t/129-ssl-socket.t | 102 +++++++++++++++++++++++++++++++++ 7 files changed, 264 insertions(+) diff --git a/README.markdown b/README.markdown index 2db20f3e05..1565487d93 100644 --- a/README.markdown +++ b/README.markdown @@ -1186,6 +1186,7 @@ Directives * [lua_ssl_certificate_key](#lua_ssl_certificate_key) * [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate) * [lua_ssl_verify_depth](#lua_ssl_verify_depth) +* [lua_ssl_key_log](#lua_ssl_key_log) * [lua_ssl_conf_command](#lua_ssl_conf_command) * [lua_http10_buffering](#lua_http10_buffering) * [rewrite_by_lua_no_postpone](#rewrite_by_lua_no_postpone) @@ -3447,6 +3448,19 @@ See also [lua_ssl_certificate](#lua_ssl_certificate), [lua_ssl_certificate_key]( [Back to TOC](#directives) +lua_ssl_key_log +--------------- + +**syntax:** *lua_ssl_key_log <file>* + +**default:** *none* + +**context:** *http, server, location* + +Enables logging of client connection SSL keys in the [tcpsock:sslhandshake](#tcpsocksslhandshake) method and specifies the path to the key log file. Keys are logged in the SSLKEYLOGFILE format compatible with Wireshark. + +[Back to TOC](#directives) + lua_ssl_conf_command -------------------- diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 733f11dc66..69e1a86303 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2925,6 +2925,16 @@ This directive was first introduced in the v0.9.11 release. See also [[#lua_ssl_certificate|lua_ssl_certificate]], [[#lua_ssl_certificate_key|lua_ssl_certificate_key]] and [[#lua_ssl_trusted_certificate|lua_ssl_trusted_certificate]]. +== lua_ssl_key_log == + +'''syntax:''' ''lua_ssl_key_log '' + +'''default:''' ''none'' + +'''context:''' ''http, server, location'' + +Enables logging of client connection SSL keys in the [[#tcpsock:sslhandshake|tcpsock:sslhandshake]] method and specifies the path to the key log file. Keys are logged in the SSLKEYLOGFILE format compatible with Wireshark. + == lua_ssl_conf_command == '''syntax:''' ''lua_ssl_conf_command '' diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index cc2d36a386..9db3e19ce5 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -379,6 +379,7 @@ typedef struct { ngx_uint_t ssl_verify_depth; ngx_str_t ssl_trusted_certificate; ngx_str_t ssl_crl; + ngx_str_t ssl_key_log; #if (nginx_version >= 1019004) ngx_array_t *ssl_conf_commands; #endif diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index 0f23c2d769..cae92b21ee 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -54,6 +54,11 @@ static ngx_int_t ngx_http_lua_merge_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *conf, ngx_http_lua_loc_conf_t *prev); static ngx_int_t ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf); +static void key_log_callback(const ngx_ssl_conn_t *ssl_conn, + const char *line); +static void ngx_http_lua_ssl_cleanup_key_log(void *data); +static ngx_int_t ngx_http_lua_ssl_key_log(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_str_t *file); #if (nginx_version >= 1019004) static char *ngx_http_lua_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); @@ -690,6 +695,13 @@ static ngx_command_t ngx_http_lua_cmds[] = { offsetof(ngx_http_lua_loc_conf_t, ssl_crl), NULL }, + { ngx_string("lua_ssl_key_log"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_lua_loc_conf_t, ssl_key_log), + NULL }, + #if (nginx_version >= 1019004) { ngx_string("lua_ssl_conf_command"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, @@ -1433,6 +1445,7 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) * conf->ssl_ciphers = { 0, NULL }; * conf->ssl_trusted_certificate = { 0, NULL }; * conf->ssl_crl = { 0, NULL }; + * conf->ssl_key_log = { 0, NULL }; */ conf->force_read_body = NGX_CONF_UNSET; @@ -1553,6 +1566,7 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) ngx_conf_merge_str_value(conf->ssl_trusted_certificate, prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); + ngx_conf_merge_str_value(conf->ssl_key_log, prev->ssl_key_log, ""); #if (nginx_version >= 1019004) ngx_conf_merge_ptr_value(conf->ssl_conf_commands, prev->ssl_conf_commands, @@ -1616,6 +1630,7 @@ ngx_http_lua_merge_ssl(ngx_conf_t *cf, && conf->ssl_certificate_keys == NGX_CONF_UNSET_PTR && conf->ssl_trusted_certificate.data == NULL && conf->ssl_crl.data == NULL + && conf->ssl_key_log.data == NULL #if (nginx_version >= 1019004) && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR #endif @@ -1723,6 +1738,12 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) return NGX_ERROR; } + if (ngx_http_lua_ssl_key_log(cf, llcf->ssl, &llcf->ssl_key_log) + != NGX_OK) + { + return NGX_ERROR; + } + #if (nginx_version >= 1019004) if (ngx_ssl_conf_commands(cf, llcf->ssl, llcf->ssl_conf_commands) != NGX_OK) @@ -1734,6 +1755,102 @@ ngx_http_lua_set_ssl(ngx_conf_t *cf, ngx_http_lua_loc_conf_t *llcf) return NGX_OK; } + +static void +key_log_callback(const ngx_ssl_conn_t *ssl_conn, const char *line) +{ + ngx_http_lua_ssl_key_log_t *ssl_key_log; + ngx_connection_t *c; + + ssl_key_log = SSL_CTX_get_ex_data(SSL_get_SSL_CTX(ssl_conn), + ngx_http_lua_ssl_key_log_index); + if (ssl_key_log == NULL) { + c = ngx_ssl_get_connection((ngx_ssl_conn_t *) ssl_conn); + ngx_ssl_error(NGX_LOG_DEBUG, c->log, 0, "get ssl key log failed"); + + return; + } + + (void) ngx_write_fd(ssl_key_log->fd, (void *) line, ngx_strlen(line)); + (void) ngx_write_fd(ssl_key_log->fd, (void *) "\n", 1); +} + + +static void +ngx_http_lua_ssl_cleanup_key_log(void *data) +{ + ngx_http_lua_ssl_key_log_t *ssl_key_log = data; + + if (ngx_close_file(ssl_key_log->fd) == NGX_FILE_ERROR) { + ngx_ssl_error(NGX_LOG_ALERT, ssl_key_log->ssl->log, 0, + ngx_close_file_n "(\"%V\") failed", ssl_key_log->name); + } +} + + +static ngx_int_t +ngx_http_lua_ssl_key_log(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file) +{ + ngx_fd_t fd; + ngx_http_lua_ssl_key_log_t *ssl_key_log; + ngx_pool_cleanup_t *cln; + + if (!file->len) { + return NGX_OK; + } + + if (ngx_conf_full_name(cf->cycle, file, 1) != NGX_OK) { + return NGX_ERROR; + } + + if (ngx_http_lua_ssl_init(cf->log) != NGX_OK) { + return NGX_ERROR; + } + + /* + * append so that existing keylog file contents can be preserved + */ + fd = ngx_open_file(file->data, NGX_FILE_APPEND, NGX_FILE_CREATE_OR_OPEN, + NGX_FILE_DEFAULT_ACCESS); + if (fd == NGX_INVALID_FILE) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, ngx_open_file_n + "(\"%V\") failed", file); + return NGX_ERROR; + } + + ssl_key_log = ngx_palloc(cf->pool, sizeof(ngx_http_lua_ssl_key_log_t)); + if (ssl_key_log == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ngx_pcalloc() failed"); + return NGX_ERROR; + } + + ssl_key_log->ssl = ssl; + ssl_key_log->fd = fd; + ssl_key_log->name = *file; + + if (SSL_CTX_set_ex_data(ssl->ctx, ngx_http_lua_ssl_key_log_index, + ssl_key_log) == 0) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_set_ex_data() failed"); + return NGX_ERROR; + } + + cln = ngx_pool_cleanup_add(cf->pool, 0); + if (cln == NULL) { + ngx_http_lua_ssl_cleanup_key_log(ssl_key_log); + return NGX_ERROR; + } + + cln->handler = ngx_http_lua_ssl_cleanup_key_log; + cln->data = ssl_key_log; + + SSL_CTX_set_keylog_callback(ssl->ctx, key_log_callback); + + return NGX_OK; +} + + #if (nginx_version >= 1019004) static char * ngx_http_lua_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data) diff --git a/src/ngx_http_lua_ssl.c b/src/ngx_http_lua_ssl.c index 8ed7b95417..25e4f1b456 100644 --- a/src/ngx_http_lua_ssl.c +++ b/src/ngx_http_lua_ssl.c @@ -14,6 +14,7 @@ int ngx_http_lua_ssl_ctx_index = -1; +int ngx_http_lua_ssl_key_log_index = -1; ngx_int_t @@ -30,6 +31,17 @@ ngx_http_lua_ssl_init(ngx_log_t *log) } } + if (ngx_http_lua_ssl_key_log_index == -1) { + ngx_http_lua_ssl_key_log_index = SSL_get_ex_new_index(0, NULL, NULL, + NULL, NULL); + + if (ngx_http_lua_ssl_key_log_index == -1) { + ngx_ssl_error(NGX_LOG_ALERT, log, 0, + "lua: SSL_get_ex_new_index() for key log failed"); + return NGX_ERROR; + } + } + return NGX_OK; } diff --git a/src/ngx_http_lua_ssl.h b/src/ngx_http_lua_ssl.h index 3d577c61fb..1ee0a3626c 100644 --- a/src/ngx_http_lua_ssl.h +++ b/src/ngx_http_lua_ssl.h @@ -41,10 +41,18 @@ typedef struct { } ngx_http_lua_ssl_ctx_t; +typedef struct { + ngx_ssl_t *ssl; + ngx_fd_t fd; + ngx_str_t name; +} ngx_http_lua_ssl_key_log_t; + + ngx_int_t ngx_http_lua_ssl_init(ngx_log_t *log); extern int ngx_http_lua_ssl_ctx_index; +extern int ngx_http_lua_ssl_key_log_index; #endif diff --git a/t/129-ssl-socket.t b/t/129-ssl-socket.t index 8a05306ab0..5651f3f817 100644 --- a/t/129-ssl-socket.t +++ b/t/129-ssl-socket.t @@ -2944,3 +2944,105 @@ SSL reused session [alert] [emerg] --- timeout: 10 + + + +=== TEST 35: lua_ssl_key_log directive +--- skip_openssl: 8: < 1.1.1 +--- http_config + server { + listen $TEST_NGINX_SERVER_SSL_PORT ssl; + server_name test.com; + ssl_certificate $TEST_NGINX_CERT_DIR/cert/test.crt; + ssl_certificate_key $TEST_NGINX_CERT_DIR/cert/test.key; + ssl_protocols TLSv1.3; + + location / { + content_by_lua_block { + ngx.exit(200) + } + } + } +--- config + server_tokens off; + lua_ssl_protocols TLSv1.3; + lua_ssl_key_log sslkey.log; + + location /t { + content_by_lua_block { + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + do + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_SERVER_SSL_PORT) + if not ok then + ngx.say("failed to connect: ", err) + return + end + + ngx.say("connected: ", ok) + + local session, err = sock:sslhandshake(nil, "test.com") + if not session then + ngx.say("failed to do SSL handshake: ", err) + return + end + + ngx.say("ssl handshake: ", type(session)) + + local req = "GET / HTTP/1.1\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.say("failed to send http request: ", err) + return + end + + ngx.say("sent http request: ", bytes, " bytes.") + + local line, err = sock:receive() + if not line then + ngx.say("failed to receive response status line: ", err) + return + end + + ngx.say("received: ", line) + + local ok, err = sock:close() + ngx.say("close: ", ok, " ", err) + + local f, err = io.open("$TEST_NGINX_SERVER_ROOT/conf/sslkey.log", "r") + if not f then + ngx.log(ngx.ERR, "failed to open sslkey.log: ", err) + return + end + + local key_log = f:read("*a") + ngx.say(key_log) + f:close() + end -- do + collectgarbage() + } + } +--- request +GET /t +--- response_body_like +connected: 1 +ssl handshake: cdata +sent http request: 53 bytes. +received: HTTP/1.1 200 OK +close: 1 nil +SERVER_HANDSHAKE_TRAFFIC_SECRET [0-9a-z\s]+ +EXPORTER_SECRET [0-9a-z\s]+ +SERVER_TRAFFIC_SECRET_0 [0-9a-z\s]+ +CLIENT_HANDSHAKE_TRAFFIC_SECRET [0-9a-z\s]+ +CLIENT_TRAFFIC_SECRET_0 [0-9a-z\s]+ + +--- log_level: debug +--- error_log eval +['lua ssl server name: "test.com"', +qr/SSL: TLSv1.3, cipher: "TLS_AES_256_GCM_SHA384 TLSv1.3/] +--- no_error_log +SSL reused session +[error] +[alert] +--- timeout: 10 From 83ae875b306cd94bd4759ded06e1a60b08cd2481 Mon Sep 17 00:00:00 2001 From: Sunny Chan Date: Wed, 2 Jul 2025 14:46:00 +0800 Subject: [PATCH 823/848] feature: add ngx_http_lua_ffi_req_shared_ssl_ciphers(). MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This function returns a uint16_t array of tls_protocol_id supported by both server and client. Co-author-by: Bjørnar Ness . --- src/ngx_http_lua_ssl_certby.c | 80 +++++++++ t/140-ssl-c-api.t | 305 +++++++++++++++++++++++++++++++++- 2 files changed, 383 insertions(+), 2 deletions(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 0c13af91c4..c5426da565 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -37,6 +37,10 @@ static u_char *ngx_http_lua_log_ssl_cert_error(ngx_log_t *log, u_char *buf, static ngx_int_t ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r); +#ifndef OPENSSL_IS_BORINGSSL +static int ngx_http_lua_is_grease_cipher(uint16_t cipher_id); +#endif + ngx_int_t ngx_http_lua_ssl_cert_handler_file(ngx_http_request_t *r, @@ -450,6 +454,18 @@ ngx_http_lua_log_ssl_cert_error(ngx_log_t *log, u_char *buf, size_t len) } +#ifndef OPENSSL_IS_BORINGSSL +static int +ngx_http_lua_is_grease_cipher(uint16_t cipher_id) +{ + /* GREASE values follow pattern: 0x?A?A where ? can be any hex digit */ + /* and both ? must be the same */ + /* Check if both bytes follow ?A pattern and high nibbles match */ + return (cipher_id & 0x0F0F) == 0x0A0A; +} +#endif + + static ngx_int_t ngx_http_lua_ssl_cert_by_chunk(lua_State *L, ngx_http_request_t *r) { @@ -837,6 +853,70 @@ ngx_http_lua_ffi_ssl_raw_server_addr(ngx_http_request_t *r, char **addr, } +int +ngx_http_lua_ffi_req_shared_ssl_ciphers(ngx_http_request_t *r, + uint16_t *ciphers, uint16_t *nciphers, int filter_grease, char **err) +{ +#ifdef OPENSSL_IS_BORINGSSL + + *err = "BoringSSL is not supported for SSL cipher operations"; + return NGX_ERROR; + +#else + ngx_ssl_conn_t *ssl_conn; + STACK_OF(SSL_CIPHER) *sk, *ck; + int sn, cn, i, n; + uint16_t cipher; + + if (r == NULL || r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + ssl_conn = r->connection->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + sk = SSL_get1_supported_ciphers(ssl_conn); + ck = SSL_get_client_ciphers(ssl_conn); + sn = sk_SSL_CIPHER_num(sk); + cn = sk_SSL_CIPHER_num(ck); + + if (sn > *nciphers) { + *err = "buffer too small"; + *nciphers = 0; + sk_SSL_CIPHER_free(sk); + return NGX_ERROR; + } + + for (*nciphers = 0, i = 0; i < sn; i++) { + cipher = SSL_CIPHER_get_protocol_id(sk_SSL_CIPHER_value(sk, i)); + + /* Skip GREASE ciphers if filtering is enabled */ + if (filter_grease && ngx_http_lua_is_grease_cipher(cipher)) { + continue; + } + + for (n = 0; n < cn; n++) { + if (SSL_CIPHER_get_protocol_id(sk_SSL_CIPHER_value(ck, n)) + == cipher) + { + ciphers[(*nciphers)++] = cipher; + break; + } + } + } + + sk_SSL_CIPHER_free(sk); + + return NGX_OK; +#endif + +} + + int ngx_http_lua_ffi_ssl_server_name(ngx_http_request_t *r, char **name, size_t *namelen, char **err) diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 81d8375bb5..15ed8abfcf 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -10,9 +10,12 @@ my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); - +} elsif ($openssl_version =~ m/BoringSSL/) { + $ENV{TEST_NGINX_USE_BORINGSSL} = 1; + plan tests => repeat_each() * (blocks() * 6 - 8); } else { - plan tests => repeat_each() * (blocks() * 5 - 1); + plan tests => repeat_each() * (blocks() * 5 - 5); + $ENV{TEST_NGINX_USE_OPENSSL} = 1; } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -77,6 +80,8 @@ ffi.cdef[[ int ngx_http_lua_ffi_ssl_client_random(ngx_http_request_t *r, unsigned char *out, size_t *outlen, char **err); + int ngx_http_lua_ffi_req_shared_ssl_ciphers(void *r, uint16_t *ciphers, + uint16_t *nciphers, int filter_grease, char **err); ]] _EOC_ } @@ -1212,6 +1217,7 @@ lua ssl server name: "test.com" === TEST 10: Raw SSL pointer +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} --- http_config server { listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; @@ -1777,3 +1783,298 @@ SUCCESS --- no_error_log [error] [alert] + + + +=== TEST 15: Get supported ciphers +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_protocols TLSv1.2; + ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384; + + server_tokens off; + + location /ciphers { + content_by_lua_block { + require "defines" + local ffi = require "ffi" + local cjson = require "cjson.safe" + local base = require "resty.core.base" + local get_request = base.get_request + + local MAX_CIPHERS = 64 + local ciphers = ffi.new("uint16_t[?]", MAX_CIPHERS) + local nciphers = ffi.new("uint16_t[1]", MAX_CIPHERS) + local err = ffi.new("char*[1]") + + local r = get_request() + local ret = ffi.C.ngx_http_lua_ffi_req_shared_ssl_ciphers(r, ciphers, nciphers, 0, err) + + if ret ~= 0 then + ngx.log(ngx.ERR, "error: ", ffi.string(err[0])) + return + end + + local res = {} + for i = 0, nciphers[0] - 1 do + local cipher_id = string.format("%04x", ciphers[i]) + table.insert(res, cipher_id) + end + + ngx.say(cjson.encode(res)) + } + } + } +--- config + server_tokens off; + location /t { + proxy_ssl_protocols TLSv1.2; + proxy_ssl_session_reuse off; + proxy_ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256; + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock:/ciphers; + } +--- request +GET /t +--- response_body_like +\["c02f","c02b"\] +--- error_log chomp +TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD" + + + +=== TEST 16: SSL cipher API error handling (no SSL) +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} +--- config + location /t { + content_by_lua_block { + require "defines" + local ffi = require "ffi" + + local ciphers = ffi.new("uint16_t[64]") + local nciphers = ffi.new("uint16_t[1]", 64) + local err = ffi.new("char*[1]") + + -- use nil request to trigger error + local ret = ffi.C.ngx_http_lua_ffi_req_shared_ssl_ciphers(nil, ciphers, nciphers, 0, err) + + ngx.say("ret: ", ret) + if err[0] ~= nil then + ngx.say("err: ", ffi.string(err[0])) + end + } + } +--- request +GET /t +--- response_body +ret: -1 +err: bad request + +--- no_error_log +[error] +[alert] + + + +=== TEST 17: Buffer overflow handling +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_protocols TLSv1.2; + ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384; + + server_tokens off; + + + location /ciphers { + content_by_lua_block { + require "defines" + local ffi = require "ffi" + local base = require "resty.core.base" + local get_request = base.get_request + local cjson = require "cjson.safe" + + local MAX_CIPHERS = 64 + local ciphers = ffi.new("uint16_t[?]", MAX_CIPHERS) + local nciphers = ffi.new("uint16_t[1]", MAX_CIPHERS) + local err = ffi.new("char*[1]") + + local r = get_request() + local ret = ffi.C.ngx_http_lua_ffi_req_shared_ssl_ciphers(r, ciphers, nciphers, 0, err) + + if ret ~= 0 then + ngx.log(ngx.ERR, "error: ", ffi.string(err[0])) + return + end + local res = {} + for i = 0, nciphers[0] - 1 do + local cipher_id = string.format("%04x", ciphers[i]) + + table.insert(res, cipher_id) + end + ngx.say(cjson.encode(res)) + } + } + } +--- config + server_tokens off; + location /t { + proxy_ssl_protocols TLSv1.2; + proxy_ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256; + proxy_ssl_session_reuse off; + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock:/ciphers; + } +--- request +GET /t +--- response_body_like +\["c02f"\] +--- error_code: 200 +--- error_log chomp +TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD" + + + +=== TEST 18: BORINGSSL error handling +--- skip_eval: 8:$ENV{TEST_NGINX_USE_OPENSSL} +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_protocols TLSv1.2 TLSv1.3; + ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384; + + server_tokens off; + + + location /ciphers { + content_by_lua_block { + require "defines" + local ffi = require "ffi" + local base = require "resty.core.base" + local get_request = base.get_request + + local MAX_CIPHERS = 64 + local ciphers = ffi.new("uint16_t[?]", MAX_CIPHERS) + local nciphers = ffi.new("uint16_t[1]", MAX_CIPHERS) + local err = ffi.new("char*[1]") + + local r = get_request() + local ret = ffi.C.ngx_http_lua_ffi_req_shared_ssl_ciphers(r, ciphers, nciphers, 0, err) + + if ret ~= 0 then + ngx.say("Error: ", ffi.string(err[0])) + return + end + + } + } + } +--- config + server_tokens off; + location /t { + proxy_ssl_protocols TLSv1.2 TLSv1.3; + proxy_ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256; + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock:/ciphers; + } +--- request +GET /t +--- response_body_like chomp +Error: BoringSSL is not supported for SSL cipher operations +--- error_code: 200 + +--- no_error_log +[error] +[alert] + + + +=== TEST 19: Get supported ciphers with GREASE filtering +--- skip_eval: 8:$ENV{TEST_NGINX_USE_BORINGSSL} +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + ssl_certificate ../../cert/test.crt; + ssl_certificate_key ../../cert/test.key; + ssl_protocols TLSv1.2; + ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384; + + server_tokens off; + + location /ciphers { + content_by_lua_block { + require "defines" + local ffi = require "ffi" + local cjson = require "cjson.safe" + local base = require "resty.core.base" + local get_request = base.get_request + + local MAX_CIPHERS = 64 + local ciphers = ffi.new("uint16_t[?]", MAX_CIPHERS) + local nciphers = ffi.new("uint16_t[1]", MAX_CIPHERS) + local err = ffi.new("char*[1]") + + local r = get_request() + -- Test without GREASE filtering + local ret = ffi.C.ngx_http_lua_ffi_req_shared_ssl_ciphers(r, ciphers, nciphers, 0, err) + if ret ~= 0 then + ngx.log(ngx.ERR, "error without filtering: ", ffi.string(err[0])) + return + end + + local res_no_filter = {} + for i = 0, nciphers[0] - 1 do + local cipher_id = string.format("%04x", ciphers[i]) + table.insert(res_no_filter, cipher_id) + end + + -- Reset buffers + nciphers[0] = MAX_CIPHERS + + -- Test with GREASE filtering + local ret = ffi.C.ngx_http_lua_ffi_req_shared_ssl_ciphers(r, ciphers, nciphers, 1, err) + if ret ~= 0 then + ngx.log(ngx.ERR, "error with filtering: ", ffi.string(err[0])) + return + end + + local res_with_filter = {} + for i = 0, nciphers[0] - 1 do + local cipher_id = string.format("%04x", ciphers[i]) + table.insert(res_with_filter, cipher_id) + end + + ngx.say("without_filter:", cjson.encode(res_no_filter)) + ngx.say("with_filter:", cjson.encode(res_with_filter)) + } + } + } +--- config + server_tokens off; + location /t { + proxy_ssl_protocols TLSv1.2; + proxy_ssl_session_reuse off; + proxy_ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256; + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock:/ciphers; + } +--- request +GET /t +--- response_body_like +without_filter:\[.*\] +with_filter:\[.*\] +--- error_log chomp +TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD" + + + + From 715e636d22108cd8ef67a2d9db816b132fb08e3b Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 9 Jul 2025 09:04:06 +0800 Subject: [PATCH 824/848] tests: update t/142-ssl-session-store.t. --- t/142-ssl-session-store.t | 1 - 1 file changed, 1 deletion(-) diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index 11deb83207..ead2a167e5 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -975,4 +975,3 @@ qr/ssl_session_store_by_lua\*: skipped since TLS version >= 1\.3 \(\d+\)/ [error] [alert] [emerg] ---- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} From 5337179b2f4afa7e2842cd02bd293d2b9bc17665 Mon Sep 17 00:00:00 2001 From: willmafh Date: Wed, 16 Jul 2025 21:21:02 +0800 Subject: [PATCH 825/848] refactor: typo fixes. --- src/ngx_http_lua_ssl_client_helloby.c | 8 ++++---- t/166-ssl-client-hello.t | 10 +++++----- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index b4377f2f1b..81b7a01f51 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -159,7 +159,7 @@ ngx_http_lua_ssl_client_hello_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, } chunkname = ngx_http_lua_gen_chunk_name(cf, "ssl_client_hello_by_lua", - sizeof("ssl_client_hello_by_lua")- 1, + sizeof("ssl_client_hello_by_lua") - 1, &chunkname_len); if (chunkname == NULL) { return NGX_CONF_ERROR; @@ -207,7 +207,7 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, if (cctx->done) { ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua_client_hello_by_lua: " + "ssl_client_hello_by_lua: " "client hello cb exit code: %d", cctx->exit_code); @@ -310,7 +310,7 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, } ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua_client_hello_by_lua: handler return value: %i, " + "ssl_client_hello_by_lua: handler return value: %i, " "client hello cb exit code: %d", rc, cctx->exit_code); c->log->action = "SSL handshaking"; @@ -402,7 +402,7 @@ ngx_http_lua_ssl_client_hello_aborted(void *data) } ngx_log_debug0(NGX_LOG_DEBUG_HTTP, cctx->connection->log, 0, - "lua_client_hello_by_lua: client hello cb aborted"); + "ssl_client_hello_by_lua: client hello cb aborted"); cctx->aborted = 1; cctx->request->connection->ssl = NULL; diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index 680011d091..53109ac3df 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -583,7 +583,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'lua_client_hello_by_lua: handler return value: -1, client hello cb exit code: 0', +'ssl_client_hello_by_lua: handler return value: -1, client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, 'lua exit with code -1', ] @@ -724,7 +724,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'lua_client_hello_by_lua: client hello cb exit code: 0', +'ssl_client_hello_by_lua: client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, 'lua exit with code -1', ] @@ -795,7 +795,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'runtime error: ssl_client_hello_by_lua(nginx.conf:28):2: bad bad bad', -'lua_client_hello_by_lua: handler return value: 500, client hello cb exit code: 0', +'ssl_client_hello_by_lua: handler return value: 500, client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, qr/context: ssl_client_hello_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, ] @@ -867,7 +867,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'runtime error: ssl_client_hello_by_lua(nginx.conf:28):3: bad bad bad', -'lua_client_hello_by_lua: client hello cb exit code: 0', +'ssl_client_hello_by_lua: client hello cb exit code: 0', qr/\[info\] .*? SSL_do_handshake\(\) failed .*?callback failed/, ] @@ -2631,7 +2631,7 @@ ssl handshake: boolean --- error_log eval [ -'lua_client_hello_by_lua: handler return value: 0, client hello cb exit code: 1', +'ssl_client_hello_by_lua: handler return value: 0, client hello cb exit code: 1', qr/\[debug\] .*? SSL_do_handshake: 1/, 'lua exit with code 0', ] From d34d3c545e80bccbd8110235d73474c11895bd2e Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 23 Jul 2025 21:43:04 +0800 Subject: [PATCH 826/848] change: ngx_http_lua_ffi_get_req_ssl_pointer() add err argument. --- src/ngx_http_lua_ssl_certby.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index c5426da565..2abee0ef4b 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -1705,9 +1705,15 @@ ngx_http_lua_ffi_ssl_verify_client(ngx_http_request_t *r, void *client_certs, ngx_ssl_conn_t * -ngx_http_lua_ffi_get_req_ssl_pointer(ngx_http_request_t *r) +ngx_http_lua_ffi_get_req_ssl_pointer(ngx_http_request_t *r, const char **err) { if (r->connection == NULL || r->connection->ssl == NULL) { + *err = "bad request"; + return NULL; + } + + if (r->connection->ssl->connection == NULL) { + *err = "bad ssl connection"; return NULL; } From 96db53ea64949ed3be35b0c98576b9e2b5007d0e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=BD=97=E6=B3=BD=E8=BD=A9?= Date: Sun, 27 Jul 2025 20:35:00 +0800 Subject: [PATCH 827/848] doc: update version info. --- README.markdown | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.markdown b/README.markdown index 1565487d93..3d5b46cac3 100644 --- a/README.markdown +++ b/README.markdown @@ -64,8 +64,8 @@ Version ======= This document describes ngx_lua -[v0.10.25](https://github.com/openresty/lua-nginx-module/tags), which was released -on 19 June 2023. +[v0.10.28](https://github.com/openresty/lua-nginx-module/tags), which was released +on 17 Jan, 2025. Videos ====== @@ -308,6 +308,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.27.x (last tested: 1.27.1) * 1.25.x (last tested: 1.25.1) * 1.21.x (last tested: 1.21.4) * 1.19.x (last tested: 1.19.3) From 9600893c310f6865efa7f799b1e6426bfd73b220 Mon Sep 17 00:00:00 2001 From: swananan Date: Sun, 27 Jul 2025 22:13:44 +0800 Subject: [PATCH 828/848] tests: fix CI configuration and SSL test issues. --- t/043-shdict.t | 4 ++-- t/140-ssl-c-api.t | 6 +---- t/142-ssl-session-store.t | 1 + util/run-ci.sh | 48 +++++++++++++++++++-------------------- 4 files changed, 28 insertions(+), 31 deletions(-) diff --git a/t/043-shdict.t b/t/043-shdict.t index 649683b9fe..3c0cc7961a 100644 --- a/t/043-shdict.t +++ b/t/043-shdict.t @@ -158,7 +158,7 @@ hello content_by_lua ' local dogs = ngx.shared.dogs dogs:set("foo", 32, 0.01) - ngx.location.capture("/sleep/0.01") + ngx.location.capture("/sleep/0.011") ngx.say(dogs:get("foo")) '; } @@ -1131,7 +1131,7 @@ nil nil content_by_lua ' local dogs = ngx.shared.dogs dogs:set("foo", 32, 0.01, 255) - ngx.location.capture("/sleep/0.01") + ngx.location.capture("/sleep/0.011") local res, flags = dogs:get("foo") ngx.say("res = ", res, ", flags = ", flags) '; diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index 15ed8abfcf..ca04a87528 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -12,7 +12,7 @@ if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); } elsif ($openssl_version =~ m/BoringSSL/) { $ENV{TEST_NGINX_USE_BORINGSSL} = 1; - plan tests => repeat_each() * (blocks() * 6 - 8); + plan tests => repeat_each() * (blocks() * 6 - 6); } else { plan tests => repeat_each() * (blocks() * 5 - 5); $ENV{TEST_NGINX_USE_OPENSSL} = 1; @@ -2074,7 +2074,3 @@ without_filter:\[.*\] with_filter:\[.*\] --- error_log chomp TLSv1.2, cipher: "ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD" - - - - diff --git a/t/142-ssl-session-store.t b/t/142-ssl-session-store.t index ead2a167e5..11deb83207 100644 --- a/t/142-ssl-session-store.t +++ b/t/142-ssl-session-store.t @@ -975,3 +975,4 @@ qr/ssl_session_store_by_lua\*: skipped since TLS version >= 1\.3 \(\d+\)/ [error] [alert] [emerg] +--- skip_eval: 6:$ENV{TEST_NGINX_USE_HTTP3} diff --git a/util/run-ci.sh b/util/run-ci.sh index b5e5407c17..bb23926165 100755 --- a/util/run-ci.sh +++ b/util/run-ci.sh @@ -67,30 +67,30 @@ function download_deps() wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz fi - git_download clone https://github.com/openresty/test-nginx.git - git_download clone https://github.com/openresty/openresty.git ../openresty - git_download clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - git_download clone https://github.com/openresty/openresty-devel-utils.git - git_download clone https://github.com/openresty/mockeagain.git - git_download clone https://github.com/openresty/lua-cjson.git lua-cjson - git_download clone https://github.com/openresty/lua-upstream-nginx-module.git ../lua-upstream-nginx-module - git_download clone https://github.com/openresty/echo-nginx-module.git ../echo-nginx-module - git_download clone https://github.com/openresty/nginx-eval-module.git ../nginx-eval-module - git_download clone https://github.com/simpl/ngx_devel_kit.git ../ndk-nginx-module - git_download clone https://github.com/FRiCKLE/ngx_coolkit.git ../coolkit-nginx-module - git_download clone https://github.com/openresty/headers-more-nginx-module.git ../headers-more-nginx-module - git_download clone https://github.com/openresty/drizzle-nginx-module.git ../drizzle-nginx-module - git_download clone https://github.com/openresty/set-misc-nginx-module.git ../set-misc-nginx-module - git_download clone https://github.com/openresty/memc-nginx-module.git ../memc-nginx-module - git_download clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module - git_download clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module - git_download clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module - git_download clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core - git_download clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache - git_download clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql - git_download clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string - git_download clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module - git_download clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 + git_download https://github.com/openresty/test-nginx.git + git_download https://github.com/openresty/openresty.git ../openresty + git_download https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx + git_download https://github.com/openresty/openresty-devel-utils.git + git_download https://github.com/openresty/mockeagain.git + git_download https://github.com/openresty/lua-cjson.git lua-cjson + git_download https://github.com/openresty/lua-upstream-nginx-module.git ../lua-upstream-nginx-module + git_download https://github.com/openresty/echo-nginx-module.git ../echo-nginx-module + git_download https://github.com/openresty/nginx-eval-module.git ../nginx-eval-module + git_download https://github.com/simpl/ngx_devel_kit.git ../ndk-nginx-module + git_download https://github.com/FRiCKLE/ngx_coolkit.git ../coolkit-nginx-module + git_download https://github.com/openresty/headers-more-nginx-module.git ../headers-more-nginx-module + git_download https://github.com/openresty/drizzle-nginx-module.git ../drizzle-nginx-module + git_download https://github.com/openresty/set-misc-nginx-module.git ../set-misc-nginx-module + git_download https://github.com/openresty/memc-nginx-module.git ../memc-nginx-module + git_download https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module + git_download https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module + git_download https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module + git_download https://github.com/openresty/lua-resty-core.git ../lua-resty-core + git_download https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache + git_download https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql + git_download https://github.com/openresty/lua-resty-string.git ../lua-resty-string + git_download https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module + git_download -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2 } function make_deps() From f40ced17f780ea6f5f479c881bda305891cc8db4 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 5 Sep 2025 17:04:20 +0800 Subject: [PATCH 829/848] bugfix: remove debugging code. --- src/ngx_http_lua_socket_udp.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ngx_http_lua_socket_udp.c b/src/ngx_http_lua_socket_udp.c index 3297cbbeca..d57a55e123 100644 --- a/src/ngx_http_lua_socket_udp.c +++ b/src/ngx_http_lua_socket_udp.c @@ -1481,7 +1481,6 @@ ngx_http_lua_udp_connect(ngx_http_lua_udp_connection_t *uc, ngx_addr_t *local) #endif if (local != NULL) { - fprintf(stderr, "=== have local address\n"); if (bind(s, local->sockaddr, local->socklen) == -1) { ngx_log_error(NGX_LOG_CRIT, &uc->log, ngx_socket_errno, "bind(%V) failed", &local->name); From 94c578a071a9c30dc57db14b927822d65251e590 Mon Sep 17 00:00:00 2001 From: swananan Date: Sun, 14 Sep 2025 18:51:47 +0800 Subject: [PATCH 830/848] bugfix: add HTTP/3 QUIC SSL Lua yield patch macro protection. --- .travis.yml | 2 +- README.markdown | 8 ++++++++ src/ngx_http_lua_ssl_certby.c | 6 ++++-- src/ngx_http_lua_ssl_client_helloby.c | 6 ++++-- src/ngx_http_lua_util.c | 2 +- src/ngx_http_lua_util.h | 2 +- 6 files changed, 19 insertions(+), 7 deletions(-) diff --git a/.travis.yml b/.travis.yml index f48c9a75bc..fc39846390 100644 --- a/.travis.yml +++ b/.travis.yml @@ -87,7 +87,7 @@ install: - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git - - git clone https://github.com/openresty/openresty.git ../openresty + - git clone https://github.com/openresty/openresty.git - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - git clone https://github.com/openresty/openresty-devel-utils.git - git clone https://github.com/openresty/mockeagain.git diff --git a/README.markdown b/README.markdown index 3d5b46cac3..03a9a66055 100644 --- a/README.markdown +++ b/README.markdown @@ -2879,6 +2879,8 @@ patches to the standard Nginx core: +**Note for HTTP/3 (QUIC) users**: When using this directive with HTTP/3 connections, certain yield operations may fail if the QUIC SSL Lua yield patch is not applied to your OpenSSL installation. OpenResty packages include this patch by default, but if you are building lua-nginx-module separately, you may need to apply the patch manually to ensure proper yield/resume functionality for HTTP/3 connections in SSL Lua phases. The patch can be found at: [nginx-1.27.1-quic_ssl_lua_yield.patch](https://github.com/openresty/openresty/blob/master/patches/nginx/1.27.1/nginx-1.27.1-quic_ssl_lua_yield.patch) + This directive was first introduced in the `v0.10.21` release. [Back to TOC](#directives) @@ -2896,6 +2898,8 @@ Equivalent to [ssl_client_hello_by_lua_block](#ssl_client_hello_by_lua_block), e When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. +**Note for HTTP/3 (QUIC) users**: When using this directive with HTTP/3 connections, certain yield operations may fail if the QUIC SSL Lua yield patch is not applied to your OpenSSL installation. OpenResty packages include this patch by default, but if you are building lua-nginx-module separately, you may need to apply the patch manually to ensure proper yield/resume functionality for HTTP/3 connections in SSL Lua phases. The patch can be found at: [nginx-1.27.1-quic_ssl_lua_yield.patch](https://github.com/openresty/openresty/blob/master/patches/nginx/1.27.1/nginx-1.27.1-quic_ssl_lua_yield.patch) + This directive was first introduced in the `v0.10.21` release. [Back to TOC](#directives) @@ -2989,6 +2993,8 @@ patches to the standard Nginx core: +**Note for HTTP/3 (QUIC) users**: When using this directive with HTTP/3 connections, certain yield operations may fail if the QUIC SSL Lua yield patch is not applied to your OpenSSL installation. OpenResty packages include this patch by default, but if you are building lua-nginx-module separately, you may need to apply the patch manually to ensure proper yield/resume functionality for HTTP/3 connections in SSL Lua phases. The patch can be found at: [nginx-1.27.1-quic_ssl_lua_yield.patch](https://github.com/openresty/openresty/blob/master/patches/nginx/1.27.1/nginx-1.27.1-quic_ssl_lua_yield.patch) + This directive was first introduced in the `v0.10.0` release. [Back to TOC](#directives) @@ -3006,6 +3012,8 @@ Equivalent to [ssl_certificate_by_lua_block](#ssl_certificate_by_lua_block), exc When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. +**Note for HTTP/3 (QUIC) users**: When using this directive with HTTP/3 connections, certain yield operations may fail if the QUIC SSL Lua yield patch is not applied to your OpenSSL installation. OpenResty packages include this patch by default, but if you are building lua-nginx-module separately, you may need to apply the patch manually to ensure proper yield/resume functionality for HTTP/3 connections in SSL Lua phases. The patch can be found at: [nginx-1.27.1-quic_ssl_lua_yield.patch](https://github.com/openresty/openresty/blob/master/patches/nginx/1.27.1/nginx-1.27.1-quic_ssl_lua_yield.patch) + This directive was first introduced in the `v0.10.0` release. [Back to TOC](#directives) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 2abee0ef4b..9a2d63fcf0 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -390,9 +390,11 @@ ngx_http_lua_ssl_cert_done(void *data) ngx_post_event(c->write, &ngx_posted_events); -#if (NGX_HTTP_V3) && OPENSSL_VERSION_NUMBER >= 0x1000205fL -# if (NGX_QUIC_OPENSSL_COMPAT) +#if (HAVE_QUIC_SSL_LUA_YIELD_PATCH && NGX_HTTP_V3) +# if OPENSSL_VERSION_NUMBER >= 0x1000205fL +# if (NGX_QUIC_OPENSSL_COMPAT) ngx_http_lua_resume_quic_ssl_handshake(c); +# endif # endif #endif } diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 81b7a01f51..42a91c1604 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -381,9 +381,11 @@ ngx_http_lua_ssl_client_hello_done(void *data) ngx_post_event(c->write, &ngx_posted_events); -#if (NGX_HTTP_V3) && defined(SSL_ERROR_WANT_CLIENT_HELLO_CB) -# if (NGX_QUIC_OPENSSL_COMPAT) +#if (HAVE_QUIC_SSL_LUA_YIELD_PATCH && NGX_HTTP_V3) +# if defined(SSL_ERROR_WANT_CLIENT_HELLO_CB) +# if (NGX_QUIC_OPENSSL_COMPAT) ngx_http_lua_resume_quic_ssl_handshake(c); +# endif # endif #endif } diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index d47c691000..5bc3b02807 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -4574,7 +4574,7 @@ ngx_http_lua_ffi_bypass_if_checks(ngx_http_request_t *r) } -#if (NGX_HTTP_V3) +#if (HAVE_QUIC_SSL_LUA_YIELD_PATCH && NGX_HTTP_V3) void ngx_http_lua_resume_quic_ssl_handshake(ngx_connection_t *c) { diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index d76508868c..eab4d4eba2 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -267,7 +267,7 @@ ngx_addr_t *ngx_http_lua_parse_addr(lua_State *L, u_char *text, size_t len); size_t ngx_http_lua_escape_log(u_char *dst, u_char *src, size_t size); -#if (NGX_HTTP_V3) +#if (HAVE_QUIC_SSL_LUA_YIELD_PATCH && NGX_HTTP_V3) void ngx_http_lua_resume_quic_ssl_handshake(ngx_connection_t *c); #endif From cc8b0672d7c655b2f2ad10c6b033df7c614077f4 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Mon, 13 Oct 2025 15:56:21 +0800 Subject: [PATCH 831/848] feature: add support for nginx-1.29.2. --- .travis.yml | 22 ++++++++++++---------- src/ngx_http_lua_ssl_client_helloby.c | 10 ++++++++++ t/014-bugs.t | 5 +++-- 3 files changed, 25 insertions(+), 12 deletions(-) diff --git a/.travis.yml b/.travis.yml index fc39846390..66997c1acc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -46,10 +46,12 @@ env: - LUAJIT_LIB=$LUAJIT_PREFIX/lib - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1 - LUA_INCLUDE_DIR=$LUAJIT_INC - - PCRE2_PREFIX=/usr/local/openresty/pcre2 + #- PCRE2_PREFIX=/usr/local/openresty/pcre2 + - PCRE2_PREFIX=/opt/pcre2 - PCRE2_LIB=$PCRE2_PREFIX/lib - PCRE2_INC=$PCRE2_PREFIX/include - - OPENSSL_PREFIX=/usr/local/openresty/openssl3 + #- OPENSSL_PREFIX=/usr/local/openresty/openssl3 + - OPENSSL_PREFIX=/opt/openssl3 - OPENSSL_LIB=$OPENSSL_PREFIX/lib - OPENSSL_INC=$OPENSSL_PREFIX/include - LIBDRIZZLE_PREFIX=/opt/drizzle @@ -60,10 +62,10 @@ env: - TEST_NGINX_SLEEP=0.006 - MALLOC_PERTURB_=9 jobs: - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.5.0 OPENSSL_PATCH_VER=3.5.0 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.5.0 OPENSSL_PATCH_VER=3.5.0 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.45 TEST_NGINX_USE_HTTP2=1 - - NGINX_VERSION=1.27.1 OPENSSL_VER=3.5.0 OPENSSL_PATCH_VER=3.5.0 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 - - NGINX_VERSION=1.27.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.45 + - NGINX_VERSION=1.29.2 OPENSSL_VER=3.5.4 OPENSSL_PATCH_VER=3.5.4 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.46 + - NGINX_VERSION=1.29.2 OPENSSL_VER=3.5.4 OPENSSL_PATCH_VER=3.5.4 TEST_NGINX_TIMEOUT=5 PCRE2_VER=10.46 TEST_NGINX_USE_HTTP2=1 + - NGINX_VERSION=1.29.2 OPENSSL_VER=3.5.4 OPENSSL_PATCH_VER=3.5.4 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.46 + - NGINX_VERSION=1.29.2 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 TEST_NGINX_QUIC_IDLE_TIMEOUT=3 PCRE2_VER=10.46 services: - memcached @@ -82,8 +84,8 @@ before_install: install: - if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi - #- if [ -n "$PCRE2_VER" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi - #- if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi + - if [ -n "$PCRE2_VER" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://github.com/PCRE2Project/pcre2/releases/download/pcre2-${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi + - if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VER/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git @@ -135,8 +137,8 @@ script: - sudo make install-libdrizzle-1.0 > build.log 2>&1 || (cat build.log && exit 1) - cd ../mockeagain/ && make CC=$CC -j$JOBS && cd .. - cd lua-cjson/ && make -j$JOBS && sudo make install && cd .. - #- if [ -n "$PCRE2_VER" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - #- if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX --libdir=lib -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + - if [ -n "$PCRE2_VER" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi + - if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX --libdir=lib -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi - if [ -n "$BORINGSSL" ]; then sudo rm -fr /usr/local/openresty/openssl3/ && sudo mkdir -p /usr/local/openresty/openssl3 && sudo tar -C /usr/local/openresty/openssl3 -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi - export NGX_BUILD_CC=$CC - sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index 42a91c1604..c56e830636 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -193,6 +193,7 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, ngx_http_lua_ssl_ctx_t *cctx; ngx_http_core_srv_conf_t *cscf; + c = ngx_ssl_get_connection(ssl_conn); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, @@ -220,6 +221,15 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, dd("first time"); +#if (nginx_version > 1029000) + /* see commit 0373fe5d98c1515640 for more details */ + rc = ngx_ssl_client_hello_callback(ssl_conn, al, arg); + + if (rc == 0) { + return rc; + } +#endif + #if (nginx_version < 1017009) ngx_reusable_connection(c, 0); #endif diff --git a/t/014-bugs.t b/t/014-bugs.t index a89efec3c1..cef20733c2 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -9,7 +9,7 @@ log_level('debug'); repeat_each(3); # NB: the shutdown_error_log block is independent from repeat times -plan tests => repeat_each() * (blocks() * 2 + 33) + 1; +plan tests => repeat_each() * (blocks() * 2 + 33); our $HtmlDir = html_dir; #warn $html_dir; @@ -1269,6 +1269,8 @@ qr/\[emerg\] \d+#\d+: unexpected "A" in/ === TEST 47: cosocket does not exit on worker_shutdown_timeout +This test must enable master process +--- SKIP --- main_config worker_shutdown_timeout 1; --- config @@ -1316,7 +1318,6 @@ if ($ENV{TEST_NGINX_USE_HTTP3}) { $expr; --- timeout: 1.2 ---- skip_eval: 2:$ENV{TEST_NGINX_USE_HTTP3} From eb2b57f859b28f13c7e549a1dbface311dba56ea Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 14 Oct 2025 11:38:17 +0800 Subject: [PATCH 832/848] tests: fixed CI. --- .travis.yml | 2 +- src/ngx_http_lua_ssl_client_helloby.c | 1 - util/build.sh | 4 ++-- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/.travis.yml b/.travis.yml index 66997c1acc..4aebcf6996 100644 --- a/.travis.yml +++ b/.travis.yml @@ -89,7 +89,7 @@ install: - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz - wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz - git clone https://github.com/openresty/test-nginx.git - - git clone https://github.com/openresty/openresty.git + - git clone https://github.com/openresty/openresty.git ../openresty - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx - git clone https://github.com/openresty/openresty-devel-utils.git - git clone https://github.com/openresty/mockeagain.git diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index c56e830636..b600ab3636 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -193,7 +193,6 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, ngx_http_lua_ssl_ctx_t *cctx; ngx_http_core_srv_conf_t *cscf; - c = ngx_ssl_get_connection(ssl_conn); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, diff --git a/util/build.sh b/util/build.sh index 9dda855223..501f74bdb9 100755 --- a/util/build.sh +++ b/util/build.sh @@ -67,6 +67,6 @@ time ngx-build $force $version \ --with-http_dav_module \ --with-select_module \ --with-poll_module \ - $opts \ - --with-debug + $opts \ + --with-debug From 03d88830de4f7578c530e2a647c82e9d5904ec4d Mon Sep 17 00:00:00 2001 From: willmafh Date: Tue, 14 Oct 2025 13:15:45 +0800 Subject: [PATCH 833/848] feature: proxy_ssl_verify_by_lua directives. --- README.markdown | 96 ++ config | 2 + doc/HttpLuaModule.wiki | 13 + src/ngx_http_lua_common.h | 59 +- src/ngx_http_lua_control.c | 6 + src/ngx_http_lua_module.c | 57 ++ src/ngx_http_lua_proxy_ssl_verifyby.c | 754 ++++++++++++++ src/ngx_http_lua_proxy_ssl_verifyby.h | 41 + src/ngx_http_lua_ssl.h | 17 +- src/ngx_http_lua_util.c | 45 + src/ngx_http_lua_util.h | 46 + t/169-proxy-ssl-verify.t | 1332 +++++++++++++++++++++++++ 12 files changed, 2446 insertions(+), 22 deletions(-) create mode 100644 src/ngx_http_lua_proxy_ssl_verifyby.c create mode 100644 src/ngx_http_lua_proxy_ssl_verifyby.h create mode 100644 t/169-proxy-ssl-verify.t diff --git a/README.markdown b/README.markdown index 03a9a66055..8c6edec844 100644 --- a/README.markdown +++ b/README.markdown @@ -1171,6 +1171,8 @@ Directives * [ssl_session_fetch_by_lua_file](#ssl_session_fetch_by_lua_file) * [ssl_session_store_by_lua_block](#ssl_session_store_by_lua_block) * [ssl_session_store_by_lua_file](#ssl_session_store_by_lua_file) +* [proxy_ssl_verify_by_lua_block](#proxy_ssl_verify_by_lua_block) +* [proxy_ssl_verify_by_lua_file](#proxy_ssl_verify_by_lua_file) * [lua_shared_dict](#lua_shared_dict) * [lua_socket_connect_timeout](#lua_socket_connect_timeout) * [lua_socket_send_timeout](#lua_socket_send_timeout) @@ -1189,6 +1191,7 @@ Directives * [lua_ssl_verify_depth](#lua_ssl_verify_depth) * [lua_ssl_key_log](#lua_ssl_key_log) * [lua_ssl_conf_command](#lua_ssl_conf_command) +* [lua_upstream_skip_openssl_default_verify](#lua_upstream_skip_openssl_default_verify) * [lua_http10_buffering](#lua_http10_buffering) * [rewrite_by_lua_no_postpone](#rewrite_by_lua_no_postpone) * [access_by_lua_no_postpone](#access_by_lua_no_postpone) @@ -3165,6 +3168,84 @@ Note that: this directive is only allowed to used in **http context** from the ` [Back to TOC](#directives) +proxy_ssl_verify_by_lua_block +----------------------------- + +**syntax:** *proxy_ssl_verify_by_lua_block { lua-script }* + +**context:** *location* + +**phase:** *right-after-server-certificate-message-was-processed* + +This directive runs user Lua code when Nginx is about to post-process the SSL server certificate message for the upstream SSL (https) connections. + +It is particularly useful to parse upstream server certificate and do some custom operations in pure lua. + +The [ngx.ssl.proxysslverify](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/proxysslverify.md) Lua modules provided by the [lua-resty-core](https://github.com/openresty/lua-resty-core/#readme) +library are particularly useful in this context. + +Below is a trivial example using the +[ngx.ssl.proxysslverify](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/proxysslverify.md) module +at the same time: + +```nginx + + server { + listen 443 ssl; + server_name test.com; + ssl_certificate /path/to/cert.crt; + ssl_certificate_key /path/to/key.key; + + location /t { + proxy_ssl_certificate /path/to/cert.crt; + proxy_ssl_certificate_key /path/to/key.key; + proxy_pass https://upstream; + + proxy_ssl_verify_by_lua_block { + local proxy_ssl_vfy = require "ngx.ssl.proxysslverify" + local cert = proxy_ssl_vfy.get_verify_cert() + + -- ocsp to verify cert + -- check crl + proxy_ssl_vfy.set_verify_result() + ... + } + } + ... + } +``` + +See more information in the [ngx.ssl.proxysslverify](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl/proxysslverify.md) +Lua modules' official documentation. + +Uncaught Lua exceptions in the user Lua code immediately abort the current SSL session, so does the +[ngx.exit](#ngxexit) call with an error code like `ngx.ERROR`. + +This Lua code execution context *does* support yielding, so Lua APIs that may yield +(like cosockets, sleeping, and "light threads") +are enabled in this context + +Note, `ngx.ctx` in proxy_ssl_verify_by_lua_block is belonging to upstream connection, not downstream connection, so it's different from `ngx.ctx` in contexts like ssl_certificate_by_lua etc. + +This directive requires OpenSSL 3.0.2 or greater. + +[Back to TOC](#directives) + +proxy_ssl_verify_by_lua_file +---------------------------- + +**syntax:** *proxy_ssl_verify_by_lua_file <path-to-lua-script-file>* + +**context:** *location* + +**phase:** *right-after-server-certificate-message-was-processed* + +Equivalent to [proxy_ssl_verify_by_lua_block](#proxy_ssl_verify_by_lua_block), except that the file specified by `` contains the Lua code, or, as from the `v0.5.0rc32` release, the [LuaJIT bytecode](#luajit-bytecode-support) to be executed. + +When a relative path like `foo/bar.lua` is given, they will be turned into the absolute path relative to the `server prefix` path determined by the `-p PATH` command-line option while starting the Nginx server. + +[Back to TOC](#directives) + lua_shared_dict --------------- @@ -3499,6 +3580,21 @@ This directive was first introduced in the `v0.10.21` release. +[Back to TOC](#directives) + +lua_upstream_skip_openssl_default_verify +-------------------- + +**syntax:** *lua_upstream_skip_openssl_default_verify on|off* + +**default:** *lua_upstream_skip_openssl_default_verify off* + +**context:** *location, location-if* + +When using proxy_ssl_verify_by_lua directive, `lua_upstream_skip_openssl_default_verify` controls whether to skip default openssl's verify function, that means using pure Lua code to verify upstream server certificate. + +This directive is turned `off` by default. + [Back to TOC](#directives) lua_http10_buffering diff --git a/config b/config index 24ebd126d6..7b1b061362 100644 --- a/config +++ b/config @@ -296,6 +296,7 @@ HTTP_LUA_SRCS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_storeby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.c \ $ngx_addon_dir/src/ngx_http_lua_ssl.c \ + $ngx_addon_dir/src/ngx_http_lua_proxy_ssl_verifyby.c \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.c \ $ngx_addon_dir/src/ngx_http_lua_input_filters.c \ $ngx_addon_dir/src/ngx_http_lua_pipe.c \ @@ -359,6 +360,7 @@ HTTP_LUA_DEPS=" \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_storeby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl_session_fetchby.h \ $ngx_addon_dir/src/ngx_http_lua_ssl.h \ + $ngx_addon_dir/src/ngx_http_lua_proxy_ssl_verifyby.h \ $ngx_addon_dir/src/ngx_http_lua_log_ringbuf.h \ $ngx_addon_dir/src/ngx_http_lua_input_filters.h \ $ngx_addon_dir/src/ngx_http_lua_pipe.h \ diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 69e1a86303..09af86f771 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -2960,6 +2960,19 @@ Note though that configuring OpenSSL directly with lua_ssl_conf_commandv0.10.21 release. +== lua_upstream_skip_openssl_default_verify == + +'''syntax:''' ''lua_upstream_skip_openssl_default_verify on|off'' + +'''default:''' ''lua_upstream_skip_openssl_default_verify off'' + +'''context:''' ''location, location-if'' + +When using proxy_ssl_verify_by_lua directive, `lua_upstream_skip_openssl_default_verify` controls whether to skip default openssl's verify function, that means using pure Lua code to verify upstream server certificate. + +This directive is turned off by default. + +[Back to TOC](#directives) == lua_http10_buffering == diff --git a/src/ngx_http_lua_common.h b/src/ngx_http_lua_common.h index 9db3e19ce5..40c330baff 100644 --- a/src/ngx_http_lua_common.h +++ b/src/ngx_http_lua_common.h @@ -131,23 +131,27 @@ typedef struct { (NGX_HTTP_LUA_FILE_TAG_LEN + 2 * MD5_DIGEST_LENGTH) -/* must be within 16 bit */ -#define NGX_HTTP_LUA_CONTEXT_SET 0x0001 -#define NGX_HTTP_LUA_CONTEXT_REWRITE 0x0002 -#define NGX_HTTP_LUA_CONTEXT_ACCESS 0x0004 -#define NGX_HTTP_LUA_CONTEXT_CONTENT 0x0008 -#define NGX_HTTP_LUA_CONTEXT_LOG 0x0010 -#define NGX_HTTP_LUA_CONTEXT_HEADER_FILTER 0x0020 -#define NGX_HTTP_LUA_CONTEXT_BODY_FILTER 0x0040 -#define NGX_HTTP_LUA_CONTEXT_TIMER 0x0080 -#define NGX_HTTP_LUA_CONTEXT_INIT_WORKER 0x0100 -#define NGX_HTTP_LUA_CONTEXT_BALANCER 0x0200 -#define NGX_HTTP_LUA_CONTEXT_SSL_CERT 0x0400 -#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE 0x0800 -#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x1000 -#define NGX_HTTP_LUA_CONTEXT_EXIT_WORKER 0x2000 -#define NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO 0x4000 -#define NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE 0x8000 +/* must be within 32 bits */ +#define NGX_HTTP_LUA_CONTEXT_SET 0x00000001 +#define NGX_HTTP_LUA_CONTEXT_REWRITE 0x00000002 +#define NGX_HTTP_LUA_CONTEXT_ACCESS 0x00000004 +#define NGX_HTTP_LUA_CONTEXT_CONTENT 0x00000008 +#define NGX_HTTP_LUA_CONTEXT_LOG 0x00000010 +#define NGX_HTTP_LUA_CONTEXT_HEADER_FILTER 0x00000020 +#define NGX_HTTP_LUA_CONTEXT_BODY_FILTER 0x00000040 +#define NGX_HTTP_LUA_CONTEXT_TIMER 0x00000080 +#define NGX_HTTP_LUA_CONTEXT_INIT_WORKER 0x00000100 +#define NGX_HTTP_LUA_CONTEXT_BALANCER 0x00000200 +#define NGX_HTTP_LUA_CONTEXT_SSL_CERT 0x00000400 +#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE 0x00000800 +#define NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH 0x00001000 +#define NGX_HTTP_LUA_CONTEXT_EXIT_WORKER 0x00002000 +#define NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO 0x00004000 +#define NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE 0x00008000 + +#ifdef HAVE_PROXY_SSL_PATCH +#define NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY 0x00010000 +#endif #define NGX_HTTP_LUA_FFI_NO_REQ_CTX -100 @@ -171,6 +175,8 @@ typedef struct ngx_http_lua_srv_conf_s ngx_http_lua_srv_conf_t; typedef struct ngx_http_lua_main_conf_s ngx_http_lua_main_conf_t; +typedef struct ngx_http_lua_loc_conf_s ngx_http_lua_loc_conf_t; + typedef struct ngx_http_lua_header_val_s ngx_http_lua_header_val_t; typedef struct ngx_http_lua_posted_thread_s ngx_http_lua_posted_thread_t; @@ -184,6 +190,9 @@ typedef ngx_int_t (*ngx_http_lua_main_conf_handler_pt)(ngx_log_t *log, typedef ngx_int_t (*ngx_http_lua_srv_conf_handler_pt)(ngx_http_request_t *r, ngx_http_lua_srv_conf_t *lscf, lua_State *L); +typedef ngx_int_t (*ngx_http_lua_loc_conf_handler_pt)(ngx_http_request_t *r, + ngx_http_lua_loc_conf_t *llcf, lua_State *L); + typedef ngx_int_t (*ngx_http_lua_set_header_pt)(ngx_http_request_t *r, ngx_http_lua_header_val_t *hv, ngx_str_t *value); @@ -369,7 +378,7 @@ struct ngx_http_lua_srv_conf_s { }; -typedef struct { +struct ngx_http_lua_loc_conf_s { #if (NGX_HTTP_SSL) ngx_ssl_t *ssl; /* shared by SSL cosockets */ ngx_array_t *ssl_certificates; @@ -383,6 +392,16 @@ typedef struct { #if (nginx_version >= 1019004) ngx_array_t *ssl_conf_commands; #endif + +#ifdef HAVE_PROXY_SSL_PATCH + ngx_http_lua_loc_conf_handler_pt proxy_ssl_verify_handler; + ngx_str_t proxy_ssl_verify_src; + u_char *proxy_ssl_verify_src_key; + u_char *proxy_ssl_verify_chunkname; + int proxy_ssl_verify_src_ref; + ngx_flag_t upstream_skip_openssl_default_verify; +#endif + #endif ngx_flag_t force_read_body; /* whether force request body to @@ -464,7 +483,7 @@ typedef struct { ngx_flag_t log_socket_errors; ngx_flag_t check_client_abort; ngx_flag_t use_default_type; -} ngx_http_lua_loc_conf_t; +}; typedef enum { @@ -628,7 +647,7 @@ typedef struct ngx_http_lua_ctx_s { int uthreads; /* number of active user threads */ - uint16_t context; /* the current running directive context + uint32_t context; /* the current running directive context (or running phase) for the current Lua chunk */ diff --git a/src/ngx_http_lua_control.c b/src/ngx_http_lua_control.c index d7e427385d..8358abcbb6 100644 --- a/src/ngx_http_lua_control.c +++ b/src/ngx_http_lua_control.c @@ -384,6 +384,9 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, | NGX_HTTP_LUA_CONTEXT_TIMER | NGX_HTTP_LUA_CONTEXT_HEADER_FILTER | NGX_HTTP_LUA_CONTEXT_BALANCER +#ifdef HAVE_PROXY_SSL_PATCH + | NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY +#endif | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO | NGX_HTTP_LUA_CONTEXT_SSL_CERT | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE @@ -395,6 +398,9 @@ ngx_http_lua_ffi_exit(ngx_http_request_t *r, int status, u_char *err, } if (ctx->context & (NGX_HTTP_LUA_CONTEXT_SSL_CERT +#ifdef HAVE_PROXY_SSL_PATCH + | NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY +#endif | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO | NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH)) diff --git a/src/ngx_http_lua_module.c b/src/ngx_http_lua_module.c index cae92b21ee..16726abbe1 100644 --- a/src/ngx_http_lua_module.c +++ b/src/ngx_http_lua_module.c @@ -31,6 +31,11 @@ #include "ngx_http_lua_ssl_certby.h" #include "ngx_http_lua_ssl_session_storeby.h" #include "ngx_http_lua_ssl_session_fetchby.h" + +#ifdef HAVE_PROXY_SSL_PATCH +#include "ngx_http_lua_proxy_ssl_verifyby.h" +#endif + #include "ngx_http_lua_headers.h" #include "ngx_http_lua_headers_out.h" #if !(NGX_WIN32) @@ -660,6 +665,30 @@ static ngx_command_t ngx_http_lua_cmds[] = { 0, (void *) ngx_http_lua_ssl_sess_fetch_handler_file }, +#ifdef HAVE_PROXY_SSL_PATCH + /* same context as proxy_pass directive */ + { ngx_string("proxy_ssl_verify_by_lua_block"), + NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS, + ngx_http_lua_proxy_ssl_verify_by_lua_block, + NGX_HTTP_LOC_CONF_OFFSET, + 0, + (void *) ngx_http_lua_proxy_ssl_verify_handler_inline }, + + { ngx_string("proxy_ssl_verify_by_lua_file"), + NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF|NGX_CONF_TAKE1, + ngx_http_lua_proxy_ssl_verify_by_lua, + NGX_HTTP_LOC_CONF_OFFSET, + 0, + (void *) ngx_http_lua_proxy_ssl_verify_handler_file }, + + { ngx_string("lua_upstream_skip_openssl_default_verify"), + NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_lua_loc_conf_t, upstream_skip_openssl_default_verify), + NULL }, +#endif + { ngx_string("lua_ssl_verify_depth"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, ngx_conf_set_num_slot, @@ -1446,6 +1475,11 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) * conf->ssl_trusted_certificate = { 0, NULL }; * conf->ssl_crl = { 0, NULL }; * conf->ssl_key_log = { 0, NULL }; + * + * conf->proxy_ssl_verify_handler = NULL; + * conf->proxy_ssl_verify_src = { 0, NULL }; + * conf->proxy_ssl_verify_chunkname = NULL; + * conf->proxy_ssl_verify_src_key = NULL; */ conf->force_read_body = NGX_CONF_UNSET; @@ -1479,6 +1513,10 @@ ngx_http_lua_create_loc_conf(ngx_conf_t *cf) #if (nginx_version >= 1019004) conf->ssl_conf_commands = NGX_CONF_UNSET_PTR; #endif +#ifdef HAVE_PROXY_SSL_PATCH + conf->proxy_ssl_verify_src_ref = LUA_REFNIL; + conf->upstream_skip_openssl_default_verify = NGX_CONF_UNSET; +#endif #endif return conf; @@ -1573,6 +1611,25 @@ ngx_http_lua_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) NULL); #endif +#ifdef HAVE_PROXY_SSL_PATCH + if (conf->proxy_ssl_verify_src.len == 0) { + conf->proxy_ssl_verify_src = prev->proxy_ssl_verify_src; + conf->proxy_ssl_verify_handler = prev->proxy_ssl_verify_handler; + conf->proxy_ssl_verify_src_ref = prev->proxy_ssl_verify_src_ref; + conf->proxy_ssl_verify_src_key = prev->proxy_ssl_verify_src_key; + conf->proxy_ssl_verify_chunkname = prev->proxy_ssl_verify_chunkname; + } + + if (conf->proxy_ssl_verify_src.len) { + if (ngx_http_lua_proxy_ssl_verify_set_callback(cf) != NGX_OK) { + return NGX_CONF_ERROR; + } + } + + ngx_conf_merge_value(conf->upstream_skip_openssl_default_verify, + prev->upstream_skip_openssl_default_verify, 0); +#endif + if (ngx_http_lua_set_ssl(cf, conf) != NGX_OK) { return NGX_CONF_ERROR; } diff --git a/src/ngx_http_lua_proxy_ssl_verifyby.c b/src/ngx_http_lua_proxy_ssl_verifyby.c new file mode 100644 index 0000000000..1695e56ffb --- /dev/null +++ b/src/ngx_http_lua_proxy_ssl_verifyby.c @@ -0,0 +1,754 @@ +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + +#ifndef DDEBUG +#define DDEBUG 0 +#endif +#include "ddebug.h" + + +#if (NGX_HTTP_SSL) + + +#include "ngx_http_lua_cache.h" +#include "ngx_http_lua_initworkerby.h" +#include "ngx_http_lua_util.h" +#include "ngx_http_ssl_module.h" +#include "ngx_http_lua_contentby.h" +#include "ngx_http_lua_directive.h" +#include "ngx_http_lua_ssl.h" + +#ifdef HAVE_PROXY_SSL_PATCH +#include "ngx_http_lua_proxy_ssl_verifyby.h" + + +static void ngx_http_lua_proxy_ssl_verify_done(void *data); +static void ngx_http_lua_proxy_ssl_verify_aborted(void *data); +static ngx_int_t ngx_http_lua_proxy_ssl_verify_by_chunk(lua_State *L, + ngx_http_request_t *r); + + +ngx_int_t +ngx_http_lua_proxy_ssl_verify_set_callback(ngx_conf_t *cf) +{ + +#ifdef LIBRESSL_VERSION_NUMBER + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "LibreSSL does not support by proxy_ssl_verify_by_lua*"); + + return NGX_ERROR; + +#else + + void *plcf; + ngx_http_upstream_conf_t *ucf; + ngx_ssl_t *ssl; + + /* + * Nginx doesn't export ngx_http_proxy_loc_conf_t, so we can't directly + * get plcf here, but the first member of plcf is ngx_http_upstream_conf_t + */ + plcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_proxy_module); + ucf = plcf; + + ssl = ucf->ssl; + + if (!ssl->ctx) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "proxy_ssl_verify_by_lua* " + "should be used with proxy_pass https url"); + + return NGX_ERROR; + } + +#if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ + || OPENSSL_VERSION_NUMBER < 0x30000020L) + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "OpenSSL too old to support " + "proxy_ssl_verify_by_lua*"); + + return NGX_ERROR; + +#else + + SSL_CTX_set_cert_verify_callback(ssl->ctx, + ngx_http_lua_proxy_ssl_verify_handler, + NULL); + return NGX_OK; + +#endif + +#endif +} + + +ngx_int_t +ngx_http_lua_proxy_ssl_verify_handler_file(ngx_http_request_t *r, + ngx_http_lua_loc_conf_t *llcf, lua_State *L) +{ + ngx_int_t rc; + + rc = ngx_http_lua_cache_loadfile(r->connection->log, L, + llcf->proxy_ssl_verify_src.data, + &llcf->proxy_ssl_verify_src_ref, + llcf->proxy_ssl_verify_src_key); + if (rc != NGX_OK) { + return rc; + } + + /* make sure we have a valid code chunk */ + ngx_http_lua_assert(lua_isfunction(L, -1)); + + return ngx_http_lua_proxy_ssl_verify_by_chunk(L, r); +} + + +ngx_int_t +ngx_http_lua_proxy_ssl_verify_handler_inline(ngx_http_request_t *r, + ngx_http_lua_loc_conf_t *llcf, lua_State *L) +{ + ngx_int_t rc; + + rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, + llcf->proxy_ssl_verify_src.data, + llcf->proxy_ssl_verify_src.len, + &llcf->proxy_ssl_verify_src_ref, + llcf->proxy_ssl_verify_src_key, + (const char *) llcf->proxy_ssl_verify_chunkname); + if (rc != NGX_OK) { + return rc; + } + + /* make sure we have a valid code chunk */ + ngx_http_lua_assert(lua_isfunction(L, -1)); + + return ngx_http_lua_proxy_ssl_verify_by_chunk(L, r); +} + + +char * +ngx_http_lua_proxy_ssl_verify_by_lua_block(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ + char *rv; + ngx_conf_t save; + + save = *cf; + cf->handler = ngx_http_lua_proxy_ssl_verify_by_lua; + cf->handler_conf = conf; + + rv = ngx_http_lua_conf_lua_block_parse(cf, cmd); + + *cf = save; + + return rv; +} + + +char * +ngx_http_lua_proxy_ssl_verify_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf) +{ +#if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ + || OPENSSL_VERSION_NUMBER < 0x30000020L) + + /* SSL_set_retry_verify() was added in OpenSSL 3.0.2 */ + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "at least OpenSSL 3.0.2 required but found " + OPENSSL_VERSION_TEXT); + + return NGX_CONF_ERROR; + +#else + + size_t chunkname_len; + u_char *chunkname; + u_char *cache_key = NULL; + u_char *name; + ngx_str_t *value; + ngx_http_lua_loc_conf_t *llcf = conf; + + /* must specify a concrete handler */ + if (cmd->post == NULL) { + return NGX_CONF_ERROR; + } + + if (llcf->proxy_ssl_verify_handler) { + return "is duplicate"; + } + + if (ngx_http_lua_ssl_init(cf->log) != NGX_OK) { + return NGX_CONF_ERROR; + } + + value = cf->args->elts; + + llcf->proxy_ssl_verify_handler = + (ngx_http_lua_loc_conf_handler_pt) cmd->post; + + if (cmd->post == ngx_http_lua_proxy_ssl_verify_handler_file) { + /* Lua code in an external file */ + + name = ngx_http_lua_rebase_path(cf->pool, value[1].data, + value[1].len); + if (name == NULL) { + return NGX_CONF_ERROR; + } + + cache_key = ngx_http_lua_gen_file_cache_key(cf, value[1].data, + value[1].len); + if (cache_key == NULL) { + return NGX_CONF_ERROR; + } + + llcf->proxy_ssl_verify_src.data = name; + llcf->proxy_ssl_verify_src.len = ngx_strlen(name); + + } else { + cache_key = ngx_http_lua_gen_chunk_cache_key(cf, + "proxy_ssl_verify_by_lua", + value[1].data, + value[1].len); + if (cache_key == NULL) { + return NGX_CONF_ERROR; + } + + chunkname = ngx_http_lua_gen_chunk_name(cf, "proxy_ssl_verify_by_lua", + sizeof("proxy_ssl_verify_by_lua") - 1, + &chunkname_len); + if (chunkname == NULL) { + return NGX_CONF_ERROR; + } + + /* Don't eval nginx variables for inline lua code */ + llcf->proxy_ssl_verify_src = value[1]; + llcf->proxy_ssl_verify_chunkname = chunkname; + } + + llcf->proxy_ssl_verify_src_key = cache_key; + + return NGX_CONF_OK; + +#endif /* SSL_ERROR_WANT_RETRY_VERIFY */ +} + + +int +ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) +{ + lua_State *L; + ngx_int_t rc; + ngx_connection_t *c; + ngx_http_request_t *r = NULL; + ngx_pool_cleanup_t *cln; + ngx_http_lua_loc_conf_t *llcf; + ngx_http_lua_ssl_ctx_t *cctx; + ngx_ssl_conn_t *ssl_conn; + + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, + SSL_get_ex_data_X509_STORE_CTX_idx()); + + c = ngx_ssl_get_connection(ssl_conn); /* upstream connection */ + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "proxy ssl verify: connection reusable: %ud", c->reusable); + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + + dd("proxy ssl verify handler, cert-verify-ctx=%p", cctx); + + if (cctx && cctx->entered_proxy_ssl_verify_handler) { + /* not the first time */ + + if (cctx->done) { + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, + "proxy_ssl_verify_by_lua: " + "cert verify callback exit code: %d", + cctx->exit_code); + + dd("lua proxy ssl verify done, finally"); + return cctx->exit_code; + } + + return SSL_set_retry_verify(ssl_conn); + } + + dd("first time"); + +#if (nginx_version < 1017009) + ngx_reusable_connection(c, 0); +#endif + + r = c->data; + + if (cctx == NULL) { + cctx = ngx_pcalloc(c->pool, sizeof(ngx_http_lua_ssl_ctx_t)); + if (cctx == NULL) { + goto failed; /* error */ + } + + cctx->ctx_ref = LUA_NOREF; + } + + cctx->connection = c; + cctx->request = r; + cctx->x509_store = x509_store; + cctx->exit_code = 1; /* successful by default */ + cctx->original_request_count = r->main->count; + cctx->done = 0; + cctx->entered_proxy_ssl_verify_handler = 1; + cctx->pool = ngx_create_pool(128, c->log); + if (cctx->pool == NULL) { + goto failed; + } + + dd("setting cctx"); + + if (SSL_set_ex_data(c->ssl->connection, ngx_http_lua_ssl_ctx_index, + cctx) == 0) + { + ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "SSL_set_ex_data() failed"); + goto failed; + } + + llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); + if (llcf->upstream_skip_openssl_default_verify == 0) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + "proxy_ssl_verify_by_lua: openssl default verify"); + + rc = X509_verify_cert(x509_store); + if (rc == 0) { + goto failed; + } + } + + /* TODO honor lua_code_cache off */ + L = ngx_http_lua_get_lua_vm(r, NULL); + + c->log->action = "loading proxy ssl verify by lua"; + + rc = llcf->proxy_ssl_verify_handler(r, llcf, L); + + if (rc >= NGX_OK || rc == NGX_ERROR) { + cctx->done = 1; + + if (cctx->cleanup) { + *cctx->cleanup = NULL; + } + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, + "proxy_ssl_verify_by_lua: handler return value: %i, " + "cert verify callback exit code: %d", rc, cctx->exit_code); + + c->log->action = "proxy pass SSL handshaking"; + return cctx->exit_code; + } + + /* rc == NGX_DONE */ + + cln = ngx_pool_cleanup_add(cctx->pool, 0); + if (cln == NULL) { + goto failed; + } + + cln->handler = ngx_http_lua_proxy_ssl_verify_done; + cln->data = cctx; + + if (cctx->cleanup == NULL) { + cln = ngx_pool_cleanup_add(c->pool, 0); + if (cln == NULL) { + goto failed; + } + + cln->data = cctx; + cctx->cleanup = &cln->handler; + } + + *cctx->cleanup = ngx_http_lua_proxy_ssl_verify_aborted; + + return SSL_set_retry_verify(ssl_conn); + +failed: + if (cctx && cctx->pool) { + ngx_destroy_pool(cctx->pool); + } + + return 0; /* verify failure or error */ +} + + +static void +ngx_http_lua_proxy_ssl_verify_done(void *data) +{ + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx = data; + + dd("lua proxy ssl verify done"); + + if (cctx->aborted) { + return; + } + + ngx_http_lua_assert(cctx->done == 0); + + cctx->done = 1; + + if (cctx->cleanup) { + *cctx->cleanup = NULL; + } + + c = cctx->connection; + + if (c->read->timer_set) { + ngx_del_timer(c->read); + } + + if (c->write->timer_set) { + ngx_del_timer(c->write); + } + + c->log->action = "proxy pass SSL handshaking"; + + ngx_post_event(c->write, &ngx_posted_events); +} + + +static void +ngx_http_lua_proxy_ssl_verify_aborted(void *data) +{ + ngx_http_lua_ssl_ctx_t *cctx = data; + + dd("lua proxy ssl verify aborted"); + + if (cctx->done) { + /* completed successfully already */ + return; + } + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, cctx->connection->log, 0, + "proxy_ssl_verify_by_lua: cert verify callback aborted"); + + cctx->aborted = 1; + cctx->connection->ssl = NULL; + cctx->exit_code = 0; + if (cctx->pool) { + ngx_destroy_pool(cctx->pool); + cctx->pool = NULL; + } +} + + +static ngx_int_t +ngx_http_lua_proxy_ssl_verify_by_chunk(lua_State *L, ngx_http_request_t *r) +{ + int co_ref; + ngx_int_t rc; + lua_State *co; + ngx_http_lua_ctx_t *ctx; + ngx_pool_cleanup_t *cln; + ngx_http_upstream_t *u; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); + + if (ctx == NULL) { + ctx = ngx_http_lua_create_ctx(r); + if (ctx == NULL) { + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; + } + + } else { + dd("reset ctx"); + ngx_http_lua_reset_ctx(r, L, ctx); + } + + ctx->entered_content_phase = 1; + + /* {{{ new coroutine to handle request */ + co = ngx_http_lua_new_thread(r, L, &co_ref); + + if (co == NULL) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "lua: failed to create new coroutine to handle request"); + + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; + } + + /* move code closure to new coroutine */ + lua_xmove(L, co, 1); + +#ifndef OPENRESTY_LUAJIT + /* set closure's env table to new coroutine's globals table */ + ngx_http_lua_get_globals_table(co); + lua_setfenv(co, -2); +#endif + + /* save nginx request in coroutine globals table */ + ngx_http_lua_set_req(co, r); + + ctx->cur_co_ctx = &ctx->entry_co_ctx; + ctx->cur_co_ctx->co = co; + ctx->cur_co_ctx->co_ref = co_ref; +#ifdef NGX_LUA_USE_ASSERT + ctx->cur_co_ctx->co_top = 1; +#endif + + ngx_http_lua_attach_co_ctx_to_L(co, ctx->cur_co_ctx); + + /* register request cleanup hooks */ + if (ctx->cleanup == NULL) { + u = r->upstream; + c = u->peer.connection; + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + + cln = ngx_pool_cleanup_add(cctx->pool, 0); + if (cln == NULL) { + rc = NGX_ERROR; + ngx_http_lua_finalize_request(r, rc); + return rc; + } + + cln->handler = ngx_http_lua_request_cleanup_handler; + cln->data = ctx; + ctx->cleanup = &cln->handler; + } + + ctx->context = NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY; + + rc = ngx_http_lua_run_thread(L, r, ctx, 0); + + if (rc == NGX_ERROR || rc >= NGX_OK) { + /* do nothing */ + + } else if (rc == NGX_AGAIN) { + rc = ngx_http_lua_content_run_posted_threads(L, r, ctx, 0); + + } else if (rc == NGX_DONE) { + rc = ngx_http_lua_content_run_posted_threads(L, r, ctx, 1); + + } else { + rc = NGX_OK; + } + + ngx_http_lua_finalize_request(r, rc); + return rc; +} + + +/* + * openssl's doc of SSL_CTX_set_cert_verify_callback: + * In any case a viable verification result value must + * be reflected in the error member of x509_store_ctx, + * which can be done using X509_STORE_CTX_set_error. + */ +int +ngx_http_lua_ffi_proxy_ssl_set_verify_result(ngx_http_request_t *r, + int verify_result, char **err) +{ +#ifdef SSL_ERROR_WANT_RETRY_VERIFY + ngx_http_upstream_t *u; + ngx_ssl_conn_t *ssl_conn; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; + X509_STORE_CTX *x509_store; + + u = r->upstream; + if (u == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + c = u->peer.connection; + if (c == NULL || c->ssl == NULL) { + *err = "bad upstream connection"; + return NGX_ERROR; + } + + ssl_conn = c->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + dd("get cctx session"); + + c = ngx_ssl_get_connection(ssl_conn); + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + if (cctx == NULL) { + *err = "bad lua context"; + return NGX_ERROR; + } + + x509_store = cctx->x509_store; + + X509_STORE_CTX_set_error(x509_store, verify_result); + + return NGX_OK; +#else + *err = "OpenSSL too old to support this function"; + + return NGX_ERROR; +#endif +} + + +int +ngx_http_lua_ffi_proxy_ssl_get_verify_result(ngx_http_request_t *r, char **err) +{ +#ifdef SSL_ERROR_WANT_RETRY_VERIFY + ngx_http_upstream_t *u; + ngx_ssl_conn_t *ssl_conn; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; + X509_STORE_CTX *x509_store; + + u = r->upstream; + if (u == NULL) { + *err = "bad request"; + return NGX_ERROR; + } + + c = u->peer.connection; + if (c == NULL || c->ssl == NULL) { + *err = "bad upstream connection"; + return NGX_ERROR; + } + + ssl_conn = c->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NGX_ERROR; + } + + dd("get cctx session"); + + c = ngx_ssl_get_connection(ssl_conn); + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + if (cctx == NULL) { + *err = "bad lua context"; + return NGX_ERROR; + } + + x509_store = cctx->x509_store; + + return X509_STORE_CTX_get_error(x509_store); +#else + *err = "OpenSSL too old to support this function"; + + return NGX_ERROR; +#endif +} + + +void +ngx_http_lua_ffi_proxy_ssl_free_verify_cert(void *cdata) +{ + X509 *cert = cdata; + + X509_free(cert); +} + + +void * +ngx_http_lua_ffi_proxy_ssl_get_verify_cert(ngx_http_request_t *r, char **err) +{ +#ifdef SSL_ERROR_WANT_RETRY_VERIFY + ngx_http_upstream_t *u; + ngx_ssl_conn_t *ssl_conn; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; + X509_STORE_CTX *x509_store; + X509 *x509; + + u = r->upstream; + if (u == NULL) { + *err = "bad request"; + return NULL; + } + + c = u->peer.connection; + if (c == NULL || c->ssl == NULL) { + *err = "bad upstream connection"; + return NULL; + } + + ssl_conn = c->ssl->connection; + if (ssl_conn == NULL) { + *err = "bad ssl conn"; + return NULL; + } + + dd("get cctx session"); + + c = ngx_ssl_get_connection(ssl_conn); + + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + if (cctx == NULL) { + *err = "bad lua context"; + return NULL; + } + + x509_store = cctx->x509_store; + + x509 = X509_STORE_CTX_get0_cert(x509_store); + + if (!X509_up_ref(x509)) { + *err = "get verify result failed"; + return NULL; + } + + return x509; +#else + *err = "OpenSSL too old to support this function"; + + return NULL; +#endif +} + + +#else /* HAVE_PROXY_SSL_PATCH */ + + +int +ngx_http_lua_ffi_proxy_ssl_set_verify_result(ngx_http_request_t *r, + int verify_result, char **err) +{ + *err = "Does not have HAVE_PROXY_SSL_PATCH to support this function"; + + return NGX_ERROR; +} + + +int +ngx_http_lua_ffi_proxy_ssl_get_verify_result(ngx_http_request_t *r, char **err) +{ + *err = "Does not have HAVE_PROXY_SSL_PATCH to support this function"; + + return NGX_ERROR; +} + + +void +ngx_http_lua_ffi_proxy_ssl_free_verify_cert(void *cdata) +{ +} + + +void * +ngx_http_lua_ffi_proxy_ssl_get_verify_cert(ngx_http_request_t *r, char **err) +{ + *err = "Does not have HAVE_PROXY_SSL_PATCH to support this function"; + + return NULL; +} + +#endif /* HAVE_PROXY_SSL_PATCH */ +#endif /* NGX_HTTP_SSL */ diff --git a/src/ngx_http_lua_proxy_ssl_verifyby.h b/src/ngx_http_lua_proxy_ssl_verifyby.h new file mode 100644 index 0000000000..3e0b178dee --- /dev/null +++ b/src/ngx_http_lua_proxy_ssl_verifyby.h @@ -0,0 +1,41 @@ +/* + * Copyright (C) Yichun Zhang (agentzh) + */ + +#ifndef _NGX_HTTP_LUA_PROXY_SSL_VERIFYBY_H_INCLUDED_ +#define _NGX_HTTP_LUA_PROXY_SSL_VERIFYBY_H_INCLUDED_ + + +#include "ngx_http_lua_common.h" + + +#if (NGX_HTTP_SSL) +#ifdef HAVE_PROXY_SSL_PATCH + +/* do not introduce ngx_http_proxy_module to pollute ngx_http_lua_module.c */ +extern ngx_module_t ngx_http_proxy_module; + +ngx_int_t ngx_http_lua_proxy_ssl_verify_handler_inline(ngx_http_request_t *r, + ngx_http_lua_loc_conf_t *llcf, lua_State *L); + +ngx_int_t ngx_http_lua_proxy_ssl_verify_handler_file(ngx_http_request_t *r, + ngx_http_lua_loc_conf_t *llcf, lua_State *L); + +char *ngx_http_lua_proxy_ssl_verify_by_lua_block(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf); + +char *ngx_http_lua_proxy_ssl_verify_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); + +int ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, + void *arg); + +ngx_int_t ngx_http_lua_proxy_ssl_verify_set_callback(ngx_conf_t *cf); + +#endif /* HAVE_PROXY_SSL_PATCH */ +#endif /* NGX_HTTP_SSL */ + + +#endif /* _NGX_HTTP_LUA_PROXY_SSL_VERIFYBY_H_INCLUDED_ */ + +/* vi:set ft=c ts=4 sw=4 et fdm=marker: */ diff --git a/src/ngx_http_lua_ssl.h b/src/ngx_http_lua_ssl.h index 1ee0a3626c..f709e6530f 100644 --- a/src/ngx_http_lua_ssl.h +++ b/src/ngx_http_lua_ssl.h @@ -16,7 +16,7 @@ typedef struct { ngx_connection_t *connection; /* original true connection */ - ngx_http_request_t *request; /* fake request */ + ngx_http_request_t *request; ngx_pool_cleanup_pt *cleanup; ngx_ssl_session_t *session; /* return value for openssl's @@ -24,20 +24,33 @@ typedef struct { ngx_str_t session_id; +#ifdef HAVE_PROXY_SSL_PATCH + X509_STORE_CTX *x509_store; + ngx_pool_t *pool; +#endif + int exit_code; /* exit code for openssl's set_client_hello_cb or - set_cert_cb callback */ + set_cert_cb callback or + SSL_CTX_set_cert_verify_callback */ int ctx_ref; /* reference to anchor request ctx data in lua registry */ +#ifdef HAVE_PROXY_SSL_PATCH + /* same size as count field of ngx_http_request_t */ + unsigned original_request_count:16; +#endif unsigned done:1; unsigned aborted:1; unsigned entered_client_hello_handler:1; unsigned entered_cert_handler:1; unsigned entered_sess_fetch_handler:1; +#ifdef HAVE_PROXY_SSL_PATCH + unsigned entered_proxy_ssl_verify_handler:1; +#endif } ngx_http_lua_ssl_ctx_t; diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 5bc3b02807..928a15beed 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1682,6 +1682,11 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, NGX_ERROR : NGX_HTTP_INTERNAL_SERVER_ERROR; done: +#ifdef HAVE_PROXY_SSL_PATCH + if (ctx->context == NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY) { + return NGX_OK; + } +#endif if (ctx->entered_content_phase && r->connection->fd != (ngx_socket_t) -1) @@ -2438,6 +2443,12 @@ ngx_http_lua_handle_exit(lua_State *L, ngx_http_request_t *r, return ctx->exit_code; } +#ifdef HAVE_PROXY_SSL_PATCH + if (ctx->context == NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY) { + return ctx->exit_code; + } +#endif + #if 1 if (!r->header_sent && !ctx->header_sent @@ -3673,12 +3684,46 @@ void ngx_http_lua_finalize_request(ngx_http_request_t *r, ngx_int_t rc) { ngx_http_lua_ctx_t *ctx; +#if (NGX_HTTP_SSL) +#ifdef HAVE_PROXY_SSL_PATCH + ngx_http_upstream_t *u; + ngx_connection_t *c; + ngx_http_lua_ssl_ctx_t *cctx; +#endif +#endif ctx = ngx_http_get_module_ctx(r, ngx_http_lua_module); if (ctx && ctx->cur_co_ctx) { ngx_http_lua_cleanup_pending_operation(ctx->cur_co_ctx); } +#if (NGX_HTTP_SSL) +#ifdef HAVE_PROXY_SSL_PATCH + u = r->upstream; + if (u) { + c = u->peer.connection; + if (c && c->ssl) { + cctx = ngx_http_lua_ssl_get_ctx(c->ssl->connection); + if (cctx && cctx->pool) { + if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) { + cctx->exit_code = 0; + } + + if (r->main->count > cctx->original_request_count) { + r->main->count--; + return; + } + + ngx_destroy_pool(cctx->pool); + cctx->pool = NULL; + + return; + } + } + } +#endif +#endif + if (r->connection->fd != (ngx_socket_t) -1) { ngx_http_finalize_request(r, rc); return; diff --git a/src/ngx_http_lua_util.h b/src/ngx_http_lua_util.h index eab4d4eba2..9d7a0bd1a3 100644 --- a/src/ngx_http_lua_util.h +++ b/src/ngx_http_lua_util.h @@ -32,6 +32,20 @@ #define NGX_HTTP_LUA_ESCAPE_HEADER_VALUE 8 +#ifdef HAVE_PROXY_SSL_PATCH + +#define NGX_HTTP_LUA_CONTEXT_YIELDABLE (NGX_HTTP_LUA_CONTEXT_REWRITE \ + | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE \ + | NGX_HTTP_LUA_CONTEXT_ACCESS \ + | NGX_HTTP_LUA_CONTEXT_CONTENT \ + | NGX_HTTP_LUA_CONTEXT_TIMER \ + | NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY \ + | NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO \ + | NGX_HTTP_LUA_CONTEXT_SSL_CERT \ + | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH) + +#else + #define NGX_HTTP_LUA_CONTEXT_YIELDABLE (NGX_HTTP_LUA_CONTEXT_REWRITE \ | NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE \ | NGX_HTTP_LUA_CONTEXT_ACCESS \ @@ -41,11 +55,15 @@ | NGX_HTTP_LUA_CONTEXT_SSL_CERT \ | NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH) +#endif /* HAVE_PROXY_SSL_PATCH */ + /* key in Lua vm registry for all the "ngx.ctx" tables */ #define ngx_http_lua_ctx_tables_key "ngx_lua_ctx_tables" +#ifdef HAVE_PROXY_SSL_PATCH + #define ngx_http_lua_context_name(c) \ ((c) == NGX_HTTP_LUA_CONTEXT_SET ? "set_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_REWRITE ? "rewrite_by_lua*" \ @@ -59,6 +77,8 @@ : (c) == NGX_HTTP_LUA_CONTEXT_INIT_WORKER ? "init_worker_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_EXIT_WORKER ? "exit_worker_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_BALANCER ? "balancer_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY ? \ + "proxy_ssl_verify_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO ? \ "ssl_client_hello_by_lua*" \ : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CERT ? "ssl_certificate_by_lua*" \ @@ -68,6 +88,32 @@ "ssl_session_fetch_by_lua*" \ : "(unknown)") +#else + +#define ngx_http_lua_context_name(c) \ + ((c) == NGX_HTTP_LUA_CONTEXT_SET ? "set_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_REWRITE ? "rewrite_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SERVER_REWRITE ? "server_rewrite_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_ACCESS ? "access_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_CONTENT ? "content_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_LOG ? "log_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_HEADER_FILTER ? "header_filter_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_BODY_FILTER ? "body_filter_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_TIMER ? "ngx.timer" \ + : (c) == NGX_HTTP_LUA_CONTEXT_INIT_WORKER ? "init_worker_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_EXIT_WORKER ? "exit_worker_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_BALANCER ? "balancer_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CLIENT_HELLO ? \ + "ssl_client_hello_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SSL_CERT ? "ssl_certificate_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SSL_SESS_STORE ? \ + "ssl_session_store_by_lua*" \ + : (c) == NGX_HTTP_LUA_CONTEXT_SSL_SESS_FETCH ? \ + "ssl_session_fetch_by_lua*" \ + : "(unknown)") + +#endif /* HAVE_PROXY_SSL_PATCH */ + #define ngx_http_lua_check_context(L, ctx, flags) \ if (!((ctx)->context & (flags))) { \ diff --git a/t/169-proxy-ssl-verify.t b/t/169-proxy-ssl-verify.t new file mode 100644 index 0000000000..4c44a8b9be --- /dev/null +++ b/t/169-proxy-ssl-verify.t @@ -0,0 +1,1332 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: + +use Test::Nginx::Socket::Lua; + +repeat_each(3); + +# All these tests need to have new openssl +my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; +my $openssl_version = eval { `$NginxBinary -V 2>&1` }; + +if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { + plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +} elsif ($openssl_version =~ m/running with BoringSSL/) { + plan(skip_all => "does not support BoringSSL"); +} elsif ($ENV{TEST_NGINX_USE_HTTP3}) { + plan tests => repeat_each() * (blocks() * 6 + 6); +} else { + plan tests => repeat_each() * (blocks() * 5 + 10); +} + +$ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); +$ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; +$ENV{TEST_NGINX_QUIC_IDLE_TIMEOUT} ||= 0.6; + +#log_level 'warn'; +log_level 'debug'; + +no_long_string(); +#no_diff(); + +run_tests(); + +__DATA__ + +=== TEST 1: invalid proxy_pass url +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("hello world") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass http://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "hello world") + } + } +--- request +GET /t +--- error_log +proxy_ssl_verify_by_lua* should be used with proxy_pass https url +--- must_die + + + +=== TEST 2: proxy_ssl_verify_by_lua in http {} block +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("hello world") + } + + more_clear_headers Date; + } + } + + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "hello world") + } +--- config + location /t { + proxy_pass http://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + } +--- request +GET /t +--- error_log +"proxy_ssl_verify_by_lua_block" directive is not allowed here +--- must_die + + + +=== TEST 3: proxy_ssl_verify_by_lua in server {} block +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("hello world") + } + + more_clear_headers Date; + } + } + +--- config + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "hello world") + } + + location /t { + proxy_pass http://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + } +--- request +GET /t +--- error_log +"proxy_ssl_verify_by_lua_block" directive is not allowed here +--- must_die + + + +=== TEST 4: simple logging +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("simple logging return") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "proxy ssl verify by lua is running!") + } + } +--- request +GET /t +--- response_body +simple logging return +--- error_log +proxy ssl verify by lua is running! +--- no_error_log +[error] +[alert] + + + +=== TEST 5: sleep +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("sleep") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + + proxy_ssl_verify_by_lua_block { + local begin = ngx.now() + ngx.sleep(0.1) + print("elapsed in proxy ssl verify by lua: ", ngx.now() - begin) + } + } +--- request +GET /t +--- response_body +sleep +--- error_log eval +qr/elapsed in proxy ssl verify by lua: 0.(?:09|1\d)\d+ while loading proxy ssl verify by lua,/, +--- no_error_log +[error] +[alert] + + + +=== TEST 6: timer +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("timer") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + + proxy_ssl_verify_by_lua_block { + local function f() + print("my timer run!") + end + local ok, err = ngx.timer.at(0, f) + if not ok then + ngx.log(ngx.ERR, "failed to create timer: ", err) + return + end + } + } +--- request +GET /t +--- response_body +timer +--- error_log +my timer run! +--- no_error_log +[error] +[alert] + + + +=== TEST 7: ngx.exit(0) - no yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("ngx.exit(0) no yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + + proxy_ssl_verify_by_lua_block { + ngx.exit(0) + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- response_body +ngx.exit(0) no yield +--- error_log +lua exit with code 0 +--- no_error_log +should never reached here +[error] +[alert] +[emerg] + + + +=== TEST 8: ngx.exit(ngx.ERROR) - no yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("ngx.exit(ngx.ERROR) no yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.exit(ngx.ERROR) + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- error_code: 502 +--- error_log eval +[ +'lua exit with code -1', +'proxy_ssl_verify_by_lua: handler return value: -1, cert verify callback exit code: 0', +qr/.*? SSL_do_handshake\(\) failed .*?certificate verify failed/, +] +--- no_error_log +should never reached here +[error] +[alert] +[emerg] + + + +=== TEST 9: ngx.exit(0) - yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("ngx.exit(0) yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.sleep(0.001) + ngx.exit(0) + + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- response_body +ngx.exit(0) yield +--- error_log +lua exit with code 0 +--- no_error_log +should never reached here +[error] +[alert] +[emerg] + + + +=== TEST 10: ngx.exit(ngx.ERROR) - yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("ngx.exit(ngx.ERROR) yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.sleep(0.001) + ngx.exit(ngx.ERROR) + + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- error_code: 502 +--- error_log eval +[ +'lua exit with code -1', +'proxy_ssl_verify_by_lua: cert verify callback exit code: 0', +qr/.*? SSL_do_handshake\(\) failed .*?certificate verify failed/, +] +--- no_error_log +should never reached here +[error] +[alert] +[emerg] + + + +=== TEST 11: lua exception - no yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("lua exception - no yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + error("bad bad bad") + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- error_code: 502 +--- error_log eval +[ +'runtime error: proxy_ssl_verify_by_lua(nginx.conf:65):2: bad bad bad', +'proxy_ssl_verify_by_lua: handler return value: 500, cert verify callback exit code: 0', +qr/.*? SSL_do_handshake\(\) failed .*?certificate verify failed/, +] +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 12: lua exception - yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("lua exception - yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.sleep(0.001) + error("bad bad bad") + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- error_code: 502 +--- error_log eval +[ +'runtime error: proxy_ssl_verify_by_lua(nginx.conf:65):3: bad bad bad', +'proxy_ssl_verify_by_lua: cert verify callback exit code: 0', +qr/.*? SSL_do_handshake\(\) failed .*?certificate verify failed/, +] +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 13: get phase +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("get phase return") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + print("get_phase: ", ngx.get_phase()) + } + } +--- request +GET /t +--- response_body +get phase return +--- error_log +get_phase: proxy_ssl_verify +--- no_error_log +[error] +[alert] + + + +=== TEST 14: subrequests disabled +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("subrequests disabled") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.location.capture("/foo") + } + } +--- request +GET /t +--- error_code: 502 +--- error_log eval +[ +'proxy_ssl_verify_by_lua(nginx.conf:65):2: API disabled in the context of proxy_ssl_verify_by_lua*', +'proxy_ssl_verify_by_lua: handler return value: 500, cert verify callback exit code: 0', +qr/.*? SSL_do_handshake\(\) failed .*?certificate verify failed/, +] +--- no_error_log +[alert] + + + +=== TEST 15: simple logging (by_lua_file) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("simple logging by lua file") + } + + more_clear_headers Date; + } + } +--- user_files +>>> a.lua +print("proxy ssl verify by lua is running!") + +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_file html/a.lua; + } +--- request +GET /t +--- response_body +simple logging by lua file +--- error_log +a.lua:1: proxy ssl verify by lua is running! +--- no_error_log +[error] +[alert] + + + +=== TEST 16: coroutine API +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("coroutine API") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + local cc, cr, cy = coroutine.create, coroutine.resume, coroutine.yield + + local function f() + local cnt = 0 + for i = 1, 20 do + print("co yield: ", cnt) + cy() + cnt = cnt + 1 + end + end + + local c = cc(f) + for i = 1, 3 do + print("co resume, status: ", coroutine.status(c)) + cr(c) + end + } + } +--- request +GET /t +--- response_body +coroutine API +--- grep_error_log eval: qr/co (?:yield: \d+|resume, status: \w+)/ +--- grep_error_log_out +co resume, status: suspended +co yield: 0 +co resume, status: suspended +co yield: 1 +co resume, status: suspended +co yield: 2 +--- no_error_log +[error] +[alert] + + + +=== TEST 17: simple user thread wait with yielding +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("simple user thread wait with yielding") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + local function f() + ngx.sleep(0.01) + print("uthread: hello in thread") + return "done" + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "uthread: failed to spawn thread: ", err) + return ngx.exit(ngx.ERROR) + end + + print("uthread: thread created: ", coroutine.status(t)) + + local ok, res = ngx.thread.wait(t) + if not ok then + print("uthread: failed to wait thread: ", res) + return + end + + print("uthread: ", res) + } + } +--- request +GET /t +--- response_body +simple user thread wait with yielding +--- no_error_log +[error] +[alert] +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out +uthread: thread created: running while loading proxy ssl verify by lua +uthread: hello in thread while loading proxy ssl verify by lua +uthread: done while loading proxy ssl verify by lua + + + +=== TEST 18: uthread (kill) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("uthread (kill)") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + local function f() + ngx.log(ngx.INFO, "uthread: hello from f()") + ngx.sleep(1) + end + + local t, err = ngx.thread.spawn(f) + if not t then + ngx.log(ngx.ERR, "failed to spawn thread: ", err) + return ngx.exit(ngx.ERROR) + end + + local ok, res = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.ERR, "failed to kill thread: ", res) + return + end + + ngx.log(ngx.INFO, "uthread: killed") + + local ok, err = ngx.thread.kill(t) + if not ok then + ngx.log(ngx.INFO, "uthread: failed to kill: ", err) + end + } + } +--- request +GET /t +--- response_body +uthread (kill) +--- no_error_log +[error] +[alert] +[emerg] +--- grep_error_log eval: qr/uthread: [^.,]+/ +--- grep_error_log_out +uthread: hello from f() while loading proxy ssl verify by lua +uthread: killed while loading proxy ssl verify by lua +uthread: failed to kill: already waited or killed while loading proxy ssl verify by lua + + + +=== TEST 19: ngx.exit(ngx.OK) - no yield +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("ngx.exit(ngx.OK) - no yield") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.exit(ngx.OK) + ngx.log(ngx.ERR, "should never reached here...") + } + } +--- request +GET /t +--- response_body +ngx.exit(ngx.OK) - no yield +--- error_log eval +[ +'proxy_ssl_verify_by_lua: handler return value: 0, cert verify callback exit code: 1', +qr/\[debug\] .*? SSL_do_handshake: 1/, +'lua exit with code 0', +] +--- no_error_log +should never reached here +[alert] +[emerg] + + + +=== TEST 20: proxy_ssl_verify_by_lua* without yield API (simple logic) +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("without yield API, simple logic") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + print("proxy ssl verify: simple test start") + + -- Simple calculations without yield + local sum = 0 + for i = 1, 10 do + sum = sum + i + end + + print("proxy ssl verify: calculated sum: ", sum) + + -- String operations + local str = "hello" + str = str .. " world" + print("proxy ssl verify: concatenated string: ", str) + + -- Table operations + local t = {a = 1, b = 2, c = 3} + local count = 0 + for k, v in pairs(t) do + count = count + v + end + print("proxy ssl verify: table sum: ", count) + + print("proxy ssl verify: simple test done") + } + } +--- request +GET /t +--- response_body +without yield API, simple logic +--- grep_error_log eval: qr/(proxy ssl verify: simple test start|proxy ssl verify: calculated sum: 55|proxy ssl verify: concatenated string: hello world|proxy ssl verify: table sum: 6|proxy ssl verify: simple test done)/ +--- grep_error_log_out +proxy ssl verify: simple test start +proxy ssl verify: calculated sum: 55 +proxy ssl verify: concatenated string: hello world +proxy ssl verify: table sum: 6 +proxy ssl verify: simple test done + +--- no_error_log +[error] +[alert] +[emerg] + + + +=== TEST 21: lua_upstream_skip_openssl_default_verify default off +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("lua_upstream_skip_openssl_default_verify default off") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "proxy ssl verify by lua is running!") + } + } +--- request +GET /t +--- error_log +proxy_ssl_verify_by_lua: openssl default verify +--- no_error_log +[error] +[alert] + + + +=== TEST 22: lua_upstream_skip_openssl_default_verify on +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("lua_upstream_skip_openssl_default_verify default off") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_ssl_conf_command VerifyMode Peer; + + lua_upstream_skip_openssl_default_verify on; + + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "proxy ssl verify by lua is running!") + } + } +--- request +GET /t +--- response_body +lua_upstream_skip_openssl_default_verify default off +--- error_log +proxy ssl verify by lua is running! +--- no_error_log +proxy_ssl_verify_by_lua: openssl default verify +[error] +[alert] + + + +=== TEST 23: ngx.ctx to pass data from downstream phase to upstream phase +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("simple logging return") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + + rewrite_by_lua_block { + ngx.ctx.greeting = "I am from rewrite phase" + } + + proxy_ssl_verify_by_lua_block { + ngx.log(ngx.INFO, "greeting: ", ngx.ctx.greeting) + } + } +--- request +GET /t +--- response_body +simple logging return +--- error_log +greeting: I am from rewrite phase +--- no_error_log +[error] +[alert] + + + +=== TEST 24: upstream connection aborted +--- http_config + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("hello world") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + proxy_connect_timeout 100ms; + + proxy_ssl_verify_by_lua_block { + ngx.sleep(0.2) + } + } +--- request +GET /t +--- error_code: 504 +--- response_body_like: 504 Gateway Time-out +--- error_log +upstream timed out (110: Connection timed out) while loading proxy ssl verify by lua +proxy_ssl_verify_by_lua: cert verify callback aborted +--- no_error_log +[alert] +--- wait: 0.5 + + + +=== TEST 25: cosocket +--- http_config + server { + listen *:80; + server_name test.com; + + server_tokens off; + location /foo { + default_type 'text/plain'; + content_by_lua_block { + ngx.sleep(0.1) + + ngx.status = 201 + ngx.say("foo") + ngx.exit(201) + } + more_clear_headers Date; + } + } + + server { + listen unix:$TEST_NGINX_HTML_DIR/nginx.sock ssl; + server_name test.com; + + ssl_certificate ../../cert/mtls_server.crt; + ssl_certificate_key ../../cert/mtls_server.key; + + location / { + default_type 'text/plain'; + + content_by_lua_block { + ngx.say("simple logging return") + } + + more_clear_headers Date; + } + } +--- config + location /t { + proxy_pass https://unix:$TEST_NGINX_HTML_DIR/nginx.sock; + proxy_ssl_verify on; + proxy_ssl_name example.com; + proxy_ssl_certificate ../../cert/mtls_client.crt; + proxy_ssl_certificate_key ../../cert/mtls_client.key; + proxy_ssl_trusted_certificate ../../cert/mtls_ca.crt; + proxy_ssl_session_reuse off; + + proxy_ssl_verify_by_lua_block { + do + local sock = ngx.socket.tcp() + sock:settimeout(2000) + + local ok, err = sock:connect("127.0.0.1", "80") + if not ok then + ngx.log(ngx.ERR, "failed to connect: ", err) + return + end + + ngx.log(ngx.INFO, "connected: ", ok) + + local req = "GET /foo HTTP/1.0\r\nHost: test.com\r\nConnection: close\r\n\r\n" + local bytes, err = sock:send(req) + if not bytes then + ngx.log(ngx.ERR, "failed to send http request: ", err) + return + end + + ngx.log(ngx.INFO, "sent http request: ", bytes, " bytes.") + + while true do + local line, err = sock:receive() + if not line then + -- ngx.log(ngx.ERR, "failed to receive response status line: ", err) + break + end + + ngx.log(ngx.INFO, "received: ", line) + end + + local ok, err = sock:close() + ngx.log(ngx.INFO, "close: ", ok, " ", err) + end -- do + -- collectgarbage() + } + } +--- request +GET /t +--- response_body +simple logging return +--- error_log +connected: 1 +sent http request: 56 bytes. +received: HTTP/1.1 201 Created +received: Server: openresty +received: Content-Type: text/plain +received: Content-Length: 4 +received: Connection: close +received: +received: foo +close: 1 nil +--- no_error_log +[error] +[alert] From dc8de6ef3f075f04ed2ab8a5dbbb13f3b8d22227 Mon Sep 17 00:00:00 2001 From: swananan Date: Sat, 18 Oct 2025 13:13:48 +0800 Subject: [PATCH 834/848] bugfix: resume QUIC handshake for OpenSSL external QUIC API builds --- src/ngx_http_lua_proxy_ssl_verifyby.c | 8 +++++--- src/ngx_http_lua_ssl_certby.c | 2 +- src/ngx_http_lua_ssl_client_helloby.c | 2 +- src/ngx_http_lua_util.c | 1 + 4 files changed, 8 insertions(+), 5 deletions(-) diff --git a/src/ngx_http_lua_proxy_ssl_verifyby.c b/src/ngx_http_lua_proxy_ssl_verifyby.c index 1695e56ffb..339522bab3 100644 --- a/src/ngx_http_lua_proxy_ssl_verifyby.c +++ b/src/ngx_http_lua_proxy_ssl_verifyby.c @@ -62,7 +62,7 @@ ngx_http_lua_proxy_ssl_verify_set_callback(ngx_conf_t *cf) return NGX_ERROR; } -#if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ +#if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ || OPENSSL_VERSION_NUMBER < 0x30000020L) ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "OpenSSL too old to support " @@ -150,7 +150,7 @@ char * ngx_http_lua_proxy_ssl_verify_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { -#if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ +#if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ || OPENSSL_VERSION_NUMBER < 0x30000020L) /* SSL_set_retry_verify() was added in OpenSSL 3.0.2 */ @@ -339,7 +339,8 @@ ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, "proxy_ssl_verify_by_lua: handler return value: %i, " - "cert verify callback exit code: %d", rc, cctx->exit_code); + "cert verify callback exit code: %d", rc, + cctx->exit_code); c->log->action = "proxy pass SSL handshaking"; return cctx->exit_code; @@ -370,6 +371,7 @@ ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) return SSL_set_retry_verify(ssl_conn); failed: + if (cctx && cctx->pool) { ngx_destroy_pool(cctx->pool); } diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 9a2d63fcf0..0667ec4293 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -392,7 +392,7 @@ ngx_http_lua_ssl_cert_done(void *data) #if (HAVE_QUIC_SSL_LUA_YIELD_PATCH && NGX_HTTP_V3) # if OPENSSL_VERSION_NUMBER >= 0x1000205fL -# if (NGX_QUIC_OPENSSL_COMPAT) +# if (NGX_QUIC_OPENSSL_COMPAT || NGX_QUIC_OPENSSL_API) ngx_http_lua_resume_quic_ssl_handshake(c); # endif # endif diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index b600ab3636..d1f86414af 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -392,7 +392,7 @@ ngx_http_lua_ssl_client_hello_done(void *data) #if (HAVE_QUIC_SSL_LUA_YIELD_PATCH && NGX_HTTP_V3) # if defined(SSL_ERROR_WANT_CLIENT_HELLO_CB) -# if (NGX_QUIC_OPENSSL_COMPAT) +# if (NGX_QUIC_OPENSSL_COMPAT || NGX_QUIC_OPENSSL_API) ngx_http_lua_resume_quic_ssl_handshake(c); # endif # endif diff --git a/src/ngx_http_lua_util.c b/src/ngx_http_lua_util.c index 928a15beed..96e9ae5b20 100644 --- a/src/ngx_http_lua_util.c +++ b/src/ngx_http_lua_util.c @@ -1682,6 +1682,7 @@ ngx_http_lua_run_thread(lua_State *L, ngx_http_request_t *r, NGX_ERROR : NGX_HTTP_INTERNAL_SERVER_ERROR; done: + #ifdef HAVE_PROXY_SSL_PATCH if (ctx->context == NGX_HTTP_LUA_CONTEXT_PROXY_SSL_VERIFY) { return NGX_OK; From 8d553f059e61182e3c6a08d723fc3b5e688a8693 Mon Sep 17 00:00:00 2001 From: willmafh Date: Sun, 19 Oct 2025 14:21:42 +0800 Subject: [PATCH 835/848] bugfix: Nginx introduces ssl client hello callback in version 1.29.2. so we need to change nginx_version value to 1029001 here --- src/ngx_http_lua_ssl_client_helloby.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_ssl_client_helloby.c b/src/ngx_http_lua_ssl_client_helloby.c index d1f86414af..283d1017f0 100644 --- a/src/ngx_http_lua_ssl_client_helloby.c +++ b/src/ngx_http_lua_ssl_client_helloby.c @@ -220,7 +220,7 @@ ngx_http_lua_ssl_client_hello_handler(ngx_ssl_conn_t *ssl_conn, dd("first time"); -#if (nginx_version > 1029000) +#if (nginx_version > 1029001) /* see commit 0373fe5d98c1515640 for more details */ rc = ngx_ssl_client_hello_callback(ssl_conn, al, arg); From 13ba02a046854812294ae6ce5d0b38ff24d38c3c Mon Sep 17 00:00:00 2001 From: willmafh Date: Mon, 20 Oct 2025 09:22:18 +0800 Subject: [PATCH 836/848] tests: fixed CI. Co-authored-by: lijunlong --- src/ngx_http_lua_proxy_ssl_verifyby.c | 93 ++++++++++++++++++++++++++- t/169-proxy-ssl-verify.t | 21 +++--- 2 files changed, 105 insertions(+), 9 deletions(-) diff --git a/src/ngx_http_lua_proxy_ssl_verifyby.c b/src/ngx_http_lua_proxy_ssl_verifyby.c index 339522bab3..1cb1d980e4 100644 --- a/src/ngx_http_lua_proxy_ssl_verifyby.c +++ b/src/ngx_http_lua_proxy_ssl_verifyby.c @@ -33,13 +33,20 @@ ngx_int_t ngx_http_lua_proxy_ssl_verify_set_callback(ngx_conf_t *cf) { -#ifdef LIBRESSL_VERSION_NUMBER +#if defined(LIBRESSL_VERSION_NUMBER) ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "LibreSSL does not support by proxy_ssl_verify_by_lua*"); return NGX_ERROR; +#elif defined(OPENSSL_IS_BORINGSSL) + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "BoringSSL does not support by proxy_ssl_verify_by_lua*"); + + return NGX_ERROR; + #else void *plcf; @@ -150,6 +157,22 @@ char * ngx_http_lua_proxy_ssl_verify_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { +#if defined(LIBRESSL_VERSION_NUMBER) + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "LibreSSL does not support by proxy_ssl_verify_by_lua*"); + + return NGX_CONF_ERROR; + +#elif defined(OPENSSL_IS_BORINGSSL) + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "BoringSSL does not support by proxy_ssl_verify_by_lua*"); + + return NGX_CONF_ERROR; + +#else + #if (!defined SSL_ERROR_WANT_RETRY_VERIFY \ || OPENSSL_VERSION_NUMBER < 0x30000020L) @@ -231,12 +254,30 @@ ngx_http_lua_proxy_ssl_verify_by_lua(ngx_conf_t *cf, ngx_command_t *cmd, return NGX_CONF_OK; #endif /* SSL_ERROR_WANT_RETRY_VERIFY */ + +#endif } int ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) { +#if defined(LIBRESSL_VERSION_NUMBER) + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + "LibreSSL does not support by proxy_ssl_verify_by_lua*"); + + return 1; + +#elif defined(OPENSSL_IS_BORINGSSL) + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + "BoringSSL does not support by proxy_ssl_verify_by_lua*"); + + return 1; + +#else + lua_State *L; ngx_int_t rc; ngx_connection_t *c; @@ -377,6 +418,8 @@ ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) } return 0; /* verify failure or error */ + +#endif } @@ -553,6 +596,20 @@ int ngx_http_lua_ffi_proxy_ssl_set_verify_result(ngx_http_request_t *r, int verify_result, char **err) { +#if defined(LIBRESSL_VERSION_NUMBER) + + *err = "LibreSSL does not support this function"; + + return NGX_ERROR; + +#elif defined(OPENSSL_IS_BORINGSSL) + + *err = "BoringSSL does not support this function"; + + return NGX_ERROR; + +#else + #ifdef SSL_ERROR_WANT_RETRY_VERIFY ngx_http_upstream_t *u; ngx_ssl_conn_t *ssl_conn; @@ -598,12 +655,28 @@ ngx_http_lua_ffi_proxy_ssl_set_verify_result(ngx_http_request_t *r, return NGX_ERROR; #endif + +#endif } int ngx_http_lua_ffi_proxy_ssl_get_verify_result(ngx_http_request_t *r, char **err) { +#if defined(LIBRESSL_VERSION_NUMBER) + + *err = "LibreSSL does not support this function"; + + return NGX_ERROR; + +#elif defined(OPENSSL_IS_BORINGSSL) + + *err = "BoringSSL does not support this function"; + + return NGX_ERROR; + +#else + #ifdef SSL_ERROR_WANT_RETRY_VERIFY ngx_http_upstream_t *u; ngx_ssl_conn_t *ssl_conn; @@ -647,6 +720,8 @@ ngx_http_lua_ffi_proxy_ssl_get_verify_result(ngx_http_request_t *r, char **err) return NGX_ERROR; #endif + +#endif } @@ -662,6 +737,20 @@ ngx_http_lua_ffi_proxy_ssl_free_verify_cert(void *cdata) void * ngx_http_lua_ffi_proxy_ssl_get_verify_cert(ngx_http_request_t *r, char **err) { +#if defined(LIBRESSL_VERSION_NUMBER) + + *err = "LibreSSL does not support this function"; + + return NGX_ERROR; + +#elif defined(OPENSSL_IS_BORINGSSL) + + *err = "BoringSSL does not support this function"; + + return NGX_ERROR; + +#else + #ifdef SSL_ERROR_WANT_RETRY_VERIFY ngx_http_upstream_t *u; ngx_ssl_conn_t *ssl_conn; @@ -713,6 +802,8 @@ ngx_http_lua_ffi_proxy_ssl_get_verify_cert(ngx_http_request_t *r, char **err) return NULL; #endif + +#endif } diff --git a/t/169-proxy-ssl-verify.t b/t/169-proxy-ssl-verify.t index 4c44a8b9be..95ecc1f3a7 100644 --- a/t/169-proxy-ssl-verify.t +++ b/t/169-proxy-ssl-verify.t @@ -8,14 +8,19 @@ repeat_each(3); my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; -if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { - plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); +if ($openssl_version =~ m/built with OpenSSL (\d+)\.(\d+)\.(\d+)/) { + my ($major, $minor, $patch) = ($1, $2, $3); + + if ($major < 3 || ($major == 3 && $minor == 0 && $patch < 2)) { + plan(skip_all => "too old OpenSSL, need >= 3.0.2, was " . + "$major.$minor.$patch"); + } else { + plan tests => repeat_each() * (blocks() * 5 + 19); + } } elsif ($openssl_version =~ m/running with BoringSSL/) { plan(skip_all => "does not support BoringSSL"); -} elsif ($ENV{TEST_NGINX_USE_HTTP3}) { - plan tests => repeat_each() * (blocks() * 6 + 6); } else { - plan tests => repeat_each() * (blocks() * 5 + 10); + die "unknown SSL"; } $ENV{TEST_NGINX_HTML_DIR} ||= html_dir(); @@ -1230,7 +1235,7 @@ proxy_ssl_verify_by_lua: cert verify callback aborted === TEST 25: cosocket --- http_config server { - listen *:80; + listen 127.0.0.1:$TEST_NGINX_RAND_PORT_1; server_name test.com; server_tokens off; @@ -1279,7 +1284,7 @@ proxy_ssl_verify_by_lua: cert verify callback aborted local sock = ngx.socket.tcp() sock:settimeout(2000) - local ok, err = sock:connect("127.0.0.1", "80") + local ok, err = sock:connect("127.0.0.1", $TEST_NGINX_RAND_PORT_1) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err) return @@ -1320,7 +1325,7 @@ simple logging return connected: 1 sent http request: 56 bytes. received: HTTP/1.1 201 Created -received: Server: openresty +received: Server: nginx received: Content-Type: text/plain received: Content-Length: 4 received: Connection: close From 529dc7ae028f5adb80e069be7404d671bd8da7aa Mon Sep 17 00:00:00 2001 From: lijunlong Date: Tue, 21 Oct 2025 08:32:45 +0800 Subject: [PATCH 837/848] feature: bumped api version to 0.10.29. --- src/api/ngx_http_lua_api.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/ngx_http_lua_api.h b/src/api/ngx_http_lua_api.h index fa59095c6a..51a9e7b714 100644 --- a/src/api/ngx_http_lua_api.h +++ b/src/api/ngx_http_lua_api.h @@ -19,7 +19,7 @@ /* Public API for other Nginx modules */ -#define ngx_http_lua_version 10028 +#define ngx_http_lua_version 10029 #define NGX_HTTP_LUA_EXPORT_CO_CTX_CLEANUP 1 From 9f1b459b9fc4a7ece7ec24397afa4fc886a15f85 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 22 Oct 2025 09:28:22 +0800 Subject: [PATCH 838/848] bugfix: failed to build with openssl < 3.0.2. --- src/ngx_http_lua_proxy_ssl_verifyby.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ngx_http_lua_proxy_ssl_verifyby.c b/src/ngx_http_lua_proxy_ssl_verifyby.c index 1cb1d980e4..9d4107359a 100644 --- a/src/ngx_http_lua_proxy_ssl_verifyby.c +++ b/src/ngx_http_lua_proxy_ssl_verifyby.c @@ -23,8 +23,10 @@ #include "ngx_http_lua_proxy_ssl_verifyby.h" +#if defined(OPENSSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x30000020uL) static void ngx_http_lua_proxy_ssl_verify_done(void *data); static void ngx_http_lua_proxy_ssl_verify_aborted(void *data); +#endif static ngx_int_t ngx_http_lua_proxy_ssl_verify_by_chunk(lua_State *L, ngx_http_request_t *r); @@ -275,6 +277,12 @@ ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) "BoringSSL does not support by proxy_ssl_verify_by_lua*"); return 1; +#elif defined(OPENSSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER < 0x30000020uL) + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + "OpenSSL(< 3.0.2) does not support by proxy_ssl_verify_by_lua*"); + + return 1; #else @@ -423,6 +431,7 @@ ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) } +#if defined(OPENSSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x30000020uL) static void ngx_http_lua_proxy_ssl_verify_done(void *data) { @@ -482,6 +491,7 @@ ngx_http_lua_proxy_ssl_verify_aborted(void *data) cctx->pool = NULL; } } +#endif static ngx_int_t From 9d930a7c2bfafe34b14b6664cdb5b5eabfcd372d Mon Sep 17 00:00:00 2001 From: lijunlong Date: Wed, 22 Oct 2025 10:46:21 +0800 Subject: [PATCH 839/848] bugfix: failed to build with openssl 1.x.x/boringssl/libressl. --- src/ngx_http_lua_proxy_ssl_verifyby.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/ngx_http_lua_proxy_ssl_verifyby.c b/src/ngx_http_lua_proxy_ssl_verifyby.c index 9d4107359a..c8d783ab4b 100644 --- a/src/ngx_http_lua_proxy_ssl_verifyby.c +++ b/src/ngx_http_lua_proxy_ssl_verifyby.c @@ -265,13 +265,26 @@ int ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) { #if defined(LIBRESSL_VERSION_NUMBER) + ngx_connection_t *c; + c = ngx_ssl_get_connection(ssl_conn); /* upstream connection */ + ngx_ssl_conn_t *ssl_conn; + + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, + SSL_get_ex_data_X509_STORE_CTX_idx()); ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "LibreSSL does not support by proxy_ssl_verify_by_lua*"); return 1; #elif defined(OPENSSL_IS_BORINGSSL) + ngx_connection_t *c; + ngx_ssl_conn_t *ssl_conn; + + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, + SSL_get_ex_data_X509_STORE_CTX_idx()); + c = ngx_ssl_get_connection(ssl_conn); /* upstream connection */ + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "BoringSSL does not support by proxy_ssl_verify_by_lua*"); @@ -279,6 +292,13 @@ ngx_http_lua_proxy_ssl_verify_handler(X509_STORE_CTX *x509_store, void *arg) return 1; #elif defined(OPENSSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER < 0x30000020uL) + ngx_connection_t *c; + ngx_ssl_conn_t *ssl_conn; + + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, + SSL_get_ex_data_X509_STORE_CTX_idx()); + c = ngx_ssl_get_connection(ssl_conn); /* upstream connection */ + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "OpenSSL(< 3.0.2) does not support by proxy_ssl_verify_by_lua*"); From 1a71cc8ea859abcbe551c7d4c052dcf582f5be9c Mon Sep 17 00:00:00 2001 From: willmafh Date: Thu, 23 Oct 2025 13:39:48 +0800 Subject: [PATCH 840/848] bugfix: typo fixes. --- src/ngx_http_lua_ssl_certby.c | 8 ++++---- src/ngx_http_lua_ssl_session_fetchby.c | 2 +- t/139-ssl-cert-by.t | 8 ++++---- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/ngx_http_lua_ssl_certby.c b/src/ngx_http_lua_ssl_certby.c index 0667ec4293..5d16834212 100644 --- a/src/ngx_http_lua_ssl_certby.c +++ b/src/ngx_http_lua_ssl_certby.c @@ -217,7 +217,7 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) if (cctx->done) { ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua_certificate_by_lua: cert cb exit code: %d", + "ssl_certificate_by_lua: cert cb exit code: %d", cctx->exit_code); dd("lua ssl cert done, finally"); @@ -319,7 +319,7 @@ ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data) } ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, - "lua_certificate_by_lua: handler return value: %i, " + "ssl_certificate_by_lua: handler return value: %i, " "cert cb exit code: %d", rc, cctx->exit_code); c->log->action = "SSL handshaking"; @@ -405,7 +405,7 @@ ngx_http_lua_ssl_cert_aborted(void *data) { ngx_http_lua_ssl_ctx_t *cctx = data; - dd("lua ssl cert done"); + dd("lua ssl cert aborted"); if (cctx->done) { /* completed successfully already */ @@ -413,7 +413,7 @@ ngx_http_lua_ssl_cert_aborted(void *data) } ngx_log_debug0(NGX_LOG_DEBUG_HTTP, cctx->connection->log, 0, - "lua_certificate_by_lua: cert cb aborted"); + "ssl_certificate_by_lua: cert cb aborted"); cctx->aborted = 1; cctx->request->connection->ssl = NULL; diff --git a/src/ngx_http_lua_ssl_session_fetchby.c b/src/ngx_http_lua_ssl_session_fetchby.c index ebce63ce28..fee039fca8 100644 --- a/src/ngx_http_lua_ssl_session_fetchby.c +++ b/src/ngx_http_lua_ssl_session_fetchby.c @@ -416,7 +416,7 @@ ngx_http_lua_ssl_sess_fetch_aborted(void *data) { ngx_http_lua_ssl_ctx_t *cctx = data; - dd("lua ssl sess_fetch done"); + dd("lua ssl sess_fetch aborted"); if (cctx->done) { /* completed successfully already */ diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index c5b1665046..82ae791787 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -582,7 +582,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'lua_certificate_by_lua: handler return value: -1, cert cb exit code: 0', +'ssl_certificate_by_lua: handler return value: -1, cert cb exit code: 0', qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, 'lua exit with code -1', ] @@ -723,7 +723,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ -'lua_certificate_by_lua: cert cb exit code: 0', +'ssl_certificate_by_lua: cert cb exit code: 0', qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, 'lua exit with code -1', ] @@ -794,7 +794,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'runtime error: ssl_certificate_by_lua(nginx.conf:28):2: bad bad bad', -'lua_certificate_by_lua: handler return value: 500, cert cb exit code: 0', +'ssl_certificate_by_lua: handler return value: 500, cert cb exit code: 0', qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, qr/context: ssl_certificate_by_lua\*, client: \d+\.\d+\.\d+\.\d+, server: \d+\.\d+\.\d+\.\d+:\d+/, ] @@ -866,7 +866,7 @@ failed to do SSL handshake: handshake failed --- error_log eval [ 'runtime error: ssl_certificate_by_lua(nginx.conf:28):3: bad bad bad', -'lua_certificate_by_lua: cert cb exit code: 0', +'ssl_certificate_by_lua: cert cb exit code: 0', qr/(\[info\] .*? SSL_do_handshake\(\) failed .*?cert cb error|routines:OPENSSL_internal:CERT_CB_ERROR)/, ] From a47ad5201d9cd7ab2256fd628b6a0ac04b1615bd Mon Sep 17 00:00:00 2001 From: "zhuo.z" Date: Thu, 23 Oct 2025 13:48:25 +0800 Subject: [PATCH 841/848] bugfix: resolve subrequest cycle issue when Lua code cache is disabled. Co-authored-by: zhuo.zhang --- src/ngx_http_lua_accessby.c | 8 +++++ src/ngx_http_lua_bodyfilterby.c | 8 +++++ src/ngx_http_lua_contentby.c | 8 +++++ src/ngx_http_lua_headerfilterby.c | 8 +++++ src/ngx_http_lua_logby.c | 8 +++++ src/ngx_http_lua_rewriteby.c | 8 +++++ t/014-bugs.t | 60 ++++++++++++++++++++++++++++++- 7 files changed, 107 insertions(+), 1 deletion(-) diff --git a/src/ngx_http_lua_accessby.c b/src/ngx_http_lua_accessby.c index 31ee82fd93..39bdcda2bf 100644 --- a/src/ngx_http_lua_accessby.c +++ b/src/ngx_http_lua_accessby.c @@ -173,6 +173,10 @@ ngx_http_lua_access_handler_inline(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->access_src_ref = LUA_REFNIL; + } + /* load Lua inline script (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->access_src.value.data, @@ -214,6 +218,10 @@ ngx_http_lua_access_handler_file(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->access_src_ref = LUA_REFNIL; + } + /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, &llcf->access_src_ref, diff --git a/src/ngx_http_lua_bodyfilterby.c b/src/ngx_http_lua_bodyfilterby.c index 179a501a59..cb5f7eb608 100644 --- a/src/ngx_http_lua_bodyfilterby.c +++ b/src/ngx_http_lua_bodyfilterby.c @@ -155,6 +155,10 @@ ngx_http_lua_body_filter_inline(ngx_http_request_t *r, ngx_chain_t *in) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->body_filter_src_ref = LUA_REFNIL; + } + /* load Lua inline script (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->body_filter_src.value.data, @@ -206,6 +210,10 @@ ngx_http_lua_body_filter_file(ngx_http_request_t *r, ngx_chain_t *in) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->body_filter_src_ref = LUA_REFNIL; + } + /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, &llcf->body_filter_src_ref, diff --git a/src/ngx_http_lua_contentby.c b/src/ngx_http_lua_contentby.c index 5e2ae55209..039f23ffcd 100644 --- a/src/ngx_http_lua_contentby.c +++ b/src/ngx_http_lua_contentby.c @@ -267,6 +267,10 @@ ngx_http_lua_content_handler_file(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->content_src_ref = LUA_REFNIL; + } + /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, &llcf->content_src_ref, @@ -297,6 +301,10 @@ ngx_http_lua_content_handler_inline(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->content_src_ref = LUA_REFNIL; + } + /* load Lua inline script (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->content_src.value.data, diff --git a/src/ngx_http_lua_headerfilterby.c b/src/ngx_http_lua_headerfilterby.c index 71553558bb..d41c055ef5 100644 --- a/src/ngx_http_lua_headerfilterby.c +++ b/src/ngx_http_lua_headerfilterby.c @@ -165,6 +165,10 @@ ngx_http_lua_header_filter_inline(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->header_filter_src_ref = LUA_REFNIL; + } + /* load Lua inline script (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->header_filter_src.value.data, @@ -210,6 +214,10 @@ ngx_http_lua_header_filter_file(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->header_filter_src_ref = LUA_REFNIL; + } + /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, &llcf->header_filter_src_ref, diff --git a/src/ngx_http_lua_logby.c b/src/ngx_http_lua_logby.c index b47058be19..cded8d8d04 100644 --- a/src/ngx_http_lua_logby.c +++ b/src/ngx_http_lua_logby.c @@ -149,6 +149,10 @@ ngx_http_lua_log_handler_inline(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->log_src_ref = LUA_REFNIL; + } + /* load Lua inline script (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->log_src.value.data, @@ -188,6 +192,10 @@ ngx_http_lua_log_handler_file(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->log_src_ref = LUA_REFNIL; + } + /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, &llcf->log_src_ref, diff --git a/src/ngx_http_lua_rewriteby.c b/src/ngx_http_lua_rewriteby.c index 2c1cc04c7a..69302ba523 100644 --- a/src/ngx_http_lua_rewriteby.c +++ b/src/ngx_http_lua_rewriteby.c @@ -177,6 +177,10 @@ ngx_http_lua_rewrite_handler_inline(ngx_http_request_t *r) llcf = ngx_http_get_module_loc_conf(r, ngx_http_lua_module); L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->rewrite_src_ref = LUA_REFNIL; + } + /* load Lua inline script (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadbuffer(r->connection->log, L, llcf->rewrite_src.value.data, @@ -217,6 +221,10 @@ ngx_http_lua_rewrite_handler_file(ngx_http_request_t *r) L = ngx_http_lua_get_lua_vm(r, NULL); + if (!llcf->enable_code_cache) { + llcf->rewrite_src_ref = LUA_REFNIL; + } + /* load Lua script file (w/ cache) sp = 1 */ rc = ngx_http_lua_cache_loadfile(r->connection->log, L, script_path, &llcf->rewrite_src_ref, diff --git a/t/014-bugs.t b/t/014-bugs.t index cef20733c2..b1c7d442ae 100644 --- a/t/014-bugs.t +++ b/t/014-bugs.t @@ -9,7 +9,7 @@ log_level('debug'); repeat_each(3); # NB: the shutdown_error_log block is independent from repeat times -plan tests => repeat_each() * (blocks() * 2 + 33); +plan tests => repeat_each() * (blocks() * 2 + 41); our $HtmlDir = html_dir; #warn $html_dir; @@ -1397,3 +1397,61 @@ If-Match: 1 --- error_code: 200 --- response_body eval qr/\Ahello\z/ + + + +=== TEST 51: subrequest cycle problem in rewrite_by_lua_file +--- http_config + lua_code_cache off; +--- config + set $main "foo"; + set $sub "bar"; + location = /main { + rewrite_by_lua_file html/main.lua; + echo $main; + } + + location = /sub { + rewrite_by_lua_file html/sub.lua; + echo $sub; + } +--- user_files +>>> main.lua +local res = ngx.location.capture("/sub") +ngx.var.main = "main " .. res.body +>>> sub.lua +ngx.var.sub = "sub" + +--- pipelined_requests eval +["GET /sub", "GET /main"] +--- response_body eval +["sub\n", "main sub\n\n"] +--- no_error_log +[error] + + + +=== TEST 52: subrequest cycle problem in content_by_lua_file +--- http_config + lua_code_cache off; +--- config + location = /main { + content_by_lua_file html/main.lua; + } + + location = /sub { + content_by_lua_file html/sub.lua; + } +--- user_files +>>> main.lua +local res = ngx.location.capture("/sub") +ngx.print("main " .. res.body) +>>> sub.lua +ngx.print("sub") + +--- pipelined_requests eval +["GET /sub", "GET /main"] +--- response_body eval +["sub", "main sub"] +--- no_error_log +[error] From 90491ed33eacdbe0db92f19ccf92e1e4d67563d9 Mon Sep 17 00:00:00 2001 From: Zeping Bai Date: Thu, 23 Oct 2025 13:49:30 +0800 Subject: [PATCH 842/848] doc: tcpsock getfd typo. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 8c6edec844..eb378ac0cd 100644 --- a/README.markdown +++ b/README.markdown @@ -8156,7 +8156,7 @@ tcpsock:getfd **context:** *rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_client_hello_by_lua** -Get the file describer of the current tcp socket. +Get the file descriptor of the current tcp socket. This method was first introduced in the `v0.10.29` release. From 47e8d8b485e12cb7cb4a17da0dc828ee83a67b45 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 23 Oct 2025 14:25:16 +0800 Subject: [PATCH 843/848] doc: update latest compatible nginx version. --- README.markdown | 1 + 1 file changed, 1 insertion(+) diff --git a/README.markdown b/README.markdown index eb378ac0cd..72d59d6131 100644 --- a/README.markdown +++ b/README.markdown @@ -308,6 +308,7 @@ Nginx Compatibility The latest version of this module is compatible with the following versions of Nginx: +* 1.29.x (last tested: 1.29.2) * 1.27.x (last tested: 1.27.1) * 1.25.x (last tested: 1.25.1) * 1.21.x (last tested: 1.21.4) From a1ec138e98801400832fee6030120777bf9bb435 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Fri, 24 Oct 2025 22:51:31 +0800 Subject: [PATCH 844/848] bugfix: didn't free ngx_regex_match_context. --- src/ngx_http_lua_regex.c | 5 +++++ t/002-content.t | 1 + t/005-exit.t | 4 ++++ t/016-resp-header.t | 6 +++--- t/023-rewrite/req-socket.t | 1 + t/025-codecache.t | 2 ++ t/041-header-filter.t | 1 + t/056-flush.t | 1 + t/058-tcp-socket.t | 3 +++ t/067-req-socket.t | 2 ++ t/071-idle-socket.t | 1 + t/072-conditional-get.t | 1 + t/100-client-abort.t | 4 ++++ t/116-raw-req-socket.t | 1 + t/167-server-rewrite.t | 1 + 15 files changed, 31 insertions(+), 3 deletions(-) diff --git a/src/ngx_http_lua_regex.c b/src/ngx_http_lua_regex.c index 646b483e1f..35fe2a055c 100644 --- a/src/ngx_http_lua_regex.c +++ b/src/ngx_http_lua_regex.c @@ -501,6 +501,11 @@ ngx_http_lua_regex_cleanup(void *data) if (ngx_regex_compile_context) { old_pool = ngx_http_lua_pcre_malloc_init(NULL); + if (ngx_regex_match_context != NULL) { + pcre2_match_context_free(ngx_regex_match_context); + ngx_regex_match_context = NULL; + } + pcre2_compile_context_free(ngx_regex_compile_context); ngx_regex_compile_context = NULL; ngx_http_lua_pcre_malloc_done(old_pool); diff --git a/t/002-content.t b/t/002-content.t index eb9d587f89..26070f5287 100644 --- a/t/002-content.t +++ b/t/002-content.t @@ -1120,3 +1120,4 @@ If-Unmodified-Since: Wed, 01 Jan 2020 07:28:00 GMT --- error_code: 412 --- no_error_log unknown phase: 0 +--- skip_eval: 2:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/005-exit.t b/t/005-exit.t index bbaeda517e..7b82a81e4c 100644 --- a/t/005-exit.t +++ b/t/005-exit.t @@ -659,6 +659,7 @@ GET /t --- response_body_like: 403 Forbidden --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -677,6 +678,7 @@ GET /t --- response_body --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -713,6 +715,7 @@ F(ngx_http_lua_header_filter_by_chunk).return { --- no_error_log [error] [alert] +--- skip_eval: 4:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -730,6 +733,7 @@ GET /t --- response_body --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/016-resp-header.t b/t/016-resp-header.t index b30090812c..f5c58fa855 100644 --- a/t/016-resp-header.t +++ b/t/016-resp-header.t @@ -2183,9 +2183,9 @@ foo: foo%0Axx:bar\r\nfoo: bar%0Dxxx:foo\r\n hi --- no_error_log [alert] ---- error_log -my Content-Length: 8589934591 -upstream prematurely closed connection while sending to client +--- error_log eval +[ "my Content-Length: 8589934591", +qr/upstream prematurely closed connection while sending to client|upstream prematurely closed connection while reading upstream/] diff --git a/t/023-rewrite/req-socket.t b/t/023-rewrite/req-socket.t index f4dd6f4e18..280001578d 100644 --- a/t/023-rewrite/req-socket.t +++ b/t/023-rewrite/req-socket.t @@ -540,3 +540,4 @@ Expect: 100-Continue \breceived: hello\b.*?\breceived: worl\b --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/025-codecache.t b/t/025-codecache.t index cd56cf57aa..6e3e335097 100644 --- a/t/025-codecache.t +++ b/t/025-codecache.t @@ -855,6 +855,7 @@ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, "decrementing the reference count for Lua VM: 2", "decrementing the reference count for Lua VM: 1", ] +--- skip_eval: 11:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -1120,6 +1121,7 @@ qq{lua tcp socket keepalive create connection pool for key "127.0.0.1:$ENV{TEST_ qr/\[alert\] \S+ lua_code_cache is off; this will hurt performance/, qr/\blua tcp socket keepalive: free connection pool [0-9A-F]+ for "127.0.0.1:/, ] +--- skip_eval: 7:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/041-header-filter.t b/t/041-header-filter.t index 84bb39fba2..23730815da 100644 --- a/t/041-header-filter.t +++ b/t/041-header-filter.t @@ -847,6 +847,7 @@ GET /t --- error_code: 302 --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/056-flush.t b/t/056-flush.t index d2b107c754..bb81c1ecf3 100644 --- a/t/056-flush.t +++ b/t/056-flush.t @@ -43,6 +43,7 @@ hiya [error] --- error_log lua reuse free buf chain, but reallocate memory because 5 >= 0 +--- skip_eval: 4:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/058-tcp-socket.t b/t/058-tcp-socket.t index e17d038038..45d3073f5e 100644 --- a/t/058-tcp-socket.t +++ b/t/058-tcp-socket.t @@ -3993,6 +3993,7 @@ hello world [error] --- error_log lua tcp socket read any +--- skip_eval: 4:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} ne "") @@ -4147,6 +4148,7 @@ hello world [error] --- error_log lua tcp socket calling receiveany() method to read at most 128 bytes +--- skip_eval: 4:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} ne "") @@ -4222,6 +4224,7 @@ orld [error] --- error_log lua tcp socket calling receiveany() method to read at most 7 bytes +--- skip_eval: 4:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} ne "") diff --git a/t/067-req-socket.t b/t/067-req-socket.t index 54d59841f9..ae6bc3f3b6 100644 --- a/t/067-req-socket.t +++ b/t/067-req-socket.t @@ -529,6 +529,7 @@ Expect: 100-Continue \breceived: hello\b.*?\breceived: worl\b --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -1176,6 +1177,7 @@ GET /t received: received: abc --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} ne "") diff --git a/t/071-idle-socket.t b/t/071-idle-socket.t index 49d45a1b54..46ad84f317 100644 --- a/t/071-idle-socket.t +++ b/t/071-idle-socket.t @@ -431,3 +431,4 @@ failed to set keepalive: unread data in buffer } --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} ne "") diff --git a/t/072-conditional-get.t b/t/072-conditional-get.t index 7cf2dcd22d..d3400c613a 100644 --- a/t/072-conditional-get.t +++ b/t/072-conditional-get.t @@ -88,3 +88,4 @@ delete thread 1 say failed: nginx output filter error --- no_error_log [error] +--- skip_eval: 5:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/100-client-abort.t b/t/100-client-abort.t index 39d3244b16..f1f6894beb 100644 --- a/t/100-client-abort.t +++ b/t/100-client-abort.t @@ -955,6 +955,7 @@ GET /t [alert] --- error_log say failed: nginx output filter error +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -983,6 +984,7 @@ GET /t [alert] --- error_log print failed: nginx output filter error +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -1042,6 +1044,7 @@ GET /t [alert] --- error_log flush succeeded +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) @@ -1072,3 +1075,4 @@ GET /t eof succeeded --- error_log eof failed: nginx output filter error +--- skip_eval: 4:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} =~ /w/) diff --git a/t/116-raw-req-socket.t b/t/116-raw-req-socket.t index 94ef077c41..78a0e8d5ce 100644 --- a/t/116-raw-req-socket.t +++ b/t/116-raw-req-socket.t @@ -977,6 +977,7 @@ GET /t msg: 1: received: hello --- no_error_log [error] +--- skip_eval: 3:defined($ENV{MOCKEAGAIN}) && ($ENV{MOCKEAGAIN} ne "") diff --git a/t/167-server-rewrite.t b/t/167-server-rewrite.t index 6aea288ef8..33a30f3842 100644 --- a/t/167-server-rewrite.t +++ b/t/167-server-rewrite.t @@ -468,6 +468,7 @@ GET /lua failed to load inlined Lua code: server_rewrite_by_lua(nginx.conf:25):2: unexpected symbol near ''for end'' --- no_error_log no_such_error +--- skip_eval: 2:$ENV{TEST_NGINX_USE_HUP} From 54e813547c54fe38697672f40a8793dc57c27cbf Mon Sep 17 00:00:00 2001 From: willmafh Date: Sun, 26 Oct 2025 12:21:13 +0800 Subject: [PATCH 845/848] tests: improve openssl version check in test cases. --- t/139-ssl-cert-by.t | 2 +- t/140-ssl-c-api.t | 2 +- t/155-tls13.t | 2 +- t/166-ssl-client-hello.t | 2 +- t/187-ssl-two-verification.t | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/t/139-ssl-cert-by.t b/t/139-ssl-cert-by.t index 82ae791787..d905cb3f16 100644 --- a/t/139-ssl-cert-by.t +++ b/t/139-ssl-cert-by.t @@ -12,7 +12,7 @@ if ($openssl_version =~ m/BoringSSL/) { } if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { - plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); + plan(skip_all => "too old OpenSSL, need >= 1.0.2e, was $1"); } else { plan tests => repeat_each() * (blocks() * 6 + 4); } diff --git a/t/140-ssl-c-api.t b/t/140-ssl-c-api.t index ca04a87528..9f647c41bc 100644 --- a/t/140-ssl-c-api.t +++ b/t/140-ssl-c-api.t @@ -9,7 +9,7 @@ my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0|1\.0\.(?:0|1[^\d]|2[a-d]).*)/) { - plan(skip_all => "too old OpenSSL, need 1.0.2e, was $1"); + plan(skip_all => "too old OpenSSL, need >= 1.0.2e, was $1"); } elsif ($openssl_version =~ m/BoringSSL/) { $ENV{TEST_NGINX_USE_BORINGSSL} = 1; plan tests => repeat_each() * (blocks() * 6 - 6); diff --git a/t/155-tls13.t b/t/155-tls13.t index 4e684cd335..54379e4412 100644 --- a/t/155-tls13.t +++ b/t/155-tls13.t @@ -9,7 +9,7 @@ my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { - plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); + plan(skip_all => "too old OpenSSL, need >= 1.1.1, was $1"); } else { plan tests => repeat_each() * (blocks() * 5); } diff --git a/t/166-ssl-client-hello.t b/t/166-ssl-client-hello.t index 53109ac3df..4372d0d1f8 100644 --- a/t/166-ssl-client-hello.t +++ b/t/166-ssl-client-hello.t @@ -9,7 +9,7 @@ my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { - plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); + plan(skip_all => "too old OpenSSL, need >= 1.1.1, was $1"); } elsif ($openssl_version =~ m/running with BoringSSL/) { plan(skip_all => "does not support BoringSSL"); } elsif ($ENV{TEST_NGINX_USE_HTTP3}) { diff --git a/t/187-ssl-two-verification.t b/t/187-ssl-two-verification.t index 312847252c..bea4aaee0f 100644 --- a/t/187-ssl-two-verification.t +++ b/t/187-ssl-two-verification.t @@ -9,7 +9,7 @@ my $NginxBinary = $ENV{'TEST_NGINX_BINARY'} || 'nginx'; my $openssl_version = eval { `$NginxBinary -V 2>&1` }; if ($openssl_version =~ m/built with OpenSSL (0\S*|1\.0\S*|1\.1\.0\S*)/) { - plan(skip_all => "too old OpenSSL, need 1.1.1, was $1"); + plan(skip_all => "too old OpenSSL, need >= 1.1.1, was $1"); } elsif ($openssl_version =~ m/running with BoringSSL/) { plan(skip_all => "does not support BoringSSL"); } else { From de02114acc8bbd4b17da933d513f736b7f45728b Mon Sep 17 00:00:00 2001 From: "Sergey A. Osokin" Date: Sun, 2 Nov 2025 21:46:46 -0500 Subject: [PATCH 846/848] optimize: add compatibility for freenginx. Commit freenginx/nginx@3329aa9 updated the ngx_http_request_t structure with the start_time; the start_sec and start_msec were removed. --- src/ngx_http_lua_subrequest.c | 6 ++++++ src/ngx_http_lua_time.c | 14 ++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/src/ngx_http_lua_subrequest.c b/src/ngx_http_lua_subrequest.c index edb24e9c0d..c9a5c08622 100644 --- a/src/ngx_http_lua_subrequest.c +++ b/src/ngx_http_lua_subrequest.c @@ -1373,7 +1373,9 @@ ngx_http_lua_subrequest(ngx_http_request_t *r, ngx_str_t *uri, ngx_str_t *args, ngx_http_request_t **psr, ngx_http_post_subrequest_t *ps, ngx_uint_t flags) { +#if !defined freenginx ngx_time_t *tp; +#endif ngx_connection_t *c; ngx_http_request_t *sr; ngx_http_core_srv_conf_t *cscf; @@ -1501,9 +1503,13 @@ ngx_http_lua_subrequest(ngx_http_request_t *r, sr->subrequests = r->subrequests - 1; #endif +#if (defined freenginx && nginx_version >= 1029000) + sr->start_time = ngx_current_msec; +#else tp = ngx_timeofday(); sr->start_sec = tp->sec; sr->start_msec = tp->msec; +#endif r->main->count++; diff --git a/src/ngx_http_lua_time.c b/src/ngx_http_lua_time.c index 72704b23a2..fa63115c4c 100644 --- a/src/ngx_http_lua_time.c +++ b/src/ngx_http_lua_time.c @@ -28,7 +28,21 @@ ngx_http_lua_ffi_now(void) double ngx_http_lua_ffi_req_start_time(ngx_http_request_t *r) { +#if (defined freenginx && nginx_version >= 1029000) + ngx_time_t *tp; + + tp = ngx_timeofday(); + tp->sec -= (ngx_current_msec - r->start_time) / 1000; + tp->msec -= (ngx_current_msec - r->start_time) % 1000; + if (tp->msec > NGX_MAX_INT_T_VALUE) { + tp->msec += 1000; + tp->sec -= 1; + } + + return tp->sec + tp->msec / 1000.0; +#else return r->start_sec + r->start_msec / 1000.0; +#endif } From 562e10690a9ce2bb8ac04cedbf18a28de64b9970 Mon Sep 17 00:00:00 2001 From: leslie Date: Sat, 8 Nov 2025 15:08:22 +0800 Subject: [PATCH 847/848] doc: fixed typo. --- README.markdown | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.markdown b/README.markdown index 72d59d6131..24f012f1c8 100644 --- a/README.markdown +++ b/README.markdown @@ -1385,7 +1385,7 @@ A zero value of `` disables the cache. Note that this feature requires OpenResty's LuaJIT with the new C API `lua_resetthread`. -This feature was first introduced in verson `v0.10.9`. +This feature was first introduced in version `v0.10.9`. [Back to TOC](#directives) @@ -6668,7 +6668,7 @@ ngx.http_time **context:** *init_worker_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*, exit_worker_by_lua*, ssl_client_hello_by_lua** -Returns a formated string can be used as the http header time (for example, being used in `Last-Modified` header). The parameter `sec` is the time stamp in seconds (like those returned from [ngx.time](#ngxtime)). +Returns a formatted string can be used as the http header time (for example, being used in `Last-Modified` header). The parameter `sec` is the time stamp in seconds (like those returned from [ngx.time](#ngxtime)). ```nginx @@ -8763,9 +8763,9 @@ By default, the corresponding Nginx handler (e.g., [rewrite_by_lua](#rewrite_by_ When the user "light thread" terminates with a Lua error, however, it will not abort other running "light threads" like the "entry thread" does. -Due to the limitation in the Nginx subrequest model, it is not allowed to abort a running Nginx subrequest in general. So it is also prohibited to abort a running "light thread" that is pending on one ore more Nginx subrequests. You must call [ngx.thread.wait](#ngxthreadwait) to wait for those "light thread" to terminate before quitting the "world". A notable exception here is that you can abort pending subrequests by calling [ngx.exit](#ngxexit) with and only with the status code `ngx.ERROR` (-1), `408`, `444`, or `499`. +Due to the limitation in the Nginx subrequest model, it is not allowed to abort a running Nginx subrequest in general. So it is also prohibited to abort a running "light thread" that is pending on one or more Nginx subrequests. You must call [ngx.thread.wait](#ngxthreadwait) to wait for those "light thread" to terminate before quitting the "world". A notable exception here is that you can abort pending subrequests by calling [ngx.exit](#ngxexit) with and only with the status code `ngx.ERROR` (-1), `408`, `444`, or `499`. -The "light threads" are not scheduled in a pre-emptive way. In other words, no time-slicing is performed automatically. A "light thread" will keep running exclusively on the CPU until +The "light threads" are not scheduled in a preemptive way. In other words, no time-slicing is performed automatically. A "light thread" will keep running exclusively on the CPU until 1. a (nonblocking) I/O operation cannot be completed in a single run, 1. it calls [coroutine.yield](#coroutineyield) to actively give up execution, or From 971cc14eded9067fcbf2e02071c432d84ace7445 Mon Sep 17 00:00:00 2001 From: lijunlong Date: Thu, 13 Nov 2025 17:39:07 +0800 Subject: [PATCH 848/848] doc: update copyright. --- README.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.markdown b/README.markdown index 24f012f1c8..961ecb593a 100644 --- a/README.markdown +++ b/README.markdown @@ -1066,7 +1066,7 @@ This module is licensed under the BSD license. Copyright (C) 2009-2017, by Xiaozhe Wang (chaoslawful) . -Copyright (C) 2009-2019, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. +Copyright (C) 2009-2025, by Yichun "agentzh" Zhang (章亦春) , OpenResty Inc. All rights reserved.