Skip to content

Commit eb88b62

Browse files
spliticesplitice
committed
Merge pull request #1 from doujiang24/shdict_cas
Shdict cas
2 parents 96eaf4a + 632f95f commit eb88b62

File tree

4 files changed

+657
-4
lines changed

4 files changed

+657
-4
lines changed

‎README.markdown‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2906,6 +2906,7 @@ Nginx API for Lua
29062906
* [ngx.shared.DICT.add](#ngxshareddictadd)
29072907
* [ngx.shared.DICT.safe_add](#ngxshareddictsafe_add)
29082908
* [ngx.shared.DICT.replace](#ngxshareddictreplace)
2909+
* [ngx.shared.DICT.cas](#ngxshareddictcas)
29092910
* [ngx.shared.DICT.delete](#ngxshareddictdelete)
29102911
* [ngx.shared.DICT.incr](#ngxshareddictincr)
29112912
* [ngx.shared.DICT.flush_all](#ngxshareddictflush_all)
@@ -5896,6 +5897,7 @@ The resulting object `dict` has the following methods:
58965897
* [add](#ngxshareddictadd)
58975898
* [safe_add](#ngxshareddictsafe_add)
58985899
* [replace](#ngxshareddictreplace)
5900+
* [cas](#ngxshareddictcas)
58995901
* [delete](#ngxshareddictdelete)
59005902
* [incr](#ngxshareddictincr)
59015903
* [flush_all](#ngxshareddictflush_all)
@@ -6114,6 +6116,62 @@ See also [ngx.shared.DICT](#ngxshareddict).
61146116

61156117
[Back to TOC](#nginx-api-for-lua)
61166118

6119+
ngx.shared.DICT.cas
6120+
-------------------
6121+
**syntax:** *success, err, forcible, current_value?, current_flags? = ngx.shared.DICT:cas(key, value, exptime, flags, old_value, old_flags?)*
6122+
6123+
**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, certificate_by_lua**
6124+
6125+
Just like the [replace](#ngxshareddictreplace) method, but only stores the key-value pair into the dictionary [ngx.shared.DICT](#ngxshareddict) if and only if the `old_value` argument and `old_flags` argument *do* match the value and flags in the dictionary [ngx.shared.DICT](#ngxshareddict).
6126+
6127+
The `old_value` argument can be `nil` only when `old_flags` argument is specified, in which case only `flags` will be checked.
6128+
6129+
If `old_flags` argument is not specified, only `value` will be checked.
6130+
6131+
The optional `old_flags` argument can be `nil`, and it means `0`.
6132+
6133+
If they do *not* match, the `success` return value will be `false` and the `err` return value will be `"not matched"`. The `current_value` return value and `current_flags` return value will be the current `value` and current `flags` in the dictionary [ngx.shared.DICT](#ngxshareddict), just like [get](#ngxshareddictget) does.
6134+
6135+
This function is often used to avoid race condition between [get](#ngxshareddictget) and [set](#ngxshareddictset) across multipe nginx worker processes, and below is an example:
6136+
6137+
```lua
6138+
6139+
local cats = ngx.shared.cats
6140+
cats:set("foo", 1, 1)
6141+
6142+
local old_value, old_flags = cats:get("foo")
6143+
6144+
while true do
6145+
local newvalue = calculate(old_value) -- some logic
6146+
local newflags = (old_flags or 0) + 1
6147+
6148+
local success, err, forcibly, current_value, current_flags
6149+
= cats:cas("foo", newvalue, 0, newflags, old_value, old_flags)
6150+
6151+
if success then
6152+
break
6153+
6154+
elseif err == "not matched" then
6155+
old_value = current_value
6156+
old_flags = current_flags
6157+
6158+
elseif err == "not found" then
6159+
-- add or some other handle
6160+
cats:add("foo", newvalue, 0, newflags)
6161+
break
6162+
6163+
else
6164+
-- "no memory" or some other error
6165+
-- just log or some other handle
6166+
break
6167+
end
6168+
end
6169+
```
6170+
6171+
See also [ngx.shared.DICT](#ngxshareddict).
6172+
6173+
[Back to TOC](#nginx-api-for-lua)
6174+
61176175
ngx.shared.DICT.delete
61186176
----------------------
61196177
**syntax:** *ngx.shared.DICT:delete(key)*

‎doc/HttpLuaModule.wiki‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4934,6 +4934,7 @@ The resulting object <code>dict</code> has the following methods:
49344934
* [[#ngx.shared.DICT.add|add]]
49354935
* [[#ngx.shared.DICT.safe_add|safe_add]]
49364936
* [[#ngx.shared.DICT.replace|replace]]
4937+
* [[#ngx.shared.DICT.cas|cas]]
49374938
* [[#ngx.shared.DICT.delete|delete]]
49384939
* [[#ngx.shared.DICT.incr|incr]]
49394940
* [[#ngx.shared.DICT.flush_all|flush_all]]
@@ -5123,6 +5124,58 @@ This feature was first introduced in the <code>v0.3.1rc22</code> release.
51235124
51245125
See also [[#ngx.shared.DICT|ngx.shared.DICT]].
51255126
5127+
== ngx.shared.DICT.cas ==
5128+
'''syntax:''' ''success, err, forcible, current_value?, current_flags? = ngx.shared.DICT:cas(key, value, exptime, flags, old_value, old_flags?)''
5129+
5130+
'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, certificate_by_lua*''
5131+
5132+
Just like the [[#ngx.shared.DICT.replace|replace]] method, but only stores the key-value pair into the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] if and only if the <code>old_value</code> argument and <code>old_flags</code> argument ''do'' match the value and flags in the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]].
5133+
5134+
The <code>old_value</code> argument can be <code>nil</code> only when <code>old_flags</code> argument is specified, in which case only <code>flags</code> will be checked.
5135+
5136+
If <code>old_flags</code> argument is not specified, only <code>value</code> will be checked.
5137+
5138+
The optional <code>old_flags</code> argument can be <code>nil</code>, and it means <code>0</code>.
5139+
5140+
If they do ''not'' match, the <code>success</code> return value will be <code>false</code> and the <code>err</code> return value will be <code>"not matched"</code>. The <code>current_value</code> return value and <code>current_flags</code> return value will be the current <code>value</code> and current <code>flags</code> in the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]], just like [[#ngx.shared.DICT.get|get]] does.
5141+
5142+
This function is often used to avoid race condition between [[#ngx.shared.DICT.get|get]] and [[#ngx.shared.DICT.set|set]] across multipe nginx worker processes, and below is an example:
5143+
5144+
<geshi lang="lua">
5145+
local cats = ngx.shared.cats
5146+
cats:set("foo", 1, 1)
5147+
5148+
local old_value, old_flags = cats:get("foo")
5149+
5150+
while true do
5151+
local newvalue = calculate(old_value) -- some logic
5152+
local newflags = (old_flags or 0) + 1
5153+
5154+
local success, err, forcibly, current_value, current_flags
5155+
= cats:cas("foo", newvalue, 0, newflags, old_value, old_flags)
5156+
5157+
if success then
5158+
break
5159+
5160+
elseif err == "not matched" then
5161+
old_value = current_value
5162+
old_flags = current_flags
5163+
5164+
elseif err == "not found" then
5165+
-- add or some other handle
5166+
cats:add("foo", newvalue, 0, newflags)
5167+
break
5168+
5169+
else
5170+
-- "no memory" or some other error
5171+
-- just log or some other handle
5172+
break
5173+
end
5174+
end
5175+
</geshi>
5176+
5177+
See also [[#ngx.shared.DICT|ngx.shared.DICT]].
5178+
51265179
== ngx.shared.DICT.delete ==
51275180
'''syntax:''' ''ngx.shared.DICT:delete(key)''
51285181

‎src/ngx_http_lua_shdict.c‎

Lines changed: 167 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ static int ngx_http_lua_shdict_set_helper(lua_State *L, int flags);
2929
static int ngx_http_lua_shdict_add(lua_State *L);
3030
static int ngx_http_lua_shdict_safe_add(lua_State *L);
3131
static int ngx_http_lua_shdict_replace(lua_State *L);
32+
static int ngx_http_lua_shdict_cas(lua_State *L);
3233
static int ngx_http_lua_shdict_incr(lua_State *L);
3334
static int ngx_http_lua_shdict_delete(lua_State *L);
3435
static int ngx_http_lua_shdict_flush_all(lua_State *L);
@@ -43,6 +44,7 @@ static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L,
4344
#define NGX_HTTP_LUA_SHDICT_ADD 0x0001
4445
#define NGX_HTTP_LUA_SHDICT_REPLACE 0x0002
4546
#define NGX_HTTP_LUA_SHDICT_SAFE_STORE 0x0004
47+
#define NGX_HTTP_LUA_SHDICT_CHECK 0x0008
4648

4749

4850
enum {
@@ -339,6 +341,9 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L)
339341
lua_pushcfunction(L, ngx_http_lua_shdict_replace);
340342
lua_setfield(L, -2, "replace");
341343

344+
lua_pushcfunction(L, ngx_http_lua_shdict_cas);
345+
lua_setfield(L, -2, "cas");
346+
342347
lua_pushcfunction(L, ngx_http_lua_shdict_incr);
343348
lua_setfield(L, -2, "incr");
344349

@@ -834,6 +839,14 @@ ngx_http_lua_shdict_replace(lua_State *L)
834839
}
835840

836841

842+
static int
843+
ngx_http_lua_shdict_cas(lua_State *L)
844+
{
845+
return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_REPLACE
846+
|NGX_HTTP_LUA_SHDICT_CHECK);
847+
}
848+
849+
837850
static int
838851
ngx_http_lua_shdict_set(lua_State *L)
839852
{
@@ -852,6 +865,7 @@ static int
852865
ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
853866
{
854867
int i, n;
868+
ngx_str_t name;
855869
ngx_str_t key;
856870
uint32_t hash;
857871
ngx_int_t rc;
@@ -869,11 +883,28 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
869883
int forcible = 0;
870884
/* indicates whether to foricibly override other
871885
* valid entries */
872-
int32_t user_flags = 0;
886+
uint32_t user_flags = 0;
887+
ngx_str_t old_value;
888+
double old_num;
889+
u_char old_c;
890+
int old_value_type;
891+
uint32_t old_user_flags = 0;
873892

874893
n = lua_gettop(L);
875894

876-
if (n != 3 && n != 4 && n != 5) {
895+
if (flags & NGX_HTTP_LUA_SHDICT_CHECK) {
896+
if (n != 6 && n != 7) {
897+
return luaL_error(L, "expecting 6 or 7 arguments, "
898+
"but only seen %d", n);
899+
}
900+
901+
if (n == 6 && lua_type(L, 6) == LUA_TNIL) {
902+
lua_pushnil(L);
903+
lua_pushliteral(L, "old_value is nil and no old_flags");
904+
return 2;
905+
}
906+
907+
} else if (n != 3 && n != 4 && n != 5) {
877908
return luaL_error(L, "expecting 3, 4 or 5 arguments, "
878909
"but only seen %d", n);
879910
}
@@ -931,7 +962,9 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
931962
break;
932963

933964
case LUA_TNIL:
934-
if (flags & (NGX_HTTP_LUA_SHDICT_ADD|NGX_HTTP_LUA_SHDICT_REPLACE)) {
965+
if (flags & (NGX_HTTP_LUA_SHDICT_ADD|NGX_HTTP_LUA_SHDICT_REPLACE)
966+
&& ! (flags & NGX_HTTP_LUA_SHDICT_CHECK))
967+
{
935968
lua_pushnil(L);
936969
lua_pushliteral(L, "attempt to add or replace nil values");
937970
return 2;
@@ -953,7 +986,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
953986
}
954987
}
955988

956-
if (n == 5) {
989+
if (n >= 5) {
957990
user_flags = (uint32_t) luaL_checkinteger(L, 5);
958991
}
959992

@@ -980,6 +1013,62 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
9801013

9811014
/* rc == NGX_OK */
9821015

1016+
if (flags & NGX_HTTP_LUA_SHDICT_CHECK) {
1017+
/* check current value flags */
1018+
old_value_type = lua_type(L, 6);
1019+
1020+
if (old_value_type != LUA_TNIL) {
1021+
1022+
if (sd->value_type != old_value_type) {
1023+
dd("value type check failed");
1024+
goto check_failed;
1025+
}
1026+
1027+
switch (old_value_type) {
1028+
case LUA_TSTRING:
1029+
old_value.data = (u_char *) lua_tolstring(L, 6,
1030+
&old_value.len);
1031+
break;
1032+
1033+
case LUA_TNUMBER:
1034+
old_value.len = sizeof(double);
1035+
old_num = lua_tonumber(L, 6);
1036+
old_value.data = (u_char *) &old_num;
1037+
break;
1038+
1039+
case LUA_TBOOLEAN:
1040+
old_value.len = sizeof(u_char);
1041+
old_c = lua_toboolean(L, 6) ? 1 : 0;
1042+
old_value.data = &old_c;
1043+
break;
1044+
}
1045+
1046+
if (old_value.len != sd->value_len) {
1047+
dd("value len check failed");
1048+
goto check_failed;
1049+
}
1050+
1051+
if (ngx_memn2cmp(old_value.data, sd->data + sd->key_len,
1052+
old_value.len, sd->value_len) != 0)
1053+
{
1054+
dd("value data check failed");
1055+
goto check_failed;
1056+
}
1057+
}
1058+
1059+
if (n == 7) {
1060+
1061+
if (lua_type(L, 7) != LUA_TNIL) {
1062+
old_user_flags = (uint32_t) luaL_checkinteger(L, 7);
1063+
}
1064+
1065+
if (sd->user_flags != old_user_flags) {
1066+
dd("user_flags check failed");
1067+
goto check_failed;
1068+
}
1069+
}
1070+
}
1071+
9831072
goto replace;
9841073
}
9851074

@@ -1166,6 +1255,80 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
11661255
lua_pushnil(L);
11671256
lua_pushboolean(L, forcible);
11681257
return 3;
1258+
1259+
check_failed:
1260+
1261+
name = ctx->name;
1262+
1263+
lua_pushboolean(L, 0);
1264+
lua_pushliteral(L, "not matched");
1265+
lua_pushboolean(L, forcible);
1266+
1267+
value_type = sd->value_type;
1268+
1269+
dd("data: %p", sd->data);
1270+
dd("key len: %d", (int) sd->key_len);
1271+
1272+
value.data = sd->data + sd->key_len;
1273+
value.len = (size_t) sd->value_len;
1274+
1275+
switch (value_type) {
1276+
case LUA_TSTRING:
1277+
1278+
lua_pushlstring(L, (char *) value.data, value.len);
1279+
break;
1280+
1281+
case LUA_TNUMBER:
1282+
1283+
if (value.len != sizeof(double)) {
1284+
1285+
ngx_shmtx_unlock(&ctx->shpool->mutex);
1286+
1287+
return luaL_error(L, "bad lua number value size found for key %s "
1288+
"in shared_dict %s: %lu", key.data, name.data,
1289+
(unsigned long) value.len);
1290+
}
1291+
1292+
ngx_memcpy(&num, value.data, sizeof(double));
1293+
1294+
lua_pushnumber(L, num);
1295+
break;
1296+
1297+
case LUA_TBOOLEAN:
1298+
1299+
if (value.len != sizeof(u_char)) {
1300+
1301+
ngx_shmtx_unlock(&ctx->shpool->mutex);
1302+
1303+
return luaL_error(L, "bad lua boolean value size found for key %s "
1304+
"in shared_dict %s: %lu", key.data, name.data,
1305+
(unsigned long) value.len);
1306+
}
1307+
1308+
c = *value.data;
1309+
1310+
lua_pushboolean(L, c ? 1 : 0);
1311+
break;
1312+
1313+
default:
1314+
1315+
ngx_shmtx_unlock(&ctx->shpool->mutex);
1316+
1317+
return luaL_error(L, "bad value type found for key %s in "
1318+
"shared_dict %s: %d", key.data, name.data,
1319+
value_type);
1320+
}
1321+
1322+
user_flags = sd->user_flags;
1323+
1324+
ngx_shmtx_unlock(&ctx->shpool->mutex);
1325+
1326+
if (user_flags) {
1327+
lua_pushinteger(L, (lua_Integer) user_flags);
1328+
return 5;
1329+
}
1330+
1331+
return 4;
11691332
}
11701333

11711334

0 commit comments

Comments
 (0)