When will lua scripts be used?

I can use Lua when I want to execute a set of Redis instructions to reduce the cost of communication.

But the pipeline command can also be used, so why not pipeline?

First of all, the limitations of pipelines, if I want to do arithmetic (thread-safe) operations in pipelines, I can’t do it.

For example: selling goods, when the inventory reaches zero, there is no need to reduce further. Otherwise it would have gone negative.

Then there is pipeline, if A command depends on the result of B, this will not be obtained.

The above scenario pipeline is not done, this time you need to use Redis Lua script.

Introduction of EVAL

EVAL and EVALSHA, starting with Redis 2.6, can evaluate Lua using the built-in Lua parser.

EVAL script numkeys key [key ...]  arg [arg ...]Copy the code
  • Script: Lua script
  • Numkeys: indicates the number of keys
  • Key: a list of KEYS that can be accessed in Lua using the global KEYS array as a base address of 1 (KEYS[1], KEYS[2]…)
  • Arg: List of arguments that can be accessed in Lua through the ARGV array of global variables at base 1 (ARGV [1], ARGV [2]…)

Example:

eval "return redis.call('set',KEYS[1],'bar')" 1 foo
Copy the code
How to call the Redis command from Lua feet?
Redis.call () : returns an error to the caller redis.pcall() : returns the caught error as a Lua tableCopy the code

Create and modify the Lua environment

Redis In order to execute Lua scripts in Redis server, Redis embedded a Lua environment in the server, and the Lua environment, and a series of changes to ensure that the Lua environment meets the needs of the Redis server.

Steps:

  1. Create a basic Lua environment for which all subsequent changes will be made.
  2. Load multiple libraries into the Lua environment
  3. Create global table REids that contain functions that operate on Redis
  4. Create random function random-with-default-seed.lua
  5. Create a sort helper function
  6. Create an error report for the redis.pcall function

1. Create the Lua environment

The server first calls Lua’s C API function, lua_open, to create a new Lua environment.

2. Load the library

Base: Assert, Error, Pairs, ToString, Pcall and other table libraries: Table. Concat, table. Insert, table. Remove, table. Sort, etc. String libraries (string) : Format, String. len, and String. reveres are generic functions that handle strings. Math. abs, math. Max, math.min, math. SQRT, math.log (debug) : Provider sets hook debug.sethook and gets hook debug.gethook. JSON format for processing UTF-8 encoding. The cjson.decode function converts a JSON-formatted string into a Lua value, while the cjson.encode function serializes a Lua value into a JSON-formatted string. Struct library for converting Lua values to C results Lua CMSGPack library for processing MessagePack format data

3. Create the Redis global table

The server will create a REIDS table in the Lua environment and set it as a global variable. The reIDS table includes the following functions: redis.call and redis.pcall functions for executing the Redis command. The Redis. Log function used to record the Redis log. And the corresponding log level handles the redis.sha1hex function used to calculate the SHA1 checksum, the redis.error_reply function used to return error messages, and the redis.status_reply function.

4. The pseudo client

In order to execute the commands included in the Lua script, the Redis command must have the corresponding client state. The Redis server creates a pseudo-client specifically for the Lua environment, and a pseudo-client handles all Redis commands in Lua scripts.

The EVAL "return redis. Call (' DSSIZE)" 0Copy the code

Lua_scripts dictionary

In addition to pseudo-clients, the other primary key created by the Redis server for Lua environments is lua_scripts. The key of this dictionary is the SHA1 checksum of a Lua script, and the value of the dictionary is the SHA1 checksum of a Lua script:

struct redisServer{ // … dict *lua_scripts; // … }

The Redis server saves all Lua scripts executed by EVAL and all Lua scripts loaded by SCRIPT LOAD into the lua_scripts dictionary.

Example:

R127.0.0.1:6371 > SCRIPT LOAD "return" hi ", "" 2 f31ba2bb6d6a0f42cc159d2e2dad55440778de3" 127.0.0.1:6371 > SCRIPT LOAD Return "1 + 1" "a27e7e8a43702b7046d4f6a7ccf5b60cef6b9bd9" 127.0.0.1:6371 > SCRIPT LOAD "return 2 * 2" "4475bfb5919b5ad16424cb50f74d4724ae833e72"Copy the code

The lua_scripts dictionary has two functions, one to implement the SCRIPT EXISTS command and the other to implement the SCRIPT copy function.

Implementation of the EVAL command

The EVAL command is executed in the following three steps:

  1. Define a Lua function in the Lua environment based on the Lua script given by the client
  2. Save the script given by the client to the lua_scripts dictionary for further use
  3. Execute the Lua script executed by the client by executing the functions just defined in the Lua environment.
127.0.0.1:6371> EVAL "return 'hello world'" 0
"hello world"
Copy the code

Defining script functions

When a client sends an EVAL command to the server to execute a Lua script, the first thing the server needs to do is to define a Lua function corresponding to the script in the Lua environment. The name of a Lua function is made up of the f_ prefix plus the script’s SHA1 checksum (four cross characters long), and the body of the function is the script itself.

We type: EVAL “return ‘hello world'” 0

For the server: the function f_5332031c6b470dc5a0dd9b4bf2030dea6d65de91 () return “hello world” end

Because the client incoming scripts as return “hello world”, and the script SHA1 checksum is 5332031 c6b470dc5a0dd9b4bf2030dea6d65de91, So function name for f_5332031c6b470dc5a0dd9b4bf2030dea6d65de91, while the body of the function to return “hello world”.

Using functions to save client-passed scripts has the following benefits:

The steps to execute the script are as simple as calling the corresponding function of the script. Function locality keeps the Lua environment clean, reduces garbage collection, and avoids the use of global variables. If a script has been defined at least once in the Lua environment, the server can call it without knowing the script itself as long as it remembers to call its SHA1 checksum.

The second thing the EVAL command does is to save the client’s incoming steps into the server’s Lua_script dictionary.

 return 'hello world'
 =====>>>
 5332031c6b470dc5a0dd9b4bf2030dea6d65de91
Copy the code

Executing script functions

After the script is saved to the LuA_script dictionary, the server must set up hooks and pass parameters before the script can be executed.

The process is as follows:

  1. Store the KEY name and argument passed in by the EVAL command into the KEY array and ARGV array, respectively, and treat the two arrays as
  2. Global variables are passed into the Lua environment.
  3. Load the Lua environment with timeout handling hooks that allow clients to pass the script if it runs out of time

The SCRIPT KILL command is used to stop the SCRIPT or the SHUTDOWN command is used to SHUTDOWN the server. 5. Execute the script function 6. Remove the timeout hook loaded previously 7. 8. Perform garbage collection on the Lua environment.

An implementation of the EVALSHA command

For every Lua script that has been successfully executed by the EVAL command, there is a corresponding Lua function in the Lua environment.

Notice the return error “SCRIPRT NOT FOUND”

SCRIPT LOAD

SCRIPT LOAD creates functions in the Lua environment and then saves the SCRIPT to the lua_scripts dictionary.

SCRIPT KILL

If the server has lua-time-limit configured, the server will set a timeout hook in the Lua environment each time the lua script is executed.

Copy scripts for master/slave servers

The master server copies the EVAL, SCRIPT FLUSH, and SCRIPT LOAD commands just as it copies the normal Redis commands, simply propagating the same commands to the slave server.

What are the optimizations for the client that Redisson implements with Lua scripts?

Parameter: useScriptCache Default Value: false Specifies whether to use Lua script cache on the Redis side. Most Redisson methods are based on Lua scripts, and turning this on can speed up the execution of such methods and save network traffic.Copy the code

Ideas:

  1. First, convert the command to SHA1 code
  2. Then run the “EVALSHA” command again, listening for the command to return the command
  3. Return “NOSCRIPT” to prove that the Redis server does not have the script, execute “SCRIPT_LOAD” command, load the script, continue to listen for “SCRIPT_LOAD” command return
  4. If “SCRIPT_LOAD” returns successfully, issue the “EVALSHA” command again

The code is as follows:

private <T, R> RFuture<R> evalAsync(NodeSource nodeSource, boolean readOnlyMode, Codec codec, RedisCommand<T> evalCommandType, String script, List<Object> keys, Object... Params) {// If useScriptCache == true and the script command EVAL if (isEvalCacheActive() && evalCommandType.getName().equals("EVAL")) { RPromise<R> mainPromise = new RedissonPromise<R>(); Object[] pps = copy(params); RPromise<R> promise = new RedissonPromise<R>(); String sha1 = calcSHA(script); RedisCommand cmd = new RedisCommand(evalCommandType, "EVALSHA"); List<Object> args = new ArrayList<Object>(2 + keys.size() + params.length); args.add(sha1); args.add(keys.size()); args.addAll(keys); args.addAll(Arrays.asList(params)); RedisExecutor<T, R> executor = new RedisExecutor<>(readOnlyMode, nodeSource, codec, CMD, args.toArray(), promise, false, connectionManager, objectBuilder, referenceType); executor.execute(); // Listener returns promise.oncomplete ((res, e) -> {if (e! = null) {if (LLDB etMessage().startswith ("NOSCRIPT")) {// perform 'SCRIPT_LOAD' RFuture<String> loadFuture = loadScript(executor.getredisClient (), script); RFuture<String> loadFuture = loadScript(executor.getredisClient (), script); Loadfuture.oncomplete ((r, ex) -> {if (ex! = null) { free(pps); mainPromise.tryFailure(ex); return; RedisCommand command = new RedisCommand(evalCommandType, "EVALSHA");} // SCRIPT_LOAD is executed successfully. List<Object> newargs = new ArrayList<Object>(2 + keys.size() + params.length); newargs.add(sha1); newargs.add(keys.size()); newargs.addAll(keys); newargs.addAll(Arrays.asList(pps)); NodeSource ns = nodeSource; if (ns.getRedisClient() == null) { ns = new NodeSource(nodeSource, executor.getRedisClient()); } async(readOnlyMode, ns, codec, command, newargs.toArray(), mainPromise, false); }); } else { free(pps); mainPromise.tryFailure(e); } return; } free(pps); mainPromise.trySuccess(res); }); return mainPromise; RPromise<R> mainPromise = createPromise(); List<Object> args = new ArrayList<Object>(2 + keys.size() + params.length); args.add(script); args.add(keys.size()); args.addAll(keys); args.addAll(Arrays.asList(params)); async(readOnlyMode, nodeSource, codec, evalCommandType, args.toArray(), mainPromise, false); return mainPromise; }Copy the code
  • Redis. Cn/commands/ev… The official documentation
  • www.cnblogs.com/chopper-poe… reference