Bats-bash Automated testing tool

Bats, a TAP-compliant version of Bash testing framework, uses an extremely easy way to verify that command-line programs are working properly.

Bats requires a minimum version of Bash to be 3.2.57, and the BATS test file is actually a Bash script file, fully written using shell syntax.

It is strongly recommended to use source code or NPM to install the latest version. Bats has been taken over by many people, so the code storage is quite chaotic. The old version is installed in some systems, and the address of the version being maintained by the community is:

https://github.com/bats-core/bats-core.git

brew install bats-coreCopy the code

yum install batsCopy the code

apt-get install batsCopy the code

1. Git for Windows Bash (MSYS2 based)

2. Windows Subsystem for Linux

3. MSYS2

4. Cygwin

In Windows, you are advised to install the software using source code or NPM.

git clone https://github.com/bats-core/bats-core.gitcd bats-core
./install.sh /usr/local
Copy the code

docker pull bats/bats
docker run -it --rm bats/bats --version
docker run -it --rm -v "$(pwd):/code" bats/bats /code/testCopy the code

npm install -g bats

Copy the code

After testing, the CentOS version is 2014, and ubuntu is not the latest version. You are advised to use the source code or NPM to install the latest version.

The syntax format of a test case is:

#! /usr/bin/env bats@test "grep --version check" {# Test case name
  run grep --version                   The external command to run
  [ $status -eq0]# assertion
  [ "${lines[0]%% *}"= ='grep' ]      # assertion
}Copy the code

Each Bats test file is evaluated n + 1 times, where n is the number of test cases in the file. When running a test script, you first count the number of test cases, then iterate through the test cases and execute each test case in a separate process.

When running test cases, Bats uses Bash’s Errexit (set-e) option so that statements written inside @test are truth assertions. If one of the assertions in a test case fails (the status code for a statement is not 0), the test case is considered to have failed.

Three, common instructions

Use to test an external command and then assert its exit status and output status. Bats includes a run directive that calls the incoming parameters as commands and stores the exit and output states in special global variables so that you can continue to add assertions to test cases.

Let’s say we’re testing cat:

#! /usr/bin/env bats@test "cat nonexistent_filename check" {# Test case name
  run cat nonexistent_filename                                              The external command to run
  [ $status -eq1]# assertion
  [ "$output"= ='cat: nonexistent_filename: No such file or directory' ]   # assertion
  [ "${lines[0]}"= ='cat: nonexistent_filename: No such file or directory' ] # assertion
}Copy the code

There are three special variables for Bats

$status is the command exit status code

$output is the standard output and standard error content of the command

$lines is an array of command output containing only one line cat nonexistent_filename

If you want to share environment variables or custom functions across multiple test files, you can use the Load directive. The extension file name of the shared file must be.bash. Load can use a relative path or an absolute path.

The relative path is written (the extension file name can be omitted) :

load test_helperCopy the code

When writing absolute path (extension filename must be included) :

load /test_helpers/test_helper.bashCopy the code

In case of failure during the test, if you want to continue, you can skip the test with the skip command:

@test "A test I don't want to execute for now" {
  skip
  run foo
  [ "$status" -eq0]}Copy the code

Can also be added to skip reasons:

@test "A test I don't want to execute for now" {
  skip "This command will return zero soon, but not now"
  run foo
  [ "$status" -eq0]}Copy the code

Or you can decide whether to skip according to the conditions:

@test "A test which should run" {  if [ foo != bar ]; then
    skip "foo isn't bar"
  fi

  run foo
  [ "$status" -eq0]}Copy the code

This is Bats’ main function and all test cases are written in this format:

@test "grep --version check" {         # Test case name
  run grep --version                   The external command to run
  [ $status -eq0]# assertion
  [ "${lines[0]%% *}"= ='grep' ]      # assertion
}Copy the code

Setup/teardown are two special functions for initialization and cleanup before and after a test case starts. Such as setting environment variables to create a test directory before you start. Take soar as an example:

setup() {
  export SOAR_DEV_DIRNAME="${BATS_TEST_DIRNAME}/.. /"
  export SOAR_BIN="${SOAR_DEV_DIRNAME}/bin/soar" 
  export SOAR_BIN_ENV="${SOAR_DEV_DIRNAME}/bin/soar -config ${SOAR_DEV_DIRNAME}/etc/soar.yaml" 
  export BATS_TMP_DIRNAME="${BATS_TEST_DIRNAME}/tmp"
  export BATS_FIXTURE_DIRNAME="${BATS_TEST_DIRNAME}/fixture"
  mkdir -p "${BATS_TMP_DIRNAME}"//TODO...... }Copy the code

Bats contains several global variables:

1. $BATS_TEST_FILENAME Specifies the absolute path to the Bats test file.

2. $BATS_TEST_DIRNAME Bats Specifies the directory where the test file resides.

3. $BATS_TEST_NAMES Array of function names for each test case.

4. $BATS_TEST_NAME Specifies the name of the function containing the current test case.

5. $BATS_TEST_DESCRIPTION Describes the current test case.

6. $BATS_TEST_NUMBER index (starting at 1) of the current test case in the test file.

7. $BATS_TMPDIR The location of the directory where temporary files are stored.

Bats will immediately interrupt execution of code written outside of @test if it fails. This can be useful in some cases, such as checking dependencies, but if any output printed outside of @test, setup, teardown must be redirected to stderr(>&2), Otherwise, the output might contaminate the TAP stream and cause THE Bats test to fail.

Code that is not tested at @test takes precedence

You can read this if Bats is stuck.

Bats separates the output stream of the test code from the TAP output stream to ensure that the TAP output is not contaminated. The output to terminal section details how to correctly print custom text using FD3.

But one known problem with using FD3 is that it can cause Bats to freeze under certain circumstances (such as when a program’s child process is running in the background). In this case, when the child process is generated, the child process inherits FD3 from the parent process, causing Bats to wait for the child process to complete execution and shut down FD3. Bats blocks for the same amount of time if the child process takes a lot of time to complete, for example, if the child process is a sleep 100 command or a background service.

To avoid this, explicitly close FD3 command_name 3>&- before starting a potentially long-running child process

For example:

1. The situation of being stuck:

@test "cat nonexistent_filename check" {            # Test case name
  run cat nonexistent_filename
  sleep 100  &                                      # background execution
  [ $status -eq[1]"$TTTT" -eq1]# assertion
  [ "$output"= ='cat: nonexistent_filename: No such file or directory' ]   # assertion
}Copy the code

2. Conditions that will not be stuck:

@test "cat nonexistent_filename check" {                # Test case name
  run cat nonexistent_filename
  sleep 100 3>&- &                                      Execute in the background and close file descriptor 3
  [ $status -eq[1]"$TTTT" -eq1]# assertion
  [ "$output"= ='cat: nonexistent_filename: No such file or directory' ]   # assertion
}Copy the code

1. Inside the @test function

(1) If you output characters from @test inside you need to redirect the output to FD3 for example echo ‘test’>&3. The output will then become part of the TAP stream. To generate 100% tap-stream format output, we recommend echo ‘# text’ >&3. Otherwise, you may experience unexpected errors when using third-party tools that analyze TAP streams.

(2)Bats uses the friendly output format (-p, –pretty) by default. The TAP stream does not output any characters from FD3 by default

(3) Regardless of the output format specified, text directly output to stdout and stderr (such as echo “aaaa”) will be treated as part of the test by @test and will only be displayed if the test fails.

2. Output inside the setup/teardown function

This part of the output behavior is the same as @test.

3. Output outside @test or setup/teardown

(1) Wherever the output character is redirected (FD1, FD2, FD3) the character is immediately displayed on the terminal

(2) Text printed this way will disable the friendly output format (-p, –pretty). It also makes the output inconsistent with the TAP stream specification.

(3) Characters output to standard error from internal pipes or redirects will always be displayed first.

Bats command line usage:

Bats 1.1.0  Usage: bats [-cr] [-f <regex>] [-p | -t] <test>...
       bats [-h | -v]

  <test> Count the number of test cases that did not run for a BATS test case file, or for a directory containing a.bats file with the suffix -c, --count-f, --filter specifies to run some test cases through regular expressions -h, --help displays help information -p, --pretty displays test case output in a friendly way (default) -r, --recursive includes test -t in subdirectories, --tap displays output in tap format -v, --version displays version number informationCopy the code

For more information see:

https://github.com/bats-core/bats-coreCopy the code

Install Bats plug-in in common development tools to add code highlighting and code completion prompts to improve development efficiency.

More development tools and plug-ins:

Github.com/bats-core/b…

bats-core:

Github.com/bats-core/b…

Bat Evaluation Process:

Github.com/sstephenson…