This is the 30th day of my participation in the August Challenge

Traditional testing vs table-driven testing

Traditional test

Disadvantages of traditional testing

Test data is mixed with test logic
The error message is not clear
Once a data error occurs, the test is over

Table-driven testing

Go uses table-driven tests. In the functional Testing section of this article, there are table driven tests, which can be seen in conjunction with this article

Table-driven testing benefits

Separate test data from test logic
Explicit error message (we can customize the output ourselves)
You can partially fail (the test doesn’t end because one of them doesn’t pass)
The syntax of the GO language makes it easier to practice table-driven testing

Go the test tool

The GO test subcommand is a test driver for the GO language package, which is organized according to certain conventions. In a package directory, files ending in _test.go are not the target of go build command compilation, but of Go test compilation

In the *_test.go file, three types of functions require special treatment: functional test functions, benchmark functions, and sample functions.

Functional Test functions are named with the prefix Test and are used to check the correctness of some program logic. Go Test runs the Test function and reports whether the result is PASS or FAIL
The name of a Benchmark function starts with Benchmark and is used to test the performance of an operation. Go Test reports the average execution time of an operation
The name of the Example function, starting with Example, to provide machine-checked documentation

The go test utility scans the *_test.go file for special functions, generates a temporary main package to call them, compiles and runs it, reports the results, and finally empties the temporary file

A functional test

For each test file, you must import the Testing package. The function signature of these functions is as follows:

func TestName(t *testing.T) {
    //...
}
Copy the code

Functional Test functions, which must start with Test, and optional suffix names must start with a capital letter

func TestSin(t *testing.T) {/*... */} func TestCos(t *testing.T) {/*... */} func TestLog(t *testing.T) {/*... * /}Copy the code

The parameter t provides the ability to report test failures and log. Below implements a string of function used to judge whether a string is palindrome (. / SRC/go language/ch11 / word/word. Go)

package word func IsPalindromeNew(str string) bool { for i := range str { if str[i] ! = str[len(str)-1-i] { return false } } return true }Copy the code

In the same directory, the file word_test.go contains two function test functions TestIsPalindromeNew and TestNonPalindromeNew. Both functions check whether IsPalindromeNew gives the correct result for a single input parameter and report an error with t. ror

func TestIsPalindromeNew(t *testing.T) { if ! IsPalindromeNew("detartrated") { t.Error(`IsPalindromeNew("detartrated") == false`) } if ! IsPalindromeNew("kayak") { t.Error(`IsPalindromeNew("kayak") == false`) } } func TestNonPalindromeNew(t *testing.T) { if  IsPalindromeNew("palindrome") { t.Error(`IsPalindromeNew("palindrome") == true`) } }Copy the code

The go test (or go build) command takes the package of the current directory as an argument without specifying package parameters. You can compile and run tests with the following commands

$CD $GOPATH/SRC/go. Language/ch11 / word $go test PASS. Ok go language/ch11 / word 0.007 sCopy the code

You can see that the program tests pass

An obvious bug with the function implemented above to determine whether a string is a palindrome string is that if all characters in the string are ASCII, that’s fine. If there were characters like Chinese, it would be a problem

Func TestChinesePalindrome(t *testing.T) {STR := "TestChinesePalindrome" if! Func TestChinesePalindrome(t *testing. IsPalindromeNew(str) { t.Errorf(`IsPalindromeNew(%q) = false`, str) } }Copy the code

Because STR is long, to avoid writing it twice, Errorf is used, which, like Printf, provides formatting

If you run the go test command, the test fails

$go test -- FAIL: TestChinesePalindrome (0.00s) word_test.go:26: IsPalindromeNew(" 123 123 123 ") = false FAIL exit status 1 FAIL go.language/ CH11 /word 0.005sCopy the code

If there are many test cases in a test suite, we can selectively execute test cases. -v prints the name and execution time of each test case in the package

$go test -v === RUN TestIsPalindromeNew -- PASS: TestIsPalindromeNew (0.00s) === RUN TestNonPalindromeNew -- PASS: TestNonPalindromeNew (0.00s) === RUN TestChinesePalindrome word_test.go:26: IsPalindromeNew(" one, two, three, two ") = false -- FAIL: TestChinesePalindrome (0.00s) FAIL Exit status 1 FAIL go. Language/CH11 / Word 0.004sCopy the code

The -run argument is a regular expression that causes the Go test to run only functions whose test function names match a given pattern

$ go test -v -run="Chinese|English" === RUN TestChinesePalindrome word_test.go:26: IsPalindromeNew(" one, two, three, two ") = false -- FAIL: TestChinesePalindrome (0.00s) FAIL Exit status 1 FAIL go. Language/CH11 / Word 0.005sCopy the code

Below are functions to refine the judgment palindrome string to support more types of characters

func IsPalindromeRune(str string) bool { strRune := []rune(str) for i, _ := range strRune { if strRune[i] ! = strRune[len(strRune) - i - 1] { return false } } return true }Copy the code

Then write a test case to test the new line of sight method for determining palindrome strings

func TestIsPalindromeRune(t *testing.T) { var tests = []struct { input string want bool }{ {"", true}, {"a", true}, {" aa ", true}, {" ab ", the false}, {" kayak ", true}, {" detartrated ", true}, {" is a test, "false}, {" test is a test and one is" the false},} for _, test := range tests { if got := IsPalindromeRune(test.input); got ! = test.want { t.Errorf("IsPalindromeRune(%q) = %v", test.input, Got)}}} // Run the following test case: $go test v-run ="TestIsPalindromeRune" === run TestIsPalindromeRune -- PASS: TestIsPalindromeRune (0.00s) PASS OK go.language/ CH11 /word 0.004sCopy the code

This type of table-based testing is common in Go. It is straightforward to add new table items as needed, and since the assertion logic is not duplicated, we can spend some effort to make the output error messages look better

The current failed test case output from a call to t. rorf does not contain the entire trace stack and does not cause an application to break down or terminate execution, unlike assertions in many other language testing frameworks. The test cases are independent of each other. If one entry in the test table causes the test to fail, the other entries will continue to be tested so that we can find multiple failures in a single test

If we really need to terminate a test function, such as confusing output due to failed initialization code or avoiding an existing error, we can terminate the test using the t.atal or t.alf functions. These functions must be called in the same Goroutine as the Test function, not in any other Goroutine created by the Test

The general format of the test error message is “f(x)=y,wantz”, where f(x) represents the operation to be performed and its input, y is the actual output result, and z is the expected result. For convenience, we will use the Go syntax for f(x). For example, in the palindrome example above, we use the Go format to display long input and avoid repeated typing. In table-based testing, the output x is important because an assertion statement is executed multiple times with different inputs. Avoid boilerplate text and redundant information for error messages

Random test

Table-based testing is convenient for testing logically interesting use cases against carefully selected input functions. Another approach is random testing, which extends the coverage of the test by building random inputs

If the input given is random, how do we know what the function outputs? There are two strategies. One way to do this is to write an additional function that uses an inefficient but clean algorithm and then check whether the output of the two implementations is consistent. Another way is to build inputs that conform to a pattern, so we know what their corresponding outputs are, right

The following example uses the second method. The randomPalindrome function produces a sequence of palindrome strings that are determined to be palindrome strings at construction time

Runes := make([]rune, n) for I := 0; func randomPalindrome(RNG *rand.Rand) string {n := rng.intn (25) runes := make([]rune, n) for I := 0; i < (n+1)/2; I ++ {r := rune(rng.intn (0x1000))// The maximum random character is '\u0999' runes[I] = r runes[n-i-1] = r} return string(runes)} func TestRandomPalindrome(t * testing.t) {// Initialize a pseudo-random number generator seed: = time.now ().utc ().unixnano (). %d", seed) rng := rand.New(rand.NewSource(seed)) for i:=0; i<1000; i++ { p := randomPalindrome(rng) if ! IsPalindromeRune(p) { t.Errorf("IsPalindromeRune(%q) = false", p) } } }Copy the code

Because of the uncertainty of random testing, in the event of a test case failure, it is important to record enough information to reproduce the problem. In this case, the input P of the IsPalindromeRune function tells us all we need to know, but for functions with more complex inputs, it’s much easier to seed the pseudo-random number generator than to dump the entire input data structure. With the seed of random numbers, we can simply modify the test code to accurately reproduce the error

By using the current time as the seed source for the pseudorandom number, new input is given every time the test is run throughout its life. This is important if your project uses an automated system to run tests periodically

White box testing

One way tests can be classified is based on internal knowledge of the package being tested. Black-box testing assumes that testers know about packages only through public apis and documentation, and that the package’s internal logic is opaque. In contrast, white-box tests have access to the package’s internal functions and data structures and can make observations and changes that regular users cannot. For example, a white-box test can check that the data type immutability of a package is maintained after each operation

The two approaches are complementary. Black box tests are generally more robust and require little modification after each program update. They also help test authors focus on the users of the package and discover flaws in the API design. Conversely, white-box testing can provide more detailed coverage of specific areas of the implementation

The function TestIsPalindromeNew above calls only the exported function IsPalindromeNew, so it is a black box test

coverage

Testing, by its very nature, is never over. Edsger Dijkstra, a renowned computer scientist, said, “Tests are designed to find bugs, not to prove they don’t exist.” No amount of testing can prove a package is bug-free. At best, they reinforce our confidence that these packages can be used in many important scenarios

The percentage of a test suite that covers the package to be tested is called test coverage. Coverage can’t be measured directly by volume, anything is dynamic, and even the tiniest program can’t be accurately measured. But there are ways to focus our testing efforts where they have the most potential

Statement coverage is one of the simplest and most widely used methods. Statement coverage of a test suite means that some statements are executed at least once in a single execution. This section will use Go’s Cover tool, which is integrated into the Go Test to measure statement coverage and help identify significant differences between tests

For example, to check whether a string is a palindrome string, go to the directory where the test file resides and run the following command to generate coverage data

$ go test -coverprofile=c.out
Copy the code

This command generates a c.out file (optionally named) under the current file. Use the Go Tool cover command to view the contents of the file on the web page

$ go tool cover -html=c.out
Copy the code

Executing this command opens a web page with the following content

Blue means covered, red means not covered

The benchmark

Benchmark functions

Benchmarking is a way to test application performance under a certain workload. In Go, the Benchmark function looks like a test function, but is prefixed with Benchmark and has a * testing.b parameter that provides most of the same methods as testing.t, with some additional performance-related methods added. It also provides an integer member N that specifies the number of times the operation being detected is executed

Here is the benchmark for the IsPalindromeNew function, which calls IsPalindromeNew N times in a loop

package word import "testing" func BenchmarkIsPalindromeNew(b *testing.B) { for i:=0; i<b.N; i++ { IsPalindromeNew("abcdedcba") } }Copy the code

Execute it using the following command. Unlike tests, no benchmarks are run by default. The mark-bench parameter specifies the benchmark to run. It is a regular expression that matches the name of a Benchmark function. Its default value does not match any function. Mode “. “makes it match all the benchmark functions in the package Word, and since there is only one benchmark function, it works the same as specifying -bench=IsPalindromeNew

$ go test -bench=. goos: darwin goarch: amd64 pkg: go.language/ch11/word cpu: Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz BenchmarkIsPalindromeNew-8 100000000 11.52 ns/op PASS OK The go language/ch11 / word 1.170 sCopy the code

The numeric suffix 8 of the benchmark name represents the value of GOMAXPROCS (which was shared in the previous article and can be read here), which is important for concurrent benchmarks. The output tells us that each call to IsPalindromeNew takes 0.01152ms, which is the average of 100 million calls. Because the benchmark runner does not initially know how long this operation will take, it starts with a small value of N for detection, and then extrapolates a large enough value of N to detect stable running times

The reason for using the benchmark function to implement the loop rather than calling the code in the test driver is that in the benchmark function, some necessary initialization code can be performed outside the loop, and this time is not added to the time of each iteration. The testing.b parameter provides methods to stop, resume, and reset the timer if the initialization code interferes with the result, but these methods are rarely used (for example, to reset the timer, use b.resettimer ()))

Now that you have benchmarking and functional testing, it’s easy to think about making your program faster. Perhaps the most obvious optimization is to make the second loop of the IsPalindromeNew function stop detection in the middle to avoid comparing two times:

n := len(str)/2 for i := 0 ; i < n; i++ { if str[i] ! = str[len(str) - 1 - i] { return false } } $ go test -bench=. goos: darwin goarch: amd64 pkg: go.language/ch11/word cpu: Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz BenchmarkIsPalindromeNew-8 384384532 3.136 ns/op PASS OK The go language/ch11 / word 1.528 sCopy the code

As you can see, the performance of the application has been greatly improved. The fastest programs are those that allocate memory the fewest times. The command line tag -benchmem includes memory allocation statistics in the report

This benchmark tells us the absolute time taken for a given operation, but in many cases the performance issue that is causing concern is the relative time taken between two different operations. For example, if a function takes 1ms to process 1000 elements, how long does it take to process 10,000 or a million elements?

Another example: what is the optimal size for an I/O buffer. Benchmarking an application with a range of sizes can help us select the smallest buffers for the best performance

Third example: Which algorithm performs best for a task? Using the same input for two different algorithms, benchmarking often reveals the strengths and weaknesses of each algorithm under significant or representative workloads

The performance comparison function is just plain old code. They usually take the form of functions with one argument, which are called by different Benchmark functions passing in different values, like this:

func benchmark(b *testing.B, size int*) {/* .. *. */} func Benchmark10(b *testing.B) { benchmark(b, 10) } func Benchmark100(b *testing.B) { benchmark(b, 100) } func Benchmark1000(b *testing.B) { benchmark(b, 1000) }Copy the code

The size argument specifies the size of the input. Each Benchmark function passes a different value, but it is a constant inside each function. Do not use b.N as the input size. The result of this benchmark is meaningless unless you think of it as the number of loops for fixed-size inputs

Performance analysis

Performance analysis is an automatic method. During program execution, performance is evaluated based on sampling of some performance events, and then the statistical report is deduced and analyzed from the sampling. The statistical report is called performance profile.

Go supports a number of performance profiling approaches, each related to a different aspect of performance, but they all need to log related events, and each has related stack information — the stack of function calls that are active at the time the event occurs. The go Test tool has built-in support for several categories of performance profiling

CPU profiling: Identify the functions that require the most CPU during execution. Threads executing on each CPU are periodically interrupted by the operating system every few milliseconds, a profiling event is recorded during each interruption, and normal execution resumes

Heap performance profiling: Identifies the statements responsible for allocating the most memory. The performance profiling library samples coroutine internal memory allocation calls, so each performance profiling event records an average of 512KB of allocated memory

Blocking performance profiling: ** Identifies those operations that block the coroutine the longest. ** Such as system calls, channels sending and receiving data, and locks. The performance analysis library logs an event each time a Goroutine is blocked by one of the above operations

Getting a performance profile of the code to be tested is easy by specifying a tag as shown below. When using more than one tag at a time, it is important to note that the mechanism for obtaining performance analysis reports is that when obtaining a report in one category, it overwrites reports in the other categories

$ go test -cpuprofile=cpu.out
$ go test -blockprofile=block.out
$ go test -memprofile=mem.out
Copy the code

Although the specifics differ for ephemeral command-line tools and long-running server programs, it is easy to add profiling support for nontest programs. Profiling is especially useful for long-running programs, so the profiling feature of the Go runtime can be enabled by programmers through the Runtime API

After we get the performance profiling results, we need to open a tool to analyze it using pprof. This is a standard part of the Go distribution, but because it is not used often, it is used indirectly through the Go Tool pprof. It has many features and options, but the basic usage has only two parameters, an executable file that produces profiling results, and profiling logs

To make the profiling process efficient and space-saving, the profiling log does not contain function names but uses their addresses. This means that the Pprof tool needs an executable to understand the data content. Although the Go Test tool normally disowns the executable generated temporarily for testing after the test is complete, when profiling is enabled, it saves and names the executable foo.test, where foo is the name of the package being tested

Using pprof

Here’s an example of performance profiling using the code used in the previous article to find the longest non-repeating substring. The code for the longest non-repeating substring looks like this:

package nonrepeating

func lengthOfNonRepeatingSubStr(s string) int {
	lastOccured := make(map[rune]int)
	start := 0
	maxLength := 0
	for i, ch := range []rune(s) {
		lastId, ok := lastOccured[ch]
		if ok && lastId >= start {
			start = lastId + 1
		}
		if i - start +1 > maxLength {
			maxLength = i - start + 1
		}
		lastOccured[ch] = i
	}

	return maxLength
}
Copy the code

Write a Benchmark test function that looks like this:

Package nonrepeating import "testing" func BenchmarkLengthOfNonRepeatingSubStr (b * testing. B) {s: = "12321" ans: = 3 for  i :=0; i < b.N; i++ { actual := LengthOfNonRepeatingSubStr(s) if actual ! = ans { b.Errorf("got %d for input %s; " + "expected %d", actual, s, ans) } } }Copy the code

Take a look at its performance test results before tuning

$ go test -bench . goos: darwin goarch: amd64 pkg: go.language/ch11/nonrepeating cpu: Intel (R) Core (TM) ng7 i7-1068 @ 2.30 GHz CPU BenchmarkLengthOfNonRepeatingSubStr 9461041-8-127.4 ns/op PASS ok The go language/ch11 / nonrepeating 1.340 sCopy the code

Now we want to see what is the slowness of this function up here? Where does it take time? Run the following command:

$ go test -bench . -cpuprofile=cpu.out
Copy the code

After executing, you will find a CPU. Out binary file generated in the current directory, because it is a binary file, do not open the contents of the file by command, because it is not easy to understand. There’s another way to look at it

$ go tool pprof cpu.out
Type: cpu
Time: Aug 30, 2021 at 2:48pm (CST)
Duration: 1.45s, Total samples = 1.17s (80.66%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof)
Copy the code

You will see the following output, and the last line is an interactive command line. You can also type other commands, such as help, to see which commands can be executed. Here is a demonstration of the simplest command, enter web. This will open an SVG file in the web page. If you type in web, it will say something like this:

failed to execute dot. Is Graphviz installed? Error: exec: "dot": executable file not found in $PATH
Copy the code

Gvedit is not installed on your computer, go to the gvedit official website to download a stable version of the installation can be (there are various operating system installation methods in the official website, you can also install the source code). After the installation is complete, set the environment variable path and add the bin folder of the gvedit installation path

The bigger the box, the more time it takes, and we can use that information to optimize the program

It can also be presented as a table (equivalent to typing text on the interactive command line above)

$ go test -run=NONE -bench=. -cpuprofile=cpu.log goos: darwin goarch: amd64 pkg: go.language/ch11/word cpu: Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz BenchmarkIsPalindromeNew-8 386407360 3.096 ns/op PASS OK Go. language/ CH11 /word 1.660s $go tool pprof -text -nodecount=10 cpu.log Type: CPU Time: Aug 30, 2021 at 3:54pm (CST) Duration: 1.43s, Total samples = 1.12s (78.19%) Showing nodes accounting for 1.08s, 96.43% of 1.12s total Showing top 10 nodes out of 30 flat flat% sum% cum% 0.30s 26.79% 26.79% 0.48s 42.86% Decoderune 0.12s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s 0.1212s Runtime. Mapaccess2_fast32 0.11 s 9.82% 68.75% s 97.32% go. 1.09 language/ch11 / nonrepeating LengthOfNonRepeatingSubStr 0.11 s 9.82% 78.57% s 31.25% 0.35 runtime. Stringtoslicerune 0.10 s 8.93% 87.50% s 8.93% runtime. 0.10 memhash32 0.06 s 5.36% 92.86% (*hmap). Growing (inline) 0.05s 1.10 s 98.21% go. Language/ch11 / nonrepeating BenchmarkLengthOfNonRepeatingSubStr 0.01 0.01 s 0.89% s 0.89% 96.43% runtime.(*bmap).overflowCopy the code

The mark-text specifies the format of the output. In the example above, a text table is shown, one function per row, that is the “hot function” sorted by the rule that consumes the most CPU. The -nodecount=10 flag limits the output to 10 lines. For more obvious performance issues, this text-formatted output is revealing enough

reference

Golang performance debugging and PPROF visualization

The Go Programming Language — Alan A. A. Donovan

Go Language Learning Notes — Rain Marks

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

GO Language Basics (30) – One article to solve the GO test

Traditional testing vs table-driven testing