Introduction to the

You’ve probably heard about Go’s excellent concurrency. Perhaps this most prominent feature is what makes Go so popular, making it ideal for writing other software like Docker, Kubernetes, and Terraform.

Before you can begin to understand how concurrency works in Go, you may need to forget what you already know from other programming languages. Go uses a very different approach.

By the time you follow this module, you already have the knowledge you need to study more advanced topics, such as concurrency. However, we need to explain why concurrency is needed first. We’ll go through the different topics one by one.

We recommend that you practice all the sample code until you have a good understanding of the concepts. As you experienced in the previous modules, practice helps you understand concepts better.

Let’s first look at what you will learn in this module.

Learning goals

In this module, you will learn the following:

How concurrency works in Go.

  • The difference between concurrency and parallelism.
  • How to use channels to communicate in concurrent programs.
  • How to write faster programs by enabling concurrency.
  • How to write dynamic programs that can use buffers to take advantage of concurrency when you want to start a limited number of concurrent calls.

A prerequisite for

  • Get ready to create the Go environment for your application.
  • Ability to create and modify.go files.
  • Ability to run Go applications using terminal prompts.
  • Understand basic data types, such as String, int, and Boolean.
  • Learn how to write basic data control flows, such as if and for statements.
  • Know how to write functions.
  • Learn how to use libraries, such as NET/HTTP.

Learn about Goroutine (Lightweight threads)

Concurrency is a combination of independent activities, just as a Web server runs autonomously even though it processes multiple user requests at the same time. Concurrency exists in many programs today. Web servers are an example, but you can also see the need for concurrency when batch processing large amounts of data.

Go has two styles for writing concurrent programs. One is the traditional style implemented by threads in other languages. In this module, you will learn about the style of Go, where values are passed between separate activities called goroutines to communicate with processes.

If this is your first time learning concurrency, we encourage you to spend some time reviewing each piece of code we’re going to write to get your hands on it.

Go implements the method of concurrency

Often, the biggest problem when writing concurrent programs is sharing data between processes. Go communicates differently from other programming languages because it passes data back and forth through channels. This means that only one activity (goroutine) has access to the data, and by design there is no contention condition. After studying Goroutine and Channel in this module, you will have a better understanding of Go’s concurrency approach.

Go’s approach can be summed up with the tagline: “Not by communicating shared memory, but by communicating shared memory.” You can learn more about shared memory through communication in the Go blog post, but we’ll continue this discussion in the next section.

As mentioned earlier, Go also provides low-level concurrency primitives. But in this module, we’ll just cover the idiomatic way Go implements concurrency.

Let’s start by exploring Goroutine.

Goroutine

Goroutine is a concurrent activity in a lightweight thread, rather than a traditional activity in an operating system. Suppose you have a program that writes output and another function that computes the sum of two numbers. A concurrent program can have several Goroutines calling both functions simultaneously.

We can say that the first goroutine executed by the program is the main() function. If you want to create other Goroutines, you must use the go keyword before calling this function, as follows:

func main(a){
    login()
    go launch()
}
Copy the code

You’ll also find that many programs like to use anonymous functions to create goroutines, as shown below:

func main(a){
    login()
    go func(a) {
        launch()
    }()
}
Copy the code

To see how this works, let’s write a simple concurrent program.

Write concurrent programs

Since we only want to focus on the concurrent part, we use an existing program to check that the API endpoint is responsive. The code is as follows:

package main

import (
    "fmt"
    "net/http"
    "time"
)

func main(a) {
    start := time.Now()

    apis := []string{
        "https://management.azure.com"."https://dev.azure.com"."https://api.github.com"."https://outlook.office.com/"."https://api.somewhereintheinternet.com/"."https://graph.microsoft.com",}for _, api := range apis {
        _, err := http.Get(api)
        iferr ! =nil {
            fmt.Printf("ERROR: %s is down! \n", api)
            continue
        }

        fmt.Printf("SUCCESS: %s is up and running! \n", api)
    }

    elapsed := time.Since(start)
    fmt.Printf("Done! It took %v seconds! \n", elapsed.Seconds())
}
Copy the code

When you run the previous code, you see the following output:

SUCCESS: https://management.azure.com is up and running!
SUCCESS: https://dev.azure.com is up and running!
SUCCESS: https://api.github.com is up and running!
SUCCESS: https://outlook.office.com/ is up and running!
ERROR: https://api.somewhereintheinternet.com/ is down!
SUCCESS: https://graph.microsoft.com is up and running!
Done! It took 1.658436834 seconds!
Copy the code

There’s nothing special here, but we can do better. Maybe we can check all the sites at once? The program can be done in 500 milliseconds instead of nearly two seconds.

Note that the part of the code we need to run concurrently is the part that makes HTTP calls to the site. In other words, we need to create a Goroutine for each API our program checks.

To create a Goroutine, we need to use the go keyword before calling the function. But we don’t have a function here. Let’s refactor the code and create a new function like this:

func checkAPI(api string) {
    _, err := http.Get(api)
    iferr ! =nil {
        fmt.Printf("ERROR: %s is down! \n", api)
        return
    }

    fmt.Printf("SUCCESS: %s is up and running! \n", api)
}
Copy the code

Notice that we no longer need the continue keyword because we are not in the for loop. To stop the flow of execution of a function, simply use the return keyword. Now we need to modify the code in the main() function to create a Goroutine for each API, as follows:

for _, api := range apis {
    go checkAPI(api)
}
Copy the code

Rerun the program and see what happens.

Looks like the program doesn’t check the API anymore, right? You might see output like this:

Done! It took 1.506 e-05 seconds!Copy the code

That’s fast! What’s going on? Notice that the last message you see says the program is complete. This is because Go creates a Goroutine for each site in the loop and immediately goes to the next line.

Even if it looks like the checkAPI function is not running, it is. It just didn’t have time to finish. Notice what happens if you add a sleep timer after the loop, as follows:

for _, api := range apis {
    go checkAPI(api)
}

time.Sleep(3 * time.Second)
Copy the code

Now, when you rerun the program, you might see output like this:

ERROR: https://api.somewhereintheinternet.com/ is down!
SUCCESS: https://api.github.com is up and running!
SUCCESS: https://management.azure.com is up and running!
SUCCESS: https://dev.azure.com is up and running!
SUCCESS: https://outlook.office.com/ is up and running!
SUCCESS: https://graph.microsoft.com is up and running!
Done! It took 3.002114573 seconds!
Copy the code

Looks like it’s working, right? Not exactly. What if you want to add a new site to the list? Maybe three seconds isn’t enough. How do you know? You can’t manage. There has to be a better way, and that’s what we’ll cover when we discuss channels in the next section.

Use a channel as a communication mechanism

A channel in Go is the communication mechanism between Goroutines. That’s why we said earlier that Go achieves concurrency by saying, “Instead of communicating through shared memory, communicate through shared memory.” Channels are used when you need to send values from one Goroutine to another. Let’s take a look at how they work and how to start using them to write concurrent Go programs.

The Channel of grammar

Since a channel is a communication mechanism for sending and receiving data, it also has types. This means that you can only send data types supported by the channel. In addition to using the keyword chan as the data type for the channel, you also need to specify the data type to be passed through the channel, such as int.

You need to use chan

every time you declare a channel or want to specify a channel as an argument in a function, such as chan int. To create a channel, use the built-in make() function, as follows:

ch := make(chan int)
Copy the code

A channel can perform two operations: sending data and receiving data. To specify the type of operation a channel has, use the channel operator <-. In addition, sending and receiving data in a channel are blocking operations. You’ll see why in a moment.

If you want a channel to send only data, you must use the <- operator after the channel. If you want a channel to receive data, you must use the <- operator before the channel, as follows:

ch <- x // sends (or write) x through channel ch
x = <-ch // x receives (or reads) data sent to the channel ch
<-ch // receives data, but the result is discarded
Copy the code

Another action you can perform in a channel is to close a channel. To close a channel, use the built-in close() function, as follows:

close(ch)
Copy the code

When you close a channel, you expect that data will no longer be sent in that channel. If you try to send data to a closed channel, the program will experience a serious error. If you try to receive data through a closed channel, you can read all the data sent. Each subsequent “read” will return a zero value.

Let’s go back to the program we created earlier and use Channel to remove the sleep function and clean it up a bit. First, let’s create a string channel in the main function, as follows:

ch := make(chan string)
Copy the code

Next, delete the Sleep row time.sleep (3 * time.second).

Now we can use channels to communicate between goroutines. Instead of printing the result in the checkAPI function, let’s refactor the code and send the message through a channel. To use a channel in this function, you need to add a channel as an argument. The checkAPI function should look like this:

func checkAPI(api string, ch chan string) {
    _, err := http.Get(api)
    iferr ! =nil {
        ch <- fmt.Sprintf("ERROR: %s is down! \n", api)
        return
    }

    ch <- fmt.Sprintf("SUCCESS: %s is up and running! \n", api)
}
Copy the code

Note that we have to use the FMT.Sprintf function because we don’t want to print any text from here. Send formatted text directly. Also notice that we use the <- operator after the channel variable to send data.

Now, you need to change the main function to send the channel variable and receive the data to print, as follows:

ch := make(chan string)

for _, api := range apis {
    go checkAPI(api, ch)
}

fmt.Print(<-ch)
Copy the code

Note that we use the <- operator before the channel to indicate that we want to read from the channel.

When you rerun the program, you should see output like this:

ERROR: https://api.somewhereintheinternet.com/ is down!

Done! It took 0.007401217 seconds!
Copy the code

At least it doesn’t have to call the sleep function to work, right? But it still doesn’t serve our purpose. We only see the output of one goroutine, and we created five goroutines. In the next section, we’ll look at why the program works this way.

There is no buffer channel

When creating a channel using the make() function, an unbuffered channel is created, which is the default behavior. An unbuffered channel blocks the send operation until someone is ready to receive the data. This is why we said earlier that both send and receive are blocking operations. This is why the program in the previous section stops immediately after receiving the first message.

We can say that fmt.print (<-ch) blocks the program because it reads from a channel and waits for some data to arrive. As soon as any data arrives, it moves on to the next line, and the program completes.

What happened to the other Goroutines? They’re still running, but they’re not listening. Also, some Goroutines were unable to send data because the program finished early. To prove this, let’s add another fmt.print (<-ch), as follows:

ch := make(chan string)

for _, api := range apis {
    go checkAPI(api, ch)
}

fmt.Print(<-ch)
fmt.Print(<-ch)
Copy the code

When you rerun the program, you should see output like this:

ERROR: https://api.somewhereintheinternet.com/ is down!
SUCCESS: https://api.github.com is up and running!
Done! It took 0.263611711 seconds!
Copy the code

Notice that you now see the output of both apis. If you keep adding more FMT.Print(<-ch) lines, you’ll end up reading all the data sent to the channel. But what happens if you try to read more data and there’s no Goroutine to send again? Such as:

ch := make(chan string)

for _, api := range apis {
    go checkAPI(api, ch)
}

fmt.Print(<-ch)
fmt.Print(<-ch)
fmt.Print(<-ch)
fmt.Print(<-ch)
fmt.Print(<-ch)
fmt.Print(<-ch)

fmt.Print(<-ch)
Copy the code

When you rerun the program, you should see output like this:

ERROR: https://api.somewhereintheinternet.com/ is down!
SUCCESS: https://api.github.com is up and running!
SUCCESS: https://management.azure.com is up and running!
SUCCESS: https://graph.microsoft.com is up and running!
SUCCESS: https://outlook.office.com/ is up and running!
SUCCESS: https://dev.azure.com is up and running!
Copy the code

It’s running, but the program isn’t finished. The last print line blocks the program because it needs to receive data. A command like Ctrl+C must be used to close the program.

This just proves that reading data and receiving data are both blocking operations. To fix this, simply change the code for the loop and then accept only the data that is determined to be sent, as follows:

for i := 0; i < len(apis); i++ {
    fmt.Print(<-ch)
}
Copy the code

Here is the final version of the program, in case your version is wrong:

package main

import (
    "fmt"
    "net/http"
    "time"
)

func main(a) {
    start := time.Now()

    apis := []string{
        "https://management.azure.com"."https://dev.azure.com"."https://api.github.com"."https://outlook.office.com/"."https://api.somewhereintheinternet.com/"."https://graph.microsoft.com",
    }

    ch := make(chan string)

    for _, api := range apis {
        go checkAPI(api, ch)
    }

    for i := 0; i < len(apis); i++ {
        fmt.Print(<-ch)
    }

    elapsed := time.Since(start)
    fmt.Printf("Done! It took %v seconds! \n", elapsed.Seconds())
}

func checkAPI(api string, ch chan string) {
    _, err := http.Get(api)
    iferr ! =nil {
        ch <- fmt.Sprintf("ERROR: %s is down! \n", api)
        return
    }

    ch <- fmt.Sprintf("SUCCESS: %s is up and running! \n", api)
}
Copy the code

When you rerun the program, you should see output like this:

ERROR: https://api.somewhereintheinternet.com/ is down!
SUCCESS: https://api.github.com is up and running!
SUCCESS: https://management.azure.com is up and running!
SUCCESS: https://dev.azure.com is up and running!
SUCCESS: https://graph.microsoft.com is up and running!
SUCCESS: https://outlook.office.com/ is up and running!
Done! It took 0.602099714 seconds!
Copy the code

The program is doing what it should. Instead of using sleep functions, you use channels. Also note that without concurrency, the program takes about 600 milliseconds to complete, rather than nearly 2 seconds.

Finally, we can say that an unbuffered channel synchronizes send and receive operations. Even with concurrency, communication is synchronous.

Learn about buffered channels

As you know, channels are unbuffered by default. This means that they accept send operations only if there is a receive operation. Otherwise, the program is permanently blocked from waiting.

Sometimes you need to do such synchronization between Goroutines. However, sometimes you just need to implement concurrency without restricting how goroutines communicate with each other.

A buffered channel sends and receives data without blocking the program, because a buffered channel behaves like a queue. When creating a channel, you can limit the size of this queue as follows:

ch := make(chan string.10)
Copy the code

Each time data is sent to a channel, elements are added to the queue. The receive operation then removes the element from the queue. When a channel is full, any send operation will wait until there is room to save data. Conversely, if the channel is empty and there is a read operation, the program is blocked until there is data to read.

Here is a simple example of understanding a buffered channel:

package main

import (
    "fmt"
)

func send(ch chan string, message string) {
    ch <- message
}

func main(a) {
    size := 4
    ch := make(chan string, size)
    send(ch, "one")
    send(ch, "two")
    send(ch, "three")
    send(ch, "four")
    fmt.Println("All data sent to the channel ...")

    for i := 0; i < size; i++ {
        fmt.Println(<-ch)
    }

    fmt.Println("Done!")}Copy the code

When you run the program, you see the following output:

All data sent to the channel ...
one
two
three
four
Done!
Copy the code

You might say we’re not doing anything different here, and you’d be right. But let’s see what happens when you change the size variable to a smaller number (you can even try to use a larger number), as follows:

size := 2
Copy the code

When you re-run the program, you will see the following error:

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.send(...)
        /Users/developer/go/src/concurrency/main.go:8
main.main()
        /Users/developer/go/src/concurrency/main.go:16 +0xf3
exit status 2
Copy the code

This error occurs because calls to the send function are continuous. You’re not creating a new Goroutine. Therefore, there are no operations to queue.

Channel has a close relationship with Goroutine. Without another Goroutine receiving data from a channel, the entire program could be blocked permanently. As you can see, this does happen.

Now let’s have some fun! We’ll create a Goroutine for the last two calls (the first two calls were properly buffered) and run the for loop four times. The code is as follows:

func main(a) {
    size := 2
    ch := make(chan string, size)
    send(ch, "one")
    send(ch, "two")
    go send(ch, "three")
    go send(ch, "four")
    fmt.Println("All data sent to the channel ...")

    for i := 0; i < 4; i++ {
        fmt.Println(<-ch)
    }

    fmt.Println("Done!")}Copy the code

When you run the program, it works as expected. We recommend always using Goroutine when using channels.

To test the case of creating a buffered channel that contains more elements than needed, let’s use the previous example used to check the API and create a buffered channel of size 10:

package main

import (
    "fmt"
    "net/http"
    "time"
)

func main(a) {
    start := time.Now()

    apis := []string{
        "https://management.azure.com"."https://dev.azure.com"."https://api.github.com"."https://outlook.office.com/"."https://api.somewhereintheinternet.com/"."https://graph.microsoft.com",
    }

    ch := make(chan string.10)

    for _, api := range apis {
        go checkAPI(api, ch)
    }

    for i := 0; i < len(apis); i++ {
        fmt.Print(<-ch)
    }

    elapsed := time.Since(start)
    fmt.Printf("Done! It took %v seconds! \n", elapsed.Seconds())
}

func checkAPI(api string, ch chan string) {
    _, err := http.Get(api)
    iferr ! =nil {
        ch <- fmt.Sprintf("ERROR: %s is down! \n", api)
        return
    }

    ch <- fmt.Sprintf("SUCCESS: %s is up and running! \n", api)
}
Copy the code

When you run the program, you see the same output as before. You can change the size of a channel, test it with a smaller or larger number, and the program still works.

Unbuffered channel and buffered channel

Now, you might be wondering when to use both types. It all depends on how you want the communication between goroutines to work. Unbuffered channel synchronous communication. They guarantee that every time data is sent, the program is blocked until someone reads it from a channel.

Instead, a buffered channel decouples the send and receive operations. They won’t stop the program, but you have to be careful with them because they can end up causing deadlocks (as described earlier). With unbuffered channels, you can control the number of Goroutines that can run concurrently. For example, you might be making calls to an API and want to control how many calls are made per second. Otherwise, you may be stopped.

The Channel direction

An interesting feature of channels in Go is that when you use a channel as an argument to a function, you can specify whether the channel wants to send or receive data. As your program grows and you may use a large number of functions, it is a good idea to document the intent of each channel so that it can be used correctly. Or, you might want to write a library and expose a channel as read-only to maintain data consistency.

To define the direction of a channel, you can define it in a similar way to when reading or receiving data. But you do this when you declare a channel in a function argument. The syntax for defining a channel type as an argument in a function looks like this:

chan<- int // it's a channel to only send data
<-chan int // it's a channel to only receive data
Copy the code

Sending data through a receipt-only channel causes an error when compiling the program.

Let’s use the following program as an example of two functions, one for reading data and the other for sending data:

package main

import "fmt"

func send(ch chan<- string, message string) {
    fmt.Printf("Sending: %#v\n", message)
    ch <- message
}

func read(ch <-chan string) {
    fmt.Printf("Receiving: %#v\n", <-ch)
}

func main(a) {
    ch := make(chan string.1)
    send(ch, "Hello World!")
    read(ch)
}
Copy the code

When you run the program, you see the following output:

Sending: "Hello World!"
Receiving: "Hello World!"
Copy the code

The program clarifies the intent of each channel in each function. If you try to use a channel to send data in a channel that is only used to receive data, you will get a compilation error. For example, try something like this:

func read(ch <-chan string) {
    fmt.Printf("Receiving: %#v\n", <-ch
    ch <- "Bye!"
}
Copy the code

When you run the program, you will see the following error:

# command-line-arguments
./main.go:12:5: invalid operation: ch <- "Bye!" (send to receive-only type <-chan string)
Copy the code

A compilation error is better than a channel misuse.

multiplexing

Finally, let’s discuss a short topic on how to interact with multiple channels simultaneously using the SELECT keyword. Sometimes, when using multiple channels, you need to wait for events to occur. For example, when an exception occurs in the data your program is processing, you can include logic to cancel the operation.

The SELECT statement works like the Switch statement, but it applies to channels. It blocks the execution of the program until it receives an event to process. If it receives multiple events, it randomly selects one.

An important aspect of the SELECT statement is that it completes execution after the event is processed. If you want to wait for more events to occur, you may need to use loops.

Let’s use the following program to see select in action:

package main

import (
    "fmt"
    "time"
)

func process(ch chan string) {
    time.Sleep(3 * time.Second)
    ch <- "Done processing!"
}

func replicate(ch chan string) {
    time.Sleep(1 * time.Second)
    ch <- "Done replicating!"
}

func main(a) {
    ch1 := make(chan string)
    ch2 := make(chan string)
    go process(ch1)
    go replicate(ch2)

    for i := 0; i < 2; i++ {
        select {
        case process := <-ch1:
            fmt.Println(process)
        case replicate := <-ch2:
            fmt.Println(replicate)
        }
    }
}
Copy the code

When you run the program, you see the following output:

Done replicating!
Done processing!
Copy the code

Notice that the replicate function is done first. That’s why you see its output in the terminal first. The main function has a loop because the SELECT statement ends as soon as it receives the event, but we are still waiting for the process function to complete.

challenge

For this challenge, you need to improve your existing program by making it faster. Try writing your own programs, even if you have to go back to the examples you used to exercise. Then, compare your solution to the solution in the next section.

Concurrency in Go is a complex issue that you’ll understand better with practice. This challenge is just one suggestion you can put into practice.

Good luck!

Faster Fibonacci number calculation using concurrent method

Calculate the Fibonacci numbers in order using the following program:

package main

import (
    "fmt"
    "math/rand"
    "time"
)

func fib(number float64) float64 {
    x, y := 1.0.1.0
    for i := 0; i < int(number); i++ {
        x, y = y, x+y
    }

    r := rand.Intn(3)
    time.Sleep(time.Duration(r) * time.Second)

    return x
}

func main(a) {
    start := time.Now()

    for i := 1; i < 15; i++ {
        n := fib(float64(i))
    fmt.Printf("Fib(%v): %v\n", i, n)
    }

    elapsed := time.Since(start)
    fmt.Printf("Done! It took %v seconds! \n", elapsed.Seconds())
}
Copy the code

You need to build two programs based on existing code:

Implement an improved version of concurrency. It takes a few seconds (no more than 15 seconds) to do this, as it does now. Use a buffered channel.

Write a new version to calculate the Fibonacci number until the user enters quit in the terminal using the fmt.scanf () function. If the user presses Enter, the new Fibonacci number should be calculated. In other words, you will no longer have a loop from 1 to 10.

Use two unbuffered channels: one for calculating the Fibonacci number and one for waiting for the user’s “exit” message. You need to use the SELECT statement.

Here is an example of interacting with a program:

1

1

2

3

5

8

13
quit
Done calculating Fibonacci!
Done! It took 12.043196415 seconds!
Copy the code

The solution

Faster Fibonacci number calculation using concurrent method

An improved version that enables concurrency and makes the program run faster is shown below:

package main

import (
    "fmt"
    "math/rand"
    "time"
)

func fib(number float64, ch chan string) {
    x, y := 1.0.1.0
    for i := 0; i < int(number); i++ {
        x, y = y, x+y
    }

    r := rand.Intn(3)
    time.Sleep(time.Duration(r) * time.Second)

    ch <- fmt.Sprintf("Fib(%v): %v\n", number, x)
}

func main(a) {
    start := time.Now()

    size := 15
    ch := make(chan string, size)

    for i := 0; i < size; i++ {
        go fib(float64(i), ch)
    }

    for i := 0; i < size; i++ {
        fmt.Printf(<-ch)
    }

    elapsed := time.Since(start)
    fmt.Printf("Done! It took %v seconds! \n", elapsed.Seconds())
}
Copy the code

The second version of the program that uses two unbuffered channels looks like this:

package main

import (
    "fmt"
    "time"
)

var quit = make(chan bool)

func fib(c chan int) {
    x, y := 1.1

    for {
        select {
            case c <- x:
                x, y = y, x+y
            case <-quit:
                fmt.Println("Done calculating Fibonacci!")
            return}}}func main(a) {
    start := time.Now()

    command := ""
    data := make(chan int)

    go fib(data)

    for {
        num := <-data
        fmt.Println(num)
        fmt.Scanf("%s", &command)
        if command == "quit" {
            quit <- true
            break
        }
    }

    time.Sleep(1 * time.Second)

    elapsed := time.Since(start)
    fmt.Printf("Done! It took %v seconds! \n", elapsed.Seconds())
}
Copy the code

conclusion

As you can see, Go implements concurrency differently from other programming languages. Go’s tagline encapsulates this approach: “Not by communicating shared memory, but by communicating shared memory.”

That simple sentence changed everything. You’ve seen that by using Goroutine and Channel, you can write concurrent programs that run faster and are easy to understand (at least once you understand how some of the features in Go work).

We’ve just covered the basics of Go’s concurrent approach. But at least you’ve got some practice, especially for this challenge.

We strongly encourage you to revisit this module to make sure you understand the basics. Then, it’s time to explore further.

Make sure you understand why channels are needed to communicate in goroutine, and the difference between unbuffered and buffered channels, especially when used. I’ll leave concurrency here and see you in the next module.

The next section,

(8) Write and test programs

Article source :www.sdk.cn/details/PEa…