The introduction

Golang’s proudest feature is the Goroutine, which is lightweight, requiring only 1KB per Go coroutine compared to 1MB per thread.

So, when you need to do concurrent processing, it’s natural to think, can I just go? Sample code is as follows

	for i:=0; i < 5; i++ {
		go func(index int) {
			fmt.Println(index)
		}(i) // Why should I be passed in?
	}
Copy the code

It is true that you can process requests concurrently, but what if one of the requests fails and you need to exit? On the one hand, you can implement this error handling yourself (I’ll write about it later), and on the other hand, you can use the golang official errGroup directly

Errorgoup is a good thing

The above sample code, if reimplemented with errGroup, would look like this

	g, _ := errgroup.WithContext(context.Background())

	for i := 0; i < 5; i++ {
	    index := i
		g.Go(func(a) error {
		    fmt.Println(index)
		    return nil // If you want to Mock out some errors, you can also return an error})}iferr = g.Wait(); err ! =nil {
		return err
	}
Copy the code

Isn’t that easy? If you’re interested, you can search the source code on your own. It’s only about 30 lines of code, minus the comments, and it’s pretty easy to understand.

Now that we have error handling, isn’t that perfect?

The problem depends on requests that need to be processed concurrently. Although coroutines are very lightweight, they still cost some resources. If you can expect hundreds or thousands of requests to be processed, you need a coroutine pool to reuse coroutines to save resources.

There are many implementations of coroutine pools on the Internet, most of which are large and comprehensive and consider many scenarios. However, in actual coding scenarios, it is likely to introduce a package just to solve a small problem, which is too heavy and may not be flexible enough.

Is there a simple template that you can copy/paste/tweak? It’s coming

A simple and practical template

Without gossip, go directly to the code first. Key parts will be annotated in the code to explain.

	var (
		err         error
		outputs     []int
		workers     = 4 // The number of coroutines can be set as required, generally not greater than runtime.numcpu ()
		workChannel = make(chan int)
		errChannel  = make(chan error, workers)
		wg          = &sync.WaitGroup{}
		mux         = sync.Mutex{}
	)
	
	worker := func(input int) (int, error){
	    retrun input, nil // If you want to Mock out some errors, you can also return an error
	}

	wg.Add(workers)
	for i := 0; i < workers; i++ {
		go func(a) {
			defer wg.Done()
			for input := range workChannel { // When the workChannel is closed, the loop exits
				output, err:=worker(input)
				iferr ! =nil {
					errChannel <- err
					break
				}

				mux.Lock() // Use lock to secure outputs to collect execution results, delete if not needed
				outputs = append(outputs, output) 
				mux.Unlock()
			}
		}()
	}

loop:
	for _, input := range inputs {
		select {
		case workChannel <- input:
		case err = <-errChannel:
			break loop
		}
	}

	close(workChannel) // Closing the workChannel allows the work coroutine to exit after processing the current task
	wg.Wait()

	// Select case (); // Select case ()
	if err == nil {
		select {
		case err = <-errChannel:
		default:}}return outputs, err
Copy the code

The code looks a bit too much, but in actual use, you can change the input and output types of worker functions as needed.

If you don’t want to copy/paste it, you can just paste it, but because golang has not yet officially released the template, you can only use inteface{}.

In other words, parallel_runner. Go, where the packaged code is needed, takes care of itself.

About the context

One might ask, why not use context.withcancel () and cancel if something goes wrong? If you’re a bit more careful, you should use it, but that also means that in worker functions you need to check ctx.done () as well. I don’t need to use it because I’m lazy…

Parallel_runner. Go = parallel_runner. Go = parallel_runner.

Write in the last

I’ve written before about the use of channels,

• How to make Golang’s channel as silky as NodeJS’s stream
• How to use Golang channel to achieve message batch processing
• How to play golang Channel out of async and await feel

The lightweight util implementation is open source in channelx, welcome to review, if you like to use the tool, please click a like or star 🙂