Why do Goroutines leaks exist and how can they be avoided?
In this day and age, software programs and applications are supposed to run fast and smoothly. Being able to run code in parallel is one of the capabilities that meet this need today. But be aware of the danger of leaks! We’ll discuss this with Jacob Walker from Ardan Labs, who wrote this blog post specifically for Golang Engineers.
Before we start reading our discussion of debugging Goroutines leaks, let’s briefly introduce a few principles that will give you a fuller understanding of the challenges this concept addresses.
Concurrent programming
Concurrent programming is a method of parallel coding that allows a collection of contiguous threads to run simultaneously. In this way, software programs can calculate faster and therefore perform better. The capabilities of today’s multi-core processors enable this kind of concurrent programming.
Goroutines:
The traditional threading approach is based on communication between threads using shared memory. Rather than specifically using locks to mediate access to shared data, Go facilitates the use of channels to move references to data between Goroutines. Thus, only one Goroutine can access data in a given period of time. Golang meets the need for concurrent programming with these Goroutines, which are basically lightweight threads controlled by Go at run time.
Let the cat out of the
But be careful! Goroutines can be stored slowly but reliably over time, thus wasting your memory and other resources without you even noticing it. Therefore, it is important to understand the risks of leaks and/or debug them early. This is a theme we discussed in a recent interview with Jacob Walker on the # Review-Interview channel at Gophers Community at Slack (40,000 + members). Read the interview below.
All in all, I’ve been working as a developer for about 10 years now, and I’ve been working on Go for the last four years almost completely. I started with version 1.3.
I work as a community engineer at Ardan Labs. My main activities are teaching Go Fundamentals and Ultimate Go. During my time there, I wrote a lot of blog posts, developed new content and helped community developers.
Sebastiaan (Interviewer) [4:16pm] :
Why have you been willing to Go for the past 4 years? Why do you like this language?
Jacob Walker [4:17pm] :
Lol yeah, I love it: Slightly_Smiling_face: My formal education was business (MBA). So when I got into development, I mostly “taught myself” using online resources. I started Web application development using HTML, JS, and PHP. This has helped me over the years, and I’ve been able to make cool things and solve interesting problems. After about 5 years of research and dabbling in other languages like Ruby and Python, I wanted to go deeper and get closer to the machine. I looked at C and C ++ and experimented there for a while, but I was excited by Go. Something about Go’s mind-set and philosophy really matches what I’m looking for and how I want to write code. What do I like about the language? There are many: slightly_smiling_face: – Simple language, inspiring code and patterns. – Consistency. Rules are applied very consistently throughout the language. – The tools are excellent. – It is generally accepted that there is usually “one right way” to do most things, so I don’t have to spend a lot of time guessing which of the dozen or so ways I should solve a problem, I usually prefer the first way.
Sebastiaan (Interviewer) [4:25pm]:
Since you (and Ardan Labs) are such a big contributor/user of Go, if you had to look back at this year, what would you say and what word would summarize your achievements?
Jacob Walker [4:26pm] :
We’ve seen explosive growth in the user base. The company’s training needs are increasing. It’s a very exciting time for the language.
Sebastiaan (Interviewer) [4:27pm]:
Ok, can you elaborate a little bit more on this “exciting” moment? What is particularly exciting about it?
Jacob Walker [4:30pm] :
Of course. It’s exciting for me to see all these new Gopher additions to the community! I love the language and it’s wonderful to share what you love with others. More and more people are beginning to understand the language and its ecosystem that experienced shepherds have long loved. Now all this is developing with positive momentum.
Sebastiaan (Interviewer) [4:32pm]:
What new features do you personally hope to release in 2019?
Jacob Walker [4:33pm] :
Well, that’s a good question. There has been a lot of discussion recently about design drafts for “Go 2” and “generics,” simplified error handling, error context, and so on. These kinds of discussions are interesting.
I don’t worry about the nuts and bolts of these new features because I know:
They still have a long way to go.
They may look quite different from existing designs when they are released, and I’ll probably be very happy to see the module issues consolidated in the near future.
Sebastiaan, Interviewer [4:38pm]:
So you wrote about the Goroutine leak. Why did you choose this topic? Is it because you’ve seen this problem so many times that it hasn’t been solved by newbie Gopher?
Jacob Walker [4:41pm] :
That’s right. Earlier, when you asked me about my love of the language, I didn’t list the reasons for concurrency. I absolutely love writing concurrent code, and Go has one of the best concurrency approaches I’ve ever seen.
The problem is that many new Gophers use it when they don’t have to, and the concurrency they use comes with some pitfalls. In my training sessions, when we reach concurrency, I alert my students to these pitfalls and how to avoid them. This information is useful to everyone in the community, so I want to turn it into a series of blog posts. The Goroutine leak you linked to earlier is the first in this series.
Sebastiaan, Interviewer [4:44pm]:
What pitfalls do you think developers are most likely to miss?
Jacob Walker [4:45 PM]
There are Goroutine leaks that are easily created by accident. I guess the most common one is Data Races. It also runs the risk of incomplete work. You can start Goroutine to do something, but it may not be able to complete when the program terminates, and then it will be cut off. So it’s hard to say which is the most common or the most dangerous. They all suck: Slightly_smiling_face:
Sebastiaan, Interviewer [4:49pm]:
Do you have any unique insights into these pitfalls that you can safely use concurrency in Go?
Jacob Walker [4:51 PM]
Of course. You can solve Goroutine leaks and incomplete work by remembering knowledge of similar proverbs never start Goroutine without knowing how to stop it. As far as I know, This sentence from Dave Cheney’s blog https://dave.cheney.net/2016/12/22/never-start-a-goroutine-without-knowing-how-it-will-stop. The community has a saying about data competition:
Do not communicate over shared memory, communicate over shared memory. It’s a bit succinct and poetic, so it may not make sense at first, but once you understand this proverb it will help you understand the memory. The key is to avoid sharing the same variables/memory on Goroutines. Instead, each variable should be maintained and managed by a Goroutine.
Sebastiaan, Interviewer [4:56pm]:
So, solving these potential problems is based on these basic principles. If you work in a team, are there some best practices in place to ensure that you always follow these principles?
Jacob Walker [4:58pm] :
What I can really say is code review. No one releases code into a production environment that hasn’t at least been thoroughly reviewed by others. Make sure everyone on the team knows these principles and knows to do so. For the first proverb, every time you see the go keyword, it makes you wonder “When will this Goroutine end?” .
Sebastiaan, Interviewer [5:01pm]:
In terms of security, what kind of problems do leaks cause?
Jacob Walker [5:03 PM]:
The only thing that comes to mind is to create a vector for denial-of-service attacks. If an attacker knows that making a request in a particular way will result in a Goroutine leak, they will do so, causing the server to run out of resources and crash.
Sebastiaan (Interviewer) [5:06pm]:
Are there additional resources available for security audits to enable you to deal with Goroutine leaks?
Jacob Walker [5:09 PM] :
Of course there is. It is best to review the code before problems occur, but once the code is in production or even in a staging or QA environment, you can use a monitoring tool such as Pprof to count the number of active Goroutines. If this number is always increasing and never decreasing, then you should think that there might be a leak somewhere.
Sebastiaan, Interviewer [5:10pm]:
Great advice. Let’s take a look at this link: https://golang.org/pkg/net/http/pprof/. Are there any other articles about Ardan LABS on the topic we discussed?
Jacob Walker [5:13pm] :
Bill recently completed a series of articles on the Goroutine scheduler mechanism. This is not leak-specific, but it provides good material for understanding how schedulers behave when they start using concurrency. https://www.ardanlabs.com/blog/2018/12/scheduling-in-go-part3.html
Sebastiaan, Interviewer [5:15pm]:
You’ve already said that your article is just the beginning of a series, so what discussion can we expect?
Jacob Walker [5:15pm] :
I’ve already posted my second post, Shows the Goroutine leak leakage is more examples of https://www.ardanlabs.com/blog/2018/12/goroutine-leaks-the-abandoned-receivers.htmlGoroutine Common causes of memory leaks in Go programs. In my previous article, I covered the Goroutine leak and provided many examples of common mistakes made by Go developers. Continuing this work, this article presents another scenario of how Goroutines might have been leaked. I may continue to use Goroutine leaks, or I may continue to use other concurrent traps. Data competition, incomplete work, and finally unnecessary complexity. Each topic requires at least one or more posts to discuss.
Sebastiaan, Interviewer [5:18pm]:
From all of these themes or challenges, what have you personally tackled that have been particularly tough?
Jacob Walker [5:21pm] :
Each of them I’ve encountered, they all have different levels of difficulty, so it’s hard to say which one is particularly tricky. Goroutine breaches and data contests are particularly tricky because you may not even know they’re happening.
Sebastiaan (Interviewer) [5:23pm]:
Ok, thank you for doing this. When is your next post on concurrency?
Jacob Walker [5:24 PM]:
Oh, now I have a deadline! : Slightly_SMILing_face: I’ll do it for a while over the next few weeks, but sometimes it may take weeks to review these posts to make sure they’re up to our standards. Maybe by the end of January? Thanks for the interview @Sebastiaan!
Originally published on Jexia.
via: https://medium.com/jexia/why-goroutines-leaks-exist-and-how-to-avoid-these-dfc572bdad08
Author: Jexia’s Editorial Team
This article is originally compiled by GCTT and published by Go Chinese
