The problem

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"runtime"
)

func main(a) {
	num := 6
	for index := 0; index < num; index++ {
		resp, _ := http.Get("https://www.baidu.com")
		_, _ = ioutil.ReadAll(resp.Body)
	}
	fmt.Printf("Number of goroutine = %d\n", runtime.NumGoroutine())
}

Copy the code

The above problem is not implementedresp.Body.Close()In the case of leaks? If it leaks, how many leaksgoroutine?

How to answer

  • Don’t makeresp.Body.Close()The leakage is certain. But the leakgoroutineThe numbers confuse me. Due to the execution of6 times, one at a timeRead and write goroutine, it is12个goroutine, plusThe main functionIs itself agoroutineSo the answer is13.
  • However, when I run the program, I find that the answer is 3, which is a little bit different. Why?

explain

  • Let’s look directly at the source code.golang 的 httpThe package.
http.Get()

-- DefaultClient.Get
----func (c *Client) do(req *Request)
------func send(ireq *Request, rt RoundTripper, deadline time.Time)
-------- resp, didTimeout, err = send(req, c.transport(), deadline) 
/ / the code in the go / 1.12.7 / libexec/SRC/net/HTTP/client: 174

func (c *Client) transport(a) RoundTripper {
	ifc.Transport ! =nil {
		return c.Transport
	}
	return DefaultTransport
}
Copy the code
  • instructionshttp.GetUse the defaultDefaultTransportManage connections.
DefaultTransportWhat does it do?
// It establishes network connections as needed
// and caches them for reuse by subsequent calls.
Copy the code
  • DefaultTransportIs used to establish network connections as needed and cache them for reuse in subsequent calls.
thenDefaultTransportWhen will the connection be established?

Then scroll down the code stack above

func send(ireq *Request, rt RoundTripper, deadline time.Time) 
--resp, err = rt.RoundTrip(req) / / the code in the go / 1.12.7 / libexec/SRC/net/HTTP/client: 250
func (t *Transport) RoundTrip(req *http.Request)
func (t *Transport) roundTrip(req *Request)
func (t *Transport) getConn(treq *transportRequest, cm connectMethod)
func (t *Transport) dialConn(ctx context.Context, cm connectMethod) (*persistConn, error){...go pconn.readLoop()  // Start a read Goroutine
	go pconn.writeLoop() // Start a write goroutine
	return pconn, nil
}
Copy the code
  • Each time a connection is made, one is startedRead goroutineandWrite a goroutine. That’s why one timehttp.Get()Will leakTwo goroutineThe source of the.
  • The source of the leak is known, and it is known that it was not implementedclose
So why notcloseWill it leak?
  • Let’s go back to what we just startedRead goroutine 的 readLoop()In the code
func (pc *persistConn) readLoop(a) {
	alive := true
	for alive {
        ...
		// Before looping back to the top of this function and peeking on
		// the bufio.Reader, wait for the caller goroutine to finish
		// reading the response body. (or for cancelation or death)
		select {
		case bodyEOF := <-waitForBodyRead:
			pc.t.setReqCanceler(rc.req, nil) // before pc might return to idle poolalive = alive && bodyEOF && ! pc.sawEOF && pc.wroteRequest() && tryPutIdleConn(trace)if bodyEOF {
				eofc <- struct{}{}
			}
		case <-rc.req.Cancel:
			alive = false
			pc.t.CancelRequest(rc.req)
		case <-rc.req.Context().Done():
			alive = false
			pc.t.cancelRequest(rc.req, rc.req.Context().Err())
		case <-pc.closech:
			alive = false}... }}Copy the code
  • In simple termsreadLoopIt’s an infinite loop as long asalivefortrue.goroutineIt will always be there
  • selectIs in thegoroutine There may beExit scenario:
    • bodyBe read out orbodyShut down
    • requestTake the initiative tocancel
    • request 的 context Donestatetrue
    • The currentpersistConnShut down

Where the first body is read or the case is closed:

alive = alive && bodyEOF && ! pc.sawEOF && pc.wroteRequest() && tryPutIdleConn(trace)Copy the code

BodyEOF comes from a channel called waitForBodyRead. The true and false values of this field directly determine the value of the alive variable (alive=true that reads the Goroutine to stay alive and loop, or exit the Goroutine).

So where does the value of this channel come from?
/ / go / 1.12.7 / libexec/SRC /.net HTTP/transport/go: 1758
		body := &bodyEOFSignal{
			body: resp.Body,
			earlyCloseFn: func(a) error {
				waitForBodyRead <- false
				<-eofc // will be closed by deferred call at the end of the function
				return nil

			},
			fn: func(err error) error {
				isEOF := err == io.EOF
				waitForBodyRead <- isEOF
				if isEOF {
					<-eofc // see comment above eofc declaration
				} else iferr ! =nil {
					ifcerr := pc.canceled(); cerr ! =nil {
						return cerr
					}
				}
				return err
			},
		}
Copy the code
  • If you executeearlyCloseFn ，waitForBodyReadThe channel input isfalse.aliveIt will also befalsethatreadLoop()thisgoroutineI quit.
  • If you executefn, including under normal circumstancesbodyRead data throwio.EOFAt the time of thecase.waitForBodyReadThe channel input istruethataliveWill betrue, thenreadLoop()thisgoroutineI don’t quit, and I execute it incidentallytryPutIdleConn(trace) 。
// tryPutIdleConn adds pconn to the list of idle persistent connections awaiting
// a new request.
// If pconn is no longer needed or not in a good state, tryPutIdleConn returns
// an error explaining why it wasn't registered.
// tryPutIdleConn does not close pconn. Use putOrCloseIdleConn instead for that.
func (t *Transport) tryPutIdleConn(pconn *persistConn) error
Copy the code
  • tryPutIdleConn 将 pconnTo the list of idle persistent connections waiting for a new request, i.e., the connection will be reused.
So the question is, when will this be implementedfn 和 earlyCloseFn?
func (es *bodyEOFSignal) Close(a) error {
	es.mu.Lock()
	defer es.mu.Unlock()
	if es.closed {
		return nil
	}
	es.closed = true
	ifes.earlyCloseFn ! =nil&& es.rerr ! = io.EOF {return es.earlyCloseFn() // Perform earlyCloseFn when closed
	}
	err := es.body.Close()
	return es.condfn(err)
}
Copy the code
  • This is actually the one we’ve heard aboutresp.Body.Close()In which it will executeearlyCloseFnThat’s right therereadLoop()In thewaitForBodyReadThe channel input isfalse.aliveIt will also befalsethatreadLoop()thisgoroutineI would quit,goroutineIt won’t leak.
b, err = ioutil.ReadAll(resp.Body)
--func ReadAll(r io.Reader) 
----func readAll(r io.Reader, capacity int64) 
------func (b *Buffer) ReadFrom(r io.Reader)


/ / go / 1.12.7 / libexec/SRC/bytes/buffer. Go: 207
func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error) {
	for{... m, e := r.Read(b.buf[i:cap(b.buf)])  // Here, it is the body that executes the read method. }}Copy the code
  • thisreadWhich is essentiallybodyEOFSignalIn the
func (es *bodyEOFSignal) Read(p []byte) (n int, err error){... n, err = es.body.Read(p)iferr ! =nil{...// There will be an error in IO.EOF, which means that you have finished reading
		err = es.condfn(err)
	}
	return
}


func (es *bodyEOFSignal) condfn(err error) error {
	if es.fn == nil {
		return err
	}
	err = es.fn(err)  // This executes fn
	es.fn = nil
	return err
}
Copy the code
  • This is actually the read that we’re familiar withbodyThe contents of.ioutil.ReadAll()And after readingbodyIs executedfnThat’s right therereadLoop()In thewaitForBodyReadThe channel input istrue.aliveIt will also betruethatreadLoop()thisgoroutineI wouldn’t have quit,goroutineIt will leak and then executetryPutIdleConn(trace)Put the connection back in the pool for reuse.

conclusion

  • So the conclusion is close, although implemented6Loop, and it doesn’t execute each timeBody.Close()Because of the executionioutil.ReadAll()The content is read out, and the connection is reused, so only one is leakedRead goroutineAnd aWrite a goroutineAnd finally addmain goroutineSo the answer isThree goroutine.
  • On the other hand, normally our code will executeioutil.ReadAll()But if you forget at this pointresp.Body.Close(), does cause leaks. But if youThe domain name is always the sameThen only one will leakRead goroutineAnd aWrite a goroutine.Is that why the code is not standard but there is no obvious memory leak.
  • So the question is, why does it have to be the same domain name? Maybe some other time, maybe some other time.

Article recommendation:

  • You don’t even know that nil slices are different from empty slices? The BAT interviewer will have to let you go back and wait for notice.
  • Is the colleague who was in the append element in the for loop yesterday still there?
  • Interviewer Golang: For select, what happens if the channel is already closed? What if there’s only one case?
  • Interviewer Golang: For select, what happens if the channel is already closed? What if there’s only one case?
  • Golang interview question: What happens if I can read and write a chan that is already closed? Why is that?
  • Golang interview question: What happens if I read and write to an uninitialized chan? Why is that?
  • Golang Interview Question: How does Reflect get the field tag? Why can’t json packages export tags for private variables?
  • Golang: What happens if you don’t tag a VARIABLE in a JSON package?
  • Golang interview question: How to avoid memory escape?
  • Golang: What about memory escapes?
  • Golang Interview question: Does memory copy occur when a string is converted into a byte array?
  • Golang interview question: Flip a string containing Chinese, numbers, and English letters
  • Golang interview question: Is copying large slices more expensive than small slices?
  • Golang interview question: Can you tell the difference between uintptr and unsafe.pointer?
If you want to learn one knowledge point every day, pay attention to my [public] [public] [number] [Golang xiao Bai growth note].