Go language bytes.Buffer source code details 2

You have to work really hard to look effortless!

Wechat search public number [long Coding road], together From Zero To Hero!

preface

The structure of the bytes.Buffer is divided into three sections: Read data, unread data, free part, forget the friends to review oh. Buffer exists, is for reading and writing services, so in this article we will learn how to read and write methods using buffer implementation!

Source code analysis

Write()

The Write method writes data from byte slice P to the buffer slice, returning the length of bytes written and the error generated

Because Write calls the grow method, a panic with ErrTooLarge occurs if the underlying buffer slice is too large to be redistributed

func (b *Buffer) Write(p []byte) (n int, err error) {
	b.lastRead = opInvalid

	Ensure that the underlying buffer slice can be written to len(p) bytes by calling tryGrowByReslice and grow.
	m, ok := b.tryGrowByReslice(len(p))
	if! ok { m = b.grow(len(p))
	}

	// Buf has len(p) free bytes from position m
	return copy(b.buf[m:], p), nil
}
Copy the code

WriteString()

The WriteString and Write methods are similar. The string s is written to the underlying buffer slice buf, returning the number of bytes successfully written and an error

func (b *Buffer) WriteString(s string) (n int, err error) {
	b.lastRead = opInvalid

	Ensure that the length of the underlying buffer slice can be written to len(s) bytes by calling tryGrowByReslice and grow.
	m, ok := b.tryGrowByReslice(len(s))
	if! ok { m = b.grow(len(s))
	}
	Buf has len(s) free bytes from position m, and copies s to the underlying buffer slice
	return copy(b.buf[m:], s), nil
}
Copy the code

WriteByte()

Similar to the Write method, Write a single byte instead of a byte slice

func (b *Buffer) WriteByte(c byte) error {
	b.lastRead = opInvalid

	Ensure that the underlying buffer slice can be written to 1 byte by calling the tryGrowByReslice and grow methods
	m, ok := b.tryGrowByReslice(1)
	if! ok { m = b.grow(1)}// m indicates the start position of the write after expansion. The value is directly assigned to the byte to be written
	b.buf[m] = c
	return nil
}
Copy the code

WriteRune()

Similar to the Write method, except that rune is written instead of byte slicing

func (b *Buffer) WriteRune(r rune) (n int, err error) {

	// if r < utf8.runeself, r is a byte, then call WriteByte directly
	if r < utf8.RuneSelf {
		b.WriteByte(byte(r))
		return 1.nil
	}
	b.lastRead = opInvalid

	// Ensure that the length of the underlying buffer slice can be written to UTf8.utfmax by calling tryGrowByReslice and grow
	m, ok := b.tryGrowByReslice(utf8.UTFMax)
	if! ok { m = b.grow(utf8.UTFMax) }Len (buf)+ utf8.utfmax. Utf8.utfmax is the maximum possible length of rune, but the current rune size may be smaller than this value
	// Call utf8.encoderune () to write rune r to buf, returning the number of bytes written
	n = utf8.EncodeRune(b.buf[m:m+utf8.UTFMax], r)

	// Update the length of buf because n<= utf8.utfmax
	b.buf = b.buf[:m+n]
	return n, nil
}
Copy the code

ReadFrom()

• The ReadFrom method reads data from Reader R and writes it to the underlying buffer slice buf, returning the number of bytes written and the error generated
• The grow method is called to increase the size of the buffer slice if the size of the buffer slice is insufficient during data reading and writing
• Read and write and this keeps going through the loop until it generates an error, and if it eventually generates an EOF error, that is, reader R reads the data at the end of the file, this method will eventually return nil error because it’s done

// The minimum free space left by the buffer slice
// This parameter is used by the ReadFrom method, which is the minimum amount of free space left by the buF when writing data from a Reader to the underlying buffer slice
const MinRead = 512


func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error) {
	b.lastRead = opInvalid

	// A for loop that reads and writes data until it encounters an EOF error or other error
	for {
    
		// Ensure that at least MinRead is set free and return the start position of the write I
		i := b.grow(MinRead)

		// The grow method changes the length to I +MinRead
		b.buf = b.buf[:i]

		// the space from I to the end of cap(buf) is all available for reader r to read and write
		// The Read method returns the number of bytes Read and the error generated. By definition, m should be processed first, then e
		m, e := r.Read(b.buf[i:cap(b.buf)])
		if m < 0 {
			panic(errNegativeRead)
		}

		// If m is greater than or equal to 0, data is being read and written to buF
		b.buf = b.buf[:i+m]
		// Number of read bytes n plus m
		n += int64(m)

		// If Reader r encounters an EOF error, it has completed reading data, error=nil
		if e == io.EOF {
			return n, nil // e is EOF, so return nil explicitly
		}

		// Another error is encountered, return error
		ife ! =nil {
			return n, e
		}
	}
}
Copy the code

The above describes the operations related to writing to the buffer. Next, we will look at the operations related to reading.

WriteTo()

The WriteTo method reads the data in the byte buffer slice and sends it to Writer W for consumption. Writer W finally returns the number of bytes consumed by Writer W and the error generated

func (b *Buffer) WriteTo(w io.Writer) (n int64, err error) {
	b.lastRead = opInvalid

	// nBytes: length of unread data
	if nBytes := b.Len(); nBytes > 0 {
		// If the length of unread data is greater than 0, the unread data from off to len() is written to Writer W
		// Write Returns the number of bytes consumed and the error generated
		m, e := w.Write(b.buf[b.off:])
		// If the consumed length is greater than the available length, it is not logical, panic
		if m > nBytes {
			panic("bytes.Buffer.WriteTo: invalid Write count")}// If m bytes are consumed, the read count is increased by m
		b.off += m

		// Consume bytes n = int64(m)
		n = int64(m)

		// If error is generated, return
		ife ! =nil {
			return n, e
		}

		// According to the IO.Writer interface definition of Write method, if the number of writes m! = nBytes, must return error! =nil
		// Hence e! =nil is absolutely true, it's going to return, so it's not going to get to this step, so it's kind of doing a double check
		ifm ! = nBytes {return n, io.ErrShortWrite
		}
	}

	// Call the Reset() method to Reset the unread data in the buffer slice
	b.Reset()
	return n, nil
}
Copy the code

Read()

The Read method reads data from the underlying buffer byte slice buf and writes it to byte slice P

// The number of bytes read by the method, and the error generated
func (b *Buffer) Read(p []byte) (n int, err error) {
	b.lastRead = opInvalid

  Error =nil if len(p)=0 and no data is readable in buf, otherwise error=EOF
	// If the unread data part is empty, there is no data to read
	if b.empty() {
		// First the byte buffer slice is reset
		b.Reset()
		Len (p)=0, error=nil
		if len(p) == 0 {
			return 0.nil
		}
		// Read less than len(p), return EOF error
		return 0, io.EOF
	}

	// If there is unread data, call copy, copy data from off to p, return the number of bytes copied
	n = copy(p, b.buf[b.off:])

	// Update the read count
	b.off += n

	// Update lastRead to opRead
	if n > 0 {
		b.lastRead = opRead
	}
	// Return the number of bytes read n, error=nil
	return n, nil
}
Copy the code

Next()

The Next method returns the first n bytes of unread data, or all unread data if the length is less than n bytes. Because the data returned by the method is a SLICE based on BUF, there is a risk of data leakage, and the data is valid until the next call to the Read or write methods, because the underlying data is modified.

func (b *Buffer) Next(n int) []byte {
	b.lastRead = opInvalid

	// m: length of unread data
	m := b.Len()

	// If the number of bytes required is greater than the length of unread data, then n=m
	if n > m {
		n = m
	}

	// Assign data to the required n bytes
	data := b.buf[b.off : b.off+n]

	// The read count is increased by n
	b.off += n
	if n > 0 {
		b.lastRead = opRead
	}
	return data
}
Copy the code

ReadByte()

Similar to the Read method, ReadByte reads a byte and returns the Read bytes and the error generated

func (b *Buffer) ReadByte(a) (byte, error) {

	// If there is no unread data, reset and return EOF error
	if b.empty() {
		// Buffer is empty, reset to recover space.
		b.Reset()
		return 0, io.EOF
	}

	// Reads a byte, then modifies the read count
	c := b.buf[b.off]
	b.off++
	b.lastRead = opRead
	return c, nil
}
Copy the code

ReadRune()

Like the Read method, ReadRune reads a UTF-8 encoded rune and returns the value, size, and error of rune

func (b *Buffer) ReadRune(a) (r rune, size int, err error) {
	// If there is no data to read, reset and return EOF error
	if b.empty() {
		b.Reset()
		return 0.0, io.EOF
	}

	// c stands for the first byte to be read
	c := b.buf[b.off]
	// If c < utf8.runeself, c is a one-byte rune, return the rune, add one to the read count
	if c < utf8.RuneSelf {
		b.off++
		b.lastRead = opReadRune1
		return rune(c), 1.nil
	}

	// From the read count position, call utf8.decoderune, which returns rune from the read count position and the corresponding number of bytes
	r, n := utf8.DecodeRune(b.buf[b.off:])

	// Change the read count
	b.off += n

	/ / modify lastRead
	b.lastRead = readOp(n)
	return r, n, nil
}
Copy the code

UnreadRune()

A rollback of a rune is valid only if the method is called after ReadRune. It is illegal to call the method after any other read method because the related read method records lastRead = opRead, not opReadRune*

func (b *Buffer) UnreadRune(a) error {

	// lastRead <= opInvalid, indicating that the last call was non-readRune and cannot be rolled back
	if b.lastRead <= opInvalid {
		return errors.New("bytes.Buffer: UnreadRune: previous operation was not a successful ReadRune")}/ / back to back
	if b.off >= int(b.lastRead) {
		b.off -= int(b.lastRead)
	}
	// The rollback can be performed only once
	b.lastRead = opInvalid
	return nil
}
Copy the code

UnreadByte()

Rollback a byte. This method is less demanding than UnreadRune, and any read-related method can rollback a byte

var errUnreadByte = errors.New("bytes.Buffer: UnreadByte: previous operation was not a successful read")

func (b *Buffer) UnreadByte(a) error {
	// only lastRead == opInvalid can be returned.
	if b.lastRead == opInvalid {
		return errUnreadByte
	}

	// Can be rolled back only once
	b.lastRead = opInvalid

	// The read count is reduced by one
	if b.off > 0 {
		b.off--
	}
	return nil
}
Copy the code

readSlice()

Private methods, readSlice reads unread data until it finds the delim character and stops, then returns the traversed data. The returned data is a reference based on the underlying buffer slice, and there is a risk of data leakage.

func (b *Buffer) readSlice(delim byte) (line []byte, err error) {

	// Call IndexByte(), starting at the off position, to find the first index where Delim appears, and return -1 if not found
	i := IndexByte(b.buf[b.off:], delim)

	// Since the index in the previous step started at 0, we need to add 1
	end := b.off + i + 1

	// All unread data needs to be returned, so end is the buffer array length and err is EOF
	if i < 0 {
		end = len(b.buf)
		err = io.EOF
	}

	// Returns the traversed data to update the read count
	line = b.buf[b.off:end]
	b.off = end

	// This operation is also opRead
	b.lastRead = opRead

	// Return data and error
	return line, err
}
Copy the code

ReadBytes()

ReadBytes iterates through the unread data until it encounters a delimiter with a byte value of Delim, then returns the traversed data (including the delimiter) and the resulting error.

ReadBytes directly calls readSlice, so only one case returns error! =nil, that is, in unread data, all unread data and an EOF error are returned if the separator is not found after traversing all data.

func (b *Buffer) ReadBytes(delim byte) (line []byte, err error) {
	// Call the readSlice() method directly, but it returns a reference to the underlying slice
	// The readSlice() method has modified the read count and lastRead
	slice, err := b.readSlice(delim)

	// Since readSlice() returns references, the data may have been modified by other method calls, so copy the data
	line = append(line, slice...)
	return line, err
}
Copy the code

ReadString()

ReadString is similar to ReadBytes except that it returns a string.

func (b *Buffer) ReadString(delim byte) (line string, err error) {
	// Call the readSlice() method directly, but it returns a reference to the underlying slice
	// The readSlice() method has modified the read count and lastRead
	slice, err := b.readSlice(delim)
	// Returns as a string
	return string(slice), err
}
Copy the code

NewBuffer()

• Instantiate a Buffer, using the passed byte slice buf as the underlying Buffer byte slice, or passing nil
• After initialization, the caller can no longer manipulate the passed byte slice BUf, otherwise data correctness will be affected
• The array of passed bytes is used to read the existing data in the slice, or to write the data in the write method, so the size of the passed byte slice should not be 0
• Bytes.buffer is out-of-the-box. In most cases, just new(Buffer) or declare a variable will do. There is no need to call the NewBuffer() method

func NewBuffer(buf []byte) *Buffer { return &Buffer{buf: buf} }
Copy the code

NewBufferString()

• NewBufferString is passed a string to initialize bytes.buffer, so the underlying byte Buffer slice has an initial value and data to read.
• Bytes.buffer is available out of the box. In most cases, just new(Buffer) or declare a variable will do, without calling the NewBufferString() method

func NewBufferString(s string) *Buffer {
	return &Buffer{buf: []byte(s)}
}
Copy the code

Use the sample

The following example uses bytes.buffer as a Buffer to complete a file copy operation.

func main(a) {
	var buffer bytes.Buffer
	srcFile, _ := os.OpenFile("test.txt", os.O_RDWR, 0666)
	n, err := buffer.ReadFrom(srcFile)
	fmt.Println(n, err)                     // 303190 <nil>
	fmt.Println(buffer.Len(), buffer.Cap()) / / 303190 523776

	targetFile, _ := os.OpenFile("target.txt", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666)
	n, err = buffer.WriteTo(targetFile)
	fmt.Println(n, err) 					          // 303190 <nil>
	fmt.Println(buffer.Len(), buffer.Cap()) / / 0, 523776
}
Copy the code

conclusion

In this article we learned about the source code implementation for reading and writing bytes.buffer. For write operations, ensure that there is sufficient free space before copying data to the unread part of the buffer. For read operations, the unread part of the data is copied out and the read count is updated.

As the saying goes, you can win a hundred battles with no danger of defeat. If you understand the principle of bytes.Buffer, you will be much more comfortable using it.

More and more

Personal blog: lifelmy.github. IO /

Wechat official account: Long Coding road