How to convert []byte to io.reader in Go?

How to convert []byte to io.reader in Go?

I saw a problem on StackOverflow where the host made a network request and the interface returned [] bytes. What do I need to do if I want to convert it to IO.Reader?

This problem is not complicated to solve, and can be easily transformed in a few lines of code. Not only that, but it can be converted back in a few lines of code.

I’m going to go through two pieces of code.

[] byte turned IO. Reader

package main

import (
	"bytes"
	"fmt"
	"log"
)

func main(a) {
	data := []byte("Hello AlwaysBeta")

	// byte slice to bytes.Reader, which implements the io.Reader interface
	reader := bytes.NewReader(data)

	// read the data from reader
	buf := make([]byte.len(data))
	if_, err := reader.Read(buf); err ! =nil {
		log.Fatal(err)
	}

	fmt.Println(string(buf))
}
Copy the code

Output:

Hello AlwaysBeta
Copy the code

This code converts []byte data to reader, then reads from Reader, and prints the output.

IO. Reader turns byte []

package main

import (
	"bytes"
	"fmt"
	"strings"
)

func main(a) {
	ioReaderData := strings.NewReader("Hello AlwaysBeta")

	// creates a bytes.Buffer and read from io.Reader
	buf := &bytes.Buffer{}
	buf.ReadFrom(ioReaderData)

	// retrieve a byte slice from bytes.Buffer
	data := buf.Bytes()

	// only read the left bytes from 6
	fmt.Println(string(data[6:))}Copy the code

Output:

AlwaysBeta
Copy the code

This code creates a reader, then reads the data to the BUF, and finally prints it out.

The above two pieces of code are converted from []byte to IO.Reader. If you compare these two pieces of code, you’ll see NewReader in both of them. And they all play a key role in the transformation process.

So the question is, what is this NewReader? Let’s take a look at the source code.

The source code parsing

The IO package of Go provides the most basic IO interfaces, among which IO.Reader and IO.Writer are the most critical interfaces, and many native architectures are built around these two interfaces.

Here are the two interfaces:

Reader interface

IO.Reader represents a Reader that reads data from a resource into a transport buffer. In the buffer, data can be streamed and used.

The interface is defined as follows:

type Reader interface {
    Read(p []byte) (n int, err error)
}
Copy the code

The Read() method reads len(p) bytes into p. It returns the number of bytes read, n, and an error message when an error occurred.

Here’s an example:

package main

import (
	"fmt"
	"io"
	"os"
	"strings"
)

func main(a) {
	reader := strings.NewReader("Clear is better than clever")
	p := make([]byte.4)

	for {
		n, err := reader.Read(p)
		iferr ! =nil {
			if err == io.EOF {
				fmt.Println("EOF:", n)
				break
			}
			fmt.Println(err)
			os.Exit(1)
		}
		fmt.Println(n, string(p[:n]))
	}
}
Copy the code

Output:

4 Clea
4 r is
4  bet
4 ter
4 than
4  cle
3 ver
EOF: 0
Copy the code

This code reads data continuously from reader, four bytes at a time, and then prints it out to the end.

It is possible that the last returned n value is less than the buffer size.

Writer interface

IO.Writer represents a Writer that reads data from a buffer and writes it to a target resource.

type Writer interface {
   Write(p []byte) (n int, err error)
}
Copy the code

The Write method writes len(p) bytes from P to the object data stream. It returns the number of bytes written from p, n, and an error message if an error occurs.

Here’s an example:

package main

import (
	"bytes"
	"fmt"
	"os"
)

func main(a) {
	// Create Buffer temporary space and write a string to Buffer
	// Write using the Write method of IO.Writer
	var buf bytes.Buffer
	buf.Write([]byte("hello world , "))

	// Concatenate a string into Buffer with Fprintf
	fmt.Fprintf(&buf, " welcome to golang !")

	// Outputs the contents of the Buffer to the standard output device
	buf.WriteTo(os.Stdout)
}
Copy the code

Output:

hello world ,  welcome to golang !
Copy the code

Buffer is a structure type used to hold written data that implements the Write method of the IO.Writer interface.

WriteTo method definition:

func (b *Buffer) WriteTo(w io.Writer) (n int64, err error)
Copy the code

The first argument to the WriteTo method is the IO.Writer interface type.

Conversion principle

Back to the conversion problem at the beginning of this article.

IO.Reader is satisfied as long as an instance implements the method Read() in IO.Reader.

Both the bytes and strings packages implement the Read() method.

// src/bytes/reader.go

// NewReader returns a new Reader reading from b.
func NewReader(b []byte) *Reader { return &Reader{b, 0.- 1}}Copy the code

// src/strings/reader.go

// NewReader returns a new Reader reading from s.
// It is similar to bytes.NewBufferString but more efficient and read-only.
func NewReader(s string) *Reader { return &Reader{s, 0.- 1}}Copy the code

When NewReader is called, the corresponding t. Reader type is returned, which is extended by IO.Reader, so the conversion is implemented.

conclusion

In the development process, it is inevitable to do some IO operations, including printouts, file reading and writing, network connection, etc.

In the Go language, a series of standard libraries are provided to handle these operations, encapsulated in the following packages:

• io: Basic I/O operation interface.
• io/ioutil: encapsulates some useful IO functions.
• fmt: Implements I/O formatting.
• bufio: Implements buffered IO.
• net.Conn: Network read and write.
• os.Stdin.os.Stdout: System standard input and output.
• os.File: System file operation.
• bytes: Byte related I/O operations.

In addition to IO.Reader and IO.Writer, the IO package also encapsulates many other basic interfaces, such as ReaderAt, WriterAt, ReaderFrom, and WriterTo, which are not covered here. This part of the code is not complex, easy to read, but also to deepen the understanding of the interface, recommended.

Well, that’s all for this article. Follow me, take you through the problem read Go source.

Recommended reading:

• Start reading Go source code

Warmly recommend:

• Classic computer books (including download methods)
• Tech blog: Core backend tech blogs, including Python, Django, Docker, Go, Redis, ElasticSearch, Kafka, Linux, and more.
• Go programmers: Go learning roadmap, including basic column, advanced column, source reading, actual combat development, interview brush, required reading list and a series of resources.
• Interview questions: Python, Go, Redis, MySQL, Kafka, data structures, algorithms, programming, networking, etc.

Reference article:

• Books.studygolang.com/The-Golang-…
• www.cnblogs.com/jiujuan/p/1…
• Segmentfault.com/a/119000001…