The most exciting parts of Java are IO and NIO. IO is a bridge between Java programs and the outside world. IO refers to all classes in the Java. IO package, which has been in existence since Java 1.0. NIO is called new IO and is a new generation of IO introduced in java1.4.

What is the essence of IO? What’s the difference between NIO and it? How do we learn IO and NIO?

This series will use the perspective of young teachers and sisters to describe the process of learning Java IO in detail, I hope you can enjoy.

Little teacher sister who also? Name unknown, but diligence love learning, unlimited potential, take a look.

The example for this article github.com/ddean2009/l…

The article is too long, so you can download the PDF of this article directly: download link java-io-all-in-one. PDF

Chapter 1 The Nature of IO

The essence of the IO

The IO is used to read data from an external system into a Java program, or to write data output from a Java program back to an external system. The external systems here might be disks, network flows, and so on.

Since all external data processing is done by the operating system kernel, Java applications simply call the corresponding interface methods in the operating system to interact with external data.

The essence of all IO is the handling of buffers. We put data into buffers for the system to write to external data, or we read data from the system Buffer that has been read from external systems. As shown in the figure below:

There’s a Buffer for user space, which is our own Java program, and there’s a Buffer for system space. In this case, the system space will directly return the data in the Buffer to improve the reading speed.

DMA and virtual address space

Before we continue, let’s cover the basic concepts in both operating systems to help us understand IO later.

Modern operating systems have a component called Direct Memory Access (DMA). What does this component do?

In the absence of DMA, if a program performs IO operations, all CPU time will be taken up and the CPU will not be able to respond to other tasks and will have to wait for the IO to complete. This is unthinkable in modern applications.

If DMA is used, the CPU can transfer IO operations to another operating system component, such as the data manager, and only notify the CPU that the IO operation is complete when the data manager completes. Almost all modern operating systems implement DMA.

Virtual address space is also called Virtual address space. In order to isolate different programs from each other and ensure the certainty of addresses in programs, modern computer systems have introduced the concept of Virtual address space. The simplest way to think about this is to map the actual physical address to the virtual address space using a technique called segmenting or paging.

In the basic flowchart for IO above, we can map the system space buffer and user space buffer to the same place in the virtual address space at the same time. This eliminates the step of copying from system space to user space. It’s going to be faster.

In order to solve the problem that the virtual space is larger than the physical memory space, modern computer technology generally uses paging technology.

Paging is a technique that divides the virtual space into multiple pages, and only allocates physical memory maps for those pages when they are needed, so that physical memory can actually be treated as a cache of virtual space addresses.

The impact of virtual space address pagination on IO is that IO operations are also based on pages.

Common page sizes are 1,024, 2,048, and 4,096 bytes.

The classification of the IO

IO can be classified into File/Block IO and Stream I/O.

For File/Block IO, data is stored on a disk, which is managed by the Filesystem. Filesystem defines the name, path, and attributes of a file.

Filesystem manages data by dividing data into individual data blocks. Some blocks store metadata for files, and some blocks store real data.

Finally, filesystem also performs paging when processing data. The filesystem page size can be the same as the memory page size, or a multiple of it, such as 2,048 or 8,192 bytes.

Not all data is in the form of blocks, we have a class of IO called stream IO.

Stream IO is like a pipe stream in which data is consumed in a sequence.

The difference between IO and NIO

IO in java1.0 is Stream IO, which can only process data byte by byte, so it is also called Stream IO.

NIO is designed to improve the efficiency of IO. It reads data in blocks.

So in Stream IO, the input is a byte, and the output is a byte, and because it’s a Stream, you can add a filter or a chain of filters, think about the Filter chain in the Web framework. In Stream IO, data can only be processed once. You can’t rollback data in Stream.

In Block IO, data is processed as blocks, so it is faster than Stream IO and can be processed backwards. However, you need to handle the buffer yourself, so it is more complex than Stream IO.

Generally, Stream IO is a blocking IO. When a thread reads or writes, it blocks.

NIO is generally non-blocking, meaning that other operations can be done while a read or write is going on, and NIO is notified when the read or write is complete.

In IO, it is mainly divided into DataOutPut and DataInput, corresponding to OUT and in of IO respectively.

There are three types of DataOutPut: Writer, OutputStream, and ObjectOutput.

Take a look at the succession:

Datainputs also come in three categories: ObjectInput, InputStream, and Reader.

Look at their inheritance:

ObjectOutput and ObjectInput are less analogies, so they won’t be listed here.

Statistics about 20 classes about, make clear the use of these 20 classes, congratulations you Java IO you understand!

NIO is a bit more complicated. First of all, in order to process the block information, you need to read the data into the buffer, so buffer is a very important concept in NIO. Let’s look at NIO’s buffer:

From the figure above we can see that NIO has a variety of buffer types for us to use.

Another very important concept is a channel. A channel is the channel through which NIO gets data:

NIO has slightly more classes to master than IO, because NIO is a bit more complicated.

It’s exciting to think that we’ve mastered IO and NIO with just a few dozen classes.

conclusion

In the following articles, we will introduce the little sister to you to know, just she is also learning Java IO, the following learning with her, please look forward to it.

Chapter 2 Try with and its underlying principles

Introduction to the

My sister is a beginner in Java, and recently she is learning to use Java IO. As a senior teacher, I naturally want to give her the most powerful support. Read on to see what problems she encountered and how they were solved.

I/O shutdown problem

This day, the little sister asked me with a depressed face: F, I learn Java IO for many days, recently wrote an example, read a file is no problem, but read a lot of files will tell me: “Can’t open so many files”, Can you help me to see what is the problem?

For more, visit www.flydean.com

Of course, I can’t refuse the request of the little sister, I immediately response: may open too many files, teach you two commands, view the maximum file open limit.

One command is ulimit -a

The second command is

ulimit -n
256
Copy the code

It seems that your maximum file limit is too small. It is only 256 files.

The younger sister said: no, brother F, I read the documents one by one, not open so many documents at the same time yo.

Well, look at your code:

BufferedReader bufferedReader = null;
        try {
            String line;
            bufferedReader = new BufferedReader(new FileReader("trywith/src/main/resources/www.flydean.com"));
            while((line = bufferedReader.readLine()) ! =null) { log.info(line); }}catch (IOException e) {
            log.error(e.getMessage(), e);
        }
Copy the code

In the end, you should close your IO in the finally section.

This code will do the job:

BufferedReader bufferedReader = null;
        try {
            String line;
            bufferedReader = new BufferedReader(new FileReader("trywith/src/main/resources/www.flydean.com"));
            while((line = bufferedReader.readLine()) ! =null) { log.info(line); }}catch (IOException e) {
            log.error(e.getMessage(), e);
        } finally {
            try {
                if(bufferedReader ! =null){ bufferedReader.close(); }}catch(IOException ex) { log.error(ex.getMessage(), ex); }}Copy the code

Little teacher sister way a thank you, silently went to change the code.

Use try with resource

After half an hour, my little sister came to me again. Brother F, now every piece of code has to add finally manually, which is really too troublesome. Many times, I am afraid that I forget to close IO, which will cause unexpected exceptions in the program. You know I’m always afraid of trouble. Is there any simple way to solve this problem?

So what’s the JDK version you’re using?

Little teacher sister is embarrassed to say: although the latest JDK has reached 14, I still use JDK8.

If you put a resource in a try, the JVM will automatically close it for you. If you put a resource in a try, the JVM will automatically close it for you.

try (BufferedReader br = new BufferedReader(new FileReader("trywith/src/main/resources/www.flydean.com")))
        {
            String sCurrentLine;
            while((sCurrentLine = br.readLine()) ! =null) { log.info(sCurrentLine); }}catch (IOException e) {
            log.error(e.getMessage(), e);
        }
Copy the code

Try with Resource Principle

Great, little sister very happy, and then began to ask me: F, what is resource? Why don’t you put it in try instead of close?

A resource is a resource and can be turned on or off. We can call any class that implements the Java.lang. AutoCloseable interface a resource.

Let’s look at the definition of AutoCloseable:

public interface AutoCloseable {
        void close(a) throws Exception;
}
Copy the code

AutoCloseable defines a close() method. The JVM will automatically call the close() method when we have AutoCloseable resources open in the try with resource when the try block is executed.

Let’s take a look at the BufferedReader close method above:

public void close(a) throws IOException {
    synchronized (lock) {
        if (in == null)
            return;
        in.close();
        in = null;
        cb = null; }}Copy the code

Custom resource

We can implement AutoCloseable to create our own resource.

For example, let’s say I’m about to go to dinner. Let’s define a resource class:

public class CustResource implements AutoCloseable {

    public void helpSister(a){
        log.info("Help little schoolgirl solve the problem!");
    }

    @Override
    public void close(a) throws Exception {
        log.info("Solve the problem, hurry to eat!");
    }

    public static void main(String[] args) throws Exception {
       try( CustResource custResource= newCustResource()){ custResource.helpSister(); }}}Copy the code

Run the output:

[main] INFO com.flydean.custResource [main] INFO com.flydean.custResource - Solve the problem and go to dinner!Copy the code

conclusion

Finally, the problem was solved, and I could eat on time.

Chapter 3 File System

Introduction to the

This time the questions are related to file creation, file permissions and file systems. Fortunately, the answers to these questions are all in my head. Let’s take a look.

File permissions and file systems

As soon as I arrived at the company in the morning, the younger sister came over and asked me in secret: Brother F, I put some important documents on the server, which are very important. Is there any way to add a protection for it and give consideration to a little privacy?

For more, visit www.flydean.com

What document is so important? It’s not your picture, is it? No one will be interested.

The younger sister said: of course not, I want to put my learning experience on, but F you know, I just started learning, a lot of ideas are not mature, I want to keep a secret first, then open.

Seeing the younger sister so ambitious, I tears, my heart is very comforted. Let’s get started.

You know, there are two kinds of operating systems in the world, Windows and Linux (Unix) systems. The two systems are very different, but both systems have a concept of a file, and of course Linux has a much broader range of files, almost all resources can be considered files.

For every file, there are corresponding file systems, which are supported by the system kernel. We don’t need to reinvent the wheel in a Java program, but simply call the system’s kernel interface.

Brother F, I understand that. We don’t make wheels over and over again. We are just the porters of wheels. So how does Java call the system kernel to create files?

The most common way to create a File is to call the createNewFile method in class File. Let’s look at the implementation of this method:

public boolean createNewFile(a) throws IOException {
        SecurityManager security = System.getSecurityManager();
        if(security ! =null) security.checkWrite(path);
        if (isInvalid()) {
            throw new IOException("Invalid file path");
        }
        return fs.createFileExclusively(path);
    }
Copy the code

The method is internally security checked, and if it passes the security check, FileSystem’s createFileAutistic method is called to create the file.

In my MAC environment, FileSystem’s implementation class is UnixFileSystem:

public native boolean createFileExclusively(String path)
        throws IOException;
Copy the code

See? Createfileautistic in UnixFileSystem is a native method that calls the underlying system interface.

Wow, now that the file is created, we can assign permissions to the file, but do Windows and Linux have the same permissions?

A good question. Java code is cross-platform, and our code needs to execute on both Windows and Linux JVMS, so we have to find common ground in their permissions.

Let’s first look at permissions for Windows files:

As you can see, there are three types of permissions for a Windows file: modify, read, and execute. We will not consider special permissions because we need to find common ground between Windows and Linux.

Let’s look at the permissions for Linux files:

 ls -al www.flydean.com 
-rw-r--r--  1 flydean  staff  15 May 14 15:43 www.flydean.com
Copy the code

Above I used an ll command to list the details of the www.flydean.com file. The first column is the file permissions.

The basic file permissions in Linux are divided into three parts: owner, group, and Others. Each part has the same read, write, and execute permissions as in Windows, which are denoted by RWX respectively.

The owner can read and write the file www.flydean.com. The owner can read and write the file www.flydean.com. The owner can read and write the file www.flydean.com, but the owner can read and write the file www.flydean.com.

To make the file readable only to yourself, execute the following command:

chmod 600 www.flydean.com
Copy the code

Little sister immediately excited: F brother, I understand this, 6 in binary is 110,600 in binary is 110000000, just correspond to RW ——-.

I was very satisfied with my little sister’s comprehension.

File creation

Although we are not kong Yiji era, do not need to know the four ways to write hui, but a more knowledge and a way, do some sufficient preparation is very necessary.

Do you know how to create files in Java?

Little sister voice: F, I only know a new File method.

I stroked my beard with satisfaction, showing my superiority.

We talked before, there are three major categories, IO is a Reader/Writer, is a kind of InputStream/OutputStream, the last one is ObjectReader/ObjectWriter.

In addition to using the first new File, we could also use an OutputStream, and of course we would use the try with Resource feature that we talked about earlier, just to make our code a little bit cleaner.

Let’s look at the first way:

public void createFileWithFile(a) throws IOException {
        File file = new File("file/src/main/resources/www.flydean.com");
        //Create the file
        if (file.createNewFile()){
            log.info("Congratulations, the file was created.");
        }else{
            log.info("Sorry, file creation failed.");
        }
        //Write Content
        try(FileWriter writer = new FileWriter(file)){
            writer.write("www.flydean.com"); }}Copy the code

Look at the second way:

public  void createFileWithStream(a) throws IOException
    {
        String data = "www.flydean.com";
        try(FileOutputStream out = new FileOutputStream("file/src/main/resources/www.flydean.com")){ out.write(data.getBytes()); }}Copy the code

The second approach seems more brief than the first.

Wait, F, NIO has already appeared in JDK7, is it possible to use NIO to create files?

This is certainly not a difficult question for me:

public void createFileWithNIO(a)  throws IOException
    {
        String data = "www.flydean.com";
        Files.write(Paths.get("file/src/main/resources/www.flydean.com"), data.getBytes());

        List<String> lines = Arrays.asList("The program stuff."."www.flydean.com");
        Files.write(Paths.get("file/src/main/resources/www.flydean.com"),
                lines,
                StandardCharsets.UTF_8,
                StandardOpenOption.CREATE,
                StandardOpenOption.APPEND);
    }
Copy the code

NIO provides the Files utility class to write to a file. When writing to a file, we can take some parameters, such as character encoding, whether to replace the file or append to the end of the file.

Permissions to files in code

The little sister has a problem again: Brother F, he has talked for a long time, but he hasn’t told me about the permission.

Don’t worry, let’s talk about permissions now:

public void fileWithPromission(a) throws IOException {
        File file = File.createTempFile("file/src/main/resources/www.flydean.com"."");
        log.info("{}",file.exists());

        file.setExecutable(true);
        file.setReadable(true.true);
        file.setWritable(true);
        log.info("{}",file.canExecute());
        log.info("{}",file.canRead());
        log.info("{}",file.canWrite());

        Path path = Files.createTempFile("file/src/main/resources/www.flydean.com"."");
        log.info("{}",Files.exists(path));
        log.info("{}",Files.isReadable(path));
        log.info("{}",Files.isWritable(path));
        log.info("{}",Files.isExecutable(path));
    }
Copy the code

As we mentioned above, the JVM can only use the functions that both Windows and Linux have for general-purpose purposes, that is, only read and write permissions and execute permissions, because Windows can also distinguish between the user and other users, so the permissions of the user are reserved.

In the example above, we use the traditional File and NIO Files to update the File permissions.

conclusion

Okay, so much for file permissions.

Chapter four File reading those things

Introduction to the

Java IO has a reader and a stream that can be used to read a file. Today, brother F will answer the questions for her.

Characters and bytes

F: Last time, you said that there are two types of IO reading: Reader and InputStream. What is the difference between these two types? Why do I see some classes that are both Reader and Stream? Such as: InputStreamReader?

Little sister, do you know the philosopher’s final three questions? Who are you? From where? Where to?

Brother F, are you confused? I’m asking you Java, what philosophy are you talking about?

In fact, philosophy is the foundation of all knowledge. Do you know how to translate the principle of science into English? The philosophy of science

What do you think is the nature of the code in a computer? The essence of code is a long string of zeros and ones that combine to form computer code, which is the binary code that the JVM can recognize to run.

For more, visit www.flydean.com

F: Well, that’s a good idea, but what does it have to do with Reader and InputStream?

Let me ask you a question. What is the smallest unit of storage in Java?

Let me see, the smallest in Java should be Boolean, true and false exactly correspond to binary 1 and 0.

A Boolean is also the smallest unit of storage in Java, but it takes up one Byte of space. The smallest unit of storage in Java is actually a Byte. If you don’t believe me, use the JOL tool I introduced earlier to verify:

[main] INFO com.flydean.JolUsage - java.lang.Boolean object internals:
 OFFSET  SIZE      TYPE DESCRIPTION                               VALUE
      0    12           (object header)                           N/A
     12     1   boolean Boolean.value                             N/A
     13     3           (loss due to the next object alignment)
Instance size: 16 bytes
Space losses: 0 bytes internal + 3 bytes external = 3 bytes total
Copy the code

Here’s the boxed Boolean, and you can see that although the Boolean ends up taking 16bytes, the Boolean inside is only 1byte.

Byte translates to byte, which is the basic unit of storage in Java.

With bytes, we can interpret characters. Characters are made up of bytes, which can have one, two, or more bytes depending on how they are encoded. We human eyes can recognize the Chinese characters ah, English and everything can be regarded as characters.

While a Reader reads characters in a given encoding, an InputStream reads lower-level bytes directly.

If it’s a text file we can use Reader. If it’s not a text file we can use InputStream.

I can teach you, and my little sister is making rapid progress.

Read by character

The first is to use FileReader to read the File, but FileReader itself does not provide any method to read the data, if you want to really read the data, Again, we’re going to use the BufferedReader to connect to the FileReader, and the BufferedReader provides a cache for reading, one line at a time:

public void withFileReader(a) throws IOException {
        File file = new File("src/main/resources/www.flydean.com");

        try (FileReader fr = new FileReader(file); BufferedReader br = new BufferedReader(fr)) {
            String line;
            while((line = br.readLine()) ! =null) {
                if (line.contains("www.flydean.com")) { log.info(line); }}}}Copy the code

With files.lines, we get a stream. From the stream, we can read the file using a lambda expression. This is called the second method:

public void withStream(a) throws IOException {
        Path filePath = Paths.get("src/main/resources"."www.flydean.com");
        try (Stream<String> lines = Files.lines(filePath))
        {
            List<String> filteredLines = lines.filter(s -> s.contains("www.flydean.com")) .collect(Collectors.toList()); filteredLines.forEach(log::info); }}Copy the code

The third is not very common, but the elder brother also wants to teach you. One way to do this is with the tool class Scanner. Scanner can be used to split files with newlines, which is also good:

public void withScanner(a) throws FileNotFoundException {
        FileInputStream fin = new FileInputStream(new File("src/main/resources/www.flydean.com"));
        Scanner scanner = new Scanner(fin,"UTF-8").useDelimiter("\n");
        String theString = scanner.hasNext() ? scanner.next() : "";
        log.info(theString);
        scanner.close();
    }
Copy the code

The way it is read by byte

Little sister listened very satisfied, hurriedly urged me: F brother, the character read way I understand, read the byte quickly.

I nodded, little sister, the essence of philosophy still remember? Bytes are the essence of Java storage. To grasp the essence of all false.

Remember the Files utility class we talked about earlier? This utility class provides a number of methods related to file operations, including a method to read all bytes, here is to read all bytes at once! Use it with caution. Use it only in scenarios with few files.

public void readBytes(a) throws IOException {
        Path path = Paths.get("src/main/resources/www.flydean.com");
        byte[] data = Files.readAllBytes(path);
        log.info("{}",data);
    }
Copy the code

For larger files, you can use a FileInputStream to read a certain number of bytes at a time:

public void readWithStream(a) throws IOException {
        File file = new File("src/main/resources/www.flydean.com");
        byte[] bFile = new byte[(int) file.length()];
        try(FileInputStream fileInputStream  = new FileInputStream(file))
        {
            fileInputStream.read(bFile);
            for (int i = 0; i < bFile.length; i++) {
                log.info("{}",bFile[i]); }}}Copy the code

Stream reads are byte by byte, which is slow, so we use NIO’s FileChannel and ByteBuffer to speed things up a bit:

public void readWithBlock(a) throws IOException {
        try (RandomAccessFile aFile = new RandomAccessFile("src/main/resources/www.flydean.com"."r");
             FileChannel inChannel = aFile.getChannel();) {
            ByteBuffer buffer = ByteBuffer.allocate(1024);
            while (inChannel.read(buffer) > 0) {
                buffer.flip();
                for (int i = 0; i < buffer.limit(); i++) {
                    log.info("{}", buffer.get()); } buffer.clear(); }}}Copy the code

S: Is there a faster way to read very, very large files?

Sure, remember last time we talked about the mapping of the virtual address space:

We can map the user’s address space and the system’s address space into the same virtual address memory at the same time, thus eliminating the performance overhead of copying:

public void copyWithMap(a) throws IOException{
        try (RandomAccessFile aFile = new RandomAccessFile("src/main/resources/www.flydean.com"."r");
             FileChannel inChannel = aFile.getChannel()) {
             MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
             buffer.load();
            for (int i = 0; i < buffer.limit(); i++)
            {
                log.info("{}", buffer.get()); } buffer.clear(); }}Copy the code

Look for the number of rows that are wrong

Little sister: Great! F, you speak well, I still have a problem: recently, I am doing file parsing, some file format is not standard, parsing halfway failed, but there is no error message exactly which line is wrong, it is difficult to locate the problem, do you have any good solution?

LineNumberReader (LineNumberReader) : LineNumberReader (LineNumberReader) : LineNumberReader (LineNumberReader) : LineNumberReader (LineNumberReader) : LineNumberReader (LineNumberReader) : LineNumberReader

public void useLineNumberReader(a) throws IOException {
        try(LineNumberReader lineNumberReader = new LineNumberReader(new FileReader("src/main/resources/www.flydean.com")))
        {
            // Output the initial number of lines
            log.info("Line {}" , lineNumberReader.getLineNumber());
            // Reset the number of rows
            lineNumberReader.setLineNumber(2);
            // Get the number of existing rows
            log.info("Line {} ", lineNumberReader.getLineNumber());
            // Read all file contents
            String line = null;
            while((line = lineNumberReader.readLine()) ! =null)
            {
                log.info("Line {} is : {}", lineNumberReader.getLineNumber() , line); }}}Copy the code

conclusion

Today, I explained the character stream and byte stream, and also explained the basic method of file reading.

Chapter five documents those things

Introduction to the

The little sister also made a lot of strange demands for brother F, to format the output, to specific code output, to locate the output, what? Burn after reading? Let’s see how brother F takes the moves one by one.

Character output and byte output

Brother F, last time you talked about IO in the middle, the reading of the file is basically finished, but the writing of the file has not been talked about. When will you give me the science popularization?

Sister: Brother F, you know THAT I have always been a model of diligence and eagerness to learn. I am a good student in the eyes of my teachers, a good example in the hearts of my classmates, and a good boy around my parents. When I was climbing the peak of science, I found that there was still half of the knowledge I had not acquired. It really made me sigh. Brother F, please pass the knowledge to me as soon as possible.

Little sister your request, brother, I should try my best to do, but I remember the last time I told IO file read has passed several days, how today you just come to me.

Little sister blushs: Brother F, this is not the use of the time encountered some problems, just want to ask you to review the knowledge again.

Let’s go over the structure of the output class again:

These are the two systems for output: Writer and OutputStream.

Writer is for characters, while Stream is for Bytes.

The most common types of Writer are FileWriter and BufferedWriter. Let’s take a look at the next basic example of writing:

public void useBufferedWriter(a) throws IOException {
        String content = "www.flydean.com";
        File file = new File("src/main/resources/www.flydean.com");

        FileWriter fw = new FileWriter(file);
        try(BufferedWriter bw = newBufferedWriter(fw)){ bw.write(content); }}Copy the code

A BufferedWriter is the encapsulation of a FileWriter. It provides a buffer mechanism to improve the writing efficiency.

In fact, BufferedWriter provides three ways to write:

public void write(int c)
public void write(char cbuf[], int off, int len)
public void write(String s, int off, int len)
Copy the code

The first method passes in an int, the second method passes in an array of characters and the position and length to start reading, and the third method passes in a string and the position and length to start reading. Is it simple and completely understandable?

F: No, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no.

The underlying storage of int is bytes. The underlying storage of char and String is also bytes. We can do a cast of int and char. Let’s see how it works:

public void write(int c) throws IOException {
        synchronized (lock) {
            ensureOpen();
            if (nextChar >= nChars)
                flushBuffer();
            cb[nextChar++] = (char) c; }}Copy the code

Remember how many bytes an int takes? 4, char takes 2 bytes. In this case, the conversion from int to char will cause accuracy loss. Only the low 2 bytes of data will be retained, and the high 2 bytes of data will be discarded.

After reading Writer, let’s look at Stream:

public void useFileOutputStream(a) throws IOException {
        String str = "www.flydean.com";
        try(FileOutputStream outputStream = new FileOutputStream("src/main/resources/www.flydean.com");
            BufferedOutputStream bufferedOutputStream= new BufferedOutputStream(outputStream)){
            byte[] strToBytes = str.getBytes(); bufferedOutputStream.write(strToBytes); }}Copy the code

Just like Writer, BufferedOutputStream wraps a FileOutputStream. Let’s take a look at the write method provided by BufferedOutputStream:

public synchronized void write(int b)
public synchronized void write(byte b[], int off, int len)
Copy the code

Compared to Writer, BufferedOutputStream’s methods are synchronized, and BufferedOutputStream operates directly on bytes.

The first write method that passes an int is also intercepted, but this time it is converted from an int to a byte.

Formatted output

Println: Println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println: println

A PrintWriter is used to format output:

public void usePrintWriter(a) throws IOException {
        FileWriter fileWriter = new FileWriter("src/main/resources/www.flydean.com");
        try(PrintWriter printWriter = new PrintWriter(fileWriter)){
            printWriter.print("www.flydean.com");
            printWriter.printf("Program those things %s"."Very good."); }}Copy the code

Output other objects

We can output String, char, and Byte. What about Integer,Long, and other basic types?

Yes, you can do that with DataOutputStream:

public void useDataOutPutStream(a)
            throws IOException {
        String value = "www.flydean.com";
        try(FileOutputStream fos = new FileOutputStream("src/main/resources/www.flydean.com")){
            DataOutputStream outStream = new DataOutputStream(newBufferedOutputStream(fos)); outStream.writeUTF(value); }}Copy the code

DataOutputStream provides writeLong writeDouble, writeFloat method and so on, can also be writeUTF!

Writes to a specific location

Brother F, sometimes we don’t need to write to the file from the beginning every time, can we define where to write?

Just use RandomAccessFile:

public void useRandomAccess(a) throws IOException {
        try(RandomAccessFile writer = new RandomAccessFile("src/main/resources/www.flydean.com"."rw")){
            writer.seek(100);
            writer.writeInt(50); }}Copy the code

A RandomAccessFile can be located by seeking and then written from the specified location using the write method.

Lock the file

Brother F, there is one last question. How can I ensure that others will not overwrite what I have written when I write the document, and there will be no conflict?

FileChannel can call tryLock to obtain a FileLock lock from which we can control access to the file.

public void useFileLock(a)
            throws IOException {
        try(RandomAccessFile stream = new RandomAccessFile("src/main/resources/www.flydean.com"."rw");
        FileChannel channel = stream.getChannel()){
            FileLock lock = null;
            try {
                lock = channel.tryLock();
            } catch (final OverlappingFileLockException e) {
                stream.close();
                channel.close();
            }
            stream.writeChars("www.flydean.com"); lock.release(); }}Copy the code

conclusion

Today, I gave my little sister a variety of ways to write documents, enough for her to learn for a while.

Chapter 6 Directories or Files

Introduction to the

Directory and file silly distinction is not clear, what is the essence of directory and file? How to manipulate directories in Java, how to traverse directories. In this article, the F Brothers will tell you one by one.

Files and directories in Linux

Brother F, I have a question recently. Java code seems to have only files without directories. Is it the god who invented Java?

Brother F: What a brave little sister! Dare to question the authority is the most important step from a boy worker to an expert. Think of brother F, since childhood, no one mentioned anything, I believe what teachers say, I listen to what experts say: the stock market must be 10,000 points, the house is for people to live in, not to be fired, crude oil treasure is of course a small white financial management essential product…. And then, there was no and then.

For more, visit www.flydean.com

Although there is no directory concept in Java, there is only a File, which can actually represent a directory:

public boolean isDirectory(a)
Copy the code

There is an isDirectory method in the File to determine whether the File is a directory.

File and directory silly silly not clear, little sister, do you think of something?

Brother F, I remember you mentioned last time that all resources under Linux can be regarded as files. Is the essence of files and directories under Linux the same?

Right, under Linux, files are first class citizens, and all resources are identified as files.

We won’t go into the underlying structure of sectors, logical blocks, pages, etc. Let’s think about what a file should contain. In addition to the data of the file itself, there is a lot of metadata, such as file permissions, owner, group, creation time and so on.

On Linux systems, the two parts are stored separately. The data itself is called a block, and the metadata is called an inode.

The inode stores the address of the block. You can use the inode to find the address of the block in the actual data store of the file to access the file. Considering that large files can take up many blocks, a single inode can store the addresses of multiple blocks, whereas a file usually uses only one inode.

In order to show the hierarchical relationship and facilitate file management, the data file of the directory is stored in the directory and the inode address of the file, thus forming a chain relationship of one ring within one ring and one ring within one ring.

The figure above shows a circular layout that searches through directories to find the files under them.

I think the reason the Java directory does not have a separate class is probably because of the reference to Linux’s underlying file layout.

Basic directory operations

Since directories and files are public to the File class in Java, it will do all the basic operations on File directories.

Basically, there are three ways to pay more attention to directories than files:

public boolean isDirectory(a)
public File[] listFiles(a) 
public boolean mkdir(a) 
Copy the code

Why three categories? Because there are several methods that are close to them, I will not list them here.

IsDirectory Determines whether the file is a directory. ListFiles Lists all files in this directory. Mkdir Creates a file directory.

F: It takes a long time to iterate through the directory before. After you explain the data structure of the directory, I feel that listFiles is not a time-consuming operation. All the data is ready, just read it out directly.

Yes, do not look at the surface of the problem, to see the underlying content of the surface. You see, I usually do not show off, in fact, I am a real pillar of strength, can be called the company excellent employee model.

Little sister :F brother, that peacetime also did not see the top commendation you what? Oh, I understand, must be afraid of the boss praised you caused others envy, will let your good big brother image collapse, it seems that the boss really understand you.

Advanced operations on directories

Ok little sister, you understand the line, the following F brother to tell you about the directory of advanced operations, such as how we copy a directory ah?

Little sister, copy directory simple F, last time you taught me:

cp -rf
Copy the code

Wouldn’t a command have settled the matter? Is there a secret in there?

Cough cough, the secret is no, little sister, I remember you said last time to Java from one to the last, today brother to introduce you to a Java copy file directory method.

The Files utility class already provides a good way to copy Files:

public static Path copy(Path source, Path target, CopyOption... options)
Copy the code

Using this method, we can copy the file.

If you want to copy a directory, you will iterate through the files in the directory, calling the copy method in a circular way.

Wait a minute, brother F, if there is a directory under the directory, how to deal with the situation that there is also a directory under the directory?

This is the trick, and let me solve it recursively:

public void useCopyFolder(a) throws IOException {
        File sourceFolder = new File("src/main/resources/flydean-source");
        File destinationFolder = new File("src/main/resources/flydean-dest");
        copyFolder(sourceFolder, destinationFolder);
    }

    private static void copyFolder(File sourceFolder, File destinationFolder) throws IOException
    {
        // If it is a dir, then create dir recursively. If it is a file, then copy it directly
        if (sourceFolder.isDirectory())
        {
            // Check whether the target dir exists
            if(! destinationFolder.exists()) { destinationFolder.mkdir(); log.info("Target dir already created: {}",destinationFolder);
            }
            for (String file : sourceFolder.list())
            {
                File srcFile = new File(sourceFolder, file);
                File destFile = newFile(destinationFolder, file); copyFolder(srcFile, destFile); }}else
        {
            // Use files. copy to copy the specific file
            Files.copy(sourceFolder.toPath(), destinationFolder.toPath(), StandardCopyOption.REPLACE_EXISTING);
            log.info("Copy target file: {}",destinationFolder); }}Copy the code

The basic idea is that when I see a directory I go through it, and when I see a file I copy it.

Directory of backache operations

Brother F, what if I want to delete files in a directory, or we want to count the number of files in the directory?

Although this is a bit of a pain, it can be fixed. The Files utility class has a method called walk that returns a Stream object, and we can use the Stream API to process the file.

Delete file:

public void useFileWalkToDelete(a) throws IOException {
        Path dir = Paths.get("src/main/resources/flydean");
        Files.walk(dir)
                .sorted(Comparator.reverseOrder())
                .map(Path::toFile)
                .forEach(File::delete);
    }
Copy the code

Statistics file:

 public void useFileWalkToSumSize(a) throws IOException {

        Path folder = Paths.get("src/test/resources");
        long size = Files.walk(folder)
                .filter(p -> p.toFile().isFile())
                .mapToLong(p -> p.toFile().length())
                .sum();
        log.info("dir size is: {}",size);
    }
Copy the code

conclusion

This article has shown you some very common and useful operations for directories.

Chapter 7 File Systems and WatchService

Introduction to the

F introduced the WatchService introduced in JDK7 NIO, but he also popularized the concept of file system, which was unexpected.

Monitoring pain points

Brother F: Have you felt a little difficult to breathe, a little chilly in the back, and a little awkward to speak?

No, little sister, are you wearing your autumn clothes inside out?

No, brother F. What I’m talking about is the feeling in my heart, the pressure I don’t even have, and a little throbbing in my heart.

Don’t beat around the bush, schoolgirl. I think we have another problem.

For more, visit www.flydean.com

I know how to use the Properties file last time. Every time I modify the Properties file, I have to restart the Java application, which is really painful. Is there any other way?

Of course there are ways, the most basic way is to open a thread to monitor the last modification time of the properties file, if it is changed, then reload, so don’t.

Write thread ah, so troublesome, is there any easier way?

WatchService is a class that JDK7 introduced in NIO.

WatchService and file system

WatchService is an interface that JDK7 introduced in NIO:

The monitored service is called WatchService, and the monitored object is called Watchable:

WatchKey register(WatchService watcher, WatchEvent.Kind
       [] events, WatchEvent.Modifier... modifiers)
        throws IOException;
WatchKey register(WatchService watcher, WatchEvent.Kind
       ... events)
        throws IOException;
Copy the code

Watchable Registers the WatchEvent of the object with the WatchService through register. The WatchService is notified whenever a WatchEvent occurs on the Watchable object.

There are four types of WatchEvents:

1. ENTRY_CREATE The target is created
2. ENTRY_DELETE The target is deleted
3. ENTRY_MODIFY The target is modified
4. OVERFLOW a special Event that is discarded or lost

The WatchKeys returned by register are the collection of WatchEvents that were listened on.

Now look at the four WatchService methods:

1. Close close watchService
2. Poll gets the next watchKey, or null if none is found
3. The poll with the time parameter waits a certain amount of time to get the next watchKey
4. Take gets the next watchKey, or waits if none is available

F: How do you build a WatchService?

The FileSystem has a method for obtaining the WatchService:

public abstract WatchService newWatchService(a) throws IOException;
Copy the code

Take a look at the FileSystem structure:

On my MAC, filesystems fall into three broad categories: UnixFileSystem, JrtFileSystem, and ZipFileSystem. I’m guessing there’s a Windows-related file system on top of it. Little sister if you are interested in can go to see.

UnixFileSystem is used to process Unix files and ZipFileSystem is used to process ZIP files. So what does JrtFileSystem do?

Alas, this again want to pull far, why every time ask a question all want to pull to the horizon….

Back when the JDK was 9, there was a big change called modular JPMS (Java Platform Module System). The Jrt was designed for modular systems. Here’s an example:

public void useJRTFileSystem(a){
        String resource = "java/lang/Object.class";
        URL url = ClassLoader.getSystemResource(resource);
        log.info("{}",url);
    }
Copy the code

In this example, we get the url of the Object class. Let’s see what the output would be if we were in JDK8:

jar:file:/Library/Java/JavaVirtualMachines/jdk18.. 0 _171.jdk/Contents/Home/jre/lib/rt.jar! /java/lang/Object.classCopy the code

The output is jar:file, which means that the Object class is placed in a JAR file, followed by the path to the JAR file.

If it is after JDK9:

jrt:/java.base/java/lang/Object.class
Copy the code

The result starts with JRT, java.base is the name of the module, followed by the path to the Object. Does it seem more concise than the traditional JAR path?

With the file system, we can get the WatchService as well as the default file system:

WatchService watchService = FileSystems.getDefault().newWatchService();
Copy the code

The use and implementation nature of WatchSerice

Brother F, how did WatchSerice come about? It’s amazing. It saves us so much work.

In fact, the JDK provides so many classes so that we don’t have to reinvent the wheel. Did I tell you earlier that the easiest way to monitor files is to open a separate thread to monitor file changes? In fact,… That’s what WatchService does!

PollingWatchService() {
        // TBD: Make the number of threads configurable
        scheduledExecutor = Executors
            .newSingleThreadScheduledExecutor(new ThreadFactory() {
                 @Override
                 public Thread newThread(Runnable r) {
                     Thread t = new Thread(null, r, "FileSystemWatcher".0.false);
                     t.setDaemon(true);
                     return t;
                 }});
    }
Copy the code

WatchService is a daemon thread that can be used to receive monitoring tasks.

Let’s see how to register a file with WatchService:

private void startWatcher(String dirPath, String file) throws IOException {
        WatchService watchService = FileSystems.getDefault().newWatchService();
        Path path = Paths.get(dirPath);
        path.register(watchService, ENTRY_MODIFY);

        Runtime.getRuntime().addShutdownHook(new Thread(() -> {
            try {
                watchService.close();
            } catch(IOException e) { log.error(e.getMessage()); }})); WatchKey key =null;
        while (true) {
            try {
                key = watchService.take();
                for(WatchEvent<? > event : key.pollEvents()) {if(event.context().toString().equals(fileName)) { loadConfig(dirPath + file); }}boolean reset = key.reset();
                if(! reset) { log.info("This file cannot be reset");
                    break; }}catch(Exception e) { log.error(e.getMessage()); }}}Copy the code

The key method above is path.register, where path is a Watchable object.

Then use watchService.take to get the generated WatchEvent, and finally process the file based on the WatchEvent.

conclusion

Tao gives birth to one, life to two, two to three, and three to all things. A simple function is actually hidden behind… Tao Te Ching, oh no, behind it lies the philosophy of tao.

Chapter 8 File File and Path

Introduction to the

What does a file have to do with a path? What secrets do files and paths hide? What are the ways to create paths under the management of the file system? Today, brother F and younger sister will give you a wonderful performance again.

Files and Paths

F: I have a question. In Java, “File” is a class, because it contains a lot of other information, but why does “Path” have a separate class? Wouldn’t it be easier just to use a String?

For more, visit www.flydean.com

Everything has a reason. There is no such thing as love or hatred without any cause or cause. It’s all really wonderful.

Let’s look at the definitions of File and path:

public class File
   implements Serializable.Comparable<File>
Copy the code

public interface Path
    extends Comparable<Path>, 可迭代<Path>, Watchable
Copy the code

First, File is a class that represents the properties and functions that all File systems have. Whether you are Windows or Linux, the File object in them should be the same.

Path is an interface. Why is Path an interface? Because Path can be divided into JrtPath, UnixPath and ZipPath according to different cases. The filesystems-corresponding to the three paths we discussed in the previous article. So the implementation of Path is different, but the File containing Path is the same.

Brother F, why is this so difficult to pronounce, please give me a plain and simple explanation.

In that case, let me explain: patriotic version, we may belong to different nationalities, but we are all Chinese. Pop version, everyone is cultural, why you so drag. Culture edition, same nine years, you ho xiu?

Looking at their implementation interfaces, File implements Serializable, which means it can be serialized, and Comparable, which means it can be sorted.

Path inherits Comparable, meaning it can be sorted. Inheritably Iterable means that it can be traversed. It can be traversed because Path can represent directories. By inherits Watchable, it means that it can be registered with WatchService for monitoring.

Different paths in the file

Brother F, there are several get methods about Path in File. Could you tell us the differences between them?

Go directly to the code:

public void getFilePath(a) throws IOException {
        File file= new File(".. /.. /www.flydean.com.txt");
        log.info("name is : {}",file.getName());

        log.info("path is : {}",file.getPath());
        log.info("absolutePath is : {}",file.getAbsolutePath());
        log.info("canonicalPath is : {}",file.getCanonicalPath());
    }
Copy the code

There are three path-related methods in the File: getPath, getAbsolutePath, and getCanonicalPath.

GetPath returns the path that was passed in as new File, and returns what was entered.

GetAbsolutePath returns the absolute path, which is the current path preceded by getPath.

GetCanonicalPath returns the reduced AbsolutePath, which is removed. Or.. Or something like that.

Look at the output:

INFO com.flydean.FilePathUsage - name is : www.flydean.com.txt INFO com.flydean.FilePathUsage - path is : .. /.. /www.flydean.com.txt INFO com.flydean.FilePathUsage - absolutePath is : /Users/flydean/learn-java-io-nio/file-path/.. /.. /www.flydean.com.txt INFO com.flydean.FilePathUsage - canonicalPath is : /Users/flydean/www.flydean.com.txtCopy the code

Build different paths

Brother F, I remember paths have relative paths, absolute paths, etc., is there a corresponding method to create a Path?

Sure, let’s look at creating an absolute path:

public void getAbsolutePath(a){
        Path absolutePath = Paths.get("/data/flydean/learn-java-io-nio/file-path"."src/resource"."www.flydean.com.txt");
        log.info("absolutePath {}",absolutePath );
    }
Copy the code

We can build an absolute path by passing in the address of the absolute path using the Paths.get method.

Also using the paths. get method, passing in non-absolute Paths can build relative Paths.

public void getRelativePath(a){
        Path RelativePath = Paths.get("src"."resource"."www.flydean.com.txt");
        log.info("absolutePath {}",RelativePath.toAbsolutePath() );
    }
Copy the code

We can also build a Path from a URI:

public void getPathfromURI(a){
        URI uri = URI.create("file:///data/flydean/learn-java-io-nio/file-path/src/resource/www.flydean.com.txt");
        log.info("schema {}",uri.getScheme());
        log.info("default provider absolutePath {}",FileSystems.getDefault().provider().getPath(uri).toAbsolutePath().toString());
    }
Copy the code

You can also build a Path from FileSystem:

public void getPathWithFileSystem(a){
            Path path1 = FileSystems.getDefault().getPath(System.getProperty("user.home"), "flydean"."flydean.txt");
           log.info(path1.toAbsolutePath().toString());

            Path path2 = FileSystems.getDefault().getPath("/Users"."flydean"."flydean.txt");
            log.info(path2.toAbsolutePath().toString());

        }
Copy the code

conclusion

There are many ways to create a Path, and there is one that works for you. Come and choose.

Chapter 9 Buffers and Buffs

Introduction to the

The only thing that can help her as she continues to learn NIO is to give her full support when she needs it. Without saying anything, today’s introduction is NIO’s base Buffer. Give me a Buff, old iron.

What is the Buffer

< p style = “paddingbottom: 0px; paddingbottom: 0px;

Buffer is the foundation of NIO. Without Buffer, there would be no NIO. Without Buffer, there would be no Java today.

Since NIO reads data in blocks, this Block can be considered a Buffer. We store the data to be read and the data to be written in the Buffer to improve the efficiency of reading and writing.

For more, visit www.flydean.com

Remember what the underlying storage unit for Java objects is?

The underlying storage unit of Java objects is Byte.

Yes, let’s look at the Buffer inheritance diagram:

Buffer is an interface with a number of implementations, including the most basic ByteBuffer and other buffers encapsulated by other basic types.

Brother F, isn’t the ByteBuffer enough? What do you need other types of buffers for?

Little sister, shan Zhen no matter how good, there are too tired of eating, occasionally also want to change radish cabbage what, you think qianlong xijiang south all do what?

ByteBuffer is easy to use, but it is the smallest unit. We have Char, int, Double, Short, and other basic types on top of it. For simplicity, we have a set of buffers for all of them.

The Buffer into the order

Brother F, since Buffer is a collection of these basic types, why not express it as a combination? Encapsulating them as an object seems a bit redundant.

Since we’re in an object-oriented world, using objects makes sense on the surface, but at the bottom, these encapsulated buffers contain some additional metadata information and provide some unexpected functionality.

The figure above lists several key concepts in Buffer, namely Capacity, Limit, Position, and Mark. The underlying Buffer is an array. Let’s take a ByteBuffer as an example. The underlying Buffer is:

final byte[] hb; 
Copy the code

• Capacity indicates the maximum number of elements the Buffer can hold. This is set at the beginning of Buffer creation and cannot be changed.
• Limit indicates the number of elements in the Buffer that can be accessed.
• Position represents the index of the next element to be accessed and can be automatically updated with the PUT and GET methods.
• Mark represents the historical index. When we call the Mark method, we set Mark to the current position. We call the reset method to restore the value of Mark to position.

Create a Buffer

Brother F, is it difficult to create so many buffers? Is there a quick way to use it?

There are two ways to create buffers: allocate, and wrap.

public void createBuffer(a){
        IntBuffer intBuffer= IntBuffer.allocate(10);
        log.info("{}",intBuffer);
        log.info("{}",intBuffer.hasArray());
        int[] intArray=new int[10];
        IntBuffer intBuffer2= IntBuffer.wrap(intArray);
        log.info("{}",intBuffer2);
        IntBuffer intBuffer3= IntBuffer.wrap(intArray,2.5);
        log.info("{}",intBuffer3);
        intBuffer3.clear();
        log.info("{}",intBuffer3);
        log.info("{}",intBuffer3.hasArray());
    }
Copy the code

Allocate: allocate: allocate: allocate: allocate: allocate: allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate: Allocate;

INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=0 lim=10 cap=10]
INFO com.flydean.BufferUsage - true
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=0 lim=10 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=2 lim=7 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=0 lim=10 cap=10]
INFO com.flydean.BufferUsage - true
Copy the code

HasArray is used to determine whether the allocate level of the Buffer is implemented by an array. As you can see, both wrap and allocate are stored at the bottom of the Buffer.

One thing to note is that finally, we call the clear method, and after the clear method is called, we find that the Buffer position and limit have been reset. This means that wrap’s three-argument method sets only initial values and can be reset.

Direct VS non-Direct

F: You mentioned two ways to create a Buffer, but the background of both kinds of Buffer is array. Is there any non-array Buffer?

Naturally, there are, but only bytebuffers. ByteBuffer has an allocateDirect method that allocates a Direct Buffer.

What’s the difference between Direct and non-direct?

A Direct Buffer is used to operate in the virtual address mapping space without copying data in user space. This is called Direct. This has the advantage of being fast. The disadvantages are that it takes more resources to allocate and destroy, and because Direct buffers are not in user space, they are not covered by garbage collection.

Therefore, the Direct Buffer is usually used only for data with large data volume and long life cycle.

Look at the code:

public void createByteBuffer(a) throws IOException {
        ByteBuffer byteBuffer= ByteBuffer.allocateDirect(10);
        log.info("{}",byteBuffer);
        log.info("{}",byteBuffer.hasArray());
        log.info("{}",byteBuffer.isDirect());

        try (RandomAccessFile aFile = new RandomAccessFile("src/main/resources/www.flydean.com"."r");
             FileChannel inChannel = aFile.getChannel()) {
            MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
            log.info("{}",buffer);
            log.info("{}",buffer.hasArray());
            log.info("{}",buffer.isDirect()); }}Copy the code

In addition to allocateDirect, you can also get a Direct MappedByteBuffer using FileChannel’s map method.

The above example outputs:

INFO com.flydean.BufferUsage - java.nio.DirectByteBuffer[pos=0 lim=10 cap=10]
INFO com.flydean.BufferUsage - false
INFO com.flydean.BufferUsage - true
INFO com.flydean.BufferUsage - java.nio.DirectByteBufferR[pos=0 lim=0 cap=0]
INFO com.flydean.BufferUsage - false
INFO com.flydean.BufferUsage - true
Copy the code

Daily operation of Buffer

Brother F, it does seem that Buffer is a bit complicated. What are the operations of Buffer?

There are many Buffer operations, which we will explain in the following section.

Write data to Buffer

To write data to Buffer, call Buffer’s put method:

public void putBuffer(a){
        IntBuffer intBuffer= IntBuffer.allocate(10);
        intBuffer.put(1).put(2).put(3);
        log.info("{}",intBuffer.array());
        intBuffer.put(0.4);
        log.info("{}",intBuffer.array());
    }
Copy the code

Since the put method returns an IntBuffer class, the Buffer put method can be concatenated just like a Stream.

Also, we can specify where to put. The above code output:

INFO com.flydean.BufferUsage - [1.2.3.0.0.0.0.0.0.0]
INFO com.flydean.BufferUsage - [4.2.3.0.0.0.0.0.0.0]
Copy the code

Read data from Buffer

Read data using the get method, but before get we need to call the flip method.

What does the Flip method do? As mentioned above, buffers have a position and a limit field. Position automatically points to the next element with the get or PUT method, and the limit indicates how many elements are available in the Buffer.

If we were to read a Buffer we would start at positon and end at LIMIT:

public void getBuffer(a){
        IntBuffer intBuffer= IntBuffer.allocate(10);
        intBuffer.put(1).put(2).put(3);
        intBuffer.flip();
        while (intBuffer.hasRemaining()) {
            log.info("{}",intBuffer.get());
        }
        intBuffer.clear();
    }
Copy the code

You can tell whether there is another element by using hasRemaining. Clear the Buffer for the next time by calling clear.

rewind Buffer

Rewind is similar to flip, except that rewind doesn’t change the limit value, it just resets position to 0.

public void rewindBuffer(a){
        IntBuffer intBuffer= IntBuffer.allocate(10);
        intBuffer.put(1).put(2).put(3);
        log.info("{}",intBuffer);
        intBuffer.rewind();
        log.info("{}",intBuffer);
    }
Copy the code

The result output above is:

INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=3 lim=10 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=0 lim=10 cap=10]
Copy the code

Compact Buffer

Buffer also has a method called compact, which assigns the value from the current position to the limit of the Buffer to a position of 0:

public void useCompact(a){
        IntBuffer intBuffer= IntBuffer.allocate(10);
        intBuffer.put(1).put(2).put(3);
        intBuffer.flip();
        log.info("{}",intBuffer);
        intBuffer.get();
        intBuffer.compact();
        log.info("{}",intBuffer);
        log.info("{}",intBuffer.array());
    }
Copy the code

The above code outputs:

INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=0 lim=3 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=2 lim=10 cap=10]
INFO com.flydean.BufferUsage - [2.3.3.0.0.0.0.0.0.0]
Copy the code

duplicate Buffer

Finally, we’ll talk about copying buffers. There are three methods: Duplicate, asReadOnlyBuffer, and slice.

Duplicate is the position, limit and mark of the original Buffer. It shares the original data with the original Buffer. So if you modify the Buffer of duplicate, the original Buffer will also be modified.

If you use asReadOnlyBuffer, you are not allowed to modify the Buffer after the copy.

Slice is also readOnly, but it copies the Buffer from position to limit-position.

public void duplicateBuffer(a){
        IntBuffer intBuffer= IntBuffer.allocate(10);
        intBuffer.put(1).put(2).put(3);
        log.info("{}",intBuffer);
        IntBuffer duplicateBuffer=intBuffer.duplicate();
        log.info("{}",duplicateBuffer);
        IntBuffer readOnlyBuffer=intBuffer.asReadOnlyBuffer();
        log.info("{}",readOnlyBuffer);
        IntBuffer sliceBuffer=intBuffer.slice();
        log.info("{}",sliceBuffer);
    }
Copy the code

Output results:

INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=3 lim=10 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=3 lim=10 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBufferR[pos=3 lim=10 cap=10]
INFO com.flydean.BufferUsage - java.nio.HeapIntBuffer[pos=0 lim=7 cap=7]
Copy the code

conclusion

Today, I introduced the principle and basic operation of Buffer.

Chapter 10 File Copy and File Filter

Introduction to the

A Linux command of the thing, little sister must let me teach her how to use Java to achieve, ah, stand on a so fine little sister, I also feel powerless, do a brother really good difficult.

Copy files using Java

Today, my little sister found me: Brother F, can you tell me how to copy files?

Copy files? Isn’t that simple? If you have read access to the file, this is all you need.

cp www.flydean.com www.flydean.com.back
Copy the code

Of course, if it is a directory, you can also use two arguments to iterate and force copy:

cp -rf srcDir distDir
Copy the code

Such a simple Linux command, don’t tell me you can’t.

Little sister smiled: F brother, I don’t want to use Linux command, I want to use Java to achieve, I am not learning Java? Learning a course to find the right opportunity to practice, teach me quickly.

In that case, let me begin. There are three ways to copy files in Java. You can use the traditional method of reading and writing files, or you can use the latest NIO method of copying files.

The traditional method is not as fast as NIO, and it is not as simple as NIO. Let’s look at how to copy files using the traditional method of reading and writing files:

    public  void copyWithFileStreams(a) throws IOException
    {
        File fileToCopy = new File("src/main/resources/www.flydean.com");
        File newFile = new File("src/main/resources/www.flydean.com.back");
        newFile.createNewFile();
        try(FileOutputStream output = newFileOutputStream(newFile); FileInputStream input =new FileInputStream(fileToCopy)){
            byte[] buf = new byte[1024];
            int bytesRead;
            while ((bytesRead = input.read(buf)) > 0)
            {
                output.write(buf, 0, bytesRead); }}}Copy the code

In the example above, we first define two files, then generate an OutputStream and an InputStream from the two files, and finally copy the files by reading data from the input into the OutputStream as a byte stream.

Traditional File IO copying is tedious and slow. Let’s look at how to do this using NIO:

public  void copyWithNIOChannel(a) throws IOException
    {
        File fileToCopy = new File("src/main/resources/www.flydean.com");
        File newFile = new File("src/main/resources/www.flydean.com.back");

        try(FileInputStream inputStream = newFileInputStream(fileToCopy); FileOutputStream outputStream =new FileOutputStream(newFile)){
            FileChannel inChannel = inputStream.getChannel();
            FileChannel outChannel = outputStream.getChannel();
            inChannel.transferTo(0, fileToCopy.length(), outChannel); }}Copy the code

Before we talked about a very important concept in NIO is channel. By building channel channels of source and target files, we can copy directly at the channel level. As shown in the example above, we call inchannell. transferTo to do the copy.

Finally, there is an easier way to copy NIO files:

public  void copyWithNIOFiles(a) throws IOException
    {
        Path source = Paths.get("src/main/resources/www.flydean.com");
        Path destination = Paths.get("src/main/resources/www.flydean.com.back");
        Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING);
    }
Copy the code

Just use the copy method provided by the Files utility class.

Using the File filter

Brother F, I still have a need, that is, I want to delete a directory inside the log file ending with. Log, is this need very common? How does brother F usually operate it?

I usually use one Linux command to do this, but if not, use two:

rm -rf *.log
Copy the code

Of course, we can do it in Java if we need to.

Java provides two filters that can be used to do this.

The two filters are java.io.FilenameFilter and java.io.FileFilter:

@FunctionalInterface
public interface FilenameFilter {
    boolean accept(File dir, String name);
}
Copy the code

@FunctionalInterface
public interface FileFilter {
    boolean accept(File pathname);
}
Copy the code

Both of these interfaces are functional, so their implementations can be replaced directly with lambda expressions.

The difference is that FilenameFilter filters the file name and the directory in which the file is located. FileFilter filters directly to the target file.

In Java, there is no concept of a directory, and a directory is represented as a File.

The two above are very similar, so let’s use FilenameFilter as an example to see how to delete a.log file:

public void useFileNameFilter(a)
    {
        String targetDirectory = "src/main/resources/";
        File directory = new File(targetDirectory);

        //Filter out all log files
        String[] logFiles = directory.list( (dir, fileName)-> fileName.endsWith(".log"));

        //If no log file found; no need to go further
        if (logFiles.length == 0)
            return;

        //This code will delete all log files one by one
        for (String logfile : logFiles)
        {
            String tempLogFile = targetDirectory + File.separator + logfile;
            File fileDelete = new File(tempLogFile);
            boolean isdeleted = fileDelete.delete();
            log.info("file : {} is deleted : {} ", tempLogFile , isdeleted); }}Copy the code

In the example above, we implement the Filter effect by passing in the Filter created by the lambda expression through the directory.list method.

Finally, we delete the filtered files. The goal was achieved.

conclusion

Little sister two problems solved, I hope today can not see her again.

Chapter 11 The Magic of Channel in NIO

Introduction to the

Little sister, do you still remember our original intention of using IO and NIO?

Brother F, isn’t the use of IO and NIO to make life better and the world full of love? It allows programmers to gracefully move data from one place to another. If you sharpen your tools, you’ll have more time to enjoy your life.

Good, if you compare data to human beings, the purpose of IO and NIO is to ship people to the United States.

Brother F, why do you want to ship it to America? The CORONAVIRUS is too serious in America now. Let’s stay in China. China is the safest country in the world!

Well, to be on the safe side, we’ll ship people to Shanghai. People are data. How do you get there? You can take a plane, you can take a car, you can take a train, you can take a plane, you can take a car, you can take a train, you can take a Buffer.

Finally, the route of the plane, the road of the car and the track of the train can be regarded as a channel.

Simply put, a channel is the channel that carries the Buffer.

There are two types of IO: File IO from a File, Stream IO from a Stream. No matter what type of IO, data can be transported over a channel.

The classification of the Channel

Although there are only two sources of data, there are many types of channels in the JDK, as shown in the figure below:

Let’s start with the most basic and top-level interface, Channel:

public interface Channel extends Closeable {
    public boolean isOpen(a);
    public void close(a) throws IOException;

}
Copy the code

The top-level Channel is simple, inheriting the Closeable interface and implementing the isOpen and CLOSE methods.

One to determine whether a channel is open, and one to close the channel.

Brother F, how can the top Channel be so simple? It is totally inconsistent with the complex set of channels.

Don’t worry, it makes sense for the JDK to do this, because it’s a top-level interface, so it has to be more abstract and generic, and it turns out that there are really only two methods that are generic.

So to deal with this, there are a number of different types defined in a Channel.

The lowest level Channel has five types, which are:

FileChannel

Of these five channels, the one related to File is the FileChannel.

FileChannel can be obtained from a RandomAccessFile, FileInputStream, or FileOutputStream by calling getChannel().

You can also call the open method in FileChannel directly and pass in the Path creation.

public abstract class FileChannel
    extends AbstractInterruptibleChannel
    implements SeekableByteChannel.GatheringByteChannel.ScatteringByteChannel
Copy the code

Let’s look at the interfaces and classes that FileChannel inherits or implements.

AbstractInterruptibleChannel implements InterruptibleChannel interface, interrupt is known to all, used to interrupt thread execution tool. Consider the following very cryptic code:

protected final void begin(a) {
        if (interruptor == null) {
            interruptor = new Interruptible() {
                    public void interrupt(Thread target) {
                        synchronized (closeLock) {
                            if (closed)
                                return;
                            closed = true;
                            interrupted = target;
                            try {
                                AbstractInterruptibleChannel.this.implCloseChannel();
                            } catch (IOException x) { }
                        }
                    }};
        }
        blockedOn(interruptor);
        Thread me = Thread.currentThread();
        if (me.isInterrupted())
            interruptor.interrupt(me);
    }
Copy the code

The code above is AbstractInterruptibleChannel core.

First, we define an instance of Interruptible with an interrupt method that closes the Channel.

If it is Interrupted, it calls Interruptible’s interrupt method to turn off the current channel.

SeekableByteChannel is used to connect to an Entry or File. It has a unique property called position, which represents the current reading position. It can be modified.

GatheringByteChannel and ScatteringByteChannel indicate that one combination of Buffer sequences can be read and written at a time (Buffer Array) :

public long write(ByteBuffer[] srcs, int offset, int length)
        throws IOException;
public long read(ByteBuffer[] dsts, int offset, int length)
        throws IOException;
Copy the code

The Selector and the Channel

Before we talk about the other channels, let’s look at a Selector associated with the following channels:

This introduces a new Channel type called SelectableChannel. The previous FileChannel connection was one-to-one, that is, one Channel for each processing thread. SelectableChannel, on the other hand, is one-to-many, meaning that a single thread can process multiple channels using selectors.

SelectableChannel Registers different selectionkeys to monitor multiple channels. We’ll talk more about using selectors later on, so stay tuned.

DatagramChannel

DatagramChannel is a Channel used to process UDP. It comes with an Open method to create instances.

Let’s look at the definition of DatagramChannel:

public abstract class DatagramChannel
    extends AbstractSelectableChannel
    implements ByteChannel.ScatteringByteChannel.GatheringByteChannel.MulticastChannel
Copy the code

ByteChannel means it’s both ReadableByteChannel and WritableByteChannel, which can write and read at the same time.

MulticastChannel represents a multicast protocol. It corresponds exactly to UDP.

SocketChannel

A SocketChannel is a channel used to process TCP. It is also created using the Open method.

public abstract class SocketChannel
    extends AbstractSelectableChannel
    implements ByteChannel.ScatteringByteChannel.GatheringByteChannel.NetworkChannel
Copy the code

The only difference between a SocketChannel and a DatagramChannel is that it implements a NetworkChannel excuse.

NetworkChannel provides some network socket operations, such as address binding.

ServerSocketChannel

ServerSocketChannel is also a NetworkChannel and is primarily used as a listener on the server side.

public abstract class ServerSocketChannel
    extends AbstractSelectableChannel
    implements NetworkChannel
Copy the code

AsynchronousSocketChannel

The last AsynchronousSocketChannel is an asynchronous Channel:

public abstract class AsynchronousSocketChannel
    implements AsynchronousByteChannel.NetworkChannel
Copy the code

Why asynchronous? Let’s look at one method:

public abstract Future<Integer> read(ByteBuffer dst);
Copy the code

As you can see, the return value is a Future, so the read method can return immediately and only take the value from the Future if we need it.

The use of the Channel

Brother F, I am dazzled by so many kinds of channels. Can you give me a specific example of a Channel?

Ok, let’s look at an example of copying files using a Channel. Although a Channel provides a transferTo method that makes copying files very simple, let’s take a more general example to see the general use of channels:

public void useChannelCopy(a) throws IOException {
        FileInputStream input = new FileInputStream ("src/main/resources/www.flydean.com");
        FileOutputStream output = new FileOutputStream ("src/main/resources/www.flydean.com.txt");
        try(ReadableByteChannel source = input.getChannel(); WritableByteChannel dest = output.getChannel()){
            ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
            while(source.read(buffer) ! = -1)
            {
                // Flip the buffer, ready to write
                buffer.flip();
                // See if there is more
                while (buffer.hasRemaining())
                {
                    dest.write(buffer);
                }
                // Clear the buffer for next timebuffer.clear(); }}}Copy the code

In the example above we read the Buffer from the InputStream and then write it to the FileOutputStream.

conclusion

Today we’ve looked at specific categories of channels, and a simple example, and we’ll look at other examples of channels later. Stay tuned.

Chapter 12 MappedByteBuffer I can fit any size file

Introduction to the

Big big big, I want big! Little teacher sister to read more and more files, how to help her, let the program in performance and speed above balance? Come and have a look with Brother F.

Virtual address space

Small teacher sister: F teacher elder brother, have you found, recently the price of hard disk is really good cheap good cheap, 1T hard disk probably want 500 pieces, average 1M 50 cents. Now that the next movie starts at 1G, does this mean we have bought into the era of big data?

Yes, the progress of hardware technology also brings the progress of software technology. The two complement each other and cannot be absent without each other.

Brother F, if you want to read G files, is there any quick and easy way?

Remember last time we talked about the virtual address space?

Let’s take the picture from last time:

Typically, our application calls the system’s interface to get a Buffer from disk space. We call our application user space, and we call the underlying system system space.

In traditional IO operations, the operating system reads a file from the disk into the system space and then copies it to the user space for users to use.

There is an extra Buffer copy process, which can be a waste of time if the amount is large enough.

So some people are thinking, copy is too cumbersome and time-consuming, let’s separate a memory area, let the system space and user space simultaneously mapped to the same block of address is not to omit the copy step?

The separate area of memory that is delimited is called the virtual address space, and the mapping of the different Spaces to the virtual address is called a Buffer Map. Java has a special MappedByteBuffer to represent this operation.

Brother F, what’s the difference between the virtual address space and memory? What virtual address space do you need when you have memory?

Virtual address Spaces have two benefits.

The first benefit is that the virtual address space is independent of the application itself, thus ensuring program isolation and address determinism in the program. For example, if a program is run in the virtual address space, its spatial address is fixed, no matter how many times it is run. If the memory address is used directly, the memory address may be available on one run and not available on the next, resulting in a potential program error.

The second advantage is that the virtual address space can be bigger than real memory address, this is the use of memory has been optimized, such as the rarely used memory write such as disk, so as to release more memory to do more meaningful things, and stored in the disk data, when you really need to load from disk into memory.

In this way, physical memory can actually be viewed as a cache of virtual space addresses.

Rounding MappedByteBuffer

MappedByteBuffer sounds amazing. How to use it?

Let’s first look at the definition of MappedByteBuffer:

public abstract class MappedByteBuffer
    extends ByteBuffer
Copy the code

It is actually an abstract class with two concrete implementations:

class DirectByteBuffer extends MappedByteBuffer implements DirectBuffer
Copy the code

class DirectByteBufferR extends DirectByteBuffer
implements DirectBuffer
Copy the code

DirectByteBuffer and DirectByteBufferR.

Brother F, what’s the difference between these two bytebuffers? What does this R mean?

R stands for ReadOnly, probably because it’s a long enough name to be a class, so it’s abbreviated. But also do not write a note, let a person look very confusing….

We get an instance of RandomAccessFile by calling the map method from the FilChannel of the RandomAccessFile.

Let’s look at the definition of the map method:

 public abstract MappedByteBuffer map(MapMode mode, long position, long size)
        throws IOException;
Copy the code

MapMode indicates the mapping mode, position indicates the starting address of the map, and size indicates the size of the ByteBuffer.

MapMode

Brother F, there are two types of files: read only and read and write. Does MapMode also include these two types?

Right. Actually, MapMode in NIO has some other interesting uses besides these two.

• Filechannel.mapmode. READ_ONLY Indicates the read-only mode
• Filechannel.mapmode. READ_WRITE Indicates the read/write mode
• FileChannel. MapMode. PRIVATE said copy – on – the write mode, this mode is similar and READ_ONLY, its operation is to copy of the original data first, and then can read and write to and Buffer after copy. However, this write does not affect the original data. You can view it as a local copy of the data, so it’s called Private.

There are three basic mapmodes. In fact, in addition to the basic MapMode, there are two extended mapmodes:

• READ_ONLY_SYNC Synchronous read
• READ_WRITE_SYNC Synchronous read and write

The maximum value of MappedByteBuffer

M: Brother F, since the MappedByteBuffer can be mapped to the virtual memory space, is this MappedByteBuffer infinite?

Of course not. First of all, the size of the virtual address space is limited. On a 32-bit CPU, a pointer can take up 4 bytes of address, so the maximum that can be represented is 0xFFFFFFFF, which is 4G.

In Java, the maximum value that long can represent is 0x7FFFFFFf, which is 2147483647 bytes, or approximately 2G. In other words, the maximum MappedByteBuffer is 2G, and a maximum of 2G data can be mapped at a time.

The use of MappedByteBuffer

Let’s take two examples of reading and writing using MappedByteBuffer.

Good!

Let’s see how to read data using the MappedByteBuffer:

public void readWithMap(a) throws IOException {
        try (RandomAccessFile file = new RandomAccessFile(new File("src/main/resources/big.www.flydean.com"), "r"))
        {
            //get Channel
            FileChannel fileChannel = file.getChannel();
            //get mappedByteBuffer from fileChannel
            MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
            // check buffer
            log.info("is Loaded in physical memory: {}",buffer.isLoaded());  // This is just a reminder, not a guarantee
            log.info("capacity {}",buffer.capacity());
            //read the buffer
            for (int i = 0; i < buffer.limit(); i++)
            {
                log.info("get {}", buffer.get()); }}}Copy the code

Then let’s look at an example of using MappedByteBuffer to write data:

public void writeWithMap(a) throws IOException {
        try (RandomAccessFile file = new RandomAccessFile(new File("src/main/resources/big.www.flydean.com"), "rw"))
        {
            //get Channel
            FileChannel fileChannel = file.getChannel();
            //get mappedByteBuffer from fileChannel
            MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0.4096 * 8 );
            // check buffer
            log.info("is Loaded in physical memory: {}",buffer.isLoaded());  // This is just a reminder, not a guarantee
            log.info("capacity {}",buffer.capacity());
            //write the content
            buffer.put("www.flydean.com".getBytes()); }}Copy the code

MappedByteBuffer

F, MappedByteBuffer uses memory mapping, so the speed of reading and writing is improved. So what should we pay attention to in use?

MappedByteBuffer does not have a close method. Even if its FileChannel is closed, the MappedByteBuffer remains open and will only be closed when the JVM is garbage collecting. And that time is uncertain.

conclusion

Again, this article introduces the use of virtual address Spaces and mappedByteBuffers.

Chapter 13 Those strange Buffers in NIO

Introduction to the

Demons and ghosts appear quickly, today F brother to help the younger sister to cut the demon demon, what BufferB, BufferL, BufferRB, BufferRL, BufferS, BufferU, BufferRS, BufferRU all give you a clear clear.

The classification of the Buffer

Isn’t it said that JDK source code is the best Java teacher? For the process does not know the source code, called cattle also in vain. However, when I was learning NIO recently, I found that some Buffer classes were written without comments, which made me very upset.

For more, visit www.flydean.com

Is there such a thing? Take brother F to see it.

F: Look at ShortBuffer, for example, its subclasses are always followed by some strange characters:

What BufferB, BufferL BufferRB BufferRL, BufferS, BufferU, BufferRS, BufferRU came, point in their source also did not say whether these classes is what to do.

There is such a thing. Just give me an hour to study it.

An hour later, after an hour of hard investigation, I found that there was no official document about the meaning of these classes, but I figured it out, as if I had discovered the little secrets between these classes, and listened to my brother.

In the previous article, we said that buffers can be classified into ShortBuffer, LongBuffer, DoubleBuffer and so on according to the type.

But according to the nature and usage habits, we can be divided into three categories, they are: ByteBufferAsXXXBuffer, DirectXXXBuffer and HeapXXXBuffer.

ByteBufferAsXXXBuffer converts ByteBuffer into a specific type of Buffer, such as CharBuffer, IntBuffer, etc.

DirectXXXBuffer is a Buffer that works with virtual memory maps.

Finally, the HeapXXXBuffer is the Buffer created above the heap space.

Big Endian and Little Endian

Little sister, brother F, what you just said is not important, I just want to know what B, L, R, S, U are after the class.

All right, before I explain all this to you, let me tell you a story.

It is said that in the late Ming Dynasty, zhejiang talented woman Wu Jiangxue wrote a poem: “Poem of Spring Scenery”

Warbler crow bank willow lane spring clear, willow lane spring clear night moon. Bright moon night clear spring lane willow, clear spring lane willow bank crow warbler.

Little sister, can you see anything special? It’s best to read it several times, aloud.

Wow, brother F, this poem reads the same from beginning to end and from end to end, it is symmetrical and has artistic conception!

Yes, this is the charm of Chinese, depending on how you read it, you get different results, and this is a problem in the computer world as well.

We know that in Java the minimum storage unit at the bottom is a Byte. A Byte is 8bits, or ox00-oxff in hexadecimal notation.

In Java, it seems that all types take more than one byte, except byte, Boolean, which takes one byte.

If we take an int for example, an int takes 4 bytes, and it’s in the range Ox00000000-OxFFFFFFFF, and if we have an int=Ox12345678, there are two ways to store it in memory.

The first Big Endian stores the highest byte at the starting address

The second type, Little Endian, stores the status byte at the starting address

In fact, Big Endian is more in line with human reading and writing habits, while Little Endian is more in line with machine reading and writing habits.

At present, PowerPC series adopts big Endian method to store data, while x86 series adopts little Endian method to store data among the two main CPU camps.

If different CPU architectures communicate directly, problems may arise due to different read orders.

Java was designed to be written once and run everywhere, so naturally it was designed.

So BufferB is the Big Endian buffer, BufferL is the Little Endian buffer.

BufferRB and BufferRL are two types of read – only buffers.

Aligned Memory

Brother F, what are these for? BufferS, BufferU, BufferRS, BufferRU.

Before we look at these classes, let’s review how objects are stored in the JVM.

Remember how we used JOL to analyze information about the JVM? The code is very, very simple:

log.info("{}", VM.current().details());
Copy the code

Output results:

## Running 64-bit HotSpot VM.
## Using compressed oop with 3-bit shift.
## Using compressed klass with 3-bit shift.
## WARNING | Compressed references base/shifts are guessed by the experiment!
## WARNING | Therefore, computed addresses are just guesses, and ARE NOT RELIABLE.
## WARNING | Make sure to attach Serviceability Agent to get the reliable addresses.
## Objects are 8 bytes aligned.
## Field sizes by type: 4.1.1.2.2.4.4.8.8 [bytes]
## Array element sizes: 4.1.1.2.2.4.4.8.8 [bytes]
Copy the code

In the output above, we can see that Objects are 8 bytes aligned, which means that all Objects are allocated bytes that are multiple of 8.

And then notice the keyword output above – aligned, making sure that the eyes are aligned, that they are the right people.

Aligned means that objects in the JVM are aligned with 8 bytes. If the objects themselves take up less than 8 bytes of space or are not multiples of 8 bytes, they are complemented.

Again, we analyze a String using JOL:

[main] INFO com.flydean.JolUsage - java.lang.String object internals:
 OFFSET  SIZE      TYPE DESCRIPTION                               VALUE
      0    12           (object header)                           N/A
     12     4    byte[] String.value                              N/A
     16     4       int String.hash                               N/A
     20     1      byte String.coder                              N/A
     21     1   boolean String.hashIsZero                         N/A
     22     2           (loss due to the next object alignment)
Instance size: 24 bytes
Space losses: 0 bytes internal + 2 bytes external = 2 bytes total
Copy the code

You can see that a String takes 24 bytes, but what really matters is 22 bytes, with two 2-bytes to complement.

The advantage of alignment is that it is easier and faster for the CPU to read data, because the CPU sets how many bytes to read at a time. If you store unaligned data, the CPU will read less efficiently.

Now you can answer some questions: BufferU means unaligned, BufferRU means read-only unaligned.

S: What about BufferS and BufferRS?

In fact, this question is very difficult to answer, but through my continuous research and exploration, I finally found the answer:

The difference between DirectShortBufferRU and DirectShortBufferRS is in two places. Let’s look at the first Order:

DirectShortBufferRU:

public ByteOrder order(a) {
        return((ByteOrder.nativeOrder() ! = ByteOrder.BIG_ENDIAN) ? ByteOrder.LITTLE_ENDIAN : ByteOrder.BIG_ENDIAN); }Copy the code

DirectShortBufferRS:public ByteOrder order(a) {
        return ((ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN)
                ? ByteOrder.LITTLE_ENDIAN : ByteOrder.BIG_ENDIAN);
    }
Copy the code

You can see that the Order of DirectShortBufferRU is consistent with the nativeOrder. The Order of DirectShortBufferRS is opposite to that of nativeOrder.

Why the opposite? Look at the difference between the two get methods:

DirectShortBufferU:public short get(a) {
        try {
            checkSegment();
            return ((UNSAFE.getShort(ix(nextGetIndex()))));
        } finally {
            Reference.reachabilityFence(this); }}Copy the code

DirectShortBufferS:public short get(a) {
        try {
            checkSegment();
            return (Bits.swap(UNSAFE.getShort(ix(nextGetIndex()))));
        } finally {
            Reference.reachabilityFence(this); }}Copy the code

DirectShortBufferS do a swap of bits when it returns.

So BufferS means the Buffer after swap, and BufferRS means the Buffer after swap that is read only.

conclusion

Not writing the notes is really killing people! Especially if the JDK doesn’t comment it itself!

Chapter 14 Saying Goodbye in Selectors

Introduction to the

NIO has three treasures :Buffer,Channel, and Selector. This article will introduce the last set of selectors in NIO’s three-piece set, and assist schoolgirls to issue a nice guy card based on the understanding of selectors. Let’s get started.

The Selector is introduced

Brother F, my peach blossom has been flourishing recently. Several brothers say hello to me inexplicably, but I am so focused on my work that I don’t want to talk about these things. After all, a career comes before a family. I can’t just say no. Are there any subtle ways to get them to drop the idea?

For more, visit www.flydean.com

I pondered this question for about 0.001 seconds, then gave the answer: just send them a nice person card and they’ll probably leave you alone.

Sister: Brother F, what if the perfect person card is still useless?

Then we’re gonna have to cut them off, cut them off. Ha ha.

Well, aren’t you studying NIO recently? It just so happens that we can simulate the process of sending a nice guy card with a Selector.

If your zhiwei and Zidan brothers want to connect with you, everyone wants to have a channel with you, then you need to create two channels.

Two channels are fine, but if you have multiple people trying to connect to you at the same time, then maintaining those channels requires maintaining connections, which wastes resources.

But these connections are not always being sent, so most of the time these connections are wasted.

If you use a Selector, you can enable only one thread to listen for changes in the messages of the channel, and that’s what a Selector is.

As you can see from the figure above, the Selector listens to three different channels and then gives them to a processor, saving resources.

Create a Selector

Let’s look at the definition of selector:

public abstract class Selector implements Closeable
Copy the code

Selector is a abstract class that implements Closeable, indicating that selectors can be turned off.

Although a Selector is an abstract class, it can be created simply by opening:

Selector selector = Selector.open();
Copy the code

If you look closely at the implementation of Open, an interesting phenomenon emerges:

public static Selector open(a) throws IOException {
        return SelectorProvider.provider().openSelector();
    }
Copy the code

The open method calls the openSelector method on the SelectorProvider.

Look at the implementation of Provider:

 public SelectorProvider run(a) {
   if (loadProviderFromProperty())
        return provider;
    if (loadProviderAsService())
        return provider;
      provider = sun.nio.ch.DefaultSelectorProvider.create();
      returnprovider; }});Copy the code

There are three kinds of situations can load a SelectorProvider, if the system attribute specifies the Java nio. Channels. Spi. SelectorProvider, then loaded from the specified attribute.

If no property is specified directly, it is loaded from ServiceLoader.

Finally, if none is found, the default DefaultSelectorProvider is used.

The usage of ServiceLoader will be covered in an article later. I’m not going to explain it much here.

Register a Selector in a Channel

ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.bind(new InetSocketAddress("localhost".9527));
        serverSocketChannel.configureBlocking(false);
        serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
Copy the code

If we are on the Server side, we need to create a ServerSocketChannel, bind the Server’s address and port, and then set Blocking to false. Because we’re using Selector, it’s actually a non-blocking IO.

Note that FileChannels cannot use selectors because it is a blocking IO.

F: Why a FileChannel is blocked? Isn’t it faster to make it non-blocking?

Student: What are we using FileChannel for? It’s not possible to read one channel for a while and then read another channel, because each channel is busy by itself until the file is read, so there’s no need to switch between the channels.

And finally, we’re going to register the Selector that we’ve created into the channel.

SelectionKey

The SelectionKey represents the event that we want to listen to.

In general, there are four kinds of events:

• Selectionkey. OP_READ Indicates that the server is ready to read data from the channel.
• Selectionkey. OP_WRITE Indicates that the server is ready to write data to the channel.
• Selectionkey. OP_CONNECT Indicates that the client attempts to connect to the server
• Selectionkey. OP_ACCEPT Indicates that the server accepts a request from a client

public static final int OP_READ = 1 << 0;
public static final int OP_WRITE = 1 << 2;
public static final int OP_CONNECT = 1 << 3;
public static final int OP_ACCEPT = 1 << 4;
Copy the code

We can see that the above four events are defined by bit operation. If the four events are combined with or operation, the interestOps in SelectionKey can be obtained.

Similar to interestOps, SelectionKey also has a readyOps.

One for interested and one for ready.

Finally, when the SelectionKey is registered, we can attach an Object. For example, we can attach the id of the channel to the Object:

SelectionKey key = channel.register(
  selector, SelectionKey.OP_ACCEPT, object);
key.attach(Object);
Object object = key.attachment();
Copy the code

Object can be passed in register, or attach method can be called.

Finally, we can get this object by using the attachment method of key.

The selector and SelectionKey

We get a ready channel by using selector. Select (), which is a blocking operation.

Then we get the SelectionKey object by calling selection.selectedKeys ().

In the SelectionKey object, we process the message by determining that the event is ready.

The general case

Next, let’s set up a ChatServer:

public class ChatServer {

    private static String BYE_BYE="Goodbye";

    public static void main(String[] args) throws IOException, InterruptedException {
        Selector selector = Selector.open();
        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.bind(new InetSocketAddress("localhost".9527));
        serverSocketChannel.configureBlocking(false);
        serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
        ByteBuffer byteBuffer = ByteBuffer.allocate(512);

        while (true) {
            selector.select();
            Set<SelectionKey> selectedKeys = selector.selectedKeys();
            Iterator<SelectionKey> iter = selectedKeys.iterator();
            while (iter.hasNext()) {
                SelectionKey selectionKey = iter.next();
                if (selectionKey.isAcceptable()) {
                    register(selector, serverSocketChannel);
                }
                if (selectionKey.isReadable()) {
                    serverResonse(byteBuffer, selectionKey);
                }
                iter.remove();
            }
            Thread.sleep(1000); }}private static void serverResonse(ByteBuffer byteBuffer, SelectionKey selectionKey)
            throws IOException {
        SocketChannel socketChannel = (SocketChannel) selectionKey.channel();
        socketChannel.read(byteBuffer);
        byteBuffer.flip();
        byte[] bytes= new byte[byteBuffer.limit()];
        byteBuffer.get(bytes);
        log.info(new String(bytes).trim());
        if(new String(bytes).trim().equals(BYE_BYE)){
            log.info("Better goodbye than no see!);
            socketChannel.write(ByteBuffer.wrap("Goodbye".getBytes()));
            socketChannel.close();
        }else {
            socketChannel.write(ByteBuffer.wrap("You're a good man.".getBytes()));
        }
        byteBuffer.clear();
    }

    private static void register(Selector selector, ServerSocketChannel serverSocketChannel)
            throws IOException {
        SocketChannel socketChannel = serverSocketChannel.accept();
        socketChannel.configureBlocking(false); socketChannel.register(selector, SelectionKey.OP_READ); }}Copy the code

Example above two points need to pay attention to, we, in our loop through. When selectionKey isAcceptable, said the server received a new client connection, this time we need to call the register method, Register another OP_READ event to the new SocketChannel and continue the traversal.

Second, we define a stop Word that closes the client channel when it is received.

Take a look at the client code:

public class ChatClient {

    private static SocketChannel socketChannel;
    private static ByteBuffer byteBuffer;

    public static void main(String[] args) throws IOException {

        ChatClient chatClient = new ChatClient();
        String response = chatClient.sendMessage("Hello!);
        log.info("response is {}", response);
        response = chatClient.sendMessage("Can't you?");
        log.info("response is {}", response);
        chatClient.stop();

    }

    public void stop(a) throws IOException {
        socketChannel.close();
        byteBuffer = null;
    }

    public ChatClient(a) throws IOException {
        socketChannel = SocketChannel.open(new InetSocketAddress("localhost".9527));
        byteBuffer = ByteBuffer.allocate(512);
    }

    public String sendMessage(String msg) throws IOException {
        byteBuffer = ByteBuffer.wrap(msg.getBytes());
        String response = null;
        socketChannel.write(byteBuffer);
        byteBuffer.clear();
        socketChannel.read(byteBuffer);
        byteBuffer.flip();
        byte[] bytes= new byte[byteBuffer.limit()];
        byteBuffer.get(bytes);
        response =new String(bytes).trim();
        byteBuffer.clear();
        returnresponse; }}Copy the code

There is nothing special about the client code, except the Buffer read.

The final output result:

Server received: INFO com.flydean.ChatServer - Hello! Client received: INFO com.flydean.ChatClient - response is you are a good person Server received: INFO com.flydean.ChatServer - can you? Client received: INFO com.flydean. Chatclient-response is goodbyeCopy the code

Explain the whole process: Zhiwei establishes a connection with the younger sister, Zhiwei says hello to the younger sister, and the younger sister sends a nice guy card to Zhiwei. Zhiwei does not give up, want to continue to entwine, little sister replied goodbye, and then their own closed the channel.

conclusion

This article describes the role of selectors and channels in issuing good cards.

Chapter 15 File encoding and Character set Unicode

Introduction to the

On a whim, the little teacher sister used a new skill that had never been used, but there was a problem that could not be solved. How many steps does it take to put an elephant in the fridge? How is the problem of garbled code solved? Come and have a look with Brother F.

Use Properties to read the file

This day, the little sister mood is very happy, whistling and singing, the standard 45 degrees overlooking people feel uncomfortable.

Little sister ah, what things so happy, say let the elder brother also touch a little festival?

Brother F, I found a new way to read files. It’s very easy to use, just like map.

public void usePropertiesFile(a) throws IOException {
        Properties configProp = new Properties();
        InputStream in = this.getClass().getClassLoader().getResourceAsStream("www.flydean.com.properties");
        configProp.load(in);
        log.info(configProp.getProperty("name"));
        configProp.setProperty("name"."www.flydean.com");
        log.info(configProp.getProperty("name"));
    }
Copy the code

The contents of the file are in the form of key=value, which is very appropriate when using configuration files. I took inspiration from the Properties configuration file in the Spring project and discovered that Java also has a class called Properties that reads properties files.

Now the younger sister will rush to answer, as expected, the student excelled in blue.

The code first

Little teacher sister you do very well, so by analogy, soon Java will be all your hands, what Scala, go, JS and so on behind the estimate is also all no matter. In a few years, you’ll be promoted to architect, and the company’s technology will thrive under your leadership.

As a senior brother, the biggest responsibility is to give young sister to encourage and confidence, to describe a better future for her, what to become a CEO, to win the high, rich, handsome and so on are all no matter. I heard that there is a word to describe this process: draw a pie.

Little sister a little guilty: but F brother, I still have a little problem not solved, a little Chinese garbled….

I nodded my head deeply: Mosaic is a stumbling block to human progress… Oh, not Mosaic, is file garbled, to figure out this problem, but also from the character set and file coding.

Character set and file encoding

Long, long ago, when I was not even born, a high-tech product called computer appeared in the Western world.

The first computers could only do simple numbers and were run using programs that punched holes. But as time went on, computers got smaller and more powerful, and punching no longer existed, it became a computer language written by humans.

Everything is changing except one thing. The story is that computers and programming languages are only in the West. In the West, 26 letters and limited punctuation are sufficient for daily communication.

In the beginning, computer storage could be very expensive. We used one byte, or 8 bits, to store all the characters we could use. After the first 1bit, we had 128 choices.

This was the original ASCII encoding, also known as the American Standard Code for Information Interchange.

After the computer spread to the world, people found that it seems that the previous ASCII code is not enough, for example, there are more than 4,000 characters commonly used in Chinese, what to do?

It doesn’t matter, just localize the ASCII encoding, called the ANSI encoding. One byte is not enough to use two bytes, the road is a person to come out, the code is also to serve people. As a result, various coding standards such as GB2312, BIG5, JIS, etc. These encodings, while compatible with ASCII encodings, are not compatible with each other.

This seriously affects the process of internationalization, so how can we realize the dream of the same earth, the same home?

So the international community stepped in and created the UNICODE character set, which defines a unique encoding for all characters in all languages. The UNICODE character set ranges from U+0000 to U+10FFFF.

F: So what does Unicode have to do with utF-8, UTF-16 and UTF-32?

I smiled and asked little sister: little sister, put the elephant into the refrigerator a few steps?

Brother F, the story of brain teaser is no longer suitable for me. There are three steps to put the elephant into the refrigerator. First, open the refrigerator, second, put the elephant in, third, close the refrigerator.

Little sister ah, as a cultural Chinese, to truly assume the national rejuvenation, scientific and technological progress of the big responsibility, your idea is very wrong, can not just think of slogans, to have a practical operational plan, otherwise when we can build qin core, Tang core and bright core?

Yes, but what does unicode have to do with it?

The Unicode character set is eventually stored in a file or in memory, so how? Should I use a fixed 1 byte, 2 bytes or variable-length bytes? According to the encoding mode, it can be divided into UTF-8, UTF-16, UTF-32 and other encoding modes.

Utf-8 is a variable-length encoding scheme that uses 1-4 bytes for storage. Utf-16 uses 2 or 4 bytes of storage, and the underlying encodings for strings after JDK9 are changed to LATIN1 and UTF16.

Utf-32 uses 4 bytes for storage. Of these three encodings, only UTF-8 is compatible with ASCII, which is why utF-8 encodings are widely used in the world (after all, computer technology is developed by westerners).

Resolve garbled characters in Properties

To solve the garble problem in your Properties, it’s easy to do so. Readers basically take a Charsets parameter, which you pass in the encoding to be read. We just pass utF-8 in:

public void usePropertiesWithUTF8(a) throws IOException{
        Properties configProp = new Properties();
        InputStream in = this.getClass().getClassLoader().getResourceAsStream("www.flydean.com.properties");
        InputStreamReader inputStreamReader= new InputStreamReader(in, StandardCharsets.UTF_8);
        configProp.load(inputStreamReader);
        log.info(configProp.getProperty("name"));
        configProp.setProperty("name"."www.flydean.com");
        log.info(configProp.getProperty("name"));
    }
Copy the code

In the above code, we use InputStreamReader to wrap InputStream, and finally solve the problem of Chinese garbled characters.

Really. The ultimate solution

F, we do this because we know that files are encoded in UTF-8. What if we don’t know? Is it UTF-8, UTF-16 or UTF-32?

Little sister asked more and more difficult questions, but this question I have prepared.

The ultimate solution is to convert all the characters of the various encodings into Unicode characters and store them in the properties file. Is there no encoding problem when we read them again?

The conversion requires the JDK’s built-in tools:

 native2ascii -encoding utf-8 file/src/main/resources/www.flydean.com.properties.utf8 file/src/main/resources/www.flydean.com.properties.cn
Copy the code

The above command converts the ENCODING of UTF-8 to Unicode.

Before conversion:

Site =www.flydean.com name= program those thingsCopy the code

After the transformation:

site=www.flydean.com
name=\u7a0b\u5e8f\u90a3\u4e9b\u4e8b
Copy the code

Run the test code again:

public void usePropertiesFileWithTransfer(a) throws IOException {
        Properties configProp = new Properties();
        InputStream in = this.getClass().getClassLoader().getResourceAsStream("www.flydean.com.properties.cn");
        configProp.load(in);
        log.info(configProp.getProperty("name"));
        configProp.setProperty("name"."www.flydean.com");
        log.info(configProp.getProperty("name"));
    }
Copy the code

Output the correct result.

If you want to do internationalization support, you do the same.

conclusion

After many hardships, he finally solved the problem of the junior sister. Brother F needs to have a rest.

The example for this article github.com/ddean2009/l…

This article can be downloaded as a PDF link in Java-io-all-in-one. PDF

In this article: The Flydean program

Link to this article: www.flydean.com/java-io-all…