Java implements file splitting and splicing

Single-threaded implementation

Document segmentation

In the old FAT32 file system, the maximum single file size had to be saved in 4G, which was not allowed for me as a movie-goer. But now Windows has NTFS and Most Linux distributions have Ext4 filesystems, which can be larger than 4 gigabytes per file. But t the two are incompatible. Linux can not recognize the FORMAT of NTFS USB flash drive, Windows can not recognize the format of Ext4 USB flash drive, only the old FAT32 compatible with both. Therefore, it is very important to split files and then splicing them. After splitting files, it is very convenient to transfer them over the network. Multithreading can also be used to HASH each part to improve processing efficiency.

For example, the size of the Fury. MKV file is 280M, and the default size of each block is 64M.

FileSlice 

Public class FileSlice {/** ** split file * @param filePath filePath * @param filePieceSize file size, expressed in bytes. If the value is -1, the default value is 64 MB * @return Public static Boolean Slice (Path filePath, int filePieceSize){return True; public static Boolean Slice (Path filePath, int filePieceSize){return True; @param howManyParts how many blocks are in the same directory by default @param howManyParts returns True on success. Public static Boolean glue(Path filePath, int howManyParts){return true; }}Copy the code

Next, implement the single-thread split method: using the diagram, it would look like this:

if (! Files.exists(filePath)){ return false; }Copy the code

Next check whether the default block size is used:

if(filePieceSize == -1){
    filePieceSize = 1024*1024*64;
}
Copy the code

Convert the path to a file object and calculate how many blocks to split:

File file = filePath.toFile();
int howManyParts = (int) Math.ceil(file.length() / (double)filePieceSize);
Copy the code

Initialize the input/output stream, error output error message, return false, get the current directory:

DataInputStream fileReader = null; try { fileReader = new DataInputStream(new FileInputStream(file)); } catch (FileNotFoundException e) { e.printStackTrace(); System.out.println(" File not found!" ); return false; } DataOutputStream fileWriter; Path dir = filePath.getParent();Copy the code

Next read the file and print it to each part file:

int readLength = -1; long total = 0; try { for (int i = 1; i <= howManyParts ; I ++){// createFile part I Path temp = files.createfile (dir.resolve(filepath.getfilename () + ".part" + I)); FileWriter = new DataOutputStream(new FileOutputStream(temp.tofile ())); While ((readLength = filereader.read (buffer))! = -1){ fileWriter.write(buffer,0,readLength); fileWriter.flush(); total += readLength; if (total == filePieceSize){ total = 0; break; Filewriter.close (); } // Close the input stream filereader.close (); } catch (IOException e) { e.printStackTrace(); System.out.println("IO error!" ); return false; }Copy the code

This function has been implemented, the next test (due to the movie Fury has 14G). It’s too big. Let’s change it.

public static void main(String[] args) throws IOException { double before = System.currentTimeMillis(); Path bigboss = path. get("D:\ Video\ I am mobile \\ I am mobile \\ Kyou.kara.Ore.wa.Ep05.Chi_Jap.HDTVrip.1280X720.mp4"); FileSlice.slice(bigboss,-1); double after = System.currentTimeMillis(); System.out.println(" Split file I'm mobile. Kyou. Kara. Ore. Wa. Ep05. Chi_Jap. HDTVrip. 1280 x720. Mp4, "+ Files. The size (bigboss) +" bytes, always takes "+ (after - before)," ms "); }Copy the code

Running results:

Split files I'm the mobile phone. Kyou. Kara. Ore. Wa. Ep05. Chi_Jap. HDTVrip. 1280 x720. Mp4, 765321889 bytes, always takes 16335.0 msCopy the code

File together

This is pretty easy, the opposite of partition is OK. Direct on the complete code:

public static boolean glue(Path filePath, int howManyParts){ if (! Files.exists(filePath)){ return false; } String filename = getOriginalFileName(filepath.getfilename ().tostring ()); If (filename == null){system.out.println (" incoming part filename parsing error!" ); return false; Byte [] buffer = new byte[1024 * 8]; Path dir = filepath.getparent (); try { DataInputStream fileReader = null; // Create the original file files.createFile (dir.resolve(filename)); DataOutputStream fileWriter = new DataOutputStream(new FileOutputStream(dir.resolve(filename).tofile ())); int readLength = -1; for (int i = 1; i <= howManyParts ; Path temp = dir.resolve(filename + ".part" + I); FileReader = new DataInputStream(new FileInputStream(temp.tofile ())); While ((readLength = filereader.read (buffer))! = -1){ fileWriter.write(buffer,0,readLength); fileWriter.flush(); } // The file in part I has been read in, close the stream filereader.close (); } // Close the output stream filewriter.close (); } catch (IOException e) { e.printStackTrace(); System.out.println("IO error!" ); return false; } return true; }Copy the code

Retest just segmented good I am mobile phone set 5

public static void main(String[] args) throws IOException { double before = System.currentTimeMillis(); Path bigboss = path. get("D:\ Video\ I am mobile \\ I am mobile \\ Kyou.kara.Ore.wa.Ep05.Chi_Jap.HDTVrip.1280X720.mp4.part1"); FileSlice.glue(bigboss,12); double after = System.currentTimeMillis(); System.out.println(" join 12 parts, use time "+ (after-before) + "ms"); }Copy the code

And then the output, at about 12s, is ok.

It took 12147.0ms to splice 12 partsCopy the code

Open play no problem, finally screenshot.