java multiple file replication methods and efficiency comparison

1. background

There are many ways for java to copy files, which can be divided into

  • Traditional byte stream read-write replication FileInputStream,FileOutputStream,BufferedInputStream,BufferedOutputStream
  • Traditional character stream read-write replication FileReader,FileWriter,BufferWriter,BufferedWriter,BufferedReader
  • FileChannel of NIO series
  • FileChannel + buffering
  • java.nio.Files.copy()
  • The FileUtils.copy method in the third-party package, such as org.apache.commons.io.FileUtils,org.codehaus.plexus.util.FileUtils, etc

So, look at the efficiency of various methods. The main measure is time. Other standards include memory overflow when copying large files

2. overview

In many cases, copying files includes copying all subdirectories and files under the folder, so the author adopts the traversal + copy method to copy files. That is to divide the whole copy process into traversal first, creating folders when traversing, and calling different copy methods when encountering files
Five ways to traverse:

  • (1)File.listFiles()
  • (2)File.list()
  • (3)org.codehaus.plexus.util.FileUtils.getFiles()
  • (4)org.apache.commons.io.FileUtils.listFiles()
  • (5) java.nio.file.Files.walkFileTree in Java NiO

Eight ways to replicate:

  • (1)FileInputStream+FileOutputStream
  • (2)BufferedInputStream+BufferedOutputStream
  • (3)FileReader+FileWriter
  • (4)BufferedReader+BufferedWriter
  • (5)FileChannel
  • (6)FileChannel+buffer
  • (7)org.apache.commons.io.FileUtils.copyFile()
  • (8)java.nio.file.Files.copy()

In addition, the author doesn't want to see the console much... So it is used with a little swing

3.jar package

1.org.apache.commons
2.org.codehaus.plexus

4. traversal

(1)listFiles()

 private static void traverseByListFiles(File srcFile,File desFile) throws IOException
{
    if(srcFile.isDirectory())
    {
        File[] files = srcFile.listFiles();
        assert files != null;
        for(File file : files)
        {
            File desFileOrDir = new File(desFile.getAbsolutePath() + File.separator + file.getName());
            if(file.isDirectory())
            {
                if(desFileOrDir.exists())
                    desFileOrDir.delete();
                desFileOrDir.mkdirs();
            }
            traverseByListFiles(file, desFileOrDir);
        }
    }
    else 
    {
        copyFile(srcFile, desFile);
    }
}

Get all the sub files and sub folders through listFiles() of srcFile, and then judge whether they are directories
If it is a directory, first determine whether there is a file (sometimes it is a folder but there is a file with the same name, delete it first), then create a folder, and then execute the function recursively
If it is not a directory, directly copy the two files as parameters. The method used will be set later

(2)list()

private static void traverseByList(File srcFile,File desFile) throws IOException
{
    if (srcFile.isDirectory())
    {
        String[] files = srcFile.list();
        assert files != null;
        for (String file : files)
        {
            File subSrcFile = new File(srcFile, file);
            File subDesFile = new File(desFile, file);
            if (subSrcFile.isDirectory())
            {
                if (subDesFile.exists())
                    subDesFile.delete();
                subDesFile.mkdirs();
            }
            traverseByList(subSrcFile, subDesFile);
        }
    }
    else
    {
        copyFile(srcFile, desFile);
    }
}

list is similar to the first type of listFiles(), but it's String []. It also judges the directory first, creates the directory, not directly copies the directory

(3)org.codehaus.plexus.util.FileUtils.getFiles

private static void traverseByGetFiles(File srcFile, File desFile) throws IOException
{
    if (srcFile.isDirectory())
    {
        java.util.List<File> fileList = org.codehaus.plexus.util.FileUtils.getFiles(srcFile,null,null);
        for (File file : fileList)
        {
            File desFileOrDir = new File(desFile.getAbsolutePath() + File.separator + file.getName());
            if(file.isDirectory())
            {
                if(desFileOrDir.exists())
                    desFileOrDir.delete();
                desFileOrDir.mkdirs();
            }
            traverseByListFiles(file, desFileOrDir);
        }
    }
    else
    {
        copyFile(srcFile, desFile);
    }
}

This is traversal with other tool classes

org.codehaus.plexus.util.FileUtils.getFiles(srcFile,null,null);

java.util.List of returned results

(4)Commons.io Toolkit

private static void traverseByCommonsIO(File srcFile, File desFile) throws IOException
{
    if (srcFile.isDirectory())
    {
        Collection<File> files = org.apache.commons.io.FileUtils.listFiles(srcFile,null,false);
        for (File file : files)
        {
            File desFileOrDir = new File(desFile.getAbsolutePath() + File.separator + file.getName());
            if(file.isDirectory())
            {
                if(desFileOrDir.exists())
                    desFileOrDir.delete();
                desFileOrDir.mkdirs();
            }
            traverseByCommonsIO(file, desFileOrDir);
        }
    }
    else 
    {
        copyFile(srcFile, desFile);
    }
}

The listFiles method of org.apache.commons.io.FileUtils is used. The parameters are the directory to traverse, a null and a false. The second parameter represents the filter, which represents the file with a specific suffix. The type is String []. The third Boolean parameter indicates whether to recursively access the subdirectory

(5)NIO--walkFileTree

In practice, SimpleFileVisitor is commonly used

private static void traverseByNIO2(File srcFile) throws IOException
{
    java.nio.file.Files.walkFileTree(srcFile.toPath(), new SimpleFileVisitor<>() {
        @Override
        public FileVisitResult visitFile(Path path, BasicFileAttributes attrs) throws IOException 
        {
            File d = new File(des.toString() + path.toAbsolutePath().toString().substring(src.toString().length()));
            new File(d.toString().substring(0, d.toString().lastIndexOf(File.separator))).mkdirs();
            copyFile(path.toFile(), d);
            return FileVisitResult.CONTINUE;
        }
    });
}

The FileVisitor interface defines four methods, which are:

public interface FileVisitor<T>
{
    FileVisitResult preVisitDirectory(T dir,BasicFileAttributes attrs)
    {
        //Before accessing dir, the type of dir is generally java.nio.Path
    }
    
    FileVisitResult postVisitDirectory(T dir,BasicFileAttributes attrs)
    {
        //Operation after accessing dir
    }
    
    FileVisitResult visitFile(T file,BasicFileAttributes attrs)
    {
        //Operation when accessing file
    }
    
    FileVisitResult visitFileFailed(T file,BasicFileAttributes attrs)
    {
        //Action on file access failure
    }
}

In the above example, only visitFile is implemented, because it is only a copy operation. First, judge whether it is the path of the source directory. If not, create a folder and then copy the file
Let's say that the return value FileVisitResult.FileVisitResult is an enumeration type, and judge whether to continue to traverse according to the return value
FileVisitResult can take value:

  • CONTINUE: CONTINUE
  • Terminate: end
  • Skip? Siblings: continue, skipping nodes in the same directory
  • Skip? Subtitle: continue, skip subdirectories, but access to child files

5. copy

(1)FileInputStream+FileOutputStream

First is the classic byte stream FileInputStream+FileOutputStream, which is relatively simple. Use FileInputStream to read and then use FileOutputStream to write, but the efficiency is not so good

private static void copyByFileStream(File srcFile,File desFile) throws IOException
{
    FileInputStream inputStream = new FileInputStream(srcFile);
    FileOutputStream outputStream = new FileOutputStream(desFile);
    byte [] b = new byte[1024];
    while(inputStream.read(b) != -1)
    {
        outputStream.write(b);
        addCopySize();
    }
    inputStream.close();
    outputStream.close();
}

Here is the difference between the three read methods. FileInputStream has three read methods:

input.read();
input.read(b);
input.read(b,off,len);

A.read()

Read byte by byte and return int. write(n) is used directly when writing;

int n = input.read();
output.write(n);

This can be said to be the slowest of the three read s.... the author tried a 2G file, and it took about 10 minutes to copy 160M

B.read(b)

Parameter is a byte [], which buffers the bytes and returns the number of bytes in the array, which is much faster than read()

byte [] b = new byte[1024];
while(input.read(b) != -1)
    output.write(b);

C.read(b,off,len)

In fact, this method is similar to read(b). read(b) is equivalent to read(b,off,len) with the parameter omitted

byte [] b = new byte[1024];
int n;
while((n = input.read(b,0,1024))!=-1)
    output.write(b,0,n);
public int read(byte b[], int off, int len) throws IOException 
{
    return readBytes(b, off, len);
}

public int read(byte b[]) throws IOException 
{
    return readBytes(b, 0, b.length);
}

Both call the same readBytes():

private native int readBytes(byte b[], int off, int len) throws IOException;

As for efficiency, let's see the results (the author uses a small file within 10G):

As you can see, no one is faster than the other (but the last error is a little too big? 7G is not enough file.)
After all, there are many errors, such as file, java version, machine itself, etc., for reference only

(2)BufferedInputStream+BufferedOutputStream

Buffer byte stream BufferedInputStream+BufferedOutputStream. Compared with FileInputStream,BufferedInputStream will first read data from the buffer, and then read data from the file when the buffer has no readable data, so it will be faster than FileInputStream

private static void copyByBufferStream(File srcFile,File desFile) throws IOException
{
    BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(srcFile));
    BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(desFile));
    byte [] b = new byte[1024];
    while(inputStream.read(b) != -1)
    {
        addCopySize();
        outputStream.write(b);
    }
    inputStream.close();
    outputStream.close();
}

Let's also talk about the three reads of BufferedInputStream (in fact, there are also readN and read(), which must be the slowest. readN authors rarely use it, so it's not listed.)

read(b);
read(b,off,len);
readAllBytes();

A.read(b)

In fact, this is no different from the FileInputStream. Just put a byte array in

B.read(b,off,len)

This... Is no different from FileInputStream, no more

C.readAllBytes()

All bytes can be read at one time. However, it is easy to use this one, and it can be used directly

output.write(input.readAllBytes());

But there is a price:

OutOfMemory error will appear, that is to say, for large documents, it is better to separate them honestly, instead of "one bite" and "several more bites"

Look at efficiency:

readAllBytes for large files (the author's this is a file within 5G) direct burst memory

readAllBytes() also detonation... This document which is not 2G. readAllBytes() does not seem very powerful. However, the efficiency of small files is acceptable.

(3)FileReader+FileWriter

Character stream reads and writes to FileReader+FileWriter. Compared with byte stream read, we can basically change byte [] to char []. Because it is read character by character, and byte stream is read byte by byte, we use byte []
Note that this cannot be used to read pictures, music and other files, or the copied file cannot be opened

private static void copyByFileReader(File srcFile,File desFile) throws IOException
{
    FileReader reader = new FileReader(srcFile);
    FileWriter writer = new FileWriter(desFile);

    char [] c = new char[1024];
    while(reader.read(c) != -1)
    {
        addCopySize();
        writer.write(c);
    }
    reader.close();
    writer.close();
}

(4)BufferedReader+BufferedWriter

BufferedReader + BufferedWriter: compared with FileReader, BufferedReader has a readLine() method, which can be read in every line and is faster than FileReader. The corresponding BufferedWriter provides a write(String) method, of course, there are also write(String s,int off,int len). The same cannot be used to read pictures

private static void copyByBufferReader(File srcFile,File desFile) throws IOException
{
    BufferedReader reader = new BufferedReader(new FileReader(srcFile));
    BufferedWriter writer = new BufferedWriter(new FileWriter(desFile));

    char [] c = new char[1024];
    while(reader.read(c) != -1)
    {
        addCopySize();
        writer.write(c);
    }
    reader.close();
    writer.close();
}

(5)NIO--FileChannel

To copy through FileChannel, first open the stream through FileInputStream and FileOutputStream, and then use getChannel() method. Finally, copy with transferTo() or transferFrom(). One statement is enough, which is very convenient and efficient

private static void copyByFileChannel(File srcFile,File desFile) throws IOException
{
    FileChannel srcChannel = new FileInputStream(srcFile).getChannel();
    FileChannel desChannel = new FileOutputStream(desFile).getChannel();
    srcChannel.transferTo(0,srcChannel.size(),desChannel);
    srcChannel.close();
    desChannel.close();
}

(6)NIO--FileChannel+ByteBuffer

FileInputStream and FileOutputStream are used to open FileChannel, and ByteBuffer is used

private static void copyByFileChannelWithBuffer(File srcFile,File desFile) throws IOException
{
    FileChannel srcChannel = new FileInputStream(srcFile).getChannel();
    FileChannel desChannel = new FileOutputStream(desFile).getChannel();
    ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
    while(srcChannel.read(buffer) != -1)
    {
        buffer.flip();
        desChannel.write(buffer);
        buffer.clear();
        addCopySize();
    }
    srcChannel.close();
    desChannel.close();
}

Flip means "flip",

buffer.flip();

Change the buffer from write mode to read mode, then write(buffer), and then clear the buffer

Look at the efficiency of these two methods:

In addition, the author found that the "upper limit" of transferTo is 2G, that is, for a single file larger than 2G, it can copy up to 2 g

So... There is no comparability for large files

(7)FileUtils.copyFile()

This is a tool class, nothing to say. The parameters are two files, representing the source and target respectively

private static void copyByCommonsIO(File srcFile,File desFile) throws IOException
{
    FileUtils.copyFile(srcFile, desFile);
}

(8)Files.copy()

This is the officially provided Files tool class. The first two parameters are Path, which represent the source and target respectively. You can set the third parameter (or omit) to represent options. For example, you can set

StandardCopyOption.REPLACE_EXISTING
private static void copyByFiles(File srcFile,File desFile) throws IOException
{
    Files.copy(srcFile.toPath(), desFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
}

Note that Files.copy will keep the hidden attributes of the file. The original hidden file is also hidden after copying. The above 7 types will not

6. other

(1)swing layout

A. grid layout

The main JFrame has a grid layout

setLayout(new GridLayout(3,1,5,3));

Three lines and one column, because there are only three buttons, select the source file (folder), select the target folder, and select the traversal method
The JFrame with traverse / copy mode is also suitable for grid layout:

showTraverseMethod.setLayout(new GridLayout(5,1,3,3));
showCopyMethod.setLayout(new GridLayout(4,2,5,5));

B. centered

setBounds(
(int) (Toolkit.getDefaultToolkit().getScreenSize().getWidth() / 2) - 200,
(int) (Toolkit.getDefaultToolkit().getScreenSize().getHeight() / 2) - 200, 
400, 400);

Height 400, width 400, use ToolKit.getDefaultToolKit().getScreenSize() to get the height and width of the screen to achieve the center

C. addition and deletion of components

Since there are only three buttons in the main JFrame, this component needs to be updated after the traversal mode is selected. The author's approach is to delete this component and add it first:

traverseMethodButton.setVisible(false);
remove(traverseMethodButton);
add(copyMethodButton);
copyMethodButton.setVisible(true);

Set it invisible, delete it, add another component, and set it visible

(2) progress bar

Progress bar makes the author miserable... In fact, it's just to create a new thread
The core code is:

new Thread(
    () ->
    {
        int percent;
        while ((percent = getCopyPercent()) < 100)
        {
            try
            {
                Thread.sleep(100);
            }
            catch(InterruptedException e)
            {
                e.printStackTrace();
            }
            copyProgressBar.setValue(percent);
        }
    }
).start();

The author's JProgressBar is directly added to a JFrame, without too complicated layout
After you get the percentage, you call setValue(). You must create a new thread operation, otherwise you can't display the progress bar.
In addition, SwingWorker is recommended for replication

SwingWorker<String,Object> copyTask = new SwingWorker<>()
{
    @Override
    protected String doInBackground()
    {
        try
        {
            if (traverseMethod[0])
                traverseByListFiles(src, des);
            else if (traverseMethod[1])
                traverseByList(src, des);
            else if (traverseMethod[2])
                traverseByGetFiles(src, des);
            else if (traverseMethod[3])
                traverseByCommonsIO(src, des);
            else if (traverseMethod[4])
                traverseByNIO2(src);
            else
            {
                showProgressBar.dispose();
                showMessage("Ergodic failure,Traversal method not found");
            }
        }
        catch (IOException e)
        {
            e.printStackTrace();
            showProgressBar.dispose();
            showMessage("Unknown error copy failed");
        }
        finish(start);
        return null;
    }
};
copyTask.execute();

7. test

I've said so much to be practical
(all the following tests are to delete the copied files and then make a new copy.)

(1)1G file

1G file File.listFiles() File.list() plexus.util.FileUtils.getFiles() commons.io.FileUtils.listFiles Files.walkFileTree
FileIntput/OutputStream 20.189s 21.152s 18.249s 20.131s 21.782s
BufferedInput/OuputStream 17.761s 23.786s 22.118s 19.646s 16.806s
FileReader/Writer 61.334s 58.3s 58.904s 58.679s 55.762s
BufferedReader/Writer 63.287s 59.546s 56.664s 58.212s 59.884s
FileChannel 20.097s 22.272s 22.751s 22.765s 20.291s
FileChannel+ByteBuffer 18.857s 22.489s 23.148s 22.337s 17.213s
FileUtils.copyFile 25.398s 21.95s 22.808s 25.325s 22.483s
Files.copy 16.272s 14.166s 17.057s 14.987s 10.653s

In fact, the vertical comparison is OK, because it doesn't need to traverse, and the horizontal comparison can be regarded as the average value
For non text files, FileReader/Writer and BufferedReader/Writer do not have much reference significance. For example, copying video files cannot be opened, and the copied files will become larger. The performance of single file Files.copy is very good, and java nio is really powerful

(2)10G file

10G file File.listFiles() File.list() plexus.util.FileUtils.getFiles() commons.io.FileUtils.listFiles Files.walkFileTree
FileIntput/OutputStream 171.427s 173.146s 172.611s 184.182s 250.251s
BufferedInput/OuputStream 203.509s 174.792s 167.727s 177.451s 217.53s
FileReader/Writer 187.55s 169.306s 226.571s 168.982s 218.303s
BufferedReader/Writer 155.134s 165.883s 166.192s 176.488s 206.306s
FileChannel 34.48s 35.445s 43.896s 41.827s 41.755s
FileChannel+ByteBuffer 175.632s 167.091s 178.455s 182.977s 183.763s
FileUtils.copyFile 203.997s 206.623s 201.01s 213.949s 208.739s
Files.copy 209.898s 186.889s 244.355s 222.336s 244.68s

This 10G file is a text file
Now let's look at this line of FileChannel. Obviously, it takes less time than others. Why?
Because the trasferTo method of a file larger than 2G.FileChannel can only write 2G files at most, it can only copy 2G files larger than 2G, so the line of FileChannel is not very comparable. For a text file, the speed of BufferedReader/Writer is the fastest, followed by FileInput/OutputStream. For a single large file, the speed of apache's FileUtils and NIO's Files.copy It's slower than FileInputStream

(3)1G directory

1G dir File.listFiles() File.list() plexus.util.FileUtils.getFiles() commons.io.FileUtils.listFiles Files.walkFileTree
FileIntput/OutputStream 23.549s 99.386s 143.388s 13.451s 10.773s
BufferedInput/OuputStream 6.306s 59.458s 20.704s 6.668s 6.616s
FileReader/Writer 49.059s 103.257s 51.995s 49.729s 51.509s
BufferedReader/Writer 59.932s 127.359s 51.731s 51.418s 50.317s
FileChannel 40.082s 71.713s 17.617s 15.782s 19.777s
FileChannel+ByteBuffer 33.355s 83.845s 19.68s 10.288s 17.152s
FileUtils.copyFile 24.163s 63.979s 8.277s 6.115s 19.513s
Files.copy 14.528s 28.215s 6.578s 5.883s 7.502s

For directories, you can consider giving up BufferedReader and FileReader. Unless all of them are text files, BufferedInput/OutputStream and Files.copy() are recommended for copying. The copying method of tool class FileUtils performs well, but the efficiency of java standard Files.copy is poor
For FileChannel and FileChannel used with buffer, 1g seems to be the same
For traversal mode... You can see that there is a big gap in the performance of the traversal method of plexus, while apache's listFiles or java nio's walkFileTree are relatively stable and fairly fast. It is recommended to use these two ways to traverse the directory

(4)10G directory

10G dir File.listFiles() File.list() plexus.util.FileUtils.getFiles() commons.io.FileUtils.listFiles Files.walkFileTree
FileIntput/OutputStream 216.822s 228.792s 227.908s 240.042s 191.863s
BufferedInput/OuputStream 218.599s 210.941s 207.375s 213.991s 167.614s
FileReader/Writer 536.747s 550.755s 550.415s 548.881s 516.684s
BufferedReader/Writer 587.612s 552.55s 549.716s 553.484s 498.18s
FileChannel 115.126s 117.538s 117.456s 118.207s 97.626s
FileChannel+ByteBuffer 225.887s 224.932s 222.077s 223.812s 180.177s
FileUtils.copyFile 233.724s 230.199s 232.133s 223.286s 189.737s
Files.copy 229.819s 227.562s 226.793s 226.78s 181.071s

FileReader and BufferedReader can be ignored. For small files, FileChannel is good. For large files, FileChannel can be used with ByteBuffer, but the effect is lower than BufferedInput/OutputStream in terms of data
Take a look at the copy of org.apache.commons.io.FileUtils and java.nio.file.Files. The difference is not too great, and the effect is close, but the difference is a little big at 1G
In traversal mode, java nio's walkFileTrees are the fastest

Of course, these tests are only for reference. Which one to use depends on the specific environment. In addition, this way separates traversal and replication. apache's FileUtils has a way to directly copy directories. Therefore, it needs individual specific tests to use which one is more suitable

8. source code

The author is lazy and still in a file. 700 lines

import java.awt.*;
import javax.swing.*;
import java.nio.*;
import java.nio.channels.*;
import java.io.*;
import java.nio.file.*;
import java.nio.file.attribute.*;
import java.util.*;
import org.apache.commons.io.*;

public class Test extends JFrame
{
    public static final long serialVersionUID = 12398129389122L;

    private JFrame showTraverseMethod = new JFrame("Traversal method");
    private JFrame showCopyMethod = new JFrame("Replication mode");

    private JButton traverseMethodButton = new JButton("Please select traversal mode");
    private JButton copyMethodButton = new JButton("Please select a copy method");
    private JButton copyButton = new JButton("Start copying");

    private JButton traverseByListFiles = new JButton("File.listFiles()");
    private JButton traverseByList = new JButton("File.list()");
    private JButton traverseByGetFiles = new JButton("(plexus)getFiles()");
    private JButton traverseByCommonsIO = new JButton("Commons IO");
    private JButton traverseByNIO2 = new JButton("NIO2");

    private JButton copyByFileStream = new JButton("File stream");
    private JButton copyByBufferStream = new JButton("Buffer stream");
    private JButton copyByFileReader = new JButton("File reader");
    private JButton copyByBufferReader = new JButton("Buffer reader");
    private JButton copyByFileChannel = new JButton("File channel");
    private JButton copyByFileChannelWithBuffer = new JButton("File channel with buffer");
    private JButton copyByCommonsIO = new JButton("Commons IO");
    private JButton copyByFiles = new JButton("Files.copy");

    public Test()
    {
        JButton src = new JButton("Select source file(Clip)");
        src.addActionListener(
            event ->
            {
                JFileChooser fileChooser = new JFileChooser();
                fileChooser.setFileSelectionMode(JFileChooser.FILES_AND_DIRECTORIES);
                fileChooser.showDialog(new Label(), "Select file(Clip)");
                FilesCopy.setSrc(fileChooser.getSelectedFile());
            }
        );
        JButton des = new JButton("Select destination folder");
        des.addActionListener(
            event ->
            {
                JFileChooser fileChooser = new JFileChooser();
                fileChooser.setFileSelectionMode(JFileChooser.FILES_AND_DIRECTORIES);
                fileChooser.showDialog(new JLabel(),"Select Folder");
                FilesCopy.setDes(fileChooser.getSelectedFile());
            }
        );

        traverseMethodButton.addActionListener(
            event ->
            {
                traverseByListFiles.addActionListener(
                    e->
                    {
                        FilesCopy.setTraverseByListFiles();
                        showTraverseMethod.dispose();
                    }
                );

                traverseByList.addActionListener(
                    e ->
                    {
                        FilesCopy.setTraverseByList();
                        showTraverseMethod.dispose();
                    }
                );

                traverseByGetFiles.addActionListener(
                    e ->
                    {
                        FilesCopy.setTraverseByGetfiles();
                        showTraverseMethod.dispose();
                    }
                );

                traverseByCommonsIO.addActionListener(
                    e ->
                    {
                        FilesCopy.setTraverseByCommonsIO();
                        showTraverseMethod.dispose();
                    }
                );

                traverseByNIO2.addActionListener(
                    e ->
                    {
                        FilesCopy.setTraverseByNIO2();
                        showTraverseMethod.dispose();
                    }
                );


                showTraverseMethod.setLayout(new GridLayout(5,1,3,3));
                showTraverseMethod.setTitle("Traversal method");
                showTraverseMethod.setBounds((int) (Toolkit.getDefaultToolkit().getScreenSize().getWidth() / 2) - 200,
                        (int) (Toolkit.getDefaultToolkit().getScreenSize().getHeight() / 2) - 200, 400, 400);
                showTraverseMethod.setVisible(true);
                showTraverseMethod.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

                showTraverseMethod.add(traverseByListFiles);
                showTraverseMethod.add(traverseByList);
                showTraverseMethod.add(traverseByGetFiles);
                showTraverseMethod.add(traverseByCommonsIO);
                showTraverseMethod.add(traverseByNIO2);

                traverseMethodButton.setVisible(false);
                remove(traverseMethodButton);
                add(copyMethodButton);
                copyMethodButton.setVisible(true);
            }
        );

        copyMethodButton.addActionListener(
            event ->
            {
                copyByFileStream.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByFileStream();
                        showCopyMethod.dispose();
                    }
                );

                copyByBufferStream.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByBufferStream();
                        showCopyMethod.dispose();
                    }
                );

                copyByFileReader.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByFileReader();
                        showCopyMethod.dispose();
                    }
                );

                copyByBufferReader.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByBufferReader();
                        showCopyMethod.dispose();
                    }
                );

                copyByFileChannel.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByFileChannel();
                        showCopyMethod.dispose();
                    }
                );

                copyByFileChannelWithBuffer.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByFileChannelWithBuffer();
                        showCopyMethod.dispose();
                    }
                );

                copyByCommonsIO.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByCommonsIO();
                        showCopyMethod.dispose();
                    }
                );

                copyByFiles.addActionListener(
                    e ->
                    {
                        FilesCopy.setCopyByFiles();
                        showCopyMethod.dispose();
                    }
                );

                showCopyMethod.setLayout(new GridLayout(4,2,5,5));
                showCopyMethod.setTitle("Replication mode");
                showCopyMethod.setBounds(
                        (int) (Toolkit.getDefaultToolkit().getScreenSize().getWidth() / 2) - 200,
                        (int) (Toolkit.getDefaultToolkit().getScreenSize().getHeight() / 2) - 200, 400, 400);
                showCopyMethod.setVisible(true);
                showCopyMethod.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                showCopyMethod.add(copyByFileStream);
                showCopyMethod.add(copyByBufferStream);
                showCopyMethod.add(copyByFileReader);
                showCopyMethod.add(copyByBufferReader);
                showCopyMethod.add(copyByFileChannel);
                showCopyMethod.add(copyByFileChannelWithBuffer);
                showCopyMethod.add(copyByCommonsIO);
                showCopyMethod.add(copyByFiles);

                copyMethodButton.setVisible(false);
                remove(copyMethodButton);
                add(copyButton);
                copyButton.setVisible(true);
            }
        );

        copyButton.addActionListener(
            event ->
            {
                if(FilesCopy.haveSelectedSrcAndDes())
                {
                    FilesCopy.copy();
                    copyButton.setVisible(false);
                    remove(copyButton);
                    add(traverseMethodButton);
                    traverseMethodButton.setVisible(true);
                }
                else
                    JOptionPane.showMessageDialog(null,"Please select the source file first(Clip)And destination folder!");
            }
        );

        setLayout(new GridLayout(3,1,5,3));
        setTitle("Copy file");
        setBounds((int) (Toolkit.getDefaultToolkit().getScreenSize().getWidth() / 2) - 200,
                (int) (Toolkit.getDefaultToolkit().getScreenSize().getHeight() / 2) - 200, 400, 400);
        setVisible(true);
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        add(src);
        add(des);
        add(traverseMethodButton);
    }

    public static void main(String[] args) {
        new Test();
    }
}

class FilesCopy
{
    private static File src = null;
    private static File des = null;
    private static long desSize = 0;
    private static long srcSize = 0;
    private static boolean [] traverseMethod = {false,false,false,false,false,false};
    private static boolean[] copyMethod = { false, false, false, false, false, false ,false,false};
    private static JFrame showProgressBar = new JFrame();
    private static JProgressBar copyProgressBar = new JProgressBar();
    private static JTextField textField = new JTextField();
    private static int index = 0;

    private static int getCopyPercent()
    {
        return (int)(desSize * 100.0 / srcSize);
    }

    private static void addCopySize() {
        desSize += 1024L;
    }

    public static void setTraverseByListFiles()
    {
        traverseMethod[0] = true;
    }

    private static void traverseByListFiles(File srcFile,File desFile) throws IOException
    {
        if(srcFile.isDirectory())
        {
            File[] files = srcFile.listFiles();
            assert files != null;
            for(File file : files)
            {
                File desFileOrDir = new File(desFile.getAbsolutePath() + File.separator + file.getName());
                if(file.isDirectory())
                {
                    if(desFileOrDir.exists())
                        desFileOrDir.delete();
                    desFileOrDir.mkdirs();
                }
                traverseByListFiles(file, desFileOrDir);
            }
        }
        else {
            copyFile(srcFile, desFile);
        }
    }

    public static void setTraverseByList()
    {
        traverseMethod[1] = true;
    }

    private static void traverseByList(File srcFile,File desFile) throws IOException
    {
        if (srcFile.isDirectory())
        {
            String[] files = srcFile.list();
            assert files != null;
            for (String file : files)
            {
                File subSrcFile = new File(srcFile, file);
                File subDesFile = new File(desFile, file);
                if (subSrcFile.isDirectory())
                {
                    if (subDesFile.exists())
                        subDesFile.delete();
                    subDesFile.mkdirs();
                }
                traverseByList(subSrcFile, subDesFile);
            }
        }
        else
        {
            copyFile(srcFile, desFile);
        }

    }

    public static void setTraverseByGetfiles()
    {
        traverseMethod[2] = true;
    }

    private static void traverseByGetFiles(File srcFile, File desFile) throws IOException
    {
        if (srcFile.isDirectory())
        {
            java.util.List<File> fileList = org.codehaus.plexus.util.FileUtils.getFiles(srcFile,null,null);
            for (File file : fileList)
            {
                File desFileOrDir = new File(desFile.getAbsolutePath() + File.separator + file.getName());
                if(file.isDirectory())
                {
                    if(desFileOrDir.exists())
                        desFileOrDir.delete();
                    desFileOrDir.mkdirs();
                }
                traverseByListFiles(file, desFileOrDir);
            }
        }
        else
        {
            copyFile(srcFile, desFile);
        }
    }

    public static void setTraverseByCommonsIO()
    {
        traverseMethod[3] = true;
    }

    private static void traverseByCommonsIO(File srcFile, File desFile) throws IOException
    {
        if (srcFile.isDirectory())
        {
            Collection<File> files = org.apache.commons.io.FileUtils.listFiles(srcFile,null,false);
            for (File file : files)
            {
                File desFileOrDir = new File(desFile.getAbsolutePath() + File.separator + file.getName());
                if(file.isDirectory())
                {
                    if(desFileOrDir.exists())
                        desFileOrDir.delete();
                    desFileOrDir.mkdirs();
                }
                traverseByCommonsIO(file, desFileOrDir);
            }
        }
        else {
            copyFile(srcFile, desFile);
        }
    }

    public static void setTraverseByNIO2()
    {
        traverseMethod[4] = true;
    }

    private static void traverseByNIO2(File srcFile) throws IOException
    {
        java.nio.file.Files.walkFileTree(srcFile.toPath(), new SimpleFileVisitor<>() {
            @Override
            public FileVisitResult visitFile(Path path, BasicFileAttributes attrs) throws IOException {
                File d = new File(des.toString() + path.toAbsolutePath().toString().substring(src.toString().length()));
                new File(d.toString().substring(0, d.toString().lastIndexOf(File.separator))).mkdirs();
                copyFile(path.toFile(), d);
                return FileVisitResult.CONTINUE;
            }
        });
    }

    public static void setCopyByFileStream()
    {
        copyMethod[0] = true;
    }

    private static void copyByFileStream(File srcFile,File desFile) throws IOException
    {
        FileInputStream inputStream = new FileInputStream(srcFile);
        FileOutputStream outputStream = new FileOutputStream(desFile);
        byte [] b = new byte[1024];
        while(inputStream.read(b) != -1)
        {
            outputStream.write(b);
            addCopySize();
        }
        inputStream.close();
        outputStream.close();
    }

    public static void setCopyByBufferStream()
    {
        copyMethod[1] = true;
    }

    private static void copyByBufferStream(File srcFile,File desFile) throws IOException
    {
        BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(srcFile));
        BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(desFile));
        byte [] b = new byte[1024];
        while(inputStream.read(b) != -1)
        {
            addCopySize();
            outputStream.write(b);
        }
        inputStream.close();
        outputStream.close();
    }

    public static void setCopyByFileReader()
    {
        copyMethod[2] = true;
    }

    private static void copyByFileReader(File srcFile,File desFile) throws IOException
    {
        FileReader reader = new FileReader(srcFile);
        FileWriter writer = new FileWriter(desFile);

        char [] c = new char[1024];
        while(reader.read(c) != -1)
        {
            addCopySize();
            writer.write(c);
        }
        reader.close();
        writer.close();
    }

    public static void setCopyByBufferReader()
    {
        copyMethod[3] = true;
    }

    private static void copyByBufferReader(File srcFile,File desFile) throws IOException
    {
        BufferedReader reader = new BufferedReader(new FileReader(srcFile));
        BufferedWriter writer = new BufferedWriter(new FileWriter(desFile));

        char [] c = new char[1024];
        while(reader.read(c) != -1)
        {
            addCopySize();
            writer.write(c);
        }
        reader.close();
        writer.close();
    }

    public static void setCopyByFileChannel()
    {
        copyMethod[4] = true;
    }

    private static void copyByFileChannel(File srcFile,File desFile) throws IOException
    {
        FileChannel srcChannel = new FileInputStream(srcFile).getChannel();
        FileChannel desChannel = new FileOutputStream(desFile).getChannel();
        srcChannel.transferTo(0,srcChannel.size(),desChannel);
        srcChannel.close();
        desChannel.close();
    }

    public static void setCopyByFileChannelWithBuffer()
    {
        copyMethod[5] = true;
    }

    private static void copyByFileChannelWithBuffer(File srcFile,File desFile) throws IOException
    {
        FileChannel srcChannel = new FileInputStream(srcFile).getChannel();
        FileChannel desChannel = new FileOutputStream(desFile).getChannel();
        ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
        while(srcChannel.read(buffer) != -1)
        {
            buffer.flip();
            desChannel.write(buffer);
            buffer.clear();
            addCopySize();
        }
        srcChannel.close();
        desChannel.close();
    }

    public static void setCopyByCommonsIO()
    {
        copyMethod[6] = true;
    }

    private static void copyByCommonsIO(File srcFile,File desFile) throws IOException
    {
        FileUtils.copyFile(srcFile, desFile);
    }

    public static void setCopyByFiles()
    {
        copyMethod[7] = true;
    }

    private static void copyByFiles(File srcFile,File desFile) throws IOException
    {
        Files.copy(srcFile.toPath(), desFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
    }

    public static void setSrc(File srcFile) {
        src = srcFile;
        if(srcFile.isDirectory())
            srcSize = org.apache.commons.io.FileUtils.sizeOfDirectory(srcFile);
        else
            srcSize = src.length();
    }

    public static void setDes(File desFile) {
        des = desFile;
        desSize = 0;
    }

    public static void setSrc(Path srcPath)
    {
        setSrc(srcPath.toFile());
    }

    public static void setDes(Path desPath)
    {
        setDes(desPath.toFile());
    }

    private static void copyFile(File srcFile,File desFile) throws IOException
    {
        if (copyMethod[0])
            copyByFileStream(srcFile,desFile);
        else if (copyMethod[1])
            copyByBufferStream(srcFile, desFile);
        else if (copyMethod[2])
            copyByFileReader(srcFile, desFile);
        else if (copyMethod[3])
            copyByBufferReader(srcFile, desFile);
        else if (copyMethod[4])
            copyByFileChannel(srcFile, desFile);
        else if (copyMethod[5])
            copyByFileChannelWithBuffer(srcFile, desFile);
        else if (copyMethod[6])
            copyByCommonsIO(srcFile, desFile);
        else if (copyMethod[7])
            copyByFiles(srcFile, desFile);
        else
            showMessage("copy failed,Replication method not found.");
    }

    private static void showMessage(String message)
    {
        JOptionPane.showMessageDialog(null, message);
    }

    public static boolean haveSelectedSrcAndDes()
    {
        return src != null && des != null;
    }

    public static void copy()
    {
        long start = System.currentTimeMillis();
        if(haveSelectedSrcAndDes())
        {
            if(src.isFile())
            {
                des = new File(des.getAbsolutePath()+File.separator+src.getName());
            }
            SwingWorker<String,Object> copyTask = new SwingWorker<>()
            {
                @Override
                protected String doInBackground()
                {
                    try
                    {
                        if (traverseMethod[0])
                            traverseByListFiles(src, des);
                        else if (traverseMethod[1])
                            traverseByList(src, des);
                        else if (traverseMethod[2])
                            traverseByGetFiles(src, des);
                        else if (traverseMethod[3])
                            traverseByCommonsIO(src, des);
                        else if (traverseMethod[4])
                            traverseByNIO2(src);
                        else
                        {
                            showProgressBar.dispose();
                            showMessage("Ergodic failure,Traversal method not found");
                        }
                    }
                    catch (IOException e)
                    {
                        e.printStackTrace();
                        showProgressBar.dispose();
                        showMessage("Unknown error copy failed");
                    }
                    finish(start);
                    return null;
                }
            };
            copyTask.execute();
            if (!copyMethod[4] && !copyMethod[6] && !copyMethod[7])
            {
                copyProgressBar.setMinimum(0);
                copyProgressBar.setMaximum(100);
                copyProgressBar.setValue(0);
                copyProgressBar.setVisible(true);
                copyProgressBar.setStringPainted(true);

                showProgressBar.add(copyProgressBar);
                showProgressBar.setTitle("Copy progress");
                showProgressBar.setBounds((int) (Toolkit.getDefaultToolkit().getScreenSize().getWidth() / 2) - 150,
                        (int) (Toolkit.getDefaultToolkit().getScreenSize().getHeight() / 2) - 50, 300, 100);
                showProgressBar.setVisible(true);
                new Thread(
                        () ->
                        {
                            int percent;
                            while ((percent = getCopyPercent()) < 100)
                            {
                                try
                                {
                                    Thread.sleep(100);
                                }
                                catch(InterruptedException e)
                                {
                                    e.printStackTrace();
                                }
                                copyProgressBar.setValue(percent);
                            }
                        }
                ).start();
            }
            else
            {

                final String [] text = {".","..","...","....",".....",".......","......",".....","....","...","..","."};
                textField.setVisible(true);
                textField.setHorizontalAlignment(JTextField.CENTER);
                textField.setEditable(false);
                showProgressBar.add(textField);
                showProgressBar.setTitle("In replication");
                showProgressBar.setBounds((int) (Toolkit.getDefaultToolkit().getScreenSize().getWidth() / 2) - 120,
                        (int) (Toolkit.getDefaultToolkit().getScreenSize().getHeight() / 2) - 40, 240, 80);
                showProgressBar.setVisible(true);

                new Thread(
                        () ->
                        {
                            while (getCopyPercent() < 100)
                            {
                                try
                                {
                                    Thread.sleep(400);
                                }
                                catch(InterruptedException e)
                                {
                                    e.printStackTrace();
                                }
                                if(index < text.length)
                                    textField.setText("In replication"+text[index++]);
                                else
                                    index = 0;
                            }
                        }
                ).start();
            }
        }
    }

    private static void finish(long start)
    {
        long end = System.currentTimeMillis();
        showProgressBar.dispose();
        showMessage("Replication success,Use time:" + (end - start) / 1000.0 + "s");

        copyProgressBar.setVisible(false);
        showProgressBar.remove(copyProgressBar);
        textField.setVisible(false);
        showProgressBar.remove(textField);

        Arrays.fill(traverseMethod, false);
        Arrays.fill(copyMethod, false);
        des = src = null;
        desSize = srcSize;
    }
}

Tags: Web Server Java Apache less Attribute

Posted on Sat, 09 Nov 2019 02:08:52 -0800 by TouranMan