JSch-Implementing Server Remote Operation with java

introduce

Some time ago, a special requirement was received, and a service for deploying services was needed. The main task is to deploy a k8s service cluster to the remote server, and the connection information of the specific server will be transmitted through the interface.

Originally, the deployment was done manually. It is nothing more than to send some necessary files scp to the target server, then SSH Remote login, perform some installation operations, and live together. There's nothing wrong with the installation process, mainly because these steps need to be implemented in code, that is, a client library supporting SSH is needed to perform these operations.

Final selection JSch(Java Secure Channel) Official website introduction:

JSch is a pure Java implementation of SSH2.
JSch allows you to connect to an sshd server and use port forwarding, X11 forwarding, file transfer, etc., and you can integrate its functionality into your own Java programs. JSch is licensed under BSD style license.

Realization

In order to accomplish the task of deploying services, several issues need to be addressed:

  • SSH connects to a remote server
  • Execute instructions on the server
  • Transfer files using scp commands
  • Edit files on the server, mainly to modify some configuration files

Here are some of the main tools and methods

Remote ssh connection

Define a Remote class to record server login information

@Data
public class Remote {

    private String user = "root";
    private String host = "127.0.0.1";
    private int port = 22;
    private String password = "";
    private String identity = "~/.ssh/id_rsa";
    private String passphrase = "";
}

There are some default values filled in here, which are more convenient for normal use.

JSch uses Session to define a remote node:

public static Session getSession(Remote remote) throws JSchException {
    JSch jSch = new JSch();
    if (Files.exists(Paths.get(remote.getIdentity()))) {
        jSch.addIdentity(remote.getIdentity(), remote.getPassphrase());
    }
    Session session = jSch.getSession(remote.getUser(), remote.getHost(),remote.getPort());
    session.setPassword(remote.getPassword());
    session.setConfig("StrictHostKeyChecking", "no");
    return session;
}

Test it:

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        System.out.println("Host({}) connected.", remote.getHost);
    }
    session.disconnect();
}

After correctly entering the server address and password, the connection succeeded.

Here I want to mention that JSch will try to login by filling in ssh_key first, and login by password only after the attempt fails, which is consistent with the usual interaction with SSH command.~

Remote instruction

The next step is to write a generic method for executing commands on Session

public static List<String> remoteExecute(Session session, String command) throws JSchException {
    log.debug(">> {}", command);
    List<String> resultLines = new ArrayList<>();
    ChannelExec channel = null;
    try{
        channel = (ChannelExec) session.openChannel("exec");
        channel.setCommand(command);
        InputStream input = channel.getInputStream();
        channel.connect(CONNECT_TIMEOUT);
        try {
            BufferedReader inputReader = new BufferedReader(newInputStreamReader(input));
            String inputLine = null;
            while((inputLine = inputReader.readLine()) != null) {
                log.debug("   {}", inputLine);
                resultLines.add(inputLine);
            }
        } finally {
            if (input != null) {
                try {
                    input.close();
                } catch (Exception e) {
                    log.error("JSch inputStream close error:", e);
                }
            }
        }
    } catch (IOException e) {
        log.error("IOcxecption:", e);
    } finally {
        if (channel != null) {
            try {
                channel.disconnect();
            } catch (Exception e) {
                log.error("JSch channel disconnect error:", e);
            }
        }
    }
    return resultLines;
}

Test it:

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        System.out.println("Host({}) connected.", remote.getHost());
    }
    
    remoteExecute(session, "pwd");
    remoteExecute(session, "mkdir /root/jsch-demo");
    remoteExecute(session, "ls /root/jsch-demo");
    remoteExecute(session, "touch /root/jsch-demo/test1; touch /root/jsch-demo/test2");
    remoteExecute(session, "echo 'It a test file.' > /root/jsch-demo/test-file");
    remoteExecute(session, "ls -all /root/jsch-demo");
    remoteExecute(session, "ls -all /root/jsch-demo | grep test");
    remoteExecute(session, "cat /root/jsch-demo/test-file");
    
    session.disconnect();
}

After execution, the log output is as follows:

Host(192.168.124.20) connected.
>> pwd
   /root
>> mkdir /root/jsch-demo
>> ls /root/jsch-demo
>> touch /root/jsch-demo/test1; touch /root/jsch-demo/test2
>> echo 'It a test file.' > /root/jsch-demo/test-file
>> ls -all /root/jsch-demo
   total 12
   drwxr-xr-x 2 root root 4096 Jul 30 03:05 .
   drwx------ 6 root root 4096 Jul 30 03:05 ..
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test1
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test2
   -rw-r--r-- 1 root root   16 Jul 30 03:05 test-file
>> ls -all /root/jsch-demo | grep test
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test1
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test2
   -rw-r--r-- 1 root root   16 Jul 30 03:05 test-file
>> cat /root/jsch-demo/test-file
   It a test file.

The execution was satisfactory and these common commands were successful.
Re-praise~

scp operation

The scp operation official gave a very detailed example scpTo+scpFrom Again, praise~
scpTo:

public static long scpTo(String source, Session session, String destination) {
    FileInputStream fileInputStream = null;
    try {
        ChannelExec channel = (ChannelExec) session.openChannel("exec");
        OutputStream out = channel.getOutputStream();
        InputStream in = channel.getInputStream();
        boolean ptimestamp = false;
        String command = "scp";
        if (ptimestamp) {
            command += " -p";
        }
        command += " -t " + destination;
        channel.setCommand(command);
        channel.connect(CONNECT_TIMEOUT);
        if (checkAck(in) != 0) {
            return -1;
        }
        File _lfile = new File(source);
        if (ptimestamp) {
            command = "T " + (_lfile.lastModified() / 1000) + " 0";
            // The access time should be sent here,
            // but it is not accessible with JavaAPI ;-<
            command += (" " + (_lfile.lastModified() / 1000) + " 0\n");
            out.write(command.getBytes());
            out.flush();
            if (checkAck(in) != 0) {
                return -1;
            }
        }
        //send "C0644 filesize filename", where filename should not include '/'
        long fileSize = _lfile.length();
        command = "C0644 " + fileSize + " ";
        if (source.lastIndexOf('/') > 0) {
            command += source.substring(source.lastIndexOf('/') + 1);
        } else {
            command += source;
        }
        command += "\n";
        out.write(command.getBytes());
        out.flush();
        if (checkAck(in) != 0) {
            return -1;
        }
        //send content of file
        fileInputStream = new FileInputStream(source);
        byte[] buf = new byte[1024];
        long sum = 0;
        while (true) {
            int len = fileInputStream.read(buf, 0, buf.length);
            if (len <= 0) {
                break;
            }
            out.write(buf, 0, len);
            sum += len;
        }
        //send '\0'
        buf[0] = 0;
        out.write(buf, 0, 1);
        out.flush();
        if (checkAck(in) != 0) {
            return -1;
        }
        return sum;
    } catch(JSchException e) {
        log.error("scp to catched jsch exception, ", e);
    } catch(IOException e) {
        log.error("scp to catched io exception, ", e);
    } catch(Exception e) {
        log.error("scp to error, ", e);
    } finally {
        if (fileInputStream != null) {
            try {
                fileInputStream.close();
            } catch (Exception e) {
                log.error("File input stream close error, ", e);
            }
        }
    }
    return -1;
}

scpFrom:

public static long scpFrom(Session session, String source, String destination) {
    FileOutputStream fileOutputStream = null;
    try {
        ChannelExec channel = (ChannelExec) session.openChannel("exec");
        channel.setCommand("scp -f " + source);
        OutputStream out = channel.getOutputStream();
        InputStream in = channel.getInputStream();
        channel.connect();
        byte[] buf = new byte[1024];
        //send '\0'
        buf[0] = 0;
        out.write(buf, 0, 1);
        out.flush();
        while(true) {
            if (checkAck(in) != 'C') {
                break;
            }
        }
        //read '644 '
        in.read(buf, 0, 4);
        long fileSize = 0;
        while (true) {
            if (in.read(buf, 0, 1) < 0) {
                break;
            }
            if (buf[0] == ' ') {
                break;
            }
            fileSize = fileSize * 10L + (long)(buf[0] - '0');
        }
        String file = null;
        for (int i = 0; ; i++) {
            in.read(buf, i, 1);
            if (buf[i] == (byte) 0x0a) {
                file = new String(buf, 0, i);
                break;
            }
        }
        // send '\0'
        buf[0] = 0;
        out.write(buf, 0, 1);
        out.flush();
        // read a content of lfile
        if (Files.isDirectory(Paths.get(destination))) {
            fileOutputStream = new FileOutputStream(destination + File.separator +file);
        } else {
            fileOutputStream = new FileOutputStream(destination);
        }
        long sum = 0;
        while (true) {
            int len = in.read(buf, 0 , buf.length);
            if (len <= 0) {
                break;
            }
            sum += len;
            if (len >= fileSize) {
                fileOutputStream.write(buf, 0, (int)fileSize);
                break;
            }
            fileOutputStream.write(buf, 0, len);
            fileSize -= len;
        }
        return sum;
    } catch(JSchException e) {
        log.error("scp to catched jsch exception, ", e);
    } catch(IOException e) {
        log.error("scp to catched io exception, ", e);
    } catch(Exception e) {
        log.error("scp to error, ", e);
    } finally {
        if (fileOutputStream != null) {
            try {
                fileOutputStream.close();
            } catch (Exception e) {
                log.error("File output stream close error, ", e);
            }
        }
    }
    return -1;
}

There is also a common method checkAck:

private static int checkAck(InputStream in) throws IOException {
    int b=in.read();
    // b may be 0 for success,
    //          1 for error,
    //          2 for fatal error,
    //          -1
    if(b==0) return b;
    if(b==-1) return b;
    if(b==1 || b==2){
        StringBuffer sb=new StringBuffer();
        int c;
        do {
            c=in.read();
            sb.append((char)c);
        }
        while(c!='\n');
        if(b==1){ // error
            log.debug(sb.toString());
        }
        if(b==2){ // fatal error
            log.debug(sb.toString());
        }
    }
    return b;
}

Test it:
We create a new file test.txt in the project root directory

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        log.debug("Host({}) connected.", remote.getHost());
    }
    
    remoteExecute(session, "ls /root/jsch-demo/");
    scpTo("test.txt", session, "/root/jsch-demo/");
    remoteExecute(session, "ls /root/jsch-demo/");
    remoteExecute(session, "echo ' append text.' >> /root/jsch-demo/test.txt");
    scpFrom(session, "/root/jsch-demo/test.txt", "file-from-remote.txt");
    
    session.disconnect();
}

The log output is as follows:
And you can see that a file file-from-remote.txt appears in the project directory. It contains more append text than the original test.txt

Host(192.168.124.20) connected.
>> ls /root/jsch-demo/
   test1
   test2
   test-file
>> ls /root/jsch-demo/
   test1
   test2
   test-file
   test.txt
>> echo ' append text.' >> /root/jsch-demo/test.txt

Remote Editing

We usually use vi to edit files on the server, which is very convenient, but it's a bit complicated to operate vi here.
The final solution is to backup the source file first, then pull the scp to the local area, and then put the scp back in place after editing.
remoteEdit method:

private static boolean remoteEdit(Session session, String source, Function<List<String>, List<String>> process) {
    InputStream in = null;
    OutputStream out = null;
    try {
        String fileName = source;
        int index = source.lastIndexOf('/');
        if (index >= 0) {
            fileName = source.substring(index + 1);
        }
        //backup source
        remoteExecute(session, String.format("cp %s %s", source, source + ".bak." +System.currentTimeMillis()));
        //scp from remote
        String tmpSource = System.getProperty("java.io.tmpdir") + session.getHost() +"-" + fileName;
        scpFrom(session, source, tmpSource);
        in = new FileInputStream(tmpSource);
        //edit file according function process
        String tmpDestination = tmpSource + ".des";
        out = new FileOutputStream(tmpDestination);
        List<String> inputLines = new ArrayList<>();
        BufferedReader reader = new BufferedReader(new InputStreamReader(in));
        String inputLine = null;
        while ((inputLine = reader.readLine()) != null) {
            inputLines.add(inputLine);
        }
        List<String> outputLines = process.apply(inputLines);
        for (String outputLine : outputLines) {
            out.write((outputLine + "\n").getBytes());
            out.flush();
        }
        //scp to remote
        scpTo(tmpDestination, session, source);
        return true;
    } catch (Exception e) {
        log.error("remote edit error, ", e);
        return false;
    } finally {
        if (in != null) {
            try {
                in.close();
            } catch (Exception e) {
                log.error("input stream close error", e);
            }
        }
        if (out != null) {
            try {
                out.close();
            } catch (Exception e) {
                log.error("output stream close error", e);
            }
        }
    }
}

Test it:

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        log.debug("Host({}) connected.", remote.getHost());
    }
    
    remoteExecute(session, "echo 'It a test file.' > /root/jsch-demo/test");
    remoteExecute(session, "cat /root/jsch-demo/test");
    remoteEdit(session, "/root/jsch-demo/test", (inputLines) -> {
        List<String> outputLines = new ArrayList<>();
        for (String inputLine : inputLines) {
            outputLines.add(inputLine.toUpperCase());
        }
        return outputLines;
    });
    remoteExecute(session, "cat /root/jsch-demo/test");
    
    session.disconnect();
}

Post-execution log output:

Host(192.168.124.20) connected.
>> echo 'It a test file.' > /root/jsch-demo/test
>> cat /root/jsch-demo/test
   It a test file.
>> cp /root/jsch-demo/test /root/jsch-demo/test.bak.1564556060191
>> cat /root/jsch-demo/test
   IT A TEST FILE.

You can see that the letters are all capitalized.

summary

These methods basically cover the scenario of our daily operation on the server, so no matter how we deploy the service, or how we operate and maintain the server, there will be no problem.

Tags: Linux Session ssh Java

Posted on Sun, 04 Aug 2019 19:34:08 -0700 by whistler