Message Receipt Solution for XMPP Protocol

Find a way in distress

The concept of message receipt is known at the beginning of instant messaging. The purpose is to solve the problem that the message is not delivered to the other party for various reasons and provide a safeguard mechanism. The main reasons for this problem are network instability, server or client anomalies, which lead to the failure to receive messages.

Because the product uses a combination of openfire and spark, I have always wanted to find a ready-made solution in this area, but by reading the summary of some developers, I mentioned that openfire has no message reply scheme. So I also saw other people's plans:

  1. Sender sends message to server
  2. The server receives the message and sends the receipt back to the sender.
  3. Sender acknowledges receipt and ends, if not, resend
  4. The server records the message and pushes it to the receiver, waiting for the receiver's receipt.
  5. Receiver receives message and receives it back to server
  6. Receiving receipt deletes message receipt record, indicating that the message has been sent.
  7. If no re-push message is received to the client within a certain period of time
  8. If the receiver receives the message, it will be de-processed. If it does not repeat, step 5-6 will be executed.

This process basically completes the function of message receipt. The key point is to establish a message confirmation mechanism between sender, server and receiver. This scheme needs to customize a set of message protocols if it is to be implemented by itself. There are many ways to implement this scheme. For XMPP, both message and iq can be sent. Of course, you can also see that this scheme will cause problems, that is, each message has to perform a set of validation, so it will increase traffic and computation.

Traffic is also very important for mobile networks, and mobile networks are prone to instability because of mobility, so naturally this part of the traffic may be larger. However, because of the instability of mobile network, message receipts are needed to confirm the status of messages and solve the problem of packet loss.

So this becomes a two-way problem, as long as the volume of the message can be minimized to reduce traffic.

But for me, how to do it is a problem, after all, to achieve a set of such functions, but also to ensure stability, otherwise the message receipt function itself is not stable. Basic design ideas are also available:

  1. The client maintains two lists (Send Receive Queue and Receive Receive Receive Queue) to save the Send/Receive Receive Message Receive situation.
  2. The server also maintains a list to record the receipt and delivery of message receipts. The server checks the list over time. If the receipt does not send a retransmitted message, it will process the message if it receives a duplicate message.
  3. The client checks the status of receipts in the two lists periodically. If the receipt is not received, it needs to be retransmitted. If the receipt is repeated, it needs to be dealt with again.

The plan is almost there, but there are new discoveries in reviewing online materials.

Dense willow trees and bright flowers

Looking at other people's summaries, we found that XMPP has an extended protocol to support message echo, which is XEP-0184 Understanding that this protocol is indeed a set of message receipt implementation methods, but...

  1. It must be supported in version 3.9 or above of openfire, which can be seen in the version log of openfire.
  2. It is only an end-to-end message receipt, and only when the receiver receives the message, it will return the receipt, which is very troublesome for the sender. If the receiver is not online, it can not know whether the message has been sent, because the server will not tell the sender that the message has been received. Only when the recipient gets the message online, the recipient sends an acknowledgement receipt to the recipient.

This looks beautiful, but it's not very useful. So I saw that my openfire is more than 4 versions, so I really support it. Then it checks that the smack package used by the client does have the implementation of XEP-0184.

//This class is a uniformly invoked class
org.jivesoftware.smackx.receipts.DeliveryReceiptManager

//This is the sender sending a receipt request to inform the client that I want a message receipt.
org.jivesoftware.smackx.receipts.DeliveryReceiptRequest

//This is the receipt confirmation returned by the recipient after receiving the message.
org.jivesoftware.smackx.receipts.DeliveryReceipt

//This is an event used by the sender to listen for acknowledgement of receipts sent back and forth by the recipient.
public interface ReceiptReceivedListener {
    /**
     * Callback invoked when a new receipt got received.
     * <p>
     * {@code receiptId} correspondents to the message ID, which can be obtained with
     * {@link org.jivesoftware.smack.packet.Stanza#getStanzaId()}.
     * </p>
     * 
     * @param fromJid the jid that send this receipt
     * @param toJid the jid which received this receipt
     * @param receiptId the message ID of the stanza(/packet) which has been received and this receipt is for
     * @param receipt the receipt
     */
    void onReceiptReceived(String fromJid, String toJid, String receiptId, Stanza receipt);
}

With these three guys, it's possible to do a message validation mechanism, but to send a Delivery Receipt Request when the client sends a message, and then wait for the message sent back by the receiver to confirm Delivery Receipt.

public class ChatDemo {

    public static void main(String[] args) {
        AbstractXMPPConnection connection = SesseionHelper.newConn("192.168.11.111", 5222, "abc", "user1", "pwd1");
        
        //Subscribe to the receipt through Delivery Receipt Manager before sending the message
        DeliveryReceiptManager drm = DeliveryReceiptManager.getInstanceFor(connection);
        drm.addReceiptReceivedListener(new ReceiptReceivedListener() {
            
            @Override
            public void onReceiptReceived(String fromJid, String toJid,
                    String receiptId, Stanza receipt) {
                System.err.println((new Date()).toString()+ " - drm:" + receipt.toXML());
                
            }
        });
        
        Message msg = new Message("100069@bkos");
        msg.setBody("Respond to my message 1.");
        msg.setType(Type.chat);
        //Put the Message in Delivery ReceiptRequest so that the receipt request can be sent back after sending the Message
        DeliveryReceiptRequest.addTo(msg);
        
        try {
            connection.sendStanza(msg);
        } catch (NotConnectedException e) {
            e.printStackTrace();
        }
        
        connection.addAsyncStanzaListener(new StanzaListener() {
            
            @Override
            public void processPacket(Stanza packet) throws NotConnectedException {
                System.out.println((new Date()).toString()+ "- processPacket:" + packet.toXML());
            }
        }, new StanzaFilter() {
            @Override
            public boolean accept(Stanza stanza) {
                return stanza instanceof Message;
            }
        });
        
        while (true) {
            try {
                Thread.sleep(5000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            
        }
    }

}

The above code is the sender's code to complete, here we do not see the recipient return the receipt process, this implementation is completed in Delivery Receipt Manager.

private DeliveryReceiptManager(XMPPConnection connection) {
    super(connection);
    ServiceDiscoveryManager sdm = ServiceDiscoveryManager.getInstanceFor(connection);
    sdm.addFeature(DeliveryReceipt.NAMESPACE);

    // Add the packet listener to handling incoming delivery receipts
    connection.addAsyncStanzaListener(new StanzaListener() {
        @Override
        public void processPacket(Stanza packet) throws NotConnectedException {
            DeliveryReceipt dr = DeliveryReceipt.from((Message) packet);
            // notify listeners of incoming receipt
            for (ReceiptReceivedListener l : receiptReceivedListeners) {
                l.onReceiptReceived(packet.getFrom(), packet.getTo(), dr.getId(), packet);
            }
        }
    }, MESSAGES_WITH_DELIVERY_RECEIPT);

    // Add the packet listener to handle incoming delivery receipt requests
    connection.addAsyncStanzaListener(new StanzaListener() {
        @Override
        public void processPacket(Stanza packet) throws NotConnectedException {
            final String from = packet.getFrom();
            final XMPPConnection connection = connection();
            switch (autoReceiptMode) {
            case disabled:
                return;
            case ifIsSubscribed:
                if (!Roster.getInstanceFor(connection).isSubscribedToMyPresence(from)) {
                    return;
                }
                break;
            case always:
                break;
            }

            final Message messageWithReceiptRequest = (Message) packet;
            Message ack = receiptMessageFor(messageWithReceiptRequest);
            if (ack == null) {
                LOGGER.warning("Received message stanza with receipt request from '" + from
                                + "' without a stanza ID set. Message: " + messageWithReceiptRequest);
                return;
            }
            connection.sendStanza(ack);
        }
    }, MESSAGES_WITH_DEVLIERY_RECEIPT_REQUEST);
}

Delivery Receipt Manager subscribes to message events and sends an ACK package when it receives a message that requires a receipt. The ack here is a message package with Delivery Receipt.

Well, this XEP-0184 is almost clear, but it's not the kind of message you want. It's more like a receipt confirmation receipt like a mobile message or an email. It is an end-to-end authentication mechanism. But if we do some interception and processing on the server side, we can achieve the state of message receipt by making an intermediate state.

The method is to intercept the XEP-0184 message at the server side, save the record at the server side if the request message is Delivery Receipt Request, and send Delivery Receipt (ack) to the sender at the same time. Then the client receives the message back to ACK and the server intercepts and updates the server record.

This method is to use xep-0184 protocol to complete the function of message receipt.

Another real Village

I don't know if it's unexpected. When I read a blog, I found a more interesting thing. That's it. XEP-0198.

What does it do?

The basic concept behind flow management is that initialized entities (a server or client) and received entities (a server) can exchange commands for more flexible management of stream s. The following two characteristics of flow management are widely concerned because they can improve network reliability and end-user experience:

  • Stanza Acknowledgements - can confirm whether a paragraph or series of Stanza has been accepted by one party.
  • Stream Resumption - The ability to quickly resume a terminated stream.

This suddenly found another village in the original ah, XMPP, after all, was originally based on TCP protocol, can complete the message arrival receipt on the basis of the flow. Its characteristics also show this point. First, it can confirm the message to ensure that the message is received by the other party. Another point is that the message can be restored (that is, retry) when it is not acknowledged. Doesn't that completely satisfy our request for message reply?

Its working process is that one end initiates a request and the other end must use the Answer.

Only in smack version 4.1.x above, and the default is not to turn on the flow management function, so you have to turn it on manually, the rest is done by smack and openfire. Execute the positive sentence before establishing TCP Connection:

XMPPTCPConnection.setUseStreamManagementResumptionDefault(true);

This code is to turn on stream recovery. Of course, stream recovery turns on Stanza confirmation. You can see the implementation of setUseStream Management Resumption Default, which calls setUseStream Management Default:

public static void setUseStreamManagementResumptionDefault(boolean useSmResumptionDefault) {
    if (useSmResumptionDefault) {
        // Also enable SM is resumption is enabled
        setUseStreamManagementDefault(useSmResumptionDefault);
    }
    XMPPTCPConnection.useSmResumptionDefault = useSmResumptionDefault;
}

The openfire server opens this function by default and has settings in openfire.xml:

  <!-- XEP-0198 properties -->  
  <stream> 
    <management> 
      <!-- Whether stream management is offered to clients by server. -->  
      <active>true</active>  
      <!-- Number of stanzas sent to client before a stream management
                 acknowledgement request is made. -->  
      <requestFrequency>5</requestFrequency> 
    </management> 
  </stream>  

Okay, that's how the message receipt function works. Unexpectedly, XMPP protocol has already supported the whole process and saved a lot of things. At the same time, websocket in openfire also supports xep-198, so the mobile phone should also be able to support it.

Reference and Reference

http://developerworks.github.io/2014/10/03/xmpp-xep-0198-stream-management/
http://blog.csdn.net/chszs/article/details/48576553

This article goes to my own blog:
https://mini188.cn/c/XMPP%E5%8D%8F%E8%AE%AE%E4%B9%8B%E6%B6%88%E6%81%AF%E5%9B%9E%E6%89%A7%E8%A7%A3%E5%86%B3%E6%96%B9%E6%A1%88

Tags: Java Mobile network Spark REST

Posted on Wed, 26 Jun 2019 11:30:06 -0700 by show8bbs