Java implementation of JTT1078 real time audio and video transmission instruction -- video streaming load package (RTP) transmission

1, On the way of realization

  • 1. Receive the 0x9101 command (real-time audio and video transmission request) issued by the platform
  • 2. Analyze the 0x9101 command and get the IP and port number of the audio and video upload data server
  • 3. Start to acquire each frame of video data of the device camera. Here, we need to pay attention to the following: the formats supported by video data are: H.264, H.265, AVS, SVAC; for details, please refer to JTT1078 protocol (Table 12)
  • 4. Encapsulate the data of each frame into (Table 19) audio and video stream and transparent data transmission protocol load package format

2, The command 0x9101 issued by the parsing platform is not mentioned here. When you get the data you need to upload the video stream

  • After the data is parsed, connect to the server.

3, Obtain video stream data and encode it into H.264 video format

I implemented it on Android platform here. Please read my article Android collects video stream data from the camera and uses MediaCodec to encode it into H264 format

4, Encapsulate each frame data of the obtained H.264 video stream with RTP package. See protocol 5.5.3 below:

5, Extract some important information

  • The bit bits defined in the table are filled in according to the large end mode: that is, the bit is arranged from left to right by string and then converted to byte
  • Field PT (load type, see Table 19): This is the wrong writing of the document, which should be as follows (Table 12)
  • Field data type: the judgment of I/P/B frame. You can refer to the blogger's I/P/B frame NALU judgment of H264 bitstream
    Simply put: the video stream data of a frame of H264 generally starts with 0x00 0x00 0x00 0x01 or 0x00 0x00 0x01, so the byte after 0x01 is the type of NALU
  • The length of data body shall not exceed 950 bytes: if the data of a frame is greater than 950 bytes, it needs to be subcontracted

Now let's see the specific implementation of the code

//Real time video packet number
private static int RTP_VIDEO_COUNT = 0;
//Calculate the distance between this I-frame and the previous I-frame
private static long LAST_I_FRAME_TIME;
//Calculate the interval between this frame and the previous frame
private static long LAST_FRAME_TIME;

/**
 * Package real-time video RTP package
 *
 * @param data       H.264 Video data of one frame
 * @param phone      SIM Card number
 * @param liveClient Connection to server
 */
public static synchronized void videoLive(byte[] data, int channelNum, String phone, LiveClient liveClient) {
    List<byte[]> dataList = new ArrayList<>();
    //Size of each package
    double everyPkgSize = 950.d;
    int length = data.length;
    if (data.length > everyPkgSize) {
        //Total number of subcontracts
        long totalPkg = Math.round(Math.ceil(length / everyPkgSize));
        for (int i = 1; i <= totalPkg; i++) {
            int end = (int) (i * everyPkgSize);
            if (end >= length) {
                end = length;
            }
            byte[] bytes = Arrays.copyOfRange(data, (int) ((i - 1) * everyPkgSize), end);
            dataList.add(bytes);
        }
    } else {
        dataList.add(data);
    }
    for (int i = 0; i < dataList.size(); i++) {
        byte[] pkgData = dataList.get(i);
        ByteBuf buffer = Unpooled.buffer();
        buffer.writeBytes(new byte[]{0x30, 0x31, 0x63, 0x64});
        //              V  P X  CC
        String vpxcc = "10 0 0 0001".replace(" ", "");
        buffer.writeByte(Integer.parseInt(vpxcc, 2));
        //            M    PT
        String mpt = "0 1100010".replace(" ", "");
        buffer.writeByte(Integer.parseInt(mpt, 2));
        //Package serial number
        buffer.writeBytes(ByteUtil.int2Word(RTP_VIDEO_COUNT));
        //SIM
        buffer.writeBytes(ByteUtil.string2Bcd(phone));
        //Logical channel number
        buffer.writeByte(channelNum);
        String dataType = "";
        //Take the 5th byte of h264, i.e. NALU type
        byte NALU = data[4];
        if ((NALU & 0x1F) == 5) {
            //This is an I frame.
            dataType = "0000";
            LAST_I_FRAME_TIME = System.currentTimeMillis();
        } else {
            dataType = "0001";
        }
        //Subcontract marking
        if (dataList.size() == 1) {
            //No packet
            dataType += "0000";
        } else if (i == 0) {
            //First package
            dataType += "0001";
        } else if (i == dataList.size() - 1) {
            //Last package
            dataType += "0010";
        } else {
            //Tundish
            dataType += "0011";
        }
        //Data type subcontracting flag
        buffer.writeByte(Integer.parseInt(dataType, 2));
        //time stamp
        buffer.writeBytes(ByteUtil.long2Bytes(System.currentTimeMillis()));
        //Last I Frame Interval
        long difIFrame = System.currentTimeMillis() - LAST_I_FRAME_TIME;
        buffer.writeBytes(ByteUtil.int2Word(difIFrame));
        //Last Frame Interval
        long difFrame = System.currentTimeMillis() - LAST_FRAME_TIME;
        buffer.writeBytes(ByteUtil.int2Word(difFrame));
        //Data body length
        buffer.writeBytes(ByteUtil.int2Word(pkgData.length));
        //Data volume
        buffer.writeBytes(pkgData);
        
        //send data
        if (liveClient != null) {
            liveClient.sendData(ByteBufUtil.toArray(buffer));
        }
        
        RTP_VIDEO_COUNT++;
    }
    LAST_FRAME_TIME = System.currentTimeMillis();
}

I really can't find out how to judge B/P frame in Java, so I only judge if it's I frame. If not, it's P frame

  • The first step is to determine whether a frame of data needs to be subcontracted, and if so, add each packet to the set
  • Step 2: fill in the corresponding field data
  • The third step is to calculate the Last I Frame Interval and the Last Frame Interval
  • Step 4 send it directly

The key point of the whole protocol is to get the video stream data and encode H.264, judge which frame the data of each frame is (I/B/P), and calculate the frame interval, so that the video stream can be uploaded

Finally, the following is attached: JTT808,JTT1078, and the example of Chongqing standard protocol data upload to Android using Netty package GitHub project address

141 original articles published, 546 praised, 540000 visitors+
His message board follow

Tags: Android Java Netty github

Posted on Fri, 06 Mar 2020 23:34:52 -0800 by suma237