Streaming media server and Hikvision Dahua camera realize video monitoring and live broadcasting solutions

With the acceleration of Internet plus Internet of things, the application of video surveillance has become more and more extensive. Among them, Hikvision Dahua and other brand cameras are frequently seen in the field of vision. Because last year also realized the video monitoring scheme on the smart site project, and the current live broadcast trend is not decreasing. Now to summarize:

Reason: it's 1-to-N point to many live broadcast mode, which generally uses server forwarding, so this end-to-end mode of WebRTC is not considered here, and WebRTC will explain the implementation ideas in the next article.

Premise: the camera of Hikvision or Dahua is needed. Dahua camera has better definition quality, but it is more expensive than that of Hikvision, so the camera of Hikvision is more popular in pocket.

1, Self built streaming media server

The first way is to build a streaming media server, and then collect and push streams to the server and pull streams to the client for playback. Take a look at a picture:

  1. First, the client software or equipment collects the video stream and voice stream, or the picture stream collected by the camera hardware (how to collect is a hardware related problem, which will not be discussed here)
  2. Then push to the streaming media server by push streaming. RTMP RMSP can be used for push streaming protocol. These two protocols are based on tcp without packet loss. But it is easy to cause high latency (specifically, it depends on whether the server network is supported by CDN).
    1 //Designable h264 or h265 Code, you can h265 Code as h264 The upgraded version of the code is more friendly in the rate volume definition mobile compensation
    2 //The general structure is as follows: rtsp://Camera user name:Password@Address: Address Parameter on port server...
    3 rtsp://admin:yjt_jiankong@192.168.0.60:554/h264/ch1/main/av_stream
    4 rtsp://admin:yjt_jiankong@192.168.0.60:554/Streaming/Channels/101?transportmode=unicast

    In the above way, the stream is pushed to the server without specifying its playback address and transcoding. Therefore, we can consider using ffmpeg, which is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. That is to say, using ffmpeg can not only collect the stream locally, but also specify which server to push to and its playback address;

    1 //ffmpeg -re -i Indicates the protocol used and the parameters of the protocol. Please refer to Baidu for the specific meaning of the parameters
    2 //Then there's the same push flow as above, which is used here rtsp,Suggested use rtmp,How does Ben Shuai feel in use rtmp Better compatibility web Front end use rtmp More convenient. For example, the front end Flash Plug-in unit. perhaps Video Labels, etc.
    3 //And then based on tcp The address of transcoding playback, for example, is: rtsp://117.250.250.250/Cameratest
    4 ffmpeg -re -i rtsp://admin:yjt_jiankong@192.168.0.60:554/h264/ch1/main/av_stream -rtsp_transport tcp -vcodec h264 -f rtsp rtsp://localhost/test
    5 ffmpeg -i rtsp://admin:yjt_jiankong@192.168.0.60:554/h264/ch1/main/av_stream -rtsp_transport tcp -vcodec h264 -f rtsp rtsp://117.250.250.250/Cameratest

    Note that the playback protocol is specified before the playback address, such as rtsp rtsp://117.250.250.250/Cameratest. If it's rtmp, then it should be: rtmp rtmp://117

  3. The streaming media server does some encoding and transcoding to distribute the stream to each client, and then plays the stream. So the question is how to implement the streaming media server? How to erect???
  4. On the installation, we can use nginx rtmp module to set up. After the installation, we can use rtmp to push the flow to it. You can also use the ffmpeg command in point 2 above to write a bat script to test the camera and set up the streaming media.
  5. The PC player uses rtmp Flash to play (learn about the Video tag in H5), and the mobile player uses HLS m3u8 rtmp to play (the specific play mode depends on the project framework). Look, someone on the Internet also uses flv + http stream for playing.
  6. In the later stage, there is a pressure of increasing concurrent playback. You can layer nginx (access layer + switch layer), or forward it for load balancing, or CDN for support. In the early stage, if we consider that CDN will be used in the later stage, we can skip the CDN live service of nginx at the beginning.

2, Access to third-party platforms

In the previous project, the camera of Hikvision was purchased, so in order to facilitate the rapid development, it was connected to the third-party platform, managed and forwarded by the third-party platform. The general process is to register the camera information (serial number verification code) to the third-party platform, and then flow directly to the third-party platform. The server only needs to obtain the API interface of the three-party platform to know the playback address and play it directly to the client. Used is Fluoride cloud.

 

 

 

We can register camera information with fluorspar cloud in our own project (that is, call fluorspar cloud interface to write a piece of data to fluorspar cloud), and then obtain the playing address of fluorspar cloud API interface where we need to use it. There is no need to process the pipe flow (recharge).

Provide a class (C code) to encapsulate the fluorite cloud request:

  1 using Newtonsoft.Json;
  2 using System;
  3 using System.Collections.Generic;
  4 using System.Linq;
  5 using System.Net.Http;
  6 using System.Net.Http.Headers;
  7 using System.Text;
  8 using System.Threading.Tasks;
  9 using YJT.Common;
 10 
 11 /*20190819 by suzong */
 12 namespace YJT.Wisdom.Api.lib
 13 {
 14     /// <summary>
 15     /// Fluorite cloud request encapsulation
 16     /// </summary>
 17     public class YsClient
 18     {
 19         private static readonly string requestUrl = "https://open.ys7.com/";
 20         private static readonly string appKey = "";//Obtained through official website registration
 21         private static readonly string appSecret = "";//Obtained through official website registration
 22 
 23         /// <summary>
 24         /// Get token
 25         /// </summary>
 26         /// <returns>{code:200,data:{accessToken:"",expireTime:Accurate to milliseconds}}</returns>
 27         public static async Task<string> GetToken()
 28         {
 29             string key = ConfigHelper.GetSetting("CacheKey:YsToken") ?? "YsAccessToken";
 30             string tokenStr = MemoryCacheHelper.Get(key)?.ToString();
 31             if (string.IsNullOrEmpty(tokenStr))
 32             {
 33                 string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/token/get?appKey={appKey}&appSecret={appSecret}");
 34                 YsResult result = JsonConvert.DeserializeObject<YsResult>(str);
 35                 //cache token Cache time is 5 days
 36                 tokenStr = result?.data?.accessToken;
 37                 MemoryCacheHelper.Set(key, tokenStr, TimeSpan.FromDays(5));
 38             }
 39             return tokenStr;
 40         }
 41 
 42         /// <summary>
 43         /// Add device
 44         /// </summary>
 45         /// <param name="deviceSerial">Equipment serial number</param>
 46         /// <param name="validateCode">Equipment verification code</param>
 47         /// <returns></returns>
 48         public static async Task<YsResult> SaveDevice(string deviceSerial, string validateCode)
 49         {
 50             if (string.IsNullOrEmpty(deviceSerial) || string.IsNullOrEmpty(validateCode))
 51                 return new YsResult() { code = "-1", msg = "Missing verification code or serial number" };
 52 
 53             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/add?accessToken={GetToken().Result}&deviceSerial={deviceSerial.ToUpper()}&validateCode={validateCode.ToUpper()}");
 54             return JsonConvert.DeserializeObject<YsResult>(str);
 55         }
 56 
 57         /// <summary>
 58         /// Turn off video encryption
 59         /// </summary>       
 60         /// <param name="deviceSerial">Equipment serial number</param>
 61         /// <param name="validateCode">Equipment verification code</param>
 62         /// <returns>{code:200}</returns>
 63         public static async Task<YsResult> OffEncryption(string deviceSerial, string validateCode)
 64         {
 65             if (string.IsNullOrEmpty(deviceSerial) || string.IsNullOrEmpty(validateCode))
 66                 return new YsResult() { code = "-1", msg = "Missing verification code or serial number" };
 67             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/encrypt/off?accessToken={GetToken().Result}&deviceSerial={deviceSerial.ToUpper()}&validateCode={validateCode.ToUpper()}");
 68             return JsonConvert.DeserializeObject<YsResult>(str);
 69         }
 70 
 71         /// <summary>
 72         /// Delete device
 73         /// </summary>
 74         /// <param name="token"></param>
 75         /// <param name="deviceSerial">Equipment serial number</param>
 76         /// <returns>{code:200}</returns>
 77         public static async Task<YsResult> DeleteDevice(string deviceSerial)
 78         {
 79             if (string.IsNullOrEmpty(deviceSerial))
 80                 return new YsResult() { code = "-1", msg = "miss sequence number" };
 81 
 82             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/delete?accessToken={GetToken().Result}&deviceSerial={deviceSerial.ToUpper()}");
 83             return JsonConvert.DeserializeObject<YsResult>(str);
 84         }
 85 
 86         /// <summary>
 87         /// Get live address WSS address #Get requests get every time 
 88         /// </summary>
 89         /// <param name="token"></param>
 90         /// <param name="deviceSerial">Equipment serial number</param>
 91         /// <param name="validateCode">Equipment verification code</param>
 92         /// <returns></returns>
 93         public static async Task<string> GetPlayWss(string deviceSerial, string validateCode)
 94         {
 95             if (string.IsNullOrEmpty(deviceSerial) || string.IsNullOrEmpty(validateCode))
 96                 return null;
 97             //{"retcode":0,"msg":"Success","data":{"tokens":["ot.cadfwa3t0dkdn62x5qf257es7dbq1cie-1vwkltfwtz-1w1jc79-9eabx2bbz"],"params":"&auth=1&biz=4&cln=100"}}
 98             string str = await HttpHelper.HttpGetAsync($"{requestUrl}jssdk/ezopen/getStreamToken?accessToken={GetToken().Result}&num=1&type=live");
 99             YsResult result = JsonConvert.DeserializeObject<YsResult>(str);
100             if (result.retcode == 0)
101             {
102                 string tokensStr = result?.data?.tokens[0];
103                 string paramStr = result?.data["params"];
104                 //wss://jsdecoder.ys7.com:20006/live?dev=Equipment serial number&chn=1&stream=2&ssn=Just got tokens[0]+Just got params String of. Act as wssUrl,This address can be added checkCode=The verification code is transmitted as video encryption.
105                 return $"wss://jsdecoder.ys7.com:20006/live?dev={deviceSerial}&chn=1&stream=2&ssn={tokensStr}{paramStr}&checkCode={validateCode}";
106             }
107             return null;
108         }
109 
110         /// <summary>
111         /// Get live address #Return RTMP address
112         /// </summary>
113         /// <param name="token"></param>
114         /// <param name="deviceSerial">Equipment serial number</param>
115         /// <returns>Return rtmp</returns>
116         public static async Task<string> GetPlayRtmp(string deviceSerial)
117         {
118             if (string.IsNullOrEmpty(deviceSerial))
119                 return null;
120             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/live/address/get?accessToken={GetToken().Result}&source={deviceSerial}:1");
121             YsResult result = JsonConvert.DeserializeObject<YsResult>(str);
122             if (result.code.Equals("200"))
123                 return result?.data[0]?.rtmp;
124             return null;
125         }
126 
127         /// <summary>
128         /// Get the permission of the device
129         /// </summary>
130         /// <param name="token"></param>
131         /// <param name="deviceSerial">Equipment serial number</param>
132         /// <returns>data:
133         ///{
134         ///    supprot_encrypt Support video image encryption 0 - I won't support it, 1 - Support
135         ///    support_modify_pwd Whether the device encryption password can be modified: 0 - I won't support it, 1 - Support
136         ///    ptz_top_bottom Whether to support the up and down rotation of the pan tilt 0 - I won't support it, 1 - Support
137         ///    ptz_left_right Whether to support the left and right rotation of pan tilt 0 - I won't support it, 1 - Support
138         ///    ptz_45 Does it support pan tilt turning 0 in 45 degree direction - I won't support it, 1 - Support
139         ///    ptz_zoom Whether to support pan tilt zoom control 0 - I won't support it, 1 - Support
140         ///    ptz_focus Whether focus mode 0 is supported - I won't support it, 1 - Support
141         ///}code: 200
142         /// </returns>
143         public static async Task<YsResult<YsRoles>> GetDeviceRole(string deviceSerial)
144         {
145             if (string.IsNullOrEmpty(deviceSerial))
146                 return new YsResult<YsRoles>() { code = "-1", msg = "miss sequence number" };
147 
148             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/capacity?accessToken={GetToken().Result}&deviceSerial={deviceSerial.ToUpper()}");
149             return JsonConvert.DeserializeObject<YsResult<YsRoles>>(str);
150         }
151 
152         /// <summary>
153         /// PTZ control start
154         /// </summary>
155         /// <param name="token"></param>
156         /// <param name="deviceSerial">Equipment serial number</param>
157         /// <param name="direction">direction (Operation command: 0 - Last, 1 - Next, 2 - Left, 3 - Right, 4 - Top left, 5 - Left bottom, 6 - Right upper, 7 - Right bottom, 8 - Zoom in, 9 - Narrow down, 10 - Near focus, 11 - Far focal length)</param>
158         /// <param name="speed">speed (Pan tilt speed: 0 - Slow, 1 - Moderate, 2 - fast)</param>
159         /// <returns>{code:200}</returns>
160         public static async Task<YsResult> CradleControlStarts(string token, string deviceSerial, int direction, int speed)
161         {
162             if (string.IsNullOrEmpty(token) || string.IsNullOrEmpty(deviceSerial))
163                 return null;
164             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/ptz/start?accessToken={token}&deviceSerial={deviceSerial}&channelNo=1&direction={direction}&speed={speed}");
165             return JsonConvert.DeserializeObject<YsResult>(str);
166         }
167 
168         /// <summary>
169         /// End of pan tilt control
170         /// </summary>
171         /// <param name="token"></param>
172         /// <param name="deviceSerial">Equipment serial number</param>
173         /// <param name="direction">direction (Operation command: 0-Last, 1-Next, 2-Left, 3-Right, 4-Top left, 5-Left bottom, 6-Right upper, 7-Right bottom, 8-Zoom in, 9-Narrow down, 10-Near focus, 11-Far focal length)</param>
174         /// <returns>{code:200}</returns>
175         public static async Task<YsResult> CradleControlEnd(string token, string deviceSerial, int direction)
176         {
177             if (string.IsNullOrEmpty(token) || string.IsNullOrEmpty(deviceSerial))
178                 return null;
179             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/ptz/stop?accessToken={token}&deviceSerial={deviceSerial}&channelNo=1&direction={direction}");
180             return JsonConvert.DeserializeObject<YsResult>(str);
181         }
182 
183         /// <summary>
184         /// Get single device information
185         /// </summary>
186         /// <param name="token"></param>
187         /// <param name="deviceSerial">Equipment serial number</param>
188         /// <returns></returns>
189         public static async Task<YsResult> GetDeviceInfo(string deviceSerial)
190         {
191             if (string.IsNullOrEmpty(deviceSerial))
192                 return null;
193             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/device/info?accessToken={GetToken().Result}&deviceSerial={deviceSerial}");
194             YsResult result = JsonConvert.DeserializeObject<YsResult>(str);
195             if (result.code.Equals("200"))
196                 return result;
197             return null;
198         }
199 
200         /// <summary>
201         /// Open live broadcast function
202         /// </summary>
203         /// <param name="deviceSerial">Equipment serial number</param>
204         /// <returns></returns>
205         public static async Task<YsResult> LiveOpen(string deviceSerial)
206         {
207             if (string.IsNullOrEmpty(deviceSerial))
208                 return null;
209             string str = await HttpHelper.HttpPostAsync($"{requestUrl}api/lapp/live/video/open?accessToken={GetToken().Result}&source={deviceSerial}:1");
210             return JsonConvert.DeserializeObject<YsResult>(str);
211         }
212 
213 
214     }
215 
216     /// <summary>
217     /// Fluorite cloud return object
218     /// </summary>
219     public class YsResult<T>
220     {
221         public string code { get; set; }
222         public T data { get; set; }
223         public string msg { get; set; }
224         public int retcode { get; set; }
225     }
226     public class YsResult : YsResult<dynamic>
227     {
228     }
229 
230     /// <summary>
231     /// Fluorite cloud equipment capability set
232     /// </summary>
233     public class YsRoles
234     {
235         /// <summary>
236         /// Support video image encryption 0 - I won't support it, 1 - Support
237         /// </summary>
238         public int supprot_encrypt { get; set; } = 0;
239         /// <summary>
240         /// Whether the device encryption password can be modified: 0 - I won't support it, 1 - Support
241         /// </summary>
242         public int support_modify_pwd { get; set; } = 0;
243         /// <summary>
244         /// Whether to support the up and down rotation of the pan tilt 0 - I won't support it, 1 - Support
245         /// </summary>
246         public int ptz_top_bottom { get; set; } = 0;
247         /// <summary>
248         /// Whether to support the left and right rotation of pan tilt 0 - I won't support it, 1 - Support
249         /// </summary>
250         public int ptz_left_right { get; set; } = 0;
251         /// <summary>
252         /// Does it support pan tilt turning 0 in 45 degree direction - I won't support it, 1 - Support
253         /// </summary>
254         public int ptz_45 { get; set; } = 0;
255         /// <summary>
256         /// Whether to support pan tilt zoom control 0 - I won't support it, 1 - Support
257         /// </summary>
258         public int ptz_zoom { get; set; } = 0;
259         /// <summary>
260         /// Whether focus mode 0 is supported - I won't support it, 1 - Support
261         /// </summary>
262         public int ptz_focus { get; set; } = 0;
263     }
264 
265 }
Fluorite cloud request encapsulation

 

3, Using the open source streaming framework

There are a lot of open source streaming media frameworks, the common ones made in China by SRS. Install the push-pull flow. It can be used in live broadcast / video recording / video customer service and other scenarios, and its positioning is the operation level Internet live server cluster. Portal: http://www.ossrs.net/srs.release/releases/ You can know what you like.

Reminder: the camera hardware you buy will have the name, serial number and verification code information of the camera. The camera manufacturer, such as Haikang, will have a tool to search the camera in the local area network (go to the official website). The background of a Web interface is used to set camera channel configuration information, etc. you can log in to the current camera background by connecting to the camera browser address bar in the LAN and entering the corresponding address.

Several free test addresses of rtsp rtmp are provided (you can let the front end use these addresses to play first):

1 rtsp://184.72.239.149/vod/mp4:BigBuckBunny_175k.mov
2 rtsp://195.200.199.8/mpeg4/media.amp
3 rtmp://media3.sinovision.net:1935/live/livestream

 End...

Tags: Linux Nginx network Mobile encoding

Posted on Tue, 07 Apr 2020 03:43:13 -0700 by FramezArt