Aiming to the actuality of more accidents happened on flying target landing in the weapon testing field, an intelligent video surveillance system based on moving target recognition was designed. The system adopts capturing front-server-decision model. At first, a method for detecting moving targets images using background difference method and frame difference method is introduced. Secondly, target recognition is studied by the technology of contour extraction and edge detection. Finally, characteristic parameters of target are extracted by feature algorithm. Based on it, key technologies involved in the system are described in detail. Additionally, corresponding algorithm is designed using OpenCV in Visual C++ 6.0, and part of the key codes are given. Simulation results shows that the system designed can meet the needs of monitor and control of flying target, and also verify the effectiveness of the algorithm.