The scale change of video frames in target tracking causes the problem that objects are lost. For solving this problem, an adaptive tracking window processing method is proposed, which uses the difference between the estimated position of the next frame and the current frame target position as the detection amount that can adaptively adjust the tracking window to achieve effective detection and tracking of targets. The experimental results show that the method can effectively reduce the probability of target and loss, prevent the target from being wrong tracking, and adapt to the target scale change.