To solve the moving target tracking loss problem of the change of illumination, similar color interference and fast moving target in video surveillance system, an moving target tracking method based on improved Vibe algorithm and Meanshift is proposed. Firstly, three-frame differences and Vibe algorithm is used to obtain foreground target to eliminate the interference of ghosts. Moreover, chroma and gradient features is used to restrain shadow. Finally, the continuous and accurate tracking of moving targets is realized in complex video surveillance scenes by integrating edge features into Meanshift algorithm and introducing motion vector to predict the next frame position of the current frame of moving target. Experiments on pedestrians, vehicles and aircraft in surveillance video show that the proposed method can not only track moving targets continuously and accurately, but also be applied in the foggy low visibility conditions.