This paper focuses on the pedestrian abnormal behavior detection in video. With regard to the shortcomings of the traditional pedestrian abnormal behavior algorithm, such as, the error detection and compatibility, this paper proposes a new algorithm. It uses the method of background segmentation and pedestrian information statistics, and the modeling poses video modular under different background, so it detects abnormal behavior of the position by analysis trajectory of the pedestrian. The algorithm proposed in this paper firstly use regional division method, then it detects target by optimized integral channel feature algorithm, lastly, the algorithm use mean-shift method to track target. The experiment results suggests that the overall performance of the algorithm proposed in this paper can be improved.