In view of the problem that the existed research on facial expression recognition could not capture the features of key facial parts, a facial expression recognition approach based on multi-scale deformable part model (DPM) was proposed in this paper. Firstly,a feature pyramid of multi-scale image was built. And then, by means of random gradient descent methods the multi-scaled facial deformable part model was obtained. According to the response value of root filter and parts filter on the feature pyramid the face and its key parts were detected. In final, the HOG features on the key facial parts were extracted. Input it into classifier to train a model. The experimental results of CK+ and JAFFE expression databases show that under various face angles and illumination conditions, the proposed approach maintains the better effect in aspects of face detection and its key parts positioning, and furthermore, the computing speed and recognition rate of facial expression is superior to other comparison algorithms.