Low contrast between background and defects make rail defects detection be difficult in machine vision system. This paper presents a novel contrast measurement method to enhance the contrast of defects and background. Dividing the rail image into several non-overlap areas, measure the contrast of each area. Enhance the contrast by mapping relatively low gray level to lower gray level, high gray level to higher gray level. The experimental results demonstrate that the proposed method is illumination independent that adaptively enhances the contrast of images in different illumination. What’s more, it satisfies the requirements of real-time for machine vision because it is simpler and easier to implement.