When the natural backgrounds such as the sky, the sea and the buildings are more complex, it is easy to disturb the objects of visual attention, and affects the detection of ship targets. So in this paper a new ship detection algorithm based on improved visual attention model was proposed. Firstly, low frequency and high frequency feature were extracted by using wavelet transform theory. Then the algorithm used improved Top-hat filter to suppress the cloud and the strong sea clutter. The direction feature was obtained by using the improved Gabor filter, and the edge-texture feature was obtained by using the DMT (discrete moment transform). The color and movement features were also obtained by converting the images from GRB color space to HSI (Hue-Saturation-Intensity) color space. Finally, the saliency map was got by weighted linear combinations, and the target area was segmented by adaptive threshold algorithm. The experimental results show that the ship detection algorithm has good detection effect.